The stochastic parrots have apparently proved the Riemann Hypothesis
scottaaronson.blogAaronson is too close to the "rationalist" cult. Maybe he should sit down with ACL2 and get a clue. Emily Bender didn't speak about all that augmentation when coining the phrase.
The boosters are out in force to gloat today.
"According to “Poe’s Law,” there is no irony or sarcasm on the Internet so obvious that it won’t fly over countless people’s heads. In line with that, lots of people on social media are now saying variants of “how dare Scott Aaronson call LLMs ‘stochastic parrots,’ that demeaning and trivializing term from 2021? Doesn’t he understand that these so-called ‘stochastic parrots’ just proved the Riemann Hypothesis?” ROFL."
Yet on this message board correct comments like the one I respond to are downvoted. Average IQ here is 90 at best, so people marvel at LLMs who seem sentient to their monkey brains.
Too many people only see the perils of something once it affects them.
The optimists think AI will be like the cotton gin where making something cheaper and more accessible actually increased the demand for slaves as the price dropped. I'm more pessimistic. A lot of jobs are going to collapse in demand in the coming years. For awhile those at or near the top will survive and possibly make a lot of money but the rising floodwaters come for everybody.
...no he's not, that's a ridiculous statement. He is a prestigious professor with tenure at a top university.
Also, mathematics is much much more than just proving theorems. As the word goes, "Good mathematicians prove theorems, great mathematicians make conjunctures, the greatest mathematicians build theories". Someone has to carve out domains of interest to humankind. And humans will always know better what is of interest to humans than machines. These kind of reactions are a form of AI psychosis.
And lastly, this result was reached by a top mathematician steering the model towards the right approaches. You might say that the next iteration of the model might not need such steering to reach this result, but that just means that the pilot seat will have shifted higher up, so to say. New frontiers of complexity will become approachable to humanity and approach we will, "exploring strange new worlds and boldly going where no one has gone before".
Funny, I'm actually optimistic for this. Let the flood wash away this messed up system that puts people in tiers and says people should only have access to what they can "afford", not to what they need. And then allows a few to be able to afford almost anything, and many to be able to afford nothing, just by being the first to X, or acquainted with Y.
Why would he be seeing that?
The combination of a mathematician and lean is powerful, and the combination of a mathematician, lean, and AI is more powerful still. Note that the mathematician is always part of that picture. That is fundamental -- no software system, no matter how big and complicated, can actually solve every math problem (this is a fundamental result in math that started the entire field of computer science). If you read TFA he basically tells you what he is actually worrying about here:
"the Lean formal verification system still has issues, from kernel bugs (which have, in the past, allowed proofs of 0=1) to uncertain soundness of the underlying type theory. In my view, these issues can and should urgently be fixed! Until they are, a sufficiently motivated AI could almost certainly exploit them to prove a false statement!"
He is worried about the proof checking system and the possibility that an AI might exploit bugs in the proof checker, not the disintegration of his entire career because AI can "just figure it all out" or whatever.
How on earth can the comment you reply to be misunderstood? It even defends Bender.
When you're worshiping something, anything that implies that it is comparable to anything else is an insult to its divinity.
The title doesn't imply that sort of thinking on Aaronsom's part at all, though. Some simple dumb thing blowing through math problems thought maybe unsolvable is a hilarious contrast.
Past performance is not indicative of future results. Airliners are still subsonic after a 100 years of aviation.
Instead we just build more of the more economical thing.
In general, past performance is actually indicative of future results. It's not definitive, and some phenomenon are random, but the trajectory of technology over time is not some random fluke that just has coincidentally trended by random chance.
There was a period in aviation development when it was widely assumed that speed will continue to rise. Just like in the development of CPUs there was such a period, quite recently.
Sure you're going to get a lot of math problems solved, but food distribution is still going to be stuck by warlords and oligarchies.
Think of it like highway designers and the "one more lane" gridlock delusion. In this case, it's more will smith eating speghetti than cheap, low cost cancer cures.
That is a deliberate decision motivated primarily by economics and secondarily by environmental concerns. In fact airliners today are somewhat slower than they were a few decades ago because the lower speeds are more fuel efficient. We certainly could build fleets of supersonic airliners if it made sense to do so, it is not as if we have no idea how to do it and in fact we built supersonic passenger jets once upon a time.
Yeah, but that's a choice made for very good reasons:
- Noise reduction
- Fuel efficiency
- Reduced pollution and lower environment impact
- Reduced cost of materials, equipment, and maintenance
- Passenger affordability
It's not that we can't make supersonic airliners or that the fact we don’t illustrates a lack of progress or performance: it's that it doesn't make economic or environmental sense to.
Gulfstream and Sukhoi even had a partnership to develop a supersonic business jet (which, if you look at the mockups, might be one of the coolest looking planes ever conceived) but they gave up after spending a fairly enormous amount of money already because it was going to cost an absolute fortune to bring to market... and there was no real market for it.
EDIT: the responses I’m getting are a bit weird. People are putting words in my mouth but all I was actually doing was pointing out the the example used wasn’t a great illustration of the point the person I was responding to was making. This isn’t Reddit so maybe we could reign it in a bit, please?
It's not we can't make compute much more denser, there's a new standard informally called as megawatt racks. But deploying them at gigantic scales bring in massive problems as well, so we can keep total energy envelope at a fixed range.
...for the sake of the reasons you outlined.
That depends which part of 'we' is running the economy. Just imagine an economy that makes mass massive amounts of money but doesn't align with human needs.
So the question is whether more intelligence will lead to faster increase in entropy. My guess is it will. Whatever recent market trends are, they are irrelevant. Whether the AI can or can't do x thing is irrelevant. What matters is whether the companies that make AI profit from making AI, and if you look closely at what's going on it sure seems like they do. If you think otherwise it may be because you're reading headlines and propaganda articles and not digging into the fundamentals.
So maybe we have a temporary correction/slow down, because these companies took on too much leverage and grew artificially too fast. That's very possible. In the big picture long term it will be a blip. The universe seems biased to increasing the volume of intelligence and I have never heard a compelling case for why this wouldn't continue for a long long time.
We still spend money on designing better airplanes, but the two lines on that graph crossed long ago.
I can say god exists because 5>3. No matter how valid my claim is that five is in-fact greater than three, if I can't convince people of the connection it's a valid proof of something, just not what I say it is.
It might be true that if the Riemann hypothesis is false, leaves cannot fall to the ground. That would mean a falling leaf demonstrates the truth of the hypothesis. Without showing people why it must be the case in a way they understand it falls short of a genuine proof.
That would make proofs subjective, and I think that is true. To many people Pythagoras has not been proved, they just believe (correctly) there is a consensus amongst those who do understand the proofs.
I don’t think that things humans can’t understand meaningfully advance human knowledge. Scientific progress is built on a foundation made of previous discoveries. What is there to build on here? An incomprehensible soup and a note from the computer that says “trust me”?
Everyone has limits to their domain knowledge that makes understanding expert material challenging. I’m guessing that at the pinnacle of any field, there might only be a handful of people who fully understand the solutions to the most complex problems.
The difficulty of doing proofs in mathematics is largely due to the fact that mathematicians refused to treat symbolic expressions as data and had to invent ever more complex paradigms to deal with them.
Lean being the final iteration of that trend.
When you add quotation and quasi quotation on top of Polish notation proofs become as mechanical as long division.
There are unfortunately no books on the topic currently, because very few people take syntax seriously enough. This is rather ironic as computer science is only possible because someone took syntax seriously enough to invent place value notation.
https://principia-math.com/projects/d16ab288ed353bc5b10e292a...
We can and should trust computers for the straightforward, boring parts. But there's a huge gulf between the people who think, "how hard could payroll be?" and actually making a system that handles all the interacting complexity of real-world problems.
The odds that computer is making a mistake in a calculation is near zero (although not impossible). Instead it's doing exactly what it is told to do.
Edit : and more generally - math has infinite resolution, whereas computers have a finite resolution, and cannot be more precise than that
Regarding the Pentium bug, I think one has to agree when it stopped being up to the human introduction of the bug. I.e. sure, many think we have designed faulty CPUs before. Some feel computers have designed faulty computers, others feel it still comes back to the human definitions. At some point though, most seem to call human mistakes human mistakes rather than where they are derived from prior to humans. I just don’t think we’ve all reached agreement of when that boundary definition is is for mistakes from the computer.
I was going to put that exact example in brackets after "although not impossible" but decided it was unnecessary.
This should be pretty clear unless you've never used arbitrary precision arithmetic.
Yes and neither can us. We have never written infinite digits down of anything. We work in finite sets of fields.
The field of Q(pi) perfectly represents various expressions of pi for example.
And to the extent they are there are always big math, rational and decimal math systems.
I don't see the comparison here. Detecting issues in a 17 million line proof is not remotely trivial, if we accidentally build a bunch of math on a failed proof, we don't have a retroactive way of getting all of that time and effort back.
I'm not saying that automated proofs like this aren't worthwhile, but the parent comment did say "functionally equivalent". The level of trust that's needed in the people who used the system to produce the output is a pretty clear functional difference in my mind between payroll and huge generated proofs. I don't think I need as much trust in someone who produces an automated proof as someone who claims that god told them something directly (since that's a lot harder to believe in general), but I also don't have a particularly large amount of trust in the average employee of Anthropic making a bold claim either.
There have always been a large number of people in the world (and still are!) that believe any number of things on the basis on faith rather than evidence, and it's not crazy to draw a parallel between "that religious authority says so" and "that Anthropic employee says so"; for the overwhelming majority of people in the world, the claims are essentially both impossible to falsify, so at the end of the day we are still relying on trust (whether from that authority directly or from a consensus of other people we trust).
Spoiler: it didn't end well for the clanker or the humans.
We are nowhere near that. Yes its getting harder, but i'm certain that within a year people will have this disciphered and translated into traditional math prose.
Keep in mind deciphering and validating traditional proofs is also quite hard. e.g. It took a long time to figure out if the proof for fermat's last theorem was valid
But in another sense, such as whether it adds to our understanding, of course it does matter.
This is not my field, but surely this has already been done before today?
First step seems to be you encode the problem according to your whims. So there's some checking to be done in terms of "does this actually encode the problem?"
For pro mathematicians I wouldn't expect that that's where they run into trouble in things like this, but still needs a look-over.
Eventually, they will almost surely produce any mathematical proof imaginable...
Start with the title of the article
I do not see what this kind of post offer here when there is already at least one thread to discuss Alpöge's result with 1000+ comments.
The feeling of a sneer comes only from the absurdity of the belief being definitively revealed.
For anyone who feels sneered at: the problem is the absurd belief, not the reveal.
"Don't sneer at the Emperor for parading around with nothing on!"
Indeed, it's not fashionable anymore to say "stochastic parrot" or "glorified auto-complete". What is fashionable today is to talk about the "convex hull of human knowledge that LLMs interpolate in"
Even if you go back to the original "stochastic parrots" paper, it wasn't saying anything of this sort. It was a paper about costs and risks (chiefly: biases from training data, lack of accountability / explainability), not a criticism of AI hype. The concern was about probabilistically parroting bias.
I think this approach and conclusion applies to a lot more than the immediate context of that conversation.
"I returned to civilization shortly after [the Trinity test] and went to Cornell to teach, and my first impression was a very strange one. I can't understand it anymore, but I felt very strongly then. I sat in a restaurant in New York, for example, and I looked out at the buildings and I began to think, you know, about how much the radius of the Hiroshima bomb damage was and so forth ... How far from here was 34th St? ... All those buildings, all smashed - and so on. And I would go along and I would see people building a bridge, or they'd be making a new road, and I thought, they're crazy, they just don't understand, they don't understand. Why are they making new things? It's so useless.
But, fortunately, it's been useless for about 30 years now, isn't it? So I've been wrong for 30 years about it being useless making bridges and I'm glad that those other people had the sense to go ahead."
I felt the same way, and I think it was a great way to encourage introspection at the end of the article.
This is just another hypothesis on top of the Riemann Hypothesis. We now have an Hypothetical proof of the Riemann Hypothesis, great. Breakthrough.
Perhaps read up on the value proposition of Lean.
RLVR isn't magic, it just bends some numbers
chain-of-thought isn't magic, it just bends some numbers
It used to be an apt description, but not anymore.
> RLVR isn't magic, it just bends some numbers
So is all machine learning, but it turns out if you bend them well enough you can get very powerful capabilities.
And also that's enough to say they aren't just trained on repeating what's in their training data, which is the reason to call them “stochastic parrots” in the first place.
And their capabilities have vastly improved since we've started doing it.
(Like how AlphaGo learned to play Go in a way that no human ever did, just by playing against “itself”).
AGI is when the output of LLMs is plausible enough to fool putatively intelligent people that they are alive
I kind of view it as how slime mold can solve a maze. Mold doesn’t think or see, but the system of a bunch of slime mold together has almost a rudimentary intelligence.
No, they cannot, and we have seen this proven time and time again.
how it was proved?
Internal to the model, it can represent intermediate relationships, combine them, and then use those results to generate a unique answer.
This is where you fall.
Imagine when you ask an LLM, "can cats fly", it reply "No".
But then, you can feed it so much fake training data based on this specific question so that it will say "Yes" to this question, despite all the genuine training data it had before that point.
You won't be able to teach a reasoning entity such contradicting facts. There is no mechanism in the LLM to reject fake data that contradicts with its existing knowledge.
So an actual intelligent entity should curate the training data first, for the LLM to show the "appearance" of intelligence.
But then it will realize… well most mammals don’t fly. And cats don’t have wings. So no, cats cannot fly. That’s rudimentary reasoning that a child could probably do, but it’s reasoning. You can try the same experience with a human child. Tell them over and over if cats can fly. They might then say yes! Then ask them to explain how, and they’ll reason and decide no, they cannot.
Of course LLMs can do reasoning much more advanced than that, as evidenced by theorem proving.
You're eliding an important difference here.
> can reason...show its reasoning
The appearance of a thing is not necessarily the same as the thing.
- Charles Babbage, 1864 memoir, Passages from the Life of a Philosopher
At its core it’s just a bunch of weights, just floats. Put together though it can solve problems and reason, including problems which has not been solved before.
Similar to the slime mold. It’s just a single cellular organism with benign, and unimpressive, DNA programming. It doesn’t have any organs, just organelles, and certainly no nervous system.
But, put a bunch together, and it can solve a maze like a rat. Interesting.
Largely I think these “stochastic parrot” type takes are just perpetrated by people who do not use LLMs. If you use LLMs to program, as virtually every software dev working does, it’s indisputable they can reason. Not always well, but they can reason.
“Predict the next token” isn’t a process or algorithm. If you traveled back in time and said “you can build AI by predicting the next token” this wouldn’t help them do anything they didn’t already know how to do.
Can humans be okay with that? Can we be happy cats?
I'm genuinely curious where the median human's experience falls throughout history. Mostly happy?
I suspect probably not
Now we have to trust the models. How do we build and maintain that trust, especially over the next decades as generations of people grow up without having to build the same intellectual muscles?
It's simple: we have no choice.
It would be entirely ok if finding very complicated proofs becomes an activity for large machines, much like adding big numbers is. After all a proof is merely a path in an enormously large graph of repeatedly applied axioms. The interesting part is seeing how ideas are connected.
It would be really nice, if it turns out to be possible to make cats a bit smarter too, while keeping heir "catiness".
With ai predicting what changes to make, there is no reason for increased intelligence to cause genetic disorders, most likely it will simply cost them more energy. So your "no free lunches in biology" is true in very literal way, but humans and their cats have free lunches!
If the story explores that then mea culpa as I haven't yet read it.
Sometime in your youth you decided what was important to you, and how to define your worth as a person.
Many of us here decided that science, creation, and discovery were among the noblest of values... And consequently came to believe that those who did not engage in those things were therfore of lower value.
We were wrong, and maybe now we have to learn that we were... It might hurt.
Similarly the looming robotics boom means that those who define themselves by the very strange, but very deeply ingrained, protestant work ethic will come to find themselves in want of purpose... And a new means by which to judge the worth of others.
The only thing I can really promise is that nobody will learn anything meaningful, and that we will simply replace our existing fallacies with new ones.
I have this feeling like we're all being pushed toward art. We'll be either poets or slaves.
But just given the current capabilities of AI, I think dishwasher is more realistic. The AIs will do maths and poetry while we do the dishes and laundry.
But of course art we make ourselves will have a point and a purpose behind it, so that's no reason to not do it.
The nobility of old felt that they inherited "noble" qualities from their ancestors, and this justified their privileges and their elevated status as moral subjects. I can't help but see the parallels with the culture tech has built around intelligence and merit. It forms much our basis for identity and moral status. When AI exceeds us, does it take away our moral status, or does it challenge our value system based on intelligence and merit?
> We were wrong, and maybe now we have to learn that we were
While I don't want to engage with that disturbing and loaded "lower value" comment -- what makes you think that scientific discovery is now devalued? LLMs are helping us increase our discovery potential, not decrease it. And if we didn't still value discovery, we wouldn't be prompting LLMs to help us.
How well do you know how your car works?
And if we meet the moment while generational churn replaces politicians we could draw a line in the sand... limit the AI at the hardware level from ever conjuring certain geometric patterns that lead it to certain tasks[1]; presumably "launch the nukes" and "add 1+1" are vastly different electrical geometry or "namespace collisions" occur in electrical state; but caveat being it's been a while since my hardware eng days so ...i could be way off being out of touch with the principles
So restrain it to never being able to consider "fully autonomous no humans in the loop" geometry by short circuiting geometry that approaches certain shapes; like grounding a car; circuit breaker is not encoded as ephemeral state but manufactured in
For utilitarian sake then, limit it to filling joann fabric and home depot and we all can be busy noodling local edge case solutions and keep up skills Norm Abrams-style in the event of systemic problems, writing zines and crafting mix tapes
Can still make art and play instruments away from a computer. Can still watch the night sky, start a campfire and cook smores. Doing it all on a laptop everyday is a recent diversion from millions of years of homo-this-n-that evolution
It's all resolvable socially and tech wise if we put in the effort. We got to the moon within a decade through similar collectivism
Media propaganda since Reagan been keeping people apart but everyone who was around for it keeps saying we were happier 25-30 years ago so maybe that's the social peak we aim for and let the robots do the mining and repetitive stress injury stuff
[1] https://iopscience.iop.org/article/10.1088/1742-6596/2987/1/...
The robots will only come for the janitors job when there is nothing higher paying left.
Easy to disrupt social knowledge work that isn't keeping people biologically healthy and alive
Janitorial work keeps human spaces clean for us.
Relative to that sort of job, VP of Whatever who sits on Slack all day been doing nothing but noodling ephemeral finance variables
But The Jetsons is still inaccurately utopian: One time when George was fired, RUDI intervened on his behalf and threatened to melt down the sprocket factory until he was rehired.
Soma is the endgame for 99% of humans.
https://www.lesswrong.com/s/d3WgHDBAPYYScp5Em/p/Py3uGnncqXuE...
Let's take this one thing as an example. So, this one dumb machine created millions of pages of crap to try to prove a theorem. That doesn't mean a human being can't understand the proof. It just means this one proof is hard to understand. Hard - not impossible.
Hard doesn't mean we can't simplify the proof. Hard doesn't mean we can't break the proof up into bite-sized chunks and prove individual pieces of it. Hard doesn't mean that someone else won't find a much simpler proof. Hard doesn't mean we can't follow chains of logic to determine meaning. All it means is this one stupid machine in this one instance created a big stupid mess. A mess we can understand, if we try.
And I suspect all future progress will actually rest on their shoulders, to the great consternation of the now-deskilled AI boosters.
Computer science was a blip in time where something revolutionary was also simple enough for one person to understand from foundations to applications. Most things aren’t that way.
And yet left goes on. People have existing for millennia not deeply understanding the natural environment around them or the systems they use. When there’s a need for deep understanding, a handful of people fulfill the need.
I empathize with you because my life has been about understanding everything I possibly can, but I was never under any illusion that it would be possible to deeply understand everything. So if that makes me a happy cat, sure, count me in.
Bidi / remote control enabled?
Special plugins? AdBlock?
- Mobile internet (IP 172.56.47.136)
- The haveibeenproxied link indicates my IP is shared (expected)
- This is stock android/chrome with a DNS blocklist of advertising domains and some invasive Samsung the things I can't fix on the device (I doubt that's the culprit given that this is a new behavior I'm observing in the outside world after 8yrs with those settings and also because it's be strange for nginx to be the thing to immediately report that trackers have to load for me to get a glimpse of the main page, especially as something I'm seeing across multiple sites, but that's not impossible).
- No VPN
- I'm a "normie" internet user. Anything remotely questionable to even the cheapest and least aligned of LLM judges and executioners happens on non-home networks.
- I see it with other devices on the same network (Chrome, Linux, MacOS, Safari, Qutebrowser, etc), which suggests to me that somehow my IP or some other shared network property is the issue. I'm curious mostly about the commonality in separate nginx installations returning a 403. That suggests there exists a common blocklist or whatnot I can inspect and fix, either because of my IP or because something in the loop is forcing ipv6-only or who knows what.
Disclosure: I work for Spur, the company behind HaveIBeenProxied.
I've been getting by at home on a T-mobile hotspot, and haveibeenproxied now reports Kookeey and Shifter residential proxies for my IPv4 shared with God knows how many other customers.
Might moving to a T-mo 5G home box help? Guessing that an IPv4 would still be shared with other people's TVs ...
But WayBackMachine has been 419ing (from nginx) my requests from my desktop these couple of days, when I’ve tried. Safari on a boring macOS installation. Same network (I mostly don’t use the cell service on my phone, when at home). Haveibeenproxied returns all clear.
Maybe an ipv6 vs ipv4 thing? I have custom dns on the desktop and I think it only resolves ipv4 (maybe?) while my phone is using my upstream’s default DNS and probably prefers ipv6.
It has occurred to me that the cost of shit like this, and of an extra 1-2 second periodically on site loads across the web to block bots (this is WAY up the last couple years) might go surprisingly far toward negating the time savings provided by LLMs…
"... the blogs/sites they reference..."
It would be helpful to list some examples
This particular blog, i.e., use of CSS and/or Javascript by Wordpress/author, is unreadable for me if using a popular graphical web browser. Unless I prefix the URL with "view-source:"
Normally I don't notice such things because I use a customised text-only browser to read HTML, not a popular graphical one
Without sourcing/running the CSS or Javascript, this blog looks fine
But at the same time, where's the RSI? If the RSI is there then make the company profitable already [though I suppose that means letting go of all the staff? But if it's improving itself, not sure what humans are in the loop for].
Genuinely curious when AI will be able to solve real-world problems, what would 20,000 hours of AI say on the topic of whether Epstein killed himself?
But I don't think it's RSI.
My mental model of RSI is essentially isn't quite asymptotic but exponential. If a system can make itself 5% smarter every week then it would be 12x as smart in one year at which point it's not clear to me if we even need governments or markets because we have a being that can direct everything.
In short I don't believe there's any world where there's true RSI intelligence and anthropic returns green paper to "shareholders" so they can buy pretty cars as status symbols.
And that is also just providing an explanation for the sake of accepting your premise, that we're not experiencing exponential improvements. In a lot of ways it sure seems like like a radical acceleration. I mean Claude Opus was only released exactly 1 year ago, and now long-standing math problems are being solved every other week. I mean if it were RSI, what would literally be different today?
Of course there is a raw hardware limit, but it's my intuition that we are under-utilizing the hardware by multiple degrees of magnitude. The whole pattern of using token prediction and realizing intelligence falls out the other end has been more or less stumbled upon by us, there's no reason to think it's optimal.
Arguing over whether a language model is really a stochastic parrot mistakes a metaphor for an actual claim.
Like arguing back and forth if a paper tiger is made out of printer paper or newspaper.
This leaves us in a very unsatisfying position as creatures who want answers to “why” questions. Intuitive answers are incomplete, and complete answers are either fundamentally impossible or completely incomprehensible or both.
Feynman said something similar with regards to magnetism, but I feel that understanding a chess position is an even better example. Magnetism depends on facts about the universe which at their most basic level “just are”, whereas even a complex chess position is theoretically entirely explainable.
There's plenty of intelligent people who can answer a question like that without devolving into a lecture.
Are you a physicist? I wonder if non-physicists just misread his frustration. The question isnt about some macroscopic physics problem that decomposes clearly into something satisfying. Its a question about one of the most basic constituents of reality. A "why?" question about something so basic has no answer other than "just because". Insofar as anything can be said, it will be so abstract and technical that it wont satisfyingly address the question.
It is a little like a lay person asking a mathematician: why is 0*1 = 0, but 0+1=1 ?
Except, somewhat more extreme than that, because such basic things as 'magnetism' are introduced into all accessible physics formalisms as assumptions.
The interviewer most likely would not have interrogated him further about the underlying properties of ice (or engaged in that entire fictitious interaction). Feynman clearly understands that there are several levels to understanding.
So just give an answer that you feel is appropriate, add a disclaimer ("this answer is hand wavy"), and move on. Why is that so hard?
He's taking it as read that the reason he is giving his extended answer is to explain why he cannot answer at all.
What he could have said to a physics student is that: wave-functions of charged particles have a property, "phase". The problem is that point values of phase cannot be observed, and if you formulate the laws of physics with raw "phase" values (like a measure of mass, at 5kg), they dont work. So something must exist to "track" phase, so that every pair of measurements "cancel out" the effect of this property. That something is the EM field. Magentism is a spatial aspect of this field. Eletricity is a space-time aspect of this field. So the magnetic force arises out of a need to "conserve phase", and that phase is how a pushing force is experienced: two particles experiencing a magentic interaction take on different phases (their wavefunctions interfere such that..).
This might "feel" like an explanation to the physics student, because they will be willing to make the assumptions needed, and they will find conservation-style explanations sufficiently satisfying. ie., they can already see a wavefn description will have a free phase term. But a lay person wouldnt feel satisfied because more unknowns have been introduced, and conserving "another unknown" is no advancement in understanding.
That's the gulf which is frustrating feynman. There's nothign to be said to the interviewer other than the force has no explanation in terms of anything he's familiar with. The question is too basic. And feynman is not going to go thru a formulation of QM in terms of wavefns in 5min -- it would be an entire lecture course.
One afternoon underway I was standing in Central Control and the main propulsion assistant, more senior to me and studying for her engineering boards, started quizzing me about the electrical plant. And I would explain things. And she would ask why. So I would explain and she would ask why. And so on. Until we got to fundamental wave equation for electromagnetism. And she kept asking why. And at one point I realized we were at the bottom. There's nothing really below electromagnetism. It just is. And I said that. "It's one of the 4 fundamental forces. It's just a mathematical description of the way the world seems to work, and it's proven to be correct, entirely, in all cases. There other forces at different energy levels, but in the world that affects you and me, it just is. It, it just is."
And at that point I realized half the engineering department was peeking in from around the consoles. And she starting crying. And I felt so bad. Like I had said something wrong, and I still don't understand what I should have done better. That was almost 30 years ago.
So Feynman's observation that you have to be in some framework that you allow something to be true strikes me as incredibly helpful. I think the MPA must not have had that kind of framework and found herself distressed when she kept reaching for things and never getting something more to hold on to. Maybe I should have had the insight to offer her more of a lifeline.
My point is that the answer you give depends on who you're talking to. It's okay to present a simplification or an incomplete picture. That's how everybody learns. Add a disclaimer and move on. There's a time and a place for technical correctness and, unless the interviewer was being critical/annoying prior to that recording, I would say the frustration is misplaced. There's no need to get frustrated at the interviewer because you can't meet your own standards of rigor.
But I have learned something very useful from this non-explanation explanation. Sometimes you have to assume stuff, and sometimes you have to dumb it down. For example, when your five year-old asks you how magnets work, you're just not going to break out Maxwell's equations. Instead you'll explain how everything is made of very tiny things called atoms, and that all atoms are like tiny magnets, but some kinds of atoms can be made to line up and then all the tiny magnet forces add up to a big magnet force. The kid'll eat it up. I know because that's roughly what my father said to me way back when. And so you'll have different explanations for different people, up to the level of your understanding.
The problem here is that not only are we missing context in that clip (had the interviewer been annoying?), but Feynman barely disguises that he can be unpleasant. He has to prep himself to tell the guy that he's just not a physicist. He has to chastise himself for being so good at this that people expect him to be able to explain anything and everything even though it's not always possible. He's kind of an ass here. But what a great answer he gives! His visible annoyance is almost necessary.
doesn’t seem useful to humans, only to other machines
https://www.pnas.org/doi/10.1073/pnas.2406675122
We can actually see some of the calculations and even learn new chess tactics from these networks. They don't need to be opaque black boxes!
Look, LLMs are models of the training data lossily compressed in a way that allows interpolating between patterns while decompressing.
A pattern can be "it's better to use int32_t than int", what a for loop in c++ looks like, that testing for primality can be done by trying divisors only until the square root of n, etc. Combining those patterns, an LLM can produce a working is_prime function in c++.
Older LLMs were only able to handle simple patterns so the code was more likely to be verbatim training data, newer LLMs work on deeper, more complex patterns so it's harder to detect where those patterns come from.
Similarly, the list of approaches a mathematician uses to approach a problem is also a pattern. One of them will be proof by induction - how to convert such a proof to Lean will is also a pattern. The idea that induction can be done separately for odd and even cases is also a pattern.
LLMs are no longer generating the final output directly - they are simulating human work but at a much larger scale. That's how they are able to generate novel results.
But it's all based on labor scraped and patternized without consent or credit, let alone monetary compensation.
And I still wouldn't call it intelligence because that gives people the idea that it's something which gives complete, correct and unbiased answers. It's still autocomplete under the hood, there's just more and more tooling built around it to keep it on track, to check intermediate results, to do arithmetic, etc. The LLMs in some cases is more like a heuristic for deterministic tools.
Humans can't do that. Why would you hold LLMs to that standard?
Humans can do amazing things with very limited hardware, LLMs make a lot of mistakes above the hardware level. The whole principle of outputting the probable next token is fundamentally different from logical reasoning, even if you can simulate logical reasoning on this shaky foundation.
Back when GPT 3.5 and 4 launched, there were a lot of people here that used that argument seriously.
The person who coined the term and published the Stochastic Parrot paper has been trying to build a career on top of the controversy, publishing books and making a lot of noise on Twitter. For a while it was working and parts of Twitter were bought into the stochastic parrot idea, but the person at the center of it has a bad habit of lashing out at everyone and alienated most of them along the way.
The concept feels antiquated now that we can all access extremely powerful AI models for $20/month and see for ourselves, but there was a time when people took this seriously. I think it was a very convenient argument for those who were in denial, and there was a large audience who wanted to consume ideas that validated their denial.
No.
as a non mathematician, you guys have some soul searching to do, it seems like its proven? I get it when there’s one lone japanese that has done the math and has a proof that no one understands, but no one can disprove it yet.
but proof that no one yet understands except an AI? it sounds like a koan, is a proof a proof if no one is convinced beyond a doubt by it?
P.S: AlphaGo was pretty much the same in the spirit btw.
That trust is valuable, because top mathematicians say that if there is a Lean proof, there is a proof. A few mathematicians reluctantly trusted HOL Light a decade ago, when the Kepler Conjecture was proven using it. That trust in Lean means that mathematicians are facing the truth now, rather than denying it.
But, from a technology perspective, Mizar and other systems existed in the 1970s. This accomplishment is all by AI.
Another prediction that might age poorly.
One day the head of the studio (the book author) had an insight. She asked the actors to drive the conversation so that they always bump the status of themselves or the other party in any direction, but only a little.
This did the magic. Not only the actors understood her perfectly, the conversation started to flow and could go endlessly while staying amusing and captivating for spectators. (Look up the book for examples.)
After they discovered that secret they started seeing the pattern (status games) everywhere up to the point they asked themselves whether there is anything else in human communication.
This was a British studio, the sketches were mostly comedy.
what? English non native so that might be the reason
I barely comprehend social etiquette on meta level of british people, so it is not just a language problem if the word used was a little too broad.
But it could work that way and I rarely noticed, since people are supposed to talk over the weather if starting smalltalk. Not necessarily about themselves.
[1] See, bumping myself.
The key for the improv actors is doing it with intention. To decide “what is the next thing I should say in this conversation” they think “how do I make my fictional self look better, or my fictional conversational partner look worse, to the audience”? And that gets them to natural-sounding phrases.
No, but the way it's said absolutely is. You can lead by intimidation, and you can lead by inspiration. The former is about trying to raise your own status and/or lower that of your subordinates. The latter is about trying to raise the status of your subordinates. Do the latter right, and it will also improve your own status in their eyes.
It's a fundamental behavior of humans confirmed by history. Psychology and sociology even more so have also studied this, but the behavior is so fundamental that wether a field "confirms it" or not means little. One classic:
https://www.amazon.com/Presentation-Everyday-Penguin-Modern-...
>most people I encounter don't do this.
Most people you encounter 100% do this.
You can still hint at how you know more about the subject, how the other person knows less (or the opposite), and imply countless other things more or less closely associated with that topic or derivative of it that raise/lower one's status.
Like you'd be talking about the pollution of the rainforests, and you casually let the others know you've travelled extensively in the Amazon, or how you did your PhD at some prestigious school on a related subject, or how you only support Fair Trade coffee or how you know this or that expert you met at a fancy conference or through an important friend. The possibilities are endless.
>But with friends talking about topic x, making points for others or oneself seems rather distracting or off-topic.
That's more like an ND view, where what's important is "the topic". In the NT game the discussion is rarely about just the topic, even if it seems so and even if the topic is important, and it's a business meeting or something like that.
People can skip around a topic for status reasons even if it's a life or death technical matter, e.g. https://thediplomat.com/2013/07/asiana-airlines-crash-a-cock...
Status truly is your god, whether you would admit so or not.”
https://www.overcomingbias.com/p/status-is-your-god(I'd still like to see some concrete examples here)
- Does intelligence is building a 10 Km brinde with shovel and grabble spending millons tons material and man power? I don't think so.
- Does intelligence is building it with a 1/100000 of the material, calculating where to put it so you don't use more than needed? I think so.
Both are effective bridges, building them might and might not require intelligence.