Claude solves two open problems in the physics of black holes
anthropic.comThose theoretical physics papers look like the real thing (i.e. not mathematical physics like the OpenAI Yang-Mills result). It looks like AI is finally branching out of math.
Is it really, though? It's still a heavily mathematical domain, rather than one involving empirical research, where AI still struggles.
"It's not much, but it's honest work."
If you attempt to formalize math involving such assumptions, one expects to encounter a proliferation of contradictions. So I’m curious to see how the contributions of AI authors is handled by the physics community. With pure math, you can ostensibly write a compact statement of a theorem in Lean, and if so, trust the process.
For people not in the know, it's common - both now and historically - for physicists to use "mathematical techniques" that have not been proven valid.
I took grad level physics courses, and this drove me bonkers. I was accustomed to the comfort mathematics provided me. If something was unclear, I could always dig deeper and find a proof. But a lot of the stuff the physics professors would do - I couldn't, and nor could they. They just responded with "intuition". Some of them had dug deep earlier in their careers and stated that the technique was likely invalid mathematically, "but still useful in getting answers experiments can verify".
I won't even talk about relying on diverging series...
The Dirac delta function is a famous example of something mathematically non-rigorous at its inception, and mathematicians objected to it, but a few decades later Laurent Schwartz developed generalized function theory, that made it work.
Or going even further back, calculus was done in terms of ratios of infinitesimals (like dy/dx) or infinite sums of them (integrals). A century or two later the formulas of calculus were proven rigorously through classical analysis (delta-epsilon proofs) but that lost sight of the original motivations. A century or two after THAT, nonstandard analysis made a rigorous theory of infinitesimals, but reliant on relatively high powered mathematical logic that didn't exist til the 1950s or so.
Right now I think quantum field theory (renormalization, path integrals etc.) have been put on a sound mathematical basis but physicists mostly don't care. And very recently, an AI proved something about Yang-Mills theory that helped rigorize part of the Standard Model of particle physics. I don't understand that last part at all.
Besides physics, a lot of the procedures of machine learning and AI are mathematically suspicious but people use it anyway because it empirically works. People in the field have told me that yeah, they expect mathematicians will someday get around to backing it up, but for now, practitioners are moving forward with what they have.
More mathematical rigour would be required for formal papers, though it wouldn’t need to be derived from first principles.
I just love to see this, particularly the response when AI then goes on to do it.
It is very good at understanding research goals, identifying appropriate datasets to work with, doing data wrangling and data science, determining appropriate metrics, running simulations, analyzing results, writing up reports with charts and tables, and proposing follow ups or “high potential next steps” which show a surprising amount of judgment, and if instructed to, executing on those too.
I use Astra and Fable (who delegate to cheaper models for coding) and have them review each others work, and only spot check things when they are done calling each other out. I rarely find issues, and when I do, it’s usually a misunderstanding of intentions.
I’ve only explored a few areas though, including computer vision, vector search, and wireless signal processing, all in applications that, as far as I can tell, are genuinely novel (though of varying levels of novelty and economic value, as the models are fond of pointing out to me!)
I prefer being in the loop and setting the direction — because I do not have unlimited tokens, hah! —- and I regularly identify new directions they had not considered or had prematurely dismissed… so I am a bit more useful than just a financier of tokens! But I can totally see how they could grinding ahead by themselves and brute force useful results, especially if given very concrete goals like a theorem to prove.
I don’t think we’re at Recursive Self Improvement yet, but I don’t doubt for a minute that this is behind the insane pace of development at Anthropic and OpenAI.
My perception is that most data collection, once designed (which does take considerable thought, as you point out), is a slog of executing rather mechanical processes over a long time. (Which is what RA’s and lab assistants are for!) For sure, regular analysis is needed to ensure the right data is being collected and no confounding factors are sneaking in, but seems to me the majority of time goes into the relatively rote work of setting up equipment, running experiments and recording the results.
I do think that with access to MCP-enabled equipment, AI would be ideal for this too, given its infinite patience. The problem, as with all other fields, is this would rob novices the opportunity of building the necessary intuition that comes from spending time in the weeds.
No, it is intellectual. Designing and implementing an experiment that eliminates confounding variables requires a great deal of creativity and ingenuity. Reality is messy and complicated.
Some notable examples include: Michelson-Morley, Millikan, Wrights, Cas-9 for CRISPR, CERN (one giant experiment).
Sadly, no. There are surely exceptions, but most data collection for experimental research is highly skilled and intellectually demanding.
The good RAs are the ones that could have aced a PhD if they'd wanted to (there's one in my lab, and they're worth their weight in gold) -- but they are rare. There's a reason that the majority of research effort in the sciences comes from PhD students and postdocs, not from RAs. This stuff is not simple, and definitely not rote.
Then I had friends studying biotech who seemed to spend hours in labs slicing up flies or preparing cultures for experimental assays.
They were all definitely very smart and hard-working, and the science was extremely deep, but setting up and carrying out the experiments seemed to be the most time-consuming aspect, which may have skewed my perceptions.
To take one of your examples -- dissecting Drosophila is an expert task and requires extremely fine motor skills (you are pulling apart soft tissues 100-200 microns in diameter), and also requires the ability to recognise and isolate different tissues visually. You're not going to ever get a robot to do this any time soon.
And that's just a tiny part of the experiment -- before that, there was thinking through the crosses; the fly husbandry; the sorting ... and after dissection, there's the fixation and antibody labelling; mounting and imaging (an art in itself).
We don't really give these skills enough credit, I agree. But the only reason why a high-performing biomed lab can do what it does, is because it's filled with extremely smart and capable individuals.
(Yes, not everything is as highly skilled as tissue dissection, and some repetitive molecular biology can now be handled with robotics and microfluidics quite well. But there's a lot that still can't be automated. I also wouldn't dream of letting an LLM near any of this -- a hallucination that causes a bug in code is one thing; a hallucination in a biomed lab facility could kill people.)
Context is king.
Because they want you to think that you will be able to do the same, too.
Except, you don't have a dedicated cluster for a week with an unrestricted model.
It was a comforting moment, for sure. But ephemeral, self-imposed ignorance is no bliss at all :(
"Neither result describes our own universe directly, and neither has practical applications, but both contribute to a long-running theoretical effort to understand how quantum mechanics and relativity fit together."
The first result is cool (but basically a complex error-prone calculation), the second is much much cooler, Claude proved the upper limit for the mass of a black-hole which is consistent with a given central charge. (thereby making it a consistent theory of quantum gravity), any theory predicting above this is not a theory of quantum gravity of that universe. This is very exciting!
https://nautil.us/what-does-any-of-this-have-to-do-with-phys...
Most who study physics at this level, especially theoretical physics, end up moving on to something else quantitative. Most of them do quit on some level: there's usually a way to keep going, and usually life's circumstances mean it's no longer worth it.
But very few actually regret the time they spent in it, from what I've seen.
The beginning (author is in an occasionally stressful job):
My girlfriend asked me [have you ever been happy?], after work over drinks at some shiny Manhattan bar, after another stressful day on the trading floor.
The conclusion interspersed with the author contemplating "what if they had pursued an academic career as a theoretical physicist" vacationing all around the US: [Prof Rajeev] ticked through a catalog of his contemporaries who'd succeeded in theoretical physics even without having the towering mathematical intellect (that Rajeev surely has) ... by focusing on problems that played to their strengths, or by taking advantage of computers, or by collaborating with peers who had complementary skills.
... I realized [Rajeev] ... sounded a lot like me, the many times I'd been asked by young strivers how to break into Wall Street or how to succeed once there ... as I gave them such practical advice and stressed how much work it took ... [many] seemed to want some secret that wasn't mine to give.
... Gradually, I heard what Rajeev was saying ... 'You did not fail, Bob. You quit' ... But Rajeev wasn't finished, "The hardest part of all is controlling your emotions." ... Was it actually as stressful for him as it had been for me? Was my failure to follow through on physics more about emotions than math?
Rajeev had concluded our last conversation by conjecturing ... that theorists like him stay the stressful course of wandering in the dark and guessing which questions to ask is because, "They must really want to know the answers."
---See also: Relentlessly Resourceful, https://www.paulgraham.com/relres.html and Bus ticket theory of genius, https://paulgraham.com/genius.html
https://news.ycombinator.com/threads?id=jeffwass&next=264747...
(another fellow PhD physicist, who went into this with all the idealism instilled by Marie Curie's biography and others, boy, was that shattered by reality)
AI cannot think in 3D. Yet. /s
Careful. Thereby making a not-inconsistent-via-bootstrap theory. Such a theory could be inconsistent for other reasons shown by other methods. The article expresses a desire to show inconsistency for M1 above c/12 for example, but it's now been shown that the boostrap method can't get you that far.
Even through after all my skepticism about AI in science, Anthropic proved they can search for and find non-trivial discoveries with their tech, they've done very little to dissuade my fear they are actively hurting the scientific community for PR.
We could treat AI as just a helpful fuzzer for open-problems that occasionally opens new interesting paths for the actual scientists, but we know the end-goal of these people is a future where there are no qualified people left to review, let alone make new discoveries independently from the machine.
"Claude did it" doesn't cut it. What's Claude anyway? Some client app talking to Anthropic API's? That's just shuttling prompts and output back & forth.
Researchers x, y or z doesn't cover it either. In this case they'll have relied on Claude for some heavy lifting.
Suggestion: name all the parts involved. "This research was done by Alice, Bob & Carol, using Claude with compute provided by Anthropic" - or something along those lines.
Credits given, clear who did what, no implicit assumptions.
Without the prompts, which seem like they might be needed to reproduce the results, I am not sure how we can ever know.
And then there is this cryptic bit on the company web page article: "Weeks after one of the agents reached a solution, a physicist, who had independently used Claude to tackle the same problem, shared his result, and a colleague realized that Hongbin’s agent had reached the same answer earlier. The two solutions used different methods, so they were a useful check on each other."
I have questions: Who was the other mystery physicist?
Does that mean there were at least 2 humans involved in the research?
Or were they not involved enough to even understand the abstract, in which case it seems like academic fraud to include any human as an author?
Is that marketing spin, lack of confidence, fear of blowback? IDK, but interesting to think about an academic world where that becomes the case
Representatives of AI labs and other organizations may email admin@hexagonmath.org to arrange a verified lab account.
https://hexagonmath.org/organizations/anthropic
Also: The main result was first obtained autonomously by an internal Claude research model. The publicly available Claude Opus 5.5 later obtained the same result independently. Claude then wrote this paper, following brief guidance from a physicist, who also checked every main result.
Wild.
Next up, the Origins of the Universe... stay tuned for our Fable e^2. More on the 10 o'clock news...
/s
Stopped reading right here. It took one. Maybe you don't know which one it took but things can't be in two places at once.
If you don't, it could have taken either route.
If you observed route A you either saw it, or saw nothing meaning it took route B.
Of course it would have taken route A until you decided to observe that route. Then it took route B.
What does it mean for Claude to solve it? What was the prompt? I didn't read it all, but the word "prompt" does not appear in the blog post, either abstract, or either paper.
When my accountant whips out a calculator, it is the accountant that defined the problem and set the solution in motion. Fantastic tools are still tools. Claude does not love me nor you no matter how its trained and encoded patterns express.
And you know the details how?
What does it mean for Jenny to spin it? Who provided the power to it? I didn't read it all, but the word "power" does not appear in the patent, or the newspaper articles about it.
To pick yet another example: you _can_ say "the car drove from California to Florida", but that'd an unusual stylistic choice and so would probably imply that you didn't know for sure who drove it or something. In ordinary conversation people would be much more likely to say "Jenny drove from California to Florida [in her car]".
Is the boiled/boiler thing related or just some other example? That isn't analogous either. The analog would be "the pressurized gases escaped the vessel" vs. "the vessel failed to contain the pressurized gases". These are different. If you want to emphasize how out-of-spec the internal conditions were, you say "the pressured gases escaped the vessel". If you want to emphasize that the chosen vessel was obviously wrong for the job, you say "the vessel failed to contain the pressurized gases". Either way though there's no risk of the reader anthropomorphizing the pressurized gases. That's different from "the agents escaped the sandbox" because that implies ... agency.
Sorry if I've misunderstood!
And "the boiler exploded" is quite straightforward, not problematic at all. "we failed to build an adequate boiler" is an extremely dumb way to write it, says something very different, and may not be true at all.
And I would note that people solve things but they don't explode, so calling that anthropomorphizing is way beyond absurd. Absurd (but not way beyond it) is saying that claiming that Claude solved it is anthropomorphizing -- that's a fallacy of affirmation of the consequent. Just because humans solve problems, that doesn't mean that anything that solves problems is human. I started programming over 60 years ago and some of the first programs I wrote solved problems. I didn't solve them, it would have been way too much work and would take too long ... that's why I wrote the programs. I solved the metaproblem of designing a program that could solve my problem. Now we have moved beyond that and have systems that solve the metaproblem. For some rather thick and ignorant people, saying that is problematic, but there's no reason that it should be.
Another program that I wrote in my early days was a random sentence generator. Because I was a college student I chose a rather risqué vocabulary. I printed out about 1/2 inch of fanfold paper with the sentences and left it on a table in the UCLA Computer Club, and it produced howls of laughter, because many of the sentences were very funny. But claiming that the program or the computer on which it ran had a sense of humor would be improper anthropomorphizing.
Philosopher Daniel Dennett developed a framework called "the intentional stance" that people who whine about saying that Claude solves problems would do well to learn about: https://en.wikipedia.org/wiki/Intentional_stance
> What was the prompt?
Find open physics problems and solve them, basically.
By the persons who trained the model.
By the persons who provided the data the model was trained on.
etc.
"We failed to build an adequate sandbox, which is why it was able to break out."
Not problematic at all.
I don't know why it's so hard for some people to understand that it is possible to build autonomous devices that gather information and make decisions based on it ... but not understanding this is dangerous.
Answering myself: yes, one of the problems was solved using Lean. The other was undisclosed. Nothing new under the sun, huh.
For any of the big things I've worked on, it isn't one "prompt", it's a goal that keeps getting updated and a bunch of back-and-forth steering of the model and sub agents, where what the human is saying to guide the model is heavily context dependent on what the model is currently doing or saying. Much of that guidance will ultimately be useless guidance on dead-end explorations. Also, the model will likely have written huge programs to solve things symbolically or simulate things numerically, and both the programs that were written, and the output of those programs are critical to understanding how the model came up with the things that led to a solution.
Now, I'd be all for publishing the full transcript, but the transcript is likely to be massive. Gigabytes of text, and reproducing the local computation (often sympy or numpy) may be hundreds or thousands of CPU-hours.
I wonder how this works out in other parts of physics codes. If you need a Monte Carlo integrator, is it common to guarantee a seed for reproducibility? How do these codes handle distributed systems for rng? Is it common to use a fully reproducible distributed rng and GPU kernel for scientific compute?
But it isn’t “just a prompt”, and sharing it doesn’t make the results reproducible (since you don’t get control over the model’s “seed” from run to run).
:)
That talk did make enough sense.
I'm still waiting to see a "picture" of a black hole that didn't involve a "pre-trained model" developed with "synthetic data" or something like that.
I'm not feeling sufficiently persuaded that I can feel confident that gravitons are a thing.
Nonetheless, I am remaining open minded to this black hole stuff.
For example, the metaphor they give about walking from Penn Station to the High Line comes off as somewhat out of touch.
nowadays i open the comments first and often dont open the article at all
i dont think this is a good thing, but i do find it interesting. I think a big part of it is how often i would read an article and develop a bunch of thougths about it and then come into the comments and see someone correct a fundamental premise of the article rendering all of my thoughts kind of useless as they are all based on misunderstanding and so i would start checking the comments to see if the article had been 'overruled' so to speak before i waste my time on it. this seems to have slowly shifted me to not knowing if the articles are even worth reading to begin with
The German software giant SAP was jokingly dubbed "Sammelstelle arbeitsloser Physiker": agency for unemployed physicists.
The article is still over my head though.
Never heard that, but it explains some UI and UX choices.
I'm not sure if that's sampling bias, but it's not true of the Dutch or Belgian or French people I have met.
As far as I can tell, the entire German economy is an agency for unemployed physicists.
edit: To be clear I don't mean this as an insult, it's a site with millions of people, and the overlap between "working with software" and "understanding Cauchy surfaces" is pretty damned small.
And that’s being generous.
https://news.ycombinator.com/item?id=37123792
The hn comment guidelines say don’t complain hn is turning into Reddit, but if dang and co. still believe that’s an “illusion”, they’re not doing much to combat it.
There are numerous classes of things, such as reimplementations of utilities that already work quite well, on some convoluted protocol that doesn't seem to actually address any of the standing issues, which get rocketed to the FP every time, it's the same people spewing koolaid in the comments every time. You can't even ask what problems it actually addresses without getting attacked. Very strange
They actually encourage it with their authoritarian enforcement of decorum (you're not allowed to criticize people for being wrong and etc). The result is everything turns into exultant startup-y bullshit.
So perhaps you are seeing what you’re biased towards, or you didn’t scroll past the first root comment?
i feel solipsistic saying this, but the more i understand about physics, the less awed i am by papers? obviously i am impressed, i know firsthand the amount of thought and calculations that goes into these sorts of results. but i dunno im just not awed. awe implies an incomprehensible greater than thou incredulity, and some of the physics in this paper feels really normal and achievable to me. i think im just too young to properly understand the effort this takes, maybe i should be awed, but im not.
i think i am trying to get at something where, it feels when ai labs put out big press releases on topics, they are making a big show of something only truly interesting to a small portion of people. if a human came up with these same results independently it would yield conference talks and grants, but ai is getting more eyes and a bigger press release just because its ai that solved it and not a human. which is impressive from an ai development standpoint! but less so from a physics standpoint (insert deintensifiers of this statement here im just stupid), so the physics feels like a prop.
still really interesting results though, again, i wish i understood enough of the underlying theories theyre connecting to feel really moved by the results. sorry for being so self centered in my analysis lol, wish i could do better but i cant.
Whilst there is no "world king" to make this decision, every nation will burn all their oil and coal, the world will warm, and people will adapt.
That has always been the future, and climate activists have mostly been wasting their time
https://elements.visualcapitalist.com/what-powered-the-world...
Efficient fuel creation from renewables could massively change this.
Also nice timing around same time as IPO
I think reality is these technologies just do not make money compared to the money they cost, at least not currently.
So they're just trying to shyster their way in as much as possible.
Really cringeworthy to see. I hope it all crumbles instead. They stole everyone's work.
One is that there are a near-infinite number of "open problems" in physics because the surface area of research programs in theoretical physics is extremely large. Before I was in the world of physics, I would have thought that "solving an open problem" meant that physicists around the world would be rejoicing and discussing the result. In reality, other than for the big, field-defining questions, only a small handful of people might care about any problem being solved. That holds true even for the Millennium problems.
Even if I were to read these papers in depth and understand them, without talking to somebody that knows the area and what has been done in it already, I would have no sense of the significance. The way to assess significance is to see how the community responds. Only time will tell whether people care about these results enough to cite them, incorporate them in future work, etc.
And so, like many other people I know, I plan to ignore all of these results until somebody I know comes and tells me to check one out: the cost of assessing any of them is too high, and there are too many other papers to read.
I would also had that in math currently you can verify proofs, like Lean, which also means the tooling itself is a good match for automation, I don't know what's the situation in theoretical physics.
Theoretical physics starts from (incomplete) observation and compatible theories plus targets an open question. There often are many solutions, but the real question is, which one holds up if the broader field is examined?
Consider what happened to the observation of the Higgs boson? There were numerous string theory flavors, all (more or less) consistent with current observations, but predicting different mass for the Higgs. Clear path for elimination of a lot of previously "correct" models.
In that empirical approach, making experiments and comparing to old/current/future work is a necessity to judge a models in a given regime. In fact, it is the differentiating factor in maths and theoretical physics.
Presumably Anthropic wouldn't have chosen to work on these if the problems weren't significant?
This seems carefully timed with their IPO.
Would spend some token on searching existing opening question for something easy, but very few people works on (i.e. the low hanging fruit).
I guess we can easily do thousands of low hanging fruit.
Next, sieve through them, try to find something headline grabbing and bundle them together.
Stop boosting products made by a scam company.
This is a rather creative problem so I think it's a very good AI benchmark.
I think egos do exist and they probably do cause problems here and there but academia has much bigger problems that hold it back.
AI is good at pattern recognition but some problems require a different kind of inference
She used to be informative. Now she just posts intentionally provocative but populist-friendly opinions, like some Rush Limbaugh of physics.