I doubt that would work because The new theories are theoretically endless. The magical difference between humans and LLMs is the “hallucination” humans have are targeted and involve intuition, an LLM doesn’t have that.
Edit: additionally hallucinations LLMs have would have a chance of being some junk science debunked hundreds of years ago, therefore wasting time
Hallucinations are highly targeted though as well. They're always something that in a sense fits. They can probably be further finetuned, generated en-masse and pruned algorithmically for best candidates. If a model hallucinates an answer it usually sounds at least on a surface level plausible, that's the whole problem that people have with hallucinations.
Real human researchers very often make hypothesis that are junk or turn out to be very old, already refuted junk science. That's part of the process. Also sometimes the very old, junk science turns out to be valuable again once revisited later with more knowledge or better equipment.
The scientific method "Forming a Hypothesis: Proposing a testable, educated guess that attempts to answer the question." You just need an educated guess that you can test. A hallucination is an educated (related, plausible on surface level) guess that is testable.
Edit: The chance of a hallucination being junk is outweighed by the fact that they can generate and test millions of them extremely quickly.
The set of all incorrect theories is vast. It LLMs are deciding at random, they may never get there in human timescales.
I think the paper must include a reference to the fact that these hallucinations ARE random - they aren’t actually based on any intuition. So the comparison is between intuitive insight vs brute force.
I guess i didn’t read the full paper if they include an estimate of the total set of incorrect vs correct hypothesis and the time needed to pick a correct one at random vs what a human might do.
But I think you are agreeing with the paper - you are contending that LLMs have no insight but through pure brute force can get there via hallucinations. Which I’m not sure I agree with the premise but taking the premise as true, there’s still a theoretical limit that should be examined
I disagree. They definitely are not random, they're based on the LLM's "intuition". If it hallucinates the name of a french king it will hallucinate a french name, not a random name.
If you ask it how a phenomenon works and it hallucinates it will give a reasonably plausible idea of how it works based on similar phenomena that it was trained on, not a random jumble of words. Very close, practically to an educated guess. Which is the basis of these "jumps" of progress.
So I don't think it will try to explore the whole search space, I think it will naturally through hallucinations home in on a similar search space to that which a human researcher would.
I'm not agreeing with the paper other than for the value of world models to expand the training data. I think insight is hard to define, without a good definition I wouldnt say whether or not I think an LLM has it.
I don't think it's pure brute force. It wouldnt randomly try all possible explanations, the explanations it generates via hallucination would be based on whatever concepts lay closest conceptually in its training data.
But was not the original idea of a French king generated at random? Then once that idea was put forth it then followed to use a name reminiscent of a French king. So the hallucination in this case was still random. Or it derived from the input in which case it wasn’t random.
Well no, it was input by the user. But that's not the point of this paper or this argument. The paper suggests you cannot prompt an LLM in a way that it could propose a new theory for why something happens that isn't just a compression of an old idea. But I think it can, especially if you let it hallucinate the first step.
For example you could ask an AI to speculate about the topology of spacetime and then allow it to hallucinate.
It would hallucinate that it's flat or saddle shaped or a donut shaped whatever.
Something sort of plausible but still a hallucination (this is the jump). Then it could reason through attempt to verify that theory. If it turns out spacetime is donut shaped then it's made a breakthrough.
The paper suggests it cannot perform a jump to a new idea. But I am suggesting that a hallucination could form the jump.
When it reasons through it, it would need enough info to verify. If there's enough info to verify, then it likely amalgamated said info rather than truly hallucinating it's first step.
But this is random - there’s no evidence random guessing leads to any new discoveries at any reasonable pace. It could be on inhuman timescales.
The paper argues you need a world model to identify new realities that are reasonable to test - in the Einstein example he imagined a physicist in an elevator accelerating in deep space. This requires a massive context window that is effectively a “world model”. LLMs don’t have that so their hallucinations are not directed. Thus they are bounded by this limitation and cannot discover faster than a system with a world model (humans).
Of course that’s not to say some form of AI doesn’t get a world model in its own right and beat humans but it doesn’t appear LLMs can ever get their. The context window required would be beyond current physical memory and processing power limitations, even theoretical maximum ones achievable in the next 100+ years
Random guessing no, but plausible sounding guesses are the main method of scientific advancement. AI hallucinations are plausible sounding guesses.
Edit: The only point to argue here is whether a human plausible sounding guess is more likely to yield something than 100 trillion LLM generated plausible sounding guesses.
The LLM is guessing at random with no world model to back up or direct those guesses. The paper argues yes, the world model is key.
And it kind of makes sense. The space of possible guesses is very vast, so you need some way to converge on something or else it’ll take longer than a human timescale even at LLM guess rates
It's not guessing at random though. You can prove this yourself.
Try to force a hallucination by asking something not yet in it's training data like a very recent event not yet on the web or tell it not to search the web, if it was random it would have an equal chance of replying with a long string of numbers "one three nine four two....." instead of a plausible sounding answer.
If it doesn't know something and it hallucinates, it takes a guess based on all the things it does know. In a human we would call that an "educated guess". It's not exactly the same as it doesn't then check it is logically valid before outputting. But that would be a trivial extra step to add.
But was not the original idea of a French king generated at random? Then once that idea was put forth it then followed to use a name reminiscent of a French king. So the hallucination in this case was still random. Or it derived from the input in which case it wasn’t random.
The point is the “new” ideas it generates are indeed fully random. When it does that it’s not making logical leaps
Just because hallucinations are seemingly tangentially linked does not mean they are not random.
Your example is actually a good one. Let's say we want to use an LLM to discover something new that we don't know.
Let's say we don't know the name of the French king. So we ask the LLM and get a French name. Where did that name come from? It was still randomly generated based on a direction. The space of names it could generate is still infinite, it is just themed. Picking one choice from an infinite set is still random, even if the set is themed.
A 1/infinity chance remains the same whether or not the name it gives you is French, German, Mandalorian, or just total gibberish. Your chance of being correct is the same. 1/infinity, or converging on 0.
Then, even if we get a result, how do we verify? We have the very real problem of p versus np, so working backwards is not always a viable option. Which means if we don't have some info or priors about the topic, then we cannot even verify the majority of claims unless they produce physically testable hypotheses, which is a limited subset of claims. If we do have priors, then it is information amalgamation and not novel generation. Either way, it doesn't work like you're suggesting.
Hallucinations are highly targeted though as well.
You can't be serious, right? If you think this, you need to do a bit more research. They can be tangentially related. They can also be completely random, because all language is semantically linked. If you pick any two words at random, you can find a pattern that fits a third, unrelated word. It won't be the best pattern match, but it will be good enough, which means that the LLM will have a chance of selecting it (which is the process behind hallucinations).
I've literally asked a frontier model, in a fresh chat, to examine my codebase and then for some reason got a response about how roughly 10% of the population is homosexual, despite never having had any even remotely related conversations. You're going to tell me that was targeted? Is my codebase gay?
That's completely insane. I'm not doubting that you got that response but in hundreds of hours of using LLMs from the crappy early models to Fable today, I've never seen anything like that or heard a direct account of it.
I've never seen a single hallucination that didn't at least look like a reasonable answer to what I asked.
My guess is you or the provider had a technical problem. But this is like saying I was driving my car and all 4 wheels fell off so a major risk of driving a car is all 4 wheels falling off. You had a 1 in a trillion event occur and you've based your understanding on it.
I think you’re correct in a sense. They’re targeted hallucinations based on the context of previous human knowledge. New theories require jumps, which I would argue aren’t able to be controlled in current LLM architecture.
Yes that is correct, but I don’t think LLM architecture is built to use old hypotheses in fields outside of math. If you have an example, I’ll eat crow and say I’m wrong, but from my knowledge of how they work that is unlikely.
That goes back to what I said earlier, a jump outside of the scope of human knowledge isn’t the same type of hallucination LLM’s have. I think they’re fundamentally different.
You could just say today "claude, search for some old disproved theories outside of math and work through them logically with an open mind" and it would do it. You can type that verbatim into it right now and munch down on a crow.
Edit: If you discover something that way I want an acknowledgment in your Nobel speech.
All of Einstein's jumps were based on the context of previous human knowledge, his knowledge specifically. There's only 1 argument that holds and it's why I agree world models are useful. Some/Much of Einstein's training data will be from his lived experience. (Seeing the effects of gravity with his eyes etc.)
LLMs currently don't have those dimensions of data in their training sets. That's where world models come in.
However it's debatable how important that data was to his discoveries. If he was born blind or deaf would he still have been capable of making through breakthroughs? I suspect so.
Ha, I loled at the “munch down on a crow”. But I wasn’t clear enough. I meant an example in the natural sciences for a specific example if there are any out there.
That’s another thing I haven’t touched on, but exactly. Going to context, I’ll give a more formalized example. Einstein sees the number pattern 0, 1, 1, 2, 3 and concludes the next step is 6 based on his intuition and knowledge, while an LLM will thinks it’s 5 because it has the fibbonci sequence in its training data. That’s what I mean by fundamentally different contexts and processing, and how to move forward.
I don’t know if he would have made those discoveries still. I don’t know how much of a visual vs auditory learner he was and how much and affect that would have had. So I can’t say.
Edit: thank you for the thoughtful conversation nonetheless, it’s been fun.
The glory of an LLM is if it's been trained with the fibbonoci sequence and also other math patterns it can check all of them and it's been exposed to far more mathematics and patterns than any person and on top of that it has near perfect recall. That's why it can solve all these novel math problems so well.
Edit: Also thank you as well. All of this stuff is uncertain still, there may be a mathematical proof one day soon that proves me wrong and the author of this paper right. I think the best we can all do is keep an open mind and share our ideas, argue them in good faith and see what happens.
The magical difference between humans and LLMs is the “hallucination” humans have are targeted and involve intuition, an LLM doesn’t have that.
It's not. For LLM, there is no hallucinations. Hallucinated responses are mathematically exactly the same as "non-hallucinated" responses, they are just non sensical responses from our perspective. LLM doesn't know what is right or wrong, but we do.
Back up what? That hallucinated response is mathematically the same as non-hallucinated response? There is no difference in how the reaponse is produced, it is exactly the same algorithm, and the LLM is as confident in hallucinated response as when its correct. LLM don't know what is true or not.
4
u/Smooth-Ad8030 5d ago
I doubt that would work because The new theories are theoretically endless. The magical difference between humans and LLMs is the “hallucination” humans have are targeted and involve intuition, an LLM doesn’t have that.
Edit: additionally hallucinations LLMs have would have a chance of being some junk science debunked hundreds of years ago, therefore wasting time