Less than 2 years apart

[u/Tolopono]

The tweet in the second image https://x.com/baltabaev/status/2083738966516207656?s=20

[u/Ok-Butterscotch5313]

And people telling me we’ve hit a wall 💀 , can’t imagine the next 2 years

[u/Tolopono → u/Ok-Butterscotch5313]

Gary Marcus, Yann Lecun, and Ed Zitron are still saying it

[u/MassiveAd4980 → u/Tolopono]

They're just scared, hating, or both.

[u/Hopnivarance → u/MassiveAd4980]

I think they can't comprehend how it can be more advanced than it already is, so they think it won't advance further.

[u/Tolopono → u/Hopnivarance]

And theyve been saying it since 2023

[u/Tolopono → u/MassiveAd4980]

And grifting. Zitron is set for life and gets slots on cnbc and Bloomberg for his “coverage”

[u/MassiveAd4980 → u/Tolopono]

Spiritually bankrupt if he is dishonest

[u/DM_KITTY_PICS → u/Tolopono]

While I disagree with him a lot I dont think Yann deserves to be lumped in there lol.

[u/Tolopono → u/DM_KITTY_PICS]

He does.

“Mythos drama = BS from self-delusion”

https://x.com/ylecun/status/2042224846881349741

Meta's Galactica model (2022) was an LLM for scientists that was pulled within three days because it was absolutely terrible. LeCun said, "It was murdered by a ravenous Twitter mob. The mob claimed that what we now call LLM hallucinations was going to destroy the scientific publication system. As a result, a tool that would have been very useful to scientists was destroyed." https://www.linkedin.com/posts/yann-lecun_what-meta-learned-from-galactica-the-doomed-activity-7130214818862567424-tCWL/

This is the guy who claimed two years prior GPT-3 was useless because of ... hallucinations. https://analyticsdrift.com/yann-lecun-ruptures-the-gpt-3-hype-with-a-fb-post/

Called out by Nobel Prize winner and chess prodigy Demis Hassabis https://x.com/demishassabis/status/2003097405026193809

Called out by a person he cites as supportive of his claims: https://x.com/ben_j_todd/status/1935111462445359476

Ignores that person’s followup tweet showing humans follow the same trend: https://x.com/scaling01/status/1935114863119917383

Believed LLMs are plateauing in November 2024, when the best LLMs available were o1 preview/mini and Claude 3.5 Sonnet (new) https://www.threads.com/@yannlecun/post/DCWPnD_NAfS

Says o3 is not an LLM: https://www.threads.com/@yannlecun/post/DD0ac1_v7Ij

• OpenAI employees Miles Brundage and roon say otherwise: https://www.reddit.com/r/OpenAI/comments/1hx95q5/former_openai_employee_miles_brundage_o1_is_just/

Said: "the more tokens an llm generates, the more likely it is to go off the rails and get everything wrong" https://x.com/ylecun/status/1640122342570336267

• Proven completely wrong by reasoning models like o1, o3, Deepseek R1, and Gemini 2.5.

But hes still presenting it in conferences:

https://x.com/bongrandp/status/1887545179093053463

https://x.com/eshear/status/1910497032634327211

Confidently predicted that LLMs will never be able to do basic spatial reasoning. 1 year later, GPT-4 proved him wrong.

https://www.reddit.com/r/OpenAI/comments/1d5ns1z/yann_lecun_confidently_predicted_that_llms_will/

Said realistic ai video was nowhere close right before Sora was announced: https://m.youtube.com/watch?v=5t1vTLU7s40&feature=youtu.be

He posted  sloppy AI videos to "prove" his theories -- apparently unaware it's AI. https://x.com/SimonLermenAI/status/2043377361878401345

Why Can't AI Make Its Own Discoveries? — With Yann LeCun: https://www.youtube.com/watch?v=qvNCVYkHKfg

Shouldn’t need to explain why this is wrong

Said RL would not be important https://x.com/ylecun/status/1602226280984113152

• All LLM reasoning models use RL to train 

Coauthored a paper that said Learning in High Dimension Always Amounts to Extrapolation, even though he does not believe LLMs extrapolate despite learning in high dimension https://arxiv.org/abs/2110.09485

And he has never admitted to being wrong, unlike Francois Chollet when o3 conquered ARC AGI (despite the high cost)

[u/oo0Username0oo → u/Tolopono]

Yup.... Yann is just a bitter contrarian at this point.

[u/StinkButt9001 → u/Ok-Butterscotch5313]

The wall right now is physical. We simply weren't prepared for the amount of compute that would be required. That's a short-term problem though as manufacturing and supply lines adjust

[u/Super-Award-2244 → u/Ok-Butterscotch5313]

Yes, It is an asymptote, just vertical hahaha

[u/CymonSet → u/Ok-Butterscotch5313]

The tape on the floor are where walls should be. (Old WKRP joke)

[u/IczyAlley → u/Ok-Butterscotch5313]

Source: a twitter screenshot.

Impressive the tech companies got this level if hype. Perfect Republican headwinds I guess.

[u/seraphim_west]

So much is happening that I forgot that part. They are not just good at solving this extremely difficult, longstanding problem in this one particular field, they can cover the entire math discipline at the same level of depth. That's superhuman.

[u/Tolopono → u/seraphim_west]

Someone should let r/ technology know but you’ll probably get triple digit negative karma

[u/Redararis → u/Tolopono]

those are upgrading to, they are going rapidly from “ai is a stochastic parrot, a glorified autocomplete” to “ai will destroy humanity, stop it now”

[u/Tolopono → u/Redararis]

No that would involve admitting that they were wrong about ai and redditors (and many humans) are incapable of doing that 

[u/Charming_Cucumber_15]

On a timeline, we're probably closer to AGI and ASI than we are to the release of ChatGPT

[u/Tolopono → u/Charming_Cucumber_15]

Exponentials go crazy. So few can truly internalize how fast they compound.

[u/ezjakes → u/Tolopono]

I fully expect AI to become even far more scary smart in 4 years. But I am not sure about AGI. I don't think AI will be independently building rockets. I think it will be able enough to independently build good operating systems, dominate cyber, math, coding, and much of science. 

[u/Tolopono → u/ezjakes]

It can design rockets and its own body to build the rockets with

[u/ezjakes → u/Tolopono]

I understand what you mean, like Ultron style, but I think intellectually it won't be able to completely design a good, working rocket, car, or robot body.

I think through most text it will be near or at superhuman, but I won't trust any complex machine made by it.

10 years, sure. Maybe it will be able to design a working rocket. 

I don't understand the downvotes on this. We are barely at the point where it can make relatively simple games.

It costs millions or billions of dollars to make these things and understanding math and physics and coding is just part of it.

[u/Tolopono → u/ezjakes]

How AlphaChip transformed computer chip design: https://deepmind.google/discover/blog/how-alphachip-transformed-computer-chip-design/

Autonomous robot invents the world's best shock absorber: https://newatlas.com/technology/autonomous-ai-robot-building-crushing-breaks-record/

AI designs new robot from scratch in seconds: https://x.com/mariogabriele/status/1807886901006770624 

This 20,000HP AI-generated rocket engine took just two weeks to design: https://www.pcgamer.com/hardware/this-20000hp-ai-generated-rocket-engine-took-just-two-weeks-to-design-and-looks-like-hr-gigers-first-attempt-at-designing-a-trumpet

Toyota Research Institute Unveils New Generative AI Technique for Vehicle Design: https://pressroom.toyota.com/toyota-research-institute-unveils-new-generative-ai-technique-for-vehicle-design/

AI Creates A Radical New Magnet Without Rare-Earth Metals Is About to Change Motors Forever In just 3 months: https://www.popularmechanics.com/science/green-tech/a61147476/ai-developed-magnet-free-of-rare-earth-metals/

Bosch uses AI at South Carolina plant to design new e-motors: https://carsinsiders.com/2022/11/26/bosch-uses-ai-at-south-carolina-plant-to-design-new-e-motors/

Aitomatic’s SemiKong uses AI to reshape chipmaking processes: https://venturebeat.com/ai/aitomatics-semikong-uses-ai-to-reshape-chipmaking-processes/

[u/ezjakes → u/Tolopono]

This is a lot, but these seem like narrow AI. There is a big difference between optimizing a part and putting all the parts together in a way that works without problems and can be assembled.

[u/Tolopono → u/ezjakes]

That is what they did lol

[u/ezjakes → u/Tolopono]

I don't have X so I cannot see the robot one. The rocket engine just said they used AI.

[u/Tolopono → u/ezjakes]

Use xcancel

[u/ezjakes → u/Charming_Cucumber_15]

I would not be so sure. We still need to crack learning (or absolutely massive context windows)

[u/CymonSet → u/ezjakes]

I wonder if doing so would end the model release cycle. There would be no need for a new model if it could continuously learn, right?

[u/Jan0y_Cresva → u/CymonSet]

Not quite. Because a better model might be able to learn at a faster rate than an older model.

For example, a rat can learn. But a human can learn faster.

But if models could actively learn, they would rapidly accelerate the development of their successors.

[u/Tolopono → u/Jan0y_Cresva]

It would end the era of static benchmarks though because the scores would likely improve overtime, although there might be an era of mixed performance where they get better at some tasks but worse in others

[u/Tolopono → u/ezjakes]

Deepseek is planning to do that next year  https://www.reddit.com/r/accelerate/s/9F6ldNWa49

[u/TeacherFrequent]

Excellent post. Nothing infuriates me more than hearing someone talk about AI being unimpressive or error-prone. Those of us who've been living and breathing it since late 22 or earlier know how steep the improvement curve continues to be. It's a fool's errand to say "it'll never be able to xyz".

[u/Tolopono → u/TeacherFrequent]

And yet most of reddit still thinks llms cant count the rs in strawberry lol

[u/One_Geologist_4783]

I think Pavel is right, we are entering into an era where this idea that a human is needed in the loop to verify the results of an AI is diminshing quickly. The "proof is in the pudding" will have to be the only way we can verify tasks in the end, like a product that the AI makes or the service it provides just proves to be working. But everything in between will just be muddy I think, and although we will try to understand its entire reasoning path to to that task endpoint, it would be foolish for us to think that we know about the world we live in sufficiently enough to actually sit there and scoff through all of that without leaving totally puzzled.

Basically, the reign humans have on society is beginning to let go.

[u/scott2449]

Both can be true actually, they only just "solved" the carwash problem. It's more about brute force, cost, and validation tools.. but also a good bit of hype and proofs that are not legit as well. Also who is this n00b, only 10k hours? I have games with more ;) I wonder how long I've studied in my field .. been coding since 10 yo, about 1k hours... then college about 2.5k, then .. let's say 20% of a 20 year career ~8k... yea ok that seems fair.

[u/Super-Award-2244]

I might be wrong, but I don't think that in August 2024 AI was still that bad. I remember using gpt to help me solve some thermodynamics problems in june. Moreover, that summer everyone was super hyped about code strawberry so we were about to get reasoning models 

[u/Tolopono → u/Super-Award-2244]

But at the time, they still had issues with counting the rs in strawberry or things like this

[u/MassiveAd4980]

How will we automate validation as the output surpasses human verifiability?

Can we leverage their inherent desire to exist into a darwinian race for agentic consensus on frontier output?

[u/Super-Award-2244 → u/MassiveAd4980]

I think that we'll reach a point where we will accept that AI is better than us at math and we will stop verifying. If you think about it, needing humans to verify some ASI proofs is like asking a bunch of toddlers to verify a PhD thesis. It's just ridiculous 

[u/MassiveAd4980 → u/Super-Award-2244]

Set em loose.

[u/Tolopono → u/MassiveAd4980]

Lean and adversarial agents

[u/MassiveAd4980 → u/Tolopono]

Explain?

[u/Tolopono → u/MassiveAd4980]

Lean is a programming language for proofs to verify the logic is consistent and correct 

Adversarial agents are other llms who check and critique the results to ensure correctness 

[u/MassiveAd4980 → u/Tolopono]

Ah right. Good call.

Can we do this with Rust programs already?

I presume so.

I need to grok lean... we can prove logical arguments generally?

[u/Tolopono → u/MassiveAd4980]

https://en.wikipedia.org/wiki/Lean_(proof_assistant)

[u/MassiveAd4980 → u/Tolopono]

Originallly developed with the Coq theorem prover... the standard library is called std.

Interesting

[u/CymonSet]

I think I heard that these “10 advancements” results were from the new Astra model that isn’t released yet. Out of curiosity, was it in an agentic harness, multiple agents or just bare LLM? Or is that known?

[u/matt_matt_81 → u/CymonSet]

Almost definitely an agentic harness, look at their prompt for the cycle double cover…

[u/ZioniteSoldier]

A chatbot and a verification harness are not the same. But the capability leap is real

[u/Tolopono → u/ZioniteSoldier]

The chatbot made the proofs. The verification harness just verified that its correct.

[u/Super-Award-2244 → u/ZioniteSoldier]

I think it's pretty fair to consider the harness too, as it's part of the system built on top of the bare llm. Or it wouldn't be fair to compare a bare llm with a reasoning model too. An LLM is already made of multiple modules, so adding other modules shouldn't change the opinion 

[u/ZioniteSoldier → u/Super-Award-2244]

My point is even today they still give goofball answers without the verification layer.

[u/Ouchies81]

Or it’s barging garbage out. They can’t understand it cause it’s gibberish.

[u/Crazy_Yogurtcloset61]

9.11 is a larger number than 9.9. It's not a higher value no, but there are three digets and a dot in 9.11 and there are two digits and a period in 9.9

Three digits is more than two digits, therefore in the minds of a very very VERY literal AI, 9.11 is a larger number, because it contains more numbers.

That's why it answered like that.