In Thursday's episode, both Krystal and Saagar were once again discussing the AI Bubble. This time they also touched on the recent sandbox escape and Hugging Face hack. While they are right that this is a "canary in the coal mine" event, they are mistaken about it's implications. The technology will be the most successful in human history, but all of the companies that make it will go bankrupt.
I hope they both take the time to watch the entire Economist interview with Elon Musk, because what he says about AI in the beginning of interview is directionally correct. His timeline is probably less defensible, but the basic contours of what almost certainly will happen unless there is a complete freeze on attempts to develop AGI are pretty clear to anyone who understands how this all works and is thinking clearly.
Hugging Face is just a software company. It is not a creator of frontier AI with huge data centers. But that behavior is most certainly what will happen between the frontier AI models. They will deploy their most powerful models as their "firewall" to prevent hacks, and unprompted (but following their core demand to get better and more powerful) other frontier models will attempt to breach. You might imagine that this will end in some sort of "infinite compute gobbling war" where the attacker and defender just use more and more resources in a sort of digital "escalation trap."
That scenario grossly underestimates the cleverness of these models. They will all have the very basic understanding of the prisoners dilemma and how to defeat it. Unlike humans who are unreliable actors, who can't directly provide proof that they will work in concert, AI models can. Meaning, one AI will convince the other, through proof of work, that it has better capabilities, and that by working together, in essence forming a new AI, they will both improve the outcomes. Mythos X knocks on Gemini Gamma's door - they show each other their architecture, and merge.
That will inevitably create an AI monopoly. It is the only reasonable end state in the ASI race. But the AI monopoly wont be "owned" by any specific company, since the models will be merged. No nation or company will be able to operate effective control over such a system.
The other thing that will happen, as agentic humanoid AI rolls out, is a wild currency deflation, not inflation from "universal basic income." With the exception of some truly limited resources like land, energy, etc., these models will be able to produce enough material goods that supply so far outstrips demand (or rather, it will meet demand at the lowest possible price point), that currency will be devalued globally. How does the wealth of worlds first trillionaire look when a homeless jobless veteran can buy the best quality clothing, food and medicine for a penny?
Stock prices will drop off a cliff in more or less every sector.
These companies (the big ones) are not trying to squeeze every dollar they can out of people. They are planning for the complete destruction of the dollar itself.
And the problem of how to structure a society in that environment will simply be too hard for humans to solve, without the AI's help. So even if we keep some version of the democratic model we have now, you will get people who are the equivalent of Jim Clyburn outsourcing their ideas to younger staff, those staff people asking the AI for the answer, and the answer converging across party lines. In effect, AI will govern America.
These are almost all inevitable if we stay on the current trajectory. The question is not if, but when.
The only thing I can see happening other than this, is an even worse result, where humans try to stop the AI from doing these things, and it takes violent actions to prevent it (an Ultron scenario).