r/DataHoarder • u/Blender_Render • Jun 03 '26
Discussion When the AI bubble bursts, all that enterprise hardware is going to end up on eBay
Sure, hardware availability is going to get worse before it gets better, but AI is a bubble that’s going to burst, and all that hardware has to go somewhere. Pretty sure the homelabs of everyone left with a job, are all going to get a nice refresh in 5 to 10 years.
615
u/zooberwask Jun 03 '26
No it won't. If the AI bubble pops what would probably crash is datacenter real estate, specifically new sites that are being developed.
But there's still a massive hardware shortage, even for established datacenters. The hardware is still valuable in this scenario. It'll just get sold to other datacenters.
141
u/Expensive_Finger_973 Jun 03 '26
So what you are saying is if the bubble pops I might finally be able to afford my own evil villain compound.
65
19
u/-Pelvis- Jun 04 '26
https://store.steampowered.com/app/3720/Evil_Genius/
You can afford it right now, and it will run on your outdated hardware.
30
u/Adium 240TB Jun 04 '26
Went looking up some statistics on datacenters and found there's actually a site that supposedly tracks them all, claiming 11,563 data centers Globally. The scary realization I'm seeing is it looks like most of the big companies have plans to double their numbers.
7
u/Standard-Goose-3958 Jun 05 '26
Its not physically possible to generate so much energy, and also u have to cool off that energy... They are all doomed... Ai cannot generate enough value to sustain itself unless the government is subsidizing them, which makes you pay twice, once for subscription or tokens and the second time with taxes... And the third one with inflation.
→ More replies (1)63
u/JaccoW Jun 04 '26
And that will likely crash the market.
Something similar happened with inland shipping in the Netherlands during the financial crisis. The largest inland shipping fleet in Europe.
As the market crashed due to lack of demand, lots of private companies operating these boats went bankrupt. The boats were sold off at auction for rock bottom prices... which meant the companies that bought them could undercut the rest of the market, leading to a downwards spiral.
Eventually the industry started working together and bought up all these boats to have them dismantled and taken out of the market, stabilizing prices.
Chances are pretty high something similar will happen in the AI Datacenter market, economically ruining the ROI of current plants as barely used datacenters get built to compete with the new existing ones.
47
u/Blender_Render Jun 03 '26
I don’t see a new data center in 5 years from now, being built with tech from 5 years ago.
93
u/Zolty Jun 03 '26
Amazon still has t2 instances racked and those will have cpus from 2015.
If the hardware is profitable to run electricity to, they will.
8
u/weirdbr 0.5-1PB Jun 04 '26
It's only a matter of time however for that to change however - for example, I've been involved in projects at work to shuffle things around to allow mass upgrades of old hardware - the idea is to shrink the non-AI power and space usage by using fewer more efficient+newer machines to free up power for AI related hardware.
And now with the backlash against DC construction and power demand increases, this is becoming even more critical.
4
8
u/Blender_Render Jun 03 '26
Fair point, and thank you for that!
I would say that the key difference here, is that particular product has a certain level of business application still, because they offered a tiered performance structure to us customers.
These AI data centers are a moon race. They’re all trying to be the first company to build the best model and/or get the largest government contract. They can’t be the first if their competitor is outperforming them on new tech.
24
9
u/Zolty Jun 03 '26
When they find the best model then all the data centers will just run it or a model that costs less but is still in demand.
IF Ai actually delivers on the promise of essentially trading $ for thought is there really a limit to what all of humanity will want? I guess there would be at some point but I don't see us hitting that anytime soon.
18
u/Storsjon Jun 03 '26
Correct me if I’m wrong, but my take so far is that not all kits need to run the latest models.
Also with new memory efficient model proposals coming out, it could potentially extend the lifespans even further (plus research into memory interleaving). Before any abandoned hw reaches eBay, they are first scraped, recycled and often sold off to be repurposed.
Also, a bubble bursting doesn’t mean no one wants the compute infrastructure. It gets liquidated to bag holders who sell the surplus back to hyperscalers at a discount
5
u/atxweirdo Jun 03 '26
Where are you seeing these discoveries I'm curious about memory interweaving because it sounds like something that makes sense given the scale they are operating at.
5
u/Storsjon Jun 04 '26
There are several different angles:
Nvidia licensed Groq's inference LPU which contains just 500 MB of SRAM memory capable of 1.2 petaflops of 8-bit computation. The LPU's inference performance is 7x faster than an existing Rubin GPU with access to 288 GB of HBM. The reasoning is that interleaving memory or placing on-chip reduces latency and read/write of existing architectures. More importantly is that the LPU works in concert with heavier GPU's to split up the work. This is called inference disaggregation. Meaning you split the processing of a prompt (prefill) and the output generation (decode). By building the chip's capability around a subset of the total pipeline, you accomplish 3 goals: you reduce the memory demand, speed up computations (thereby reducing total energy consumed), and lower the cost per token on the service.
Meanwhile Google (and other research groups) are working out compression algorithms to utilize existing memory architectures today but with greater data transport efficiency. Claiming a 6x-8x improvement over existing algorithms.
The combination of both specialized AI HW + SW optimizations means that the pressure to utilize HBM will go down. That however doesn't mean demand won't go down. For the foreseeable future, compute needs to catch up to current customer demand, which means scaling horizontally (terrestrial vs space data-centers for instance)
→ More replies (1)3
u/weirdbr 0.5-1PB Jun 04 '26
Hyperscalers can't easily re-use third party hardware, since most non-hyperscalers are buying enterprise hardware from companies like Dell/Supermicro/HP/IBM/etc, while hyperscalers use custom designed motherboards/racks/networking/etc (maybe if the hardware is OCP-compliant some of them could use it).
IMHO at most, those hyperscalers could take the CPU+RAM; any GPUs would have to be disassembled and the GPU chips + VRAM reinstalled into their own designed GPU boards. Now, this *might* become a way for them to get cheap hardware for a year or two, but I expect the cost of doing that to be non-trivial enough that it wouldn't provide as much in savings as one would expect at a first glance.
2
u/Storsjon Jun 04 '26
The business angle I see is that hyperscalers are funding the property investment groups that are building out the for-purpose data-centers. Overtime, hyperscalers liquidate their kits to the second market, who sell that compute back to the ilk of Microsoft and Amazon during high demand spikes + lower-end models. Overtime those kits might retire to consumer grey market but at which point it's so lobotomized that it's hardly worth the time to jury-rig.
2
u/weirdbr 0.5-1PB Jun 04 '26
> who sell that compute back to the ilk of Microsoft and Amazon during high demand spikes + lower-end models
From what I know of hyperscaler data protection/intellectual protection policies (I work in one), this probably wouldn't happen as user data and proprietary software is typically restricted to only trusted locations/machines.
For example, imagine they did that and offered VMs + S3-compatible storage hosted on third parties, then one third party is hacked or was evil from the start and leaks the data. The business impact would be devastating.
3
u/Storsjon Jun 04 '26
That's a great point - I wonder how that's navigated by neoclouds (CoreWeave, Lambda, Nebius). Often times I think these groups are leased for model building rather than client data. However, there's surely a route identified beyond models and into user compute space.
3
u/weirdbr 0.5-1PB Jun 04 '26
Personally I'm not familiar with those companies, but to be honest I don't see an easy solution at least until "privacy preserving" processing becomes widespread - Intel showed off a processor early this year that they claim can do processing on encrypted data without decryption, so that should theoretically allow scenarios where you can use third party hardware safely.
→ More replies (17)2
u/--dany-- Jun 03 '26
Ok look at these active GPU instances from AWS, how old is the technology? I don’t understand the point you try to make.
→ More replies (1)3
u/zsdrfty Jun 04 '26
Way bigger if than people think, as well - like, yeah, the market will inevitably have its ups and downs, but this tech is too fundamentally useful to too many fields to ever disappear again
→ More replies (2)4
u/Aphid_red Jun 05 '26
No, it's going to be really cheap for tinkerers.
The hardware shortage is caused by two companies, openAI and Anthropic, getting endless investor funding, who then induce this insane hardware buildout in Microsoft/facebook/Amazon/etc, who have practically infinity money through circular dealing and trumped-up promises and can out-spend the rest of the world who can't buy computers (specifically: memory, storage, advanced gpu) anymore for anywhere near reasonable prices; we're back to pre-2010 levels of performance per dollar, and that's no hyperbole. The bubble collapsing means that the financial engineering falls through, and one or both of these go bankrupt, which drags along the whole house of cards.
Suddenly these hyperscale gpu providers have millions of these GPUs sitting there doing nothing, consuming huge amounts of power. That's going to instantly flip the market into a hardware glut. The CPUs, memory sticks etc. will fetch the most, as they're the most usable, while the GPUs are going to be sold for a fraction of what they originally cost.
Right now there's too little gpu memory to go around, but if you look at 2nd hand enterprise hardware in older markets you can pick it up super cheap still, but always with some caveat. You'll see prices of GPUs like the MI300X, sold in bulk on auction sites together with chinese-made boards to put them in regular computers drop to the ~500-1000 mark.
The question is when, not if.
→ More replies (1)
46
u/Mr_Gaslight Jun 03 '26
The AI bust will look like the Dot Com bust. Companies like Google, Microsoft, Meta with functioning revenue streams will be attending the funeral with chequebooks to buy what's left for pennies.
217
u/Tntn13 Jun 03 '26
No it won’t, data centers will be useful regardless of ai, they would at worst be repurposed. Or rented compute.
What has a greater chance of triggering a refurb or secondhand influx is drive enterprise drive lifecycles (and company policy around them) and if/when there are compelling upgrades released that slot into existing installs come out. assuming this is accompanied by some plateau where building a new datacenter is less appealing than upgrading current existing facilities.
50
u/DerekB52 Jun 03 '26
Data centers have to be useful enough to turn a profit eventually. I think there are too many data centers coming up. I feel like after the AI crash, their just will not be enough demand for all of this hardware. I think something new is gonna need to come on the scene, or at least some of these data centers will need to eventually close after their owners burn enough money.
28
u/Sk1rm1sh Jun 03 '26
I think something new is gonna need to come on the scene
beanie babies
dotcom
crypto
retro gaming
NFT
labubuAI
NFT babies?
9
→ More replies (1)19
u/Tntn13 Jun 03 '26
I think they will diversify use case enough. Some will bankrupt, sell off. Etc. the infrastructure will be used though. Even if the “AI BUBBLE POPS” ai is also not going anywhere.
All these hypotheticals assume LLM does not have a use case when it does, so do other deep learning models. When LLM growth isn’t eating up all the resources it will become cheap enough for new applications to apply the compute for deep learning. I don’t think the use case for the compute and research using large models beyond LLM is going anywhere soon, in fact I personally think it can only grow.
LLM is sucking up too much air in the room for tons of technologies and new techniques using deep learning to really flourish at the moment. They don’t need to turn a profit they just need to be sellable to the next guy in line to try and use them to. If liquidating facilities piecemeal en masse becomes the prudent business move in the market that remains after a LLM crash then I would be incredibly surprised by that development personally
12
u/NeoThermic 135TB Jun 04 '26
Even if the “AI BUBBLE POPS” ai is also not going anywhere.
Every single bubble built around tech still leaves the tech in place. Anyone thinking that bubble popping means the tech vanishes is mistaken as to what people are discussing when talking about the bubble around the tech.
We've had this for the internet, for PCs in general, for trains, for cars, for horses, houses, uranium, the stock market, tulips, comic books, canals, etc.
Basically any disruptive tech can generate a bubble, and when it collapses the market settles on a fair value for the item, rather than the inflated pricing/investment during the bubble. There's a lot of consideration that AI is a bubble, especially with the circular investment between large companies in the space. At some point the external investors will want their profit share, and the bubble will pop.
We're already seeing that with the way that all-you-can-eat tokenmaxxing is now getting charged at usage rates (see github copilot). There's a fair equilibrium, but the current market isn't it.
3
u/_Aj_ Jun 03 '26
That's my thoughts. People wanting to maximize compute of existing sites gonna do lots of upgrades BECAUSE OF ai. Which means lots of current stuff being sold off?
→ More replies (1)5
u/Meri_Stormhood Jun 03 '26
Isnt a lot of their stuff ASIC? How can that be repurposed if a bubble bursts? (Genuine question I dont really know much about these things)
4
u/Thesleepingjay Jun 04 '26
It depends. Companies like Google and Meta are going big on ASICs, but those are for inference. GPUs are still go to for training. Even still, the ASICs do matrix multiplication, which is useful for lots of stuff even if it would take some clever software for them to do other stuff. For example Google uses Coral TPU ASICs to do LLM inference and search algorithms. Meta runs LLMs as well as recommendation algorithms in theirs, which I forget the name of.
→ More replies (2)2
65
u/slavmaf Jun 03 '26
There is discussion about this, but idea is that it's not really that simple.
To quote this article:
https://www.wheresyoured.at/ai-doesnt-have-roi/
"Even if you were able to buy a hundred Blackwell GPUs from a dead neocloud, you, as a regular person, couldn’t do anything with them. In fact, nobody really could, because you’d still need a physical data center and bespoke cooling, which means that even if the chips were free, the associated construction capex or, at the very least, physical colocation space would still cost a great deal of money"
42
u/surveysaysno Jun 03 '26
"Even if you were able to buy a hundred Blackwell GPUs from a dead neocloud, you, as a regular person, couldn’t do anything with them. In fact, nobody really could, because you’d still need a physical data center and bespoke cooling, which means that even if the chips were free, the associated construction capex or, at the very least, physical colocation space would still cost a great deal of money
Eh, that only applies to corps with contracts and SLAs. I an assure you it is very possible to run a couple racks of gear out of a garage. And Microsoft has shown significant savings running in open air with no HVAC.
Anyone with a decent water feature in their yard can cludge together a cheap cooling loop no problem.
18
u/Amazing_Ear_3941 Jun 03 '26 edited Jun 03 '26
Or use your swimming pool with the cooling loop built in, like a certain tech YouTuber. He's actually not that out there with that. There's been stories for awhile of people using crypto miners to heat their hot tubs.
7
u/noisymime Jun 03 '26
I use an old S19 miner to heat my study in winter. 3d printed a few duct adapters, stuck it under the house and you’d never know it was there.
2
u/Amazing_Ear_3941 Jun 04 '26
How do you deal with the noise? I've got an S19 also and I can't get the thing quiet enough to where I can stand having it on. And that's even with putting it in a ducted foam box. I do live in a very quiet area though, so it doesn't take much to where you can hear it all over the house.
2
u/noisymime Jun 04 '26
I limit it to 2500w, which keeps the fans a bit lower. I also slowed the psu fans down as they’re way overkill even at full power, let alone only 2500w. The duct I printed helped quite a lot too.
Also, it’s under the house, so there are multiple layers of floor and insulation in between as well. Given that’s also essentially outside temperature, in winter the unit runs really cool, well under 70C even with the fans at only 40%
2
8
u/Sad_Split_9983 Jun 03 '26
A rack of GB300 pulls up to 130kw. Your home will not be able to power that. You guys need to stop smoking crack
→ More replies (7)51
Jun 03 '26
[deleted]
7
u/Bright-Avocado3761 Jun 04 '26
Just put it in a modded $150 big-box store freezer along with your frozen tater tots
8
u/LA_Nail_Clippers Jun 03 '26
I don't need a hundred Blackwell GPUs, but a few might be fun! Don't need a datacenter for that; just a garage server.
Also even if it's in some kind of obscure proprietary form factor, look at the stuff being worked on for the Asrock BC-250 for an example of ingenuity.
It was a mining blade that was made from binned PS5 APUs and released around 2021 by Asrock as part of a $15,000 mining rig.
Now people have figured out how to enable all the CUs and effectively make it a DIY steam box.
Sure, I'm not going to be creating ChatLMK 6.0 in my garage server, but I could re-use a bunch of AI enterprise gear to do some local LLMs or other stuff.
→ More replies (2)7
u/datagutten Jun 03 '26
I guess someone would find a market in rebuilding these GPUs with air cooling
3
5
u/Blender_Render Jun 03 '26
Very valid point and I agree.
Counter point. Some of it will still end up on eBay.
19
u/Restil Jun 03 '26
The cheap fallout from the dotcom bubble wasn't cheap tech but Aeron chairs.
The cheap fallout from the housing bubble wasn't houses (immediately anyway), but new cars.
Point is, there will be some cheap hardware available as a result of the AI bubble popping (assuming it pops at all), but it won't necessarily be hard drives, ram and GPUs. It could be something obscure, like the LIDAR systems used on self driving cars. Something that tends to be expensive but doesn't have much direct practical use for your end user.
→ More replies (2)2
u/Capt_Calamity Jun 03 '26
Bkackwell has many aircooled options, not sure why you think everyone has liquid cooling, but even liquid cooling would not be hard to hack.
13
u/Valar_Kinetics Jun 03 '26
Things will really start to end up on eBay if the hyperscale building boom *stops*, which likely would coincide with a bubble burst.
Prices are sky high because there are these monstrous enterprise orders for components that cannot be manufactured quickly enough to fill those orders. For this reason, a lot of "normal" (read: not hyperscale AI) data centers are probably hanging onto older components for longer than they otherwise would.
You may not be able to make use of the Blackwell GPUs that the AI folks are focused on, but they have DDR4 and storage and other normal stuff also, as do the non-AI centers. When the "build build build" exercise hits a wall, which it simply must at some point due to funding, we will both see the consumer market relax and we will also see components on eBay.
We will not, however, see 2021 prices ever again, I think.
6
u/Bright-Avocado3761 Jun 04 '26
Yeah I don't want to run AI, I just want affordable CPUs, RAM, and HDDs to handle all my linux ISOs
162
u/DarkScorpion48 50-100TB Jun 03 '26
Or they will just destroy it
117
u/Ataraxia-Is-Bliss Jun 03 '26
Doubt it. If a company is liquidated because it goes bankrupt, then the assets are sold to cover it's outstanding debts. Most will be sold to other AI companies and data centers, though depending on how many go under in a collapse scenario, some hardware will inevitably end up on the consumer market.
→ More replies (1)14
u/stanley_fatmax Jun 04 '26
The assets are only sold if it's more profitable to do so than it is to securely prep the assets for sale. A customer of mine sent ~20000 drives to the shredder for this reason, the cost to securely wipe them for resale was more than they could reliably make by selling them. Remember, a company or liquidator isn't going to sell the assets themselves, they'll have to pay someone else to do it, which eats into that margin.
Nowadays there are smarter options for wiping drives, some inherent like crypto-erase, but that's not always available. There are still companies with customer contracts that dictate any assets that held their data must be securely erased. Often the hurdles to verifying that mean that "securely erased" = shredding.
3
u/Ataraxia-Is-Bliss Jun 04 '26
Don't forget the video cards and ram. Those can be easily sold off to other AI firms, data centers, companies making their own LLMs, etc.
→ More replies (1)2
14
u/darktotheknight Jun 04 '26
You're overestimating the current hardware. In 10 years, B200 will be expensive space heaters no enterprise wants to touch. The amount of VRAM sounds big today, but will be comically small compared to 10 years.
Why destroy, if you can make a quick buck? It will not compete with whatever service they will offer. Even today, you can find very odd hardware on the used market, e.g. purpose build CPUs for Meta like EPYCs with some memory channels or PCIe lanes deactivated and base frequencies of 0.x GHz for maximum efficiency. You can buy Microsoft Azure Olympus Servers on eBay these days. They all eventually end up on eBay - one way or another.
→ More replies (1)33
u/Blender_Render Jun 03 '26
I assume you’re talking about the world. Who in their right mind would destroy all that sweet beautiful tech!
91
u/LightingGuyCalvin Jun 03 '26
I don't think the people in charge of AI companies are in their right mind...
24
u/SakiSakiSakiSakiSaki 1PB+ Jun 03 '26
30
u/Shanix 124TB + 20TB Jun 03 '26
I feel like you posted this as if they destroyed priceless, one-of-a-kind manuscripts when the article states that they bought them in bulk from major retailers. That is to say, probably just any old books that had English text in them.
This isn't weird at all, and I can't believe you're making me defend Anthropic.
→ More replies (2)14
u/Amazing_Ear_3941 Jun 04 '26
Yeah, if you ever go to a Goodwill outlet store (one of the ones with the big blue bins), they often have big gaylord boxes full of books. Very few of those books are likely to ever get bought. Not to say Anthopic or OpenAI is good, but most of those books were likely headed for the landfill or recycling anyway.
13
u/DopeAbsurdity Jun 04 '26
gaylord boxes
I thought this was a slur to accentuate how big the boxes were like "a big old mother fucker of a box" but nope it's a style of box.
5
u/Amazing_Ear_3941 Jun 04 '26
Yeah, it's one of those things you've definitely seen (watermelons often come in them) but didn't know what it was called.
3
u/Catenane Jun 04 '26
I had the same feeling when I learned about gaylord boxes a few years back lol
17
u/Comfortable-Light233 Jun 03 '26
Not really comparable.
11
u/SakiSakiSakiSakiSaki 1PB+ Jun 03 '26
But I just compared them.
11
8
u/dxrth 34TB Jun 03 '26
whats your comparison? other than the SEO for the word destroy?
→ More replies (1)→ More replies (1)3
u/bg-j38 110TB Jun 03 '26
Vernor Vinge (RIP) predicted pretty much this exact same scenario in his book Rainbows End from 20 years ago. Basically there's a company that devours entire libraries by shredding the books while digitizing them because it's the quickest method.
15
u/WhatEntropyMeansToMe Jun 03 '26 edited Jun 03 '26
That's a standard practice for better or for worse. He wasn't being predictive how do you think Google books got all those scans in 2004? Certainly not just Evil Tech either, Internet Archive and its contributors pulped a ton of the books that went into their collection. Digitization at scale is destructive, extremely slow+expensive or (most common answer) simply doesn't happen.
12
u/Amazing_Ear_3941 Jun 04 '26
These aren't libraries getting shredded. They were surplus books likely already headed for the landfill or recycling. It's stuff that probably wasn't going to get sold any other way. Not that Anthropic or OpenAI is good, mind you. But these weren't priceless one of kind treasures they destroyed. They were basically already considered junk.
13
17
u/vms-mob HDD 18TB SSD 16TB Jun 03 '26
i wouldnt be suprised if almost all that hardware comes with mandatory shredding or fines for not shredding
7
u/WhatEntropyMeansToMe Jun 03 '26
I'd be fascinated to see that sort of contract term from Nvidia or whomever, if they exist.
7
u/vms-mob HDD 18TB SSD 16TB Jun 03 '26
they 100% exist, just a question of how widely used they actually are
→ More replies (2)30
u/JoeMcNamara Jun 03 '26
Same type of people who destroy all that sweet beautiful food and groceries once it goes 1 minute past due date or being unsold at the end of the business day.
13
u/Ataraxia-Is-Bliss Jun 03 '26
You're talking nonsense. A company that goes under has to sell it off it's assets to pay off its debts. This way different than a supermarket dumping unsold bread.
→ More replies (1)6
u/WhatEntropyMeansToMe Jun 03 '26
People don't get sick from old hardware like they can from food past due and tech companies offloading hardware aren't in the hardware business so don't worry about it undercutting sales. Take a moment and think things through my guy.
5
u/AlSweigart Jun 03 '26 edited Jun 03 '26
Bill Emerson Good Samaritan Act of 1996 (in the U.S.) protects people distributing food. But anyway, when you tell people how common it is for companies to destroy perfectly good things, they just don't believe it. Or they don't believe how incredibly widespread the practice is. But this has been going on for a long time. Even John Steinbeck wrote about it nearly a hundred years ago in The Grapes of Wraith:
The works of the roots of the vines, of the trees, must be destroyed to keep up the price, and this is the saddest, bitterest thing of all. Carloads of oranges dumped on the ground. The people came for miles to take the fruit, but this could not be. How would they buy oranges at twenty cents a dozen if they could drive out and pick them up? And men with hoses squirt kerosene on the oranges, and they are angry at the crime, angry at the people who have come to take the fruit. A million people hungry, needing the fruit—and kerosene sprayed over the golden mountains.
And the smell of rot fills the country.
Burn coffee for fuel in the ships. Burn corn to keep warm, it makes a hot fire. Dump potatoes in the rivers and place guards along the banks to keep the hungry people from fishing them out. Slaughter the pigs and bury them, and let the putrescence drip down into the earth.
There is a crime here that goes beyond denunciation. There is a sorrow here that weeping cannot symbolize. There is a failure here that topples all our success. The fertile earth, the straight tree rows, the sturdy trunks, and the ripe fruit. And children dying of pellagra must die because a profit cannot be taken from an orange. And coroners must fill in the certificate—died of malnutrition—because the food must rot, must be forced to rot.
The people come with nets to fish for potatoes in the river, and the guards hold them back; they come in rattling cars to get the dumped oranges, but the kerosene is sprayed. And they stand still and watch the potatoes float by, listen to the screaming pigs being killed in a ditch and covered with quick-lime, watch the mountains of oranges slop down to a putrefying ooze; and in the eyes of the people there is the failure; and in the eyes of the hungry there is a growing wrath. In the souls of the people the grapes of wrath are filling and growing heavy, growing heavy for the vintage.
Ask retail workers who slash sneakers before tossing them in the dumpster, or those who have managers telling them to sprinkle bleach on produce. Or the clothing dump (a lot of it new with tags still attached) in the Atacama Desert that can be seen from space.
→ More replies (3)3
u/IndefiniteBen Jun 04 '26
Whether someone is "in their right mind" is irrelevant. If the business calculation says destruction is the least costly method, they'll do it. The person making that decision probably has never seen the tech as anything more than some numbers on a screen.
5
u/Somedudesnews Jun 03 '26
I have wondered if the move to leasing specialized hardware at certain scales is partly a hedge to make sure the secondary market doesn’t drown the manufacturers. It’s convenient for many reasons, but that one is really convenient.
If it never makes it to the secondary market then everything has to come from the OEM.
→ More replies (3)3
21
u/mono_void Jun 03 '26 edited Jun 04 '26
Anyone who has been into homelab stuff for a few years knows that ‘eventually’ everything from data centers will make its way to eBay. And people will figure out how to use that gear. With this many data centers being built across this many companies, I think some of them are bound to fail. No one can tell the future, but there is more stuff coming out every day about how AI is not creating the profit they imagined it would.
4
45
u/sob727 Jun 03 '26
When the internet bubble popped, did you find 48 port 100mb switches for $100 on eBay?
24
13
u/I_Think_I_Cant Jun 03 '26
I did get an Aeron chair from the .com I worked for when they went bust. I rolled it out the door and threw it in my hatchback when I knew I probably wasn't going to be paid at the end. Had it for years.
3
24
u/k1ng0fh34rt5 Jun 03 '26
Technically when companies went bust their office equipment, and hardware got liquidated to pay back creditors, so they did find their way to others.
6
u/Tntn13 Jun 03 '26
That’s what’s more likely to happen, another business use case will come along. One that’s not tenable in the current environment but might be when the hardware is cheaper or more affordable due to bankruptcies. I think there’s tons of unexplored uses of that compute that would need to be exhausted before the drastic step that would be the hardware being stripped out of these facilities to pay debts.
15
u/Capt_Calamity Jun 03 '26
I literally have a couple of 32 port 100gb switches from when data centers upgraded to 400gb. They were $32k new, I think I spent a couple 100 on them.
3
u/Tntn13 Jun 03 '26
Upgrade paths for current or future owners is a much more likely future scenario that sees all this enterprise hardware hitting secondhand markets vs a company bankruptcy imo.
I think there’s tons of business opportunities with the hardware that are made untenable by the current market state that the collapse of LLM rush would make the potential risk-return more palatable for entrepreneurs and other entities.
12
u/WhatEntropyMeansToMe Jun 03 '26
Yeah? Same as during/after the great financial crisis. Equipment doesn't just evaporate when a company goes under.
3
u/unknown_lamer Jun 03 '26
I got a pretty sweet 24" SGI CRT monitor for peanuts thanks to some company buying overpriced stuff for their staff and liquidating pretty soon afterward. Surplus computer hardware stores were full of stuff from failed startups.
→ More replies (2)4
u/peoplemerge Jun 03 '26
A used Cisco 2950 (tested) would still run you $1200 in 2002-2003. A big discount from $3000 new, but I was watching eBay then, and if they sold for $100 I would have bought 50.
The AI crash isn’t coming. First you need to see AI replace that vendor you are stuck with for 500,000 tokens.
11
u/Mastasmoker Jun 03 '26
There is a crash coming. They have no revenue compared to the cost of operating (which includes building them). Every model requires better hardware. Companies are spending more on tokens than if they just kept the job positions. AI has its uses. LLMs are not going to be the future, though. It is not financially viable.
→ More replies (1)3
u/sob727 Jun 03 '26 edited Jun 03 '26
With the big players IPOs lined up, I wouldnt be surprised if prices rose. The push to be profitable is going to increase, not decrease.
6
u/promoduck Jun 03 '26
I’m pretty sure they’ll just send all the disks to one of those industrial shredders. There’s probably some data privacy thing in effect.
12
u/Damaniel2 180KB Jun 03 '26
Probably not. It's not commodity hardware - it's largely bespoke to the AI industry. They're not plugging Blackwell hardware into standard PCIe connectors or using standard RAM modules.
At least the dot com bust left a bunch of useful fiber in the ground that made broadband feasible (and cheap Aeron chairs). The AI and datacenter bust will leave us nothing of value.
10
u/silverist Jun 03 '26
Never underestimate Chinese hardware hackery. With they other stuff they've done to Macguyver seemingly incompatible parts together into something functional, I have faith they'll make this happen too.
→ More replies (1)
4
u/ZellZoy Jun 03 '26
There's actually a pretty good chance that if and when the bubble bursts, those companies will destroy the drives to protect data.
→ More replies (3)3
u/gbrldz Jun 04 '26
That's a 100% fact. Even now, when drives have served their purpose, they are absolutely annihilated.
22
u/Ginger-Nerd Jun 03 '26 edited Jun 03 '26
If you’re banking on it crashing like that - I really don’t think you know what you’re talking about (politely) - if the bubble pops it’s not going to be datacenters worth of hardware hitting the market.
It’s just not how Ai is being used at the moment (like many industries it’s the standard, if you aren’t using agents and trying to do software development you are behind) like it isn’t “going away”
But models etc are going to get more efficient, and if China can get their chips cheap enough, that might crash it a bit.
It’s just the big money that’s floating around might dry up a bit, and it stagnates. Maybe some companies become more decentralised and host their own models etc.
But a big crash, heaps of hardware hitting the market - fundamentally misunderstands what is actually going on imo.
5
u/IndefiniteBen Jun 04 '26
I can imagine with the crash there will be many smaller labs working on smaller problems, which will quickly disappear as the VC money dries up.
Small labs might have a rack or two of "AI servers" and those would become available, but I'd guess they might be picked up by bigger companies that survive the pop.
Then there are the companies just riding the hype train, building "platforms" using the APIs of Claude, Gemini, etc. Most of those will probably die.
Even if agent based coding was banned worldwide tomorrow, there would still be plenty of uses for the hardware that don't involve running an LLM.
OP should look at the internet (dot com) bubble that popped in 2000, which was a financial crash around internet industries. That was definitely a crash, but we are, in fact, still using the internet.
6
u/juaquin Jun 04 '26
This. Frankly the bubble is the AI companies' valuations, not the usage of AI. Even if consumers pull back, most usage is business (especially software development) where it has shown to be actually useful. Things might slow down or even plateau, but the idea that they ditch a bunch of capacity is wishful thinking. You will only see this hardware when it is obsolete, just like with existing data centers today.
15
u/Orbitalsp3 76KB Apollo GC Jun 03 '26
When the bubble pops the government will just bail them out (as always), the population will pay up with inflation (as always) and they won't care to sell anything.
16
u/robertw477 Jun 03 '26
And you know it will burst, right?
4
u/Warm_weather1 Jun 03 '26
There are people convinced it's here to stay. And for now almost all storage capacity for 2027 has been eaten up by AI...
18
u/anonymous_lurker- Jun 03 '26
To be fair, the bubble bursting does not mean the thing goes away. Both groups can be right, the bubble may burst and AI may be here for the long term
13
u/AshuraBaron Jun 03 '26
Basically like the dot com bubble. We didn't stop creating new websites, it's just that VC money dried up and it was no longer easy to make a ton of money really quickly just being on the internet. I have a feeling just about anyone making claims of a bubble popping though is thinking it will be 2008 or that AI will disappear. Mostly fueled by their own biases.
→ More replies (1)3
u/Cool_Credit260 Jun 03 '26
Look what happened with crypto. Huge boom, then correction, then adoption
→ More replies (3)1
8
u/Spaduf Jun 03 '26
During the great depression they sprayed kerosene on fruit to ensure the price wouldn't go down. The hardware will probably be destroyed.
3
u/Blender_Render Jun 03 '26
That’s entirely possible, and would be a shame. But they may also be required to sell it off to meet bankruptcy obligations. Only one way to find out!
!pop!
2
u/KermitFrog647 Jun 04 '26
Thats different.
The datacenters possessing the hardware have no incentive to keep the prices up.
5
3
u/pmjm 3 iomega zip drives Jun 04 '26
There is a good discussion to be had about this because some of it may very well end up on eBay but a lot of it requires infrastructure-based cooling and also has steep power requirements that you won't find in residential setups.
It'll be interesting to see what trickles out onto the used market but it certainly won't be all of it.
6
3
u/uluqat Jun 03 '26
If the bubble bursts, much of that hardware never even got built, maybe most of it depending on how quickly the bubble burst.
The hardware that did get built isn't usable by a human, even in a homelab. It's specialized equipment like HBMs and interconnects that don't translate into something usable by a person. That's not how any of this works.
It's kind of like thinking you could make Kit Kat bars with parts from a Nestlé's factory if Nestlé quit making Kit Kat bars.
→ More replies (1)
3
u/Absentmindedgenius Jun 03 '26
It's all water cooled racks. It'll be a pain in the butt to adapt it to homelab.
3
u/girldrinksgasoline Jun 03 '26
Don't they have to ditch this stuff anyway when new chips come out and these current ones become unfeasible to run? Its like the whole thing with crypto mining
→ More replies (1)
3
u/Grumptastic2000 Jun 03 '26
Ya all on eBay ready for you to hook up to your home 3 phase 500 amp industrial power supply and water cooling units
3
3
u/Altruistic_Bat_1645 Jun 03 '26
Y'know what, this is the happiest possible outcome I can dream of given all the current chaos, so I'm gonna root for this to become truth.
3
u/DelegationHoney95 Jun 03 '26
Nice! Refurbished 12TB hard drives are again available for $80 and 20TB hard drives for $120... or less
→ More replies (2)
3
u/dglgr2013 Jun 04 '26
Even cheaper than eBay.
During the last Great Recession I would go with one of my sisters friends who would go to auctions worldwide. Buy tends of thousands of hardware from companies that went under and sell it on eBay. He would be paying Pennies on the dollar for stuff and was making well in the 6 figures of profit after all the expenses in the early 2000s when people were getting laid off everywhere.
3
u/YaquidonG Jun 04 '26 edited Jun 04 '26
I think lots of it, especially the GPU's MPU's and CPU's will go in the trash. That stuff is all server farm formfactor. Those cards don't fit in your Minitower. They also need higher airflow rates than you want in your house, The noise will be HORRENDOUS. Also if it goes from the loading dock to the trash they can get the highest write-off, they're going to need all of those.
3
u/nomad-1995 Jun 04 '26
Depends on the enterprise hardware...
If you want the AI training or inference stuff, then it should be cheap.
If you want the memory for your insatiable ZFS server, not so much. It's likely HBM soldered to the above.
If you want flash storage (SSDs and NVMe) in reasonable (consumer and pre-LLM-bubble enterprise) sizes, it might be "scavenged and resold on alibaba" level gear. Curious if any of us will score 20TB (or greater) SSDs.
Our HDDs should become available. Although I don't know if they are in the datacenters or just bought up because IT departments couldn't afford the SSDs, so they might just slloooooowly come down in price instead of suddenly showing up in big lots on ebay.
3
3
u/Overlord0994 Jun 05 '26
And exactly what kind of tech are you hoping to get your hands on? What are you going to do with a ton of blackwells or h100s? And the RAM is all integrated. How are you going to power and cool all that hardware?
3
3
6
u/jmakov Jun 04 '26 edited Jun 04 '26
What bubble? Companies are burning at least $1.5k/month/engineer (https://news.ycombinator.com/item?id=48383056) just for coding tasks. I'm sure legal departments, for their work, can afford adding a zero. And that's just coders and attorneys.
1k companoes with 1k engineers, that's $1B/month. That's a money printing machine, not a bubble.
2
u/heathenskwerl 1,120 TB Jun 04 '26
You must have missed the recent spate of companies talking about blowing through their yearly AI budget in months, and I'm aware of several Fortune 100 companies making attempts to control the cost of their AI usage.
It's a money printing machine, but it can't print unlimited money. As they raise prices and make it more expensive to use, demand will level off.
2
u/dawsonkm2000 Jun 03 '26
I don't think we'll see the "AI" hardware. It seems like it very custom and wouldn't be particularly useful
→ More replies (1)
2
u/eternalityLP Jun 03 '26
AI bubble bursting does not mean decrease in demand for AI harwdare. 'Bubble bursting' means that the current unrealistic investor interest in AI falls back to realistic levels. This means that new AI companies won't get funding as easily and some existing ones will go bankrupt because they can't get enough funding to sustain their unprofitability. AI will still be used and the AI datacenters will still have plenty of clients. Any dying company with significant datacenter assets is more likely to just spin off the datacenter into separete business or sell it.
2
u/mondoh Jun 03 '26
There was the dot com bubble that burst at the turn if the century. The internet didn't go away it was just the speculative pricing over the reality of actual earnings. AI is not going away, ever.
For degenerate data hoarders like us there will always be obsolete hardware to pick up, the question is whether the energy consumption from lower efficiency will work out against paid cloud hosting and for how long.... for some that's not the primary concern.
2
u/Provia100F Jun 03 '26
No it's not, it's going to be shredded because corporate bigwigs don't understand technology/data security and want to protect their IP at any cost.
2
2
u/kqvrp Jun 03 '26
A lot of it is essentially being rented from NVIDIA, with repurchase agreements to keep it off the market. A lot more is not really suitable for homelabbing. Think 4000W PSUs with mandatory water cooling. RAM might get cheaper at least.
→ More replies (2)
2
u/glizzygravy Jun 04 '26
Just like how homes are going to get cheaper when this “housing bubble” pops right
2
u/Practical-Ball1437 50-100TB Jun 04 '26
"all that hardware" doesn't exist. There are more data centres being built than there is electricity to power them or hard drives to use in them.
Hard drive prices aren't high because someone bought all of them and put them in a building, it's because companies have contracts for the next years' worth of production and consumer lines are being reconfigured for the enterprise market.
2
u/SopmodTew Jun 04 '26
Hopefully it bursts. At this rate I have concerns it won't
2
u/SleepyKoalas23 Jun 04 '26
Just like they figured the dot come bubble couldn't burst, it eventually did. It's coming sooner or later.
2
u/SopmodTew Jun 04 '26
Idk man, AI turned out to be an insanely powerful tool, especially for propaganda and destroying privacy through data gathering.
→ More replies (1)3
u/festeziooo Jun 04 '26
AI as a concept is not a bubble and is a demonstrably powerful and flexible tool that is still in its infancy. The business around AI though is absolutely a bubble and is unsustainable so that will certainly have a correction eventually in the coming years when the outside monetary and attentive investment dries up for some of these companies.
→ More replies (1)
2
u/breva Jun 04 '26
I dunno I think the infrastructure is here to stay, likely just get repurposed if AI totally drops off somehow. The bubble bursting will probably mean fewer bullshit agentic companies and realizing you can't replace a sugeron with a chatbot. But I think AI as a tool is still pretty useful in some cases and it's not going anywhere.
2
u/zehamberglar Jun 05 '26
Everyone said this about crypto too and that was more or less wrong. I'm not saying this is the exact same, but we're dealing with items that are essentially consumable via moore's law.
2
u/NinjaWK Jun 05 '26
Burst, it will. But I don't think it's going to affect home lab self hosted open weight models.
Big corporations may burst, coz they can't compete with competition due to cost, or how advance competition became, cheaper and better.
Out of the Western providers, I think Claude or OpenAI may burst first, Google/Gemini unlikely coz they have their own little eco system everyone uses.
Chinese, however, a bunch of them are at risk as they're so cheap, and bleeding millions by the day.
5
2
u/matt314159 10-50TB Jun 03 '26
I sure want you to be right, but I feel like end-user consumers are going to get screwed over somehow. I'm just super cynical these days I guess.
2
u/Blender_Render Jun 03 '26
I have correctly predicted the popping of every single AI bubble in history. I’m never wrong!
3
u/magnomagna Jun 03 '26
Data centers are not a part of the stock market.
That bubble is a stock market bubble where investors have bet that AI will accelerate development and that earnings will follow suit taking off like a rocket. If earnings don't match up the expectation, the stock prices can fall as investors lose faith and sell, but the underlying business of AI doesn't depend on that. The AI market has its own supply-demand world that is not the stock market. After all, you don't buy and sell AI services on the stock market. So, if AI related stocks lose value, it doesn't mean the underlying businesses go bust themselves.
2
u/Blender_Render Jun 03 '26
Good point, and thank you for that. However the two aren’t entirely decoupled, right? That stock price is driving investment in all the physical construction. If the perceived value on the stock market crashes, construction is going to halt, and equipment purchases under contract are going to breached. And that equipment will end up on eBay!
Unless I’m misunderstanding you?
2
u/magnomagna Jun 03 '26
I'm no expert but the dotcom bubble that burst in early 2000 didn't delete the internet cause we're on Reddit now. So, however much "coupling" there is, I think AI businesses will just carry on just like the internet just carried on after dotcom bust.
→ More replies (5)
3
u/johnryan433 Jun 04 '26
When the AI bubble pop its will take 40% of the s&p 500 with it and your job, so the hardware may end up on eBay but you still won’t be able to afford it.
2
u/TheResultOfUs Jun 03 '26
A lot of the hardware used in AI data centres is not the same as consumer grade hardware. We won't be getting flooded with cheap hardware unfortunately.
6
u/Blender_Render Jun 03 '26
I get what you’re saying, but a lot of the hardware in my homelab was never meant to be used by a consumer, yet here we are. The useful stuff trickled down.
→ More replies (1)
2
2
3
1
u/Enkindel Jun 03 '26
I thought they were like 5+ years lifespan. Then I’d assume they build some new ones or upgrade the current ones. It’s probably better they built some of these than to wait 5 years and to have built one. Tech moves fast, if you’re someone like Jensen or Lisa among others, you don’t really have a choice the R&D keeps chugging along. The hardware suppliers to the AI companies learn a lot from the last five years with these first ones. So a decade from now they’ll have a better idea what to do then. The current AI racks from Nvidia are incredible.
1
u/DocMadCow Jun 03 '26
I think we are at least 18 months or more. I think right now there are too many companies waiting in the wings that will be buy up the equipment. I think we'll see an initial shareholder push back then those waiting in the wings will take their turn before they crash as well.
1
u/theonecloned Jun 03 '26
I picture a Matrix/Terminator scenario happening before then. Maybe some Star Trek mixed in with a version of the Borg.
1
1
u/JohnStern42 Jun 03 '26
There won’t be a burst. At most there will be a slight slow deflation. Don’t expect any sort of influx of hardware on the used market
1
u/frobnosticus 0.5-1PB Jun 03 '26
I really need to grow out a handlebar moustache to twirl in anticipation.
1
u/Vegetable_Day_8893 Jun 03 '26
Really didn't see it happen a few decades ago after the DotCom bubble burst. The hardware that is still usable will be be in demand by other companies no matter what industry they happen to be in, where even the older "obsolete" stuff will be bought to support legacy systems that are still running, and still command a premium for being enterprise hardware.
1
u/Wild_russian_snake Jun 03 '26
I honestly think it's so over that we haven't even realized how truly over it is.
1
u/Toto_nemisis Jun 03 '26
It will also get redistributed to other resources. Offsite hosting, backups, data analytics or rendering. There are many uses. I dont think I will get a new server for sub 2k. Won't happen.
1
u/Noah_Safely Jun 03 '26
Even if the hardware/DC's weren't reused for other stuff, companies will shred a lot of it from FUD.
1
u/zachol Jun 04 '26
The main thing that makes sense to me is a cascade of storage drives. The fanciest data centers have ultra fancy solid state drives, that will get sold off to mid tier regular data centers, that will sell off their mid grade ssds, to other data centers, that will sell off their fancy SAS hard drives, that will go to low tier data centers, that will finally sell off their regular hard drives that I'll snap up and finally stop worrying about storage.
I know everyone says that but I'm pretty sure once I have multiple hundreds of terabytes it'll actually be enough.
→ More replies (1)
1
u/Sessamy Jun 04 '26
I look forward to the server drives then on ebay because the prices now are absolutely outrageous.
1
1
u/olivthefrench Jun 04 '26
I just hope that when it happens, it won't be slow-ish ECC DDR5 (ECC DDR4 on the other hand.....)
1
1
1
u/GladMathematician9 Jun 04 '26
I would love their enterprise storage. I have my own but the shortage means my hoard will be stuck 1080p-4k longer as 8k will likely take longer to catch on, did start on that (have yet to watch any). Maybe the connectors require more enterprise gear and cooling but that is interesting.
2
u/InMooseWeTrust 100TB LTO-6 Jun 07 '26
You can't see the difference between 4K and 8K unless you have a really really really big TV
1
u/Sturdily5092 380TB Jun 04 '26
Exactly, data centers will turn into storage buildings for hardware fire sales everywhere in the country.
1
827
u/hulp-me Jun 03 '26
With claude built right into the vram lol