Oracle's recent force majeure invocation shines a spotlight on what seems to be an endemic problem: datacenter builders have been signing commitments to bring a certain amount of capacity online by a certain date, without first conducting sufficient due diligence about whether they would be able to secure all the resources needed to meet those commitments.
This means they are now caught in a mad scramble to source all sorts of stuff, including RAM, potentially as a matter of life and death.
What's less clear to me, though, is whether datacenter builders will be able to make good on these purchase agreements. It seems these projects have every opportunity to fail, and the path to success keeps getting narrower. What are the odds that chip manufacturers ultimately get caught holding a bag of chips that nobody can buy in one hand, and a bag of IOUs from bankrupt customers in the other?
Yeah there was just a story about how there are hundreds of billions of dollars in "non-cancellable" debt obligations to build AI data centers. But they're building far beyond current demand, I feel like the collapse is inevitable.
I'm sensing sarcasm but is it not true that the scale of obligations is unsustainable?
As for preparation, I'm not sure. I have a 401k that I'm worried about. I'm too young to really remember the dotcom bubble, let alone have any stake in the stock market at the time.
There are signs that we are, and are not, in an AI bubble.
Compared to the dotcom boom, firms are well positioned in terms of capital - they have revenue and profits.
However, there's still the absurd speculative valuations, risk, and high expenditure (on said data centres). Also, Andrew Bailey from the Bank of England, personal opinion - https://www.bbc.co.uk/news/articles/cv8e30enrkxyo
My health insurance company recently rolled out a major revamp of their website, and it now uses GiB as if they were KiB.
At my knitting group last night one of the conversation topics was how everyone's computer feels really unstable these days. One person stopped volunteering at a local charity because, after a recent website rewrite, she has too much trouble with the page locking up when she's trying to sign up for hours.
Last weekend I got talking to another parent at my kid's gymnastics class about how her computer's really struggling since they rolled out some "modernization" rewrites of internal business applications, and it's making it hard to get her work done.
What I'm getting at is, if we do have to get used to memory being more expensive, that might be very disruptive. Programmers will have to do a major about-face on how they've been writing software in the post-AI era.
On the other hand, memory optimization is so much easier with AI. Profiling an app, reworking a function to be twice as long but more efficient, A/B testing libraries across browsers to check which uses the least extra memory. All of these things are fairly easy but so tedious, which makes them perfect for AI.
> What I'm getting at is, if we do have to get used to memory being more expensive, that might become very disruptive because software developers will have to do a major about-face on how they've been writing software in the post-AI era.
I'm guessing you meant to say "pre-AI era."
But don't kid yourself. What that means is right before they prompt "and don't make mistakes," they'll prompt "use less memory."
Not sure I buy that? There's a hardware/process argument that DRAM scaling is hitting a wall (for largely the same reason that logic scaling is, even though the fabs are different).
But DRAM has always been almost completely dominated by consumer device needs, and consumer requirements leveled off hard maybe a decade ago. Once you had a 4G system, things were mostly OK. Pervasive VM use and general bloat pushed that up by maybe a factor of two or three since, but that's it.
AI is different. It wants that capacity for different reasons and puts it in different places. And almost none of us really saw that coming more than ~18 months ago or so. Certainly not to the extend that you'd conclude "we've been ignoring it".
or you can wait until one of those frontier labs IPO goes bust in a really bad way where it pumps 10x on day 1 and then everyone dumps every share out there and half a million people are holding bags. that ll surely wipe out all the AI hype
Yes it is, they're talking about induced demand. There are lots of resources where, when usage of the resource becomes more efficient, people end up using more rather than less.
> I have this wild theory that this has not just todo with AI but with weapons production overall.
Why? Weapons are low-volume compared to commercial products, and they probably use trailing edge or weird bespoke components.
I was digging around the other day, and at least the initial versions of the Tomahawk cruise missile had the computing power of an original IBM PC. That just shows how little computing power is needed for a lot of advanced weapons capabilities.
If training and inference is hardware constrained, and you can train and server better and bigger models on the same hardware with memory optimizations, that's exactly what I would expect companies to do.
What was the benefit of all this AI stuff again? Because all I see is middle and lower class being fucked over, repeatedly. Increasing prices of consumer electronics, job losses, suffering quality of goods and services, consolidation of power in favor of elites.
Think about the app sitting on some developer's laptop, necer getting deployed anywhere else! Or the thousands getting uploaded to app stores to slurp and sell data!
This means they are now caught in a mad scramble to source all sorts of stuff, including RAM, potentially as a matter of life and death.
What's less clear to me, though, is whether datacenter builders will be able to make good on these purchase agreements. It seems these projects have every opportunity to fail, and the path to success keeps getting narrower. What are the odds that chip manufacturers ultimately get caught holding a bag of chips that nobody can buy in one hand, and a bag of IOUs from bankrupt customers in the other?
how does one prepare for the moment it actually happens? what would it look like?
what were the immediate consequences of dotcom for the normal folk?
As for preparation, I'm not sure. I have a 401k that I'm worried about. I'm too young to really remember the dotcom bubble, let alone have any stake in the stock market at the time.
Compared to the dotcom boom, firms are well positioned in terms of capital - they have revenue and profits.
However, there's still the absurd speculative valuations, risk, and high expenditure (on said data centres). Also, Andrew Bailey from the Bank of England, personal opinion - https://www.bbc.co.uk/news/articles/cv8e30enrkxyo
At my knitting group last night one of the conversation topics was how everyone's computer feels really unstable these days. One person stopped volunteering at a local charity because, after a recent website rewrite, she has too much trouble with the page locking up when she's trying to sign up for hours.
Last weekend I got talking to another parent at my kid's gymnastics class about how her computer's really struggling since they rolled out some "modernization" rewrites of internal business applications, and it's making it hard to get her work done.
What I'm getting at is, if we do have to get used to memory being more expensive, that might be very disruptive. Programmers will have to do a major about-face on how they've been writing software in the post-AI era.
I'm guessing you meant to say "pre-AI era."
But don't kid yourself. What that means is right before they prompt "and don't make mistakes," they'll prompt "use less memory."
But DRAM has always been almost completely dominated by consumer device needs, and consumer requirements leveled off hard maybe a decade ago. Once you had a 4G system, things were mostly OK. Pervasive VM use and general bloat pushed that up by maybe a factor of two or three since, but that's it.
AI is different. It wants that capacity for different reasons and puts it in different places. And almost none of us really saw that coming more than ~18 months ago or so. Certainly not to the extend that you'd conclude "we've been ignoring it".
Why? Weapons are low-volume compared to commercial products, and they probably use trailing edge or weird bespoke components.
I was digging around the other day, and at least the initial versions of the Tomahawk cruise missile had the computing power of an original IBM PC. That just shows how little computing power is needed for a lot of advanced weapons capabilities.