Hasn't seemed to catch up with Tesla, which is still highly valued despite making a pretty mediocre car compared to the competition. Even if one makes the argument that Tesla vehicles are of good quality, it's still a high valuation that seems to show no sign of dropping.
> Bottom line is that H100 prices are near 3 year highs, A100s are still profitable to run, B200 prices are increasing, no one has enough compute.
Then why aren't the hardware manufacturers of components needed by AI companies making plans yesterday to bring new fabs online to meet demand? That isn't a gotcha question, I genuinely want to know. The money involved isn't that much compared to the money changing hands between Nvidia Microsoft, OpenAI, etc., and it's not like once in-progress data center construction is complete they won't need to buy more RAM and GPUs, especially with any new advances in technology that might happen.
Inevitably someone will reply that hardware manufacturers don't want to be stuck losing money on a facility because the bubble popped and demand disappeared, but if Anthropic and OpenAI are going to "run laps around current big tech", it should be a no-brainer to increase production capacity.
A new fab will need to be filled with advanced equipment like lithography machines. They are the most complex thing humanity has every built.
There is one supplier of EUV lithography machines in the world, ASML. They are basically acting as an integrator for hundreds of highly specialized components manufactured to unimaginable levels of precision. Each of them has roughly one eligible supplier in the world who are operating at full capacity. To expand, they'll need yet another set of specialized and almost impossible to build equipment.
So the supply chain moves incredibly slowly, and the slowness is intrinsic due to the complexity and depth of the supply chain. It can't be fixed with just money. IIRC ASML is aiming to merely double their production of EUV lithography machines by 2030.
Sure, I didn't mean to suggest that it would be easy or fast to increase manufacturing capabilities, just that the confidence I'm seeing around AI should extend to the manufacturers (if that confidence for the future growth and success of OpenAI and Anthropic is warranted). That is, the business decision to increase RAM and GPU supply should be "easy".
They are. They're making as many fabs as they can as fast as they can.
The bottleneck is ASML, who can only make so many EUV machines. No one else can make EUV machines.
Scaling chip fabs and chip equipment is much harder. And you have to understand that chip fabs go bankrupt if demand suddenly drops so they have to be more cautious by default.
If you're really compute constrained do you really need EUV machines? You can make do with DUV fabrication nodes, albeit at somewhat higher cost. The trailing edge is where a lot of the mass impactful innovation is, e.g. trying to replicate more advanced EUV nodes with DUV multiple patterning.
That’s what’s happening. Companies who were planning a move to advanced nodes for non AI chips are delaying it. All the advanced nodes are going to AI or smartphone chips only.
There was a good episode on Dwarkesh's podcast about this in the last few weeks, just a deep dive into the semiconductor industry and what the bottlenecks are.
They used to, but this generation has been hit by at least 4 independant crises that have made them more expensive over time:
1. Moore's law dying means that older nodes are still very useful, so there's still lots of people bidding for capacity on these 8 or whatever year old foundry nodes who are willing to pay lots of money, meaning that base costs of the CPU and GPU haven't fallen as quickly as one would expect even in the absence of 2-4.
2. The Pandemic and then Russia's invasion of Ukraine screwed with supply chains quite a bit, and caused an inflationary spike.
3. Trump's tariffs affected the profitability of these consoles, and inserted a lot of uncertainty into them because nobody knows how the tariffs situation will evolve over time, or if Sony will get reimbursed or not for the illegal tariffs that were levied against them. The uncertainty and general animosity towards the USA has also caused the US Dollar to slump in value relative to a lot of currencies, which then pressures Sony to raise prices.
4. The current RAM, SSD, and GPU shortages caused by LLM hyperscalers is again spiking the costs of their components
I'm also very skeptical of "everything eventually passes" as it pertains to hardware prices. Right now, prices are high because supply can't keep up with demand. But if/when supply increases to meet demand or demand decreases, there's no reason for companies to drop prices now that consumers have become accustomed to them.
Exactly. Production of RAM, SSDs, etc is spread out enough that no one company/country/fab has a stranglehold on the market. Right now anyone with a memory fab has a money printer. More people will build fabs, just like they did last time. It takes a bit but they'll get built.
> I was under the impression that only ASML could build SOTA fabs.
You're correct that this is true, but wrong in that it is not relevant.
SOTA fabs are used for cutting edge CPU/GPU chips. You do not need a SOTA fab to create DDR5 memory or SSD memory. There are a number of companies able to create the lithography machines to do so, and dozens of actual operators of those machines.
Why would that concern you unless you are working on the cutting edge and the very limits of that hardware?
The current generation is insanely fast. I am planning to get a gaming PC for my wife and a mix of gaming + workstation PC for me (or maybe just base it off of the Ryzen 9950x3D and call it a day). We plan to hold on to them for 10 years.
I don't care if anything 6x faster comes out. For what I need the current generation is even an overkill.
I'd even go as far as to say that it would be quite OK if that's the very last generation and no further hardware development ever happens.
I am on the edge of current available hardware and do feel the desire to upgrade. As stated before, I am unhappy with the current maximum when combing frame generation, resolution, and graphics quality.
My dream spec is UE5 at 120hz on an 8k oled. I think that sounds like a super sick experience I would buy tens of thousands of dollars of hardware for.
What's boring to me is how abstract many of the "AI success stories" tend to be, even on here. A whole blog post about some new way to use LLMs, or a best practice, or whatever, and no link to the code or dotfiles. I understand that how you prompt is a big part of things but all the major providers have a lot of configuration options. There are whole ecosystems of plugins.
It's just not very interesting or useful to me to read about how you got AI to output better quality code or how you can program from your phone now without going into detail. And so many of the conversations are showing off the wins without talking about the tools, configurations, or other parts of the setup that made it possible.
It's still around and up to 6th edition! Catalyst Game hasn't been the best steward of the IP, with the rules still being internally inconsistent and usually needing a lot of house rules to fix.
> Countries across the world will have to treat US as unpredictable from now on
Anyone who has studied American history knows the US has been unreliable. Just look at how they made and then broke treaties with Native Americans. It's part of the foundation of the country.
Within Geopolitical commentaries that I used to watch, A famous quote by Henry Kissinger is often repeated.
"to be an enemy of america can be dangerous, but to be a friend is fatal"
So yeah, America has never been trustworthy in a way but it still had its upsides and it still had some laws and checks and people still believed in some aspects of the American dream somewhat, Not anymore.
But now?,it has never been this less trustworthy either in a way to the whole world.
In some recent recruiter calls for hybrid positions in New York, I asked if the employer would pay for roundtrip Amtrak tickets 3x/week from where I live (a ~1.5 hour train ride). Of course the answer was no and I knew that already, but if the company policy is that all employees must live within 50 miles of the office, surely they know that a 50 mile commute by car could be as long or longer than an 85 mile train ride.
That's so weird. Employers have always paid for employees' commuting expenses here in Japan [0]. It's not even mandated by law and there's a legal limit of 150k yen / month, roughly $1000 though most companies limit it to 25k (~$200). Still it's enough that commuting by bullet train is a thing [1].
> The theory is that if we close the gap in regulatory burden between public companies and large private companies, then maybe we'll see more IPOs like back in the 90's, before Sarbanes-Oxley and other new laws.
Yes, maybe. The optimal number of scandals is sadly not zero, and any given piece of legislation tends to overreact, fighting the last battle without seeing all the potential second-order consequences. Even the most carefully-crafted laws are worth giving another look, periodically.
Note that FTX, for example, was privately held. If it had been born in the nineties, the norm would be for it to go public, and have at least a modicum of disclosure; staying private would have been weird, a red flag. Instead, "our generation's Enron" had no public markets oversight whatsoever, SOX or otherwise.
So yeah, it's necessary to find a balance. You are choosing between a little regulation on a lot of companies, or a lot of regulation on a smaller and smaller chunk of the economy each year.
> fighting the last battle without seeing all the potential second-order consequences. Even the most carefully-crafted laws are worth giving another look, periodically.
Dare I say the special interests that ghost write the bulk of the text of any given legislation are specifically banking on those second and Nth order consequences.
Not directly related to FTX, but to the public vs. private discourse, if I'm not mistaken there are now a lot of pension funds and related financial institutions which have redirected a big part of their funds towards privately-owned unicorns/big companies through indirect means, and if those privately-owned unicorns/big companies were to do some shady things those pension funds would be much less in the know compared to if they'd invested their money in public companies.
One could make the valid point that those pension funds shouldn't have been (indirectly) invested in those privately-owned unicorns to begin with, but doing that would have most probably come with opportunity costs for those pension funds (as for some reason or another private big companies have been seen as bringing in more money for each dollar spent compared to big public companies, at least when it comes to the last 8-10 years).
As the OP implies, there needs to be some sort of balance between public and private companies, each of them need to be, in effect, more like the other in the eyes of the State/taxman, State-run regulators and the like.
Not quite, but only because the FTX case was weird. Many individuals from around the world were users. They didn't sign up to be investors, or even to be depositors in a banking sense, and so not all of them were qualified/accredited investors. However, SBF unilaterally and secretly treated them like investors, borrowing from them to finance various schemes. So no, FTX's fallout was not limited in that way.
The people and venture funds that officially owned FTX were a narrower group, and I assume they were all qualified investors. But the thing about our disclosure regime is that protecting the official owners of the company is only one goal, the one that serves as the pretext. Informally, various regs on public companies are designed to bring sunlight more generally, and to prevent a wider array of crimes and shenanigans than just defrauding the company's owners. Public companies also have rules and norms around governance which, had FTX been been subject to them, would have made a difference.
> It seems the private/public split along the lines of "public companies should be more scrutinized" worked as intended.
Only if the intention was also, "...and public companies should be an ever-shrinking share of the economy". There are a number of reasons why one might not have intended that. Ordinary investors miss out on early growth, and the good side-effect of general sunshine and governance norms only covers a sliver of the economy, missing many of the most dynamic firms that could use some scrutiny.
reply