The obvious story is the leaderboard; the better story is the infrastructure underneath.

Moonshot did not close the distance with American labs by acquiring more compute. It did so by building software that treats compute as precious (because for Moonshot, compute is precious). The company’s inference stack, an open-source system called Mooncake, was engineered to squeeze usable capacity out of hardware the company could not expand. A DeepMind researcher, Anika Somaia, made the point bluntly the day K3’s leaderboard result landed: Mooncake exists because Moonshot doesn’t have GPUs.

This piece makes three arguments. First, that Mooncake and the K3 architecture are a case study in what scarcity does to engineering culture. Second, that American AI infrastructure, sitting on the largest clusters ever assembled, tolerates waste that would be existential for a Chinese lab. Third, that the policy conclusion most people are drawing from K3 is backwards. Export controls did not fail to stop Moonshot. They functioned as a forced R&D subsidy for Chinese systems software, while leaving untouched the thing America actually holds: the capacity to serve models at scale.

What the reaction got right and wrong

The stakes here are not subtle. The United States has spent four years restricting China’s access to AI chips on the theory that compute determines capability, hyperscalers are spending over $650 billion this year in the same direction, and the Philadelphia Semiconductor Index dropped into a bear market within a day of K3’s release because the theory suddenly looked shaky. If a 300-person lab under export controls can match American frontier models, the premise behind both the policy and the capex is wrong, and a lot of capital and statecraft is aimed at the wrong target.

The reaction on X sorted into two camps within hours. One camp declared the compute moat dead, that “the frontier is no longer something money can buy.” The other camp explained the result away as benchmark tuning and distillation from American models.

imo, both camps are arguing about the wrong layer. The first camp is right that training compressed but wrong beyond (K3’s pricing shows Moonshot hitting a wall forced by capacity limits). The second camp is right that leaderboard placement overstates the model (K3 trails American models on the hardest reasoning benchmarks), but wrong to treat the engineering as fake (the architecture work is real, published, and already being adopted by American serving stacks). What neither camp discusses is the part I work in, which is what the constraint did to Moonshot’s infrastructure engineering, and what the absence of constraint has done to American models. That is the focus of the rest of this piece.

What Mooncake actually is

Every LLM request has two phases that impact the underlying hardware differently.

• The first phase, prefill, reads your entire prompt and builds an internal working memory of it, a structure called the KV cache. Prefill is compute-hungry: the chip’s arithmetic units are the bottleneck.

• The second phase, decode, generates the response one token at a time, and each token requires reading that cached memory back. Decode is bandwidth-hungry: the bottleneck is how fast the chip can move data, not how fast it can multiply.

Most serving systems historically ran both phases on the same GPUs, which means the hardware is mismatched to the work at least half the time. A GPU doing decode has arithmetic units idling. A GPU interrupted by a new prompt’s prefill stalls the users mid-generation. Most ate this inefficiency and bought more GPUs.

Moonshot could not buy more. Mooncake, the serving platform behind the Kimi chatbot, splits the two phases onto separate pools of machines, then builds the entire system around the KV cache as the central object. Prefill nodes compute the cache and ship it to decode nodes. Cached prompt prefixes are stored not on scarce GPU memory but on the CPU RAM, SSDs, and network cards that sit underused in every GPU server, forming a distributed cache pool out of hardware the company already owned. A scheduler routes each incoming request to wherever the largest fraction of its prompt is already cached, so the expensive prefill work is done once and reused. Mooncake’s engineers write about overload as their normal operating condition rather than an edge case. The system predicts which requests it cannot serve within latency targets and rejects them early, before they waste compute that a servable request could have used.

Read more

“Critical infrastructure” is doing a lot of work in Washington this year. It is the label that determines which AI labs can deploy which models, to whom, and under what terms. The timing raised the stakes. Anthropic had filed a confidential IPO prospectus earlier that month disclosing a $47 billion revenue run rate and a valuation near a trillion, which means the government switched off the flagship product of a company weeks away from one of the largest public offerings in history.

In February, U.S. Defense Secretary Pete Hegseth met with Anthropic CEO Dario Amodei and threatened to invoke the Defense Production Act if Anthropic did not accept Pentagon terms by Friday. The threat was DPA Title I, the compulsion authority that would let the government force Anthropic to accept and prioritize defense contracts. Hegseth paired the deadline with a threat to label Anthropic a “supply chain risk,” a designation that would force other defense contractors to certify they do not use Claude, and by March the company was in court over a directive banning its models from federal agencies. Other leading AI labs had accepted Pentagon terms for unclassified work, and xAI took the classified track. Anthropic was the holdout. The two threats never squared with each other. A DPA order rests on the premise that a company’s technology is so essential the state must compel access to it, while a supply chain risk label says the same company is too dangerous to use. For most of the spring, Anthropic was officially both. By June, the hyperscaler hosting Anthropic's models flagged a vulnerability to Treasury, Commerce restricted the strongest models, and Anthropic spent two weeks negotiating back onto a permission list it does not control. The regime uses the vocabulary of critical infrastructure protection, but functions as industrial policy applied to a single AI lab.

A category that was never built for this

The term “critical infrastructure” has a specific federal pedigree. The 1996 Presidential Commission on Critical Infrastructure Protection identified eight sectors whose disruption would cause “debilitating” national consequences. In 2013, Presidential Policy Directive 21 codified the sixteen-sector model that still anchors the system today.

The sixteen sectors run from Chemical and Dams to Energy, Information Technology, and Water and Wastewater Systems, and the roster has not changed since 2013.

This is the structure now being stretched to cover AI. The Commerce Department’s restoration of Mythos to “organizations that operate and defend critical infrastructure” draws the eligibility line along these sixteen designations, even though none were drawn with AI in mind.

What Mythos actually established

The June 12 suspension was unprecedented federal action against a commercial AI product, but not surprising given the administration’s escalation against Anthropic. The mechanism used to restrict semiconductor manufacturing equipment now applies to commercial AI software, with Commerce controlling the access. Commerce ran this through export-control licensing vocabulary rather than a published rule. The June 12 directive imposed license requirements on exports, reexports, and transfers of the models where the government saw diversion risk toward military-intelligence end uses, and the June 26 restoration letter lifted the license requirement for Annex A entities, including deemed exports to their foreign national employees. Treating an API session with a foreign national as a deemed export is the novel move. In plain terms, the government decided that letting a foreign citizen type a prompt into the model counts as exporting the model itself. There is no published rule behind that position, and Lutnick reserved the right to adjust the scope at any time. A June 2 executive order had created a voluntary pre-release review path, and the government reached past it for a letter. The next test arrives in August, when a cybersecurity executive order requires federal agencies to build a formal process for assessing what AI models can do to computer systems.

Leave a comment

Annex A will leak, and I expect it within six months. Once companies on the list start naming themselves for marketing reasons, and companies off it start leaking that fact as competitive intelligence, the selection criteria will have to be defended publicly for the first time. Public criteria are litigable criteria. The excluded companies have standing and motive, allied governments will ask why their operators are missing, and Congress will want the selection record.

The geopolitical move that has not been priced in

The cleanest international read is that Washington has decided to act as the gatekeeper for frontier AI globally, and to do it through executive authority rather than statute.

Anthropic’s earlier June expansion of Mythos covered partners in more than fifteen allied countries; after June 12, those partners lost access overnight. The decision about which of them got access back came from a US cabinet secretary, not from a treaty or a multilateral process. European officials and other allied governments have publicly expressed frustration, and in my view this will accelerate non-US AI infrastructure development on a five to ten year horizon.

Every European, Asian, and Middle Eastern operator now has direct evidence that depending on a US lab means depending on US Treasury, Commerce, and DoD permission, and that the permission can be revoked within days of a competitor’s phone call. The rational response is to build domestic AI compute capacity and, for the largest states, to fund domestic frontier training. The Mythos episode gives every program manager outside the United States political cover to ask for more budget.

The trade-off the United States has chosen is domestic control over international AI market share. The American Dynamism crowd spent three years arguing the government should pick strategic industries and back them hard. The government picked, and they went quiet. I read the silence as a sign that this happened faster than anyone wants to acknowledge.

The other implication is the China question. Every restriction of US frontier AI access makes the parallel Chinese stack more attractive to non-aligned states. The Mythos suspension will appear in Beijing pitch decks for the next two years, alongside the chip export controls. The Chinese AI infrastructure benefits from every public moment that confirms US AI is gated by US politics.

What this means for the data center buildout

Every new GPU buildout in the United States is now exposed to politically determined customer access. The contracts and the underwriting, however, do not yet reflect that. Pricing the risk in looks like regulatory suspension carve-outs in anchor tenant agreements, take-or-pay floors that survive a Commerce action, and lenders asking whether tenant revenue is insured against export-control events. None of that language exists in today’s templates.

A neocloud or colo operator that signs a long-term anchor tenant agreement with an AI lab is taking on counterparty risk. This risk now also includes the possibility that the lab’s product gets suspended by Commerce with little notice. That risk is real. Commerce demonstrated it in June, and February’s DPA threat against the same company showed a second lever, compulsion rather than suspension.

The flip side is that explicit recognition of AI compute as critical infrastructure would open procurement and grant pathways that do not exist today. NSM-22 directs federal agencies toward more assertive use of procurement and grant rules to support critical infrastructure resilience. If AI compute becomes a designated sector or a recognized subsector, operators could gain access to mechanisms ranging from CHIPS-style grant funding to DPA Title III investment authority.

The operational implication is more pointed. The Energy sector already has Sector Risk Management Agency leadership through DOE and meaningful federal coordination on resilience. The Information Technology sector does not, in any practical sense, treat hyperscale AI training as the strategic asset class it has become. That mismatch resolves one of three ways. A new SRMA-adjacent body for AI compute gets created, the existing IT and Critical Manufacturing structures get extended explicitly, or nothing moves until an incident forces the issue. The sector list has sat still since 2013, so inertia is the base case, and either of the first two outcomes would materially change the regulatory exposure of every operator that hosts frontier model training.

Every operator building right now is choosing a side, whether they've thought about it or not: build inside the US and follow its rules, or build somewhere those rules can’t reach. Most haven’t made that choice on purpose. Instead, they’re picking sites based on cheap power and fast interconnects. Neither of those removeed our access to AI in June.

The seventeenth sector

Commerce decided who gets Mythos by asking whether an organization operates or defends critical infrastructure. Some compute providers may well have made the list, since the hyperscalers were among Glasswing’s earliest partners. But being on the list and being designated critical infrastructure are different things. A water utility qualifies because its sector has held a federal designation since 2013, which comes with a risk management agency, federal resilience coordination, and procurement programs. A data center operator that made Annex A is there because Commerce picked that specific company, and Lutnick has already said he can revise the list whenever he wants. Meanwhile the GPUs that run Mythos, the facilities that power and cool them, and the people who operate them have no sector designation at all. The government is controlling access to the model while the infrastructure that produces the model has no formal status in the system being used to control it.

The exclusion costs most on the people side, because hardware failures are priced into every buildout model and operator scarcity is not. Every new gigawatt of capacity needs staff who can bring hardware online and keep fleets healthy, and that pool is growing far slower than the capacity is. The suspension proved the government can act fast when it decides a model layer matters. Nothing in this episode suggests anyone in Washington has looked one layer down.

Subscribe now

The perimeter is the product

Fable 5 came back on July 1, still the strongest model anyone can rent. On SWE-bench Pro, the benchmark that measures whether a model can do real software engineering work rather than answer trivia, it scores 80.3% against 58.6% for GPT-5.5. Mythos 5, the same model with the safety restrictions removed, stays inside Project Glasswing, Anthropic’s invite-only program for vetted cyber defenders and infrastructure operators, which now operates inside the perimeter Commerce drew with Annex A. There it scores 78% on ExploitBench against 40% for Opus 4.8, which is what Fable falls back to on cyber queries, and in blocking mode Fable made no progress on offensive cyber tasks at all. Commerce kept Mythos 5 caged for everyone off the Annex A list, and the NSA was reportedly preparing to use Mythos for its own cyber operations during the same weeks the model sat under suspension.

Winning used to require training the best model. Now it also requires government permission to sell it. While a startup can close a capability gap with compute and talent, it unlikely has the same Treasury and Commerce relationships as an incumbent.

Four things will show which way this goes. Whether Annex A leaks and which companies are on it; whether the August executive order produces a working assessment process or another one-off letter; whether any data center operator gets onto a future access list directly instead of through a customer; and whether anyone signs an anchor tenant agreement with an export-control clause in it. (imo, contract language is the clearest evidence that the market has started pricing this).

I believe AI compute will get formally designated as critical infrastructure sooner rather than later, probably after the Annex A list leaks and forces the issue. When it happens, the companies already on the government’s approved lists will call the arrangement safety policy, and the companies shut out will call it regulatory capture. Both descriptions will be accurate. Commerce staff assembled the list privately, with no vote and no published criteria, and every incumbent with access has a reason to keep it that way.

Sovereign compute is the same shape of problem with the timeline already collapsed. The frontier of machine intelligence, in the literal sense of the most capable systems for reasoning and prediction this species has ever built, now lives in a small number of buildings in California, Texas, and increasingly Abu Dhabi. The people running those buildings are friendly to me. They might not be friendly in ten years. Every layer of the stack underneath those models, the silicon, the optical interconnects, the data centers, the orchestration software, the training data pipelines, the model weights themselves, is concentrated in roughly the same set of hands. The question of who gets to think at the frontier has already been answered for a while. The question of whether anyone else retains agency over that answer is the entire fight.

Sovereign compute is not primarily about data residency (although data residency matters), nor about language and culture (although both matter). Underneath all the policy noise sits a question of whether a country, an institution, or a person retains agency over the intelligence that increasingly runs their world, or hands it over to a small number of foreign firms in exchange for convenience and speed. Those that have figured this out are moving fast. Those that have not are about to discover what dependency feels like at the layer of the stack that decides what their citizens are allowed to know.

How we got here

Sovereign compute has three distinct origin stories that have only recently merged into one conversation, and confusing them is how most of the press coverage gets the argument wrong.

The first origin story is national supercomputing as a Cold War instrument. The earliest case for a government owning its own most powerful computer had nothing to do with culture or residency. The Cray-1 went to Los Alamos in 1976 to simulate nuclear yield. The Soviets built BESM machines for the same reason. Computing capability at scale was a strategic asset from the moment it became technically possible, and the institutions running the largest machines, including Los Alamos, Oak Ridge, NCAR, the UK Met Office, and RIKEN, have carried that lineage straight into the present. When ORNL or LANL or NERSC discusses a sovereign system today, the institution is pulling on a thread that runs back through Eisenhower’s massive retaliation doctrine.

The second origin story is sovereign cloud, which began roughly a decade ago and was a fight about data, not about compute. It crystallized after the 2018 CLOUD Act and the 2020 Schrems II decision. American hyperscalers had built their European businesses on the premise that data physically located in Frankfurt or Dublin or Paris was under European law. The CLOUD Act made that a polite fiction. American law enforcement can compel American companies to produce data stored anywhere in the world, regardless of where the bits sit. The watershed moment for that entire debate came in June 2025, when Microsoft France confirmed under oath at a French Senate hearing that it cannot guarantee data sovereignty against US authorities, even for data stored in France under a French-marketed sovereign offering. The polite fiction died on the record.

Subscribe now

The third origin story is the post-ChatGPT realization that frontier AI is a strategic capability concentrated in roughly five firms across two jurisdictions. Epoch AI’s data puts the United States at the majority of global GPU cluster performance, China second, and the rest of the world holding small fractions individually. If you are a serious country and those numbers cross the desk of your defense minister, you do something about it. Jensen Huang crystallized the political pitch at the World Government Summit in Dubai in February 2024: every country needs to own the production of its own intelligence; you cannot allow that to be done by other people. Two years later, the Bangkok Declaration of February 2026, signed by more than a hundred countries, formalized AI sovereignty as a shared policy objective across most of the world.

The logical argument

The clean version of the logical argument is that the AI stack has eight or nine layers, every layer has chokepoints, and no country in the world holds all of them. So the policy question is not whether you achieve “full sovereignty,” because no one will, including the United States. The question is which layers you hold yourself, which layers you partner on, and which layers you tolerate dependency for.

At the bottom sit critical minerals (gallium, germanium, neon, rare earths) where China holds dominant positions in extraction and refining. Wafer fabrication is held by TSMC at the frontier, with Samsung and Intel a generation behind. Lithography sits at a single point of failure called ASML. Memory is split between Micron, SK Hynix, and Samsung. Compute silicon is dominated by Nvidia, with AMD, Cerebras, Groq, Tenstorrent, and Huawei Ascend holding most of the rest. Networking is concentrated in Broadcom, Marvell, and a small set of optical players. Data centers and the energy underneath them turn out to be surprisingly hard to scale. Orchestration and inference serving are dominated by a small set of US clouds. Foundation models at the frontier come from roughly five US labs (OpenAI, Anthropic, Google, Meta, xAI) and a handful of Chinese counterparts (DeepSeek, Alibaba, Moonshot). Applications are the most diffuse layer, increasingly captured by the model providers themselves.

Full-stack AI sovereignty is infeasible for almost any country because AI is a transnational stack with concentrated choke points. The implication people draw from that, which is that sovereignty talk is therefore unserious, is wrong. Industry standard, it seems, is what some call managed interdependence: pick the layers where you have genuine comparative advantage, build there, partner for the rest, and use the building to give yourself negotiating power on the things you cannot build. The globalization argument. Vish Nandlall’s June 2026 analysis calls this the bottleneck blueprint: successful national strategies converge on controlling specific bottlenecks, not on owning everything.

The United Kingdom is the clearest version of this approach. The UK has not tried to construct an indigenous Nvidia or a domestic foundation model lab. It has built Isambard-AI on Grace Hopper silicon, stood up a Sovereign AI Unit in April that operates as a £500 million state-backed venture fund, and partnered openly with Anthropic and Nvidia on the layers where domestic capability is not viable. The UK has taken explicit positions on the model layer, the deployment layer, and the talent layer, and is being honest about importing the rest.

The opposite extreme is France. The French state launched Project Andromède in 2009 with €150 million of state money to build a sovereign cloud, split into two competing entities (Cloudwatt and Numergy) that posted combined revenue of €8 million against billion-euro business plans before being quietly shut down by 2020. Macron’s €109 billion AI Action Summit number in February last year included roughly a third to nearly half from a single Abu Dhabi sovereign wealth fund with no legal obligation to deliver. Mistral is the real asset France has, and in early 2026 Mistral raised €830 million in debt from BNP Paribas, Crédit Agricole, HSBC, and MUFG to buy roughly 13,800 Nvidia chips for a Paris data center, on top of the 18,000 Grace Blackwell systems already underwriting Mistral Compute. Mistral is also closing what started as Apache 2.0 open weights, with its most capable systems now behind commercial licenses and sovereignty contracts. The lesson there is not that France should give up on sovereignty, but that sovereignty everywhere produces a worse outcome than sovereignty in the layers where a country can actually win.

Share CipherTalk

The economic argument

The economic case for sovereign compute is more boring and more important than the policy version makes it sound. It is fundamentally about value capture. AI is going to be a meaningful fraction of GDP growth this decade in every economy where it is allowed to run, and the economic value generated by AI gets divided between

• the firms running the models,

• the firms providing the compute,

• the firms providing the silicon, and

• the customers using the output.

If your country sits on the customer side of that ledger at every layer, you’re facing inbalance payments problem that compounds over time.

The simplest version of the math: an enterprise customer in Germany spending €100 million a year on inference today is sending most of that money to American clouds, who pass a substantial percent on to Nvidia and TSMC. A sovereign alternative captures some of that spend domestically, generates tax revenue, creates engineering jobs, and accumulates operational expertise that compounds. Even a partial sovereign stack, owning the inference layer on top of imported silicon, retains a real share of the value chain instead of routing it offshore.

This is also where the critique lands hardest, because most sovereign compute programs do not actually capture value, they spend it. The EU AI Gigafactories program will deploy roughly €20 billion in public and private capital across four to five facilities, each with about 100,000 advanced processors, on top of the 19 AI Factories EuroHPC already has operational or selected. Each gigafactory is a multi-billion-euro capital project. The unit economics question that determines whether the program works is whether those facilities run at hyperscaler-class utilization rates, above 70 percent, with paying enterprise workloads. If they end up running at 30 percent utilization with grant-funded academic experiments, the program produces what one analyst called cathedrals in the desert: beautiful infrastructure that does not generate the cash flows to justify itself. A GPU farm does not spontaneously generate a thriving AI platform, a publicly funded supercomputer does not automatically birth a hypergrowth startup, and a sovereign label on a data center does nothing to erase underlying dependence on Nvidia hardware, American software orchestration, and US cloud distribution channels. Utilization quality matters more than nameplate FLOPS, and most of the public European numbers conflate the two.

The Gulf model is the most economically interesting because it is the most explicit about value capture. The UAE and Saudi Arabia are not pretending to build a domestic Nvidia. They are buying the silicon at scale, building the data centers at scale, and positioning themselves as regional compute exporters into a 2000-mile radius that covers Africa, South Asia, and parts of Europe. Stargate UAE is a 1-gigawatt cluster inside a 5-gigawatt campus, operated by G42 in partnership with OpenAI, Oracle, Nvidia, SoftBank, and Cisco, with the first 200-megawatt phase coming online in 2026. G42 and HUMAIN each received Commerce Department approval in November 2025 to import compute equivalent to 35,000 GB300 systems each, the highest-end chip Nvidia makes. The trade was straightforward: deep security commitments and supply chain assurances in exchange for chip access. That is not sovereignty in the maximalist sense. It is value capture at the infrastructure layer, paid for with security alignment.

The United States is doing the same thing in reverse. After the Trump administration rescinded the Biden-era AI Diffusion Rule in May 2025, the replacement approach has been country-by-country negotiation, where chip access is the carrot and the implicit threat is loss of access to the American AI stack altogether.

India sits in a third bucket. The IndiaAI Mission has deployed more than 38,000 GPUs against an original target of 10,000, built a national compute portal that subsidizes access for startups and researchers, and selected Sarvam to build a domestic 120-billion-parameter open-source foundation model. The economic bet is on the model and applications layers, on top of imported compute, with the longer game on indigenous silicon still in front of them. Canada closed applications on its AI Sovereign Compute Infrastructure Program on June 1, 2026. Across the US, Europe, the Gulf, and Asia, announced sovereign compute commitments now approach a trillion dollars in headline figures, though much of that total is buildout intention rather than funded spend.

The philosophical and emotional argument

The emotional case is the one nobody writes about clearly, because it sounds soft next to economics and policy. It is also the part that explains why this issue moves so fast in capitals once leadership actually thinks about it, and the part that connects sovereign compute to the deepest commitments of the American constitutional project.

Federalist No. 10, written by Madison in November 1787, was an argument against the concentration of power, even when that power is held by competent and well-intentioned people. The whole American constitutional architecture rests on that bet. The Founders distributed power between branches, between federal and state, between government and the press, and between citizens and any concentrated interest, on the theory that diffusion produces better outcomes than centralization even when the centralization is benevolent. Madison’s specific worry was factions, his term for any organized interest that could capture the levers of government and turn them against the rest of the country. His architectural answer was that a sufficiently large and federated republic would make any single faction structurally unable to dominate. Concentrated power was the danger, in any direction, foreign or domestic.

Mill made a parallel argument about freedom of thought specifically. On Liberty holds that thought concentrated in any single source, however reasonable that source might be, narrows what is knowable. The right to seek and receive information from a plurality of sources is the precondition for the exercise of any other freedom. You cannot vote intelligently, contract intelligently, or live intelligently if a single institution decides what arguments you ever hear.

Foreign-controlled AI sits in tension with both of those traditions in a way that should be obvious to anyone who takes them seriously. Intelligence is not a neutral commodity. Intelligence shapes what questions get asked, what answers are visible, what arguments are considered legitimate, and what frames the people who use it come to think in. A foundation model trained primarily on English internet text, fine-tuned by a small group of American engineers, and deployed through interfaces optimized for American users does not produce neutral output. It produces output that reflects the assumptions of the people who built it. Those assumptions might be reasonable, charitable, even broadly correct. They are still not the assumptions of every country whose citizens are now using the model to write their essays, debug their code, summarize their meetings, draft their laws, and plan their lives.

Self-determinism is claimed by libertarians, though I think they have been slower to make this particular argument publicly. The libertarian intuition has always been that concentrated infrastructure is a single point of capture, regardless of who holds it. The fact that the holders are private firms rather than governments does not change the analysis. If three or five companies decide what eight billion people can ask, learn, and reason about, that is the most concentrated cognitive infrastructure in human history. The First Amendment tradition is going to have to extend in some form to the upstream layer where ideas get shaped before they are spoken, and serious work on that question has already begun at the Knight Institute and in the Stanford Federalist Society’s 2026 symposium, which explicitly framed foreign-built models as a risk to American freedom. A FIRE poll from January 2026 captured the consumer-side instinct accurately: Americans overwhelmingly want free speech protected in AI regulation, with 72 percent concerned about AI laws being used to suppress political criticism. The same instinct, generalized to the production of intelligence itself, is what sovereign compute is responding to in every country that takes it seriously.

The visceral reaction in Paris and New Delhi and Riyadh is the same visceral reaction countries have always had to dependency on foreign infrastructure. Countries have always treated certain capabilities as too important to outsource: currency, military, the institutions that shape what their children learn. Outsourcing the substrate of intelligence, to a small number of firms in a foreign country, feels wrong on a register that has nothing to do with cost-benefit analysis. The reason Macron’s “third way” rhetoric resonates in Europe is not that anyone has done the math on European AI ROI; it is that a continent watching itself become a customer of a technology it once expected to invent is having a reaction that registers below the level of policy. The reaction is pro-agency, not anti-American, in most cases. Countries do not want to discover, ten years from now, that the most important strategic capability of the century is rented from someone else. And inside the United States, the same logic could apply in reverse. A small number of American firms controlling the substrate of cognition for the entire planet is a Federalist 10 problem with the polarity flipped. Anyone who takes the original American constitutional project seriously should think deeply about the current concentration, regardless of which flag flies over the data center.

There is a personal version of this too, and I think people in my line of work feel it more than they say. I work on the layer of infrastructure that sits below the models, because the models themselves are increasingly outside the reach of any company that did not start that race in 2018. The compute, the operations, the reliability of new systems, the way fleets of heterogeneous silicon actually run in production, those are the layers where smaller firms can still take meaningful positions. Building there is, for me, a way of holding agency over a future that would otherwise be determined elsewhere. That is the same instinct that drew me to technology in the first place.

If you do not feel some version of that pull, sovereign compute will read as a policy abstraction. If you do feel it, you already know what this piece is about.

TL;DR

My honest guess is that sovereign compute as a category is going to produce a small number of national programs that genuinely move the needle, a larger number of programs that mostly subsidize national champions whose product nobody wants, and a much larger number of expensive data centers that run at thirty percent utilization while the press releases age. The CNAS Sovereign AI Index from April 2026 already shows the divergence opening up between countries with infrastructure that actually runs production workloads and countries with press releases. The countries that do this well will share three properties: realistic assessment of which layers of the stack they can actually own, willingness to import the rest without pretending otherwise, and operational competence at running heterogeneous infrastructure across vendors and generations of silicon.

That last property is, in my experience, the one most often underestimated. Rack-and-cabling a thousand-node cluster is mostly procurement. Keeping it running at production utilization across multiple GPU generations, multiple vendors, multiple firmware revisions, and the inevitable silent-data-corruption events that show up at scale is operations work that almost nobody in the sovereign compute conversation talks about. We work with national labs, neoclouds, and defense folks whose fleets look exactly like the fleets that every sovereign compute program is about to inherit, and whose operational problems are exactly the problems those programs will discover on day one of going live.

That is the boring version of what owning the intelligence stack looks like. The work is operational competence in the layers nobody puts on a slide: bring-up, firmware, telemetry, the substrate beneath the model. Teams that build that competence end up with compute that runs. Everyone else ends up with subsidized cathedrals and infrastructure that belongs to someone else.

I want to trace that distance across three eras: scientific simulation, AI model training, and live AI inference. Each one got faster in marketing and more fragile in the rack, and each invented a new trick to survive its own fragility. Every one of those tricks exists because the launch-day number was never the operating number. The classical supercomputing world spent thirty years learning this and building the discipline to live with it. The AI market skipped that schooling, the chip vendors had no reason to stop it, and inference is the era where the bill comes due.

FLOPS is a marketing unit. Goodput is the operating unit.

The metric the field grew up on is FLOPS, floating-point operations per second: how many calculations on decimal numbers a machine performs each second. The industry scoreboard, the Top500, ranks the largest computers in the world by one version of it, a standard benchmark run in 64-bit “double precision,” the high-accuracy format scientific math depends on. It is clean, easy to compare, and close to useless for predicting whether your job finishes.

The figure that matters in production is goodput: the useful work a system completes after you subtract everything lost to failures, restarts, and waiting. A machine that peaks at a petaflop and falls over every two hours delivers less than a slower one that runs for a week untouched. The industry sells the peak number anyway, because it fits on a slide and moves a market cap. When a company raises billions for a cluster, its model assumes a utilization rate, the share of theoretical output it turns into real work. Peak FLOPS sets the figure everyone quotes, goodput sets the figure the business earns, and the space between them is where capital quietly evaporates.

Peak FLOPS sets the figure everyone quotes, and at least two separate taxes stand between it and the figure the business earns.

The first tax is efficiency while the machine is healthy. Even on a cluster where nothing has failed, a training job turns only part of the chip’s peak into useful model math, because the chips spend real time waiting on memory, synchronizing with each other, and sitting idle between pipeline stages. The field measures this as model FLOPs utilization, or MFU: the share of peak the job actually uses. On large training runs it sits around 40 percent, and lower for the newest mixture-of-experts designs. The second tax is the one this piece is about, goodput: the share of the clock that was productive rather than lost to failure and recovery. The two multiply. A run at 40 percent MFU and 90 percent uptime delivers about a third of the number on the slide, and on a well-operated cluster the efficiency tax is the larger of the two. Reliability only overtakes it once the machine grows large enough to break, which is where the rest of this history goes.

Classical HPC treated failure as a line item

The first era was high-performance computing: scientific simulation on machines built from thousands of processors wired to act as one. Climate, fluid dynamics, molecular dynamics, nuclear stockpile work. These jobs are tightly coupled, meaning the processors stay in lockstep and one failed part can stall the whole run, and they are correctness-bound, because you are modeling physics and the arithmetic has to be exact. That is why the era lived in 64-bit precision.

Read more

It’s interesting whether Apollo, or Diameter Capital Partners, or any of the other lenders now financing the AI buildout this way, would want to exercise that right.

The harder question is what they would actually be holding if they did.

A GPU cluster bears little resemblance to a building. Its value at any given moment depends on how it has been provisioned, how it is currently performing, and whether the team that knows its quirks is still there. All of that sits off the lender's balance sheet, beyond the reach of anyone they can call.

This is one of the center problems of the AI infrastructure boom. I cannot determine why no one is talking about it. Tens of billions of dollars in debt is now collateralized by chips whose value depends on operational state, and the operational state is invisible to the people pricing the debt.

This week’s CipherTalk is about what happens to a specific kind of debt when the collateral itself can walk out the door with the operations team.

The cluster does not run itself

At the scale these GPU clusters operate, hardware and systems break constantly. Keeping them productive is a craft.

Modern data center GPUs fail at roughly 9% annually. The number traces to Meta’s Llama 3 technical report, which documented 419 unforeseen disruptions across 16,384 H100s over 54 days of training, of which 148 were GPU failures and 72 were HBM3 memory failures. At 200,000 GPUs, that annualized rate works out to approximately 50 GPU failures every day. At xAI’s stated million-GPU target, Epoch AI projects a failure roughly every three minutes. These are not catastrophic events. They are the steady state.

The failure modes that matter for a credit person are the ones that do not look like failures. Silent data corruption (SDC) is the most expensive, where a faulty GPU produces wrong answers without crashing anything, which means a multi-day training run can complete normally and the resulting model weights are quietly poisoned. Cascading failures are the second category, where one bad GPU crashes a training job spread across thousands of others, costing days of compute. Then there are the routine ones: thermal throttle, ECC memory errors, NVLink flap, GPUs falling off the bus.

NVIDIA built NVSentinel because traditional monitoring detects these problems but rarely fixes them. Crusoe built AutoClusters because queue wait time is the largest controllable variable in cluster goodput. Without these tools, remediation timelines run hours to days.

The job of an operations team is to keep all of this in steady state. They know which racks run hot in summer, which cooling loops have been flaky since the last firmware update, which jobs to re-route when a node degrades but has not failed yet. None of that knowledge is written down. It lives in the team.

This is the asset that serves as collateral for tens of billions of dollars in debt and counting.

Share

How the chips became the collateral

In the last eighteen months, AI infrastructure went from being financed by corporate debt, to being financed by the chips themselves.

The xAI Colossus 2 SPV is the cleanest example. The structure is roughly $7.5 billion in equity, with up to $2 billion of that contributed by NVIDIA itself, and $12.5 billion in debt. The special purpose vehicle (SPV) purchases NVIDIA GPUs and leases them to xAI on a five-year term. Apollo and Diameter sit on the debt tranche. Valor Equity Partners leads the equity. The debt is collateralized by the chips, not by xAI’s broader balance sheet.

Look at the pricing: xAI’s $5B round was priced at up to 12.5%. CoreWeave's GPU-backed deals priced at roughly 8.5% above the benchmark rate, before terms tightened as lenders got more comfortable with the structure.

If we assume here these are not unsophisticated lenders, then we have to assume they are charging what they think the risk costs. The premium is then, the price of guessing.

The scope is wider than one company. CoreWeave alone holds $18.8 billion in GPU-collateralized debt across multiple SPVs. FluidStack’s $50 billion deal with Anthropic uses a different wrapper, with Google providing a backstop on the lease payments, but the underlying logic is the same.

Every neocloud and most major AI labs are now financed this way.

What real collateral looks like

Every other major asset class that gets used as collateral at this scale has decades of price discovery infrastructure behind it. GPUs have almost none of it.

Aircraft have ISTAT-certified appraisers, a global registry, standardized maintenance logs, ferry pilots, and an active secondary market dating back to the 1970s. Ships have BICA. Cars have NADA. Class A office space has standardized cap rates and vacancy comps. Oil has had a forward curve since the early 1980s.

GPUs have Silicon Data’s H100 Rental Index on Bloomberg terminals, which launched in 2024, and Ornn AI, which raised $5.7 million in October 2025 to build the first regulated exchange for GPU compute derivatives. That is the entire price discovery infrastructure for an asset class now backing tens of billions of dollars in debt.

The price moves underneath all of this are wild. H100 hourly rental rates went from roughly $8 per hour in early 2024 to $1.70 by October 2025, then surged 40% back up to $2.35 by March 2026 on a wave of inference demand nobody had priced in. SemiAnalysis put it bluntly: lenders who used six-year depreciation schedules now look smarter than the analysts who chastised them for being too generous. They were guessing, and they happened to land closer to the right answer than the people calling them reckless. No aircraft lender or shipping lender would underwrite five-year debt against an asset whose price swings like that without a way to hedge it. They would not be allowed to.

CoreWeave’s GPU-backed loans price at roughly 8.5 percentage points above the benchmark rate. For comparison, a typical aircraft loan prices at 1 to 2 points above benchmark, and a commercial mortgage usually sits below that. The extra 6 to 7 points is what lenders charge to bear a risk they cannot measure. There is no GPU futures market, no standardized residual value curve, and no way to lock in a forward rental rate. The premium is is the price of underwriting in the dark.

That spread should compress as the market matures. Hedging instruments will appear. Residual value curves will get more standardized. Secondary markets for used GPUs will deepen. When that happens, the cost of capital for AI infrastructure drops meaningfully, which changes who can build at scale. The companies that benefit are not the ones with the cheapest GPUs today. They are the ones positioned to access cheap debt once the financing infrastructure catches up to the asset class.

Six years, four years, or somewhere worse

The public fight over how fast GPUs depreciate is a tell about how confident the people writing the books actually are.

CoreWeave depreciates GPUs over six years. Nebius, with the same business model and the same hardware, depreciates the same chips over four. AWS, Microsoft, and Google all moved their server useful-life assumptions from three to four years up to six years in 2023, a change that reduced reported depreciation expense by roughly $18 billion annually across $300 billion of combined capex. CoreWeave made the same accounting change in January 2023, before going public, lowering reported expense by hundreds of millions of dollars per year.

NVIDIA announced in 2025 that it is moving from a two-year product cycle to a one-year cycle. The chips backing all of this debt are about to become previous-generation twice as fast.

Subscribe now

Michael Burry’s claim is that hyperscalers will cumulatively understate depreciation by approximately $176 billion between 2026 and 2028. He projects Oracle will overstate earnings by roughly 27% and Meta by roughly 21% by 2028. Burry’s motives aside, the math is independently checkable. If the true useful life of frontier-training GPUs is closer to two to four years and the books say six, the gap between paper value and recovery value is real and it is enormous. The recent inference demand surge complicates this. If H100s genuinely have productive life past frontier training, six years may not be wrong. If demand softens again in 2026 or 2027, the writedowns hit at exactly the moment lenders need their collateral to be worth something.

Three values for the same chip

GPU collateral has three different values, and the market is currently pricing only one of them.

Face value is what the SPV says, the purchase price minus straight-line depreciation on whatever schedule the borrower picked. This is the number that determines loan-to-value covenants and the amount of debt the deal can support.

Liquidation value is what a buyer pays in distress. Secondary market data shows moderately-used 2 to 3 year old GPUs trading at 50% to 70% of new pricing under normal conditions. In a default scenario where multiple neoclouds are stressed simultaneously, the buyer pool collapses at the same moment supply spikes, plausibly putting recovery at 30% to 50% of face value in a fire sale.

Going-concern value is what the cluster is worth as a working asset to the next tenant, which depends entirely on whether operational handoff works.

This is where the operational reality from the first section returns. The lender exercising step-in rights inherits a colocation facility owned by someone else, with that facility’s own contracts and constraints. They inherit credentials and topology knowledge that historically lived with the borrower’s operations team, which walked out the door at default. They inherit a market where rental rates already moved 60% in one direction and 40% back the other in eighteen months, with no hedging instrument available. They inherit an asset class where 50 chips a day fail and somebody has to know which racks have been flaky for the last quarter.

The spread between face value and going-concern value is the entire risk that nobody has hedged.

The signal in who is not in the room

The most telling positions in this market are the ones not being taken.

KKR has been the most aggressive private equity firm in data centers, with the CyrusOne acquisition alongside Global Infrastructure Partners in 2022 for $15 billion, the Global Technical Realty commitment in 2026 for $1.5 billion, and the STT GDC deal in February 2026 for $5.1 billion at a 75% stake. KKR’s digital infrastructure book is a central pillar of $186 billion in real assets. The firm is not in the AIP consortium that bought Aligned Data Centers, not in any xAI SPV, and not in CoreWeave’s debt facilities. KKR owns the buildings, the power, the cooling, and the land, the infrastructure layer that holds value regardless of which AI lab wins or which chip generation dominates.

Peter Thiel sold his entire NVIDIA stake in Q3 2025 and rotated into Apple and Microsoft. The chips are not the durable asset, and the financing structure pricing them as durable will eventually have to reckon with what the chips actually are.

Aircraft became financeable because someone built the registry, the appraisers, and the maintenance logs. Ships became financeable because someone built BICA. The interest premium on these deals exists because no one can answer two basic questions: Is the cluster still working? And: Will it still be working in three years?

In US commentary, that result is usually absorbed into a familiar frame: better AI performance tracks access to NVIDIA’s compute stack. But Zhipu operates under US export restrictions and cannot legally rely on large-scale NVIDIA GPU clusters. The company reports training on Huawei Ascend 910B chips instead. (Not been independently verified.)

That mismatch breaks the default interpretation. Hardware hierarchy no longer cleanly maps onto model performance.

The question shifts: if supremacy is no longer about which chips dominate, then what? What constraints shape the systems built on top of them?

The AI stack follows a steep power law in value distribution, concentrated at the bottom.

Most of the value sits at the bottom: electricity generation (Level 0) and silicon (Level 1). Everything above that, networking, orchestration, models, products, is comparatively flexible. Everything below is physical, slow, and constrained.

Policy discussions typically split this into two separate competitions: energy capacity and semiconductor dominance. That separation misreads the system.

Energy behaves like infrastructure constraint, not a competitive variable. Chip leadership increasingly reflects that constraint rather than escaping it.

That tension appears directly in the Zhipu case. Chip provenance no longer reliably predicts model capability.

The energy gap is not a gap that closes

China added roughly 430 GW of new wind and solar capacity in 2025 and reached 1,840 GW of combined wind and solar by year-end. That is more than the European Union, United States, and India combined. In Ningxia and Gansu, industrial power trades at roughly five cents per kilowatt-hour. In parts of the US it reaches forty.

The US grid is moving in the opposite direction. Peak demand growth by NERC is projected to be 224 GW over the next decade, a 69% jump over last year’s forecast, and the highest compound annual growth rate since NERC began tracking in 1995. Interconnection queues, the process by which new power plants get approved to connect to the grid, stretch past five years.

This reflects two decades of different industrial choices, and policymakers somehow treat catching up as the interesting question. The US is very unlikely to catch China at Level 0 in this decade. Permitting, interconnection, and politics function as features of the US system rather than bugs. They have compounded for four decades. Large power transformer lead times run 128 weeks on average, roughly two and a half years, with some units extending past four years. China holds roughly 60% of global transformer production capacity and has become the default supplier when Western lead times stretch. Substation capacity is slow to build and geographically fixed by grid topology.

If Level 0 is lost, the question worth asking is what that does to Level 1. The silicon implication is direct: chip roadmaps are planned five to seven years in advance. The energy environment a chip ships into is baked into its architecture at the design stage, not adjusted at deployment.

What a scarcity-optimized chip looks like

US constraints begin with electricity. Compute capacity depends on available power, not silicon supply. Under that condition, performance increases require concentration of computation. Density becomes the primary design target. Everything downstream follows that constraint.

NVIDIA Blackwell reflects that condition.

Blackwell relies on chip-on-wafer-on-substrate packaging, combining multiple dies into a single system-level unit. Manufacturing scale depends on TSMC. High-bandwidth memory comes from a narrow supplier base, including SK hynix, with HBM3E stacked vertically to reduce physical distance between compute and memory.

Each GPU consumes about 1,200 watts. A full rack crosses 120 kilowatts under standard specifications and rises higher in deployed systems. Air cooling fails at that thermal load. Liquid cooling infrastructure replaces it across leading installations.

Each layer reflects the same constraint: incremental power carries high cost, and floor space inside data centers carries similar pressure. Engineering focus shifts toward packing more computation per unit of energy and space.

That structure produces coupling across the stack. Advanced packaging depends on tightly synchronized suppliers, while leading-edge fabrication depends on tools from ASML and design software from Synopsys. Capability rises alongside fragility, since performance depends on a narrow set of interlocked systems.

Performance gains follow that structure. Blackwell-class systems deliver roughly four times the compute of previous generations under similar power budgets. Each gain traces back to the same constraint: extraction of additional compute from fixed energy input.

Change that input, and design pressure moves elsewhere.

Subscribe now

What an abundance-optimized chip looks like

China builds under a different constraint environment. Electricity is cheaper, capital is coordinated, and scale matters more than efficiency.

Huawei’s Ascend 910C runs on SMIC N+2, a trailing-node process several generations behind TSMC leading-edge fabrication. Reported yields for large AI dies range from 20 to 40 percent in 2025, compared with 60 percent or higher for frontier-class chips.

That yield gap matters less when electricity and deployment scale are the binding constraints. In parts of China, industrial power sits near five cents per kilowatt-hour. Capacity expansion follows state planning rather than private return thresholds.

Under those conditions, cost shifts away from per-chip efficiency and toward total system throughput over time.

Lower-quality silicon can be offset with volume. Multiple chips replace a single high-efficiency unit, and system performance depends on aggregate output rather than individual chip performance.

Architecture reflects that constraint. Larger dies, older process nodes, and scale-out networking replace density optimization. Huawei’s Lingqu fabric prioritizes cluster expansion rather than per-node performance. Parallel execution compensates for limits in lithography that leading-edge Western systems address through EUV-based transistor scaling via ASML.

Ascend 910C sits inside that design space. It does not track the same optimization target as NVIDIA Blackwell, so direct comparison on efficiency alone produces a distorted picture of intent and outcome.

GLM-5.1 at the frontier layer

The chip-level argument becomes concrete at the model layer. Earlier this month Zhipu AI released GLM-5.1, a large open-weight system positioned among frontier coding models.

Zhipu originates as a Tsinghua spinout and operates under US export restrictions, including placement on the US Entity List in 2025. That designation blocks access to NVIDIA GPUs for training at scale. The company reports reliance on Huawei Ascend hardware instead, with no confirmed use of NVIDIA training clusters.

The claim aligns with a broader shift in Chinese AI infrastructure: increasing use of Huawei Ascend chips and a software stack built outside CUDA, under tightening export constraints from NVIDIA supply chains.

Under US hardware assumptions, the underlying stack appears degraded. The Ascend 910B runs on SMIC’s 7nm DUV process, several generations behind TSMC leading-edge production. Reported yields for large AI dies sit between 30 and 50 percent. FP16 throughput is estimated around 600 TFLOPS, below NVIDIA H100-class performance and far behind Blackwell-generation systems.

None of that prevents frontier behavior.

GLM-5.1 briefly reached the top of the SWE-Bench Pro leaderboard. Days later, Anthropic reclaimed the position with a newer system update. The ordering change matters less than the interval between them. A training run built on lower-yield silicon, older process nodes, and constrained supply chains produced a model that temporarily matched frontier performance.

The implication sits in the structure rather than the headline. Compute scale, not per-chip efficiency, carried the result. As cluster size increased and power consumption expanded, per-unit hardware disadvantages carried less weight in the final outcome.

The metric problem

The comparison breaks at the point both systems are evaluated under a shared metric set that no longer matches deployment reality.

Standard analysis ranks NVIDIA-class and Ascend-class hardware using perf-per-watt or perf-per-dollar-of-silicon. Those measures remain internally consistent, but they assume a shared optimization target across environments.

That assumption fails.

In US deployments, constraints concentrate around grid access, interconnection delay, and long-horizon electricity cost. In Chinese deployments, constraints concentrate around aggregate throughput under state-directed capital allocation and subsidized power.

Different constraints produce different objectives.

US systems compress compute into dense hardware to maximize performance per watt. Chinese systems distribute compute across larger clusters to maximize total throughput per unit of deployed capital and available energy.

Once optimization targets diverge, hardware ranking stops carrying predictive power. A chip that leads under one constraint set loses meaning under another, not because performance changes, but because the objective function changes.

What appears as a performance gap often reflects incompatible measurement systems rather than comparable outcomes.

Export controls and unintended system design

US export controls on advanced compute, centered on restrictions involving NVIDIA GPUs and related supply chains, aim to constrain access to frontier-scale training hardware.

The effect has been partial restriction combined with rapid substitution.

Chinese systems have shifted toward domestic stacks built around Huawei Ascend chips and fabrication through SMIC. Rather than reducing capability linearly, constraints redirect system design.

Instead of density-maximization around NVIDIA-style architectures, compute shifts toward scale-out clusters, older process nodes, and higher aggregate power usage per unit of performance.

Export controls therefore behave less like a cap on capability and more like a filter on architecture. They constrain the shape of compute systems rather than their eventual scale.

The result is divergence in system design rather than convergence toward a shared baseline.

How to read chip systems

The SMIC–TSMC gap persists across process generations and does not need to close for Chinese deployment models to function at scale. At industrial electricity prices near five cents per kilowatt-hour, yield loss and per-chip inefficiency translate into different system economics than in US data center environments.

Similarly, the Ascend–Blackwell perf-per-watt gap persists because both systems optimize different variables under different constraints. US roadmaps prioritize energy efficiency per unit of compute. Chinese roadmaps prioritize aggregate throughput under supply chain and capital constraints.

Most comparative analysis compresses these into a single ranking. That produces clarity in hierarchy but distortion in prediction.

The US density advantage reflects an energy constraint rather than a universal hardware optimum. It is a response to scarcity rather than a neutral endpoint of engineering. If that constraint relaxes through grid expansion, nuclear deployment, or transmission buildout, architectures shaped around it lose their organizing pressure.

The systems built under abundance will not resemble current frontier chips. The logic that produced them does not transfer cleanly across regimes.

I used to think that too. I spent years working in quantum sensing, where hardware operates so close to physical limits that every measurement is a fight with noise. You cannot abstract past atomic-scale interference. You have to understand what the sensor is doing at a physical level, in real time, or you get nothing useful out of it.

That requirement, understanding what hardware is physically doing in real time, now applies to everyone building AI infrastructure. Most people do not know it yet.

This piece covers why the efficiency gains people are counting on may not be physically available, why chips are getting harder to keep running after deployment, where the research frontier actually stands, and why the industry's response of building better chips may be aimed at the wrong layer of the problem.

It is long and it is technical in places. I wrote it because the gap between what people believe about hardware and what hardware actually does in production is where a lot of money and time quietly disappears. The physics here is relevant to decisions you are making now.

The material accident underneath everything

Computing looks the way it does because of a chemical coincidence. Silicon oxidizes into silicon dioxide, a nearly perfect insulator, and the interface between them is atomically clean. That combination of properties, discovered in the mid-twentieth century, made it possible to build billions of tiny electrical switches (transistors) on a single chip cheaply and reliably. Not because silicon is the best possible material for computation, but because it worked first, and working first creates compounding path dependence that is almost impossible to escape.

Every chip in every data center descends from that material accident. The transistor, the switch at the core of all digital logic, has been shrinking for fifty years, and each generation was faster, cheaper, and more power-efficient than the last. That era is ending, and the reasons are physical.

As chips get smaller, the margin for error gets smaller with them. Each generation of silicon is more capable and more fragile at the same time.

The efficiency gains people are counting on may not exist

There is a hard floor on how little energy computation can use. In 1961, Rolf Landauer at IBM showed that erasing a single bit of information must release a minimum amount of heat, no matter how clever the engineering. This is not a design constraint; it is a consequence of thermodynamics.

The reason this matters to anyone building or buying infrastructure: current chips burn roughly 300,000 times more energy per operation than that theoretical minimum. When people say hardware will keep getting more efficient, that gap is what they’re implicitly pointing to.

Process nodes like “2nm” or “3nm” are marketing labels, not physical measurements.

Read more

The AI industry sold itself as software-native. The firmware hiring spree is evidence that framing was wrong from the start.

The conventional read is that AI labs are software companies that happen to consume enormous amounts of compute. They rent hardware from CoreWeave and Microsoft, let the cloud provider worry about the physical machines, and point their own engineers at the model. OpenAI has committed $22.4B to a single GPU cloud provider on top of its arrangements with Microsoft and Oracle. The division of labor sounds clean.

What doesn’t fit that read is the hiring. Scroll through OpenAI’s job postings now and you’ll find roles for systems bring-up specialists, firmware engineers, hardware health monitoring architects, and people responsible for what happens when a physical server fails at 3 a.m. during a training run. These are not software jobs. They are the kinds of roles you’d expect at the companies that manufacture servers, not the companies that rent them. I’ve been watching this pattern spread across the labs and I think it points to something the industry narrative has gotten wrong from the start. Not a new problem, but a misread of how this infrastructure actually works.

Why Now

OpenAI did not always need BMC engineers. For most of its history the company ran primarily on Azure, where Microsoft managed the physical infrastructure and OpenAI’s engineers never had to think about what happened below the OS. The job postings did not look like this two years ago.

(BMC, the baseboard management controller, is the secondary computer inside every server that monitors hardware health and handles power cycling independently of the main OS.)

What changed was a failed attempt at vertical integration. Sam Altman concluded that OpenAI needed to own its own infrastructure to reach AGI on the timeline the company believed necessary. Stargate, announced in January 2025 as a $500 billion program to build dedicated AI campuses, was supposed to make OpenAI a full infrastructure operator. This month, these plans collapsed. Lenders refused to underwrite massive construction costs for a company yet to turn a profit.

The fallback position was bare metal at scale: not owning data centers, not renting virtualized VMs, but signing multi-billion dollar contracts for dedicated physical hardware with providers like CoreWeave and Oracle. That position sits precisely at the point where the hardware management problem transfers from the provider to the tenant. OpenAI did not choose this because it wanted the operational complexity. It landed here because the financing failed and bare metal rental was the best available option. The firmware hiring spree is a lagging indicator of that strategic pivot, not a proactive investment in infrastructure capability.

Share

Bare Metal Is Not a Cloud

I want to address something I see cited constantly as the reason frontier labs use bare metal: hypervisor overhead. It is probably the least important reason.

The standard cloud model runs compute on virtual machines managed by a hypervisor, which abstracts the physical hardware and lets multiple tenants share one machine. That abstraction is genuinely useful. It’s the reason for the rise of AWS, and is why general-purpose cloud computing works. VMware’s own benchmarks show virtualized configurations reaching 103% of bare-metal training throughput in some workloads. The performance gap is real but small. For frontier AI training, the problem is not what the hypervisor costs in performance. It is what the hypervisor hides.

What bare metal actually provides is visibility into the full hardware stack. On a managed VM, the hypervisor normalizes away firmware quirks. A managed OS image constrains what kernel version you can run; shared networking masks the behavior of the InfiniBand fabric under load. CoreWeave’s clusters use bare-metal Kubernetes: Kubernetes-native orchestration on dedicated physical hardware, with no virtualization layer in between. That means OpenAI’s infrastructure engineers can see the full hardware stack. It also means they are responsible for it. The cloud provider swaps failed nodes. The tenant’s firmware engineers figure out why the node failed and whether the training checkpoint is salvageable.

There is a key nuance worth knowing here.

CoreWeave is genuinely bare metal from the GPU's perspective, but multi-tenancy still requires isolation. They handle this by offloading networking, storage, encryption, and security onto NVIDIA BlueField DPUs attached to each node, a side-channel processor that handles the cloud management work a hypervisor would normally do, without sitting in the GPU's data path. Because there is no hypervisor, OpenAI gets direct access all the way down to the BMC — the secondary computer inside every server that monitors hardware health, handles power cycling, and reports component failures. On a standard cloud VM, the hypervisor abstracts all of that away and the cloud provider owns it. On CoreWeave's bare metal, OpenAI owns it. That is precisely why the job postings exist. The visibility that bare metal provides comes with full responsibility for everything the BMC sees.

Hardware Fails Constantly.

Most people underestimate how much of the AI training problem is a hardware reliability problem in disguise.

Meta's Llama 3 paper found that during a 54-day training run on a 16,000 GPU cluster, hardware issues caused 78% of unexpected interruptions. Their reliability engineering team found that failures in SRAMs, HBM memory, processing grids, and network switch hardware caused over 66% of training interruptions across their AI clusters. These are not software bugs that get patched in the next deploy. They are physics at scale.

Note: this data covers failures during training only, not inference. The inference failure picture is almost certainly different. My intuition is that inference failures are considerably worse, and worth a separate piece.

Silent data corruptions are the category worth understanding. A GPU exhibiting one does not throw an XID error or trigger an alert. It miscomputes quietly. The wrong numbers propagate through gradient updates, and the model trains on corrupted state for hours or days before the loss curves show something is wrong. Finding the offending node requires hardware diagnostics at a layer most ML engineers never reach, and that no cloud provider will diagnose on the customer’s behalf. As of late 2025, liquid cooling failures became the leading hardware incident category at scale, driven by cooling distribution unit issues, coolant contamination, and air locks. Failure modes that require understanding the physical system, not the software sitting on top of it.

The cloud provider replaces the failed node. The lab’s engineers figure out why training produced NaN loss for six hours before the node was flagged, and whether the last checkpoint is salvageable.

Subscribe now

The Abstraction That Was Never There

None of this is new. Hardware has always failed at scale. What is new is that an entire industry built its cost models and staffing plans on the assumption that someone else would handle it.

The failure has three interlocking causes. First, training runs are now long enough and large enough that hardware fault rates, which are probabilistic per node per unit time, become near-certainties at cluster scale. A two-month training job on thousands of GPUs will see hardware failures. Second, AI workloads stress hardware far beyond what general-purpose compute does. AI training pushes memory bandwidth and power draw to near-saturation levels for weeks or months continuously, producing accelerated silicon aging (threshold voltage drift, electromigration effects) that is qualitatively different from what these chips experience under general-purpose compute. Third, distributed training is tightly synchronized: one node failure idles every healthy GPU in the job while recovery proceeds.

Cloud providers handle physical uptime. What they cannot handle, and make no contractual claim to handle, is the interaction between a tenant’s CUDA build, their collective communications library configuration, and the firmware version on a BMC that was silently updated during a maintenance window. OpenAI’s SRE role for frontier systems calls explicitly for owning node bring-up from bare metal through firmware upgrades. That is not a function any cloud provider performs for you. It is an in-house engineering function that only appears in a job description once a company has learned, at cost, that the abstraction layer is not as complete as it looked from outside.

Compilers and Security

These labs still need compiler engineers and better security engineers is its own separate argument. It is correct for a less obvious reason than people think.

On compilers: the interesting thing is not that GPU vendors ship imperfect toolchains. It is that the gap between what the toolchain produces and what is theoretically achievable on the hardware has a dollar value at cluster scale. FP8 precision formats, custom attention kernels, and fused operations require compiler work that no off-the-shelf toolkit handles correctly at the precision and throughput frontier labs need. A suboptimal compilation choice on a 10,000 GPU cluster running for two months is not a footnote. OpenAI’s ASIC firmware role asks for high-throughput, low-latency firmware code and hands-on investigation of bring-up and production issues on in-house silicon. That is a job description that says: the toolchain shipped with the hardware is not good enough.

On security: every server has a small secondary computer embedded in its motherboard that runs independently of the main OS. It handles remote power cycling, temperature monitoring, and out-of-band access even when the machine is otherwise off. That is the BMC, the baseboard management controller. A compromised BMC sits on a separate management network and can access the physical system regardless of what the host OS is doing. Model weights are the most valuable IP these companies produce. The engineers who understand BMC security well enough to defend it at scale do not come from the standard ML hiring pool, and they are not plentiful.

What the Hiring Pattern Signals

My read is that this is not an AI-specific problem. It is a large-scale compute problem that the AI labs are just the most visible example of right now.

Every operator of dense GPU infrastructure, whether a frontier model lab, a national laboratory running scientific workloads, or a defense program running inference at the edge, faces the same gap: the hardware layer is too complex and too failure-prone to be treated as a solved problem by the vendor or the cloud provider. The AI labs are discovering this loudly because their compute bills are enormous and their training runs are public knowledge. The same gap exists anywhere hardware is pushed this hard.

The engineers being hired, firmware specialists, hardware bring-up engineers, BMC security engineers, come from server ODMs, hyperscalers, and defense contractors. They are not the ML engineering profile. Their presence at model labs is a quiet acknowledgment that the compute stack underneath frontier AI does not abstract away the hard problems. It relocates them. The cloud provider boundary stops at physical uptime. Everything above, firmware behavior, health telemetry, driver compatibility, compiler performance, management plane security, is the operator’s engineering problem.

Products like TorchPass, which handles live GPU migration to spare nodes during hardware failure, show what more mature cloud tooling might eventually cover. But the compiler and security problems are structural. Custom silicon, custom kernels, and model weight security at scale have no off-the-shelf solution. The operators that build the in-house engineering capability to handle the hardware layer will carry an advantage over those that assume it belongs to someone else, because it doesn’t.

As clusters scale past 100,000 GPUs and training runs stretch past 90 days, the failure math gets worse with every node added. The engineering required to manage it is not going to commoditize on any near-term timeline.

I’m Meg McNulty, co-founder of Cosmic Labs. Cosmic automates hardware intelligence for the operators running dense compute infrastructure: data centers, supercomputing clusters, laboratories, defense programs. The problems in this piece are the ones we work on every day: silent hardware failures with no alerting surface, the gap between what a cluster reports and what is actually happening inside it, and the firmware and management layer that no cloud provider will debug for you.

I write about AI infrastructure because the distance between what the industry narrative says and what operators actually experience is where the most interesting problems live. If something in this piece resonates, or you’re seeing things in your own infrastructure I didn’t cover, reach out: meg@cosmiclabs.io.

Yesterday at GTC, Nvidia officially launched NemoClaw, the open-source agent platform it had been pitching to enterprise software companies for weeks. The platform runs on AMD, Intel, and Google chips. A company whose valuation rests on hardware lock-in is now shipping software with none.

OpenClaw is an AI agent you interact with through WhatsApp, Slack, Discord, or iMessage. You text it a task and it executes: booking flights, managing files, writing code, browsing the web, pulling data from connected services. It runs locally on your own hardware, works with any major model, and surpassed React’s GitHub star count in 60 days after going viral in late January. Created by Austrian developer Peter Steinberger as a weekend project, it became the fastest-growing open-source repository in history.

Nvidia can keep every customer and still watch margins fall. All it takes is for those customers to have somewhere else they could go.

Why Nvidia Would Give This Away

Nvidia’s data center business generates margins around 70%. Those margins exist because customers are locked into CUDA, Nvidia’s programming layer that sits between AI code and the chips that run it. CUDA itself is free, but two decades of libraries, tools, and optimization work mean that switching to a competitor’s hardware requires rewriting and retuning code that already works. Most companies don’t bother.

So why ship software that makes chips interchangeable? Because Nvidia sees the lock-in eroding, and if hardware becomes a commodity, the profits migrate to whoever owns the software layer above it. NemoClaw is a bet that orchestrating agents matters more than running them.

NemoClaw is a bet that orchestrating agents matters more than running them. That bet only makes sense if Nvidia believes the hardware moat is already leaking.

Subscribe now

What NemoClaw Adds

OpenClaw’s architecture combines what researchers call the “lethal trifecta“: access to private data, the ability to communicate externally, and the ability to process untrusted content. Within three weeks of going viral, security teams found over 40,000 instances exposed on the public internet. A critical remote code execution vulnerability meant one malicious link could compromise an entire machine. Twenty percent of the skills in ClawHub, OpenClaw’s plugin marketplace, were found to deliver malware. Microsoft’s security team advised that OpenClaw is “not appropriate to run on a standard personal or enterprise workstation.”

Enterprises wanted the capability but couldn’t accept the risk.

NemoClaw wraps OpenClaw in enterprise controls. OpenShell sandboxes agent execution with least-privilege access, restricting file system scope and network connections through YAML configuration rules. A privacy router inspects outbound requests and blocks sensitive data from reaching cloud-hosted models. Nemotron, Nvidia’s family of open models, handles local inference for tasks that shouldn’t leave the premises. The stack installs with a single command and runs on any hardware.

That last part is the tell. NemoClaw doesn’t require Nvidia chips. It runs on AMD, Intel, or Google silicon.

Why The Lock-In Is Eroding

Training is where models learn from data; it happens once, requires massive compute bursts, and Nvidia’s GPUs remain unmatched. Inference is where trained models run continuously, handling billions of queries and generating revenue. Inference now consumes two-thirds of all AI compute, up from half in 2025 and a third in 2023. The shift matters because inference workloads are easier to optimize for specialized chips, and those chips are now reaching production scale.

Nvidia’s $20B acquisition of Groq in December was a bet on that shift. Groq’s chips were built specifically for inference, hitting 750 tokens per second where standard GPUs manage around 100. Nvidia bought the threat before it scaled.

Meanwhile, the switching costs that protect CUDA are dropping. Google is building PyTorch compatibility for its TPU chips, which means developers can move code from Nvidia GPUs to Google TPUs without rewriting it. Anthropic has built infrastructure that runs Claude across Nvidia, Google, and Amazon chips simultaneously; their CPO told CNBC this multi-chip approach was the only way to meet demand. Google is now selling TPU access externally, with Anthropic’s deal for one million Ironwood chips representing the largest commitment to non-Nvidia AI hardware to date.

Leave a comment

The Threat Is Margin Compression

Google and Amazon price their custom chips 30-40% below Nvidia for inference workloads. When Anthropic can credibly shift workloads between three vendors, the pricing conversation changes. Nvidia can keep every customer and still watch margins fall. All it takes is for those customers to have somewhere else they could go.

How fast will this play out? Slower than the narrative suggests, but faster than incumbents hope. Microsoft has spent billions on its Maia chip program and still delayed production to 2026. Google and Amazon succeeded because they started a decade ago. Internal AWS data from early 2024 showed Trainium handling just 0.5% of the AI workloads that Nvidia GPUs handled. Custom chips work well for predictable internal tasks like search or voice assistants. They don’t yet work for enterprise customers running unpredictable workloads at variable scale.

But the 2026-2027 window is when external TPU and Trainium sales grow large enough to appear as line items in earnings reports. The AI infrastructure market will grow from $242 billion to over $1.2T by 2030. Nvidia can lose ten points of share and still grow revenue. But 70% margins at 85% share is a different business than 55% margins at 75% share.

What GTC Tells Us

GTC runs through Thursday, and the technical announcements reinforce the pattern. Nvidia introduced “disaggregated inference,” splitting workloads between Rubin GPUs for prompt processing and Groq LPUs for token generation. The architecture exists because Nvidia now owns two chip lines optimized for different phases of inference, and combining them claims 35x higher throughput per megawatt. Three months ago, Groq was a competitor. Now it’s a component.

Huang announced $1 trillion in expected orders for Blackwell and Vera Rubin through 2027, double last year’s projection. He compared OpenClaw to Linux and Kubernetes: infrastructure every company will eventually need a strategy for. The demand story is intact. The margin story is the one worth watching.

What Breaks and What Ships

Gartner predicts 40% of agent AI projects will be abandoned by 2027. The failures will share a pattern: tight coupling between agent logic and specific hardware or cloud providers. The projects that ship will treat chips the way applications treat databases, as interchangeable backends behind a consistent interface. NemoClaw offers that interface. So do a growing number of startups building inference optimization and multi-cloud orchestration. The margin that used to live in silicon could migrate to whoever owns that layer.

That’s the signal from this week. Nvidia announced record demand and a hardware roadmap measured in trillions of dollars, but the strategic tell was a piece of open-source software designed to work on competitors’ chips. Hardware lock-in built the $4.4 trillion market cap. Software lock-in is the plan to protect it.

I used to think that too.

Before starting Cosmic, I worked on quantum sensing and navigation systems, where hardware operates at the absolute edge of what physics permits. That experience made me skeptical of abstractions. When someone tells me a system performs a certain way, my first question is: what physical phenomenon is doing that work, and what happens when the physics stops cooperating?

Right now, billions of dollars are flowing into custom AI chips. And today, I work with infrastructure teams across national laboratories and large-scale compute environments. The pattern is consistent: hardware that was right at procurement is already drifting from the workloads running on it. I think a very large amount of the custom AI silicon being taped out today will be functionally stranded within 36 months. Not because the chips are bad, but because the physics and the algorithms are moving in directions that will make the hardware assumptions underneath them wrong. This piece explains why.

The industry has massively underinvested in the operational layer between silicon and software. The stranding problem described in this piece is going to make that gap a lot more expensive.

The Transistor Is Fighting Physics

A transistor is a gate. Apply voltage, current flows. Remove it, current stops. Stack a billion on a chip the size of a thumbnail and you have a computer. Making transistors smaller made them faster and cheaper for decades, because electrons at those scales behaved like classical particles and the engineering intuitions held.

That era is over. Modern process nodes like “2nm” and “3nm” are marketing labels, not measurements. Actual channel widths sit between five and fifteen nanometers, roughly twenty to fifty silicon atoms across. At those dimensions, electrons behave like waves and can tunnel through energy barriers they shouldn’t classically cross. The transistor never fully turns off. Across billions of transistors, that leakage becomes a real fraction of total power draw even at idle.

A 2024 paper in Nature Nanotechnology from Queen Mary University and Oxford showed that direct source-to-drain tunneling degrades switching behavior and caps operating frequency due to rising static power dissipation. The same team built a transistor whose channel is a single zinc porphyrin molecule, using destructive quantum interference (two electron pathways through the molecule cancel each other out) to suppress leakage. This is quantum mechanics recruited as the switching mechanism rather than fought against. It’s a laboratory result, not manufacturable, but it demonstrates a path that works with physics.

Subscribe now

There is also a thermodynamic constraint. In 1961, Rolf Landauer showed that erasing one bit of information must release heat, a consequence of the second law of thermodynamics. This was experimentally confirmed in 2012, and the minimum energy (called the Landauer bound) sits at roughly 2.9 × 10⁻²¹ joules per bit at room temperature. Current transistors dissipate about one femtojoule per switch, a gap of over 300,000x. But a 2025 paper measuring erasure energy in real silicon DRAM (dynamic random-access memory, the working memory inside every computer) at single-electron resolution found the Landauer limit wasn’t reached even under effectively infinite-time operation, because DRAM cells can’t prepare their initial charge state in thermal equilibrium. Since that circuit topology appears broadly across electronic design, the theoretical floor for energy efficiency in real circuits may be significantly higher than the Landauer bound suggests, which means projections that extrapolate current improvement curves toward the Landauer floor are probably wrong.

These physical constraints mean each generation of silicon is more expensive to fabricate, more thermally fragile, and harder to repurpose when the workload it was designed for changes. That last point is where the money is.

Why Custom AI Silicon Gets Stranded

An ASIC (application-specific integrated circuit) strips out general-purpose overhead and builds exactly the circuits a particular workload needs. For dense matrix multiplication (the core math operation in AI training and inference), the efficiency gains over a GPU (graphics processing unit, the general-purpose chips most AI runs on today) can be an order of magnitude per watt. That’s real. Groq, Cerebras, and dozens of stealth startups have raised billions on this premise. Internal efforts at the hyperscalers are following the same logic.

The problem is the clock. ASICs take two to three years from architecture freeze to first silicon. Model architectures are now shifting faster than that.

When most of today’s AI ASICs were designed, the dominant paradigm was dense transformer inference: big matrix multiplies, regular memory access patterns, predictable compute-to-memory ratios. That’s no longer the only game. Mixture-of-experts models (architectures where only a small subset of the network activates for each input) like DeepSeek-V3 activate only a fraction of their parameters per token, which means compute is sparse and irregular, not dense and uniform. State space models (a class of sequence models that process inputs through a compressed internal state rather than attending to every prior token) and hybrid architectures are replacing quadratic attention with linear-time alternatives that have fundamentally different hardware profiles. Inference-time compute scaling, where models spend variable amounts of compute per query depending on difficulty, breaks the fixed-throughput assumptions ASICs are designed around.

A chip optimized for dense matrix multiply doesn’t help much when the workload is sparse routing across 256 experts, or when the bottleneck is memory bandwidth for the model’s compressed internal state, or when inference compute varies 10x between queries. The silicon is still good. The assumptions baked into it are wrong.

Consider Groq's LPU. The architecture is built around deterministic execution: every operation is statically scheduled by the compiler before runtime, all model weights live in on-chip SRAM with no external memory, and the hardware assumes a fixed, predictable dataflow through sequential transformer layers. For dense, single-model inference on a standard transformer, this design delivers extraordinary latency. But deterministic static scheduling is the opposite of what you want when inference-time compute varies per query, or when MoE routing sends different tokens to different experts based on input content that can't be known at compile time. The architecture's greatest strength, total elimination of runtime variability, becomes its limitation when the workload is variable. In December, NVIDIA agreed to pay $20 billion to license Groq's technology and hire its founder and senior leadership, in what analysts described as a de facto acquisition structured to avoid antitrust scrutiny. You can read that two ways: NVIDIA saw durable value in the inference IP, or the market signaled that the standalone path for a single-workload inference ASIC was narrowing fast enough that consolidation was the better exit.

Power makes this worse. At sub-10nm nodes, a phenomenon called “dark silicon” means that more than half of a chip’s transistors must be powered off at any moment to stay within thermal limits. The industry response has been to pack specialized accelerators that power on only when their workload arrives. But when the workload evolves, those accelerators become expensive dead area.

And yield is the financial multiplier. Defect density is roughly constant per unit area of silicon, so larger dies catch more defects per die. A wafer of large dies might yield 42%; the same wafer cut into smaller dies might yield 85%. Advanced-node AI chips tend to be large. The broken ones cost the same to manufacture as the working ones.

Quantum Error Correction: Real Science, Wrong Timeline

A qubit (quantum bit, the basic unit of quantum information) can exist in a blend of 0 and 1, but that blend is extraordinarily fragile. Error correction spreads each logical qubit across many physical qubits so damage can be detected and repaired, but the math only works if physical error rates stay below a critical threshold.

In 2024, Google’s Willow processor (a superconducting chip) and a Microsoft-Quantinuum trapped-ion system (which uses individual charged atoms held in place by electromagnetic fields) both demonstrated below-threshold error correction in hardware. In November 2025, a Harvard-led team published in Nature the first system combining all core components of fault-tolerant quantum computing in a single 448-atom processor.

These are genuine scientific milestones. But even Willow’s best logical error rate (around 0.14% per cycle for its distance-7 code) is still orders of magnitude above the ~10⁻⁶ levels needed for useful algorithms. The gap between “below threshold” and “commercially useful” is enormous, and the press coverage collapses it in ways that are moving real capital toward quantum and away from the near-term hardware operations problems that are actually solvable right now. Practical quantum computation remains uncertain by years, probably a decade or more. For anyone making infrastructure investment decisions on a five-year horizon, quantum is not a factor.

The physical world doesn’t hold still for your procurement cycle.

Every section above circles the same point. I see this in practice constantly. A chip that passed qualification at 25°C behaves differently at 68°C in a dense enclosure six months into sustained inference. A fleet of nominally identical accelerators drifts in performance, thermal behavior, and failure modes until jobs that ran on one node fail silently on another. Tooling built for one chip generation breaks when the next one ships. None of this is surprising if you think about hardware as physics rather than abstraction, but most infrastructure teams don’t have time for that framing. They’re firefighting.

That’s the space we at Cosmic operate in: making hardware that’s already in the ground actually work, across vendors and chip generations, even as the workloads running on it move out from under it. But the broader point isn’t about us. It’s that the industry has massively underinvested in the operational layer between silicon and software. The stranding problem described in this piece is going to make that gap a lot more expensive.

Then DeepSeek dropped R1 in January 2025, and the comfortable consensus cracked open.

A Chinese lab, locked out of the best American GPUs by export controls, trained a model that rivaled GPT-4 on a fraction of the hardware budget. They did it by squeezing performance out of export-restricted H800 chips using PTX-level optimizations and Mixture-of-Experts architectures that activate only the parameters a given query needs. The message: brute-force GPU spending is not the only path to state-of-the-art AI.

NVIDIA’s market cap dropped $589 billion in a single trading session. More importantly, the moment gave oxygen to every startup and hyperscaler that had been quietly building alternative silicon or rethinking how AI infrastructure fits together. Chip companies like Etched, Cerebras, Groq, and D-Matrix suddenly found their phones ringing off the hook. And companies like WEKA, which had been building the high-performance data layer that actually feeds all of these processors, found themselves at the center of a conversation that finally matched their thesis: the problem was never any single chip. It was how you architect the full stack around them.

Share CipherTalk

The inference shift is rewriting the economics

Here is the number that should change how you think about this market: by 2027, inference is projected to consume 80-90% of total AI compute spending. Training a model happens once (or a few times). Running it for millions of users happens forever.

This is where specialized silicon gets interesting. Training rewards flexibility because researchers constantly experiment with new architectures and hyperparameters. You want a GPU for that. Inference, by contrast, is repetitive and predictable. You know the model architecture, the data types, the batch sizes. That predictability is exactly what ASICs are designed to exploit.

Read more

Meta alone is committing up to $135 billion on AI infrastructure in 2026. Combined with Amazon ($200B), Alphabet ($175–185B), Microsoft ($120B+), and Oracle ($50B), the total approaches $700 billion. Goldman Sachs projects $1.15 trillion deployed between 2025 and 2027. Morgan Stanley models $2.9 trillion through 2028.

The spending numbers are by now almost routine in their enormity. But this capital buys compute. It doesn’t buy the ability to keep that compute running.

When servers cost $5,000, this was tolerable. Today, a single NVIDIA DGX system costs over $400,000, and an hour of unplanned downtime on a large GPU partition can destroy $100,000 or more in compute value. The math no longer works.

The AI infrastructure stack has been automated, layer by layer, over the last two decades. Cloud providers abstracted compute. CI/CD pipelines automated software deployment. Observability platforms monitor application performance. Provisioning happens through code. Every layer in the stack got its automation pass — except the one at the bottom.

Below the operating system, at the layer where hardware actually fails, there is no unified observability, no automated control, and no intelligence. Each vendor ships its own management tools, which speak only to its own hardware. NVIDIA’s DCGM tells you about NVIDIA GPUs. Dell’s iDRAC tells you about Dell servers. Arista’s tooling covers Arista network equipment. None of them talk to each other. When something goes wrong — and at scale, something is always going wrong — an engineer has to manually pull data from three or four separate management planes, cross-reference the signals, form a hypothesis about what failed, execute a fix, and validate. The full cycle routinely takes two to four hours, and it depends entirely on someone with specific knowledge of that hardware stack being available.

Subscribe now

When servers cost $5,000, this was tolerable. Today, a single NVIDIA DGX system costs over $400,000, and an hour of unplanned downtime on a large GPU partition can destroy $100,000 or more in compute value. The math no longer works.

The scale problem compounds the economic one. When Meta trained Llama 3 405B on 16,384 H100 GPUs, the run experienced 466 job interruptions over 54 days, 419 of them unexpected. Approximately 78% of those unexpected interruptions were caused by confirmed or suspected hardware failures. That works out to roughly one unexpected interruption every three hours, on a cluster that by today’s standards would be considered modest. xAI’s Colossus facility in Memphis now operates 555,000 GPUs, targeting over one million by late 2026. At 100,000 GPUs, statistical expectation is one hardware failure every 30 minutes. At 500,000, roughly one every six minutes. The problem doesn’t scale linearly. It scales combinatorially, because each additional component introduces new interaction effects with the power, cooling, and network systems around it.

The workforce to handle this at scale doesn’t exist and isn’t being trained fast enough. Industry analyses project roughly 340,000 unfilled data center technical positions in 2026, with only about 15% of applicants meeting minimum qualifications. Companies are recruiting from nuclear energy, the military, and aerospace to fill gaps. The construction boom is moving faster than the operator pipeline.

Share

So what the Meta-NVIDIA deal signals isn’t the scale of investment in AI compute alone. It’s that the operational challenge beneath that compute is about to get much harder. Every rack that ships is another collection of components that needs to be provisioned, monitored, diagnosed, and repaired. At millions of units, across multiple vendors, running continuously, manual operations aren’t a strategy. They’re a ceiling.

The AI infrastructure stack has been automated from the top down for twenty years. The physical layer is next. The models that can diagnose hardware faults from correlated telemetry across vendors and predict failures before they cause downtime now exist, validated in production, at scale.

The economics have crossed the threshold where the cost of the problem exceeds the cost of automating it by orders of magnitude. And the human capital to handle it manually isn’t there.

Leave a comment

Every major GPU deal announced over the next eighteen months is also an announcement of operational demand that doesn’t have a solution yet. The gap between the scale of investment and the sophistication of hardware operations is widening. That gap is going to close. The question worth watching is how.

Everyone is talking about the size of the number. Fewer people are asking what happens when that money hits the physical world and the physical world can’t absorb it.

The Deal That Fell Apart

Last September, Nvidia and OpenAI announced a $100 billion partnership to build 10 gigawatts of AI data centers. It was framed as a landmark: the two most important companies in AI, locking arms to build infrastructure at a scale the industry had never attempted. Five months later, no contract has been signed and no money has changed hands. Nvidia CEO Jensen Huang conceded the agreement was “never a commitment.” OpenAI has since struck separate deals with AMD and Broadcom for competing chip architectures. The $100 billion figure that anchored so much of the market’s confidence in AI infrastructure was, it turns out, a letter of intent that both sides were already hedging against.

This matters most for Oracle, which is arguably the most exposed company in the entire AI buildout. Oracle signed a $300 billion five-year cloud contract with OpenAI. It is the primary builder of Stargate data centers. It is sitting on more than $100 billion in debt. And Bloomberg reported in December that several of its data centers for OpenAI have already slipped from 2027 to 2028, citing labor and materials shortages. When the Nvidia-OpenAI deal stalled, Oracle’s stock dropped, and the company posted on X: “We remain highly confident in OpenAI’s ability to raise funds and meet its commitments.” When a company has to publicly reassure the market that its biggest customer can pay, investors notice.

The circular financing question deserves more scrutiny than it is getting. Nvidia invests in OpenAI. OpenAI uses that capital to lease Nvidia chips through Oracle’s data centers. Oracle borrows to build those data centers. If any link in that chain weakens, the exposure cascades. Bank of America noted that hyperscalers are now spending 94% of their operating cash flow on capex, and Meta and Oracle alone issued $75 billion in bonds and loans in the fall of 2025, more than double the annual average over the past decade.

None of this means the demand is fake. AWS grew 24% year over year last quarter, its fastest growth in 13 quarters. Andy Jassy says capacity is monetized as fast as it gets installed. But the financial structure backing the buildout is more fragile than the headline spending numbers suggest, and the fragility is downstream, in the physical world where the money has to become concrete and copper and running machines.

Subscribe now

The Five-Month Fantasy

Here is something you will not find in any earnings call or analyst report. People building new data center sites right now cannot get power guarantees two years out, even though the construction itself takes that long. Utilities and grid operators are being asked to commit capacity for facilities that will not be operational for 24 months, and they cannot do it because they do not have the generation, the transmission, or the regulatory approvals lined up on that timeline.

At the same time, the customers commissioning these builds want sites operational in five months.

That gap between what buyers expect and what the physical world can deliver is the most important dynamic in AI infrastructure right now. Microsoft disclosed an $80 billion backlog of Azure orders it cannot fulfill because of power constraints. Meta broke ground last week on a 1-gigawatt campus in Lebanon, Indiana, backed by more than $10 billion. Its planned facility in Louisiana could eventually scale to 5 gigawatts, roughly the output of five nuclear power plants, for a single company. The U.S. grid was largely built between the 1950s and 1970s, and approximately 70% of it is approaching end of life. Companies are exploring small modular reactors, direct utility partnerships, and in some cases buying power plants outright. xAI put a $20 billion data center in Southaven, Mississippi, largely because the power was available there. The geography of AI is being determined by where the electrons are, not where the engineers live.

And even where the electrons exist, the people do not. The U.S. construction industry is short roughly 439,000 skilled workers, with most of the gap in the exact trades data centers need: electricians, pipe fitters, and commissioning technicians. Over 400 data centers are under development across the country. A single campus now requires up to 5,000 workers at peak construction. Wages are up 25 to 30 percent, with electricians earning well over $100,000, and contractors still report project backlogs approaching 11 months. Meanwhile, 23,000 experienced workers retire from the construction industry every year, and there is no pipeline replacing them at the rate this buildout requires.

The companies spending hundreds of billions to automate knowledge work are bottlenecked by the physical trades they have collectively underinvested in for decades.

Who Gets to Build the Future, and on Whose Terms

Then there is the political layer, which adds a dimension most infrastructure analysis ignores.

The Pentagon is threatening to sever its relationship with Anthropic because the company will not allow fully unrestricted military use of its AI model. Claude is the only frontier AI model currently deployed on classified U.S. military networks. OpenAI, Google, and xAI have all agreed to lift their safety guardrails for Pentagon use. Anthropic has not. Defense Secretary Pete Hegseth is reportedly close to designating Anthropic a “supply chain risk,” a label normally reserved for foreign adversaries. An anonymous senior Pentagon official told Axios: “It will be an enormous pain in the ass to disentangle, and we are going to make sure they pay a price for forcing our hand.”

This standoff matters for the infrastructure story because it exposes who is buying all this compute and why. A significant share of the demand driving the buildout is coming from government and defense applications. The Stargate project was announced at the White House with President Trump standing alongside the CEOs. If the political relationship between Washington and the leading AI labs determines which models get deployed on which networks, it also determines which data centers get built, where, and for whom. AI infrastructure is becoming national industrial policy, not a market that sorts itself out through supply and demand alone.

Leave a comment

The Last Mile Nobody Talks About

There is one more constraint downstream from all of this that gets almost no attention. Once a data center is built and powered, the hardware inside it still has to be installed, configured, tested, and brought online. Every rack, every server, every GPU cluster requires someone who understands the specific tooling for that vendor’s hardware, that generation of chip, and that particular system configuration.

The pool of embedded systems engineers who can do hardware bring-up at scale is even smaller than the construction workforce. These are the people who connect software to physical machines, who debug firmware, who take a cluster from powered-on to production-ready. There are no bootcamps for this work. The knowledge is hard-won and typically locked inside vendor-specific workflows that break with each new chip generation.

But bring-up is only the beginning. Once a facility is running, the operational challenges compound. Provisioning new hardware into production environments is still a painfully manual process at most organizations, often taking days or weeks per device when it should take minutes. When something fails, and in a facility running tens of thousands of GPUs something is always failing, mean time to recovery determines whether that rack is generating revenue or burning electricity for nothing. Recovery workflows are frequently manual, ad hoc, and dependent on a handful of engineers who happen to know how a particular vendor’s tooling works for that particular chip generation. When the next generation of silicon ships, much of that institutional knowledge becomes obsolete, and teams start over.

This is the part of the AI infrastructure story that almost never makes it into earnings calls or analyst reports. The hyperscalers talk about how many gigawatts they are building. They do not talk about what happens when a firmware update bricks 200 nodes at 2 AM, or when a new GPU architecture arrives and the existing provisioning tools do not support it, or when a field technician at a remote site has to call an engineer in another time zone to walk through a recovery procedure that has never been documented. Uptime at scale is not a software problem or a hardware problem. It is a systems problem that lives at the intersection of both, and the number of people in the world who can solve it is vanishingly small.

The path from “we will spend $200 billion” to “this infrastructure is generating revenue” runs through all of these constraints: grid capacity, construction labor, provisioning speed, recovery automation, and the engineers who keep hardware running once it is deployed. The companies that figure out how to compress the full deployment and operations cycle will capture disproportionate value from this spending wave. Everyone else will be waiting on electricians.

CipherTalk covers AI infrastructure, compute, and the physical systems behind the technology reshaping how we live and work. If this was useful, share it with someone who should be reading it. [Subscribe here] to get these posts in your inbox every week.

But the SemiAnalysis piece, and most of the analyst coverage around it, focuses almost entirely on the supplier side. Who’s winning the HBM (high bandwidth memory) race, which fabs are converting capacity, where margins are headed. The downstream consequences get treated as footnotes. They shouldn’t be. This shortage is functioning as a structural tax on consumer hardware, on mid-tier cloud buyers, and on anyone trying to deploy AI without hyperscale procurement power.

What we’re living through isn’t a cyclical memory shortage. It’s a permanent reallocation of the world’s silicon wafer capacity from consumer electronics to AI infrastructure.

Why This Shortage Is Different

Memory has always been cyclical. Prices spike, manufacturers overbuild, prices crash, repeat. This time the mechanism is architectural: manufacturing one gigabyte of HBM consumes roughly four times the wafer capacity of one gigabyte of standard DRAM (dynamic random access memory… a type of fast, volatile, and affordable computer memory that uses transistors and capacitors to temporarily store data for the CPU). GDDR7 requires about 1.7x. That ratio matters more than anything else in this story, because it means a single HBM stack bound for an Nvidia data center GPU removes four times as much manufacturing capacity from the pool that would otherwise supply phones, PCs, and cars.

TrendForce estimates that by equivalent wafer usage, AI will consume nearly 20% of global DRAM output in 2026. Annual DRAM capacity growth runs only 10-15%. And the three major DRAM manufacturers (Samsung, SK Hynix, and Micron) aren’t trying to fix the consumer supply problem. They’re actively choosing not to.

The Inference Shift Makes It Worse

The industry narrative around AI’s move from training to inference carries an implicit assumption: inference is less hardware-intensive, so memory pressure should ease. For memory, the opposite is true.

Training is compute-bound. A 70-billion parameter model requires about 840GB for mixed-precision training, but the GPU spends most of its time doing math. Inference needs far less capacity (about 140GB for the same model, a 6x reduction) but is memory-bandwidth-bound instead. Every token the model generates requires reading the full set of weights from HBM. It also reads a running log of every prior token in the conversation, called the KV cache, which grows longer with each exchange. If the memory can’t feed data to the GPU fast enough, the GPU stalls.

Google engineers Xiaoyu Ma and David Patterson published a paper arguing that the current AI hardware philosophy (chips designed to maximize raw computing power) is actually mismatched to LLM decode inference, where the constraint is moving data, not computing it.

Training runs are large but finite. You train a model, then you’re done. Inference happens billions of times across a growing server fleet, with longer-context models and agentic workloads that produce larger KV caches per query. TrendForce projects that by 2029, inference surpasses training as the primary driver of AI server demand. The shift doesn’t relieve memory pressure. It spreads demand across a much larger surface area of deployed hardware.

Who Gets Punished

The most telling signal came two months ago. Micron announced in early December that it would shut down its 29-year-old Crucial consumer memory business entirely. The official language was polite (”improve supply and support for our larger, strategic customers”) but the real message is hard to miss: consumer memory is no longer worth the wafer capacity it occupies.

Micron isn’t alone. Samsung and SK Hynix are winding down DDR4 production. Global DRAM inventory collapsed from 13-17 weeks in late 2024 to just 2-4 weeks by October 2025, and the situation has only tightened since.

Under the moderate scenario, PCs contract 4.9% in 2026 with average selling prices up 4-6%. Under the pessimistic case, an 8.9% decline with ASPs up 6-8%.

Nvidia is reportedly cutting RTX 50-series GPU production 30-40% in H1 2026 because GDDR7 is being deprioritized by suppliers who would rather fill HBM and server DDR5 orders.

And then there’s the detail that should get the most attention but hasn’t. TrendForce reports that low-end smartphones are likely returning to 4GB base RAM in 2026. A 4GB Android phone in 2026, running a heavier OS with larger apps, performs worse than a 6GB phone from 2024. Consumer hardware isn’t stagnating. It’s regressing, and the freed-up wafer capacity is going straight to data center buildouts.

The Financial Picture and the Access Gap

Memory companies are now posting gross margins above TSMC’s, something that would have been unthinkable two years ago. TrendForce reported that Samsung’s memory division and SK Hynix delivered gross margins of roughly 63-67% in Q4 2025, compared to TSMC’s guided 60%. The HBM market is controlled by three companies: SK Hynix holds about 57% share, Samsung about 22%, and Micron about 21% (per Counterpoint Q3 2025 data). All three have sold out capacity through 2026.

Server OEMs like Dell and HPE face squeezed margins on fixed-price enterprise contracts. Module assemblers have lost a chip supplier. PC and smartphone OEMs are absorbing cost increases that eat into margins or get passed to consumers who then buy fewer units.

But the more important consequence is strategic. The memory shortage functions as a regressive tax on AI adoption for everyone below hyperscale. Google, Amazon, Meta, and Microsoft secured memory allocations years ahead. Mid-size companies trying to deploy inference on-premise are paying spot-market premiums that run double or triple contract rates. The memory crisis is widening the compute access gap between the largest tech companies and everyone else, and almost no one in the supplier-focused analyst coverage is writing about it.

New cleanroom capacity takes two to three years minimum. Samsung’s P5 fab arrives around 2028 and SK Hynix’s M15X targets mid-2027. Some analysts model shortages persisting into 2028. The signal that would indicate a turn: memory manufacturers reconverting even a small percentage of HBM-allocated capacity back to commodity DRAM, or a major hyperscaler canceling or delaying a planned GPU cluster. Neither has happened.

SemiAnalysis is right on the supply fundamentals. Where the coverage falls short is on the demand side of the ledger. The memory supercycle is not lifting the broader tech industry. It is concentrating resources in the data center and draining them from everywhere else.

Five days later, users are still reporting problems. California’s governor is launching an investigation into whether TikTok censored political content. Oracle says everything is fine.

The power came back in 30 minutes. So what is taking five days?

When I was in aerospace, we had a test environment that was supposed to mirror the flight hardware exactly. It did, on day one. Then an engineer applied a debug patch during a late-night session and never rolled it back. A vendor shipped a slightly different firmware version mid-program and nobody caught it. A workaround for a timing issue became permanent because removing it would require recertification.

By the time we discovered the divergence, we had validated months of flight software against the wrong baseline. That was a $2M mistake and a six-month schedule slip. The root cause was that the system for tracking configuration was completely separate from the system where configuration actually lived. The documentation said one thing. The hardware said another. Nobody knew until something broke.

Multiply that by 170 million users trying to reconnect at once, and you have what almost certainly happened to TikTok.

The Three States Problem

Every large infrastructure system has three versions of the truth that are supposed to be identical but never are: what the documentation says is running, what the automation actually deployed to the servers, and what is executing on the hardware right now.

In theory, these three match perfectly. In practice, they drift apart constantly, and the drift is invisible until something breaks.

An engineer logs into a server to debug a performance problem and changes a system setting. The latency improves, so she moves on to the next fire. The documentation never gets updated, and the running system has now diverged from the official record without anyone logging it. A software update gets pushed to a new set of servers, but the deployment script times out halfway through, leaving half the machines on version 2.3.1 and half on version 2.3.0. Both versions work fine, so nobody notices for months. A backup system references a traffic routing configuration that got renamed six months ago, and since the primary system uses the new name, nobody realizes the backup path has never been tested since the rename.

Now the power goes out.

What Recovery Actually Looks Like

When power returns, the system tries to fail over to backup configurations. Services that were assigned to specific servers cannot find their targets because the address lookup system cached outdated records during the outage. Machines that come back online pull configuration from a central source that does not match what was actually running before the outage. Applications restart with settings that reference security credentials that were rotated months ago but never propagated to the backup systems.

Every small divergence becomes a blocking dependency, and each fix reveals two more inconsistencies underneath it. Recovery is not a matter of turning the power back on. Recovery means figuring out what was actually running before the outage, attempting to recreate that state, and then realizing you cannot recreate it because nobody recorded what that state was.

The power came back in 30 minutes. The configuration archaeology takes a week.

Share

Why AI Makes This Exponentially Worse

Traditional data center equipment used 10 to 15 kilowatts of power per unit. AI equipment uses 100 to 132 kilowatts. The next generation of systems arriving this year will hit 240 kilowatts, and by 2028, projections show single units consuming 1 megawatt, which is enough electricity to power 800 homes.

The jump from 15 kilowatts to 132 kilowatts brings an entirely different set of configuration requirements. AI equipment requires liquid cooling because air cannot remove heat fast enough at these densities, which means managing flow rates, coolant temperatures, pump speeds, pressure levels, and fluid reservoirs. Each parameter represents a new setting that can drift without anyone noticing. Most AI deployments run hybrid systems with 70 to 80 percent liquid cooling and 20 to 30 percent air cooling, and these two systems have separate configuration requirements that must stay synchronized with each other.

Meanwhile, the engineers who understood the old systems are leaving. Amazon Web Services laid off 27,000 employees between 2022 and 2025, and institutional knowledge walked out the door while system complexity was exploding in the opposite direction. The engineers debugging outages today were not there when the systems were designed.

AI model training iterations also move faster than infrastructure teams can document them. A research team spins up a new cluster configuration for a training run, it works, and they move on. Six months later that cluster is running production workloads and nobody remembers what made it different from the standard setup.

Why Automation Tools Cannot See Reality

The technology industry spent a decade building tools to automate infrastructure deployment. The idea was simple: define what you want your systems to look like in a configuration file, and let software automatically make it so. Billions of dollars have been invested in this approach, and every major technology company uses these tools.

The architecture has a fatal flaw in that it only works in one direction. These tools are excellent at pushing settings out to servers, but they have no ability to verify what is actually running. The deployment system knows what it sent, but it does not know if the settings were applied correctly, it does not know if someone changed them afterward, and it does not know if a process crashed and restarted with different parameters.

The entire model assumes the world does not change after deployment touches it. The world changes constantly. Someone logs in to fix an urgent problem. A software update fails partway through. A service crashes and restarts with stale settings from a cached configuration. The industry invested billions in tools for pushing state, but the corresponding tools for verifying that the state actually stuck do not exist.

Why This Gap Still Exists

The infrastructure software market is worth over $300 billion. Thousands of companies sell tools for deployment, monitoring, and orchestration, and enterprises spend millions on automation. The October 2025 AWS outage cost businesses $75 million per hour, the Microsoft 365 outage earlier this month halted global operations, and the CrowdStrike incident in 2024 caused $5.4 billion in losses.

These outages are the predictable result of a gap in the market that nobody has closed. The companies that figure out continuous state verification will be sitting on something enormously valuable, because every major outage in the last two years, if you read the post-mortems carefully, comes back to some version of “we thought the system was configured one way, but it was actually configured differently.”

What This Means for Oracle

Oracle built its entire 2025 narrative around being the trusted infrastructure partner for AI. The company has $124 billion in debt funding a $50 billion capital expenditure plan, and it has a $523 billion contract backlog that it needs to execute against. TikTok alone represents $800 million in annual revenue, roughly 5 percent of Oracle Cloud Infrastructure.

Share CipherTalk

The TikTok deal was supposed to prove Oracle could handle security-sensitive, compliance-heavy workloads at consumer scale, which was the pitch to healthcare systems, financial institutions, and government agencies, and the justification for the OpenAI partnership and the massive Stargate data center buildout in Texas.

Then a winter storm hit and Oracle could not recover a single application in under a week.

Watch Oracle’s enterprise sales pipeline over the next two quarters. The executives evaluating Oracle for AI infrastructure just watched a live demonstration of what happens when Oracle’s operations fail, and they are not going to forget. Amazon, Microsoft, and Google will reference this outage in every competitive deal through 2026.

Oracle is not going bankrupt, but the “trusted infrastructure partner” narrative just took damage that marketing cannot repair. Only flawless execution can fix it, and this outage proved that execution is exactly where Oracle falls short.

The Systemic Risk No One Is Pricing

Oracle got caught because TikTok is a consumer app. When enterprise workloads fail, the damage gets buried in SLAs and service credits. AWS, Azure, and Google Cloud have the same architectural vulnerability - they’ve just been lucky about which workloads were running when something broke.

The AI infrastructure buildout represents over $200 billion in capital allocation in 2025. Due diligence focuses on power, real estate, and demand forecasts. Almost nobody is asking whether operators can recover systems to a known state after failure. The answer, for nearly everyone, is no. That risk is not in any valuation model.

The next major outage won’t be a cyberattack. It will be a configuration that drifted six months ago and went unnoticed until a failover exposed it. When that hits a foundation model training run three weeks deep, the losses will make TikTok look like a rounding error.

The Question

Most infrastructure teams cannot answer this question with confidence:

“Is the configuration running in production right now identical to what is documented?”

If you cannot say yes immediately, you are one winter storm away from finding out what TikTok just learned.

What is the worst configuration disaster you have witnessed? The 3 AM phone calls. The “wait, why is this different?” moments. I want to hear them.

In the past six months, OpenAI signed a $10 billion chip deal with Broadcom. Amazon is deploying half a million custom Trainium chips for Anthropic. Google’s TPU capacity sold out across three regions, demand exceeding supply by 340%. Cerebras, the wafer-scale insurgent, is preparing a $22 billion IPO. Microsoft’s custom Maia accelerators are now in production.

The AI industry just crossed a threshold that changes everything about who wins the hardware race. Understanding why requires grasping a distinction that most people outside the industry miss: training versus inference.

Subscribe now

The Split That Changes Everything

Training teaches an AI model to recognize patterns. You feed it enormous datasets, adjust billions of parameters through iterative computation, and eventually produce something like GPT-4 or Claude. Training is computationally brutal. It requires months of processing on thousands of the most powerful chips available.

Inference is using that trained model. Every ChatGPT response, every AI-generated image, every autonomous vehicle decision runs inference. The model is already trained; now it processes your specific input and produces an output.

The economic reality is stark: training a frontier model might require dozens to hundreds of chips for weeks or months. Deploying that model to millions of users requires tens of thousands of chips running continuously. OpenAI’s inference cluster is reportedly over 10x larger than its training infrastructure. ChatGPT alone handles hundreds of millions of queries per week.

In early 2026, global spending on AI inference officially surpassed spending on training. Industry analysts call it the “Inference Flip.” And it changes which chips matter most.

Why GPUs Won Training (And Why That’s Changing)

Nvidia’s dominance emerged from a historical accident. GPUs were designed for rendering video game graphics: millions of simple calculations performed simultaneously. Researchers discovered in 2012 that the same parallel architecture was perfect for training neural networks. AlexNet, the model that kicked off the modern AI era, ran on Nvidia GPUs.

Since then, Nvidia has relentlessly optimized for AI training. The company controls roughly 90% of the AI training chip market. Its CUDA software ecosystem locks developers into Nvidia hardware. The new Blackwell architecture supports massive model training with NVLink 6 technology enabling seamless interconnection of 72-card clusters.

GPUs are general-purpose. They’re designed to handle everything from video game textures to scientific simulations. That flexibility comes with overhead: complex caching, branch prediction, thread management. A GPU is a Swiss Army knife.

For inference, you don’t need a Swiss Army knife. You need pure precision.

Enter the ASIC

An ASIC (Application-Specific Integrated Circuit) is a chip designed to do one thing extremely well. Unlike GPUs, which can handle diverse workloads, ASICs are hard-wired for specific calculations. They strip away all the architectural baggage that GPUs carry.

The result is dramatically better efficiency. Google’s TPU v5e delivers 3x efficiency versus Nvidia’s H100 for inference workloads. Amazon’s Trainium 2 offers 30-40% better price-performance. ASICs can be 40% better than GPUs for the same task, according to Alchip’s CEO.

Energy efficiency matters enormously at scale. AI data centers are driving up electricity prices across the country. A chip that consumes 67% less power while delivering comparable performance represents massive cost savings when you’re running millions of queries.

ASICs are inflexible. Design cycles take 12-24 months. If your AI architecture changes faster than your chip design, you end up with expensive silicon that can’t run your new models. GPUs handle this risk by being general-purpose. They can run whatever algorithms you throw at them.

Inference workloads are more stable than training workloads. The transformer architecture that powers most large language models hasn’t fundamentally changed in years. Once a model is trained, its inference requirements are predictable. That predictability makes ASICs viable.

Google’s Quiet Victory

While everyone watches Nvidia, Google may have already won the inference war.

In October, Anthropic signed a deal worth tens of billions to access up to one million Google TPUs. Meta is in advanced negotiations for a multibillion-dollar TPU deployment, starting with cloud rentals in 2026 and on-premise installations in 2027. Midjourney reportedly slashed costs 65% after switching to TPUs. Google, which kept TPUs locked inside its own cloud for a decade, is now selling directly into customer data centers.

The TPU v7, codenamed Ironwood, delivers 10x improvement over the previous generation and is purpose-built for inference workloads. Google Cloud CEO Thomas Kurian cited “strong price-performance” when announcing the Anthropic deal. Industry analysts estimate TPUs deliver 4x cost-performance versus Nvidia GPUs for inference-heavy workloads.

The implications for Amazon are uncomfortable. Anthropic, despite AWS being its “primary training partner” with $8 billion invested, chose to massively expand its Google TPU usage. The company now runs a diversified stack across Google TPUs, Amazon Trainium, and Nvidia GPUs. Jefferies analyst Blayne Curtis noted that Anthropic’s growing use of Google’s chips “doesn’t reflect well” on Amazon’s Trainium processors.

AWS isn’t retreating. At re:Invent 2025, Amazon announced Trainium3, its first 3nm AI chip, and previewed Trainium4 with NVLink Fusion integration. The company canceled Inferentia, betting that Trainium can handle both training and inference. Project Rainier continues to deploy hundreds of thousands of Trainium 2 chips for Anthropic.

The deeper signal: Google’s decade of TPU development is paying off. The company that pioneered the transformer architecture in 2017 now offers the most mature ASIC ecosystem in the industry, with nine generations of refinement. Amazon started building custom AI silicon later and is still catching up on software maturity. The Neuron SDK works, but it lacks the years of optimization that make TPUs feel native to researchers.

The Great Unbundling

Every major AI company has now reached the same conclusion: they can’t depend entirely on Nvidia.

Meta has MTIA. Microsoft has Maia and Athena.

OpenAI, Nvidia’s most important customer, is building custom chips with Broadcom and TSMC. The first custom processors, using TSMC’s 3nm process, are targeting mass production this year. The chip will primarily handle inference, reducing OpenAI’s dependence on Nvidia for production workloads.

This isn’t about cutting Nvidia out entirely. Training still requires GPUs, and Nvidia’s training chips remain unmatched. The hyperscalers are bifurcating their infrastructure: GPUs for training and rapid prototyping, custom ASICs for production inference.

IDC predicts ASIC share in inference scenarios will grow from 15% to 40% by 2026, potentially reaching 70-80% of production inference by 2028. Nvidia will likely maintain 90%+ share of training but could fall to 20-30% of inference.

Cerebras: The Architectural Outlier

Not everyone is building conventional ASICs. Cerebras Systems has taken a radically different approach.

Last week, OpenAI signed a $10 billion deal with Cerebras Systems for 750 megawatts of inference compute through 2028. The announcement validated a radically different approach to AI chips.

Traditional chips are carved from silicon wafers, each wafer yielding dozens of individual processors. Cerebras uses the entire wafer as a single chip. The Wafer-Scale Engine 3 is literally the size of a dinner plate: 4 trillion transistors, 900,000 AI-optimized cores, 125 petaflops of compute.

The advantage comes from eliminating interconnect overhead. Traditional GPU clusters spend enormous energy shuttling data between chips. Cerebras keeps the entire model on-chip, bypassing the memory wall that constrains conventional architectures. The company claims 21x faster inference than Nvidia’s flagship systems. OpenAI is particularly interested in using Cerebras for coding workloads, where response latency directly affects developer productivity.

The deal transforms Cerebras’ business. Previously, UAE-based G42 accounted for 87% of revenue, creating concentration risk that spooked public market investors. OpenAI diversifies that exposure and positions the company for a Q2 2026 IPO at a $22 billion valuation, nearly triple its $8.1 billion valuation from September.

The contrast with Groq is instructive. Groq, another inference-focused chipmaker, was absorbed by Nvidia in December for $20 billion in a deal structured to avoid antitrust scrutiny. Cerebras chose partnership over acquisition, betting that independence and an IPO offer more upside than getting folded into an incumbent.

Sam Altman has been an investor in Cerebras since 2017. OpenAI evaluated the technology that year, and Elon Musk attempted to acquire the company in 2018. The relationship finally converted to a commercial deal after Cerebras demonstrated last August that OpenAI’s gpt-oss models ran faster on wafer-scale chips than on conventional GPUs.

Share CipherTalk

The Stargate Factor

The scale of upcoming AI infrastructure makes all these dynamics more consequential.

The Stargate Project, announced in January, plans to invest $500 billion over four years building AI data centers across the United States. The initial equity funders (SoftBank, OpenAI, Oracle, and MGX) are deploying $100 billion immediately. The first Texas facility, near Abilene, will house Nvidia GB200 chips with deployment beginning this summer.

The memory requirements alone reshape global supply chains. Samsung and SK Hynix have preliminary agreements to supply Stargate up to 900,000 wafers monthly, potentially 40% of global DRAM output. Memory prices are expected to rise another 40% through Q2 2026.

Amazon’s Project Rainier, built specifically for Anthropic, aims to be the “world’s largest” AI compute cluster, larger even than Stargate. When TIME visited the data center, executives demonstrated racks of Trainium 2 chips, each fridge-sized unit containing 64 processors.

These aren’t data center expansions. They’re infrastructure bets on what Anthropic CEO Dario Amodei calls “powerful AI” arriving as early as 2026. The companies building these facilities believe they might be where AGI is birthed.

TSMC: The Choke Point

Every chip company’s strategy ultimately depends on one variable: TSMC capacity.

Taiwan Semiconductor Manufacturing Company fabricates roughly 90% of the world’s most advanced chips. Nvidia, AMD, Apple, Qualcomm, Cerebras, OpenAI are all fabless. They design. TSMC builds. There is no alternative at the bleeding edge.

TSMC’s 2nm process enters mass production this year, with capacity already sold out. Pricing runs 10-20% above the current 3nm node. The company’s CoWoS advanced packaging technology, which integrates processors with high-bandwidth memory, is the current bottleneck.

Nvidia has locked up over half of CoWoS capacity through 2027. Google reportedly cut its target from 4 million to 3 million TPUs for 2026 because it couldn’t secure enough packaging slots. Morgan Stanley now projects TSMC will reach 120,000-130,000 CoWoS wafers per month by end of 2026, up from earlier estimates of 100,000.

The geopolitical implications are obvious. A Taiwan contingency isn’t a tail risk. It’s an existential threat to the entire technology stack. TSMC’s Arizona fab is ramping, but capacity is limited and the technology lags Taiwan by a generation.

Now Add Tariffs

Into this complex landscape, drop the Trump administration’s semiconductor policy, which manages to be both aggressive and contradictory.

On January 14th, eight days ago, President Trump signed a proclamation imposing a 25% tariff on certain advanced AI chips, specifically Nvidia’s H200 and AMD’s MI325X. The administration framed it as protecting national security.

Simultaneously, the Commerce Department revised its licensing posture for these chips from “presumption of denial” to “case-by-case review,” effectively greenlighting H200 sales to China that had been blocked under Biden’s export controls. Nvidia is preparing to ship 82,000 GPUs to Chinese customers.

The tariff only applies to chips routed through the United States before re-export. It’s a revenue mechanism, not a restriction. Trump announced the arrangement on social media in December: “25% will be paid to the United States of America.”

Congressional China hawks are pushing back. Yesterday, the House Foreign Affairs Committee voted 42-2 to advance a bill requiring export licenses for advanced AI chips. Trump’s former Asia advisor Matt Pottinger testified last week that the administration is on the “wrong track.” White House AI advisor David Sacks fired back on X.

Meanwhile, buried in the Federal Register, the Office of the U.S. Trade Representative announced new tariffs on Chinese semiconductor imports covering diodes, transistors, and integrated circuits, taking effect June 2027. The initial rate is zero percent. The actual rate won’t be announced until 30 days before implementation.

The policy captures a real tension. America wants to dominate AI. American companies need to sell chips to fund R&D. China is building its own stack regardless of export controls. Huawei’s Ascend chips violate U.S. restrictions according to Commerce Department assessments, but they exist. Export controls buy time. They don’t stop technological development.

The New Competitive Landscape

Here’s what’s actually happening beneath the policy noise.

Nvidia remains dominant in training. The company holds a $275 billion backlog, roughly 90% market share in AI training accelerators, and privileged access to TSMC capacity. The Rubin architecture extends its roadmap through 2027.

Inference is fragmenting. Hyperscalers are building custom ASICs. Specialized startups like Cerebras and Groq (now absorbed by Nvidia for $20 billion) are competing for the remainder. The market is bifurcating.

Vertical integration accelerates. OpenAI, Google, Amazon, Microsoft, and Meta are all building custom silicon. They’re not replacing Nvidia entirely, but they’re reducing dependence and capturing margin.

The memory supercycle is real. HBM4 enters production this year. DRAM prices are surging. Samsung, SK Hynix, and Micron are the only companies that matter, and all are capacity-constrained.

Consolidation continues. Intel acquired SambaNova for $1.6 billion. Nvidia absorbed Groq. Smaller AI chip startups face an existential choice: scale quickly, get acquired, or fade.

What This Means

If you’re building AI products, your inference costs are about to change dramatically. The companies running dual-track procurement (GPUs for training, ASICs for production) are seeing 30-50% lower total cost of ownership. The companies still 100% on Nvidia are carrying hidden technical debt.

If you’re investing, Nvidia isn’t going anywhere. The bull case assumes 80%+ share of the entire AI compute market indefinitely. That share is eroding in inference, which will be 75% of future AI compute. Adjust expectations accordingly.

If you’re watching geopolitics, semiconductor policy is now embedded in every strategic plan. Cerebras restructured its cap table to pass national security review. Nvidia navigates shifting China rules quarterly. No chip company operates outside this reality.

The semiconductor industry is approaching $1 trillion in annual sales. AI server spending alone may hit $312 billion this year. The infrastructure buildout is unprecedented.

The architecture of that buildout is shifting. Training made Nvidia. Inference may unmake its monopoly, not through competition on training, but through the simple economics of running AI at scale.

The Inference Flip is here. The companies that recognized it early are building alternatives. The companies that didn’t are about to find out what it costs.

The gap between what AI can do in a controlled environment and what AI can do in your driveway or at 30,000 feet comes down to something most people never think about: the low-level tooling that connects software to physical machines, and the shrinking pool of engineers who know how to use it.

The infrastructure teams, embedded engineers, and people who actually have to make these systems work have a story to tell. And it has almost nothing to do with models or compute.

The Invisible Complexity

Here is what the deployment stack looks like for a single rack of servers in a modern data center:

Count the orange boxes. Each one represents a proprietary tool that requires specialized knowledge. HPE iLO speaks a different language than Dell iDRAC. NVIDIA’s CUDA ecosystem shares almost nothing with AMD’s ROCm. Cisco’s network operating system has no overlap with Arista’s or Juniper’s.

An engineer who has spent years mastering one vendor’s stack walks into a different data center and starts over from scratch. The knowledge does not transfer. The muscle memory does not apply.

This fragmentation extends far beyond data centers. Modern vehicles contain over 500 million lines of code distributed across 80 to 150 electronic control units. Aircraft require DO-178C certification that can take five or more years and cost in excess of $25 million per system. Edge devices run on heterogeneous architectures combining CPUs, neural processing units, and digital signal processors, each requiring its own configuration and debugging approach.

The Human Bottleneck

The people who understand these systems are disappearing. According to IEEE Spectrum, 58% of data center operators worldwide struggle to find qualified talent. CBS News reports that 400,000 skilled trade positions remain unfilled in America, with projections showing that number reaching 2 million by 2033. The Uptime Institute estimates that half of all data center engineers could retire within the next three years, while demand for these specialists grows by 300,000 over the same period.

This matters because physical AI requires physical infrastructure. Every autonomous vehicle needs edge compute. Every smart factory needs embedded systems. Every eVTOL needs avionics that someone has to configure, certify, and debug. The software is ready. The models are ready. The humans who can connect software to hardware are not.

The consequences show up in the numbers. Data Center Watch reported that in Q2 2025 alone, $98 billion in data center projects were blocked or delayed, exceeding all disruptions tracked since 2023 combined. A QNX study found that 52% of Vice Presidents of Engineering at automotive companies cite integration complexity as their top development challenge. AFuzion research shows that 65% of eVTOL certification delays stem from disputes over components that blur the line between hardware and software.

Why You Should Care

The self-driving features that keep getting delayed? Integration complexity. The urban air mobility revolution that was supposed to arrive by 2025? Certification bottlenecks. The AI features you are waiting for? Capacity constraints at data centers that cannot hire enough engineers to bring new racks online.

Jim Rowan, the former CEO of Volvo, put it bluntly: “You need to be able to write from layer one of the silicon all the way up to the application layer of the car in order to control it properly. There are three companies in the world that have managed to do that: Tesla, Rivian and Volvo. There’s a lot of good car companies but none of them have figured it out. It’s a big deal, and freaking hard to get this done.”

The Boeing 737 MAX disasters demonstrated what happens when hardware-software integration goes wrong. A software system called MCAS, designed to compensate for an airframe change, relied on a single sensor. When that sensor failed, 346 people died. One expert estimated that eight lines of code could have prevented both crashes. The problem was not that no one knew how to write those lines. The problem was that the complexity of the system obscured the need for them.

Physical AI multiplies this complexity by orders of magnitude. An autonomous vehicle processes terabytes of sensor data per hour. A smart factory coordinates hundreds of embedded controllers. An eVTOL integrates flight controls, battery management, and propulsion systems that all need to work together without fail. The stakes of getting the hardware-software interface wrong are measured in lives, not lost revenue.

The Abstraction That Does Not Exist

Software engineering solved this problem for computers decades ago. Developers write code once and run it on Intel, AMD, and ARM processors without modification. Operating systems abstract the hardware. Compilers handle the translation. Productivity follows.

Below the operating system, no equivalent abstraction exists for physical systems. Every vendor ships proprietary tools. Every hardware generation requires relearning. Every deployment becomes a bespoke integration project.

Industry Lines of Code / Complexity Key Bottleneck Automotive 500M+ lines across 80-150 ECUs 52% cite integration complexity as top challenge Aerospace DO-178C certification: 5+ years 65% of eVTOL delays from hybrid component disputes Data Centers 17+ vendor tools per rack 58% cannot find qualified talent Edge/IoT CPU + NPU + DSP per device Hardware fragmentation across platforms

Construction spending tells part of the story. JLL reports that global data center construction costs have risen 39% in five years, from $7.7 million per megawatt in 2020 to $10.7 million today. But the money exists. Microsoft committed $80 billion to AI infrastructure in 2025 alone. The constraint is not capital. The constraint is that the industry cannot hire its way out of a problem created by decades of vendor fragmentation.

What Would Have to Change

For physical AI to deliver on its promise, the industry needs what I call “the CUDA for everything below the OS.” NVIDIA built CUDA to give developers a unified way to program GPUs. That abstraction unlocked the deep learning revolution. The same abstraction needs to happen for embedded systems, baseboard management controllers, network operating systems, and device configuration.

An engineer should be able to declare what a system needs to accomplish and have the tooling translate that intent into vendor-specific commands. Deployment timelines could compress from weeks to days. The existing pool of infrastructure engineers could accomplish more. The capacity the entire AI industry requires could actually come online.

The vendors have no incentive to build this. The consulting firms profit from complexity. The hyperscalers can afford to hire their way around the problem, though even they are struggling. Everyone else waits for a future that the tooling cannot yet deliver.

The physical AI era that the industry keeps celebrating depends on infrastructure that already runs into human limits. Until the tooling catches up, the software-defined vehicles and electric aircraft and edge AI systems will arrive later than the keynotes promise. The constraint on your AI future might be a 55-year-old embedded engineer in Wichita who knows how to certify flight control software, and whether her employer can convince her to postpone retirement.

A robotics company I won’t name showed me their manipulation model running flawlessly, tying surgical knots in simulation with sub-milimeter precision. Eighteen months of research. World-class team. The model worked.

Then they tried to deploy the model to their actual robots.

Six months later, they were still debugging. The model that ran perfectly in the cloud choked when touching real silicon. Timing issues, memory constraints, thermal throttling, sensor calibration drift. The neura

l network was fine. The hardware was fine. The boundary between them was a disaster.

That company raised hundreds of millions of dollars. And they lost half a year to a problem that has no name, no category, no solution on the market.

This story repeats across every industry building physical AI. And nobody’s talking about why.

Not-so-subtle disclosure: I've written about infrastructure bottlenecks for years on CipherTalk. This one is different. My cofounder and I started Cosmic Labs to build what this essay describes. Consider this the thesis.

The Language Problem

Something most people in tech don’t realize is that software and hardware ‘speak’ completely different languages. Not programming languages, but the fundamental way they communicate.

When your web app talks to a server, the conversation happens in clean abstractions. HTTP requests, JSON payloads, REST APIs… the underlying complexity hides behind decades of infrastructure investment. You don’t think about packets or TCP handshakes or routing tables. You call a function and get data back.

When AI needs to talk to hardware (a GPU cluster, a robot’s motor controller, a vehicle’s sensor array), the abstractions disappear. The “conversation” happens in something closer to native electrical signals: precise timing requirements, register-level commands that vary by manufacturer and chip revision, protocols designed in the 1980s that never anticipated modern AI workloads.

Those “clean abstractions” exist because someone built them.

An abstraction is a layer that hides complexity. Instead of managing every detail directly, developers interact with a simplified interface and trust the layer beneath to handle the rest.

In the 1990s and 2000s, the internet scaled because engineers built these layers deliberately. Each one handled one job and exposed a clean interface to the layer above. Application developers didn’t need to understand routing or sockets. The whole system became modular, interchangeable, and efficient.

Hardware never got that treatment. The economics didn’t support it. Unlike software, where one protocol could serve millions of users, hardware is fragmented: different vendors, different chips, different protocols for different industries. Aerospace had its standards, automotive had its standards, industrial equipment had its standards. No single abstraction layer could serve all of them, and no single market was large enough to justify building one. Every integration is custom. Every deployment is bespoke.

So - the robotics company I mentioned? Their model worked perfectly when “hardware” meant an NVIDIA GPU in a data center. But their production robots used custom motor controllers, specific sensor chips, a particular FPGA configuration. Nobody at NVIDIA documented how to make those components play together. Nobody anywhere documented that. The knowledge existed in maybe fifty people worldwide, and those fifty people have job offers stacked to the ceiling.

Subscribe now

The Same Problem at Every Scale

A data center and a robot look nothing alike. Different sizes, different environments, different purposes. But the engineering bottlenecks? Both require complex hardware configuration before they can do anything useful. The data center needs hundreds of components aligned and communicating. The robot needs dozens of processors initialized and calibrated. In both cases, the knowledge required to do this lives in documentation that is incomplete, outdated, or wrong.

The implication: any system that can reason about hardware configuration at data center scale can reason about it at the edge. The specific chips differ. The protocols differ. The underlying problem structure does not. What is required is an intelligence layer for hardware, wherever compute meets the physical world.

The $600 Billion Assumption

Let me frame the stakes differently.

Right now, the largest companies in the world are spending over $320 billion per year building AI infrastructure. With data centers, GPU clusters, networking equipment, power plants… estimates suggest this could exceed $500 billion in 2026.

The assumption behind this spending: AI will eventually generate returns that justify the investment.

But what that assumption requires: AI has to reach the physical world.

A language model that lives in the cloud generates value by producing text. But the much larger opportunity (the one that justifies hundreds of billions per year in capex) is AI that operates physical systems. Robots that work in factories, vehicles that drive themselves, data centers that configure themselves.

Every single one of those applications requires AI to cross the hardware boundary. And right now, that crossing looks like this:

Sequoia Capital’s David Cahn has been asking what he calls “AI’s $600 billion question”: where will the revenue come from to justify all this infrastructure spending?

The bottleneck isn’t compute or algorithms. The bottleneck is the last mile: getting software out of research environments and onto physical systems at scale. Data centers that need to configure thousands of heterogeneous servers. Automotive companies pushing models to vehicle fleets. Robotics startups deploying to factory floors. Aerospace programs integrating AI into flight systems. Every one of these deployments hits the same wall: The hardware boundary has no solution.

Subscribe now

Why This Hasn’t Been Solved

The people who know how to deploy AI to hardware are approaching retirement age. They learned their craft on systems from the 1980s. They carry decades of hard-won intuition about failure modes and edge cases and undocumented behaviors.

The embedded systems engineering discipline has been supply-constrained for twenty years. Universities don’t produce enough graduates. The learning curve is brutal (5-7 years to senior competency). And 80% of embedded engineering job postings remain unfilled for months, sometimes years. The talent math:

• ↑ Physical AI investment: exponential growth, demand for hardware deployment expertise exploding

• ↓ Senior embedded engineers: flat to declining, retirements outpacing new graduates

• ↓ AI/ML engineers who understand hardware: rare and getting rarer, software culture dominates

• ↓ Documented best practices: fragmented across vendors, no standardization, no transferable knowledge

What Changes Everything

Hardware configuration has resisted automation for forty years because the work is investigative. Engineers read thousand-page datasheets, reason about unexpected behavior, and synthesize information across components that were never designed to work together, forming hypotheses and discovering the documentation was wrong.

The engineers who can do this are rare, and every data center, automotive and robotics company competes for the same small talent pool, while heterogeneous compute explodes the problem: systems now combine CPUs, GPUs, FPGAs, and custom ASICs, each with its own configuration surface. The bottleneck is a hard constraint on what organizations can build.

Rule-based automation never had a chance because the configuration space is too large, the edge cases too numerous, the documentation too unstructured for explicit knowledge encoding. Modern language models change the equation by reading datasheets the way an engineer does, tracking dependencies, reasoning about behavior. Two years ago this was impossible. But models hallucinate, and for mission-critical hardware, “usually right” is worthless.

Hardware offers ground truth. Unlike software, you can interrogate the physical system directly. This is embodied reasoning. The hard problem is maintaining coherent, multi-step reasoning while simultaneously controlling probes across dozens of protocols in real time, bidirectionally, on hardware that does not wait. The hard part is also building models that know how to do this, and gathering enough data to do it well.

There’s a reason there’s only one CUDA. Despite billions invested, no one has successfully abstracted hardware at scale. Nvidia succeeded because they controlled both the hardware and the software stack, and spent two decades building the ecosystem. Everyone else tried to build the abstraction layer without the underlying data, without the hardware relationships, without the feedback loops. They failed.

This is why the system has to be two layers. One layer reasons over documentation and system state. One layer touches physical reality. Neither works alone.

Leave a comment

What Comes Next

AI is moving from generating text to controlling machines. That transition bottlenecks at hardware.

But this constraint is starting to lift.

Organizations that could never afford six-month integration cycles can ship in weeks. Space programs that bottlenecked on three engineers who understood the full system can parallelize across teams. Robotics companies can iterate on hardware as fast as they iterate on software.

Every system that touches the physical world—data centers, satellites, autonomous vehicles, manufacturing lines—moves faster when hardware configuration stops being the long pole.

Hundreds of billions of dollars are flowing into physical AI and compute infrastructure. The returns depend on deploying that hardware, not just buying it. Deployment depends on configuration. Configuration depends on the engineers who understand it.

We need a system that learns hardware the way those engineers do, then scales what they know to every board, every rack, every factory floor.

And if you’re intersted in learning more about what I’ve been building… check this out.

-M

The numbers contradict themselves. That contradiction is the story.

Since 2022, the cost of running a query through a large language model has dropped by a factor of 1,000. GPT-3.5 once cost $12 per million output tokens; today, GPT-4o mini runs at $0.60. DeepSeek entered the market undercutting competitors by 90%. Epoch AI reports price drops ranging from 9x to 900x per year depending on the benchmark. The sticker price of intelligence appears to be in freefall.

The invoice tells a different story. Ten million dollars per megawatt to build the data centers running those models. Forty percent of a facility’s electricity devoted solely to preventing thermal collapse. Lead times of 36 to 48 weeks for switchgear and chillers. Mean time to repair stretching into hours when a GPU rack drawing 140 kilowatts throws a thermal fault at 3 AM.

Token prices are a mirage, the polished front-end of a back-end hemorrhaging capital, talent, and kilowatt-hours. The gap between what AI costs to use and what it costs to exist is where the next trillion-dollar battle will be fought.

The Stack Nobody Talks About

When people discuss AI infrastructure, they usually mean GPUs, cloud instances, and API endpoints. They think above the kernel, the software abstraction layer that makes hardware look like a well-behaved API call.

Below the kernel is a different universe.

Down there, engineers deal with bootloaders that have not been synchronized with Linux kernel fixes since 2019. Device trees that describe hardware topology in a language most software engineers have never encountered. JTAG interfaces and oscilloscopes. Firmware that controls how a processor talks to memory, how sensors convert analog signals to digital data, and how a network interface handshakes with the physical layer of a data center’s spine.

This is the domain of embedded systems, and the true configuration nightmare lives here. Unlike software, which can be patched with a git push, firmware is burned into read-only memory. A misconfigured register can brick a board. A timing mismatch can cause intermittent failures that take weeks to diagnose. No stack trace exists. No graceful degradation. When hardware below the OS breaks, it breaks catastrophically.

The uncomfortable truth: AI data centers are increasingly defined by this layer. A single Nvidia Blackwell Ultra rack will hit 140 kilowatts in 2025. Vera Rubin NVL144 systems may require 300-plus kilowatts by 2026. Google’s Project Deschutes has already unveiled a one-megawatt rack design. At these power densities, the hardware abstraction layer becomes a thermal abstraction layer, and the physical world does not abstract cleanly.

The Vendor Integration Hellscape

Building a data center in 2025 resembles less the construction of a building than the orchestration of a symphony where every musician speaks a different language and half of them are three time zones away.

The players: utility companies for grid access, fiber providers for connectivity, HVAC manufacturers for cooling, UPS vendors for power continuity, server OEMs for compute, storage vendors for persistence, networking companies for switching and routing, liquid cooling specialists for thermal management, and a constellation of sensors, controllers, and monitoring systems that all need to communicate with each other.

The problem is not that these systems fail to work. The problem is that they fail to work together without extraordinary effort.

Consider the cooling transition unfolding right now. Air cooling, which still accounts for 54% of the data center cooling market, physically cannot dissipate heat at AI-scale power densities. Cooling a 100-kilowatt rack with air would require a wind tunnel. So the industry is racing toward liquid cooling: direct-to-chip cold plates, rear-door heat exchangers, immersion systems where servers are submerged in synthetic oil.

Integrating liquid cooling into existing facilities means rethinking everything. The plumbing. The electrical distribution. The raised floors designed for airflow, not fluid dynamics. The monitoring systems built to track temperature, not flow rates and coolant pressure. According to Schneider Electric, air-based cooling already accounts for up to 40% of a typical data center’s total electricity use. Liquid cooling can cut that dramatically, but only if deployment actually happens.

Deployment means vendor integration. It means getting the cooling distribution unit to talk to the building management system. It means training technicians who have spent their careers swapping CRAC filters to now monitor microfluidic cold plates. It means supply chains that can deliver 300 W/cm² heat flux solutions when the industry was optimizing for 30 W/cm² five years ago.

Share

MTTR: The Metric That Matters

In a world where downtime costs $5,600 to $9,000 per minute, mean time to repair becomes the gravitational constant of infrastructure economics.

MTTR measures more than fixing things fast. It encompasses the entire cascade: detecting the failure, diagnosing the root cause, getting the right technician with the right parts to the right rack, executing the repair, and validating that the fix worked. A four-hour MTTR might be world-class for a custom aerospace system. For a financial data center, four hours ends careers.

The challenge with AI infrastructure is that failure modes are multiplicative. A traditional server fails in predictable ways: disk, power supply, memory. An AI training cluster fails in ways that ripple through interconnected GPU fabrics, high-bandwidth memory hierarchies, and distributed training frameworks that assume everything is working perfectly.

When something goes wrong below the OS (a firmware bug, a hardware timing issue, a thermal excursion that triggers protective throttling) the symptoms often manifest above the OS as mysterious performance degradation. The training run does not crash; it runs 40% slower. The inference latency does not spike; it drifts higher until SLAs breach. By the time anyone notices, the damage is done.

The repair process demands bridging multiple worlds. Software engineers who understand the training framework. Hardware engineers who understand the silicon. Facilities engineers who understand the power and cooling. Network engineers who understand the fabric topology. Each speaks their own language, uses their own tools, and maintains their own model of what “working” means.

The Construction Paradox

Something counterintuitive is happening: the physical construction of data centers is getting more automated. Digital twins simulate airflow and thermal loads before ground breaks. Modular designs allow prefabricated components to be assembled like building blocks. AI-based construction planning reduces clashes, cuts delays, and streamlines cost estimation.

Hardware configuration inside those buildings? Still largely manual. Still fragmented across vendor tools. Still dependent on tribal knowledge passed between engineers.

This asymmetry exists because of the kernel boundary.

Above the kernel, everything looks like software. APIs. Abstractions. Version control. Infrastructure as code. A Kubernetes cluster can be defined in a YAML file and spun up in minutes. The entire cloud computing revolution was built on making hardware look like software.

Below the kernel, everything remains hardware. Pin configurations. Register maps. Timing constraints. Signal integrity. The tools are oscilloscopes and logic analyzers, not IDEs and debuggers. Debugging involves probing physical signal lines, not setting breakpoints.

Construction sits above the kernel. Moving physical objects according to a plan is tractable for automation. Hardware bring-up sits below the kernel. Coaxing silicon to behave according to datasheet specifications written by someone who may no longer be employed is not.

This explains why a data center can be built in 18 months and then spend another 6 months in commissioning. The concrete pours fast. The silicon refuses to cooperate.

The Economics of Exponentiality

Some math that AI evangelists prefer to skip:

A 30-megawatt data center costs roughly $300 million to build at $10 million per megawatt. Annual operating expenses (maintenance, electricity, labor, water) run about 35-45% of capital costs, or $105-135 million per year. To generate a 10% IRR, that facility needs to produce around $100 million in annual revenue.

Now consider what that facility is actually doing. It converts electricity into floating-point operations. The efficiency of that conversion (measured in FLOPs per watt, or utilization rates, or tokens served per dollar of capex) determines whether the economics work.

At full utilization, a GPU cluster generates extraordinary returns. At 10% utilization, it converts capital into waste heat. The difference between 10% and 90% utilization often comes down to the unglamorous work of keeping hardware operational: fast failure detection, rapid repair cycles, predictive maintenance, and the ability to hot-swap components without disrupting adjacent workloads. And the faster each GPU is brought to life + configured, the faster it starts making money.

This is why the “agents are cheaper than employees” narrative misses the point. Yes, inference is cheap per query. Yes, thousands of AI agents can run for the cost of one knowledge worker. But those agents need hardware. That hardware needs power, cooling, networking, and maintenance. The true cost of an AI agent is not the token price but the amortized cost of the infrastructure that makes those tokens possible. And right now, that token price is too subsidized to reflect the true cost of compute.

The price of a token can drop 1,000x and still leave you underwater if infrastructure costs grow faster than utilization.

The Energy Reality

By 2030, data centers could consume 2,200 TWh of electricity globally, equivalent to India’s entire power consumption. In the United States alone, AI servers went from consuming 2 TWh in 2017 to over 40 TWh in 2023.

This is not a future problem. It is a present constraint.

Northern Virginia, the largest data center market in the world, is already hitting power availability limits. New construction is slowing because the grid cannot deliver enough electrons. In PJM’s Mid-Atlantic region, data centers accounted for more than 60% of capacity market price increases, adding $9.3 billion in costs that are being passed to residential customers.

The ISPs and utilities are no longer infrastructure partners. They are bottlenecks. A data center can have the latest GPUs, the most advanced cooling, and the fastest networking, but if the grid cannot deliver stable power, none of it matters.

The thermal side is equally constrained. Cooling systems account for 40% of a facility’s energy use. That represents 1.2% of U.S. energy consumption devoted not to processing data, but to removing the heat that processing generates. The physics are unforgiving: every watt of compute becomes a watt of heat that needs to go somewhere.

Liquid cooling can reduce that energy overhead by up to 50%. But adoption is slow because, again, it requires rethinking everything: the facility design, the maintenance procedures, the vendor relationships, the training pipeline. Most data centers are not ready for liquid cooling of any type, whether immersion or direct-to-chip.

The Valuation Disconnect

Consider the cognitive dissonance.

Token prices are dropping 50-200x per year for equivalent capability. Yet data center deal volume more than doubled from $26 billion in 2023 to $57 billion in 2024. The cost of a ChatGPT query is approaching zero. The cost of the infrastructure running ChatGPT is approaching infinity.

Part of this is Jevons’ Paradox: cheaper tokens lead to more token consumption, which increases total infrastructure demand. A query that once returned 200 tokens now returns 2,000 tokens because reasoning models “think out loud” before responding.

The deeper issue is that token prices and infrastructure costs exist on different timescales. Token prices can drop overnight when a new model is released. Infrastructure costs are locked in for decades once ground breaks.

A hyperscaler building a billion-dollar campus today is betting that demand will persist through 2035 and beyond. That bet depends on the AI industry continuing to grow, but also on the infrastructure industry’s ability to keep pace with power density, cooling requirements, and maintenance complexity.

If the infrastructure cannot scale, token prices become academic. Cheap tokens cannot be served on hardware that does not exist.

What Comes Next

The winners in AI infrastructure will not be the companies with the best GPUs. They will be the companies that solve the unglamorous problems: vendor integration that actually works, MTTR measured in minutes instead of hours, thermal management that scales to megawatt racks, and commissioning processes that do not require six months of hand-tuning.

The future is not more automation of construction. Construction is already getting automated. The future is automation of configuration, the below-the-kernel work of making hardware behave reliably at scale.

Today, that work is done by a shrinking pool of engineers who understand both hardware and software, who can debug firmware at 3 AM with an oscilloscope and sheer stubbornness. Tomorrow, it will need to be done by systems that can sense, diagnose, and repair hardware issues faster than humans can respond.

Not because humans lack capability. Because the math demands it. At $9,000 per minute of downtime, every hour of MTTR costs $540,000. At petawatt-scale deployments, there are not enough engineers to keep the lights on.

The token price will keep dropping. The headline cost of AI will keep falling. Underneath it all, the infrastructure will keep getting more expensive, more complex, and more critical.

The mirage will persist. The bill will come due.

All figures cited are from publicly available sources as of late 2025. Infrastructure costs vary significantly by region, facility tier, and deployment type.

The poster session on the second day of NeurIPS (a big annual ML conference) had the usual chaos: researchers jockeying for position, coffee cups abandoned on windowsills, the low hum of a hundred simultaneous conversations about gradient descent and attention mechanisms.

But one exchange stuck with me. A young chemist presenting work on catalyst discovery mentioned, almost offhandedly, that she’d be starting at Periodic Labs in January. Her collaborator, still at MIT, asked if she’d keep her adjunct appointment. “Maybe,” she said. “Depends on whether I have time.”

That casual fluidity between institutions would have been remarkable a decade ago. At NeurIPS last week in San Diego, it was everywhere. The boundaries that once separated academic research from commercial application, that kept university labs and corporate R&D in distinct orbits, have become porous in ways that are reshaping how science gets done, who funds it, and where the next generation of breakthroughs will come from.

The New Physics

A category of company is emerging that doesn’t fit neatly into existing taxonomies. Call it physical AI, or science AI, or what Lila Sciences boldly terms “scientific superintelligence.” These ventures share a common conviction: that the next frontier of artificial intelligence lies not in generating text or images, but in understanding and manipulating the physical world.

• Periodic Labs surfaced in October with $300M raised and a pedigree that reads like a roster of AI royalty. Liam Fedus helped create ChatGPT. Ekin Dogus Cubuk led materials and chemistry research at Google DeepMind, where his team discovered 2.2 million new inorganic crystals. Their thesis is stark: large language models trained on internet data will plateau. Genuine scientific reasoning requires AI systems that can formulate hypotheses, design experiments, and learn from physical outcomes. More than twenty researchers left OpenAI, DeepMind, Meta, and Apple to bet their careers on that idea.

• Physical Intelligence approaches the problem from the robotics side. Founded in 2024 by former DeepMind researchers and academics from Stanford and Berkeley, the company raised $600 million in November at a $5.6 billion valuation. Alphabet’s CapitalG led the round. Jeff Bezos came back for more. The goal, as CEO Karol Hausman has described it, is building “a single generalist brain that can control any robot.” Their reinforcement learning techniques have doubled robotic throughput in testing, with machines now handling tasks from espresso preparation to laundry folding.

• Lila Sciences emerged from Flagship Pioneering, the firm behind Moderna, with a $200 million seed round in March. A $350 million Series A followed in October, with Nvidia participating, bringing total funding north of $550 million and valuation past $1.3 billion. The company operates what it calls AI Science Factories: autonomous laboratories where robotic systems conduct thousands of experiments without human intervention, generating proprietary datasets no competitor can access. The early results include novel antibodies, catalysts for green hydrogen production, and carbon capture materials that outperform commercial alternatives.

• PhysicsX takes yet another angle. Founded by Robin Tuluie, former head of R&D at Mercedes and Renault F1, and Jacomo Corbo, former chief scientist at QuantumBlack, the London-based company builds AI tools for engineering simulation. A $135 million Series B in June pushed valuation to just under $1 billion. Revenue has more than quadrupled over two years. The core offering: physics predictions that run 10,000 to a million times faster than traditional numerical simulation.

  

Following the Money

The capital flowing into physical AI reflects broader patterns in venture funding, but with distinct characteristics that signal something beyond typical hype cycles.

U.S. startup funding rose 75.6% in the first half of 2025, reaching $162.8B. AI deals now represent more than half of total venture capital allocation globally. Robotics startups have pulled in over $6B this year, on pace to exceed 2024.

What distinguishes physical AI investment is the composition of the cap tables. PhysicsX counts Siemens, Temasek, and Applied Materials among its backers, alongside traditional venture firms like Atomico and General Catalyst. That mix of industrial giants, sovereign wealth, and Silicon Valley capital suggests conviction across investor categories that rarely align.

Defense applications are also accelerating the trend. Northrop Grumman partnered with Luminary Cloud to apply physics-based AI to spacecraft design, compressing development timelines from years to months. The underlying model, built on Nvidia’s PhysicsNeMo framework, generates high-fidelity thruster simulations in seconds. Juan Alonso, Luminary’s CTO and chair of aeronautics at Stanford, frames the opportunity bluntly: “We can’t find the data for the latest rocket thruster on the internet.”

Leave a comment

The Great Reshuffling

Scientific talent is leaving academia faster than at any point in recent memory, and the causes trace directly to Washington.

The money matters. Research scientists at Series D startups command stock grants between $2 million and $4 million. Universities can’t match that. But compensation alone doesn’t explain the speed. Federal research funding collapsed this year. The NIST layoffs in February gutted the U.S. AI Safety Institute. Johns Hopkins cut 2,000 workers after $800 million in federal funding disappeared. Graduate programs rescinded PhD offers. Scientists applied for international jobs at a 32% higher rate than last year.

The private sector is catching them. In November, Yann LeCun, Meta’s chief AI scientist and a Turing Award winner, announced he’s leaving to start his own company. When Periodic Labs launched in October, more than twenty researchers from OpenAI, DeepMind, Meta, and Apple joined them. These aren’t junior hires chasing better salaries. These are people who built the field.

So is this what policy intended? The CHIPS Act passed to rebuild domestic manufacturing. Defense modernization assumes access to engineering talent. But the same administration that championed those goals gutted the institutions that produce the people. One read: they expect the private sector to absorb researchers more productively. And maybe that’s right. The startups profiled above move faster and have produced real results. The other read: foundational science takes decades to pay off, and venture capital doesn’t wait that long. Bell Labs could afford patience because AT&T was a regulated monopoly. Startups answer to different pressures.

What Gets Built

The practical iYou are not allowed to have any words, and the photos must be overlapping. It must be like inspired by this. mplications of physical AI extend well beyond academic interest. These systems promise to transform how products get designed, how materials get discovered, and how scientific research itself gets conducted.

PhysicsX already works on turbine optimization and aerospace applications, cutting simulation times from hours to seconds. Their Large Physics Models, trained on data from Siemens simulations, encode engineering knowledge in ways that compound over time. Lila’s autonomous labs have produced discoveries across chemistry, biology, and materials science. Physical Intelligence pushes toward robots capable of operating in unstructured environments, learning from experience rather than explicit programming.

The energy transition offers perhaps the clearest application domain. Catalyst discovery for green hydrogen. Materials optimization for carbon capture. Battery chemistry exploration. Grid management. These problems share characteristics that make them well-suited to AI-driven approaches: vast search spaces, expensive experimentation, and high economic stakes.

Defense applications carry similar logic. When Northrop Grumman can iterate on thruster designs in seconds rather than months, development cycles compress and design possibilities expand. The same capability applied to autonomous systems, materials science, or electronic warfare creates advantages that compound over time.

The Institutional Question

Whether venture-backed labs can sustain foundational research remains genuinely uncertain. The concern among academics is that commercial pressure biases toward near-term applications at the expense of fundamental inquiry. The counterargument, articulated by founders like Lila’s Geoffrey von Maltzahn, is that internet-scale training data has inherent limits. Breakthroughs require generating new data through experimentation, which requires resources that exceed what most universities can provide.

Periodic Labs positions itself explicitly as a successor to Bell Labs, the institution that produced the transistor, the laser, Unix, and seven Nobel Prizes. The comparison is aspirational, but it captures something real about the ambition. These companies are not building incremental improvements. They are attempting to change how scientific discovery happens.

The hybrid models emerging at NeurIPS suggest the binary framing of academia versus industry may already be obsolete. Researchers hold simultaneous appointments. Startups fund university collaborations. Corporate labs publish in top venues. The boundaries blur not because anyone planned it, but because the problems demand resources and talent that no single institution can provide.

Share

What Comes Next

The chemist I met at the poster session will start at Periodic Labs next month. Her MIT collaborator may or may not keep the adjunct appointment. Neither seemed particularly anxious about the ambiguity. The interesting problems are wherever the interesting problems are.

That pragmatism captures something about the current moment. The institutional frameworks that organized scientific research for decades are not collapsing so much as becoming optional. Talent flows toward capability, capital follows talent, and capability concentrates in new configurations that don’t map cleanly onto existing categories.

Whether this produces a new golden age of applied science or a fragmented landscape where capability concentrates in a handful of well-funded ventures while the broader research ecosystem atrophies likely depends on decisions that haven’t been made yet. Policy choices about research funding. Corporate choices about long-term investment. Individual choices about where to build careers.

What seems clear, walking out of NeurIPS into the sunny San Diego afternoon, is that the old equilibrium is gone. The new one is still taking shape.