In 2024, spending $1 on AI infrastructure felt simple: buy Nvidia, ride the GPU supercycle, collect returns. By 2027, that same dollar has passed through eight distinct hardware layers — each with radically different economics — and the profit has concentrated in places most investors weren't watching in 2024.
The AI profit pool is not shrinking. It is migrating.
The 2024 Stack: Chip Designers Captured Everything
In 2024, the AI hardware buildout was overwhelmingly a chip story. Nvidia's H100 GPU carried gross margins above 75%, making it the most profitable mass-produced hardware product in modern semiconductor history. The entire value chain downstream — server assemblers, rack integrators, cooling vendors — operated on margins ranging from high single digits to the low twenties.
Of every $1 of AI server spend, roughly $0.36 was retained as gross profit across the entire stack. Chip designers captured $0.14, HBM memory suppliers captured $0.09, and optical transceivers captured $0.05. Server OEMs, rack integrators, and power vendors split the remaining $0.08. The rack assembler's margin was essentially a rounding error on the chip designer's margin.
That concentration was driven by scarcity. Nvidia's CUDA software ecosystem created switching costs that allowed the company to price far above fabrication cost. TSMC's CoWoS advanced packaging was sold out through 2025. HBM3E from SK Hynix was in allocation. When supply is constrained and demand is structurally growing, the bottleneck owner captures the rent.
Segment-by-Segment: 2024 Gross Margins
Chip Design (Nvidia, Broadcom XPU, AMD): 60–75% gross margin. Nvidia alone earned more operating profit from AI chips in 2024 than the entire server industry combined.
Silicon Foundry (TSMC): 53–55% gross margin. CoWoS capacity ran at near-100% utilization, supporting pricing power unusual for a manufacturer.
HBM Memory (SK Hynix, Samsung, Micron): 35–45% gross margin for HBM specifically, versus 15–20% for commodity DRAM. SK Hynix controlled 53–62% of the HBM market. Prices jumped 60%+ between 2024 and 2025.
Optical Transceivers (Coherent, Lumentum): 25–35% gross margin. The shift from copper to optical interconnects inside data centers accelerated in 2024, but the market remained fragmented at the low end.
Server Racks and OEMs (Dell, HPE, Supermicro): 8–18% gross margin. Dell's AI server revenue grew 757% year-over-year in one quarter of 2025, but margins on those servers ran below 15%. Revenue growth in a low-margin layer is still growth in a low-margin layer.
Natural Gas Turbines and Power Infrastructure (GE Vernova, Siemens Energy): Emerging story in 2024. Gas turbine lead times began stretching to 2–3 years, but power had not yet reached dominant bottleneck status.
Cooling Systems: Commodity market with thin margins. Liquid cooling was beginning to emerge as a premium segment, but fragmented and early.
The 2027 Stack: Power and Physics Become the Binding Constraint
By 2027, the profit pool map has been redrawn by three forces: custom silicon at scale, memory bifurcation, and the emergence of power infrastructure as the genuine binding constraint.
Chip Design: Still high-margin but composition has changed. Nvidia's merchant GPU share declined from ~86% in 2025 as Broadcom's custom XPU business scales — Hock Tan guided to AI semiconductor revenue above $100 billion in 2027, supported by a $73 billion backlog. Google TPUs, Amazon Trainium, and Meta's MTIA represent real alternative compute. Custom ASIC shipments grew 44.6% in 2026 versus 16.1% for merchant GPUs.
HBM Memory: Bifurcated. HBM4 from SK Hynix commands premium pricing for the most demanding training clusters. Standard DRAM has partially corrected after the 2025–2026 surge. Memory chip sales are projected at $1 trillion in 2027, but margin varies enormously by tier.
Optical Transceivers: One of the clearest profit pool winners of the 2027 cycle. The bottleneck moved from GPUs to connectivity. Every AI rack requires 800G and above optical interconnects. Co-packaged optics (CPO) — where the transceiver is integrated directly into the switch ASIC — is being deployed at hyperscale. Coherent, Lumentum, and the CPO ecosystem represent some of the highest structural margin improvements in the entire stack between 2024 and 2027.
Natural Gas Turbines: The most dramatic profit pool shift. In 2024, power was an afterthought. In 2027, it is the binding constraint. The IEA estimates one advanced AI rack can consume power equivalent to 65 households. Hyperscale campuses target 500 MW to 1 GW. Gas turbine lead times have stretched to 2–3 years. GE Vernova booked $2.4 billion in data-center-related orders in a single quarter of 2026. Alphabet paid $4.75 billion for a power company. Meta signed 6.6 gigawatts of nuclear agreements. The companies that can deliver electrons quickly — Bloom Energy (55–90-day fuel cell deployment), Cameco, GE Vernova — are capturing a profit pool that barely existed in 2024.
Server Racks: Still the lowest-margin segment. Semiconductors account for more than 95% of the value in a leading AI server rack. The rack is a delivery mechanism for a decision already made at the chip layer. Dell's forecast of $60 billion in AI server revenue for fiscal 2027 confirms massive scale — at margins that will still run in the single digits on that revenue.
The Reallocation in Summary
The 2024 profit pool concentrated in chip design, with secondary capture in HBM memory and advanced packaging. By 2027, the pool has spread: chip design remains dominant but splits between merchant GPUs and custom silicon; optics has moved from niche to critical infrastructure; power has emerged as a genuine high-margin constraint business; and memory has bifurcated into premium (HBM4) and commodity tiers.
The structural principle is consistent: the bottleneck always captures the rent. In 2024, the bottleneck was the GPU. In 2027, the bottleneck has moved to power delivery, optical connectivity, and custom silicon allocation. The next bottleneck — likely 2028–2030 — will be at the application and distribution layer. But that is a different article.