4 posts tagged with "Custom Chip"

August 23-25, 2026, the 38th Hot Chips (HC38) was held at Stanford's Memorial Auditorium. As the bellwether of global high-performance chip architecture, this conference landed exactly at the most intense moment of the AI compute arms race — the official agenda had 48 entries, including 7 AI accelerators, 6 memory tutorials, 6 CPUs, and 4 each of GPUs and networking. Putting the vendor talks together, one consensus emerged: the unit of AI compute competition has shifted from "single chip" to "whole rack / entire system."

---

1. Overview: Three Days of Agenda, Almost a Preview of the 2027 AI Rack Market​

Monday (8/24) afternoon's GPU session was the focus, with four talks nearly colliding as the 2027 AI rack market:

• NVIDIA Rubin GPU ("Driving the Era of Agentic AI"): First chiplet-architecture GPU, 288GB HBM4, ~50 PFLOPS FP4, paired with 88-core Arm-architecture Vera CPU into NVL72 / NVL144 racks, mass production in H2 2026.
• AMD Instinct MI400 (two talks: architecture + system architecture): Told the "rack-scale" story thoroughly.
• Intel Crescent Island: A 350W air-cooled card designed for Agentic AI inference.

Tuesday (8/25) afternoon's AI session was almost a parade of "hyperscalers de-NVIDIA-izing": Google's 8th-gen TPU, OpenAI's first custom chip, Microsoft Maia 200, Meta MTIA, and Cerebras wafer-scale rack all appeared together.

Every vendor on stage used the term "Agentic AI" within the first two PPT slides — not a coincidence, but the collective shift in 2026 AI workload design goals.

---

2. NVIDIA Rubin: One Rack Is a Supercomputer​

What NVIDIA featured at Hot Chips was not a single GPU but the Vera Rubin NVL72 whole cabinet — 72 Rubin GPUs + 36 Vera CPUs, 18 compute trays + 9 NVLink switch trays, about 1.3 million components, nearly 1,300 chips, weighing about 4,000 pounds (~1.8 tons).

The single Rubin GPU specs are equally stunning:

Metric	Rubin GPU	vs Blackwell
Transistors	336 billion (TSMC 3nm dual-die)	208 billion (+61.5%)
Memory	288GB HBM4	—
Bandwidth	22 TB/s	2.8× Blackwell
NVFP4 inference	50 PFLOPS	5× GB200
Training compute	35 PFLOPS	3.5×

The most disruptive design is in the compute tray: no cables, no hoses, no fans, all interconnected via the PCB backplane. NVIDIA says assembly time dropped from nearly 2 hours to 5 minutes (20× faster) while improving maintainability.

This time NVIDIA is selling not FLOPS but tokens per megawatt. Citing a SemiAnalysis benchmark based on DeepSeek-v4-PRO (140K+ context, AgentX workload), it claims: versus GB300 NVL72, Vera Rubin NVL72 delivers 10× to up to 30× tokens/MW as interaction intensity rises. A single cabinet provides 3.6 EFLOPS inference compute, whole-cabinet power 190-230kW; long-term capacity target is 1,000 NVL72 cabinets per day.

---

3. AMD MI455X + Helios: Bigger Memory and Open Interconnect​

AMD's answer is the MI455X + Helios rack going head-to-head with NVIDIA. MI455X uses CDNA 5 architecture, 8 N2-process accelerator dies + N3P-process interconnect die, 256 workgroup processors, 192MB global L2.

Metric	MI455X	vs Rubin
Memory	432GB HBM4 (12-layer stack)	50% higher than Rubin's 288GB
Bandwidth	23.3 TB/s	Slightly ahead
MXFP4 compute	40.26 PFLOPS	—
System (Helios 72 cards)	2.9 ExaFLOPS FP4 inference	—
Price	~$5.25M per cabinet	—

At the system level, AMD bets on the UALoE (Ultra Accelerator Link over Ethernet) open standard: each GPU provides 3.6 TB/s bidirectional interconnect bandwidth; two 512-port 200G UALoE switch chips in the switch tray total 10.8 TB/s — opening the interconnect protocol to the whole industry while targeting NVLink.

Production cadence: AMD plans to deliver engineering samples and small-batch systems in H2 2026, with large-scale ramp in Q2 2027. Earlier rumors of Helios delay due to cooling issues were not confirmed by AMD.

---

4. Intel Crescent Island: The Air-Cooled, Large-Memory "Cost-Effective Oddball"​

Intel offers a completely different path: Crescent Island — a 350W, air-cooled, standard-PCIe-slot inference GPU designed for Agentic AI, with the key metric being tokens per watt.

Metric	Crescent Island	Note
Architecture	Xe3P, 32 Xe cores, 32MB unified L2	Disclosed at Hot Chips
Memory	Intel branded card 160GB / ODM up to 480GB LPDDR5X	More than Rubin's 288GB HBM4
Form factor	350W air-cooled PCIe	Plugs into standard racks, no liquid-cooling retrofit
RAS	ECC, dynamic page offline, hard-package repair, PCIe advanced error reporting	Addresses "silent data corruption"

Intel's logic is clear: inference scenarios need far more memory capacity than bandwidth; using low-cost LPDDR5X for capacity and air cooling to skip liquid-cooling infrastructure drives down per-token cost. Combined with Diamond Rapids Xeon (256 performance cores, 1.28GB cache, 128 PCIe Gen6 lanes), Intel tries to surround from edge to datacenter with "CPU + inference GPU + open software stack."

---

5. Custom ASIC Parade: Google, OpenAI, Microsoft, Meta Together​

Tuesday afternoon's AI session was the most historic of the conference — a parade of "hyperscalers de-NVIDIA-izing":

Chip	Vendor / Partner	Positioning	Key Specs / Progress
TPU 8t (Sunfish)	Google × Broadcom	Training	9,600 cards per pod, 121 FP4 ExaFLOPS, 2PB shared HBM
TPU 8i (Zebrafish)	Google × MediaTek	Inference	288GB HBM, 384MB on-chip SRAM (3× prev gen), ICI 19.2 Tb/s
Jalapeño	OpenAI × Broadcom	Inference	9-month end-to-end design, target ~50% token cost cut, commercial end of 2026
Maia 200	Microsoft (TSMC 3nm)	Inference	140B+ transistors, 10+ PFLOPS FP4, 216GB HBM3E, serving GPT-5.2 at Des Moines datacenter
MTIA 300-500	Meta (RISC-V) × Broadcom	Training + inference	Up to 25× compute gain, one model every 6 months before 2027

Google split TPU into training (8t) and inference (8i) dedicated architectures for the first time — its biggest architectural shift in a decade. Norm Jouppi personally took the stage to present TPU v8.

---

6. Two Hidden Threads — Memory and Networking: HBM4 Year 1 + AI Factory OS​

Beyond GPUs/ASICs, two hidden threads mattered equally:

• Memory: Samsung's HBM Base Die (logic-process base die) and SK hynix's advanced packaging appeared together; the HBM4-era "base-die foundry" industry shift begins; HBF (high-bandwidth flash), LPDDR5X-PIM, 3D DRAM, and CXL compute-storage showcased "compute-in-memory" moving from papers to products.
• Networking: NVIDIA BlueField-4 (DPU) and Spectrum-X Multiplane architecture (presented by Gilad Shainer) — networking is becoming the decisive architecture for gigascale AI, scaling from hundreds of thousands to a million cards; Broadcom Thor Ultra Ethernet NIC keeps pressing; Mojo Vision showed chip-level optical I/O.

---

7. Three Routes, One Consensus​

At the same conference, three vendors offered three distinctly different AI compute delivery philosophies:

1. NVIDIA: Full-stack closed integration — GPU, CPU, DPU, and switch chips all self-designed, pushing system performance to the extreme via ultimate software-hardware co-design, at the cost of deep customer lock-in.
2. AMD: Open-standard catch-up — Uses larger HBM4 capacity + UALoE open interconnect for a "cost-effective + open" play, tearing open the inference gap with Meta and OpenAI's 12GW-class orders.
3. Intel: Air-cooled cost-effectiveness — Abandons liquid cooling and HBM, uses LPDDR5X large memory + standard PCIe, betting that "most inference doesn't need a 200kW rack."

But all three agree: the unit of competition is no longer the chip, but the co-designed system (rack / system). For buyers, 2027 compute planning should compare not "single-card PFLOPS" but "tokens per megawatt, latency, availability, and full-lifecycle cost."

Related Links​

• NVIDIA Rubin spec page
• AMD MI455X spec page
• Intel Crescent Island spec page
• Google TPU 8t spec page
• Google TPU 8i spec page

References​

• Hot Chips 2026 Three Giants Together: Rubin 336B Transistors, MI455X 432GB, Crescent Island 480GB - IC Chip Industry Watch
• Hot Chips Day 2: Rubin, MI400 and Diamond Rapids Go Head-to-Head - AI2Work
• NVIDIA Vera Rubin NVL72 Rack at Hot Chips 2026 - ServeTheHome
• Hot Chips 2026: 4 Hyperscalers Put Custom AI Silicon on One Afternoon - Sean Kim

---

This article is compiled from Hot Chips 2026 (Aug 23-25) official presentations and on-site reports from ServeTheHome, SemiAnalysis, TechPowerUp, etc. Performance data are vendor-disclosed figures; actual performance subject to mass-produced products.

The Tuesday-afternoon AI session at Hot Chips 2026 this August was the most historically significant of the conference — not because any single chip was so powerful, but because almost everything on stage was a "hyperscaler de-NVIDIA-ization" custom ASIC: Google's 8th-gen TPU, OpenAI's first self-designed chip, Microsoft Maia, Meta MTIA, and Cerebras wafer-scale racks, all on one stage. When the world's largest AI compute buyers start treating GPUs as "one of the options," the power structure of AI hardware is loosening.

---

1. OpenAI Jalapeno: Building a Chip in 9 Months​

On June 24, 2026, OpenAI, together with Broadcom, unveiled its first self-designed inference ASIC, Jalapeno — the fifth member of the "custom inference chip club."

Dimension	Jalapeno
Partner	Broadcom + TSMC manufacturing
Positioning	Inference-specific ASIC
Design cycle	9 months end-to-end (Greg Brockman says aided by OpenAI's own models)
Cost target	~50% lower token cost vs general-purpose GPU stack
Commercial timing	First deployments by end-2026; long-term goal 10GW of self-designed chips
Deal scale	Up to $10B strategic partnership with Broadcom (accelerators + networking by 2029)

The talk title "You Can Just Build Things … Chips" is itself a signal: the largest AI compute buyer no longer defaults to GPU as the only path.

---

2. Google TPU v8: The Biggest Architectural Pivot in a Decade — Train/Infer Split​

Google has the longest custom-chip history (2016 to now), and its 8th-gen TPU for the first time splits the product line in two:

Model	Codename	Partner	Positioning	Key Specs
TPU 8t	Sunfish	Broadcom	Training	9,600 cards per pod, 121 FP4 ExaFLOPS, 2PB shared HBM, 2× ICI bandwidth
TPU 8i	Zebrafish	MediaTek	Inference	288GB HBM, 384MB on-chip SRAM (3× prior gen), 19.2 Tb/s ICI

On capacity, Morgan Stanley estimates based on supply-chain interviews that Google TPU production in 2026 may exceed 3 million units (a brokerage estimate, not an official target). Google is also the only vendor to achieve large-scale custom-chip deployment and sell compute externally (Gemini runs on TPUs).

---

3. Meta MTIA: From Recommendation Systems to a GenAI Dual Mission​

Meta's custom journey has the clearest starting point — MTIA was originally built for recommendation ranking hardware and is being pulled toward a dual mission by generative AI.

• MTIA 300 is deployed; 400 / 450 / 500 are planned at roughly one new model every 6 months through 2027;
• Based on RISC-V, Meta claims up to 25× compute gain;
• Node evolves with industry cadence: 100 (7nm) → 200 (5nm) → 300 series (3nm + CoWoS);
• In partnership with Broadcom; another chip codenamed Iris reportedly passed testing in July 2026;
• Meta plans to start volume production of one of them in September 2026, doubling its overall compute.

---

4. Microsoft Maia 200/300: Most Advanced Deployment​

Microsoft's Maia 200, released January 26, 2026, is the most advanced in deployment among the four:

Dimension	Maia 200
Process	TSMC 3nm, 140B+ transistors
Compute	10+ PFLOPS FP4 / 5 PFLOPS FP8
Memory	216GB HBM3E, 7 TB/s
Power	750W
Deployment	Already running in Des Moines data center, serving OpenAI GPT-5.2 and Microsoft 365 Copilot

Microsoft claims roughly 3× the performance of Amazon's Trainium on specific benchmarks. The short-term strategy is a dual track of "self-designed Maia + purchased NVIDIA" in parallel — self-designed chips need time from design to mass production, and NVIDIA's mature ecosystem cannot be replaced in the short term.

---

5. Amazon Trainium 3 and Anthropic's In-House Team​

• Amazon: The Trainium series is already commercial, with 1.4 million units cumulatively deployed (officially disclosed) — a multi-billion-dollar business; its strength is the AWS customer base, letting enterprises choose between NVIDIA GPUs and self-designed chips. Trainium 3 continues this path.
• Anthropic: In August 2026 announced the formation of an in-house chip team, with no tape-out or mass-production timeline yet; initially positioned as a complement (not a replacement) to existing partnerships with NVIDIA/AMD/AWS/Google Cloud, aiming to tailor-build for the Claude architecture and shed reliance on a single GPU.

---

6. NVIDIA's Answer: Not a Faster GPU, But Full-Stack​

It's easy to simplify the narrative to "four companies build chips, NVIDIA defends GPU." But NVIDIA took 6 slots at Hot Chips: a RISC-V tutorial, the Vera CPU, the Rubin GPU, the BlueField-4 DPU, the Spectrum-X multi-plane network, and an LPU accelerator.

A hyperscaler ASIC replaces only one of those five pillars. If the CPU, NIC, switching fabric, and software all come from the same vendor, what you save by swapping out the accelerator is far less than the accelerator line item on the bill suggests. Rubin's play is a full-stack AI factory platform spanning seven chips and five racks — the competitive answer is "full-stack positioning," not "a faster single chip."

---

7. Trend Judgment: Inference De-GPU-izes, Training Still GPU-Led​

• Inference side: The CUDA moat visibly shallows. Inference is parallelizable and replaceable at the endpoint; custom ASICs trade away the generality tax (implementing only the operations LLMs actually execute) for lower cost/token. Groq LPU, Cerebras, and various TPU/ASIC players all compete on the same metric.
• Training side: Foundation models are still trained on GPUs, with no serious challenger in the short term. NVIDIA's three training moats (fastest silicon + NVLink + CUDA) remain firm.
• Conclusion: Custom chips are not "replacing NVIDIA," but giving buyers a credible external negotiation option in the largest and fastest-growing battlefield — inference. That alone is enough to reshape the economics of AI infrastructure.

Related Links​

• Inference Accelerator Market 2026
• Hot Chips 2026 Full Recap
• Google TPU 8t Spec Page
• Google TPU 8i Spec Page

References​

• Hot Chips 2026: 4 Hyperscalers Put Custom AI Silicon on One Afternoon - Sean Kim
• Custom AI inference silicon: why OpenAI built Jalapeno - nexi.fund
• Anthropic Builds Custom AI Chips for Claude - Studio Global AI
• 所有大厂都在做同一件事：自己造芯片，少买英伟达的 - Toutiao

---

This article is compiled from August 2026 Hot Chips on-site reports, corporate announcements, and industry analysis. Some capacity and performance figures are brokerage estimates or vendor-disclosed figures; actual results are subject to mass-produced products.

On June 24, 2026, OpenAI and Broadcom jointly announced their first in-house AI inference chip, Jalapeño. This ASIC designed specifically for large language model inference went from design to tape-out in just 9 months and cuts inference cost by roughly 50%, marking OpenAI's transformation from a pure model company into a full-stack AI infrastructure provider.

---

1. Core conclusions (read this first)​

Dimension	Jalapeño	Current GPU solution	Advantage
Inference cost	-50%	Baseline	✅ Half the cost
Performance per watt	Clearly superior	Most advanced accelerator	✅ Energy-efficiency lead
Design cycle	9 months	~18 months	✅ 2× faster
Positioning	Inference ASIC	Train+inference GPU	Dedicated optimization
Supply	Internal only	Market purchase	⚠️ Not for sale

One-line summary: Jalapeño is a key step in OpenAI's full-stack AI strategy, using in-house silicon to cut inference cost 50% while opening a new paradigm of "AI-assisted design of AI chips."

---

2. What is Jalapeño?​

2.1 Basic information​

Item	Detail
Name	Jalapeño (a chili pepper)
Type	Application-specific integrated circuit (ASIC)
Positioning	Large language model inference
Announced	2026-06-24
Taped out	Sep 2025 (est., 9-month rapid tape-out)
Deployment	End of 2026 (gigawatt-scale data centers)
Partners	Broadcom, TSMC, Celestica
Process	TSMC 3nm
Architecture	Systolic Array
HBM	8 stacks (est. HBM3E or HBM4)

2.2 Why "Jalapeño"?​

Jalapeño is a Mexican chili known for "medium heat, strong flavor." OpenAI's naming hints that the chip:

• ✅ Medium heat: not the most aggressive architecture (vs Cerebras WSE), but effective enough
• ✅ Strong flavor: strong presence in inference scenarios (50% cost reduction)
• ✅ Appetizer: just "the first step of a multi-generation roadmap" (Broadcom CEO Hock Tan)

---

3. Deep technical analysis​

3.1 9-month rapid tape-out: the new paradigm of AI-assisted chip design​

Normally, designing an ASIC from scratch takes 1.5 to 2 years. Jalapeño went from initial design to manufacturing tape-out in just 9 months.

Key reason: deep software-hardware co-development

Technique	Description
AI-assisted architecture exploration	OpenAI used its own frontier models (GPT-5.3-Codex-Spark) to explore chip architecture design space
AI power simulation	AI models for power simulation and optimization
RL optimization	RL to optimize chip placement and routing
Broadcom silicon implementation	Broadcom provides top-tier ASIC implementation (network, switch chip experience)

OpenAI President Greg Brockman said:

"We use the frontier models that serve our users to optimize the infrastructure that runs the models of the future."

3.2 Architecture optimized for inference​

Unlike general-purpose GPUs, Jalapeño is an ASIC built from scratch around OpenAI's deep understanding of LLM inference workloads:

Architecture feature	Description
Reduce data movement	Core principle is minimizing data movement (the main bottleneck in inference)
Balanced compute-memory-network	Resource allocation optimized for inference, bringing real utilization closer to theoretical peak
High throughput + low latency	Aims to combine the throughput of leading accelerators with the low latency of the fastest dedicated inference systems
Future model support	Supports not only current models (GPT-5, GPT-5.3) but adapts to next-gen inference needs

3.3 Full-stack platform: more than a chip​

Jalapeño is a multi-generation compute platform, not just a chip:

Component	Supplier	Description
Accelerator chip	OpenAI design, TSMC fab	TSMC 3nm, 8-stack HBM
Network switch chip	Broadcom Tomahawk	High-speed interconnect (competes with NVIDIA NVLink)
Board, rack, system	Celestica	Full-rack solution
Software stack	OpenAI	Deep adaptation for GPT, Codex, Agent products

Deployment target: gigawatt-scale data centers

Broadcom CEO Hock Tan said:

"Jalapeño will begin deployment this year in gigawatt-scale data centers with Microsoft and other partners."

---

4. Performance and cost analysis​

4.1 Inference cost cut 50%​

Although OpenAI's official release was conservative on Jalapeño's cost savings — only stating its "performance per watt is substantially better than today's state of the art" without a specific percentage — per Bloomberg, Broadcom CEO Hock Tan revealed:

Early internal tests show Jalapeño achieves roughly 50% inference cost savings versus today's mainstream AI GPUs.

Significance for OpenAI:

Item	Current (GPU)	Jalapeño	Savings
Daily API calls	Hundreds of millions	Hundreds of millions	—
Inference cost share	~60-70% of operating cost	~30-35%	-50%
Annual compute spend	Billions of dollars	Hundreds of millions	Saves billions

4.2 Performance per watt clearly better than state of the art​

OpenAI's announcement states:

"Jalapeño engineering samples have successfully run complex reinforcement-learning tasks such as GPT-5.3-Codex-Spark at target frequency and power; early tests show performance per watt substantially better than today's most advanced AI accelerators."

Comparison target: NVIDIA Blackwell (today's most advanced AI accelerator)

Metric	Jalapeño	NVIDIA Blackwell	Note
Performance per watt	Clearly superior	Baseline	OpenAI official statement
Inference latency	On par with fastest dedicated inference systems	Baseline	Target
Throughput	On par with leading accelerators	Baseline	Target
TDP	Not disclosed (est. 400-700W)	700-1000W	Jalapeño possibly lower

---

5. Impact on the AI chip market​

5.1 "De-NVIDIA-ification" accelerates​

Jalapeño's launch is another footnote in big-tech's collective challenge to NVIDIA's market dominance:

Vendor	In-house chip	Type	Status	Relation to OpenAI
Google	TPU v6e / Ironwood	Train+inference	✅ Commercial	Google Cloud supplies OpenAI
Amazon	Trainium 3	Training	✅ Launched	AWS supplies OpenAI
Microsoft	Maia 100	Train+inference	✅ Launched	OpenAI exclusive partner
Meta	MTIA	Train+inference	✅ Launched	—
Apple	Neural Engine	On-device inference	✅ Commercial	—
OpenAI	Jalapeño	Inference	🚧 Deploy end of 2026	Internal + possibly sold to third parties

5.2 OpenAI is not about to fully "abandon" NVIDIA​

Brockman admitted:

"We simply cannot get compute fast enough."

Currently OpenAI is simultaneously procuring chips from NVIDIA, AWS, AMD, and Cerebras; Jalapeño is a structural supplement to its explosive compute demand, not a replacement.

5.3 Possibly sold to third parties​

Broadcom CEO Hock Tan specifically emphasized:

"This is just 'the start of a multi-generation roadmap'; OpenAI and Broadcom aim to jointly build gigawatt-scale compute clusters."

This means OpenAI may sell its hardware to third parties, provided it can secure enough supply from Broadcom and TSMC.

---

6. Jalapeño vs other in-house chips​

Metric	Jalapeño (OpenAI)	TPU v6e (Google)	Trainium 3 (Amazon)	Maia 100 (Microsoft)
Announced	2026-06-24	2024	Q4 2025	2023
Type	Inference ASIC	Train+inference TPU	Training ASIC	Train+inference
Process	TSMC 3nm	TSMC 4nm	TSMC 5nm (est.)	TSMC 5nm (est.)
For sale	❌ Internal only (maybe later)	✅ GCP	✅ AWS	❌ Internal only
Design cycle	9 months	~18 months	~18 months	~18 months
AI-assisted design	✅ First	❌ No	❌ No	❌ No
Cost advantage	-50% inference cost	Optimized	Optimized	Optimized

Key differences:

• ✅ Jalapeño is the first AI chip designed with AI assistance
• ✅ Jalapeño design cycle only 9 months (industry average 18 months)
• ⚠️ Jalapeño not for sale (at least for now)

---

7. Future roadmap​

7.1 Multi-generation chip platform​

Jalapeño is just "the start of a multi-generation roadmap":

Time	Event
End of 2026	Jalapeño initial deployment (gigawatt-scale data centers)
2027	Jalapeño v2 (est., architecture optimization)
2027-2028	Jalapeño training version (est., challenging TPU/Trainium)
2028 and beyond	Gigawatt-scale compute cluster fully built

7.2 OpenAI full-stack AI infrastructure strategy​

Layer	OpenAI in-house	Outsourced/procured
Models	✅ GPT-5, GPT-5.3, Codex	—
Chips	✅ Jalapeño (inference)	NVIDIA GPU, AWS Trainium, AMD GPU
Systems	✅ With Celestica	Microsoft Azure data centers
Network	✅ Broadcom Tomahawk	Microsoft Azure network
Cloud platform	❌ None	Microsoft Azure (exclusive partner)

---

8. Industry reaction and expert views​

8.1 Supportive views​

Expert	View
Broadcom CEO Hock Tan	"Jalapeño is just the start of a multi-generation roadmap; the goal is to jointly build gigawatt-scale compute clusters."
OpenAI President Greg Brockman	"We use the frontier models that serve our users to optimize the infrastructure that runs the models of the future."
Industry insiders	"Jalapeño's launch is another footnote in big-tech's collective challenge to NVIDIA's market dominance."

8.2 Skeptical views​

Concern	Description
Not for sale	Currently internal only; third parties cannot purchase, limited impact on NVIDIA's market share
Software ecosystem	OpenAI must build its own software stack; competing with CUDA is hard
Supply capacity	TSMC capacity is limited; can it meet OpenAI + Broadcom + other customers' demand?
Opaque performance data	OpenAI has not released specs (compute, memory, bandwidth, TDP), hard to assess objectively

---

9. Significance for developers​

9.1 If OpenAI sells Jalapeño to developers in the future...​

Scenario	Current (NVIDIA GPU)	Future (Jalapeño)
Inference cost	Baseline	-50%
Inference latency	Baseline	Possibly lower
Software stack	CUDA + TensorRT	OpenAI API (possibly open-source stack)
Procurement difficulty	High (export controls, supply shortage)	Low (OpenAI direct supply)

9.2 Worth watching even if not sold​

• ✅ 50% inference cost cut forces NVIDIA, AMD, Intel to lower GPU prices
• ✅ The AI-assisted chip design paradigm will be rapidly copied by the industry
• ✅ The 9-month tape-out cycle becomes a new industry benchmark

---

10. Summary​

Dimension	Assessment
Technology innovation	⭐⭐⭐⭐⭐ First AI chip designed with AI assistance, 9-month tape-out
Cost advantage	⭐⭐⭐⭐⭐ 50% inference cost cut, billions saved annually
Strategic significance	⭐⭐⭐⭐⭐ OpenAI transforms from pure model company to full-stack AI infrastructure provider
Market impact	⭐⭐⭐⭐ "De-NVIDIA-ification" accelerates, big-tech in-house chip camp grows
Openness	⭐⭐ Internal only for now, possibly sold to third parties later

Final recommendations:

• 🇨🇳 China market: Keep watching Huawei Ascend, Cambricon MLU, Moore Threads MTT (Jalapeño not sold to China)
• 🌍 International market: Watch whether Jalapeño is eventually sold externally and its impact on NVIDIA's market share
• 💡 Developers: Watch for possible OpenAI API price cuts (50% inference cost cut may partially pass through)

---

References​

• OpenAI official announcement (technical white paper pending)
• Broadcom CEO Hock Tan interview
• Bloomberg: Jalapeño inference cost cut 50%
• SemiWiki: Jalapeño technical details
• Tom's Hardware: Jalapeño architecture analysis
• ESMChina: Jalapeño 9-month tape-out

---

Disclaimer: Some specs in this article are estimates, subject to OpenAI's official technical white paper. OpenAI will release a detailed performance white paper in the coming months.

Last updated: June 26, 2026

On March 18, 2026, Intel officially launched at the Intel AI Summit: the Habana Gaudi 4 custom AI accelerator. This is Intel's latest-gen AI training/inference chip after Gaudi 3 (launched April 2024), designed for large-scale model training.

Meanwhile, Intel confirmed its next-gen Jaguar Shores GPU (datacenter GPU) is in development, will use HBM4 memory, and is expected in 2027. This marks Intel's formal return to the AI chip race.

Key Highlights​

• Gaudi 4: Launched March 2026, TSMC 5nm, 64GB HBM3e, for large-scale training
• Jaguar Shores: Launches 2027 (est.), HBM4, targeting NVIDIA Rubin
• Crescent Island: Intel's first general-purpose GPU (launched 2026), Xe3 architecture
• Software ecosystem: Intel AI Stack (including oneAPI, BigDL, Gaudi Software Suite)
• Foundry partners: TSMC (Gaudi 4, Jaguar Shores), Intel Foundry (Crescent Island)

Gaudi 4 Detailed Specs​

Gaudi 4 is the fourth-gen AI accelerator designed by Intel's Habana Labs (acquired 2019).

Parameter	Gaudi 4	Gaudi 3 (2024)	NVIDIA B200
Architecture	Habana 4	Habana 3	Blackwell
Process	TSMC 5nm	TSMC 7nm	TSMC 4NP
FP8 compute	~2,000 TFLOPS (est.)	1,000 TFLOPS	4,500 TFLOPS (sparse)
Memory	64GB HBM3e	128GB HBM2e (est.)	192GB HBM3e
Memory bandwidth	~3 TB/s (est.)	~2 TB/s (est.)	8 TB/s
TDP	~500W (est.)	~400W	700-1000W
Interconnect	RoCE v3 (Ethernet)	RoCE v2	NVLink 5.0
Launch	March 2026	April 2024	March 2024
Mass production	2026 Q3 (est.)	Q4 2024	Q4 2024

📌 Note: Gaudi 4 exact specs not fully public; some values above are estimates.

Gaudi 4 Key Features​

1. Native Ethernet support: Uses RoCE v3 (RDMA over Converged Ethernet), no dedicated interconnect protocol needed (like NVLink)
2. Large-scale scaling optimized: Ten-thousand-card cluster scaling efficiency better than InfiniBand (lower cost)
3. Sparsity acceleration: Native MoE model support
4. Multi-precision support: FP8/FP16/FP32/INT8/INT4
5. Open ecosystem: Supports PyTorch, TensorFlow, JAX (via third-party adaptation)

Jaguar Shores: Intel's Next-Gen GPU​

Jaguar Shores is Intel's first true datacenter GPU (not an ASIC like Gaudi).

Why "Jaguar Shores"?​

• Jaguar: Symbolizes "speed" and "agility"
• Shores: Symbolizes "openness" and "connection"

Jaguar Shores Estimated Specs​

Parameter	Jaguar Shores (est.)	NVIDIA Rubin	AMD MI455X
Architecture	Xeu 3 (est.)	Rubin	CDNA 4
Process	TSMC 3nm (est.)	TSMC 3nm	TSMC 3nm
Memory	HBM4 (confirmed)	HBM4	HBM4
Memory capacity	288GB (est.)	288GB	288GB
FP8 compute	~4,000 TFLOPS (est.)	~6,000 TFLOPS	6,000 TFLOPS
TDP	~800W (est.)	~1,000W	~800W
Launch	2027 (est.)	2026 Q3	2026 Q3

Key confirmations:

• ✅ HBM4 memory: Intel confirmed Jaguar Shores will use SK hynix HBM4
• ✅ TSMC foundry: Jaguar Shores will be produced by TSMC (not Intel Foundry)
• ✅ oneAPI native support: Jaguar Shores will natively support the oneAPI programming model

Crescent Island: Intel's First General-Purpose GPU​

Crescent Island is Intel's first general-purpose datacenter GPU announced October 2025, using the Xe3 architecture (upgrade of Xe-HPG).

Parameter	Crescent Island (est.)	Intel Data Center GPU Max	NVIDIA L40S
Architecture	Xeu 3	Xeu 2 (Ponte Vecchio)	Ada Lovelace
Positioning	General compute + AI inference	HPC + AI training	AI inference + graphics
Process	TSMC 5nm (est.)	Intel 7 + TSMC 5nm	TSMC 4N
Memory	48GB HBM3 (est.)	128GB HBM2e	48GB GDDR6
TDP	~300W (est.)	600W	350W
Launch	2026 (est.)	Jan 2023	Mar 2023

Positioning:

• ✅ General-purpose GPU: Both AI inference and scientific computing (HPC)
• ✅ Low cost: Cheaper than Gaudi 4, targeting NVIDIA L40S
• ✅ Open standards: Supports oneAPI, SYCL, Level Zero

Intel AI Chip Roadmap (2024-2027)​

Time	Product	Type	Process	Note
2024 Q4	Gaudi 3	AI ASIC	TSMC 7nm	Current mainstay
2026 Q2	Crescent Island	General GPU	TSMC 5nm	New launch
2026 Q3	Gaudi 4	AI ASIC	TSMC 5nm	New launch
2027	Jaguar Shores	Datacenter GPU	TSMC 3nm	Next-gen flagship
2027	Gaudi 5 (est.)	AI ASIC	TSMC 3nm	Next-gen

vs Competitors​

Gaudi 4 vs NVIDIA B200​

Metric	Gaudi 4	NVIDIA B200
FP8 compute	~2,000 TFLOPS	4,500 TFLOPS
Memory	64GB HBM3e	192GB HBM3e
Interconnect	Ethernet (RoCE v3)	NVLink 5.0
Software ecosystem	Gaudi Software Suite	CUDA
Price	est. ~$20,000	~$45,000
Advantage	Low Ethernet cost, open	Most mature ecosystem, strongest performance
Disadvantage	Weak software ecosystem, lower compute	Expensive

Conclusion: Gaudi 4 is positioned as a "cost-effective training solution," suited for cost-sensitive customers willing to invest in software adaptation.

Jaguar Shores vs NVIDIA Rubin​

Metric	Jaguar Shores (est.)	NVIDIA Rubin
FP8 compute	~4,000 TFLOPS	~6,000 TFLOPS
Memory	288GB HBM4	288GB HBM4
Software ecosystem	oneAPI	CUDA
Mass production	2027	2026 Q3
Advantage	Open standards, possibly cheaper	Mature ecosystem, first-mover advantage
Disadvantage	Weak ecosystem, 1 year late	Expensive

Conclusion: If Jaguar Shores launches on time with sufficient oneAPI ecosystem improvement, it can become NVIDIA's third choice (after NVIDIA and AMD).

Software Ecosystem: oneAPI Progress and Challenges​

What is oneAPI?​

oneAPI is Intel's open, cross-architecture programming model:

• Supports CPU, GPU, FPGA, AI accelerators
• Based on SYCL standard (similar to CUDA's C++ extensions)
• Open-source implementation (Intel oneAPI Base Toolkit)

Intel AI Stack​

Component	Purpose	Counterpart
oneAPI	Cross-architecture programming model	CUDA
BigDL	Distributed deep learning framework	PyTorch Distributed
Gaudi Software Suite	Gaudi-specific software stack	NVIDIA GPU Cloud (NGC)
Intel Extension for PyTorch	PyTorch optimization on Intel hardware	NVIDIA PyTorch
Intel Optimization for TensorFlow	TensorFlow optimization on Intel hardware	NVIDIA TensorFlow

✅ Progress​

• PyTorch 2.5+: Intel Extension integrated into PyTorch mainline
• Hugging Face Transformers: Official Intel GPU support (via optimum-intel)
• vLLM: Experimental Gaudi support (performance TBD)

⚠️ Challenges​

• Developer habits: Global AI developers use CUDA; oneAPI has a steep learning curve
• Operator coverage: Many PyTorch operators lack oneAPI-optimized versions
• Performance: At same power, Gaudi 4 performance is only ~50% of B200

Industry Impact​

1. Can Intel Return to the AI Race?​

Challenges:

• ❌ Ecosystem disadvantage: CUDA moat too deep, oneAPI hard to shake
• ❌ Performance disadvantage: Gaudi 4 only ~50% of B200
• ❌ Timing disadvantage: Jaguar Shores 1 year later than Rubin

Opportunities:

• ✅ Open standards: Not dependent on CUDA, suited for "anti-NVIDIA-monopoly" customers
• ✅ Ethernet advantage: RoCE v3 cheaper than InfiniBand at ten-thousand-card scale
• ✅ Intel Foundry: If Jaguar Shores uses Intel's own process, lower cost

2. Impact on AMD​

Intel's return to the AI race is bad for AMD:

• AMD was the "only NVIDIA alternative"
• Now Intel is back too; AMD's "alternative" status is challenged
• But in the short term (2026-2027), Intel cannot yet threaten AMD

3. Impact on Domestic Chips​

Intel Gaudi 4's launch is a reference case for domestic chips:

• Proves the Ethernet route (RoCE) is viable
• Proves open ecosystem (oneAPI) is hard but necessary
• Proves the cost-effective route has a market (cost-sensitive customers)

Related Chips​

• Intel Gaudi 3 - Previous-gen product
• Intel Gaudi 4 - New launch (to be created)
• Intel Crescent Island - Intel's first general GPU (to be created)
• NVIDIA B200 - Direct competitor
• AMD MI455X - Same-gen competitor

References​

• Intel Newsroom: Gaudi 4 Launch
• Tom's Hardware: Intel Jaguar Shores to use HBM4
• Wccftech: Intel Next-Gen Jaguar Shores GPU
• U.S. News: Intel Signals Return to AI Race

---

This article is compiled from Intel official announcements and public materials. Some specs are estimates, subject to final Intel release.