Signals Inbox·August 26, 2026·AI Chips

What is OpenAI's Jalapeño chip exactly?

OpenAI’s Jalapeño is its first custom LLM inference processor: a working HBM4-based ASIC and rack-scale system built to serve models faster and more efficiently, while giving OpenAI much tighter control over the compute behind ChatGPT, Codex and agents.

Q1What did OpenAI actually build with Jalapeño?

OpenAI’s Jalapeño is a custom chip for running large language models, especially the low-latency inference behind products such as ChatGPT, Codex and AI agents.

OpenAI calls it its first “Intelligence Processor.” In more conventional semiconductor language, Jalapeño is a custom inference ASIC: a processor designed around a narrower set of AI workloads than a general Nvidia GPU.

The chip is already running real models in OpenAI’s lab. OpenAI initially disclosed engineering samples running at production target frequency and power, including GPT-5.3-Codex-Spark. It has now gone much further and published measured InferenceX results on GPT-OSS 120B, DeepSeek R1 670B and Kimi K2.5 1T. SemiAnalysis was invited into the lab and says it watched the InferenceX workloads run on the hardware.

Today, one thing is clear: Jalapeño has moved beyond the “mysterious OpenAI chip project” stage. Working silicon exists, large models run on it, and OpenAI is preparing to put it into its own compute infrastructure.

Q2Why did OpenAI build Jalapeño for inference instead of training?

OpenAI built Jalapeño around inference because serving models is becoming a huge, continuous compute problem, especially as AI products shift from simple chat toward reasoning and agents.

Training a frontier model can consume enormous amounts of hardware for weeks or months. Once that model exists, though, inference starts every time someone asks ChatGPT a question, sends an API request or gives Codex a task. Agents make the problem bigger because one user request can trigger many model calls in sequence.

Power is increasingly the constraint. SemiAnalysis says OpenAI is currently more constrained by available data-center power than by floor space or willingness to buy hardware. At that scale, squeezing more useful tokens from each megawatt directly increases how much AI OpenAI can serve.

That explains why Jalapeño is so focused on performance per watt and latency rather than simply maximizing theoretical FLOPS. OpenAI still needs GPUs for training, but inference is large and repetitive enough that designing custom hardware for it can change the economics of the whole business.

Q3Is Jalapeño really OpenAI’s chip if Broadcom builds it?

Yes, Jalapeño is genuinely an OpenAI-designed accelerator, while Broadcom handles much of the semiconductor engineering needed to turn that architecture into a manufacturable product.

OpenAI controls the architecture and makes the decisions around the workloads, memory behavior, kernels, serving system and overall machine. Broadcom contributes silicon implementation, connectivity and Tomahawk networking. TSMC manufactures the advanced silicon, while Celestica works on boards, racks and system integration.

That setup is normal for custom data-center chips. Google does not need to own a semiconductor fab for TPU to be considered a Google processor, and OpenAI does not need to manufacture wafers itself to own Jalapeño’s architecture.

The distinction becomes clearer when we look at what OpenAI is actually trying to own. It wants the parts that determine how efficiently its models run: how data moves through the chip, how memory is organized, how processors communicate, how kernels are written and how thousands of accelerators behave together. Broadcom gives OpenAI a much faster route from those ideas to industrial-scale hardware.

Q4Is Jalapeño basically an OpenAI version of an Nvidia GPU?

Jalapeño is much more specialized than an Nvidia GPU, although OpenAI deliberately kept it flexible enough to handle many different LLM architectures and inference workloads.

SemiAnalysis describes a weight-stationary systolic matrix engine using MXFP formats, alongside 64-bit scalar cores and FP32/INT32 vector processing. Support for relatively small matrix dimensions also helps Jalapeño avoid some of the efficiency cliffs that can hit accelerators built around very large systolic arrays.

The production B0 design is expected to reach 13.4 PFLOPS of MXFP4 compute from a reticle-sized compute die manufactured on TSMC N3P. Each package gets 216 GiB of HBM4 and about 15.4 TB/s of memory bandwidth. Rated package power is 700 W, while OpenAI says the earlier A0 hardware stayed at or below 550 W during the workloads used in its published tests.

Jalapeño therefore sits somewhere between the flexibility of a GPU and the extreme specialization of a chip designed around one fixed model. OpenAI narrowed the target to modern LLM inference while keeping plenty of freedom inside that category.

Jalapeño specifications

Q5Can Jalapeño run AI models that OpenAI did not build?

Yes, and this is one of the strongest pieces of evidence that Jalapeño is broader than a chip hard-wired around GPT.

OpenAI has now shown the processor running DeepSeek R1 670B and Kimi K2.5 1T alongside GPT-OSS 120B. DeepSeek and Moonshot AI obviously did not design those models around OpenAI’s future hardware, yet Jalapeño performs very well on both.

SemiAnalysis even reports that OpenAI had no internal MLA kernel implementation when it decided to benchmark DeepSeek. The software stack had to support the new workload quickly, and Codex helped produce the required kernels.

There are limits. Every new model family can still require new kernels and model-specific optimization, so Jalapeño cannot magically run any architecture at peak efficiency from day one.

But the evidence we have now points toward a fairly general LLM inference processor. OpenAI seems to have specialized around recurring properties of transformer inference, attention, mixture-of-experts workloads, memory traffic and inter-chip communication rather than around one private GPT architecture.

Q6Why is Jalapeño so fast at LLM inference?

Jalapeño gets much of its advantage by cutting the time and energy spent moving data around, which is often more important in LLM inference than adding more raw arithmetic.

Modern inference spends a lot of time moving model weights, reading and writing the KV cache, synchronizing processors and waiting for data. A chip can have enormous theoretical compute and still use only a fraction of it if those delays keep the math units idle.

OpenAI designed Jalapeño so its cores and HBM are divided into slices, with each compute slice getting a low-latency local view of its own memory. A dedicated collective network handles predictable communication between slices. SemiAnalysis says the simplified memory and network hierarchy removes several fixed overheads that GPUs normally have to hide through larger batches and more parallel work.

At the core level, OpenAI also chose an out-of-order design with L1 cache and prefetching. That is unusual among AI accelerators, which often rely heavily on software-managed scratchpads.

The practical goal is to keep the hardware busy even when the workload is small, interactive or awkwardly shaped. That helps explain why Jalapeño looks particularly strong at the low-latency end of the benchmark curve.

Q7Why did OpenAI keep prompt processing and token generation on the same Jalapeño chips?

OpenAI kept prefill and decode on the same Jalapeño pool because real inference traffic changes constantly, making a fixed split between specialized hardware pools surprisingly awkward to operate.

Prefill is the stage where the model processes the prompt, so it tends to use a lot of compute. Decode generates the answer token by token and leans much more heavily on memory bandwidth. One obvious approach is to dedicate some hardware to each phase.

OpenAI chose differently. Prompt lengths change, output lengths change, reasoning models generate different amounts of work, cache hit rates move around and agent workloads can look very different from ordinary chat. A pool sized perfectly for yesterday’s traffic can leave decode processors idle while prefill requests queue, or the reverse.

There is also a data-transfer penalty. Prefill creates the KV cache that decode immediately needs. Sending that cache to another machine adds network traffic and latency.

So OpenAI accepts some local inefficiency in exchange for a fleet where every Jalapeño processor can take whichever type of work arrives next. For a service with a constantly changing mix of ChatGPT, API and agent traffic, that looks like a very practical choice.

Q8How big is an OpenAI Jalapeño system?

Jalapeño is built to operate as a rack-scale computer, with 128 accelerators in one ASIC rack and as many as 2,048 processors connected inside the larger scale-up domain described by SemiAnalysis.

A Jalapeño accelerator rack contains 16 trays, nicknamed Vindaloo, with eight chips in each tray. A paired host rack contains AMD EPYC CPUs. Broadcom Tomahawk 6 networking connects the accelerators, while Celestica is involved in the physical system design.

Those 128 processors give one rack roughly 27.6 TB of HBM4 based on the published 216 GiB per chip. Using the B0 target of 13.4 PFLOPS MXFP4, a rack would also contain roughly 1.7 exaflops of theoretical low-precision compute.

OpenAI can then connect 16 of these accelerator racks, reaching a 2,048-chip scale-up domain. SemiAnalysis says the design mixes copper connections inside the local domain with optical networking across the larger system.

The scale is the point. OpenAI designed a complete inference machine around the processor, not just a chip.

Jalapeño system configuration

Q9Does OpenAI’s Jalapeño actually beat Nvidia right now?

Jalapeño currently beats the Nvidia Blackwell systems in OpenAI’s published InferenceX comparisons by a meaningful margin on both power efficiency and latency.

On GPT-OSS 120B, OpenAI reports roughly 85,448 mixed tokens per second per kilowatt for Jalapeño versus 44,960 for GB200, close to a 1.9x advantage. End-to-end latency was 1.03 seconds against 1.80 seconds.

DeepSeek R1 produced a similar efficiency win: 19,641 versus 11,781 mixed tokens per second per kilowatt against GB300, while latency fell from 5.99 seconds to 1.65 seconds. On Kimi K2.5, Jalapeño delivered around 1.5x the peak performance per watt of GB300 and cut end-to-end latency from 5.31 seconds to 1.56 seconds.

Those are strong numbers for a first-generation custom chip. SemiAnalysis also says Jalapeño reached more than 700 output tokens per second per user on DeepSeek at concurrency one, while GPT-OSS and Kimi could reach around 1,400 tokens per second per user in some low-concurrency configurations.

So the Blackwell result is pretty clear. Jalapeño is already a serious inference accelerator rather than a custom chip that merely saves OpenAI some supplier margin.

Jalapeño vs Nvidia in OpenAI’s InferenceX results

Q10Are OpenAI’s Jalapeño benchmarks actually trustworthy?

The Jalapeño results are credible enough to take seriously today, although we still do not have the kind of independent testing that comes once hardware is widely available.

OpenAI supplied the benchmark numbers. That normally deserves caution. In this case, however, SemiAnalysis visited OpenAI’s lab and says it verified the InferenceX runs in person. It also confirmed that model quality on GSM8K remained comparable with Nvidia systems, which helps rule out an obvious way of inflating speed by sacrificing output quality.

The bigger weakness is the workload itself. The public results use a nominal 8,000-token input and 1,000-token output. SemiAnalysis calls this much easier to optimize than AgentX, its newer long-context, multi-turn benchmark designed to stress routing, prefix caching, cache management and the messier behavior of actual agents.

Jalapeño has not publicly run AgentX yet. The tested models also lag the newest open-model frontier, where Nvidia and AMD have already published results on larger architectures.

Our confidence should therefore differ depending on the claim. We can be highly confident that Jalapeño is real and unusually fast on the workloads shown. We still need harder production-style tests before saying OpenAI has the best inference system across every modern agent workload.

Q11Is Nvidia Rubin the real comparison for Jalapeño now?

Yes. Nvidia’s HBM4-based Vera Rubin platform is the comparison that really matters now, and using Blackwell alone makes Jalapeño look stronger than the competitive situation actually is.

SemiAnalysis makes this point explicitly. Jalapeño and Rubin belong to roughly the same hardware generation, both use HBM4, and Vera Rubin systems have already started reaching customers. Blackwell remains commercially important, but beating GB200 or GB300 tells us less about where OpenAI stands against Nvidia’s latest architecture.

The early Rubin comparison is still impressive. SemiAnalysis says Jalapeño’s single-token-prediction output throughput per megawatt exceeds Nvidia and CoreWeave’s published Rubin results even though those Rubin numbers benefit from multi-token prediction. On total cost of ownership, however, SemiAnalysis currently estimates Rubin and Jalapeño at roughly the same output tokens per dollar.

The cleaner conclusion is that OpenAI appears to have built an inference architecture competitive with Nvidia’s newest generation on its first attempt. We cannot yet say Jalapeño has clearly beaten Rubin overall because both platforms are still early, software is improving quickly and comparable benchmark coverage remains thin.

Q12Did OpenAI really design Jalapeño in nine months, and did AI actually help?

OpenAI’s nine-month Jalapeño claim refers to the intense design-to-tapeout phase, and there is real evidence that AI shortened both the chip-design process and the software work that followed.

SemiAnalysis traces the wider effort back to team building around the middle of 2024 and reports that the complete CoWoS design taped out in November 2025, closer to 16 months from early hiring. OpenAI’s narrower nine-month figure starts with initial chip design and ends at manufacturing tapeout.

Either way, the pace is unusually fast for first-generation advanced silicon.

More interestingly, OpenAI has disclosed specific areas where AI helped. SemiAnalysis reports an 8% reduction in SIMD area and a 10% reduction in matrix-engine area from AI-assisted optimization. OpenAI also says selected GPT-OSS attention and mixture-of-experts blocks generated with AI ran 1.5 to 1.8 times faster than the previous human-expert implementations.

The software progressed quickly too. Three open-weight models outside Jalapeño’s original production plan reached high performance within roughly two months using Codex-assisted kernel development. SemiAnalysis separately observed more than 2x throughput improvement at some operating points over less than two weeks of software tuning.

So “AI designed the chip” would oversell what happened. The stronger story is that OpenAI used AI inside several normally slow engineering loops and appears to be compressing the time between a hardware idea and optimized working code.

Q13Does Jalapeño weaken Nvidia’s CUDA advantage?

Jalapeño shows that a frontier AI lab can get surprisingly far without CUDA when it controls its models, compiler tools, kernels and serving stack, which makes Nvidia’s software moat less absolute at the very top of the market.

OpenAI programs Jalapeño through Gluon, a low-level kernel language built around ideas from Triton. SemiAnalysis says some kernels run to thousands of lines and can be tuned almost like assembly. OpenAI also built its own serving stack and uses Codex heavily to search for faster implementations.

That approach would be painful for an ordinary company. OpenAI has several advantages: it knows which workloads will consume the hardware, it can modify the serving software, it can generate enormous amounts of kernel code with its own models, and it does not need millions of outside developers to support the platform.

As seen above, OpenAI even brought model architectures that were never part of Jalapeño’s original plan onto the chip quickly. That is probably the most important challenge to the traditional CUDA argument. Custom accelerators used to face a huge software tax every time workloads changed; AI coding systems may shrink that tax dramatically.

CUDA remains a huge advantage for Nvidia across the wider market. Inside a handful of frontier labs with enough scale and engineering talent, though, it may no longer be the barrier it once was.

Q14Will OpenAI actually replace Nvidia with Jalapeño?

No. OpenAI is building a much larger multi-supplier compute strategy, and Jalapeño currently gives it another major source of inference capacity rather than a reason to stop buying GPUs.

OpenAI’s commitments make the direction obvious. Its Broadcom program targets 10 gigawatts of OpenAI-designed accelerator systems through 2029. OpenAI also has an agreement covering up to 6 gigawatts of AMD GPUs and a separate Nvidia partnership targeting at least 10 gigawatts of Nvidia systems.

OpenAI has even reiterated lately that it plans to keep widely deploying Nvidia and other partner accelerators for both training and inference.

The interesting change is therefore inside the workload mix. If Jalapeño keeps performing well, OpenAI can shift a growing share of predictable, high-volume inference onto its own silicon while keeping Nvidia and AMD hardware for training, new model architectures, capacity bursts and workloads where programmability matters more.

Nobody outside OpenAI is expected to buy Jalapeño cards either. The chip is being built for OpenAI’s own infrastructure. Users may eventually get ChatGPT or API responses generated on Jalapeño without ever knowing which processor handled the request.

Q15So what is OpenAI’s Jalapeño chip exactly?

OpenAI’s Jalapeño is now best understood as the first working piece of OpenAI’s own inference-computing platform: custom silicon, memory, networking, racks and software designed together around the way modern LLMs actually run.

The chip itself is impressive, but the bigger story sits above it. OpenAI has gone from buying compute to designing a meaningful part of the machine that produces that compute. It has working first-generation silicon, competitive benchmark results, a software stack that is improving very quickly and a multi-generation roadmap already extending into Gen 2 and Gen 3.

Production is the part we still have to watch. OpenAI says it plans to begin deploying Jalapeño inside its compute infrastructure by the end of 2026. SemiAnalysis expects the production ramp to build through 2027, with much of the first-generation volume arriving later in that ramp. Going from engineering samples to hundreds of megawatts means dealing with yields, HBM supply, rack deployment, reliability, networking and data-center operations, none of which can be proven by a benchmark.

For now, though, calling Jalapeño an experiment undersells it. OpenAI has built a genuinely competitive LLM inference chip on its first generation, and the newest evidence suggests the real question has already moved on: how much of OpenAI’s future inference fleet can this architecture take over once production catches up?