Signals Inbox·August 26, 2026·AI Chips

Is OpenAI’s Jalapeño chip better than Nvidia?

OpenAI’s Jalapeño is already better than Nvidia Blackwell for low-latency LLM inference, especially per watt. Against Rubin, the answer is still open, and against Nvidia as a whole, it is clearly no.

Q1When we say Jalapeño is “better than Nvidia,” what are we actually comparing?

Today, OpenAI’s Jalapeño looks better than Nvidia Blackwell for one very specific job: running large language models quickly while getting as much inference as possible from each watt of power.

That definition matters. Jalapeño was built around LLM inference, including the low-latency workloads behind ChatGPT, Codex and AI agents. Nvidia’s GPUs cover a much wider range of jobs, from frontier-model training to inference, scientific computing and accelerated computing in general.

So we should judge Jalapeño where OpenAI actually intends to use it. The useful tests are latency, throughput per watt, cost per token and performance on real agent workloads. On the first two, Jalapeño already has strong results against Blackwell. Cost looks promising. Agent workloads remain the biggest unanswered question. And once we move from Blackwell to Nvidia’s newer Rubin generation, the contest becomes much tighter.

Q2Did OpenAI’s Jalapeño actually beat Nvidia Blackwell?

Yes. OpenAI’s Jalapeño beat Nvidia GB200 and GB300 clearly in the latest public InferenceX tests.

OpenAI ran GPT-OSS 120B, DeepSeek R1 670B and Kimi K2.5 1T on Jalapeño and compared the results with Nvidia Blackwell systems. Across those three models, Jalapeño produced roughly 1.5 to 1.9 times more peak inference throughput per kilowatt. End-to-end latency was roughly 1.7 to 3.6 times lower.

The power-adjusted result is the impressive part. Jalapeño has a 700-watt rated package, while the GB300 configuration in the DeepSeek and Kimi comparisons uses a 1,400-watt rating. OpenAI also says Jalapeño stayed at or below roughly 550 watts of measured sustained power during the tested workloads.

We recalculated the approximate raw throughput from OpenAI’s published throughput-per-kilowatt figures and package ratings. That gives a more nuanced picture. Jalapeño slightly beats GB200 in raw GPT-OSS throughput, while one GB300 package still produces more raw peak throughput on DeepSeek and Kimi. Jalapeño gets much more work from the available power.

Jalapeño vs Nvidia Blackwell in the latest public InferenceX tests

Q3Are the claims that Jalapeño is 50x or 100x better than Nvidia real?

The 50x and 100x Jalapeño figures come from real benchmark points, but reading them as a 50x or 100x raw-speed advantage over Nvidia would be wrong.

OpenAI measured how much traffic each system could handle while maintaining the same very low token latency. On GPT-OSS, Jalapeño delivered about 54 times more throughput than GB200 at the latency Nvidia previously achieved. On DeepSeek R1, the gap exceeded 100 times at one matched token-latency point.

Those huge ratios appear because Nvidia’s throughput falls sharply when the system is forced to operate at extremely low latency. GPUs can recover a lot of throughput by batching many users together, although the individual requests then wait longer. Jalapeño stays efficient much closer to the low-concurrency end of the curve.

That is useful for ChatGPT or an interactive coding agent, where OpenAI cares about how much traffic it can serve without making each user wait. Still, the cleaner overall comparison remains the roughly 1.5 to 1.9 times advantage in peak throughput per watt. We would avoid describing Jalapeño as “100x faster than Nvidia.”

Q4Would Jalapeño actually make ChatGPT or an AI agent feel faster?

Yes. Jalapeño’s strongest advantage against Blackwell may be the speed experienced by each individual user rather than its maximum aggregate throughput.

On DeepSeek R1, Jalapeño reached roughly 700 generated tokens per second per user at its lowest token-to-token latency point. The comparable GB300 figure was about 169 tokens per second. On Kimi K2.5, Jalapeño reached roughly 694 tokens per second versus about 182 on GB300.

The end-to-end numbers tell the same story. DeepSeek R1 completed the benchmark request in about 1.65 seconds on Jalapeño compared with 5.99 seconds on GB300. Kimi took about 1.56 seconds versus 5.31 seconds.

A three- or four-second gap is already noticeable in chat. It becomes much more important when an agent repeatedly calls models, tools and sub-agents. If a task requires dozens of sequential model steps, latency gets paid again and again. That is why we think Jalapeño’s low-latency result is more commercially interesting than the spectacular 50x or 100x benchmark points.

Q5Can we actually trust OpenAI’s Jalapeño benchmarks?

We trust the basic conclusion that Jalapeño beat Blackwell under the tested conditions, although the current evidence still falls short of proving that Jalapeño wins across production AI workloads.

OpenAI used InferenceX, the public inference benchmark built by SemiAnalysis, rather than inventing a private benchmark specifically for Jalapeño. More importantly, SemiAnalysis researchers went into OpenAI’s lab and watched the InferenceX runs. They also checked model quality using GSM8K evaluations and reported comparable results with Nvidia.

There is still an important limitation. SemiAnalysis says the benchmark numbers were provided by OpenAI, and its researchers did not independently operate the entire system or run the full InferenceX suite. The published tests use nominal 8k-token inputs and 1k-token outputs in single-turn workloads.

That is enough for us to take the performance seriously. It is too narrow to extrapolate confidently to every model, context length and production traffic pattern.

Q6Is Jalapeño better than Nvidia Rubin too?

Not yet. The first comparison is surprisingly uncomfortable for Nvidia, though.

SemiAnalysis compared Jalapeño with the best public Rubin results available from Nvidia and CoreWeave. On output-token throughput per megawatt, Jalapeño’s single-token-prediction results came out ahead of those early Rubin results, even though Rubin was already using multi-token techniques. SemiAnalysis’s current total-cost model also puts Jalapeño and Vera Rubin roughly head-to-head on output tokens per dollar.

That is a much harder comparison than Blackwell. Rubin and Jalapeño are from roughly the same hardware generation, both use HBM4 and both were designed with reasoning and agentic inference in mind.

Rubin also keeps improving. Nvidia’s newest performance disclosures show Vera Rubin NVL72 reaching up to 30 times the throughput per megawatt of GB300 NVL72 on some AgentX coding-agent workloads. Those results cannot be placed directly beside Jalapeño because OpenAI has not released an AgentX run yet.

So the evidence currently supports a stronger claim than “Jalapeño beats old Nvidia hardware.” OpenAI has already built an inference accelerator that belongs in the same performance conversation as Rubin. A clean Rubin victory for either side still needs a proper head-to-head workload.

Q7Why is Jalapeño so efficient compared with Nvidia GPUs?

OpenAI’s Jalapeño gets much of its inference advantage from reducing the waiting, movement and fixed overhead that can keep compute hardware idle during small, latency-sensitive LLM workloads.

Large-language-model inference constantly moves weights, activations and KV-cache data while coordinating work across cores and chips. OpenAI designed Jalapeño around those specific patterns. The architecture gives each core slice a low-latency view of its portion of HBM and uses dedicated communication paths for predictable collective operations.

Interestingly, Jalapeño does not simply overpower Rubin on the specification sheet. SemiAnalysis reports 15.4 TB/s of HBM4 bandwidth per Jalapeño package, while Nvidia specifies 22 TB/s for Rubin. SemiAnalysis also reports 13.4 PFLOPS of MXFP4 compute for Jalapeño’s newer B0 compute die versus 17.5 PFLOPS of dense NVFP4 for a Rubin compute die.

Yet Jalapeño performs extremely well in measured inference. That tells us OpenAI is extracting a high percentage of the hardware it actually bought instead of relying on a larger theoretical peak.

The current benchmark also ran on the earlier A0 stepping. SemiAnalysis says the B0 stepping now in fabrication contains changes expected to improve performance per watt by roughly another 25%. We would treat that as an engineering projection until production silicon confirms it, but Jalapeño clearly still has headroom.

Q8Is Jalapeño secretly optimized only for OpenAI models?

No. Jalapeño’s strongest argument for being a real LLM accelerator today is that it also performs well on models built outside OpenAI.

Two of the three main public benchmarks use DeepSeek R1 and Moonshot AI’s Kimi K2.5. Jalapeño produced its strongest latency advantages on those outside models, with roughly four times better minimum token latency than GB300 on DeepSeek and nearly four times on Kimi.

OpenAI also says three open-weight model families that were absent from Jalapeño’s original production plan were brought to high performance in roughly two months. That is an important test for a custom accelerator. LLM architectures change quickly, so a chip tied too closely to one model generation can become obsolete before the hardware has paid for itself.

We should still be precise about the kind of flexibility OpenAI has demonstrated. Jalapeño looks general across modern LLM inference. Nvidia GPUs remain far more general across computing as a whole.

Q9Can Jalapeño beat Nvidia on long-context AI agents?

We still do not know whether OpenAI’s Jalapeño beats Nvidia on the long-context agent workloads that could become the biggest inference market.

The missing benchmark is AgentX. SemiAnalysis built AgentX around production-style coding-agent sessions with long contexts, multiple turns, tool-call pauses, changing concurrency and heavy KV-cache reuse. Those workloads put much more pressure on the entire serving system than a clean 8k-input/1k-output request.

Nvidia has already started publishing Rubin results on AgentX. Its latest tests show Vera Rubin NVL72 delivering up to 30 times more throughput per megawatt than GB300 NVL72 on selected agent workloads. OpenAI has yet to publish the equivalent Jalapeño run.

That gap is important. Jalapeño’s architecture looks well suited to agents because OpenAI designed it around low latency, locality and rapidly changing inference workloads. We simply have not seen the benchmark that proves those advantages survive one-million-token contexts, repeated cache reuse and multi-turn production traffic.

This is the test we'd watch most closely right now.

Q10Is Jalapeño actually cheaper than Nvidia for OpenAI?

Jalapeño will probably become cheaper for OpenAI at scale, but the best current comparison puts its cost efficiency surprisingly close to Rubin rather than miles ahead.

SemiAnalysis’s total-cost analysis finds Jalapeño and Vera Rubin producing roughly similar output tokens per dollar today. That may initially sound disappointing given Jalapeño’s much stronger power numbers, but the software setups are different. Jalapeño produced its current results without speculative decoding, while the Rubin results already use techniques that can improve token economics substantially.

So OpenAI has software upside left. SemiAnalysis estimates speculative decoding can improve cost per output token by more than threefold in suitable configurations. We would not assume Jalapeño automatically captures all of that gain, but its current cost position was achieved before several mature inference tricks have been added.

Power is another part of the economics. SemiAnalysis says OpenAI is currently constrained heavily by available data-center power. In that situation, tokens per megawatt can matter more than the sticker price of an accelerator. A system that generates 50% or 80% more useful inference from an already powered facility can create capacity that OpenAI cannot easily obtain by adding another building.

Custom silicon also changes who collects the margin. OpenAI still pays Broadcom, HBM suppliers, TSMC, packaging companies, networking suppliers and system integrators. But it no longer has to buy every inference accelerator through Nvidia’s merchant-GPU economics.

Our read is that Jalapeño’s financial case looks strong even before it produces dramatically cheaper tokens than Rubin. Reaching roughly Rubin-class token economics on a first-generation internal chip is already a meaningful result.

Q11Can OpenAI make enough Jalapeño chips for them to matter?

Jalapeño is currently far too early in its production ramp to change Nvidia’s position by volume, and manufacturing is now a bigger question than whether the chip works.

OpenAI says it plans to begin putting Jalapeño into its own infrastructure by the end of the year. SemiAnalysis expects production to ramp progressively through 2027, with most currently scheduled output weighted toward the final quarter of that year. The immediate fleet will therefore look tiny beside the enormous Nvidia base OpenAI already uses.

The system itself has already been designed for much larger deployments. SemiAnalysis describes racks containing 128 Jalapeño accelerators and a scale-up network connecting as many as 2,048 accelerators across 16 racks. OpenAI and Broadcom have a multi-generation custom-accelerator program targeting 10 gigawatts of deployment by the end of 2029.

The next hurdle is operational. OpenAI has to manufacture enough good chips, install racks quickly, keep them reliable, mature monitoring and recovery software and run the hardware continuously inside production data centers.

Nvidia has spent years getting very good at those boring parts. Jalapeño’s first benchmarks show that OpenAI can design a competitive chip. The next year should tell us whether OpenAI can operate a competitive hardware platform.

Q12Can OpenAI really get around Nvidia’s CUDA advantage?

Inside OpenAI’s own infrastructure, Jalapeño is already showing that CUDA may be a much smaller barrier than it is for companies buying hardware on the open market.

OpenAI controls the models, inference software and hardware at the same time. It programs Jalapeño through its own stack, including Gluon, which builds on Triton, and it increasingly uses Codex to write and optimize low-level kernels.

The speed of that software work stands out. OpenAI says it brought three previously unplanned open-weight models to high performance in about two months. For selected GPT-OSS attention and mixture-of-experts blocks, AI-generated implementations ran roughly 1.5 to 1.8 times faster than the existing human-expert versions.

SemiAnalysis saw the same pace during its benchmarking work. Jalapeño throughput at some operating points improved by more than two times over a period of less than two weeks. The team also expanded from TP8 to TP32 configurations and got a full rack-scale setup working on a large model within eight days.

CUDA remains incredibly valuable for the thousands of companies that need one mature platform to support whatever workload appears next. OpenAI has a different problem. It only needs Jalapeño’s software stack to support the models OpenAI actually intends to run, and its own coding models can help close software gaps quickly.

That makes custom silicon much more viable for OpenAI than it would be for a normal cloud customer trying to leave Nvidia.

Q13Can Jalapeño train OpenAI’s next frontier model?

No. Jalapeño currently focuses on inference, which leaves Nvidia deeply embedded in OpenAI’s frontier-model training infrastructure.

This sharply limits how far we should take the “OpenAI beat Nvidia” argument. Training a new frontier model requires enormous compute clusters, mature distributed software and hardware that can sustain large compute-heavy workloads. Nvidia built Rubin for training, post-training and inference, while Jalapeño’s architecture and published results focus on serving already-trained LLMs.

OpenAI’s own spending confirms the distinction. Earlier this year, the company expanded its Nvidia relationship with 2 gigawatts of Vera Rubin capacity specifically for training, alongside another 3 gigawatts dedicated to Nvidia inference.

Jalapeño can therefore take an increasingly valuable slice of OpenAI’s compute bill without eliminating the need for Nvidia. Training remains one of the biggest pieces Nvidia controls.

Q14Is OpenAI actually trying to replace Nvidia with Jalapeño?

OpenAI is building bargaining power and workload flexibility around Nvidia while still planning to buy Nvidia systems at enormous scale.

The commitments are unusually revealing. OpenAI and Broadcom have a 10-gigawatt roadmap for OpenAI-designed accelerators. Separately, OpenAI and Nvidia announced a framework for at least 10 gigawatts of Nvidia systems, with Vera Rubin powering the first gigawatt. OpenAI then added the more concrete 3 gigawatts of dedicated Nvidia inference and 2 gigawatts of training capacity announced earlier this year.

Those numbers make a complete Nvidia replacement very hard to argue. OpenAI is simultaneously building one of the world’s largest custom-chip programs and planning one of the world’s largest Nvidia deployments.

The strategy makes more sense by workload. High-volume inference that maps well to Jalapeño can move onto OpenAI silicon. Nvidia can continue serving training, new models that have not yet been optimized for Jalapeño and workloads where Nvidia offers better economics.

For OpenAI, even shifting a large minority of inference onto Jalapeño would be strategically valuable. Nvidia would have to compete for workloads that OpenAI previously had little choice but to place on Nvidia hardware.

Q15So, is OpenAI’s Jalapeño chip actually better than Nvidia?

Our answer today is yes for Blackwell-class LLM inference, unproven against Rubin, and no for the broader Nvidia platform.

The Blackwell part is already fairly clear. Jalapeño beat GB200 and GB300 across three large models on the combination OpenAI cares about most: low latency and throughput per watt. As seen above, the advantage reached roughly 1.5 to 1.9 times at peak throughput per kilowatt and as much as 3.6 times on end-to-end latency. SemiAnalysis watched the benchmark runs and found the first-generation chip genuinely competitive rather than a paper launch.

Rubin is where we would stop making a definitive claim. SemiAnalysis’s early comparison actually puts Jalapeño ahead on some throughput-per-megawatt measurements and roughly level on current token economics. Nvidia, meanwhile, has already shown major Rubin gains on the much harder AgentX workloads that OpenAI has yet to publish for Jalapeño. Until both systems run comparable long-context agent tests, saying Jalapeño has beaten Rubin would go beyond the evidence.

The broader Nvidia comparison is easier. Jalapeño cannot currently take over OpenAI’s frontier training, it has barely started its production ramp, and OpenAI itself is committing gigawatts of new capacity to Nvidia. Nvidia also has a mature platform spanning GPUs, CPUs, networking, software and thousands of workloads that Jalapeño was never designed to cover.

The important change is narrower. Nvidia can no longer assume that the inference workload of one of the world’s largest AI companies automatically belongs on Nvidia GPUs. OpenAI has built a first-generation chip that already beats Blackwell where OpenAI cares most and is close enough to Rubin to make the next benchmark genuinely interesting. If later Jalapeño generations keep that pace while the fleet moves from engineering samples to gigawatt-scale production, Nvidia will still be central to OpenAI, but it will have to earn a much larger share of OpenAI’s inference business.

Jalapeño vs Nvidia: our verdict today