Diffusion vs Autoregressive Language Models

Two hosted diffusion models on our runtime feed differ by more than five times in first-answer latency.

Celeris-1 starts in 0.64 seconds. Mercury 2 takes 3.53 seconds. Both stream far faster than typical autoregressive endpoints once they begin.

We treat parallel decoding as an engineering opportunity, not proof of lower latency, higher quality, or cheaper serving.

Architecture sets the chance. The serving stack decides how much of it reaches a user.

The same architecture is not the same clock

Our runtime feed puts numbers on it. Here are the fastest first-answer rows we currently measure, with a diffusion model among them:

Table 1 Download CSV

Model	Latency (first answer)	Output speed	Type	Overall score
Gemini 2.5 Flash	0.47s	221 t/s	Non-Reasoning	47
Grok 4.1 Fast	0.54s	138 t/s	Non-Reasoning	51
GPT-4.1 nano	0.63s	181 t/s	Non-Reasoning	42
GPT-OSS 20B	0.65s	313 t/s	Non-Reasoning	42
Gemini 3 Flash	0.75s	199 t/s	Non-Reasoning	60
GPT-4.1 mini	0.76s	80 t/s	Non-Reasoning	44

Look at the output-speed column rather than the latency column. Celeris-1 starts answering at roughly the same moment as the quick autoregressive rows around it, then streams at a rate several times higher than any of them. That is the architecture's promise arriving intact.

Now the same architecture, served differently. On the same feed and the same review date, Mercury 2 measured 3.53 seconds to first answer with 802 tokens per second, against Celeris-1's 0.64 seconds and 2,086 tokens per second. Both are hosted diffusion models. One of them is more than five times slower to begin while still streaming four times faster than a typical autoregressive endpoint once it starts.

Nothing about masked denoising explains a gap that size. Step counts, batching policy, kernels, hardware, and routing do.

Autoregressive decoding stays sequential for a reason

Autoregressive models receive a prompt, predict a probability distribution for the next token, select one token, append it, and repeat. Every new token depends on the generated prefix. That dependency makes the core decoding loop sequential.

Serving systems still exploit parallel hardware. They batch requests, process prompt tokens together during prefill, reuse cached prefixes, and sometimes use a smaller draft model to propose several tokens for verification. Those techniques can produce high throughput without changing the model's left-to-right factorization.

Maturity and predictability are the practical strengths. Most model APIs, tool-call protocols, streaming interfaces, quantization methods, and inference engines were built around autoregressive decoding. The ecosystem knows how to operate it, which is worth more than it sounds when something breaks at three in the morning.

Diffusion revises positions. It does not skip serving

Text diffusion models start from a sequence containing masked or noisy positions. Across multiple steps, they predict replacements, keep high-confidence tokens, and revise uncertain positions. Several tokens can change during one invocation.

LLaDA describes a masked-diffusion language model trained to reconstruct randomly masked tokens. Dream publishes another diffusion reasoning implementation. DiffusionGemma uses a mixture-of-experts design and exposes open experimental weights.

These systems do not all use one algorithm. Masking schedules, sequence length, confidence thresholds, step counts, bidirectional context, and post-training differ between them. "Diffusion LLM" is a family label, not a shared service profile.

Updating multiple positions can reduce the number of sequential decoding steps. It can also spend computation on positions that later change. Models needing many denoising passes give back part of the theoretical gain.

User-visible latency also includes more than decoding:

Table 2 Download CSV

Stage	Autoregressive pressure	Diffusion pressure
Prompt prefill	Process the input before decoding	Process the input before denoising
Output generation	Sequential token dependency	Multiple refinement steps
Streaming	Natural token-by-token chunks	Requires a policy for stable, displayable text
Tool calls	Arguments grow left to right	Arguments may be revised before commitment
Serving	Mature optimized engines	Newer kernels and scheduling paths

Memory behavior can differ as well. Parallel refinement may change how serving systems allocate compute across a sequence, while autoregressive decoding maintains a growing cache of prior token state. Production effects depend on implementation, batch, sequence length, and hardware, and the architecture label alone predicts none of it.

Read our numbers with their limits attached. Each row is one route measured on one feed, not a service-level guarantee, and a provider can change any of it without renaming the model. The speed dashboard carries the current rows.

Provider speed numbers are not one benchmark

Inception reports 1,009 tokens per second for Mercury 2 on NVIDIA Blackwell. Google reports 1,479 tokens per second of sampling speed for Gemini Diffusion and lists overhead separately. DiffusionGemma's page reports more than 1,000 tokens per second on one H100.

Provider-published results use different models, prompts, hardware, serving paths, and timing boundaries. Dividing one by another does not yield a trustworthy speed advantage.

Our own Mercury 2 row sits below Inception's published figure, which is the ordinary result of measuring a public endpoint under different conditions rather than evidence that anyone is wrong. Vendor numbers describe what the architecture can do on a chosen configuration. Runtime numbers describe what an API returned on a particular day, and applications experience the second one.

Our diffusion LLM tracker ranks only models that share the current hosted runtime feed. Other rows stay visible and unranked, which preserves useful evidence without manufacturing precision.

Quality still comes from training, not the decoder

Diffusion does not turn an undertrained model into a strong reasoner. Autoregression does not prevent a model from serving an interactive product. Architecture interacts with scale, data, objectives, post-training, tool instruction, and inference budget.

Look at the overall-score column above. Both hosted diffusion models score in the forties on our current ranking, well below the frontier autoregressive models. Speed is real and measured. Parity on task quality is not established, and buying the throughput means accepting that trade knowingly.

Price interacts with the same decision. Mercury 2 currently lists at $0.25/$0.75 per million input and output tokens, which is inexpensive enough that the interesting question is not whether it is cheap but whether it is good enough for the specific job.

Evaluate exact checkpoints on exact tasks. For a coding agent, run repository-level fixes and tool calls. For search, test citation support and abstention. For voice, measure first audible response and interruption recovery. General benchmarks screen candidates and cannot substitute for the failure modes of an application.

Missing evidence should stay missing. A provider table can inform a trial while the model remains outside an independent quality rank.

Streaming is where the product actually changes

Autoregressive output is append-only: once a token is streamed, the model normally does not edit it. Diffusion decoders may revise positions until they stabilize. Serving layers must decide when a span is safe to release.

Commit policy affects perceived latency and correctness. Commit too late and the system hides the architecture's speed. Commit too early and text may need corrections, which is awkward for users and dangerous for JSON, code, or tool arguments.

Ask a provider directly how its commit policy works. That rule is rarely documented, it changes between versions, and it is the implementation detail most likely to surprise an application built on the assumption that streamed text is final. Test it by streaming a long structured response and diffing the intermediate chunks against the finished output.

Voice makes it harder still, because spoken audio cannot be silently edited. Implementations need stable chunks before synthesis, or a strategy confining revision to text that has not been said yet. Anyone pairing a diffusion model with a voice agent should test that boundary first, since a revision arriving after synthesis is a defect the caller hears.

Choose a hosted diffusion model when its measured first-answer time, output rate, price, task success, and API behavior beat the alternatives for the deployed workload. High sustained throughput at low cost is a genuine advantage for long generations, bulk transformation, and drafting, where the answer is long and the quality bar is a revision away.

Choose an open diffusion model when research access or self-hosting justifies the newer serving path.

Choose an autoregressive model when it wins the same test, or when mature tooling, broad provider access, stable streaming, and operational familiarity matter more than experimental throughput. For short interactive answers, the first-answer column decides the experience and a fast autoregressive endpoint frequently wins it.

Then repeat the test after a provider update, because the gap between two diffusion models on our own feed is larger than the gap between the architectures.

Measurable behavior is the useful distinction. Architecture explains part of it. Production evidence makes the decision.