Diffusion LLMs Ranked by Evidence

Only two diffusion language models have current, comparable hosted runtime rows: Celeris-1 and Mercury 2.

Celeris-1 leads those rows on both output rate and first-answer delay.

We leave the other entries unranked because their evidence uses different workloads and timing boundaries.

Marketing treats the category as a leaderboard. Evidence does not.

Only two hosted rows can share a rank

Table 1 Download CSV

Rank	Model	First answer	Output rate	Evidence status	Production access
1	Celeris-1	0.64 s	2,086 tok/s	Cross-provider runtime row	Hosted API, US access
2	Mercury 2	3.53 s	802 tok/s	Cross-provider runtime row	Hosted API
—	Gemini Diffusion	Not comparable	1,479 tok/s sampling claim	Google-published experimental test	Demo, no general API
—	DiffusionGemma	Not comparable	More than 1,000 tok/s claim	Google-published H100 test	Open experimental model
—	LLaDA	Not comparable	No hosted row	Peer-reviewed research preprint	Open research implementation
—	Dream 7B	Not comparable	No hosted row	Research paper and code	Open research implementation

Those first two rows share the current runtime feed and can be ordered within it. Celeris-1 is not yet eligible for the overall quality leaderboard because its sourced task coverage is narrow. Mercury 2 now has a public quality score from source-attached benchmark rows, but that score answers a different question. Speed evidence can identify a runtime leader without proving the better model.

Remaining speed figures do not share the same test. Google's Gemini Diffusion page labels 1,479 tokens per second as sampling speed and lists 0.84 seconds of separate overhead. DiffusionGemma's page says more than 1,000 tokens per second on one H100. Neither result is a hosted cross-provider measurement, so neither receives a synthetic rank.

We use an evidence ladder to decide what belongs in the numbered rows. A current hosted runtime measurement can rank against another row from the same feed. A provider test can describe its own model when hardware, workload, and timing boundary are visible. A paper can establish an evaluation for the exact checkpoint and protocol. None can silently move into another tier.

Table 2 Download CSV

Evidence tier	What it supports	What it does not support
Shared hosted runtime	Ordering inside the same snapshot	A permanent speed guarantee
Provider performance report	A dated result on the stated setup	Cross-provider rank without a shared method
Research paper	Reproducible checkpoint and protocol claims	Managed-service latency or availability
Architecture description	How the provider says generation works	Task quality or production throughput

Missing rows stay missing rather than becoming zeros. Research models with no managed endpoint are unranked, not slower than every commercial service.

A faster decode is not a better model

Autoregressive models commit to the next token, append it to the sequence, and repeat. Diffusion language models start from a noisy or masked representation and revise several token positions over a series of denoising steps. That parallelism can reduce the number of sequential model invocations needed to produce a response.

Parallel decoding does not eliminate order from language. Implementations still need to preserve dependencies, decide which positions are ready, and spend additional work correcting uncertain tokens. Serving software, hardware utilization, prompt processing, batching, and the requested reasoning budget can dominate the user-visible result.

Architecture alone cannot predict time to first answer. It also cannot predict task quality. Diffusion changes how tokens are generated. It does not make an undertrained checkpoint a stronger reasoner, and it does not make a hosted row eligible for a quality rank it has not earned.

Celeris has one source-attached MMLU-Pro result and does not yet have enough verified categories for an overall rank. A fast knowledge-test response does not establish coding, multilingual, tool-use, or visual performance. Mercury 2's quality score is real and source-attached. It still answers a different question from the speed table.

Celeris leads speed. Mercury has the broader hosted path

Celeris-1 records 2,086 output tokens per second and 0.64 seconds to first answer in the current hosted runtime feed. Celeris separately reports 1,664 tokens per second at p50 on a 1,000-token provider workload. We keep the two results separate because the prompts and timing method differ.

Celeris-1 accepts text and images, returns text, exposes a 131,072-token window, and uses an OpenAI-compatible endpoint. Official pricing is $0.20 per million input tokens and $0.70 per million output tokens. Public API access is currently limited to the United States.

Inception's Mercury 2 release reports 1,009 tokens per second on NVIDIA Blackwell, a 128K context window, tool use, schema-aligned JSON, and OpenAI-compatible chat completions. The official rate is a provider result. The hosted row here measures 802 tokens per second, which is still high, alongside a 3.53-second first-answer delay.

Those two facts can coexist. Output rate measures how quickly tokens arrive after generation begins. First-answer latency measures the wait before usable output appears. A workload that needs a long answer may value Mercury's throughput. A router that returns one short label may care almost entirely about the initial delay.

Inception prices Mercury 2 at $0.25 per million input tokens and $0.75 per million output tokens. That makes it inexpensive to replay against a real task set. Provider quality tables are still not independent evidence. The model page now links the admissible rows to exact public sources and keeps unresolved historical rows out of the score.

If you need a hosted OpenAI-compatible path today, those two rows are the shortlist. Celeris wins the measured clock, subject to US-only public access and thin task coverage. Mercury wins if you need the broader hosted quality evidence, or if first-answer delay matters less than sustained output.

Provider sampling claims stay off the numbered table

Google DeepMind's Gemini Diffusion page presents an experimental text model and an interactive demo. Its table reports 1,479 tokens per second of sampling speed, 0.84 seconds of overhead, and provider-run task results. Google explicitly separates sampling from overhead, which is helpful, but the deployment is not a generally available API that can enter the hosted test unchanged.

Google has demonstrated a high-throughput diffusion model in a controlled product preview. That evidence does not support placing Gemini Diffusion above or below Celeris-1 on a shared production leaderboard.

DiffusionGemma is the more accessible Google research path. It is an open, experimental mixture-of-experts model with 26 billion total parameters and 3.8 billion active parameters. Google reports more than 1,000 tokens per second on one H100. Teams can inspect and serve it, but must supply their own production measurements.

LLaDA established a masked-diffusion approach for large language models and released training and evaluation details. Later work extends the family, but paper results should remain attached to their exact checkpoint and protocol. No current managed endpoint sits in the hosted runtime feed.

Dream 7B publishes model code and checkpoints for diffusion-based reasoning. Its value in this tracker is reproducibility: researchers can inspect the implementation and run controlled comparisons. That is different from a service-level claim about tail latency, concurrency, or regional availability.

Calling either research model slow because it lacks a hosted row would be wrong. Calling it production-fast without one would be wrong too.

The rank is only as good as the artifact

Start with the path the application needs. Short interactive responses should weight first-answer latency and replay short prompts. Celeris-1 is the current measured leader, subject to its US-only availability and limited quality coverage. Long generated responses should measure both first answer and sustained output. Mercury 2's throughput and price make it a practical hosted candidate.

Weights, code, and controlled experiments belong on DiffusionGemma, LLaDA, or Dream when a managed endpoint is not the point. Treat Gemini Diffusion as an experiment until the API, model identifier, quotas, and production terms are public.

For every path, hold prompts, output length, region, concurrency, and hardware constant. Report median and tail behavior. Then run the same task-success gates used for autoregressive models.

Use at least three output lengths. Twenty-token routing responses emphasize first-answer delay. Five-hundred-token explanations mix the opening wait with generation. Two-thousand-token results make sustained output dominant. Record time to the first usable chunk and to the valid completed response, especially for JSON and tool calls that cannot be consumed halfway through.

Concurrency can reverse a single-request result. Send isolated requests first, then repeat at the expected parallel load. Keep rate-limit errors, timeouts, and retries in the report. Systems that serve one request at 2,000 tokens per second but queue the tenth request for several seconds may still be the wrong production choice.

Deployment terms are part of the ranking decision. Celeris-1 currently restricts public API access to the United States. Gemini Diffusion has no general production endpoint. Open models require hardware, an inference engine, monitoring, and an operator. Mercury 2 is managed, but the team should confirm regional routing, quotas, and support before treating a successful development request as production availability.

Recheck the model identifier and evidence date when repeating the test. “Mercury,” “Mercury 2,” a research checkpoint, and a provider demo may share an architecture family while representing different systems. The row must name the exact artifact that produced it.

Useful leaderboards admit when a row does not belong.