This LLM Doesn't Write One Token at a Time

Every LLM you use writes one token at a time.

This one doesn't.

Inception just launched Mercury 3, a diffusion LLM. Instead of writing left to right, it drafts the whole answer in parallel and then refines it, a bit like an image model sharpening a picture.

The numbers:

→ 1,068 tokens per second on Nvidia GPUs

→ 1M-token context window

→ $0.10 per million input tokens at launch (50% off)

→ OpenAI-compatible, so it's close to a drop-in swap

Inception claims it beats GPT-6 Luna and Claude Haiku 5.5 on agentic coding. Haiku still wins on longer coding tasks. These are vendor numbers, so test before you trust them.

Why speed matters this much: agents wait on the model at every step. In a voice call, a browser agent or a tutoring app, a few seconds per step adds up to an experience that feels broken.

For real-time products, "smart enough and fast" often beats "smartest".

Where would 1,000 tokens per second change your product?

2 comments

This LLM Doesn't Write One Token at a Time