In a September 24, 2026 post on its official blog, vLLM announced that its inference engine now officially supports Gumbel-max-based text watermarking. The goal is straightforward: make the provenance of model-generated text traceable, without distorting the model's output distribution and without slowing down inference. According to official measurements, Qwen3.5-27B with watermarking enabled scores within error bars on all benchmarks, with throughput changing between −1.1% and +2.0% — essentially negligible.
How the watermark works: marking the randomness
Every time a language model generates a token, it scores each candidate token and then samples randomly. The watermarking idea doesn't touch the text — it marks the "randomness" step itself: a secret key, the recent context tokens, and the candidate token's ID are fed through a pseudorandom function to compute a reproducible set of random values, added to the logits as Gumbel noise, and the token with the maximum value is picked.
The key mathematical property: taking argmax over Gumbel noise is exactly equivalent to ordinary sampling from the original probability distribution. In other words, at any single step, a token's probability of being chosen is identical with or without the watermark — the output distribution is not twisted, and the model doesn't systematically favor certain words or styles. Anyone holding the key can recompute those random values; without the key, it just looks like an ordinary random sample.
Detection: verification in reverse
Detection needs no model weights or logits — just the secret key and the tokenizer. Convert the text under test back to token IDs, and for each token recompute the pseudorandom value from its preceding context and the key: for unwatermarked text, these values are uniform random; for watermarked text, the chosen tokens systematically skew toward larger values. Summing each token's score follows a Gamma distribution, yielding a p-value — the smaller it is, the less likely the result is "a coincidence."
Longer text with more high-entropy steps gives a stronger signal: creative writing approaches near-100% detection by about 100 tokens. Copy-pasting a whole passage doesn't erase the evidence; local paraphrasing disturbs scores near the edited tokens, but once the edits slide out of the context window, the remaining signal stays intact.
Two hard problems: speculative decoding and diversity
Turning the paper into a shippable engine took solving two hard problems. The first is speculative decoding: if the draft and target models share one watermark, their distribution overlap drops and the acceptance rate suffers. vLLM's answer is dual keys (PR #56122) — accepted draft tokens use one key, the target model's residual and bonus tokens use another — preserving acceptance rate at the cost of combining both keys' scores at detection time, which dilutes the signal.
The second is output diversity: per-token distortion-freeness only guarantees the distribution of "one step," not of the whole sequence. If a context repeats, a fixed key produces identical random values, and the model can spiral into infinite repetition. vLLM solves it with generation-time context deduplication (PR #56233): when a context repeats, watermarking is skipped, preserving sequence-level distortion-freeness, with measured end-to-end throughput dropping by at most 0.19%.
On the engineering side, the watermark plugs directly into Model Runner v2's sampling pipeline (PR #54053), with pseudorandom generation, the Gumbel transform, and argmax fused into a single GPU kernel, avoiding huge temporary VRAM overhead.
Who will adopt it first
The first beneficiaries are companies and enterprises self-hosting vLLM: when serving APIs externally, they can stamp a verifiable "birth certificate" on outputs — for provenance, compliance trails, or telling their own model apart from impostors. The limits should be stated plainly: detection depends on key possession, so it can't be publicly verified; short, heavily templated text gives a weak signal; and it counters "provenance denial," not a determined paraphraser.
Text provenance has been talked about for years; vLLM has now turned it into a switch inside the inference engine — distortion-free, low overhead, production-ready. After this step, the debate over "should AI-generated content be traceable" has at least one fewer technical excuse.
