> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sglang.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Text Watermarking

SGLang provides opt-in Aaronson-Gumbel watermarking for sampled text. The watermark replaces random token selection with a keyed, deterministic choice that follows the same output distribution under the ideal independent-uniform hash model, so anyone holding the secret key can later test whether a token sequence carries the watermark. Detection needs only the key and the token IDs; it does not need the model or a GPU.

This feature can support transparency controls such as the machine-readable marking and detectability described in [Article 50 of the EU AI Act](https://eur-lex.europa.eu/eli/reg/2024/1689/oj). It is one technical signal, not a complete compliance program or proof of authorship.

## How it works

### Generation

At each sampling step, SGLang first applies frequency, presence, and repetition penalties, logit bias, and any structured-output grammar mask, followed by the request's temperature and its top-k, top-p, and min-p truncation, then renormalizes. Let `p_i` be the resulting probability of candidate token `i`; grammar-masked and truncated tokens have `p_i = 0` and can never be selected. For each candidate, SGLang hashes the key, the preceding token context, and the token ID into a uniform value `u_i` in `(0, 1)` and selects

```text theme={null}
argmax over i with p_i > 0 of  log(u_i) / p_i
```

When the `u_i` are independent uniforms, this rule selects token `i` with probability exactly `p_i`, so a single step is distributed like ordinary sampling. Ties go to the lowest token ID.

A position is watermarked only when all of the following hold; otherwise the request samples that position normally:

* The request has watermarking enabled (see [Per-request control](#per-request-control)).
* The request is not greedy. `temperature=0` is normalized to `top_k=1`, and `top_k=1` positions are never watermarked.
* The context is not empty.
* The context has not already been watermarked earlier in the same request. Runtime identifies a context by its 32-bit context hash.
* The request has watermarked fewer than `min(4,096, per-request token-pool capacity)` contexts.
* The largest truncated probability is at most the server's `max_probability`. The default `1.0` disables this gate.

Only a watermarked position enters the request's context history. A position skipped by the probability gate therefore leaves its context eligible for a later, higher-entropy occurrence, and once a request reaches its context budget the remaining tokens use ordinary sampling. The repeated-context rule keeps a repeated context from forcing the same token indefinitely.

The context for a position is the last `h` tokens before it, where `h` is the request's context window, taken from the prompt followed by the tokens generated so far. The first generated token therefore uses the last `h` prompt tokens. A context is shorter than `h` only when fewer than `h` tokens precede the position.

### Hash contract

All arithmetic wraps to unsigned 32 bits. The primitives are the MurmurHash3 x86 32-bit body mix and finalizer:

```text theme={null}
mix(state, value):
  value = value * 0xcc9e2d51
  value = rotl32(value, 15)
  value = value * 0x1b873593
  state = rotl32(state XOR value, 13)
  return state * 5 + 0xe6546b64

fmix32(value):
  value = value XOR (value >> 16)
  value = value * 0x85ebca6b
  value = value XOR (value >> 13)
  value = value * 0xc2b2ae35
  return value XOR (value >> 16)
```

A key is 1 to 16 hexadecimal digits with an optional `0x` prefix, interpreted as an unsigned 64-bit integer; `key_lo` and `key_hi` are its low and high 32-bit words. For a context of `L` token IDs `c_1 ... c_L` and a candidate token `t`:

```text theme={null}
context_hash = fmix32(mix(...mix(mix(0, c_1), c_2)..., c_L) XOR (4 * L))
token_hash   = fmix32(mix(mix(mix(mix(0, key_lo), key_hi), context_hash), t) XOR 16)
u            = (token_hash + 0.5) / 2^32
```

The context hash covers only the context tokens; keys enter only through the token hash and, in dual-key mode, through a per-context coin that chooses which key produces the token hash:

```text theme={null}
coin      = fmix32(mix(mix(mix(mix(mix(0, a_lo), a_hi), b_lo), b_hi), context_hash) XOR 20)
threshold = floor(P * 2^32)
key       = key A if coin < threshold else key B
```

`a_lo`/`a_hi` and `b_lo`/`b_hi` are the words of key A and key B, and `P` is the server's `mixing_probability`.

### Detection

A detector holding the key scores each generated token `t` with `s = -log(1 - u)`, where `u` comes from the hash contract above using that token's context. Without the watermark, each `s` is an independent `Exponential(1)` sample. With the watermark, the generator preferred tokens with large `u`, so the scores are inflated.

Over `n` scored contexts, the score `S = sum(s)` follows `Gamma(shape=n, scale=1)` under the null hypothesis that the text is not watermarked with this key. The detector reports the upper-tail probability `P[Gamma(n, 1) >= S]` as the p-value, its natural logarithm (finite even when the p-value underflows to `0.0`), and the normal approximation `z = (S - n) / sqrt(n)`.

To match generation, a detector must follow these rules:

1. Use the exact prompt token IDs that preceded the completion and the request's context window `h`. With prompt IDs, the first completion token's context is the last `h` prompt tokens. Without prompt IDs, the first `h` completion tokens have unknown contexts and are skipped.
2. Score each distinct context once. The detector compares exact context token tuples, whereas generation compares 32-bit context hashes. After a hash collision, generation skips a later distinct context that the detector still scores. That extra score is an ordinary null sample, so the mismatch can only weaken detection, never create evidence.
3. Stop after the generation request's context budget and report a prefix test. The default detector budget is 4,096 distinct contexts. If the server's per-request token-pool capacity is lower, pass that lower value as `max_contexts`; scoring the unforced tail adds null samples that dilute the statistic without adding evidence. With a maximum-probability gate below `1.0`, the model-free detector cannot identify skipped positions, so this fixed prefix is conservative and may omit signal from later forced positions.

Positions that generation left unforced (greedy positions, and positions skipped by `max_probability`) are indistinguishable from ordinary text to the detector, which scores them as null samples; they dilute the statistic but cannot produce false evidence.

A low p-value is evidence that the text carries the keyed watermark. It is not a calibrated probability that a particular author or model produced the text. Applications must choose a decision threshold and measure false-positive and false-negative rates on their own traffic.

## Server configuration

`--enable-watermark` turns the feature on; every other watermark setting lives in one JSON object passed with `--watermark-config`, either as a file path or inline. Restrict the file to the server account and launch a CUDA server:

```json theme={null}
{
  "key": "0123456789abcdef",
  "key_b": "fedcba9876543210",
  "context_window": 4,
  "mixing_probability": 0.5,
  "max_probability": 1.0,
  "default_enabled": true,
  "enforce_all": false
}
```

```bash theme={null}
chmod 600 /run/secrets/sglang-watermark.json
python -m sglang.launch_server \
  --model-path Qwen/Qwen3-8B \
  --enable-watermark \
  --watermark-config /run/secrets/sglang-watermark.json
```

| Field | Default | Behavior |
| - | - | - |
| `key` | unset | Server key (key A), 1 to 16 hex digits, for requests that enable watermarking without their own key. |
| `key_b` | unset | Enables dual-key mode with a second server key. Requires `key`. |
| `context_window` | `4` | Default context window and the maximum that a request may select, from 1 to 64 tokens. |
| `mixing_probability` | `0.5` | In dual-key mode, the probability of selecting key A for each context. Must be strictly between 0 and 1; a non-default value requires `key_b`. |
| `max_probability` | `1.0` | Skips watermarking when the largest truncated probability exceeds this value. Must be greater than 0 and at most 1; `1.0` disables the gate. |
| `default_enabled` | `false` | Watermarks requests that omit the `watermark` field with the server key. Requests may opt out. Requires `key`. |
| `enforce_all` | `false` | Watermarks requests that omit the `watermark` field and rejects `{"enabled": false}`. Requires `key`. |

Every field is optional and any other field is rejected. The config must be a JSON object of at most 4,096 bytes. A value that starts with `{` is parsed as inline JSON; any other value is read as a path to a regular file, and SGLang logs a warning when that file is readable by group or other users. Inline JSON is visible in the process command line, including `/proc/*/cmdline` on Linux, so pass keys through a file. `--watermark-config` requires `--enable-watermark`. A server key alone does not watermark every request; without `default_enabled` or `enforce_all`, requests must opt in.

SGLang redacts the keys and the `--watermark-config` value from the recorded launch command, startup logs, request logs, server information responses, and request crash-dump payloads. Process and GPU core dumps may contain in-memory keys. Protect those artifacts, the configuration file, and the transport used by clients that send per-request keys.

## Per-request control

The native `/generate` API and the OpenAI-compatible `/v1/completions` and `/v1/chat/completions` APIs accept a `watermark` object with three optional fields:

| Field | Type | Meaning |
| - | - | - |
| `enabled` | boolean | `true` enables watermarking with the server key; `false` opts out. |
| `key` | hex string | Watermarks with this key instead of the server key. A key implicitly enables watermarking. |
| `context_window` | integer | Context window for this request, from 1 to the server's `context_window`. |

On `/generate`, pass `watermark` either at the top level of the request or inside `sampling_params`, but not both; a batched request may pass one object per prompt as a list. On the OpenAI-compatible APIs, pass it as a top-level request field:

```bash theme={null}
curl http://127.0.0.1:30000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Qwen/Qwen3-8B",
    "messages": [{"role": "user", "content": "Explain photosynthesis."}],
    "temperature": 1.0,
    "top_p": 0.95,
    "return_token_ids": true,
    "watermark": {"enabled": true, "context_window": 4}
  }'
```

The server resolves each request as follows:

| Server mode | No `watermark` field | `{"enabled": false}` | `{"enabled": true}` | `{"key": "HEX"}` |
| - | - | - | - | - |
| Watermarking disabled | Off | Off | HTTP 400 | HTTP 400 |
| `--enable-watermark`, no server key | Off | Off | HTTP 400 | Request key |
| `--enable-watermark` with a server key | Off | Off | Server key | Request key |
| Add `default_enabled` | Server key | Off | Server key | Request key |
| Add `enforce_all` | Server key | HTTP 400 | Server key | Request key |

The request also returns HTTP 400 when:

* the `watermark` object sets neither `enabled` nor `key`, or contains an unknown field;
* `key` is combined with `"enabled": false`;
* a key is not 1 to 16 hexadecimal digits;
* `context_window` is not a positive integer or exceeds the server maximum;
* a watermarked request uses beam search.

Request-level keys are credentials carried in the request body. Use an authenticated TLS endpoint whenever they cross a host boundary, and avoid recording request bodies in proxies or application logs.

## Dual-key mode

`key_b` in `--watermark-config` applies dual-key selection to every watermarked request. The request's resolved key, whether the server key or a request key, is key A, and the server's second key is key B. A request key therefore replaces key A while key B stays active. Each eligible context uses key A with probability `mixing_probability` and key B otherwise, according to the coin in the [hash contract](#hash-contract); truncation, repeated-context, and greedy rules are unchanged. Without key B, selection uses the single-key path.

A detector holding both keys splits the distinct contexts by the coin, scores each context with its selected key, and tests the union, which is the strongest statistic; it can also test each partition separately. A detector holding only one key can score every distinct context with that key, but the contexts selected for the other key contribute only null samples and dilute the signal.

## Using the detector

`WatermarkDetector` implements the [detection rules](#detection) above. Construct it with the same key, key B, mixing probability, and context window that generation used:

A minimal [reference detection server](https://github.com/sgl-project/sglang/tree/main/examples/watermark) exposes the in-tree detector over HTTP and reads keys only from `--watermark-config`.

```python theme={null}
WatermarkDetector(
    key,                      # hex key A
    *,
    key_b=None,               # hex key B for dual-key mode
    mixing_probability=0.5,   # must match the server's mixing_probability
    context_window=4,         # the request's effective window, 1 to 64
    max_contexts=4096,        # prefix-test length, 1 to 4096
)
```

* `detect_tokens(token_ids, *, prompt_token_ids=())` scores completion token IDs.
* `detect_text(text, tokenizer, *, prompt_token_ids=())` first encodes the text with `tokenizer.encode(text, add_special_tokens=False)`. Any object with that method works, including Hugging Face tokenizers.
* `detect(token_ids, *, key, key_b=None, prompt_token_ids=(), mixing_probability=0.5, context_window=4)` is a one-call shortcut.

Token IDs must be unsigned 32-bit integers; any iterable of Python or NumPy integers is accepted.

Every call returns a `WatermarkDetection`:

| Field | Meaning |
| - | - |
| `combined` | The test to report. Single-key: every distinct context scored with key A. Dual-key: each context scored with its coin-selected key. |
| `key_a_all_positions` | Every distinct context scored with key A. |
| `key_b_all_positions`, `key_a_partition`, `key_b_partition` | Dual-key only (`None` otherwise): every context scored with key B, and each coin partition scored with its own key. |
| `prefix_limit_reached` | `True` when scoring stopped at `max_contexts` distinct contexts. |
| `tokens_examined` | Completion tokens covered by the test, including repeated and skipped ones. |
| `repeated_contexts` | Positions skipped because their exact context was already scored. |
| `skipped_initial_tokens` | Leading completion tokens skipped because no prompt IDs were given. |

Each statistic is a `WatermarkStatistics` with `num_contexts`, `score`, `z_score`, `p_value`, and `log_p_value`.

Prefer the server's token IDs to re-tokenized text, because re-encoding decoded text does not always reproduce the generated IDs. With `"return_token_ids": true`, non-streaming chat completions return `prompt_token_ids` and `response_token_ids` on each choice, and completions return `prompt_token_ids` and `token_ids`.

### In an SGLang environment

```python theme={null}
import requests

from sglang.srt.sampling.watermarking import WatermarkDetector

response = requests.post(
    "http://127.0.0.1:30000/v1/chat/completions",
    json={
        "model": "Qwen/Qwen3-8B",
        "messages": [{"role": "user", "content": "Explain photosynthesis."}],
        "temperature": 1.0,
        "top_p": 0.95,
        "return_token_ids": True,
        "watermark": {"enabled": True},
    },
).json()
choice = response["choices"][0]

detector = WatermarkDetector("0123456789abcdef", context_window=4)
result = detector.detect_tokens(
    choice["response_token_ids"],
    prompt_token_ids=choice["prompt_token_ids"],
)
print(result.combined.num_contexts, result.combined.z_score, result.combined.log_p_value)
```

To score text, pass the model's tokenizer and, when available, the prompt IDs:

```python theme={null}
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-8B")
result = detector.detect_text(
    choice["message"]["content"],
    tokenizer,
    prompt_token_ids=choice["prompt_token_ids"],
)
```

For a dual-key server, pass both keys and the server's mixing probability:

```python theme={null}
detector = WatermarkDetector(
    "0123456789abcdef",
    key_b="fedcba9876543210",
    mixing_probability=0.5,
    context_window=4,
)
result = detector.detect_tokens(completion_ids, prompt_token_ids=prompt_ids)
print(result.combined.p_value, result.key_a_partition.p_value, result.key_b_partition.p_value)
```

### Standalone, without SGLang or PyTorch

`detector.py` and `config.py` import only the Python standard library and `msgspec`, and the package's `__init__.py` imports only the detector. NumPy arrays are accepted as token-ID iterables, but NumPy is not required. To run detection in a separate application, copy the package directory and install `msgspec`:

```bash theme={null}
mkdir -p detector_app
cp -R python/sglang/srt/sampling/watermarking detector_app/watermarking
python -m pip install msgspec
```

The copy also contains `core.py`, the server runtime, which imports PyTorch; nothing on the detector import path loads it. Run the application with `detector_app` on the import path and import the detector from the copied package:

```python theme={null}
from watermarking.detector import WatermarkDetector

single = WatermarkDetector("0123456789abcdef", context_window=4)
result = single.detect_tokens(completion_ids, prompt_token_ids=prompt_ids)

dual = WatermarkDetector("0123456789abcdef", key_b="fedcba9876543210")
result = dual.detect_tokens(completion_ids, prompt_token_ids=prompt_ids)
```

Text detection needs only an object with `encode(text, *, add_special_tokens)` that returns a list of IDs. For example, with the lightweight `tokenizers` package:

```python theme={null}
from tokenizers import Tokenizer

from watermarking.detector import WatermarkDetector


class TokenizerAdapter:
    def __init__(self, name):
        self._tokenizer = Tokenizer.from_pretrained(name)

    def encode(self, text, *, add_special_tokens):
        return self._tokenizer.encode(text, add_special_tokens=add_special_tokens).ids


detector = WatermarkDetector("0123456789abcdef", context_window=4)
result = detector.detect_text(text, TokenizerAdapter("Qwen/Qwen3-8B"), prompt_token_ids=prompt_ids)
```

A detector written from scratch instead of reusing this package must implement the [hash contract](#hash-contract) and all three [detection rules](#detection): prompt alignment, exact-tuple deduplication, and the 4,096-context prefix.

## Compatibility

| Feature | Status | Behavior and validation |
| - | - | - |
| CUDA graphs | Verified | Decode graphs, eager fallback, and fused state updates use the same selector contract. |
| Overlap scheduling | Verified | Prompt initialization is carried with the delayed sampling batch. |
| Tensor parallelism | Verified | TP2 and TP4 ordinary decode, plus NEXTN with TP2, produce detectable output without rank divergence. |
| Mixed enabled and disabled requests | Verified | Keys, context windows, histories, and enablement are resolved per request. |
| JSON schema and regex grammars | Verified | Grammar masking runs before watermark selection; strict outputs remain valid and detectable. |
| Streaming chat completions | Verified | Streaming and non-streaming requests use the same watermark state and token selection. |
| Parallel sampling (`n > 1`) | Verified | Each generated choice owns independent request state and history. |
| Output log probabilities | Verified | Ordinary and speculative decode report the pre-watermark target distribution. Logits are cloned only for requests that ask for log probabilities. |
| Top-k, top-p, and min-p | Verified | The selector reproduces SGLang's combined truncation support before keyed selection. |
| Maximum-probability gate | Verified | Positions above `max_probability` use ordinary sampling and do not consume the repeated-context budget. |
| Logit bias and frequency, presence, and repetition penalties | Verified | These transforms run before watermark selection. |
| Chunked prefill | Verified | Each prefill chunk rebuilds the prompt tail from the request's complete logical fill IDs. |
| Radix-cache prompt reuse | Verified | Prompt-tail initialization uses logical request IDs rather than only newly computed tokens. |
| Decode retraction | Verified | Re-prefill restores the distinct-context history before generation resumes. |
| Request abort and request-slot reuse | Verified | A later prefill resets ring-buffer and context-history state for the reused slot. |
| Multimodal text generation | Verified | Qwen3-VL text tokens use the same watermark path after multimodal prefill. |
| DP attention | Verified | DP2/TP2 normal and NEXTN decoding were verified with Qwen3-8B on two H200 GPUs, including idle-rank batches. |
| NGRAM speculative decoding | Verified | Draft-node contexts are reconstructed before verification sampling. |
| EAGLE and EAGLE3 speculative decoding | Verified | Tree nodes use their own ancestor-derived contexts. |
| NEXTN speculative decoding | Verified | NEXTN resolves to the EAGLE verification path. |
| Pipeline-parallel speculative decoding | Rejected | Startup validation fails closed. |
| Other speculative algorithms | Rejected | Startup validation fails closed. |
| Speculative rejection sampling | Rejected | Startup validation fails closed. |
| Custom logit processors | Rejected | Startup validation fails closed because processor ordering is not defined. |
| Token-oracle sampling backend | Rejected | Startup validation fails closed because oracle-selected tokens bypass the watermark selector. |
| Beam search | Rejected | Request validation rejects watermarked beam-search requests. |
| Diffusion LLM | Rejected | Startup validation fails closed. |
| PD disaggregation | Rejected | Startup validation fails closed because watermark state is not transferred between workers. |
| Embedded Rust API server | Rejected | Startup validation fails closed. |
| Non-CUDA devices | Rejected | Startup validation fails closed. |

In speculative decoding, verification clones the target logits before forcing, so target-model log-probability computation still sees the pre-watermark distribution. Each draft node gets its own context, rebuilt from its ancestors in the verify tree. Repeated-context admission is conservative across the whole tree: once a context hash appears in an earlier row, later rows with the same hash are not forced, even on sibling branches. This keeps duplicate entries out of the history, but reconvergent wide trees may carry less signal.

## Limitations and performance

* Greedy requests carry no signal, and very low-entropy distributions leave few meaningful alternatives for the selector.
* A `max_probability` value below 1 keeps ordinary selection at low-entropy positions and can improve speculative draft acceptance, but the unforced positions dilute the detector statistic.
* Short completions provide few distinct contexts and therefore weak statistical power.
* Paraphrasing, rewriting, translation, and re-tokenization can weaken or remove the signal.
* Detection needs the matching key or keys, mixing probability, context window, and tokenizer, and is strongest with the exact prompt token IDs.
* A request watermarks at most `min(4,096, per-request token-pool capacity)` distinct contexts; later tokens use ordinary sampling.
* The hashes are MurmurHash3, a keyed but non-cryptographic mix. There is no indistinguishability or unforgeability guarantee, and anyone holding the key can both detect and forge the watermark.
* Speculative decoding can accept fewer draft tokens when a draft proposal does not match the watermarked target token. The impact depends on tree width and workload.
* Mixed batches choose the selector path from the batch-wide maximum top-k. A transient unlimited-top-k request can keep that batch on the full-sort path until its sampling metadata is rebuilt.
* Retraction reconstructs a conservative distinct-context history. Repeated retractions can reduce later signal and add scheduler-side replay work.

On one single-GPU node of each type with public `Qwen/Qwen3-8B`, TP1, TRT-LLM MHA, ordinary decoding, temperature 1.0, top-p 0.95, unlimited top-k, and 800-token sampled outputs, the fused selector measured the following mean output throughput over three repetitions per cell:

| Node | Concurrency | Watermark off | Watermark on | Change |
| - | -: | -: | -: | -: |
| GB300 | 1 | 270.25 token/s | 260.44 token/s | -3.63% |
| GB300 | 8, per stream | 224.88 token/s | 212.84 token/s | -5.35% |
| B300 | 1 | 258.84 token/s | 247.69 token/s | -4.31% |
| B300 | 8, per stream | 196.26 token/s | 182.04 token/s | -7.24% |

The GB300 concurrency-eight row was remeasured after the score-precision fix on a default-off server by comparing omitted requests with request-level opt-in. These measurements describe one model, GPU, sampling configuration, and output length. Benchmark your deployment before setting capacity targets. Finite-top-k selector latency at batch size one is also hardware-dependent: in one microbenchmark shape (`top_k=20`), the fast path was 58.8% slower than a full sort on H200 with bfloat16 logits and 6.4% faster on GB300 with float32 logits, and at batch size eight it was faster on both.

On a 2x GB300 node with public `Qwen/Qwen3.5-9B`, TP2, NEXTN 3/1/4, concurrency 8, 1,024-token random inputs, 512-token outputs, temperature 0.8, top-p 0.95, and top-k 20, the fused speculative path measured the following means over three repetitions:

| Watermark | Mean TPOT | Aggregate output throughput | Mean accept length |
| - | -: | -: | -: |
| Off | 2.246 ms | 3,188.9 token/s | 2.759 |
| On | 2.451 ms | 2,922.2 token/s | 2.783 |
| Change | +9.17% | -8.36% | +0.85% |

Each speculative row still requires keyed selection, but tree-context reconstruction, repeated-context admission, accepted-context recording, and ring-buffer append run as fused GPU operations. The reported cost remains workload- and acceptance-dependent.

## See also

The fused Triton selector is adapted from Meta's Apache-2.0 [TextSeal](https://github.com/facebookresearch/textseal) implementation and re-implemented on the hash contract above, so detection follows this page rather than TextSeal's own tooling.

The runtime and reference detector live in `python/sglang/srt/sampling/watermarking/`. Registered coverage lives in `test/registered/kernels/ops/sampling/test_watermark_*.py` and `test/registered/e2e/openai_server/test_watermark.py`.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.