Encoder Parallelism - SGLang Documentation

While the DiT denoises, the text and image encoders are idle — and while they encode, the whole DiT replica is idle. --encoder-parallel decides how to use those otherwise-unused GPUs for the encoding stage.

--encoder-parallel {auto,fold,dp,replicate}

Mode	What it does	Use when
`auto`	Picks `fold`, `dp`, or `replicate` per encoder from its width and the request’s batch width	Default for `generate`; you want the decision made per encoder
`fold`	TP-shards the encoder weights across the idle DiT replica	One wide encoder dominates a single-request encode
`dp`	Each rank encodes its slice of the prompt batch, then the outputs are all-gathered	Default for `serve`; needs `--batching-max-size > 1` to engage
`replicate`	Every rank encodes the whole batch redundantly	You want the encoding stage to match single-GPU numerics exactly

The two accelerated modes are mutually exclusive per encoder: folding shards the weights for the lifetime of the loaded model, so a folded encoder cannot also be data-parallel.

Which Mode Wins

Measured on H100 across T5 (hidden 4096), Qwen3 (2560), and CLIP-L (768) at batch 1–8 and replica sizes 2 and 4:

Folding pays when the encoder is wide enough that sharding its GEMMs beats the per-layer all-reduce it adds. T5 gains; Qwen3 (+35%) and CLIP-L (+50%) get slower, so folding is gated at hidden ≥ 4096. Its benefit also saturates as the replica grows, since each rank’s slice keeps shrinking.
Data-parallel pays only when the encode is compute-bound, which needs a wide encoder (hidden ≥ 1024 — CLIP-L is slower at every batch and replica measured) and more than one prompt in a single encode call.
Replication is the right answer whenever neither condition holds, which is most single-request latency work.

auto encodes exactly these rules, so prefer it unless you are pinning a configuration you measured yourself.

Numerics

fold and replicate are bitwise-identical to single-GPU encoding: folding shards a GEMM and reduces it, which is the same arithmetic the unsharded kernel performs. dp is not bitwise-identical. Each rank runs the full unsharded encoder on a smaller batch, so the GEMM tiling and reduction order differ from the batched reference — the same floating-point reordering class as choosing a different attention backend or parallelism strategy, not a precision loss. The gathered result is mathematically equivalent, and per-request results stay deterministic for a fixed batch shape, but embeddings will not match a replicate run bit-for-bit, and long video sampling can amplify the difference into visible frame differences. Use replicate (or fold) when you need bit-exact reproducibility against a single-GPU reference, e.g. when refreshing consistency baselines.

Recommended Commands

Throughput serving. serve already defaults to dp, but a single encode call must carry more than one prompt for it to engage, so raise the batching ceiling too — an encoder flag deliberately does not change DiT batching for you:

sglang serve \
  --model-path Qwen/Qwen-Image-2512 \
  --model-type diffusion \
  --num-gpus 2 \
  --encoder-parallel dp \
  --batching-max-size 2

Single-request latency with one wide text encoder:

sglang serve \
  --model-path Wan-AI/Wan2.2-TI2V-5B-Diffusers \
  --model-type diffusion \
  --num-gpus 4 \
  --ulysses-degree 4 \
  --encoder-parallel fold

Bit-exact reproducibility against a single-GPU reference:

sglang serve \
  --model-path Qwen/Qwen-Image-2512 \
  --model-type diffusion \
  --num-gpus 2 \
  --encoder-parallel replicate

Interaction With Other Flags

Tensor / data parallel: dp requires a replicated encoder, so it is skipped when --tp-size > 1 or --dp-size > 1.
Dynamic batching: dp only pays with a wide batch, so selecting it raises the default batching ceiling. See Inference Batching.
Sequence parallelism: independent — SP splits the DiT’s latent sequence, encoder parallelism splits the encoding stage. See Sequence Parallelism.

​Which Mode Wins

​Numerics

​Recommended Commands

​Interaction With Other Flags

Which Mode Wins

Numerics

Recommended Commands

Interaction With Other Flags