Skip to main content
Use the CLI for one-off generation with sglang generate or to start a persistent HTTP server with sglang serve.

​
Overlay repos for non-diffusers models

If --model-path points to a supported non-diffusers source repo, SGLang can resolve it through a self-hosted overlay repo. SGLang first checks a built-in overlay registry. Concrete built-in mappings can be added over time without changing the CLI surface. Override example:
Command
export SGLANG_DIFFUSION_MODEL_OVERLAY_REGISTRY='{
  "Wan-AI/Wan2.2-S2V-14B": {
    "overlay_repo_id": "your-org/Wan2.2-S2V-14B-overlay",
    "overlay_revision": "main"
  }
}'

sglang generate \
  --model-path Wan-AI/Wan2.2-S2V-14B \
  --config configs/wan_s2v.yaml
The overlay repo should be a complete diffusers-style/componentized repo You can also pass the overlay repo itself as --model-path if it contains _overlay/overlay_manifest.json. Notes:
  1. SGLANG_DIFFUSION_MODEL_OVERLAY_REGISTRY is only an optional override for development and debugging. It accepts either a JSON object or a path to a JSON file, and can extend or replace built-in entries for the current process.
  2. On the first load, SGLang will:
    • download overlay metadata from the overlay repo
    • download the required files from the original source repo
    • materialize a local standard component repo under ~/.cache/sgl_diffusion/materialized_models/
  3. Later loads reuse the materialized local repo. The materialized repo is what the runtime loads as a normal componentized model directory.

​
Quick Start

​
Generate

Command
sglang generate \
  --model-path Qwen/Qwen-Image \
  --prompt "A beautiful sunset over the mountains" \
  --save-output

​
Serve

Command
sglang serve \
  --model-path Wan-AI/Wan2.1-T2V-1.3B-Diffusers \
  --num-gpus 4 \
  --ulysses-degree 2 \
  --ring-degree 2 \
  --port 30010
For request and response examples, see OpenAI-Compatible API.
Use sglang generate --help and sglang serve --help for the full argument list. The CLI help output is the source of truth for exhaustive flags.

​
Common Options

​
Model and runtime

  • --model-path {MODEL}: model path or Hugging Face model ID
  • --lora-path {PATH} and --lora-nickname {NAME}: load a LoRA adapter
  • --num-gpus {N}: number of GPUs to use
  • --tp-size {N}: tensor parallelism size, mainly for encoders
  • --sp-degree {N}: sequence parallelism size
  • --ulysses-degree {N} and --ring-degree {N}: USP parallelism controls
  • --attention-backend {BACKEND}: attention backend for native SGLang pipelines
  • --attention-backend-config {CONFIG}: attention backend configuration

​
Sampling and output

  • --prompt {PROMPT} and --negative-prompt {PROMPT}
  • --image-path {PATH} [{PATH} ...]: input image(s) for image-to-video or image-to-image generation
  • --num-inference-steps {STEPS} and --seed {SEED}
  • --height {HEIGHT}, --width {WIDTH}, --num-frames {N}, --fps {FPS}
  • --output-path {PATH}, --output-file-name {NAME}, --save-output, --return-frames
For frame interpolation and upscaling, see Post-Processing.

​
Quantized transformers

For quantized transformer checkpoints, prefer:
  • --model-path for the base pipeline
  • --transformer-path for a quantized transformers transformer component folder
  • --transformer-weights-path for a quantized safetensors file, directory, or repo
See Quantization for supported quantization families and examples.

​
Configuration Files

Use --config to load JSON or YAML configuration. Command-line flags override values from the config file.
Command
sglang generate --config config.yaml
Example:
Config
model_path: FastVideo/FastHunyuan-diffusers
prompt: A beautiful woman in a red dress walking down a street
output_path: outputs/
num_gpus: 2
sp_size: 2
tp_size: 1
num_frames: 45
height: 720
width: 1280
num_inference_steps: 6
seed: 1024
fps: 24
precision: bf16
vae_precision: fp16
vae_tiling: true
vae_sp: true
enable_torch_compile: false

​
Generate

sglang generate runs a single generation job and exits when the job finishes.
Command
sglang generate \
  --model-path Wan-AI/Wan2.2-T2V-A14B-Diffusers \
  --text-encoder-cpu-offload \
  --pin-cpu-memory \
  --num-gpus 4 \
  --ulysses-degree 2 \
  --ring-degree 2 \
  --prompt "A curious raccoon" \
  --save-output \
  --output-path outputs \
  --output-file-name "a-curious-raccoon.mp4"
HTTP server-only arguments are ignored by sglang generate.
For diffusers pipelines, Cache-DiT can be enabled with SGLANG_CACHE_DIT_ENABLED=true or --cache-dit-config. See Cache-DiT.

​
Serve

sglang serve starts the HTTP server and keeps the model loaded for repeated requests.
Command
sglang serve \
  --model-path Wan-AI/Wan2.1-T2V-1.3B-Diffusers \
  --text-encoder-cpu-offload \
  --pin-cpu-memory \
  --num-gpus 4 \
  --ulysses-degree 2 \
  --ring-degree 2 \
  --port 30010

​
Cloud Storage

SGLang Diffusion can upload generated images and videos to S3-compatible object storage after generation.
Command
export SGLANG_CLOUD_STORAGE_TYPE=s3
export SGLANG_S3_BUCKET_NAME=my-bucket
export SGLANG_S3_ACCESS_KEY_ID=your-access-key
export SGLANG_S3_SECRET_ACCESS_KEY=your-secret-key
export SGLANG_S3_ENDPOINT_URL=https://minio.example.com
See Environment Variables for the full set of storage options.

​
Component Path Overrides

Override individual pipeline components such as vae, transformer, or text_encoder with --<component>-path.
Command
sglang serve \
  --model-path black-forest-labs/FLUX.2-dev \
  --vae-path fal/FLUX.2-Tiny-AutoEncoder
The component key must match the key in the model’s model_index.json, and the path must be either a Hugging Face repo ID or a complete component directory.

​
Diffusers Backend

Use --backend diffusers to force vanilla diffusers pipelines when no native SGLang implementation exists or when a model requires a custom pipeline class.

​
Key Options

ArgumentValuesDescription
—backendauto, sglang, diffusersChoose native SGLang, force native, or force diffusers
—diffusers-attention-backendflash, _flash_3_hub, sage, xformers, nativeAttention backend for diffusers pipelines
—trust-remote-codeflagRequired for models with custom pipeline classes
—vae-tiling and —vae-slicingflagLower memory usage for VAE decode
—dit-precision and —vae-precisionfp16, bf16, fp32Precision controls
—enable-torch-compileflagEnable torch.compile
—cache-dit-config{PATH}Cache-DiT config for diffusers pipelines

​
Example

sglang generate \
  --model-path AIDC-AI/Ovis-Image-7B \
  --backend diffusers \
  --trust-remote-code \
  --diffusers-attention-backend flash \
  --prompt "A serene Japanese garden with cherry blossoms" \
  --height 1024 \
  --width 1024 \
  --num-inference-steps 30 \
  --save-output \
  --output-path outputs \
  --output-file-name ovis_garden.png
For pipeline-specific arguments not exposed in the CLI, pass diffusers_kwargs in a config file.