sglang Python package, using the shared sglang generate and sglang serve commands.
It provides native SGLang pipelines, diffusers backend support, a server with OpenAI-compatible media APIs and SGLang action APIs, and an optimized kernel stack built on both precompiled sgl-kernel operators and JIT kernels for key inference paths.
Key Features
- Broad model support across media generation and robot action policies, including Wan, Hunyuan, Qwen-Image, FLUX, Z-Image, GLM-Image, Cosmos3, Pi0.5, and Praxis-1
- Fast inference with
sgl-kernel, JIT kernels, scheduler improvements, and caching acceleration - Multiple interfaces:
sglang generate,sglang serve, OpenAI-compatible media APIs, and action HTTP/WebSocket APIs - Multi-platform support for NVIDIA, AMD, Intel XPU, Ascend, Apple Silicon, and Moore Threads
Quick Start
Start with the recommended Docker setup for Linux GPU deployments. The installation guide also covers pip/uv, source installation, and other platforms. Run the following commands inside the container or your activated Python environment. Generate an image:Start Here
- Installation: install SGLang Diffusion and platform dependencies
- Supported Models: browse supported model families, tasks, and public checkpoints
- CLI: run one-off generation jobs or launch a persistent server
- OpenAI-Compatible API: send image and video requests to the HTTP server
- Action Generation API: generate robot action chunks and discover policy metadata
- VLA Cookbook: deploy action policies with model-specific observation schemas and control-loop settings
- ComfyUI plugin: use SGLang from ComfyUI in server mode or as a per-step DiT backend
- Performance Overview: choose speed, memory, parallelism, caching, and quality-tradeoff levers
- Caching Acceleration: use Cache-DiT, TeaCache, or Spectrum to reduce denoising cost
- Quantization: configure component checkpoint and causal KV-cache quantization
- Realtime and Causal Video Models: understand session state, causal caches, and realtime-only controls
- Contributing: contribution workflow, adding new models, and CI perf baselines
Additional Documentation
- Post-Processing: frame interpolation and upscaling
- Models with AR Stage: run hybrid diffusion pipelines like GLM-Image with a separate AR encoder server
- Models with Prompt Enhancement: run ERNIE-Image with either built-in PE or a separate PE server
- Deployment and Performance Modes: choose
--performance-mode, offload, FSDP, CFG parallelism, SP, and TP - Attention Backends: choose the best backend for your model and hardware
- Sequence Parallelism: configure SP, Ulysses, and ring-based splitting for long sequences
- Encoder Parallelism: fold, data-parallel, or replicate the text/image encoders across idle GPUs
- Inference Batching: batch compatible native diffusion requests during serving
- Progressive Resolution Generation: run early denoising steps at lower latent resolution for selected pipelines
- Environment Variables: platform, caching, storage, and debugging configuration
Developer Documentation
- Support New Models: implementation guide for new diffusion pipelines
- CI Performance Baselines: generate and update performance baselines used in CI
