Deployment
Install SGLang
Install SGLang
For all methods and hardware platforms, see the official SGLang installation guide. The two paths below match the Python / Docker toggle in the command panel.For how to launch the image, see Install → Method 3: Using Docker, substituting the inner
- Docker
- Python (pip / uv)
Command
sglang serve ... with what the command generator below produces.Playground
The Playground is where you experiment with SGLang features beyond the verified matrix. The Deploy panel above only emits combinations the SGLang team has signed off on; the Playground lets you turn on prefill/decode disaggregation and hierarchical KV caching (HiCache) on top of whichever cell the Deploy panel is currently showing.1. Model Introduction
Hy4-flash is Tencent’s ~220B-parameter (~12B active) Mixture-of-Experts model with 512 routed experts, interleaving DeepSeek-style sparse MLA attention with sliding-window attention and adding Engram n-gram memory tables. Available models:- tencent/Hy4-flash — BF16
- tencent/Hy4-flash-FP8 — FP8 (e4m3, 128×128 block scales)
2. Advanced Usage
2.1 HiCache (Hierarchical KV Caching)
Hy4-flash supports SGLang’s hierarchical KV cache, including the host-memory (L2) tier and storage (L3) backends. Toggle it in the HiCache card of the Playground above to add the flags to your launch command.2.2 PD Disaggregation
Prefill and decode can run on separate SGLang instances. Pick a role and transfer backend in the PD Disagg card of the Playground above; it also generates thesgl-router command that fronts both roles.