Installation
Install From Source
Currently SGLang XPU only supports installation from source. Please refer to “Getting Started on Intel GPU” to install XPU dependency.Command
[tool.uv.*] sections of pyproject.toml. To install with pip instead of uv, pass the package indexes explicitly:
Command
Install Using Docker
It is recommended to use Docker for setting up the SGLang environment.Pull from Docker Hub
Pull the prebuilt docker image of SGLang package releases fromlmsysorg/sglang repository.
The XPU image tags end with xpu suffix.
The image pulling command is like:
Command
Build from Dockerfile
The SGLang XPU Dockerfile is provided to facilitate the installation. Replace<secret> below with your HuggingFace access token.
Command
Launch of the Serving Engine
Example command to launch SGLang serving:MXFP4 MoE Quantization
Native MXFP4 MoE checkpoints (OCP microscaling FP4: packede2m1 weights plus
per-32-element ue8m0 block scales) run on Intel GPUs through the
sgl-kernel-xpu W4A16 grouped GEMM. The expert weights stay in the checkpoint’s
packed layout end to end — there is no dequantization to BF16 — so GPT-OSS-20B
loads in roughly 13 GB rather than the ~42 GB a BF16 upcast would need, which is
what lets it fit on a single 24 GB card.
The mxfp4 method is registered automatically on --device xpu, which already
requires sgl-kernel-xpu — no extra flag is needed:
Requirements and limitations:
Benchmarking with Requests
You can benchmark the performance via thebench_serving script.
Run the command in another terminal.
curl) or via your own script.
XPU Graph [Experimental]
SGLang enables XPU graph capture to reduce per-step kernel-launch overhead.Enable Decode Graph
Decode graph capture is opt-in on XPU. Enable it explicitly:Enable Prefill Graph
Prefill graph capture is opt-in on XPU and must be enabled explicitly. The available backends arefull and breakable.
full
Captures the full prefill forward as onetorch.xpu.XPUGraph per
token-length bucket. For full prefill, the graph also fixes the
number of request slots baked into each capture. If you do not set that
explicitly, SGLang derives it from chunked_prefill_size // 512 and
clamps it by the phase-level max_bs.
full prefill, pass it through
--cuda-graph-config, for example:
breakable
Captures the transformer stack as segmentedXPUGraphs with eager break points
at attention / MoE boundaries, without torch.compile:
--cuda-graph-config JSON argument:
Enable torch.compile for Decode
--enable-torch-compile adds a torch.compile pass on top of the decode
XPU graph: the model forward is compiled first, and the compiled forward is
then captured as an XPUGraph. This can reduce per-kernel overhead further
but increases startup time.
Disable XPU Graph
Both phases are disabled by default. To explicitly disable them anyway:Customize Capture Buckets
By default, prefill capture sizes are derived from--chunked-prefill-size.
To specify explicit token-length buckets:
Server Args
* Prefill graph is auto-disabled on XPU unless you lock the backend explicitly
via
--cuda-graph-backend-prefill or --cuda-graph-config.
Limitations
Decode Context Parallel (DCP) on Intel XPU [Experimental]
DCP shards the KV cache across the ranks of a TP group by token position. It grows the servable context only whentp_size exceeds the model’s KV heads, which TP would
otherwise replicate, and decode gets slower, so use it only when the context you need
does not fit.
intel_xpu attention backend and ranks that share their DCP group’s
KV heads: Qwen3.5 models, or models with at most tp_size / dcp_size KV heads.
Unsupported features are refused at startup. If startup fails with
UR_RESULT_ERROR_OUT_OF_RESOURCES, lower --mem-fraction-static.
With P/D disaggregation, DCP on XPU skips the DCP pack fast path: the pack
kernel runs only on CUDA, so KV is transferred per token instead.
Memory Saver (release/resume memory occupation) on Intel XPU [Experimental]
SGLang can temporarily release most of the GPU memory it holds — model weights and/or KV cache — and reclaim it later without restarting the process. This is the samerelease_memory_occupation / resume_memory_occupation feature
available on CUDA, used for RL rollout/training hand-off and for freeing the
device between inference bursts.
This is backed by the torch_memory_saver
package — the same package used on CUDA — which gained an Intel XPU backend
built natively on Level Zero (keeping virtual addresses fixed while
releasing/re-committing physical pages via zeVirtualMemUnmap /
zeVirtualMemMap).
Install torch_memory_saver. Unlike CUDA (prebuilt wheel), the XPU backend
is built from source against your local oneAPI + torch+xpu runtime (the .so
links libsycl.so.<N>, which must match the installed intel-sycl-rt).
TMS_PLATFORM=xpu forces the XPU backend, and --no-build-isolation lets the
build import your installed torch so it can match the libsycl major to it:
The published wheels are CUDA-only, so install from git and let it build. The ref
below is the v0.0.10b2 release, pinned so the build is reproducible.
--enable-memory-saver (the XPU backend is
selected automatically); optionally add --enable-weights-cpu-backup to keep
weights in host RAM across a release. The release/resume endpoints are admin
endpoints, so the server also needs SGLANG_ADMIN_API_KEY (or --admin-api-key)
and the calls must carry that key (exported as SGLANG_ADMIN_API_KEY below):
engine.release_memory_occupation(tags=...) /
engine.resume_memory_occupation(tags=...)) and the weights / kv_cache tags
behave the same as on CUDA. Pauseable CUDA-graph capture is not used on XPU, so
the cuda_graph tag is a no-op there.
Verifying memory was freed: neithertorch.xpu.memory_allocated()nortorch.xpu.mem_get_info()drops when physical pages are released — the first is allocator accounting, and the second stays flat because torch keeps the freed block cached. Query actual device memory via sysman (ZES_ENABLE_SYSMAN=1) instead.
Prefill-Decode (P/D) Disaggregation on Intel XPU [Experimental]
SGLang supports prefill-decode disaggregation on Intel XPU with the Mooncake and NIXL KV-transfer backends. Both are covered by the XPU nightly P/D test. Tested configurations:Mooncake backend
Mooncake supports Intel XPU through its TENT engine. The PyPImooncake-transfer-engine wheel does not include it, so either use the XPU Docker image, which builds Mooncake from a pinned upstream commit (see docker/xpu.Dockerfile), or build Mooncake from source with icpx and -DUSE_XPU=ON -DUSE_TENT=ON. The image builds Mooncake without its store (-DWITH_STORE=OFF), so the Mooncake store HiCache L3 backend (--hicache-storage-backend mooncake) is not available on XPU.
Prerequisites: Mooncake with XPU support (above) and the sgl-router binary, which the XPU Docker image installs at /usr/local/bin/sgl-router (outside the image, install it with pip install sgl-router or see installing sgl-router).
Set MC_USE_TENT=1 on both the prefill and decode servers, and leave MOONCAKE_PROTOCOL at its default. On one node, the TENT engine then copies KV card to card through Level Zero IPC, so no RDMA NIC is needed.
Both servers must see both cards: give them the same ZE_AFFINITY_MASK and pick each server’s card with --base-gpu-id. If a server cannot see the peer’s card, TENT logs ... is on a device not visible here ... using another route and does not copy card to card.
Inside a container, run with --cap-add SYS_PTRACE, or --privileged as in the Docker example above. TENT imports the peer’s buffer through pidfd_getfd(), which Docker blocks by default; without it TENT logs cannot map peer buffer ... Operation not permitted ... using another route, and requests still succeed but KV no longer goes card to card.
Start the prefill server (GPU 0):
Qwen/Qwen2.5-1.5B-Instruct as the model.
SGLANG_MOONCAKE_CUSTOM_MEM_POOL (NVLINK, BAREX, INTRA_NODE_NVLINK) selects Mooncake’s CUDA allocators and is CUDA-only; do not set it on XPU.
NIXL backend
Prerequisites: NIXL on a UCX built with the Level Zero transport (--with-ze), and the sgl-router binary (see the Mooncake prerequisites above). The PyPI nixl wheel bundles a UCX without it, so either use the XPU Docker image, which builds UCX, Abseil, and NIXL from pinned upstream commits (see docker/xpu.Dockerfile), or build them from source the same way.
Inside a container, run with --cap-add SYS_PTRACE, or --privileged as in the Docker example above: UCX’s ze_ipc transport maps the peer’s device memory through pidfd_getfd(), which Docker blocks by default, and the first transfer fails with Operation not permitted.
Start the prefill server (GPU 0):
Note: If UCX’s shared-memory transport fails to attach a peer’s segment (it opens/proc/<pid>/fd/<fd>, which needs ptrace rights), setUCX_POSIX_USE_PROC_LINK=nto have it open the segment by its/dev/shmpath instead.
