> ## Documentation Index > Fetch the complete documentation index at: https://docs.sglang.io/llms.txt > Use this file to discover all available pages before exploring further. # Attention Backends > Select and configure attention backends for SGLang diffusion pipelines. This document describes the attention backends available in sglang diffusion (`sglang.multimodal_gen`) and how to select them. ## Overview Attention backends are defined by `AttentionBackendEnum` (`sglang.multimodal_gen.runtime.platforms.interface.AttentionBackendEnum`) and selected via the CLI flag `--attention-backend`. Backend selection is performed by the shared attention layers (e.g. `LocalAttention` / `USPAttention` / `UlyssesAttention` in `sglang.multimodal_gen.runtime.layers.attention.layer`) and therefore applies to any model component using these layers (e.g. diffusion transformer / DiT and encoders). When using the diffusers backend, `--attention-backend` is passed through to diffusers' `set_attention_backend` (e.g., `flash`, `_flash_3_hub`, `sage`, `xformers`, `native`). * **CUDA**: prefers FlashAttention (FA3/FA4) when supported; otherwise falls back to PyTorch SDPA. * **ROCm**: uses FlashAttention when available; otherwise falls back to PyTorch SDPA. * **Intel XPU**: uses XPU Flash Attention backend (fp16/bf16, head sizes 64/96/128/192/256); otherwise falls back to PyTorch SDPA. * **MUSA**: uses FlashAttention when available; also supports Sage Attention when installed; otherwise falls back to PyTorch SDPA. * **MPS**: always uses PyTorch SDPA. * **NPU**: for ring attention uses FA otherwise uses PyTorch SDPA. ## Backend options For SGLang-native pipelines, the CLI accepts the lowercase names of `AttentionBackendEnum`. The table below lists the backends implemented by the built-in platforms. `fa3`/`fa4` are accepted as aliases for `fa`.
CLI value Enum value Notes
`fa` / `fa3` / `fa4` `FA` FlashAttention. fa3/fa4 are normalized to fa during argument parsing (ServerArgs.**post\_init**).
`torch_sdpa` `TORCH_SDPA` PyTorch scaled\_dot\_product\_attention.
`sliding_tile_attn` `SLIDING_TILE_ATTN` Sliding Tile Attention (STA). Requires st\_attn. Configure via --attention-backend-config.
`sage_attn` `SAGE_ATTN` Requires sageattention. Upstream SageAttention CUDA extensions target SM80/SM86/SM89/SM90/SM120 (compute capability 8.0/8.6/8.9/9.0/12.0); see upstream setup.py: [https://github.com/thu-ml/SageAttention/blob/main/setup.py](https://github.com/thu-ml/SageAttention/blob/main/setup.py).
`sage_attn_3` `SAGE_ATTN_3` Requires SageAttention3 installed per upstream instructions.
`video_sparse_attn` `VIDEO_SPARSE_ATTN` Requires vsa. Configure sparsity via --attention-backend-config.
`vmoba_attn` `VMOBA_ATTN` Requires kernel.attn.vmoba\_attn.vmoba. Configure via --attention-backend-config.
`aiter` `AITER` Requires aiter.
aiter\_sage AITER\_SAGE Requires aiter.
sla\_attn SLA\_ATTN Sparse Linear Attention. Requires SpargeAttn. Install with pip install git+[https://github.com/thu-ml/SpargeAttn.git](https://github.com/thu-ml/SpargeAttn.git) --no-build-isolation.
sage\_sla\_attn SAGE\_SLA\_ATTN SageAttention + Sparse Linear Attention. Requires SpargeAttn (same install as SLA).
`sparse_video_gen_2_attn` `SPARSE_VIDEO_GEN_2_ATTN` Requires svg. See installation instructions at [https://github.com/svg-project/Sparse-VideoGen](https://github.com/svg-project/Sparse-VideoGen).
## Selection priority The selection order in `runtime/layers/attention/selector.py` is: 1. `global_force_attn_backend(...)` / `global_force_attn_backend_context_manager(...)` 2. Component override from `--component-attention-backends` while that component is being constructed 3. CLI `--attention-backend` (`ServerArgs.attention_backend`) 4. Auto selection (platform capability, dtype, and installed packages) ## Configuration Some backends require additional configuration. You can pass these parameters via `--attention-backend-config`. This argument accepts: * A path to a JSON or YAML configuration file. * A JSON string (e.g., `'{"sparsity": 0.5}'`). * Key-value pairs (e.g., `"sparsity=0.5,enable_x=true"`). ### Supported Configuration Parameters **Sliding Tile Attention (`sliding_tile_attn`)**
Parameter Type Description Default
`mask_strategy_file_path` `str` **Required.** Path to the mask strategy JSON file. -
`sta_mode` `str` Mode of STA. `STA_inference`
`skip_time_steps` `int` Number of steps to use full attention before switching to sparse attention. `15`
**Video Sparse Attention (`video_sparse_attn`)**
Parameter Type Description Default
`sparsity` `float` Validation sparsity (0.0 - 1.0). `0.0`
**V-MoBA (`vmoba_attn`)**
Parameter Type Description Default
`temporal_chunk_size` `int` Chunk size for temporal dimension. -
`temporal_topk` `int` Top-K tokens to select in temporal dimension. -
`spatial_chunk_size` `list[int]` Chunk size for spatial dimension (H, W). -
`spatial_topk` `int` Top-K tokens to select in spatial dimension. -
`st_chunk_size` `list[int]` Chunk size for spatiotemporal dimension (T, H, W). -
`st_topk` `int` Top-K tokens to select in spatiotemporal dimension. -
`moba_select_mode` `str` Selection mode (e.g., `threshold`). `threshold`
`moba_threshold` `float` Threshold value for selection. `0.25`
`moba_threshold_type` `str` Type of thresholding (e.g., `query_head`). `query_head`
`first_full_step` `int` Number of initial steps to use full attention. `12`
`first_full_layer` `int` Number of initial layers to use full attention. `0`
`temporal_layer` `int` Number of temporal layers. `1`
`spatial_layer` `int` Number of spatial layers. `1`
`st_layer` `int` Number of spatiotemporal layers. `1`
## Platform support matrix
Backend CUDA ROCm XPU MUSA MPS NPU Notes
`fa` Yes Yes ✅ ✅ ❌ ✅ CUDA requires SM80+ and fp16/bf16. XPU uses its own flash attention backend. FlashAttention is only used when the required runtime is installed; otherwise it falls back to torch\_sdpa. No extra installations are required for NPU
`torch_sdpa` Yes Yes Yes Yes ✅ ✅ Most compatible option across platforms.
`sliding_tile_attn` Yes No No No ❌ ❌ CUDA-only. Requires st\_attn. Configure via --attention-backend-config.
`sage_attn` Yes No No Yes ❌ ❌ Optional dependency on CUDA and MUSA. Falls back to FlashAttention if sageattention is not installed.
`sage_attn_3` Yes No No No ❌ ❌ CUDA-only (optional dependency).
`video_sparse_attn` Yes No No No ❌ ❌ CUDA-only. Requires vsa. Configure sparsity via --attention-backend-config.
sla\_attn Yes No No No ❌ ❌ CUDA-only. Requires SpargeAttn.
sage\_sla\_attn Yes No No No ❌ ❌ CUDA-only. Requires SpargeAttn.
vmoba\_attn Yes No No No ❌ ❌ CUDA-only. Requires kernel.attn.vmoba\_attn.vmoba. Configure via --attention-backend-config.
aiter ❌ ✅ No No ❌ ❌ Requires aiter.
aiter\_sage ❌ ✅ No No ❌ ❌ Requires aiter.
`sparse_video_gen_2_attn` Yes No No No ❌ ❌ CUDA-only. Requires svg.
## Usage ### Select a backend via CLI ```bash theme={null} sglang generate \ --model-path \ --prompt "..." \ --attention-backend fa ``` ```bash theme={null} sglang generate \ --model-path \ --prompt "..." \ --attention-backend torch_sdpa ``` ### Override one component Use component overrides when a specific module needs different attention semantics from the main transformer: ```bash theme={null} sglang generate \ --model-path \ --prompt "..." \ --attention-backend fa \ --component-attention-backends text_encoder=torch_sdpa ``` Component keys match pipeline module names from `model_index.json`, such as `text_encoder`, `text_encoder_2`, `transformer`, `transformer_2`, or `connectors`. ### Using Sliding Tile Attention (STA) ```bash theme={null} # Pass the mask strategy file path via config sglang generate \ --model-path \ --prompt "..." \ --attention-backend sliding_tile_attn \ --attention-backend-config "mask_strategy_file_path=/abs/path/to/mask_strategy.json" ``` ### Notes for ROCm / MPS * ROCm: use `--attention-backend torch_sdpa` or `fa` depending on what is available in your environment. * MPS: the platform implementation always uses `torch_sdpa`.