Session-Aware Radix Cache

Session-aware radix caching improves cache hits for long-lived, multi-turn workloads under memory pressure. It registers reusable KV to a session and evicts unreferenced KV before KV still referenced by an active session. Session references are soft protection, not memory pins. Referenced KV can still be evicted when reclaiming unreferenced KV is insufficient.

Enable the cache

This feature is implemented only by UnifiedRadixCache.

Command

SGLANG_ENABLE_UNIFIED_RADIX_TREE=1 python3 -m sglang.launch_server \
  --model-path MODEL_PATH \
  --enable-session-radix-cache

Pass and close the session

Your application must pass the same top-level session_id on every request in a session. The ID labels cache references only; it does not append or reconstruct conversation context, so each request must still contain the intended prompt.

Command

curl http://localhost:30000/generate \
  -H "Content-Type: application/json" \
  -d '{
    "text": "FULL_PROMPT_FOR_THIS_TURN",
    "sampling_params": {"max_new_tokens": 128},
    "session_id": "agent-42"
  }'

When a request finishes, SGLang automatically registers its reusable cache leaves under the session_id. This reference-only workflow does not require an /open_session call. Call /close_session when the application session ends, including error and cancellation paths:

Command

curl -X POST http://localhost:30000/close_session \
  -H "Content-Type: application/json" \
  -d '{"session_id": "agent-42"}'

Closing removes the session’s references but does not immediately free its KV. The KV remains reusable and returns to the normal eviction order.

Eviction behavior

The cache tracks references independently for each UnifiedRadixCache component. Device and host eviction use the same session preference.

Component	Referenced data	Eviction order
Full attention	The reusable prefix path from the registered leaf to the root	Unreferenced nodes first, then referenced nodes with fewer session references, then the configured policy such as LRU
Sliding-window attention (SWA)	The reusable tail covering the sliding window plus page-alignment allowance	Two LRU passes: unreferenced nodes first, then referenced nodes if more space is required
Mamba	The reusable state on the registered leaf	Two LRU passes: unreferenced nodes first, then referenced nodes if more space is required

UnifiedRadixCache still applies component cascade rules. Evicting an internal Full node also evicts its SWA and Mamba data; evicting SWA also evicts Mamba data; evicting Mamba affects only Mamba. Evicting a leaf removes all component data on that leaf.

Server Arguments

Hyperparameter Tuning

⌘I

Basic Usage

Advanced Features

Supported Models

Developer Guide

References

Enable the cache

Pass and close the session

Eviction behavior

​Enable the cache

​Pass and close the session

​Eviction behavior

Enable the cache

Pass and close the session

Eviction behavior