Laguna-S-2.1 - SGLang Documentation

Deployment

Install SGLang

Laguna-S-2.1 uses the same laguna model architecture as Laguna-XS-2.1, which is fully supported in SGLang main. The model ships custom config code on the Hub, so --trust-remote-code is required (included in the launch commands).

Python (pip / uv)
Docker

Command

pip install -U uv
uv venv --python 3.12 && source .venv/bin/activate

git clone https://github.com/sgl-project/sglang.git
cd sglang
uv pip install -e python

Then run the Python output of the command panel below in that environment.

Command

docker pull lmsysorg/sglang:latest

For how to launch the image, see Install → Method 3: Using Docker. Substitute the inner sglang serve ... with what the command generator below produces.

Pick your hardware + quantization + strategy to generate the launch command. The two serving strategies cover the common operating points:

Low-latency — DFlash speculative decoding with a matched draft model. Pick for chat and interactive agents.
High-throughput — plain serving. Best for batch workloads, where speculation’s draft + rejection overhead costs more than it saves.

On the 8-GPU HGX platforms (H200 / B300) all quantizations run --tp 8. The 4-GPU GB300 node runs --tp 4 throughout. NVFP4 is Blackwell-only (B300 / GB300 only).

Playground

The Playground is where you experiment with SGLang features beyond the verified matrix. The Deploy panel above only emits combinations that have been signed off; the Playground lets you turn on additional knobs (TP degree, parsers) on top of whichever cell the Deploy panel is currently showing.

1. Model Introduction

Laguna-S-2.1 is an open-weight 118B-parameter hybrid sliding-window-attention MoE model (~8B active per token) from poolside, built for agentic coding and long-horizon software engineering. It sits between Laguna XS 2.1 (33B/3B active) and Laguna M.1 (222B/23B active) in the Laguna family. Key Features:

Sparse MoE: 48 layers, 256 routed experts, top-10 routing, plus 1 shared expert.
Hybrid attention: 36 sliding-window layers (window 512) interleaved with 12 full-attention layers (1:3 global-to-SWA ratio); 8 KV heads, head dim 128; per-head sigmoid output gating with per-layer-type rotary scales.
Long context: 1,048,576 tokens.
DFlash drafts: matched draft models ship per quantization for low-latency serving.
Hybrid reasoning: <think>…</think> toggled per request via chat_template_kwargs={"enable_thinking": …}.

Available quantizations:

Precision	Target model	Draft model
BF16	`poolside/Laguna-S-2.1`	`poolside/Laguna-S-2.1-DFlash`
FP8	`poolside/Laguna-S-2.1-FP8`	`poolside/Laguna-S-2.1-DFlash-FP8`
NVFP4	`poolside/Laguna-S-2.1-NVFP4`	`poolside/Laguna-S-2.1-DFlash-NVFP4`
INT4	`poolside/Laguna-S-2.1-INT4`	`poolside/Laguna-S-2.1-DFlash-INT4`

The drafts are small BF16 models, each calibrated against its quantized target — always pair a target with its matched draft (mixing precisions degrades accept-length). License: OpenMDW-1.1 Resources: Hugging Face · Technical report · API platform

2. Configuration Tips

Attention backend Leave --attention-backend unset for High-throughput cells — auto-select is correct (fa3 on Hopper, trtllm_mha on Blackwell). With DFlash active, auto-select instead falls back to flashinfer, which breaks this hybrid-SWA model at tp ≥ 4 on Blackwell (reproduced on Laguna-XS-2.1, greedy GSM8K 76% → 28%), so the Low-latency commands pin the target backend explicitly. Leave --speculative-draft-attention-backend unset. Other attention backend choices have not been fully validated on Laguna; keep the default. BF16 memory on H200 BF16 on H200 leaves less headroom for CUDA-graph capture and NCCL allocations than FP8/INT4. The High-throughput BF16 command carries --mem-fraction-static 0.80. FP8, INT4, and all B300/GB300 cells use the default heuristic. FP8 shared expert SGLANG_SHARED_EXPERT_TP1=1 is required for FP8 cells on all hardware — confirmed on both H200 (TP=8) and GB300 (TP=4). The FP8 checkpoint block-quantizes the shared expert (128×128 scales), which cannot TP-shard cleanly at either TP degree on S-2.1. This env var replicates the shared expert instead of sharding it. INT4 keeps the shared expert in BF16 (no flag needed); BF16 is unquantized. Note: this differs from Laguna-XS-2.1 where TP=4 does not require the flag — the constraint is architecture-specific. FP8 and NVFP4 DFlash drafts Fixed upstream on 2026-07-21: all DFlash draft configs now use a flat top-level rope_theta (the rope_parameters block was removed). If a server crashes at draft-model load with KeyError: 'rope_theta', you are serving a draft checkpoint cached before 2026-07-21 — re-download it (e.g. hf download poolside/Laguna-S-2.1-DFlash-FP8) to pick up the corrected config. DFlash memory Low-latency cells carry --mem-fraction-static 0.7 (sufficient even for BF16 on H200). Dense cells use the default heuristic (except BF16 on H200 — see above). BF16 reasoning length BF16 reasons approximately 2× longer than FP8/INT4 on AIME25 (median 34.8 k vs 16.9 k tokens), consistently truncating at max_tokens=64000. FP8/INT4 truncate at ≈ 2%. For a valid BF16 AIME25 score, serve with max_tokens ≥ 131072 (the model supports a 1 M context window). Chat template On transformers ≥ 5.10 the standalone chat_template.jinja auto-loads — no flag needed (the server logs Auto-detected template features: reasoning_parser=poolside_v1, ...). On older transformers (≤ ~5.8) pass --chat-template <model-dir>/chat_template.jinja explicitly. Thinking Off by default; opt in per request with extra_body={"chat_template_kwargs": {"enable_thinking": True}}. The template gates on enable_thinking — the generic thinking key is ignored. Served model id The server registers the model under whatever you pass to --model-path; a client’s model field must match it (poolside/Laguna-S-2.1, or the -FP8 / -NVFP4 / -INT4 id).

3. Advanced Usage

3.1 DFlash Speculative Decoding

DFlash is a block-wise speculative decoder: the draft proposes a block of tokens and the target verifies the whole block in one forward pass — output quality is the target’s by construction. The speedup lever is accept-length, the number of draft tokens surviving verification per target step. Best for interactive / few-stream serving. Under batch-saturated load prefer High-throughput: once the GPU is compute-bound, draft + rejected-token overhead costs aggregate throughput. The generated commands always pair the draft calibrated for the selected target precision.

3.2 Reasoning

Launch with --reasoning-parser poolside_v1 (baked into every generated command). Reasoning is opt-in via enable_thinking=True; the <think> trace lands in message.reasoning_content, separate from the final answer in message.content.

Reasoning Example (Python)

Example

from openai import OpenAI

client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")

response = client.chat.completions.create(
    model="poolside/Laguna-S-2.1",
    messages=[{"role": "user", "content": "What is 15% of 240? Explain briefly."}],
    max_tokens=4096,
    extra_body={"chat_template_kwargs": {"enable_thinking": True}},
)

message = response.choices[0].message
print("=============== Reasoning ===============")
print(message.reasoning_content)
print("=============== Answer ==================")
print(message.content)

Give generous max_tokens when thinking is enabled — hard problems regularly reason for thousands of tokens. Keep thinking off for short-form tasks.

3.3 Tool Calling

Launch with --tool-call-parser poolside_v1 (baked into every generated command). The parser converts Laguna’s <tool_call> output into the standard OpenAI tool_calls structure. Tool calling works with reasoning off (the default).

Tool Calling Example (Python)

Example

from openai import OpenAI

client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather for a location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {"type": "string", "description": "The city name"},
                    "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
                },
                "required": ["location"],
            },
        },
    }
]

response = client.chat.completions.create(
    model="poolside/Laguna-S-2.1",
    messages=[{"role": "user", "content": "What's the weather in Beijing?"}],
    tools=tools,
)

message = response.choices[0].message
if message.tool_calls:
    for call in message.tool_calls:
        print(f"Tool: {call.function.name}")
        print(f"Args: {call.function.arguments}")

​Deployment

​Playground

​1. Model Introduction

​2. Configuration Tips

​3. Advanced Usage

​3.1 DFlash Speculative Decoding

​3.2 Reasoning

​3.3 Tool Calling

Deployment

Playground

1. Model Introduction

2. Configuration Tips

3. Advanced Usage

3.1 DFlash Speculative Decoding

3.2 Reasoning

3.3 Tool Calling