Intern-S2-Mobius - SGLang Documentation

Deployment

Install SGLang

For all methods and hardware platforms, see the official SGLang installation guide. The two paths below match the Python / Docker toggle in the command panel.

Python (pip / uv)
Docker

Command

pip install --upgrade pip
pip install uv
uv pip install sglang

Then run the Python output of the command panel below in that environment.

Command

docker pull lmsysorg/sglang:dev

Intern-S2-Mobius was upstreamed in PR #33691 (merged 2026-08-08) — it lives on lmsysorg/sglang:dev (nightly) until the next release cut. For how to launch the image, see Install → Method 3: Using Docker. Substitute the inner sglang serve ... with what the command generator below produces.

Pick your hardware + recipe to generate the launch command. The two serving strategies cover the main operating points:

Low-Latency — MTP (NEXTN) speculative decoding on. Fastest reply for a single user.
High-Throughput — spec off, more tokens per second when many users share the server.

Speed numbers are measured with --random-range-ratio 1.0, --flush-cache, on 2×H200 TP=2 against main @ e0828ee3 + PR #33691 head (since merged 2026-08-08 — lmsysorg/sglang:dev is the live equivalent). GSM8K is the full 1319-example test split; GPQA is Diamond 198 problems × 8 repeats (pass@1 avg-of-8). Both ran with no server-side sampling override, so the checkpoint’s generation_config.json defaults applied (temperature 1.0, top_p 0.95, top_k 20). The B200 recipes are inferred from the H200 ones and unverified — same flags, just a TP=2 or TP=1 Blackwell equivalent.

Playground

The Playground is where you experiment with SGLang features beyond the verified matrix. The Deploy panel above only emits combinations the SGLang team has signed off on; the Playground lets you turn on additional knobs on top of whichever cell the Deploy panel is currently showing.

1. Model Introduction

Intern-S2-Mobius is InternLM’s 35B scientific multimodal foundation model built on the Mobius-v0 architecture (continually pre-trained from Qwen3.5-35B, then SFT and RL post-trained). Instead of binding knowledge storage and reasoning computation layer by layer as conventional Transformers do, Mobius organizes knowledge into a globally shared Memory that multiple Reasoners iteratively query against, yielding two native capabilities:

Backward Residual Connection — shallow and deep reasoning stages can reach knowledge across the model rather than relying only on forward layer-wise flow.
Dynamic Latent Reasoning — recurrent latent iteration refines hidden states before decoding, internalizing part of the deliberation process and shrinking visible chain-of-thought. The reported result is roughly a 4× end-to-end inference speedup over the Qwen3.5-35B baseline while holding comparable scores on general reasoning benchmarks and improving on scientific tasks (Biology-Instructions, Mol-Instructions, MolecularIQ).

On the serving side the model is a hybrid: 30 of 40 transformer layers use GDN (Gated Delta Net) linear attention (kimi-linear-family), with a full-attention layer every 4th layer (full_attention_interval: 4 → 10 full-attention layers), and the bottom of the stack is MoE-routed (2,560 routed experts × 512 intermediate, 8 active per token); a separate MoE-256 / top-8 MTP (NEXTN) layer feeds speculative decoding. It takes images via a vision tower and recognizes the standard <|vision_start|>…<|vision_end|> + <|image_pad|> markers. Context length is 262,144 tokens.

Variant	Architecture	Context	License
Intern-S2-Mobius	Mobius-v0 · GDN ×30 + full ×10 · MoE-2560 / top-8 · MTP · BF16	262,144	Apache-2.0

Recommended generation: temperature=0.8, top_p=1.0, top_k=50, min_p=0.0 — the values the model card recommends. Note these are not what the checkpoint ships in generation_config.json (temperature=1.0, top_p=0.95, top_k=20), and SGLang applies that file by default (--sampling-defaults model) — so send the recommended values explicitly per request if you want them. Resources: HuggingFace · GitHub (InternLM/Intern-S2-Mobius).

2. Configuration Tips

Trust remote code is required. Intern-S2-Mobius ships a custom configuration_interns2_mobius.py / modeling_interns2_mobius.py on its HF repo; every recipe adds --trust-remote-code.
Speculative decoding schedule. The checkpoint ships one MTP layer. Enable MTP for the lowest latency (--speculative-algorithm NEXTN --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4). We measured accept-length ~3.9/4 draft tokens at 8K-in / 1K-out, roughly tripling single-stream decode speed (median TPOT 9.79 ms → 3.13 ms at conc=1, 14.26 ms → 6.84 ms at conc=16) and roughly doubling mid-concurrency total throughput (9358 → 18029 tokens/s at conc=16, 21395 → 26033 tokens/s at conc=64). The high-throughput recipe stays spec-off because once you can batch wide, its saturation point is higher (34786 tokens/s at conc=256 vs the spec recipe’s peak at conc=64).
Mamba pool sizing. GDN layers live in a separate Mamba state pool; the --mamba-full-memory-ratio (defaults to 0.9) controls the split between the 10 full-attention layers’ KV pool and the 30 GDN layers’ conv+SSM state pool. Default split comfortably handles conc=64 on a 2×H200 node; if you need higher concurrency than --max-running-requests allows for your workload, raise --mamba-full-memory-ratio slightly (each +1% mamba ratio costs full-attn KV).
Vision input. Images are accepted via the standard image_url chat message type. Vision tokens are counted into the prompt (prompt_tokens_details.image_tokens shows the count), and the model honors <|vision_start|> / <|vision_end|> boundaries exactly.
B200 sizing. B200 (192 GB HBM) fits the BF16 weights + KV + Mamba pool on a single GPU with --tp 1. The B200 cells in the panel inherit the H200 recipe with only --tp changed — unverified; treat them as a starting point until the Intern-S2-Mobius team publishes a Blackwell measurement.

3. Advanced Usage

The outputs below are verbatim captures from a live server (sampling per the checkpoint’s generation_config.json, temperature 1.0). Re-running the same request yields a semantically equivalent but textually different trace — treat them as representative, not deterministic.

3.1 Reasoning

InternS2-Mobius is a hybrid-reasoning model — thinking traces start with “Thinking Process:” before the final answer. Enable the qwen3 reasoning parser (toggle Reasoning Parser in the Parsers card of the Playground above) to split thinking into message.reasoning_content and the answer into message.content.

Reasoning Example (Python)

Example

from openai import OpenAI

client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")
resp = client.chat.completions.create(
    model="internlm/Intern-S2-Mobius",
    messages=[{"role": "user", "content": "What is 15% of 240?"}],
)
msg = resp.choices[0].message
print("Reasoning:", getattr(msg, "reasoning_content", None))
print("Answer:", msg.content)

Example Output

Output

Reasoning: Thinking Process:

1.  **Identify the core question:** The user is asking for 15% of 240.
2.  **Determine the calculation method:** To find a percentage of a
    number, multiply the number by the percentage expressed as a decimal
    or fraction.
    *   Percentage: 15%
    *   Decimal: 0.15
    *   Fraction: 15/100
3.  **Perform the calculation:** $240 \times 0.15$
    *   Method 1: $240 \times 0.10 = 24$ (10%) and
        $240 \times 0.05 = 12$ (5%). Then add them: $24 + 12 = 36$.
    *   Method 2: $240 \times 15 = 3600$. Divide by 100 -> 36.
4.  **Verify the result:** The calculation is correct.
5.  **Formulate the answer:** State the final number clearly.cw

Answer:

15% of 240 is **36**.

Here is the math:
$240 \times 0.15 = 36$

3.2 Tool Calling

Enable the qwen3_coder tool-call parser (toggle Tool Call Parser in the Parsers card of the Playground above) to surface structured tool calls via message.tool_calls. Intern-S2-Mobius emits <tool_call>…<function=name>…<parameter=key>…value…</parameter>… — this is exactly the format qwen3_coder parses; without the parser the call is left as raw text in content. On this thinking-mode model the turn also fills reasoning_content, so print both fields. Auto-resolution works out of the box. Intern-S2-Mobius’s chat template contains the <function= / <parameter= markers the auto-detector keys on, so --reasoning-parser auto --tool-call-parser auto resolves to qwen3 / qwen3_coder without any extra config (verified on this build by tailing the server log’s “Auto-detected …” lines + a live tools request). You can pass the literal qwen3_coder slug, but you don’t have to.

Tool Calling Example (Python)

Example

from openai import OpenAI

client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")
tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get the current weather for a location",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {"type": "string", "description": "The city name"},
            },
            "required": ["location"],
        },
    },
}]
resp = client.chat.completions.create(
    model="internlm/Intern-S2-Mobius",
    messages=[{"role": "user", "content": "What is the weather in Beijing?"}],
    tools=tools,
)
msg = resp.choices[0].message
print("Reasoning:", getattr(msg, "reasoning_content", None))
print("Tool calls:", msg.tool_calls)

Example Output

Output

Reasoning: The user is asking for the weather in Beijing. I have access
to a get_weather function that can get the current weather for a
location. The function requires a "location" parameter which should be
the city name. In this case, the user specified "Beijing", so I should
use that as the location parameter.

Tool calls: [
  {
    "id": "call_545b5956b4c3457286261490",
    "index": 0,
    "type": "function",
    "function": {"name": "get_weather", "arguments": "{\"location\": \"Beijing\"}"}
  }
]

finish_reason: tool_calls

3.3 Vision Input

Intern-S2-Mobius takes images via the OpenAI-compatible image_url content type. Vision input works with the same server the Deploy panel produces — no extra model-specific flags needed.

Vision Example (Python)

Example

from openai import OpenAI

client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")
resp = client.chat.completions.create(
    model="internlm/Intern-S2-Mobius",
    messages=[{
        "role": "user",
        "content": [
            {"type": "image_url",
             "image_url": {"url": "https://raw.githubusercontent.com/open-mmlab/mmdeploy/main/tests/data/tiger.jpeg"}},
            {"type": "text", "text": "Describe this image in one sentence."},
        ],
    }],
)
msg = resp.choices[0].message
print("Reasoning:", getattr(msg, "reasoning_content", None))
print("Answer:", msg.content)

Example Output

Output

Reasoning: The user wants a one-sentence description of the image.
Key elements: Tiger, lying down, grass, looking at camera.
Drafting: A tiger is lying in the green grass looking directly at the
camera.
Refining for flow and detail: A majestic tiger with striking orange and
black stripes rests calmly on a bed of lush green grass, staring
intently directly at the viewer.

Answer:

A majestic tiger with striking orange and black stripes rests calmly on a
bed of lush green grass, staring intently directly at the viewer.

​Deployment

​Playground

​1. Model Introduction

​2. Configuration Tips

​3. Advanced Usage

​3.1 Reasoning

​3.2 Tool Calling

​3.3 Vision Input

Deployment

Playground

1. Model Introduction

2. Configuration Tips

3. Advanced Usage

3.1 Reasoning

3.2 Tool Calling

3.3 Vision Input