Deployment
Install SGLang
Install SGLang
For all methods and hardware platforms, see the official SGLang installation guide. The two paths below match the Python / Docker toggle in the command panel.Then run the Python output of the command panel below in that environment.
- Python (pip / uv)
- Docker
Command
Speed numbers exist for the RTX 5090 and DGX Spark cells — a single card at
random 1024/1024, recorded as Mean. The H200 and RTX PRO 6000 cells are
pending measurement, and no accuracy numbers have been taken on any platform
yet. The DSpark overlay carries no speed numbers on any card.
Playground
The Playground is where you experiment with SGLang features beyond the recipes above. The Deploy panel emits this model’s documented launch recipes; the Playground lets you turn on additional knobs on top of whichever cell the Deploy panel is currently showing.1. Model Introduction
MiniCPM5-2B is the second model in OpenBMB’s MiniCPM5 series, following MiniCPM5-1B. It is a dense 2B-class Transformer built for on-device assistants, local deployment, coding agents and tool-use workflows — scenarios where a compact model with a small deployment footprint is preferred. OpenBMB reports 2B-class open-source SOTA within its comparison set (average 53.9), with its clearest advantages in code reasoning, math reasoning, long-context understanding, tool use and agentic tasks. The checkpoint uses the standardLlamaForCausalLM architecture — no custom kernels and no model-code fork — so SGLang loads it through the stock Llama path. It is 2,516,756,480 parameters (1,981,982,720 non-embedding) over 42 layers with GQA (16 query heads, 2 KV heads), and a native context length of 131,072 tokens. Post-training runs SFT → RL → On-Policy Distillation, which merges 16 RL expert models into the single released checkpoint. Weights are released under the Apache-2.0 license.
| Checkpoint | Precision | Role on this page |
|---|---|---|
| openbmb/MiniCPM5-2B | BF16 | The served model in every cell above (final release, post-trained with RL + OPD). |
| openbmb/MiniCPM5-2B-DSpark | BF16 draft | Draft model loaded by the DSPARK row of the Deploy panel; not served on its own. |
temperature=1.0, top_p=0.95 (informational — SGLang reads the checkpoint’s generation_config.json; do not hardcode these in client code).
Resources: HuggingFace · ModelScope · GitHub · Tech report.
2. Advanced Usage
2.1 Thinking Mode
Thinking is controlled by the chat template’senable_thinking flag, passed per request through chat_template_kwargs. The qwen3 reasoning parser — already in every generated command, and toggleable from the Parsers card in the Playground above — splits the <think> segment into message.reasoning_content and leaves the final answer in message.content. Drop the flag and the thinking text stays inline in content, closing tag and all.
Thinking On / Off (Python)
Thinking On / Off (Python)
Example
Example Output
Example Output
Output
2.2 Tool Calling
MiniCPM5-2B emits XML-style tool calls (<function name="..."><param name="...">...</param></function>), and SGLang’s built-in minicpm5 detector converts them to OpenAI-compatible tool_calls, parallel calls included. The parser is already in every command the Deploy panel generates; the Tool Call Parser chip in the Playground’s Parsers card is an opt-out, not an opt-in.
Tool Calling (Python)
Tool Calling (Python)
Example
Example Output
Example Output
Output
