LongCat-2.0 - SGLang Documentation

Deployment

Install SGLang

For all methods and hardware platforms, see the official SGLang installation guide. LongCat-2.0 support is on SGLang main; use a nightly wheel or rolling nightly Docker image until the next tagged release includes it. The two paths below match the Python / Docker toggle in the command panel.

Python (pip / uv)
Docker

Command

pip install --upgrade pip
pip install uv

# Choose the nightly wheel index for your CUDA runtime.
SGLANG_WHL_INDEX=https://docs.sglang.ai/whl/cu130  # B300 / CUDA 13
# SGLANG_WHL_INDEX=https://docs.sglang.ai/whl/cu129  # CUDA 12.9

uv pip install --prerelease=allow --extra-index-url "${SGLANG_WHL_INDEX}" "sglang[all]"

Then run the Python output of the command panel below in that environment.

Command

# Choose the rolling nightly image for your hardware.
SGLANG_DOCKER_IMAGE=lmsysorg/sglang:dev-cu13  # B300 / CUDA 13
# SGLANG_DOCKER_IMAGE=lmsysorg/sglang:dev  # Other supported hardware

docker pull "${SGLANG_DOCKER_IMAGE}"

For how to launch the image, see Install -> Method 3: Using Docker. Substitute the inner sglang serve ... with what the command generator below produces.

Pick your hardware + recipe to generate the launch command. LongCat-2.0 currently exposes one model-card-aligned serving strategy:

Balanced - the validated B300 recipe and the 2-node H200/B200/H20 topology use TP/EP parallelism with LongCat sparse attention prefill.

The B300 single-node recipe was validated end-to-end with CUDA graph capture enabled. H200, B200, and H20 are shown as 2-node recipes because LongCat-2.0-FP8 needs 16 ranks for those GPU memory profiles.

Playground

The Playground is where you experiment with SGLang features beyond the verified matrix. The Deploy panel above only emits combinations the SGLang team has signed off on; the Playground lets you turn on additional knobs on top of whichever cell the Deploy panel is currently showing.

1. Model Introduction

LongCat-2.0-FP8 is the FP8 checkpoint of Meituan LongCat-2.0, a large sparse Mixture-of-Experts language model with 1.6T total parameters and about 48B activated parameters per token. It combines LongCat Sparse Attention (LSA), expert parallel MoE layers, and an n-gram/token-table embedding path for serving long-context workloads efficiently.

Model	Architecture	Serving precision
LongCat-2.0-FP8	Sparse MoE · LongCat Sparse Attention · n-gram embedding	FP8 weights, BF16 KV cache

Resources: LongCat-2.0-FP8.

2. Configuration Tips

Remote code. Use --trust-remote-code for the Hugging Face checkpoint.
Topology. The 8x B300 recipe uses TP=8 and EP=8. H200, B200, and H20 use a 2-node 16 GPU layout with TP=16 and EP=16; the command panel injects the multi-node rank flags for you.
LongCat sparse attention. Keep --nsa-prefill-backend fa3 with --chunked-prefill-size 2048 for the model-card-aligned prefill path.
Memory. The recipe uses --kv-cache-dtype bfloat16 and starts at --mem-fraction-static 0.92. Tune memory only after the generated command launches cleanly on your cluster.
Weight loading. --model-loader-extra-config '{"enable_multithread_load":true,"num_threads":12}' loads checkpoint shards in parallel and reduces startup time.
FP8 backend selection. Do not pass --fp8-gemm-runner-backend manually. SGLang selects the correct backend for the LongCat FP8 scale layout.
Host, port, and ranks. Use the command panel environment fields for HOST_IP, PORT, NODE0_IP, and NODE_RANK instead of hardcoding them in the recipe.

3. Advanced Usage

3.1 Test the deployment

Chat completion example (cURL)

Command

curl http://localhost:30000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "meituan-longcat/LongCat-2.0-FP8",
    "messages": [
      {"role": "user", "content": "A shop has 17 apples and sells 8. Then it buys 6 more. How many apples are there? Answer with only the final number."}
    ],
    "max_tokens": 32,
    "chat_template_kwargs": {"enable_thinking": false}
  }'

Expected output

Output

OpenAI-compatible client (Python)

Example

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:30000/v1",
    api_key="EMPTY",
)

response = client.chat.completions.create(
    model="meituan-longcat/LongCat-2.0-FP8",
    messages=[
        {
            "role": "user",
            "content": "Solve: A shop has 17 apples and sells 8, then buys 6 more. Answer with only the final number.",
        }
    ],
    max_tokens=32,
    extra_body={"chat_template_kwargs": {"enable_thinking": False}},
)

print(response.choices[0].message.content)

Example output

Output

4. Validation

The B300 recipe was validated with meituan-longcat/LongCat-2.0-FP8 on 8x B300 using the command generated above.

Evaluation	Examples	Accuracy
GSM8K	200	98.0%
GSM8K	1314	95.8904109589041%

CUDA graph was enabled, and decode CUDA graph capture completed successfully during serving validation.

​Deployment

​Playground

​1. Model Introduction

​2. Configuration Tips

​3. Advanced Usage

​3.1 Test the deployment

​4. Validation

Deployment

Playground

1. Model Introduction

2. Configuration Tips

3. Advanced Usage

3.1 Test the deployment

4. Validation