LFM3

​

Deployment

docker pull lmsysorg/sglang:dev-lfm3

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<your-hf-token>" \
    --ipc=host \
    lmsysorg/sglang:dev-lfm3 \
    sglang serve <use args below>

​

Playground

• TP override — both variants are served at TP=1 in the Deploy cells; TP=2 is verified on the text model.
• PD disaggregation and HiCache — the generic SGLang prefill/decode split and hierarchical KV cache, not yet run end-to-end on LFM3.

​

1. Model Introduction

• Hybrid gated short conv + attention layout: the 1.7B text model stacks 13 gated short convolution blocks, 5 sliding-window GQA blocks, and 6 full-attention GQA blocks. Only the 6 full-attention layers grow KV state with sequence length — the 5 sliding-window layers cap theirs at 256 tokens, and each convolution layer carries a fixed 6-token state — so long-context memory stays small.
• Long context: 256K tokens on the text model; 128K on the VL model.
• Thinking by default: both checkpoints emit a <think>...</think> chain-of-thought before answering. Pass enable_thinking=False through the chat template to answer directly (see §3.2).
• Tool calling: function calls are emitted in an XML-ish <tool_call><function=name><parameter=arg>...</parameter></function></tool_call> format that SGLang parses into standard message.tool_calls (see §3.3).
• Vision and video (VL): LFM3-VL pairs the LFM3 language backbone with a Liquid hybrid convolution/attention vision encoder (14 layers: 8 gated depthwise 2D convolution + 6 bidirectional attention, 16×16 patches with 2D rotary embeddings) and a 25M projector. Images and videos are processed at their native resolution; every 2×2 patch block merges into one language token. Videos are sampled at 2 fps with per-frame timestamps — the first LFM release with video input.

​

2. Configuration Tips

• Remote code (--trust-remote-code): the checkpoints carry auto_map remote modeling code, so the flag is required — it is already part of the generated command.
• Attention backend: the LFM3 model overrides default the attention backend to flashinfer on NVIDIA GPUs — no --attention-backend flag is needed. For sliding-window correctness the overrides also switch FlashInfer prefill to paged mode automatically (SGLANG_FLASHINFER_USE_PAGED=True), because FlashInfer’s ragged planner does not apply the 256-token window within prefill chunks. The triton backend also passes the model PR’s text/vision checks on H100, but FlashInfer is the default and the backend these cells were verified with.
• AMD (ROCm): the generated command pins --attention-backend aiter (the LFM3 overrides’ default on HIP) with SGLANG_USE_AITER=1. The gated-conv blocks take the PyTorch fallback path on ROCm — the CuTe DSL conv kernel is CUDA-only (sm90 / sm100 / sm120-class) — so expect lower conv throughput until a ROCm kernel lands. No published ROCm image carries LFM3 support yet: install SGLang from source and use the Python output of the command panel.
• Reasoning parser (--reasoning-parser auto): LFM3 wraps its chain-of-thought in <think>...</think> tags. The generated command passes --reasoning-parser auto, which resolves the qwen3 parser for this checkpoint and splits thinking into reasoning_content. Without a reasoning parser the thinking stays inline in content.
• Tool-call parser (--tool-call-parser auto): LFM3’s <tool_call><function=...><parameter=...> format matches the qwen3_coder parser’s sentinels. The generated command passes --tool-call-parser auto, which resolves qwen3_coder for this checkpoint (the server log prints the resolved parser), and calls surface as standard message.tool_calls. If the log names a different parser, pass --tool-call-parser qwen3_coder explicitly.
• Thinking by default: the chat template enables thinking (enable_thinking=true) unless you opt out, and long thinking traces are normal — a mean of ~1100 reasoning tokens per GSM8K answer was measured, so request-level token budgets must accommodate the chain-of-thought, not just the final answer. Pass extra_body={"chat_template_kwargs": {"enable_thinking": False}} to answer directly (the generation prompt then ends with an empty <think></think> block).
• Hardware requirements: both LFM3 checkpoints run on a single GPU (TP=1); the BF16 weights take under 4 GB, leaving most of GPU memory for KV cache and multimodal features. TP=2 is verified on the text model (Playground override) but not required.
• No speculative decoding: the server rejects speculative-decoding configurations for this architecture at startup.
• Precision: BF16 (or FP16) checkpoints only — there is no packed-quantization (FP8/FP4) variant.
• Video input (VL): LFM3-VL samples videos at 2 fps, pairs consecutive frames, and prefixes each frame with its timestamp. Pass videos as video_url content blocks the same way as images (see §3.4).

​

3. Advanced Usage

​

3.1 Basic Usage

from openai import OpenAI

client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")

​

3.2 Reasoning

resp = client.chat.completions.create(
    model="LiquidAI/LFM3-1.7B",
    messages=[{"role": "user", "content": "What is 17 * 23?"}],
)
msg = resp.choices[0].message
print("Reasoning:", msg.reasoning_content)
print("Answer:", msg.content)

Reasoning: …
Answer: 391

​

3.3 Tool Calling

resp = client.chat.completions.create(
    model="LiquidAI/LFM3-1.7B",
    messages=[{"role": "user", "content": "What's the weather in Warsaw in celsius?"}],
    tools=[{
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get current weather for a location",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {"type": "string"},
                    "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
                },
                "required": ["city"],
            },
        },
    }],
)
msg = resp.choices[0].message
print("Reasoning:", msg.reasoning_content)
print("Content:", msg.content)
for call in msg.tool_calls or []:
    print(call.function.name, call.function.arguments)
print("finish_reason:", resp.choices[0].finish_reason)

Reasoning: …
Content: None
get_weather {"city":"Warsaw","unit":"celsius"}
finish_reason: tool_calls

​

3.4 Vision and Video Input

import base64

with open("test_pattern.png", "rb") as f:
    data_uri = "data:image/png;base64," + base64.b64encode(f.read()).decode()

resp = client.chat.completions.create(
    model="LiquidAI/LFM3-VL-2B",
    messages=[{
        "role": "user",
        "content": [
            {"type": "image_url", "image_url": {"url": data_uri}},
            {"type": "text", "text": "Describe this image in one sentence."},
        ],
    }],
)
print(resp.choices[0].message.content)

The image shows a dark blue flag-like background with a red circle on the left and a yellow square on the right, accompanied by the white text "LFM3" centered along the bottom edge.

import base64

with open("test_clip.mp4", "rb") as f:
    data_uri = "data:video/mp4;base64," + base64.b64encode(f.read()).decode()

resp = client.chat.completions.create(
    model="LiquidAI/LFM3-VL-2B",
    messages=[{
        "role": "user",
        "content": [
            {"type": "video_url", "video_url": {"url": data_uri}},
            {"type": "text", "text": "Describe what happens in this video."},
        ],
    }],
)
print(resp.choices[0].message.content)

The video is a short abstract, glitch‑style animation in which bright diagonal streaks of color and pixelated shapes (including a plus/cross made of gray pixels) sweep and shift across a background of vertical rainbow stripes (red, green, yellow, blue, magenta and cyan), while a small digital timer in the top‑left corner counts upward.