Ornith-1.5 - SGLang Documentation

Deployment

These recipes target a single Linux node with NVIDIA CUDA GPUs. The Ornith-1.5 model cards recommend SGLang >=0.5.9. Both parsers are included in every launch command: qwen3 separates reasoning from the answer, and qwen3_coder parses tool calls.

Install SGLang

Python
Docker

Command

pip install --upgrade "sglang>=0.5.9"

Run the Python command from the deployment panel in this environment. See the installation guide for platform-specific requirements.

Command

docker pull lmsysorg/sglang:latest

Select Docker in the deployment panel to generate the container command.

Choose your hardware, model size, and checkpoint format. The 9B recipe uses one GPU, 35B-A3B uses two, and 122B and 397B use eight. BF16 397B requires H200 in this matrix; its FP8 checkpoint is also available on H100. Select BF16 for a BF16 checkpoint or FP8 for an FP8 checkpoint. All commands use Hugging Face repository IDs and default to port 30000.

1. Model introduction

Ornith-1.5 is an agentic model family published by Ornith AI. The 9B, 35B-A3B, 122B, and 397B checkpoints use Qwen3.5 architectures. The family extends Ornith-1.0’s self-improvement loop by jointly optimizing task generation, scaffold construction, and solution rollouts through reinforcement learning. The model cards describe applications in coding, reasoning, and tool-using agents. The recipes expose an OpenAI-compatible API with separate reasoning output, parsed tool calls, and a 262,144-token context window. The table lists the checkpoints covered by this recipe.

Checkpoint	Format	Recipe
Ornith-1.5-9B	BF16	Dense; 1× H100 or H200
Ornith-1.5-35B-A3B	BF16	MoE, ~3B active; 2× H100 or H200
Ornith-1.5-35B-A3B-FP8	FP8	2× H100 or H200
Ornith-1.5-122B	BF16	MoE, 256 experts with 8 routed per token; 8× H100 or H200
Ornith-1.5-397B	BF16	MoE; 8× H200
Ornith-1.5-397B-FP8	FP8	8× H100 or H200

The official 9B, 35B-A3B, 122B, and 397B model cards declare the MIT license. Resources: Ornith-1.5 model repositories · Ornith-1.5 blog

2. Configuration tips

Model IDs: use the ornith-ai namespace. The 35B repository includes the -A3B suffix. FP8 selections use the actual -FP8 repository rather than the BF16 repository shown in their model-card quickstarts.
Context and memory: every recipe uses --context-length 262144. The 9B, 35B-A3B, and 397B recipes use --mem-fraction-static 0.85; the 122B recipe uses 0.80. Lower the context length if you need more memory headroom.
Reasoning: --reasoning-parser qwen3 exposes reasoning as message.reasoning_content, with the answer in message.content.
Tool calling: --tool-call-parser qwen3_coder exposes parsed calls in message.tool_calls. Your application executes the requested function and submits its result in a follow-up tool message.
Sampling: 35B-A3B and 397B recommend temperature=0.6, top_p=0.95, and top_k=20. For 9B and 122B, the model cards recommend temperature=1.0 and presence_penalty=1.5 for general tasks, or temperature=0.6 and presence_penalty=0.0 for precise coding; both use top_p=0.95, top_k=20, min_p=0.0, and repetition_penalty=1.0. These settings are distinct from benchmark-specific settings. The Python examples below use the precise-coding sampling settings.

3. Usage examples

These requests use the precise-coding sampling settings shared by 9B and 122B. Set model to the deployment panel’s --served-model-name, such as Ornith-1.5-9B or Ornith-1.5-122B. The examples use the default port; match the client port if you change the launch port.

Chat completion with reasoning (Python)

Example

from openai import OpenAI

client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")
response = client.chat.completions.create(
    model="Ornith-1.5-9B",
    messages=[{"role": "user", "content": "Write a compact Python is_prime function."}],
    temperature=0.6,
    top_p=0.95,
    presence_penalty=0.0,
    max_tokens=1024,
    extra_body={"top_k": 20, "min_p": 0.0, "repetition_penalty": 1.0},
)
message = response.choices[0].message
print("Reasoning:", getattr(message, "reasoning_content", None))
print("Answer:", message.content)

Tool-call request (Python)

Example

from openai import OpenAI

client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")
response = client.chat.completions.create(
    model="Ornith-1.5-9B",
    messages=[{"role": "user", "content": "Run the parser module's unit tests."}],
    tools=[{
        "type": "function",
        "function": {
            "name": "run_tests",
            "description": "Run the project's test suite.",
            "parameters": {
                "type": "object",
                "properties": {"target": {"type": "string"}},
                "required": ["target"],
            },
        },
    }],
    tool_choice="auto",
    temperature=0.6,
    top_p=0.95,
    presence_penalty=0.0,
    max_tokens=2048,
    extra_body={"top_k": 20, "min_p": 0.0, "repetition_penalty": 1.0},
)
message = response.choices[0].message
print("Reasoning:", getattr(message, "reasoning_content", None))
print("Tool calls:", message.tool_calls)

Agent integration

Configure your agent or coding CLI to use the same OpenAI-compatible endpoint:

API base URL: http://localhost:30000/v1.
API key: EMPTY for a local server without API-key authentication.
Model: the selected checkpoint’s --served-model-name, such as Ornith-1.5-122B.

Use the served model name rather than the Hugging Face repository ID when the two differ. Keep both parsers enabled so the agent receives separate reasoning and structured tool calls. Your agent executes tools and returns their results to the server in subsequent messages.

​Deployment

​1. Model introduction

​2. Configuration tips

​3. Usage examples

​Agent integration

Deployment

1. Model introduction

2. Configuration tips

3. Usage examples

Agent integration