Skip to main content

Ctrl+K

Get Started

Install SGLang

Basic Usage

Sending Requests
OpenAI-Compatible APIs
Ollama-Compatible API
Offline Engine API
SGLang Native APIs
Sampling Parameters
Popular Model Usage (DeepSeek, GPT-OSS, GLM, Llama, MiniMax, Qwen, and more)
Diffusion Language Models (dLLMs)
Diffusion

Advanced Features

Server Arguments
Hyperparameter Tuning
Attention Backend
Speculative Decoding
Structured Outputs
Structured Outputs For Reasoning Models
Tool Parser
Reasoning Parser
Quantization
Quantized KV Cache
Expert Parallelism
LoRA Serving
PD Disaggregation
EPD Disaggregation
Pipeline Parallelism for Long Context
Hierarchical KV Caching (HiCache)
Query VLM with Offline Engine
DP for Multi-Modal Encoder in SGLang
Cuda Graph for Multi-Modal Encoder in SGLang
SGLang Model Gateway
Deterministic Inference
Observability
Checkpoint Engine Integration
SGLang for RL Systems

Supported Models

Text Generation
Image Generation
Retrieval & Ranking
Specialized Models
Extending SGLang

Hardware Platforms

AMD GPUs
CPU Servers
TPU
NVIDIA Jetson Orin
Ascend NPUs
XPU

Developer Guide

Contribution Guide
Development Guide Using Docker
Development Guide for JIT Kernels
Benchmark and Profiling
Bench Serving Guide
Evaluating New Models with SGLang

References

Troubleshooting and Frequently Asked Questions
Environment Variables
Production Metrics
Production Request Tracing
Multi-Node Deployment
Custom Chat Template
Frontend Language
Post-Training Integration
Learn More and Join the Community

Repository
Show source
Suggest edit
Open issue

.rst

Extending SGLang

Extending SGLang#

Adding new models and alternative backends.

How to Support New Models
Transformers fallback in SGLang
Use Models From ModelScope

previous

Reward Models

next

How to Support New Models

By SGLang Team

© Copyright 2023-2026, SGLang.

Last updated on Feb 07, 2026.