Skip to main content

Ctrl+K

Get Started

Install SGLang

Basic Usage

Sending Requests
OpenAI-Compatible APIs
Ollama-Compatible API
Offline Engine API
SGLang Native APIs
DeepSeek OCR (OCR-1 / OCR-2)
Sampling Parameters
Popular Model Usage (DeepSeek, GPT-OSS, GLM, Llama, MiniMax, Qwen, and more)

Advanced Features

Server Arguments
Hyperparameter Tuning
Attention Backend
Speculative Decoding
Structured Outputs
Structured Outputs For Reasoning Models
Tool Parser
Reasoning Parser
Quantization
Quantized KV Cache
Expert Parallelism
LoRA Serving
PD Disaggregation
EPD Disaggregation
Pipeline Parallelism for Long Context
Hierarchical KV Caching (HiCache)
Query VLM with Offline Engine
DP for Multi-Modal Encoder in SGLang
Cuda Graph for Multi-Modal Encoder in SGLang
SGLang Model Gateway
Deterministic Inference
Observability
Checkpoint Engine Integration
SGLang for RL Systems

Supported Models

Large Language Models
Multimodal Language Models
Diffusion Language Models
Diffusion Models
Embedding Models
Reward Models
Rerank Models
Classification API
How to Support New Models
Transformers fallback in SGLang
Use Models From ModelScope
MindSpore Models

Hardware Platforms

AMD GPUs
CPU Servers
TPU
NVIDIA Jetson Orin
Ascend NPUs
XPU

Developer Guide

Contribution Guide
Development Guide Using Docker
Development Guide for JIT Kernels
Benchmark and Profiling
Bench Serving Guide
Evaluating New Models with SGLang

References

Troubleshooting and Frequently Asked Questions
Environment Variables
Production Metrics
Production Request Tracing
Multi-Node Deployment
Custom Chat Template
Frontend Language
Post-Training Integration
Learn More and Join the Community

Repository
Show source
Suggest edit
Open issue

.rst

Ascend NPUs

Ascend NPUs#

SGLang installation with NPUs support
Support Models on Ascend NPU
DeepSeek examples
Qwen3 examples
Contribution Guide
Best Practice on Ascend NPU

previous

NVIDIA Jetson Orin

next

SGLang installation with NPUs support

By SGLang Team

© Copyright 2023-2026, SGLang.

Last updated on Feb 01, 2026.