Skip to main content

Ctrl+K

Get Started

Install SGLang

Basic Usage

Sending Requests
OpenAI-Compatible APIs
Ollama-Compatible API
Offline Engine API
SGLang Native APIs
Sampling Parameters
Popular Model Usage (DeepSeek, GPT-OSS, GLM, Llama, Qwen, and more)

Advanced Features

Server Arguments
Hyperparameter Tuning
Attention Backend
Speculative Decoding
Structured Outputs
Structured Outputs For Reasoning Models
Tool Parser
Reasoning Parser
Quantization
Quantized KV Cache
Expert Parallelism
LoRA Serving
PD Disaggregation
EPD Disaggregation
Pipeline Parallelism for Long Context
Hierarchical KV Caching (HiCache)
Query Vision Language Model
DP for Multi-Modal Encoder in SGLang
SGLang Model Gateway (formerly SGLang Router)
Deterministic Inference
Observability
Checkpoint Engine Integration

Supported Models

Large Language Models
Multimodal Language Models
Diffusion Language Models
Embedding Models
Reward Models
Rerank Models
Classification API
How to Support New Models
Transformers fallback in SGLang
Use Models From ModelScope
MindSpore Models

Hardware Platforms

AMD GPUs
CPU Servers
TPU
NVIDIA Jetson Orin
Ascend NPUs
XPU

Developer Guide

Contribution Guide
Development Guide Using Docker
Benchmark and Profiling
Bench Serving Guide

References

Troubleshooting and Frequently Asked Questions
Environment Variables
Production Metrics
Production Request Tracing
Multi-Node Deployment
Custom Chat Template
Frontend Language
Post-Training Integration
Learn More and Join the Community

Repository
Show source
Suggest edit
Open issue

.rst

Ascend NPUs

Ascend NPUs#

SGLang installation with NPUs support
DeepSeek examples
Qwen3 examples

previous

NVIDIA Jetson Orin

next

SGLang installation with NPUs support

By SGLang Team

© Copyright 2023-2025, SGLang.

Last updated on Dec 17, 2025.