Skip to main content

​
Production Metrics

SGLang exposes the following metrics via Prometheus. You can enable them by adding --enable-metrics when launching the server. You can query them by:
Command
See Production Metrics and Production Request Tracing for more details.

​
Logging

By default, SGLang does not log any request contents. You can log them by using --log-requests. You can control the verbosity by using --log-request-level. See Logging for more details. You can change verbosity at runtime:
Command

​
Request Dump and Replay

You can dump all requests and replay them later for benchmarking or other purposes. To start dumping, use the following command to send a request to a server:
Command
The server will dump the requests into a pickle file for every 100 requests. To replay the request dump, use scripts/playground/replay_request_dump.py.

​
Crash Dump and Replay

Sometimes the server might crash, and you may want to debug the cause of the crash. SGLang can preserve recent request data for replay and collect CUDA device coredumps for low-level debugging. Set the crash diagnostics folder with --crash-dump-folder /tmp/crash_dump. When SGLang handles a crash, it writes completed requests retained by the crash-dump buffer plus in-flight requests to /tmp/crash_dump/<hostname>/crash_dump_<timestamp>.pkl. The file also contains the server arguments and launch command. Replay it with scripts/playground/replay_request_dump.py. On NVIDIA CUDA, the option also sets default environment variables before CUDA initializes. These defaults enable device coredumps on CUDA exceptions and allow SGLang to trigger device coredumps for live scheduler processes when it handles a crash. CUDA device coredumps are written to /tmp/crash_dump/<hostname>/core.cuda.<timestamp>.<pid>. Explicitly configured CUDA coredump environment variables take precedence, including a custom CUDA_COREDUMP_FILE path. This option does not configure OS process core dumps.