> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sglang.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Environment Variables

SGLang supports various environment variables related to Ascend NPU that can be used to configure its runtime behavior.
This document provides a list of commonly used environment variables and aims to stay updated over time.

## Directly Used in SGLang

<table>
  <thead>
    <tr>
      <th>Environment Variable</th>
      <th>Description</th>
      <th>Default Value</th>
    </tr>
  </thead>

  <tbody>
    <tr>
      <td><code>SGLANG\_NPU\_USE\_MLAPO</code></td>
      <td>Adopts the <code>MLAPO</code> fusion operator in attention <br /> preprocessing stage of the MLA model.</td>
      <td><code>false</code></td>
    </tr>

    <tr>
      <td><code>SGLANG\_USE\_FIA\_NZ</code></td>
      <td>Reshapes KV Cache for FIA NZ format.<br /> <code>SGLANG\_USE\_FIA\_NZ</code> must be enabled with <code>SGLANG\_NPU\_USE\_MLAPO</code></td>
      <td><code>false</code></td>
    </tr>

    <tr>
      <td><code>SGLANG\_NPU\_USE\_MULTI\_STREAM</code></td>
      <td>Enable dual-stream computation of shared experts <br /> and routing experts in DeepSeek models.<br /> Enable dual-stream computation in DeepSeek DSA Indexer.</td>
      <td><code>false</code></td>
    </tr>

    <tr>
      <td><code>SGLANG\_NPU\_DISABLE\_ACL\_FORMAT\_WEIGHT</code></td>
      <td>Disable cast model weight tensor to a specific NPU <br /> ACL format.</td>
      <td><code>false</code></td>
    </tr>

    <tr>
      <td><code>SGLANG\_DEEPEP\_NUM\_MAX\_DISPATCH\_TOKENS\_PER\_RANK</code></td>
      <td>The maximum number of dispatched tokens on each rank.</td>
      <td><code>128</code></td>
    </tr>
  </tbody>
</table>

## Used in DeepEP Ascend

<table>
  <thead>
    <tr>
      <th>Environment Variable</th>
      <th>Description</th>
      <th>Default Value</th>
    </tr>
  </thead>

  <tbody>
    <tr>
      <td><code>DEEPEP\_NORMAL\_LONG\_SEQ\_PER\_ROUND\_TOKENS</code></td>
      <td>Enable long-sequence token pipelining in dispatch stage. Indicates <br /> the number of tokens transmitted per round on each rank.</td>
      <td><code>8192</code></td>
    </tr>

    <tr>
      <td><code>DEEPEP\_NORMAL\_LONG\_SEQ\_ROUND</code></td>
      <td>Enable long-sequence token pipelining in dispatch stage. Indicates <br /> the number of rounds transmitted on each rank.</td>
      <td><code>1</code></td>
    </tr>

    <tr>
      <td><code>DEEPEP\_NORMAL\_COMBINE\_ENABLE\_LONG\_SEQ</code></td>
      <td>Enable long-sequence token pipelining in combine stage. <br /> The value <code>0</code> means disabled.</td>
      <td><code>0</code></td>
    </tr>

    <tr>
      <td><code>MOE\_ENABLE\_TOPK\_NEG\_ONE</code></td>
      <td>Needs to be enabled when the expert ID to be processed by <br /> DEEPEP contains -1.</td>
      <td><code>0</code></td>
    </tr>

    <tr>
      <td><code>DEEP\_NORMAL\_MODE\_USE\_INT8\_QUANT</code></td>

      <td>
        <strong>Deprecated — will be removed in a future release.</strong>

        <br />

        When set to <code>1</code>, quantizes intermediate activations to INT8 in
        <br />the DeepEP dispatch operator during normal mode, reducing communication
        <br />volume for W8A8-quantized MoE models. This variable will become a no-op;
        <br />the quantization behavior will be inferred automatically.
      </td>

      <td><code>0</code></td>
    </tr>
  </tbody>
</table>

## Others

<table>
  <thead>
    <tr>
      <th>Environment Variable</th>
      <th>Description</th>
      <th>Default Value</th>
    </tr>
  </thead>

  <tbody>
    <tr>
      <td><code>TASK\_QUEUE\_ENABLE</code></td>
      <td>Used to control the optimization level of the dispatch queue<br /> about the task\_queue operator. <a href="https://www.hiascend.com/document/detail/zh/Pytorch/730/comref/Envvariables/docs/zh/environment_variable_reference/TASK_QUEUE_ENABLE.md">Detail</a></td>
      <td><code>1</code></td>
    </tr>

    <tr>
      <td><code>INF\_NAN\_MODE\_ENABLE</code></td>
      <td>Controls whether the chip uses saturation mode or INF\_NAN mode. <a href="https://www.hiascend.com/document/detail/zh/CANNCommunityEdition/800alpha001/apiref/envref/envref_07_0056.html">Detail</a></td>
      <td><code>1</code></td>
    </tr>

    <tr>
      <td><code>STREAMS\_PER\_DEVICE</code></td>
      <td>Configures the maximum number of streams for the stream pool. <a href="https://www.hiascend.com/document/detail/zh/Pytorch/720/comref/Envvariables/Envir_041.html">Detail</a></td>
      <td><code>32</code></td>
    </tr>

    <tr>
      <td><code>PYTORCH\_NPU\_ALLOC\_CONF</code></td>
      <td>Controls the behavior of the cache allocator. <br />This variable changes memory usage and may cause performance fluctuations. <a href="https://www.hiascend.com/document/detail/zh/Pytorch/700/comref/Envvariables/Envir_012.html">Detail</a></td>

      <td />
    </tr>

    <tr>
      <td><code>ASCEND\_MF\_STORE\_URL</code></td>
      <td>The address of config store in MemFabric during PD separation, <br />which is generally set to the IP address of the P primary node<br /> with an arbitrary port number.</td>

      <td />
    </tr>

    <tr>
      <td><code>ASCEND\_LAUNCH\_BLOCKING</code></td>
      <td>Controls whether synchronous mode is enabled during operator execution. <a href="https://www.hiascend.com/document/detail/zh/Pytorch/710/comref/Envvariables/Envir_006.html">Detail</a></td>
      <td><code>0</code></td>
    </tr>

    <tr>
      <td><code>HCCL\_OP\_EXPANSION\_MODE</code></td>
      <td>Configures the expansion position for communication algorithm scheduling. <a href="https://www.hiascend.com/document/detail/zh/CANNCommunityEdition/800alpha001/apiref/envref/envref_07_0094.html">Detail</a></td>

      <td />
    </tr>

    <tr>
      <td><code>HCCL\_BUFFSIZE</code></td>
      <td>Controls the size of the buffer area for shared data between two NPUs. <br />The unit is MB, and the value must be greater than or equal to 1. <a href="https://www.hiascend.com/document/detail/zh/Pytorch/60RC3/ptmoddevg/trainingmigrguide/performance_tuning_0047.html">Detail</a></td>
      <td><code>200</code></td>
    </tr>

    <tr>
      <td><code>HCCL\_SOCKET\_IFNAME</code></td>
      <td>Configures the name of the network card used by the Host <br />during HCCL initialization. <a href="https://www.hiascend.com/document/detail/zh/canncommercial/81RC1/apiref/envvar/envref_07_0075.html">Detail</a></td>

      <td />
    </tr>

    <tr>
      <td><code>GLOO\_SOCKET\_IFNAME</code></td>
      <td>Configures the network interface name for GLOO communication.</td>

      <td />
    </tr>
  </tbody>
</table>