Krea-2 - SGLang Documentation

1. Model Introduction

Krea-2 is a high-quality text-to-image diffusion model from Krea. It ships in two variants that share the same backbone and differ only in their sampling recipe:

Krea-2-Turbo - a distilled, few-step model that produces photorealistic images in only 8 inference steps with no classifier-free guidance (guidance_scale = 1.0), ideal for fast and interactive generation.
Krea-2-Raw - the base (non-distilled) model that trades speed for maximum fidelity, using a longer schedule (~52 steps) with classifier-free guidance (guidance_scale ≈ 4.5).

Both variants are built on a single-stream MMDiT with a Qwen3-VL text encoder and the Qwen-Image VAE, and are distributed in the standard diffusers layout (a model_index.json plus sharded transformer/, text_encoder/, vae/, tokenizer/, and scheduler/ folders). SGLang loads them natively - just point --model-path at the repo, no conversion step required. Key Features:

Two variants, one pipeline: switch between fast (Turbo) and high-fidelity (Raw) by changing only the model path and the sampling settings.
Photorealistic generation at 1024x1024 and other resolutions.
Native diffusers loading: components (DiT, text encoder, VAE, scheduler) are read straight from the repo’s model_index.json.

For more details, see the Krea-2-Turbo and Krea-2-Raw HuggingFace pages.

2. SGLang-diffusion Installation

SGLang-diffusion offers multiple installation methods. You can choose the most suitable installation method based on your hardware platform and requirements. Please refer to the official SGLang-diffusion installation guide for installation instructions.

3. Model Deployment

This section covers deploying Krea-2-Turbo for fast, high-quality image generation.

3.1 Basic Configuration

Krea-2-Turbo generates high-quality images in only 8 inference steps. Launch the server with:

Command

sglang serve \
  --model-path krea/Krea-2-Turbo \
  --num-gpus 1 \
  --port 30000

The step count and guidance scale are request-time settings (see API Usage); Krea-2-Turbo defaults to 8 steps with guidance_scale = 1.0.

3.2 Configuration Tips

Currently supported optimizations are listed here.

--num-gpus: Number of GPUs to use.
Multi-GPU (tensor and/or sequence parallelism): see Section 3.3.

3.3 Multi-GPU: tensor and sequence parallelism

Krea-2 supports two multi-GPU axes that can be combined; --num-gpus must equal tp_size × ulysses_degree.

Tensor parallelism (--tp-size N) shards the DiT weights across GPUs, lowering per-GPU VRAM. Krea-2’s attention heads (48 query / 12 KV) and text heads (20) are divisible by a tp size of 1, 2, or 4.
Sequence parallelism / Ulysses (--ulysses-degree N) shards the image-token sequence across GPUs while keeping the text prefix replicated. It does not shard weights (per-GPU VRAM is unchanged), but its output is bitwise-identical to single-GPU. It currently requires a single prompt per request (ragged/padded multi-prompt batches under SP are not supported — use --tp-size for those).

Command

# Tensor parallel (2 GPUs) — lowest per-GPU VRAM (DiT weights sharded)
sglang serve --model-path krea/Krea-2-Turbo --num-gpus 2 --tp-size 2 --port 30000

# Sequence parallel / Ulysses (2 GPUs) — output bitwise-identical to single-GPU
sglang serve --model-path krea/Krea-2-Turbo --num-gpus 2 --ulysses-degree 2 --port 30000

# Hybrid TP × SP (4 GPUs) — composes both axes
sglang serve --model-path krea/Krea-2-Turbo --num-gpus 4 --tp-size 2 --ulysses-degree 2 --port 30000

Measured on 2× H200 (Krea-2-Turbo, 8 steps, 1024×1024): --tp-size 2 and --ulysses-degree 2 each give ~1.7× denoise speedup over single-GPU; the hybrid TP=2 × SP=2 reaches ~2.8× on 4 GPUs. Choosing: on memory-constrained GPUs prefer --tp-size (it shards the ~24 GB DiT, e.g. ~38 GB → ~27 GB per GPU on 2 GPUs); on large-VRAM GPUs sequence parallelism is marginally faster and numerically exact, and the two compose for the highest throughput.

4. API Usage

For complete API documentation, please refer to the official API usage guide.

4.1 Generate an Image

Generate an image with the OpenAI-compatible images API:

Example

import base64
from openai import OpenAI

client = OpenAI(api_key="EMPTY", base_url="http://localhost:30000/v1")

response = client.images.generate(
    model="krea/Krea-2-Turbo",
    prompt="a red fox sitting in fresh snow, golden hour, photorealistic",
    n=1,
    response_format="b64_json",
)

# Save the generated image
image_bytes = base64.b64decode(response.data[0].b64_json)
with open("output.png", "wb") as f:
    f.write(image_bytes)

You can also generate a single image from the command line:

Command

sglang generate --model-path krea/Krea-2-Turbo \
    --prompt "a red fox sitting in fresh snow, golden hour, photorealistic" \
    --num-inference-steps 8 --height 1024 --width 1024 --save-output

4.2 Advanced Usage

4.2.1 Cache-DiT Acceleration

SGLang integrates Cache-DiT, a caching acceleration engine for Diffusion Transformers (DiT), to speed up inference with minimal quality loss. Enable it by setting SGLANG_CACHE_DIT_ENABLED=true. For more details, see the SGLang Cache-DiT documentation. Cache-DiT works for both Krea-2 variants with no extra configuration: SGLang tracks each request’s classifier-free-guidance mode, so Krea-2-Turbo (no CFG, guidance_scale = 1.0) and Krea-2-Raw (CFG, guidance_scale ≈ 4.5) both cache correctly and automatically. Basic Usage

Command

SGLANG_CACHE_DIT_ENABLED=true sglang serve \
  --model-path krea/Krea-2-Turbo \
  --num-gpus 1 \
  --port 30000

Measured per-image denoise speedup with the default cache settings (NVIDIA H200, 1024x1024, seed 0):

Variant	Inference steps	Denoise (no cache → cache)	Speedup
Krea-2-Turbo (no CFG)	8	1.27s → 0.92s	~1.4x
Krea-2-Raw (CFG 4.5)	50	18.0s → 6.3s	~2.9x

Caching has the most headroom on Raw’s longer schedule; the 8-step distilled Turbo has only a few cacheable steps after warmup. Advanced Usage

DBCache Parameters: DBCache controls block-level caching behavior:

Parameter	Env Variable	Default	Description
Fn	`SGLANG_CACHE_DIT_FN`	1	Number of first blocks to always compute
Bn	`SGLANG_CACHE_DIT_BN`	0	Number of last blocks to always compute
W	`SGLANG_CACHE_DIT_WARMUP`	4	Warmup steps before caching starts
R	`SGLANG_CACHE_DIT_RDT`	0.24	Residual difference threshold
MC	`SGLANG_CACHE_DIT_MC`	3	Maximum continuous cached steps

TaylorSeer Configuration: TaylorSeer improves caching accuracy using Taylor expansion (best suited to the longer Raw schedule; not recommended for the 8-step Turbo):

Parameter	Env Variable	Default	Description
Enable	`SGLANG_CACHE_DIT_TAYLORSEER`	false	Enable TaylorSeer calibrator
Order	`SGLANG_CACHE_DIT_TS_ORDER`	1	Taylor expansion order (1 or 2)

Combined Configuration Example (Krea-2-Raw, default cache settings shown explicitly):

Command

SGLANG_CACHE_DIT_ENABLED=true \
SGLANG_CACHE_DIT_FN=1 \
SGLANG_CACHE_DIT_BN=0 \
SGLANG_CACHE_DIT_WARMUP=4 \
SGLANG_CACHE_DIT_RDT=0.24 \
SGLANG_CACHE_DIT_MC=3 \
sglang serve --model-path krea/Krea-2-Raw

4.2.2 Memory & CPU Offload

Krea-2’s DiT is ~24 GB in bf16 (the bulk of the model). On memory-constrained GPUs you can keep less of it resident:

--dit-layerwise-offload: stream the DiT’s transformer blocks layer-by-layer with async host-to-device prefetch overlap, so only a small working set stays on the GPU. This is the primary way to fit Krea-2 on a single consumer / 32 GB-class card, at a modest latency cost. Tune the memory/latency trade-off with --dit-offload-prefetch-size (0.0 prefetches one layer for the lowest memory; larger values prefetch more layers — faster but more memory).
--dit-cpu-offload: keep the whole DiT in host memory. Combine it with --dit-layerwise-offload for the lowest peak GPU memory (weights stay on host and only the layers needed for the current step are brought on-device).
--text-encoder-cpu-offload: offload the Qwen3-VL text encoder (it is idle during the denoise loop).
--vae-cpu-offload: offload the VAE.
--pin-cpu-memory: pin host memory for offload. Add only as a temporary workaround if you hit CUDA error: invalid argument.

On large-VRAM GPUs (e.g. H200), keep everything resident (offloads off) for the fastest latency.

5. Benchmark

Test Environment:

Hardware: NVIDIA H200 GPU (1x)
Model: krea/Krea-2-Turbo (8 inference steps)
sglang diffusion version: 0.5.13

Server Command (used for both benchmarks below):

Command

sglang serve --model-path krea/Krea-2-Turbo --port 30000

5.1 Generate an image

Benchmark Command:

Command

python3 -m sglang.multimodal_gen.benchmarks.bench_serving \
    --model krea/Krea-2-Turbo --dataset vbench --task text-to-image \
    --num-prompts 1 --max-concurrency 1

Result:

Output

================= Serving Benchmark Result =================
Task:                                    text-to-image
Model:                                   krea/Krea-2-Turbo
Dataset:                                 vbench
--------------------------------------------------
Benchmark duration (s):                  1.56
Request rate:                            inf
Max request concurrency:                 1
Successful requests:                     1/1
--------------------------------------------------
Request throughput (req/s):              0.64
Latency Mean (s):                        1.5600
Latency Median (s):                      1.5600
Latency P99 (s):                         1.5600
--------------------------------------------------
Peak Memory Max (MB):                    37466.00
Peak Memory Mean (MB):                   37466.00
Peak Memory Median (MB):                 37466.00
============================================================

5.2 Generate images with high concurrency

Benchmark Command:

Command

python3 -m sglang.multimodal_gen.benchmarks.bench_serving \
    --model krea/Krea-2-Turbo --dataset vbench --task text-to-image \
    --num-prompts 20 --max-concurrency 20

Result:

Output

================= Serving Benchmark Result =================
Task:                                    text-to-image
Model:                                   krea/Krea-2-Turbo
Dataset:                                 vbench
--------------------------------------------------
Benchmark duration (s):                  31.47
Request rate:                            inf
Max request concurrency:                 20
Successful requests:                     20/20
--------------------------------------------------
Request throughput (req/s):              0.64
Latency Mean (s):                        16.5000
Latency Median (s):                      16.5200
Latency P99 (s):                         31.1300
--------------------------------------------------
Peak Memory Max (MB):                    37468.00
Peak Memory Mean (MB):                   37466.40
Peak Memory Median (MB):                 37466.00
============================================================

​1. Model Introduction

​2. SGLang-diffusion Installation

​3. Model Deployment

​3.1 Basic Configuration

​3.2 Configuration Tips

​3.3 Multi-GPU: tensor and sequence parallelism

​4. API Usage

​4.1 Generate an Image

​4.2 Advanced Usage

​4.2.1 Cache-DiT Acceleration

​4.2.2 Memory & CPU Offload

​5. Benchmark

​5.1 Generate an image

​5.2 Generate images with high concurrency

1. Model Introduction

2. SGLang-diffusion Installation

3. Model Deployment

3.1 Basic Configuration

3.2 Configuration Tips

3.3 Multi-GPU: tensor and sequence parallelism

4. API Usage

4.1 Generate an Image

4.2 Advanced Usage

4.2.1 Cache-DiT Acceleration

4.2.2 Memory & CPU Offload

5. Benchmark

5.1 Generate an image

5.2 Generate images with high concurrency