Skip to main content

​
1. Quick start

Use a SGLang Diffusion Docker environment. Until your image includes this integration, install the model-support PR inside the container:
Install model support
You need access to the selected Hugging Face checkpoint. The picker distinguishes available implementation paths from hardware recipes that have not been measured.

​
2. Model capabilities

MOSS-Image is a text-to-image model using Qwen3.5 text conditioning and a FLUX.2 image decoder. Its native pipeline accepts natural-language prompts and structured layout descriptions through the same Images API. Structured prompts are passed to the text encoder as text, not interpreted as a separate rendering language. Choose Base for reproducing the reference comparison. The SFT identifier resolves to the same pipeline family, but its checkpoint has not been validated here. Generation is supported; image editing is not implemented. Keep the scheduler, guidance recipe, attention backend, and precision fixed when comparing outputs.

​
3. Guidance and precision

Defaults are 1024 x 1024, 50 steps, flow shift 5, CFG 3, and an empty negative prompt. Width and height must be divisible by 16. DiT and text-encoder weights use BF16; latents, CFG arithmetic, scheduler updates, and VAE decode use FP32. Autocast is disabled by the pipeline configuration; no extra CLI flag is needed. The default two-pass recipe skips unconditional layer 22 inside the slg_start=0.01, slg_stop=1.0 window. It leaves the first step at full CFG. Set uncond_skip_layers=[] for ordinary CFG. Three-pass SLG adds a skipped conditional prediction and changes the sampling algorithm; it is not a lossless speed optimization.
Three-pass request overrides
For the official pipeline’s start-at-zero behavior, explicitly set slg_start=0.0 and uncond_skip_layers=[22], including when using three-pass SLG. Guidance fields are request-scoped and do not require a server restart. quality=exact preserves the native rounding policy, but does not make different attention backends identical. quality=lossless can additionally enable shared VAE fast paths. Compare each configuration against its matched reference rather than treating either quality name as a cross-backend guarantee.

​
4. Deployment features

Component and layerwise offload reuse the standard residency manager. Start with Auto placement; use --component-residency all=resident only when weights and request activations fit. Offload saves VRAM at the cost of host RAM and transfers. The text encoder also supports layerwise offload through its native layer list. Sequence parallelism uses the shared shard/pad/gather helpers and USP attention; the text refiner stays replicated. Ulysses degree must divide the 24 DiT heads. Ring requires a backend exposing softmax LSE, such as FlashAttention. CFG parallelism uses the shared branch dispatcher, including the optional third SLG branch. Multi-GPU checkpoint recipes remain experimental until measured on the selected hardware and workload. TP, Cache-DiT, quantization, LoRA, compilation, and breakable CUDA graphs do not have validated recipes in this integration. Do not assume that a framework-wide flag establishes model support. See performance optimization for the shared feature contracts.

​
5. Batching

Dynamic batching uses the shared scheduler and is opt-in through the maximum batch size. For example, on an H200:
Compatible requests may have different prompts and seeds; resolution, step count, negative prompt, and guidance settings must match. The maximum size limits total outputs, not just HTTP requests. A request can also use n to generate multiple images from one prompt. Higher batch sizes need more activation VRAM and can increase individual-request latency; the default remains one output at a time. At quality=lossless (the default) or high, the DiT groups samples by their actual text length and restores their original output order. Different lengths run in separate groups without padding or switching attention backends. Empty negative prompts can share one group even when positive prompts differ. The text refiner retains per-sample GEMM shapes to reduce numerical drift. Batched DiT GEMM shapes can change floating-point rounding, which denoising can amplify into visible changes; cross-batch-size pixel identity is not guaranteed. quality=exact retains the per-sample DiT path instead. Neither mode promises batch-invariant text encoding or VAE decoding. Benchmark your prompts and batch sizes before deployment. On one H200, four concurrent requests with different prompts and seeds 42-45 measured the following warmed HTTP throughput. Both paths use BF16 DiT/text encoder, FP32 VAE, default attention backends, quality=lossless, resident components, maximum batch size 4, and a 20 ms batching delay. Loading is excluded; PNG encoding, transfer, and client decoding are included. Results average three measured waves at 512 pixels and two at 1024, after one warmup wave per resolution. The environment used PyTorch 2.14.1+cu130, Transformers 5.19.0, and eager execution. At 512 pixels, agreement with the per-sample DiT path ranged from 28.3 to 46.2 dB PSNR and 0.939 to 0.996 SSIM. All four 1024-pixel outputs were pixel-identical. These measure output agreement, not semantic quality or a guarantee for other prompts. Visible differences remain possible; use quality=exact to retain per-sample DiT execution. Fixed-batch repeated requests are checked separately from cross-batch agreement.

​
6. Run in ComfyUI