JoyAI-Echo - SGLang Documentation

1. Model Introduction

JoyAI-Echo is an 8-step long-form audio-video model built on LTX-2. Its paired memory bank carries decoded visual context and audio latents across prompt changes, making it strongest for multi-shot, minute-scale sequences that need continuity in both picture and soundtrack. Choose JoyEcho over a standard LTX pipeline when shots must share audiovisual memory. Its distilled 832×480 path prioritizes long-form continuity and throughput rather than the higher-resolution two-stage quality modes offered by LTX-2.3. SGLang materializes the Echo 1.0 monolithic release through the built-in JoyAI-Echo overlay. Prepare the pinned checkpoint below before running the examples.

Aspect	Standard LTX-2.3	JoyEcho
Pipeline	`LTX2Pipeline` / `LTX2TwoStageHQPipeline`	`JoyEchoPipeline` (default for this model)
Denoising	Multi-step flow matching + CFG	LTX-2 DMD distilled path (8 steps, `guidance_scale=1.0`)
Multi-shot	Not supported	Paired audio–video memory bank across shots
Sequence parallelism	LTX-2 SP (video/audio sharded)	Ulysses SP (`ulysses_degree=2`): single-shot and multi-shot + memory bank
Post-processing	Optional two-stage HQ upscaling	Per-shot mp4 output

Review the model license on the JoyAI-Echo Hugging Face page before production or commercial use. SGLang support does not grant additional model usage rights.

2. SGLang-diffusion Installation

Install SGLang with diffusion dependencies:

uv pip install "sglang[diffusion]" --prerelease=allow

For platform-specific setup, see the SGLang Diffusion installation guide.

3. Model Deployment

3.1 Prepare the Echo 1.0 checkpoint

The native overlay requires JoyAI-Echo-release.safetensors. The upstream repository’s Echo 1.5 revision does not contain that file. Download the Echo 1.0 revision into the Hugging Face cache:

JOY_ECHO_MODEL_PATH=$(python - <<'PY'
from huggingface_hub import snapshot_download

print(snapshot_download(
    repo_id="jdopensource/JoyAI-Echo",
    revision="4187f9a53c6eff3a76c51e79bd27f70d10f7591b",
    allow_patterns=["JoyAI-Echo-release.safetensors", "*.json", "*.md", "LICENSE"],
))
PY
)

Use the returned cache path as --model-path and keep --model-id jdopensource/JoyAI-Echo when using local weights. The model ID lets BCG select JoyEcho’s support policy. Initial startup also downloads and materializes the overlay and its text encoder dependencies.

3.2 Serve the model

JoyEcho uses the default JoyEchoPipeline registered for jdopensource/JoyAI-Echo. A single high-VRAM GPU (for example H100 or H200) is enough for the common 832x480 / 121-frame / 8-step setting.

sglang serve \
  --model-path "$JOY_ECHO_MODEL_PATH" \
  --model-id jdopensource/JoyAI-Echo

Optional environment variable for long runs:

export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True

For multi-GPU serving, tensor parallelism (TP) and Ulysses sequence parallelism (SP) are supported. JoyEcho SP uses an asymmetric layout: video target latents are time-sharded across ranks, while audio (including memory tokens) is replicated on every rank so cross-attention stays temporally aligned. Multi-shot runs with enable_memory_bank=true are supported on SP.

sglang serve \
  --model-path "$JOY_ECHO_MODEL_PATH" \
  --model-id jdopensource/JoyAI-Echo \
  --num-gpus 2 \
  --ulysses-degree 2

JoyEcho SP currently targets Ulysses-only parallelism (ulysses_degree=2, ring_degree=1). Ring SP is not validated for this pipeline. For sglang generate, add --num-gpus 2 --ulysses-degree 2 to the commands in section 4.

4. Model Invocation

4.1 Default sampling

Setting	Default
Resolution	832x480
Frames	121
FPS	25
Steps	8
Guidance scale	1.0
Seed	12345

4.2 Single-shot text-to-video

sglang generate \
  --model-path "$JOY_ECHO_MODEL_PATH" \
  --model-id jdopensource/JoyAI-Echo \
  --prompt "A curious raccoon walks through a sunlit forest path" \
  --height 480 --width 832 --num-frames 121 --fps 25 \
  --num-inference-steps 8 --seed 42 \
  --save-output

Disable the memory bank for standalone clips with a config file:

cat > /tmp/joy_echo_single.json <<'EOF'
{
  "prompt": "A curious raccoon walks through a sunlit forest path",
  "enable_memory_bank": false,
  "seed": 42,
  "height": 480,
  "width": 832,
  "num_frames": 121,
  "fps": 25,
  "num_inference_steps": 8
}
EOF

sglang generate --config /tmp/joy_echo_single.json \
  --model-path "$JOY_ECHO_MODEL_PATH" --model-id jdopensource/JoyAI-Echo \
  --save-output

4.3 Multi-shot generation

JoyEcho does not generate all shots in one forward pass. Each shot is one generation request. Continuity is carried by an in-process memory bank on the pipeline instance. Typical workflow:

Shot 0 — memory bank is empty; the model generates a standalone A/V clip.
After decode — decoded video frames and packed audio latents are committed to the memory bank (up to 7 slots by default).
Shot 1+ — prior-shot frames are re-encoded and prepended as a memory prefix before denoising.
Per-shot seeding — official semantics use prompt_seed = base_seed + shot_index.

Pass multiple prompts as a list in a config file:

cat > /tmp/joy_echo_4shot.json <<'EOF'
{
  "prompt": [
    "Shot 0: A raccoon wakes up in a cozy attic.",
    "Shot 1: The raccoon climbs down and opens the back door.",
    "Shot 2: It walks through a rainy alley under neon signs.",
    "Shot 3: The raccoon finds a warm bakery window and stops."
  ],
  "enable_memory_bank": true,
  "reset_memory_bank": true,
  "seed": 42,
  "height": 480,
  "width": 832,
  "num_frames": 121,
  "fps": 25,
  "num_inference_steps": 8
}
EOF

sglang generate --config /tmp/joy_echo_4shot.json \
  --model-path "$JOY_ECHO_MODEL_PATH" --model-id jdopensource/JoyAI-Echo \
  --save-output

You can also pass prompts from a text file (one prompt per line) with --prompt-path:

sglang generate \
  --model-path "$JOY_ECHO_MODEL_PATH" \
  --model-id jdopensource/JoyAI-Echo \
  --prompt-path /tmp/joy_echo_shots.txt \
  --seed 42 \
  --height 480 --width 832 --num-frames 121 --fps 25 \
  --num-inference-steps 8 \
  --save-output

4.4 Memory bank controls

Parameter	Default	Meaning
`enable_memory_bank`	`true`	Read/write the paired A/V memory bank between shots.
`reset_memory_bank`	`true`	Clear the bank and shot counter at the start of a new session (`request_id` change or first shot).

Set enable_memory_bank=false when you want independent shots without cross-shot continuity.

4.5 Measured two-H200 single-shot configuration

For short independent clips, keep the text and audio/video components on GPU with --component-residency=all=resident. Full-stage profiles showed that this removes repeated host-to-device weight copies between component uses. The following configuration was measured on two H200s with Ulysses degree 2, TP1, PyTorch 2.11.0+cu130, 640x384, 33 frames, 8 steps and seed 42. It disables compilation and the memory bank:

cat > joy_echo_h200.json <<'EOF'
{"enable_memory_bank": false}
EOF

CUDA_VISIBLE_DEVICES=0,1 sglang generate \
  --model-path "$JOY_ECHO_MODEL_PATH" --model-id jdopensource/JoyAI-Echo \
  --config joy_echo_h200.json --prompt "A curious raccoon" \
  --width 640 --height 384 --num-frames 33 --num-inference-steps 8 --seed 42 \
  --num-gpus 2 --ulysses-degree 2 \
  --performance-mode manual --enable-torch-compile=false --quality lossless \
  --component-residency=all=resident --warmup-mode request \
  --save-output --perf-dump-path joy_echo_h200.json.perf

For BCG, add --enable-breakable-cuda-graph --warmup-resolutions 640x384 --warmup-num-frames 33. Check for successful [Diffusion BCG] captured logs and absence of request signature misses. Keep the same model ID, resolution, frame count and quality as warmup. Two fresh-process saved requests per configuration, after request warmup:

Lossless mode	Auto residency E2E	All resident E2E	Reduction	Peak reserved per rank, auto → resident
Eager	2.563 s	2.366 s	7.65%	46.43 → 69.32 GiB
BCG	1.229 s	1.058 s	13.88%	48.77–48.88 → 69.32 GiB

Loading, warmup and profiling are excluded from these timings. The paired eager profiles remove 71 pinned host-to-device copies (13.07 GB, 278.86 ms on the profiled rank), with the same 77,045 kernel launches. All eight lossless outputs have pixel-identical video frames; audio differences are comparable to baseline repeat variability. These results cover this compact single-shot workload, rather than the default 121-frame or multi-shot memory-bank workload. Use quality=lossless for this recipe. High-mode output did not pass the separate quality comparison, and high + BCG is rejected by the runtime. If a larger request exceeds available memory, return to auto residency or keep only selected components resident.

5. Practical Tips

Use --num-inference-steps 8 and --guidance-scale 1.0 to match the official JoyEcho DMD distilled path.
Multi-shot prompts can be passed as a prompt list, via prompt_path, or as sequential API calls on the same server instance.
The memory bank caps at 7 slots; from shot 8 onward the oldest slots roll off.
For 2-GPU latency, try Ulysses SP (--num-gpus 2 --ulysses-degree 2) on both single-shot and multi-shot runs. Use TP when you need a different sharding strategy or more than two GPUs.
Set PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True for long multi-shot SP sessions.
JoyEcho outputs per-shot mp4 files with synchronized audio. There is no built-in two-stage HQ upscaling path like LTX-2.3 HQ.

​1. Model Introduction

​2. SGLang-diffusion Installation

​3. Model Deployment

​3.1 Prepare the Echo 1.0 checkpoint

​3.2 Serve the model

​4. Model Invocation

​4.1 Default sampling

​4.2 Single-shot text-to-video

​4.3 Multi-shot generation

​4.4 Memory bank controls

​4.5 Measured two-H200 single-shot configuration

​5. Practical Tips

1. Model Introduction

2. SGLang-diffusion Installation

3. Model Deployment

3.1 Prepare the Echo 1.0 checkpoint

3.2 Serve the model

4. Model Invocation

4.1 Default sampling

4.2 Single-shot text-to-video

4.3 Multi-shot generation

4.4 Memory bank controls

4.5 Measured two-H200 single-shot configuration

5. Practical Tips