> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sglang.io/llms.txt
> Use this file to discover all available pages before exploring further.

# JoyAI-Echo

> Run JoyAI-Echo multi-shot audio–video generation with SGLang Diffusion.

export const DiffusionModelTags = ({tags = []}) => {
  const normalizedTags = Array.isArray(tags) ? tags : [tags];
  return <div className="not-prose sgd-model-tags">
      {normalizedTags.map(tag => <span key={tag} className="sgd-chip">
          {tag}
        </span>)}
    </div>;
};

<DiffusionModelTags tags={["video + audio", "multi-shot", "minute-scale", "memory bank", "8-step"]} />

## 1. Model Introduction

[JoyAI-Echo](https://huggingface.co/jdopensource/JoyAI-Echo) is an 8-step long-form audio-video model built on LTX-2. Its paired memory bank carries decoded visual context and audio latents across prompt changes, making it strongest for multi-shot, minute-scale sequences that need continuity in both picture and soundtrack.

Choose JoyEcho over a standard LTX pipeline when shots must share audiovisual memory. Its distilled 832×480 path prioritizes long-form continuity and throughput rather than the higher-resolution two-stage quality modes offered by LTX-2.3.

SGLang materializes the Echo 1.0 monolithic release through the built-in [JoyAI-Echo overlay](https://huggingface.co/Niehen6174/JoyAI-Echo-overlay). Prepare the pinned checkpoint below before running the examples.

| Aspect               | Standard LTX-2.3                          | JoyEcho                                                                   |
| -------------------- | ----------------------------------------- | ------------------------------------------------------------------------- |
| Pipeline             | `LTX2Pipeline` / `LTX2TwoStageHQPipeline` | `JoyEchoPipeline` (default for this model)                                |
| Denoising            | Multi-step flow matching + CFG            | LTX-2 DMD distilled path (8 steps, `guidance_scale=1.0`)                  |
| Multi-shot           | Not supported                             | Paired audio–video memory bank across shots                               |
| Sequence parallelism | LTX-2 SP (video/audio sharded)            | Ulysses SP (`ulysses_degree=2`): single-shot and multi-shot + memory bank |
| Post-processing      | Optional two-stage HQ upscaling           | Per-shot mp4 output                                                       |

<Warning>
  Review the model license on the [JoyAI-Echo Hugging Face page](https://huggingface.co/jdopensource/JoyAI-Echo) before production or commercial use. SGLang support does not grant additional model usage rights.
</Warning>

## 2. SGLang-diffusion Installation

Install SGLang with diffusion dependencies:

```bash theme={null}
uv pip install "sglang[diffusion]" --prerelease=allow
```

For platform-specific setup, see the [SGLang Diffusion installation guide](/docs/sglang-diffusion/installation).

## 3. Model Deployment

### 3.1 Prepare the Echo 1.0 checkpoint

The native overlay requires `JoyAI-Echo-release.safetensors`. The upstream repository's Echo 1.5 revision does not contain that file. Download the [Echo 1.0 revision](https://huggingface.co/jdopensource/JoyAI-Echo/tree/4187f9a53c6eff3a76c51e79bd27f70d10f7591b) into the Hugging Face cache:

```bash theme={null}
JOY_ECHO_MODEL_PATH=$(python - <<'PY'
from huggingface_hub import snapshot_download

print(snapshot_download(
    repo_id="jdopensource/JoyAI-Echo",
    revision="4187f9a53c6eff3a76c51e79bd27f70d10f7591b",
    allow_patterns=["JoyAI-Echo-release.safetensors", "*.json", "*.md", "LICENSE"],
))
PY
)
```

Use the returned cache path as `--model-path` and keep `--model-id jdopensource/JoyAI-Echo` when using local weights. The model ID lets BCG select JoyEcho's support policy. Initial startup also downloads and materializes the overlay and its text encoder dependencies.

### 3.2 Serve the model

JoyEcho uses the default `JoyEchoPipeline` registered for `jdopensource/JoyAI-Echo`. A single high-VRAM GPU (for example H100 or H200) is enough for the common 832x480 / 121-frame / 8-step setting.

```bash theme={null}
sglang serve \
  --model-path "$JOY_ECHO_MODEL_PATH" \
  --model-id jdopensource/JoyAI-Echo
```

Optional environment variable for long runs:

```bash theme={null}
export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
```

For multi-GPU serving, tensor parallelism (TP) and **Ulysses sequence parallelism (SP)** are supported. JoyEcho SP uses an **asymmetric layout**: video target latents are time-sharded across ranks, while audio (including memory tokens) is **replicated** on every rank so cross-attention stays temporally aligned. Multi-shot runs with `enable_memory_bank=true` are supported on SP.

```bash theme={null}
sglang serve \
  --model-path "$JOY_ECHO_MODEL_PATH" \
  --model-id jdopensource/JoyAI-Echo \
  --num-gpus 2 \
  --ulysses-degree 2
```

<Note>
  JoyEcho SP currently targets **Ulysses-only** parallelism (`ulysses_degree=2`, `ring_degree=1`). Ring SP is not validated for this pipeline. For `sglang generate`, add `--num-gpus 2 --ulysses-degree 2` to the commands in section 4.
</Note>

## 4. Model Invocation

### 4.1 Default sampling

| Setting        | Default |
| -------------- | ------- |
| Resolution     | 832x480 |
| Frames         | 121     |
| FPS            | 25      |
| Steps          | 8       |
| Guidance scale | 1.0     |
| Seed           | 12345   |

### 4.2 Single-shot text-to-video

```bash theme={null}
sglang generate \
  --model-path "$JOY_ECHO_MODEL_PATH" \
  --model-id jdopensource/JoyAI-Echo \
  --prompt "A curious raccoon walks through a sunlit forest path" \
  --height 480 --width 832 --num-frames 121 --fps 25 \
  --num-inference-steps 8 --seed 42 \
  --save-output
```

Disable the memory bank for standalone clips with a config file:

```bash theme={null}
cat > /tmp/joy_echo_single.json <<'EOF'
{
  "prompt": "A curious raccoon walks through a sunlit forest path",
  "enable_memory_bank": false,
  "seed": 42,
  "height": 480,
  "width": 832,
  "num_frames": 121,
  "fps": 25,
  "num_inference_steps": 8
}
EOF

sglang generate --config /tmp/joy_echo_single.json \
  --model-path "$JOY_ECHO_MODEL_PATH" --model-id jdopensource/JoyAI-Echo \
  --save-output
```

### 4.3 Multi-shot generation

JoyEcho does **not** generate all shots in one forward pass. Each shot is one generation request. Continuity is carried by an in-process **memory bank** on the pipeline instance.

Typical workflow:

1. **Shot 0** — memory bank is empty; the model generates a standalone A/V clip.
2. **After decode** — decoded video frames and packed audio latents are committed to the memory bank (up to 7 slots by default).
3. **Shot 1+** — prior-shot frames are re-encoded and prepended as a memory prefix before denoising.
4. **Per-shot seeding** — official semantics use `prompt_seed = base_seed + shot_index`.

Pass multiple prompts as a list in a config file:

```bash theme={null}
cat > /tmp/joy_echo_4shot.json <<'EOF'
{
  "prompt": [
    "Shot 0: A raccoon wakes up in a cozy attic.",
    "Shot 1: The raccoon climbs down and opens the back door.",
    "Shot 2: It walks through a rainy alley under neon signs.",
    "Shot 3: The raccoon finds a warm bakery window and stops."
  ],
  "enable_memory_bank": true,
  "reset_memory_bank": true,
  "seed": 42,
  "height": 480,
  "width": 832,
  "num_frames": 121,
  "fps": 25,
  "num_inference_steps": 8
}
EOF

sglang generate --config /tmp/joy_echo_4shot.json \
  --model-path "$JOY_ECHO_MODEL_PATH" --model-id jdopensource/JoyAI-Echo \
  --save-output
```

You can also pass prompts from a text file (one prompt per line) with `--prompt-path`:

```bash theme={null}
sglang generate \
  --model-path "$JOY_ECHO_MODEL_PATH" \
  --model-id jdopensource/JoyAI-Echo \
  --prompt-path /tmp/joy_echo_shots.txt \
  --seed 42 \
  --height 480 --width 832 --num-frames 121 --fps 25 \
  --num-inference-steps 8 \
  --save-output
```

### 4.4 Memory bank controls

| Parameter            | Default | Meaning                                                                                            |
| -------------------- | ------- | -------------------------------------------------------------------------------------------------- |
| `enable_memory_bank` | `true`  | Read/write the paired A/V memory bank between shots.                                               |
| `reset_memory_bank`  | `true`  | Clear the bank and shot counter at the start of a new session (`request_id` change or first shot). |

Set `enable_memory_bank=false` when you want independent shots without cross-shot continuity.

### 4.5 Measured two-H200 single-shot configuration

For short independent clips, keep the text and audio/video components on GPU with `--component-residency=all=resident`. Full-stage profiles showed that this removes repeated host-to-device weight copies between component uses.

The following configuration was measured on two H200s with Ulysses degree 2, TP1, PyTorch 2.11.0+cu130, 640x384, 33 frames, 8 steps and seed 42. It disables compilation and the memory bank:

```bash theme={null}
cat > joy_echo_h200.json <<'EOF'
{"enable_memory_bank": false}
EOF

CUDA_VISIBLE_DEVICES=0,1 sglang generate \
  --model-path "$JOY_ECHO_MODEL_PATH" --model-id jdopensource/JoyAI-Echo \
  --config joy_echo_h200.json --prompt "A curious raccoon" \
  --width 640 --height 384 --num-frames 33 --num-inference-steps 8 --seed 42 \
  --num-gpus 2 --ulysses-degree 2 \
  --performance-mode manual --enable-torch-compile=false --quality lossless \
  --component-residency=all=resident --warmup-mode request \
  --save-output --perf-dump-path joy_echo_h200.json.perf
```

For BCG, add `--enable-breakable-cuda-graph --warmup-resolutions 640x384 --warmup-num-frames 33`. Check for successful `[Diffusion BCG] captured` logs and absence of request signature misses. Keep the same model ID, resolution, frame count and quality as warmup.

Two fresh-process saved requests per configuration, after request warmup:

| Lossless mode | Auto residency E2E | All resident E2E | Reduction | Peak reserved per rank, auto → resident |
| ------------- | -----------------: | ---------------: | --------: | --------------------------------------: |
| Eager         |            2.563 s |          2.366 s |     7.65% |                       46.43 → 69.32 GiB |
| BCG           |            1.229 s |          1.058 s |    13.88% |                 48.77–48.88 → 69.32 GiB |

Loading, warmup and profiling are excluded from these timings. The paired eager profiles remove 71 pinned host-to-device copies (13.07 GB, 278.86 ms on the profiled rank), with the same 77,045 kernel launches. All eight lossless outputs have pixel-identical video frames; audio differences are comparable to baseline repeat variability. These results cover this compact single-shot workload, rather than the default 121-frame or multi-shot memory-bank workload.

Use `quality=lossless` for this recipe. High-mode output did not pass the separate quality comparison, and high + BCG is rejected by the runtime. If a larger request exceeds available memory, return to auto residency or keep only selected components resident.

## 5. Practical Tips

* Use `--num-inference-steps 8` and `--guidance-scale 1.0` to match the official JoyEcho DMD distilled path.
* Multi-shot prompts can be passed as a `prompt` list, via `prompt_path`, or as sequential API calls on the same server instance.
* The memory bank caps at **7 slots**; from shot 8 onward the oldest slots roll off.
* For **2-GPU latency**, try **Ulysses SP** (`--num-gpus 2 --ulysses-degree 2`) on both single-shot and multi-shot runs. Use **TP** when you need a different sharding strategy or more than two GPUs.
* Set `PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True` for long multi-shot SP sessions.
* JoyEcho outputs per-shot mp4 files with synchronized audio. There is no built-in two-stage HQ upscaling path like LTX-2.3 HQ.