PaddleOCR-VL

​

Deployment

pip install --upgrade pip
pip install uv
uv pip install sglang

docker pull lmsysorg/sglang:dev

​

Playground

​

1. Model Introduction

​

2. Configuration Tips

• Page resolution is the main cost knob. The vision tower and the prefill both scale with the patch count of a page. max_pixels is expressed in 28x28 units (patch size 14 with a 2x2 merge), so max_pixels / 784 is the image-token budget per page. The checkpoint’s own default is 1280 tokens; the Page Resolution selector in the Deploy panel emits the corresponding --mm-process-config value. Lower it for clean born-digital PDFs, raise it for dense scans and small print.
• Prompt selects the task. PaddleOCR-VL is prompt-conditioned rather than instruction-following — use the exact task strings in §3.1. A free-form question will not behave like a chat model.
• Leave --trust-remote-code off. The checkpoints ship their own configuration_paddleocr_vl.py / processing_paddleocr_vl.py, but transformers 5.12 supports paddleocr_vl natively — and the bundled remote image processor is the slower of the two implementations (measured 87.4 ms vs 39.1 ms per 1080p page). Passing the flag pins SGLang to the remote copy. Serving without it produced byte-identical OCR output on every page we checked and about 5% more requests per second at 32-way concurrency.
• Preprocessing is parallelized for you. A full-resolution page costs tens of milliseconds of CPU to resize, normalize and patchify, which caps throughput long before the GPU saturates, so this model runs the image processor across several workers by default. --mm-processor-worker-num overrides the count; raising it past the default did not help in our measurements.
• Keep the radix cache on for repeated pages. Unlike whole-document batch OCR over unique scans, a workload that re-asks about the same page (different task prompts on one image) reuses the image prefix. Add --disable-radix-cache only if every request carries a different page.
• The saturated-throughput flags earn their place. A page is ~2700 tokens, so the default 8192-token prefill budget packs only three of them into a forward. Raising it to 16384 and letting decode ride along in the same batch (--enable-mixed-chunk, --num-continuous-decode-steps 2) measured +11% requests per second at 32-way concurrency and cut queued TTFT by 23%, with single-stream latency unchanged. Measured on an H200; on a smaller card lower --chunked-prefill-size until it fits.
• Prefill CUDA graph is on for this model. SGLang normally switches the breakable prefill graph off for every multimodal architecture; PaddleOCR-VL is allowlisted back in, which is worth 16.1 ms → 11.5 ms of single-stream TTFT on text-only prompts. Image-carrying batches are rejected at graph replay and run eager, so this helps mixed and text traffic, not pure page parsing. No flag needed.
• Tensor parallelism is optional. The weights are under 2 GB in BF16; TP>1 only shortens the vision-encoder and prefill critical path, at the cost of a collective per layer. Measure before adopting it.
• Context length. The backbone advertises 131072 positions, but a parsed page rarely needs more than a few thousand tokens. The recipe pins --context-length 16384 so the KV pool stays small and concurrency stays high; raise it only if you batch many pages into one request.

​

Measured on one H200

​

3. Advanced Usage

​

3.1 Task prompts

from openai import OpenAI

client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")

response = client.chat.completions.create(
    model="PaddlePaddle/PaddleOCR-VL-1.6",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "OCR:"},
                {
                    "type": "image_url",
                    "image_url": {"url": "https://example.com/your_page.png"},
                },
            ],
        }
    ],
    max_tokens=2048,
)

print(response.choices[0].message.content)

Pending update — paste the server's verbatim output for your page here.

​

3.2 Parsing a multi-page document

import base64
from concurrent.futures import ThreadPoolExecutor

import pymupdf
from openai import OpenAI

client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")


def render(page, dpi=200):
    pixmap = page.get_pixmap(dpi=dpi)
    return base64.b64encode(pixmap.tobytes("png")).decode("ascii")


def parse(page_png_b64):
    response = client.chat.completions.create(
        model="PaddlePaddle/PaddleOCR-VL-1.6",
        messages=[
            {
                "role": "user",
                "content": [
                    {"type": "text", "text": "OCR:"},
                    {
                        "type": "image_url",
                        "image_url": {
                            "url": f"data:image/png;base64,{page_png_b64}"
                        },
                    },
                ],
            }
        ],
        max_tokens=2048,
    )
    return response.choices[0].message.content


document = pymupdf.open("your_document.pdf")
pages = [render(page) for page in document]

with ThreadPoolExecutor(max_workers=16) as pool:
    for index, text in enumerate(pool.map(parse, pages)):
        print(f"--- page {index + 1} ---")
        print(text)

Pending update — paste the server's verbatim output for your document here.