OCR & document extraction

There are two ways to pull text out of documents on Gopura: point any vision chat model at an image (priced per token), or use the dedicated OCR endpoint (per-page pricing, PDFs natively).

Recipe: OCR with a vision model

Every vision-capable chat model doubles as an OCR engine — send the document as an image and ask for an exact transcription. Modern vision models beat classic OCR on layout, tables, and handwriting, and the cheap ones make it almost free: reading a full receipt costs around $0.0001 with alibaba/qwen3.7-flash.

curl https://api.gopura.net/v1/chat/completions \
  -H "Authorization: Bearer $GOPURA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "alibaba/qwen3.7-flash",
    "messages": [{
      "role": "user",
      "content": [
        {"type": "text", "text": "Transcribe ALL text in this image exactly as written, line by line. Output only the text."},
        {"type": "image_url", "image_url": {"url": "data:image/png;base64,<BASE64>"}}
      ]
    }]
  }'

Or with the OpenAI SDK — no code changes beyond the base URL:

from openai import OpenAI
import base64

client = OpenAI(base_url="https://api.gopura.net/v1", api_key=GOPURA_API_KEY)

image_b64 = base64.b64encode(open("receipt.png", "rb").read()).decode()
result = client.chat.completions.create(
    model="alibaba/qwen3.7-flash",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "Transcribe ALL text exactly. Preserve the layout as markdown."},
            {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{image_b64}"}},
        ],
    }],
)
print(result.choices[0].message.content)
print(result.usage.cost)  # exact billed cost, in-band

Prompt tips that matter for accuracy:

• Say "exactly as written" — otherwise models may paraphrase or fix typos.
• Ask for markdown when the document has tables; vision models reproduce table structure well.
• For structured extraction (dates, totals, line items), ask for JSON and use a model with response_format support — you get OCR + parsing in one call.

Good model picks (qwen3.7-flash is probe-verified — word-perfect on a printed invoice test):

PDFs: the chat endpoint takes images only. Rasterize PDF pages to images client-side (e.g. pdftoppm -png doc.pdf page) and send one message per page — or use the dedicated OCR endpoint below, which takes PDFs natively.

Dedicated OCR API

For document pipelines, POST /v1/ocr is backed by a purpose-built OCR model (mistral/mistral-ocr, Mistral OCR 4.1): PDFs and images in, clean markdown out per page, with tables, headers, and embedded images preserved — billed per processed page ($0.005), exact cost in-band as everywhere on Gopura.

curl https://api.gopura.net/v1/ocr \
  -H "Authorization: Bearer $GOPURA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mistral/mistral-ocr",
    "document": {
      "type": "document_url",
      "document_url": "data:application/pdf;base64,<BASE64_PDF>"
    }
  }'

{
  "pages": [
    { "index": 0, "markdown": "# Invoice 4217\n\n| Item | Price |…", "images": [], "dimensions": {…} }
  ],
  "model": "mistral/mistral-ocr",
  "usage_info": { "pages_processed": 3, "doc_size_bytes": 182034 },
  "usage": { "pages": 3, "cost": 0.015 }
}

document.type is document_url for PDFs (https or data URL) or image_url for single images (billed as one page). Optional passthroughs: pages (process a subset), table_format ("markdown" or "html"), include_image_base64 (return embedded figures).

Engines: Mistral and Datalab

Two OCR engines run behind the same wire — pick by model id:

Datalab models take the same document input (https URLs or base64 data URLs) and add two extras: the response carries quality_score — the engine's 0–5 parse-quality self-score — and requests may pin "processing_location": "us" | "eu" for data-residency control. The table_format and include_image_base64 options are Mistral-only (Datalab requests naming them get a clean 400); embedded-figure extraction on Datalab models is not supported yet. Long documents on :accurate can take a while — the gateway enforces a 180-second ceiling and returns a 504 with nothing billed if the engine blows through it.

One billing quirk, mirrored honestly: Datalab prices each request rounded up to the whole cent, and Gopura bills the same way for these models (margin on top). A 1-page datalab/marker call is $0.0125; ten balanced pages are $0.05 — the listed per-page rate is exact whenever pages × rate lands on a cent boundary.

Layout geometry (include_layout, Datalab only)

Set "include_layout": true on a Datalab model and each page in the response also carries its pixel dimensions and the document's layout blocks — each with a type, a bounding box, and its text. Useful for drawing regions over a preview, or locating which part of a page a value came from. It costs the same as a normal call (still billed per page) and returns the page text too.

curl https://api.gopura.net/v1/ocr \
  -H "Authorization: Bearer $GOPURA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "datalab/marker",
    "document": { "type": "image_url", "image_url": "data:image/png;base64,<BASE64>" },
    "include_layout": true
  }'

{
  "pages": [
    {
      "index": 0,
      "markdown": "…",
      "dimensions": { "width": 1624, "height": 2100 },
      "blocks": [
        { "type": "SectionHeader", "bbox": [34, 42, 583, 84], "text": "Weekly timesheet" },
        { "type": "Table", "bbox": [34, 170, 698, 375], "text": "| Day | Hours |…" }
      ]
    }
  ],
  "model": "datalab/marker",
  "usage": { "pages": 1, "cost": 0.0125 }
}

bbox is [x0, y0, x1, y1] in pixels, in the frame given by that page's dimensions — scale those onto your rendered preview to place the boxes. The blocks are Datalab's layout regions (headings, text lines, tables, table cells), not fields parsed by an extraction schema — for structured fields use /v1/extract or a document pack. include_layout is Datalab-only; a Mistral request naming it gets a clean 400.

OCR models carry pricing.page in the live catalog — filter for them programmatically:

curl -s https://api.gopura.net/v1/models \
  | jq '.data[] | select(.pricing.page != null) | {id, pricing}'