Vision

Models with vision can read screenshots, charts, documents and photographs alongside your text.

Chat Completions#

Use the array form of content and mix text with image_url parts. Order matters: put the image before the question if the question refers to it.

Image by URL

{
  "model": "openai/gpt-5.6-sol",
  "messages": [
    {
      "role": "user",
      "content": [
        { "type": "image_url", "image_url": { "url": "https://example.com/receipt.jpg" } },
        { "type": "text", "text": "What is the total on this receipt?" }
      ]
    }
  ]
}

Image from disk, as a data URL

import { readFileSync } from "node:fs";

const bytes = readFileSync("./receipt.jpg");
const dataUrl = `data:image/jpeg;base64,${bytes.toString("base64")}`;

const res = await client.chat.completions.create({
  model: "openai/gpt-5.6-sol",
  messages: [
    {
      role: "user",
      content: [
        { type: "image_url", image_url: { url: dataUrl } },
        { type: "text", text: "What is the total on this receipt?" },
      ],
    },
  ],
});

Messages#

The Anthropic dialect uses explicit source objects.

Base64 source

{
  "model": "anthropic/claude-opus-5",
  "max_tokens": 1024,
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "image",
          "source": {
            "type": "base64",
            "media_type": "image/jpeg",
            "data": "/9j/4AAQSkZJRgABAQAA..."
          }
        },
        { "type": "text", "text": "What is the total on this receipt?" }
      ]
    }
  ]
}

The media type is detected from the bytes

If your declared media_type disagrees with the actual file, xKiro trusts the bytes. A JPEG labelled image/png is a common source of provider errors, and this removes that class of failure entirely.

Sending an image to a model without vision#

The request is not rejected. xKiro removes the image and replaces it with a note telling the model it could not see an image, so the model says so instead of inventing a description.

Images in the current turn are replaced with an instruction not to guess, and to suggest switching to a vision model.
Images earlier in the conversation are replaced with a short marker so the history still makes sense.

This is deliberate. Tools and editor extensions frequently attach screenshots the user never intended to send; failing the whole request over an incidental attachment would throw away the question they actually asked.

Formats and limits

JPEG, PNG, GIF and WebP. Non-image files are rejected with 400.
Images consume tokens roughly in proportion to their pixel area. Downscaling a screenshot before sending is the cheapest optimisation available.
Very large images may be resized by the provider. If fine print matters, crop to the region of interest rather than sending the full page.

Getting better answers#

Ask a specific question. "Describe this image" produces generic prose.
For documents and tables, ask for structured output — a JSON shape you name — instead of a paragraph you then have to parse.
Send several images in one message when they should be compared; send them separately when they are unrelated.

To generate images rather than read them, see Image generation.