Skip to main content

Advanced: Multimodal Input and Image Q&A

Images enter a session as file_id after upload and review: send a mixed text-and-image message, verify recognition with a controlled image, and the current model and input-format limits.

This page covers the full chain for handing an image to an agent: upload the image, wait until review passes, send a message containing both text and the image, and verify the answer actually responds to the image content.

Scenario: a business flow submits an image and a question

A business flow receives an image (screenshot, photo, chart) and a question, hands it to an agent for an answer, and writes the answer back into the business system. The chain itself is short, with two engineering points worth attention: the image must pass upload review before sending; and whether the answer actually looked at the image must be verified — HTTP 200 and a cleanly ended turn prove nothing about image recognition.

Image input: the file_id chain

1. Upload the image

Upload via multipart and keep the returned File ID: 
curl -s -X POST "$BASE/files" \
  -H "Authorization: Bearer $DASHSCOPE_API_KEY" \
  -F "file=@red-64.png"

2. Poll until available

After upload the file enters review; poll within a deadline until the status is available: 
curl -s "$BASE/files/$FILE_ID" \
  -H "Authorization: Bearer $DASHSCOPE_API_KEY"
If the status turns into a rejection, stop the flow and do not send the message.

3. Send a mixed text-and-image message

Send one message event to the session, with text and image together in the content array: 
{
  "input": [{
    "type": "message",
    "role": "user",
    "content": [
      {"type": "text", "text": "Identify the dominant color of the image. Answer just the color name."},
      {"type": "image", "file_id": "<available_image_file_id>"}
    ]
  }]
}

4. Verify the answer

Check the assistant answer and the turn-end status. For a programmatically generated solid red PNG (64×64), the expected answer is Red — using a controlled image with a known answer is the key to this step: end_turn only says the turn ended cleanly, not that the model saw the image. Verified 2026-10-15: qwen3.8-max answered Red correctly over the file_id chain with no tool calls; auto and qwen3.7-plus also completed recognition in 2026-10-09 testing. Check the console for the current model list.
performance does not support multimodal input. In testing, performance explicitly reported it could not view the image. Use auto or a model explicitly documented to support images instead; updating an agent does not change the version an existing session has locked — create a new session for a model change to take effect.
Image input currently uses file_id, not Base64 image_data. Sending Base64 data earlier on, the request was accepted and the image content block was retained in the event history, but the model reported it could not see the image; switching to an uploaded file_id made recognition work. Re-test the Base64 path only if support is explicitly announced later.

type: "image" vs type: "file"

Both content blocks carry a file_id, but they serve different purposes:
  • type: "image": feeds the image into the model's vision channel — for "seeing", answering questions about image content;
  • type: "file" (file_id + filename): materializes the file into the session sandbox filesystem for tools to read and process — the model does not necessarily "see" the content itself.
Use the former for image Q&A; use the latter when the agent should process a file with scripts. For the mounting and reading chain once a file is in the sandbox, see Getting Started: Build a Data Analysis Agent.

Summary

  • The image chain is four steps: multipart upload → poll until available → send the mixed message → verify the answer;
  • Verify with a controlled image: only a known answer distinguishes "really recognized it" from "the turn ended cleanly";
  • Models have limits: performance does not support multimodal input; create a new session after changing models;
  • Input format is file_id: Base64 image_data is currently unsupported.

Next steps