Video Understanding

The video understanding model can answer based on the video you provide, supporting both video URL and Base64 encoding as input methods, and is suitable for scenarios such as video analysis.

Quick Start

For preparations such as obtaining an API Key, please refer to First API Call.

Quickly experience the video understanding effect by passing the model through the video URL method. The sample code is as follows.

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ.get("MIMO_API_KEY"),
    base_url="https://api.xiaomimimo.com/v1"
)

completion = client.chat.completions.create(
    model="mimo-v2.6-pro",
    messages=[
        {
            "role": "system",
            "content": "You are MiMo, an AI assistant developed by Xiaomi. Today is date: Tuesday, December 16, 2025. Your knowledge cutoff date is December 2024."
        },
        {
            "role": "user",
            "content": [
                {
                    "type": "video_url",
                    "video_url": {
                        "url": "https://example-files.cnbj1.mi-fds.com/example-files/video/video_example.mp4"
                    },
                    "fps": 2,
                    "media_resolution": "default"
                },
                {
                    "type": "text",
                    "text": "please describe the content of the video"
                }
            ]
        }
    ],
    max_completion_tokens=1024
)

print(completion.model_dump_json())

Response

{
    "id": "fbfdb524-f4c8-41e6-b0a3-31520f6198c7_0b8b3ccec31e40868b76109308ed9329",
    "choices": [
        {
            "finish_reason": "stop",
            "index": 0,
            "message": {
                "content": "The video features a small, white kitten sitting on a polished wooden floor indoors. The scene is brightly lit by natural sunlight streaming in from a window on the left, which is partially covered by a beige curtain. The light casts distinct shadows on the floorboards. To the right, a large green potted plant (resembling a Monstera) is visible in the background. The kitten, which has striking blue eyes, sits facing forward but glances slightly to the side. It briefly opens its mouth, as if meowing or yawning. The overall atmosphere is warm, calm, and sunny.",
                "role": "assistant",
                "tool_calls": null,
                "reasoning_content": "The user wants a description of the video content.\n\n1.  **Analyze the video:**\n    *   **Subject:** A small, white kitten.\n    *   **Setting:** Indoors, near a window. There's a wooden floor. A curtain hangs to the left. A potted plant (Monstera) is visible in the background to the right.\n    *   **Lighting:** Bright natural sunlight is streaming in from the left (likely a window), casting strong shadows and highlights on the floor and the kitten.\n    *   **Action:** The kitten is sitting on the floor. It's looking slightly to its left (viewer's right). It opens its mouth briefly, possibly meowing or yawning, but there's no audio. Then it looks forward again. It's a very short clip, essentially a still shot with minor movement.\n    *   **Details:** Blue eyes, white fur, pink nose. The curtain is beige/white striped. The floor is polished wood.\n\n2.  **Draft the description:**\n    *   Start with the main subject: A small white kitten.\n    *   Describe the location: Sitting on a wooden floor near a window.\n    *   Describe the lighting: Sunlight is streaming in, creating a warm atmosphere.\n    *   Describe the background elements: A curtain on the left, a green plant on the right.\n    *   Describe the kitten's features: White fur, blue eyes.\n    *   Describe the action: The kitten looks around and opens its mouth slightly.\n\n3.  **Refine the description:**\n    *   \"The video features a small white kitten with blue eyes sitting on a polished wooden floor.\"\n    *   \"The setting appears to be a cozy room with bright sunlight streaming in through a window on the left.\"\n    *   \"A beige curtain hangs beside the window.\"\n    *   \"To the right, a green potted plant is visible in the background.\"\n    *   \"The kitten is facing forward but looks slightly to the side, opening its mouth briefly as if meowing or yawning.\"\n    *   \"The lighting creates distinct shadows on the floor.\"\n\n4.  **Final Polish:** Combine into a coherent paragraph. Ensure it covers the visual aspects clearly.\n\n    *   *Draft:* The video shows a white kitten with bright blue eyes sitting on a shiny wooden floor. Sunlight is pouring in from a window on the left, illuminating the kitten and the floor. There is a curtain to the left and a large leafy plant in the background on the right. The kitten opens its mouth slightly and looks around. The scene is calm and well-lit.\n\n    *   *More detailed:* The video captures a serene moment featuring a small, white kitten sitting on a polished wooden floor bathed in natural sunlight. The light streams in from a window on the left, partially covered by a beige curtain, casting long shadows across the floor. To the right, the large green leaves of a monstera plant add a touch of nature to the indoor setting. The kitten, with its striking blue eyes, faces the camera but glances to the side, briefly opening its mouth as if letting out a soft meow or a yawn. The overall atmosphere is warm and peaceful.\n\n5.  **Output generation:** Present the description clearly to the user."
            }
        }
    ],
    "created": 1790008279,
    "model": "mimo-v2.6-pro",
    "object": "chat.completion",
    "usage": {
        "completion_tokens": 807,
        "prompt_tokens": 1260,
        "total_tokens": 2067,
        "completion_tokens_details": {
            "reasoning_tokens": 684
        },
        "prompt_tokens_details": {
            "audio_tokens": 19,
            "cached_tokens": 1216,
            "video_tokens": 1144
        }
    }
}

Supported models

Currently, the mimo-v2.6-flash, mimo-v2.6-pro, mimo-v2.6-pro-ultraspeed models are supported.

Video Input Method

Supported video input methods are as follows:

Video URL Input: A publicly accessible video URL address must be provided.
Base64 Encoding Input: Convert the video to a Base64-encoded string before inputting it.

Video URL Input

Videos can be directly passed in via a publicly accessible video URL address, which is suitable for scenarios where the video is already stored in a publicly accessible environment. The size of a single video file cannot exceed 300 MB.

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ.get("MIMO_API_KEY"),
    base_url="https://api.xiaomimimo.com/v1"
)

completion = client.chat.completions.create(
    model="mimo-v2.6-pro",
    messages=[
        {
            "role": "system",
            "content": "You are MiMo, an AI assistant developed by Xiaomi. Today is date: Tuesday, December 16, 2025. Your knowledge cutoff date is December 2024."
        },
        {
            "role": "user",
            "content": [
                {
                    "type": "video_url",
                    "video_url": {
                        "url": "https://example-files.cnbj1.mi-fds.com/example-files/video/video_example.mp4"
                    },
                    "fps": 2,
                    "media_resolution": "default"
                },
                {
                    "type": "text",
                    "text": "please describe the content of the video"
                }
            ]
        }
    ],
    max_completion_tokens=1024
)

print(completion.model_dump_json())

Base64 Encoding Input

Convert the video file to a Base64-encoded string and then pass it in, which is suitable for scenarios where the video cannot be accessed via a public network URL. The size of the converted Base64-encoded string cannot exceed 50 MB.

Please include the prefix before Base64 encoding: data:{MIME_TYPE};base64,$BASE64_VIDEO

{MIME_TYPE}: The MIME type (media type) of the video, used to identify the video format, and needs to be replaced with the MIME value corresponding to the actual video.
$BASE64_VIDEO: Pure Base64-encoded string of the video file (without any prefix).

In the following example, both {MIME_TYPE} and $BASE64_VIDEO are placeholders, please replace them with actual valid data before running.

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ.get("MIMO_API_KEY"),
    base_url="https://api.xiaomimimo.com/v1"
)

completion = client.chat.completions.create(
    model="mimo-v2.6-pro",
    messages=[
        {
            "role": "system",
            "content": "You are MiMo, an AI assistant developed by Xiaomi. Today is date: Tuesday, December 16, 2025. Your knowledge cutoff date is December 2024."
        },
        {
            "role": "user",
            "content": [
                {
                    "type": "video_url",
                    "video_url": {
                        "url": "data:{MIME_TYPE};base64,$BASE64_VIDEO"
                    },
                    "fps": 2,
                    "media_resolution": "default"
                },
                {
                    "type": "text",
                    "text": "please describe the content of the video"
                }
            ]
        }
    ],
    max_completion_tokens=1024
)

print(completion.model_dump_json())

Instructions for Use

Video Restrictions

Video Formats: MP4, MOV, AVI, WMV.

Video Formats variants are numerous, and it cannot be guaranteed that all files can be recognized. Please verify through testing that the files can be recognized normally.

Video Size:
- When passed in as a URL: single video file size does not exceed 300 MB.
- When passed in as Base64 encoding: The size of the Base64 encoded string of a single video does not exceed 50 MB.
Number of videos: When multiple videos are input, the number of videos is limited by the model's context length, and the total number of tokens for all video and text must be less than the model's context length.

Note: For calculating video tokens, please refer to Explanation of Video Token Usage. For the model context length, please refer to Pricing and Rate Limits.

Control the fineness of video understanding

You can control the granularity of video understanding through the two fields fps and media_resolution respectively.

fps is the frame number of images extracted from the video per second, used to control the fineness of understanding the time dimension of the video. The default value is 2, with a range of [0.1, 10].
- The higher the value, the denser the frame extraction, and the more refined the model's perception of frame changes, movements, and temporal details;
- The lower the value, the sparser the frame extraction, the faster the processing speed, and the less Token consumption.
media_resolution refers to the resolution level of video frames, used to control the visual understanding fineness of a single frame. The default value is default.
- default: The default level, balancing recognition effectiveness and processing efficiency;
- max: The highest resolution level, which enhances the recognition ability for small objects and detailed textures.

Explanation of Video Token Usage

Video tokens are divided into video_tokens (visual) and audio_tokens (audio).

video_tokens Calculation please refer to the following code. The estimated results are for reference only, and the actual usage is subject to the API response.

"""
Estimate the number of tokens consumed by an API call based on video duration and resolution.
Two parameters control the level of detail:
  - fps: Frames extracted per second. Default 2, range [0.1, 10]. Higher values yield
    finer temporal granularity at the cost of more tokens.
  - media_resolution: Per-frame resolution tier. "default" balances quality and efficiency;
    "max" improves fine-grained detail recognition.
"""

import math

def estimate_video_tokens(
    duration: float,
    width: int,
    height: int,
    fps: float = 2.0,
    media_resolution: str = "default",
    mute: bool = False,
) -> int:
    """
    Estimate the token count for a video input.

    Args:
        duration:         Video duration in seconds.
        width:            Video width in pixels.
        height:           Video height in pixels.
        fps:              Frame extraction rate. Default 2, range [0.1, 10].
        media_resolution: "default" or "max".
        mute:             If True, audio tokens are excluded.

    Returns:
        Estimated total token count.
    """
    # ---- Constants ----
    PATCH, MERGE, T_PATCH = 16, 2, 2
    SPATIAL = PATCH * MERGE                         # 32
    PIX_PER_TOKEN = SPATIAL ** 2                    # 1024
    MAX_TOTAL_TOKENS = 131072
    TOTAL_MAX_PIX = MAX_TOTAL_TOKENS * PIX_PER_TOKEN
    MIN_PIX, MAX_PIX = 8192, 8388608
    MAX_FRAMES = 2048
    DEFAULT_MAX_FRAME_TOKEN = 300

    # ---- 1. Number of extracted frames ----
    nframes = math.ceil(duration * fps)
    nframes = min(nframes, MAX_FRAMES)
    nframes = max(math.ceil(nframes / T_PATCH) * T_PATCH, T_PATCH)

    # ---- 2. Per-frame pixel budget ----
    max_pix = TOTAL_MAX_PIX * T_PATCH // nframes
    if media_resolution != "max":
        max_pix = min(max_pix, DEFAULT_MAX_FRAME_TOKEN * PIX_PER_TOKEN)
    max_pix = max(MIN_PIX, min(max_pix, MAX_PIX))

    # ---- 3. Resolution scaling ----
    h, w = height, width
    if min(h, w) < SPATIAL:
        if h < w:
            w = int(w * SPATIAL / h); h = SPATIAL
        else:
            h = int(h * SPATIAL / w); w = SPATIAL
    h_bar = round(h / SPATIAL) * SPATIAL
    w_bar = round(w / SPATIAL) * SPATIAL
    if h_bar * w_bar > max_pix:
        beta = math.sqrt(h * w / max_pix)
        h_bar = math.floor(h / beta / SPATIAL) * SPATIAL
        w_bar = math.floor(w / beta / SPATIAL) * SPATIAL
    elif h_bar * w_bar < MIN_PIX:
        beta = math.sqrt(MIN_PIX / (h * w))
        h_bar = math.ceil(h * beta / SPATIAL) * SPATIAL
        w_bar = math.ceil(w * beta / SPATIAL) * SPATIAL

    # ---- 4. Token calculation ----
    grids = nframes // T_PATCH                       # temporal grid count
    tokens_per_grid = (h_bar // PATCH) * (w_bar // PATCH) // (MERGE ** 2)
    vision = grids * tokens_per_grid
    timestamps = grids * (5 if fps > 2 else 3)       # timestamp text tokens
    special = grids * 2 + 2                           # special markers

    # ---- 5. Audio tokens ----
    audio = 0
    if not mute:
        spec_len = int(duration * 24000) // 240 + 1
        t = (spec_len - 1) // 2 + 1
        t = t // 2 + int(t % 2 != 0)
        audio = math.ceil(t / 4) + 2                 # +2 for audio special tokens

    return vision + timestamps + special + audio

# ============ Example ============
if __name__ == "__main__":
    # A 1080p, 60-second video
    tokens = estimate_video_tokens(duration=60, width=1920, height=1080)
    print(f"Default params (fps=2, default): {tokens:,} tokens")

    tokens = estimate_video_tokens(duration=60, width=1920, height=1080, fps=5)
    print(f"High frame rate (fps=5, default): {tokens:,} tokens")

    tokens = estimate_video_tokens(duration=60, width=1920, height=1080, media_resolution="max")
    print(f"High resolution (fps=2, max):     {tokens:,} tokens")

    tokens = estimate_video_tokens(duration=60, width=1920, height=1080, mute=True)
    print(f"Muted           (fps=2, mute):    {tokens:,} tokens")

audio_tokens Calculation please refer to the following code. The estimated results are for reference only, and the actual usage is subject to the API response.
```
Total tokens ≈ Audio duration (in seconds) * 6.25
```

Price

Billing: The total cost is calculated based on the number of input, input (cache hits), and output tokens; for pricing, please refer to Pricing and Rate Limits.
- Video Token consumption can be calculated through Explanation of Video Token Usage. The estimated results are for reference only, and the actual usage is subject to the API response.
View Bill: You can view your bill and usage on the Billing page in the Console.

FAQ

Does it support local file upload?

mimo-v2.6-flash, mimo-v2.6-pro, mimo-v2.6-pro-ultraspeed models don't currently support uploading local video files. For supported upload methods, please refer to Video Input Method.

Update Time October 20, 2026