Batch API

Batch Inference (Batch API) is an offline, large-scale data processing solution for workloads that do not require real-time responses. The interface is OpenAI-compatible and is well suited for scenarios such as model evaluation, data labeling, and batch regression testing. You can create asynchronous jobs via the SDK or the console. Key advantages:

• ⭐ Lower cost: Batch Inference (Batch API) is priced at 50% of the real-time API

Models supported for batch inference: mimo-v2.6-pro, mimo-v2.6-flash,when using the batch inference service for model calls, the model name must be in lowercase.

• ⭐ Off-peak smart scheduling: After a job is submitted, the system automatically schedules it for execution during off-peak hours, making full use of idle compute

• ⭐ Flexible usage: Two ways to use it — API and console — supporting full lifecycle job management through either the API or the Open Platform console

• ⭐ OpenAI compatible: The API protocol is identical to OpenAI's, making migration from OpenAI Batch simple

Applicable Scenarios

Batch Inference API Pricing

Xiaomi MiMo API pay-as-you-go billing uses a regular Open Platform API Key and deducts from your account balance based on actual token usage. It is not interchangeable with Token Plan package quotas.

Mainland China pricing

Overseas pricing

Getting Started with Batch Inference

Prerequisites

Before using Batch Inference (Batch API), complete the following steps:

Step 1: Prepare the File

Prepare your data file in JSONL format (JSON Lines — one JSON object per line) following the file format requirements below. Each line contains the details of a single API request.

File Examples

Below is an example of an input file containing 2 requests (a .jsonl file), corresponding to the 3 supported endpoint formats. Note: when using batch inference via the console, only the OpenAI | completions file format is currently supported for upload. Using the API is more flexible and supports all 3 file formats.

OpenAI | completions, download sample file

{"custom_id": "request-1", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "mimo-v2.6-flash", "messages": [{"role": "user", "content": "Hello"}]}}
{"custom_id": "request-2", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "mimo-v2.6-pro", "messages": [{"role": "user", "content": "World"}]}}

OpenAI | responses, download sample file

{"custom_id": "request-1", "method": "POST", "url": "/v1/responses", "body": {"model": "mimo-v2.6-pro", "input": "Hello"}}
{"custom_id": "request-2", "method": "POST", "url": "/v1/responses", "body": {"model": "mimo-v2.6-flash", "input": "please introduce yourself"}}

Anthropic | messages, download sample file

{"custom_id": "request-1", "method": "POST", "url": "/anthropic/v1/messages", "body": {"model": "mimo-v2.6-pro", "messages": [{"role": "user", "content": [{"type": "text", "text": "Hello"}]}]}}
{"custom_id": "request-2", "method": "POST", "url": "/anthropic/v1/messages", "body": {"model": "mimo-v2.6-flash", "messages": [{"role": "user", "content": [{"type": "text", "text": "please introduce yourself"}]}]}}

File Specification Requirements

• Basic file requirements

Default maximum file size is 128 MB. A single file may only contain requests for one batch inference endpoint.

• Field rules

  • Every request must include a custom_id field, a string that is unique within the file, used to map each request to its corresponding result.

  • Each request is sent and returns its result independently. If multiple requests share the same prompt, include the same prompt in each request.

  • The body field of each request must be consistent with the request body of the underlying model invocation API and must be a valid JSON Object.

Step 2: Create a Batch Inference Job

1. On the Batch Inference page , click Create Batch Inference Job.

2. On the create job page, upload your JSONL file, fill in a job description, and set the maximum wait time (1–14 days).

When using batch inference via the console, only the OpenAI | completions file format is currently supported for upload. Using the API is more flexible and supports all 3 file formats.

• OpenAI | completions

• OpenAI | responses

• Anthropic | messages

1. When finished, click Create.

Step 3: Manage Jobs

1. View:

   • On the job list page, view the progress (processed requests / total requests) and status of each job. Or enter the job detail page for more information.

   • Search by job description or ID to quickly locate a target job.

2. Manage:

   • Cancel: Jobs in the "In Progress" state can be cancelled from the Actions column.

   • Troubleshoot errors: "Failed" jobs allow you to download the error file for details. File downloads are available from both the job list Actions column and the job detail page.

Step 4: Download Results

After a job completes, files can be downloaded from the job list Actions column and the job detail page:

• Success file: records all successful requests and their response results.

• Error file (if any) : records all failed requests and their error details.

Both files include the custom_id field, used to match against your original input data, associate results, or locate errors.

Step 5: View Usage Statistics (Optional)

On the Billing Details page, filter and view usage statistics for batch inference.

View data overview: Select a time range, set Inference Type to Batch Inference, select an API Key, and view the batch inference model invocation overview.

• Jobs created via the API: use the API Key you created; select that Key to view usage

• Jobs created via the console: attached to the platform's default API Key by default; no need to use your own API Key; select Other to view usage

API Reference

Step 1: Upload the Job File to the File Service

You can use Curl to upload the job file to the file service's bucket. The gateway platform will subsequently read the request information from the file for batch inference.

Request example

curl https://batch-api-${region}.xiaomimimo.com/v1/files \
-H "Authorization: Bearer $ARK_API_KEY" \
-F 'purpose=batch' \ 
-F 'file=@/Users/doc/demo.jsonl' \     # file path

Request parameters

• purpose — Upload file category

  • batch (batch inference file)

• file=@ — Local path of the file to upload

Sample request response

{
    "id": "file-8a761b15a195",
    "object": "file",
    "purpose": "batch",
    "filename": "input.jsonl",
    "bytes": 327,
    "status": "active",
    "error": null,
    "metadata": null,
    "mime_type": "application/jsonl",
    "created_at": 1780034477,
    "expire_at": 1780038877,
    "preprocess_configs": null
}

Step 2: Create a Batch Inference Job

Request example

curl -X POST https://batch-api-${region}.xiaomimimo.com/v1/batches \
  -H "Authorization: Bearer $ARK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input_file_id": "$FILE_ID",
    "endpoint": "/v1/chat/completions",
    "completion_window": "24h"
  }'

Response example

{
  "id": "batch_c089f12dde664f07aeadd8c7",
  "object": "batch",
  "endpoint": "/v1/chat/completions",
  "errors": null,
  "input_file_id": "file-0f4add6a5001",
  "completion_window": "24h",
  "status": "validating",
  "output_file_id": null,
  "error_file_id": null,
  "created_at": 1711402400,
  "in_progress_at": null,
  "expires_at": 1711488800,
  "finalizing_at": null,
  "completed_at": null,
  "failed_at": null,
  "expired_at": null,
  "cancelling_at": null,
  "cancelled_at": null,
  "request_counts": { "total": 0, "completed": 0, "failed": 0 }
}

Step 3: Query Batch Inference Job Status

Request example

curl https://batch-api-${region}.xiaomimimo.com/v1/batches/$BATCH_ID \
  -H "Authorization: Bearer $ARK_API_KEY"

Batch inference job statuses and their descriptions:

Step 4: Download Batch Inference Job Results

Success file (output_file)

One result object per line, mapped to the input via custom_id:

{"id":"batch_req_xxx","custom_id":"request-1","response":{"status_code":200,"body":{"id":"chatcmpl-xxx","choices":[{"message":{"content":"Hello!"}}]}},"error":null}
{"id":"batch_req_yyy","custom_id":"request-2","response":{"status_code":200,"body":{"id":"chatcmpl-yyy","choices":[{"message":{"content":"World!"}}]}},"error":null}

Error file (error_file)

Generated only when there are failed requests:

{"id":"batch_req_zzz","custom_id":"request-3","response":null,"error":{"code":"inference_failed","message":"400 Bad Request"}}

The input **custom_id** appears verbatim in the output, used to align inputs with outputs.

After the batch inference job finishes running, you can download the result files via curl. The result files fall into 2 categories:

Download the output file

curl https://batch-api-${region}.xiaomimimo.com/v1/files/${result_file_id}/content \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -o output.jsonl

Output file format (JSONL, one result per line):

{"id":"f7ed2","custom_id":"request-1","response":{"status_code":200,"body":{"id":"chatcmpl-xxx","choices":[{"message":{"content":"Hello!"}}]}},"error":null}
{"id":"a3b21","custom_id":"request-2","response":{"status_code":200,"body":{"id":"chatcmpl-yyy","choices":[{"message":{"content":"World!"}}]}},"error":null}

Download the error file

If there are failed requests, error_file_id is not empty:

curl https://batch-api-${region}.xiaomimimo.com/v1/files/${error_file_id}/content \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -o errors.jsonl

Error file format:

{"id":"db7da","custom_id":"request-3","response":null,"error":{"code":"inference_failed","message":"400 Bad Request"}}

Other API Examples

Cancel a Batch Inference Job

Request example

curl -X POST https://batch-api-${region}.xiaomimimo.com/v1/batches/batch_c089f12dde664f07aeadd8c7/cancel \
  -H "Authorization: Bearer YOUR_API_KEY"

Response example

{
  "id": "batch_c089f12dde664f07aeadd8c7",
  "object": "batch",
  "endpoint": "/v1/chat/completions",
  "errors": null,
  "input_file_id": "file-0f4add6a5001",
  "completion_window": "24h",
  "status": "cancelling",
  "output_file_id": null,
  "error_file_id": null,
  "created_at": 1711402400,
  "in_progress_at": 1711402410,
  "expires_at": 1711488800,
  "finalizing_at": null,
  "completed_at": null,
  "failed_at": null,
  "expired_at": null,
  "cancelling_at": 1711402600,
  "cancelled_at": null,
  "request_counts": { "total": 2, "completed": 1, "failed": 0 }
}

OpenAI SDK Compatibility

Batch Inference (Batch API) is OpenAI-protocol compatible and can be used directly with the OpenAI Python SDK:

from openai import OpenAI

client = OpenAI(
    api_key="your-mimo-api-key",
    base_url="https://batch-api-${region}.xiaomimimo.com/v1"
)

# Upload the file
file = client.files.create(
    file=open("input.jsonl", "rb"),
    purpose="batch"
)

# Create the batch
batch = client.batches.create(
    input_file_id=file.id,
    endpoint="/v1/chat/completions",
    completion_window="24h"
)

# Query status
batch = client.batches.retrieve(batch.id)
print(f"Status: {batch.status}")
print(f"Completed: {batch.request_counts.completed}/{batch.request_counts.total}")

# Download results
if batch.output_file_id:
    result = client.files.content(batch.output_file_id)
    with open("output.jsonl", "wb") as f:
        f.write(result.content)

FAQ

Why doesn't the job start immediately after submission?

Batch Inference (Batch API) uses an off-peak scheduling strategy: the system schedules jobs automatically based on online resource availability, with no manual intervention required. When resources are tight, job startup and execution may be delayed.

How do I retry a failed job?

Batch Inference (Batch API) currently does not support in-job retry or resume. If a job fails, you need to:

1. Download the error file;

2. Review the failure reasons in the error file;

3. Create a new job and resubmit.

What should I do if the job is partially successful?

1. If some requests in your job succeeded and others failed, the success file contains all successful request results, and the error file contains the failed requests and their error reasons. Successfully completed requests are billed normally.

2. You can download the error file, review the failure reasons, and create a new job to resubmit.

What happens if my balance is insufficient?

1. Balance is 0 at job creation: you can still browse and go through the job creation flow; the job creation will fail

2. Balance becomes insufficient while the job is running: the job fails. The portion already completed generates a success file and is billed normally; the portion not executed generates a failure file and is not charged

Can I use it without real-name verification?

No. Real-name verification must be completed before using Batch Inference (Batch API). Unverified users who visit the batch inference page will see a guide page and be redirected to the real-name verification page.

Is streaming output supported?

No. Batch scenarios are asynchronous processing and do not apply to the streaming protocol.

Is Token Plan deduction supported?

No. Batch Inference (Batch API) only deducts from your account's cash balance based on actual token usage and is not interchangeable with Token Plan package quotas.

How long are result files retained?

Input files and result files are retained for 30 days by default and are automatically cleaned up after expiry. Please download them promptly.

How are completed, failed, and cancelled jobs billed?

• Only successful requests are billed. Requests that fail during file parsing or job execution incur no charges

• Requests successfully completed before cancellation are billed normally