Timeouts, Rate Limits, and Retries

Documentation maintained by 1ZheAI. Sources and corrections.

Found an error? Contact support with the page address and a description. Do not send API keys or account credentials.

Fix configuration errors first, then handle transient failures. Limit retry count, concurrency, and total wait time together to avoid sending the same failed request many times in a short period.

Which situations should be retried

Situation	Recommendation
400 / 401 / 403 / 404	Fix the request, permissions, or model first; do not automatically repeat the original request
429	Check the response message; when Retry-After is present, follow the wait time and reduce concurrency
500 / 502 / 503 / 504 / 529	Limited retries are allowed; if failures persist, stop and report the request ID
Connection failure	Check DNS, proxy, and network; retry within the time allowed by the task
Timeout or stream interruption after the request was sent	The result may be uncertain; check logs first; do not assume the upstream has not processed it

The retry scope of client libraries may not exactly match the table above. Whichever layer you use for retries, explicitly set the policy at that layer.

Use the SDK’s built-in retries first

Python and Node.js use different timeout units: Python uses seconds, Node.js uses milliseconds. The example below sets a 120-second timeout and up to 2 automatic retries; change it to 0 when debugging initial integration.

Python

Python

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["ONEZHE_API_KEY"],
    base_url="https://1zheai.com/v1",
    timeout=120.0,
    max_retries=2,
)

Node.js

Node.js

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.ONEZHE_API_KEY,
  baseURL: "https://1zheai.com/v1",
  timeout: 120_000,
  maxRetries: 2,
});

The timeout configuration is not an upper bound on the total duration of the entire business task: retries and waits will increase the total duration. If the outer layer will re-execute the task, set a total deadline and cancellation mechanism separately.

When implementing retries yourself

Disable the SDK’s automatic retries to avoid stacking. Use progressively increasing wait intervals with random jitter; explicitly limit attempt count, maximum wait, and concurrency queue. When parsing Retry-After, note that it may be either seconds or an HTTP date.

If partial streaming results have already been shown to the user, do not automatically replay from the beginning and append to the old text. Keep the “incomplete” state and let the user decide to regenerate or continue.

For image generation, tool execution, and other tasks that may produce external results, pay special attention to duplicate submissions. Do not assume that setting an arbitrary request ID will make the server automatically deduplicate.

Record sufficient debugging information

Record time, endpoint, model, status code, duration, attempt number, and the request ID returned by the server. Avoid logging authentication headers and full private prompts. Use this information to cross-reference call logs .

SDK settings are documented in OpenAI Python SDK and Node SDK . For specific errors, see error codes and troubleshooting .