Timeouts, rate limits and retries

Configure SDK timeouts and limited retries, distinguish retryable errors from configuration errors, and avoid duplicate submissions.

Fix configuration errors first, then handle transient failures. Limit retry count, concurrency, and total wait time together to avoid sending the same failed request many times in a short period.

Which cases should be retried

Case	Recommendation
400 / 401 / 403 / 404	Fix the request, permissions, or model first; do not automatically repeat the original request
429	Check the response message; when `Retry-After` is present, follow the wait time and reduce concurrency
500 / 502 / 503 / 504 / 529	Limited retries are allowed; if failures persist, stop and report the request ID
Connection failure	Check DNS, proxy, and network; retry within the time allowed by the task
Timeout or stream interruption after the request was sent	The result may be uncertain; check logs first; do not assume the upstream has not processed it

The retry scope of client libraries may not exactly match the table above. Whichever layer you use for retries, explicitly set the policy at that layer.

Use the SDK’s built-in retries first

Python and Node.js use different timeout units: Python uses seconds, Node.js uses milliseconds. The example below sets a 120-second timeout and up to 2 automatic retries; when debugging an initial integration, you can change it to 0.

Python
Node.js

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["ONEZHE_API_KEY"],
    base_url="https://1zheai.com/v1",
    timeout=120.0,
    max_retries=2,
)

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.ONEZHE_API_KEY,
  baseURL: "https://1zheai.com/v1",
  timeout: 120_000,
  maxRetries: 2,
});

The timeout setting is not an upper bound on the total time for the entire business task: retries and waits increase the total time. If an outer layer will re-execute the task, set a separate total deadline and cancellation mechanism.

When implementing retries yourself

Disable the SDK’s automatic retries to avoid stacking. Use progressively increasing wait intervals with random jitter; explicitly limit the number of attempts, maximum wait, and concurrency queue. When parsing Retry-After, note that it may be either seconds or an HTTP date.

If partial streaming results have already been shown to the user, do not automatically replay from the beginning and append after the old text. Keep the “incomplete” state and let the user decide whether to regenerate or continue.

For image generation, tool execution, and other tasks that may produce external results, pay special attention to duplicate submissions. Do not assume that setting an arbitrary request ID will make the server automatically deduplicate.

Record enough debugging information

Record the time, endpoint, model, status code, duration, attempt number, and the request ID returned by the server. Avoid logging authentication headers and full private prompts. Use this information to cross-reference the call logs .

For SDK settings, see the OpenAI Python SDK and Node SDK . For specific errors, see error codes and troubleshooting .