Using the API

Errors

Errors use the OpenAI envelope, so existing error handling keeps working. An extra provider field tells you which upstream was involved.

The envelope

{
  "error": {
    "message": "Insufficient credits to serve this request",
    "type": "insufficient_quota",
    "code": "insufficient_credits",
    "provider": null
  }
}

Status codes

Status	Type	What to do
`400`	invalid_request_error	Fix the request body. Retrying will not help.
`401`	authentication_error	Check the key exists, is active and is sent as a bearer token.
`402`	insufficient_quota	Top up the balance, or raise the monthly limit on the key.
`404`	invalid_request_error	The model slug is unknown or currently disabled. Check the catalog.
`429`	rate_limit_error	Back off and retry. The limit is per key.
`500`	api_error	A gateway fault. Safe to retry with backoff.
`502`	api_error	Every provider for that model failed. Retry, or pin a different provider.

Common codes

Code	Cause
model_not_found	The slug does not exist in the catalog.
model_unavailable	The model exists but no provider is currently able to serve it.
insufficient_credits	The balance cannot cover the estimated cost of the request.
rate_limit_exceeded	The key exceeded its requests-per-minute allowance.
invalid_api_key	The key is unknown, disabled or revoked.
provider_error	An upstream returned an error that was not worth retrying elsewhere.

Failed requests are free

A request that never produced tokens is not billed, including every failed attempt made before a successful failover.

Retrying well

Retry on 429, 500 and 502 with exponential backoff and jitter. Do not retry 400, 401, 402 or 404, since the outcome will be identical.

import time
from openai import OpenAI, APIStatusError

RETRYABLE = {429, 500, 502, 503}

def with_retries(fn, attempts=4):
    for attempt in range(attempts):
        try:
            return fn()
        except APIStatusError as error:
            if error.status_code not in RETRYABLE or attempt == attempts - 1:
                raise
            time.sleep(2 ** attempt * 0.5)