A production API can look healthy while a leaked key generates charges, or while a valid customer changes an object ID and reads another customer's invoice. In a multi-provider LLM gateway, the exposure points multiply. Upstream provider credentials, token-based spend, failover routing, and prompt logs all need controls that work under pressure, not just controls that look complete in a design document.
API security has become a primary engineering boundary because APIs carry a large share of modern internet traffic. A 2023 review reported that API calls represented 54% of Cloudflare's total requests and referenced earlier research finding that more than 80% of web traffic was API communication, as documented in this review of the OWASP API Security Top 10. OWASP ranks Broken Object Level Authorization first, which means authentication alone isn't enough.
The api security best practices below move from the credentials a caller presents, through the requests they send, to the monitoring that catches what prevention misses. Each practice is framed as a control with a named failure mode, a verification step, and a revocation or rollback path that an on-call engineer can execute.
Table of Contents
- 1. Scope and rotate API keys
- 2. Enforce authentication and object-level authorization
- 3. Encrypt transport and harden configuration
- 4. Apply rate limits, quotas, and spend controls
- 5. Validate input and limit request size
- 6. Protect upstream provider credentials
- 7. Make failover bounded and validate upstream responses
- 8. Redact logs and preserve useful audit trails
- 9. Minimize data and control retention
- 10. Monitor anomalies and rehearse incident response
- 10-Point API Security Best Practices Comparison
- Turn these controls into a quarterly security review
1. Scope and rotate API keys
Failure mode: one shared bearer key gives an attacker access to every environment, service, model, and balance.
API keys should identify a narrowly defined caller, not act as a permanent master password. Create separate credentials for development, staging, production, and distinct services. Scope each key to the operations it needs, assign a spend limit, show the secret only once, and store a hash rather than the recoverable value when your platform controls key storage.
A multi-provider LLM gateway makes attribution especially important. A key used by a background summarization worker shouldn't also authorize an administrative dashboard or a customer-facing chat route. Per-key limits let you attribute usage, disable one integration without disrupting others, and contain a leaked credential before it becomes an unbounded billing problem.
Verify the boundary before deployment
Run a negative test for every scope. A read-only key should fail on a write endpoint, a staging key should fail against production, and a revoked key should receive an authentication failure immediately. Scan commits and pull requests with secret detection, and inspect compiled frontend and mobile artifacts because environment variables don't protect a secret once it's bundled for an untrusted client.
Use a secrets manager or protected runtime configuration. Don't place keys in query parameters, browser JavaScript, mobile binaries, screenshots, tickets, or example curl commands that might reach a public repository.
Operational rule: rotate the credential first, investigate the leaked value second. Preserving a compromised key for forensics leaves the door open.
A rotation runbook should name the owner, create the replacement, deploy it, confirm traffic uses the replacement, revoke the old key, and check usage afterward. Providers such as Stripe document restricted and rolling keys, while GitHub's secret scanning partner program can notify or auto-revoke participating provider tokens when they appear in public repositories. These examples illustrate the control pattern, but your own gateway still needs an explicit rollback path.
After deployment, compare key-level request volume, models, providers, and spend with the service's expected profile. An unexpected model or region is a useful signal even when authentication succeeds.
2. Enforce authentication and object-level authorization
Failure mode: the API authenticates a caller but trusts an object ID supplied in the request.
OWASP places Broken Object Level Authorization, or BOLA, at API1 and Broken Function Level Authorization at API5 in its 2023 taxonomy. The practical distinction matters. Authentication answers “who is calling?” Authorization must answer whether that principal may read this invoice, invoke this model, alter this API key, or call this administrative function.
For every request, derive the tenant and principal from the verified credential. Never accept tenant_id, account_id, or organization_id from the request as proof of ownership. Bind the authenticated identity to the data-access query, then enforce the same binding on reads, writes, usage records, invoices, model requests, API keys, and log searches.
Verify horizontal and vertical access
Automated negative tests should authenticate as user A, request user B's object, and expect no data. They should also authenticate as an ordinary user and attempt an administrative operation. Test direct API calls, not only frontend flows, because a hidden button isn't an authorization control.
Opaque identifiers and UUIDs can make enumeration harder, but they don't replace server-side policy. Where revealing resource existence creates risk, return a normalized 404 response rather than confirming that another tenant's object exists with a detailed 403 response.
A central policy layer is easier to audit than scattered checks. It should evaluate the authenticated principal, tenant, resource, action, and relevant attributes before the handler reaches the data source. Response schemas must also be allowlisted so mass assignment or automatic serialization can't expose fields the caller isn't entitled to see.
For implementation details, keep the gateway's credential lifecycle aligned with the Openbase authentication documentation. The same principle applies whether the caller uses an API key, OAuth token, or service identity.
Revoke or roll back by disabling the affected key or policy, isolating the tenant if necessary, and rerunning cross-tenant tests before restoring access. A successful login is not evidence that the authorization boundary works.
3. Encrypt transport and harden configuration
Failure mode: a key, prompt, response, or error detail travels through a weak transport or an unsafe default.
Require HTTPS for every API route that carries credentials, prompts, completions, or business data. Configure TLS 1.2 or newer, prefer TLS 1.3 where compatible, and reject plaintext HTTP rather than redirecting API requests. A redirect can expose the original request path and headers before encryption begins.
Set Strict-Transport-Security for browser-facing domains, restrict CORS to known origins, and avoid treating CORS as an access-control mechanism. Non-browser callers can ignore it. API keys belong in the Authorization header or another protected header, never in a query string that may enter proxy, browser, or analytics logs.
Remove information from the failure path
Disable debug routes, interactive API explorers in production when they expose operational details, default credentials, and verbose stack traces. Return a stable error body such as an application code and safe message. Log the diagnostic exception server-side with access controls and redaction.
For an LLM gateway, upstream provider names, internal hostnames, request headers, and raw provider responses shouldn't automatically pass through to the customer. A normalized error contract lets clients handle failures consistently while preserving internal details for incident analysis.
Verify the boundary after each infrastructure change. Use a TLS configuration checker such as Qualys SSL Labs or testssl.sh, send an HTTP request and confirm rejection, test CORS with an unapproved origin, and inspect responses for security headers and accidental secrets. Managed policies from Cloudflare or AWS can reduce configuration drift, but they still need testing.
Rollback means restoring the last known-good TLS or gateway policy, not reopening plaintext traffic to keep a failing client alive. If a legacy client can't meet the minimum transport policy, isolate it behind a controlled migration path and assign an owner for removal.
4. Apply rate limits, quotas, and spend controls
Failure mode: a valid caller exhausts compute, provider capacity, or money through volume, concurrency, retries, or oversized model output.
A request limit alone isn't enough for an LLM gateway. Track several dimensions:
- Per-key request rate: stops one credential from flooding the edge.
- Concurrency: prevents a caller from holding too many upstream operations open.
- Token budget: controls the expensive unit that drives model usage.
- Output ceiling: set
max_tokensor the provider equivalent on every request. - Spend cap: stops a compromised or misconfigured integration from consuming the available balance.
Return 429 Too Many Requests with Retry-After when the caller exceeds a limit. Clients should use exponential backoff with jitter. Immediate retries turn a temporary provider response into a retry storm, especially when the gateway has automatic failover.
Match the limit to the operation
A model-selection endpoint, login route, prompt submission, and administrative export don't deserve the same quota. Apply limits per key, identity, tenant, provider, and operation where the risk requires it. A low-and-slow attacker can distribute activity across accounts or IP addresses, so a single global counter won't detect every abuse pattern.
Set hard caps for non-production keys and alert before exhaustion. During a launch, review token consumption, model mix, provider distribution, and failed retries frequently. If a new deployment unexpectedly increases output length, max_tokens gives the team a controllable ceiling while the code is corrected.
The Openbase rate limit documentation provides the relevant integration surface for clients using its gateway. Regardless of provider, verify that limit responses are consistent, that retry headers are accurate, and that a client can't bypass a key-level limit by changing an unrelated request field.
To revoke the control, disable the affected key or tenant quota, reduce concurrency at the gateway, and stop automated failover if it is multiplying requests. Restore limits gradually after confirming that token growth and retry volume have returned to the expected profile.
5. Validate input and limit request size
Failure mode: malformed or attacker-controlled input reaches a parser, model router, database, command, or template that wasn't designed to handle it.
Treat every field as untrusted. Validate request bodies against OpenAPI or JSON Schema, reject unknown fields unless pass-through is deliberately documented, and enforce types, lengths, ranges, nesting depth, and body size at both the edge and application layers.
For chat-completion requests, validate message roles, content types, tool-call structures, model slugs, sampling parameters, and provider-specific options. Maintain an allowlist for model identifiers. If a caller submits an unknown or unauthorized model, reject it before routing rather than allowing a loosely matched slug to select an unexpected provider.
Keep prompts as data
Prompt text can contain instructions, delimiters, markup, or strings that resemble commands. Treat it as data and pass it through parameterized interfaces. Don't interpolate prompt content into shell commands, SQL, filesystem paths, server-side templates, or dynamic configuration.
Request-size controls should account for compressed and decompressed bodies where compression is accepted. A small compressed payload can expand into a resource exhaustion problem. Set parser limits, timeout limits, and response limits before expensive processing begins.
Verify with malformed JSON, highly nested objects, unknown parameters, oversized prompts, invalid tool calls, and unsupported model slugs. Confirm that the service returns a safe client error without a stack trace and that the rejected request doesn't reach the provider. Use contract tests so a schema change can't subtly widen the accepted surface.
Rollback by restoring the prior schema or temporarily disabling the newly introduced field. Don't weaken validation globally because one client hasn't migrated. Give that client a versioned contract and an explicit sunset owner.
6. Protect upstream provider credentials
Failure mode: the gateway is secure at its public edge, but a stolen upstream key gives direct access to a provider account.
A multi-provider gateway holds credentials that can invoke models and create spend outside the gateway's own account boundary. Store those secrets in a dedicated secrets manager or KMS, encrypt them at rest, and grant access only to the service that needs to call the provider. Provider keys shouldn't sit in source control, container images, writable application disks, or ordinary database rows.
Separate production and staging credentials. Scope each provider key where the provider supports it, assign independent spend limits, and restrict which service identity can retrieve it. Load secrets at process start or through a controlled refresh path, keep them in memory only when practical, and avoid writing plaintext copies to disk.
Make rotation a tested deployment
A rotation runbook should cover creation, secret-manager update, process refresh, health verification, old-key revocation, and failover. Rotate one provider credential while another provider remains available, then test both normal routing and degraded routing. A correct secret rotation that causes an outage still becomes an operational incident.
Verify that the gateway never returns provider credentials in errors, traces, metrics, or debug endpoints. Monitor provider usage for unexpected hosts, regions, models, or volume. A provider key appearing outside the gateway's expected execution environment should trigger containment, not merely a ticket.
AWS Secrets Manager documents automated rotation patterns, while HashiCorp Vault supports short-lived dynamic credentials for suitable integrations. These tools reduce manual handling, but the revocation path remains yours. If a provider key is suspected to be exposed, disable it at the provider, replace it in the secrets manager, restart or refresh affected workers, and inspect usage from the exposure window.
7. Make failover bounded and validate upstream responses
Failure mode: retries and failover multiply side effects, route to an attacker-controlled destination, or hide the original provider failure until the gateway exhausts capacity.
Third-party responses are untrusted input. Validate status codes, content types, response schemas, maximum response size, and completion structure before returning data to a caller. Strip provider-specific diagnostics that reveal internal routing, account details, or upstream infrastructure.
Never derive a routing target directly from user input. Keep an allowlist of provider hosts and model-to-provider mappings. A provider-prefixed model slug can select among approved routes, but it must not become an arbitrary URL or hostname.
Define what may be retried
Retry only conditions that are plausibly transient, such as 408, 429, 5xx responses, and connection timeouts. Don't retry malformed requests or 404 responses. Set a total request deadline, a small retry budget, jittered backoff, and a circuit breaker that stops sending traffic to an unhealthy provider.
Idempotency is essential when a request can create a side effect. For generation requests, repeated attempts can create duplicate audit events, charges, or downstream tool actions even when the user sees one response. Use an idempotency key and a consistent transaction policy across providers, or restrict retries to operations with no external side effects.
Verify failover with a controlled provider outage. Confirm that the gateway selects only an approved backup, records the provider that served the request, respects the total deadline, and doesn't retry a client error. Envoy's outlier detection and retry budgets illustrate how infrastructure can limit retry storms, while circuit breakers provide a clear open, half-open, and closed state for operations teams.
Rollback by disabling the failing provider route, lowering its priority, or opening its circuit manually. Restore it only after health checks and a small canary confirm that latency, errors, response validation, and billing attribution are stable.
8. Redact logs and preserve useful audit trails
Failure mode: the logs needed to investigate a key compromise become the source of a second compromise.
Log enough metadata to answer who called, what happened, where it went, and what it cost. Useful fields include a masked key identifier, tenant and principal context, model, provider, input and output token counts, latency, time to first token where available, status code, retry count, and request correlation ID.
Never log the full Authorization header, provider key, raw secret, or unrestricted prompt and completion bodies by default. LLM prompts can contain customer records, credentials, source code, or regulated information. Make body logging opt-in, configurable per key or tenant, and explicit about retention.
Separate operational logs from audit records
Application logs support debugging. Audit logs record security-sensitive events such as key creation, scope changes, revocation, tenant-policy changes, provider credential rotation, administrative access, and export activity. Store audit events separately with restricted read and write access, integrity protection, and a defined retention policy.
Verify redaction with synthetic secrets in staging. Put fake bearer tokens, provider keys, passwords, and sensitive prompt content through every logging path, including exceptions, retries, tracing, dead-letter queues, and provider error handling. Check dashboards and alert payloads too. Redaction that works in application logs but fails in a trace exporter isn't complete.
Stripe's dashboard illustrates the useful pattern of displaying only a masked portion of a key rather than the secret itself. Datadog and Splunk provide ingestion-time scanning and redaction controls, but application-level filtering should happen before data leaves the service.
If a log pipeline leaks content, stop body logging, revoke any exposed credentials, restrict access to the affected index, and preserve only the minimum evidence needed for investigation. Then deploy a tested redaction rule before re-enabling the stream.
9. Minimize data and control retention
Failure mode: the gateway stores sensitive prompts and completions long after the feature needed them, expanding the impact of every future incident.
Default to not storing prompt and completion bodies. Store the metadata required for billing, reliability, and abuse investigation, then make content retention an explicit customer or tenant choice. The right default reduces exposure without preventing teams from enabling controlled debugging for a defined incident or development workflow.
Encrypt retained data at rest with managed keys and restrict access by role and tenant. Document retention periods, deletion behavior, subprocessors, regional processing, and upstream provider handling. Ask each provider for its current retention and training-use terms, then record the answer in the vendor register instead of relying on an old assumption.
Map the complete data path
For an LLM request, document the route from client to gateway, prompt logger, queue, provider, response store, analytics system, backup, and support tooling. Mark where data is copied, transformed, cached, or exported. A deletion request that removes the primary row but leaves a prompt in a trace system isn't a complete deletion workflow.
Validate deletion with a test tenant. Create synthetic prompts, run them through every enabled integration, execute deletion, and search each approved storage system for residual content. Verify that encryption keys, access policies, and backups follow the organization's retention and deletion requirements.
The Openbase security information is a useful reference point when evaluating a gateway's security posture, but teams still need to confirm terms that apply to their own account, data, region, and use case. Requirements for GDPR, HIPAA, or SOC 2 scope should be mapped before regulated workloads are onboarded.
Rollback means disabling body storage or a new analytics sink, stopping new writes, and preserving only approved metadata. Don't solve an uncertain retention problem by copying the same sensitive data into another system for investigation.
10. Monitor anomalies and rehearse incident response
Failure mode: preventive controls fail, but no one sees the misuse until the account, tenant, or provider balance is exhausted.
Monitor authentication failures, denied authorization decisions, request volume, concurrency, token consumption, spend, source locations, error rates, model mix, provider selection, retry counts, and latency. A sudden rise in 401 responses from new source addresses can indicate credential stuffing. A steady increase in token use across ordinary-looking requests can indicate automation or a prompt-size regression.
Baselines must be specific to the key, tenant, service, and operation. A production ingestion worker and an interactive dashboard have different normal behavior. Alert on deviations from each key's rolling seven-day usage baseline, then attach the alert to a runbook with an owner and severity.
Detect business-flow abuse, not only floods
OWASP distinguishes sensitive business-flow abuse from ordinary authorization failures. A caller may be authenticated and technically permitted to invoke a workflow, yet still automate it at an abusive scale. For an LLM gateway, watch for unusual provider failover, repeated expensive prompts, excessive tool invocations, retry loops, token-budget exhaustion, and coordinated activity spread across keys or accounts.
Use operation-level controls such as idempotency keys, per-identity and per-resource quotas, cost budgets, circuit breakers, anomaly thresholds, and audit events that distinguish an upstream outage from repeated automated misuse. Generic IP rate limits won't reliably catch low-and-slow behavior distributed across providers or credentials.
Verify detection with tabletop exercises and synthetic events. One exercise should cover a leaked customer key, another an exposed upstream provider credential, and another a degraded provider that triggers failover. The team should be able to identify the key, revoke it, rotate provider credentials, disable a route, preserve evidence, notify affected customers, and confirm recovery.
OWASP's public methodology notes that the 2023 update received no data sufficient for statistical analysis of sensitive-business-flow abuse. That limitation is a reason to test your own workflows rather than assume a generic control has been effective.
10-Point API Security Best Practices Comparison
| Item | Implementation complexity 🔄 | Resource requirements ⚡ | Expected outcomes 📊 | Ideal use cases 💡 | Key advantages ⭐ |
|---|---|---|---|---|---|
| Scoped, Rotatable API Key Management | Medium, rotation workflows and scoping required | Medium, secrets manager, automation, CI integration | Limits blast radius, enables fast revocation and spend attribution | Multi-environment deployments, team-isolated services | Fine-grained revocation, zero-downtime rotation, clearer billing ⭐⭐⭐ |
| Strong Authentication & Object-Level Authorization | High, per-endpoint object checks and auth logic | High, auth service, token validation, test coverage | Prevents cross-tenant leaks and unauthorized access | Multi-tenant SaaS, sensitive-data APIs | Robust tenant isolation, centralized permission checks ⭐⭐⭐⭐ |
| Transport Encryption & Secure API Configuration | Low–Medium, TLS and header hardening, config management | Low, managed TLS, config scanners, CI checks | Protects data in transit and reduces information disclosure | All public APIs and browser clients | Strong baseline protection, easy validation with standard tools ⭐⭐⭐ |
| Rate Limiting, Quotas & Spend Controls | Medium, distributed limits, quota accounting | Medium, rate-limit infra, billing hooks, dashboards | Prevents runaway costs, preserves capacity, predictable billing | LLM/public APIs, pay-per-use services | Cost control, fair usage, graceful throttling (429/Retry-After) ⭐⭐⭐ |
| Input Validation & Request Size Limits | Medium, schemas and allowlists to maintain | Low–Medium, validators, API gateway enforcement | Blocks malformed input, reduces parser DoS, clearer errors | APIs accepting free-form input, chat-completions | Early rejection of bad requests, improved reliability ⭐⭐⭐ |
| Secure Upstream Provider Credential Management | Medium, secrets rotation, access policies | Medium, KMS/secrets manager, audit logs | Limits provider exposure, enables rotation without deploys | Gateways/proxies that call third-party APIs | Centralized secrets, least-privilege access, auditability ⭐⭐⭐ |
| Secure Failover & Validation of Upstream Responses | High, timeouts, retries, circuit breakers, validation | Medium, resilience libs, monitoring, schema checks | Prevents SSRF, avoids amplification, hides provider internals | Multi-provider failover, high-availability gateways | Resilience under failure, safe response normalization ⭐⭐⭐ |
| Secure Logging, Redaction & Audit Trails | Medium, redaction rules, immutable audit pipelines | Medium–High, logging storage, redaction tooling, retention | Enables forensics without secrets; supports compliance | Enterprise/regulatory environments, billing audits | Forensic capability, masked identifiers, audit compliance ⭐⭐⭐ |
| Data Minimization, Encryption at Rest & Retention Controls | Medium, policies, KMS, deletion workflows | Medium, KMS, regional storage, retention tooling | Reduces breach impact, meets data residency/compliance needs | Regulated customers, enterprise contracts | Lower compliance risk, customer trust, reduced storage cost ⭐⭐⭐ |
| Anomaly Detection, Monitoring & Incident Response | High, baselining, alerting, runbooks, on-call | High, monitoring stack, on-call staff, analytics | Early detection of compromises; faster containment (lower MTTR) | Large-scale APIs, teams needing rapid incident response | Rapid detection of abuse, operational readiness, playbooks ⭐⭐⭐⭐ |
Turn these controls into a quarterly security review
Start with the controls that cap damage fastest: scoped keys with spend limits, object-level authorization tests, and a tested key-revocation runbook. They address the most direct paths from a compromised credential or unchecked identifier to financial loss or cross-tenant exposure. Confirm that every production key has an owner, a scope, a spend boundary, and a documented replacement path.
Then turn the ten practices into a recurring review. Inventory every active endpoint, model route, provider credential, logging destination, deprecated version, and administrative function. Re-test authentication and authorization with negative cases. Submit oversized and malformed payloads. Confirm that TLS policies, CORS rules, error bodies, rate limits, retry budgets, circuit breakers, and provider allowlists still match the intended design.
Rotate provider credentials under controlled conditions and verify that failover works without duplicating side effects. Review key-level spend, token consumption, model mix, source locations, denied actions, and retry patterns. Check that prompts, completions, authorization headers, and upstream errors remain redacted across application logs, traces, queues, dashboards, and support exports.
A quarterly cadence shouldn't become a substitute for runtime response. Security posture drifts whenever teams add an endpoint, enable a model, change a provider, alter a schema, or introduce a new logging sink. Put ownership beside every control, record the last verification date, and assign a rollback action that an on-call engineer can execute without waiting for the original implementer.
Openbase can illustrate this operating model for teams using a multi-provider LLM gateway. Its platform offers a single gateway key and balance across provider integrations, automatic failover, per-key controls, and usage records that include model, provider, token, latency, and cost information. Those capabilities don't remove the need for your own authorization tests, retention decisions, or incident runbooks, but they give engineering teams concrete control points to review.
Use this list as a quarterly security review, not a one-time launch checklist. The strongest API security program is the one that can demonstrate each control, reproduce its failure mode safely, and revoke or roll back the affected path before an incident becomes a customer-impacting event.
Openbase provides an OpenAI-compatible gateway for accessing models from multiple providers through a unified key, balance, billing view, failover path, and granular usage logging. Review Openbase to evaluate how its key controls, provider routing, and cost visibility could fit your API security operating model.