The global average cost of a data breach rose to USD 4.88 million in 2024, marking the largest annual jump since the pandemic. Security and privacy are now strategic priorities because an AI integration can turn one exposed prompt, credential, or log into a business-wide financial and regulatory problem.
AI development moves quickly because teams can connect an application to a language model with a small amount of code. Governance moves more slowly because it must answer harder questions: which data may enter a prompt, which provider may process it, where copies are retained, who can inspect the interaction, and what happens when the model behaves unexpectedly?
That friction creates a familiar engineering temptation. A team prototypes with real customer data, stores full prompts for debugging, gives a shared API key to several environments, and postpones the security review until the feature has users. The design may work, but every shortcut creates another copy, access path, or assumption that security and privacy teams must later unwind.
Table of Contents
- The Stakes of Security and Privacy in AI
- Understanding the AI Threat Landscape
- Technical Controls for Data Protection
- Navigating Compliance and Data Residency
- Managing Logs and Retention Risks
- Implementing Zero-Retention Architectures
- Building a Governance Strategy for AI
The Stakes of Security and Privacy in AI
A breach is no longer an abstract event reserved for large financial institutions. IBM's 2024 Cost of a Data Breach report found that the worldwide average breach cost reached USD 4.88 million, up from USD 4.45 million in 2023, a 10% increase and the largest annual jump since the pandemic. The same report placed the average cost for U.S. organizations at USD 9.36 million, while breaches that took more than 200 days to identify and contain averaged USD 5.46 million.
Those figures change the engineering conversation. A prompt pipeline isn't merely a feature endpoint. It may carry customer records, source code, internal documents, credentials, health information, or commercially sensitive plans into a chain of gateways, model providers, caches, telemetry systems, and backups. A failure in any one of those layers can create both a security incident and a privacy incident.
Why AI changes the risk calculation
Traditional applications usually expose predictable functions. An AI application accepts language as an input, interprets it probabilistically, and may use that interpretation to retrieve data, call tools, or generate an action. A malicious instruction hidden in a document can therefore influence a model differently from a conventional malformed request.
Consider an internal support assistant with access to a knowledge base. An attacker submits a document containing instructions that tell the assistant to reveal retrieved content or call an external tool. The model may treat those instructions as part of its task unless the application separates trusted instructions, untrusted content, and authorization decisions.
The result is a combined problem:
- Security controls must prevent unauthorized access, tool use, credential exposure, and service abuse.
- Privacy controls must limit what data enters the system, who can process it, how long copies survive, and whether a provider can reuse it.
- Operational controls must detect misuse without collecting the very payloads the organization is trying to protect.
Teams that treat these as separate checklists often miss the overlap. A gateway can authenticate a request while still logging sensitive content. A model provider can be contractually approved while an application sends more personal data than the task requires. A privacy policy can promise deletion while backups and analytics retain unexamined copies.
Engineers can use an AI security reference for developers as one input to a broader architecture review, but the core responsibility remains with the system owner. The safest design makes the permitted data flow explicit before implementation begins.
Understanding the AI Threat Landscape
Start with a simple analogy. An API key is like a house key, not like a username. Anyone who obtains it may be able to enter a service, consume resources, or access data within its permissions. Putting that key in source code, a browser bundle, a support ticket, or an unfiltered log is equivalent to leaving a physical key where strangers can copy it.
Encryption solves a different problem. Encryption in transit protects data while it travels between the application, gateway, and provider. Encryption at rest protects stored data on disks, databases, queues, and backups. Neither control decides whether the application should have sent the data in the first place, and neither replaces authentication or authorization.
Four risks that belong in the threat model
Prompt injection occurs when untrusted text attempts to override the application's intended instructions. The text might come from a user, a retrieved webpage, an uploaded file, or a tool response. Treat every external input as data, not as an authority that can redefine permissions.
Credential leakage happens when keys, access tokens, or connection details appear in code, error messages, prompts, or logs. A model doesn't need to understand a credential perfectly for the exposure to matter. An attacker who searches an accessible log store may use it directly.
Data exfiltration is the unauthorized movement of sensitive information outside its approved boundary. A model might reveal retrieved records in a response, pass content to a tool, or include confidential text in a diagnostic event. The application must enforce output and tool policies rather than assuming the model will protect data by itself.
Model abuse includes unauthorized resource consumption and attempts to use an AI endpoint outside its intended purpose. Authentication limits who can call the service. Rate limits, quotas, request validation, and per-environment controls limit what an authenticated caller can do.
Build the boundary before choosing the model
Draw the complete request path. Include the client, application server, AI gateway, model provider, retrieval store, tool services, observability platform, support systems, and backups. Mark where data is transformed, duplicated, retained, and viewed by humans.
Then answer four questions for every connection:
- Who is calling? Use workload identities or short-lived credentials where practical, rather than one shared secret.
- What may cross the boundary? Define permitted data classes and remove unnecessary fields before the request leaves the application.
- What may the receiver do? Separate model inference from privileged actions such as sending mail, changing records, or issuing refunds.
- What evidence is retained? Log enough metadata to investigate failures, but avoid storing full sensitive payloads by default.
The model is an untrusted component within this design. It can help interpret content, but it shouldn't become the final authority for access control, data classification, or irreversible business actions.
Technical Controls for Data Protection
A secure AI gateway applies several controls in sequence rather than relying on one protective layer. The application authenticates the caller, classifies the data, removes unnecessary fields, sends the request over an encrypted connection, and applies policy to the response. The gateway then records limited operational metadata without retaining the original content unless a documented purpose requires it.
Protect credentials at the source
Keep provider credentials on the server side. Never place them in mobile applications, browser code, client-side configuration, screenshots, or prompts. Give each environment and service its own credential so a development compromise doesn't automatically become a production compromise.
Use a secret manager, restrict permissions, rotate credentials, and revoke them when a service or employee no longer needs access. A gateway can centralize provider access, but it shouldn't become an excuse for every internal service to share one unrestricted key. Assign separate limits and identities to applications so usage anomalies point to a responsible workload.
Teams can review an authentication implementation guide while mapping these controls to their own identity platform. The important design decision is separation: authentication identifies the caller, authorization determines the allowed model, data class, tool, and spending boundary.
Reduce the data before encrypting it
Encryption protects a payload, but it doesn't make excessive collection acceptable. If an assistant only needs a customer's order status, send an order identifier and the relevant status fields, not the complete account record. If a document contains a national identifier, remove or replace it before retrieval and inference unless the task genuinely requires it.
Tokenization replaces a sensitive value with a controlled placeholder. A payment reference can become a token that has meaning inside the application but no useful meaning to the model. The application can map the token back only inside a tightly controlled service, which reduces the amount of sensitive information exposed to downstream systems.
This connects engineering controls with privacy obligations. The GDPR privacy rights overview describes how GDPR became a global benchmark after taking effect on May 25, 2018. By April 2023, the GDPR Enforcement Tracker recorded 1,602 fines totaling about €2.7 billion, demonstrating that privacy enforcement creates material consequences rather than theoretical obligations.
Apply controls to outputs and tools
Validate model-generated tool arguments against a schema. Check the user's authorization in application code before executing the action. Require human approval for high-impact operations, and prevent the model from deciding that an approval requirement no longer applies.
Use content filtering and structured output validation where they fit the task. These controls won't eliminate prompt injection, but they make it harder for untrusted text to cross into an authorized action. Security and privacy improve when the model has narrow capabilities and the application enforces the final decision.
Navigating Compliance and Data Residency
Compliance begins with a data-flow question, not a vendor logo. Identify the categories of information the system processes, the purpose for processing, the people who may access it, the countries involved, and the locations where copies can persist. A request may pass through an application region, an AI gateway, a provider's inference environment, a logging service, and a backup system.
Residency is a system property
Data residency is not just the location listed on a provider's marketing page. Ask where prompts and outputs are processed, where support personnel can access them, where telemetry is stored, how backups are handled, and whether subprocessors receive any content. A provider may offer regional storage while processing requests elsewhere, or retain operational metadata in a separate service.
Document those distinctions in the architecture record. Classify workloads by sensitivity and assign an approved processing path to each class. A public marketing draft can follow a different route from an employee record or confidential legal document.
Privacy and security teams should also review provider terms for training, retention, human review, subprocessors, deletion, and incident notification. A no-training promise doesn't automatically mean no retention. A short retention period doesn't mean the application can send unrestricted personal data.
Logs and training create downstream copies
AI systems often generate more artifacts than engineers expect. Prompts may enter request traces, error reports, evaluation datasets, caching layers, analytics exports, and support tickets. If a provider uses inputs or outputs for a separate purpose, the original data may gain a new lifecycle that the application owner didn't plan.
The API compliance guidance for LLM integrations emphasizes data minimization, scoped retention, and explicit control over provider handling. That means a privacy review should inspect defaults, not just contractual language. Teams need configurable retention windows, deletion workflows, and zero-retention or opt-out modes for sensitive workloads.
The practical test is simple: can the organization explain every copy of a prompt, who can access it, why it exists, and when it disappears? If the answer is unclear, the system has a governance gap even when the primary database is well protected.
Architecture rule: Treat residency, retention, and provider reuse as separate decisions. Recording one favorable setting doesn't answer the other two.
A technical design may satisfy a regional requirement while still exposing data through unrestricted administrator access or verbose diagnostics. Compliance works best when legal requirements become enforceable configuration, identity, and data-flow rules.
Managing Logs and Retention Risks
Logs help engineers diagnose latency, failed tool calls, malformed responses, and provider outages. They also create a second data system with its own access controls, retention schedule, replication behavior, and breach impact. In an AI application, the most useful debugging field can be the most sensitive field, because a prompt may contain the user's entire task and a response may reproduce confidential information.
The National Cyber Security Centre guidance on API logging and monitoring says passwords, API keys, access tokens, and personally identifiable information shouldn't be logged. It also recommends redaction or masking and encryption in transit and at rest.
Design a useful event without the payload
Log the request identifier, application identity, model route, outcome, latency, token counters where appropriate, policy decisions, and error category. Hash or otherwise de-identify identifiers when a stable correlation value is enough. Store a small, approved excerpt only when a documented diagnostic need exists, and make that exception visible to reviewers.
Redaction must happen before persistence. Filtering a dashboard after the log store has already received the raw prompt doesn't solve the exposure. Apply field-level rules in the application or gateway, then test them with representative sensitive inputs and malformed requests.
For temporary debugging, use a controlled capture path with explicit approval, restricted access, short retention, and an automatic shutdown condition. Don't turn production logging into a permanent evaluation dataset by accident.
Log Safety Checklist
| Practice | Risk Level |
|---|---|
| Store API keys, passwords, or access tokens in logs | Critical |
| Persist full prompts and responses by default | High |
| Redact sensitive fields before persistence | Lower |
| Encrypt logs in transit and at rest | Lower |
| Record metadata such as route, status, and latency | Lower |
| Permit temporary payload capture without approval | High |
| Give broad administrator access to raw AI logs | High |
| Review retention and deletion behavior | Lower |
A zero-retention design still needs operational evidence. If no content is stored, engineers can rely on request IDs, policy outcomes, timing, error classes, and controlled replay with synthetic data. This approach may make some investigations less convenient, but it keeps observability from becoming an uncontrolled data-exfiltration path.
Implementing Zero-Retention Architectures
Zero retention is a design choice, not a label that can be accepted without inspection. It may mean that the gateway doesn't persist message content, that an upstream provider doesn't retain inputs or outputs, or that both conditions apply. Ask which definition a provider uses and verify the behavior across normal requests, errors, retries, abuse investigations, backups, and support workflows.
Compare the operating models
A full-payload logging model gives developers detailed replay data and makes debugging convenient. Its control level is low because prompts and responses may spread across several systems. It can support evaluation, but only with strong classification, redaction, access controls, and a defined purpose.
A metadata-only model records route, status, usage, latency, and policy decisions while excluding message content. It offers a stronger privacy boundary and still supports reliability and billing analysis. Developers lose direct replay, so they need synthetic fixtures, client-side correlation, and controlled reproduction.
A zero-retention provider model prevents the gateway and approved upstream providers from keeping sensitive inputs and outputs beyond request processing, subject to the actual provider terms and configuration. It provides stronger control for confidential workloads, but it requires better application-side debugging and more disciplined incident procedures.
Operational trade-off: The less content you retain, the more intentional your test data, correlation IDs, and failure telemetry must become.
Put the policy into the request path
Define a policy for each data class. Public and synthetic data may use ordinary observability. Confidential data should use minimized payloads, restricted routes, and metadata-only events. Highly sensitive data may require tokenization, an approved provider path, zero-retention settings, and a prohibition on human review.
Make the gateway reject requests that violate the policy rather than relying on developer memory. Enforce provider allowlists, model restrictions, maximum input size, tool permissions, and environment-specific credentials. Ensure retries don't move sensitive data to an unapproved provider without detection.
Openbase offers a unified API gateway model with consolidated model access, provider failover, per-request usage visibility, and separate API keys for environment isolation. Those capabilities can support an AI control plane, but the organization still needs to configure retention, provider handling, access policies, and data minimization according to its workload requirements.
Test deletion as an operation, not a promise. Submit a controlled marker through the system, inspect application and gateway stores, check analytics and support paths, and confirm how backups and legal holds affect removal. Repeat the exercise after configuration changes and provider changes.
Building a Governance Strategy for AI
A product team once treated its internal assistant as a small productivity feature. Engineers connected retrieval, model inference, and a ticketing tool, then asked for approval after the prototype worked. The review found no single catastrophic flaw. Instead, it found several ordinary gaps: broad source access, shared credentials, full prompt logging, unclear provider retention, and no owner for model-generated ticket updates.
The team didn't need a slower development process. It needed a paved path with decisions made before each integration. Governance becomes an accelerator when developers know which data, models, providers, tools, and logging modes are already approved.
Five decisions every team should make
Access policies define who may use each AI capability and which actions require approval. Bind permissions to people, services, and environments. A user who can ask a summarization assistant shouldn't automatically be able to invoke a tool that changes a customer record.
Data classification gives developers a practical route through ambiguity. Mark data as public, internal, confidential, or highly sensitive, then map each category to permitted providers, retention modes, tools, and human-review rules.
Vendor review should cover more than uptime and price. Review provider retention, training use, residency, subprocessors, encryption, incident handling, access logging, deletion, and the behavior of retries and failover. Reassess the route when the model or provider changes.
Incident response needs AI-specific signals. Prepare to revoke keys, disable a tool, block a model route, preserve metadata, identify affected prompts without collecting more content, and notify the right privacy and security owners.
Continuous audit checks whether the deployed system still matches its approved design. Review access grants, provider configurations, retention settings, tool schemas, policy exceptions, and unusual usage. An audit should produce corrective work, not just a document.
The business case remains clear. IBM's breach data connects financial impact with detection speed, response maturity, and exposure. A governance program therefore needs both preventive controls and usable detection. If an organization can't tell which application sent a request, which provider handled it, or whether a response triggered a privileged action, it can't respond quickly enough.
Public concern reinforces the need for usable defaults. YouGov's 2026 Data Privacy Day survey reported that 61% of Americans consider limiting access to personal data very important, while 33% take only moderate care protecting it. The survey also found that 56% worry wearables could expose lifestyle details, 28% regularly review privacy and security settings, and 29% limit app permissions. Those figures point to a design lesson for enterprise systems: users may care strongly about privacy while lacking the time or clarity to manage complex controls manually.
AI governance must account for the same friction inside engineering teams. A 2026 ISACA summary of privacy findings states that 13% of privacy professionals currently use AI in their privacy function, while 38% plan to do so within 12 months. It also reports that nearly half consider technical privacy teams understaffed and only 22% expect privacy budgets to rise. Governance can't depend on a large specialist team reviewing every prompt. It needs enforceable platform defaults, clear ownership, and escalation paths for exceptions.
The winning architecture is neither unrestricted experimentation nor a compliance queue that blocks every release. It gives teams approved routes, narrow permissions, minimized data, safe telemetry, tested provider settings, and a response plan. That is how security and privacy become part of delivery velocity instead of an obstacle discovered after launch.
Openbase provides a unified API for accessing multiple language-model providers, with consolidated credentials and billing, provider failover, per-request usage visibility, and separate API keys for environment isolation. Review your AI data flows and retention requirements, then visit Openbase to evaluate whether its gateway model fits the controls your team needs.