A Safer Pattern for AI API Credentials in Kubernetes

Kubernetes gives teams several ways to deliver confidential values to workloads. It does not decide whether one long-lived AI-provider key should be shared by every Pod that needs model access.

The Kubernetes documentation is direct about the boundary. A Secret can hold a password, token, or key, but Secret data is stored unencrypted in etcd by default unless the cluster is configured for encryption at rest. Base64 encoding is not encryption. A user who can create a Pod that consumes a Secret may also be able to expose its value through that Pod.

Kubernetes' official good practices for Secrets therefore recommend encryption at rest, least-privilege RBAC, restricted get, list, and watch access, namespace isolation, short-lived secrets, and limiting a Secret mount to only the containers that need it.

Start with the workload boundary

An AI workload should not receive a credential merely because another workload in the namespace needs the same provider. Define the boundary explicitly:

• one ServiceAccount per independently operated workload;
• RBAC that grants access only to the required Secret objects;
• a separate Secret mount for the specific container that makes outbound requests;
• network policy and egress controls appropriate to the cluster; and
• logs that exclude authorization headers, environment dumps, and prompt bodies by default.

Avoid granting broad Secret list access for convenience. Kubernetes notes that listing Secrets implicitly exposes their contents. Also review who can create Pods: denying direct Secret reads is not sufficient if the same principal can launch a Pod that mounts the Secret.

Reduce the value of what is distributed

The strongest cluster configuration still leaves a bearer credential inside a running workload. Reduce its blast radius at the upstream boundary.

For supported AI APIs, that can mean giving each workload its own provider project or service credential with provider-side restrictions. It can also mean placing a proxy in the request path and distributing a narrower proxy token instead of the upstream provider key. In either design, use distinct credentials for development, test, and production rather than copying one Secret between namespaces or clusters.

Rotation should be planned as an overlap-and-cutover process: introduce the replacement, update consumers, verify successful use, revoke the previous value, and confirm that no workload still depends on it. Restart behavior matters when credentials are injected as environment variables; volume-mounted values and application reload behavior should be tested rather than assumed.

Where Till fits—and where it does not

Till's hosted beta stores tenant provider connections encrypted at rest and issues provider-agnostic scoped bearer tokens with required activation ceilings. Separate tokens can improve attribution and let operators revoke or constrain one Kubernetes workload without rotating the upstream provider credential for every workload.

That does not make a Till token safe to expose. Store it as a Kubernetes Secret, apply the same RBAC and encryption guidance, and avoid logging it. Till does not configure cluster encryption, RBAC, network policy, or workload identity. It is also a single-region service, so applications need an explicit failure policy for proxy or provider unavailability.

Kubernetes secures how a confidential value is represented and delivered. Provider-side controls and workload-scoped credentials determine what that value can do. A defensible design needs both.

← Back to blog