Secrets Management for AI Projects: Stop Hardcoding Your API Keys

Hardcoded API keys in AI projects are the single most common security failure in the field. I have reviewed dozens of AI codebases from teams at various stages of maturity, and the pattern is depressingly consistent. The ANTHROPIC_API_KEY lives in a Python file as a string literal. The OPENAI_API_KEY is pasted directly into a Jupyter notebook. The PINECONE_API_KEY sits in a config dictionary that gets committed to Git. Each of these is a ticking clock, counting down to the moment someone pushes the repository to GitHub and an automated scanner harvests the key within minutes.

AI projects are uniquely vulnerable because they consume more third-party API keys than almost any other type of software. A typical production AI service touches an LLM provider (Anthropic, OpenAI), a vector database (Pinecone, Weaviate), an embedding service, a cloud storage bucket, a database, and potentially a monitoring service. That is six or more secrets, each of which costs real money if compromised and requires emergency rotation if leaked.

This guide covers the progression from local development to production deployment, starting with .env files and ending with cloud-native secret managers.

The .env Pattern for Local Development

The twelve-factor app methodology (published over a decade ago and still the most practical architecture guide for cloud applications) states that configuration should be stored in the environment. For AI projects, this means every secret, every API endpoint, every model identifier, and every feature flag lives in environment variables, never in source code.

The .env file is the local implementation of this principle. It sits in your project root, is listed in .gitignore (always), and is loaded at application startup. The python-dotenv library reads this file and injects its values into os.environ, making them available throughout your application via os.environ["ANTHROPIC_API_KEY"] or os.getenv("ANTHROPIC_API_KEY").

But raw os.getenv() calls scattered throughout your codebase create their own problems. There is no validation, no type checking, no default values, and no single place to see what environment variables your application requires. This is where pydantic-settings transforms the pattern.

Pydantic-Settings: Typed, Validated Configuration

Pydantic-settings (the BaseSettings class from the pydantic-settings package) lets you define your application's configuration as a typed Python class. Each field has a type, a default value (or is marked as required), and optional validation. The class automatically reads values from environment variables, .env files, and even secret files (useful for Docker secrets).

The advantage for AI projects is immediate. Instead of calling os.getenv("LLM_TEMPERATURE") and getting a string that you need to cast to a float and validate is between 0 and 2, you define a field llm_temperature: float = Field(default=0.2, ge=0.0, le=2.0). Pydantic validates the value at startup. If someone sets LLM_TEMPERATURE=5 in their .env file, the application fails immediately with a clear error message instead of silently sending bad parameters to the LLM API.

For secrets specifically, Pydantic-settings supports a secrets_dir parameter that reads values from files. This integrates directly with Docker Swarm secrets and Kubernetes secrets, where each secret is mounted as a file in a directory.

Cloud Secret Managers for Production

.env files work for local development. They are completely inappropriate for production. Production secrets belong in cloud-native secret managers that provide encryption at rest, access control, audit logging, and automated rotation.

AWS Secrets Manager stores secrets as encrypted key-value pairs with automatic rotation support. For AI services deployed on AWS, it integrates with Lambda, ECS, and EKS through IAM role-based access. Your application retrieves secrets at startup using the boto3 client, with no secret values in environment variables, configuration files, or container images.

GCP Secret Manager follows the same pattern with Google Cloud's IAM system. For services deployed on Cloud Run (a common choice for AI APIs), the secrets can be mounted as environment variables or volumes directly in the deployment configuration, avoiding any code-level secret fetching.

OCI Vault (Oracle Cloud Infrastructure) provides a key management and secrets service that integrates with OCI Container Instances and OCI Functions. For teams using OCI's Autonomous Database with AI Vector Search, the Vault integration means database credentials and API keys are managed through a single, auditable service.

The critical principle across all three providers is the same: your application code never contains secret values. Secrets are injected by the platform (through environment variables, mounted files, or API calls at startup) and rotated independently of deployments.

The Emergency Rotation Playbook

Despite your best practices, secrets will eventually leak. A developer pastes an API key into a Slack message. A debug log captures a request header containing a bearer token. A CI/CD pipeline logs environment variables during a failed build.

The response must be immediate and systematic. Rotate the compromised key (not just the one you think leaked, but any key that was in the same .env file or secret group). Audit the access logs on the provider side to determine if the key was used by unauthorized parties. Update the secret in your cloud secret manager. Redeploy any services that consume the rotated secret. Then conduct a brief retrospective to close the gap that allowed the leak.

Automated rotation reduces the blast radius of leaks. If your keys rotate every 90 days automatically, a leaked key has a limited window of usefulness. AWS Secrets Manager and GCP Secret Manager both support rotation Lambda/Cloud Functions that generate new keys and update the consuming services without downtime.

Key Takeaways

Secrets management is not a "nice to have" for AI projects. It is the difference between a security posture and a security incident. The progression is clear: .env files with python-dotenv for local development, pydantic-settings for typed and validated configuration at every stage, cloud secret managers (AWS Secrets Manager, GCP Secret Manager, OCI Vault) for production, and an emergency rotation playbook for when things go wrong. Every secret your AI service touches costs real money if compromised. Treat them accordingly.

⚡ Version note: This guide covers pydantic-settings v2.x and current cloud secret manager APIs as of March 2026. Cloud provider interfaces evolve frequently. Always consult the official documentation for the latest integration patterns.

Follow Usama Nawaz for weekly deep dives on building production grade AI systems.