Vault Secrets Management

HashiCorp Vault configuration and secrets management in Aurora

Aurora uses HashiCorp Vault to securely store user credentials, cloud provider tokens, and API keys.

Overview

Vault provides:

• Secure storage: Encrypted secrets at rest and in transit
• User scoping: Each user’s secrets are isolated under users/{user_id}/
• Dynamic secrets: Cloud provider credentials are resolved at runtime
• Audit logging: Track secret access and modifications

Architecture

Application → Vault KV v2 Engine → Encrypted Storage
                ↓
         aurora (mount)
                ↓
         users/{user_id}/
                ↓
    {secret_name} → secret data

Configuration

Environment Variables

VAULT_ADDR · string · default: "http://vault:8200" · required

Vault server address

VAULT_TOKEN · string · required

Vault root token for authentication. Auto-generated by make init.

On first startup, check the vault-init container logs for the root token.

VAULT_KV_MOUNT · string · default: "aurora"

KV v2 secrets engine mount point

VAULT_KV_BASE_PATH · string · default: "users"

Base path for user secrets

Persistent Storage

Vault data is persisted using Docker volumes:

• vault-data: Main Vault data storage
• vault-init: Initialization state and unseal keys

Auto-Initialization

The vault-init container automatically:

1. Initializes Vault on first startup
2. Stores unseal keys in the vault-init volume
3. Unseals Vault automatically
4. Generates a root token

First Run Setup

On first startup:

# Start Aurora
make init
make prod-prebuilt

# Check vault-init logs for the root token
docker logs aurora-vault-init-1 | grep "Root Token"

# Update .env with the root token
VAULT_TOKEN=hvs.XXXXXXXXXXXXXXXXXXXX

# Restart containers
make down
make prod-prebuilt

Secret Management

Secret Path Format

Secrets are stored in the KV v2 engine with the following structure:

aurora/                    # KV mount
  └── users/              # Base path
      └── {user_id}/      # User-scoped secrets
          ├── aws/        # Cloud provider credentials
          ├── gcp/
          ├── azure/
          └── {custom}/   # Custom secret names

Database References

In the Aurora database, secrets are stored as references rather than actual values:

vault:kv/data/aurora/users/{user_id}/{secret_name}

At runtime, Aurora resolves these references by:

1. Parsing the reference format
2. Authenticating to Vault with VAULT_TOKEN
3. Fetching the secret data
4. Using the credentials

Example: Storing AWS Credentials

from utils.vault import store_user_secret, get_user_secret

# Store AWS credentials
store_user_secret(
    user_id="user123",
    secret_name="aws",
    data={
        "access_key_id": "AKIAIOSFODNN7EXAMPLE",
        "secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCY"
    }
)

# Retrieve AWS credentials
aws_creds = get_user_secret(user_id="user123", secret_name="aws")
print(aws_creds["access_key_id"])  # AKIAIOSFODNN7EXAMPLE

Vault CLI Operations

You can interact with Vault directly using the CLI:

Access Vault Shell

# Enter vault container
docker exec -it aurora-vault-1 sh

# Set token
export VAULT_TOKEN=hvs.XXXXXXXXXXXXXXXXXXXX

List Secrets

# List all user secret keys
vault kv list aurora/users/

# List secrets for a specific user
vault kv list aurora/users/user123/

Read a Secret

# Read secret data
vault kv get aurora/users/user123/aws

# Read in JSON format
vault kv get -format=json aurora/users/user123/aws

Write a Secret

# Write a simple secret
vault kv put aurora/users/user123/test-secret value="hello"

# Write multiple fields
vault kv put aurora/users/user123/database \
  host="db.example.com" \
  username="admin" \
  password="secret123"

Delete a Secret

# Soft delete (can be recovered)
vault kv delete aurora/users/user123/test-secret

# Permanently delete all versions
vault kv metadata delete aurora/users/user123/test-secret

Test Vault Connection

# Check Vault status
vault status

# Test authentication
vault token lookup

# Test KV engine
vault kv put aurora/users/test-user/test-secret value="test"
vault kv get aurora/users/test-user/test-secret
vault kv delete aurora/users/test-user/test-secret

API Integration

Storing User Secrets

When users connect cloud providers through the UI, Aurora:

1. Receives OAuth tokens or API keys
2. Stores them in Vault under the user’s path
3. Saves a Vault reference in the database

# In server/connectors/gcp_connector.py
from utils.vault import store_user_secret

def save_gcp_credentials(user_id: str, credentials: dict):
    # Store in Vault
    store_user_secret(
        user_id=user_id,
        secret_name="gcp",
        data={
            "token": credentials["access_token"],
            "refresh_token": credentials["refresh_token"],
            "project_id": credentials["project_id"]
        }
    )
    
    # Save reference in database
    save_to_db(
        user_id=user_id,
        provider="gcp",
        credential_path="vault:kv/data/aurora/users/" + user_id + "/gcp"
    )

Retrieving Secrets

from utils.vault import get_user_secret

def get_gcp_client(user_id: str):
    # Retrieve from Vault
    creds = get_user_secret(user_id=user_id, secret_name="gcp")
    
    # Use credentials
    return build_gcp_client(
        token=creds["token"],
        project_id=creds["project_id"]
    )

Security Best Practices

Token Management

The root token has unlimited access to Vault. In production:

1. Store it securely (password manager, secrets manager)
2. Rotate it regularly
3. Use AppRole authentication for services
4. Never commit it to version control

Production Considerations

1. Use AppRole Authentication: Instead of the root token, create AppRole credentials for Aurora services
2. Enable Audit Logging: Track all secret access
3. Implement Secret Rotation: Regularly rotate cloud provider credentials
4. Backup Vault Data: Regularly backup the vault-data volume
5. Use TLS: Enable TLS for Vault API communication

AppRole Example (Production)

# Enable AppRole authentication
vault auth enable approle

# Create a policy for Aurora
vault policy write aurora-policy - <<EOF
path "aurora/users/*" {
  capabilities = ["create", "read", "update", "delete", "list"]
}
EOF

# Create an AppRole
vault write auth/approle/role/aurora \
  secret_id_ttl=0 \
  token_ttl=1h \
  token_max_ttl=4h \
  policies="aurora-policy"

# Get Role ID and Secret ID
vault read auth/approle/role/aurora/role-id
vault write -f auth/approle/role/aurora/secret-id

# Use in Aurora (update .env)
VAULT_ROLE_ID=<role_id>
VAULT_SECRET_ID=<secret_id>

Troubleshooting

Vault Not Initialized

# Check vault-init logs
docker logs aurora-vault-init-1

# If init failed, remove volumes and restart
make down
docker volume rm aurora-vault-data aurora-vault-init
make prod-prebuilt

Vault Sealed

# Check status
docker exec aurora-vault-1 vault status

# Unseal manually (if auto-unseal failed)
docker exec -it aurora-vault-1 vault operator unseal

Connection Errors

# Check Vault is running
docker ps | grep vault

# Check Vault logs
docker logs aurora-vault-1

# Test connection from aurora-server
docker exec aurora-server-1 wget -qO- http://vault:8200/v1/sys/health

Permission Denied

# Verify token is valid
docker exec -e VAULT_TOKEN=$VAULT_TOKEN aurora-vault-1 vault token lookup

# Check policy permissions
docker exec -e VAULT_TOKEN=$VAULT_TOKEN aurora-vault-1 vault policy read default

Migration from Database Storage

If you’re migrating from storing secrets directly in the database:

1. Export existing secrets from the database
2. Store in Vault using the format above
3. Update database records to Vault references
4. Delete plaintext secrets from the database

# Migration script example
from utils.vault import store_user_secret

def migrate_user_secrets(user_id: str, db_secrets: dict):
    for provider, credentials in db_secrets.items():
        # Store in Vault
        store_user_secret(
            user_id=user_id,
            secret_name=provider,
            data=credentials
        )
        
        # Update DB with Vault reference
        update_db_credential(
            user_id=user_id,
            provider=provider,
            credential_path=f"vault:kv/data/aurora/users/{user_id}/{provider}"
        )
        
        # Delete plaintext from DB
        delete_plaintext_credential(user_id=user_id, provider=provider)

Web UI

Vault’s web UI is available at http://localhost:8200/ui

1. Enter your VAULT_TOKEN
2. Navigate to aurora/users/
3. View, create, and manage secrets

The Vault UI is useful for debugging but should be disabled or restricted in production.