> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/Arvo-AI/aurora/llms.txt
> Use this file to discover all available pages before exploring further.

# Vault Secrets Management

> HashiCorp Vault configuration and secrets management in Aurora

Aurora uses HashiCorp Vault to securely store user credentials, cloud provider tokens, and API keys.

## Overview

Vault provides:

* **Secure storage**: Encrypted secrets at rest and in transit
* **User scoping**: Each user's secrets are isolated under `users/{user_id}/`
* **Dynamic secrets**: Cloud provider credentials are resolved at runtime
* **Audit logging**: Track secret access and modifications

## Architecture

```
Application → Vault KV v2 Engine → Encrypted Storage
                ↓
         aurora (mount)
                ↓
         users/{user_id}/
                ↓
    {secret_name} → secret data
```

## Configuration

### Environment Variables

<ParamField path="VAULT_ADDR" type="string" default="http://vault:8200" required>
  Vault server address
</ParamField>

<ParamField path="VAULT_TOKEN" type="string" required>
  Vault root token for authentication. Auto-generated by `make init`.

  On first startup, check the `vault-init` container logs for the root token.
</ParamField>

<ParamField path="VAULT_KV_MOUNT" type="string" default="aurora">
  KV v2 secrets engine mount point
</ParamField>

<ParamField path="VAULT_KV_BASE_PATH" type="string" default="users">
  Base path for user secrets
</ParamField>

## Persistent Storage

Vault data is persisted using Docker volumes:

* **vault-data**: Main Vault data storage
* **vault-init**: Initialization state and unseal keys

### Auto-Initialization

The `vault-init` container automatically:

1. Initializes Vault on first startup
2. Stores unseal keys in the `vault-init` volume
3. Unseals Vault automatically
4. Generates a root token

### First Run Setup

On first startup:

```bash theme={null}
# Start Aurora
make init
make prod-prebuilt

# Check vault-init logs for the root token
docker logs aurora-vault-init-1 | grep "Root Token"

# Update .env with the root token
VAULT_TOKEN=hvs.XXXXXXXXXXXXXXXXXXXX

# Restart containers
make down
make prod-prebuilt
```

## Secret Management

### Secret Path Format

Secrets are stored in the KV v2 engine with the following structure:

```
aurora/                    # KV mount
  └── users/              # Base path
      └── {user_id}/      # User-scoped secrets
          ├── aws/        # Cloud provider credentials
          ├── gcp/
          ├── azure/
          └── {custom}/   # Custom secret names
```

### Database References

In the Aurora database, secrets are stored as **references** rather than actual values:

```
vault:kv/data/aurora/users/{user_id}/{secret_name}
```

At runtime, Aurora resolves these references by:

1. Parsing the reference format
2. Authenticating to Vault with `VAULT_TOKEN`
3. Fetching the secret data
4. Using the credentials

### Example: Storing AWS Credentials

```python theme={null}
from utils.vault import store_user_secret, get_user_secret

# Store AWS credentials
store_user_secret(
    user_id="user123",
    secret_name="aws",
    data={
        "access_key_id": "AKIAIOSFODNN7EXAMPLE",
        "secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCY"
    }
)

# Retrieve AWS credentials
aws_creds = get_user_secret(user_id="user123", secret_name="aws")
print(aws_creds["access_key_id"])  # AKIAIOSFODNN7EXAMPLE
```

## Vault CLI Operations

You can interact with Vault directly using the CLI:

### Access Vault Shell

```bash theme={null}
# Enter vault container
docker exec -it aurora-vault-1 sh

# Set token
export VAULT_TOKEN=hvs.XXXXXXXXXXXXXXXXXXXX
```

### List Secrets

```bash theme={null}
# List all user secret keys
vault kv list aurora/users/

# List secrets for a specific user
vault kv list aurora/users/user123/
```

### Read a Secret

```bash theme={null}
# Read secret data
vault kv get aurora/users/user123/aws

# Read in JSON format
vault kv get -format=json aurora/users/user123/aws
```

### Write a Secret

```bash theme={null}
# Write a simple secret
vault kv put aurora/users/user123/test-secret value="hello"

# Write multiple fields
vault kv put aurora/users/user123/database \
  host="db.example.com" \
  username="admin" \
  password="secret123"
```

### Delete a Secret

```bash theme={null}
# Soft delete (can be recovered)
vault kv delete aurora/users/user123/test-secret

# Permanently delete all versions
vault kv metadata delete aurora/users/user123/test-secret
```

### Test Vault Connection

```bash theme={null}
# Check Vault status
vault status

# Test authentication
vault token lookup

# Test KV engine
vault kv put aurora/users/test-user/test-secret value="test"
vault kv get aurora/users/test-user/test-secret
vault kv delete aurora/users/test-user/test-secret
```

## API Integration

### Storing User Secrets

When users connect cloud providers through the UI, Aurora:

1. Receives OAuth tokens or API keys
2. Stores them in Vault under the user's path
3. Saves a Vault reference in the database

```python theme={null}
# In server/connectors/gcp_connector.py
from utils.vault import store_user_secret

def save_gcp_credentials(user_id: str, credentials: dict):
    # Store in Vault
    store_user_secret(
        user_id=user_id,
        secret_name="gcp",
        data={
            "token": credentials["access_token"],
            "refresh_token": credentials["refresh_token"],
            "project_id": credentials["project_id"]
        }
    )
    
    # Save reference in database
    save_to_db(
        user_id=user_id,
        provider="gcp",
        credential_path="vault:kv/data/aurora/users/" + user_id + "/gcp"
    )
```

### Retrieving Secrets

```python theme={null}
from utils.vault import get_user_secret

def get_gcp_client(user_id: str):
    # Retrieve from Vault
    creds = get_user_secret(user_id=user_id, secret_name="gcp")
    
    # Use credentials
    return build_gcp_client(
        token=creds["token"],
        project_id=creds["project_id"]
    )
```

## Security Best Practices

### Token Management

<Warning>
  The root token has unlimited access to Vault. In production:

  1. Store it securely (password manager, secrets manager)
  2. Rotate it regularly
  3. Use AppRole authentication for services
  4. Never commit it to version control
</Warning>

### Production Considerations

1. **Use AppRole Authentication**: Instead of the root token, create AppRole credentials for Aurora services
2. **Enable Audit Logging**: Track all secret access
3. **Implement Secret Rotation**: Regularly rotate cloud provider credentials
4. **Backup Vault Data**: Regularly backup the `vault-data` volume
5. **Use TLS**: Enable TLS for Vault API communication

### AppRole Example (Production)

```bash theme={null}
# Enable AppRole authentication
vault auth enable approle

# Create a policy for Aurora
vault policy write aurora-policy - <<EOF
path "aurora/users/*" {
  capabilities = ["create", "read", "update", "delete", "list"]
}
EOF

# Create an AppRole
vault write auth/approle/role/aurora \
  secret_id_ttl=0 \
  token_ttl=1h \
  token_max_ttl=4h \
  policies="aurora-policy"

# Get Role ID and Secret ID
vault read auth/approle/role/aurora/role-id
vault write -f auth/approle/role/aurora/secret-id

# Use in Aurora (update .env)
VAULT_ROLE_ID=<role_id>
VAULT_SECRET_ID=<secret_id>
```

## Troubleshooting

### Vault Not Initialized

```bash theme={null}
# Check vault-init logs
docker logs aurora-vault-init-1

# If init failed, remove volumes and restart
make down
docker volume rm aurora-vault-data aurora-vault-init
make prod-prebuilt
```

### Vault Sealed

```bash theme={null}
# Check status
docker exec aurora-vault-1 vault status

# Unseal manually (if auto-unseal failed)
docker exec -it aurora-vault-1 vault operator unseal
```

### Connection Errors

```bash theme={null}
# Check Vault is running
docker ps | grep vault

# Check Vault logs
docker logs aurora-vault-1

# Test connection from aurora-server
docker exec aurora-server-1 wget -qO- http://vault:8200/v1/sys/health
```

### Permission Denied

```bash theme={null}
# Verify token is valid
docker exec -e VAULT_TOKEN=$VAULT_TOKEN aurora-vault-1 vault token lookup

# Check policy permissions
docker exec -e VAULT_TOKEN=$VAULT_TOKEN aurora-vault-1 vault policy read default
```

## Migration from Database Storage

If you're migrating from storing secrets directly in the database:

1. **Export existing secrets** from the database
2. **Store in Vault** using the format above
3. **Update database records** to Vault references
4. **Delete plaintext secrets** from the database

```python theme={null}
# Migration script example
from utils.vault import store_user_secret

def migrate_user_secrets(user_id: str, db_secrets: dict):
    for provider, credentials in db_secrets.items():
        # Store in Vault
        store_user_secret(
            user_id=user_id,
            secret_name=provider,
            data=credentials
        )
        
        # Update DB with Vault reference
        update_db_credential(
            user_id=user_id,
            provider=provider,
            credential_path=f"vault:kv/data/aurora/users/{user_id}/{provider}"
        )
        
        # Delete plaintext from DB
        delete_plaintext_credential(user_id=user_id, provider=provider)
```

## Web UI

Vault's web UI is available at [http://localhost:8200/ui](http://localhost:8200/ui)

1. Enter your `VAULT_TOKEN`
2. Navigate to `aurora/users/`
3. View, create, and manage secrets

<Note>
  The Vault UI is useful for debugging but should be disabled or restricted in production.
</Note>