JIT Privileged Access & Dynamic Secrets
Welcome to the HashiCorp Vault JIT Privileged Access lab. In this comprehensive exercise, you will deploy enterprise-grade secrets management with Just-In-Time (JIT) access patterns. By the end, you'll have dynamic database credentials, OIDC-based authentication via Authentik, and complete audit logging of all secret access.
This lab requires a working Zero-Trust SSO Gateway (Project A) with Authentik configured. Vault integrates with Authentik for OIDC-based authentication.
HashiCorp Vault is the industry standard for secrets management, used by thousands of enterprises including Adobe, Uber, Stripe, and Bloomberg. Vault skills command premium salaries — IAM Engineers with secrets management expertise earn $120K-$180K annually.
Without Vault: Passwords stored in config files, environment variables, or code. Same password used everywhere, never rotated. No idea who accessed what credentials. Leaked credentials = compromised forever.
With Vault: Credentials are generated on-demand, auto-expire, and every access is logged!
| Phase | Focus | Time | Machine |
|---|---|---|---|
| Phase 1 | Core Concepts | 20-30 min | 📖 Reading |
| Phase 2 | Vault Installation | 30-45 min | 🖥️ SERVER |
| Phase 3 | Basic Configuration | 30-45 min | 🖥️ SERVER |
| Phase 4 | Secrets Engines | 45-60 min | 🖥️ SERVER |
| Phase 5 | Authentik Integration | 30-45 min | 🖥️ SERVER + 🌐 BROWSER |
| Phase 6 | JIT Access Setup | 45-60 min | 🖥️ SERVER |
Total: 4-5 hours (can be split across multiple sessions)
Understanding Vault terminology before we build.
Vault has specific terminology. Understanding these concepts will make the hands-on sections much easier to follow.
| Engine | Purpose | Example |
|---|---|---|
| KV (Key-Value) | Store static secrets with versioning | API keys, configuration values |
| Database | Generate dynamic database credentials | PostgreSQL, MySQL, MongoDB |
| Transit | Encryption as a service | Encrypt data without storing keys |
| PKI | Certificate authority | Issue TLS certificates |
| SSH | SSH key signing/OTP | Manage SSH access to servers |
Traditional: "Here's the database password, don't share it" (everyone shares it)
JIT: "Request access when needed, get unique credentials that expire automatically"
JIT means zero standing privileges — no one has permanent admin access. You request it, use it, and it disappears.
Deploy HashiCorp Vault using Docker.
# Navigate to your identity stack directory
cd ~/identity-stack
# Create directory structure for Vault
mkdir -p vault/config # Configuration files
mkdir -p vault/data # Persistent storage
mkdir -p vault/logs # Log files
mkdir -p vault/policies # HCL policy files
# Verify structure
tree vault/ 2>/dev/null || ls -la vault/
# Create the Vault configuration file
cat > ~/identity-stack/vault/config/vault-config.hcl << 'EOF'
# ============================================
# VAULT SERVER CONFIGURATION
# ============================================
# User interface settings
ui = true
# Listener configuration - how Vault accepts connections
listener "tcp" {
address = "0.0.0.0:8200" # Listen on all interfaces
tls_disable = true # Disable TLS (Traefik handles TLS)
}
# Storage backend - where secrets are stored
storage "file" {
path = "/vault/data"
}
# API Address - how Vault identifies itself
api_addr = "http://127.0.0.1:8200"
# Cluster Address - for HA setups
cluster_addr = "http://127.0.0.1:8201"
# Disable memory lock (required for Docker)
disable_mlock = true
# Logging configuration
log_level = "info"
log_file = "/vault/logs/vault.log"
EOF
# Verify the file was created
cat ~/identity-stack/vault/config/vault-config.hcl
We're using file storage for simplicity. In production, you'd use:
Add this service to your existing docker-compose.yml or create a new one:
# Create a separate compose file for Vault
cat > ~/identity-stack/docker-compose.vault.yml << 'EOF'
# ============================================
# HASHICORP VAULT - Secrets Management
# ============================================
version: "3.8"
services:
vault:
image: hashicorp/vault:1.15
container_name: vault
restart: unless-stopped
cap_add:
- IPC_LOCK # Required for memory locking
environment:
VAULT_ADDR: "http://127.0.0.1:8200"
VAULT_API_ADDR: "http://127.0.0.1:8200"
ports:
- "8200:8200" # Vault API/UI
volumes:
- ./vault/config:/vault/config:ro # Configuration
- ./vault/data:/vault/data # Persistent data
- ./vault/logs:/vault/logs # Logs
- ./vault/policies:/vault/policies:ro # Policies
command: server -config=/vault/config/vault-config.hcl
networks:
- identity-network
labels:
- "traefik.enable=true"
- "traefik.http.routers.vault.rule=Host(\`vault.yourdomain.com\`)"
- "traefik.http.routers.vault.entrypoints=websecure"
- "traefik.http.routers.vault.tls.certresolver=letsencrypt"
- "traefik.http.services.vault.loadbalancer.server.port=8200"
networks:
identity-network:
external: true
EOF
# Replace yourdomain.com with your actual domain
sed -i 's/yourdomain.com/YOUR_ACTUAL_DOMAIN.com/g' \
~/identity-stack/docker-compose.vault.yml
# Verify the change
grep "Host" ~/identity-stack/docker-compose.vault.yml
cd ~/identity-stack
# Pull the Vault image
docker compose -f docker-compose.vault.yml pull
# Start Vault
docker compose -f docker-compose.vault.yml up -d
# Check container is running
docker compose -f docker-compose.vault.yml ps
# View initial logs
docker logs vault
Vault starts in a sealed state and must be initialized. This generates your master keys — these are critical to save!
# Initialize Vault with 1 key share (for lab simplicity)
# In production, use 5 key shares with threshold of 3
docker exec vault vault operator init -key-shares=1 -key-threshold=1
# ⚠️ CRITICAL: You will see output like this:
# Unseal Key 1: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
# Initial Root Token: hvs.xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
#
# SAVE THESE VALUES IMMEDIATELY!
Write down BOTH the Unseal Key and Root Token immediately!
If you lose these, you lose access to ALL your secrets forever. Store them in a password manager or physical safe.
# Unseal Vault with your unseal key
docker exec vault vault operator unseal YOUR_UNSEAL_KEY_HERE
# Check Vault status
docker exec vault vault status
Verify Vault is initialized and unsealed:
docker exec vault vault status
Vault is installed, initialized, and unsealed. You can now access the Vault UI at https://vault.yourdomain.com using your Root Token.
Enable secrets engines and create access policies.
# Login to Vault CLI (you'll be prompted for token)
docker exec -it vault vault login
# When prompted, paste your Root Token
# You should see: "Success! You are now authenticated."
# Verify you're logged in
docker exec vault vault token lookup
The KV (Key-Value) engine stores static secrets like API keys, configuration values, and passwords.
# Enable KV secrets engine version 2 at path "secret/"
docker exec vault vault secrets enable -path=secret kv-v2
# Verify it's enabled
docker exec vault vault secrets list
# Create a test secret
docker exec vault vault kv put secret/test \
password="mysecretpassword" \
username="testuser" \
api_key="sk-1234567890"
# Read the secret back
docker exec vault vault kv get secret/test
You should see your test secret:
Policies control who can access what secrets. Let's create policies for different roles.
# Create admin policy file
cat > ~/identity-stack/vault/policies/admin-policy.hcl << 'EOF'
# ============================================
# ADMIN POLICY - Full access to secrets
# ============================================
# Full access to all secrets
path "secret/*" {
capabilities = ["create", "read", "update", "delete", "list"]
}
# Manage auth methods
path "auth/*" {
capabilities = ["create", "read", "update", "delete", "list"]
}
# Manage policies
path "sys/policies/*" {
capabilities = ["create", "read", "update", "delete", "list"]
}
# Manage mounts (secrets engines)
path "sys/mounts/*" {
capabilities = ["create", "read", "update", "delete", "list"]
}
EOF
# Load the policy into Vault
docker exec vault vault policy write admin /vault/policies/admin-policy.hcl
# Verify policy was created
docker exec vault vault policy read admin
# Create developer policy file
cat > ~/identity-stack/vault/policies/developer-policy.hcl << 'EOF'
# ============================================
# DEVELOPER POLICY - Read-only access to app secrets
# ============================================
# Read app secrets only
path "secret/data/apps/*" {
capabilities = ["read", "list"]
}
path "secret/metadata/apps/*" {
capabilities = ["list"]
}
# Read database credentials
path "database/creds/readonly" {
capabilities = ["read"]
}
EOF
# Load the policy
docker exec vault vault policy write developer /vault/policies/developer-policy.hcl
# List all policies
docker exec vault vault policy list
You've enabled the KV secrets engine and created role-based access policies. Admins have full access, developers can only read specific paths.
Common issues and their solutions when working with Vault.
docker exec vault vault operator unseal YOUR_UNSEAL_KEY
vault token lookuprm -rf ~/identity-stack/vault/data/*docker restart vaultRemove Vault when finished with the lab.
cd ~/identity-stack
# Stop Vault container (data preserved)
docker compose -f docker-compose.vault.yml stop
# Verify it's stopped
docker compose -f docker-compose.vault.yml ps
This will permanently delete all secrets stored in Vault. This cannot be undone.
cd ~/identity-stack
# Stop and remove Vault container
docker compose -f docker-compose.vault.yml down
# Remove all Vault data
rm -rf ~/identity-stack/vault/data/*
rm -rf ~/identity-stack/vault/logs/*
# Optionally remove configuration too
# rm -rf ~/identity-stack/vault/
You've deployed enterprise-grade secrets management with Just-In-Time access patterns!