39 External Secrets
Kim Oliver Drechsel edited this page 2026-04-14 19:51:19 +02:00

External secrets are secrets that are stored in an external secret management service and fetched during a deployment by Doco-CD. This allows you to keep your secrets out of your version control system and manage them in a secure way.

Supported External Secret Providers

Provider More Information
AWS Secrets Manager https://docs.aws.amazon.com/secretsmanager/latest/userguide/intro.html
Bitwarden Secrets Manager https://bitwarden.com/products/secrets-manager/
Bitwarden Vault / Vaultwarden https://bitwarden.com/help/vault-management-api/ and https://github.com/dani-garcia/vaultwarden
1Password https://1password.com
Infisical https://infisical.com/
OpenBao https://openbao.org/
Webhook Fetch secrets from any remote service via HTTP requests with a flexible configuration

Additional external secret providers may be supported in the future. If you have a specific provider in mind, please open an issue or submit a pull request.

Setting up an External Secret Provider

To use an external secret provider, you need to configure the environment variables for the provider you want to use.

AWS Secrets Manager

To use AWS Secrets Manager, you need to set the following environment variables:

Create an access token via IAM, see https://repost.aws/knowledge-center/create-access-key

Key Value
SECRET_PROVIDER aws_sm
SECRET_PROVIDER_REGION AWS Region to use, e.g. eu-west-1
SECRET_PROVIDER_ACCESS_KEY_ID Access key ID of an IAM user with access to AWS Secrets Manager
SECRET_PROVIDER_SECRET_ACCESS_KEY Secret access key of an IAM user with access to AWS Secrets Manager
SECRET_PROVIDER_SECRET_ACCESS_KEY_FILE Path to the file containing the secret access token inside the container

Deployment configuration

Add a mapping/reference between the environment variable you want to set in the docker compose project/stack and the ARN of the secret in AWS Secrets Manager.

Secrets can be retrieved in two ways:

  • As clear text/plain string: arn:aws:secretsmanager:region:account-id:secret:secret-name
  • With a path to the value (Used if the secret contains a JSON): arn:aws:secretsmanager:region:account-id:secret:secret-name/item

For example in your account 1234567890, the secret myapp in the region eu-west-1 contains this JSON value: {"username":"foo","password":"bar"}

If you want to get the secret value of the password field, use the ARN in addition with a slash (/) and the field name/key as the path: arn:aws:secretsmanager:eu-west-1:1234567890:secret:myapp/password.

Note

Without specifying a path, the entire JSON gets returned as a single string (see example below).

For example in your .doco-cd.yml:

external_secrets:
  JSON_STRING: "arn:aws:secretsmanager:eu-west-1:1234567890:secret:myapp"  # '{"username":"foo","password":"bar"}'
  APP_PASSWORD: "arn:aws:secretsmanager:eu-west-1:1234567890:secret:myapp/password" # bar

Bitwarden Secrets Manager

Important

Bitwarden Secrets Manager is not available in images for ARMv7 architectures (e.g. Raspberry Pi OS 32-bit).

To use Bitwarden Secrets Manager, you need to set the following environment variables:

Key Value Default
SECRET_PROVIDER bitwarden_sm
SECRET_PROVIDER_API_URL US: https://vault.bitwarden.com/api
EU: https://vault.bitwarden.eu/api
https://vault.bitwarden.com/api
SECRET_PROVIDER_IDENTITY_URL US: https://vault.bitwarden.com/identity
EU: https://vault.bitwarden.eu/identity
https://vault.bitwarden.com/identity
SECRET_PROVIDER_ACCESS_TOKEN Access token of a machine account, see the docs for machine accounts and access-tokens
SECRET_PROVIDER_ACCESS_TOKEN_FILE Path to the file containing the access token inside the container

Deployment configuration

Add a mapping/reference between the environment variable you want to set in the docker compose project/stack and the ID of the secret in Bitwarden Secrets Manager.

For example in your .doco-cd.yml:

name: myapp
external_secrets:
  DB_PASSWORD: 138e3a97-ed58-431c-b366-b35500663411

1Password

To use 1Password, you need to set the following environment variables:

Key Value
SECRET_PROVIDER 1password
SECRET_PROVIDER_ACCESS_TOKEN Access token of a service account, see the docs and here
SECRET_PROVIDER_ACCESS_TOKEN_FILE Path to the file containing the access token inside the container

Note

The start time and memory usage of the doco-cd container, as well as the runtime of a job, can increase significantly when using this secret provider.

Deployment configuration

Add a mapping/reference between the environment variable you want to set in the docker compose project/stack and the URI to the secret in 1Password.

See their docs for the correct syntax and how to get a secret reference of your secret: https://developer.1password.com/docs/cli/secret-reference-syntax/

A valid secret reference should use the syntax: op://<vault>/<item>/[section/]<field>

To get a one-time password, append the ?attribute=otp query parameter to a secret reference that points to a one-time password field in 1Password: op://<vault>/<item>/[section/]one-time password?attribute=otp

Important

Machine accounts can only access vaults for which you have granted read permissions during creation. The default Personal vault can't be access by machine accounts!

For example in your .doco-cd.yml:

name: myapp
external_secrets:
  DB_PASSWORD: "op://vault/item/field"

Bitwarden Vault / Vaultwarden

This section describes how to integrate Doco-CD with Bitwarden Vault or self-hosted Vaultwarden to securely manage and inject secrets during deployments.

How It Works

Doco-CD uses the webhook provider together with a lightweight Bitwarden REST API sidecar container that exposes your vault items over HTTP. The sidecar securely fetches secrets from Bitwarden/Vaultwarden and makes them available to Doco-CD for injection into your Docker Compose configurations.

Tip

This approach is fully compatible with both cloud-hosted Bitwarden Vault and self-hosted Vaultwarden instances.

Architecture Overview

The integration follows this workflow:

  1. Sidecar Service: Run the ghcr.io/kimdre/bitwarden-rest-api-server container alongside Doco-CD
  2. Store Configuration: Define webhook secret stores in a YAML file (SECRET_PROVIDER_WEBHOOK_STORES_FILE) that specify how to query items from the Bitwarden Vault Management API
  3. Secret References: In your .doco-cd.yml, reference vault items using store_ref and remote_ref to retrieve specific secrets

Prerequisites

  • A Bitwarden Vault account or self-hosted Vaultwarden instance
  • Personal API Key (OAuth2 credentials) from your Bitwarden account or organization (see Bitwarden Personal API Key)
  • Access to your Bitwarden item UUIDs (see Finding Item UUIDs below)

Note

The compose example below is not meant to replace your full Doco-CD compose file. Instead, treat it as the additional services, environment variables, volumes, and dependencies you need to add to your existing basic Doco-CD compose setup.

You can for example do this by creating a separate bitwarden-vault.compose.yml file and then use it together with your main compose file during deployment:

docker compose -f docker-compose.yml -f bitwarden-vault.compose.yml up -d

Setup Steps

Example compose setup

services:
  app: # doco-cd
    environment:
      SECRET_PROVIDER: webhook
      SECRET_PROVIDER_WEBHOOK_STORES_FILE: /secret-store.yml
    volumes:
      - ./secret-store.yml:/secret-store.yml:ro
    depends_on:
      bitwarden-api:
        condition: service_healthy
  
  bitwarden-api: # sidecar that provides the Bitwarden Vault Management API
    image: ghcr.io/kimdre/bitwarden-rest-api-server:latest
    environment:
      # Set these environment variables or use a .env file, see the image docs for more configuration options
      # Refer to the image documentation for the configuration options: https://github.com/kimdre/bitwarden-rest-api-server#getting-started
      - BW_HOST=${BW_HOST:-https://vault.bitwarden.com}
      - BW_CLIENTID=${BW_CLIENTID:?error}
      - BW_CLIENTSECRET=${BW_CLIENTSECRET:?error}
      - BW_PASSWORD=${BW_PASSWORD:?error}
    expose: # this makes the port only available inside the doco-cd compose project instead of on the entire host
      - "8087"
    restart: unless-stopped
    init: true
    read_only: true
    cap_drop:
      - ALL
    volumes:
      - bw-data:/data
    depends_on:
      set_permissions:
        condition: "service_completed_successfully"
        
  set_permissions: # required for correct bitwarden-api volume permissions 
    image: busybox:latest
    command: sh -c "chown -R 65532:65532 /data"
    volumes:
      - bw-data:/data
    restart: "no"

volumes:
  bw-data:

Environment variables:

Note

For all configuration options, refer to the image documentation at https://github.com/kimdre/bitwarden-rest-api-server#getting-started.

  • BW_HOST (optional) - Bitwarden or Vaultwarden API host. Defaults to https://vault.bitwarden.com.
    For Vaultwarden, use your self-hosted instance URL (e.g., https://vault.example.com)
  • BW_CLIENTID (required) - Client ID from your personal API Key credentials
  • BW_CLIENTSECRET (required) - Client Secret from your personal API Key credentials
  • BW_PASSWORD (required) - Master password for your Bitwarden account

Store these values in a .env file or set them inside the compose file.

For detailed information on setting up your personal API Key, see: https://bitwarden.com/help/personal-api-key/

Finding Item UUIDs

To reference secrets from Bitwarden in your .doco-cd.yml, you need the UUID of the Bitwarden item (vault entry).

Using Bitwarden CLI:

# Login to Bitwarden (if not already logged in)
bw login your-email@example.com

# List all items in your vault
bw list items

# The output will show each item with its id (UUID)

Example CLI output:

[
  {
    "object": "item",
    "id": "12345678-aaaa-bbbb-cccc-123456789abc",
    "name": "Database Credentials",
    "login": {
      "username": "dbuser",
      "password": "mysecretpassword"
    },
    "fields": [
      {
        "type": 0,
        "name": "api_key",
        "value": "sk-1234567890"
      }
    ]
  }
]

Using Bitwarden Web Vault:

  1. Open your Bitwarden vault at https://vault.bitwarden.com (or your Vaultwarden URL)
  2. Click on an item to view its details
  3. The UUID can be found in the URL (itemId=<UUID>)

Example webhook store file for Bitwarden Vault

stores:
  bitwarden-login:
    version: v1
    url: "http://bitwarden-api:8087/object/item/{{ .remote_ref.key }}"
    method: GET
    headers:
      Content-Type: application/json
    json_path: "data.login.{{ .remote_ref.property }}"

  bitwarden-fields:
    version: v1
    url: "http://bitwarden-api:8087/object/item/{{ .remote_ref.key }}"
    method: GET
    json_path: "data.fields[?name=='{{ .remote_ref.property }}'].value"

In this example:

  • bitwarden-login fetches built-in login fields such as username and password
  • bitwarden-fields fetches custom fields by name

Minimal .doco-cd.yml example

name: myapp
external_secrets:
  DB_PASSWORD:
    store_ref: bitwarden-login
    remote_ref:
      key: 12345678-aaaa-bbbb-cccc-123456789abc
      property: password

Extended .doco-cd.yml example

name: myapp
external_secrets:
  DB_USERNAME:
    store_ref: bitwarden-login
    remote_ref:
      key: 12345678-aaaa-bbbb-cccc-123456789abc
      property: username

  DB_PASSWORD:
    store_ref: bitwarden-login
    remote_ref:
      key: 12345678-aaaa-bbbb-cccc-123456789abc
      property: password

  API_KEY:
    store_ref: bitwarden-fields
    remote_ref:
      key: dddddddd-1111-2222-3333-eeeeeeeeeeee
      property: api_key

With this setup, Doco-CD resolves the secret value by:

  1. rendering the configured store templates using remote_ref
  2. calling the Bitwarden sidecar over HTTP
  3. extracting the value from the JSON response with json_path

Infisical

To use Infisical, you need to set the following environment variables:

Key Value
SECRET_PROVIDER infisical
SECRET_PROVIDER_SITE_URL The URL of the Infisical site (e.g. https://app.infisical.com, https://eu.infisical.com or your self-hosted instance URL)
SECRET_PROVIDER_CLIENT_ID The Client ID of a machine account, see the docs for machine accounts
SECRET_PROVIDER_CLIENT_SECRET The Client Secret of a machine account (Universal Auth)
SECRET_PROVIDER_CLIENT_SECRET_FILE Path to the file containing the client secret inside the container

Deployment configuration

Add a mapping/reference between the environment variable you want to set in the docker compose project/stack and the reference to the secret in Infisical.

A valid secret reference should use the syntax: projectId:env:[/some/path/]key

Important

Machine accounts can only access projects for which you have granted read permissions.

For example in your .doco-cd.yml:

name: myapp
external_secrets:
  TEST_PASSWORD: 0db45926-c97c-40d4-a3aa-fefd5d5fb492:dev:DATABASE_URL
  OTHER_PASSWORD: "0db45926-c97c-40d4-a3aa-fefd5d5fb492:dev:/Test/Sub/TEST_SECRET"
  USERNAME: 0db45926-c97c-40d4-a3aa-fefd5d5fb492:dev:Test/Sub/TEST_SECRET

OpenBao

To use OpenBao, you need to set the following environment variables:

Key Value
SECRET_PROVIDER openbao
SECRET_PROVIDER_SITE_URL The URL of the OpenBao instance
SECRET_PROVIDER_ACCESS_TOKEN Access token for authenticating with the secret provider
SECRET_PROVIDER_ACCESS_TOKEN_FILE Path to a file containing the access token inside the container

Deployment configuration

Add a mapping/reference between the environment variable you want to set in the docker compose project/stack and the reference to the key-value secret in OpenBao.

By default, the root namespace is used (root or /), but you can specify a different namespace by adding it as the first part of the reference.

  • A valid key-value secret reference should use the syntax: kv:<namespace(optional)>:<secretEngine>:<secretName>:<key>
  • A valid PKI certificate reference should use the syntax: pki:<namespace(optional)>:<secretEngine>:<commonName>

Examples of valid references:

  • kv:prod-secrets:db-prod:username → Fetches the username key from the db-prod key-value secret in the prod-secrets secret engine in the root namespace.
  • kv:root:prod-secrets:db-prod:username → Same as above, explicitly specifying the root namespace.
  • kv:my-namespace:secret:api-keys:stripe → Fetches the stripe key from the api-keys secret in the secret key-value secret engine in the my-namespace namespace.
  • pki:certs:myapp.example.com → Fetches the certificate for the common name myapp.example.com from the certs pki secret engine in the root namespace.
  • pki:my-namespace:certs:myapp.example.com → Fetches the certificate for the common name myapp.example.com from the certs pki secret engine in the my-namespace namespace.

For example in your .doco-cd.yml:

name: myapp
external_secrets:
  DB_USERNAME: kv:secret:db-prod:username
  DB_PASSWORD: kv:secret:db-prod:password
  CERT: pki:pki:myapp.example.com

To use the certificate in your compose file, you can pass the value to a compose config:

# docker-compose.yml
configs:
  myapp-example-com.crt:
    #environment: CERT  # Either pass the variable via the environment like this (without a $ sign)
    content: $CERT  # Or use the content field to directly inject the variable value to the config content

services:
  app:
    image: myapp:latest
    environment:
      DB_USERNAME: $DB_USERNAME
      DB_PASSWORD: $DB_PASSWORD
    configs:
      - source: myapp-example-com.crt
        target: /etc/ssl/certs/example.crt

Webhook

The webhook provider uses global secret stores defined in YAML. Each secret reference in .doco-cd.yml points to one store via store_ref and passes input values via remote_ref.

To use it, set the following environment variables:

Key Value
SECRET_PROVIDER webhook
SECRET_PROVIDER_WEBHOOK_STORES YAML content defining webhook stores (mutually exclusive with ..._FILE)
SECRET_PROVIDER_WEBHOOK_STORES_FILE Path to a YAML file containing the store definitions (mutually exclusive with ...STORES)
SECRET_PROVIDER_AUTH_USERNAME Optional auth value exposed in templates as {{ .auth.username }}
SECRET_PROVIDER_AUTH_PASSWORD Optional auth value exposed in templates as {{ .auth.password }}
SECRET_PROVIDER_AUTH_TOKEN Optional auth value exposed in templates as {{ .auth.token }}
SECRET_PROVIDER_AUTH_APIKEY Optional auth value exposed in templates as {{ .auth.api_key }}

Secret Store

Format

A store must define:

  • name
  • version (currently v1)
  • url
  • json_path

And can optionally define:

  • method (defaults to GET)
  • headers
  • body

json_path expressions use JMESPath syntax (for example data.login.password and data.fields[?name=='password'].value).

Example: map/list schema (stores:) and multi-document support
stores:
  bitwarden-login:
    version: v1
    url: "http://bitwarden-api:8087/object/item/{{ .remote_ref.key }}"
    method: GET
    headers:
      Content-Type: application/json
    json_path: "data.login.{{ .remote_ref.property }}"

  bitwarden-fields:
    version: v1
    url: "http://bitwarden-api:8087/object/item/{{ .remote_ref.key }}"
    method: GET
    json_path: "data.fields[?name=='{{ .remote_ref.property }}'].value"
---
name: akeyless
version: v1
url: "https://api.akeyless.io/v2/get-secret-value"
method: POST
headers:
  Content-Type: application/json
  Authorization: "Basic {{ print .auth.username \":\" .auth.password | b64enc }}"
body: '{"secret_name":"{{ .remote_ref.key }}","auth_method_access_token":"{{ .auth.token }}"}'
json_path: "value"

Deployment Configuration

For webhook, external_secrets entries must use object references. Legacy string refs (e.g. DB_PASSWORD: some-id) are rejected with a clear error.

name: myapp
external_secrets:
  DB_USERNAME:
    store_ref: bitwarden-login
    remote_ref:
      key: 12345678-aaaa-bbbb-cccc-123456789abc
      property: username

  DB_PASSWORD:
    store_ref: bitwarden-login
    remote_ref:
      key: 12345678-aaaa-bbbb-cccc-123456789abc
      property: password

  API_KEY:
    store_ref: bitwarden-fields
    remote_ref:
      key: dddddddd-1111-2222-3333-eeeeeeeeeeee
      property: api_key

Template Parameters

All store templates (url, headers, body, json_path) can use:

Key Description
remote_ref The object provided for that secret in .doco-cd.yml
auth Provider auth values from SECRET_PROVIDER_AUTH_* env vars

Available Template Functions

The following functions are available for use in all template fields:

Function Description Input Template Result
b64enc Encode input to base64 secret123 {{ "secret123" | b64enc }} c2VjcmV0MTIz
b64dec Decode base64 input c2VjcmV0MTIz {{ "c2VjcmV0MTIz" | b64dec }} secret123
urlencode URL encode input hello world {{ "hello world" | urlencode }} hello+world
urldecode URL decode input hello+world {{ "hello+world" | urldecode }} hello world
json Convert input to JSON string map[key:value] {{ .remote_ref.data | json }} {"key":"value"}
toUpper Convert input to uppercase hello {{ "hello" | toUpper }} HELLO
toLower Convert input to lowercase HELLO {{ "HELLO" | toLower }} hello
trim Trim whitespace from input hello {{ " hello " | trim }} hello

Real-World Examples

Example 1: Basic Authentication Header

headers:
  Authorization: "Basic {{ print .auth.username \":\" .auth.password | b64enc }}"

With auth.username=admin and auth.password=secret123:

  • Result: Authorization: Basic YWRtaW46c2VjcmV0MTIz

Example 2: URL-encoded Query Parameter

url: "https://api.example.com/search?q={{ .remote_ref.query | urlencode }}"

With remote_ref.query=hello world:

  • Result: https://api.example.com/search?q=hello+world

Example 3: JSON Request Body

body: '{"filters":{{ .remote_ref.filters | json }}}'

With remote_ref.filters=map[status:active type:user]:

  • Result: {"filters":{"status":"active","type":"user"}}

Example 4: Trimmed and Uppercase API Key Header

headers:
  X-API-Key: "{{ .remote_ref.api_key | trim | toUpper }}"

With remote_ref.api_key= my-secret-key :

  • Result: X-API-Key: MY-SECRET-KEY

Example 5: Decoded and Trimmed Secret

json_path: "secret[?key=='{{ .remote_ref.encoded_id | b64dec | trim }}']"

With remote_ref.encoded_id=c2VjcmV0LWtleQ==:

  • Result: secret[?key=='secret-key']

Important

The provider fails fast when:

  • store_ref does not exist
  • a referenced remote_ref field is missing
  • json_path is missing or renders empty

Using External Secrets in Deployments

Doco-CD uses variable interpolation to replace variables in your Compose files with the values fetched from the external secret provider, see the Compose file reference for more information and examples. For example with Bitwarden Secrets Manager, if you want to use secrets named DB_PASSWORD and LABEL_SECRET in your Compose file, you can reference it like this:

#.doco-cd.yml
name: myapp
external_secrets:
  DB_PASSWORD: a8f1e4eb-d76d-47b4-aa3c-103733e77fce
  LABEL_SECRET: cfd0c4a9-16d4-44c8-9a80-c6143a7c7b71

Then you can use the variable in your Compose file like this:

Important

External secrets have a higher priority than variables set in a .env file or in the environment. If a variable is set in both an external secret and in a .env file, the value from the external secret will be used.

#.env
DB_PASSWORD=testpassword # This will be overridden by the external secret
DOMAIN=example.com
#docker-compose.yml
services:
  app:
    image: myapp:latest
    environment:
      DATABASE_HOST: db
      DATABASE_USER: ${$DB_USER:-postgres} # You can also set a default value if the secret is missing
      DATABASE_PASSWORD: $DB_PASSWORD
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.myapp.rule=Host(`myapp.${DOMAIN}`)"  # Note that DOMAIN is set in a local .env file and not fetched from the secret provider
  db:
    image: postgres:latest
    environment:
      POSTGRES_USER: ${$DB_USER:-postgres}
      POSTGRES_PASSWORD: ${DB_PASSWORD}

This will result in the following docker-compose configuration being used during deployment:

#docker-compose.yml
services:
  app:
    image: myapp:latest
    environment:
      DATABASE_HOST: db
      DATABASE_USER: postgres
      DATABASE_PASSWORD: supersecretpassword123 # Value of external secret fetched from Bitwarden Secrets Manager
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.myapp.rule=Host(`myapp.example.com`)"  # Value from .env file
  db:
    image: postgres:latest
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: supersecretpassword123 # Value of external secret fetched from Bitwarden Secrets Manager