Hashicorp Vault - Agent Injector - does it make sense? Hashicorp Vault - Agent Injector - does it make sense? kubernetes kubernetes

Hashicorp Vault - Agent Injector - does it make sense?


I think it definitely makes sense as the purpose is to avoid hard coding credentials in the spec.

inject agent saves the file with PLAIN text password in the /vaultpath, so everybody can see this secret.

Even in bare metal servers or cloud instances, credentials are saved in plain text. In case of k8s, it is inside a container. In both scenarios, you would control who can access your instances or k8s pods. Only authorised personnel shall have the required access to pods in production cluster(s).

how can I rotate credentials for application

The vault agent injector runs a sidecar container along with your app container in the same pod. Its purpose is to look for any changes in the vault secret at regular intervals. If you do a kubectl describe po <pod-name> you'll find a sidecar container vault-agent running.

kubectl get po app-example-deployment-7c4b45cf8-4fkr7NAME                                     READY   STATUS    RESTARTS   AGEapp-example-deployment-7c4b45cf8-4fkr7   2/2     Running   0          166m

kubectl describe pod app-example-deployment-7c4b45cf8-4fkr7 :

...vault-agent:    Container ID:  docker://b6f9df32ed903d684c972401f41e15a8f6b1bec62aa111bfd9c693159af1ff09    Image:         vault:1.7.0    Image ID:      docker-pullable://vault@sha256:635cf1c3f9b10fe03aad375f94cc61f63d74a189662165285a8bf1c189ea04b8    Port:          <none>    Host Port:     <none>    Command:      /bin/sh      -ec    Args:      echo ${VAULT_CONFIG?} | base64 -d > /home/vault/config.json && vault agent -config=/home/vault/config.json    State:          Running      Started:      Tue, 13 Apr 2021 15:40:10 +0100    Ready:          True    Restart Count:  0    Limits:      cpu:     500m      memory:  128Mi    Requests:      cpu:     250m      memory:  64Mi    Environment:      VAULT_LOG_LEVEL:   info      VAULT_LOG_FORMAT:  standard...

After successfully fetching the secret from vault during deployment:

kubectl exec -it app-example-deployment-7c4b45cf8-4fkr7 -c app -- cat /vault/secrets/db-credsmongodb+srv://testUser:testPass@test-5xxxx.mongodb.net/testDb

If I change the kv secret in Vault with password set to, let's say "testPass2", I need not do anything as the vault-agent sidecar container will update it for me automatically.

kubectl exec -it app-example-deployment-7c4b45cf8-4fkr7 -c app -- cat /vault/secrets/db-credsmongodb+srv://testUser:testPass2@test-5xxxx.mongodb.net/testDb

In the vault-agent sidecar container logs, you'll see something like.

kubectl logs app-example-deployment-7c4b45cf8-4fkr7 -c vault-agent --follow2021-04-13T14:40:10.426Z [INFO]  sink.file: creating file sink2021-04-13T14:40:10.426Z [INFO]  sink.file: file sink configured: path=/home/vault/.vault-token mode=-rw-r-----==> Vault agent started! Log data will stream in below:==> Vault agent configuration:                     Cgo: disabled               Log Level: info                 Version: Vault v1.7.0             Version Sha: 4e222b85c40a810b74400ee3c54449479e32bb9f2021-04-13T14:40:10.426Z [INFO]  template.server: starting template server[INFO] (runner) creating new runner (dry: false, once: false)2021-04-13T14:40:10.427Z [INFO]  auth.handler: starting auth handler2021-04-13T14:40:10.427Z [INFO]  auth.handler: authenticating2021-04-13T14:40:10.427Z [INFO]  sink.server: starting sink server[INFO] (runner) creating watcher2021-04-13T14:40:10.437Z [INFO]  auth.handler: authentication successful, sending token to sinks2021-04-13T14:40:10.437Z [INFO]  auth.handler: starting renewal process2021-04-13T14:40:10.437Z [INFO]  template.server: template server received new token[INFO] (runner) stopping[INFO] (runner) creating new runner (dry: false, once: false)[INFO] (runner) creating watcher[INFO] (runner) starting2021-04-13T14:40:10.437Z [INFO]  sink.file: token written: path=/home/vault/.vault-token2021-04-13T14:40:10.439Z [INFO]  auth.handler: renewed auth token[INFO] (runner) rendered "(dynamic)" => "/vault/secrets/db-creds"2021-04-13T15:23:43.315Z [INFO]  auth.handler: renewed auth token[INFO] (runner) rendered "(dynamic)" => "/vault/secrets/db-creds"2021-04-13T16:07:16.191Z [INFO]  auth.handler: renewed auth token