Setup Vault secret and authenticate with Kubernetes Service Account with use For Portworx encrypted volumes [Service Account Method]

Portworx Volumes
1

Setting Key Value Secret in Vault for Portworx Volume Encryption

Prerequisites :
Follow the steps to setup Vault and ready your Portworx Cluster to use Vault as secrets provider. Vault

# vault secrets enable -path=kv/portworx/alpha -version=2 kv
Success! Enabled the kv secrets engine at: kv/portworx/alpha/

# vault secrets list --detailed

Path                  Plugin       Accessor              Default TTL    Max TTL    Force No Cache    Replication    Seal Wrap    Options           Description                                                UUID
----                  ------       --------              -----------    -------    --------------    -----------    ---------    -------           -----------                                                ----
cubbyhole/            cubbyhole    cubbyhole_8b127211    n/a            n/a        false             local          false        map[]             per-token private secret storage                           3c25434d-eb6c-759a-c537-e5d4d8aeaa43
identity/             identity     identity_54bb167e     system         system     false             replicated     false        map[]             identity store                                             b51807bf-15d0-2b85-e5f2-0e209b1c5f06
kv/portworx/alpha/     kv           kv_18287242           system         system     false             replicated     false        map[version:2]    n/a                                                        1ce35677-2df8-d1fe-dff1-84eee7224510
sys/                  system       system_a8cd80fd       n/a            n/a        false             replicated     false        map[]             system endpoints used for control, policy and debugging    7149649d-8469-073f-8dfa-91139e24234d

`

  • Put the secret

# vault kv put kv/portworx/alpha/test-secret password=mypass1234
Success! Data written to: kv/portworx/alpha/test-secret

  • Check contents of the secret
    #vault kv get kv/portworx/alpha/test-secret 
====== Data ======
Key         Value
---         -----
password    mypass1234
  • Set and Read Vault Policy for Kubernetes to access Vault

# vault policy write portworx portworx-policy.hcl
Success! Uploaded policy: admin

# vault policy read portworx

# Read and List capabilities on mount to determine which version of kv backend is supported
 path "sys/mounts"
 {
 capabilities = ["read", "list"]
 }
# V1 backends (Using custom backend)
 # Provide full access to the portworx subkey
 # Provide -> VAULT_BASE_PATH=portworx to PX (optional)
 # Provide -> VAULT_BACKEND_PATH=custom-backend (required)
 path "kv/portworx/alpha/portworx/*"
 {
 capabilities = ["create", "read", "update", "delete", "list"]
 }
# V2 backends (Using custom backend )
 # Provide full access to the data/portworx subkey
 # Provide -> VAULT_BASE_PATH=portworx to PX (optional)
 # Provide -> VAULT_BACKEND_PATH=custom-backend (required)
 path "kv/portworx/alpha/data/*"
 {
 capabilities = ["create", "read", "update", "delete", "list"]
 }
  • Read

# vault read sys/auth

Key            Value
---            -----
kubernetes/    map[accessor:auth_kubernetes_7edfd487 config:map[default_lease_ttl:0 force_no_cache:false max_lease_ttl:0 token_type:default-service] description: local:false options:<nil> seal_wrap:false type:kubernetes uuid:580a9648-07fe-bc80-3cac-08f8c4c7caef]
token/         map[accessor:auth_token_09ee968b config:map[default_lease_ttl:0 force_no_cache:false max_lease_ttl:0 token_type:default-service] description:token based credentials local:false options:<nil> seal_wrap:false type:token uuid:22f56a2f-29bf-f479-486e-520dc7986243]
  • Check Auth List

# vault auth list

Path           Type          Accessor                    Description
----           ----          --------                    -----------
kubernetes/    kubernetes    auth_kubernetes_7edfd487    n/a
token/         token         auth_token_09ee968b         token based credentials
  • Set environment variables:
export VAULT_SA_NAME=$(kubectl get sa vault-auth -n kube-system -o jsonpath="{.secrets[*]['name']}")
export SA_JWT_TOKEN=$(kubectl get secret $VAULT_SA_NAME -n kube-system -o jsonpath="{.data.token}" | base64 --decode; echo)
export SA_CA_CRT=$(kubectl get secret $VAULT_SA_NAME -n kube-system  -o jsonpath="{.data['ca\.crt']}" | base64 --decode; echo)
  • Update Vault authentication

# vault write auth/kubernetes/role/portworx bound_service_account_names=portworx bound_service_account_namespaces=kube-system policies=portworx ttl=24h

Success! Data written to: auth/kubernetes/role/portworx
  • Read the Vault Authentication status with Kubernetes

# vault read auth/kubernetes/role/portworx

Key                                 Value
---                                 -----
bound_service_account_names         [portworx]
bound_service_account_namespaces    [kube-system]
policies                            [portworx]
token_bound_cidrs                   []
token_explicit_max_ttl              0s
token_max_ttl                       0s
token_no_default_policy             false
token_num_uses                      0
token_period                        0s
token_policies                      [portworx]
token_ttl                           24h
token_type                          default
ttl                                 24h
  • Create px-vault secret in the same namespace as Portworx.
echo http://70.0.71.11:8200 | base64
echo kv/portworx/alpha | base64
echo kubernetes | base64
echo portworx | base64

apiVersion: v1
kind: Secret
metadata:
  name: px-vault
  namespace: kube-system
type: Opaque
data:
  VAULT_ADDR: aHR0cDovLzcwLjAuNzEuMTE6ODIwMA==
  VAULT_BACKEND_PATH: a3YvcG9ydHdvcngvYWxwaGE=
  VAULT_AUTH_METHOD: a3ViZXJuZXRlcw==
  VAULT_AUTH_KUBERNETES_ROLE: cG9ydHdvcng=
  • Restart Portworx Service on all PX nodes.

kubectl label nodes --all px/service=restart
OR
kubectl label nodes px-node1 px-node2 px-node3 px/service=restart