Skip to content

HashiCorp Vault

Use case: Database credentials management

Acquiring database credentials from the Eurodat Client

In EuroDaT, Hashicorp Vault is used for managing the credentials of a PostgreSQL database role with custom rights. The purpose of the Vault in this case is to automatically generate the credentials for the user and invalidate the credentials after a specific time. These credentials can be received from the client controller by calling the endpoint:

  • At client controller Swagger UI in the Credential Endpoint Resource
  • Assuming the client controller runs on localhost, on port 12345, with curl:

```bash curl -X 'GET' \ 'https://localhost:12345/api/v1/client-controller/credential-service/database/external/' \ -H 'accept: application/json'

Interaction with Vault in the EuroDaT Cluster

The diagram illustrates the interaction between various EuroDaT components and HashiCorp Vault, highlighting Vault's involvement in three key functions: client registration, credential retrieval, and the registration of the internal Postgres role with every setup of a new transaction database (DB). The vault-workflow is thus embedded in the transaction-database-workflow (illustrated in blue, top figure), the client-registration-workflow (illustrated in pink, top figure) and the credential-retrieval-workflow (bottom figure).

The two Access-Tokens are integral to form a connection with Vault and build the first two steps of every Vault-workflow:

  • Acquisition of the JSON Web Token (JWT)
  • Retrieval of the Vault token

To get credentials for a role only an additional Get-request is necessary. However, this credential flow is responsible for the final creation of the time-limited credentials. The other two vault workflows are more complex comprising 5 steps which are triggered every time Vault grants credentials for database roles. A detailed description of all steps can be found below the diagram.

VaultFlowDiagram

Initiating a Transaction:

  • To start a transaction the /api/client-controller/apps-endpoint in the client-cluster initiates the /api/controller/apps-endpoint in the eurodat-cluster.
  • Activation of the database-service/transaction-database-endpoint generates a unique transaction ID and a dedicated database for the transaction. The transaction is saved by calling the transaction-service/<transactionId>-endpoint.
  • Subsequently, the system creates an internal Postgres role with specified access to this transaction database.
  • To finalize the access grant, the credential-service/database-endpoint is invoked, triggering the vault credential flow.
  • The vault credential flow comprises four steps:
  • Utilizing the Keycloak client eurodat-vault within the realm eurodat-vault, a JWT token is acquired.
  • Using the obtained JWT token, the credential-service obtains a Vault token by querying the POST-/auth/jwt/login-endpoint of HashiCorp Vault.
  • With the Vault token the credential-service configures a Vault "Connection" through the POST-/database/config-endpoint of HashiCorp Vault for the internal Postgres role linked to the transaction database.
  • After repeating steps 1. and 2., the newly obtained Vault token is employed to register a role associated with the connection in HashiCorp Vault using the POST-/database/roles-endpoint. The Postgres role <transactionId>_internal is linked to the Vault role defined as internal_database_<transactionId>.
  • To create time-limited credentials for the Postgres role the /database/creds-endpoint must be called.
  • This workflow is illustrated by the blue workflow

Client Registration:

  • To register a new client, the /api/controller/clients-endpoint in the eurodat-cluster is invoked.
  • Following the initiation of external users in Keycloak and the creation of a Postgres role, a POST request is sent to the credential-service/database/external-endpoint to grant access to this role, triggering the vault credential flow.
  • The vault credential flow for client registration involves four stages:
  • Leveraging the Keycloak client eurodat-vault within the realm eurodat-vault to obtain a JWT token.
  • Using the obtained JWT token, the credential-service acquires a Vault token by querying the POST-/auth/jwt/login-endpoint of HashiCorp Vault.
  • With the Vault token, the credential-service configures a connection for the external Postgres role of the client in registration through the POST-/database/config-endpoint of HashiCorp Vault.
  • After repeating steps 1. and 2., the newly obtained Vault token is utilized to register a role associated with each connection in HashiCorp Vault using the POST-/database/roles-endpoint. The Postgres role db_<clientId>_external is linked to the Vault role defined as external_<clientId>_database
  • To create time-limited credentials for the Postgres role the /database/creds-endpoint must be called.
  • This workflow is illustrated by the pink workflow

Credential Retrieval:

  • To obtain credentials, the /api/client-controller/credential-service/database/external-endpoint in the client-cluster triggers a GET request to the /api/credential-service/database/external-endpoint in the eurodat-cluster, initiating the vault credential flow.
  • The vault credential flow for credential retrieval encompasses three steps:
  • Using the Keycloak client eurodat-vault within the realm eurodat-vault to obtain a JWT token.
  • Utilizing the obtained JWT token, the credential-service obtains a Vault token by querying the POST-/auth/jwt/login-endpoint of HashiCorp Vault.
  • Using the obtained Vault token, the credential-service calls the database/creds-endpoint of HashiCorp Vault to retrieve credentials for the registered role.
  • Consequently, calling the /database/creds-endpoint creates time-limited credentials for the Postgres role.