Integrate with Vault

Introduction

HashiCorp Vault is a secrets management solution that brokers access for both humans and machines, through programmatic access, to systems. For more details about Vault please visit their website here.

ModelOp Center provides the ability to integrate with Vault to secure the credentials for the core ModelOp Center services. It extends the overall credentials framework that supports third party providers to the core ModelOp Center services.

 

The image above depicts the way ModelOp Center’s credentials framework leverages the Vault integration to retrieve secrets from Vault in order to connect to different core services.

Vault Integration Setup

Prerequisites

Installed moc-builder

This guide assumes moc-builder has been installed. Please request download instructions for moc-builder here. Before configuring moc-builder, the ServiceNow instance will need a service account created and configured. For an example of how to set up a service account see the instructions at https://community.servicenow.com/community?id=community_blog&sys_id=b4fca2a5dbd0dbc01dcaf3231f961900.

For simplicity purposes, the below sections will point out to environment variables that need to be found on the pods assuming a procedure like the following:

First, open the config.json file within moc-builder’s config directory and navigate to the mlc-manager section. It will contain the following:

"mlc-manager": { ... "env": { ... "mlc.my-variable": "value", ... }, ... },

To add or change the value of a given environment variable (like mlc.my-variable on the example above) replace the value with the desired (or add the variable and value if not present already.

Likewise to change the value of Kubernetes secrets injected, open up the file located in moc-builder at secrets/secrets.json and replace the value with the new value:

{ ... "JIRA_USERNAME": "demo", "JIRA_PASSWORD": "modelop", ... }

NOTE: For the below Vault configurations to take precedence over the default Kubernetes secrets we might require removing the default environment property at config.json being injected to the service in addition to removing the entry from the secrets.json file.

Core services

This guide assumes a basic setup of the core ModelOp Center services has been performed prior to attempt to retrieve the service’s credentials using Vault.

The ModelOp Center services supported for integration with vault are listed below:

  1. Git (via model-manager)

  2. S3 (via model-manager)

  3. MongoDb (via model-manager)

  4. Jenkins (via model-manager)

  5. Jira (via mlc-manager)

  6. Service Now (via mlc-manager)

  7. Veracode (via mlc-manager)

Package configuration

As per the list above the Vault integration requires the “credentials-vault” jar to be present in the classpath of the model-manager and/or mlc-manager microservices, depending on the core service that we need it for.

Image setup

Make sure to use the image tag that supports Vault integration, on your config.json use a tag that matches your ModelOp Center version

For example:

modelop/model-manager:2.3-vault modelop/mlc-manager:2.3-vault

Entrypoint setup

To achieve this with moc-builder we need to include the environment variable ENTRYPOINT_SCRIPT = classpath-entrypoint.sh as we initialize the two images mentioned, depending on the target core services we’ll need to retrieve credentials from Vault.

NOTE: The above will have to be done on model-manager and/or mlc-manager as needed by the core service.

Vault client setup

ModelOp Center Vault integration supports token authentication with Vault server.

  1. The Token may be injected as an environment variable vault.token or may be passed in a file and providing the file name in the environment variable vault.token_file.

  2. The remote URL to connect to Vault server is provided with the environment property vault.uri in the form of a fully formed address.

  3. A default path using the environment property vault.defaultPath must be provided to retrieve the keys if no custom mapping is specified.

In general the Vault configuration can look as shown in the example below:

NOTE: The above will have to be done on model-manager and/or mlc-manager as needed by the core service.

Configuring Credentials for ModelOp Center Core Services

By default Vault secrets will take precedence over the default Kubernetes secret approach, in other words the SpringBoot properties being loaded with the secrets as part of the environment variables injection on the application is considered part of the core-service key only after looking it up on Vault first. Hence you can potentially store only the passwords on Vault and the users (or other core services usage keys) can be loaded from the environment.

By default the ModelOp Center integration with Vault will search for specific keys on the default path configured. If it finds a the key it will try to use it as the value of the credentials used to connect to such core service. Please see each section below for a full list of keys looked up per core service.

Additionally, if the keys found on the Vault secrets are different from the ones expected on the list below, we can provide a list of mappings as part of the environment variables of the mlc-manager and/or model-manager like in the example below:

Example values for the variables above specified between <tags>:

  1. Vault path of secret - As configured on Vault, this is where the secret resides and the host base URI doesn’t need to be included. For example “secret/data/modelop-services”.

  2. Key as found on Vault - If the required by the core service is not found on Vault, this is the key on Vault we need to map to our core services. For example “JENKINS_PASSWORD”.

  3. Core service usage key - From the sections below, this is what each core service requires as usage key for each of the properties. For example “model-manage.jenkins-credentials.storedCredentials.password”.

Vault “usage key” per core service

To fill in on the example above vault.mappings or to configure Vault using only the vault.defaultPath please refer to the usageKeys per core service below.

Git credentials from Vault

Usage keys for use with Vault:

  1. model-manage.git.storedCredentials.server - The server to match for credentials, please note you can leverage useHttpPath on your git config here if that is the desired behavior.

  2. model-manage.git.storedCredentials.username - Git username to connect to the remote repo.

  3. model-manage.git.storedCredentials.password - Git remote password, may be the Personal Access Token (PAT) as required by account.

  4. model-manage.git.storedCredentials.identity - For ssh login, this is the location of the identity file.

  5. model-manage.git.storedCredentials.passphrase - For ssh login, this is the passphrase of the identity file.

S3 credentials from Vault

Usage keys for use with Vault:

  1. model-manage.s3-credentials.storedCredentials.path - The prefix path of the s3 file to match for credentials.

  2. model-manage.s3-credentials.storedCredentials.authenticationType - From options: BASIC, ASSUME_ROLE, ASSUME_ROLE_WITH_SAML, ASSUME_ROLE_WITH_WEB_IDENTITY.

  3. model-manage.s3-credentials.storedCredentials.accessKeyId - (BASIC) The account’s access key.

  4. model-manage.s3-credentials.storedCredentials.secretAccessKey - (BASIC) The account’s secret access key.

  5. model-manage.s3-credentials.storedCredentials.sessionPrefix - (ASSUME_ROLE_WITH_WEB_IDENTITY) A prefix for the identifier for the assumed role session.

  6. model-manage.s3-credentials.storedCredentials.roleArn - (ASSUME_ROLE_WITH_WEB_IDENTITY) The Amazon Resource Name (ARN) of the role that the caller is assuming.

  7. model-manage.s3-credentials.storedCredentials.webIdentityToken - (ASSUME_ROLE_WITH_WEB_IDENTITY) The OAuth 2.0 access token or OpenID Connect ID token that is provided by the identity provider.

  8. model-manage.s3-credentials.storedCredentials.principalArn - (not supported yet) The Amazon Resource Name (ARN) of the SAML provider in IAM that describes the IdP.

  9. model-manage.s3-credentials.storedCredentials.samlAssertion - (not supported yet) The base-64 encoded SAML authentication response provided by the IdP.

  10. model-manage.s3-credentials.storedCredentials.duration - (not supported yet) The duration, in seconds, of the role session.

Mongo credentials from Vault

Usage keys for use with Vault:

  1. spring.data.mongodb.uri - The mongoDB connection string, it may contain all other elements herein. See here for more info on the format.

  2. spring.data.mongodb.database - The mongoDB database to connect. If provided, this overrides the value specified within the connection string uri.

  3. spring.data.mongodb.username - The mongoDB username to use to connect. If provided, this overrides the value specified within the connection string uri.

  4. spring.data.mongodb.password - The mongoDB password to use to connect. If provided, this overrides the value specified within the connection string uri.

Jenkins credentials from Vault

Usage keys for use with Vault:

  1. model-manage.jenkins-credentials.storedCredentials.server - The server to match for credentials, this needs to match the job’s JenkinsRuntimeInfo.jenkinsUrl.

  2. model-manage.jenkins-credentials.storedCredentials.username - Jenkins username to use to connect.

  3. model-manage.jenkins-credentials.storedCredentials.password - Jenkins password to use to connect.

Jira credentials from Vault

Usage keys for use with Vault:

  1. mlc.jira-task-monitor.server - The server to use for credentials for Jira ticketing system.

  2. mlc.jira-task-monitor.username - Jira username to use to connect.

  3. mlc.jira-task-monitor.password - Jira password to use to connect.

Service Now credentials from Vault

Usage keys for use with Vault:

  1. mlc.servicenow-task-monitor.server - The server to use for credentials for Service Now ticketing system.

  2. mlc.servicenow-task-monitor.username - Service Now username to use to connect.

  3. mlc.servicenow-task-monitor.password - Service Now password to use to connect.

Veracode credentials from Vault

Usage keys for use with Vault:

  1. mlc.veracode-task-monitor.server - The server to use for credentials to connect to Veracode.

  2. mlc.veracode-task-monitor.username - Veracode username to use to connect.

  3. mlc.veracode-task-monitor.password - Veracode password to use to connect.