Integrate with Vault

Introduction

HashiCorp Vault is a secrets management solution that brokers access for both humans and machines, through programmatic access, to systems. For more details about Vault please visit their website here.

ModelOp Center provides the ability to integrate with Vault to secure the credentials for core ModelOp Center services. It extends the overall credentials framework that supports third party providers to the core ModelOp Center services.

 

The image above depicts the way ModelOp Center’s credentials framework leverages the Vault integration to retrieve secrets from Vault in order to connect to different core services. The sagemaker-service, not depicted on the diagram, also supports vault credentials.

Vault Integration Setup

Prerequisites

Core services

This guide assumes a basic setup of the core ModelOp Center services has been performed prior to attempt to retrieve the service’s credentials using Vault.

The ModelOp Center services supported for integration with vault are listed below:

  1. Git (via model-manager)

  2. S3 (via model-manager)

  3. MongoDb (via model-manager)

  4. Jenkins (via model-manager)

  5. Jira (via mlc-manager)

  6. Service Now (via mlc-manager)

  7. Veracode (via mlc-manager)

  8. SageMaker Service (via sagemaker-service)

Package Configuration

As per the list above the Vault integration requires the “credentials-vault” jar to be present in the classpath of the model-manager, mlc-manager, and/or sagemaker-service microservices, depending on the core service that it is needed for.

Vault Client Setup

ModelOp Center Vault integration supports token authentication with Vault server.

  1. The Token may be injected as the property (or environment variable) vault.token, or may be passed in a file by providing the file name in the property vault.token_file.

  2. The remote URL to connect to Vault server is provided with the property vault.uri in the form of a fully formed address.

  3. A default path using the environment property vault.defaultPath must be provided to retrieve the keys if no custom mapping is specified.

In general the Vault configuration can look as shown in the example below:

vault: token_file: /modelop/vault-token uri: https://vault:8200 defaultPath: secret/data/modelop-services

NOTE: The above will have to be done on model-manager, mlc-manager, and/or sagemaker-service as needed by the core service.

Configuring Credentials for ModelOp Center Core Services

By default, Vault secrets will take precedence over the default Kubernetes secrets approach. In other words, the SpringBoot properties being loaded with secrets as part of property injection in the application are considered part of a core-service key only after looking it up in Vault first. Hence you can potentially store only the passwords on Vault and the users (or other core services usage keys) can be loaded from the properties.

By default, the ModelOp Center integration with Vault will search for specific keys on the configured default path. If it finds a matching key it will try to use the corresponding value in the credentials to connect to the matching core service. Please see each section below for a full list of keys looked up per core service.

Additionally, if the keys found in the Vault secrets are different from the ones expected on the list below, we can provide a list of mappings as part of the properties of the mlc-manager, model-manager, and/or sagemaker-service as in the example below:

vault: mappings.1.path: <Vault path of secret> mappings.1.vaultKey: <key as found on Vault> mappings.1.usageKey: <core service usage key> mappings.2.path: <Vault path of secret> mappings.2.vaultKey: <key as found on Vault> mappings.2.usageKey: <core service usage key>

Example values for the variables above specified between <tags>:

  1. Vault path of secret - As configured on Vault, this is where the secret resides; the host base URI doesn’t need to be included. For example “secret/data/modelop-services”.

  2. Key as found on Vault - If the required key for the core service is not found on Vault, this is the key on Vault we need to map to our core services. For example “JENKINS_PASSWORD”.

  3. Core service usage key - From the sections below, this is what each core service requires as usage key for each of the properties. For example “model-manage.jenkins-credentials.storedCredentials.password”.

NOTE: The mappings list above should continue the number sequence as we add more core services from Vault.

NOTE: The above will have to be done on model-manager, mlc-manager, and/or sagemaker-service as needed by the core service.

Vault “usage key” per Core Service

To fill in on the example above vault.mappings or to configure Vault using only the vault.defaultPath please refer to the usageKeys per core service below.

Git Credentials from Vault

Usage keys for use with Vault:

  1. model-manage.git.storedCredentials.server - The server to match for credentials, please note you can leverage useHttpPath on your git config here if that is the desired behavior.

  2. model-manage.git.storedCredentials.username - Git username to connect to the remote repo.

  3. model-manage.git.storedCredentials.password - Git remote password, may be the Personal Access Token (PAT) as required by account.

  4. model-manage.git.storedCredentials.identity - For ssh login, this is the location of the identity file.

  5. model-manage.git.storedCredentials.passphrase - For ssh login, this is the passphrase of the identity file.

S3 Credentials from Vault

Usage keys for use with Vault:

  1. model-manage.s3-credentials.storedCredentials.path - The prefix path of the s3 file to match for credentials.

  2. model-manage.s3-credentials.storedCredentials.authenticationType - From options: BASIC, ASSUME_ROLE, ASSUME_ROLE_WITH_SAML, ASSUME_ROLE_WITH_WEB_IDENTITY.

  3. model-manage.s3-credentials.storedCredentials.accessKeyId - (BASIC) The account’s access key.

  4. model-manage.s3-credentials.storedCredentials.secretAccessKey - (BASIC) The account’s secret access key.

  5. model-manage.s3-credentials.storedCredentials.sessionPrefix - (ASSUME_ROLE_WITH_WEB_IDENTITY) A prefix for the identifier for the assumed role session.

  6. model-manage.s3-credentials.storedCredentials.roleArn - (ASSUME_ROLE_WITH_WEB_IDENTITY) The Amazon Resource Name (ARN) of the role that the caller is assuming.

  7. model-manage.s3-credentials.storedCredentials.webIdentityToken - (ASSUME_ROLE_WITH_WEB_IDENTITY) The OAuth 2.0 access token or OpenID Connect ID token that is provided by the identity provider.

  8. model-manage.s3-credentials.storedCredentials.principalArn - (not supported yet) The Amazon Resource Name (ARN) of the SAML provider in IAM that describes the IdP.

  9. model-manage.s3-credentials.storedCredentials.samlAssertion - (not supported yet) The base-64 encoded SAML authentication response provided by the IdP.

  10. model-manage.s3-credentials.storedCredentials.duration - (not supported yet) The duration, in seconds, of the role session.

Mongo Credentials from Vault

Usage keys for use with Vault:

  1. spring.data.mongodb.uri - The mongoDB connection string, it may contain all other elements herein. See here for more info on the format.

  2. spring.data.mongodb.database - The mongoDB database to connect. If provided, this overrides the value specified within the connection string uri.

  3. spring.data.mongodb.username - The mongoDB username to use to connect. If provided, this overrides the value specified within the connection string uri.

  4. spring.data.mongodb.password - The mongoDB password to use to connect. If provided, this overrides the value specified within the connection string uri.

Jenkins Credentials from Vault

Usage keys for use with Vault:

  1. model-manage.jenkins-credentials.storedCredentials.server - The server to match for credentials, this needs to match the job’s JenkinsRuntimeInfo.jenkinsUrl.

  2. model-manage.jenkins-credentials.storedCredentials.username - Jenkins username to use to connect.

  3. model-manage.jenkins-credentials.storedCredentials.password - Jenkins password to use to connect.

Jira Credentials from Vault

Usage keys for use with Vault:

  1. mlc.jira-task-monitor.server - The server to use for credentials for Jira ticketing system.

  2. mlc.jira-task-monitor.username - Jira username to use to connect.

  3. mlc.jira-task-monitor.password - Jira password to use to connect.

Service Now Credentials from Vault

Usage keys for use with Vault:

  1. mlc.servicenow-task-monitor.server - The server to use for credentials for Service Now ticketing system.

  2. mlc.servicenow-task-monitor.username - Service Now username to use to connect.

  3. mlc.servicenow-task-monitor.password - Service Now password to use to connect.

Veracode Credentials from Vault

Usage keys for use with Vault:

  1. mlc.veracode-task-monitor.server - The server to use for credentials to connect to Veracode.

  2. mlc.veracode-task-monitor.username - Veracode username to use to connect.

  3. mlc.veracode-task-monitor.password - Veracode password to use to connect.

SagerMaker Credentials from Vault

Usage key for use with Vault:

  1. sagemaker-service.sagemaker-credentials.storedCredentials.group - The ModelOp group to which the credentials apply

  2. sagemaker-service.sagemaker-credentials.storedCredentials.region - The AWS region for the credentials

  3. sagemaker-service.sagemaker-credentials.storedCredentials.authenticationType - From options: BASIC, ASSUME_ROLE, ASSUME_ROLE_WITH_SAML, ASSUME_ROLE_WITH_WEB_IDENTITY.

  4. sagemaker-service.sagemaker-credentials.storedCredentials.accessKeyId - (BASIC) The account’s access key.

  5. sagemaker-service.sagemaker-credentials.storedCredentials.secretAccessKey - (BASIC) The account’s secret access key.

  6. sagemaker-service.sagemaker-credentials.storedCredentials.sessionPrefix - (ASSUME_ROLE_WITH_WEB_IDENTITY) A prefix for the identifier for the assumed role session.

  7. sagemaker-service.sagemaker-credentials.storedCredentials.roleArn - (ASSUME_ROLE_WITH_WEB_IDENTITY) The Amazon Resource Name (ARN) of the role that the caller is assuming.

  8. sagemaker-service.sagemaker-credentials.storedCredentials.webIdentityToken - (ASSUME_ROLE_WITH_WEB_IDENTITY) The OAuth 2.0 access token or OpenID Connect ID token that is provided by the identity provider.

  9. sagemaker-service.sagemaker-credentials.storedCredentials.principalArn - (not supported yet) The Amazon Resource Name (ARN) of the SAML provider in IAM that describes the IdP.

  10. sagemaker-service.sagemaker-credentials.storedCredentials.samlAssertion - (not supported yet) The base-64 encoded SAML authentication response provided by the IdP.

  11. sagemaker-service.sagemaker-credentials.storedCredentials.duration - (not supported yet) The duration, in seconds, of the role session.

 

 

Related Articles