Integrate with Vault
ModelOp Center seamlessly integrates with existing credential management systems, such as Vault, to allow enterprises to leverage existing IT investments.
Table of Contents
Introduction
HashiCorp Vault is a secrets management solution that brokers access for both humans and machines, through programmatic access, to systems. For more details about Vault please visit their website here.
ModelOp Center provides the ability to integrate with Vault to secure the credentials for core ModelOp Center services. It extends the overall credentials framework that supports third party providers to the core ModelOp Center services.
The image above depicts the way ModelOp Center’s credentials framework leverages the Vault integration to retrieve secrets from Vault in order to connect to different core services. The sagemaker-service, not depicted on the diagram, also supports vault credentials.
Vault Integration Setup
Prerequisites
Core services
This guide assumes a basic setup of the core ModelOp Center services has been performed prior to attempt to retrieve the service’s credentials using Vault.
The ModelOp Center services supported for integration with vault are listed below:
Git (via model-manager)
S3 (via model-manager)
MongoDb (via model-manager)
Jenkins (via model-manager)
Jira (via mlc-manager)
Service Now (via mlc-manager)
Veracode (via mlc-manager)
SageMaker Service (via sagemaker-service)
Package Configuration
As per the list above the Vault integration requires the “credentials-vault” jar to be present in the classpath of the model-manager
, mlc-manager
, and/or sagemaker-service
microservices, depending on the core service that it is needed for.
Vault Client Setup
ModelOp Center Vault integration supports token authentication with Vault server.
The Token may be injected as the property (or environment variable)
vault.token
, or may be passed in a file by providing the file name in the propertyvault.token_file
.The remote URL to connect to Vault server is provided with the property
vault.uri
in the form of a fully formed address.A default path using the environment property
vault.defaultPath
must be provided to retrieve the keys if no custom mapping is specified.
In general the Vault configuration can look as shown in the example below:
vault:
token_file: /modelop/vault-token
uri: https://vault:8200
defaultPath: secret/data/modelop-services
NOTE: The above will have to be done on model-manager
, mlc-manager
, and/or sagemaker-service
as needed by the core service.
Configuring Credentials for ModelOp Center Core Services
By default, Vault secrets will take precedence over the default Kubernetes secrets approach. In other words, the SpringBoot properties being loaded with secrets as part of property injection in the application are considered part of a core-service key only after looking it up in Vault first. Hence you can potentially store only the passwords on Vault and the users (or other core services usage keys) can be loaded from the properties.
By default, the ModelOp Center integration with Vault will search for specific keys on the configured default path. If it finds a matching key it will try to use the corresponding value in the credentials to connect to the matching core service. Please see each section below for a full list of keys looked up per core service.
Additionally, if the keys found in the Vault secrets are different from the ones expected on the list below, we can provide a list of mappings as part of the properties of the mlc-manager
, model-manager
, and/or sagemaker-service
as in the example below:
vault:
mappings.1.path: <Vault path of secret>
mappings.1.vaultKey: <key as found on Vault>
mappings.1.usageKey: <core service usage key>
mappings.2.path: <Vault path of secret>
mappings.2.vaultKey: <key as found on Vault>
mappings.2.usageKey: <core service usage key>
Example values for the variables above specified between <tags>:
Vault path of secret - As configured on Vault, this is where the secret resides; the host base URI doesn’t need to be included. For example “secret/data/modelop-services”.
Key as found on Vault - If the required key for the core service is not found on Vault, this is the key on Vault we need to map to our core services. For example “JENKINS_PASSWORD”.
Core service usage key - From the sections below, this is what each core service requires as usage key for each of the properties. For example “model-manage.jenkins-credentials.storedCredentials.password”.
NOTE: The mappings list above should continue the number sequence as we add more core services from Vault.
NOTE: The above will have to be done on model-manager
, mlc-manager
, and/or sagemaker-service
as needed by the core service.
Vault “usage key” per Core Service
To fill in on the example above vault.mappings
or to configure Vault using only the vault.defaultPath
please refer to the usageKey
s per core service below.
Git Credentials from Vault
Usage keys for use with Vault:
model-manage.git.storedCredentials.server
- The server to match for credentials, please note you can leverage useHttpPath on your git config here if that is the desired behavior.model-manage.git.storedCredentials.username
- Git username to connect to the remote repo.model-manage.git.storedCredentials.password
- Git remote password, may be the Personal Access Token (PAT) as required by account.model-manage.git.storedCredentials.identity
- For ssh login, this is the location of the identity file.model-manage.git.storedCredentials.passphrase
- For ssh login, this is the passphrase of the identity file.
S3 Credentials from Vault
Usage keys for use with Vault:
model-manage.s3-credentials.storedCredentials.path
- The prefix path of the s3 file to match for credentials.model-manage.s3-credentials.storedCredentials.authenticationType
- From options: BASIC, ASSUME_ROLE, ASSUME_ROLE_WITH_SAML, ASSUME_ROLE_WITH_WEB_IDENTITY.model-manage.s3-credentials.storedCredentials.accessKeyId
- (BASIC) The account’s access key.model-manage.s3-credentials.storedCredentials.secretAccessKey
- (BASIC) The account’s secret access key.model-manage.s3-credentials.storedCredentials.sessionPrefix
- (ASSUME_ROLE_WITH_WEB_IDENTITY) A prefix for the identifier for the assumed role session.model-manage.s3-credentials.storedCredentials.roleArn
- (ASSUME_ROLE_WITH_WEB_IDENTITY) The Amazon Resource Name (ARN) of the role that the caller is assuming.model-manage.s3-credentials.storedCredentials.webIdentityToken
- (ASSUME_ROLE_WITH_WEB_IDENTITY) The OAuth 2.0 access token or OpenID Connect ID token that is provided by the identity provider.model-manage.s3-credentials.storedCredentials.principalArn
- (not supported yet) The Amazon Resource Name (ARN) of the SAML provider in IAM that describes the IdP.model-manage.s3-credentials.storedCredentials.samlAssertion
- (not supported yet) The base-64 encoded SAML authentication response provided by the IdP.model-manage.s3-credentials.storedCredentials.duration
- (not supported yet) The duration, in seconds, of the role session.
Mongo Credentials from Vault
Usage keys for use with Vault:
spring.data.mongodb.uri
- The mongoDB connection string, it may contain all other elements herein. See here for more info on the format.spring.data.mongodb.database
- The mongoDB database to connect. If provided, this overrides the value specified within the connection stringuri
.spring.data.mongodb.username
- The mongoDB username to use to connect. If provided, this overrides the value specified within the connection stringuri
.spring.data.mongodb.password
- The mongoDB password to use to connect. If provided, this overrides the value specified within the connection stringuri
.
Jenkins Credentials from Vault
Usage keys for use with Vault:
model-manage.jenkins-credentials.storedCredentials.server
- The server to match for credentials, this needs to match the job’sJenkinsRuntimeInfo.jenkinsUrl
.model-manage.jenkins-credentials.storedCredentials.username
- Jenkins username to use to connect.model-manage.jenkins-credentials.storedCredentials.password
- Jenkins password to use to connect.
Jira Credentials from Vault
Usage keys for use with Vault:
mlc.jira-task-monitor.server
- The server to use for credentials for Jira ticketing system.mlc.jira-task-monitor.username
- Jira username to use to connect.mlc.jira-task-monitor.password
- Jira password to use to connect.
Service Now Credentials from Vault
Usage keys for use with Vault:
mlc.servicenow-task-monitor.server
- The server to use for credentials for Service Now ticketing system.mlc.servicenow-task-monitor.username
- Service Now username to use to connect.mlc.servicenow-task-monitor.password
- Service Now password to use to connect.
Veracode Credentials from Vault
Usage keys for use with Vault:
mlc.veracode-task-monitor.server
- The server to use for credentials to connect to Veracode.mlc.veracode-task-monitor.username
- Veracode username to use to connect.mlc.veracode-task-monitor.password
- Veracode password to use to connect.
SagerMaker Credentials from Vault
Usage key for use with Vault:
sagemaker-service.sagemaker-credentials.storedCredentials.group
- The ModelOp group to which the credentials applysagemaker-service.sagemaker-credentials.storedCredentials.region
- The AWS region for the credentialssagemaker-service.sagemaker-credentials.storedCredentials.authenticationType
- From options: BASIC, ASSUME_ROLE, ASSUME_ROLE_WITH_SAML, ASSUME_ROLE_WITH_WEB_IDENTITY.sagemaker-service.sagemaker-credentials.storedCredentials.accessKeyId
- (BASIC) The account’s access key.sagemaker-service.sagemaker-credentials.storedCredentials.secretAccessKey
- (BASIC) The account’s secret access key.sagemaker-service.sagemaker-credentials.storedCredentials.sessionPrefix
- (ASSUME_ROLE_WITH_WEB_IDENTITY) A prefix for the identifier for the assumed role session.sagemaker-service.sagemaker-credentials.storedCredentials.roleArn
- (ASSUME_ROLE_WITH_WEB_IDENTITY) The Amazon Resource Name (ARN) of the role that the caller is assuming.sagemaker-service.sagemaker-credentials.storedCredentials.webIdentityToken
- (ASSUME_ROLE_WITH_WEB_IDENTITY) The OAuth 2.0 access token or OpenID Connect ID token that is provided by the identity provider.sagemaker-service.sagemaker-credentials.storedCredentials.principalArn
- (not supported yet) The Amazon Resource Name (ARN) of the SAML provider in IAM that describes the IdP.sagemaker-service.sagemaker-credentials.storedCredentials.samlAssertion
- (not supported yet) The base-64 encoded SAML authentication response provided by the IdP.sagemaker-service.sagemaker-credentials.storedCredentials.duration
- (not supported yet) The duration, in seconds, of the role session.
Related Articles