Integrate with Azure Blob Store

Owned by Dave Trier
Jun 15, 2022
3 min read

ModelOp Center seamlessly integrates with existing Azure Blob Store instances to allow enterprises to leverage existing IT investments in their data platforms.

Table of Contents

 

Introduction

ModelOp Center provides the ability to integrate with Azure Blob Storage as an external file repository. In other words, when models are imported or assets are uploaded they can be placed in an Azure Blob Storage container.

Azure Blob Storage Overview

Three main components need to be configured to enable the Azure Blob Storage service:

  • The profile

  • The external-file-repo-config

  • BucketInfoList details

The profile

The azure-blob-storage-client profile must be active to make Azure configuration available (note, you may need to add to other active profiles as shown below). Furthermore, you can use mixtures of Azure/S3 and allow different groups to use different stores by configuring the BucketInfoList detail (see below).

spring: profiles: active: minio-s3-client,azure-blob-storage-client,aws-s3-client,secured,jwt

 

The External File Repo Config

Specify the connection settings used by Azure Blob Storage

Property

Details

Property

Details

azure-defaultStorageAccountName

Default Storage Account Name used when there is no match under stored credentials

azure-defaultStorageAccountKey

Default Storage Account Key used when there is no match under stored credentials

azure-defaultBucket

Default Storage Bucket Name used when there is no match under stored credentials

storedCredentials.storageAccountName

Storage Account Name used when there is a path match

storedCredentials.storageAccountKey

Storage Account Key used when there is a path match. Note: When using USERNAME_PASSWORD authentication type the user password should be set in this field.

storedCredentials.path

Path used to determine if there is a match

storedCredentials.authenticationType

Alternate Auth Type, default value is SHARED_KEY Use value USERNAME_PASSWORD when using Username/Password authentication

storedCredentials.port

(Optional) Port value used in the connection, used for meta data only with Azure

storedCredentials.secure

(Optional) Secure value used in the connection, used for meta data only with Azure

storedCredentials.location

(Optional) Location value used in the connection, used for meta data only with Azure

storedCredentials.clientId

ClientID value, used with Username/Password authentication

storedCredentials.username

Username value, used with Username/Password authentication.

external-file-repo-config: azure: azure-defaultStorageAccountName: ****ACCOUNT NAME HERE**** azure-defaultStorageAccountKey: ****KEY HERE**** azure-defaultBucket: modelop azure-credentials: storedCredentials: - storageAccountName: ****ACCOUNT NAME HERE**** storageAccountKey: ****KEY HERE**** path: //authenticationType: // NOT REQUIRED Username/Password Auth only //clientId: // NOT REQUIRED Username/Password Auth only //username: // NOT REQUIRED Username/Password Auth only s3: ...

 

BucketList Info Details

When using both S3 and Azure repository you must specify what groups are associated with what storage type, make sure the storageType property is specified in the bucketInfoList detail. If this is not specified, the value will fallback to the default group value, and finally to use the S3_REPOSITORY.

model-manage: ... buckets: bucketInfoList: - group: modelop bucket: modelop path: storageType: S3_REPOSITORY - group: azureGroup <== NOTE: Indicates groups associated with Azure storage bucket: modelop path: storageType: AZURE_REPOSITORY <== NOTE: Azure storage type indicated here

Note: You may need to clear or comment out the model-manage.buckets.bucketInfoList in other configuration files to make sure it does not clear values

Â