Integrate with Sagemaker

Owned by Dave Trier
Last updated: Sept 01, 2022 by Kira Diaz (Unlicensed)
5 min read

ModelOp allows users to connect with SageMaker in order to view and operate their models, jobs, and endpoint configurations / endpoints.

Table of Contents

 

SageMaker preparation

Within SageMaker AWS

Model

Navigate to your Models in AWS and select your target model.

 

As you can see, the model has certain settings, such as:

  1. Name, ARN, Creation Time

  2. A container in which a training job was run

  3. Certain network properties and tags

You will be using your target model’s name and your AWS credentials to upload to ModelOp center.

Endpoint configuration / Endpoints

Endpoint configuration

Our endpoint configuration is equivalent to a Snapshot/Deployable model in ModelOp Center, when creating an endpoint you must associate it with your target model. You can also use the relevant tags for your instance. Below is an example of an endpoint configuration.

 

Endpoint configurations include:

  1. Name, ARN, Creation Time

  2. Production variants such as a model

    1. An Endpoint configuration could have 1 or more models associated with it.

  3. Tags

 

Endpoint

Endpoints are the equivalent of a Deployed Model running on a run time in ModelOp Center, on the endpoint configuration there is a section present that is used to create this endpoint as you can see in the above image in the top right corner.

 

Endpoints include:

  1. Name, ARN, Creation Time, Status

  2. Endpoint Configuration Settings from your Endpoint Configuration

  3. Monitor

 

Configuration

Here we must have the SageMaker Service running and configure the credentials below.

Depending on what authentication type you use there will be different credentials, we support BASIC and ASSUME_ROLE_WITH_WEB_IDENTITY types.

 

For BASIC authentication type-

sagemaker-credentials: storedCredentials: - group: authenticationType: BASIC accessKeyId: xxxxxxxxxxxxxxx secretAccessKey: xxxxxxxxxxxxxxxxxxxxxxxxxx region: us-east-2

  • accessKeyId and secretAccessKey:

    • These act as a username and password for AWS SageMaker so you will need both to authenticate. These will be unique to you.

  • region:

    • This is where your instance is located, you will be able to find this in your AWS account webpage or on the header bar on the AWS webpage.

 

For ASSUME_ROLE_WITH_WEB_IDENTITY authentication type-

sagemaker-credentials: eksServiceAccountCredentials: - path: roleArn: <AWS-ROLE-ARN> webIdentityTokenFile: <AWS-Injected-file>

  • roleArn:

    • This is the Role Amazon Resource Name, find in AWS console in your desired resource

  • webIdentityTokenFile:

    • Here you provide a file containing your token in order to authenticate you.

 

Import onto ModelOp UI

We must now register the SageMaker model in ModelOp Center. Navigate to the Business Models tab and select “Add Business Model” in the top right corner. Here we will need to provide the model name, it is recommended that the credentials be configured within the environment as seen in the Integration section above. If not you can chose Enter Credentials and enter the region, access key and secret key at this step.

 

When a SageMaker model is registered with ModelOp Center, it is represented and handled as a Stored Model.

  1. The same model specific information we saw on AWS console, we have available in ModelOp Center, including the model name, ARN, tags, creation time, and links to training jobs.

  2. Within ModelOp Center, an Endpoint Configuration is represented as a snapshot. Each snapshot includes its name, tags, creation time, endpoint configuration specific fields, model details, tests and custom metadata. If there is no endpoint configuration for your model one will be created by default.

  3. Next, we have the Endpoint which is represented as a SageMaker Runtime with a snapshot deployed. The engine details show the most relevant information that we saw on AWS console.

  4. All previously ran jobs will be able to be viewed and new jobs can be created within the ModelOp Center.

  5. Finally, when we imported the model, all of the TRANSFORM and TRAINING jobs from AWS, for this particular model, were imported as well.

All of the entities that we looked at were first created in AWS and then represented in ModelOp Center.

 

 

Import Elements

Importing a model will import the following elements:

  • Model Information on the Registry

  • Related Training Jobs

  • Related Batch Transform Jobs

  • Related Endpoint configurations

  • Related Endpoints

 

Job import will include:

  • Job execution dates, execution time, and other available basic info

  • Input data s3 url

  • Output data s3 url

  • Link to the SageMaker console for the job

 

Endpoint Config Import will include:

  • Basic details of the endpoint config.

  • Link to the SageMaker console for the endpoint config

 

Endpoint Import will include:

  • Basic details of the endpoint.

  • Link to the SageMaker console for the endpoint

 

MLCs

SageMaker deploy model

In this MLC you have an option to run a transform job and/or deploy a SageMaker endpoint. First you must trigger the MLC by:

  • Importing a SageMaker model with existing Endpoint Configurations.

OR

  • Creating an Endpoint Configuration in SageMaker on an already imported model.

 

Once the MLC is triggered it will follow this flow:

  1. Verifying required assets (documentation, schema, training data, test result comparator, test data asset).

  2. If Test Data Asset available, Run a test SageMaker transform job.

  3. Create Jira approval ticket

  4. If approved, a SageMaker Endpoint will be created, which translates to a new Runtime.

 

In these steps we see that a transform job is ran if Test Data Asset is present, once the job has completed successfully it is sent to be deployed. Alternatively, if you do not wish to run a transform job do not include Test Data in your model and simply approve the ticket requesting deployment and the model will be sent on the path to deploy. When a SageMaker model is deployed this creates a SageMaker Endpoint.

 

There are cases that this MLC fails, these are:

  1. The MLC will trigger for every new Snapshot, but will quickly exit if the model is not a SageMaker model.

  2. If the SageMaker Transform job is run and failed, it will create a Jira ticket notification, and won't try to deploy to an Endpoint.