Operationalize a Model in a ModelOp Runtime - ONLINE Deployment

The following are the steps to deploy a model as REST via ModelOp Center:

Pre-Requisites:

Prior to any model deployment, the following steps should be conducted by the Enterprise AI Architect and ModelOps team.

1. Runtime Environments/Stages. Define the Runtime Environments/Stages required for operationalizing models for a given team and/or class of models. For example, if this team requires a model to be deployed and tested in Development, then SIT, then UAT, before being promoted to Production.

2. Operationalization MLC. The Enterprise AI Architect and/or ModelOps Engineer will create the model life cycles (MLC’s) required for operationalization models for that team and/or class of models. This operationalization MLC will take into account the technical and governance requirements as a model is operationalized, which may include running tests, obtaining approvals, promoting the model to the next higher Runtime Environment (e.g. from Dev to QA).

Note, each Business Unit / Team may have different requirements and therefore these steps may be updated or repeated as the overall program progresses.

Steps:

1. Prepare Runtimes. Identify the target Runtimes across the requisite Environments. For each target Runtime, complete the following:

   1. Add Runtime Endpoints. If any runtimes will be used to provide an online model, add the required Endpoints.

      1. To add an endpoint, follow these instructions: Adding Endpoints to a ModelOp Runtime

      2. For runtimes that will be used for Batch runs of the model, do not put endpoints on the model.

   2. Add “Environment/Stage Tags”. Based on the environments/stages required (see pre-requisites), add the necessary “environment/stage tag” to the runtime.

      1. Example: add a “DEV” tag to the Runtime in their development environment, an “SIT” tag to the Runtime in their SIT environment, a “UAT” tag to the Runtime in their UAT environment, and ultimately a “PROD” tag to the Runtime in their Prod environment

   3. Add “Model Service Tags”. The Model “Service” tag will be used to identify that this specific runtime is designed to be a target runtime for that particular model. Add the appropriate “Model Service Tag” to the runtime.

      1. Example: add a “cc-fraud” Model Service Tag to the runtime for a 3rd party credit card model to the “Dev”, “SIT”, “UAT”, and “Prod” runtimes.

2. Create a Snapshot of the model.

   1. From the Production Model Inventory, open the specific model that you would like to snapshot. Select the “Create Snapshot” button from the top right.

   2. 

4. 1. Add the “Model Service Tag”. When creating the snapshot, in the tags section, add the same “Model Service Tag” that was used for the requisite Runtimes. This will ensure that the MLC will know the correct Runtimes to which the model should be deployed throughout the life cycle.

      1. Example: add a “cc-fraud” Model Service Tag to the snapshot.

   2. (optional) Confirm the Target Runtimes. While the MLC will attempt to automatically find the correct runtime based on the “Model Service Tag”, during snapshot creation, the ModelOps engineer may want to select the specific runtimes to which the model should be deployed. This may be required when the ModelOps engineer knows that this specific snapshot should only run on a particular Runtime.

      1. Example: this snapshot of the “cc-fraud” model must run on a particular Spark Runtime. The ModelOps engineer selects this Runtime to ensure that the MLC knows to use that Runtime.

   3. Confirm Snapshot Creation.

Deploying a Model into a Persistent ModelOp Runtime

You can deploy a model into a runtime provided by ModelOp or you can deploy into an external runtime environment of your choice (e.g. Spark). This article focuses on deploying into the ModelOp Runtime.

Regardless which runtime you choose, ModelOp streamlines the model deployment process once the model is ready. The typical steps include:

1. Prepare ModelOp Runtime for a Persistent Deployment - ensure there is a Runtime with the correct dependencies, and set up the Runtime to integrate with the data pipeline.

2. Define the MLC Process - an MLC Process automates and regulates the models registered with the ModelOp Center. For more information, see Model Lifecycle Manager: Automation.

3. Submit a Model for Deployment - this triggers the MLC Process to guide the model through the tests and approvals to ensure the quality of the predictions the model provides to business.

1. Prepare ModelOp Runtime for a (Persistent) Deployment

A ModelOp Runtime must be configured with an input endpoint and an output endpoint. The input endpoint provides an entry point for records from the data pipeline to the model. The output endpoint is where the consuming data pipeline or application receives the output of the model scoring. When a record activates the input endpoint, the record is decoded based on the Encoding setting, and sent to the Scoring Function. The output of the Scoring Function for that record is then sent to the output endpoint. The type of endpoint dictates the scoring method: on-demand (e.g. REST) vs. streaming (e.g. Kafka). Batch scoring is accomplished using the Batch Job which can run in a specified Runtime for dependencies and environment differentiation.

To manually configure a Runtime:

Note: This can be automated using an MLC Process (see: Model Lifecycle Manager: Automation).

1. Log on to the Command Center and navigate to Runtimes > Runtime Dashboard.

   

2. Select the specific Runtime that you would like to configure.

3. In the Tags field, type the name of the model you want to run in this ModelOp Runtime, press Enter, and then click Update Runtime.

4. Click Endpoints > Add New Input Endpoint and set the following:

   1. (Optional) Name and Description

   2. Endpoint Type: select REST Endpoint, Kafka Subscriber etc.

   3. Encoding: select JSON, CSV, Binary, etc.

   4. (Optional) Port: 0, 1, 85535, etc.

5. Click Update Runtime.

6. Click Endpoints > Add New Output Endpoint and set the following:

   1. (Optional) Name and Description

   2. Endpoint Type: select REST Endpoint, Kafka Subscriber etc.

   3. Encoding: select JSON, CSV, Binary, etc.

   4. (Optional) Port: 0, 1, 85535, etc.

7. Click Update Runtime.

2. Define the MLC Process

The MLC Process is typically set up to automate and regulate the deployment process across a team, group of models, or other grouping sharing the same standardized approach to ModelOps. See ModelOp Life Cycle Manager: Automation for how to set up the MLC Process that controls the path to production.

3. Submit a Model for Deployment

These instructions describe how to manually submit a model for deployment using the Command Center. This assume there is an MLC Process using Deployable Model Changed to trigger a deployment process as described in the On Model Changed example.

Note: Typically you deploy models with delegates in an MLC Process.

1. Click on the Models tab and select the model you want to deploy.

2. Click Submit Model.

3. To confirm the model has deployed, click Model Details and look for it in the list.

4. If the deployment failed, click Tasks and Alerts to view the associated task.

Related Articles

• Monitor a Deployed Model

• Model Efficacy Metrics and Monitoring

• Model Lifecycle Manager: Automation

• Model Batch Jobs and Tests

Next Article: Model Batch Jobs and Tests >