Model Assets

Owned by Dave Trier
Last updated: Aug 16, 2023
5 min read

Overview

ModelOp Center is designed to be agnostic to the type of model, the platform on which the model runs, and the data platforms with which the model interacts during various portions of the model lifecycle. Central to the execution of a model are the model-specific assets. Types of Model Assets that may be used during the Model Life Cycle Include:

  • Trained model artifacts

  • Requirements (Dependencies) lists

  • Data assets

  • Configuration files

 

Asset Types

Data Assets

ModelOp Center “Assets” support various data technologies, including:

  • AWS S3 or S3-Compliant S3 Files

  • Azure Blob Storage Files

  • GCP Storage Buckets

  • HDFS Files

  • SQL data sources

 

While assets can be used in a variety of activities throughout a model life cycle, below is a summary of typical activities where a User or an MLC may use a Data Asset:

Action

Data Required

Available via:

MOC Asset Examples (note: these examples are not the only ways to define assets)

Add an Asset (to a Business Use Case, Model, or Snapshot)

All types

CLI: s3 & embedded assets only

UI: all

API: all

Applicable to S3, Azure Blob, GCP Storage Buckets, HDFS, and SQL Asset

Run a Metrics Job

(e.g. “Back Test” using labeled data)

Test Data

CLI, UI, API, MLC

S3:

•Test_Data: s3://<model_base>/TestData.csv

HDFS:

•Test_Data: hdfs://<model_base>/TestData.csv

SQL Asset:

•Test_Data: SELECT * FROM <read_only_Test_Data_table> WHERE <conditions>

Run a Performance Metrics Job

•Comparator Data

CLI: would trigger the API

API, MLC

S3:

•Compare_Data: s3://<model_base>/ComparatorData.csv

HDFS:

•Compare_Data: hdfs://<model_base>/ComparatorData.csv

SQL:

•Compare_Data: Training: SELECT * FROM <read_only_Comparator_table> WHERE <conditions> … may need to have tag to specify input vs. output comparator data

Run a Distribution Comparison (e.g. Drift) Job

•Training/Baseline Data

•Comparator Data

CLI: would trigger the API

API, MLC

S3:

•Training_Data: s3://<model_base>/TrainingData.csv

•Compare_Data: s3://<model_base>/ComparatorData.csv

HDFS:

•Training_Data: hdfs://<model_base>/TrainingData.csv

•Compare_Data: hdfs://<model_base>/ComparatorData.csv

SQL:

•Training_Data: SELECT * FROM <read_only_Training_table> WHERE <conditions>

•Compare_Data: Training: SELECT * FROM <read_only_Comparator_table> WHERE <conditions> … may need to have tag to specify input vs. output comparator data

Run a Bias Detection Job

•Evaluation Data

CLI, UI, API, MLC

S3:

•Test_Data: s3://<model_base>/EvaluationData.csv

HDFS:

•Test_Data: hdfs://<model_base>/EvaluationData.csv

SQL Asset:

•Test_Data: SELECT * FROM <read_only_Eval_table> WHERE <conditions>

Run a Training Job

•Training Data

MLC, UI, API

S3:

•Training_Data: s3://<model_base>/TrainingData.csv

HDFS:

•Training_Data: hdfs://<model_base>/TrainingData.csv

SQL Asset:

•Training_Data: SELECT * FROM <read_only_Training_table> WHERE <conditions>

 

Other Assets

Trained Model Artifacts

ModelOp Center supports Trained Model Artifacts stored in S3 buckets, Azure Blob Store, GCP Storage Buckets, or Artifactory. When adding asset that is used as Trained Model Artifact, please select the asset role to be “Weights File” or “Model Binary File” depending on the use case.

Schemas

ModelOp Center runtime supports data schemas which helps to define the data inputs/outputs for testing, monitoring, scoring, and governance. The schema can be used for input data and/or output data. When uploading a schema, select “Model Schema” asset role.

Requirements

ModelOp runtime comes with basic pre-installed libraries for different model and languages. If the model requires additional libraries to be installed by the ModelOp Runtime, it supports defining model library requirements in requirements.txt file. The ModelOp runtime will check the requirements file before running the model and install the missing libraries. When uploading the requirements file, please select the asset role of “Requirements”.

 

Adding Model Assets

During Model Import

Please see Register a Model to get details about adding assets during model import.

Via ModelOp UI

  1. Go to the Model Details page by clicking on the model from the Models page

     

  2. View the existing Assets by clicking on Assets tab:

     

  3. Click on “Add Assets” , which presents multiple options for adding assets:

     

  4. The user is presented with multiple options for adding assets: as a local file upload, by providing a reference (url) to an existing location, as a REST asset, or by providing as SQL query:

 

Upload a Local File:

  1. Click on the “File Upload”

  2. Select the file from your local computer’s file system and hit “Open”

  3. The file will be uploaded as a new asset on the model

To Add an Asset by Reference:

  1. Click “URL” from the Add Asset menu

    1.  

  2. Provide the URL to the file.

  3. Select the type of asset: S3, Google Cloud Bucket, Azure Blob Store, HDFS. Note that your system administrator may have already pre-configured the type of asset store for your group

     

  4. Fill in the other pertinent storage information

  5. Click “Save” to save the asset

  6. The file will be uploaded as a new asset on the model

To Add a REST Asset

REST assets are those data sets or other assets that can be retrieved via standard REST API calls

  1. Click “REST” from the Add Asset menu

    1.  

  2. Fill in the Asset Name, Asset Role, Request Method, Target URL, Query Params to be sent in the request.

    1. If the REST asset does NOT use standard HAL paging, select the “Requires Custom Paging” option and fill in the details

    2.  

  3. Click “Save”

  4. The REST asset will be uploaded as a new asset on the model

To Add a SQL Asset

  1. Click “SQL” from the Add Asset menu

  2. Please see Integrate with SQL Databases to understand how ModelOp Center integrates with SQL databases. Once you get familiar with how ModelOp Center handles SQL Assets, follow these steps to add SQL assets to a given model.

Via CLI

Please see the asset command for adding assets using the CLI.

 

 

 

Next Article: Model Life Cycle Management: Overview >