Versions Compared

Old Version 1

changes.mady.by.user Dave Trier

Saved on

compared with

New Version 2

changes.mady.by.user Dave Trier

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

ModelOp Center “Assets” support various data technologies, including:

  • AWS S3 or S3-Compliant S3 Files

  • Azure Blob Storage Files

  • GCP Storage Buckets

  • HDFS Files

  • SQL data sources

...

Action

Data Required

Available via:

MOC Asset Examples [ASSET_ROLE : EXAMPLE}(note: these examples are not the only ways to define assets)

Add an Asset (to a StoredModelBusiness Use Case, Model, or Snapshot)

All types

CLI: s3 & embedded assets only

UI: all

API: all

Applicable to S3, Azure Blob, GCP Storage Buckets, HDFS, and SQL Asset

Run a Metrics Job

(e.g. “Back Test” using labeled data)

•Evaluation Test Data

CLI, UI, API, MLC

S3:

•Test_Data: s3://<model_base>/EvaluationDataTestData.csv

HDFS:

•Test_Data: hdfs://<model_base>/EvaluationDataTestData.csv

SQL Asset:

•Test_Data: SELECT * FROM <read_only_Test_EvalData_table> WHERE <conditions>

Run a Data Drift Performance Metrics Job

•Training/Baseline Data

•Comparator Data

CLI: would trigger the API

API, MLC

S3:

•Training_Data: s3://<model_base>/TrainingData.csv•Compare_Data: s3://<model_base>/ComparatorData.csv

HDFS:•Training_Data: hdfs://<model_base>/TrainingData.csv

•Compare_Data: hdfs://<model_base>/ComparatorData.csv

SQL:

•Training_Data: SELECT * FROM <read_only_Training_table> WHERE <conditions>•Compare_Data: Training: SELECT * FROM <read_only_Comparator_table> WHERE <conditions> … may need to have tag to specify input vs. output comparator data

Run a Model Concept Drift Distribution Comparison (e.g. Drift) Job

•Training/Baseline Data

•Comparator Data

CLI: would trigger the API

API, MLC

S3:

•Training_Data: s3://<model_base>/TrainingData.csv

•Compare_Data: s3://<model_base>/ComparatorData.csv

HDFS:

•Training_Data: hdfs://<model_base>/TrainingData.csv

•Compare_Data: hdfs://<model_base>/ComparatorData.csv

SQL:

•Training_Data: SELECT * FROM <read_only_Training_table> WHERE <conditions>

•Compare_Data: Training: SELECT * FROM <read_only_Comparator_table> WHERE <conditions> … may need to have tag to specify input vs. output comparator data

Run a Bias Detection Job

•Evaluation Data

CLI, UI, API, MLC

S3:

•Test_Data: s3://<model_base>/EvaluationData.csv

HDFS:

•Test_Data: hdfs://<model_base>/EvaluationData.csv

SQL Asset:

•Test_Data: SELECT * FROM <read_only_Eval_table> WHERE <conditions>

Run a Training Job

•Training Data

MLC, UI, API

S3:

•Training_Data: s3://<model_base>/TrainingData.csv

HDFS:

•Training_Data: hdfs://<model_base>/TrainingData.csv

SQL Asset:

•Training_Data: SELECT * FROM <read_only_Training_table> WHERE <conditions>

...

ModelOp Center supports Trained Model Artifacts stored in S3 bucket or HDFS (does not support as SQL Asset)buckets, Azure Blob Store, GCP Storage Buckets, or Artifactory. When adding asset that is used as Trained Model Artifact, please select the asset role to be “Weights File” or “Model Binary File” depending on the use case.

...

ModelOp Center runtime supports data schemas which helps to enforce specif type of on different fieldsdefine the data inputs/outputs for testing, monitoring, scoring, and governance. The schema can be used for input data and/or output data. When uploading a schema, select “Model Schema” asset role.

...

Please see Register a Model to get details about adding assets during model import.

Via ModelOp UI

  1. Go to the Model Details page by clicking on the model from the Models page

...

  1. Image Added

  2. View the existing Assets by clicking on Assets tab:

    Image Modified

  3. Click on “Add Assets” , which opens a dialog box to select a local file.

    Image Modified

  4. The user is presented with multiple options for adding assets: as a local file upload, by providing a reference to an existing location, or by providing as SQL query:

    1. Image Modified

For Local File Upload:

  1. Click on the “Add File Upload”.

  2. Click on “Upload file”:

    1. Image Modified

  3. Select the file from your local computer

  4. Select to Upload the file:

    1. Image Modified

  5. The file will be uploaded as a new asset on the model

To Add Asset by Reference:

  1. Provide the URL to the file.

  2. Select the type of asset: S3, HDFS, Azure Blob Store

    Image Modified

  3. Click Add Asset

  4. The file will be uploaded as a new asset on the model

To Add a SQL Asset

Please see Integrate with SQL Databases to understand how ModelOp Center integrates with SQL databases. Once you get familiar with how ModelOp Center handles SQL Assets, follow these steps to add SQL assets to a given model.

...