Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 2 Next »

This article describes how ModelOp Center enables model versioning for all types of models.

Table of Contents

Introduction

ModelOp Center provides comprehensive versioning for all models under management, inclusive of versioning of all items that compose a model. For each version of a model, ModelOp Center takes a snapshot of all the elements that are part of the Standard Model Definition within ModelOp Center’s Governance Inventory, such that you have long-term audibility and reproducibility.

Details

System of Record

All of the elements of the Standard Model Definition are versioned using ModelOp Center’s Model Manager micro service, and for external artifacts (e.g. Model Source Code), are backed by your preferred enterprise-standard versioning tooling.

For example:

  • Model Source Code → backed by Git (all git-compliant systems: Github, Bitbucket, Gitlab, etc.)

  • Model Assets → backed by standard artifact repositories (such as S3, Artifactory, etc.)

  • Metadata & Other Elements → backed and versioned by ModelOp Center’s Model Manager

Provenance (Traceability)

As a result, ModelOp Center provides systematic traceability of all of the core elements of a model. The immutable snapshot contains the exact asset (or reference) so that an auditor can easily trace back to the exact source that was reviewed, approved, and used for production decisioning.

  • Source code traceability: reference to the exact commitID that is used

  • Model asset traceability: reference to the exact immutable copy of the coefficients/weights/serialized-model-artifact that is used

  • Dependent libraries: reference to the exact list of dependent libraries that are used

  • Schemas: reference to the exact schema definition that is used

  • Job (Test): reference to the exact test jobs that were run systematically (not via a human) as part of the testing/approval process, with the ability to reproduce the tests using the snapshot information

  • Test Results: exact results of the systematic tests for each snapshot

  • Approvals: reference to each approval that was raised for a given snapshot, with traceability to the ticketing system

Model Snapshots

Behind the scenes, ModelOp Center’s data model creates a new model snapshot that is an immutable copy of all the elements (or references to the elements) that compose that model at a given point in time. These snapshots are persisted in perpetuity for long term auditability. For more details about the ModelOp Center metadata and data model, see Model Metadata Details.

To Create a New Model Snapshot

A new model version is created when a User “submits” a model for productionization. This can be done via the ModelOp Center UI or the Jupyter or RStudio plugins.

Create Version via the ModelOp Center UI

  1. From the Inventory, open the specific model that you would like to snapshot. Select the “Create Snapshot” button from the top right.

image-20240716-220506.png

2. Confirm the Name, Access Group, and optionally add a Description and Tags for the snapshot.

image-20240716-220857.png

3. Enter target runtime selection criteria by entering in text for the “Tags” section or enter a “Runtime name”. The list of matching runtimes will be filtered below based on the selections.

image-20240716-221008.png

4. Review the Snapshot details. When ready, select “Create Snapshot”.

image-20240716-221035.png

5. Upon successful snapshot creation, the Model Snapshot page will be opened for the newly created snapshot. Typically an MLC is triggered upon snapshot, and will be shown immediately:

image-20240716-221213.png

The appropriate Model Life Cycle will automatically begin orchestrating all of the steps necessary to operationalize the model. This includes deployment of the model as REST into the designated ModelOp runtimes or the best matching available ModelOp runtime.

6. Note, model life cycle updates and any errors in the model life cycle execution will be populated in the “Snapshot Notifications” of the Snapshot “Overview” tab. As well, depending on the environment configuration, Jira and/or ServiceNow tickets may also be raised in the event of an issue. Tickets related to the model will appear in the “Associated Tickets” pane of the Snapshot “Overview” tab.

image-20240716-221406.png

Next Article: Using the ModelOp Runtime for Model Execution >

Create Version via the Jupyter Plugin

To create a new version via the Jupyter Plugin, please see the “Submit a Model” section of the Submit Model Version using Jupyter.

Create Version via the RStudio Plugin

To create a new version via the RStudio Plugin, please see the “Submit a Model” section of the Submit Model Version using RStudio.

Next Article: Model Governance: Model Metadata Details >

  • No labels