Model Governance: Model Management

Owned by Dave Trier
Last updated: Aug 26, 2023
4 min read

This article describes how to use the ModelOp Command Center as the central repository for governing models, including how ModelOp Center provides a standard representation of a model regardless of the model factory from which it came, or the infrastructure upon which it will run.

Table of Contents

 

Robust Model Management - Standard Model Definition

ModelOp Center provides the most robust and extensible definition of a model to allow for consistent management, monitoring, and governance of all models across the enterprise.

Elements of the Standard Model Definition

The ModelOp Standard Model Definition includes all of the metadata, technical model details, version information, MLC’s, and test results related to a given model. This information is listed in the Business Model Details page within ModelOp Center:

Overview Tab

  • “Snapshots” (aka Versions) section: Lists all snapshots of the model, their current deployment status and associated stage (environment), last modification date, and created date. Click on a given snapshot to see the snapshot details.

 

  • Notifications section: Lists all the notifications related to this particular model. If a ticket (e.g. Jira or ServiceNOW) is associated with the model, a link to that ticket will be displayed. Click on the icon to view more details about the notification.

 

  • Production Status: If the model is currently deployed in production, the business value and status “heatmap” will be displayed as well:

 

See the “Model Governance: Model Versioning” page for more details on model snapshots (versions).

 

Details tab

Provides detailed information, metadata, and entry points for a model:

  • Model Details: provides the overview information for a model. Click the “Edit” button to modify, as appropriate.

  • Additional Metadata: lists the custom metadata that has been added by a Customer, typically populated via an MLC or a specific integration

 

 

  • Functions (Optional for the ModelOp Runtime): When using a ModelOp Runtime or a Custom Monitor, the Functions define the entry points into the model for initialization, training, metrics testing, and scoring. For example, the Init Function is invoked upon deployment of a model and the Score Function is invoked when scoring. Click the “Edit” button to modify. Note that ModelOp will attempt to provide a drop-down of available functions from the main source code that has been possible.

 

 

Compliance tab

Allows users to generate a detailed Audit report. To run a report:

  • Select the specific Snapshot of interest and select “Run Report”

 

  • The detailed report will be displayed which includes all of the model details, tickets of high severity, test / monitor results, documentation, assets, deployments, and related MLC instances for the given snapshot.

 

Click the “Download” button to create a PDF version of the report:

 

 

Assets

Lists all of the specific assets associated with the model, which can be source code, trained model artifacts, model configuration files, associated data (training, test, etc.), documents, decision tables, etc.

 

 

Repository

Provides more information about the backing repository for the assets. Typically this is a git repository (e.g. Github, Bitbucket, etc.), but models can also be stored in other repositories such as Artifactory, SageMaker, or DataIku repositories.

For git-based repositories, there is more information about the last sync that occurred with the backing git repository

 

 

Schemas

Lists the schemas associated with the model. Note that there is only one input and one output schema supported.

For more information on creating input and output schemas, see the Model Schemas page.

For more details on the metadata that is collected and persisted for a Model, see the “Model Governance: Model Metadata” reference document.

Supported Languages & Frameworks

While ModelOp Center supports almost any model language, framework, and overall model factory, below is a sampling of some of the more common ones that are supported in ModelOp Center. Each of these are encoded in ModelOp Center’s standard model definition:

 

Next Article: Model Governance: Production Model Inventory >