Production Model Inventory
ModelOp Center provides a Production Model Inventory where you can view all models across the enterprise. Each user can see only the models to which they have access. The Production Model Inventory serves as a central entry point into your models where you can view details, modify, and work further on operationalizing your models.
To access the Production Model Inventory:
Click Models on side bar.
2. The list of available models appears, including the Model Name, Type of Model (e.g. Python3, R), Model Creator, Created Date, and Last Modifier.
To search for a Model by Name:
Click the Search icon in the top right.
2. In the “Filter by Name” field, begin typing in the name of your model.
3. The list of models automatically filters based on your input.
4. To filter by model “tag”, in the “Filter by tag” field, begin typing in the name of the tag.
A drop-down with available tags will be presented.
Select the tag for your model
The list of models automatically filters based on your input:
To compare 2 or more models (Champion/Challenger):
This section describes the Champion/Challenger feature in ModelOp Center, and how to use it to help determine the best model method or approach to promote to production.
Table of Contents
Introduction
Data Science requires experimentation using a variety of methods, approaches, frameworks, and potentially even coding languages. While most data scientists will conduct their experimentation in their model development environment, data scientists--and the managerial and governance reviewers--often want to review that a candidate model for production performs better than any currently running version of the model. The Champion/Challenger feature in ModelOp Center presents a side-by side comparison of the performance of different models--or versions of the same model--to help users analyze which model or model version is best suited for production usage.
The following phases lay the groundwork for doing a Champion/Challenger comparison.
Define the evaluation metrics for the model. See Model Monitoring: Overview.
Automate evaluation tests - this is done either by manually running metrics jobs (see Running a Metrics Job Manually) or through an automated MLC Process. You can build an MLC Process to automatically execute metrics tests for a particular version of a model. See Model Lifecycle Management Overview for more information.
Conduct a side-by-side comparison of the test results in the Champion/Challenger page of the Command Center (see next section).
Champion/Challenger Comparison
For the previously generated metrics results (see steps 1 & 2 above), use the following steps to activate the Champion/Challenger feature in the ModelOp Center to compare test results side-by-side.
Within the Production Model Inventory, select the checkbox in the “Champion/Challenger” column for each model that you would like to include in the comparison:
2. Hit the “Compare Models” button
3. The Champion/Challenger comparison screen will open to compare the latest Test Results for those selected models
See Model Governance: Standard Model Definition for more information about the information that is contained for each model within the Production Model Inventory.
Next Article: Model Monitoring: Overview >
Within the Production Model Inventory, select the checkbox in the “Champion/Challenger” column for each model that you would like to include in the comparison:
2. Hit the “Compare Models” button
3. The Champion/Challenger comparison screen will open to compare the latest Test Results for those selected models
See Model Governance: Standard Model Definition for more information about the information that is contained for each model within the Production Model Inventory.
Next Article: Model Governance: Interpretability >