Getting Oriented with ModelOp Center’s Command Center
- Carl Yorke (Unlicensed)
- Brendan Kelly (Deactivated)
- Dave Trier
This article describes the components of the Command Center which appear on the Dashboard. The ModelOp Command Center is the user interface that allows access to Model Op's features, functions, and assets.
Table of Contents
Introduction
The ModelOp Command Center displays a single screen with access to all of the tools to monitor and manage your models. The components on the Dashboard include the three panes in the middle of the screen, the notifications pane below, and the four icons in the menu on the left.
Models Pane
On the Models pane, you can see the current status of the models in your system. The Models pane shows:
The number of models that are currently registered. This is the number of models being managed.
The number of models that are currently running in an ModelOp Runtime
The number of model tests that have failed
The number of those failures that are errors in the model as opposed to system or MLC errors
You can click on a category that is populated to filter the notifications to see details related to that category. For instance, if Test Failures shows the number 2, you can click on the number to filter the notifications below to just those associated with test failures.
Processes Pane
The Processes pane displays how many MLC processes are currently deployed and any incidents or errors associated with the MLC processes.
The MLC Processes
Each MLC Process has built in monitoring tasks that report events and errors to the Command Center. This pane displays the following:
The number of defined MLC Processes
The number of reported incidents
The number of active MLC Processes
The number of MLC Process errors
Runtimes Pane
The Runtimes pane displays information about the runtimes available to the system. These are the environments where models are deployed.
This pane displays how many runtimes are available, active, idle, and if there are any runtime errors.
Notifications Pane
The notifications pane displays a real-time list of all notifications across the ModelOp Center system. This includes informational notifications as well as alerts that require immediate attention. Per the prior sections, the notifications pane can be filtered by clicking one of the items in the Models, Processes, or Runtimes pane.
Main Menu Bar
The Main Menu Bar is meant to provide quick access to all of the Models, MLC Processes, Runtimes, and Tasks across the enterprise ModelOps program. See below for details of each section.
Tasks and Alerts
The Tasks and Alerts screen has two tabs, My Tasks and Approvals.
My Tasks
Tasks are defined as User Tasks in the MLC Process. They are typically configured to assign tasks to a specific group. A person within this group can then decide to take ownership of that task and execute it to completion.
Tasks are things a user has to do - such as approve a model or acknowledge a failed test - so that the MLC Process can move forward.
The My Tasks page filters tasks in two categories:
All Open Tasks
Tasks in Progress
Approvals
My active Tasks - # outstanding
ds_team - awaiting assignment - # outstanding
python_approvers - awaiting assignment - # outstanding
r_approvers - awaiting assignment - # outstanding
other_approvers - awaiting assignment - 0 outstanding
Runtimes
The Runtimes screen has the following tabs:
Jobs | Displays how many jobs are running, how many have failed, how many were successful and the total. |
Runtime Dashboard | Displays high level status for all currently deployed runtimes, each in it’s own pane. Click on the pane for access to details about that Runtime. Tabs under the Runtime Dashboard pane include:
|
Runtime Stats | Lists all available runtimes. Click through a runtime for details about that runtime. |
Models
Tabs under the Models icon are Model Details, Model Tests and Champion/Challenger tabs.
Model Details
The following table describes the details available under the Model Details tab. These fields are available only after you select a model from the list.
Model Info | Displays the name of the model, the description, and any tags applied to the model. |
Model Source | Source code that defines the model. |
Attachments | Attachments are typically training artifacts that get loaded into the model, but can be any other file required to execute the model. |
Input Schemas | Defines the parameters of what is passed to the model. |
Output Schemas | Defines the parameters and format of what is passed from the model. |
Model Functions | Defines the entry points into the model for initialization, training, metrics testing, and scoring. |
Model Platform | Lists the dependencies used by the model. |
Model Tests
Lists the models in the system. Click a model to access the results of tests on that model.
Click on a test to see visualizations of the tests. The visualizations available depend on the tests built into a MLC Process.
Champion/Challenger
Champion/Challenger is an industry standard technique to compare the performance of competing model strategies, or the performance of a recently updated model against a previous version of the same model.
Lists the models in the system. Click a model to access the results of tests on that model and choose two tests two compare them against each other.
See the Champion/Challenger article for more details.
Model Life Cycle
The Model Life Cycle screen has two tabs, MLC Overview and MLC Processes.
The MLC Overview tab displays information about all the activities being handled by the MLC Manager.
Active MLC Processes
Total Incidents
Total Tasks
Runtime information
Oldest Tasks information
The MLC Processes tab displays the diagrams of the MLC Processes that are currently deployed into the MLC Manager.