Introduction

The ModelOp Center UI displays a single screen with access to all of the tools to monitor and manage your models. Below you will find information regarding the main modules and navigation within ModelOp Center

Executive Dashboard

The home screen of ModelOp Center contains the main Executive Dashboard, which provides visibility into all Models running in production across all environments, as well as insight into the status of all models against business, operational, data science, and risk KPI’s.

Open

Page Navigation (Breadcrumb)

The top portion of the ModelOp Center UI will maintain a “breadcrumb” of your current location within the site hierarchy, allowing for easy navigation throughout the site.

Open

Main Menu Bar

The Main Menu Bar is meant to provide quick access to all of the Models, Model Life Cycles (MLC’s), Risk and Compliance Module, and Operations modules. See below for details of each section.

NOTE: only those users in the Administrators group will have access to the Model Lifecycles section

Models Module

The Models menu item is composed of the Models Overview, Business Models, and Monitors pages.

Models Overview

The Models Overview page provides an overview of where all models are in their deployment cycle (e.g. Dev, QA, Prod) by Organization and by Runtime. Additionally, the page specifies model-specific notifications for actioning.

Business Models (“Evergreen Model Inventory”)

Clicking the “Business Models” menu item opens the Evergreen Model Inventory (sometimes referred by others as “Production Model Inventory”). The production model inventory contains the full list of registered models within ModelOp Center, regardless of the model type, model factory in which it was produced, current state of the model (registered vs. in QA vs. deployed).

Search. A user may search for a model by Name or Tag by using the Search bar:

Filter. A user can filter the models based on the filter categories on the right:

Searching for Models in Production:

NOTE: starting in v3.2, a user may choose to search based on models that are “In Production” using the “in production” toggle. By selecting this option, there are additional filter options for the search:

See Model Details. A user may click on any row to view the specific details of that model (see subsequent “Model Details” page):

Compare Models. Additionally, to compare one or more models, a user may select “Enable champion challenger” toggle button. This will expose the “Champion/Challenger” column, allowing the user compare two or models models using the “Champion/Challenger” check boxes. Champion/Challenger is an industry standard technique to compare the performance of competing model strategies, or the performance of a recently updated model against a previous version of the same model.

See the Champion/Challenger article for more details.

Business Model Details

When a Business Model is selected from the Evergreen Model Inventory, the Business Model details page is opened, which provides all of the metadata, technical model details, assets, documentation, and snapshots (versions) of a given model. This page is meant to provide an update-to-date view of all the elements that compose a model. See the Model Management page for more information.

Business Model Snapshot Details

Clicking on a given Snapshot from the Business Model Details page, the Business Model Snapshot page is opened provides all of the metadata, technical model details, assets, documentation, MLC’s, deployments, and monitoring/test results for a given Snapshot (Version) of a model. The snapshot is an immutable copy of the model at a specific point in time, providing enterprises the ability to have auditability for the exact model “package” that was deployed at a given instance in time.

Model Life Cycle (MLC) Module

Within the Model Lifecycles Module, there is a Model Lifecycle Overview page and a Model Lifecycles list page.

Model Lifecycles Overview

Clicking the “Model Lifecycles Overview” main menu item opens the MLC Overview, which contains a view into the active MLC Processes, MLC Process Instances, the latest MLC Incidents, and trends of issues over time.

Model Lifecycle List

Clicking the “Model Lifecycles List” main menu item opens the list of available Model Lifecycles within the system:

A user can toggle between the list view and card view by selecting the toggle in the top-right:

A user may choose to open a given MLC to view more details about the process by simply clicking one of the MLC “tiles”. This navigates the user to the MLC Process Definition page.

Model Life Cycle (MLC) Process Definition

The MLC Process Definition page provides an overview of a given MLC process definition. A process definition is essentially the process “blueprint” of which there may be many instances of that process “blueprint” running for each model in the system, e.g.

The MLC Process Definition includes:

• Information about the Process Definition: name, version, state (active/suspended), and description. The user can click the “see more” button to see all of the details of the description of the MLC.

• The actual MLC process diagram itself. Note that, as a process instance runs, the activities within a given process are highlighted in green.

• Process Instances (bottom-left): provides a list of the active “instances” of this process definition that are currently running.

• Camunda Incidents (bottom-right): provides a list of any MLC (Camunda) incidents that have occurred for this process definition. Typically these incidents are technical issues, such as an external system (e.g. Jira or ServiceNOW) cannot be reached.

From the “Process Instance” list in this page, a user may open the MLC Process Instance details (see next section).

Model Life Cycle (MLC) Process Instance

As mentioned above, a live automation that is running is called a Process Instance. Typically, each model has one or more automations running for that model, each of which is a process instance.

The MLC Process Instance includes:

• Information about the Process Definition: name, version, state (active/suspended), and start date.

• The actual MLC process diagram itself. Note that, as a process instance runs, the activities within a given process are highlighted in green.

• Steps (bottom-left): provides a text list of all the steps that have run within this process instance.

• Camunda Incidents (bottom-right): provides a list of any MLC (Camunda) incidents that have occurred for this process definition. Typically these incidents are technical issues, such as an external system (e.g. Jira or ServiceNOW) cannot be reached.

Compliance Module

The Compliance Module provides an overarching Risk and Compliance Dashboard, as well as a Reporting section to generate detailed model reports.

Compliance Overview

The Compliance Overview page provides high-level dashboards for Risk and Compliance executives to view:

• All models that are being used for business decisioning by Risk Classification and Business Unit/Organization

• Where all models are in their lifecycle by Business Unit/Organization

• Any risk and compliance breaches in Production

• Any models that are past their Validation Expiration Date or nearing their Validation Expiration Date.

Reporting Overview

The Reporting Overview page allows users to create a detailed audit report about all models that have been deployed. The user simply selects the desired model snapshot, the specific deployment date and clicks the “Run Report” button:

The resulting report contains all of the details of that specific model that was deployed at that specific date. A user can download the report as a PDF, by clicking the Download button in the top-right corner.

Operations Module

The Operations Module provides an overall Operations Dashboard, as well as details of the Runtimes, Jobs, and Notifications.

Operations Dashboard:

The Operations Dashboard provides a high-level view into all of the current running models, jobs, and open tickets. Additionally, it provides a view of the deployments by runtimes, and recent Open Tickets and Notifications.

Jobs List

The Jobs list shows all of the Jobs that have been orchestrated and are being managed by ModelOp Center. Note that Jobs can be orchestrated to run on ModelOp Runtimes, or they can be run on “external” Runtimes, such as a Spark or AWS SageMaker runtimes.

A user can use the filters in the top right to find the Jobs of interest:

Job Details

The Job details page provides key information about the job including: Job name, ID, category, type, runtime location, execution time, business model and the specific snapshot that is running, any associated model, job inputs and outputs, and the job logs:

Quick Links: Users can quickly jump to the Business Model, Business Model Snapshot, or Test Results (if applicable) by using the links at the top of the Job Details Page:

Runtimes List

The Runtimes page shows all of the Runtimes that have been registered and are being managed by ModelOp Center. Note that Runtimes can consist of ModelOp Runtimes, or they can be “external” Runtimes, such as a Spark or AWS SageMaker runtime.

For each Runtime, the following information is displayed:

A user may navigate to an individual runtime by clicking on a runtime row in the list.

Runtime Details

The Runtime Details page provides information about the current status, configuration, and deployment details for a particular runtime.

This includes:

• Overview tab :

  • Engine Details - displays the name, description and type of the runtime, identifying tags, its current state, release number, and when it was last updated

  • URL to the Deployed Model - displays information about the model that is currently deployed on the runtime, if applicable

  • Total Jobs - information about jobs that have been executed on the Runtime

  • Existing Endpoints - list of existing input and output endpoints that have been configured for the Runtime

• Endpoints tab: lists the assigned input and output endpoints. Authorized users can also add new input and output endpoints on this screen.

• Platform tab: platform information including system and framework-specific libraries installed on the Runtime

• Live View tab: live statistics and details about the endpoint throughput, CPU utilization, and system resource usage

Deployments List

The Deployments list contains a running view of all Deployments across all model types: Batch, REST, Streaming, across all environments and runtimes.

A user may filter the deployments based on the State (e.g. DEPLOYED, REMOVED), by using the filter in the top-right.

Notifications List

The Notifications list contains a running view of all notifications throughout the entire system, including Model, Runtime, Job, and Process notifications.

A user may filter the notifications by Type and/or Severity using the filters in the top right.

Next Article: Creating Production-Ready Models >