Versions Compared

Old Version 22

changes.mady.by.user Brendan Kelly (Deactivated)

Saved on

compared with

New Version 23

changes.mady.by.user Brendan Kelly (Deactivated)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

This article describes how to use the ModelOp Command Center as the central station to monitor your models. It also describes the MLC processes that generate the alerts, and how to react to alerts, tasks and notifications reported by the MLC.

The primary audience for this article is the ModelOps Support Team.

Files | Questions?

Table of Contents

Table of Contents

ModelOp

Command

Center

DashboardModelOps Command Center Dashboard provides

Messages

Messages provide visibility into the health of the system and any events, conditions or tasks that need attention. You can customize an MLC process to test for technical problems within the model, underlying business problems that are revealed in the data set, and for the efficacy of a model using metrics that warn the operator when the metrics fall outside of the configured range.

Operators in the MLC are configured to produce messages that are accessible through the Dashboard.

Brendan Kelly (Deactivated) screenshot with overlay

ModelOp Center Messages

The types messages generated from the monitoring infrastructure models across the organizations in the single-pane of glass Dashboard.

The types messages generated include:

  • Alerts - test failures and model errors that require a response.

  • Tasks - user tasks such as approve a model, acknowledge a failed test, etc.

    • For details about viewing and responding to test failures, see Addressing User Tasks on this page.

  • Notifications - includes system status, engine status and errors, and model errors

Model Test Failures

  • Click Test Failure in the Models pane.

  • Triage

    • Model Errors or Engine Errors

  • Click Model Error.

  • Triage

    • Addressing

    • . Generated by ModelOp Center

    Responding to Notifications & Alerts

    Notifications and Alerts provide visibility into the models across teams and the organization. Notifications are generated by ModelOp Center as well as by MLC Processes. They provide information on what is happening within the system. Alerts function similarly, but require investigation and response. Alerts are generated by the MLC Process and have a severity of ERROR. You can use logic in the process to determine when they’re generated for ModelOps Support to respond to: test exceeds threshold, deployment failures, etc.

    Responding to Notifications & Alerts:

    1. Select the Notification from table to learn more information. For example, selecting a Runtime Notification will take you to the Runtime Detail View of the Runtime.

    2. Selecting into an Alert will provide information about the context of the Alert. In this example, you can see the details of the model that failed the back-test.

    3. You can also interact with this Alert to send it to have the issue resolved. In this case, you can see the ModelOp Support Team can notify the Developer about this issue.

    <screenshot>

    User Tasks

    User tasks are displayed under the Tasks & Alerts tab in the column on the left. Tasks are filtered by All Open Tasks and Tasks in Progress. ModelOps has a light-weight task management tool and also integrates with the task management tool of your choisechoice, such as ServiceNow or Jira.

    Image Removed

    1. Click the Tasks and Alerts icon in the left columnsidebar. Tasks are filtered by All Open Tasks and Tasks in Progress. 

    (NOTE: Replace this image with one that has some of the fields populated)

    RunTime/ Engine Monitoring

    1. Click into the User Task for context on the task to be performed.

    2. You have the option to assign the issue to yourself.

    3. At this point, you can complete the required task and the MLC Process will update.

    <screenshot>

    Runtime Monitoring

    You can click into a detailed real-time view of the Runtime information. This includes information for understanding the infrastructure, data pipeline, lineage and other information about the model and runtime status.

    The Runtime monitor displays the following information about the Runtime environment:

    • Endpoint throughput - volume of data through the runtime and model

    • CPU Utilization - User CPU utilization and Kernel CPU usage

    • RealSystem Resource Usage - real-time system resource memory usage

    • A diagram Lineage of the MLC process currently managing the modeldeployment - MLC Process meta-data that details the deployment information and history

    • Logs - A live scroll of the logs

    Image Removed

    http://mocaasin.modelop.com/#/engine_overview/engine_view<screenshot>

    Related Articles