V3.1 Release Notes

What’s New:

ModelOp Center v3.1 continues to build upon the strong AI governance and risk foundation by incorporating additional comprehensive risk monitoring, documentation generation, and governance processes. Additionally, v3.1 introduces support for Okta and AzureAD as Oauth2 providers.

Govern:

  • Governance Process Orchestration: enhancements to the out-of-the-box templates that enable orchestration of the entire model governance process across a variety of different use cases/business units to ensure adherence to overall AI governance procedures and processes.

  • Model Risk - Annual Review MLC’s: out-of-the-box templates for orchestrating the typical steps required for annual model reviews from a risk and governance perspective

  • Risk Documentation Generation Enhancements: support for manual generation of model documentation using both automated and manually generated test results

  • MLC Incident Management: integrated tracking of MLC-specific technical incidents, including troubleshooting the incident and resuming the process instance

 

Test, Monitor, & Visualize

  • Regression Diagnostics: additional set of out-of-the-box tests that produce a comprehensive analysis of Normality, MultiCollinearity, Linearity, AutoCorrelation, and Homoscedasticity.

  • Automated Risk Testing: updates to the out-of-the-box automated tests for risk review including Performance, Drift, Stability, Fairness, Normality, MultiCollinearity, Linearity, AutoCorrelation, and Homoscedasticity, etc. Ability to compare against standard thresholds for these metrics, with drill down into the detailed metrics calculations.

  • Manual Test Execution: enhancements to the UI to allow for manually executing any test or monitor, using either out-of-the-box tests/monitors or custom monitors. Test results are captured and made available in the UI

 

General:

  • User Experience: minor User Interface enhancements including:

    • New dashboard that displays all Batch deployments by environment, status, etc.

    • Updated Executive Dashboard to allow for drill down into individual monitoring results

    • Ability to filter Access Group display names

  • AWS Cognito: enhancements to ModelOp Center’s support for AWS Cognito as an OAuth2 provider

  • Okta: support for integrating ModelOp Center with Okta as an OAuth2 provider

  • AzureAD (MVP): support for integrating ModelOp Center with AzureAD as an OAuth2 provider

  • Gitlab Pipelines (beta): support for integrating with Gitlab pipelines for CICD processes

 

Specific Details:

 

Full List of Enhancements:

  • Removed schema checking option for Training & Metrics jobs in the Job Wizard

  • Updated the delete endpoint capability to remove the REST roundtrip output endpoint

  • Updated the Job details page to show more information about the Job Result

  • Updated UI to improve handling of OAuth2 token refresh

  • Added popout feature for Log view component for the Runtime Live View

  • Added pre-install of unixODBC to enable pyodbc installation on ModelOp Runtime

  • Added support for monitoring data in Databricks via SparkSQL

  • Updated getRuntimesList endpoint to support DocumentDB

  • Added support for Kerberos-based Kafka integration

  • Added ability to see all Deployed snapshots, including Batch based models

  • Added support for more than 10 columns in the Dashboard heatmap

  • Added capability to clone existing associated models from a prior snapshot when creating a new snapshot

  • Updated Monitors Model Inventory column sorting and filtering

  • Optimized MLC execution by moving to the use of incidents for delegates/external tasks

  • Added support for client to client communication throught JWT

  • Optimized ModelOp Center core services to more accurately indicate status during startup

  • Updated Dashboard to require a DeployedModel instead of finding the latest snapshot

  • Updated the UI to provide a dedicated Scheduler for dashboard execution

  • Added support for refreshing OAuth2 tokens for Ping Fedederate

  • Updated UI for live log tracing

  • Added capability to manually run a test in the UI

  • Added standard thresholds for common risk tests

  • Updated Model Import screen to allow for custom display name

  • Added ability to manually generate documentation from existing test results

  • Added support for provisioning ModelOp Center using Terraform for AWS Fargate-based clusters

  • Added a ModelOp Center preinstall utility

  • Updated Dashboard heatmap to allow direct click-through to results

  • Added support for Helm-based installation of ModelOp Center

  • Added additional OOTB tests/monitors, including ones for Autocorrelation, Homoscedasticity, Linearity, and Multicollinearity metrics

  • Simplified the process for calling the ModelOp Monitors python package from the OOTB registered monitors

  • Fixed DIV issue on the Compliance Overview Page

  • Updated OOTB MLC's to incorporate new Regression-model risk tests

  • Updated document service to provide better error messages when certain values are missing during document generation

  • Added new MLC's to support annual review automations and model risk testing

  • Added comprehensive risk test for Regression models

  • Updated OOTB deployment MLC's to ignore runtime matching for OOTB monitors

  • Added support in the UI for scheduling the Dashboard

  • Fixed issue to support statsmodels version issues with new OOTB monitors/tests

  • Fixed issue with kerberos authentication to Cloudera clusters if the principal is not yarn

  • Optimized MLC execution to handle Business Model vs. System Failure notifications

  • Updated ModelOp Center installation to support multiple OOTB model packages

  • Updated API endpoint to retrieve "Models in Production" to optimize performance

  • Updated document service to allow for multiple model test results to be passed in for manual documentation generation

  • Added MLC delegate to attach a schedule to a Deployed Model

  • Added support for filtering in the tags used in document service

  • Added document template for Annual Model Reviews

  • Updated OOTB monitors to simplify how the backend ModelOp monitors python package is called

  • Addressed identified vulnerabilities with the hadoop-common and hadoop-hdfs libraries

  • Optimized Job execution by moving job validation checking to Job Monitor

  • Fixed issue with expired Oauth2 refresh token for Ping Federate

  • Added support to authenticate the Jupyter notebook plugin with Keycloak

  • Addressed perfomance issues with AWS-RDS MySQL support for MLC Manager

  • Added support for configuring AWS Cognito via Helm

  • Added support for obtaining AWS Cognito tokens for Customer testing accounts

  • Updated AWS Cognito integration to allow for configurable display name for users

 

Bug Fixes:

  • Expanded support for larger files for large model asset upload (up to 1GB)

  • Fixed javascript TypeError in Runtime Live View Page

  • Removed capability that allows assets to be moved to a modelOp-controlled HDFS folder

  • Fixed javascript Console error during snapshot creation

  • Fixed issue with display for adding assets by URL for models backed by Azure Blob Storage

  • Added sort by organization to Business Model Inventory page

  • Fixed issue with scheduler signal name dropdown box

  • Fixed issue with tag population for document generation service

  • Added support to list out all deployments: batch, REST, kafka

  • Updated the Help and Documentation links

  • Fixed tag filtering issue in the Business Model Inventory

  • Fixed issue with findAllByStateInAndEngineInProductionTrueOrderByCreatedDateDesd endpoint

  • Fixed issue with navigation breadcrumb in the Jobs UI page

  • Fixed UI issue with saving Kafka endpoint configuration

  • Fixed issue when uploading assets to a secure (https) jira instance

  • Updated issue with the CreatedBy field used in secured environments

  • Fixed UI issue with re-running an existing Monitoring job.

  • Fixed sorting issue for Monitors Model Inventory

  • Implemented performance improvements for the Reporting Page

  • Updated Job Wizard to immediately filter to Azure-based assets for Azure-backed model

  • Fixed issue with UI routing to Camunda Cockpit in Openshift

  • Fixed issue with Sagemaker model duplicate imports

  • Fixed UI issue with uploading external file assets when selecting HDFS as a backing store

  • Fixed issue in retrieving Spark job status from a Kerberized Cloudera cluster

  • Fixed issue with UI MLC tab of the snapshot to avoid excessive "flicker" updates

  • Fixed issue with UI navigation breadcrumbs in MLC process instance page

  • Fixed issue where kafka endpoints disappear in the UI after model is deployed

  • Fixed issue with MLC tab of the snapshots page not refreshing properly

  • Updated error message during model import from git when the git repo contains corrupted files