V2.0 Release Notes

Owned by Dave Trier
Last updated: Mar 30, 2021
5 min read

Overview:

ModelOp is extremely proud to release the 2nd generation of our flagship product: ModelOp Center! ModelOp Center v2.0 includes a substantial number of new features--running on top of an updated architecture-- to provide our Customers more flexibility in how they Deploy, Monitor, and Govern all of their models across the enterprise. We encourage you to visit our ModelOps Guide for an overview of all the essentials to running an enterprise ModelOps program, as well as our updated ModelOp Center architecture for an overview of the new services incorporated into the v2.0 architecture.

What’s New:

General:

  • Refreshed Web UI (“Command Center”): incorporated a new overall layout and design to streamline the user experience and provide additional details about all models, their model lifecycles, and the model runtimes. This includes an Operations-focused landing page to surface critical issues and notifications for proactive attention to ensure your models are providing the highest value possible.

  • CLI: revamped command line interface (CLI) to allow those that prefer interacting via a terminal to have easy access to interact with ModelOp Center, including getting model and job status, registering new models, adding new assets, and many more features.

  • Model Factory Plugins: added new Jupyter and DataIku plugins to provide Data Scientists streamlined access to register and update models within ModelOp Center directly from their favorite model factory.

  • Updated architecture: updated the core architecture that includes a number of new micro services, an enhanced service registry, a best-in-class gateway, and other enhancement to further enable scale and extensibility for the enterprise.

Deploy

  • Model Registration:

    • Jupyter: added a new GUI-based Jupyter plugin that provides an intuitive wizard-based approach to register a new model, as well as mechanisms to update existing models, add attachments such as trained model artifacts, and to submit a model for productionization.

    • CLI: streamlined the approach to register new models via the CLI, with flexible options for adding in schemas, as appropriate.

    • DataIku DSS: added support to integrate with DataIku’s API’s to onboard code-based Python models created in DataIku DSS.

    • API: added new API to support registration and persistence of new models, allowing for systematic onboarding of new models, as required.

  • Model Lifecycle (MLC) Automation:

    • MLC Manager: new micro service that automates and manages the common processes in the lifecycle of a model, effectively providing the underpinning for scaled Model Operations (ModelOps). The MLC manager allows for the creation of customized model lifecycle (MLC) processes via a drag and drop user interface, including allowing for defining custom business rules around triggering proactive notifications.

    • MLC Templates: starter model lifecycle templates to automate and manage common processes in the lifecycle of a model, such as model onboarding, model promotion, on-going backtesting, etc.

  • Champion / Challenger: added support for both automated and manual champion / challenger comparisons across model versions or different models.

Score & Monitor

  • Batch Jobs: added support for manually or automatically executing batch jobs, which may include batch scoring jobs, batch validation/metrics jobs, or batch training jobs. Each batch run is monitored, tracked, and has its results persisted with the model’s metadata for traceability.

  • Runtimes: added support for generic runtimes, which may consist of ModelOp Center’s runtime (called “ModelOp Center Runtime”) or external runtimes (e.g. DataIku). ModelOp Center can configure, manage, and monitor model execution across these runtimes, allowing for centralized monitoring across all of your existing and future investments in data/analytic platforms.

  • Endpoints: added support for generic endpoints to support “always on” running models, including support for REST and Kafka-based endpoints. This complements the ability to run regular batch jobs (see above) for production and testing loads.

  • Notifications & Alerting: added support for real-time notifications and configurable alerts based on business rules. Notifications and alerts surface vital information about models, their model lifecycles, runtimes, jobs, and other operations-focused items to provide a comprehensive view of the state of your models across the enterprise.

Govern

  • Model Metadata: extended substantially our core model management and metadata collection capabilities. This includes:

    • Standard Model Representation: expanded “standard model representation”, which is how we define and collect all of the elements that compose a model in an agnostic way, regardless of the language, framework, data science development tooling, etc.

    • Metadata Collection: expanded support for on-going and continual metadata collection about a model, including collecting and tracking metadata about a model’s intended usage, source code, externalized schemas, trained artifacts, dependencies, and other core elements that compose a model.

  • Externalized Artifacts: added support for storing large model artifacts in S3-compliant object stores, allowing organizations to leverage cost-efficient approaches to managing the large trained model artifacts (and other model artifacts) that are required for model execution for certain classes of models.

  • Model Test Results: added support for generic test results, allowing for the automated or manual execution of defined statistical, technical, or business testing. Data scientists and other users have the ability to define the specific metrics that they would like to see, as each model and use case vary and thus may require a specific testing approach. All test results are persisted with the specific version of the model for full auditability.

  • Model Reproducibility Automation: added support for automatically reproducing training and/or validation results in a target environment from the original model assets. This capability supports the ModelOps best practices to enable systematic reproducibility of core functions of a model (training, testing, scoring) to support both quality control and auditability. All reproducible runs are persisted with that specific version of the model allowing for rapid and comprehensive response to any audit inquiry.

  • Model Traceability: added support--via the MLC manager--to track each step in a model’s lifecycle for each version of a model to provide a full lineage of a model from when it is registered, to when it is tested, validated, approved, deployed, improved, and eventually retired.

Extended Support:

With the release of v2.0, there a several v1.x items that will begin to be deprecated over time. However, ModelOp will offer extended support for these components for licensed Customers until Customers are upgraded to v2.0.

  • v1.x CLI: both the Go and Python v1.x CLI will be deprecated over time with the introduction of v2.0 software. Note that the latest v2.0 CLI should be downloaded for usage with our latest v2.0 release.

  • v1.x Python SDK: the v1.x Python SDK will be deprecated over time with the introduction of v2.0. The Python SDK is being replaced by the much more extensive and robust set of API’s offered via the v2.0 architecture. For more details on the v2.0 API’s, please see the v2.0 Technical Guide materials.