...
Model Registration:
Jupyter: added a new GUI-based Jupyter plugin that provides an intuitive wizard-based approach to register a new model, as well as mechanisms to update existing models, add attachments such as trained model artifacts, and to submit a model for productionization.
CLI: streamlined the approach to register new models via the CLI, with flexible options for adding in schemas, as appropriate.
DataIku DSS: added support to integrate with DataIku’s API’s to onboard code-based Python models created in DataIku DSS.
API: added new API to support registration and persistence of new models, allowing for systematic onboarding of new models, as required.
Model Lifecycle (MLC) Automation:
MLC EngineManager: new micro service that automates and manages the common processes in the lifecycle of a model, effectively providing the underpinning for scaled Model Operations (ModelOps). The MLC engine manager allows for the creation of customized model lifecycle (MLC) processes via a drag and drop user interface, including allowing for defining custom business rules around triggering proactive notifications.
MLC Templates: starter model lifecycle templates to automate and manage common processes in the lifecycle of a model, such as model onboarding, model promotion, on-going backtesting, etc.
Champion / Challenger: added support for both automated and manual champion / challenger comparisons across model versions or different models.
...
Batch Jobs: added support for manually or automatically executing batch jobs, which may include batch scoring jobs, batch validation/metrics jobs, or batch training jobs. Each batch run is monitored, tracked, and has its results persisted with the model’s metadata for traceability.
Runtimes: added support for generic runtimes, which may consist of ModelOp Center’s runtime (called “ModelOp Center Engine”Runtime”) or external runtimes (e.g. DataIku). ModelOp Center can configure, manage, and monitor model execution across these runtimes, allowing for centralized monitoring across all of your existing and future investments in data/analytic platforms.
Endpoints: added support for generic endpoints to support “always on” running models, including support for REST and Kafka-based endpoints. This complements the ability to run regular batch jobs (see above) for production and testing loads.
Notifications & Alerting: added support for real-time notifications and configurable alerts based on business rules. Notifications and alerts surface vital information about models, their model lifecycles, runtimes, jobs, and other operations-focused items to provide a comprehensive view of the state of your models across the enterprise.
...
Model Metadata: extended substantially our core model management and metadata collection capabilities. This includes:
Standard Model Representation: expanded “standard model representation”, which is how we define and collect all of the elements that compose a model in an agnostic way, regardless of the language, framework, data science development tooling, etc.
Metadata Collection: expanded support for on-going and continual metadata collection about a model, including collecting and tracking metadata about a model’s intended usage, source code, externalized schemas, trained artifacts, dependencies, and other core elements that compose a model.
Externalized Artifacts: added support for storing large model artifacts in S3-compliant object stores, allowing organizations to leverage cost-efficient approaches to managing the large trained model artifacts (and other model artifacts) that are required for model execution for certain classes of models.
Model Test Results: added support for generic test results, allowing for the automated or manual execution of defined statistical, technical, or business testing. Data scientists and other users have the ability to define the specific metrics that they would like to see, as each model and use case vary and thus may require a specific testing approach. All test results are persisted with the specific version of the model for full auditability.
Model Reproducibility Automation: added support for automatically reproducing training and/or validation results in a target environment from the original model assets. This capability supports the ModelOps best practices to enable systematic reproducibility of core functions of a model (training, testing, scoring) to support both quality control and auditability. All reproducible runs are persisted with that specific version of the model allowing for rapid and comprehensive response to any audit inquiry.
Model Traceability: added support--via the MLC enginemanager--to track each step in a model’s lifecycle for each version of a model to provide a full lineage of a model from when it is registered, to when it is tested, validated, approved, deployed, improved, and eventually retired.
...