...
Model Submitted - a deployable model object has been created, which is a snapshot in time of the model with the typical goal of moving the model into production. Clicking “Create New Snapshot” in the Model Details page triggers the start of this MLC process.
Training - based on a metadata flag, a Training Job can be initiated to train the model. A service task automatically polls checking for the Training Job to finish before proceeding.
Testing - based on a metadata flag “run_test” (read from a snapshot level tag), an automated, reproducible Metrics Job executes, and then the results are persisted in ModelOp Center. If the test process fails a Jira ticket will be created with failure information and another run test attempt will be made if Jira ticket is moved to Done.
Approval Based on Test Results - based on a metadata flag “jira” (read from a snapshot level tag), the previous test results are analyzed if a DMN file is associated with the model. The details of the model, including all of the core information about the model, the changes to the model and the test results, are passed on to the reviewer on the Jira ticket.
Model Deployment - the MLC Process finds receives a ModelOp Runtime that is tagged with the appropriate tag, and attempts to deploy the model onto that runtime. Or list of ModelOp Runtimes if the matching runtime has no endpoints defined, then the model will be deployed as batch, leaving it ready for scoring on the matching runtime at the time of batch execution otherwise it will an Online deployment.
Error handling - when models are rejected, errors occur running the tests, or the model fails to deploy, the process creates Jira tasks to review the reasons for failure so they can take the appropriate actions.
A rejection Jira ticket is created with the details for the reviewer if:
We intended to run a Metrics Test Job (step 3) the model is missing test data
The job execution failed
The analyzed test results didn’t pass the DMN criteria
The generated Jira review was rejected
An error Jira ticket is created if other general errors are found during the process of deployment.
...