Page Comparison

...

The Run Back Test is a simple monitor that runs a test against a new set of labeled data for a given model.

...

Image Removed

1. Start Event - a triggered signal event initiates the monitor. This signal (com.modelop.mlc.definitions.Signals_Run_Back_Test_Jira / com.modelop.mlc.definitions.Signals_Run_Back_Test_ServiceNow) can be triggered by a rest API providing the variables used during the process.

2. Get model - based on the MODEL_ID signal variable, the process will fetch the snapshot

3. Get data - Based on the variables provided using the signal, the input/output will be decided. If the signal has INPUT_FILE the process will use as input to the job or find an asset with TEST_DATA role on the model. Similarly, if OUTPUT_FILE is provided with the signal, it will be used for storing the job’s output. Otherwise, it will create an embedded output file and use it for the job.

4. Run and Analyze test - runs a Metrics Test batch job to evaluate the model with the new data. Based on the results of the test, if the model has an associated DMN file, it will be used to determine success criteria.

5. Test Passed - generates a notification stating that the test passed

6. Test Failed - if test fails a Jira/ServiceNow ticket is created. The details of the model, including all of the core information about the model, the changes to the model and the test results with failure, are passed on to the reviewer on the Jira/Servicenow ticket.

7. Error Handling - The following are the scenarios if any error or exception occurs in the process

...