Versions Compared

Old Version 2

changes.mady.by.user Daniyal Saeed

Saved on

compared with

New Version Current

changes.mady.by.user Dave Trier

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

This article describes how ModelOp Center’s MLC’s can be used to orchestrate and automate the annual review processSagemaker Model Implementation.

Table of Contents

Table of Contents
maxLevel2
minLevel1

...

The ModelOp Life Cycle Manager (MLC Manager) orchestrates and automates the the various steps involved in conducting an annual review of a model for Model Completeness validation, Model Risk evaluation and Governance processes.

...

  1. Scheduler - the process is triggered by Use Case Registeration MLC based on ModelType “SAGEMAKER”.

  2. Model PIA Approval - the process creates PIA Approval Request Jira ticket and continues when the ticket is moved to Done status.

  3. Create Model Snapshot - A snapshot of the business model is created

  4. Run Standard Risk Tests - the process validates if the model assets with the BASELINE_DATA and COMPARATOR_DATA assets roles in preparation to run Standard Risk Tests on the model. If the assets are present in the model the MLC runs the Standard Risk Tests and creates a notification if any of the monitors in Standard Risk Tests fail to run. If the test job fails, a Jira ticket is created with the failed job error message. A test can be re-run if the Jira ticket is moved to Done.

  5. Approval Based on Test Results - A Jira ticket is created with document generated from the model test results for validation. If ticket is moved to Done the flow continues and adds the generated document to the model snapshot.

  6. Production Approval - Jira ticket is created for production deployment approval.

  7. Marked as Deployed- If ticket is moved to Done the snapshot will Mark as Deployed in Production

  8. Error handling -

    1. An error is logged if the flow is unable to get the business model

    2. An error notification is created for the following:
      i. Standard Risk Test snapshot is not found.
      ii. the first Jira ticket with document has been moved to Rejected.
      iii. any other exception occurs in the flow.

Related Articles

Next Article: Operationalizing Models: Overview Next Article: Example Workflows - LLM Model Implementations >