Introduction

To comply with governance and risk policies, model owners, validators, governance officers, and other stakeholders need to create documentation about a model throughout its life cycle. This can include model design documentation, model review (validation) documentation, quarterly reports, annual reviews, and other document types. Often, these documents incorporate model overview information, metadata, and a variety of different metrics about the model, which, in total, can lead to 10’s to 100’s of pages of documentation.

ModelOp Center streamlines the documentation generation process by automatically populating an existing documentation template with a model’s metadata, metrics, and test result details. This approach saves significant time for model constituents, but also drives consistency across the entire enterprise--consistency of the documentation process, documentation templates, and documentation content.

Supported Documentation Types

Overall, ModelOp is able to generate two types of documents: (1) Model Cards (Markdown - .MD format) (2) Microsoft Word Documents (.DOCX format). The below article applies to both - the only difference is the template is Markdown vs. Microsoft Words

Starter Template

To get started, users can start with this ModelOp-provided documentation templates:

1. Model Card Template

2. Microsoft Word Documentation Template

Inputs for Auto-Documentation Generation

The following information about a model can be automatically included in a Model Documentation:

• Use Case / Model Information: any of the core information / metadata about a given Use Case / Model Implementation is accessible to be included in the auto-generated documentation. This includes any of the information in the full storedModel object. For full details of what metadata is available for a storedModel object, please visit the Model Metadata Details page

• Snapshot Information: any of the core information / metadata about a given Model Snapshot is accessible to be included in the auto-generated documentation. This includes any of the information in the full deployableModel object. For full details of what metadata is available for a deployableModel object, please visit the Model Metadata Details page

• Model Test Results: any of the information / data included in a Model Test Result is accessible to be included in the auto-generated documentation. For details about the Model Test Result, see the Monitoring Overview page of this product documentation.

Creating a Document Template

Overview

ModelOp Center can automatically populate model information/content into a Word document template through the use of ModelOp tags. These are defined placeholders that are inserted into a Microsoft Word template to indicate where ModelOp Center should populate certain pieces of model information. Note that any Word document can be used – most customers start with their existing Documentation template and simply insert the ModelOp tags in the appropriate location. In this way, a user can build a highly polished and customized Model Document that will automatically be populated with model content.

Open

In the above screenshot, note how a standard Microsoft Word document was created and formatted to the user’s preferences, and the ModelOp tags were simply inserted into the appropriate sections of the template.

ModelOp Tag Structure

A ModelOp tag is simply constructed using the following format:

Open

As a simple example, a Use Case or Model Implementation’s metadata is represented in the backend of ModelOp as a storedModel object. To pull pieces of this metadata into the Microsoft Word template, the user simply needs to write the name of the fields from the storedModel object into the tag:

Accessing ModelOp Information in a Tag

Overview

As mentioned above, any of the information about a Use Case, Model Implementation, Snapshot, or Model Test Result can be inserted as a ModelOp tag into a word document. This includes all of the nested information within each of these objects.

Accessing ModelOp Object Information

Using the base tag structure, a user can then access any field within a given ModelOp object by using one of these approaches:

• Simple (name) - in the simplest case where the information is a base field on the ModelOp object, the user need only specify the name of the field

• Nested (name1.name2.name3) if the information is in a nested structure, the user can traverse the nested structure with a period separating the parent from child objects using the name1.name2.name3 format.

• Indexed (name[index]) - if a field within the ModelOp object contains an array, the name[index] format can be used to specify the appropriate (zero-relative) entry in the array

• Mapped (name(key)) - if a field within the ModelOp object contains a map, the name(key) format can be used to specify the appropriate field based on the key.

• Combined (name1.name2[index].name3(key)) - combining mapped, nested, and indexed references is also supported.

Example Usages

Single-Value (Nested)

A simple, but common example is to pull the core metadata information about a Use Case or Model Implementation into a Word Document template. As a refresher, this metadata is represented in the backend of ModelOp as a storedModel object. To pull pieces of the Use Case or Model Implementation Model’s metadata into the Microsoft Word template, the user simply needs to write the name of the field from the storedModel object into the Word Document template as a ModelOp tag, using <<modelop.storedModel>> as the base and then the name of the specific field, with all elements separated by a period.

It is easiest to show with an example:

• On the left-hand side of this picture is the storedModel Object for a given Use Case or Model Implementation. This can be obtained by querying the REST API for that storedModel. As one can see, the storedModel Object has numerous key/value pairs, including complex nested objects.

• On the right-hand side of this picture are the corresponding ModelOp tags that can be inserted into a Word Document template so that the values will automatically be populated from the Business Model into the Word document.

Single-Value (Nested + Mapped)

Another simple, but common example, is to pull a single Test/Monitor metric value using a key. As a refresher, a model’s metrics are stored as a modelTestResult object within a given business model Snapshot. To pull Test/Monitor metrics into the Microsoft Word template, the user simply needs to write the name of the field from the modelTestResult object into the Word Document template as a ModelOp tag, using <<modelop.modelTestResult>> as the base and then the name of the specific metric, with all elements separated by a period.

Again, it is easiest to show with an example:

• On the left-hand side of this picture is the modelTestResult Object for a given Model’s Snapshot. This can be obtained by querying the REST API for that modelTestResult. As one can see, the modelTestResult Object has numerous key/value pairs, including complex nested objects.

• On the right-hand side of this picture are the corresponding ModelOp tags that can be inserted into a Word Document template so that the values will automatically be populated from the Model Test Result object into the Word document.

Multiple Metrics from an Array (Nested + Mapped + Indexed)

If there are multiple Metrics in an array, a user can leverage Indexes of the array to pull out the specific values. Furthermore, ModelOp Center can pull these values into a simple Word table, if the ModelOp tag is placed within a Word document table object.

Again, let’s look at an example:

• On the left-hand side of this picture is the modelTestResult Object for a given Model’s Snapshot. Note that there is a specific metric that contains an array of objects. The user is interested in pulling the first object in the array.

• On the right-hand side of this picture are the corresponding ModelOp tags. Note that the specific metric is referred to by metric key (in this case “linearity”), and the first object in the array is referenced with “[0]”

Tables

Tables can easily be generated in a Word template from a model’s test result (i.e. from the modelTestResult Object for a given Model’s Snapshot). This can be done by:

1. Preparing the custom test/monitor output: as with any custom test/monitor, when creating the test/monitor, the user should output a dictionary of results from the metrics function of the custom test/monitor (see https://modelopdocs.atlassian.net/wiki/spaces/ARCHIVE/pages/1726840843 for more details). Specifically, to create a table, the structure the output of the test/monitor dictionary should include a field with the following format:

2. Add a Table in the Document Template: in one’s Microsoft Word template, insert a “1x1 Table” in the appropriate location (In Microsoft Word, from the Insert menu, select to add a table):

3. Add ModelOp Tags in the Table: add the appropriate ModelOp tags into the Microsoft Word template. Using the metric output example in step 1, the tags would be:

Note: the key in the output structure (in this case “generic_table”) should be included in parenthesis.

4. Result: Here is an example of the Microsoft Word template and the resulting Table for the example metrics listed above:

Line Graphs

Line graphs can easily be generated in a Word template from a model’s test result (i.e. from the modelTestResult Object for a given Model’s Snapshot). This can be done by:

1. Preparing the custom test/monitor output: as with any custom test/monitor, when creating the test/monitor, the user should output a dictionary of results from the metrics function of the custom test/monitor (see https://modelopdocs.atlassian.net/wiki/spaces/ARCHIVE/pages/1726840843 for more details). For a line graph, ModelOp supports multiple types of line graphs, each of which would follow one of the below structures in the metrics output:

c. Decimal Line Graph: specifically, to create a Decimal Line Graph, the structure the output of the test/monitor dictionary should include a field with the following format:

2. Add ModelOp Tags in the Table: add the appropriate ModelOp tags into the Microsoft Word template. Using the metric output example in step 1, the tags would be:

Note: the key in the output structure (in this case “generic_line_graph”) should be included in parenthesis.

4. Result: Here is an example of the Microsoft Word template and the resulting Table for the example metrics listed above:

Bar Charts

Bar graphs can also be generated in a Word template from a model’s test result (i.e. from the modelTestResult Object for a given Model’s Snapshot). This can be done by:

1. Preparing the custom test/monitor output: as with any custom test/monitor, when creating the test/monitor, the user should output a dictionary of results from the metrics function of the custom test/monitor (see https://modelopdocs.atlassian.net/wiki/spaces/ARCHIVE/pages/1726840843 for more details). For a line graph, ModelOp supports multiple types of line graphs, each of which would follow one of the below structures in the metrics output:

2. Add ModelOp Tags in the Table: add the appropriate ModelOp tags into the Microsoft Word template. Using the metric output example in step 1, the tags would be:

Note: the key in the output structure (in this case “generic_bar_graph”) should be included in parenthesis.

4. Result: Here is an example of the Microsoft Word template and the resulting Table for the example metrics listed above:

For a deeper look at OOTB monitors, see https://modelopdocs.atlassian.net/wiki/spaces/ARCHIVE/pages/1726840843.

Next Article: Executive Dashboard >