This article describes how ModelOp Center automatically generates documentation using an existing documentation template

Table of Contents

1 Introduction
- 1.1 Starter Template
2 Inputs for Auto-Documentation Generation
3 Creating a Document Template
- 3.1 Overview
- 3.2 ModelOp Tag Structure
- 3.3 Accessing ModelOp Information in a Tag
  - 3.3.1 Overview
  - 3.3.2 Accessing ModelOp Object Information
4 Example Usages
- 4.1 Single-Value (Nested)
- 4.2 Single-Value (Nested + Mapped)
- 4.3 Multiple Metrics from an Array (Nested + Mapped + Indexed)
- 4.4 Tables
- 4.5 Line Graphs
- 4.6 Bar Charts

Introduction

To comply with governance and risk policies, model owners, validators, governance officers, and other stakeholders need to create documentation about a model throughout its life cycle. This can include model design documentation, model review (validation) documentation, quarterly reports, annual reviews, and other document types. Often, these documents incorporate model overview information, metadata, and a variety of different metrics about the model, which, in total, can lead to 10’s to 100’s of pages of documentation.

ModelOp Center streamlines the documentation generation process by automatically populating an existing documentation template with a model’s metadata, metrics, and test result details. This approach saves significant time for model constituents, but also drives consistency across the entire enterprise--consistency of the documentation process, documentation templates, and documentation content.

Starter Template

To get started, users can start with this ModelOp-provided documentation template, which provides multiple examples of pulling in metadata and metrics into a document report.

Inputs for Auto-Documentation Generation

The following information about a model can be automatically included in a Model Documentation:

Business Model Information: any of the core information / metadata about a given Business Model is accessible to be included in the auto-generated documentation. This includes any of the information in the full storedModel object. For full details of what metadata is available for a storedModel object, please visit the Model Metadata Details page
Snapshot Information: any of the core information / metadata about a given Model Snapshot is accessible to be included in the auto-generated documentation. This includes any of the information in the full deployableModel object. For full details of what metadata is available for a deployableModel object, please visit the Model Metadata Details page
Model Test Results: any of the information / data included in a Model Test Result is accessible to be included in the auto-generated documentation. For details about the Model Test Result, see the Monitoring Overview page or the specific Monitor examples in the Monitor section of this product documentation.

Creating a Document Template

Overview

ModelOp Center can automatically populate model information/content into a Word document template through the use of ModelOp tags. These are defined placeholders that are inserted into a Microsoft Word template to indicate where ModelOp Center should populate certain pieces of model information. Note that any Word document can be used – most customers start with their existing Documentation template and simply insert the ModelOp tags in the appropriate location. In this way, a user can build a highly polished and customized Model Document that will automatically be populated with model content.

In the above screenshot, note how a standard Microsoft Word document was created and formatted to the user’s preferences, and the ModelOp tags were simply inserted into the appropriate sections of the template.

ModelOp Tag Structure

A ModelOp tag is simply constructed using the following format:

As a simple example, a Business Model’s metadata is represented in the backend of ModelOp as a storedModel object. To pull pieces of the Business Model’s metadata into the Microsoft Word template, the user simply needs to write the name of the fields from the storedModel object into the tag:

Accessing ModelOp Information in a Tag

Overview

As mentioned above, any of the information about a Business Model, Snapshot, or Model Test Result can be inserted as a ModelOp tag into a word document. This includes all of the nested information within each of these objects.

The base ModelOp tag structure of each of the ModelOp core objects is as follows:

Business Model → <<modelop.storedModel>>
Snapshot → <<modelop.snapshot>>
Model Test Result → <<modelop.modelTestResult>>

Accessing ModelOp Object Information

Using the base tag structure, a user can then access any field within a given ModelOp object by using one of these approaches:

Simple (name) - in the simplest case where the information is a base field on the ModelOp object, the user need only specify the name of the field
Nested (name1.name2.name3) if the information is in a nested structure, the user can traverse the nested structure with a period separating the parent from child objects using the name1.name2.name3 format.
Indexed (name[index]) - if a field within the ModelOp object contains an array, the name[index] format can be used to specify the appropriate (zero-relative) entry in the array
Mapped (name(key)) - if a field within the ModelOp object contains a map, the name(key) format can be used to specify the appropriate field based on the key.
Combined (name1.name2[index].name3(key)) - combining mapped, nested, and indexed references is also supported.

Example Usages

Single-Value (Nested)

A simple, but common example is to pull the core metadata information about a Business Model into a Word Document template. As a refresher, a Business Model’s metadata is represented in the backend of ModelOp as a storedModel object. To pull pieces of the Business Model’s metadata into the Microsoft Word template, the user simply needs to write the name of the field from the storedModel object into the Word Document template as a ModelOp tag, using <<modelop.storedModel>> as the base and then the name of the specific field, with all elements separated by a period.

In general, the tag structure is is composed of three parts (all separated by a period):

<<modelop>> → the base for all ModelOp objects
<<storedModel>> → the name of the ModelOp object
<<field>> → the specific field in the storedModel object of interest

It is easiest to show with an example:

On the left-hand side of this picture is the storedModel Object for a given Business Model. This can be obtained by querying the REST API for that storedModel. As one can see, the storedModel Object has numerous key/value pairs, including complex nested objects.
On the right-hand side of this picture are the corresponding ModelOp tags that can be inserted into a Word Document template so that the values will automatically be populated from the Business Model into the Word document.

Single-Value (Nested + Mapped)

Another simple, but common example, is to pull a single Test/Monitor metric value using a key. As a refresher, a model’s metrics are stored as a modelTestResult object within a given business model Snapshot. To pull Test/Monitor metrics into the Microsoft Word template, the user simply needs to write the name of the field from the modelTestResult object into the Word Document template as a ModelOp tag, using <<modelop.modelTestResult>> as the base and then the name of the specific metric, with all elements separated by a period.

In general, the tag structure is is composed of three parts (all separated by a period):

<<modelop>> → the base for all ModelOp objects
<<modelTestResult>> → the name of the ModelOp object
<<metric>> → the specific field in the storedModel object of interest

Again, it is easiest to show with an example:

On the left-hand side of this picture is the modelTestResult Object for a given Business Model’s Snapshot. This can be obtained by querying the REST API for that modelTestResult. As one can see, the modelTestResult Object has numerous key/value pairs, including complex nested objects.
On the right-hand side of this picture are the corresponding ModelOp tags that can be inserted into a Word Document template so that the values will automatically be populated from the Model Test Result object into the Word document.

Multiple Metrics from an Array (Nested + Mapped + Indexed)

If there are multiple Metrics in an array, a user can leverage Indexes of the array to pull out the specific values. Furthermore, ModelOp Center can pull these values into a simple Word table, if the ModelOp tag is placed within a Word document table object.

Again, let’s look at an example:

On the left-hand side of this picture is the modelTestResult Object for a given Business Model’s Snapshot. Note that there is a specific metric that contains an array of objects. The user is interested in pulling the first object in the array.
On the right-hand side of this picture are the corresponding ModelOp tags. Note that the specific metric is referred to by metric key (in this case “linearity”), and the first object in the array is referenced with “[0]”

Tables

Tables can easily be generated in a Word template from a model’s test result (i.e. from the modelTestResult Object for a given Business Model’s Snapshot). This can be done by:

Preparing the custom test/monitor output: as with any custom test/monitor, when creating the test/monitor, the user should output a dictionary of results from the metrics function of the custom test/monitor (see https://modelopdocs.atlassian.net/wiki/spaces/ARCHIVE/pages/1726840843 for more details). Specifically, to create a table, the structure the output of the test/monitor dictionary should include a field with the following format:

  "generic_table": [
	  {"data1" : 1, "data2" : 2, "data3" : 3},
	  {"data1" : 2, "data2" : 3, "data3": 4},
	  {"data1" :  3, "data2" : 4, "data3" : 5}
	]

Add a Table in the Document Template: in one’s Microsoft Word template, insert a “1x1 Table” in the appropriate location (In Microsoft Word, from the Insert menu, select to add a table):

Add ModelOp Tags in the Table: add the appropriate ModelOp tags into the Microsoft Word template. Using the metric output example in step 1, the tags would be:

Note: the key in the output structure (in this case “generic_table”) should be included in parenthesis.

Result: Here is an example of the Microsoft Word template and the resulting Table for the example metrics listed above:

Line Graphs

Line graphs can easily be generated in a Word template from a model’s test result (i.e. from the modelTestResult Object for a given Business Model’s Snapshot). This can be done by:

Preparing the custom test/monitor output: as with any custom test/monitor, when creating the test/monitor, the user should output a dictionary of results from the metrics function of the custom test/monitor (see https://modelopdocs.atlassian.net/wiki/spaces/ARCHIVE/pages/1726840843 for more details). For a line graph, ModelOp supports multiple types of line graphs, each of which would follow one of the below structures in the metrics output:

a. Time Line Graph: specifically, to create a Time Line Graph, the structure the output of the test/monitor dictionary should include a field with the following format:

  	"time_line_graph": {
	  "title" : "Example Line Graph - Timeseries Data",
	  "x_axis_label": "X Axis",
	  "y_axis_label": "Y Axis",
	  "data": {
		"data1": [["2023-02-27T20:10:20",100], ["2023-03-01T20:10:20",200], ["2023-03-03T20:10:20", 300]],
		"data2": [["2023-02-28T20:10:20", 350], ["2023-03-02T20:10:20", 250], ["2023-03-04T20:10:20", 150]]
	  }
	}

b. Generic Line Graph: specifically, to create a Generic Line Graph, the structure the output of the test/monitor dictionary should include a field with the following format:

  "generic_line_graph": {
	"title" : "Example Line Graph - XY Data",
	"x_axis_label": "X Axis",
	"y_axis_label": "Y Axis",
	"data": {
	  "data1": [[1,100], [3,200], [5, 300]],
	  "data2": [[2, 350], [4, 250], [6, 150]]
	}
  }

c. Decimal Line Graph: specifically, to create a Decimal Line Graph, the structure the output of the test/monitor dictionary should include a field with the following format:

Add ModelOp Tags in the Table: add the appropriate ModelOp tags into the Microsoft Word template. Using the metric output example in step 1, the tags would be:

Note: the key in the output structure (in this case “generic_line_graph”) should be included in parenthesis.

Result: Here is an example of the Microsoft Word template and the resulting Table for the example metrics listed above:

Bar Charts

Bar graphs can also be generated in a Word template from a model’s test result (i.e. from the modelTestResult Object for a given Business Model’s Snapshot). This can be done by:

Preparing the custom test/monitor output: as with any custom test/monitor, when creating the test/monitor, the user should output a dictionary of results from the metrics function of the custom test/monitor (see https://modelopdocs.atlassian.net/wiki/spaces/ARCHIVE/pages/1726840843 for more details). For a line graph, ModelOp supports multiple types of line graphs, each of which would follow one of the below structures in the metrics output:

a. Vertical Bar Graph: specifically, to create a Vertical Bar Graph, the structure the output of the test/monitor dictionary should include a field with the following format:

b. Horizontal Bar Graph: specifically, to create a Horizontal Bar Graph, the structure the output of the test/monitor dictionary should include a field with the following format:

Add ModelOp Tags in the Table: add the appropriate ModelOp tags into the Microsoft Word template. Using the metric output example in step 1, the tags would be:

Note: the key in the output structure (in this case “generic_bar_graph”) should be included in parenthesis.

Result: Here is an example of the Microsoft Word template and the resulting Table for the example metrics listed above:

For a deeper look at OOTB monitors, see https://modelopdocs.atlassian.net/wiki/spaces/ARCHIVE/pages/1726840843.