Introduction

ModelOp Center allows integration with Git platforms in order to externalize the management of model assets as well as allow for a distributed development of such resources. This in place, integration allows for the development of such assets to be done from the platform (IDE) of choice of the Data Scientist providing greater flexibility using a widely accepted technology such as Git.

ModelOp Center allows different configured levels of interaction with Git:

1. No Git interaction.: Model assets are not stored in a Git repository but rather are stored in a local database.

2. Local Git repository. : It is possible to create and work in a local Git repository which will be used by within ModelOp Center to commit changes to assetswhere changes to the asset will be committed changes. A remote git repository can be added to the asset at any point.

3. Remote Git repository. We can It is also possible to configure a remote git repository to sync external changes back into ModelOp Center working directory. It is also possible optionally enable/disable the feature to push to upstreamA configuration option controls whether the commits in the repo are pushed to the remote.

Remote import

Given a remote repository url to clone, ModelOp Center clones down the repo and creates a “Model” with all files represented as assets within it. All non-source files will be turned into are imported as external assets and loaded into the configured external file asset repository instance (e.g. S3 / MinIO). All source assets will be loaded into the “Model” instance, and have their repository information loaded appropriately.

• Source files are automatically determined (by file extension):

  • Model source: ".py", ".py3", ".c", ".r", ".pfa", ".ipynb", ".ppfa", ".java", ".m"

  • Test result comparator: ".dmn", ".cmmn"

• Non-source files are considered any other type of files.

ModelOp Center allows to import a git repository through ModelOp Center Dashboard

1. Click on the “Models” menu item.

2. On the top right corner click on “Import”.

3. Provide the details of the “Remote Repository Url” and “Branch”.
   Please note for the URL to extract the one depending on your Git platform, generally you can find this URL by going to the “Clone” button and selecting the “HTTPS clone” option.

4. Click on “Import model from Git”

Info

For the above steps to work on private repositories with login restrictions please see the sections below

Git integration setup

In order to be able to configure a private remote repository for any model in ModelOp Center, it’s necessary to go through certain configuration settings

Git credentials

In order for ModelOp Center to know which username and password to use to connect to the remote repository, we have the following properties:

Container: "model-manager":Note: these steps assume the repository is public without authentication. The following sections detail how to configure the integration when authentication is required.

Git Integration

In order to integrate authenticated remote repository, ModelOp Center will need to be configured to utilize a Service Account to pull from the specified repository. This Service Account should be able to read and/ or write for the repositories to imported and integrated with ModelOp Center.

Git Credentials

The username and password to integrate with the Git repository are set as properties in the container definition for model-manager. These can be obscured using Kubernetes Secrets or other existing credential management systems.

model-manage.git.username=<The git username>
model-manage.git.password=<The username's passphrase>

The above configuration will be used to access all remote repositories from ModelOp Center, which requires the Service Account be added to integrated repositories.

Local Repository Settings

ModelOp Center also provides additional parameters , so that we can to configure the behavior of our the local repository.

Container: "model-manager":

model-manage.repo.push=false   # Enable/Disable to push back to the remote repository after an update has been applied to an asset in ModelOp Center
model-manage.repo.change-poll-rate=120000   # Time interval between remote repository polls for updates.
model-manage.repo.base-directory=/tmp/model-manage-repos   # Default container's base for git repositories

Git Config

It is possible to add git-specific configurations to the way control how ModelOp Center’s git repository behaves. Please refer to git scm for more details on the git configuration and how it can be used to achieve certain behavior.

The idea behind ModelOp Center’s git config customization is that any git config variable can be mapped in the following way.

The example desired git config file being the following:

  [http]
    sslVerify = true
  [http "https://example.com/scm/demo/repo.git"]
    sslVerify = false
  [format]
    notes = foo
    notes = bar

Then the expected ModelOp Center configuration would be the following:

Container: "model-manager"
  model-manage.git.config.http.sslVerify=true
  model-manage.git.config.http.https://example.com/scm/demo/repo.git.sslVerify=false
  model-manage.git.config.format.notes.0=foo
  model-manage.git.config.format.notes.1=bar

Please refer to git-scm for more comprehensive list of variables that can be set through git config.

Load on startup

Status
title demo only

Optionally you Model Manage can configure configured to automatically import repositories upon startup, . This is mainly used for demo purposes.

  model-manage.git.load-on-startup.repos.0.repositoryRemote="https://github.com/modelop/first-repo.git"
  model-manage.git.load-on-startup.repos.0.repositoryBranch="master"
  model-manage.git.load-on-startup.repos.1.repositoryRemote="https://github.com/modelop/second-repo.git"
  model-manage.git.load-on-startup.repos.1.repositoryBranch="master"
  ...

View

Asset Git Details

ModelOp Center provides multiple ways to see the details of the git integration for various assets.

ModelOp Center Dashboard

You can view the The model’s assets repository configuration in can be seen within the ModelOp Center Dashboard.

1. Click on the “Models” menu item.

2. From the list on the main panel, select the desired model to inspect.

3. From the sub-menus on the left, click on “<> Sources”

4. On the main panel, select the “Metadata” tab on the title

5. On the left panel, you can inspect the various source files as needed.

Jupyter Notebook

Plugin

Git configuration is available directly within a Notebook via the Jupyter Notebook Plugin. When registering or opening a model, you can view , these details are available in the “View” button, a ModelOp specific Cell Toolbar button.

1. On Within the Jupyter Notebook, click on the menu “View”.

2. Click on “Cell Toolbar”.

3. Select the “ModelOp Model” toolbar option.

4. On the desired model asset (cell) click on the toolbar button “Asset Details”. This button will be visible for ModelOp registered models only.

   1. If the selected asset didn’t does not have repository configured previously, it will allow the user to configure it, similarly as when registering the model for the first time.
      

      

   2. If the selected asset already had a repository configured it will allow to modify the “Remote URL” and “Branch” for the asset, these values can be modified as well.