Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

ModelOp Center provides plugins into common data science development tools such as Jupyter, to allow data scientists to interact with ModelOp Center from their preferred development environment.

Table of Contents

Table of Contents

What’s New in v2.1:

  1. Including “Model Functions” definition feature to “Add Model” wizard.

    1. Inspects the code searching for functions to include as options of the dropdown.

    2. Read “smart comments” and pre-sets functions.

    3. Include help about “Model Functions” and “Smart Tags” in information icon.

  2. Editing existing model.

    1. Basic Model Info.

    2. Model Functions (same way).

    3. Platform summary.

    4. Catches unsaved changes in code before saving.

  3. ModelOp toolbar update.

    1. Get rid of “Model Details” button (it was for debugging purposes).

    2. The “Asset Details” button lets you edit the asset name and repo info.

  4. Include Live Search on the “Load Model” screen.

    1. Allow search by model name

    2. Allow search by tag

  5. Support for external parameterized ModelOp Center base URL.

    1. Environment variable setup.

    2. Parameter in NBextensions config.

    3. Enable CORS support to invoke REST API from Jupyter external host.

    4. Enable external libraries compatibility when running on an custom instance.

    5. Add support for Jupyter versions prior to 5.7.0, incompatibility between jquery 3+ and jquery-ui <1.12

  6. Validating file extension to source code asset filename (when creating model and when editing asset file name)

  7. Visualizing ModelOp Jupyter plugin version in all dialogs

  8. Visual feedback on the attachments file upload status

  9. Support for asset role definition on the attachments (during model creation and during file upload).

  10. Enable plugin for Jupyterhub notebook.

  11. Bug Fixes:

    1. When a model was just created the “submit” screen didn’t show the last changes (the initial commit).

    2. Update local cell metadata when saving a model (to avoid keeping outdated data).

    3. Validate to include only code cells in the model creation

    4. Include the model summary in the ‘Save’ screen.

    5. Updating a model overwrote MOC with the current (modified) local copy. That got fixed on “Model Save”, “Model functions editing”, “Asset button edit”

Using ModelOp Center Jupyter plugin

...

Overview

The ModelOp Center (MOC) is flexible on how models are registered so that data scientists can leverage their preferred environments for model creation. In other words it is agnostic to the toolkit used to create the model.

Jupyter (http://jupyter.org/) is a popular tool among data scientists and other users of Python and R for building models. It is an evolution of the IPython project, and provides a Notebook interface similar to that of Mathematica or SAGE.

...

The Jupyer Plugin enables the integration between a Jupyter Notebook and ModelOp Center to allow Data Scientists to register and interact with models directly from the Notebook. This allows the Data Scientists to work from their IDE of choice.

Installation and configuration instructions

...

Include Page
Jupyter plugin installation

...

Installation

These installation instructions assume the following:

  • Jupyter notebook is already installed.

  • Maven is already installed.

  • You have access to ModelOp’s jupyter-plugin repository.

Code Block
languagebash
git clone https://github.com/modelop/jupyter-plugin.git
cd jupyter-plugin
mvn clean install
jupyter nbextension install target/jupyter-plugin-2.1-SNAPSHOT.zip
jupyter nbextension enable modelop-plugin/modelop-plugin

Then you can run your Jupyter notebook instance (please note the configuration section before running):

Code Block
jupyter notebook

Configuration

Before being able to use the plugin in you Jupyter instance, please make sure your Jupyter knows where your ModelOp Center can be reached by providing the base URL in either of the following ways.

  1. Setting up the MOC_INSTANCE_URL environment variable with the base URL, e.g.: “http://modelop:8090” before running the jupyter notebook. Please note for Jupyterhub installations that the environment variables aren’t automatically passed to the notebook, but rather need to be whitelisted. See here for details on how to achieve this.

  2. Using the nbextensions configurator found in contrib nbextensions package. Once available this will provide an additional tab “Nbextensions” where you will be able to:

    1. Enable/Disable the plugin altogether.

    2. Override the environment variable with an arbitrary URL provided at the configurator itself.

    3. Specify the environment variable name if MOC_INSTANCE_URL is not desired to hold the base URL

      Image Removed

...

and configuration instructions
Jupyter plugin installation and configuration instructions

ModelOp specific functions available witdhin Jupyter

Once installed, when the user opens or creates a new Jupyter notebook file, the Jupyter notebook interface is provided with a

...

set of toolbar buttons that interact with ModelOp Center and will be expanded in the sections below.

...

Image Added

Registering a new model

...

From the toolbar, click on the “Register a new model with ModelOp Center” icon.

...

The “Add Model to Model Operations Center” window opens. Click in the left side of the code cells to designate that the code is part of the model. The selected cells will have a highlighted green border on the left.

...

...

...

Click Next. In the “Cell details” section:

  1. Choose your remote repository configuration. Make sure you have already configured ModelOp Center with access to your git repositories.
    If the Git Remote text field is blank it will create a local (to ModelOp Center) git repository (no remote repository associated).
    If the Git Remote text field is provided (with a valid accesible git repository URL), the local copy will be associated with given remote repository.
    Check Use default repo option if you want the same git remote and branch for all the source code assets (cells). Otherwise, you can specify each asset with a different remote url or branch.
    You can entirely disable git handling by switching off the Remote repo configuration toggle. This will in turn handle the source code assets using ModelOp Center internal storage only.

  2. Provide a File Name, and select Primary Model Source if appropriate.
    The file name will be complemented with a default extension, but it can be provided by the input text field as part of the file name.
    A model must have one and only one primary source code, which is where it will look for the model functions (more on model functions in the next step).

    Image Removed

...

Click Next. Specify the model functions used in your code. For additional information on the model functions please see here.
ModelOp Center allows the user to associate the triggering operations with specific code functions. Additionally it can identify if the functions already have predefined model functions associated via “smart tags”. These optional “smart tags” should be presented as comments in the line above the function definition and the supported comments are:
- modelop.init
- modelop.score
- modelop.metrics
- modelop.training
For example:

Code Block
languagepy
# modelop.init
def begin():
...

You will find that the functions identified from source with a smart tag, are already associated with one of the functions and this selection is not modifiable (if you wish to change the function associated, please change or remove the smart tag comment).
Also the function associated via smart tag can no longer be used as another model function, but the other functions are still displayed as options for the other associations.

...

...

Click Next. On the next screen, add any attachments that go with the model and select an asset role for each of the files to upload. This will help ModelOp Center runtime identify the accompanying assets for different purposes accross the Model Life Cycle. Leave the asset role blank to let ModelOp Center automatically identify the asset based on the file type.

...

...

Click Next. Review the summary. If you are satisfied, click Submit Model

Opening an existing model

...

From the toolbar, click on the “Load model from ModelOp Center“ icon

Select the desired model from the list or filter using the live search bar. Note that search text can match the name of the model or the tags of the model.

...

The plugin will then create a new cells at the bottom of the active notebook with the source code assets' contents (each asset on a new cell).

Updating an existing model

There are multiple ways in which a model can be updated, depending on what you are ultimately achieving.

Editing the source code

...

From the toolbar, click on the “Update current model in ModelOp Center“ icon.

Please make sure the previously select a cell which is part of a model already loaded or created using either of the two options above (Registering a new model / Opening an existing model). The screen prompted will confirm the latest changes worked on the cell code before updating the model. This option copies back the source code of the selected cell into the ModelOp Center.

...

Updating model metadata fields

...

From the toolbar, click on the “Edit current model” icon.

Please make sure you previously select the notebook cell with an associated model, and the plugin will prompt you with a screen that allows to change the following:

  1. Edit the Model Information

    1. Change Model Name

    2. Change the model Description

  2. Edit the model functions associated with the primary source code. Please refer to “Registering a new model” section of this guide or here for more details. Please note that in order to avoid versioning conflicts, make sure to save all code modifications as indicated in the step “Editing the source code” above before changing the model functions. This will prevent recently introduced function definitions in the code to be associated before the code itself is synced back to ModelOp Center.

This step also sends ModelOp Center with information of the underlying platform in which this is run (system details, python details and libraries loaded).

Adding attachments

...

From the toolbar, click on the “Add attachment for existing model” icon.

Please make sure you previously select the notebook cell with an associated model, and the plugin will prompt you with a file browser and a screen to attach selected file to the model in the active cell.

Select the role for the asset to upload or leave blank to be automatically determined by ModelOp Center based on the file type.

...

Submit model version

...

From the toolbar, click on the “Submit a model version to ModelOp Center” icon.

Please make sure you previously select the notebook cell with an associated model, and the plugin will prompt you with a screen confirming the last changes made to the model.

...

Include Page
Registering a model using Jupyter
Registering a model using Jupyter

Opening an existing model

'

Include Page
Opening an existing model in Jupyter
Opening an existing model in Jupyter

Updating an existing model

Include Page
Updating an existing model using Jupyter
Updating an existing model using Jupyter

Submit model version

Include Page
Submit model version using Jupyter
Submit model version using Jupyter

Table of Contents

Table of Contents