(v1) Integrate with Jupyter
ModelOp Center provides plugins into common data science development tools such as Jupyter, to allow data scientists to interact with ModelOp Center from their preferred development environment.
Using ModelOp Center Jupyter plugin
Overview
The ModelOp Center (MOC) is flexible on how models are registered so that data scientists can leverage their preferred environments for model creation. In other words it is agnostic to the toolkit used to create the model.
Jupyter (http://jupyter.org/) is a popular tool among data scientists and other users of Python and R for building models. It is an evolution of the IPython project, and provides a Notebook interface similar to that of Mathematica or SAGE. The Jupyer Plugin enables the integration between a Jupyter Notebook and ModelOp Center to allow Data Scientists to register and interact with models directly from the Notebook. This allows the Data Scientists to work from their IDE of choice.
Installation and configuration instructions
ModelOp Center contains a package with the Jupyter plugin installation zip file. Please contact your ModelOp representative to get the latest version.
Installation
These installation instructions assume the following:
Jupyter notebook is already installed.
Maven is already installed.
You have access to ModelOp’s jupyter-plugin repository.
git clone https://github.com/modelop/jupyter-plugin.git
cd jupyter-plugin
mvn clean install
jupyter nbextension install target/jupyter-plugin-2.1-SNAPSHOT.zip
jupyter nbextension enable modelop-plugin/modelop-plugin
Next, start the Jupyter notebook instance (please note the configuration section before running):
jupyter notebook
Configuration
Before being able to use the plugin in the Jupyter instance, please make sure Jupyter knows where ModelOp Center can be reached by providing the base URL in either of the following ways.
Setting up the
MOC_INSTANCE_URL
environment variable with the base URL, e.g.: “http://modelop:8090” before running the jupyter notebook. Please note for Jupyterhub installations that the environment variables aren’t automatically passed to the notebook, but rather need to be whitelisted. See here for details on how to achieve this.Using the nbextensions configurator found in contrib nbextensions package. Once available this will provide an additional tab “Nbextensions” where you will be able to:
Enable/Disable the plugin altogether.
Override the environment variable with an arbitrary URL provided at the configurator itself.
Specify the environment variable name if
MOC_INSTANCE_URL
is not desired to hold the base URL
ModelOp specific functions available witdhin Jupyter
Once installed, when the user opens or creates a new Jupyter notebook file, the Jupyter notebook interface is provided with a set of toolbar buttons that interact with ModelOp Center and will be expanded in the sections below.
Registering a new model
From the toolbar, click on the “Register a new model with ModelOp Center” icon.
The “Add Model to Model Operations Center” window opens. Click in the left side of the code cells to designate that the code is part of the model. The selected cells will have a highlighted green border on the left.
Click Next. In the “Model Information” section, type the Model Name. Optionally, you can add a brief Description of the model.
Click Next. In the “Cell details” section:
Choose the remote repository configuration. Make sure ModelOp Center has been configured with access to the git repositories.
If the Git Remote text field is blank it will create a local (to ModelOp Center) git repository (no remote repository associated).
If the Git Remote text field is provided (with a valid accessible git repository URL), the local copy will be associated with given remote repository.
Check Use default repo option if you want the same git remote and branch for all the source code assets (cells). Otherwise, specify each asset with a different remote url or branch.
You can entirely disable git handling by switching off the Remote repo configuration toggle. This will in turn handle the source code assets using ModelOp Center internal storage only.Provide a File Name, and select Primary Model Source if appropriate.
The file name will be complemented with a default extension, but it can be provided by the input text field as part of the file name.
A model must have one and only one primary source code, which is where it will look for the model functions (more on model functions in the next step).
Click Next. Specify the model functions used in your code. (For additional information on the model functions please see here.) ModelOp Center allows the user to associate the triggering operations with specific code functions. Additionally it can identify if the functions already have predefined model functions associated via smart tags. These optional smart tags should be presented as comments in the line above the function definition and the supported comments are:
modelop.init
modelop.score
modelop.metrics
modelop.training
For example:# modelop.init def begin(): ...
You will find that the functions identified from source with a smart tag are already associated with one of the functions, and this selection is not modifiable. If you wish to change the associated model function, either you can edit or remove the smart tag comment, or you can modify the associated model function in the ModelOp Center UI.
The function associated via smart tag cannot be used for multiple model functions, but the remaining functions can be selected as options for the other associations:Click Next. On the next screen, add any attachments associated the model and select an asset role for each of the files to upload. This will help ModelOp Center runtime identify the accompanying assets for different purposes across the Model Life Cycle. Leave the asset role blank to let ModelOp Center automatically identify the asset based on the file type.
Click Next. Review the summary and click Submit Model.
Opening an existing model
'
From the toolbar, click on the “Load model from ModelOp Center“ icon.
Then, either select the desired model from the list or filter models using the live search bar. Note that your search can match the name of the model or the tags of the model.
The plugin will then create a new cells at the bottom of the active notebook with the source code assets' contents (each asset on a new cell).
Updating an existing model
There are multiple ways to update an existing model using the Jupyter plugin.
Pre-requisites
After registering a model with ModelOp Center, it is possible to update the model when it is open in the Notebook. Registration can be done with the steps outlined in Registering a new model or Opening an existing model (see Register a Model and Opening an existing model in Jupyter).
Editing the source code
Once the model has been loaded from ModelOp Center, make the modifications to the source code as desired using standard Jupyter cell edit techniques.
When the changes are complete, click on the “Update current model in ModelOp Center“ icon from the ModelOp Center toolbar.
A prompt will ask confirm the changes made to cell contents before updating the model. This option copies back the source code of the selected cell into ModelOp Center.
Updating model metadata fields
From the toolbar, click on the “Edit current model” icon.
Please ensure have selected the notebook cell with an associated model, and the plugin will prompt with a screen that allows to change the following:
Edit the Model Information
Change Model Name
Change the model Description
Edit the model functions associated with the primary source code. Please refer to “Registering a new model” section of this guide or here for more details. Please note that in order to avoid versioning conflicts, make sure to save all code modifications as indicated in the step “Editing the source code” above before changing the model functions. This will prevent recently introduced function definitions in the code to be associated before the code itself is synced back to ModelOp Center.
This step also sends information of the underlying Notebook environment in which this is run (system details, python details and libraries loaded) to ModelOp Center
Adding attachments
From the toolbar, click on the “Add attachment for existing model” icon.
Please make sure the notebook cell with an associated model is selected, and the plugin will prompt you with a file browser and a screen to attach selected file to the model in the active cell.
Select the role for the asset to upload or leave blank to be automatically determined by ModelOp Center based on the file type.
Click the “Attach” button to upload the attachment.
Submit model version
From the toolbar, click on the “Submit a model version to ModelOp Center” icon.
Please make sure you previously select the notebook cell with an associated model, and the plugin will prompt you with a screen confirming the last changes made to the model.
This action will submit a version for deploy, creating a deployable model. Please note that the model may or may not be automatically deployed to a model runtime depending on the logic defined by any active Model Life Cycle (MLC) processes.
Table of Contents
- 1 Using ModelOp Center Jupyter plugin
- 1.1 Overview
- 1.2 Installation and configuration instructions
- 1.2.1 Installation
- 1.2.2 Configuration
- 1.3 ModelOp specific functions available witdhin Jupyter
- 1.3.1 Registering a new model
- 1.3.2 Opening an existing model
- 1.3.3 Updating an existing model
- 1.3.3.1 Pre-requisites
- 1.3.3.2 Editing the source code
- 1.3.3.3 Updating model metadata fields
- 1.3.3.4 Adding attachments
- 1.3.4 Submit model version