Introduction

The ModelOp Runtime is a containerized service that is capable of running arbitrary model code. The runtime supports models written in several languages, but python3 and R are the most common. Model code, as well as associated model assets, are materialized inside the engine, at which point, some language specific model runner code executes the model. The runtime itself takes care of retrieving input data, and is also responsible for writing yielded output data to the configured endpoints.

Note that MOC itself supports other model runtimes, including Spark and AWS Sagemaker. Models aimed for execution by those platforms are not handled by the ModelOp Runtime.

ModelOp Runtime Concepts

The runtime handles two basic types of models: batch jobs, and deployed models.

Batch jobs

Batch jobs are self contained model runs where the input data is defined by the batch job directly, and output data is similarly contained in the completed batch job run. Batch jobs have a fairly straightforward lifecycle in an engine - they are queued up internally and then run one at a time until each is complete. Output data can either be attached directly to the job or pushed dynamically to an external file asset store (like S3).

Deployed models

Deployed models do not contain their own input data. Instead, they are typically attached to an endpoint (like a REST endpoint) where input data can arrive at any time, be scored dynamically, and the yielded to the configured output endpoint. This is typically done using a single REST endpoint that synchronously provides a record to be scored, and then waits for the output of the model before returning that data.

Installing/Configuring a ModelOp Runtime

The runtime runs inside a container. It is configured via a combination of a environment variables and an application.json file. Details on configuration can be found here: ModelOp Runtime Details | Configuration-Options . Additionally, at least one control port must be exposed so that the rest of the MOC ecosystem can talk to the engine. By default, this is port 8003. Additionally, when running a deployed model, the engine can open a separate port that can be used exclusively for scoring against the model. This port is configurable and must be exposed independently of the control port.

Scoring with a ModelOp Runtime

When a model is deployed against a REST endpoint, a POST command can be used to score data. Details of the command are provided here: ModelOp Runtime Details | Score-data-via-the-REST-endpoint . Data received via the POST command is funneled into the model’s action function, and whatever data is yielded from that function is then funneled back and returned in the response to the REST call.

Running Metrics with a ModelOp Runtime

Metrics jobs are, for the most part, just regular batch jobs. The main difference is that all the input records are supplied at the same time to the action function. So while the action function for a normal scoring batch job can be expected to be called once for each input record, the action function for a metrics job can be expected to be called only once.

Automatically Importing Requirements into a ModelOp Runtime

In certain situations, the infrastructure/security team allows for dynamic installations of libraries/packages on a model-specific basis within the ModelOp runtime. ModelOp Center allows a user to define the specific dependent library/package requirements in a requirements.txt file, and will automatically install those designated libraries from an internal or external package repository on deployment of a model.

To make use of this feature:

• Pre-requisites: ensure that pip install is enabled for containers within the orchestration environment that ModelOp runs. This pip install can refer to library locations internally (e.g. an internal artifactory repository) or to external locations (e.g. PyPI).

• Create a requirements.txt file in the industry-standard format to list dependent libraries and the specific versions

• Upon registration of the model, ModelOp Center will automatically assign the role of ‘Required libraries’ to model assets named requirements.txt. However, the user may update any asset with the role.

• During deployment of a model to a ModelOp runtime, if a model contains an asset with the ‘Required libraries’ role, the ModelOp Runtime will attempt to install the libraries described therein.

  • This feature is currently only supported for python models, and the format of the asset must be that of a typical requirements.txt file.

  • The Runtime will attempt to install the libraries indicated by the contents of each required libraries asset before running the model code.

  • Each model execution is done in an isolated environment, so requirements installed for model A will not be present for model B.

Next Article: ModelOp Runtime Details