...
Model Execution Mode: a model “deployment” may consist of either:
Batch (Ephemeral) Deployment: pushing a model to a Runtime to execute a Batch Job, after which the Runtime is cleared of the model to be made available for other jobs.
Online (Persistent) Deployment: pushing a model to a Runtime that has persistent input and output endpoints for continual model scoring.
Model Runtime: a model runtime (sometimes referred to as “model serving”) is an environment in which a model--or model metrics--can be executed. A model is physically deployed into a runtime for execution, which, depending on the type of deployment, may be an ephemeral deployment (e.g. for a Batch job) or a long-running process (e.g. REST).
Examples: Python-enabled Docker container, Spark, Sagemaker, Azure ML Studio, DataIku, Domino Data Labs
Runtime Endpoints: when a model is deployed for online inferences (REST, streaming), a model runtime requires endpoints to which the requesting/consuming application can connect.
Examples:
REST: models deployed as REST require endpoints to allow for the synchronous REST-based communication
Kafka: models may receive inference requests from a Kafka topic and may push the model output (inferences) back onto a Kakfa topic. Endpoints are required to enable this integration
Runtime Environments (“Stages”): within a typical enterprise, there are multiple environments to facilitate development, testing, production staging, production, production-failover, etc. Depending on the life cycle of a model, the model may be required to move through one or more of these environments as part of standard IT/QA/Compliance processes.
Environment/Stages Tags: given that each enterprise has multiple execution platforms (e.g. Spark, Docker, etc.), each of which might have different runtime environments/stages, ModelOp Center uses the concept of “Environment/Stage tags”Stage”. This allows the ModelOps engineer to tag that a particular environment runtime is to be used for Development, while another might be used for UAT, or SIT or Production, etc. Through this approach, a MLC can be designed generically so that all models are automatically deployed and tested through each of the required stages in the standard IT/QA/Compliance process.
Example: IT requires that a model be deployed and tested in a Development, then SIT, and then UAT environment before it can be deployed to production. The ModelOps engineer would therefore want to add a mark as “DEV” tag to the Runtime in their development environment, an as “SIT” tag to the Runtime in their SIT environment, a as “UAT” tag to the Runtime in their UAT environment, and ultimately a “PROD” tag to the Runtime in their Prod environment. The MLC can then orchestrate the promotion of the model automatically across those stages, deploying the model to the appropriate runtime based on the Environment/Stage tag.
Model Service: as mentioned above, a model may be “servicing” (providing scores/inferences) for multiple consuming business applications or processes.
Model Service Tags: Therefore, from a Production Model Management perspective, ModelOp Center tracks these models by the use of “Model Service” tags to allow models to be enabled in multiple environments, potentially utilizing different consumption modes (e.g. Batch vs. REST), but ModelOp Center still is able to track and maintain exactly where each and every version of a model is running in production. The Model Life Cycle (MLC) uses the “Model Service” tag to determine exactly where a model should be deployed throughout its life cycle.
Examples:
3rd Party CC Fraud Model: a fraud model has been created that detects 3rd party credit card fraud. In order to make sure that the MLC knows where to enable this model, a “Model Service tag” such as “cc-fraud” would be added to the Model upon Snapshotting.
HELOC Credit Risk Model: a Credit Risk model for a home equity line of credit has been created. a “heloc-credit-line” tag should be added to this model to ensure that the MLC knows in which environments the model should be deployed throughout the model’s life cycle.
...
Model Operationalization is automated in ModelOp Center through the use of model life cycles (MLC’s). ModelOp Center provides a set of MLC’s out of the box, but enterprises can customize their own MLC’s and upload them into ModelOp Center to orchestrate their specific pathways to production and on-going management. The following articles provide details on how to Operationalize models in both online and batch mode within ModelOp Center.
Next Article: Operationalize a Model - REST >