...

    - path: ""
      authenticationType: "BASIC"
      accessKeyId: "my_key"
      secretAccessKey: "my_secret"

    - path: "s3.us-east-2.amazonaws.com"
      roleArnEnvVariable: "TEST_ROLE_ARN"
      webIdentityTokenFileEnvVariable: "TEST_TOKEN_FILE"
      sessionPrefix: "testprefixABC"

  - host: "host1"
    username: "username1"
    password: "password1"

moc-builder

When leveraging moc-builder to generate a MOC environment, your environment’s config/config.json file can be updated with the desired values for the aforementioned environment variables. A subsequent compose command will use the values in the config.json when building kubernetes manifests.

application.json

{
  "sql-credentials": [
    {
      "host": "sql-server-abc",
      "username": "my_user",
      "password": "my_password"
    },
    {
      "host": "sql-other-server-def",
      "username": {"from_file": "/config/the-username"},
      "password": {"from_env": "THE_ENV_VAR_WHERE_MY_PASSWORD_IS"}
    }
  ],
  "s3-credentials": {
    "storedCredentials": [
      {
        "path": "",
        "authenticationType": "BASIC",
        "accessKeyId" : "my_key",
        "secretAccessKey" : "my_secret"
      }
    ],
    "eksServiceAccountCredentials": [
      {
        "path": "s3.us-east-2.amazonaws.com",
        "roleArnEnvVariable" : "TEST_ROLE_ARN",
        "webIdentityTokenFileEnvVariable" : "TEST_TOKEN_FILE",
        "sessionPrefix" : "testprefixABC"
      }
    ]
  },
  "azure-credentials": {
    "storedCredentials": [
      {
        "account": "my_account",
        "authenticationType": "SHARED_KEY",
        "path": "",
        "sharedKey": "mysharedkey=="
      }
    ]
  }
}

The ‘sql-credentials’ section allows username and password credentials to be associated with specific sql database hosts. Credentials can be hardcoded directly in the config file, located in a file, or located in an environment variable.

The ‘s3-credentials’ section allows credentials to be hardcoded directly (under storedCredentials) or determined dynamically using the AWS STS service. The credentials are chosen based on the longest matching path.

API’s

The following are a list of commonly used runtime APIs. The provided curl commands assume the runtime is running at localhost:8003. When communicating with a runtime through a MOC gateway, the localhost:8003 will likely look like this: gateway:8090/engine-name, where engine-name is the name the runtime used when it registered with the registry via the env variable: MODELOP_ENGINE_NAME.

Reset the engine

Reset the engine, causing the current model to be unloaded, and the various endpoints reset. This will also cause the current batch job to complete immediately, and move on to the next batch job.

curl -X POST http://localhost:8003/2/engine/reset

Load a deployed model

Load the deployed model contained in the ‘my-deployable-model.json’ file.

...

    - bootstrap: my.kafka.server
      client-type: producer
      topic: my-topic
      security_protocol: ssl
      ssl_ca_location: /ssl-stuff/ca-cert
      ssl_keystore_location: /ssl-stuff/kafka.client.keystore.pkcs12
      ssl_keystore_password: "password

    - bootstrap: my.kafka.server
      client-type: consumer
      topic: my-topic
      batch_size: 100

  - host: "host1"
    username: "username1"
    password: "password1"

moc-builder

When leveraging moc-builder to generate a MOC environment, your environment’s config/config.json file can be updated with the desired values for the aforementioned environment variables. A subsequent compose command will use the values in the config.json when building kubernetes manifests.

application.json

{
  "sql-credentials": [
    {
      "host": "sql-server-abc",
      "username": "my_user",
      "password": "my_password"
    },
    {
      "host": "sql-other-server-def",
      "username": {"from_file": "/config/the-username"},
      "password": {"from_env": "THE_ENV_VAR_WHERE_MY_PASSWORD_IS"}
    }
  ],
  "s3-credentials": {
    "storedCredentials": [
      {
        "path": "",
        "authenticationType": "BASIC",
        "accessKeyId" : "my_key",
        "secretAccessKey" : "my_secret"
      }
    ],
    "eksServiceAccountCredentials": [
      {
        "path": "s3.us-east-2.amazonaws.com",
        "roleArnEnvVariable" : "TEST_ROLE_ARN",
        "webIdentityTokenFileEnvVariable" : "TEST_TOKEN_FILE",
        "sessionPrefix" : "testprefixABC"
      }
    ]
  },
  "azure-credentials": {
    "storedCredentials": [
      {
        "account": "my_account",
        "authenticationType": "SHARED_KEY",
        "path": "",
        "sharedKey": "mysharedkey=="
      }
    ]
  }
}

The ‘sql-credentials’ section allows username and password credentials to be associated with specific sql database hosts. Credentials can be hardcoded directly in the config file, located in a file, or located in an environment variable.

The ‘s3-credentials’ section allows credentials to be hardcoded directly (under storedCredentials) or determined dynamically using the AWS STS service. The credentials are chosen based on the longest matching path.

API’s

The following are a list of commonly used runtime APIs. The provided curl commands assume the runtime is running at localhost:8003. When communicating with a runtime through a MOC gateway, the localhost:8003 will likely look like this: gateway:8090/engine-name, where engine-name is the name the runtime used when it registered with the registry via the env variable: MODELOP_ENGINE_NAME.

Reset the engine

Reset the engine, causing the current model to be unloaded, and the various endpoints reset. This will also cause the current batch job to complete immediately, and move on to the next batch job.

curl -X POST http://localhost:8003/2/engine/reset

Load a deployed model

Load the deployed model contained in the ‘my-deployable-model.json’ file.

curl -X PUT --data-binary "@my-deployable-model.json" -H "Content-Type: application/json" http://localhost:8003/api/deployedModels/{UUID}

There is an optional wait_for_init query parameter for the /api/deployedModels/{UUID} endpoint. The default value of true means to not return from the REST call until the model’s init function has completed successfully. A value of false means to return immediately once the model and the specified endpoints have been loaded, but before the init function returns.

Load a batch job

Load a batch job into the runtime.

curl -X POST --data-binary "@my-batch-job.json" -H "Content-Type: application/json" http://localhost:8003/api/modelBatchJobs/{UUID}

Delete a batch job

Delete the batch job with the specified UUID.

curl -X DELETE http://localhost:8003/api/modelBatchJobs/{UUID}

List all batch jobs

Return all the batch jobs known to the runtime.

curl http://localhost:8003/api/deployedModels/{UUID}

There is an optional wait_for_init query parameter for the /api/deployedModels/{UUID} endpoint. The default value of true means to not return from the REST call until the model’s init function has completed successfully. A value of false means to return immediately once the model and the specified endpoints have been loaded, but before the init function returns.

Load a batch job

Load a batch job into the runtimemodelBatchJobs/

List a specific batch job

Return the batch job with the specified UUID.

curl http://localhost:8003/api/modelBatchJobs/{UUID}

Score data via the REST endpoint

Submit the contents of input.json to the runtime. Note that in this example, the timeout has been set to 100 seconds, overriding the default set by the MODELOP_ROUNDTRIP_DEFAULT_TIMEOUT environment variable.

curl -X POST --data-binary "@my-batch-job@input.json" -H "Content-Type: application/json" http://localhost:8003/api/modelBatchJobs/{UUID}

Delete a batch job

Delete the batch job with the specified UUID.

curl -X DELETE http://localhost:8003/api/modelBatchJobs/{UUID}

List all batch jobs

Return all the batch jobs known to the runtimeroundtrip/0/1?timeout=100000

Get current engine status

Retrieve information about the currently running model, attached streams, and overall state of the engine.

curl http://localhost:8003/api1/modelBatchJobs/

List a specific batch job

Return the batch job with the specified UUID.

curl http://localhost:8003/api/modelBatchJobs/{UUID}

Score data via the REST endpoint

Submit the contents of input.json to the runtime. Note that in this example, the timeout has been set to 100 seconds, overriding the default set by the MODELOP_ROUNDTRIP_DEFAULT_TIMEOUT environment variable.

curl -X POST --data-binary "@input.json" -H "Content-Type: application/json" http://localhost:8003/api/roundtrip/0/1?timeout=100000

Get current engine status

Retrieve information about the currently running model, attached streams, and overall state of the engine.

curl http://localhost:8003/1/job/status

Sample response:

...

...

job/status

Sample response:

{
  "streams": [
    {
      "slot": 0,
      "name": "rest",
      "eof": false,
      "descriptor": {
        "Transport": {
          "secured": false,
          "roundtrip": true,
          "port": 9003,
          "Type": "REST",
          "Mode": "simple"
        },
        "slotSchema": 0"$inherit",
        "nameName": "rest",
        "eofEncoding": false"JSON",
        "descriptorDescription": { "Application Restful Endpoint"
      }
    },
   "Transport": {
      "slot": 1,
      "name": "secured": falserest_out",
          "roundtripeof": truefalse,
          "portdescriptor": 9003,{
          "TypeTransport": "REST",{
          "ModeType": "simplesynthetic"
        },
        "Schema": "$inherit",
        "NameEncoding": "restJSON",
      }
 "Encoding": "JSON",   }
  ],
  "Descriptionstate": "Application Restful Endpoint"
      }
    }running",
  "model":  {
 
    "slotsource": 1,"[model source code...]",
    "namesnapshots": "rest_outnone",
      "eofslots": false,[
      "descriptor": {
        "Transportslot": {0,
          "Typeschema": "synthetic"
        }null,
        "Schemarecordsets": "$inherit"false,
        "Encodingaction": "JSONaction"
      },
      {
  }   ],   "stateslot": "running"1,
  "model": {     "sourceschema": "[model source code...]"null,
    "snapshots": "none",     "slotsrecordsets": [
 false,
    {         "slotaction": 0,null
        "schema": null,}
    ],
    "recordsetsname": false,
   "test_model.py",
    "actionmtype": "actionpython3",
    "attachments": []
  },
  "jets": [
  {  {
      "slotsandbox": 1"123472369",

       "schemapid": null55,
 
      "recordsetsbusy": false
    }
  ],
    "initialTags": [],
   "actiongroup": null
      }
    ],
    "name": "test_model.py",
    "mtype": "python3",
    "attachments": []
  },
  "jets": [
    {
      "sandbox": "123472369",
      "pid": 55,
      "busy": false
    }
  ],
  "initialTags": [],
  "group": "modelop"
}

Model Definition Injection

Both the python and R runners support an optional parameter for the init function. Normally, the init function is defined with 0 arguments, but if a single argument is defined, the runner will populate that argument with a ‘model definition’ object. In python, this object is a dictionary. In R, this object is a list. In both cases, the object has two keys. There will be a ‘rawJson’ key that contains the unparsed json string that contains either the job or the deployedModel object in its serialized form. There will also be either a ‘job’ or a ‘deployedModel' key, the value of which will either be a deserialized job or deployedModel object. In python, the deserialized job or deployedModel will be the result of calling json.loads() on the contents of the ‘rawJson’ field. In R, it will be the result of calling the jsonlite fromJSON function.

Python:

def begin(model_def):
  print(model_def['rawJson'])
  print(model_def['job'])

R:

begin <- function(model_def) {
  print(model_def$rawJson)
  print(model_def$deployedModel)
}

To see an example of the above, go to https://github.com/merhi-odg/model_definition_test.

Adding Endpoints to the ModelOp Runtime

To add endpoints to the ModelOp Runtime:

1. Open the ModelOp Runtime to which you want to add endpoints

...

2. Select the “Create Endpoint” button for either the input or output endpoints

...

3. Click the button on the right to add either an Input endpoint or Output endpoint. Choose either REST or Kafka:

...

REST Endpoints

1. Add a Name, optional Description, Encoding type, and optional port. Select “Save Endpoint”

...

"modelop"
}

Model Definition Injection

Both the python and R runners support an optional parameter for the init function. Normally, the init function is defined with 0 arguments, but if a single argument is defined, the runner will populate that argument with a ‘model definition’ object. In python, this object is a dictionary. In R, this object is a list. In both cases, the object has two keys. There will be a ‘rawJson’ key that contains the unparsed json string that contains either the job or the deployedModel object in its serialized form. There will also be either a ‘job’ or a ‘deployedModel' key, the value of which will either be a deserialized job or deployedModel object. In python, the deserialized job or deployedModel will be the result of calling json.loads() on the contents of the ‘rawJson’ field. In R, it will be the result of calling the jsonlite fromJSON function.

Python:

def begin(model_def):
  print(model_def['rawJson'])
  print(model_def['job'])

R:

begin <- function(model_def) {
  print(model_def$rawJson)
  print(model_def$deployedModel)
}

To see an example of the above, go to https://github.com/merhi-odg/model_definition_test.

Adding Endpoints to the ModelOp Runtime

To add endpoints to the ModelOp Runtime:

1. Open the ModelOp Runtime to which you want to add endpoints

...

2. Select the “Create Endpoint” button for either the input or output endpoints

...

3. Click the button on the right to add either an Input endpoint or Output endpoint. Choose either REST or Kafka:

...

REST Endpoints

1. Add a Name, optional Description, Encoding type, and optional port. Select “Save Endpoint”

...

2. Repeat for the Output Endpoint, as required.

Kafka Endpoints

1. Kafka endpoints require several fields, as detailed in the screenshot and table below.

...

2. Repeat for the Output Endpoint, as required.

Kafka Endpoints

1. Kafka endpoints require several fields, as detailed in the screenshot and table below.

...

...

Field

...

Type

...

Description

...

Example

...

Name

...

string

...

A user-supplied name for the endpoint

...

“MyKafkaEndpoint1”

...

Description

...

string

...

A user-supplied description for the endpoint

...

“Kafka Endpoint for consumer transactions”

...

Encoding

...

drop-down

...

The encoding used for serialization/deserialization of the kafka messages

...

json

...

BootstrapServers

...

string

...

The name and port of the Kafka bootstrap servers.

...

"kafka1:9002"

...

Group (Consumer Group)

(optional)

...

string

...

To provide a horizontally scalable solution, a Consumer Group can be used to allow load balancing of requests across multiple ModelOp runtimes. By defining a Consumer Group, ModelOp Center ensures that only one ModelOp runtime in the Consumer Group services a given message.

...

"fastscore-1"

...

Principal (optional)

...

string

...

An authenticated user in a secure cluster

...

"kafka/kafka@REALM"

...

Topic

...

string

...

The Kafka topic to consume or push to, depending on if input vs. output endpoint

...

"MyKafkaTopic"

...

Keyfile (optional)

...

string

...

The location of the keytab file containing pairs of Kerberos principals and encrypted keys

...

"/fastscore.keytab"

...

Adding files to a runtime container for purposes of Kerberos or mTLS authentication

Files can be added to a k8s secret or configmap, and then the secret or configmap can be mounted to a specific location in the runtime container’s file system.

1. Create a secret or configmap that contains the necessary files.

   1. This can be done using kubectl: kubectl create configmap [configmap_name] --from-file [path/to/directory]

   2. This can be done via helm

      1. Place the files you want in your configmap inside a subdirectory of the helm chart: moc/my-files/some.file

      2. In the helm values/override file:

         Code Block
         language json
         moc: configMaps: my-configmap: my-files/*

2. Mount the secret or configmap on a runtime:

   Code Block
   language json
   moc: runtimes: "my-runtime": configMaps: my-configmap: /path/to/files/in/container

This will create a configmap called my-configmap that contains a file called some.file and will mount that file into "my-runtime" such that inside the runtime’s container there will exist a file at /path/to/files/in/container/some.file.

Adding Tags to a ModelOp Runtime

1. Open the ModelOp Runtime to which you want to add tag(s).

2. In the “tags” section begin typing the tag that you would like to add:

...

3. Once done typing, either hit enter on your keyboard or click your mouse in any other location on the screen.

...