...

The engine is configured via two main mechanismsseveral mechanisms. The preferred mechanism is via spring configuration through spring cloud config server. There are also a number of (deprecated) environment variables that affect the behavior, as well as an application.json file that can be configured with credentials the engine may need when interacting with external services.

Environment Variables

moc-builder

When leveraging moc-builder to generate a MOC environment, your environment’s config/config.json file can be updated with the desired values for the aforementioned environment variables. A subsequent compose command will use the values in the config.json when building kubernetes manifests.

application.json

{
  "sql-credentials": [
    {
      "host": "sql-server-abc",
      "username": "my_user",
      "password": "my_password"
    },
    {
      "host": "sql-other-server-def",
      "username": {"from_file": "/config/the-username"},
      "password": {"from_env": "THE_ENV_VAR_WHERE_MY_PASSWORD_IS"}
    }
  ],
  "s3-credentials": {
    "storedCredentials": [
      {
        "path": "",
        "authenticationType": "BASIC",
        "accessKeyId" : "my_key",
        "secretAccessKey" : "my_secret"
      }
    ],
    "eksServiceAccountCredentials": [
      {
        "path": "s3.us-east-2.amazonaws.com",
        "roleArnEnvVariable" : "TEST_ROLE_ARN",
        "webIdentityTokenFileEnvVariable" : "TEST_TOKEN_FILE",
        "sessionPrefix" : "testprefixABC"
      }
    ]
  }
}

The ‘sql-credentials’ section allows username and password credentials to be associated with specific sql database hosts. Credentials can be hardcoded directly in the config file, located in a file, or located in an environment variable.

The ‘s3-credentials’ section allows credentials to be hardcoded directly (under storedCredentials) or determined dynamically using the AWS STS service. The credentials are chosen based on the longest matching path.

API’s

The following are a list of commonly used runtime APIs. The provided curl commands assume the runtime is running at localhost:8003. When communicating with a runtime through a MOC gateway, the localhost:8003 will likely look like this: gateway:8090/engine-name, where engine-name is the name the runtime used when it registered with the registry via the env variable: MODELOP_ENGINE_NAME.

Reset the engine

Reset the engine, causing the current model to be unloaded, and the various endpoints reset. This will also cause the current batch job to complete immediately, and move on to the next batch job.

curl -X POST http://localhost:8003/2/engine/reset

Load a deployed model

Load the deployed model contained in the ‘my-deployable-model.json’ file.

curl -X PUT --data-binary "@my-deployable-model.json" -H "Content-Type: application/json" http://localhost:8003/api/deployedModels/{UUID}

There is an optional wait_for_init query parameter for the /api/deployedModels/{UUID} endpoint. The default value of true means to not return from the REST call until the model’s init function has completed successfully. A value of false means to return immediately once the model and the specified endpoints have been loaded, but before the init function returns.

Load a batch job

Load a batch job into the runtime.

curl -X POST --data-binary "@my-batch-job.json" -H "Content-Type: application/json" http://localhost:8003/api/modelBatchJobs/{UUID}

Delete a batch job

Delete the batch job with the specified UUID.

curl -X DELETE http://localhost:8003/api/modelBatchJobs/{UUID}

List all batch jobs

Return all the batch jobs known to the runtime.

curl http://localhost:8003/api/modelBatchJobs/

List a specific batch job

Return the batch job with the specified UUID.

curl http://localhost:8003/api/modelBatchJobs/{UUID}

Score data via the REST endpoint

Submit the contents of input.json to the runtime. Note that in this example, the timeout has been set to 100 seconds, overriding the default set by the MODELOP_ROUNDTRIP_DEFAULT_TIMEOUT environment variable.

curl -X POST --data-binary "@input.json" -H "Content-Type: application/json" http://localhost:8003/api/roundtrip/0/1?timeout=100000

Get current engine status

Retrieve information about the currently running model, attached streams, and overall state of the engine.

curl http://localhost:8003/1/job/status

Sample response:

    - path: ""
      authenticationType: "BASIC"
      accessKeyId: "my_key"
      secretAccessKey: "my_secret"

    - path: "s3.us-east-2.amazonaws.com"
      roleArnEnvVariable: "TEST_ROLE_ARN"
      webIdentityTokenFileEnvVariable: "TEST_TOKEN_FILE"
      sessionPrefix: "testprefixABC"

  - host: "host1"
    username: "username1"
    password: "password1"

moc-builder

When leveraging moc-builder to generate a MOC environment, your environment’s config/config.json file can be updated with the desired values for the aforementioned environment variables. A subsequent compose command will use the values in the config.json when building kubernetes manifests.

application.json

{
  "

...

sql-credentials": [
    {
      "

...

host": 

...

"sql-server-abc",
      "

...

username": "

...

my_user",
      "

...

password": 

...

"my_password"
    },
 

...

...

  {
      "

...

host": 

...

"sql-other-server-def",
      

...

"username": {"from_file": 

...

"/config/the-username"},
      

...

"password": {"from_env": "THE_ENV_VAR_WHERE_MY_PASSWORD_IS"}
    }
  ],
  "

...

s3-credentials": 

...

{
    "storedCredentials": [
    

...

...

...

{
        "

...

path": "

...

"

...

,
        "

...

authenticationType": "

...

BASIC",
        "

...

accessKeyId" : "

...

my_key",
        "

...

secretAccessKey" : "

...

my_secret"

...

      }
 

...

...

...

...

],
    

...

"eksServiceAccountCredentials": [
   

...

   {
 

...

       "

...

path": 

...

"s3.us-east-2.amazonaws.com",
        "

...

roleArnEnvVariable" : "

...

TEST_ROLE_

...

ARN",
        "

...

webIdentityTokenFileEnvVariable" : 

...

"TEST_TOKEN_FILE",
        "

...

sessionPrefix" : 

...

"testprefixABC"
      }
    

...

Model Definition Injection

Both the python and R runners support an optional parameter for the init function. Normally, the init function is defined with 0 arguments, but if a single argument is defined, the runner will populate that argument with a ‘model definition’ object. In python, this object is a dictionary. In R, this object is a list. In both cases, the object has two keys. There will be a ‘rawJson’ key that contains the unparsed json string that contains either the job or the deployedModel object in its serialized form. There will also be either a ‘job’ or a ‘deployedModel' key, the value of which will either be a deserialized job or deployedModel object. In python, the deserialized job or deployedModel will be the result of calling json.loads() on the contents of the ‘rawJson’ field. In R, it will be the result of calling the jsonlite fromJSON function.

Python:

def begin(model_def):
  print(model_def['rawJson'])
  print(model_def['job'])

R:

begin <- function(model_def) {
  print(model_def$rawJson)
  print(model_def$deployedModel)
}

To see an example of the above, go to https://github.com/merhi-odg/model_definition_test.

Adding Endpoints to the ModelOp Runtime

To add endpoints to the ModelOp Runtime:

1. Open the ModelOp Runtime to which you want to add endpoints

2. Select the “Create Endpoint” button for either the input or output endpoints

...

3. Select to add an endpoint, either REST or Kafka

...

4. Add a Name, optional Description, Encoding type, and optional port. Select “Save Endpoint”

...

5

]
  }
}

The ‘sql-credentials’ section allows username and password credentials to be associated with specific sql database hosts. Credentials can be hardcoded directly in the config file, located in a file, or located in an environment variable.

The ‘s3-credentials’ section allows credentials to be hardcoded directly (under storedCredentials) or determined dynamically using the AWS STS service. The credentials are chosen based on the longest matching path.

API’s

The following are a list of commonly used runtime APIs. The provided curl commands assume the runtime is running at localhost:8003. When communicating with a runtime through a MOC gateway, the localhost:8003 will likely look like this: gateway:8090/engine-name, where engine-name is the name the runtime used when it registered with the registry via the env variable: MODELOP_ENGINE_NAME.

Reset the engine

Reset the engine, causing the current model to be unloaded, and the various endpoints reset. This will also cause the current batch job to complete immediately, and move on to the next batch job.

curl -X POST http://localhost:8003/2/engine/reset

Load a deployed model

Load the deployed model contained in the ‘my-deployable-model.json’ file.

curl -X PUT --data-binary "@my-deployable-model.json" -H "Content-Type: application/json" http://localhost:8003/api/deployedModels/{UUID}

There is an optional wait_for_init query parameter for the /api/deployedModels/{UUID} endpoint. The default value of true means to not return from the REST call until the model’s init function has completed successfully. A value of false means to return immediately once the model and the specified endpoints have been loaded, but before the init function returns.

Load a batch job

Load a batch job into the runtime.

curl -X POST --data-binary "@my-batch-job.json" -H "Content-Type: application/json" http://localhost:8003/api/modelBatchJobs/{UUID}

Delete a batch job

Delete the batch job with the specified UUID.

curl -X DELETE http://localhost:8003/api/modelBatchJobs/{UUID}

List all batch jobs

Return all the batch jobs known to the runtime.

curl http://localhost:8003/api/modelBatchJobs/

List a specific batch job

Return the batch job with the specified UUID.

curl http://localhost:8003/api/modelBatchJobs/{UUID}

Score data via the REST endpoint

Submit the contents of input.json to the runtime. Note that in this example, the timeout has been set to 100 seconds, overriding the default set by the MODELOP_ROUNDTRIP_DEFAULT_TIMEOUT environment variable.

curl -X POST --data-binary "@input.json" -H "Content-Type: application/json" http://localhost:8003/api/roundtrip/0/1?timeout=100000

Get current engine status

Retrieve information about the currently running model, attached streams, and overall state of the engine.

curl http://localhost:8003/1/job/status

Sample response:

{
  "streams": [
    {
      "slot": 0,
      "name": "rest",
      "eof": false,
      "descriptor": {
        "Transport": {
          "secured": false,
          "roundtrip": true,
          "port": 9003,
          "Type": "REST",
          "Mode": "simple"
        },
        "Schema": "$inherit",
        "Name": "rest",
        "Encoding": "JSON",
        "Description": "Application Restful Endpoint"
      }
    },
    {
      "slot": 1,
      "name": "rest_out",
      "eof": false,
      "descriptor": {
        "Transport": {
          "Type": "synthetic"
        },
        "Schema": "$inherit",
        "Encoding": "JSON"
      }
    }
  ],
  "state": "running",
  "model": {
    "source": "[model source code...]",
    "snapshots": "none",
    "slots": [
      {
        "slot": 0,
        "schema": null,
        "recordsets": false,
        "action": "action"
      },
      {
        "slot": 1,
        "schema": null,
        "recordsets": false,
        "action": null
      }
    ],
    "name": "test_model.py",
    "mtype": "python3",
    "attachments": []
  },
  "jets": [
    {
      "sandbox": "123472369",
      "pid": 55,
      "busy": false
    }
  ],
  "initialTags": [],
  "group": "modelop"
}

Model Definition Injection

Both the python and R runners support an optional parameter for the init function. Normally, the init function is defined with 0 arguments, but if a single argument is defined, the runner will populate that argument with a ‘model definition’ object. In python, this object is a dictionary. In R, this object is a list. In both cases, the object has two keys. There will be a ‘rawJson’ key that contains the unparsed json string that contains either the job or the deployedModel object in its serialized form. There will also be either a ‘job’ or a ‘deployedModel' key, the value of which will either be a deserialized job or deployedModel object. In python, the deserialized job or deployedModel will be the result of calling json.loads() on the contents of the ‘rawJson’ field. In R, it will be the result of calling the jsonlite fromJSON function.

Python:

def begin(model_def):
  print(model_def['rawJson'])
  print(model_def['job'])

R:

begin <- function(model_def) {
  print(model_def$rawJson)
  print(model_def$deployedModel)
}

To see an example of the above, go to https://github.com/merhi-odg/model_definition_test.

Adding Endpoints to the ModelOp Runtime

To add endpoints to the ModelOp Runtime:

1. Open the ModelOp Runtime to which you want to add endpoints

2. Select the “Create Endpoint” button for either the input or output endpoints

...

3. Select to add an endpoint, either REST or Kafka

...

REST Endpoints

1. Add a Name, optional Description, Encoding type, and optional port. Select “Save Endpoint”

...

2. Repeat for the Output Endpoint, as required.

Kafka Endpoints

1. Kafka endpoints require several fields, as detailed in the screenshot and table below.

...

2. Repeat for the Output Endpoint, as required.

...