Upgrading moc from 2.5 → 3.1.4

Used moc-builder (version 4) to provision and deploy 2.5
Steps taken:
- in the moc-builder folder executed
python3 src/moc_builder.py env create --fresh oauth-2.5
cp resources/external_pf.crt environments/oauth-2.5/certs
- Copied the file config-2.5.yaml to environments/oauth-2.5/ renaming to config.yaml and overwriting the existing file
- Executed the command to compose:
python3 src/moc_builder.py manifest compose oauth-2.4 docker-desktop --oauth --services jenkins jira jupyter rstudio --data minio mongodb mysql postgres --log-level DEBUG --ootb latest
- Finally did an install with:
python3 src/moc_builder.py install deploy oauth-2.5 -f --log-level DEBUG
- Added the following indexes to mongoDB:
db.job.createIndex( { createdDate: 1 } )
db.job.createIndex( { jobStatus: 1 } )
db.job.createIndex( { jobType: 1 } )
db.job.createIndex( { "model.id": 1 } )
db.job.createIndex( { "referenceModel.id": 1 } )
db.notification.createIndex( { createdDate: 1 } )
db.modelTestResult.createIndex( { "deployedModel.deployableModel.id" : 1 } )
db.modelTestResult.createIndex( { "job.id" : 1 } )
db.modelTestResult.createIndex( { createdDate : 1 } )
db.deployableModel.createIndex( { "storedModel.id": 1 } )
db.deployableModel.createIndex( { createdDate: 1 } )
db.storedModels.createIndex( { createdDate: 1 } )
- Ran some integration tests logged in and tested to make sure things were ok.
Deleted all deployments except for MongoDB and Postgres
kubectl delete deployments engine-1 engine-2 engine-test gateway jenkins jira jupyter minio mlc-manager model-manager mysql registry reporting-service rstudio spark-runtime-1
Upgrade to 3.1.4
Backup
Database Backup
Before you begin the upgrade process you will need to first backup your current Mongo and PostgreSQL databases. Please see instructions here on how to backup your databases:
How To Backup/Restore ModelOp Center's Databases
K8 Resources backup
Back up your current environment be executing:
kubectl get deployments,services,configmaps,secrets -l app=modelop-center -o yaml > moc-config-backup.yaml
Shut down MOC
Prior to version 3.0 ModelOp Center was deployed using MOC builder. From 3.0 forward ModelOp center is now deployed using our helm charts.
As a result, before you can upgrade one must first shutdown the current running instance of ModelOp Center running in your cluster as well as delete all K8 resources e.t.c originally provisioned using MOC builder.
To shutdown moc execute the following:
Delete deployments
kubectl delete deployments engine-1 engine-2 engine-3 engine-4 engine-5 engine-6 engine-7 engine-8 gateway jenkins jira jupyter minio mlc-manager registry reporting-service rstudio spark-runtime-1 model-manager
Delete configMaps
kubectl delete configMap ootb
Delete Services
kubectl delete services engine-1 engine-2 engine-3 engine-4 engine-5 engine-6 engine-7 engine-8 gateway jenkins jira jupyter minio mlc-manager registry reporting-service rstudio spark-runtime-1 model-manager
Execute DB Migration script
When upgrading MOC from 2.5 -> 3.1.4 an update of Camunda from 7.13 to 7.17 is required
To Migrate Camunda from 7.13 to 7.17, we would need to execute the available database patch script on your postgresql db: postgres-camunda-schema-upgrade.sql
In addition you would also need to execute the following DB script on your mongoDB database:
db.deployableModel.aggregate(
     [
         { $project: { "mlc" : { $map: { input: "$modelMLCS", as: "mlc", in: {$mergeObjects: "$$mlc" } } }, "deployableModelId": "$_id"} },
         { $match: { "mlc.0": { $exists: true } } },
         { $unwind: "$mlc" },
         { $project: { "_id": 0, "processInstance": "$mlc.processInstance", "activityInstances": "$mlc.activityInstances", "deployableModelId" : "$deployableModelId", "historyType" : "DEPLOYABLE_MODEL_HISTORY"} },
         { $out: "tempDeployableModelMLC" }
     ])

db.jobHistory.aggregate(
     [
         { $project: { "mlc" : { $map: { input: "$jobMLCS", as: "mlc", in: {$mergeObjects: "$$mlc" } } }, "jobId": "$jobId"} },
         { $match: { "mlc.0": { $exists: true } } },
         { $unwind: "$mlc" },
         { $project: { "_id": 0, "processInstance": "$mlc.processInstance", "activityInstances": "$mlc.activityInstances", "jobId" : "$jobId", "historyType" : "JOB_HISTORY"} },
         { $out: "tempJobMLC" }
     ])

db.deployedModel.aggregate(
     [
         { $match: { "modelLifecycleProcessInstance": { $exists: true }}},
         { $project: { "_id": 0, "processInstance": "$modelLifecycleProcessInstance", "activityInstances": "$modelLifecycleActivityInstances", "deployedModelId": "$_id", "historyType": "DEPLOYED_MODEL_HISTORY"}},
         { $out: "tempDeployedModelMLC"}
     ]
)

db.tempDeployableModelMLC.find().forEach(function (mlc) {
     db.modelMLC.insert(mlc)
})
db.tempDeployedModelMLC.find().forEach(function (mlc) {
     db.modelMLC.insert(mlc)
})
db.tempJobMLC.find().forEach(function (mlc) {
     db.modelMLC.insert(mlc)
})
Install Modelop Center using helm
Next step would be to install helm. See the following link on steps to install ModelOp Center using helm:
Installing ModelOp Center Using Helm
Update helm overrides file
When installing with helm you have the ability to configure modelOp center. Put in the appropriate configuration settings related to your environment as well as specifying the version to install 3.1.4
MOC Validation
Once MOC comes up login and validate things work as expected.

Upgrading moc from 2.4.1-rc-11 → 2.5

Used moc-builder (version 4) to provision and deploy 2.4
Steps taken:
- in the moc-builder folder executed
python3 src/moc_builder.py env create --fresh oauth-2.4
cp resources/external_pf.crt environments/oauth-2.4/certs
- Copied the file config-2.4.yaml to environments/oauth-2.4/ renaming to config.yaml and overwriting the existing file
- Executed the command to compose:
python3 src/moc_builder.py manifest compose oauth-2.4 docker-desktop --oauth --services jenkins jira jupyter rstudio --data minio mongodb mysql postgres --log-level DEBUG --ootb latest
- Finally did an install with:
python3 src/moc_builder.py install deploy oauth-2.4 -f --log-level DEBUG
- Added the following indexes to mongoDB:
db.job.createIndex( { createdDate: 1 } )
db.job.createIndex( { jobStatus: 1 } )
db.job.createIndex( { jobType: 1 } )
db.job.createIndex( { "model.id": 1 } )
db.job.createIndex( { "referenceModel.id": 1 } )
db.notification.createIndex( { createdDate: 1 } )
db.modelTestResult.createIndex( { "deployedModel.deployableModel.id" : 1 } )
db.modelTestResult.createIndex( { "job.id" : 1 } )
db.modelTestResult.createIndex( { createdDate : 1 } )
db.deployableModel.createIndex( { "storedModel.id": 1 } )
db.deployableModel.createIndex( { createdDate: 1 } )
db.storedModels.createIndex( { createdDate: 1 } )
- Ran some integration tests logged in and tested to make sure things were ok.
Deleted all deployments except for MongoDB and Postgres
kubectl delete deployments engine-1 engine-2 engine-test gateway jenkins jira jupyter minio mlc-manager model-manager mysql registry reporting-service rstudio spark-runtime-1
Upgrade to 2.5
Executed the command:
python3 src/moc_builder.py env create --fresh oauth-2.5
cp resources/external_pf.crt environments/oauth-2.5/certs
- Copied the file config-2.5.yaml to environments/oauth-2.5/ renaming to config.yaml and overwriting the existing file
- executed the command:
python3 src/moc_builder.py manifest compose oauth-2.5 docker-desktop --oauth --services jenkins jira jupyter rstudio --data minio mongodb mysql postgres --log-level DEBUG --ootb latest
* Since we want to keep the mongoDB we need to modify the mongodb.yaml file in environments/oauth-2.5/manifests/ and override the existing file that was created.
Copy mongodb.yaml environments/oauth-2.5/manifests/
- Finally install with:
python3 src/moc_builder.py install deploy oauth-2.5 -f --log-level DEBUG
- Added the following indexes to mongoDB:
db.deployableModel.createIndex( { createdDate: -1, group : -1 }, { name: "createdDate_-1_group_-1" } )
db.deployableModel.createIndex( { "modelMLCS.processInstance.rootProcessInstanceId": 1, group : 1 }, { name: "modelMLCS.processInstance.rootProcessInstanceId_1_group_1" } )
db.deployableModel.createIndex( { "associatedModels.schedules.active": 1, "lastModifiedDate" : 1, group : 1 }, { name: "associatedModels.schedules.active_1_lastModifiedDate_1_group_1" } )

db.deployedModel.createIndex( { createdDate: -1, group : -1 }, { name: "createdDate_-1_group_-1" } )
db.deployedModel.createIndex( { "engine.name": 1, "state" : 1, group : 1 }, { name: "engine.name_1_state_1_group_1" } )
db.deployedModel.createIndex( { group : 1 }, { name: "group_1" } )
db.deployedModel.createIndex( { deployTime: -1, group : -1 }, { name: "deployTime_-1_group_-1" } )

db.job.createIndex( { "model.id" : 1, group : 1 }, { name: "modelId_and_group" } )
db.job.createIndex( { "referenceModel.id" : 1, group : 1 }, { name: "referenceModel_and_group" } )
db.job.createIndex( { createdDate : -1, group : 1 }, { name: "createdDate_and_group" } )
db.job.createIndex( { jobStatus : 1, group : 1 , jobType : 1,deployableModelId : 1 }, { name: "status_group_type_deployableModel" } )

db.jobHistory.createIndex( { jobId : 1, group : 1 }, { name: "jobId_and_group" } )
db.jobHistory.createIndex( { "jobMLCS.processInstance.rootProcessInstanceId" : 1, group : 1 }, { name: "rootProcId_and_group" } )
db.jobHistory.createIndex( { group : 1 }, { name: "group" } )
Login and you should now be running 2.5.
- Performed some tests.

ModelOp Documentation Hub V3.1

Upgrading from MOC 2.5 to Helm-Based Installs

Upgrading moc from 2.5 → 3.1.4

Upgrading moc from 2.4.1-rc-11 → 2.5