Remove Pipelines - from DEV

[wg102]

This is to deprecated Kubeflow-Pipelines features.

It might include multiple steps.

• Remove the pipelines from the menu Remove pipeline card kubeflow#133
• Remove the deployment form the aaw-kubeflow-manifest
• Checkout if there are any resources left after (config maps, secrets, deployments, ertc.)
• Remove the link between pipeline repo and argocd see https://github.com/StatCan/aaw-argocd-manifests/blob/aaw-dev-cc-00/kubeflow/kubeflow.jsonnet
• Deal with pipelines-profiles-controller
• [x]Figure out a testing plan for dev

** Update **
For easy tracking, a new ticket was created for testing purposes.

• Test dev
• Execute the plan for prod
• Clean up the leftover resources (config maps, secrets, deployments , svc, service account, rolebindings)

[wg102]

Following the deletions, it was noticed that a few pods, and different types of resources are not deleted. I will try to manually delete them to see if they recreated. I am not sure why it happened, but it will most likely be similar on Prod.

[wg102]

This was not completed on time yesterday because of the network issues, it took minutes to query simple resources making the debugging virtually impossible.

[wg102]

There is one additional issue, in Dev when opening the dev tools I see the 2 following error message:

[wg102]

Needed to delete the services, that automatically deleted the secrets.
Rolebindings and serviceaccounts get recreated as soon as they are deleted.

[Souheil-Yazji]

Removed ml-pipelines rolebinding creation from aaw-kubeflow-profiles-controller. Redeployed controller to dev.

[wg102]

The role bindings delete after the update in charts/argocd-manifest

[wg102]

We have all the changes required to remove pipelines, but it will most likely still need manual deletion of resources on prod. It might be possible to create a script to run it instead.

[wg102]

I am missing changes on the Kubeflow repo, to remove the cards about the pipelines.

Once the changes are done, cleanup will be needed. As far as I currently see on prod:

• Configmaps: 11
• Secrets 134
• Svc: 251
• Serviceaccount: Access denied
• Deployments: 15 (or 7... depending how you read it)
• Rolebindings: Access denied

[wg102]

Converted the prod PRs to draft, while waiting for answers from the kubeflow folks. Also, we need to establish a testing plan for dev, to make sure there is nothing wrong with the way we removed the pipelines.

[StanHatko]

I documented this in the pull request for the AAW user documentation #1470, where I said pipelines are in the process of being removed.

[wg102]

This will need to be tested in the #1472