Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update offline bundle instructions from CSI to CSM Operator #235

Merged
merged 4 commits into from
Dec 21, 2023
Merged

Update offline bundle instructions from CSI to CSM Operator #235

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
b198b03
Update offline bundle instructions from CSI to CSM Operator
KerryKovacevic Dec 14, 2023
557589b
Update Helm references for case consistency
KerryKovacevic Dec 14, 2023
af679a5
Update driver name
KerryKovacevic Dec 18, 2023
b2b0091
Update documentation
KerryKovacevic Dec 19, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion dell-csi-helm-installer/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ Installing any of the Dell EMC CSI Drivers requires a few utilities to be instal
| Dependency | Usage |
| ------------- | ----- |
| `kubectl` | Kubectl is used to validate that the Kubernetes system meets the requirements of the driver. |
| `helm` | Helm v3 is used as the deployment tool for Charts. See, [Install HELM 3](https://helm.sh/docs/intro/install/) for instructions to install HELM 3. |
| `helm` | Helm v3 is used as the deployment tool for Charts. See, [Install Helm 3](https://helm.sh/docs/intro/install/) for instructions to install Helm 3. |
| `sshpass` | sshpass is used to check certain pre-requisities in worker nodes (in chosen drivers). |


Expand Down
184 changes: 89 additions & 95 deletions dell-csi-helm-installer/csi-offline-bundle.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,7 @@

## Description

The `csi-offline-bundle.sh` script can be used to create a package usable for offline installation of the Dell EMC CSI Storage Providers, via either Helm
or the Dell CSI Operator.
The `csi-offline-bundle.sh` script can be used to create a package for the offline installation of Dell CSI storage providers for deployment via Helm.

This includes the following drivers:
* [PowerFlex](https://github.com/dell/csi-vxflexos)
Expand All @@ -12,8 +11,8 @@ This includes the following drivers:
* [PowerStore](https://github.com/dell/csi-powerstore)
* [Unity](https://github.com/dell/csi-unity)

As well as the Dell CSI Operator
* [Dell CSI Operator](https://github.com/dell/dell-csi-operator)
The `csm-offline-bundle.sh` script can be used to create a package for the offline installation of Dell CSI storage providers for deployment via the CSM Operator.
* [Dell CSM Operator](https://github.com/dell/csm-operator)

## Dependencies

Expand Down Expand Up @@ -46,84 +45,93 @@ To perform an offline installation of a driver or the Operator, the following st
This needs to be performed on a linux system with access to the internet as a git repo will need to be cloned, and container images pulled from public registries.

The build an offline bundle, the following steps are needed:
1. Perform a `git clone` of the desired repository. For a helm based install, the specific driver repo should be cloned. For an Operator based deployment, the Dell CSI Operator repo should be cloned
2. Run the `csi-offline-bundle.sh` script with an argument of `-c` in order to create an offline bundle
1. Perform a `git clone` of the desired repository. For a Helm based install, the specific driver repo should be cloned. For an Operator based deployment, the Dell CSM Operator repo should be cloned
2. Run the offline bundle script with an argument of `-c` in order to create an offline bundle
- For Helm installs, the `csi-offline-bundle.sh` script will be found in the `dell-csi-helm-installer` directory
- For Operator installs, the `csi-offline-bundle.sh` script will be found in the `scripts` directory
- For Operator installs, the `csm-offline-bundle.sh` script will be found in the `scripts` directory

The script will perform the following steps:
- Determine required images by parsing either the driver Helm charts (if run from a cloned CSI Driver git repository) or the Dell CSI Operator configuration files (if run from a clone of the Dell CSI Operator repository)
- Determine required images by parsing either the driver Helm charts (if run from a cloned CSI Driver git repository) or the Dell CSM Operator configuration files (if run from a clone of the Dell CSM Operator repository)
- Perform an image `pull` of each image required
- Save all required images to a file by running `docker save` or `podman save`
- Build a `tar.gz` file containing the images as well as files required to installer the driver and/or Operator

The resulting offline bundle file can be copied to another machine, if necessary, to gain access to the desired image registry.

For example, here is the output of a request to build an offline bundle for the Dell CSI Operator:
For example, here is the output of a request to build an offline bundle for the Dell CSM Operator:
```
[user@anothersystem /home/user]# git clone https://github.com/dell/dell-csi-operator.git
[user@anothersystem /home/user]# git clone https://github.com/dell/csm-operator.git

```
```
[user@anothersystem /home/user]# cd dell-csi-operator
[user@anothersystem /home/user]# cd csm-operator
```
```
[user@system /home/user/dell-csi-operator]# scripts/csi-offline-bundle.sh -c
[user@system /home/user/csm-operator]# bash scripts/csm-offline-bundle.sh -c

*
* Building image manifest file

Processing file /root/csm-operator/operatorconfig/driverconfig/common/default.yaml
Processing file /root/csm-operator/bundle/manifests/dell-csm-operator.clusterserviceversion.yaml

*
* Pulling container images

dellemc/csi-isilon:v1.2.0
dellemc/csi-isilon:v1.3.0.000R
dellemc/csipowermax-reverseproxy:v1.0.0.000R
dellemc/csi-powermax:v1.2.0.000R
dellemc/csi-powermax:v1.4.0.000R
dellemc/csi-powerstore:v1.1.0.000R
dellemc/csi-unity:v1.3.0.000R
dellemc/csi-vxflexos:v1.1.5.000R
dellemc/csi-vxflexos:v1.2.0.000R
dellemc/dell-csi-operator:v1.1.0.000R
quay.io/k8scsi/csi-attacher:v2.0.0
quay.io/k8scsi/csi-attacher:v2.2.0
quay.io/k8scsi/csi-node-driver-registrar:v1.2.0
quay.io/k8scsi/csi-provisioner:v1.4.0
quay.io/k8scsi/csi-provisioner:v1.6.0
quay.io/k8scsi/csi-resizer:v0.5.0
quay.io/k8scsi/csi-snapshotter:v2.1.1

*
* Saving images

* Pulling and saving container images

dellemc/csi-isilon:v2.8.0
dellemc/csi-metadata-retriever:v1.6.0
dellemc/csipowermax-reverseproxy:v2.6.0
dellemc/csi-powermax:v2.9.0
dellemc/csi-powerstore:v2.9.0
dellemc/csi-unity:v2.8.0
dellemc/csi-vxflexos:v2.9.0
dellemc/csm-authorization-sidecar:v1.9.0
dellemc/csm-metrics-powerflex:v1.5.0
dellemc/csm-metrics-powerscale:v1.2.0
dellemc/csm-topology:v1.5.0
dellemc/dell-csi-replicator:v1.7.0
dellemc/dell-replication-controller:v1.7.0
dellemc/sdc:4.5
docker.io/dellemc/dell-csm-operator:v1.3.0
gcr.io/kubebuilder/kube-rbac-proxy:v0.8.0
nginxinc/nginx-unprivileged:1.20
otel/opentelemetry-collector:0.42.0
registry.k8s.io/sig-storage/csi-attacher:v4.3.0
registry.k8s.io/sig-storage/csi-external-health-monitor-controller:v0.9.0
registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.8.0
registry.k8s.io/sig-storage/csi-provisioner:v3.5.0
registry.k8s.io/sig-storage/csi-resizer:v1.8.0
registry.k8s.io/sig-storage/csi-snapshotter:v6.2.2

*
* Copying necessary files

/dell/git/dell-csi-operator/config
/dell/git/dell-csi-operator/deploy
/dell/git/dell-csi-operator/samples
/dell/git/dell-csi-operator/scripts
/dell/git/dell-csi-operator/README.md
/dell/git/dell-csi-operator/LICENSE
/root/csm-operator/deploy
/root/csm-operator/operatorconfig
/root/csm-operator/samples
/root/csm-operator/scripts
/root/csm-operator/README.md
/root/csm-operator/LICENSE

*
* Compressing release

dell-csi-operator-bundle/
dell-csi-operator-bundle/samples/
dell-csm-operator-bundle/
dell-csm-operator-bundle/deploy/
dell-csm-operator-bundle/deploy/operator.yaml
dell-csm-operator-bundle/deploy/crds/
dell-csm-operator-bundle/deploy/crds/storage.dell.com_containerstoragemodules.yaml
dell-csm-operator-bundle/deploy/olm/
dell-csm-operator-bundle/deploy/olm/operator_community.yaml
...
<listing of files included in bundle>
...
dell-csi-operator-bundle/LICENSE
dell-csi-operator-bundle/README.md
dell-csm-operator-bundle/README.md
dell-csm-operator-bundle/LICENSE

*
* Complete

Offline bundle file is: /dell/git/dell-csi-operator/dell-csi-operator-bundle.tar.gz
Offline bundle file is: /root/csm-operator/dell-csm-operator-bundle.tar.gz

```

Expand All @@ -134,7 +142,7 @@ This needs to be performed on a linux system with access to an image registry th
To prepare for driver or Operator installation, the following steps need to be performed:
1. Copy the offline bundle file to a system with access to an image registry available to your Kubernetes/OpenShift cluster
2. Expand the bundle file by running `tar xvfz <filename>`
3. Run the `csi-offline-bundle.sh` script and supply the `-p` option as well as the path to the internal registry with the `-r` option
3. Run the `csm-offline-bundle.sh` script and supply the `-p` option as well as the path to the internal registry with the `-r` option

The script will then perform the following steps:
- Load the required container images into the local system
Expand All @@ -143,69 +151,57 @@ The script will then perform the following steps:
- Modify the Helm charts or Operator configuration to refer to the newly tagged/pushed images


An example of preparing the bundle for installation (192.168.75.40:5000 refers to a image registry accessible to Kubernetes/OpenShift):
An example of preparing the bundle for installation:
```
[user@anothersystem /tmp]# tar xvfz dell-csi-operator-bundle.tar.gz
dell-csi-operator-bundle/
dell-csi-operator-bundle/samples/
[user@anothersystem /tmp]# tar xvfz dell-csm-operator-bundle.tar.gz
dell-csm-operator-bundle/
dell-csm-operator-bundle/deploy/
dell-csm-operator-bundle/deploy/operator.yaml
dell-csm-operator-bundle/deploy/crds/
dell-csm-operator-bundle/deploy/crds/storage.dell.com_containerstoragemodules.yaml
dell-csm-operator-bundle/deploy/olm/
dell-csm-operator-bundle/deploy/olm/operator_community.yaml
...
<listing of files included in bundle>
...
dell-csi-operator-bundle/LICENSE
dell-csi-operator-bundle/README.md
dell-csm-operator-bundle/README.md
dell-csm-operator-bundle/LICENSE
```
```
[user@anothersystem /tmp]# cd dell-csi-operator-bundle
[user@anothersystem /tmp]# cd dell-csm-operator-bundle
```
```
[user@anothersystem /tmp/dell-csi-operator-bundle]# scripts/csi-offline-bundle.sh -p -r 192.168.75.40:5000/operator
[user@anothersystem /tmp/dell-csm-operator-bundle]# bash scripts/csm-offline-bundle.sh -p -r localregistry:5000/dell-csm-operator/
Preparing a offline bundle for installation

*
* Loading docker images

Loaded image: docker.io/dellemc/csi-powerstore:v2.9.0
Loaded image: docker.io/dellemc/csi-isilon:v2.9.0
...
...
Loaded image: registry.k8s.io/sig-storage/csi-resizer:v1.9.2
Loaded image: registry.k8s.io/sig-storage/csi-snapshotter:v6.3.2

*
* Tagging and pushing images

dellemc/csi-isilon:v1.2.0 -> 192.168.75.40:5000/operator/csi-isilon:v1.2.0
dellemc/csi-isilon:v1.3.0.000R -> 192.168.75.40:5000/operator/csi-isilon:v1.3.0.000R
dellemc/csipowermax-reverseproxy:v1.0.0.000R -> 192.168.75.40:5000/operator/csipowermax-reverseproxy:v1.0.0.000R
dellemc/csi-powermax:v1.2.0.000R -> 192.168.75.40:5000/operator/csi-powermax:v1.2.0.000R
dellemc/csi-powermax:v1.4.0.000R -> 192.168.75.40:5000/operator/csi-powermax:v1.4.0.000R
dellemc/csi-powerstore:v1.1.0.000R -> 192.168.75.40:5000/operator/csi-powerstore:v1.1.0.000R
dellemc/csi-unity:v1.3.0.000R -> 192.168.75.40:5000/operator/csi-unity:v1.3.0.000R
dellemc/csi-vxflexos:v1.1.5.000R -> 192.168.75.40:5000/operator/csi-vxflexos:v1.1.5.000R
dellemc/csi-vxflexos:v1.2.0.000R -> 192.168.75.40:5000/operator/csi-vxflexos:v1.2.0.000R
dellemc/dell-csi-operator:v1.1.0.000R -> 192.168.75.40:5000/operator/dell-csi-operator:v1.1.0.000R
quay.io/k8scsi/csi-attacher:v2.0.0 -> 192.168.75.40:5000/operator/csi-attacher:v2.0.0
quay.io/k8scsi/csi-attacher:v2.2.0 -> 192.168.75.40:5000/operator/csi-attacher:v2.2.0
quay.io/k8scsi/csi-node-driver-registrar:v1.2.0 -> 192.168.75.40:5000/operator/csi-node-driver-registrar:v1.2.0
quay.io/k8scsi/csi-provisioner:v1.4.0 -> 192.168.75.40:5000/operator/csi-provisioner:v1.4.0
quay.io/k8scsi/csi-provisioner:v1.6.0 -> 192.168.75.40:5000/operator/csi-provisioner:v1.6.0
quay.io/k8scsi/csi-resizer:v0.5.0 -> 192.168.75.40:5000/operator/csi-resizer:v0.5.0
quay.io/k8scsi/csi-snapshotter:v2.1.1 -> 192.168.75.40:5000/operator/csi-snapshotter:v2.1.1
dellemc/csi-isilon:v2.9.0 -> localregistry:5000/dell-csm-operator/csi-isilon:v2.9.0
dellemc/csi-metadata-retriever:v1.6.0 -> localregistry:5000/dell-csm-operator/csi-metadata-retriever:v1.6.0
...
...
registry.k8s.io/sig-storage/csi-resizer:v1.9.2 -> localregistry:5000/dell-csm-operator/csi-resizer:v1.9.2
registry.k8s.io/sig-storage/csi-snapshotter:v6.3.2 -> localregistry:5000/dell-csm-operator/csi-snapshotter:v6.3.2

*
* Preparing operator files within /tmp/dell-csi-operator-bundle

changing: dellemc/csi-isilon:v1.2.0 -> 192.168.75.40:5000/operator/csi-isilon:v1.2.0
changing: dellemc/csi-isilon:v1.3.0.000R -> 192.168.75.40:5000/operator/csi-isilon:v1.3.0.000R
changing: dellemc/csipowermax-reverseproxy:v1.0.0.000R -> 192.168.75.40:5000/operator/csipowermax-reverseproxy:v1.0.0.000R
changing: dellemc/csi-powermax:v1.2.0.000R -> 192.168.75.40:5000/operator/csi-powermax:v1.2.0.000R
changing: dellemc/csi-powermax:v1.4.0.000R -> 192.168.75.40:5000/operator/csi-powermax:v1.4.0.000R
changing: dellemc/csi-powerstore:v1.1.0.000R -> 192.168.75.40:5000/operator/csi-powerstore:v1.1.0.000R
changing: dellemc/csi-unity:v1.3.0.000R -> 192.168.75.40:5000/operator/csi-unity:v1.3.0.000R
changing: dellemc/csi-vxflexos:v1.1.5.000R -> 192.168.75.40:5000/operator/csi-vxflexos:v1.1.5.000R
changing: dellemc/csi-vxflexos:v1.2.0.000R -> 192.168.75.40:5000/operator/csi-vxflexos:v1.2.0.000R
changing: dellemc/dell-csi-operator:v1.1.0.000R -> 192.168.75.40:5000/operator/dell-csi-operator:v1.1.0.000R
changing: quay.io/k8scsi/csi-attacher:v2.0.0 -> 192.168.75.40:5000/operator/csi-attacher:v2.0.0
changing: quay.io/k8scsi/csi-attacher:v2.2.0 -> 192.168.75.40:5000/operator/csi-attacher:v2.2.0
changing: quay.io/k8scsi/csi-node-driver-registrar:v1.2.0 -> 192.168.75.40:5000/operator/csi-node-driver-registrar:v1.2.0
changing: quay.io/k8scsi/csi-provisioner:v1.4.0 -> 192.168.75.40:5000/operator/csi-provisioner:v1.4.0
changing: quay.io/k8scsi/csi-provisioner:v1.6.0 -> 192.168.75.40:5000/operator/csi-provisioner:v1.6.0
changing: quay.io/k8scsi/csi-resizer:v0.5.0 -> 192.168.75.40:5000/operator/csi-resizer:v0.5.0
changing: quay.io/k8scsi/csi-snapshotter:v2.1.1 -> 192.168.75.40:5000/operator/csi-snapshotter:v2.1.1
* Preparing files within /root/dell-csm-operator-bundle

changing: dellemc/csi-isilon:v2.9.0 -> localregistry:5000/dell-csm-operator/csi-isilon:v2.9.0
changing: dellemc/csi-metadata-retriever:v1.6.0 -> localregistry:5000/dell-csm-operator/csi-metadata-retriever:v1.6.0
...
...
changing: registry.k8s.io/sig-storage/csi-resizer:v1.9.2 -> localregistry:5000/dell-csm-operator/csi-resizer:v1.9.2
changing: registry.k8s.io/sig-storage/csi-snapshotter:v6.3.2 -> localregistry:5000/dell-csm-operator/csi-snapshotter:v6.3.2

*
* Complete
Expand All @@ -215,5 +211,3 @@ Preparing a offline bundle for installation
### Perform either a Helm installation or Operator installation

Now that the required images have been made available and the Helm Charts/Operator configuration updated, installation can proceed by following the instructions that are documented within the driver or Operator repo.


2 changes: 1 addition & 1 deletion dell-csi-helm-installer/csi-offline-bundle.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -315,7 +315,7 @@ if [ "${MODE}" == "helm" ]; then
"${REPODIR}/LICENSE"
)
else
DRIVERNAME="dell-csi-operator"
DRIVERNAME="dell-csm-operator"
DISTBASE="${REPODIR}"
DRIVERDIR="${DRIVERNAME}-bundle"
DISTDIR="${DISTBASE}/${DRIVERDIR}"
Expand Down
16 changes: 8 additions & 8 deletions test/helm/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Various test helm charts and test scripts
# Various test Helm charts and test scripts

## Helm charts
| Name | Usage |
Expand All @@ -14,18 +14,18 @@
| deletepvcs.sh | Script to delete all PVCS in a namespace
| get.volume.ids | Script to list the volume IDs for all PVS
| logit.sh | Script to print number of pods and pvcs in a namespace
| starttest.sh | Used to instantiate one of the helm charts above. Requires argument of helm chart
| stoptest.sh | Stops currently running helm chart and deletes all PVCS
| starttest.sh | Used to instantiate one of the Helm charts above. Requires argument of Helm chart
| stoptest.sh | Stops currently running Helm chart and deletes all PVCS
| snaptest.sh | Script to create volume and snapshot from volume

## Usage
The starttest.sh script is used to deploy helm charts that test the deployment of a simple pod
with various storage configurations. The stoptest.sh script will delete the helm chart and cleanup after the test.
The starttest.sh script is used to deploy Helm charts that test the deployment of a simple pod
with various storage configurations. The stoptest.sh script will delete the Helm chart and cleanup after the test.
Procedure
1. Navigate to the test/helm directory, which contains the starttest.sh and various helm charts.
1. Navigate to the test/helm directory, which contains the starttest.sh and various Helm charts.

2. Run the starttest.sh script with an argument of the specific helm chart to deploy and test. For example:
2. Run the starttest.sh script with an argument of the specific Helm chart to deploy and test. For example:
> ./starttest.sh -t 2vols
3. After the test has completed, run the stoptest.sh script to delete the helm chart and cleanup the volumes.
3. After the test has completed, run the stoptest.sh script to delete the Helm chart and cleanup the volumes.
> ./stoptest.sh -t 2vols

8 changes: 4 additions & 4 deletions test/scale_longevity/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ for CSI driver

## Scalability
The scaletest.sh script can be used to kick off scalability tests.
This will run one series of 3 helm deployments, causing a number of pods and replicas to be
This will run one series of 3 Helm deployments, causing a number of pods and replicas to be
created, each with a certain number of volumes. Once the specified number of replicas have
been launched, the deployments will be scaled down and removed.

Expand All @@ -26,11 +26,11 @@ An example of running the scalability tests for Isilon is:
```
./scaletest.sh -n test -r 33 -v 30
```
That test will deploy 3 helm charts, with 33 replicas of each pod and 30 volumes per pod (a total of 99 pods and 2970 volumes). Once the pods are running, the system will scale down and terminate the test.
That test will deploy 3 Helm charts, with 33 replicas of each pod and 30 volumes per pod (a total of 99 pods and 2970 volumes). Once the pods are running, the system will scale down and terminate the test.

## Longevity
The longevity.sh script can be used to kick off longevity tests.
This will run a series of 3 helm deployments over and over, causing a number of pods and replicas to be created and deleted until the test is stopped. To stop the test, create a file named 'stop' in the directory. Upon stopping, the deployments will be scaled down and removed.
This will run a series of 3 Helm deployments over and over, causing a number of pods and replicas to be created and deleted until the test is stopped. To stop the test, create a file named 'stop' in the directory. Upon stopping, the deployments will be scaled down and removed.

Log files will be created in the directory, titled `longevity-YYYYMMDD-HHMMSS.log`

Expand All @@ -49,6 +49,6 @@ An example of running the longevity tests for Isilon is:
```
./longevity.sh -n test -r 10 -v 10
```
That test will deploy 3 helm charts, with 10 replicas of each pod and 10 volumes per pod (a total of 30 pods and 300 volumes). Once the pods are running, the system will scale down to zero and repeat the process until stopped.
That test will deploy 3 Helm charts, with 10 replicas of each pod and 10 volumes per pod (a total of 30 pods and 300 volumes). Once the pods are running, the system will scale down to zero and repeat the process until stopped.

Note that values specified on the command line override values defined in the env file.
Loading