diff --git a/docs/csi_driver/using.md b/docs/csi_driver/using.md index 3f34b65a..07150239 100644 --- a/docs/csi_driver/using.md +++ b/docs/csi_driver/using.md @@ -834,6 +834,7 @@ These are some common issues and gotchas that are useful to know about when plan - Using the same network interface for NFS and block IO has shown suboptimal performance. Use FC for the block storage for the best performance. - A single NFS server instance is capable of 100GigE wirespeed with large sequential workloads and up to 200,000 IOPS with small IO using bare-metal nodes and multiple clients. - Using ext4 as the backing filesystem has shown better performance with simultaneous writers to the same file. +- Additional configuration and considerations may be required when using the NFS Server Provisioner with Red Hat OpenShift. See [NFS Server Provisioner Considerations](../partners/redhat_openshift/index.md#nfs_server_provisioner_considerations) for OpenShift. See [diagnosing NFS Server Provisioner issues](diagnostics.md#nfs_server_provisioner_resources) for further details. diff --git a/docs/partners/redhat_openshift/examples/scc/hpe-csi-scc.yaml b/docs/partners/redhat_openshift/examples/scc/hpe-csi-scc.yaml new file mode 100644 index 00000000..cf5d94cf --- /dev/null +++ b/docs/partners/redhat_openshift/examples/scc/hpe-csi-scc.yaml @@ -0,0 +1,103 @@ +--- +kind: SecurityContextConstraints +apiVersion: security.openshift.io/v1 +metadata: + name: hpe-csi-controller-scc +allowHostDirVolumePlugin: true +allowHostIPC: true +allowHostNetwork: true +allowHostPID: true +allowHostPorts: true +readOnlyRootFilesystem: true +requiredDropCapabilities: [] +runAsUser: + type: RunAsAny +seLinuxContext: + type: RunAsAny +users: +- system:serviceaccount:hpe-storage:hpe-csi-controller-sa +volumes: +- hostPath +- emptyDir +- projected +--- +kind: SecurityContextConstraints +apiVersion: security.openshift.io/v1 +metadata: + name: hpe-csi-node-scc +allowHostDirVolumePlugin: true +allowHostIPC: true +allowHostNetwork: true +allowHostPID: true +allowHostPorts: true +allowPrivilegeEscalation: true +allowPrivilegedContainer: true +allowedCapabilities: +- SYS_ADMIN +defaultAddCapabilities: [] +fsGroup: + type: RunAsAny +groups: [] +priority: +readOnlyRootFilesystem: false +requiredDropCapabilities: [] +runAsUser: + type: RunAsAny +seLinuxContext: + type: RunAsAny +supplementalGroups: + type: RunAsAny +users: +- system:serviceaccount:hpe-storage:hpe-csi-node-sa +volumes: +- emptyDir +- hostPath +- projected +- configMap +--- +kind: SecurityContextConstraints +apiVersion: security.openshift.io/v1 +metadata: + name: hpe-csi-csp-scc +allowHostDirVolumePlugin: true +readOnlyRootFilesystem: true +runAsUser: + type: RunAsAny +seLinuxContext: + type: RunAsAny +#supplementalGroups: +# type: RunAsAny +users: +- system:serviceaccount:hpe-storage:hpe-csp-sa +volumes: +- hostPath +- emptyDir +- projected +--- +kind: SecurityContextConstraints +apiVersion: security.openshift.io/v1 +metadata: + name: hpe-csi-nfs-scc +allowPrivilegedContainer: true +allowPrivilegeEscalation: true +allowedCapabilities: +- SYS_ADMIN +- DAC_READ_SEARCH +defaultAddCapabilities: [] +fsGroup: + type: RunAsAny +groups: [] +readOnlyRootFilesystem: false +requiredDropCapabilities: [] +runAsUser: + type: RunAsAny +seLinuxContext: + type: RunAsAny +supplementalGroups: + type: RunAsAny +users: +- system:serviceaccount:hpe-nfs:hpe-csi-nfs-sa +volumes: +- persistentVolumeClaim +- configMap +- projected diff --git a/docs/partners/redhat_openshift/img/redhat-certified.png b/docs/partners/redhat_openshift/img/redhat-certified.png index 2f3eed61..679ebe8c 100644 Binary files a/docs/partners/redhat_openshift/img/redhat-certified.png and b/docs/partners/redhat_openshift/img/redhat-certified.png differ diff --git a/docs/partners/redhat_openshift/img/webcon-1.png b/docs/partners/redhat_openshift/img/webcon-1.png index d29fe695..2f86e790 100644 Binary files a/docs/partners/redhat_openshift/img/webcon-1.png and b/docs/partners/redhat_openshift/img/webcon-1.png differ diff --git a/docs/partners/redhat_openshift/img/webcon-2.png b/docs/partners/redhat_openshift/img/webcon-2.png index 3329d22f..06ebc51e 100644 Binary files a/docs/partners/redhat_openshift/img/webcon-2.png and b/docs/partners/redhat_openshift/img/webcon-2.png differ diff --git a/docs/partners/redhat_openshift/img/webcon-3-1.png b/docs/partners/redhat_openshift/img/webcon-3-1.png index a6297dcf..e070ac67 100644 Binary files a/docs/partners/redhat_openshift/img/webcon-3-1.png and b/docs/partners/redhat_openshift/img/webcon-3-1.png differ diff --git a/docs/partners/redhat_openshift/img/webcon-3.png b/docs/partners/redhat_openshift/img/webcon-3.png index f71a7840..4cadc3ab 100644 Binary files a/docs/partners/redhat_openshift/img/webcon-3.png and b/docs/partners/redhat_openshift/img/webcon-3.png differ diff --git a/docs/partners/redhat_openshift/img/webcon-4.png b/docs/partners/redhat_openshift/img/webcon-4.png index 17de7ea4..0c519f21 100644 Binary files a/docs/partners/redhat_openshift/img/webcon-4.png and b/docs/partners/redhat_openshift/img/webcon-4.png differ diff --git a/docs/partners/redhat_openshift/img/webcon-5.png b/docs/partners/redhat_openshift/img/webcon-5.png index 2d7f9cd3..33d8cd22 100644 Binary files a/docs/partners/redhat_openshift/img/webcon-5.png and b/docs/partners/redhat_openshift/img/webcon-5.png differ diff --git a/docs/partners/redhat_openshift/img/webcon-6.png b/docs/partners/redhat_openshift/img/webcon-6.png index 131aa326..343e685b 100644 Binary files a/docs/partners/redhat_openshift/img/webcon-6.png and b/docs/partners/redhat_openshift/img/webcon-6.png differ diff --git a/docs/partners/redhat_openshift/img/webcon-7.png b/docs/partners/redhat_openshift/img/webcon-7.png index e9b5af4a..5150d394 100644 Binary files a/docs/partners/redhat_openshift/img/webcon-7.png and b/docs/partners/redhat_openshift/img/webcon-7.png differ diff --git a/docs/partners/redhat_openshift/index.md b/docs/partners/redhat_openshift/index.md index ba7a55e4..8e2cae48 100644 --- a/docs/partners/redhat_openshift/index.md +++ b/docs/partners/redhat_openshift/index.md @@ -1,6 +1,5 @@ - - # Overview + HPE and Red Hat have a long standing partnership to provide jointly supported software, platform and services with the absolute best customer experience in the industry. Red Hat OpenShift uses open source Kubernetes and various other components to deliver a PaaS experience that benefits both developers and operations. This packaged experience differs slightly on how you would deploy and use the HPE volume drivers and this page serves as the authoritative source for all things HPE primary storage and Red Hat OpenShift. @@ -54,6 +53,7 @@ Since the CSI Operator only provides "Basic Install" capabilities. The following - The `ConfigMap` "hpe-linux-config" that controls host configuration is immutable - The NFS Server Provisioner can not be used with Operators deploying `PersistentVolumeClaims` as part of the installation. See [#295](https://github.com/hpe-storage/csi-driver/issues/295) on GitHub. +- Deploying the NFS Server Provisioner to a `Namespace` other than "hpe-nfs" requires a separate SCC applied to the `Namespace`. See [#nfs_server_provisioner_considerations](NFS Server Provisioner Considerations). ### Deployment @@ -86,29 +86,23 @@ Once the steps have been followed for the particular version transition: The HPE CSI Driver needs to run in privileged mode and needs access to host ports, host network and should be able to mount hostPath volumes. Hence, before deploying HPE CSI Operator on OpenShift, please create the following `SecurityContextConstraints` (SCC) to allow the CSI driver to be running with these privileges. -Download the SCC to where you have access to `oc` and the OpenShift cluster: - ```text -curl -sL https://raw.githubusercontent.com/hpe-storage/co-deployments/master/operators/hpe-csi-operator/deploy/scc.yaml > hpe-csi-scc.yaml +oc new-project hpe-storage --display-name="HPE CSI Driver for Kubernetes" ``` -Change `my-hpe-csi-operator` to the name of the project (e.g. `hpe-csi-driver` below) where the CSI Operator is being deployed. - -```text -oc new-project hpe-csi-driver --display-name="HPE CSI Driver for Kubernetes" -sed -i'' -e 's/my-hpe-csi-driver-operator/hpe-csi-driver/g' hpe-csi-scc.yaml -``` +!!! important + The rest of this implementation guide assumes the default "hpe-storage" `Namespace`. If a different `Namespace` is desired. Update the `ServiceAccount` `Namespace` in the SCC below. -Deploy the SCC: +Deploy or [download]({{ config.site_url}}partners/redhat_openshift/examples/scc/hpe-csi-scc.yaml) the SCC: ```text -oc create -f hpe-csi-scc.yaml -securitycontextconstraints.security.openshift.io/hpe-csi-scc created +oc apply -f {{ config.site_url}}partners/redhat_openshift/examples/scc/hpe-csi-scc.yaml +securitycontextconstraints.security.openshift.io/hpe-csi-controller-scc created +securitycontextconstraints.security.openshift.io/hpe-csi-node-scc created +securitycontextconstraints.security.openshift.io/hpe-csi-csp-scc created +securitycontextconstraints.security.openshift.io/hpe-csi-nfs-scc created ``` -!!! important - Make note of the project name as it's needed for the Operator deployment in the next steps. - #### OpenShift web console Once the SCC has been applied to the project, login to the OpenShift web console as `kube:admin` and navigate to **Operators -> OperatorHub**. @@ -132,7 +126,7 @@ Once the SCC has been applied to the project, login to the OpenShift web console *Click 'Create Instance'.* ![Configure instance](img/webcon-6.png) -*Normally, no customizations are needed, click 'Create'.* +*Normally, no customizations are needed, scroll all the way down and click 'Create'.* By navigating to the Developer view, it should now be possible to inspect the CSI driver and Operator topology. @@ -146,7 +140,7 @@ See [Caveats](#caveats) below for information on creating `StorageClasses` in Re This provides an example Operator deployment using `oc`. If you want to use the web console, proceed to the [previous section](#openshift_web_console). -It's assumed the SCC has been applied to the project and have `kube:admin` privileges. As an example, we'll deploy to the `hpe-csi-driver` project as described in previous steps. +It's assumed the SCC has been applied to the project and have `kube:admin` privileges. As an example, we'll deploy to the `hpe-storage` project as described in previous steps. First, an `OperatorGroup` needs to be created. @@ -155,10 +149,10 @@ apiVersion: operators.coreos.com/v1 kind: OperatorGroup metadata: name: hpe-csi-driver-for-kubernetes - namespace: hpe-csi-driver + namespace: hpe-storage spec: targetNamespaces: - - hpe-csi-driver + - hpe-storage ``` Next, create a `Subscription` to the Operator. @@ -168,7 +162,7 @@ apiVersion: operators.coreos.com/v1alpha1 kind: Subscription metadata: name: hpe-csi-operator - namespace: hpe-csi-driver + namespace: hpe-storage spec: channel: stable installPlanApproval: Manual @@ -180,13 +174,13 @@ spec: Next, approve the installation. ```text -oc -n hpe-csi-driver patch $(oc get installplans -n hpe-csi-driver -o name) -p '{"spec":{"approved":true}}' --type merge +oc -n hpe-storage patch $(oc get installplans -n hpe-storage -o name) -p '{"spec":{"approved":true}}' --type merge ``` The Operator will now be installed on the OpenShift cluster. Before instantiating a CSI driver, watch the roll-out of the Operator. ```text -oc rollout status deploy/hpe-csi-driver-operator -n hpe-csi-driver +oc rollout status deploy/hpe-csi-driver-operator -n hpe-storage Waiting for deployment "hpe-csi-driver-operator" rollout to finish: 0 of 1 updated replicas are available... deployment "hpe-csi-driver-operator" successfully rolled out ``` @@ -235,6 +229,24 @@ v2.storage.hpe.com Please refer to the OLM Lifecycle Manager documentation on how to safely [Uninstall your operator](https://olm.operatorframework.io/docs/tasks/uninstall-operator/). +# NFS Server Provisioner Considerations + +When deploying NFS servers on OpenShift there's currently two things to keep in mind for a successful deployment. + +## Non-standard hpe-nfs Namespace + +If NFS servers are deployed in a different `Namespace` than the default "hpe-nfs" by using the "nfsNamespace" `StorageClass` parameter, the "hpe-csi-nfs-scc" SCC needs to be updated to include the `Namespace` `ServiceAccount`. + +This example adds "my-namespace" NFS server `ServiceAccount` to the SCC: + +```text +oc patch scc hpe-csi-nfs-scc --type=json -p='[{"op": "add", "path": "/users/-", "value": "system:serviceaccount:my-namespace:hpe-csi-nfs-sa" }]' +``` + +## Operators Requesting NFS Persistent Volume Claims + +Object references in OpenShift are not compatible with the NFS Server Provisioner. If a user deploys an Operator of any kind that creates a NFS server backed `PVC`, the operation will fail. Instead, pre-provision the `PVC` manually for the Operator instance to use. + # Unsupported Helm Chart Install In the event Red Hat releases a new release of OpenShift between HPE CSI driver releases or if interest arises to run the HPE CSI Driver on an uncertified version of OpenShift, it's possible to install the CSI driver using the Helm chart instead.