Files: Update to latest rabbitmq

[scblack321]

Task: Update to latest rabbitmq. Currently Files uses 3.8. Latest version currently is 3.11.
Make sure that rabbitmq is using a fixed data directory, e.g. /var/lib/rabbitmq/mnesia/rabbit@files-rabbitmq

History:

Files is currently setup to use to use rabbitmq 3.8.11-management.
Note that this configuration is in the deployment files in repo tapis-project/tapis-deployer under directlory templates/files.

Recently it was discovered that for files rabbitmq would create a new data directory each time it was re-deployed in kubernetes. This leads to the possibility that the pvc volume could fill up.
This is unlike the Jobs service, where the rabbitmq data directory is always the same. This turned out to be (at least in part) because Jobs sets the env variable HOSTNAME and Files did not. In kubernetes pods the HOSTNAME by default can change with each deployment.
Just before this issue was created the Files k8s template deploy rabbitmq files were updated to match those of Jobs, so Files is also using a fixed rabbitmq data dir.

Another discovery was that when moving to rabbitmq 3.11.7 setting the HOSTNAME in the environment was not enough to result in a fixed data dir. Explicitly setting certain rabbitmq env variables did work:

 RABBITMQ_NODENAME=tapis-files
 RABBITMQ_MNESIA_DIR=/var/lib/rabbitmq/mnesia/tapis-files
 RABBITMQ_PLUGINS_EXPAND_DIR=/var/lib/rabbitmq/mnesia/tapis-files-plugins-expand
 RABBITMQ_PID_FILE=/var/lib/rabbitmq/mnesia/[email protected]
 RABBITMQ_FEATURE_FLAGS_FILE=/var/lib/rabbitmq/mnesia/tapis-files@localhost-feature_flags

Note that although it is undesirable that these have hard coded absolute paths we already have this entry in the rabbitmq deployment files:

         volumeMounts:
            - name: files-rabbitmq-data
              mountPath: "/var/lib/rabbitmq/mnesia"