-
Notifications
You must be signed in to change notification settings - Fork 5
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #1280 from Zarquan/20240207-zrq-handover-notes
20240207 zrq handover notes
- Loading branch information
Showing
4 changed files
with
804 additions
and
2 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,4 @@ | ||
| #!/bin/sh | ||
| #!/bin/bash | ||
| # | ||
| # <meta:header> | ||
| # <meta:licence> | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,4 @@ | ||
| #!/bin/sh | ||
| #!/bin/bash | ||
| # | ||
| # <meta:header> | ||
| # <meta:licence> | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,311 @@ | ||
| # | ||
| # <meta:header> | ||
| # <meta:licence> | ||
| # Copyright (c) 2024, ROE (http://www.roe.ac.uk/) | ||
| # | ||
| # This information is free software: you can redistribute it and/or modify | ||
| # it under the terms of the GNU General Public License as published by | ||
| # the Free Software Foundation, either version 3 of the License, or | ||
| # (at your option) any later version. | ||
| # | ||
| # This information is distributed in the hope that it will be useful, | ||
| # but WITHOUT ANY WARRANTY; without even the implied warranty of | ||
| # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | ||
| # GNU General Public License for more details. | ||
| # | ||
| # You should have received a copy of the GNU General Public License | ||
| # along with this program. If not, see <http://www.gnu.org/licenses/>. | ||
| # </meta:licence> | ||
| # </meta:header> | ||
| # | ||
| #zrq-notes-time | ||
| #zrq-notes-indent | ||
| #zrq-notes-crypto | ||
| #zrq-notes-ansible | ||
| #zrq-notes-osformat | ||
| #zrq-notes-zeppelin | ||
| # | ||
|
|
||
| Target: | ||
|
|
||
| Reproduce the deployment process starting from Windows laptop, | ||
| using a DigitalOcean VM as the client. | ||
|
|
||
| Result: | ||
|
|
||
| Work in progress ... | ||
|
|
||
| # ----------------------------------------------------- | ||
| # | ||
|
|
||
| Login to Openstack Horizon at Cambridge (keystone auth) | ||
| https://arcus.openstack.hpc.cam.ac.uk/ | ||
|
|
||
| Generate application credentials for each cloud (red, green, blue and data). | ||
| Fill in the appropriate sections in the clouds.yaml file on your laptop. | ||
|
|
||
|
|
||
| # ----------------------------------------------------- | ||
| # Create a local env file that will configure paths. | ||
| # This makes things easier to refer to later on. | ||
| #[user@laptop] | ||
|
|
||
| edit "aglais.env" | ||
|
|
||
| #!/bin/bash | ||
| # Environment settings for gaia-dmp project. | ||
|
|
||
| AGLAIS_REPO='[email protected]:millingw/gaia-dmp.git' | ||
| AGLAIS_CODE="${HOME}/gaia-dmp" | ||
|
|
||
| PATH="${PATH}:${AGLAIS_CODE}/bin" | ||
| export PATH | ||
|
|
||
|
|
||
| # ----------------------------------------------------- | ||
| # | ||
|
|
||
| Login to DigitalOcean and create a VM to use as the client. | ||
| Add public ssh key to the VM. | ||
|
|
||
|
|
||
| # ----------------------------------------------------- | ||
| # Transfer our **private** ssh key, clouds.yaml and env file to the VM. | ||
| #[user@laptop] | ||
|
|
||
| scp .ssh/id_rsa [email protected]:.ssh/id_rsa | ||
|
|
||
| scp clouds.yaml [email protected]:clouds.yaml | ||
|
|
||
| scp aglais.env [email protected]:aglais.env | ||
|
|
||
|
|
||
| # ----------------------------------------------------- | ||
| # Login to the VM using ssh. | ||
| #[user@windows] | ||
|
|
||
| ssh [email protected] | ||
|
|
||
|
|
||
| # ----------------------------------------------------- | ||
| # Setup your ssh-agent. | ||
| #[user@client-vm] | ||
|
|
||
| # This is not needed if you are using a Linux client. | ||
| # Linux automatically runs a ssh agent that will respond to authentication requests. | ||
| # This doesn't work on Windows, so we need to start an agent on the client VM. | ||
|
|
||
| eval $(ssh-agent) | ||
|
|
||
| > .... | ||
| > .... | ||
|
|
||
|
|
||
| cat << EOF | ||
| SSH_AGENT_PID [${SSH_AGENT_PID}] | ||
| SSH_AUTH_SOCK [${SSH_AUTH_SOCK}] | ||
| EOF | ||
|
|
||
| > .... | ||
| > .... | ||
|
|
||
|
|
||
| # ----------------------------------------------------- | ||
| # Add your ssh key to the agent. | ||
| #[user@client-vm] | ||
|
|
||
| # This is not needed if you are using a Linux client. | ||
| # Linux automatically runs a ssh agent that will respond to authentication requests. | ||
| # This doesn't work on Windows, so we need to start an agent on the client VM. | ||
|
|
||
| ssh-add "${HOME}/.ssh/id_rsa" | ||
|
|
||
| > .... | ||
| > .... | ||
|
|
||
|
|
||
| ssh-add -l | ||
|
|
||
| > .... | ||
| > .... | ||
|
|
||
|
|
||
| # ----------------------------------------------------- | ||
| # Install client software | ||
| #[user@client-vm] | ||
|
|
||
| apt-get install -y podman | ||
|
|
||
| > .... | ||
| > .... | ||
|
|
||
|
|
||
| apt-get install -y git | ||
|
|
||
| > .... | ||
| > .... | ||
|
|
||
|
|
||
| # ----------------------------------------------------- | ||
| # Clone our main source code. | ||
| #[user@client-vm] | ||
|
|
||
| source "${HOME:?}/aglais.env" | ||
|
|
||
| git clone "${AGLAIS_REPO}" "${AGLAIS_CODE}" | ||
|
|
||
| > .... | ||
| > .... | ||
|
|
||
|
|
||
| # ----------------------------------------------------- | ||
| # Sanity check the current live host. | ||
| #[user@client-vm] | ||
|
|
||
| ssh [email protected] \ | ||
| ' | ||
| date | ||
| hostname | ||
| ' | ||
|
|
||
| > .... | ||
| > .... | ||
|
|
||
|
|
||
| # ----------------------------------------------------- | ||
| # Create a container to work with. | ||
| #[user@client-vm] | ||
|
|
||
| source "${HOME:?}/aglais.env" | ||
|
|
||
| ansi-client 'blue' | ||
|
|
||
| > .... | ||
| > .... | ||
|
|
||
|
|
||
| # ----------------------------------------------------- | ||
| # Deploy everything. | ||
| #[root@ansibler] 3h ago | ||
|
|
||
| source /deployments/hadoop-yarn/bin/deploy.sh | ||
|
|
||
| > .... | ||
| > .... | ||
|
|
||
|
|
||
| # | ||
| # Problem - old shares remaining in blue project. | ||
| # These shouldn't be there. | ||
| # | ||
| # My theory: | ||
| # A broken build process wasn't able connect to the data cloud to list the existing shares. | ||
| # As a result, it created new /home and /user shares for each user in the blue project. | ||
| # | ||
| # Now that they are there, they don't get deleted because the names start with 'data' rather than 'blue' | ||
| # and safety feature in the delete-all script filters on name startswith '${cloudname}'. | ||
| # | ||
|
|
||
| # | ||
| # Checked and double checked that the IDs in blue and data are not the same. | ||
| # Explicity typing the cloud name in each command because ${cloudname} could be wrong. | ||
| # | ||
|
|
||
|
|
||
| openstack --os-cloud iris-gaia-blue share list | ||
|
|
||
| > +--------------------------------------+---------------------------------+------+-------------+-----------+-----------+-----------------+------+-------------------+ | ||
| > | ID | Name | Size | Share Proto | Status | Is Public | Share Type Name | Host | Availability Zone | | ||
| > +--------------------------------------+---------------------------------+------+-------------+-----------+-----------+-----------------+------+-------------------+ | ||
| > | d5be916e-6079-4f16-8cb2-103b40ad597b | iris-gaia-data-home-AZijlstra | 1 | CEPHFS | available | False | ceph01_cephfs | | nova | | ||
| > | c5d48d1d-1118-4f29-b2ec-c14f2659cacd | iris-gaia-data-home-DCrake | 1 | CEPHFS | available | False | ceph01_cephfs | | nova | | ||
| > | 92b2eb56-2f49-4101-a6ed-639cddb19208 | iris-gaia-data-home-DMorris | 1 | CEPHFS | available | False | ceph01_cephfs | | nova | | ||
| > | 5e915548-36e1-4021-a5b9-a3b75e7e8933 | iris-gaia-data-home-MSemczuk | 1 | CEPHFS | available | False | ceph01_cephfs | | nova | | ||
| > | a1692ea3-b441-413c-b6c7-aea7b7d89776 | iris-gaia-data-home-NHambly | 1 | CEPHFS | available | False | ceph01_cephfs | | nova | | ||
| > | 2dbde819-9911-4e71-ae8a-87d2bbf6fc3a | iris-gaia-data-home-SGoughKelly | 1 | CEPHFS | available | False | ceph01_cephfs | | nova | | ||
| > | 50eb3326-078f-4a8b-b3aa-d2ec78382d6c | iris-gaia-data-home-SVoutsinas | 1 | CEPHFS | available | False | ceph01_cephfs | | nova | | ||
| > | 548be29b-c735-4fa2-9b33-19168247e599 | iris-gaia-data-user-DCrake | 10 | CEPHFS | available | False | ceph01_cephfs | | nova | | ||
| > | ed0183ad-f7a5-4203-90f3-95fddf0b236f | iris-gaia-data-user-DMorris | 10 | CEPHFS | available | False | ceph01_cephfs | | nova | | ||
| > | 2f83003d-d3ef-4ff0-b794-69f82585130c | iris-gaia-data-user-NHambly | 10 | CEPHFS | available | False | ceph01_cephfs | | nova | | ||
| > | 2ae18c67-bc56-4bf1-ba7e-05d55c2571c8 | iris-gaia-data-user-SVoutsinas | 10 | CEPHFS | available | False | ceph01_cephfs | | nova | | ||
| > +--------------------------------------+---------------------------------+------+-------------+-----------+-----------+-----------------+------+-------------------+ | ||
|
|
||
|
|
||
| openstack --os-cloud iris-gaia-data share list | ||
|
|
||
| +--------------------------------------+------------------------------------+-------+-------------+-----------+-----------+-----------------+------+-------------------+ | ||
| | ID | Name | Size | Share Proto | Status | Is Public | Share Type Name | Host | Availability Zone | | ||
| +--------------------------------------+------------------------------------+-------+-------------+-----------+-----------+-----------------+------+-------------------+ | ||
| | c3c83cf6-5897-4194-b150-a29e83022a13 | aglais-data-gaia-dr3-2048 | 4196 | CEPHFS | available | False | ceph01_cephfs | | nova | | ||
| | 89467d18-212e-4207-ba67-b3597892186d | aglais-data-gaia-dr3-2048-20221107 | 8192 | CEPHFS | available | False | ceph01_cephfs | | nova | | ||
| | 298ad303-9d81-4540-b4f0-d099ade46be2 | aglais-data-gaia-edr3-2048 | 1024 | CEPHFS | available | False | ceph01_cephfs | | nova | | ||
| | d07a403d-12aa-4b72-9a2e-9136d29721fb | aglais-data-panstarrs-ps1 | 300 | CEPHFS | available | False | ceph01_cephfs | | nova | | ||
| | 9faa8e39-ba47-474f-8abd-d6303fb9436e | aglais-data-twomass-allsky | 40 | CEPHFS | available | False | ceph01_cephfs | | nova | | ||
| | 417fb77f-5659-46e3-a074-7c1d7c18a0fe | aglais-data-wise-allwise | 350 | CEPHFS | available | False | ceph01_cephfs | | nova | | ||
| | ........ | ........ | ... | ...... | ......... | ..... | ............. | | .... | | ||
| | 55b28681-5efe-4c40-90a0-d3e408632fb7 | iris-gaia-data-home-DMorris | 1 | CEPHFS | available | False | ceph01_cephfs | | nova | | ||
| | ........ | ........ | ... | ...... | ......... | ..... | ............. | | .... | | ||
| | 38cc5c54-7bd5-4322-9fdd-95ae3a9212af | iris-gaia-data-home-NHambly | 1 | CEPHFS | available | False | ceph01_cephfs | | nova | | ||
| | ........ | ........ | ... | ...... | ......... | ..... | ............. | | .... | | ||
| | 7e095f97-7bc8-43c0-8971-45b8d732c3cf | iris-gaia-data-home-SVoutsinas | 1 | CEPHFS | available | False | ceph01_cephfs | | nova | | ||
| | ........ | ........ | ... | ...... | ......... | ..... | ............. | | .... | | ||
| | ........ | ........ | ... | ...... | ......... | ..... | ............. | | .... | | ||
| | ........ | ........ | ... | ...... | ......... | ..... | ............. | | .... | | ||
| +--------------------------------------+------------------------------------+-------+-------------+-----------+-----------+-----------------+------+-------------------+ | ||
|
|
||
| # | ||
| # The names are the same, but the ID values are different. | ||
| # Therefore, safe to delete the rouge ones from the blue project. | ||
| # | ||
|
|
||
| > .... | ||
| > | 92b2eb56-2f49-4101-a6ed-639cddb19208 | iris-gaia-data-home-DMorris | 1 | CEPHFS | available | False | ceph01_cephfs | | nova | | ||
| > .... | ||
| > | ed0183ad-f7a5-4203-90f3-95fddf0b236f | iris-gaia-data-user-DMorris | 10 | CEPHFS | available | False | ceph01_cephfs | | nova | | ||
| > .... | ||
|
|
||
|
|
||
| > .... | ||
| > | 55b28681-5efe-4c40-90a0-d3e408632fb7 | iris-gaia-data-home-DMorris | 1 | CEPHFS | available | False | ceph01_cephfs | | nova | | ||
| > .... | ||
| > | 493b34ad-cbec-42ca-9308-36bc09b79528 | iris-gaia-data-user-DMorris | 1025 | CEPHFS | available | False | ceph01_cephfs | | nova | | ||
| > .... | ||
|
|
||
| # | ||
| # Manually deleted them from the blue project using the Horizon GUI. | ||
| # | ||
|
|
||
| openstack --os-cloud iris-gaia-blue share list | ||
|
|
||
| > - | ||
|
|
||
|
|
||
| openstack --os-cloud iris-gaia-data share list | ||
|
|
||
| +--------------------------------------+------------------------------------+-------+-------------+-----------+-----------+-----------------+------+-------------------+ | ||
| | ID | Name | Size | Share Proto | Status | Is Public | Share Type Name | Host | Availability Zone | | ||
| +--------------------------------------+------------------------------------+-------+-------------+-----------+-----------+-----------------+------+-------------------+ | ||
| | c3c83cf6-5897-4194-b150-a29e83022a13 | aglais-data-gaia-dr3-2048 | 4196 | CEPHFS | available | False | ceph01_cephfs | | nova | | ||
| | 89467d18-212e-4207-ba67-b3597892186d | aglais-data-gaia-dr3-2048-20221107 | 8192 | CEPHFS | available | False | ceph01_cephfs | | nova | | ||
| | 298ad303-9d81-4540-b4f0-d099ade46be2 | aglais-data-gaia-edr3-2048 | 1024 | CEPHFS | available | False | ceph01_cephfs | | nova | | ||
| | d07a403d-12aa-4b72-9a2e-9136d29721fb | aglais-data-panstarrs-ps1 | 300 | CEPHFS | available | False | ceph01_cephfs | | nova | | ||
| | 9faa8e39-ba47-474f-8abd-d6303fb9436e | aglais-data-twomass-allsky | 40 | CEPHFS | available | False | ceph01_cephfs | | nova | | ||
| | 417fb77f-5659-46e3-a074-7c1d7c18a0fe | aglais-data-wise-allwise | 350 | CEPHFS | available | False | ceph01_cephfs | | nova | | ||
| | ........ | ........ | ... | ...... | ......... | ..... | ............. | | .... | | ||
| | 55b28681-5efe-4c40-90a0-d3e408632fb7 | iris-gaia-data-home-DMorris | 1 | CEPHFS | available | False | ceph01_cephfs | | nova | | ||
| | ........ | ........ | ... | ...... | ......... | ..... | ............. | | .... | | ||
| | 38cc5c54-7bd5-4322-9fdd-95ae3a9212af | iris-gaia-data-home-NHambly | 1 | CEPHFS | available | False | ceph01_cephfs | | nova | | ||
| | ........ | ........ | ... | ...... | ......... | ..... | ............. | | .... | | ||
| | 7e095f97-7bc8-43c0-8971-45b8d732c3cf | iris-gaia-data-home-SVoutsinas | 1 | CEPHFS | available | False | ceph01_cephfs | | nova | | ||
| | ........ | ........ | ... | ...... | ......... | ..... | ............. | | .... | | ||
| | ........ | ........ | ... | ...... | ......... | ..... | ............. | | .... | | ||
| | ........ | ........ | ... | ...... | ......... | ..... | ............. | | .... | | ||
| +--------------------------------------+------------------------------------+-------+-------------+-----------+-----------+-----------------+------+-------------------+ | ||
|
|
||
| # | ||
| # OK, so delete-all doesn't actually delete-all, it has some safety features | ||
| # .. like filtering on object name. | ||
| # | ||
|
|
||
|
|
||
|
|
Oops, something went wrong.