From 6179730cda4e353ffec5ba2f4c35c218223a184c Mon Sep 17 00:00:00 2001 From: Andreas Schrogl Date: Tue, 23 Apr 2024 16:57:29 +0200 Subject: [PATCH 1/3] updates flavor-manager.md and added update.md Signed-off-by: Andreas Schrogl --- .../index.md} | 32 +++++++------- .../day2-operations/image-manager/update.md | 44 +++++++++++++++++++ 2 files changed, 61 insertions(+), 15 deletions(-) rename docs/guides/operations-guide/openstack/day2-operations/{image-manager.md => image-manager/index.md} (92%) create mode 100644 docs/guides/operations-guide/openstack/day2-operations/image-manager/update.md diff --git a/docs/guides/operations-guide/openstack/day2-operations/image-manager.md b/docs/guides/operations-guide/openstack/day2-operations/image-manager/index.md similarity index 92% rename from docs/guides/operations-guide/openstack/day2-operations/image-manager.md rename to docs/guides/operations-guide/openstack/day2-operations/image-manager/index.md index 845f056633..c277d76fc0 100644 --- a/docs/guides/operations-guide/openstack/day2-operations/image-manager.md +++ b/docs/guides/operations-guide/openstack/day2-operations/image-manager/index.md @@ -126,10 +126,11 @@ the help of the OpenStack Image Manager. provided_until: none tags: [] versions: - - version: '0.6.0' - url: https://github.com/cirros-dev/cirros/releases/download/0.6.0/cirros-0.6.0-x86_64-disk.img - checksum: "sha256:94e1e2c94dbbae7d4bdc38e68590a1daf73c9de2d03dd693857b4b0a042548e8" - build_date: 2022-09-28 + - version: '0.6.2' + url: https://github.com/cirros-dev/cirros/releases/download/0.6.2/cirros-0.6.2-x86_64-disk.img + checksum: "sha256:07e44a73e54c94d988028515403c1ed762055e01b83a767edf3c2b387f78ce00" + build_date: 2023-05-30 + ``` 3. Run the OpenStack Image Manager. It is assumed that a profile with the name `openstack` exists in the @@ -169,7 +170,7 @@ in parentheses upon each rotation (except for the latest entry). ```yaml images: - - name: Ubuntu 16.04 + - name: Ubuntu 24.04 format: qcow2 login: ubuntu min_disk: 8 @@ -183,25 +184,26 @@ images: hw_scsi_model: virtio-scsi hw_watchdog_action: reset os_distro: ubuntu - os_version: '16.04' + os_version: '24.04' tags: [] versions: - - version: '20180928' - url: https://cloud-images.ubuntu.com/xenial/20180928/xenial-server-cloudimg-amd64-disk1.img - - version: '20181004' - url: https://cloud-images.ubuntu.com/xenial/20181004/xenial-server-cloudimg-amd64-disk1.img + - version: '20240416' + url: https://cloud-images.ubuntu.com/noble/20240416/noble-server-cloudimg-amd64.img + - version: '20240422' + url: https://cloud-images.ubuntu.com/noble/20240422/noble-server-cloudimg-amd64.img ``` + This configuration creates the following images: -* **Ubuntu 16.04 (20180928)** -* **Ubuntu 16.04** +* **Ubuntu 24.04 (20240416)** +* **Ubuntu 24.04** If a newer build is added, the following rotation takes place: -* **Ubuntu 16.04 (20180928)** does not change -* **Ubuntu 16.04** becomes **Ubuntu 16.04 (20181004)** -* the new image becomes **Ubuntu 16.04** +* **Ubuntu 24.04 (20240416)** does not change +* **Ubuntu 24.04** becomes **Ubuntu 24.04 (20240422)** +* the new image becomes **Ubuntu 24.04** By default the last three images will be visible. When a fourth image is added, the visibility of the last image in the list is changed to `community` and the image can be deleted in the future. diff --git a/docs/guides/operations-guide/openstack/day2-operations/image-manager/update.md b/docs/guides/operations-guide/openstack/day2-operations/image-manager/update.md new file mode 100644 index 0000000000..7d4edc7c04 --- /dev/null +++ b/docs/guides/operations-guide/openstack/day2-operations/image-manager/update.md @@ -0,0 +1,44 @@ +--- +sidebar_label: Automated updates +sidebar_position: 1 +--- + +# Image Manager update.py + +## Overview + +The OpenStack Image Manager update.py Script updates the `/etc/images/*.yaml` Files to the always latest release. + + +## Installation + +Prepare to use the `update.py` script. + +``` +git clone https://github.com/osism/openstack-image-manager/ +cd openstack-image-manager +pipenv install +pipenv shell +``` + +## Usage + +``` +python update.py --help + + Usage: update.py [OPTIONS] + +╭─ Options ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮ +│ --debug Enable debug logging │ +│ --dry-run Do not perform any changes │ +│ --minio-access-key TEXT Minio access key [env var: MINIO_ACCESS_KEY] [default: None] │ +│ --minio-secret-key TEXT Minio secret key [env var: MINIO_SECRET_KEY] [default: None] │ +│ --minio-server TEXT Minio server [env var: MINIO_SERVER] [default: swift.services.a.regiocloud.tech] │ +│ --minio-bucket TEXT Minio bucket [env var: MINIO_BUCKET] [default: openstack-images] │ +│ --swift-prefix TEXT Swift prefix [env var: SWIFT_PREFIX] [default: swift/v1/AUTH_b182637428444b9aa302bb8d5a5a418c/] │ +│ --install-completion Install completion for the current shell. │ +│ --show-completion Show completion for the current shell, to copy it or customize the installation. │ +│ --help Show this message and exit. │ +╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ +``` + From 210274e1d1625b0a35d789ea81461b2617c994c3 Mon Sep 17 00:00:00 2001 From: Andreas Schrogl Date: Wed, 24 Apr 2024 11:21:24 +0200 Subject: [PATCH 2/3] update.md - add usage and output, example yaml file Signed-off-by: Andreas Schrogl --- .../day2-operations/image-manager/update.md | 116 +++++++++++++++++- 1 file changed, 114 insertions(+), 2 deletions(-) diff --git a/docs/guides/operations-guide/openstack/day2-operations/image-manager/update.md b/docs/guides/operations-guide/openstack/day2-operations/image-manager/update.md index 7d4edc7c04..78f48c0cfe 100644 --- a/docs/guides/operations-guide/openstack/day2-operations/image-manager/update.md +++ b/docs/guides/operations-guide/openstack/day2-operations/image-manager/update.md @@ -7,7 +7,9 @@ sidebar_position: 1 ## Overview -The OpenStack Image Manager update.py Script updates the `/etc/images/*.yaml` Files to the always latest release. +The OpenStack Image Manager `update.py` Script updates the `/etc/images/*.yaml` files to the always latest release of the Distributions, set S3 Mirror Urls and uploads the Images to the mirror. + +These updated yaml files are later processed by the Image Manger itself. ## Installation @@ -24,7 +26,7 @@ pipenv shell ## Usage ``` -python update.py --help +python contrib/update.py --help Usage: update.py [OPTIONS] @@ -42,3 +44,113 @@ python update.py --help ╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ ``` +:::note + +* At this time the update.py expects all yamls Files at /etc/images/, which can't be configured at the moment. +* Mirroring can't be disabled at the moment. + +::: + + +Best is to run this Script by cron, to update all Distribution Files periodically to the latest release and afterwards run Openstack Image Manager. +The Distribution Images yaml files must exists before running the script, you can use the files from Github repo under `etc/images/` as template for your first run. + + +``` +$ python contrib/update.py +2024-04-24 09:29:44 | INFO | main:300 - Processing file /etc/images/centos.yml +2024-04-24 09:29:44 | INFO | update_image:179 - Checking image CentOS Stream 9 +2024-04-24 09:29:44 | INFO | update_image:182 - Latest download URL is https://cloud.centos.org/centos/9-stream/x86_64/images/CentOS-Stream-GenericCloud-9-HEREBE\d+\.\dDRAGONS.x86_64.qcow2 +2024-04-24 09:29:44 | INFO | update_image:185 - Getting checksums from https://cloud.centos.org/centos/9-stream/x86_64/images/CHECKSUM +2024-04-24 09:29:44 | INFO | get_latest_default:62 - Latest URL is now https://cloud.centos.org/centos/9-stream/x86_64/images/CentOS-Stream-GenericCloud-9-20240422.0.x86_64.qcow2 +2024-04-24 09:29:44 | INFO | get_latest_default:63 - Latest filename is now CentOS-Stream-GenericCloud-9-20240422.0.x86_64.qcow2 +2024-04-24 09:29:44 | INFO | update_image:192 - Checksum of current CentOS-Stream-GenericCloud-9-20240422.0.x86_64.qcow2 is sha256:47dd9ad7048afe96bc6cc0b3fd8922f290e99c29d251affcd22d0afecfe0e337 +2024-04-24 09:29:44 | INFO | update_image:208 - Our checksum is sha256:47dd9ad7048afe96bc6cc0b3fd8922f290e99c29d251affcd22d0afecfe0e337 +2024-04-24 09:29:44 | INFO | update_image:211 - Image CentOS Stream 9 is up-to-date, nothing to do +2024-04-24 09:29:44 | INFO | main:300 - Processing file /etc/images/debian.yml +2024-04-24 09:29:44 | INFO | update_image:179 - Checking image Debian 11 +2024-04-24 09:29:44 | INFO | update_image:182 - Latest download URL is https://cdimage.debian.org/cdimage/cloud/bullseye/latest/debian-11-genericcloud-amd64.raw +2024-04-24 09:29:44 | INFO | update_image:185 - Getting checksums from https://cdimage.debian.org/cdimage/cloud/bullseye/latest/SHA512SUMS +2024-04-24 09:29:45 | INFO | update_image:192 - Checksum of current debian-11-genericcloud-amd64-20240211-1654.raw is sha512:bdccf01b778a602024918e27bb8cfd84be32104609651f457ac1db10ee5d2a490d0c60e21ce3c0a7704e7ca439281724d0d7e48d279c9fc3a5133a7283e321e4 +2024-04-24 09:29:45 | INFO | update_image:208 - Our checksum is sha512:bdccf01b778a602024918e27bb8cfd84be32104609651f457ac1db10ee5d2a490d0c60e21ce3c0a7704e7ca439281724d0d7e48d279c9fc3a5133a7283e321e4 +2024-04-24 09:29:45 | INFO | update_image:211 - Image Debian 11 is up-to-date, nothing to do +2024-04-24 09:29:45 | INFO | update_image:179 - Checking image Debian 12 +2024-04-24 09:29:45 | INFO | update_image:182 - Latest download URL is https://cdimage.debian.org/cdimage/cloud/bookworm/daily/latest/debian-12-genericcloud-amd64-daily.raw +2024-04-24 09:29:45 | INFO | update_image:185 - Getting checksums from https://cdimage.debian.org/cdimage/cloud/bookworm/daily/latest/SHA512SUMS +2024-04-24 09:29:46 | INFO | update_image:192 - Checksum of current debian-12-genericcloud-amd64-daily-20240424-1727.raw is sha512:f4850b3910adb80801649399d4f89be08974a05a198aba7093f6e72d38d82183bc5b36183fb8dd34cd48a3e226d46802d8a8d85e8b5714b67c52e7ea642f085e +2024-04-24 09:29:46 | INFO | update_image:208 - Our checksum is sha512:5401f8c6361bb2a82c2c24b4b4606d95e77229152a80e61f9c613bc88e25de9257057d0ed68b0256b745c4059162a54970fe4a8daf456b2eb67b4f5db5c97fcc +2024-04-24 09:29:46 | INFO | update_image:229 - New values are {'version': '20240424', 'build_date': datetime.date(2024, 4, 24), 'checksum': 'sha512:f4850b3910adb80801649399d4f89be08974a05a198aba7093f6e72d38d82183bc5b36183fb8dd34cd48a3e226d46802d8a8d85e8b5714b67c52e7ea642f085e', 'url': 'https://cdimage.debian.org/cdimage/cloud/bookworm/daily/20240424-1727/debian-12-genericcloud-amd64-daily-20240424-1727.raw'} +2024-04-24 09:29:46 | INFO | main:300 - Processing file /etc/images/rockylinux.yml +2024-04-24 09:29:46 | INFO | update_image:179 - Checking image Rocky 9 +2024-04-24 09:29:46 | INFO | update_image:182 - Latest download URL is https://download.rockylinux.org/pub/rocky/9/images/x86_64/Rocky-9-GenericCloud.latest.x86_64.qcow2 +2024-04-24 09:29:46 | INFO | update_image:185 - Getting checksums from https://download.rockylinux.org/pub/rocky/9/images/x86_64/Rocky-9-GenericCloud.latest.x86_64.qcow2.CHECKSUM +2024-04-24 09:29:47 | INFO | update_image:192 - Checksum of current Rocky-9-GenericCloud.latest.x86_64.qcow2 is sha256:7713278c37f29b0341b0a841ca3ec5c3724df86b4d97e7ee4a2a85def9b2e651 +2024-04-24 09:29:47 | INFO | update_image:208 - Our checksum is sha256:7713278c37f29b0341b0a841ca3ec5c3724df86b4d97e7ee4a2a85def9b2e651 +2024-04-24 09:29:47 | INFO | update_image:211 - Image Rocky_9 is up-to-date, nothing to do +2024-04-24 09:29:47 | INFO | main:300 - Processing file /etc/images/ubuntu.yml +2024-04-24 09:29:47 | INFO | update_image:179 - Checking image Ubuntu 22.04 +2024-04-24 09:29:47 | INFO | update_image:182 - Latest download URL is https://cloud-images.ubuntu.com/jammy/current/jammy-server-cloudimg-amd64.img +2024-04-24 09:29:47 | INFO | update_image:185 - Getting checksums from https://cloud-images.ubuntu.com/jammy/current/SHA256SUMS +2024-04-24 09:29:47 | INFO | update_image:192 - Checksum of current jammy-server-cloudimg-amd64.img is sha256:62af6445fd2c31f68a069151938a7dcb49158644cae531dd22efc36c1c15a710 +2024-04-24 09:29:47 | INFO | update_image:208 - Our checksum is sha256:62af6445fd2c31f68a069151938a7dcb49158644cae531dd22efc36c1c15a710 +2024-04-24 09:29:47 | INFO | update_image:211 - Image Ubuntu_22.04 is up-to-date, nothing to do +2024-04-24 09:29:47 | INFO | update_image:179 - Checking image Ubuntu 22.04 Minimal +2024-04-24 09:29:47 | INFO | update_image:182 - Latest download URL is https://cloud-images.ubuntu.com/minimal/releases/jammy/release/ubuntu-22.04-minimal-cloudimg-amd64.img +2024-04-24 09:29:47 | INFO | update_image:185 - Getting checksums from https://cloud-images.ubuntu.com/minimal/releases/jammy/release/SHA256SUMS +2024-04-24 09:29:48 | INFO | update_image:192 - Checksum of current ubuntu-22.04-minimal-cloudimg-amd64.img is sha256:bd99c64ad9d926eb5769f9f2cfd96ae4989a029bd64bd3e7e7deb8cff4251c65 +2024-04-24 09:29:48 | INFO | update_image:208 - Our checksum is sha256:bd99c64ad9d926eb5769f9f2cfd96ae4989a029bd64bd3e7e7deb8cff4251c65 +2024-04-24 09:29:48 | INFO | update_image:211 - Image Ubuntu 22.04 Minimal is up-to-date, nothing to do +2024-04-24 09:29:48 | INFO | update_image:179 - Checking image Ubuntu 24.04 +2024-04-24 09:29:48 | INFO | update_image:182 - Latest download URL is https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img +2024-04-24 09:29:48 | INFO | update_image:185 - Getting checksums from https://cloud-images.ubuntu.com/noble/current/SHA256SUMS +2024-04-24 09:29:48 | INFO | update_image:192 - Checksum of current noble-server-cloudimg-amd64.img is sha256:32a9d30d18803da72f5936cf2b7b9efcb4d0bb63c67933f17e3bdfd1751de3f3 +2024-04-24 09:29:48 | INFO | update_image:208 - Our checksum is sha256:d7ba8d5d1d073f2dc8351973bf4f35157c846a0ea6ee16fb2a9f45a78953e4a7 +2024-04-24 09:29:48 | INFO | update_image:229 - New values are {'version': '20240423', 'build_date': datetime.date(2024, 4, 23), 'checksum': 'sha256:32a9d30d18803da72f5936cf2b7b9efcb4d0bb63c67933f17e3bdfd1751de3f3', 'url': 'https://cloud-images.ubuntu.com/noble/20240423/noble-server-cloudimg-amd64.img'} +``` + +These yaml files are now extended with additional fields and the update.py will take care of the versions, checksum, url and build date to the latest release in the yaml file on every run. + +* latest_checksum_url - URL of the distros checksum file +* latest_url - URL of the distros latest image +* mirror_url - URL of the Image File at the local S3 Mirror + +```yaml title="someexample.yaml" +--- +images: + - name: Debian 12 + enable: true + shortname: debian-12 + format: qcow2 + login: debian + min_disk: 8 + min_ram: 512 + status: active + visibility: public + multi: true + meta: + architecture: x86_64 + hw_disk_bus: scsi + hw_rng_model: virtio + hw_scsi_model: virtio-scsi + hw_watchdog_action: reset + hypervisor_type: qemu + os_distro: debian + os_version: '12' + replace_frequency: quarterly + uuid_validity: last-3 + provided_until: none + tags: [] + latest_checksum_url: https://cdimage.debian.org/cdimage/cloud/bookworm/daily/latest/SHA512SUMS + latest_url: + https://cdimage.debian.org/cdimage/cloud/bookworm/daily/latest/debian-12-genericcloud-amd64-daily.qcow2 + versions: + - build_date: 2024-04-11 + checksum: + sha512:3d6f26616e2c8b705993ddef874232887cebe42f1e70fcc020827ac88e8990177d537d34538c71ae2afd3b8baca953fff71eaa7ef71e752e82532c93dcdca436 + url: + https://cdimage.debian.org/cdimage/cloud/bookworm/daily/20240411-1714/debian-12-genericcloud-amd64-daily-20240411-1714.qcow2 + mirror_url: + https://swift.services.a.regiocloud.tech/swift/v1/AUTH_b182637428444b9aa302bb8d5a5a418c/openstack-images/debian-12/20240411-debian-12.qcow2 + version: '20240411' + +``` From 66618dfd4012a3470d04e3efa13631ddeee282df Mon Sep 17 00:00:00 2001 From: Andreas Schrogl Date: Wed, 24 Apr 2024 11:34:42 +0200 Subject: [PATCH 3/3] update.md - changed some text and added links Signed-off-by: Andreas Schrogl --- .../openstack/day2-operations/image-manager/index.md | 2 +- .../openstack/day2-operations/image-manager/update.md | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/guides/operations-guide/openstack/day2-operations/image-manager/index.md b/docs/guides/operations-guide/openstack/day2-operations/image-manager/index.md index c277d76fc0..5ba3e556f2 100644 --- a/docs/guides/operations-guide/openstack/day2-operations/image-manager/index.md +++ b/docs/guides/operations-guide/openstack/day2-operations/image-manager/index.md @@ -291,7 +291,7 @@ This makes us independent of the availability of the images in the individual up ### Updating images -Some of the images are automatically updated by a CI job. The latest available build at the time of the CI job execution is mirrored and +Some of the images are automatically updated by a [CI job](update). The latest available build at the time of the CI job execution is mirrored and made available as the current version. Currently, the following images are updated once a week (every Sunday at 0 am): diff --git a/docs/guides/operations-guide/openstack/day2-operations/image-manager/update.md b/docs/guides/operations-guide/openstack/day2-operations/image-manager/update.md index 78f48c0cfe..597c9cdec6 100644 --- a/docs/guides/operations-guide/openstack/day2-operations/image-manager/update.md +++ b/docs/guides/operations-guide/openstack/day2-operations/image-manager/update.md @@ -52,8 +52,8 @@ python contrib/update.py --help ::: -Best is to run this Script by cron, to update all Distribution Files periodically to the latest release and afterwards run Openstack Image Manager. -The Distribution Images yaml files must exists before running the script, you can use the files from Github repo under `etc/images/` as template for your first run. +Best is to run this Script by cron or a CI job, to update all Distribution Files periodically to the latest release and afterwards run [Openstack Image Manager](../image-manager/). +The Distribution Image yaml files must exists before running the script, you can use the files from Github repo at `etc/images/` as template for your first run. ```