Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Slowness and strange behaviour of OCM handling OCI Images #293

Closed
Skarlso opened this issue Oct 21, 2024 · 2 comments
Closed

Slowness and strange behaviour of OCM handling OCI Images #293

Skarlso opened this issue Oct 21, 2024 · 2 comments
Assignees
@Skarlso
Labels
area/ipcei Important Project of Common European Interest component/ocm-cli OCM Command Line Interface component/ocm-core Open Component Model Core aka. go API kind/bugfix Bug
Milestone
2024-Q4

Comments

@Skarlso
Copy link
Contributor

Skarlso commented Oct 21, 2024 •
edited
Loading

There are some problem dealing with newly created TAGs and images and there is also a significant slowness of the new ORAS code to deal with OCI images. The following terminal actions tell the story:

➜  podinfo git:(main) ✗ time ocm15 transfer commontransportarchive --copy-resources ./transport-archive ghcr.io/skarlso/ocm-transfer-test-0.15.0-1
transferring component "ocm.software/podinfo"...
  transferring version "ocm.software/podinfo:1.0.5"...
    version "ocm.software/podinfo:1.0.5" already present -> skip transport
transferring component "ocm.software/podinfo/backend"...
  transferring version "ocm.software/podinfo/backend:1.0.0"...
    version "ocm.software/podinfo/backend:1.0.0" already present -> skip transport
transferring component "ocm.software/podinfo/frontend"...
  transferring version "ocm.software/podinfo/frontend:1.0.0"...
    version "ocm.software/podinfo/frontend:1.0.0" already present -> skip transport
transferring component "ocm.software/redis"...
  transferring version "ocm.software/redis:1.0.0"...
  ...resource 0 config[configdata.ocm.software]...
  ...resource 1 image[ociImage](library/redis:6.0.1)...
  ...resource 2 manifests[kustomize.ocm.fluxcd.io]...
  ...adding component version...
ocm15 transfer commontransportarchive --copy-resources ./transport-archive   1,50s user 2,00s system 2% cpu 2:01,97 total
➜  podinfo git:(main) ✗ time ocm16 transfer commontransportarchive --copy-resources ./transport-archive ghcr.io/skarlso/ocm-transfer-test-0.16.0-1
transferring component "ocm.software/podinfo"...
  transferring version "ocm.software/podinfo:1.0.5"...
  ...adding component version...
transferring component "ocm.software/podinfo/backend"...
  transferring version "ocm.software/podinfo/backend:1.0.0"...
  ...resource 0 config[configdata.ocm.software]...
  ...resource 1 image[ociImage](stefanprodan/podinfo:6.2.0)...
  ...resource 2 manifests[kustomize.ocm.fluxcd.io]...
  ...adding component version...
transferring component "ocm.software/podinfo/frontend"...
  transferring version "ocm.software/podinfo/frontend:1.0.0"...
  ...resource 0 config[configdata.ocm.software]...
  ...resource 1 image[ociImage](stefanprodan/podinfo:6.2.0)...
  ...resource 2 manifests[kustomize.ocm.fluxcd.io]...
  ...adding component version...
transferring component "ocm.software/redis"...
  transferring version "ocm.software/redis:1.0.0"...
  ...resource 0 config[configdata.ocm.software]...
  ...resource 1 image[ociImage](library/redis:6.0.1)...
  ...resource 2 manifests[kustomize.ocm.fluxcd.io]...
  ...adding component version...
ocm16 transfer commontransportarchive --copy-resources ./transport-archive   2,35s user 2,81s system 2% cpu 3:48,74 total
➜  podinfo git:(main) ✗ ocm oci artifacts download ghcr.io/skarlso/ocm-transfer-test-22/stefanprodan/podinfo@sha256:fbc7e3038e8f8235d4d0c04484ee5d3019eb941a9643b733e60c18443d228f3a -O downloaded30
➜  podinfo git:(main) ✗ ocm-transfer-test-0.15.0-1/stefanprodan/podinfo / sha256:fbc7e3038e8f8235d4d0c04484ee5d3019eb941a9643b733e60c18443d228f3a

➜  podinfo git:(main) ✗ ocm15 oci artifacts download ghcr.io/skarlso/ocm-transfer-test-0.15.0-1/stefanprodan/podinfo@sha256:fbc7e3038e8f8235d4d0c04484ee5d3019eb941a9643b733e60c18443d228f3a -O donwload-v0.15.0

donwload-v0.15.0: downloaded
➜  podinfo git:(main) ✗ ocm16 oci artifacts download ghcr.io/skarlso/ocm-transfer-test-0.15.0-1/stefanprodan/podinfo@sha256:fbc7e3038e8f8235d4d0c04484ee5d3019eb941a9643b733e60c18443d228f3a -O donwload-v0.15.0-with-16
donwload-v0.15.0-with-16: downloaded
➜  podinfo git:(main) ✗ time ocm15 oci artifacts download ghcr.io/skarlso/ocm-transfer-test-0.15.0-1/stefanprodan/podinfo@sha256:fbc7e3038e8f8235d4d0c04484ee5d3019eb941a9643b733e60c18443d228f3a -O donwload-v0.15.0-time

donwload-v0.15.0-time: downloaded
ocm15 oci artifacts download  -O donwload-v0.15.0-time  0,14s user 0,16s system 10% cpu 2,876 total
➜  podinfo git:(main) ✗ time ocm16 oci artifacts download ghcr.io/skarlso/ocm-transfer-test-0.15.0-1/stefanprodan/podinfo@sha256:fbc7e3038e8f8235d4d0c04484ee5d3019eb941a9643b733e60c18443d228f3a -O donwload-v0.15.0-time-16
donwload-v0.15.0-time-16: downloaded
ocm16 oci artifacts download  -O donwload-v0.15.0-time-16  0,21s user 0,31s system 3% cpu 17,191 total
➜  podinfo git:(main) ✗ time ocm15 oci artifacts download ghcr.io/skarlso/ocm-transfer-test-0.16.0-1/stefanprodan/podinfo@sha256:fbc7e3038e8f8235d4d0c04484ee5d3019eb941a9643b733e60c18443d228f3a -O download-v0.16.0-time

Error: getting indexed artifact sha256:f60e14b08375a64528113dd8808b16030c771f626e66961dfaf511b74d6f68dc: oci artifact "ghcr.io/skarlso/ocm-transfer-test-0.16.0-1/stefanprodan/podinfo@sha256:f60e14b08375a64528113dd8808b16030c771f626e66961dfaf511b74d6f68dc" not found in skarlso/ocm-transfer-test-0.16.0-1/stefanprodan/podinfo
ocm15 oci artifacts download  -O download-v0.16.0-time  0,10s user 0,07s system 10% cpu 1,659 total
➜  podinfo git:(main) ✗ time ocm16 oci artifacts download ghcr.io/skarlso/ocm-transfer-test-0.16.0-1/stefanprodan/podinfo@sha256:fbc7e3038e8f8235d4d0c04484ee5d3019eb941a9643b733e60c18443d228f3a -O download-v0.16.0-time

Error: mkdir download-v0.16.0-time: file exists
ocm16 oci artifacts download  -O download-v0.16.0-time  0,12s user 0,06s system 14% cpu 1,216 total
➜  podinfo git:(main) ✗ time ocm16 oci artifacts download ghcr.io/skarlso/ocm-transfer-test-0.16.0-1/stefanprodan/podinfo@sha256:fbc7e3038e8f8235d4d0c04484ee5d3019eb941a9643b733e60c18443d228f3a -O download-v0.16.0-with-16-time
Error: getting indexed artifact sha256:f60e14b08375a64528113dd8808b16030c771f626e66961dfaf511b74d6f68dc: oci artifact "ghcr.io/skarlso/ocm-transfer-test-0.16.0-1/stefanprodan/podinfo@sha256:f60e14b08375a64528113dd8808b16030c771f626e66961dfaf511b74d6f68dc" not found in skarlso/ocm-transfer-test-0.16.0-1/stefanprodan/podinfo
ocm16 oci artifacts download  -O download-v0.16.0-with-16-time  0,12s user 0,06s system 12% cpu 1,497 total

The SHA is taken each time directly from the repository by checking what the created tag corresponds to. In case of 0.16.0 using the hash isn't working.

Using the tag also doesn't work:

time ocm16 oci artifacts download ghcr.io/skarlso/ocm-transfer-test-0.16.0-1/stefanprodan/podinfo:6.2.0 -O download-v0.16.0-with-16-time-tag
Error: getting indexed artifact sha256:f60e14b08375a64528113dd8808b16030c771f626e66961dfaf511b74d6f68dc: oci artifact "ghcr.io/skarlso/ocm-transfer-test-0.16.0-1/stefanprodan/podinfo@sha256:f60e14b08375a64528113dd8808b16030c771f626e66961dfaf511b74d6f68dc" not found in skarlso/ocm-transfer-test-0.16.0-1/stefanprodan/podinfo
ocm16 oci artifacts download  -O download-v0.16.0-with-16-time-tag  0,10s user 0,07s system 9% cpu 1,681 total
➜  podinfo git:(main) ✗ time ocm16 oci artifacts download ghcr.io/skarlso/ocm-transfer-test-0.16.0-1/stefanprodan/podinfo:6.2.0 -O download-v0.16.0-with-16-time-tag

Using with component-descriptor:

time ocm16 oci artifacts download ghcr.io/skarlso/ocm-transfer-test-0.16.0-1/component-descriptors/ocm.software/podinfo@sha256:6ab290e0e34815a3176955ecd0c10d836bc0369da8a1a89326e861af650b7587 -O download-v0.16.0-with-16-time-tag-3-component
download-v0.16.0-with-16-time-tag-3-component: downloaded
ocm16 oci artifacts download  -O download-v0.16.0-with-16-time-tag-3-componen  0,13s user 0,06s system 8% cpu 2,328 total

Downloading the component directly works. Whatever the reason for the layering is we should NOT break functionality working previously.

Looking at the time we also see that the oras one (16) works significantly slower during transfer actions.
@Skarlso Skarlso self-assigned this Oct 21, 2024
@Skarlso Skarlso converted this from a draft issue Oct 21, 2024
@github-actions github-actions bot added the area/ipcei Important Project of Common European Interest label Oct 21, 2024
@Skarlso
Copy link
Contributor Author

Skarlso commented Oct 21, 2024

More information from others testing this and seeing this problem: open-component-model/ocm#994

@morri-son morri-son added component/ocm-cli OCM Command Line Interface component/ocm-core Open Component Model Core aka. go API kind/bugfix Bug labels Oct 21, 2024
@morri-son morri-son added this to the 2024-Q4 milestone Oct 21, 2024
@Skarlso
Copy link
Contributor Author

Skarlso commented Oct 24, 2024

Reverted the ORAS change here open-component-model/ocm#1005

@Skarlso Skarlso closed this as completed Oct 24, 2024
@ocmbot ocmbot bot moved this from 🍺 Done to 🔒Closed in OCM Backlog Board Nov 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Skarlso Skarlso

Labels
area/ipcei Important Project of Common European Interest component/ocm-cli OCM Command Line Interface component/ocm-core Open Component Model Core aka. go API kind/bugfix Bug
Projects
OCM Backlog Board
Status: 🔒Closed
Milestone
2024-Q4
Development

No branches or pull requests
3 participants
@Skarlso @jakobmoellerdev @morri-son