Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

boto3 >=1.36.0 breaks most non-AWS S3 providers #35

Open
calwe24 opened this issue Jan 21, 2025 · 1 comment
Open

boto3 >=1.36.0 breaks most non-AWS S3 providers #35

calwe24 opened this issue Jan 21, 2025 · 1 comment
Labels
bug Something isn't working

Comments

@calwe24
Copy link

calwe24 commented Jan 21, 2025

Package version (if known): N/A

Describe the bug

boto3 recently updated to enforce checksum validation on all requests by default - including S3. This broke many S3 providers as they don't support this: boto/boto3#4392

Any operation that interacts with the S3 bucket will now fail, such as attempting to upload a file. The browser recieves an HTML 500 error, and the Invenio logs will complain about 'MissingContentLength'. It can be fixed by either downgrading boto3, or setting the env variables AWS_REQUEST_CHECKSUM_CALCULATION and AWS_RESPONSE_CHECKSUM_VALIDATION to WHEN_REQUIRED. It was discussed in the Invenio discord server that these should be added as option that can be set in invenio.cfg, rather than as env variables.

Expected behavior

Screenshots (if applicable)

Additional context

@calwe24 calwe24 added the bug Something isn't working label Jan 21, 2025
@tmorrell
Copy link

I think the env variables are the way to go (versus pinning), since folks with a non-AWS S3 provider will know they already have to do some configuration. I think we should add these config options to the docs here https://inveniordm.docs.cern.ch/customize/s3/#amazon-s3, in a new section below Amazon S3.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants