Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GCP Storage Bucket binding: Bulk file transfer #3655

Open
rahulpoddar-fyndna opened this issue Jan 22, 2025 · 0 comments
Open

GCP Storage Bucket binding: Bulk file transfer #3655

rahulpoddar-fyndna opened this issue Jan 22, 2025 · 0 comments
Labels
kind/bug Something isn't working

Comments

@rahulpoddar-fyndna
Copy link

We move files from GCP object store to PV for processing and there are more then 100+ files (ranging from 15 to 30GB) to be transferred daily.

Dapr supports one by one file transfer and hence not been able to use dapr object store binding for this. One by one file transfer takes long time to complete.

Using GCP SDK (java) completes the 100+ file transfers in around 1 minutes or less.

We would like to go with dapr binding for this requirement as well.

Can we have feature similar to how GCP sdk provides for transfer of files from a object store to PV. Provision to transfer files in parallel from one object store to PV makes it very fast.
Likewise, support for Azure and AWS will be helpful.

@rahulpoddar-fyndna rahulpoddar-fyndna added the kind/bug Something isn't working label Jan 22, 2025
@rahulpoddar-fyndna rahulpoddar-fyndna changed the title Bulk file transfer in parallel from Object store to PV GCP Storage Bucket binding: Bulk file transfer Jan 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant