-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Performance bottleneck: too small chunks from uplink_download_read #21
Comments
Jeff: seems like that should be easy to fix with a call to https://pkg.go.dev/io#ReadFull (or something similar) and some careful error handling.. |
Do we know where the 7408 value coming from? |
Bill says: It likely has something to do with EncryptionParameters / Encryption block size and the math 29* 256 -16 https://github.com/storj/uplink/blob/5c11ad6846b6a602cfa489225a5d7d7eef883896/project.go#L74 |
It looks like this was previously created as an issue but possibly not fully diagnosed before closing storj/storj#3499 |
I don't have a good setup for testing a potential fix: https://github.com/storj/uplink-c/tree/download-read-all |
TopperDEL tested w/ .NET and says there's no discernible performance impact in .NET |
I will benchmark it in PHP this weekend |
When calling uplink_download_read() it only gives back 7408 bytes. I've tested this with current main branch.
For some languages/platforms there is a big overhead calling native functions and the large number of calls becomes a performance bottleneck.
The text was updated successfully, but these errors were encountered: