You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When using s3fs, I saw this concerning message in the logs:
Future exception was never retrieved
future: <Future finished exception=ConnectionError('Connection lost')>
aiohttp.client_exceptions.ClientOSError: [Errno 32] Broken pipe
The above exception was the direct cause of the following exception:
ConnectionError: Connection lost
Luckily, I was using s3fs to output model checkpoints for training a deep learning model, and the next checkpoint succesfully saved, however, we should really probably properly handle this exception by either forwarding it to the caller, or alternatively, engaging in a low level retry to restore the connection and resume the upload.
The text was updated successfully, but these errors were encountered:
When using s3fs, I saw this concerning message in the logs:
Luckily, I was using s3fs to output model checkpoints for training a deep learning model, and the next checkpoint succesfully saved, however, we should really probably properly handle this exception by either forwarding it to the caller, or alternatively, engaging in a low level retry to restore the connection and resume the upload.
The text was updated successfully, but these errors were encountered: