Merge branch 'feature/51-azure-multipart-support'

README.md

@@ -266,7 +266,47 @@ from the YAML string taking precedence over values from the YAML file.
 
 #### Transfer Adapters
 
-TBD
+Git LFS servers and clients can implement and negotiate different [transfer adapters]
+(https://github.com/git-lfs/git-lfs/blob/master/docs/api/basic-transfers.md). Typically,
+Git LFS will only define a `basic` transfer mode and support that. `basic` is simple 
+and efficient for direct-to-storage uploads for backends that support uploading using 
+a single `PUT` request.  
+
+To support more complex, and especially multi-part uploads (uploads done using more
+than one HTTP request, each with a different part of a large file) directly to backends
+that support that, Giftless adds support for a non-standard `multipart-basic` transfer 
+mode. Note that this can only work with specific backends that support this type of 
+functionality. 
+
+##### Enabling Multipart Transfer Mode
+
+You can enable multipart transfers by adding the following lines to your Giftless config
+file:
+
+```yaml
+TRANSFER_ADAPTERS:
+  # Add the following lines:
+  multipart-basic:
+    factory: giftless.transfer.multipart:factory
+    options:
+      storage_class: giftless.storage.azure:AzureBlobsStorage
+      storage_options:
+        connection_string: "somesecretconnectionstringhere"
+        container_name: my-multipart-storage
+```
+
+You must specify a `storage_class` that supports multipart transfers (implements the `MultipartStorage`
+interface). Currently, these are:
+* `giftless.storage.azure:AzureBlobsStorage` - Azure Blob Storage
+
+The following additional options are available for `multipart-basic` transfer adapter:
+* `action_lifetime` - The maximal lifetime in seconds for signed multipart actions; Because multipart 
+uploads tend to be of very large files and can easily take hours to complete, we recommend setting this
+to a few hours; The default is 6 hours. 
+* `max_part_size` - Maximal length in bytes of a single part upload. The default is 10MB.
+
+See the specific storage adapter for additional backend-specific configuration options to be added under
+`storage_options`.
 
 #### Authenticators
 

giftless/transfer/multipart.py

@@ -10,6 +10,7 @@
 from giftless.util import get_callable
 
 DEFAULT_PART_SIZE = 10240000  # 10mb
+DEFAULT_ACTION_LIFETIME = 6 * 3600  # 6 hours
 
 
 class MultipartTransferAdapter(PreAuthorizingTransferAdapter, ViewProvider):
@@ -75,7 +76,8 @@ def _check_object(self, prefix: str, oid: str, size: int):
             raise exc.InvalidObject('Object size does not match')
 
 
-def factory(storage_class, storage_options, action_lifetime: int, max_part_size: int = DEFAULT_PART_SIZE):
+def factory(storage_class, storage_options, action_lifetime: int = DEFAULT_ACTION_LIFETIME,
+            max_part_size: int = DEFAULT_PART_SIZE):
     """Factory for multipart transfer adapter with storage
     """
     try: