diff --git a/README.md b/README.md index 7afcd0f..a5b5eb7 100644 --- a/README.md +++ b/README.md @@ -13,7 +13,7 @@ You can add the latest stable version of this library to your application using me.desair.tus tus-java-server - 1.0.0-1.3 + 1.0.0-2.0 The main entry point of the library is the `me.desair.tus.server.TusFileUploadService.process(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse)` method. You can call this method inside a `javax.servlet.http.HttpServlet`, a `javax.servlet.Filter` or any REST API controller of a framework that gives you access to `HttpServletRequest` and `HttpServletResponse` objects. In the following list, you can find some example implementations: @@ -49,9 +49,10 @@ The first step is to create a `TusFileUploadService` object using its constructo * `withDownloadFeature()`: Enable the unofficial `download` extension that also allows you to download uploaded bytes. * `addTusExtension(TusExtension)`: Add a custom (application-specific) extension that implements the `me.desair.tus.server.TusExtension` interface. For example you can add your own extension that checks authentication and authorization policies within your application for the user doing the upload. * `disableTusExtension(String)`: Disable the `TusExtension` for which the `getName()` method matches the provided string. The default extensions have names "creation", "checksum", "expiration", "concatenation", "termination" and "download". You cannot disable the "core" feature. +* `withUploadIdFactory(UploadIdFactory)`: Provide a custom `UploadIdFactory` implementation that should be used to generate identifiers for the different uploads. The default implementation generates identifiers using a UUID (`UUIDUploadIdFactory`). Another example implementation of a custom ID factory is the system-time based `TimeBasedUploadIdFactory` class. -For now this library only provides file system-based storage and locking options. You can however provide your own implementation of a `UploadStorageService` and `UploadLockingService` using the methods `withUploadStorageService(UploadStorageService)` and `withUploadLockingService(UploadLockingService)` in order to support different types of upload storage. +For now this library only provides filesystem based storage and locking options. You can however provide your own implementation of a `UploadStorageService` and `UploadLockingService` using the methods `withUploadStorageService(UploadStorageService)` and `withUploadLockingService(UploadLockingService)` in order to support different types of upload storage. ### 2. Processing an upload To process an upload request you have to pass the current `javax.servlet.http.HttpServletRequest` and `javax.servlet.http.HttpServletResponse` objects to the `me.desair.tus.server.TusFileUploadService.process()` method. Typical places were you can do this are inside Servlets, Filters or REST API Controllers (see [examples](#quick-start-and-examples)). @@ -59,10 +60,14 @@ To process an upload request you have to pass the current `javax.servlet.http.Ht Optionally you can also pass a `String ownerKey` parameter. The `ownerKey` can be used to have a hard separation between uploads of different users, groups or tenants in a multi-tenant setup. Examples of `ownerKey` values are user ID's, group names, client ID's... ### 3. Retrieving the uploaded bytes and metadata within the application -Once the upload has been completed by the user, the business logic layer of your application needs to retrieve and do something with the uploaded bytes. This can be achieved by using the `me.desair.tus.server.TusFileUploadService.getUploadedBytes(String uploadURL)` method. The passed `uploadURL` value should be the upload url used by the client to which the file was uploaded. Therefor your application should pass the upload URL of completed uploads to the backend. Optionally, you can also pass an `ownerKey` value to this method in case your application chooses to process uploads using owner keys. Examples of values that can be used as an `ownerKey` are: an internal user identifier, a session ID, the name of the subpart of your application... +Once the upload has been completed by the user, the business logic layer of your application needs to retrieve and do something with the uploaded bytes. For example it could read the contents of the file, or move the uploaded bytes to their final persistent storage location. Retrieving the uploaded bytes in the backend can be achieved by using the `me.desair.tus.server.TusFileUploadService.getUploadedBytes(String uploadURL)` method. The passed `uploadURL` value should be the upload url used by the client to which the file was uploaded. Therefor your application should pass the upload URL of completed uploads to the backend. Optionally, you can also pass an `ownerKey` value to this method in case your application chooses to process uploads using owner keys. Examples of values that can be used as an `ownerKey` are: an internal user identifier, a session ID, the name of the subpart of your application... + +Using the `me.desair.tus.server.TusFileUploadService.getUploadInfo(String uploadURL)` method you can retrieve metadata about a specific upload process. This includes metadata provided by the client as well as metadata kept by the library like creation timestamp, creator ip-address list, upload length... The method `UploadInfo.getId()` will return the unique identifier of this upload encapsulated in an `UploadId` instance. The original (custom generated) identifier object of this upload can be retrieved using `UploadId.getOriginalObject()`. A URL safe string representation of the identifier is returned by `UploadId.toString()`. It is highly recommended to consult the [JavaDoc of both classes](https://tus.desair.me/). ### 4. Upload cleanup -After having processed the uploaded bytes on the server backend, it's important to cleanup the uploaded bytes. This can be done by calling the `me.desair.tus.server.TusFileUploadService.deleteUpload(String uploadURI)` method. This will remove the uploaded bytes and any associated upload information from the storage backend. Alternatively, a client can also remove an (in-progress) upload using the [termination extension](https://tus.io/protocols/resumable-upload.html#termination). +After having processed the uploaded bytes on the server backend (e.g. copy them to their final persistent location), it's important to cleanup the (temporary) uploaded bytes. This can be done by calling the `me.desair.tus.server.TusFileUploadService.deleteUpload(String uploadURI)` method. This will remove the uploaded bytes and any associated upload information from the storage backend. Alternatively, a client can also remove an (in-progress) upload using the [termination extension](https://tus.io/protocols/resumable-upload.html#termination). + +Next to removing uploads after they have been completed and processed by the backend, it is also recommended to schedule a regular maintenance task to clean up any expired uploads or locks. Cleaning up expired uploads and locks can be achieved using the `me.desair.tus.server.TusFileUploadService.cleanup()` method. ## Compatible Client Implementations This tus protocol implementation has been [tested](https://github.com/tomdesair/tus-java-server-spring-demo) with the [Uppy file upload client](https://uppy.io/). This repository also contains [many automated integration tests](https://github.com/tomdesair/tus-java-server/blob/master/src/test/java/me/desair/tus/server/ITTusFileUploadService.java) that validate the tus protocol server implementation using plain HTTP requests. So in theory this means we're compatible with any tus 1.0.0 compliant client. @@ -71,4 +76,4 @@ This tus protocol implementation has been [tested](https://github.com/tomdesair/ This artifact is versioned as `A.B.C-X.Y` where `A.B.C` is the version of the implemented tus protocol (currently 1.0.0) and `X.Y` is the version of this library. ## Contributing -This library comes without any warranty and is released under a [MIT license](https://github.com/tomdesair/tus-java-server/blob/master/LICENSE). If you encounter any bugs or if you have an idea for a useful improvement you are welcome to [open a new issue](https://github.com/tomdesair/tus-java-server/issues) or to [create a pull request](https://github.com/tomdesair/tus-java-server/pulls) with the proposed implementation. Please note that any contributed code needs to be accompanied by automated unit and/or integration tests and comply with the [define code-style](https://github.com/tomdesair/tus-java-server/blob/master/checkstyle.xml). +This library comes without any warranty and is released under a [MIT license](https://github.com/tomdesair/tus-java-server/blob/master/LICENSE). If you encounter any bugs or if you have an idea for a useful improvement you are welcome to [open a new issue](https://github.com/tomdesair/tus-java-server/issues) or to [create a pull request](https://github.com/tomdesair/tus-java-server/pulls) with the proposed implementation. Please note that any contributed code needs to be accompanied by automated unit and/or integration tests and comply with the [defined code-style](https://github.com/tomdesair/tus-java-server/blob/master/checkstyle.xml). diff --git a/pom.xml b/pom.xml index 239e53c..2fc18ae 100644 --- a/pom.xml +++ b/pom.xml @@ -4,7 +4,7 @@ me.desair.tus tus-java-server - 1.0.0-1.4-SNAPSHOT + 1.0.0-2.0-SNAPSHOT jar ${project.groupId}:${project.artifactId} @@ -178,11 +178,8 @@ ${surefireArgLine} - - - off - true - + + ${project.build.directory}/surefire-reports @@ -202,11 +199,6 @@ ${failsafeArgLine} ${project.build.directory}/surefire-reports - - - off - true - diff --git a/src/main/java/me/desair/tus/server/HttpHeader.java b/src/main/java/me/desair/tus/server/HttpHeader.java index 02c197b..48a8dce 100644 --- a/src/main/java/me/desair/tus/server/HttpHeader.java +++ b/src/main/java/me/desair/tus/server/HttpHeader.java @@ -92,6 +92,12 @@ public class HttpHeader { */ public static final String TUS_CHECKSUM_ALGORITHM = "Tus-Checksum-Algorithm"; + /** + * The X-Forwarded-For (XFF) HTTP header field is a common method for identifying the originating IP address of a + * client connecting to a web server through an HTTP proxy or load balancer. + */ + public static final String X_FORWARDED_FOR = "X-Forwarded-For"; + private HttpHeader() { //This is an utility class to hold constants } diff --git a/src/main/java/me/desair/tus/server/TusFileUploadService.java b/src/main/java/me/desair/tus/server/TusFileUploadService.java index 33b735a..4e7441e 100644 --- a/src/main/java/me/desair/tus/server/TusFileUploadService.java +++ b/src/main/java/me/desair/tus/server/TusFileUploadService.java @@ -17,6 +17,7 @@ import me.desair.tus.server.exception.TusException; import me.desair.tus.server.expiration.ExpirationExtension; import me.desair.tus.server.termination.TerminationExtension; +import me.desair.tus.server.upload.UUIDUploadIdFactory; import me.desair.tus.server.upload.UploadIdFactory; import me.desair.tus.server.upload.UploadInfo; import me.desair.tus.server.upload.UploadLock; @@ -44,7 +45,7 @@ public class TusFileUploadService { private UploadStorageService uploadStorageService; private UploadLockingService uploadLockingService; - private UploadIdFactory idFactory = new UploadIdFactory(); + private UploadIdFactory idFactory = new UUIDUploadIdFactory(); private final LinkedHashMap enabledFeatures = new LinkedHashMap<>(); private final Set supportedHttpMethods = EnumSet.noneOf(HttpMethod.class); private boolean isThreadLocalCacheEnabled = false; @@ -93,6 +94,24 @@ public TusFileUploadService withMaxUploadSize(Long maxUploadSize) { return this; } + /** + * Provide a custom {@link UploadIdFactory} implementation that should be used to generate identifiers for + * the different uploads. Example implementation are {@link me.desair.tus.server.upload.UUIDUploadIdFactory} and + * {@link me.desair.tus.server.upload.TimeBasedUploadIdFactory}. + * + * @param uploadIdFactory The custom {@link UploadIdFactory} implementation + * @return The current service + */ + public TusFileUploadService withUploadIdFactory(UploadIdFactory uploadIdFactory) { + Validate.notNull(uploadIdFactory, "The UploadIdFactory cannot be null"); + String previousUploadURI = this.idFactory.getUploadURI(); + this.idFactory = uploadIdFactory; + this.idFactory.setUploadURI(previousUploadURI); + this.uploadStorageService.setIdFactory(this.idFactory); + this.uploadLockingService.setIdFactory(this.idFactory); + return this; + } + /** * Provide a custom {@link UploadStorageService} implementation that should be used to store uploaded bytes and * metadata ({@link UploadInfo}). @@ -108,7 +127,7 @@ public TusFileUploadService withUploadStorageService(UploadStorageService upload uploadStorageService.setIdFactory(this.idFactory); //Update the upload storage service this.uploadStorageService = uploadStorageService; - prepareCacheIfEnable(); + prepareCacheIfEnabled(); return this; } @@ -125,7 +144,7 @@ public TusFileUploadService withUploadLockingService(UploadLockingService upload uploadLockingService.setIdFactory(this.idFactory); //Update the upload storage service this.uploadLockingService = uploadLockingService; - prepareCacheIfEnable(); + prepareCacheIfEnabled(); return this; } @@ -138,9 +157,9 @@ public TusFileUploadService withUploadLockingService(UploadLockingService upload */ public TusFileUploadService withStoragePath(String storagePath) { Validate.notBlank(storagePath, "The storage path cannot be blank"); - withUploadStorageService(new DiskStorageService(idFactory, storagePath)); - withUploadLockingService(new DiskLockingService(idFactory, storagePath)); - prepareCacheIfEnable(); + withUploadStorageService(new DiskStorageService(storagePath)); + withUploadLockingService(new DiskLockingService(storagePath)); + prepareCacheIfEnabled(); return this; } @@ -152,7 +171,7 @@ public TusFileUploadService withStoragePath(String storagePath) { */ public TusFileUploadService withThreadLocalCache(boolean isEnabled) { this.isThreadLocalCacheEnabled = isEnabled; - prepareCacheIfEnable(); + prepareCacheIfEnabled(); return this; } @@ -450,7 +469,7 @@ private void updateSupportedHttpMethods() { } } - private void prepareCacheIfEnable() { + private void prepareCacheIfEnabled() { if (isThreadLocalCacheEnabled && uploadStorageService != null && uploadLockingService != null) { ThreadLocalCachedStorageAndLockingService service = new ThreadLocalCachedStorageAndLockingService( @@ -461,4 +480,5 @@ private void prepareCacheIfEnable() { this.uploadLockingService = service; } } + } diff --git a/src/main/java/me/desair/tus/server/concatenation/ConcatenationPostRequestHandler.java b/src/main/java/me/desair/tus/server/concatenation/ConcatenationPostRequestHandler.java index 1a41d5e..cc32eb1 100644 --- a/src/main/java/me/desair/tus/server/concatenation/ConcatenationPostRequestHandler.java +++ b/src/main/java/me/desair/tus/server/concatenation/ConcatenationPostRequestHandler.java @@ -44,7 +44,7 @@ public void process(HttpMethod method, TusServletRequest servletRequest, //reset the length, just to be sure uploadInfo.setLength(null); uploadInfo.setUploadType(UploadType.CONCATENATED); - uploadInfo.setConcatenationParts(Utils.parseConcatenationIDsFromHeader(uploadConcatValue)); + uploadInfo.setConcatenationPartIds(Utils.parseConcatenationIDsFromHeader(uploadConcatValue)); uploadStorageService.getUploadConcatenationService().merge(uploadInfo); diff --git a/src/main/java/me/desair/tus/server/core/CorePatchRequestHandler.java b/src/main/java/me/desair/tus/server/core/CorePatchRequestHandler.java index e57b0f0..e8e5695 100644 --- a/src/main/java/me/desair/tus/server/core/CorePatchRequestHandler.java +++ b/src/main/java/me/desair/tus/server/core/CorePatchRequestHandler.java @@ -55,6 +55,11 @@ public void process(HttpMethod method, TusServletRequest servletRequest, if (found) { servletResponse.setHeader(HttpHeader.UPLOAD_OFFSET, Objects.toString(uploadInfo.getOffset())); servletResponse.setStatus(HttpServletResponse.SC_NO_CONTENT); + + if (!uploadInfo.isUploadInProgress()) { + log.info("Upload with ID {} at location {} finished successfully", + uploadInfo.getId(), servletRequest.getRequestURI()); + } } else { log.error("The patch request handler could not find the upload for URL " + servletRequest.getRequestURI() + ". This means something is really wrong the request validators!"); diff --git a/src/main/java/me/desair/tus/server/creation/CreationPostRequestHandler.java b/src/main/java/me/desair/tus/server/creation/CreationPostRequestHandler.java index 38d7ffe..4367aed 100644 --- a/src/main/java/me/desair/tus/server/creation/CreationPostRequestHandler.java +++ b/src/main/java/me/desair/tus/server/creation/CreationPostRequestHandler.java @@ -47,11 +47,12 @@ public void process(HttpMethod method, TusServletRequest servletRequest, servletResponse.setHeader(HttpHeader.LOCATION, url); servletResponse.setStatus(HttpServletResponse.SC_CREATED); - log.debug("Create upload location {}", url); + log.info("Created upload with ID {} at {} for ip address {} with location {}", + info.getId(), info.getCreationTimestamp(), info.getCreatorIpAddresses(), url); } private UploadInfo buildUploadInfo(HttpServletRequest servletRequest) { - UploadInfo info = new UploadInfo(); + UploadInfo info = new UploadInfo(servletRequest); Long length = Utils.getLongHeader(servletRequest, HttpHeader.UPLOAD_LENGTH); if (length != null) { diff --git a/src/main/java/me/desair/tus/server/upload/TimeBasedUploadIdFactory.java b/src/main/java/me/desair/tus/server/upload/TimeBasedUploadIdFactory.java new file mode 100644 index 0000000..ea56dad --- /dev/null +++ b/src/main/java/me/desair/tus/server/upload/TimeBasedUploadIdFactory.java @@ -0,0 +1,33 @@ +package me.desair.tus.server.upload; + +import java.io.Serializable; + +import org.apache.commons.lang3.StringUtils; + +/** + * Alternative {@link UploadIdFactory} implementation that uses the current system time to generate ID's. + * Since time is not unique, this upload ID factory should not be used in busy, clustered production systems. + */ +public class TimeBasedUploadIdFactory extends UploadIdFactory { + + @Override + protected Serializable getIdValueIfValid(String extractedUrlId) { + Long id = null; + + if (StringUtils.isNotBlank(extractedUrlId)) { + try { + id = Long.parseLong(extractedUrlId); + } catch (NumberFormatException ex) { + id = null; + } + } + + return id; + } + + @Override + public synchronized UploadId createId() { + return new UploadId(System.currentTimeMillis()); + } + +} diff --git a/src/main/java/me/desair/tus/server/upload/UUIDUploadIdFactory.java b/src/main/java/me/desair/tus/server/upload/UUIDUploadIdFactory.java new file mode 100644 index 0000000..cce50a6 --- /dev/null +++ b/src/main/java/me/desair/tus/server/upload/UUIDUploadIdFactory.java @@ -0,0 +1,29 @@ +package me.desair.tus.server.upload; + +import java.io.Serializable; +import java.util.UUID; + +/** + * Factory to create unique upload IDs. This factory can also parse the upload identifier + * from a given upload URL. + */ +public class UUIDUploadIdFactory extends UploadIdFactory { + + @Override + protected Serializable getIdValueIfValid(String extractedUrlId) { + UUID id = null; + try { + id = UUID.fromString(extractedUrlId); + } catch (IllegalArgumentException ex) { + id = null; + } + + return id; + } + + @Override + public synchronized UploadId createId() { + return new UploadId(UUID.randomUUID()); + } + +} diff --git a/src/main/java/me/desair/tus/server/upload/UploadId.java b/src/main/java/me/desair/tus/server/upload/UploadId.java new file mode 100644 index 0000000..b025b46 --- /dev/null +++ b/src/main/java/me/desair/tus/server/upload/UploadId.java @@ -0,0 +1,80 @@ +package me.desair.tus.server.upload; + +import java.io.Serializable; +import java.io.UnsupportedEncodingException; +import java.util.Objects; + +import org.apache.commons.codec.DecoderException; +import org.apache.commons.codec.net.URLCodec; +import org.apache.commons.lang3.Validate; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * The unique identifier of an upload process in the tus protocol + */ +public class UploadId implements Serializable { + + private static final String UPLOAD_ID_CHARSET = "UTF-8"; + private static final Logger log = LoggerFactory.getLogger(UploadId.class); + + private String urlSafeValue; + private Serializable originalObject; + + /** + * Create a new {@link UploadId} instance based on the provided object using it's toString method. + * @param inputObject The object to use for constructing the ID + */ + public UploadId(Serializable inputObject) { + String inputValue = (inputObject == null ? null : inputObject.toString()); + Validate.notBlank(inputValue, "The upload ID value cannot be blank"); + + this.originalObject = inputObject; + URLCodec codec = new URLCodec(); + //Check if value is not encoded already + try { + if (inputValue.equals(codec.decode(inputValue, UPLOAD_ID_CHARSET))) { + this.urlSafeValue = codec.encode(inputValue, UPLOAD_ID_CHARSET); + } else { + //value is already encoded, use as is + this.urlSafeValue = inputValue; + } + } catch (DecoderException | UnsupportedEncodingException e) { + log.warn("Unable to URL encode upload ID value", e); + this.urlSafeValue = inputValue; + } + } + + /** + * The original input object that was provided when constructing this upload ID + * @return The original object used to create this ID + */ + public Serializable getOriginalObject() { + return this.originalObject; + } + + @Override + public String toString() { + return urlSafeValue; + } + + @Override + public boolean equals(Object o) { + if (this == o) { + return true; + } + if (!(o instanceof UploadId)) { + return false; + } + + UploadId uploadId = (UploadId) o; + //Upload IDs with the same URL-safe value should be considered equal + return Objects.equals(urlSafeValue, uploadId.urlSafeValue); + } + + @Override + public int hashCode() { + //Upload IDs with the same URL-safe value should be considered equal + return Objects.hash(urlSafeValue); + } +} diff --git a/src/main/java/me/desair/tus/server/upload/UploadIdFactory.java b/src/main/java/me/desair/tus/server/upload/UploadIdFactory.java index 230960e..791f00c 100644 --- a/src/main/java/me/desair/tus/server/upload/UploadIdFactory.java +++ b/src/main/java/me/desair/tus/server/upload/UploadIdFactory.java @@ -1,6 +1,6 @@ package me.desair.tus.server.upload; -import java.util.UUID; +import java.io.Serializable; import java.util.regex.Matcher; import java.util.regex.Pattern; @@ -8,10 +8,10 @@ import org.apache.commons.lang3.Validate; /** - * Factory to create unique upload IDs. This factory can also parse the upload identifier + * Interface for a factory that can create unique upload IDs. This factory can also parse the upload identifier * from a given upload URL. */ -public class UploadIdFactory { +public abstract class UploadIdFactory { private String uploadURI = "/"; private Pattern uploadUriPattern = null; @@ -31,6 +31,14 @@ public void setUploadURI(String uploadURI) { this.uploadUriPattern = null; } + /** + * Return the URI of the main tus upload endpoint. Note that this value possibly contains regex parameters. + * @return The URI of the main tus upload endpoint. + */ + public String getUploadURI() { + return uploadURI; + } + /** * Read the upload identifier from the given URL. *

@@ -40,39 +48,37 @@ public void setUploadURI(String uploadURI) { * @param url The URL provided by the client * @return The corresponding Upload identifier */ - public UUID readUploadId(String url) { + public UploadId readUploadId(String url) { Matcher uploadUriMatcher = getUploadUriPattern().matcher(StringUtils.trimToEmpty(url)); String pathId = uploadUriMatcher.replaceFirst(""); - UUID id = null; + Serializable idValue = null; if (StringUtils.isNotBlank(pathId)) { - try { - id = UUID.fromString(pathId); - } catch (IllegalArgumentException ex) { - id = null; - } + idValue = getIdValueIfValid(pathId); } - return id; + return idValue == null ? null : new UploadId(idValue); } /** - * Return the URI of the main tus upload endpoint. Note that this value possibly contains regex parameters. - * @return The URI of the main tus upload endpoint. + * Create a new unique upload ID + * @return A new unique upload ID */ - public String getUploadURI() { - return uploadURI; - } + public abstract UploadId createId(); /** - * Create a new unique upload ID - * @return A new unique upload ID + * Transform the extracted path ID value to a value to use for the upload ID object. + * If the extracted value is not valid, null is returned + * @param extractedUrlId The ID extracted from the URL + * @return Value to use in the UploadId object, null if the extracted URL value was not valid */ - public synchronized UUID createId() { - return UUID.randomUUID(); - } + protected abstract Serializable getIdValueIfValid(String extractedUrlId); - private Pattern getUploadUriPattern() { + /** + * Build and retrieve the Upload URI regex pattern + * @return A (cached) Pattern to match upload URI's + */ + protected Pattern getUploadUriPattern() { if (uploadUriPattern == null) { //We will extract the upload ID's by removing the upload URI from the start of the request URI uploadUriPattern = Pattern.compile("^.*" diff --git a/src/main/java/me/desair/tus/server/upload/UploadInfo.java b/src/main/java/me/desair/tus/server/upload/UploadInfo.java index 628f3a4..7be6200 100644 --- a/src/main/java/me/desair/tus/server/upload/UploadInfo.java +++ b/src/main/java/me/desair/tus/server/upload/UploadInfo.java @@ -1,19 +1,25 @@ package me.desair.tus.server.upload; import java.io.Serializable; -import java.nio.charset.Charset; import java.util.Arrays; import java.util.Date; import java.util.List; import java.util.Map; import java.util.TreeMap; -import java.util.UUID; +import javax.servlet.http.HttpServletRequest; + +import me.desair.tus.server.util.Utils; +import org.apache.commons.codec.Charsets; import org.apache.commons.codec.binary.Base64; import org.apache.commons.lang3.StringUtils; import org.apache.commons.lang3.builder.EqualsBuilder; import org.apache.commons.lang3.builder.HashCodeBuilder; +/** + * Class that contains all metadata on an upload process. This class also holds the metadata provided by the client + * when creating the upload. + */ public class UploadInfo implements Serializable { private static final String APPLICATION_OCTET_STREAM = "application/octet-stream"; @@ -24,34 +30,77 @@ public class UploadInfo implements Serializable { private Long offset; private String encodedMetadata; private Long length; - private UUID id; + private UploadId id; private String ownerKey; + private Long creationTimestamp; + private String creatorIpAddresses; private Long expirationTimestamp; - private List concatenationParts; + private List concatenationPartIds; private String uploadConcatHeaderValue; + /** + * Default constructor to use if an upload is created without HTTP request + */ public UploadInfo() { + creationTimestamp = getCurrentTime(); offset = 0L; encodedMetadata = null; length = null; } + /** + * Constructor to use if the upload is created using an HTTP request (which is usually the case) + * @param servletRequest The HTTP request that creates the new upload + */ + public UploadInfo(HttpServletRequest servletRequest) { + this(); + creatorIpAddresses = Utils.buildRemoteIpList(servletRequest); + } + + /** + * The current byte offset of the bytes that already have been stored for this upload on the server. + * The offset is the position where the next newly received byte should be stored. This index is zero-based. + * @return The offset where the next new byte will be written + */ public Long getOffset() { return offset; } + /** + * Set the position where the next newly received byte should be stored. This index is zero-based. + * @param offset The offset where the next new byte should be written + */ public void setOffset(Long offset) { this.offset = offset; } + /** + * Get the encoded Tus metadata string as it was provided by the Tus client at creation of the upload. + * The encoded metadata string consists of one or more comma-separated key-value pairs where the key is + * ASCII encoded and the value Base64 encoded. See https://tus.io/protocols/resumable-upload.html#upload-metadata + * @return The encoded metadata string as received from the client + */ public String getEncodedMetadata() { return encodedMetadata; } + /** + * Set the encoded Tus metadata string as it was provided by the Tus client at creation of the upload. + * The encoded metadata string consists of one or more comma-separated key-value pairs where the key is + * ASCII encoded and the value Base64 encoded. See https://tus.io/protocols/resumable-upload.html#upload-metadata + * @return The encoded metadata string as received from the client + */ public void setEncodedMetadata(String encodedMetadata) { this.encodedMetadata = encodedMetadata; } + /** + * Get the decoded metadata map provided by the client based on the encoded Tus metadata string received on + * creation of the upload. The encoded metadata string consists of one or more comma-separated key-value pairs + * where the key is ASCII encoded and the value Base64 encoded. The key and value MUST be separated by a space. + * See https://tus.io/protocols/resumable-upload.html#upload-metadata + * @return The encoded metadata string as received from the client + */ public Map getMetadata() { Map metadata = new TreeMap<>(String.CASE_INSENSITIVE_ORDER); for (String valuePair : splitToArray(encodedMetadata, ",")) { @@ -77,25 +126,43 @@ public Map getMetadata() { return metadata; } + /** + * Did the client provide any metadata when creating this upload? + * @return True if metadata is present, false otherwise + */ public boolean hasMetadata() { return StringUtils.isNotBlank(encodedMetadata); } + /** + * Get the total length of the byte array that the client wants to upload. This value is provided by the client + * when creating the upload (POST) or when uploading a new set of bytes (PATCH). + * @return The number of bytes that the client specified he will upload + */ public Long getLength() { return length; } + /** + * Set the total length of the byte array that the client wants to upload. The client can provided this value + * when creating the upload (POST) or when uploading a new set of bytes (PATCH). + * @param length The number of bytes that the client specified he will upload + */ public void setLength(Long length) { this.length = (length != null && length > 0 ? length : null); } + /** + * Did the client already provide a total upload length? + * @return True if the total upload length is known, false otherwise + */ public boolean hasLength() { return length != null; } /** * An upload is still in progress: - * - as long as we did not receive information on the total length + * - as long as we did not receive information on the total length (see {@link UploadInfo#getLength()}) * - the total length does not match the current offset * @return true if the upload is still in progress, false otherwise */ @@ -103,92 +170,130 @@ public boolean isUploadInProgress() { return length == null || !offset.equals(length); } - public void setId(UUID id) { + /** + * Set the unique identifier of this upload process + * The unique identifier is represented by a {@link UploadId} instance + * @param id The unique identifier to use + */ + public void setId(UploadId id) { this.id = id; } - public UUID getId() { + /** + * Get the unique identifier of this upload process + * The unique identifier is represented by a {@link UploadId} instance + * @return The unique upload identifier of this upload + */ + public UploadId getId() { return id; } + /** + * Set the owner key for this upload. + * This key uniquely identifies the owner of the uploaded bytes. The user of this library is free to interpret the + * meaning of "owner". This can be a user ID, a company division, a group of users, a tenant... + * @param ownerKey The owner key to assign to this upload + */ public void setOwnerKey(String ownerKey) { this.ownerKey = ownerKey; } + /** + * Get the owner key for this upload. + * This key uniquely identifies the owner of the uploaded bytes. The user of this library is free to interpret the + * meaning of "owner". This can be a user ID, a company division, a group of users, a tenant... + * @return The unique identifying key of the owner of this upload + */ public String getOwnerKey() { return ownerKey; } + /** + * Indicates the timestamp after which the upload expires in milliseconds since January 1, 1970, 00:00:00 GMT + * @return The expiration timestamp in milliseconds + */ public Long getExpirationTimestamp() { return expirationTimestamp; } + /** + * Calculate the expiration timestamp based on the provided expiration period. + * @param expirationPeriod The period the upload should remain valid + */ public void updateExpiration(long expirationPeriod) { expirationTimestamp = getCurrentTime() + expirationPeriod; } + /** + * The timestamp this upload was created in number of milliseconds since January 1, 1970, 00:00:00 GMT + * @return Creation timestamp of this upload object + */ + public Long getCreationTimestamp() { + return creationTimestamp; + } + + /** + * Get the ip-addresses that were involved when this upload was created. + * The returned value is a comma-separated list based on the remote address of the request and the + * X-Forwareded-For header. The list is constructed as "client, proxy1, proxy2". + * @return A comma-separated list of ip-addresses + */ + public String getCreatorIpAddresses() { + return creatorIpAddresses; + } + + /** + * Return the type of this upload. An upload can have types specified in {@link UploadType}. + * The type of an upload depends on the Tus concatenation extension: + * https://tus.io/protocols/resumable-upload.html#concatenation + * @return The type of this upload as specified in {@link UploadType} + */ public UploadType getUploadType() { return uploadType; } + /** + * Set the type of this upload. An upload can have types specified in {@link UploadType}. + * The type of an upload depends on the Tus concatenation extension: + * https://tus.io/protocols/resumable-upload.html#concatenation + * @param uploadType The type to set on this upload + */ public void setUploadType(UploadType uploadType) { this.uploadType = uploadType; } - public void setConcatenationParts(List concatenationParts) { - this.concatenationParts = concatenationParts; + /** + * Set the list of upload identifiers of which this upload is composed of. + * @param concatenationPartIds The list of child upload identifiers + */ + public void setConcatenationPartIds(List concatenationPartIds) { + this.concatenationPartIds = concatenationPartIds; } - public List getConcatenationParts() { - return concatenationParts; + /** + * Get the list of upload identifiers of which this upload is composed of. + * @return The list of child upload identifiers + */ + public List getConcatenationPartIds() { + return concatenationPartIds; } + /** + * Set the original value of the "Upload-Concat" HTTP header that was provided by the client + * @param uploadConcatHeaderValue The original value of the "Upload-Concat" HTTP header + */ public void setUploadConcatHeaderValue(String uploadConcatHeaderValue) { this.uploadConcatHeaderValue = uploadConcatHeaderValue; } + /** + * Get the original value of the "Upload-Concat" HTTP header that was provided by the client + * @return The original value of the "Upload-Concat" HTTP header + */ public String getUploadConcatHeaderValue() { return uploadConcatHeaderValue; } - @Override - public boolean equals(Object o) { - if (this == o) { - return true; - } - - if (o == null || getClass() != o.getClass()) { - return false; - } - - UploadInfo info = (UploadInfo) o; - - return new EqualsBuilder() - .append(getOffset(), info.getOffset()) - .append(getEncodedMetadata(), info.getEncodedMetadata()) - .append(getLength(), info.getLength()) - .append(getId(), info.getId()) - .append(getOwnerKey(), info.getOwnerKey()) - .append(getExpirationTimestamp(), info.getExpirationTimestamp()) - .append(getUploadType(), info.getUploadType()) - .append(getUploadConcatHeaderValue(), info.getUploadConcatHeaderValue()) - .isEquals(); - } - - @Override - public int hashCode() { - return new HashCodeBuilder(17, 37) - .append(getOffset()) - .append(getEncodedMetadata()) - .append(getLength()) - .append(getId()) - .append(getOwnerKey()) - .append(getExpirationTimestamp()) - .append(getUploadType()) - .append(getUploadConcatHeaderValue()) - .toHashCode(); - } - /** * Try to guess the filename of the uploaded data. If we cannot guess the name * we fall back to the ID. @@ -234,6 +339,48 @@ public boolean isExpired() { return expirationTimestamp != null && expirationTimestamp < getCurrentTime(); } + @Override + public boolean equals(Object o) { + if (this == o) { + return true; + } + + if (!(o instanceof UploadInfo)) { + return false; + } + + UploadInfo that = (UploadInfo) o; + + return new EqualsBuilder() + .append(getUploadType(), that.getUploadType()) + .append(getOffset(), that.getOffset()) + .append(getEncodedMetadata(), that.getEncodedMetadata()) + .append(getLength(), that.getLength()) + .append(getId(), that.getId()) + .append(getOwnerKey(), that.getOwnerKey()) + .append(getCreatorIpAddresses(), that.getCreatorIpAddresses()) + .append(getExpirationTimestamp(), that.getExpirationTimestamp()) + .append(getConcatenationPartIds(), that.getConcatenationPartIds()) + .append(getUploadConcatHeaderValue(), that.getUploadConcatHeaderValue()) + .isEquals(); + } + + @Override + public int hashCode() { + return new HashCodeBuilder(17, 37) + .append(getUploadType()) + .append(getOffset()) + .append(getEncodedMetadata()) + .append(getLength()) + .append(getId()) + .append(getOwnerKey()) + .append(getCreatorIpAddresses()) + .append(getExpirationTimestamp()) + .append(getConcatenationPartIds()) + .append(getUploadConcatHeaderValue()) + .toHashCode(); + } + /** * Get the current time in the number of milliseconds since January 1, 1970, 00:00:00 GMT */ @@ -253,7 +400,7 @@ private String decode(String encodedValue) { if (encodedValue == null) { return null; } else { - return new String(Base64.decodeBase64(encodedValue), Charset.forName("UTF-8")); + return new String(Base64.decodeBase64(encodedValue), Charsets.UTF_8); } } } diff --git a/src/main/java/me/desair/tus/server/upload/UploadLockingService.java b/src/main/java/me/desair/tus/server/upload/UploadLockingService.java index 31223b0..6dc52af 100644 --- a/src/main/java/me/desair/tus/server/upload/UploadLockingService.java +++ b/src/main/java/me/desair/tus/server/upload/UploadLockingService.java @@ -1,7 +1,6 @@ package me.desair.tus.server.upload; import java.io.IOException; -import java.util.UUID; import me.desair.tus.server.exception.TusException; @@ -29,7 +28,7 @@ public interface UploadLockingService { * @param id The ID of the upload to check * @return True if the upload is locked, false otherwise */ - boolean isLocked(UUID id); + boolean isLocked(UploadId id); /** * Set an instance if IdFactory to be used for creating identities and extracting them from uploadURIs diff --git a/src/main/java/me/desair/tus/server/upload/UploadStorageService.java b/src/main/java/me/desair/tus/server/upload/UploadStorageService.java index 8e651d2..f5d18c0 100644 --- a/src/main/java/me/desair/tus/server/upload/UploadStorageService.java +++ b/src/main/java/me/desair/tus/server/upload/UploadStorageService.java @@ -3,7 +3,6 @@ import java.io.IOException; import java.io.InputStream; import java.io.OutputStream; -import java.util.UUID; import me.desair.tus.server.exception.TusException; import me.desair.tus.server.exception.UploadNotFoundException; @@ -28,7 +27,7 @@ public interface UploadStorageService { * @return The matching upload info * @throws IOException When the service is not able to retrieve the upload information */ - UploadInfo getUploadInfo(UUID id) throws IOException; + UploadInfo getUploadInfo(UploadId id) throws IOException; /** * The URI which is configured as the upload endpoint @@ -88,7 +87,7 @@ InputStream getUploadedBytes(String uploadURI, String ownerKey) * @throws IOException When retrieving the bytes from the storage layer fails * @throws UploadNotFoundException When the proved id is not linked to an upload */ - InputStream getUploadedBytes(UUID id) throws IOException, UploadNotFoundException; + InputStream getUploadedBytes(UploadId id) throws IOException, UploadNotFoundException; /** * Copy the uploaded bytes to the given output stream diff --git a/src/main/java/me/desair/tus/server/upload/cache/ThreadLocalCachedStorageAndLockingService.java b/src/main/java/me/desair/tus/server/upload/cache/ThreadLocalCachedStorageAndLockingService.java index 6e71fb7..3c25ffc 100644 --- a/src/main/java/me/desair/tus/server/upload/cache/ThreadLocalCachedStorageAndLockingService.java +++ b/src/main/java/me/desair/tus/server/upload/cache/ThreadLocalCachedStorageAndLockingService.java @@ -5,9 +5,10 @@ import java.io.OutputStream; import java.lang.ref.WeakReference; import java.util.Objects; -import java.util.UUID; + import me.desair.tus.server.exception.TusException; import me.desair.tus.server.exception.UploadNotFoundException; +import me.desair.tus.server.upload.UploadId; import me.desair.tus.server.upload.UploadIdFactory; import me.desair.tus.server.upload.UploadInfo; import me.desair.tus.server.upload.UploadLock; @@ -45,7 +46,7 @@ public ThreadLocalCachedStorageAndLockingService(UploadStorageService storageSer } @Override - public UploadInfo getUploadInfo(UUID id) throws IOException { + public UploadInfo getUploadInfo(UploadId id) throws IOException { UploadInfo uploadInfo; WeakReference ref = uploadInfoCache.get(); if (ref == null || (uploadInfo = ref.get()) == null || !id.equals(uploadInfo.getId())) { @@ -114,7 +115,7 @@ public InputStream getUploadedBytes(String uploadURI, String ownerKey) throws IO } @Override - public InputStream getUploadedBytes(UUID id) throws IOException, UploadNotFoundException { + public InputStream getUploadedBytes(UploadId id) throws IOException, UploadNotFoundException { return storageServiceDelegate.getUploadedBytes(id); } @@ -180,7 +181,7 @@ public void cleanupStaleLocks() throws IOException { } @Override - public boolean isLocked(UUID id) { + public boolean isLocked(UploadId id) { return lockingServiceDelegate.isLocked(id); } diff --git a/src/main/java/me/desair/tus/server/upload/concatenation/VirtualConcatenationService.java b/src/main/java/me/desair/tus/server/upload/concatenation/VirtualConcatenationService.java index 25bdca5..844d9b3 100644 --- a/src/main/java/me/desair/tus/server/upload/concatenation/VirtualConcatenationService.java +++ b/src/main/java/me/desair/tus/server/upload/concatenation/VirtualConcatenationService.java @@ -31,7 +31,7 @@ public VirtualConcatenationService(UploadStorageService uploadStorageService) { @Override public void merge(UploadInfo uploadInfo) throws IOException, UploadNotFoundException { if (uploadInfo != null && uploadInfo.isUploadInProgress() - && uploadInfo.getConcatenationParts() != null) { + && uploadInfo.getConcatenationPartIds() != null) { Long expirationPeriod = uploadStorageService.getUploadExpirationPeriod(); @@ -70,7 +70,7 @@ public InputStream getConcatenatedBytes(UploadInfo uploadInfo) throws IOExceptio @Override public List getPartialUploads(UploadInfo info) throws IOException, UploadNotFoundException { - List concatenationParts = info.getConcatenationParts(); + List concatenationParts = info.getConcatenationPartIds(); if (concatenationParts == null || concatenationParts.isEmpty()) { return Collections.emptyList(); diff --git a/src/main/java/me/desair/tus/server/upload/disk/AbstractDiskBasedService.java b/src/main/java/me/desair/tus/server/upload/disk/AbstractDiskBasedService.java index d0a97cb..a9cd430 100644 --- a/src/main/java/me/desair/tus/server/upload/disk/AbstractDiskBasedService.java +++ b/src/main/java/me/desair/tus/server/upload/disk/AbstractDiskBasedService.java @@ -4,9 +4,9 @@ import java.nio.file.Files; import java.nio.file.Path; import java.nio.file.Paths; -import java.util.UUID; import me.desair.tus.server.TusFileUploadService; +import me.desair.tus.server.upload.UploadId; import org.apache.commons.lang3.Validate; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -29,7 +29,7 @@ protected Path getStoragePath() { return storagePath; } - protected Path getPathInStorageDirectory(UUID id) { + protected Path getPathInStorageDirectory(UploadId id) { if (!Files.exists(storagePath)) { init(); } diff --git a/src/main/java/me/desair/tus/server/upload/disk/DiskLockingService.java b/src/main/java/me/desair/tus/server/upload/disk/DiskLockingService.java index b289502..71eafb9 100644 --- a/src/main/java/me/desair/tus/server/upload/disk/DiskLockingService.java +++ b/src/main/java/me/desair/tus/server/upload/disk/DiskLockingService.java @@ -6,10 +6,10 @@ import java.nio.file.Files; import java.nio.file.Path; import java.nio.file.attribute.FileTime; -import java.util.UUID; import me.desair.tus.server.exception.TusException; import me.desair.tus.server.exception.UploadAlreadyLockedException; +import me.desair.tus.server.upload.UploadId; import me.desair.tus.server.upload.UploadIdFactory; import me.desair.tus.server.upload.UploadLock; import me.desair.tus.server.upload.UploadLockingService; @@ -30,8 +30,12 @@ public class DiskLockingService extends AbstractDiskBasedService implements Uplo private UploadIdFactory idFactory; - public DiskLockingService(UploadIdFactory idFactory, String storagePath) { + public DiskLockingService(String storagePath) { super(storagePath + File.separator + LOCK_SUB_DIRECTORY); + } + + public DiskLockingService(UploadIdFactory idFactory, String storagePath) { + this(storagePath); Validate.notNull(idFactory, "The IdFactory cannot be null"); this.idFactory = idFactory; } @@ -39,7 +43,7 @@ public DiskLockingService(UploadIdFactory idFactory, String storagePath) { @Override public UploadLock lockUploadByUri(String requestURI) throws TusException, IOException { - UUID id = idFactory.readUploadId(requestURI); + UploadId id = idFactory.readUploadId(requestURI); UploadLock lock = null; @@ -58,7 +62,7 @@ public void cleanupStaleLocks() throws IOException { FileTime lastModifiedTime = Files.getLastModifiedTime(path); if (lastModifiedTime.toMillis() < System.currentTimeMillis() - 10000L) { - UUID id = UUID.fromString(path.getFileName().toString()); + UploadId id = new UploadId(path.getFileName().toString()); if (!isLocked(id)) { Files.deleteIfExists(path); @@ -70,7 +74,7 @@ public void cleanupStaleLocks() throws IOException { } @Override - public boolean isLocked(UUID id) { + public boolean isLocked(UploadId id) { boolean locked = false; Path lockPath = getLockPath(id); @@ -92,10 +96,11 @@ public boolean isLocked(UUID id) { @Override public void setIdFactory(UploadIdFactory idFactory) { + Validate.notNull(idFactory, "The IdFactory cannot be null"); this.idFactory = idFactory; } - private Path getLockPath(UUID id) { + private Path getLockPath(UploadId id) { return getPathInStorageDirectory(id); } diff --git a/src/main/java/me/desair/tus/server/upload/disk/DiskStorageService.java b/src/main/java/me/desair/tus/server/upload/disk/DiskStorageService.java index 59a0202..34f5627 100644 --- a/src/main/java/me/desair/tus/server/upload/disk/DiskStorageService.java +++ b/src/main/java/me/desair/tus/server/upload/disk/DiskStorageService.java @@ -17,11 +17,11 @@ import java.util.Collections; import java.util.List; import java.util.Objects; -import java.util.UUID; import me.desair.tus.server.exception.InvalidUploadOffsetException; import me.desair.tus.server.exception.TusException; import me.desair.tus.server.exception.UploadNotFoundException; +import me.desair.tus.server.upload.UploadId; import me.desair.tus.server.upload.UploadIdFactory; import me.desair.tus.server.upload.UploadInfo; import me.desair.tus.server.upload.UploadLockingService; @@ -51,15 +51,20 @@ public class DiskStorageService extends AbstractDiskBasedService implements Uplo private UploadIdFactory idFactory; private UploadConcatenationService uploadConcatenationService; - public DiskStorageService(UploadIdFactory idFactory, String storagePath) { + public DiskStorageService(String storagePath) { super(storagePath + File.separator + UPLOAD_SUB_DIRECTORY); + setUploadConcatenationService(new VirtualConcatenationService(this)); + } + + public DiskStorageService(UploadIdFactory idFactory, String storagePath) { + this(storagePath); Validate.notNull(idFactory, "The IdFactory cannot be null"); this.idFactory = idFactory; - setUploadConcatenationService(new VirtualConcatenationService(this)); } @Override public void setIdFactory(UploadIdFactory idFactory) { + Validate.notNull(idFactory, "The IdFactory cannot be null"); this.idFactory = idFactory; } @@ -84,7 +89,7 @@ public UploadInfo getUploadInfo(String uploadUrl, String ownerKey) throws IOExce } @Override - public UploadInfo getUploadInfo(UUID id) throws IOException { + public UploadInfo getUploadInfo(UploadId id) throws IOException { try { Path infoPath = getInfoPath(id); return Utils.readSerializable(infoPath, UploadInfo.class); @@ -100,7 +105,7 @@ public String getUploadURI() { @Override public UploadInfo create(UploadInfo info, String ownerKey) throws IOException { - UUID id = createNewId(); + UploadId id = createNewId(); createUploadDirectory(id); @@ -228,7 +233,7 @@ public UploadConcatenationService getUploadConcatenationService() { public InputStream getUploadedBytes(String uploadURI, String ownerKey) throws IOException, UploadNotFoundException { - UUID id = idFactory.readUploadId(uploadURI); + UploadId id = idFactory.readUploadId(uploadURI); UploadInfo uploadInfo = getUploadInfo(id); if (uploadInfo == null || !Objects.equals(uploadInfo.getOwnerKey(), ownerKey)) { @@ -239,7 +244,7 @@ public InputStream getUploadedBytes(String uploadURI, String ownerKey) } @Override - public InputStream getUploadedBytes(UUID id) throws IOException, UploadNotFoundException { + public InputStream getUploadedBytes(UploadId id) throws IOException, UploadNotFoundException { InputStream inputStream = null; UploadInfo uploadInfo = getUploadInfo(id); if (UploadType.CONCATENATED.equals(uploadInfo.getUploadType()) && uploadConcatenationService != null) { @@ -305,19 +310,19 @@ private List getUploads(UploadInfo info) throws IOException, UploadN return uploads; } - private Path getBytesPath(UUID id) throws UploadNotFoundException { + private Path getBytesPath(UploadId id) throws UploadNotFoundException { return getPathInUploadDir(id, DATA_FILE); } - private Path getInfoPath(UUID id) throws UploadNotFoundException { + private Path getInfoPath(UploadId id) throws UploadNotFoundException { return getPathInUploadDir(id, INFO_FILE); } - private Path createUploadDirectory(UUID id) throws IOException { + private Path createUploadDirectory(UploadId id) throws IOException { return Files.createDirectories(getPathInStorageDirectory(id)); } - private Path getPathInUploadDir(UUID id, String fileName) throws UploadNotFoundException { + private Path getPathInUploadDir(UploadId id, String fileName) throws UploadNotFoundException { //Get the upload directory Path uploadDir = getPathInStorageDirectory(id); if (uploadDir != null && Files.exists(uploadDir)) { @@ -327,8 +332,8 @@ private Path getPathInUploadDir(UUID id, String fileName) throws UploadNotFoundE } } - private synchronized UUID createNewId() throws IOException { - UUID id; + private synchronized UploadId createNewId() throws IOException { + UploadId id; do { id = idFactory.createId(); //For extra safety, double check that this ID is not in use yet diff --git a/src/main/java/me/desair/tus/server/upload/disk/ExpiredUploadFilter.java b/src/main/java/me/desair/tus/server/upload/disk/ExpiredUploadFilter.java index 3178532..f7c7536 100644 --- a/src/main/java/me/desair/tus/server/upload/disk/ExpiredUploadFilter.java +++ b/src/main/java/me/desair/tus/server/upload/disk/ExpiredUploadFilter.java @@ -4,8 +4,8 @@ import java.nio.file.DirectoryStream; import java.nio.file.Path; import java.util.Objects; -import java.util.UUID; +import me.desair.tus.server.upload.UploadId; import me.desair.tus.server.upload.UploadInfo; import me.desair.tus.server.upload.UploadLockingService; import org.slf4j.Logger; @@ -28,9 +28,9 @@ public class ExpiredUploadFilter implements DirectoryStream.Filter { @Override public boolean accept(Path upload) throws IOException { - UUID id = null; + UploadId id = null; try { - id = UUID.fromString(upload.getFileName().toString()); + id = new UploadId(upload.getFileName().toString()); UploadInfo info = diskStorageService.getUploadInfo(id); if (info != null && info.isExpired() && !uploadLockingService.isLocked(id)) { diff --git a/src/main/java/me/desair/tus/server/util/Utils.java b/src/main/java/me/desair/tus/server/util/Utils.java index dbe50e5..d1ce330 100644 --- a/src/main/java/me/desair/tus/server/util/Utils.java +++ b/src/main/java/me/desair/tus/server/util/Utils.java @@ -21,6 +21,7 @@ import javax.servlet.http.HttpServletRequest; +import me.desair.tus.server.HttpHeader; import org.apache.commons.lang3.StringUtils; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -50,6 +51,20 @@ public static Long getLongHeader(HttpServletRequest request, String header) { } } + /** + * Build a comma-separated list based on the remote address of the request and the + * X-Forwareded-For header. The list is constructed as "client, proxy1, proxy2". + * @return A comma-separated list of ip-addresses + */ + public static String buildRemoteIpList(HttpServletRequest servletRequest) { + String ipAddresses = servletRequest.getRemoteAddr(); + String xforwardedForHeader = getHeader(servletRequest, HttpHeader.X_FORWARDED_FOR); + if (xforwardedForHeader.length() > 0) { + ipAddresses = xforwardedForHeader + ", " + ipAddresses; + } + return ipAddresses; + } + public static List parseConcatenationIDsFromHeader(String uploadConcatValue) { List output = new LinkedList<>(); @@ -109,9 +124,13 @@ public static FileLock lockFileShared(FileChannel channel) throws IOException { return lockFile(channel, true); } - public static void sleep(long sleepTime) { + /** + * Sleep the specified number of milliseconds + * @param sleepTimeMillis The time to sleep in milliseconds + */ + public static void sleep(long sleepTimeMillis) { try { - Thread.sleep(sleepTime); + Thread.sleep(sleepTimeMillis); } catch (InterruptedException e) { log.warn("Sleep was interrupted"); // Restore interrupted state... @@ -134,4 +153,5 @@ private static FileLock lockFile(FileChannel channel, boolean shared) throws IOE return lock; } + } diff --git a/src/test/java/me/desair/tus/server/ITTusFileUploadService.java b/src/test/java/me/desair/tus/server/ITTusFileUploadService.java index b060f5e..79d4d18 100644 --- a/src/test/java/me/desair/tus/server/ITTusFileUploadService.java +++ b/src/test/java/me/desair/tus/server/ITTusFileUploadService.java @@ -61,8 +61,7 @@ public static void destroyDataFolder() throws IOException { @Before public void setUp() { - servletRequest = new MockHttpServletRequest(); - servletResponse = new MockHttpServletResponse(); + reset(); tusFileUploadService = new TusFileUploadService() .withUploadURI(UPLOAD_URI) .withStoragePath(storagePath.toAbsolutePath().toString()) @@ -72,6 +71,13 @@ public void setUp() { .withChunkedTransferDecoding(true); } + protected void reset() { + servletRequest = new MockHttpServletRequest(); + servletRequest.setRemoteAddr("192.168.1.1"); + servletRequest.addHeader(HttpHeader.X_FORWARDED_FOR, "10.0.2.1, 123.231.12.4"); + servletResponse = new MockHttpServletResponse(); + } + @Test public void testSupportedHttpMethods() { assertThat(tusFileUploadService.getSupportedHttpMethods(), containsInAnyOrder( @@ -193,6 +199,7 @@ public void testProcessCompleteUpload() throws Exception { hasEntry("filename", "world_domination_plan.pdf") ) ); + assertThat(info.getCreatorIpAddresses(), is("10.0.2.1, 123.231.12.4, 192.168.1.1")); //Try retrieving the uploaded bytes without owner key try { @@ -370,6 +377,7 @@ public void testProcessUploadTwoParts() throws Exception { hasEntry("filename", "world_domination_plan.pdf") ) ); + assertThat(info.getCreatorIpAddresses(), is("10.0.2.1, 123.231.12.4, 192.168.1.1")); //Make sure cleanup does not interfere with this test tusFileUploadService.cleanup(); @@ -428,6 +436,7 @@ public void testProcessUploadTwoParts() throws Exception { hasEntry("filename", "world_domination_plan.pdf") ) ); + assertThat(info.getCreatorIpAddresses(), is("10.0.2.1, 123.231.12.4, 192.168.1.1")); //Get uploaded bytes from service try (InputStream uploadedBytes = tusFileUploadService.getUploadedBytes(location, OWNER_KEY)) { @@ -489,6 +498,7 @@ public void testProcessUploadDeferredLength() throws Exception { hasEntry("filename", "world_domination_plan.pdf") ) ); + assertThat(info.getCreatorIpAddresses(), is("10.0.2.1, 123.231.12.4, 192.168.1.1")); //Make sure cleanup does not interfere with this test tusFileUploadService.cleanup(); @@ -929,6 +939,7 @@ public void testConcatenationCompleted() throws Exception { hasEntry("filename", "world_domination_map_concatenated.pdf") ) ); + assertThat(info.getCreatorIpAddresses(), is("10.0.2.1, 123.231.12.4, 192.168.1.1")); //Download the upload reset(); @@ -1353,9 +1364,4 @@ protected void assertResponseStatus(final int httpStatus) { assertThat(servletResponse.getStatus(), is(httpStatus)); } - protected void reset() { - servletRequest = new MockHttpServletRequest(); - servletResponse = new MockHttpServletResponse(); - } - } \ No newline at end of file diff --git a/src/test/java/me/desair/tus/server/ITTusFileUploadServiceCached.java b/src/test/java/me/desair/tus/server/ITTusFileUploadServiceCached.java index cf11d92..6bff86d 100644 --- a/src/test/java/me/desair/tus/server/ITTusFileUploadServiceCached.java +++ b/src/test/java/me/desair/tus/server/ITTusFileUploadServiceCached.java @@ -4,7 +4,8 @@ import static org.hamcrest.MatcherAssert.assertThat; import javax.servlet.http.HttpServletResponse; -import me.desair.tus.server.upload.UploadIdFactory; + +import me.desair.tus.server.upload.TimeBasedUploadIdFactory; import me.desair.tus.server.upload.UploadLockingService; import me.desair.tus.server.upload.UploadStorageService; import me.desair.tus.server.upload.cache.ThreadLocalCachedStorageAndLockingService; @@ -21,15 +22,16 @@ public class ITTusFileUploadServiceCached extends ITTusFileUploadService { @Before public void setUp() { super.setUp(); - tusFileUploadService = tusFileUploadService.withThreadLocalCache(true); + tusFileUploadService = tusFileUploadService + .withThreadLocalCache(true) + .withUploadIdFactory(new TimeBasedUploadIdFactory()); } @Test public void testProcessUploadDoubleCached() throws Exception { - UploadIdFactory idFactory = new UploadIdFactory(); String path = storagePath.toAbsolutePath().toString(); - UploadStorageService uploadStorageService = new DiskStorageService(idFactory, path); - UploadLockingService uploadLockingService = new DiskLockingService(idFactory, path); + UploadStorageService uploadStorageService = new DiskStorageService(path); + UploadLockingService uploadLockingService = new DiskLockingService(path); ThreadLocalCachedStorageAndLockingService service2 = new ThreadLocalCachedStorageAndLockingService( @@ -41,6 +43,9 @@ public void testProcessUploadDoubleCached() throws Exception { tusFileUploadService.withUploadStorageService(service2); tusFileUploadService.withUploadLockingService(service2); + assertThat(service2.getUploadURI(), is(UPLOAD_URI)); + assertThat(uploadStorageService.getUploadURI(), is(UPLOAD_URI)); + testConcatenationCompleted(); } diff --git a/src/test/java/me/desair/tus/server/concatenation/ConcatenationHeadRequestHandlerTest.java b/src/test/java/me/desair/tus/server/concatenation/ConcatenationHeadRequestHandlerTest.java index a52d30c..a64b3ec 100644 --- a/src/test/java/me/desair/tus/server/concatenation/ConcatenationHeadRequestHandlerTest.java +++ b/src/test/java/me/desair/tus/server/concatenation/ConcatenationHeadRequestHandlerTest.java @@ -14,6 +14,7 @@ import me.desair.tus.server.HttpHeader; import me.desair.tus.server.HttpMethod; +import me.desair.tus.server.upload.UploadId; import me.desair.tus.server.upload.UploadInfo; import me.desair.tus.server.upload.UploadStorageService; import me.desair.tus.server.upload.UploadType; @@ -66,7 +67,7 @@ public void supports() throws Exception { @Test public void testRegularUpload() throws Exception { UploadInfo info1 = new UploadInfo(); - info1.setId(UUID.randomUUID()); + info1.setId(new UploadId(UUID.randomUUID())); info1.setUploadConcatHeaderValue("Impossible"); info1.setUploadType(UploadType.REGULAR); @@ -83,7 +84,7 @@ public void testRegularUpload() throws Exception { @Test public void testPartialUpload() throws Exception { UploadInfo info1 = new UploadInfo(); - info1.setId(UUID.randomUUID()); + info1.setId(new UploadId(UUID.randomUUID())); info1.setUploadConcatHeaderValue("partial"); info1.setUploadType(UploadType.PARTIAL); @@ -100,7 +101,7 @@ public void testPartialUpload() throws Exception { @Test public void testConcatenatedUploadWithLength() throws Exception { UploadInfo info1 = new UploadInfo(); - info1.setId(UUID.randomUUID()); + info1.setId(new UploadId(UUID.randomUUID())); info1.setUploadConcatHeaderValue("final; 123 456"); info1.setLength(10L); info1.setOffset(10L); @@ -123,7 +124,7 @@ public void testConcatenatedUploadWithLength() throws Exception { @Test public void testConcatenatedUploadWithoutLength() throws Exception { UploadInfo info1 = new UploadInfo(); - info1.setId(UUID.randomUUID()); + info1.setId(new UploadId(UUID.randomUUID())); info1.setUploadConcatHeaderValue("final; 123 456"); info1.setLength(10L); info1.setOffset(8L); diff --git a/src/test/java/me/desair/tus/server/concatenation/ConcatenationPostRequestHandlerTest.java b/src/test/java/me/desair/tus/server/concatenation/ConcatenationPostRequestHandlerTest.java index 967772d..622efd1 100644 --- a/src/test/java/me/desair/tus/server/concatenation/ConcatenationPostRequestHandlerTest.java +++ b/src/test/java/me/desair/tus/server/concatenation/ConcatenationPostRequestHandlerTest.java @@ -15,6 +15,7 @@ import me.desair.tus.server.HttpHeader; import me.desair.tus.server.HttpMethod; +import me.desair.tus.server.upload.UploadId; import me.desair.tus.server.upload.UploadInfo; import me.desair.tus.server.upload.UploadStorageService; import me.desair.tus.server.upload.UploadType; @@ -69,7 +70,7 @@ public void testRegularUpload() throws Exception { TusServletResponse response = new TusServletResponse(this.servletResponse); UploadInfo info1 = new UploadInfo(); - info1.setId(UUID.randomUUID()); + info1.setId(new UploadId(UUID.randomUUID())); when(uploadStorageService.getUploadInfo(eq(info1.getId().toString()), nullable(String.class))).thenReturn(info1); @@ -89,7 +90,7 @@ public void testPartialUpload() throws Exception { TusServletResponse response = new TusServletResponse(this.servletResponse); UploadInfo info1 = new UploadInfo(); - info1.setId(UUID.randomUUID()); + info1.setId(new UploadId(UUID.randomUUID())); when(uploadStorageService.getUploadInfo(eq(info1.getId().toString()), nullable(String.class))).thenReturn(info1); @@ -110,7 +111,7 @@ public void testFinalUpload() throws Exception { TusServletResponse response = new TusServletResponse(this.servletResponse); UploadInfo info1 = new UploadInfo(); - info1.setId(UUID.randomUUID()); + info1.setId(new UploadId(UUID.randomUUID())); when(uploadStorageService.getUploadInfo(eq(info1.getId().toString()), nullable(String.class))).thenReturn(info1); diff --git a/src/test/java/me/desair/tus/server/concatenation/validation/PartialUploadsExistValidatorTest.java b/src/test/java/me/desair/tus/server/concatenation/validation/PartialUploadsExistValidatorTest.java index fa2275e..1d23023 100644 --- a/src/test/java/me/desair/tus/server/concatenation/validation/PartialUploadsExistValidatorTest.java +++ b/src/test/java/me/desair/tus/server/concatenation/validation/PartialUploadsExistValidatorTest.java @@ -9,6 +9,7 @@ import me.desair.tus.server.HttpHeader; import me.desair.tus.server.HttpMethod; import me.desair.tus.server.exception.InvalidPartialUploadIdException; +import me.desair.tus.server.upload.UploadId; import me.desair.tus.server.upload.UploadInfo; import me.desair.tus.server.upload.UploadStorageService; import org.junit.Before; @@ -49,10 +50,10 @@ public void supports() throws Exception { @Test public void testValid() throws Exception { UploadInfo info1 = new UploadInfo(); - info1.setId(UUID.randomUUID()); + info1.setId(new UploadId(UUID.randomUUID())); UploadInfo info2 = new UploadInfo(); - info2.setId(UUID.randomUUID()); + info2.setId(new UploadId(UUID.randomUUID())); when(uploadStorageService.getUploadInfo(info1.getId().toString(), null)).thenReturn(info1); when(uploadStorageService.getUploadInfo(info2.getId().toString(), null)).thenReturn(info2); @@ -68,7 +69,7 @@ public void testValid() throws Exception { @Test(expected = InvalidPartialUploadIdException.class) public void testInvalidUploadNotFound() throws Exception { UploadInfo info1 = new UploadInfo(); - info1.setId(UUID.randomUUID()); + info1.setId(new UploadId(UUID.randomUUID())); when(uploadStorageService.getUploadInfo(info1.getId())).thenReturn(info1); @@ -82,7 +83,7 @@ public void testInvalidUploadNotFound() throws Exception { @Test(expected = InvalidPartialUploadIdException.class) public void testInvalidId() throws Exception { UploadInfo info1 = new UploadInfo(); - info1.setId(UUID.randomUUID()); + info1.setId(new UploadId(UUID.randomUUID())); when(uploadStorageService.getUploadInfo(info1.getId().toString(), null)).thenReturn(info1); diff --git a/src/test/java/me/desair/tus/server/concatenation/validation/PatchFinalUploadValidatorTest.java b/src/test/java/me/desair/tus/server/concatenation/validation/PatchFinalUploadValidatorTest.java index 8711475..6ba9d85 100644 --- a/src/test/java/me/desair/tus/server/concatenation/validation/PatchFinalUploadValidatorTest.java +++ b/src/test/java/me/desair/tus/server/concatenation/validation/PatchFinalUploadValidatorTest.java @@ -11,6 +11,7 @@ import me.desair.tus.server.HttpMethod; import me.desair.tus.server.exception.PatchOnFinalUploadNotAllowedException; +import me.desair.tus.server.upload.UploadId; import me.desair.tus.server.upload.UploadInfo; import me.desair.tus.server.upload.UploadStorageService; import me.desair.tus.server.upload.UploadType; @@ -52,15 +53,15 @@ public void supports() throws Exception { @Test public void testValid() throws Exception { UploadInfo info1 = new UploadInfo(); - info1.setId(UUID.randomUUID()); + info1.setId(new UploadId(UUID.randomUUID())); info1.setUploadType(UploadType.REGULAR); UploadInfo info2 = new UploadInfo(); - info2.setId(UUID.randomUUID()); + info2.setId(new UploadId(UUID.randomUUID())); info2.setUploadType(UploadType.PARTIAL); UploadInfo info3 = new UploadInfo(); - info3.setId(UUID.randomUUID()); + info3.setId(new UploadId(UUID.randomUUID())); info3.setUploadType(null); when(uploadStorageService.getUploadInfo(eq(info1.getId().toString()), @@ -101,7 +102,7 @@ public void testValidNotFound() throws Exception { @Test(expected = PatchOnFinalUploadNotAllowedException.class) public void testInvalidFinal() throws Exception { UploadInfo info1 = new UploadInfo(); - info1.setId(UUID.randomUUID()); + info1.setId(new UploadId(UUID.randomUUID())); info1.setUploadType(UploadType.CONCATENATED); when(uploadStorageService.getUploadInfo(eq(info1.getId().toString()), diff --git a/src/test/java/me/desair/tus/server/core/CorePatchRequestHandlerTest.java b/src/test/java/me/desair/tus/server/core/CorePatchRequestHandlerTest.java index 04a3a17..af9a1af 100644 --- a/src/test/java/me/desair/tus/server/core/CorePatchRequestHandlerTest.java +++ b/src/test/java/me/desair/tus/server/core/CorePatchRequestHandlerTest.java @@ -18,6 +18,7 @@ import me.desair.tus.server.HttpHeader; import me.desair.tus.server.HttpMethod; import me.desair.tus.server.exception.UploadNotFoundException; +import me.desair.tus.server.upload.UploadId; import me.desair.tus.server.upload.UploadInfo; import me.desair.tus.server.upload.UploadStorageService; import me.desair.tus.server.util.TusServletRequest; @@ -64,7 +65,7 @@ public void supports() throws Exception { @Test public void processInProgress() throws Exception { UploadInfo info = new UploadInfo(); - info.setId(UUID.randomUUID()); + info.setId(new UploadId(UUID.randomUUID())); info.setOffset(2L); info.setLength(10L); when(uploadStorageService.getUploadInfo(nullable(String.class), nullable(String.class))).thenReturn(info); @@ -88,7 +89,7 @@ public void processInProgress() throws Exception { @Test public void processFinished() throws Exception { UploadInfo info = new UploadInfo(); - info.setId(UUID.randomUUID()); + info.setId(new UploadId(UUID.randomUUID())); info.setOffset(10L); info.setLength(10L); when(uploadStorageService.getUploadInfo(nullable(String.class), nullable(String.class))).thenReturn(info); @@ -117,7 +118,7 @@ public void processNotFound() throws Exception { @Test public void processAppendNotFound() throws Exception { UploadInfo info = new UploadInfo(); - info.setId(UUID.randomUUID()); + info.setId(new UploadId(UUID.randomUUID())); info.setOffset(10L); info.setLength(8L); when(uploadStorageService.getUploadInfo(nullable(String.class), nullable(String.class))).thenReturn(info); diff --git a/src/test/java/me/desair/tus/server/creation/CreationPatchRequestHandlerTest.java b/src/test/java/me/desair/tus/server/creation/CreationPatchRequestHandlerTest.java index 3e3d7d2..1763adc 100644 --- a/src/test/java/me/desair/tus/server/creation/CreationPatchRequestHandlerTest.java +++ b/src/test/java/me/desair/tus/server/creation/CreationPatchRequestHandlerTest.java @@ -17,6 +17,7 @@ import me.desair.tus.server.HttpHeader; import me.desair.tus.server.HttpMethod; import me.desair.tus.server.exception.UploadNotFoundException; +import me.desair.tus.server.upload.UploadId; import me.desair.tus.server.upload.UploadInfo; import me.desair.tus.server.upload.UploadStorageService; import me.desair.tus.server.util.TusServletRequest; @@ -133,7 +134,7 @@ public void processNotFound() throws Exception { @Test public void processAppendNotFound() throws Exception { UploadInfo info = new UploadInfo(); - info.setId(UUID.randomUUID()); + info.setId(new UploadId(UUID.randomUUID())); info.setOffset(10L); when(uploadStorageService.getUploadInfo(nullable(String.class), nullable(String.class))).thenReturn(info); diff --git a/src/test/java/me/desair/tus/server/creation/CreationPostRequestHandlerTest.java b/src/test/java/me/desair/tus/server/creation/CreationPostRequestHandlerTest.java index aa4cc2b..2ff05c9 100644 --- a/src/test/java/me/desair/tus/server/creation/CreationPostRequestHandlerTest.java +++ b/src/test/java/me/desair/tus/server/creation/CreationPostRequestHandlerTest.java @@ -15,6 +15,7 @@ import me.desair.tus.server.HttpHeader; import me.desair.tus.server.HttpMethod; +import me.desair.tus.server.upload.UploadId; import me.desair.tus.server.upload.UploadInfo; import me.desair.tus.server.upload.UploadStorageService; import me.desair.tus.server.util.TusServletRequest; @@ -71,7 +72,7 @@ public void processWithLengthAndMetadata() throws Exception { servletRequest.addHeader(HttpHeader.UPLOAD_LENGTH, 10L); servletRequest.addHeader(HttpHeader.UPLOAD_METADATA, "encoded-metadata"); - final UUID id = UUID.randomUUID(); + final UploadId id = new UploadId(UUID.randomUUID()); when(uploadStorageService.create(ArgumentMatchers.any(UploadInfo.class), nullable(String.class))).then( new Answer() { @Override @@ -101,7 +102,7 @@ public void processWithLengthAndNoMetadata() throws Exception { servletRequest.addHeader(HttpHeader.UPLOAD_LENGTH, 10L); //servletRequest.addHeader(HttpHeader.UPLOAD_METADATA, null); - final UUID id = UUID.randomUUID(); + final UploadId id = new UploadId(UUID.randomUUID()); when(uploadStorageService.create(ArgumentMatchers.any(UploadInfo.class), nullable(String.class))).then( new Answer() { @Override @@ -131,7 +132,7 @@ public void processWithNoLengthAndMetadata() throws Exception { //servletRequest.addHeader(HttpHeader.UPLOAD_LENGTH, null); servletRequest.addHeader(HttpHeader.UPLOAD_METADATA, "encoded-metadata"); - final UUID id = UUID.randomUUID(); + final UploadId id = new UploadId(UUID.randomUUID()); when(uploadStorageService.create(ArgumentMatchers.any(UploadInfo.class), nullable(String.class))).then( new Answer() { @Override @@ -161,7 +162,7 @@ public void processWithNoLengthAndNoMetadata() throws Exception { //servletRequest.addHeader(HttpHeader.UPLOAD_LENGTH, null); //servletRequest.addHeader(HttpHeader.UPLOAD_METADATA, null); - final UUID id = UUID.randomUUID(); + final UploadId id = new UploadId(UUID.randomUUID()); when(uploadStorageService.create(ArgumentMatchers.any(UploadInfo.class), nullable(String.class))).then( new Answer() { @Override diff --git a/src/test/java/me/desair/tus/server/creation/ITCreationExtension.java b/src/test/java/me/desair/tus/server/creation/ITCreationExtension.java index 7634487..3aeb3ba 100644 --- a/src/test/java/me/desair/tus/server/creation/ITCreationExtension.java +++ b/src/test/java/me/desair/tus/server/creation/ITCreationExtension.java @@ -20,6 +20,7 @@ import me.desair.tus.server.exception.InvalidUploadLengthException; import me.desair.tus.server.exception.MaxUploadLengthExceededException; import me.desair.tus.server.exception.PostOnInvalidRequestURIException; +import me.desair.tus.server.upload.UploadId; import me.desair.tus.server.upload.UploadInfo; import org.junit.Before; import org.junit.Test; @@ -37,7 +38,7 @@ public class ITCreationExtension extends AbstractTusExtensionIntegrationTest { //that sit in front of the web app private static final String UPLOAD_URL = UPLOAD_URI + "/"; - private UUID id; + private UploadId id; @Before public void setUp() throws Exception { @@ -46,7 +47,7 @@ public void setUp() throws Exception { tusFeature = new CreationExtension(); uploadInfo = null; - id = UUID.randomUUID(); + id = new UploadId(UUID.randomUUID()); servletRequest.setRequestURI(UPLOAD_URI); reset(uploadStorageService); when(uploadStorageService.getUploadURI()).thenReturn(UPLOAD_URI); diff --git a/src/test/java/me/desair/tus/server/download/DownloadGetRequestHandlerTest.java b/src/test/java/me/desair/tus/server/download/DownloadGetRequestHandlerTest.java index 98e06de..53e0662 100644 --- a/src/test/java/me/desair/tus/server/download/DownloadGetRequestHandlerTest.java +++ b/src/test/java/me/desair/tus/server/download/DownloadGetRequestHandlerTest.java @@ -17,6 +17,7 @@ import me.desair.tus.server.HttpHeader; import me.desair.tus.server.HttpMethod; import me.desair.tus.server.exception.UploadInProgressException; +import me.desair.tus.server.upload.UploadId; import me.desair.tus.server.upload.UploadInfo; import me.desair.tus.server.upload.UploadStorageService; import me.desair.tus.server.util.TusServletRequest; @@ -62,7 +63,7 @@ public void supports() throws Exception { @Test public void testWithCompletedUploadWithMetadata() throws Exception { - final UUID id = UUID.randomUUID(); + final UploadId id = new UploadId(UUID.randomUUID()); UploadInfo info = new UploadInfo(); info.setId(id); @@ -88,7 +89,7 @@ public void testWithCompletedUploadWithMetadata() throws Exception { @Test public void testWithCompletedUploadWithoutMetadata() throws Exception { - final UUID id = UUID.randomUUID(); + final UploadId id = new UploadId(UUID.randomUUID()); UploadInfo info = new UploadInfo(); info.setId(id); @@ -110,7 +111,7 @@ public void testWithCompletedUploadWithoutMetadata() throws Exception { @Test(expected = UploadInProgressException.class) public void testWithInProgressUpload() throws Exception { - final UUID id = UUID.randomUUID(); + final UploadId id = new UploadId(UUID.randomUUID()); UploadInfo info = new UploadInfo(); info.setId(id); diff --git a/src/test/java/me/desair/tus/server/termination/TerminationDeleteRequestHandlerTest.java b/src/test/java/me/desair/tus/server/termination/TerminationDeleteRequestHandlerTest.java index b98cc23..8713c97 100644 --- a/src/test/java/me/desair/tus/server/termination/TerminationDeleteRequestHandlerTest.java +++ b/src/test/java/me/desair/tus/server/termination/TerminationDeleteRequestHandlerTest.java @@ -14,6 +14,7 @@ import javax.servlet.http.HttpServletResponse; import me.desair.tus.server.HttpMethod; +import me.desair.tus.server.upload.UploadId; import me.desair.tus.server.upload.UploadInfo; import me.desair.tus.server.upload.UploadStorageService; import me.desair.tus.server.util.TusServletRequest; @@ -70,7 +71,7 @@ public void testWithNotExistingUpload() throws Exception { @Test public void testWithExistingUpload() throws Exception { - final UUID id = UUID.randomUUID(); + final UploadId id = new UploadId(UUID.randomUUID()); UploadInfo info = new UploadInfo(); info.setId(id); diff --git a/src/test/java/me/desair/tus/server/upload/TimeBasedUploadIdFactoryTest.java b/src/test/java/me/desair/tus/server/upload/TimeBasedUploadIdFactoryTest.java new file mode 100644 index 0000000..0904455 --- /dev/null +++ b/src/test/java/me/desair/tus/server/upload/TimeBasedUploadIdFactoryTest.java @@ -0,0 +1,113 @@ +package me.desair.tus.server.upload; + +import static org.hamcrest.Matchers.greaterThan; +import static org.hamcrest.Matchers.hasToString; +import static org.hamcrest.Matchers.is; +import static org.hamcrest.Matchers.lessThan; +import static org.hamcrest.Matchers.not; +import static org.hamcrest.Matchers.nullValue; +import static org.junit.Assert.assertThat; + +import me.desair.tus.server.util.Utils; +import org.junit.Before; +import org.junit.Test; + +public class TimeBasedUploadIdFactoryTest { + + private UploadIdFactory idFactory; + + @Before + public void setUp() { + idFactory = new TimeBasedUploadIdFactory(); + } + + @Test(expected = NullPointerException.class) + public void setUploadURINull() throws Exception { + idFactory.setUploadURI(null); + } + + @Test + public void setUploadURINoTrailingSlash() throws Exception { + idFactory.setUploadURI("/test/upload"); + assertThat(idFactory.getUploadURI(), is("/test/upload")); + } + + @Test + public void setUploadURIWithTrailingSlash() throws Exception { + idFactory.setUploadURI("/test/upload/"); + assertThat(idFactory.getUploadURI(), is("/test/upload/")); + } + + @Test(expected = IllegalArgumentException.class) + public void setUploadURIBlank() throws Exception { + idFactory.setUploadURI(" "); + } + + @Test(expected = IllegalArgumentException.class) + public void setUploadURINoStartingSlash() throws Exception { + idFactory.setUploadURI("test/upload/"); + } + + @Test(expected = IllegalArgumentException.class) + public void setUploadURIEndsWithDollar() throws Exception { + idFactory.setUploadURI("/test/upload$"); + } + + @Test + public void readUploadId() throws Exception { + idFactory.setUploadURI("/test/upload"); + + assertThat(idFactory.readUploadId("/test/upload/1546152320043"), + hasToString("1546152320043")); + } + + @Test + public void readUploadIdRegex() throws Exception { + idFactory.setUploadURI("/users/[0-9]+/files/upload"); + + assertThat(idFactory.readUploadId("/users/1337/files/upload/1546152320043"), + hasToString("1546152320043")); + } + + @Test + public void readUploadIdTrailingSlash() throws Exception { + idFactory.setUploadURI("/test/upload/"); + + assertThat(idFactory.readUploadId("/test/upload/1546152320043"), + hasToString("1546152320043")); + } + + @Test + public void readUploadIdRegexTrailingSlash() throws Exception { + idFactory.setUploadURI("/users/[0-9]+/files/upload/"); + + assertThat(idFactory.readUploadId("/users/123456789/files/upload/1546152320043"), + hasToString("1546152320043")); + } + + @Test + public void readUploadIdNoUUID() throws Exception { + idFactory.setUploadURI("/test/upload"); + + assertThat(idFactory.readUploadId("/test/upload/not-a-time-value"), is(nullValue())); + } + + @Test + public void readUploadIdRegexNoMatch() throws Exception { + idFactory.setUploadURI("/users/[0-9]+/files/upload"); + + assertThat(idFactory.readUploadId("/users/files/upload/1546152320043"), + is(nullValue())); + } + + @Test + public void createId() throws Exception { + UploadId id = idFactory.createId(); + assertThat(id, not(nullValue())); + Utils.sleep(10); + assertThat(Long.parseLong(id.getOriginalObject().toString()), + greaterThan(System.currentTimeMillis() - 1000L)); + assertThat(Long.parseLong(id.getOriginalObject().toString()), + lessThan(System.currentTimeMillis())); + } +} \ No newline at end of file diff --git a/src/test/java/me/desair/tus/server/upload/UploadIdFactoryTest.java b/src/test/java/me/desair/tus/server/upload/UUIDUploadIdFactoryTest.java similarity index 97% rename from src/test/java/me/desair/tus/server/upload/UploadIdFactoryTest.java rename to src/test/java/me/desair/tus/server/upload/UUIDUploadIdFactoryTest.java index 9510a6f..e7dd0a5 100644 --- a/src/test/java/me/desair/tus/server/upload/UploadIdFactoryTest.java +++ b/src/test/java/me/desair/tus/server/upload/UUIDUploadIdFactoryTest.java @@ -9,13 +9,13 @@ import org.junit.Before; import org.junit.Test; -public class UploadIdFactoryTest { +public class UUIDUploadIdFactoryTest { private UploadIdFactory idFactory; @Before public void setUp() { - idFactory = new UploadIdFactory(); + idFactory = new UUIDUploadIdFactory(); } @Test(expected = NullPointerException.class) diff --git a/src/test/java/me/desair/tus/server/upload/UploadIdTest.java b/src/test/java/me/desair/tus/server/upload/UploadIdTest.java new file mode 100644 index 0000000..6253cd9 --- /dev/null +++ b/src/test/java/me/desair/tus/server/upload/UploadIdTest.java @@ -0,0 +1,85 @@ +package me.desair.tus.server.upload; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertNotEquals; + +import java.util.UUID; + +import org.junit.Test; + +public class UploadIdTest { + + @Test + public void getOriginalObjectUUID() { + UUID id = UUID.randomUUID(); + UploadId uploadId = new UploadId(id); + assertEquals(id.toString(), uploadId.toString()); + assertEquals(id, uploadId.getOriginalObject()); + } + + @Test + public void getOriginalObjectLong() { + UploadId uploadId = new UploadId(1337L); + assertEquals("1337", uploadId.toString()); + assertEquals(1337L, uploadId.getOriginalObject()); + } + + @Test(expected = NullPointerException.class) + public void testNullConstructor() { + new UploadId(null); + } + + @Test(expected = IllegalArgumentException.class) + public void testBlankConstructor() { + new UploadId(" \t"); + } + + @Test + public void toStringNotYetUrlSafe() { + UploadId uploadId = new UploadId("my test id/1"); + assertEquals("my+test+id%2F1", uploadId.toString()); + } + + @Test + public void toStringNotYetUrlSafe2() { + UploadId uploadId = new UploadId("id+%2F1+/+1"); + assertEquals("id+%2F1+/+1", uploadId.toString()); + } + + @Test + public void toStringAlreadyUrlSafe() { + UploadId uploadId = new UploadId("my+test+id%2F1"); + assertEquals("my+test+id%2F1", uploadId.toString()); + } + + @Test + public void toStringWithInternalDecoderException() { + String test = "Invalid % value"; + UploadId id = new UploadId(test); + assertEquals("Invalid % value", id.toString()); + } + + @Test + public void equalsSameUrlSafeValue() { + UploadId id1 = new UploadId("id%2F1"); + UploadId id2 = new UploadId("id/1"); + UploadId id3 = new UploadId("id/1"); + + assertEquals(id1, id2); + assertEquals(id2, id3); + assertEquals(id1, id1); + assertNotEquals(id1, null); + assertFalse(id1.equals(UUID.randomUUID())); + } + + @Test + public void hashCodeSameUrlSafeValue() { + UploadId id1 = new UploadId("id%2F1"); + UploadId id2 = new UploadId("id/1"); + UploadId id3 = new UploadId("id/1"); + + assertEquals(id1.hashCode(), id2.hashCode()); + assertEquals(id2.hashCode(), id3.hashCode()); + } +} \ No newline at end of file diff --git a/src/test/java/me/desair/tus/server/upload/UploadInfoTest.java b/src/test/java/me/desair/tus/server/upload/UploadInfoTest.java index ef5435e..ca695e4 100644 --- a/src/test/java/me/desair/tus/server/upload/UploadInfoTest.java +++ b/src/test/java/me/desair/tus/server/upload/UploadInfoTest.java @@ -3,6 +3,9 @@ import static me.desair.tus.server.util.MapMatcher.hasSize; import static org.hamcrest.CoreMatchers.allOf; import static org.hamcrest.CoreMatchers.is; +import static org.hamcrest.CoreMatchers.nullValue; +import static org.hamcrest.Matchers.greaterThan; +import static org.hamcrest.Matchers.lessThan; import static org.hamcrest.collection.IsMapContaining.hasEntry; import static org.junit.Assert.assertFalse; import static org.junit.Assert.assertThat; @@ -12,8 +15,11 @@ import java.util.Stack; import java.util.UUID; +import me.desair.tus.server.HttpHeader; +import me.desair.tus.server.util.Utils; import org.apache.commons.lang3.time.DateFormatUtils; import org.junit.Test; +import org.springframework.mock.web.MockHttpServletRequest; public class UploadInfoTest { @@ -119,37 +125,37 @@ public void testEquals() throws Exception { info1.setLength(10L); info1.setOffset(5L); info1.setEncodedMetadata("Encoded-Metadata"); - info1.setId(UUID.fromString("1911e8a4-6939-490c-b58b-a5d70f8d91fb")); + info1.setId(new UploadId("1911e8a4-6939-490c-b58b-a5d70f8d91fb")); UploadInfo info2 = new UploadInfo(); info2.setLength(10L); info2.setOffset(5L); info2.setEncodedMetadata("Encoded-Metadata"); - info2.setId(UUID.fromString("1911e8a4-6939-490c-b58b-a5d70f8d91fb")); + info2.setId(new UploadId("1911e8a4-6939-490c-b58b-a5d70f8d91fb")); UploadInfo info3 = new UploadInfo(); info3.setLength(9L); info3.setOffset(5L); info3.setEncodedMetadata("Encoded-Metadata"); - info3.setId(UUID.fromString("1911e8a4-6939-490c-b58b-a5d70f8d91fb")); + info3.setId(new UploadId("1911e8a4-6939-490c-b58b-a5d70f8d91fb")); UploadInfo info4 = new UploadInfo(); info4.setLength(10L); info4.setOffset(6L); info4.setEncodedMetadata("Encoded-Metadata"); - info4.setId(UUID.fromString("1911e8a4-6939-490c-b58b-a5d70f8d91fb")); + info4.setId(new UploadId("1911e8a4-6939-490c-b58b-a5d70f8d91fb")); UploadInfo info5 = new UploadInfo(); info5.setLength(10L); info5.setOffset(5L); info5.setEncodedMetadata("Encoded-Metadatas"); - info5.setId(UUID.fromString("1911e8a4-6939-490c-b58b-a5d70f8d91fb")); + info5.setId(new UploadId("1911e8a4-6939-490c-b58b-a5d70f8d91fb")); UploadInfo info6 = new UploadInfo(); info6.setLength(10L); info6.setOffset(5L); info6.setEncodedMetadata("Encoded-Metadata"); - info6.setId(UUID.fromString("1911e8a4-6939-490c-c58b-a5d70f8d91fb")); + info6.setId(new UploadId("1911e8a4-6939-490c-c58b-a5d70f8d91fb")); assertTrue(info1.equals(info1)); assertTrue(info1.equals(info2)); @@ -167,13 +173,13 @@ public void testHashCode() throws Exception { info1.setLength(10L); info1.setOffset(5L); info1.setEncodedMetadata("Encoded-Metadata"); - info1.setId(UUID.fromString("1911e8a4-6939-490c-b58b-a5d70f8d91fb")); + info1.setId(new UploadId("1911e8a4-6939-490c-b58b-a5d70f8d91fb")); UploadInfo info2 = new UploadInfo(); info2.setLength(10L); info2.setOffset(5L); info2.setEncodedMetadata("Encoded-Metadata"); - info2.setId(UUID.fromString("1911e8a4-6939-490c-b58b-a5d70f8d91fb")); + info2.setId(new UploadId("1911e8a4-6939-490c-b58b-a5d70f8d91fb")); assertTrue(info1.hashCode() == info2.hashCode()); } @@ -190,7 +196,7 @@ public void testGetNameAndTypeWithMetadata() throws Exception { @Test public void testGetNameAndTypeWithoutMetadata() throws Exception { UploadInfo info = new UploadInfo(); - final UUID id = UUID.randomUUID(); + final UploadId id = new UploadId(UUID.randomUUID()); info.setId(id); assertThat(info.getFileName(), is(id.toString())); @@ -216,8 +222,12 @@ protected long getCurrentTime() { assertFalse(info2.isExpired()); final Stack dateStack = new Stack<>(); + //Current time stamp to check expiration dateStack.push(DateFormatUtils.ISO_8601_EXTENDED_DATETIME_FORMAT.parse("2018-01-23T10:43:11").getTime()); + //Current time stamp to calculate expiration dateStack.push(DateFormatUtils.ISO_8601_EXTENDED_DATETIME_FORMAT.parse("2018-01-20T10:43:11").getTime()); + //Creation time stamp + dateStack.push(DateFormatUtils.ISO_8601_EXTENDED_DATETIME_FORMAT.parse("2018-01-20T10:40:39").getTime()); UploadInfo info3 = new UploadInfo() { @Override @@ -229,4 +239,37 @@ protected long getCurrentTime() { assertTrue(info3.isExpired()); } + @Test + public void testGetCreationTimestamp() throws Exception { + UploadInfo info = new UploadInfo(); + Utils.sleep(10); + + assertThat(info.getCreationTimestamp(), greaterThan(System.currentTimeMillis() - 500L)); + assertThat(info.getCreationTimestamp(), lessThan(System.currentTimeMillis())); + } + + @Test + public void testGetCreatorIpAddressesNull() throws Exception { + UploadInfo info = new UploadInfo(); + assertThat(info.getCreatorIpAddresses(), nullValue()); + } + + @Test + public void testGetCreatorIpAddressesWithoutXForwardedFor() throws Exception { + MockHttpServletRequest servletRequest = new MockHttpServletRequest(); + servletRequest.setRemoteAddr("10.11.12.13"); + + UploadInfo info = new UploadInfo(servletRequest); + assertThat(info.getCreatorIpAddresses(), is("10.11.12.13")); + } + + @Test + public void testGetCreatorIpAddressesWithXForwardedFor() throws Exception { + MockHttpServletRequest servletRequest = new MockHttpServletRequest(); + servletRequest.setRemoteAddr("10.11.12.13"); + servletRequest.addHeader(HttpHeader.X_FORWARDED_FOR, "24.23.22.21, 192.168.1.1"); + + UploadInfo info = new UploadInfo(servletRequest); + assertThat(info.getCreatorIpAddresses(), is("24.23.22.21, 192.168.1.1, 10.11.12.13")); + } } \ No newline at end of file diff --git a/src/test/java/me/desair/tus/server/upload/concatenation/UploadInputStreamEnumerationTest.java b/src/test/java/me/desair/tus/server/upload/concatenation/UploadInputStreamEnumerationTest.java index 4678de4..0fee31a 100644 --- a/src/test/java/me/desair/tus/server/upload/concatenation/UploadInputStreamEnumerationTest.java +++ b/src/test/java/me/desair/tus/server/upload/concatenation/UploadInputStreamEnumerationTest.java @@ -11,6 +11,7 @@ import java.util.LinkedList; import java.util.UUID; +import me.desair.tus.server.upload.UploadId; import me.desair.tus.server.upload.UploadInfo; import me.desair.tus.server.upload.UploadStorageService; import org.apache.commons.io.IOUtils; @@ -28,13 +29,13 @@ public class UploadInputStreamEnumerationTest { @Test public void hasMoreElements() throws Exception { UploadInfo info1 = new UploadInfo(); - info1.setId(UUID.randomUUID()); + info1.setId(new UploadId(UUID.randomUUID())); UploadInfo info2 = new UploadInfo(); - info2.setId(UUID.randomUUID()); + info2.setId(new UploadId(UUID.randomUUID())); UploadInfo info3 = new UploadInfo(); - info3.setId(UUID.randomUUID()); + info3.setId(new UploadId(UUID.randomUUID())); when(uploadStorageService.getUploadedBytes(info1.getId())) .thenReturn(IOUtils.toInputStream("Upload 1", StandardCharsets.UTF_8)); @@ -58,13 +59,13 @@ public void hasMoreElements() throws Exception { @Test public void hasMoreElementsException() throws Exception { UploadInfo info1 = new UploadInfo(); - info1.setId(UUID.randomUUID()); + info1.setId(new UploadId(UUID.randomUUID())); UploadInfo info2 = new UploadInfo(); - info2.setId(UUID.randomUUID()); + info2.setId(new UploadId(UUID.randomUUID())); UploadInfo info3 = new UploadInfo(); - info3.setId(UUID.randomUUID()); + info3.setId(new UploadId(UUID.randomUUID())); when(uploadStorageService.getUploadedBytes(info1.getId())) .thenReturn(IOUtils.toInputStream("Upload 1", StandardCharsets.UTF_8)); @@ -86,13 +87,13 @@ public void hasMoreElementsException() throws Exception { @Test public void hasMoreElementsNotFound() throws Exception { UploadInfo info1 = new UploadInfo(); - info1.setId(UUID.randomUUID()); + info1.setId(new UploadId(UUID.randomUUID())); UploadInfo info2 = new UploadInfo(); - info2.setId(UUID.randomUUID()); + info2.setId(new UploadId(UUID.randomUUID())); UploadInfo info3 = new UploadInfo(); - info3.setId(UUID.randomUUID()); + info3.setId(new UploadId(UUID.randomUUID())); when(uploadStorageService.getUploadedBytes(info1.getId())) .thenReturn(IOUtils.toInputStream("Upload 1", StandardCharsets.UTF_8)); @@ -114,10 +115,10 @@ public void hasMoreElementsNotFound() throws Exception { @Test public void hasMoreElementsNullElement() throws Exception { UploadInfo info1 = new UploadInfo(); - info1.setId(UUID.randomUUID()); + info1.setId(new UploadId(UUID.randomUUID())); UploadInfo info3 = new UploadInfo(); - info3.setId(UUID.randomUUID()); + info3.setId(new UploadId(UUID.randomUUID())); when(uploadStorageService.getUploadedBytes(info1.getId())) .thenReturn(IOUtils.toInputStream("Upload 1", StandardCharsets.UTF_8)); diff --git a/src/test/java/me/desair/tus/server/upload/concatenation/VirtualConcatenationServiceTest.java b/src/test/java/me/desair/tus/server/upload/concatenation/VirtualConcatenationServiceTest.java index 6484862..c6676ef 100644 --- a/src/test/java/me/desair/tus/server/upload/concatenation/VirtualConcatenationServiceTest.java +++ b/src/test/java/me/desair/tus/server/upload/concatenation/VirtualConcatenationServiceTest.java @@ -14,6 +14,7 @@ import java.util.UUID; import me.desair.tus.server.exception.UploadNotFoundException; +import me.desair.tus.server.upload.UploadId; import me.desair.tus.server.upload.UploadInfo; import me.desair.tus.server.upload.UploadStorageService; import org.apache.commons.io.IOUtils; @@ -40,18 +41,18 @@ public void setUp() { @Test public void merge() throws Exception { UploadInfo child1 = new UploadInfo(); - child1.setId(UUID.randomUUID()); + child1.setId(new UploadId(UUID.randomUUID())); child1.setLength(5L); child1.setOffset(5L); UploadInfo child2 = new UploadInfo(); - child2.setId(UUID.randomUUID()); + child2.setId(new UploadId(UUID.randomUUID())); child2.setLength(10L); child2.setOffset(10L); UploadInfo infoParent = new UploadInfo(); - infoParent.setId(UUID.randomUUID()); - infoParent.setConcatenationParts(Arrays.asList(child1.getId().toString(), child2.getId().toString())); + infoParent.setId(new UploadId(UUID.randomUUID())); + infoParent.setConcatenationPartIds(Arrays.asList(child1.getId().toString(), child2.getId().toString())); when(uploadStorageService.getUploadInfo(child1.getId().toString(), infoParent.getOwnerKey())) .thenReturn(child1); @@ -72,18 +73,18 @@ public void merge() throws Exception { @Test public void mergeNotCompleted() throws Exception { UploadInfo child1 = new UploadInfo(); - child1.setId(UUID.randomUUID()); + child1.setId(new UploadId(UUID.randomUUID())); child1.setLength(5L); child1.setOffset(5L); UploadInfo child2 = new UploadInfo(); - child2.setId(UUID.randomUUID()); + child2.setId(new UploadId(UUID.randomUUID())); child2.setLength(10L); child2.setOffset(8L); UploadInfo infoParent = new UploadInfo(); - infoParent.setId(UUID.randomUUID()); - infoParent.setConcatenationParts(Arrays.asList(child1.getId().toString(), child2.getId().toString())); + infoParent.setId(new UploadId(UUID.randomUUID())); + infoParent.setConcatenationPartIds(Arrays.asList(child1.getId().toString(), child2.getId().toString())); when(uploadStorageService.getUploadInfo(child1.getId().toString(), infoParent.getOwnerKey())) .thenReturn(child1); @@ -104,18 +105,18 @@ public void mergeNotCompleted() throws Exception { @Test public void mergeWithoutLength() throws Exception { UploadInfo child1 = new UploadInfo(); - child1.setId(UUID.randomUUID()); + child1.setId(new UploadId(UUID.randomUUID())); child1.setLength(null); child1.setOffset(5L); UploadInfo child2 = new UploadInfo(); - child2.setId(UUID.randomUUID()); + child2.setId(new UploadId(UUID.randomUUID())); child2.setLength(null); child2.setOffset(8L); UploadInfo infoParent = new UploadInfo(); - infoParent.setId(UUID.randomUUID()); - infoParent.setConcatenationParts(Arrays.asList(child1.getId().toString(), child2.getId().toString())); + infoParent.setId(new UploadId(UUID.randomUUID())); + infoParent.setConcatenationPartIds(Arrays.asList(child1.getId().toString(), child2.getId().toString())); when(uploadStorageService.getUploadInfo(child1.getId().toString(), infoParent.getOwnerKey())) .thenReturn(child1); @@ -136,18 +137,18 @@ public void mergeWithoutLength() throws Exception { @Test(expected = UploadNotFoundException.class) public void mergeNotFound() throws Exception { UploadInfo child1 = new UploadInfo(); - child1.setId(UUID.randomUUID()); + child1.setId(new UploadId(UUID.randomUUID())); child1.setLength(5L); child1.setOffset(5L); UploadInfo child2 = new UploadInfo(); - child2.setId(UUID.randomUUID()); + child2.setId(new UploadId(UUID.randomUUID())); child2.setLength(10L); child2.setOffset(10L); UploadInfo infoParent = new UploadInfo(); - infoParent.setId(UUID.randomUUID()); - infoParent.setConcatenationParts(Arrays.asList(child1.getId().toString(), child2.getId().toString())); + infoParent.setId(new UploadId(UUID.randomUUID())); + infoParent.setConcatenationPartIds(Arrays.asList(child1.getId().toString(), child2.getId().toString())); when(uploadStorageService.getUploadInfo(child1.getId().toString(), infoParent.getOwnerKey())) .thenReturn(child1); @@ -162,18 +163,18 @@ public void mergeNotFound() throws Exception { @Test public void mergeWithExpiration() throws Exception { UploadInfo child1 = new UploadInfo(); - child1.setId(UUID.randomUUID()); + child1.setId(new UploadId(UUID.randomUUID())); child1.setLength(5L); child1.setOffset(5L); UploadInfo child2 = new UploadInfo(); - child2.setId(UUID.randomUUID()); + child2.setId(new UploadId(UUID.randomUUID())); child2.setLength(10L); child2.setOffset(8L); UploadInfo infoParent = new UploadInfo(); - infoParent.setId(UUID.randomUUID()); - infoParent.setConcatenationParts(Arrays.asList(child1.getId().toString(), child2.getId().toString())); + infoParent.setId(new UploadId(UUID.randomUUID())); + infoParent.setConcatenationPartIds(Arrays.asList(child1.getId().toString(), child2.getId().toString())); when(uploadStorageService.getUploadInfo(child1.getId().toString(), infoParent.getOwnerKey())) .thenReturn(child1); @@ -204,8 +205,8 @@ public void mergeWithExpiration() throws Exception { @Test public void getUploadsEmptyFinal() throws Exception { UploadInfo infoParent = new UploadInfo(); - infoParent.setId(UUID.randomUUID()); - infoParent.setConcatenationParts(null); + infoParent.setId(new UploadId(UUID.randomUUID())); + infoParent.setConcatenationPartIds(null); when(uploadStorageService.getUploadInfo(infoParent.getId().toString(), infoParent.getOwnerKey())) .thenReturn(infoParent); @@ -225,18 +226,18 @@ public void getConcatenatedBytes() throws Exception { String upload2 = "concatenated upload!"; UploadInfo child1 = new UploadInfo(); - child1.setId(UUID.randomUUID()); + child1.setId(new UploadId(UUID.randomUUID())); child1.setLength((long) upload1.getBytes().length); child1.setOffset((long) upload1.getBytes().length); UploadInfo child2 = new UploadInfo(); - child2.setId(UUID.randomUUID()); + child2.setId(new UploadId(UUID.randomUUID())); child2.setLength((long) upload2.getBytes().length); child2.setOffset((long) upload2.getBytes().length); UploadInfo infoParent = new UploadInfo(); - infoParent.setId(UUID.randomUUID()); - infoParent.setConcatenationParts(Arrays.asList(child1.getId().toString(), child2.getId().toString())); + infoParent.setId(new UploadId(UUID.randomUUID())); + infoParent.setConcatenationPartIds(Arrays.asList(child1.getId().toString(), child2.getId().toString())); when(uploadStorageService.getUploadInfo(child1.getId().toString(), infoParent.getOwnerKey())) .thenReturn(child1); @@ -260,18 +261,18 @@ public void getConcatenatedBytesNotComplete() throws Exception { String upload2 = "concatenated upload!"; UploadInfo child1 = new UploadInfo(); - child1.setId(UUID.randomUUID()); + child1.setId(new UploadId(UUID.randomUUID())); child1.setLength((long) upload1.getBytes().length); child1.setOffset((long) upload1.getBytes().length - 2); UploadInfo child2 = new UploadInfo(); - child2.setId(UUID.randomUUID()); + child2.setId(new UploadId(UUID.randomUUID())); child2.setLength((long) upload2.getBytes().length); child2.setOffset((long) upload2.getBytes().length - 2); UploadInfo infoParent = new UploadInfo(); - infoParent.setId(UUID.randomUUID()); - infoParent.setConcatenationParts(Arrays.asList(child1.getId().toString(), child2.getId().toString())); + infoParent.setId(new UploadId(UUID.randomUUID())); + infoParent.setConcatenationPartIds(Arrays.asList(child1.getId().toString(), child2.getId().toString())); when(uploadStorageService.getUploadInfo(child1.getId().toString(), infoParent.getOwnerKey())) .thenReturn(child1); @@ -295,18 +296,18 @@ public void getConcatenatedBytesNotFound() throws Exception { String upload2 = "concatenated upload!"; UploadInfo child1 = new UploadInfo(); - child1.setId(UUID.randomUUID()); + child1.setId(new UploadId(UUID.randomUUID())); child1.setLength((long) upload1.getBytes().length); child1.setOffset((long) upload1.getBytes().length - 2); UploadInfo child2 = new UploadInfo(); - child2.setId(UUID.randomUUID()); + child2.setId(new UploadId(UUID.randomUUID())); child2.setLength((long) upload2.getBytes().length); child2.setOffset((long) upload2.getBytes().length - 2); UploadInfo infoParent = new UploadInfo(); - infoParent.setId(UUID.randomUUID()); - infoParent.setConcatenationParts(Arrays.asList(child1.getId().toString(), child2.getId().toString())); + infoParent.setId(new UploadId(UUID.randomUUID())); + infoParent.setConcatenationPartIds(Arrays.asList(child1.getId().toString(), child2.getId().toString())); when(uploadStorageService.getUploadInfo(child1.getId().toString(), infoParent.getOwnerKey())) .thenReturn(child1); diff --git a/src/test/java/me/desair/tus/server/upload/disk/DiskLockingServiceTest.java b/src/test/java/me/desair/tus/server/upload/disk/DiskLockingServiceTest.java index d723eb0..a8c6b81 100644 --- a/src/test/java/me/desair/tus/server/upload/disk/DiskLockingServiceTest.java +++ b/src/test/java/me/desair/tus/server/upload/disk/DiskLockingServiceTest.java @@ -17,6 +17,7 @@ import java.nio.file.attribute.FileTime; import java.util.UUID; +import me.desair.tus.server.upload.UploadId; import me.desair.tus.server.upload.UploadIdFactory; import me.desair.tus.server.upload.UploadLock; import org.apache.commons.io.FileUtils; @@ -57,11 +58,11 @@ public static void destroyDataFolder() throws IOException { public void setUp() { reset(idFactory); when(idFactory.getUploadURI()).thenReturn(UPLOAD_URL); - when(idFactory.createId()).thenReturn(UUID.randomUUID()); - when(idFactory.readUploadId(nullable(String.class))).then(new Answer() { + when(idFactory.createId()).thenReturn(new UploadId(UUID.randomUUID())); + when(idFactory.readUploadId(nullable(String.class))).then(new Answer() { @Override - public UUID answer(InvocationOnMock invocation) throws Throwable { - return UUID.fromString(StringUtils.substringAfter(invocation.getArguments()[0].toString(), + public UploadId answer(InvocationOnMock invocation) throws Throwable { + return new UploadId(StringUtils.substringAfter(invocation.getArguments()[0].toString(), UPLOAD_URL + "/")); } }); @@ -82,7 +83,7 @@ public void lockUploadByUri() throws Exception { public void isLockedTrue() throws Exception { UploadLock uploadLock = lockingService.lockUploadByUri("/upload/test/000003f1-a850-49de-af03-997272d834c9"); - assertThat(lockingService.isLocked(UUID.fromString("000003f1-a850-49de-af03-997272d834c9")), is(true)); + assertThat(lockingService.isLocked(new UploadId("000003f1-a850-49de-af03-997272d834c9")), is(true)); uploadLock.release(); } @@ -92,7 +93,7 @@ public void isLockedFalse() throws Exception { UploadLock uploadLock = lockingService.lockUploadByUri("/upload/test/000003f1-a850-49de-af03-997272d834c9"); uploadLock.release(); - assertThat(lockingService.isLocked(UUID.fromString("000003f1-a850-49de-af03-997272d834c9")), is(false)); + assertThat(lockingService.isLocked(new UploadId("000003f1-a850-49de-af03-997272d834c9")), is(false)); } @Test diff --git a/src/test/java/me/desair/tus/server/upload/disk/DiskStorageServiceTest.java b/src/test/java/me/desair/tus/server/upload/disk/DiskStorageServiceTest.java index 97e7368..f9ed2cd 100644 --- a/src/test/java/me/desair/tus/server/upload/disk/DiskStorageServiceTest.java +++ b/src/test/java/me/desair/tus/server/upload/disk/DiskStorageServiceTest.java @@ -27,6 +27,7 @@ import me.desair.tus.server.exception.InvalidUploadOffsetException; import me.desair.tus.server.exception.UploadNotFoundException; +import me.desair.tus.server.upload.UploadId; import me.desair.tus.server.upload.UploadIdFactory; import me.desair.tus.server.upload.UploadInfo; import me.desair.tus.server.upload.UploadLockingService; @@ -73,11 +74,11 @@ public static void destroyDataFolder() throws IOException { public void setUp() { reset(idFactory); when(idFactory.getUploadURI()).thenReturn(UPLOAD_URL); - when(idFactory.createId()).thenReturn(UUID.randomUUID()); - when(idFactory.readUploadId(nullable(String.class))).then(new Answer() { + when(idFactory.createId()).thenReturn(new UploadId(UUID.randomUUID())); + when(idFactory.readUploadId(nullable(String.class))).then(new Answer() { @Override - public UUID answer(InvocationOnMock invocation) throws Throwable { - return UUID.fromString(StringUtils.substringAfter(invocation.getArguments()[0].toString(), + public UploadId answer(InvocationOnMock invocation) throws Throwable { + return new UploadId(StringUtils.substringAfter(invocation.getArguments()[0].toString(), UPLOAD_URL + "/")); } }); @@ -127,7 +128,7 @@ public void getUploadInfoById() throws Exception { info.setLength(10L); info.setEncodedMetadata("Encoded Metadata"); - info = storageService.create(info, null); + info = storageService.create(info, "John"); assertTrue(Files.exists(getUploadInfoPath(info.getId()))); @@ -138,11 +139,14 @@ public void getUploadInfoById() throws Exception { assertThat(readInfo.getOffset(), is(0L)); assertThat(readInfo.getLength(), is(10L)); assertThat(readInfo.getEncodedMetadata(), is("Encoded Metadata")); + assertThat(readInfo.getCreationTimestamp(), is(info.getCreationTimestamp())); + assertThat(readInfo.getUploadType(), is(info.getUploadType())); + assertThat(readInfo.getOwnerKey(), is(info.getOwnerKey())); } @Test public void getUploadInfoByFakeId() throws Exception { - UploadInfo readInfo = storageService.getUploadInfo(UUID.randomUUID()); + UploadInfo readInfo = storageService.getUploadInfo(new UploadId(UUID.randomUUID())); assertThat(readInfo, is(nullValue())); } @@ -300,7 +304,7 @@ public void appendOnFakeUpload() throws Exception { //Create our fake upload UploadInfo info = new UploadInfo(); - info.setId(UUID.randomUUID()); + info.setId(new UploadId(UUID.randomUUID())); info.setLength((long) (content.getBytes().length)); //Write the content of the upload @@ -472,7 +476,7 @@ public void terminateInProgressUpload() throws Exception { @Test public void cleanupExpiredUploads() throws Exception { - when(uploadLockingService.isLocked(any(UUID.class))).thenReturn(false); + when(uploadLockingService.isLocked(any(UploadId.class))).thenReturn(false); String content = "This is the content of my upload"; @@ -491,15 +495,15 @@ public void cleanupExpiredUploads() throws Exception { assertFalse(Files.exists(getStoragePath(info.getId()))); } - private Path getUploadInfoPath(UUID id) { + private Path getUploadInfoPath(UploadId id) { return getStoragePath(id).resolve("info"); } - private Path getUploadDataPath(UUID id) { + private Path getUploadDataPath(UploadId id) { return getStoragePath(id).resolve("data"); } - private Path getStoragePath(UUID id) { + private Path getStoragePath(UploadId id) { return storagePath.resolve("uploads").resolve(id.toString()); } } \ No newline at end of file diff --git a/src/test/java/me/desair/tus/server/upload/disk/ExpiredUploadFilterTest.java b/src/test/java/me/desair/tus/server/upload/disk/ExpiredUploadFilterTest.java index 0f839ad..e10cb6b 100644 --- a/src/test/java/me/desair/tus/server/upload/disk/ExpiredUploadFilterTest.java +++ b/src/test/java/me/desair/tus/server/upload/disk/ExpiredUploadFilterTest.java @@ -11,6 +11,7 @@ import java.text.ParseException; import java.util.UUID; +import me.desair.tus.server.upload.UploadId; import me.desair.tus.server.upload.UploadInfo; import me.desair.tus.server.upload.UploadLockingService; import org.apache.commons.lang3.time.DateFormatUtils; @@ -39,7 +40,7 @@ public void setUp() { @Test public void accept() throws Exception { UploadInfo info = createExpiredUploadInfo(); - info.setId(UUID.randomUUID()); + info.setId(new UploadId(UUID.randomUUID())); info.setOffset(2L); info.setLength(10L); info.updateExpiration(100L); @@ -52,8 +53,8 @@ public void accept() throws Exception { @Test public void acceptNotFound() throws Exception { - when(diskStorageService.getUploadInfo(any(UUID.class))).thenReturn(null); - when(uploadLockingService.isLocked(any(UUID.class))).thenReturn(false); + when(diskStorageService.getUploadInfo(any(UploadId.class))).thenReturn(null); + when(uploadLockingService.isLocked(any(UploadId.class))).thenReturn(false); assertFalse(uploadFilter.accept(Paths.get(UUID.randomUUID().toString()))); } @@ -61,7 +62,7 @@ public void acceptNotFound() throws Exception { @Test public void acceptCompletedUpload() throws Exception { UploadInfo info = createExpiredUploadInfo(); - info.setId(UUID.randomUUID()); + info.setId(new UploadId(UUID.randomUUID())); info.setOffset(10L); info.setLength(10L); info.updateExpiration(100L); @@ -76,7 +77,7 @@ public void acceptCompletedUpload() throws Exception { @Test public void acceptInProgressButNotExpired() throws Exception { UploadInfo info = new UploadInfo(); - info.setId(UUID.randomUUID()); + info.setId(new UploadId(UUID.randomUUID())); info.setOffset(2L); info.setLength(10L); info.updateExpiration(172800000L); @@ -90,7 +91,7 @@ public void acceptInProgressButNotExpired() throws Exception { @Test public void acceptLocked() throws Exception { UploadInfo info = createExpiredUploadInfo(); - info.setId(UUID.randomUUID()); + info.setId(new UploadId(UUID.randomUUID())); info.setOffset(8L); info.setLength(10L); info.updateExpiration(100L); @@ -104,7 +105,7 @@ public void acceptLocked() throws Exception { @Test public void acceptException() throws Exception { UploadInfo info = createExpiredUploadInfo(); - info.setId(UUID.randomUUID()); + info.setId(new UploadId(UUID.randomUUID())); info.setOffset(8L); info.setLength(10L); info.updateExpiration(100L); diff --git a/src/test/resources/simplelogger.properties b/src/test/resources/simplelogger.properties new file mode 100644 index 0000000..cf99f09 --- /dev/null +++ b/src/test/resources/simplelogger.properties @@ -0,0 +1,34 @@ +# SLF4J's SimpleLogger configuration file +# Simple implementation of Logger that sends all enabled log messages, for all defined loggers, to System.err. + +# Default logging detail level for all instances of SimpleLogger. +# Must be one of ("trace", "debug", "info", "warn", or "error"). +# If not specified, defaults to "info". +org.slf4j.simpleLogger.defaultLogLevel=info + +# Logging detail level for a SimpleLogger instance named "xxxxx". +# Must be one of ("trace", "debug", "info", "warn", or "error"). +# If not specified, the default logging detail level is used. +#org.slf4j.simpleLogger.log.xxxxx= + +# Set to true if you want the current date and time to be included in output messages. +# Default is false, and will output the number of milliseconds elapsed since startup. +org.slf4j.simpleLogger.showDateTime=true + +# The date and time format to be used in the output messages. +# The pattern describing the date and time format is the same that is used in java.text.SimpleDateFormat. +# If the format is not specified or is invalid, the default format is used. +# The default format is yyyy-MM-dd HH:mm:ss:SSS Z. +#org.slf4j.simpleLogger.dateTimeFormat=yyyy-MM-dd HH:mm:ss:SSS Z + +# Set to true if you want to output the current thread name. +# Defaults to true. +#org.slf4j.simpleLogger.showThreadName=true + +# Set to true if you want the Logger instance name to be included in output messages. +# Defaults to true. +org.slf4j.simpleLogger.showLogName=true + +# Set to true if you want the last component of the name to be included in output messages. +# Defaults to false. +org.slf4j.simpleLogger.showShortLogName=true \ No newline at end of file