You are viewing a plain text version of this content. The canonical link for it is here.
Posted to oak-commits@jackrabbit.apache.org by ma...@apache.org on 2019/12/03 23:36:25 UTC
svn commit: r1870774 - in /jackrabbit/oak/trunk:
oak-doc/src/site/markdown/features/direct-binary-access.md
oak-jackrabbit-api/src/main/java/org/apache/jackrabbit/api/binary/BinaryUpload.java
Author: mattryan
Date: Tue Dec 3 23:36:25 2019
New Revision: 1870774
URL: http://svn.apache.org/viewvc?rev=1870774&view=rev
Log:
OAK-8695: Clarify direct upload doc to indicate that not all URIs must be used.
Modified:
jackrabbit/oak/trunk/oak-doc/src/site/markdown/features/direct-binary-access.md
jackrabbit/oak/trunk/oak-jackrabbit-api/src/main/java/org/apache/jackrabbit/api/binary/BinaryUpload.java
Modified: jackrabbit/oak/trunk/oak-doc/src/site/markdown/features/direct-binary-access.md
URL: http://svn.apache.org/viewvc/jackrabbit/oak/trunk/oak-doc/src/site/markdown/features/direct-binary-access.md?rev=1870774&r1=1870773&r2=1870774&view=diff
==============================================================================
--- jackrabbit/oak/trunk/oak-doc/src/site/markdown/features/direct-binary-access.md (original)
+++ jackrabbit/oak/trunk/oak-doc/src/site/markdown/features/direct-binary-access.md Tue Dec 3 23:36:25 2019
@@ -117,7 +117,7 @@ Also note that clients should always che
The direct binary upload process is split into 3 phases:
1. **Initialize:** A remote client makes request to the Jackrabbit-based application to request an upload, which calls `initiateBinaryUpload(long, int)` and returns the resulting information to the remote client.
-2. **Upload:** The remote client performs the actual binary upload directly to the binary storage provider. The BinaryUpload returned from the previous call to `initiateBinaryUpload(long, int)` contains detailed instructions on how to complete the upload successfully. For more information, see the `BinaryUpload` documentation.
+2. **Upload:** The remote client performs the actual binary upload directly to the binary storage provider. The `BinaryUpload` returned from the previous call to `initiateBinaryUpload(long, int)` contains a list of URIs that can be used to upload. Clients use these URIs to upload; however, it is not required for clients to use all the URIs in the list. The number of URIs that must be used is subject to the size of the binary being uploaded and the minimum and maximum sizes returned in the `BinaryUpload` object, which constitutes detailed instructions on how to complete the upload successfully. For more information, see the `BinaryUpload` documentation.
3. **Complete:** The remote client notifies the Jackrabbit-based application that step 2 is complete. The upload token returned in the first step (obtained by calling `BinaryUpload.getUploadToken()`) is passed by the client to `completeBinaryUpload(String)`. This will provide the application with a regular JCR Binary that can then be used to write JCR content including the binary (such as an `nt:file` structure) and persist it.
Modified: jackrabbit/oak/trunk/oak-jackrabbit-api/src/main/java/org/apache/jackrabbit/api/binary/BinaryUpload.java
URL: http://svn.apache.org/viewvc/jackrabbit/oak/trunk/oak-jackrabbit-api/src/main/java/org/apache/jackrabbit/api/binary/BinaryUpload.java?rev=1870774&r1=1870773&r2=1870774&view=diff
==============================================================================
--- jackrabbit/oak/trunk/oak-jackrabbit-api/src/main/java/org/apache/jackrabbit/api/binary/BinaryUpload.java (original)
+++ jackrabbit/oak/trunk/oak-jackrabbit-api/src/main/java/org/apache/jackrabbit/api/binary/BinaryUpload.java Tue Dec 3 23:36:25 2019
@@ -77,19 +77,49 @@ import org.osgi.annotation.versioning.Pr
* <h3>Steps</h3>
* <ol>
* <li>
- * If {@code (fileSize / maxPartSize) > numUploadURIs}, then the
- * client cannot proceed and will have to request a new set of URIs
- * with the right fileSize as {@code maxSize}
- * </li>
- * <li>
- * If {@code fileSize < minPartSize}, then take the first provided
- * upload URI to upload the entire binary, with
- * {@code partSize = fileSize}
- * </li>
- * <li>
- * (optional) If the client has more information to optimize, the
- * {@code partSize} can be chosen, under the condition that all of these are
- * true:
+ * If {@code (fileSize / maxPartSize) > numUploadURIs}, then the client
+ * cannot proceed and will have to request a new set of URIs with the
+ * right fileSize as {@code maxSize}.
+ * </li>
+ * <li>
+ * Calculate the {@code partSize} and the number of URIs to use.
+ * <br>
+ * The easiest way to do this is to use the {@code maxPartSize} as the
+ * value for {@code partSize}. As long as the size of the actual binary
+ * upload is less than or equal to the size passed to
+ * {@link JackrabbitValueFactory#initiateBinaryUpload(long, int)}, a
+ * non-null BinaryUpload object returned from that call means you are
+ * guaranteed to be able to upload the binary successfully, using the
+ * provided {@code uploadURIs}, so long as the value you use for
+ * {@code partSize} is {@code maxPartSize}.
+ * Note that it is not required to use of all the URIs provided in
+ * {@code uploadURIs} if not all URIs are required to upload the entire
+ * binary with the selected {@code partSize}.
+ * <br>
+ * However, there are some exceptions to consider:
+ * <ol>
+ * <li>
+ * If {@code fileSize < minPartSize}, then take the first
+ * provided upload URI to upload the entire binary, with
+ * {@code partSize = fileSize}. Note that it is not required to
+ * use all of the URIs provided in {@code uploadURIs}.
+ * </li>
+ * <li>
+ * If {@code fileSize / partSize == numUploadURIs}, all part
+ * URIs must to be used. The {@code partSize} to use for all
+ * parts except the last would be calculated using:
+ * <pre>partSize = (fileSize + numUploadURIs - 1) / numUploadURIs</pre>
+ * It is also possible to simply use {@code maxPartSize} as the
+ * value for {@code partSize} in this case, for every part
+ * except the last.
+ * </li>
+ * </ol>
+ * Optionally, a client may select a different {@code partSize},
+ * for example if the client has more information about the
+ * conditions of the network or other information that would
+ * make a different {@code partSize} preferable. In this case a
+ * different value may be chosen, under the condition that all
+ * of the following are true:
* <ol>
* <li>{@code partSize >= minPartSize}</li>
* <li>{@code partSize <= maxPartSize}
@@ -98,23 +128,35 @@ import org.osgi.annotation.versioning.Pr
* </ol>
* </li>
* <li>
- * Otherwise all part URIs are to be used. The {@code partSize}
- * to use for all parts except the last would be calculated using:
- * <pre>partSize = (fileSize + numUploadURIs - 1) / numUploadURIs</pre>
- * </li>
- * <li>
- * Upload: segment the binary into {@code partSize}, for each segment take the
- * next URI from {@code uploadURIs} (strictly in order), proceed with a standard
- * HTTP PUT for each, and for the last part use whatever segment size is left
+ * Upload: segment the binary into {@code partSize}, for each segment
+ * take the next URI from {@code uploadURIs} (strictly in order),
+ * proceed with a standard HTTP PUT for each, and for the last part use
+ * whatever segment size is left.
* </li>
* <li>
- * If a segment fails during upload, retry (up to a certain timeout)
+ * If a segment fails during upload, retry (up to a certain timeout).
* </li>
* <li>
* After the upload has finished successfully, notify the application,
* for example through a complete request, passing the {@link
* #getUploadToken() upload token}, and the application will call {@link
- * JackrabbitValueFactory#completeBinaryUpload(String)} with the token
+ * JackrabbitValueFactory#completeBinaryUpload(String)} with the token.
+ * <br>
+ * The only timeout restrictions for calling
+ * {@link JackrabbitValueFactory#completeBinaryUpload(String)} are those
+ * imposed by the cloud blob storage service on uploaded blocks. Upload
+ * tokens themselves do not time out, which allows you to be very
+ * lenient in allowing uploads to complete, and very resilient in
+ * handling temporary network issues or other issues that might impact
+ * the uploading of one or more blocks.
+ * <br>
+ * In the case that the upload cannot be finished (for example, one or
+ * more segments cannot be uploaded even after a reasonable number of
+ * retries), do not call
+ * {@link JackrabbitValueFactory#completeBinaryUpload(String)}.
+ * Instead, simply restart the upload from the beginning by calling
+ * {@link JackrabbitValueFactory#initiateBinaryUpload(long, int)} when
+ * the situation preventing a successful upload has been resolved.
* </li>
* </ol>
*