[jclouds/jclouds] JCLOUDS-847: Poor upload performance for putBlob (#1274)

[dordoriko <no...@github.com>]

This change improves the performance of writing to sockets with the
default Java URL connection HTTP client, by enlarging the buffer used
for socket writes from an implicit hard-coded 4KB / 8KB buffer to a
configurable 32KB buffer.

The buffer size is now controlled by the following property with the
following default value:

jclouds.output-socket-buffer-size: 32768

The implementation is based on a variant of ByteStreams.copy (written as
ByteStreams2.copy) which accepts the buffer size as an argument, unlike
the original Guava code that uses a hard-coded size.

The change was done directly within the loop that copies the input
stream to the output stream, and not by wrapping a BufferedOutputStream
around the existing output stream, in order to avoid copying the payload
twice.

On some platforms this change can improve both the putBlob throughput
and the total CPU consumption.
You can view, comment on, or merge this pull request online at:

  https://github.com/jclouds/jclouds/pull/1274

-- Commit Summary --

  * JCLOUDS-847: Poor upload performance for putBlob

-- File Changes --

    M core/src/main/java/org/jclouds/Constants.java (9)
    M core/src/main/java/org/jclouds/apis/internal/BaseApiMetadata.java (2)
    M core/src/main/java/org/jclouds/http/internal/JavaUrlHttpCommandExecutorService.java (8)
    M core/src/main/java/org/jclouds/io/ByteStreams2.java (23)
    M core/src/test/java/org/jclouds/http/internal/TrackingJavaUrlHttpCommandExecutorService.java (4)
    M providers/dynect/src/main/java/org/jclouds/dynect/v3/config/DynECTHttpApiModule.java (4)
    M providers/profitbricks/src/main/java/org/jclouds/profitbricks/http/ResponseStatusFromPayloadHttpCommandExecutorService.java (4)

-- Patch Links --

https://github.com/jclouds/jclouds/pull/1274.patch
https://github.com/jclouds/jclouds/pull/1274.diff

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/1274

Re: [jclouds/jclouds] JCLOUDS-847: Poor upload performance for putBlob (#1274)

[Ignasi Barrera <no...@github.com>]

nacx commented on this pull request.

LGTM
We have migrated to the Apache GitHub org. Mind closing this PR and reopening again here? https://github.com/apache/jclouds

> @@ -325,7 +325,14 @@
     * </code>
     */
    public static final String PROPERTY_TIMEOUTS_PREFIX = "jclouds.timeouts.";
-   
+
+   /**
+    * Integer property. Default (32768).
+    * <p/>
+    * Buffer size for socket write.

Explain here that this is only considered int he default java http driver

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/1274#pullrequestreview-230145896

Re: [jclouds/jclouds] JCLOUDS-847: Poor upload performance for putBlob (#1274)

[dordoriko <no...@github.com>]

closing, reopened as https://github.com/apache/jclouds/pull/26 on the correct project (apache/jclouds).

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/1274#issuecomment-486344685

Re: [jclouds/jclouds] JCLOUDS-847: Poor upload performance for putBlob (#1274)

[dordoriko <no...@github.com>]

Closed #1274.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/1274#event-2297944124

Re: [jclouds/jclouds] JCLOUDS-847: Poor upload performance for putBlob (#1274)

[dordoriko <no...@github.com>]

dordoriko commented on this pull request.



> @@ -325,7 +325,14 @@
     * </code>
     */
    public static final String PROPERTY_TIMEOUTS_PREFIX = "jclouds.timeouts.";
-   
+
+   /**
+    * Integer property. Default (32768).
+    * <p/>
+    * Buffer size for socket write.

closing this PR, reopened as https://github.com/apache/jclouds/pull/26 - thanks!

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/1274#discussion_r278237606