You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@jclouds.apache.org by Zack Shoylev <no...@github.com> on 2016/06/20 20:39:20 UTC

[jclouds/jclouds] Changes the upload behavior to parallel, completing a TODO (#975)

You can view, comment on, or merge this pull request online at:

  https://github.com/jclouds/jclouds/pull/975

-- Commit Summary --

  * Changes the upload behavior to parallel, a TODO

-- File Changes --

    M apis/openstack-swift/src/main/java/org/jclouds/openstack/swift/v1/blobstore/RegionScopedSwiftBlobStore.java (56)

-- Patch Links --

https://github.com/jclouds/jclouds/pull/975.patch
https://github.com/jclouds/jclouds/pull/975.diff

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/975

Re: [jclouds/jclouds] Changes the upload behavior to parallel, completing a TODO (#975)

Posted by Andrew Gaul <no...@github.com>.
> @@ -44,6 +47,16 @@ public PutOptions(boolean multipart) {
>        this.multipart = multipart;
>     }
>  
> +   public PutOptions(boolean multipart, ExecutorService customExecutor) {

Do we need this variant?

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/975/files/0b784d15fa6f35ceb82028e1823e85c31ccf513d#r68281936

Re: [jclouds/jclouds] Changes the upload behavior to parallel, completing a TODO (#975)

Posted by Andrew Gaul <no...@github.com>.
>           ++partNumber;
> +         BlobUploader b =
> +               new BlobUploader(mpu, partNumber, payload, parts);
> +         executor.execute(b);
> +      }
> +      executor.shutdown();
> +      try {
> +         executor.awaitTermination(Integer.MAX_VALUE, TimeUnit.DAYS);
> +      } catch (InterruptedException e) {
> +         e.printStackTrace();
> +      }
> +
> +      return completeMultipartUpload(mpu, new ArrayList( parts ));
> +   }
> +
> +   private class BlobUploader implements Runnable {

Use `AutoValue` and make `static`?

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/975/files/70ba6e0f5aa5c660cab27eb6a450ca7f1e435235#r67774608

Re: [jclouds/jclouds] Changes the upload behavior to parallel, completing a TODO (#975)

Posted by Andrew Gaul <no...@github.com>.
>           ++partNumber;
> +         BlobUploader b =
> +               new BlobUploader(mpu, partNumber, payload, parts);
> +         executor.execute(b);
> +      }
> +      executor.shutdown();

For a while we have talked about geting away from jclouds managing threads; we want the contexts to be more lightweight and default to caller context/single-thread execution.  For performance-critical code like threaded multipart upload and clear container we can either take an explicit `ExecutorService` or push it into the `ContextBuilder`.  The latter is more discoverable but feels more clunky, but in the end the user should manage the lifetime.

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/975/files/70ba6e0f5aa5c660cab27eb6a450ca7f1e435235#r67949555

Re: [jclouds/jclouds] Changes the upload behavior to parallel, completing a TODO (#975)

Posted by Andrew Gaul <no...@github.com>.
> @@ -579,22 +590,79 @@ public long countBlobs(String containerName, ListContainerOptions options) {
>        throw new UnsupportedOperationException();
>     }
>  
> -   // copied from BaseBlobStore
> +   // TODO: Move statics up to BaseBlobStore to reuse?
> +   @com.google.inject.Inject(optional = true)
> +   @Named("jclouds.mpu.parallel.degree")
> +   @VisibleForTesting
> +   private int numThreads = 16;
> +
> +   private ExecutorService jcloudsBigBlobUploadTreadPool = MoreExecutors.getExitingExecutorService(

Incredibly, none of the multipart configuration variables actually *do* anything, except for this one.  We should just remove all of them.

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/975/files/0b784d15fa6f35ceb82028e1823e85c31ccf513d#r68612932

Re: [jclouds/jclouds] Changes the upload behavior to parallel, completing a TODO (#975)

Posted by Zack Shoylev <no...@github.com>.
>           ++partNumber;
> +         BlobUploader b =
> +               new BlobUploader(mpu, partNumber, payload, parts);
> +         executor.execute(b);
> +      }
> +      executor.shutdown();
> +      try {
> +         executor.awaitTermination(Integer.MAX_VALUE, TimeUnit.DAYS);
> +      } catch (InterruptedException e) {
> +         e.printStackTrace();
> +      }
> +
> +      return completeMultipartUpload(mpu, new ArrayList( parts ));
> +   }
> +
> +   private class BlobUploader implements Runnable {

Making it static means uploadMultipartPart in BlobStore will have be made static, otherwise it's referencing a dynamic context from a static one. But if I do that it would mean changes in other providers and so forth. There might be other problems with that, too.

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/975/files/70ba6e0f5aa5c660cab27eb6a450ca7f1e435235#r67949493

Re: [jclouds/jclouds] Changes the upload behavior to parallel, completing a TODO (#975)

Posted by Andrew Gaul <no...@github.com>.
> @@ -87,6 +110,15 @@ public BlobAccess getBlobAccess() {
>        return blobAccess;
>     }
>  
> +   public ExecutorService getCustomExecutor() {
> +      return customExecutor;
> +   }
> +
> +   public PutOptions setCustomExecutor(ExecutorService customExecutor) {
> +      this.customExecutor = customExecutor;

Call `Preconditions.checkNotNull`.

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/975/files/0b784d15fa6f35ceb82028e1823e85c31ccf513d#r68282022

Re: [jclouds/jclouds] Changes the upload behavior to parallel, completing a TODO (#975)

Posted by Zack Shoylev <no...@github.com>.
merged

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/975#issuecomment-230003838

Re: [jclouds/jclouds] Changes the upload behavior to parallel, completing a TODO (#975)

Posted by Zack Shoylev <no...@github.com>.
> @@ -579,22 +590,79 @@ public long countBlobs(String containerName, ListContainerOptions options) {
>        throw new UnsupportedOperationException();
>     }
>  
> -   // copied from BaseBlobStore
> +   // TODO: Move statics up to BaseBlobStore to reuse?
> +   @com.google.inject.Inject(optional = true)
> +   @Named("jclouds.mpu.parallel.degree")
> +   @VisibleForTesting
> +   private int numThreads = 16;
> +
> +   private ExecutorService jcloudsBigBlobUploadTreadPool = MoreExecutors.getExitingExecutorService(

Using an existing executor means we can't use the property variable for number of threads, though, right? Since the variable is already documented, I think there is an expectation that such a separate executor should exist.

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/975/files/0b784d15fa6f35ceb82028e1823e85c31ccf513d#r68583312

Re: [jclouds/jclouds] Changes the upload behavior to parallel, completing a TODO (#975)

Posted by Zack Shoylev <no...@github.com>.
@zack-shoylev pushed 1 commit.

89bff7f  Review fixes


---
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/jclouds/jclouds/pull/975/files/70ba6e0f5aa5c660cab27eb6a450ca7f1e435235..89bff7f1d8e279bdbfbe455250a7e37115460f64

Re: [jclouds/jclouds] Changes the upload behavior to parallel, completing a TODO (#975)

Posted by Andrew Gaul <no...@github.com>.
>     @Beta
>     protected String putMultipartBlob(String container, Blob blob, PutOptions overrides) {
> -      List<MultipartPart> parts = Lists.newArrayList();
> +      if (overrides.getCustomExecutor() != null) {
> +         return putMultipartBlob(container, blob, overrides, overrides.getCustomExecutor());
> +      } else {
> +         return putMultipartBlob(container, blob, overrides, jcloudsBigBlobUploadTreadPool);
> +      }
> +   }
> +
> +   // copied from BaseBlobStore

We need similar code in `BaseBlobStore` for other providers now that we change `PutOptions`.  This will also resolve [JCLOUDS-227](https://issues.apache.org/jira/browse/JCLOUDS-227).

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/975/files/0b784d15fa6f35ceb82028e1823e85c31ccf513d#r68281631

Re: [jclouds/jclouds] Changes the upload behavior to parallel, completing a TODO (#975)

Posted by Andrew Gaul <no...@github.com>.
>        long contentLength = checkNotNull(blob.getMetadata().getContentMetadata().getContentLength(),
>              "must provide content-length to use multi-part upload");
>        MultipartUploadSlicingAlgorithm algorithm = new MultipartUploadSlicingAlgorithm(
>              getMinimumMultipartPartSize(), getMaximumMultipartPartSize(), getMaximumNumberOfParts());
>        long partSize = algorithm.calculateChunkSize(contentLength);
>        MultipartUpload mpu = initiateMultipartUpload(container, blob.getMetadata(), partSize, overrides);
>        int partNumber = 1;
> +
> +      int numThreads = 16;
> +      if (System.getProperty("jclouds.mpu.parallel.degree") != null) {
> +         numThreads = Integer.getInteger(System.getProperty("jclouds.mpu.parallel.degree"));

This gets the property twice, with the second call failing.  Instead you want: `Integer.getInteger("jclouds.mpu.parallel.degree"), 16)`.

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/975/files/70ba6e0f5aa5c660cab27eb6a450ca7f1e435235#r67774064

Re: [jclouds/jclouds] Changes the upload behavior to parallel, completing a TODO (#975)

Posted by Andrew Gaul <no...@github.com>.
>           ++partNumber;
> +         BlobUploader b =
> +               new BlobUploader(mpu, partNumber, payload, parts);
> +         executor.execute(b);
> +      }
> +      executor.shutdown();

Instead of creating a new `ExecutorService` per `putBlob` request, we should take an optional one as an argument.  This would allow callers to control the degree of parallelism with multiple `putBlob` calls.

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/975/files/70ba6e0f5aa5c660cab27eb6a450ca7f1e435235#r67774299

Re: [jclouds/jclouds] Changes the upload behavior to parallel, completing a TODO (#975)

Posted by Andrew Gaul <no...@github.com>.
> @@ -579,22 +590,79 @@ public long countBlobs(String containerName, ListContainerOptions options) {
>        throw new UnsupportedOperationException();
>     }
>  
> -   // copied from BaseBlobStore
> +   // TODO: Move statics up to BaseBlobStore to reuse?
> +   @com.google.inject.Inject(optional = true)
> +   @Named("jclouds.mpu.parallel.degree")
> +   @VisibleForTesting
> +   private int numThreads = 16;
> +
> +   private ExecutorService jcloudsBigBlobUploadTreadPool = MoreExecutors.getExitingExecutorService(

We should not add random `ExecutorService` to our code.  Either we reuse the deprecated `userExecutor` and `ioExecutor` or make a caller provide this via a parameter.

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/975/files/0b784d15fa6f35ceb82028e1823e85c31ccf513d#r68280252

Re: [jclouds/jclouds] Changes the upload behavior to parallel, completing a TODO (#975)

Posted by Andrew Gaul <no...@github.com>.
> @@ -44,6 +47,16 @@ public PutOptions(boolean multipart) {
>        this.multipart = multipart;
>     }
>  
> +   public PutOptions(boolean multipart, ExecutorService customExecutor) {
> +      this.multipart = multipart;
> +      this.customExecutor = customExecutor;
> +   }
> +
> +   public PutOptions(ExecutorService customExecutor) {
> +      this.multipart = true;
> +      this.customExecutor = customExecutor;

Call `Preconditions.checkNotNull`.

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/975/files/0b784d15fa6f35ceb82028e1823e85c31ccf513d#r68281873

Re: [jclouds/jclouds] Changes the upload behavior to parallel, completing a TODO (#975)

Posted by Andrew Gaul <no...@github.com>.
> +      })));
> +   }
> +
> +   private class BlobUploader implements Callable<MultipartPart> {
> +      private MultipartUpload mpu;
> +      private int partNumber;
> +      private Payload payload;
> +
> +      BlobUploader(MultipartUpload mpu, int partNumber, Payload payload) {
> +         this.mpu = mpu;
> +         this.partNumber = partNumber;
> +         this.payload = payload;
> +      }
> +
> +      @Override
> +      public final MultipartPart call() {

Remove `final` here, add to class and fields.

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/975/files/0b784d15fa6f35ceb82028e1823e85c31ccf513d#r68280327

Re: [jclouds/jclouds] Changes the upload behavior to parallel, completing a TODO (#975)

Posted by Zack Shoylev <no...@github.com>.
Closed #975.

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/975#event-710760366

Re: [jclouds/jclouds] Changes the upload behavior to parallel, completing a TODO (#975)

Posted by Andrew Gaul <no...@github.com>.
>        long contentLength = checkNotNull(blob.getMetadata().getContentMetadata().getContentLength(),
>              "must provide content-length to use multi-part upload");
>        MultipartUploadSlicingAlgorithm algorithm = new MultipartUploadSlicingAlgorithm(
>              getMinimumMultipartPartSize(), getMaximumMultipartPartSize(), getMaximumNumberOfParts());
>        long partSize = algorithm.calculateChunkSize(contentLength);
>        MultipartUpload mpu = initiateMultipartUpload(container, blob.getMetadata(), partSize, overrides);
>        int partNumber = 1;
> +
> +      int numThreads = 16;
> +      if (System.getProperty("jclouds.mpu.parallel.degree") != null) {
> +         numThreads = Integer.getInteger(System.getProperty("jclouds.mpu.parallel.degree"));

Also shouldn't we inject this property instead of using the system properties?

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/975/files/70ba6e0f5aa5c660cab27eb6a450ca7f1e435235#r67775087

Re: [jclouds/jclouds] Changes the upload behavior to parallel, completing a TODO (#975)

Posted by Zack Shoylev <no...@github.com>.
@zack-shoylev pushed 1 commit.

65e8b67  Review fixes


---
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/jclouds/jclouds/pull/975/files/0b784d15fa6f35ceb82028e1823e85c31ccf513d..65e8b67ae8c9aa2a9fd152d957f4df0a02ac27fe

Re: [jclouds/jclouds] Changes the upload behavior to parallel, completing a TODO (#975)

Posted by Zack Shoylev <no...@github.com>.
> @@ -44,6 +47,16 @@ public PutOptions(boolean multipart) {
>        this.multipart = multipart;
>     }
>  
> +   public PutOptions(boolean multipart, ExecutorService customExecutor) {

Yeah that seems redundant.

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/975/files/0b784d15fa6f35ceb82028e1823e85c31ccf513d#r68397108

Re: [jclouds/jclouds] Changes the upload behavior to parallel, completing a TODO (#975)

Posted by Zack Shoylev <no...@github.com>.
>        long contentLength = checkNotNull(blob.getMetadata().getContentMetadata().getContentLength(),
>              "must provide content-length to use multi-part upload");
>        MultipartUploadSlicingAlgorithm algorithm = new MultipartUploadSlicingAlgorithm(
>              getMinimumMultipartPartSize(), getMaximumMultipartPartSize(), getMaximumNumberOfParts());
>        long partSize = algorithm.calculateChunkSize(contentLength);
>        MultipartUpload mpu = initiateMultipartUpload(container, blob.getMetadata(), partSize, overrides);
>        int partNumber = 1;
> +
> +      int numThreads = 16;
> +      if (System.getProperty("jclouds.mpu.parallel.degree") != null) {
> +         numThreads = Integer.getInteger(System.getProperty("jclouds.mpu.parallel.degree"));

Also I am not very sure if we should be using this property at all and I am not sure about how it is documented (the website states it has a default value of 4, actually, not 16, but it is also not used elsewhere in our code).

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/975/files/70ba6e0f5aa5c660cab27eb6a450ca7f1e435235#r67881183

Re: [jclouds/jclouds] Changes the upload behavior to parallel, completing a TODO (#975)

Posted by Andrew Gaul <no...@github.com>.
👍 

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/975#issuecomment-229843594

Re: [jclouds/jclouds] Changes the upload behavior to parallel, completing a TODO (#975)

Posted by Zack Shoylev <no...@github.com>.
>           ++partNumber;
> +         BlobUploader b =
> +               new BlobUploader(mpu, partNumber, payload, parts);
> +         executor.execute(b);
> +      }
> +      executor.shutdown();
> +      try {
> +         executor.awaitTermination(Integer.MAX_VALUE, TimeUnit.DAYS);
> +      } catch (InterruptedException e) {
> +         e.printStackTrace();
> +      }
> +
> +      return completeMultipartUpload(mpu, new ArrayList( parts ));
> +   }
> +
> +   private class BlobUploader implements Runnable {

As for AutoValue, I am not sure if it's worth it, and I am not sure it will work well considering payload and multipartupload are mutable... I think...

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/975/files/70ba6e0f5aa5c660cab27eb6a450ca7f1e435235#r67957832

Re: [jclouds/jclouds] Changes the upload behavior to parallel, completing a TODO (#975)

Posted by Andrew Gaul <no...@github.com>.
> +
> +   private class BlobUploader implements Runnable {
> +      private ConcurrentLinkedQueue<MultipartPart> list;
> +      private MultipartUpload mpu;
> +      private int partNumber;
> +      private Payload payload;
> +
> +      BlobUploader(MultipartUpload mpu, int partNumber, Payload payload, ConcurrentLinkedQueue<MultipartPart> list) {
> +         this.mpu = mpu;
> +         this.partNumber = partNumber;
> +         this.payload = payload;
> +         this.list = list;
> +      }
> +
> +      @Override
> +      public void run() {

If you make this a `Callable` can you use the return value instead of the explicit `list`?

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/975/files/70ba6e0f5aa5c660cab27eb6a450ca7f1e435235#r67775010

Re: [jclouds/jclouds] Changes the upload behavior to parallel, completing a TODO (#975)

Posted by Zack Shoylev <no...@github.com>.
>           ++partNumber;
> +         BlobUploader b =
> +               new BlobUploader(mpu, partNumber, payload, parts);
> +         executor.execute(b);
> +      }
> +      executor.shutdown();

OK, this makes sense, but might make it pretty complex for users. What about making the thread-pool for large blob uploads static instead? Thoughts? Do both? Not very sure what is the best design here. So many options :)

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/975/files/70ba6e0f5aa5c660cab27eb6a450ca7f1e435235#r67882871

Re: [jclouds/jclouds] Changes the upload behavior to parallel, completing a TODO (#975)

Posted by Andrew Gaul <no...@github.com>.
> @@ -36,6 +38,7 @@
>  
>     private BlobAccess blobAccess = BlobAccess.PRIVATE;
>     private boolean multipart = false;
> +   private ExecutorService customExecutor = null;

Instead use `MoreExecutors.sameThreadExecutor` so all code can rely on this.

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/975/files/0b784d15fa6f35ceb82028e1823e85c31ccf513d#r68281843

Re: [jclouds/jclouds] Changes the upload behavior to parallel, completing a TODO (#975)

Posted by Andrew Gaul <no...@github.com>.
>        for (Payload payload : slicer.slice(blob.getPayload(), partSize)) {
> -         MultipartPart part = uploadMultipartPart(mpu, partNumber, payload);
> -         parts.add(part);
> -         ++partNumber;
> +         BlobUploader b =
> +               new BlobUploader(mpu, partNumber++, payload);
> +         parts.add(executor.submit(b));
> +      }
> +
> +      return completeMultipartUpload(mpu, Lists.newArrayList(Iterables.transform(parts, new Function<Future<MultipartPart>, MultipartPart>() {
> +         @Override
> +         public MultipartPart apply(Future<MultipartPart> input) {

Call `Futures.allAsList().get()` instead.  Possibly with `Futures.getUnchecked` to handle exceptions.

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/975/files/0b784d15fa6f35ceb82028e1823e85c31ccf513d#r68281233

Re: [jclouds/jclouds] Changes the upload behavior to parallel, completing a TODO (#975)

Posted by Andrew Gaul <no...@github.com>.
> -         ++partNumber;
> +         BlobUploader b =
> +               new BlobUploader(mpu, partNumber++, payload);
> +         parts.add(executor.submit(b));
> +      }
> +
> +      return completeMultipartUpload(mpu, Lists.newArrayList(Iterables.transform(parts, new Function<Future<MultipartPart>, MultipartPart>() {
> +         @Override
> +         public MultipartPart apply(Future<MultipartPart> input) {
> +            try {
> +               return input.get();
> +            } catch (InterruptedException e) {
> +               e.printStackTrace();
> +               throw new RuntimeException(e);
> +            } catch (ExecutionException e) {
> +               e.printStackTrace();

Don't `printStackTrace`.

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/975/files/0b784d15fa6f35ceb82028e1823e85c31ccf513d#r68281268

Re: [jclouds/jclouds] Changes the upload behavior to parallel, completing a TODO (#975)

Posted by Zack Shoylev <no...@github.com>.
>     @Beta
>     protected String putMultipartBlob(String container, Blob blob, PutOptions overrides) {
> -      List<MultipartPart> parts = Lists.newArrayList();
> +      if (overrides.getCustomExecutor() != null) {
> +         return putMultipartBlob(container, blob, overrides, overrides.getCustomExecutor());
> +      } else {
> +         return putMultipartBlob(container, blob, overrides, jcloudsBigBlobUploadTreadPool);
> +      }
> +   }
> +
> +   // copied from BaseBlobStore

Hmm yes. Do you think it would be possible to move all the thread-pool code to BaseBlobStore, or will it cause issues with some providers?

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/975/files/0b784d15fa6f35ceb82028e1823e85c31ccf513d#r68396994

Re: [jclouds/jclouds] Changes the upload behavior to parallel, completing a TODO (#975)

Posted by Andrew Gaul <no...@github.com>.
>     @Beta
>     protected String putMultipartBlob(String container, Blob blob, PutOptions overrides) {
> -      List<MultipartPart> parts = Lists.newArrayList();
> +      if (overrides.getCustomExecutor() != null) {
> +         return putMultipartBlob(container, blob, overrides, overrides.getCustomExecutor());
> +      } else {
> +         return putMultipartBlob(container, blob, overrides, jcloudsBigBlobUploadTreadPool);
> +      }
> +   }
> +
> +   // copied from BaseBlobStore

We should include it in `BaseBlobStore`, although `RegionScopedSwiftBlobStore` does not subclass it.

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/975/files/0b784d15fa6f35ceb82028e1823e85c31ccf513d#r68612639

Re: [jclouds/jclouds] Changes the upload behavior to parallel, completing a TODO (#975)

Posted by Zack Shoylev <no...@github.com>.
@zack-shoylev pushed 1 commit.

0b784d1  Add custom executor to PutOptions


---
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/jclouds/jclouds/pull/975/files/89bff7f1d8e279bdbfbe455250a7e37115460f64..0b784d15fa6f35ceb82028e1823e85c31ccf513d

Re: [jclouds/jclouds] Changes the upload behavior to parallel, completing a TODO (#975)

Posted by Zack Shoylev <no...@github.com>.
@andrewgaul Have another look. Also, I'd like to merge this before I start adding similar code in the abstraction layer (separate PR).

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds/pull/975#issuecomment-229410240