You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@jclouds.apache.org by Roman Coedo <no...@github.com> on 2014/08/04 22:41:07 UTC

[jclouds-examples] Add glacier example (#58)

You can merge this Pull Request by running:

  git pull https://github.com/rcoedo/jclouds-examples master

Or you can view, comment on it, or merge it online at:

  https://github.com/jclouds/jclouds-examples/pull/58

-- Commit Summary --

  * Add glacier example

-- File Changes --

    A glacier/README.md (17)
    A glacier/pom.xml (83)
    A glacier/src/main/assembly/jar-with-dependencies.xml (42)
    A glacier/src/main/java/org/jclouds/examples/glacier/MainApp.java (255)

-- Patch Links --

https://github.com/jclouds/jclouds-examples/pull/58.patch
https://github.com/jclouds/jclouds-examples/pull/58.diff

---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-examples/pull/58

Re: [jclouds-examples] Add glacier example (#58)

Posted by Andrew Phillips <no...@github.com>.
> +         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
> +  <modelVersion>4.0.0</modelVersion>
> +  <groupId>org.apache.jclouds.examples</groupId>
> +  <artifactId>glacier</artifactId>
> +  <version>1.8.0</version>
> +  <name>glacier</name>
> +
> +  <properties>
> +    <jclouds.version>1.8.0</jclouds.version>
> +  </properties>
> +
> +  <dependencies>
> +    <dependency>
> +      <groupId>org.apache.jclouds.labs</groupId>
> +      <artifactId>glacier</artifactId>
> +      <version>1.8.0</version>

Use `${project.version}`?

---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-examples/pull/58/files#r16025973

Re: [jclouds-examples] Add glacier example (#58)

Posted by Roman Coedo <no...@github.com>.
> +            return false;
> +         default:
> +            System.out.println("Not a valid option");
> +            break;
> +         }
> +      }
> +      catch(InputMismatchException e){
> +         System.out.println("Not a valid option");
> +      }
> +      return true;
> +   }
> +
> +   private static ByteSource buildData(long size) {
> +      byte[] array = new byte[1024];
> +      Arrays.fill(array, (byte) 'a');
> +      return ByteSources.repeatingArrayByteSource(array).slice(0, size);

Is this part of the new SNAPSHOT version? I can't import the TestUtils class while using 1.8.0 version.

---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-examples/pull/58/files#r16032611

Re: [jclouds-examples] Add glacier example (#58)

Posted by Andrew Gaul <no...@github.com>.
> +
> +      // Init
> +      BlobStoreContext context = ContextBuilder.newBuilder("glacier")
> +            .credentials(identity, credentials)
> +            .buildView(BlobStoreContext.class);
> +
> +      try {
> +         while (chooseOption(context));
> +      } finally {
> +         context.close();
> +      }
> +   }
> +
> +   private static void putAndRetrieveBlobExample(BlobStoreContext context) throws IOException {
> +      // Get the blobstore
> +      BlobStore blobstore = context.getBlobStore();

Can you pass in the `BlobStore` view instead of its context?

---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-examples/pull/58/files#r16017625

Re: [jclouds-examples] Add glacier example (#58)

Posted by Andrew Gaul <no...@github.com>.
> +            return false;
> +         default:
> +            System.out.println("Not a valid option");
> +            break;
> +         }
> +      }
> +      catch(InputMismatchException e){
> +         System.out.println("Not a valid option");
> +      }
> +      return true;
> +   }
> +
> +   private static ByteSource buildData(long size) {
> +      byte[] array = new byte[1024];
> +      Arrays.fill(array, (byte) 'a');
> +      return ByteSources.repeatingArrayByteSource(array).slice(0, size);

Sorry you are right; `org.jclouds.utils.TestUtils` is not part of the distribution nor should it be.

---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-examples/pull/58/files#r16032770

Re: [jclouds-examples] Add glacier example (#58)

Posted by Andrew Gaul <no...@github.com>.
> +            .description("retrieval job")
> +            .build();
> +
> +      // Initiate job
> +      String jobId = client.initiateJob(vaultName, archiveRetrievalJobRequest);
> +      try {
> +         // Poll until the job is done
> +         new BasePollingStrategy(client).waitForSuccess(vaultName, jobId);
> +
> +         // Get the job output
> +         Payload result = client.getJobOutput(vaultName, jobId);
> +
> +         // Print the result
> +         InputStream is = result.openStream();
> +         try {
> +            String data = CharStreams.toString(new InputStreamReader(is, "UTF-8"));

`Charsets.UTF_8`?

---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-examples/pull/58/files#r16017769

Re: [jclouds-examples] Add glacier example (#58)

Posted by Andrew Gaul <no...@github.com>.
> +import org.jclouds.glacier.blobstore.strategy.internal.BasePollingStrategy;
> +import org.jclouds.glacier.domain.ArchiveRetrievalJobRequest;
> +import org.jclouds.glacier.domain.JobRequest;
> +import org.jclouds.io.ByteSources;
> +import org.jclouds.io.Payload;
> +import org.jclouds.io.payloads.ByteSourcePayload;
> +
> +import com.google.common.base.Charsets;
> +import com.google.common.base.Throwables;
> +import com.google.common.io.ByteSource;
> +import com.google.common.io.CharStreams;
> +
> +/**
> + * Demonstrates the use of Glacier provider and BlobStore.
> + *
> + * Usage is: java MainApp \"identity\" \"credential\"

backslashes necessary?

---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-examples/pull/58/files#r16017885

Re: [jclouds-examples] Add glacier example (#58)

Posted by Andrew Gaul <no...@github.com>.
> +            return false;
> +         default:
> +            System.out.println("Not a valid option");
> +            break;
> +         }
> +      }
> +      catch(InputMismatchException e){
> +         System.out.println("Not a valid option");
> +      }
> +      return true;
> +   }
> +
> +   private static ByteSource buildData(long size) {
> +      byte[] array = new byte[1024];
> +      Arrays.fill(array, (byte) 'a');
> +      return ByteSources.repeatingArrayByteSource(array).slice(0, size);

Can you call `TestUtils.randomByteSource().limit(size)` instead?

---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-examples/pull/58/files#r16017703

Re: [jclouds-examples] Add glacier example (#58)

Posted by Andrew Gaul <no...@github.com>.
> +      JobRequest archiveRetrievalJobRequest = ArchiveRetrievalJobRequest.builder()
> +            .archiveId(archiveId)
> +            .description("retrieval job")
> +            .build();
> +
> +      // Initiate job
> +      String jobId = client.initiateJob(vaultName, archiveRetrievalJobRequest);
> +      try {
> +         // Poll until the job is done
> +         new BasePollingStrategy(client).waitForSuccess(vaultName, jobId);
> +
> +         // Get the job output
> +         Payload result = client.getJobOutput(vaultName, jobId);
> +
> +         // Print the result
> +         InputStream is = result.openStream();

`Strings2.toStringAndClose`?

---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-examples/pull/58/files#r16017803

Re: [jclouds-examples] Add glacier example (#58)

Posted by Andrew Phillips <no...@github.com>.
> +            multipartUploadExample(context);
> +            break;
> +         case 3:
> +            interruptionExample(context);
> +            break;
> +         case 4:
> +            providerExample(context);
> +            break;
> +         case 5:
> +            return false;
> +         default:
> +            System.out.println("Not a valid option");
> +            break;
> +         }
> +      }
> +      catch(InputMismatchException e){

[minor] Space before `{` here and throughout this `try` block?

---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-examples/pull/58/files#r16025982

Re: [jclouds-examples] Add glacier example (#58)

Posted by Andrew Gaul <no...@github.com>.
> +            .credentials(identity, credentials)
> +            .buildView(BlobStoreContext.class);
> +
> +      try {
> +         while (chooseOption(context));
> +      } finally {
> +         context.close();
> +      }
> +   }
> +
> +   private static void putAndRetrieveBlobExample(BlobStoreContext context) throws IOException {
> +      // Get the blobstore
> +      BlobStore blobstore = context.getBlobStore();
> +
> +      // Create a container
> +      String containerName = UUID.randomUUID().toString() + "_putAndRetrieveBlobExample";

Reverse to "putAndRetrieveBlobExample_" + UUID.randomUUID().toString(), so multiple runs sort near each other.  Perhaps add a "jclouds" prefix?

---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-examples/pull/58/files#r16017861

Re: [jclouds-examples] Add glacier example (#58)

Posted by Andrew Phillips <no...@github.com>.
> +
> +      // Create a container
> +      final String containerName = UUID. randomUUID().toString() + "_interruptionExample";
> +      blobstore.createContainerInLocation(null, containerName); // Create a vault
> +
> +      // Create a blob
> +      ByteSource payload = ByteSource.wrap("data".getBytes(Charsets.UTF_8));
> +      Blob blob = blobstore.blobBuilder("ignored") // The blob name is ignored in Glacier
> +            .payload(payload)
> +            .contentLength(payload.size())
> +            .build();
> +
> +      // Put the blob in the container
> +      final String blobId = blobstore.putBlob(containerName, blob);
> +
> +      // New thread

Is explicit thread management of this kind still "good style"? I thought the idea was to use executors for all this kind of stuff?

---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-examples/pull/58/files#r16025985

Re: [jclouds-examples] Add glacier example (#58)

Posted by Andrew Gaul <no...@github.com>.
Pushed to master as cae84da833968ba930859201b00e6e97b78ef27b.

---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-examples/pull/58#issuecomment-51745809

Re: [jclouds-examples] Add glacier example (#58)

Posted by Andrew Phillips <no...@github.com>.
> +    distributed under the License is distributed on an "AS IS" BASIS,
> +    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> +    See the License for the specific language governing permissions and
> +    limitations under the License.
> +
> +-->
> +<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> +         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
> +  <modelVersion>4.0.0</modelVersion>
> +  <groupId>org.apache.jclouds.examples</groupId>
> +  <artifactId>glacier</artifactId>
> +  <version>1.8.0</version>
> +  <name>glacier</name>
> +
> +  <properties>
> +    <jclouds.version>1.8.0</jclouds.version>

If we're going to use `<version>` above to track the jclouds version, we don't need this?

---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-examples/pull/58/files#r16025970

Re: [jclouds-examples] Add glacier example (#58)

Posted by Andrew Phillips <no...@github.com>.
Thanks, @andrewgaul and @rcoedo!

---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-examples/pull/58#issuecomment-52099518

Re: [jclouds-examples] Add glacier example (#58)

Posted by Andrew Gaul <no...@github.com>.
> +
> +      // Create a container
> +      final String containerName = UUID. randomUUID().toString() + "_interruptionExample";
> +      blobstore.createContainerInLocation(null, containerName); // Create a vault
> +
> +      // Create a blob
> +      ByteSource payload = ByteSource.wrap("data".getBytes(Charsets.UTF_8));
> +      Blob blob = blobstore.blobBuilder("ignored") // The blob name is ignored in Glacier
> +            .payload(payload)
> +            .contentLength(payload.size())
> +            .build();
> +
> +      // Put the blob in the container
> +      final String blobId = blobstore.putBlob(containerName, blob);
> +
> +      // New thread

Discussing offline with @rcoedo, we will revisit this when we have a better handle on general blobstore cancellation.

---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-examples/pull/58/files#r16038862

Re: [jclouds-examples] Add glacier example (#58)

Posted by Roman Coedo <no...@github.com>.
> +
> +      // Create a container
> +      final String containerName = UUID. randomUUID().toString() + "_interruptionExample";
> +      blobstore.createContainerInLocation(null, containerName); // Create a vault
> +
> +      // Create a blob
> +      ByteSource payload = ByteSource.wrap("data".getBytes(Charsets.UTF_8));
> +      Blob blob = blobstore.blobBuilder("ignored") // The blob name is ignored in Glacier
> +            .payload(payload)
> +            .contentLength(payload.size())
> +            .build();
> +
> +      // Put the blob in the container
> +      final String blobId = blobstore.putBlob(containerName, blob);
> +
> +      // New thread

Wouldn't be executors useless here? In this case I'm not going to reuse the thread.

---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-examples/pull/58/files#r16032757

Re: [jclouds-examples] Add glacier example (#58)

Posted by Andrew Gaul <no...@github.com>.
Closed #58.

---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-examples/pull/58#event-151230728

Re: [jclouds-examples] Add glacier example (#58)

Posted by Andrew Gaul <no...@github.com>.
> +
> +      // Create a container
> +      final String containerName = UUID. randomUUID().toString() + "_interruptionExample";
> +      blobstore.createContainerInLocation(null, containerName); // Create a vault
> +
> +      // Create a blob
> +      ByteSource payload = ByteSource.wrap("data".getBytes(Charsets.UTF_8));
> +      Blob blob = blobstore.blobBuilder("ignored") // The blob name is ignored in Glacier
> +            .payload(payload)
> +            .contentLength(payload.size())
> +            .build();
> +
> +      // Put the blob in the container
> +      final String blobId = blobstore.putBlob(containerName, blob);
> +
> +      // New thread

It is useless in this specific example, but most callers will work with an `ExecutorService` and `Future`.

---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-examples/pull/58/files#r16034877