You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jclouds.apache.org by Adrian Cole <no...@github.com> on 2013/09/30 08:58:38 UTC
[jclouds-labs-openstack] Openstack swift blobstore (#33)
This adds support for openstack-swift grizzly BlobStore.
Once this is in, we can start working on JCLOUDS-299: Multi-Region Support for BlobStore
Note: the live tests will fail a couple items on either HPCS or RAX, depending on what's unsupported on that cloud. Usually, this is related to either signing or public container acls.
Once this is in, we can create subprojects to address the vendor specifics, like we do now.
You can merge this Pull Request by running:
git pull https://github.com/adriancole/jclouds-labs-openstack openstack-swift-blobstore
Or you can view, comment on it, or merge it online at:
https://github.com/jclouds/jclouds-labs-openstack/pull/33
-- Commit Summary --
* JCLOUDS-308. Add Temporary Url Support to openstack-swift
* JCLOUDS-306. expose Container.anybodyRead
* JCLOUDS-298. expose container metadata via ObjectList and port Object PUT code from legacy codebase
* JCLOUDS-300. Add BlobStore abstraction to openstack-swift api
-- File Changes --
M openstack-swift/pom.xml (13)
M openstack-swift/src/main/java/org/jclouds/openstack/swift/v1/SwiftApiMetadata.java (9)
A openstack-swift/src/main/java/org/jclouds/openstack/swift/v1/TemporaryUrlSigner.java (92)
A openstack-swift/src/main/java/org/jclouds/openstack/swift/v1/binders/SetPayload.java (49)
A openstack-swift/src/main/java/org/jclouds/openstack/swift/v1/blobstore/RegionScopedSwiftBlobStore.java (209)
A openstack-swift/src/main/java/org/jclouds/openstack/swift/v1/blobstore/RegionScopedTemporaryUrlBlobSigner.java (98)
A openstack-swift/src/main/java/org/jclouds/openstack/swift/v1/blobstore/config/SignUsingTemporaryUrls.java (78)
A openstack-swift/src/main/java/org/jclouds/openstack/swift/v1/blobstore/config/SwiftBlobStoreContextModule.java (34)
A openstack-swift/src/main/java/org/jclouds/openstack/swift/v1/blobstore/functions/ToBlobMetadata.java (71)
A openstack-swift/src/main/java/org/jclouds/openstack/swift/v1/blobstore/functions/ToListContainerOptions.java (52)
A openstack-swift/src/main/java/org/jclouds/openstack/swift/v1/blobstore/functions/ToResourceMetadata.java (48)
A openstack-swift/src/main/java/org/jclouds/openstack/swift/v1/blobstore/functions/ToSwiftObject.java (55)
A openstack-swift/src/main/java/org/jclouds/openstack/swift/v1/blobstore/internal/SubmissionAsyncBlobStore.java (284)
M openstack-swift/src/main/java/org/jclouds/openstack/swift/v1/domain/Account.java (5)
M openstack-swift/src/main/java/org/jclouds/openstack/swift/v1/domain/Container.java (37)
A openstack-swift/src/main/java/org/jclouds/openstack/swift/v1/domain/ObjectList.java (47)
M openstack-swift/src/main/java/org/jclouds/openstack/swift/v1/features/AccountApi.java (20)
M openstack-swift/src/main/java/org/jclouds/openstack/swift/v1/features/BulkApi.java (2)
M openstack-swift/src/main/java/org/jclouds/openstack/swift/v1/features/ObjectApi.java (25)
M openstack-swift/src/main/java/org/jclouds/openstack/swift/v1/functions/ParseContainerFromHeaders.java (3)
M openstack-swift/src/main/java/org/jclouds/openstack/swift/v1/functions/ParseObjectListFromResponse.java (21)
A openstack-swift/src/test/java/org/jclouds/openstack/swift/v1/TemporaryUrlSignerLiveTest.java (91)
A openstack-swift/src/test/java/org/jclouds/openstack/swift/v1/TemporaryUrlSignerMockTest.java (75)
A openstack-swift/src/test/java/org/jclouds/openstack/swift/v1/blobstore/integration/SwiftBlobIntegrationLiveTest.java (75)
A openstack-swift/src/test/java/org/jclouds/openstack/swift/v1/blobstore/integration/SwiftBlobLiveTest.java (38)
A openstack-swift/src/test/java/org/jclouds/openstack/swift/v1/blobstore/integration/SwiftBlobSignerLiveTest.java (38)
A openstack-swift/src/test/java/org/jclouds/openstack/swift/v1/blobstore/integration/SwiftContainerIntegrationLiveTest.java (53)
A openstack-swift/src/test/java/org/jclouds/openstack/swift/v1/blobstore/integration/SwiftContainerLiveTest.java (38)
A openstack-swift/src/test/java/org/jclouds/openstack/swift/v1/blobstore/integration/SwiftServiceIntegrationLiveTest.java (38)
M openstack-swift/src/test/java/org/jclouds/openstack/swift/v1/features/AccountApiMockTest.java (20)
M openstack-swift/src/test/java/org/jclouds/openstack/swift/v1/features/ContainerApiMockTest.java (3)
M openstack-swift/src/test/java/org/jclouds/openstack/swift/v1/features/CreatePublicContainerLiveTest.java (29)
M openstack-swift/src/test/java/org/jclouds/openstack/swift/v1/features/ObjectApiLiveTest.java (5)
M openstack-swift/src/test/java/org/jclouds/openstack/swift/v1/features/ObjectApiMockTest.java (16)
M openstack-swift/src/test/java/org/jclouds/openstack/swift/v1/internal/BaseSwiftApiLiveTest.java (21)
M openstack-swift/src/test/resources/logback.xml (2)
-- Patch Links --
https://github.com/jclouds/jclouds-labs-openstack/pull/33.patch
https://github.com/jclouds/jclouds-labs-openstack/pull/33.diff
Re: [jclouds-labs-openstack] Openstack swift blobstore (#33)
Posted by Andrew Phillips <no...@github.com>.
> + to.setPublicUri(from.uri());
> + }
> + to.setUri(from.uri());
> + to.setETag(from.etag());
> + to.setName(from.name());
> + to.setLastModified(from.lastModified());
> + to.setContentMetadata(from.payload().getContentMetadata());
> + to.getContentMetadata().setContentMD5(base16().lowerCase().decode(from.etag()));
> + to.setUserMetadata(from.metadata());
> + String directoryName = ifDirectoryReturnName.execute(to);
> + if (directoryName != null) {
> + to.setName(directoryName);
> + to.setType(StorageType.RELATIVE_PATH);
> + } else {
> + to.setType(StorageType.BLOB);
> + }
Do we have or need tests for this logic?
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-labs-openstack/pull/33/files#r7235177
Re: [jclouds-labs-openstack] Openstack swift blobstore (#33)
Posted by Andrew Phillips <no...@github.com>.
> + }
> +
> + @Override
> + public boolean blobExists(String container, String name) {
> + return blobMetadata(container, name) != null;
> + }
> +
> + @Override
> + public String putBlob(String container, Blob blob) {
> + return putBlob(container, blob, PutOptions.NONE);
> + }
> +
> + @Override
> + public String putBlob(String container, Blob blob, PutOptions options) {
> + if (options.isMultipart()) {
> + throw new UnsupportedOperationException();
Add message?
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-labs-openstack/pull/33/files#r7235171
Re: [jclouds-labs-openstack] Openstack swift blobstore (#33)
Posted by CloudBees pull request builder plugin <no...@github.com>.
[jclouds-labs-openstack-pull-requests #62](https://jclouds.ci.cloudbees.com/job/jclouds-labs-openstack-pull-requests/62/) SUCCESS
This pull request looks good
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-labs-openstack/pull/33#issuecomment-25340911
Re: [jclouds-labs-openstack] Openstack swift blobstore (#33)
Posted by Andrew Phillips <no...@github.com>.
> + }
> + if ((from.getDir() == null) && (!from.isRecursive())) {
> + options.path("");
> + }
> + if ((from.getDir() != null) && (from.isRecursive())) {
> + options.prefix(from.getDir().endsWith("/") ? from.getDir() : from.getDir() + "/");
> + }
> + if ((from.getDir() != null) && (!from.isRecursive())) {
> + options.path(from.getDir());
> + }
> + if (from.getMarker() != null) {
> + options.marker(from.getMarker());
> + }
> + if (from.getMaxResults() != null) {
> + options.limit(from.getMaxResults());
> + }
Do we have or need tests for this logic?
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-labs-openstack/pull/33/files#r7235179
Re: [jclouds-labs-openstack] Openstack swift blobstore (#33)
Posted by Andrew Phillips <no...@github.com>.
> assertEquals(objects, parsedObjectsForUrl(server.getUrl("/").toString()));
> + assertEquals(objects.container().name(), "myContainer");
> + assertTrue(objects.container().anybodyRead().get());
Add message?
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-labs-openstack/pull/33/files#r7235183
Re: [jclouds-labs-openstack] Openstack swift blobstore (#33)
Posted by Andrew Phillips <no...@github.com>.
> <logger name="jclouds.wire">
> <level value="DEBUG" />
> <appender-ref ref="WIREFILE" />
> </logger>
> +-->
Intentional?
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-labs-openstack/pull/33/files#r7235185
Re: [jclouds-labs-openstack] Openstack swift blobstore (#33)
Posted by Adrian Cole <no...@github.com>.
commit cbac63e184d6e24ba3503cb7d95edb6e2df6f9d4
Author: Adrian Cole <ad...@gmail.com>
Date: Sun Sep 29 23:53:04 2013 -0700
JCLOUDS-300. Add BlobStore abstraction to openstack-swift api
commit 0426ebf0c00113f3affc608c2ba83da7bb5554be
Author: Adrian Cole <ad...@gmail.com>
Date: Sun Sep 29 23:49:50 2013 -0700
JCLOUDS-298. expose container metadata via ObjectList and port Object PUT code from legacy codebase
commit 523b134256a781e7c8a5aaf314301cee7f7bd702
Author: Adrian Cole <ad...@gmail.com>
Date: Sun Sep 29 23:47:23 2013 -0700
JCLOUDS-306. expose Container.anybodyRead
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-labs-openstack/pull/33#issuecomment-25341422
Re: [jclouds-labs-openstack] Openstack swift blobstore (#33)
Posted by Andrew Phillips <no...@github.com>.
> <jclouds.osgi.export>org.jclouds.openstack.swift.v1*;version="${project.version}"</jclouds.osgi.export>
> - <jclouds.osgi.import>
> - org.jclouds.blobstore.internal;version="${jclouds.version}",
> - org.jclouds.rest.internal;version="${jclouds.version}",
> - org.jclouds.labs*;version="${project.version}",
> - org.jclouds*;version="${jclouds.version}",
> - *
> - </jclouds.osgi.import>
> + <jclouds.osgi.import>org.jclouds*;version="${project.version}",*</jclouds.osgi.import>
This is in labs-openstack, so should that not be `jclouds.version`? During a release, project.version will not be the same as the core version.
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-labs-openstack/pull/33/files#r7235169
Re: [jclouds-labs-openstack] Openstack swift blobstore (#33)
Posted by CloudBees pull request builder plugin <no...@github.com>.
[jclouds-labs-openstack-pull-requests #63](https://jclouds.ci.cloudbees.com/job/jclouds-labs-openstack-pull-requests/63/) SUCCESS
This pull request looks good
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-labs-openstack/pull/33#issuecomment-25341391
Re: [jclouds-labs-openstack] Openstack swift blobstore (#33)
Posted by Andrew Phillips <no...@github.com>.
> for (String regionId : api.configuredRegions()) {
> - api.containerApiInRegion(regionId).createIfAbsent(containerName, anybodyRead());
> - api.containerApiInRegion(regionId).get(containerName);
> -
> - ObjectApi objectApi = api.objectApiInRegionForContainer(regionId, containerName);
> - objectApi.replace(name, newStringPayload("swifty"), ImmutableMap.<String, String> of());
> -
> - InputStream publicStream = objectApi.get(name, new GetOptions()).uri().toURL().openStream();
> -
> - assertEquals(Strings2.toStringAndClose(publicStream), "swifty");
> + api.containerApiInRegion(regionId).createIfAbsent(name, anybodyRead());
> + assertTrue(api.containerApiInRegion(regionId).get(name).anybodyRead().get());
Add message?
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-labs-openstack/pull/33/files#r7235181