You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@jclouds.apache.org by Steve Kingsland <st...@opower.com> on 2014/09/24 17:00:22 UTC
Example of setting Blob ACL using S3 provider?
I'm trying to get code working which sets an ACL on an object when I upload
it to the BlobStore. I'm using the Jclouds S3 provider, and the closest
documentation I can find is from the "Using S3" example code at
http://jclouds.apache.org/guides/aws/:
// when you need access to s3-specific features,// use the
provider-specific contextAWSS3Client s3Client =
AWSS3Client.class.cast(context.getProviderSpecificContext().getApi());
// make the object world readableString publicReadWriteObjectKey =
"public-read-write-acl";S3Object object = s3Client.newS3Object();
object.getMetadata().setKey(publicReadWriteObjectKey);object.setPayload("hello
world");s3Client.putObject(bucket, object,
withAcl(CannedAccessPolicy.PUBLIC_READ));
context.close();
Problem is, the getProviderSpecificContext()
<http://demobox.github.io/jclouds-maven-site-1.5.0/1.5.0/jclouds-multi/apidocs/org/jclouds/blobstore/BlobStoreContext.html#getProviderSpecificContext()>
method
was apparently removed in Jclouds 1.6. I think I can work around this using
contextBuilder.buildApi(S3Client.class) and the S3Object class to set the
ACL.
But then if I have to use an S3-specific API to set an ACL, why not just
use the com.amazonaws.services.s3.AmazonS3Client client directly? How is
Jclouds actually benefitting me, if I'm using it to code directly to the S3
provider?
*Steve Kingsland*
Senior Software Engineer
*Opower* <http://www.opower.com/>
*We’re hiring! See jobs here <http://www.opower.com/careers>*
Re: Example of setting Blob ACL using S3 provider?
Posted by Ignasi Barrera <ig...@gmail.com>.
Ah yes, sorry. IIRC it is deprecated to indicate that it will
disappear in future releases. 1.7 introduces the new ApiContext and
most providers have been migrated to use that. It should be safe to
use it in 1.6 even if it is deprecated.
On 26 September 2014 18:12, Steve Kingsland <st...@opower.com> wrote:
> Ah, you mean this:
> http://jclouds.apache.org/reference/javadoc/1.7.x/org/jclouds/View.html#unwrapApi(java.lang.Class)
>
> Looks like it was just added in Jclouds 1.7, which explains why I'm not
> seeing it locally. :(
>
>
>
> Steve Kingsland
>
>
> Senior Software Engineer
>
> Opower
>
>
> We’re hiring! See jobs here
>
>
> On Fri, Sep 26, 2014 at 11:59 AM, Ignasi Barrera <na...@apache.org> wrote:
>>
>> You should be able to create the portable BlobStoreContext and use the
>> "unwrapApi" method to get the provider specific api you want to use
>> (S3Client or AWSS3Client).
>>
>> El 26/09/2014 17:38, "Steve Kingsland" <st...@opower.com>
>> escribió:
>>
>>> Thanks, now I understand the difference between APIs and providers
>>> better. I had previously thought that I needed to create an S3Client
>>> instance via contextBuilder.buildApi(S3Client.class). But it looks like I
>>> can get a reference to it from the basic BlobStoreContext instead:
>>>
>>> BlobStoreContext context =
>>> ContextBuilder.newBuilder("s3")....buildView(BlobStoreContext.class);
>>>
>>> S3Client s3Client = context.<RestContext<S3Client,
>>> S3AsyncClient>>unwrap().getApi();
>>>
>>> The only problem is that RestContext is deprecated (in Jclouds 1.6.3),
>>> and I'm not sure how to get the the S3Client object using ApiContext
>>> instead.
>>>
>>>
>>>
>>> Steve Kingsland
>>>
>>>
>>> Senior Software Engineer
>>>
>>> Opower
>>>
>>>
>>> We’re hiring! See jobs here
>>>
>>>
>>> On Fri, Sep 26, 2014 at 2:17 AM, Andrew Phillips <an...@apache.org>
>>> wrote:
>>>>>
>>>>> although I'm not entirely sure what the difference is between the
>>>>> AWSS3Client
>>>>> the docs are using, and the S3Client that I've used
>>>>
>>>>
>>>> One is the API-specific client, the other the provider-specific client
>>>> (see [1]). I.e. S3Client should work with all "S3-compatible" providers,
>>>> whereas the AWSS3Client could expose extra functionality that *only* Amazon
>>>> offers.
>>>>
>>>> Hope that helps!
>>>>
>>>> ap
>>>>
>>>> PS: PRs to update outdated docs much appreciated...thanks!
>>>>
>>>> [1] http://jclouds.apache.org/start/concepts/
>>>
>>>
>
Re: Example of setting Blob ACL using S3 provider?
Posted by Steve Kingsland <st...@opower.com>.
Ah, you mean this:
http://jclouds.apache.org/reference/javadoc/1.7.x/org/jclouds/View.html#unwrapApi(java.lang.Class)
Looks like it was just added in Jclouds 1.7, which explains why I'm not
seeing it locally. :(
*Steve Kingsland*
Senior Software Engineer
*Opower* <http://www.opower.com/>
*We’re hiring! See jobs here <http://www.opower.com/careers>*
On Fri, Sep 26, 2014 at 11:59 AM, Ignasi Barrera <na...@apache.org> wrote:
> You should be able to create the portable BlobStoreContext and use the
> "unwrapApi" method to get the provider specific api you want to use
> (S3Client or AWSS3Client).
> El 26/09/2014 17:38, "Steve Kingsland" <st...@opower.com>
> escribió:
>
> Thanks, now I understand the difference between APIs and providers better.
>> I had previously thought that I needed to create an S3Client instance via
>> contextBuilder.buildApi(S3Client.class). But it looks like I can get a
>> reference to it from the basic BlobStoreContext instead:
>>
>> BlobStoreContext context =
>> ContextBuilder.newBuilder("s3")....buildView(BlobStoreContext.class);
>>
>> S3Client s3Client = context.<RestContext<S3Client,
>> S3AsyncClient>>unwrap().getApi();
>>
>> The only problem is that RestContext is deprecated (in Jclouds 1.6.3),
>> and I'm not sure how to get the the S3Client object using ApiContext
>> instead.
>>
>>
>>
>> *Steve Kingsland*
>>
>> Senior Software Engineer
>>
>> *Opower* <http://www.opower.com/>
>>
>>
>> *We’re hiring! See jobs here <http://www.opower.com/careers>*
>>
>> On Fri, Sep 26, 2014 at 2:17 AM, Andrew Phillips <an...@apache.org>
>> wrote:
>>
>>> although I'm not entirely sure what the difference is between the
>>>> AWSS3Client
>>>> the docs are using, and the S3Client that I've used
>>>>
>>>
>>> One is the API-specific client, the other the provider-specific client
>>> (see [1]). I.e. S3Client should work with all "S3-compatible" providers,
>>> whereas the AWSS3Client could expose extra functionality that *only* Amazon
>>> offers.
>>>
>>> Hope that helps!
>>>
>>> ap
>>>
>>> PS: PRs to update outdated docs much appreciated...thanks!
>>>
>>> [1] http://jclouds.apache.org/start/concepts/
>>>
>>
>>
Re: Example of setting Blob ACL using S3 provider?
Posted by Ignasi Barrera <na...@apache.org>.
You should be able to create the portable BlobStoreContext and use the
"unwrapApi" method to get the provider specific api you want to use
(S3Client or AWSS3Client).
El 26/09/2014 17:38, "Steve Kingsland" <st...@opower.com>
escribió:
> Thanks, now I understand the difference between APIs and providers better.
> I had previously thought that I needed to create an S3Client instance via
> contextBuilder.buildApi(S3Client.class). But it looks like I can get a
> reference to it from the basic BlobStoreContext instead:
>
> BlobStoreContext context =
> ContextBuilder.newBuilder("s3")....buildView(BlobStoreContext.class);
>
> S3Client s3Client = context.<RestContext<S3Client,
> S3AsyncClient>>unwrap().getApi();
>
> The only problem is that RestContext is deprecated (in Jclouds 1.6.3),
> and I'm not sure how to get the the S3Client object using ApiContext
> instead.
>
>
>
> *Steve Kingsland*
>
> Senior Software Engineer
>
> *Opower* <http://www.opower.com/>
>
>
> *We’re hiring! See jobs here <http://www.opower.com/careers>*
>
> On Fri, Sep 26, 2014 at 2:17 AM, Andrew Phillips <an...@apache.org>
> wrote:
>
>> although I'm not entirely sure what the difference is between the
>>> AWSS3Client
>>> the docs are using, and the S3Client that I've used
>>>
>>
>> One is the API-specific client, the other the provider-specific client
>> (see [1]). I.e. S3Client should work with all "S3-compatible" providers,
>> whereas the AWSS3Client could expose extra functionality that *only* Amazon
>> offers.
>>
>> Hope that helps!
>>
>> ap
>>
>> PS: PRs to update outdated docs much appreciated...thanks!
>>
>> [1] http://jclouds.apache.org/start/concepts/
>>
>
>
Re: Example of setting Blob ACL using S3 provider?
Posted by Steve Kingsland <st...@opower.com>.
Thanks, now I understand the difference between APIs and providers better.
I had previously thought that I needed to create an S3Client instance via
contextBuilder.buildApi(S3Client.class). But it looks like I can get a
reference to it from the basic BlobStoreContext instead:
BlobStoreContext context =
ContextBuilder.newBuilder("s3")....buildView(BlobStoreContext.class);
S3Client s3Client = context.<RestContext<S3Client,
S3AsyncClient>>unwrap().getApi();
The only problem is that RestContext is deprecated (in Jclouds 1.6.3), and
I'm not sure how to get the the S3Client object using ApiContext instead.
*Steve Kingsland*
Senior Software Engineer
*Opower* <http://www.opower.com/>
*We’re hiring! See jobs here <http://www.opower.com/careers>*
On Fri, Sep 26, 2014 at 2:17 AM, Andrew Phillips <an...@apache.org> wrote:
> although I'm not entirely sure what the difference is between the
>> AWSS3Client
>> the docs are using, and the S3Client that I've used
>>
>
> One is the API-specific client, the other the provider-specific client
> (see [1]). I.e. S3Client should work with all "S3-compatible" providers,
> whereas the AWSS3Client could expose extra functionality that *only* Amazon
> offers.
>
> Hope that helps!
>
> ap
>
> PS: PRs to update outdated docs much appreciated...thanks!
>
> [1] http://jclouds.apache.org/start/concepts/
>
Re: Example of setting Blob ACL using S3 provider?
Posted by Andrew Phillips <an...@apache.org>.
> although I'm not entirely sure what the difference is between the AWSS3Client
> the docs are using, and the S3Client that I've used
One is the API-specific client, the other the provider-specific client
(see [1]). I.e. S3Client should work with all "S3-compatible"
providers, whereas the AWSS3Client could expose extra functionality
that *only* Amazon offers.
Hope that helps!
ap
PS: PRs to update outdated docs much appreciated...thanks!
[1] http://jclouds.apache.org/start/concepts/
Re: Example of setting Blob ACL using S3 provider?
Posted by Steve Kingsland <st...@opower.com>.
Andrew,
I'd be happy to create a ticket for using the BlobStore API to set
object-level ACLs on the putBlob() call, if you think it's achievable
across all providers that Jclouds supports. I won't be able to implement
this myself, though, because like I said Jclouds has advanced beyond the
point where I'm able to use the latest version, because of the Guava
version it depends on (see JCLOUDS-427).
I don't totally follow what you're talking about with signed URLs; is this
explained somewhere on jclouds.apache.org that I could read more about it,
and see if it fits my needs?
Also, perhaps I made too big a deal of the ACL issue; it's actually only 1
line of code, and it's working quite nicely for me locally:
S3Object s3Object = s3Client.newS3Object();
s3Object.setPayload(payload);
s3Object.getMetadata().setKey(objectName);
PutObjectOptions putObjectOptions =
*PutObjectOptions.Builder.withAcl(CannedAccessPolicy.PUBLIC_READ);*
s3Client.putObject(bucketName, s3Object, putObjectOptions);
The only "catch" I struggled with was how to get a handle to the S3Client
instance; the docs on the site were out-of date. (I'll submit a pull
request to fix them up, although I'm not entirely sure what the difference
is between the AWSS3Client the docs are using, and the S3Client that I've
used. OK if I change them to use S3Client instead?)
Thanks for your help!
*Steve Kingsland*
Senior Software Engineer
*Opower* <http://www.opower.com/>
*We’re hiring! See jobs here <http://www.opower.com/careers>*
On Wed, Sep 24, 2014 at 7:03 PM, Andrew Gaul <ga...@apache.org> wrote:
> Many providers support ACLs and the jclouds portable abstraction could
> provide support for a subset of them, mostly the public read and write
> variants. I opened a JIRA issue for container ACLs:
>
> https://issues.apache.org/jira/browse/JCLOUDS-660
>
> Could you open an issue for object ACLs? Could you also research this
> issue further and potentially implement the desired functionality
> itself? The portable abstraction is growing and it should provide
> support for copying blobs in the next major release.
>
> In the mean-time, you might consider signed URLs instead of object ACLs.
> Signed URLs allow your application to vend a time-limited read or write
> token to a client which then interacts with the object store directly.
> This approach offers finer-grained permissions and most jclouds
> providers support this feature.
>
> On Wed, Sep 24, 2014 at 02:28:53PM -0400, Steve Kingsland wrote:
> > Thanks Andrew, it now makes sense that ACLs would be a provider-specific
> > feature. Maybe my confusion arose simply because the S3-specific docs [1]
> > were a bit out-of-date, with regards to getting the provider-specific
> API?
> > (referencing a method which no longer exists)
> >
> > AFA supporting ACLs on the AWSS3PutOptions, I'm afraid that adding this
> to
> > the latest version of jclouds wouldn't help me at all. I'm stuck on 1.6.3
> > because of a Guava incompatibility between Jclouds (see JCLOUDS-427) and
> > the version of HBase we're using (see HBASE-9667). But that's fine;
> instead
> > of passing around a BlobStoreContext in my code, it's a straightforward
> > change to use an S3Client object instead, which gives me access to all
> the
> > S3-specific features (like ACLs) that I need.
> >
> >
> > [1] http://jclouds.apache.org/guides/aws/
> >
> >
> >
> > *Steve Kingsland*
> >
> > Senior Software Engineer
> >
> > *Opower* <http://www.opower.com/>
> >
> >
> > *We’re hiring! See jobs here <http://www.opower.com/careers>*
> >
> > On Wed, Sep 24, 2014 at 1:10 PM, Andrew Phillips <an...@apache.org>
> wrote:
> >
> > > was apparently removed in Jclouds 1.6. I think I can work around this
> using
> > >> contextBuilder.buildApi(S3Client.class) and the S3Object class to set
> the
> > >> ACL.
> > >>
> > >> But then if I have to use an S3-specific API to set an ACL, why not
> just
> > >> use the com.amazonaws.services.s3.AmazonS3Client client directly? How
> is
> > >> Jclouds actually benefitting me, if I'm using it to code directly to
> the
> > >> S3
> > >> provider?
> > >>
> > >
> > > The challenge here is that ACLs are not something that is supported by
> all
> > > the blobstores jclouds supports, so is not included in the BlobStore
> > > abstraction. In other words, coding with ACLs on blobs is implicitly
> making
> > > your code provider-dependent already.
> > >
> > > In order to support provider-specific features like that, jclouds
> indeed
> > > supports access to the underlying API, as you describe (see [1] for
> more
> > > details). This is indeed not all that different from using the provider
> > > client directly, but if the number of provider-specific calls you need
> to
> > > make are small, using jclouds will allow you to move to another
> provider
> > > relatively easily if you can re-implement the provider-specific calls
> or
> > > remove them.
> > >
> > > A "middle-ground approach" that jclouds takes for some options is the
> > > ability to pass provider-specific options to the abstract interface.
> E.g.
> > > something like:
> > >
> > > Blob myBlob = ...
> > > PutOptions options = AWSS3PutOptions.Builder.
> > > storageClass(...).otherOption...;
> > > blobstore.putBlob("myContainer", myBlob, options); // [2]
> > >
> > > This is *also* AWS-specific code, but not quite as tied to the specific
> > > underlying API. Unfortunately, "withAcl" is not an option currently
> > > supported on AWSS3PutOptions [3] (it *is* supported on PutObjectOptions
> > > [4], but that's the options class for the S3-specific call).
> > >
> > > That should be a relatively easy fix, though - would you be interested
> in
> > > submitting a PR for that?
> > >
> > > Hope that helps!
> > >
> > > ap
> > >
> > > [1] http://jclouds.apache.org/start/concepts/
> > > [2] http://javadocs.jclouds.cloudbees.net/org/jclouds/
> > > blobstore/BlobStore.html#putBlob(java.lang.String,
> > > org.jclouds.blobstore.domain.Blob, org.jclouds.blobstore.options.
> > > PutOptions)
> > > [3] http://javadocs.jclouds.cloudbees.net/org/jclouds/aws/
> > > s3/blobstore/options/AWSS3PutOptions.html
> > > [4] http://javadocs.jclouds.cloudbees.net/org/jclouds/s3/
> > > options/PutObjectOptions.html
> > >
>
> --
> Andrew Gaul
> http://gaul.org/
>
Re: Example of setting Blob ACL using S3 provider?
Posted by Andrew Gaul <ga...@apache.org>.
Many providers support ACLs and the jclouds portable abstraction could
provide support for a subset of them, mostly the public read and write
variants. I opened a JIRA issue for container ACLs:
https://issues.apache.org/jira/browse/JCLOUDS-660
Could you open an issue for object ACLs? Could you also research this
issue further and potentially implement the desired functionality
itself? The portable abstraction is growing and it should provide
support for copying blobs in the next major release.
In the mean-time, you might consider signed URLs instead of object ACLs.
Signed URLs allow your application to vend a time-limited read or write
token to a client which then interacts with the object store directly.
This approach offers finer-grained permissions and most jclouds
providers support this feature.
On Wed, Sep 24, 2014 at 02:28:53PM -0400, Steve Kingsland wrote:
> Thanks Andrew, it now makes sense that ACLs would be a provider-specific
> feature. Maybe my confusion arose simply because the S3-specific docs [1]
> were a bit out-of-date, with regards to getting the provider-specific API?
> (referencing a method which no longer exists)
>
> AFA supporting ACLs on the AWSS3PutOptions, I'm afraid that adding this to
> the latest version of jclouds wouldn't help me at all. I'm stuck on 1.6.3
> because of a Guava incompatibility between Jclouds (see JCLOUDS-427) and
> the version of HBase we're using (see HBASE-9667). But that's fine; instead
> of passing around a BlobStoreContext in my code, it's a straightforward
> change to use an S3Client object instead, which gives me access to all the
> S3-specific features (like ACLs) that I need.
>
>
> [1] http://jclouds.apache.org/guides/aws/
>
>
>
> *Steve Kingsland*
>
> Senior Software Engineer
>
> *Opower* <http://www.opower.com/>
>
>
> *We’re hiring! See jobs here <http://www.opower.com/careers>*
>
> On Wed, Sep 24, 2014 at 1:10 PM, Andrew Phillips <an...@apache.org> wrote:
>
> > was apparently removed in Jclouds 1.6. I think I can work around this using
> >> contextBuilder.buildApi(S3Client.class) and the S3Object class to set the
> >> ACL.
> >>
> >> But then if I have to use an S3-specific API to set an ACL, why not just
> >> use the com.amazonaws.services.s3.AmazonS3Client client directly? How is
> >> Jclouds actually benefitting me, if I'm using it to code directly to the
> >> S3
> >> provider?
> >>
> >
> > The challenge here is that ACLs are not something that is supported by all
> > the blobstores jclouds supports, so is not included in the BlobStore
> > abstraction. In other words, coding with ACLs on blobs is implicitly making
> > your code provider-dependent already.
> >
> > In order to support provider-specific features like that, jclouds indeed
> > supports access to the underlying API, as you describe (see [1] for more
> > details). This is indeed not all that different from using the provider
> > client directly, but if the number of provider-specific calls you need to
> > make are small, using jclouds will allow you to move to another provider
> > relatively easily if you can re-implement the provider-specific calls or
> > remove them.
> >
> > A "middle-ground approach" that jclouds takes for some options is the
> > ability to pass provider-specific options to the abstract interface. E.g.
> > something like:
> >
> > Blob myBlob = ...
> > PutOptions options = AWSS3PutOptions.Builder.
> > storageClass(...).otherOption...;
> > blobstore.putBlob("myContainer", myBlob, options); // [2]
> >
> > This is *also* AWS-specific code, but not quite as tied to the specific
> > underlying API. Unfortunately, "withAcl" is not an option currently
> > supported on AWSS3PutOptions [3] (it *is* supported on PutObjectOptions
> > [4], but that's the options class for the S3-specific call).
> >
> > That should be a relatively easy fix, though - would you be interested in
> > submitting a PR for that?
> >
> > Hope that helps!
> >
> > ap
> >
> > [1] http://jclouds.apache.org/start/concepts/
> > [2] http://javadocs.jclouds.cloudbees.net/org/jclouds/
> > blobstore/BlobStore.html#putBlob(java.lang.String,
> > org.jclouds.blobstore.domain.Blob, org.jclouds.blobstore.options.
> > PutOptions)
> > [3] http://javadocs.jclouds.cloudbees.net/org/jclouds/aws/
> > s3/blobstore/options/AWSS3PutOptions.html
> > [4] http://javadocs.jclouds.cloudbees.net/org/jclouds/s3/
> > options/PutObjectOptions.html
> >
--
Andrew Gaul
http://gaul.org/
Re: Example of setting Blob ACL using S3 provider?
Posted by Steve Kingsland <st...@opower.com>.
Thanks Andrew, it now makes sense that ACLs would be a provider-specific
feature. Maybe my confusion arose simply because the S3-specific docs [1]
were a bit out-of-date, with regards to getting the provider-specific API?
(referencing a method which no longer exists)
AFA supporting ACLs on the AWSS3PutOptions, I'm afraid that adding this to
the latest version of jclouds wouldn't help me at all. I'm stuck on 1.6.3
because of a Guava incompatibility between Jclouds (see JCLOUDS-427) and
the version of HBase we're using (see HBASE-9667). But that's fine; instead
of passing around a BlobStoreContext in my code, it's a straightforward
change to use an S3Client object instead, which gives me access to all the
S3-specific features (like ACLs) that I need.
[1] http://jclouds.apache.org/guides/aws/
*Steve Kingsland*
Senior Software Engineer
*Opower* <http://www.opower.com/>
*We’re hiring! See jobs here <http://www.opower.com/careers>*
On Wed, Sep 24, 2014 at 1:10 PM, Andrew Phillips <an...@apache.org> wrote:
> was apparently removed in Jclouds 1.6. I think I can work around this using
>> contextBuilder.buildApi(S3Client.class) and the S3Object class to set the
>> ACL.
>>
>> But then if I have to use an S3-specific API to set an ACL, why not just
>> use the com.amazonaws.services.s3.AmazonS3Client client directly? How is
>> Jclouds actually benefitting me, if I'm using it to code directly to the
>> S3
>> provider?
>>
>
> The challenge here is that ACLs are not something that is supported by all
> the blobstores jclouds supports, so is not included in the BlobStore
> abstraction. In other words, coding with ACLs on blobs is implicitly making
> your code provider-dependent already.
>
> In order to support provider-specific features like that, jclouds indeed
> supports access to the underlying API, as you describe (see [1] for more
> details). This is indeed not all that different from using the provider
> client directly, but if the number of provider-specific calls you need to
> make are small, using jclouds will allow you to move to another provider
> relatively easily if you can re-implement the provider-specific calls or
> remove them.
>
> A "middle-ground approach" that jclouds takes for some options is the
> ability to pass provider-specific options to the abstract interface. E.g.
> something like:
>
> Blob myBlob = ...
> PutOptions options = AWSS3PutOptions.Builder.
> storageClass(...).otherOption...;
> blobstore.putBlob("myContainer", myBlob, options); // [2]
>
> This is *also* AWS-specific code, but not quite as tied to the specific
> underlying API. Unfortunately, "withAcl" is not an option currently
> supported on AWSS3PutOptions [3] (it *is* supported on PutObjectOptions
> [4], but that's the options class for the S3-specific call).
>
> That should be a relatively easy fix, though - would you be interested in
> submitting a PR for that?
>
> Hope that helps!
>
> ap
>
> [1] http://jclouds.apache.org/start/concepts/
> [2] http://javadocs.jclouds.cloudbees.net/org/jclouds/
> blobstore/BlobStore.html#putBlob(java.lang.String,
> org.jclouds.blobstore.domain.Blob, org.jclouds.blobstore.options.
> PutOptions)
> [3] http://javadocs.jclouds.cloudbees.net/org/jclouds/aws/
> s3/blobstore/options/AWSS3PutOptions.html
> [4] http://javadocs.jclouds.cloudbees.net/org/jclouds/s3/
> options/PutObjectOptions.html
>
Re: Example of setting Blob ACL using S3 provider?
Posted by Andrew Phillips <an...@apache.org>.
> was apparently removed in Jclouds 1.6. I think I can work around this using
> contextBuilder.buildApi(S3Client.class) and the S3Object class to set the
> ACL.
>
> But then if I have to use an S3-specific API to set an ACL, why not just
> use the com.amazonaws.services.s3.AmazonS3Client client directly? How is
> Jclouds actually benefitting me, if I'm using it to code directly to the S3
> provider?
The challenge here is that ACLs are not something that is supported by
all the blobstores jclouds supports, so is not included in the
BlobStore abstraction. In other words, coding with ACLs on blobs is
implicitly making your code provider-dependent already.
In order to support provider-specific features like that, jclouds
indeed supports access to the underlying API, as you describe (see [1]
for more details). This is indeed not all that different from using
the provider client directly, but if the number of provider-specific
calls you need to make are small, using jclouds will allow you to move
to another provider relatively easily if you can re-implement the
provider-specific calls or remove them.
A "middle-ground approach" that jclouds takes for some options is the
ability to pass provider-specific options to the abstract interface.
E.g. something like:
Blob myBlob = ...
PutOptions options = AWSS3PutOptions.Builder.storageClass(...).otherOption...;
blobstore.putBlob("myContainer", myBlob, options); // [2]
This is *also* AWS-specific code, but not quite as tied to the
specific underlying API. Unfortunately, "withAcl" is not an option
currently supported on AWSS3PutOptions [3] (it *is* supported on
PutObjectOptions [4], but that's the options class for the S3-specific
call).
That should be a relatively easy fix, though - would you be interested
in submitting a PR for that?
Hope that helps!
ap
[1] http://jclouds.apache.org/start/concepts/
[2]
http://javadocs.jclouds.cloudbees.net/org/jclouds/blobstore/BlobStore.html#putBlob(java.lang.String, org.jclouds.blobstore.domain.Blob,
org.jclouds.blobstore.options.PutOptions)
[3]
http://javadocs.jclouds.cloudbees.net/org/jclouds/aws/s3/blobstore/options/AWSS3PutOptions.html
[4]
http://javadocs.jclouds.cloudbees.net/org/jclouds/s3/options/PutObjectOptions.html
Re: Example of setting Blob ACL using S3 provider?
Posted by Steve Kingsland <st...@opower.com>.
That does help, thanks Ignasi! In my case, I need to connect to an
S3-compliant object store that my company is hosting internally. Right now
it's Ceph, but they want me to interact with it using the S3 APIs, so the
specific server becomes an implementation detail. The server might change,
but it's unlikely that the API would (e.g. switching from S3 to the Swift
API).
Given that, and that I need to use the Jclouds S3 API directly to set ACLs
(and to check if a Blob exists), I'm trying to decide between using
Jclouds' S3 provider API, or using the Amazon S3 client directly. In both
cases I have working code which lets me do what I need to do. Right now my
needs are fairly simple, but I can't help wondering if I'd be better served
using the "official" S3 Java client that Amazon provides.
Several other projects at my company are already using S3 hosted on AWS, so
if I'm going to diverge from the Amazon client for accessing our internal
S3, I need to have a good reason. And having portable abstractions,
consistent API patterns, and clean dependencies doesn't really matter if
I'm only ever going to use the S3 API for communicating with an (internally
or externally hosted) object store. As far as the other "jclouds built-in
HTTP features" you speak of, is there a features matrix somewhere for me to
compare these benefits to what the amazon S3 client provides (that jclouds
doesn't)?
*Steve Kingsland*
Senior Software Engineer
*Opower* <http://www.opower.com/>
*We’re hiring! See jobs here <http://www.opower.com/careers>*
On Wed, Sep 24, 2014 at 11:42 AM, Ignasi Barrera <na...@apache.org> wrote:
> It always depends on your needs.
>
> jclouds provides a portable abstraction layer that allows you to talk
> to different clouds with the same code. It provides portable Compute,
> Blobstore and Load Balancing models and APIs you can use.
>
> Each cloud, however, has its own semantics and specific features, and
> although jclouds does a good job in putting them together behind the
> portable abstractions, some provider specific things have to be done
> with the provider specific APIs. IT wouldn't make sense to promote
> every specific feature of every provider to the portable layer.
>
> That said, jclouds gives you the freedom to use the portable
> abstractions or the provider specific APIs. Is up to you. But even
> when you use the provider specific APIs to do concrete things, jclouds
> provides menu benefits:
>
> * You still have a consistent pattern to use APIs. All apis are
> constructed the same way and used in a similar fashion. You don't need
> to learn how a new library works if you want to use the specific API
> of several providers.
> * You keep your dependencies clean, as provider specific APIs don't
> bring new players.
> * You also benefit from the jclouds built-in HTTP features: smart
> retry policies, consistent error handling (guess how different APIs
> can populate similar errors? jclouds does a good job standardizing
> these behaviors), transparent pagination (you don't have to worry
> about getting the N page or even if the list is paginated, jclouds
> does it for you transparently), and more.
> * It deals with authentication, sessions, expired tokens, etc.
>
>
> In the end it is up to your use case. If you only need to talk with
> one cloud provider and there exist a library for it, perhaps it is
> better to just use that library, but jclouds is more than an "api
> aggregator" and does a great job also when using the provider specific
> APIs.
>
> HTH!
>
> I.
>
> On 24 September 2014 17:00, Steve Kingsland <st...@opower.com>
> wrote:
> > I'm trying to get code working which sets an ACL on an object when I
> upload
> > it to the BlobStore. I'm using the Jclouds S3 provider, and the closest
> > documentation I can find is from the "Using S3" example code at
> > http://jclouds.apache.org/guides/aws/:
> >
> > // when you need access to s3-specific features,
> > // use the provider-specific context
> > AWSS3Client s3Client =
> >
> AWSS3Client.class.cast(context.getProviderSpecificContext().getApi());
> >
> > // make the object world readable
> > String publicReadWriteObjectKey = "public-read-write-acl";
> > S3Object object = s3Client.newS3Object();
> >
> > object.getMetadata().setKey(publicReadWriteObjectKey);
> > object.setPayload("hello world");
> > s3Client.putObject(bucket, object,
> withAcl(CannedAccessPolicy.PUBLIC_READ));
> >
> > context.close();
> >
> > Problem is, the getProviderSpecificContext() method was apparently
> removed
> > in Jclouds 1.6. I think I can work around this using
> > contextBuilder.buildApi(S3Client.class) and the S3Object class to set the
> > ACL.
> >
> > But then if I have to use an S3-specific API to set an ACL, why not just
> use
> > the com.amazonaws.services.s3.AmazonS3Client client directly? How is
> Jclouds
> > actually benefitting me, if I'm using it to code directly to the S3
> > provider?
> >
> >
> > Steve Kingsland
> >
> >
> > Senior Software Engineer
> >
> > Opower
> >
> >
> > We’re hiring! See jobs here
>
Re: Example of setting Blob ACL using S3 provider?
Posted by Ignasi Barrera <na...@apache.org>.
It always depends on your needs.
jclouds provides a portable abstraction layer that allows you to talk
to different clouds with the same code. It provides portable Compute,
Blobstore and Load Balancing models and APIs you can use.
Each cloud, however, has its own semantics and specific features, and
although jclouds does a good job in putting them together behind the
portable abstractions, some provider specific things have to be done
with the provider specific APIs. IT wouldn't make sense to promote
every specific feature of every provider to the portable layer.
That said, jclouds gives you the freedom to use the portable
abstractions or the provider specific APIs. Is up to you. But even
when you use the provider specific APIs to do concrete things, jclouds
provides menu benefits:
* You still have a consistent pattern to use APIs. All apis are
constructed the same way and used in a similar fashion. You don't need
to learn how a new library works if you want to use the specific API
of several providers.
* You keep your dependencies clean, as provider specific APIs don't
bring new players.
* You also benefit from the jclouds built-in HTTP features: smart
retry policies, consistent error handling (guess how different APIs
can populate similar errors? jclouds does a good job standardizing
these behaviors), transparent pagination (you don't have to worry
about getting the N page or even if the list is paginated, jclouds
does it for you transparently), and more.
* It deals with authentication, sessions, expired tokens, etc.
In the end it is up to your use case. If you only need to talk with
one cloud provider and there exist a library for it, perhaps it is
better to just use that library, but jclouds is more than an "api
aggregator" and does a great job also when using the provider specific
APIs.
HTH!
I.
On 24 September 2014 17:00, Steve Kingsland <st...@opower.com> wrote:
> I'm trying to get code working which sets an ACL on an object when I upload
> it to the BlobStore. I'm using the Jclouds S3 provider, and the closest
> documentation I can find is from the "Using S3" example code at
> http://jclouds.apache.org/guides/aws/:
>
> // when you need access to s3-specific features,
> // use the provider-specific context
> AWSS3Client s3Client =
> AWSS3Client.class.cast(context.getProviderSpecificContext().getApi());
>
> // make the object world readable
> String publicReadWriteObjectKey = "public-read-write-acl";
> S3Object object = s3Client.newS3Object();
>
> object.getMetadata().setKey(publicReadWriteObjectKey);
> object.setPayload("hello world");
> s3Client.putObject(bucket, object, withAcl(CannedAccessPolicy.PUBLIC_READ));
>
> context.close();
>
> Problem is, the getProviderSpecificContext() method was apparently removed
> in Jclouds 1.6. I think I can work around this using
> contextBuilder.buildApi(S3Client.class) and the S3Object class to set the
> ACL.
>
> But then if I have to use an S3-specific API to set an ACL, why not just use
> the com.amazonaws.services.s3.AmazonS3Client client directly? How is Jclouds
> actually benefitting me, if I'm using it to code directly to the S3
> provider?
>
>
> Steve Kingsland
>
>
> Senior Software Engineer
>
> Opower
>
>
> We’re hiring! See jobs here