You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jclouds.apache.org by Fotis Psilos <pa...@gmail.com> on 2014/01/17 13:48:36 UTC
Architecture of jClouds
Hi,
I am new to jClouds and I am working on getting an insight in the jClouds
architecture. I am particularly interested in the blobstore service.
I would like to dive into the detailed implementation of the framework and
understand how to add a new blobstore provider. Particularly I am trying to
figure out how the concrete implementation is mapped to the abstract
classes and how the concrete methods are called.
I understand that jClouds generates the http calls for each concrete
provider but I cannot find the exact class that does this and where it gets
the information for the required attributes from.
Any information related to those questions is more that welcome :)
regards,
Fotis
Re: Architecture of jClouds
Posted by Andrew Phillips <ap...@qrmedia.com>.
> cloud services such as: payment service, image manipulation service etc.
> Such services are gaining popularity in PaaS marketplaces ( Heroku,
> engineyard, Amazon) and thought it would be nice to create a cloud API
> (similar to jcouds) for each of those services.
If you're looking at the potential intersection of jclouds and PaaS,
the following two old threads might be useful:
https://groups.google.com/forum/#!msg/jclouds/cGSxOKEHCzw/6Vv4yXghYYYJ
Nirmal Fernando has also recently submitted a provider for docker to
jclouds-labs, which has been generating a very enthusiastic response:
http://apache.markmail.org/thread/g4sfgahigav757a6
ap
Re: Architecture of jClouds
Posted by Fotis Psilos <pa...@gmail.com>.
I am doing my phd in cloud app portability and looking into cloud APIs.
Since there are already similar solutions for cloud storage, message
queuing, dtabases I am looking into applying the jClouds design into other
cloud services such as: payment service, image manipulation service etc.
Such services are gaining popularity in PaaS marketplaces ( Heroku,
engineyard, Amazon) and thought it would be nice to create a cloud API
(similar to jcouds) for each of those services.
But first I am trying to fully understand the rationale of jClouds design.
So you ll be hearing from me quite often since I sitll have quite some
questions related to the detailed design. I hope I m not spamming. :)
On 18 February 2014 11:31, Andrew Phillips <ap...@qrmedia.com> wrote:
> I see that the BaseBlobStore and BaseProviderMetadata classes can have some
>> implemented methods. But then what is the use of the interface on top of
>> that?
>>
>
> The Base* classes are intended mainly as convenience classes for
> *implementers* of providers. The implemented methods you've found provide
> common functionality that otherwise would probably have to be
> re-implemented in very similar ways in each provider.
>
> The interfaces are intended to be used/programmed against by *users*. So
> users would pretty much always be recommended to write:
>
> BlobStore blobStore = ctx.getBlobStore();
>
> rather than
>
> BaseBlobStore blobStore = (BaseBlobStore) ctx.getBlobStore(); //
> arararrgh!!
>
> Are you looking to implement your own blobstore provider, by any chance?
>
> ap
>
Re: Architecture of jClouds
Posted by Andrew Phillips <ap...@qrmedia.com>.
> I see that the BaseBlobStore and BaseProviderMetadata classes can have some
> implemented methods. But then what is the use of the interface on top of
> that?
The Base* classes are intended mainly as convenience classes for
*implementers* of providers. The implemented methods you've found
provide common functionality that otherwise would probably have to be
re-implemented in very similar ways in each provider.
The interfaces are intended to be used/programmed against by *users*.
So users would pretty much always be recommended to write:
BlobStore blobStore = ctx.getBlobStore();
rather than
BaseBlobStore blobStore = (BaseBlobStore) ctx.getBlobStore(); // arararrgh!!
Are you looking to implement your own blobstore provider, by any chance?
ap
Re: Architecture of jClouds
Posted by Fotis Psilos <pa...@gmail.com>.
Hi guys
I am trying to get an insight in the design of the jClouds framework
I am particulalry looking into the blobstore service
I have a question related to the purpose of having an Interface
(ProviderMetadata or BlobStore) and then an abstract class (
BaseProviderMetadata or BaseBlobtore) instead of using only an abstract
class
I see that the BaseBlobStore and BaseProviderMetadata classes can have some
implemented methods. But then what is the use of the interface on top of
that?
( I am also on IRC but due to the time difference (GMT+2) I cannot always
find the right people online)
Thank you
On 14 February 2014 18:15, Andrew Phillips <ap...@qrmedia.com> wrote:
> Are they only used to get the name of the method whcih subsequently is fed
>> to the url of the http request???
>>
>
> jclouds uses these methods to automatically generate HTTP invocations. The
> class that achieves most of this is the RestAnnotationProcessor. [1]
>
>
> Another question is why dynamic invocation is used? Is it used to enable
>> runtime deployment of new providers?
>>
>
> One of the reasons is similar to why deserializers such as Gson, or ORM
> tools such as Hibernate, also use dynamic processing of annotations: it
> means that the writer of a jclouds provider does not need to care about the
> underlying mechanics of HTTP call invocation, but can focus on simply
> defining the API endpoints that need to be called.
>
> If you have any questions about the design, feel free to jump on IRC
> (#jclouds at Freenode)!
>
> ap
>
> [1] https://github.com/jclouds/jclouds/blob/master/core/src/
> main/java/org/jclouds/rest/internal/RestAnnotationProcessor.java
>
Re: Architecture of jClouds
Posted by Andrew Phillips <ap...@qrmedia.com>.
> Are they only used to get the name of the method whcih subsequently is fed
> to the url of the http request???
jclouds uses these methods to automatically generate HTTP invocations.
The class that achieves most of this is the RestAnnotationProcessor. [1]
> Another question is why dynamic invocation is used? Is it used to enable
> runtime deployment of new providers?
One of the reasons is similar to why deserializers such as Gson, or
ORM tools such as Hibernate, also use dynamic processing of
annotations: it means that the writer of a jclouds provider does not
need to care about the underlying mechanics of HTTP call invocation,
but can focus on simply defining the API endpoints that need to be
called.
If you have any questions about the design, feel free to jump on IRC
(#jclouds at Freenode)!
ap
[1]
https://github.com/jclouds/jclouds/blob/master/core/src/main/java/org/jclouds/rest/internal/RestAnnotationProcessor.java
Re: Architecture of jClouds
Posted by Fotis Psilos <pa...@gmail.com>.
Hi guys,
I have a question related to the BlobStore Service. I am trying to figure
out how the provider specific operations are implemented. I noticed that
for Azure and S3 blobstore there are org.jclouds.azureblob.AzureBlobClient
and org.jclouds.s3.S3Client interfaces respectively which are supposed to
implement the operations. However I didnt find any concrete implmentation
of those interfaces.
Are they only used to get the name of the method whcih subsequently is fed
to the url of the http request???
Is it only me that finds jClouds internal design bit complicated ? :)
I have spent days over days to figure out the exact sequence of a blobstore
operation and still I am not that confident.
Another question is why dynamic invocation is used? Is it used to enable
runtime deployment of new providers?
Thank you ,
Fotis
On 17 January 2014 15:57, Andrew Phillips <ap...@qrmedia.com> wrote:
> Hi Fotis
>
>
> I understand that jClouds generates the http calls for each concrete
>> provider but I cannot find the exact class that does this and where it
>> gets
>> the information for the required attributes from.
>>
>
> Glad you're interested in jclouds! For a general overview of the
> architectural concepts, you can try the 'Core Concepts' document on the
> wiki page [1].
>
> As regards the lower-level details, the HTTP call generation largely
> happens in the RestAnnotationProcessor [2]. In many cases, concrete
> implementations are encapsulated in "strategies" (e.g. [3, 4]) that are
> injected abstraction layer classes like BaseComputeService [5] by Guice.
>
> Luckily, you'll find that BaseBlobStore is somewhat less complex than
> BaseComputeService ;-)
>
> This is only a very high-level sketch: please feel free to hop on to IRC
> for further discussion!
>
> ap
>
> [1] https://wiki.apache.org/jclouds/Core%20Concepts
> [2] https://github.com/jclouds/jclouds/blob/master/core/src/
> main/java/org/jclouds/rest/internal/RestAnnotationProcessor.java
> [3] https://github.com/jclouds/jclouds/blob/master/providers/
> aws-ec2/src/main/java/org/jclouds/aws/ec2/compute/strategy/
> AWSEC2GetNodeMetadataStrategy.java
> [4] https://github.com/jclouds/jclouds/blob/master/apis/
> swift/src/main/java/org/jclouds/openstack/swift/
> blobstore/strategy/internal/ParallelMultipartUploadStrategy.java
> [5] https://github.com/jclouds/jclouds/blob/master/compute/
> src/main/java/org/jclouds/compute/internal/BaseComputeService.java
>
Re: Architecture of jClouds
Posted by Andrew Phillips <ap...@qrmedia.com>.
Hi Fotis
> I understand that jClouds generates the http calls for each concrete
> provider but I cannot find the exact class that does this and where it gets
> the information for the required attributes from.
Glad you're interested in jclouds! For a general overview of the
architectural concepts, you can try the 'Core Concepts' document on
the wiki page [1].
As regards the lower-level details, the HTTP call generation largely
happens in the RestAnnotationProcessor [2]. In many cases, concrete
implementations are encapsulated in "strategies" (e.g. [3, 4]) that
are injected abstraction layer classes like BaseComputeService [5] by
Guice.
Luckily, you'll find that BaseBlobStore is somewhat less complex than
BaseComputeService ;-)
This is only a very high-level sketch: please feel free to hop on to
IRC for further discussion!
ap
[1] https://wiki.apache.org/jclouds/Core%20Concepts
[2]
https://github.com/jclouds/jclouds/blob/master/core/src/main/java/org/jclouds/rest/internal/RestAnnotationProcessor.java
[3]
https://github.com/jclouds/jclouds/blob/master/providers/aws-ec2/src/main/java/org/jclouds/aws/ec2/compute/strategy/AWSEC2GetNodeMetadataStrategy.java
[4]
https://github.com/jclouds/jclouds/blob/master/apis/swift/src/main/java/org/jclouds/openstack/swift/blobstore/strategy/internal/ParallelMultipartUploadStrategy.java
[5]
https://github.com/jclouds/jclouds/blob/master/compute/src/main/java/org/jclouds/compute/internal/BaseComputeService.java