You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jclouds.apache.org by "Ross Gardler (MS OPEN TECH)" <Ro...@microsoft.com> on 2015/02/05 00:11:24 UTC
Contributing Azure support
Hi JClouds folks,
We'd like to contribute Azure support to JClouds. We already have a fairly complete implementation of compute and storage and plan to work further on this implementation. However, this implementation is built against an the Azure Java SDK rather than the REST API. From conversations with some folks familiar with JClouds I understand that this is not the usual way of doing things here.
There are good reasons why we've implemented it this way. We can go into those if you want, but it will make no real difference since that's the working code we have available. It is feasible that we'd consider moving this over to a REST API if active community members are going to help with the work, but if it's just our ongoing contributions then continuing with the work already done is our preference. Rather than discussing it endlessly I'd propose a code talks approach.
Having poked around a fair bit it looks to me like your model is to have new code worked on in a lab at jclouds-labs. Given the SDK vs REST API approach it would seem sensible to start this effort as a lab anyway. That is using a lab would give time for both code maturity and for the community to evaluate the approach.
I also note that there is some existing work on Azure compute in https://github.com/jclouds/jclouds-labs/tree/master/azurecompute it would make sense to have our code in the repo so that those working on the two different implementations can avoid duplicating effort wherever possible and community members have access to both implementations in order to provide feedback.
How do you want to proceed? Shall we go ahead and make the contribution via that repository? This will provide an early opportunity for feedback and code commentary.
(NB both myself and Eduard Koller are ASF committers so the legal side of the process is clear, just asking about the community contribution process here in JClouds land)
Ross
RE: Contributing Azure support
Posted by "Ross Gardler (MS OPEN TECH)" <Ro...@microsoft.com>.
Makes sense.
Future contributions shouldn't be as big. We can be patient while you review this one :-)
Sent from my Windows Phone
________________________________
From: Ignasi Barrera<ma...@apache.org>
Sent: 2/10/2015 1:02 AM
To: dev@jclouds.apache.org<ma...@jclouds.apache.org>
Subject: Re: Contributing Azure support
Thanks Eduard!
I will have a look at the PR later today/tomorrow. It is quite big, so
it may take some time to review and I want to make sure I have time to
review it properly.
Regarding the issue, there was already JCLOUDS-664 [1] to track this.
As mentioned in previous threads, there are a couple contributors that
were already working on Azure and their work is being tracked there. I
think it would be better to have both approaches (the MS SDK and the
rest api one) tracked on the same issue, so people have the whole
context and understand why there are two implementations. If you're OK
with this, I'll close JCLOUDS-818 as a duplicate.
Thanks!
Ignasi
[1] https://issues.apache.org/jira/browse/JCLOUDS-664
On 9 February 2015 at 20:50, Eduard Koller (MS OPEN TECH)
<ed...@microsoft.com> wrote:
> Hi everyone,
>
> I created the following JIRA issue and pull request for the initial contribution
>
> https://issues.apache.org/jira/browse/JCLOUDS-818
> https://github.com/jclouds/jclouds-labs/pull/132
>
> Thank you,
> Eduard
>
>
>
> -----Original Message-----
> From: Ross Gardler (MS OPEN TECH) [mailto:Ross.Gardler@microsoft.com]
> Sent: Friday, February 6, 2015 8:55 AM
> To: dev@jclouds.apache.org
> Subject: RE: Contributing Azure support
>
> At this time we have no intention of migrating to REST, that may change, but I'm not able to guarantee it at this time. If the community want to do the REST work then we'll work alongside that. This is a tried and tested mechanisms in Apache projects to allow parallel approaches are explored simultaneously. One approach will win out, either we'll agree REST is the way to go, our we'll agree there are other approaches that work.
>
> We could spend thousands of words on discussing this in the abstract case, but I'd rather get the code out there and discuss it in concrete form.
>
> There are very good reasons for doing it using the SDK, there are very good reasons for insisting on the REST approach. We should address each of these reasons as they come up in code. I don't believe it is helpful to make a blanket statement of x is better than y.
>
> Internally we are evaluating the REST vs SDK approach. Once that evaluation is complete we can engage in productive discussion about the best approach. But, at this time, I don't want to imply it is a given that we will spend our time contributing to a REST implementation. It isn't clear that such an approach is the right thing for JClouds (it's also unclear whether or approach is the right thing in the general case).
>
> What I am committing to is spending time addressing the issues at hand, in the most open and constructive way possible. I hope the JClouds community will be equally open to this approach. Together we will find the optimal approach.
>
> I believe our goal is the same as yours. A truly valuable implementation of JClouds Azure, where value is measured in terms of both code maintenance and usability.
>
> Ross
>
> Sent from my Windows Phone
> ________________________________
> From: Ignasi Barrera<ma...@apache.org>
> Sent: 2/6/2015 1:24 AM
> To: dev@jclouds.apache.org<ma...@jclouds.apache.org>
> Subject: Re: Contributing Azure support
>
> The only thing that worries me about the SDK vs REST api is the uncertainity of "when" that change (required for promotion) will be made.
>
> We have always stated and made efforts to make it clear that providers in labs are not production ready. We can make non-backwards compatible changes there, completely refactor the interfaces, or even drop providers if there is no hope they can be promoted. However, despite all these well-known statements, people still uses providers in labs, and we have to (up to a certain level) care about that. As Andrea said, Apache Brooklyn will use the Azure support and, as other users, it will need it to be "jclouds compliant"; it will expect to provide the same feature set and architecture than other providers.
>
> We can live without this while in labs, but... when would the change/refactor to the rest API happen? This uncertainity is my only worry. As you said, in X months the initial contributors might not be active maintainers (it is ok, we can maintain and evolve it as long as we have a testing account), but if at that point the provider still uses the SDK, we'll be at a difficult position: we might not be able to assume the effort to rework the provider but there might be active users of it. We'll have a provider with very little hope to be promoted, but users depending on it, and would be bad for everyone.
>
> Experience shows that we should work on the tough parts when everyone is full of energy, and that moment is now, when starting the development.
>
> Also there are being active contributions to the existing Azure compute, and Andrea and Bhathiya are working on it now. They're doing a good job and I think we have to take that into account too, so I would propose the following:
>
> * Raise a pull request to add your provider to labs. This way we can see the ComputeService implementation and discuss it.
> * Use that implementation as a reference of "what we want", and the API calls that are needed to implement it.
> * Then implement those REST api calls in the existing azure compute provider.
> * And finally migrate the existing ComputeService implementation to use those rest api calls.
>
> This way I think we can join efforts: Andrea and Bhathiya already have good knowledge of how jclouds works, and them (and I) can start building the rest api calls based on the discussions of the ComputeService implementation. With your knowledge of how Azure works and how it should be used we can build the requirements for the ComputeService implementation, and join forces to build the underlying layer.
>
> Does this sound like a good plan?
>
> On 5 February 2015 at 19:18, Andrew Phillips <ap...@qrmedia.com> wrote:
>>> Eduard (who has also joined this list) will work on the initial
>>> contribution in the coming days and we can start doing concrete stuff.
>>
>>
>> Great! Looking forward to seeing the code doing the talking ;-)
>>
>> Regards
>>
>> ap
Re: Contributing Azure support
Posted by Ignasi Barrera <na...@apache.org>.
Thanks Eduard!
I will have a look at the PR later today/tomorrow. It is quite big, so
it may take some time to review and I want to make sure I have time to
review it properly.
Regarding the issue, there was already JCLOUDS-664 [1] to track this.
As mentioned in previous threads, there are a couple contributors that
were already working on Azure and their work is being tracked there. I
think it would be better to have both approaches (the MS SDK and the
rest api one) tracked on the same issue, so people have the whole
context and understand why there are two implementations. If you're OK
with this, I'll close JCLOUDS-818 as a duplicate.
Thanks!
Ignasi
[1] https://issues.apache.org/jira/browse/JCLOUDS-664
On 9 February 2015 at 20:50, Eduard Koller (MS OPEN TECH)
<ed...@microsoft.com> wrote:
> Hi everyone,
>
> I created the following JIRA issue and pull request for the initial contribution
>
> https://issues.apache.org/jira/browse/JCLOUDS-818
> https://github.com/jclouds/jclouds-labs/pull/132
>
> Thank you,
> Eduard
>
>
>
> -----Original Message-----
> From: Ross Gardler (MS OPEN TECH) [mailto:Ross.Gardler@microsoft.com]
> Sent: Friday, February 6, 2015 8:55 AM
> To: dev@jclouds.apache.org
> Subject: RE: Contributing Azure support
>
> At this time we have no intention of migrating to REST, that may change, but I'm not able to guarantee it at this time. If the community want to do the REST work then we'll work alongside that. This is a tried and tested mechanisms in Apache projects to allow parallel approaches are explored simultaneously. One approach will win out, either we'll agree REST is the way to go, our we'll agree there are other approaches that work.
>
> We could spend thousands of words on discussing this in the abstract case, but I'd rather get the code out there and discuss it in concrete form.
>
> There are very good reasons for doing it using the SDK, there are very good reasons for insisting on the REST approach. We should address each of these reasons as they come up in code. I don't believe it is helpful to make a blanket statement of x is better than y.
>
> Internally we are evaluating the REST vs SDK approach. Once that evaluation is complete we can engage in productive discussion about the best approach. But, at this time, I don't want to imply it is a given that we will spend our time contributing to a REST implementation. It isn't clear that such an approach is the right thing for JClouds (it's also unclear whether or approach is the right thing in the general case).
>
> What I am committing to is spending time addressing the issues at hand, in the most open and constructive way possible. I hope the JClouds community will be equally open to this approach. Together we will find the optimal approach.
>
> I believe our goal is the same as yours. A truly valuable implementation of JClouds Azure, where value is measured in terms of both code maintenance and usability.
>
> Ross
>
> Sent from my Windows Phone
> ________________________________
> From: Ignasi Barrera<ma...@apache.org>
> Sent: 2/6/2015 1:24 AM
> To: dev@jclouds.apache.org<ma...@jclouds.apache.org>
> Subject: Re: Contributing Azure support
>
> The only thing that worries me about the SDK vs REST api is the uncertainity of "when" that change (required for promotion) will be made.
>
> We have always stated and made efforts to make it clear that providers in labs are not production ready. We can make non-backwards compatible changes there, completely refactor the interfaces, or even drop providers if there is no hope they can be promoted. However, despite all these well-known statements, people still uses providers in labs, and we have to (up to a certain level) care about that. As Andrea said, Apache Brooklyn will use the Azure support and, as other users, it will need it to be "jclouds compliant"; it will expect to provide the same feature set and architecture than other providers.
>
> We can live without this while in labs, but... when would the change/refactor to the rest API happen? This uncertainity is my only worry. As you said, in X months the initial contributors might not be active maintainers (it is ok, we can maintain and evolve it as long as we have a testing account), but if at that point the provider still uses the SDK, we'll be at a difficult position: we might not be able to assume the effort to rework the provider but there might be active users of it. We'll have a provider with very little hope to be promoted, but users depending on it, and would be bad for everyone.
>
> Experience shows that we should work on the tough parts when everyone is full of energy, and that moment is now, when starting the development.
>
> Also there are being active contributions to the existing Azure compute, and Andrea and Bhathiya are working on it now. They're doing a good job and I think we have to take that into account too, so I would propose the following:
>
> * Raise a pull request to add your provider to labs. This way we can see the ComputeService implementation and discuss it.
> * Use that implementation as a reference of "what we want", and the API calls that are needed to implement it.
> * Then implement those REST api calls in the existing azure compute provider.
> * And finally migrate the existing ComputeService implementation to use those rest api calls.
>
> This way I think we can join efforts: Andrea and Bhathiya already have good knowledge of how jclouds works, and them (and I) can start building the rest api calls based on the discussions of the ComputeService implementation. With your knowledge of how Azure works and how it should be used we can build the requirements for the ComputeService implementation, and join forces to build the underlying layer.
>
> Does this sound like a good plan?
>
> On 5 February 2015 at 19:18, Andrew Phillips <ap...@qrmedia.com> wrote:
>>> Eduard (who has also joined this list) will work on the initial
>>> contribution in the coming days and we can start doing concrete stuff.
>>
>>
>> Great! Looking forward to seeing the code doing the talking ;-)
>>
>> Regards
>>
>> ap
RE: Contributing Azure support
Posted by "Eduard Koller (MS OPEN TECH)" <ed...@microsoft.com>.
Hi everyone,
I created the following JIRA issue and pull request for the initial contribution
https://issues.apache.org/jira/browse/JCLOUDS-818
https://github.com/jclouds/jclouds-labs/pull/132
Thank you,
Eduard
-----Original Message-----
From: Ross Gardler (MS OPEN TECH) [mailto:Ross.Gardler@microsoft.com]
Sent: Friday, February 6, 2015 8:55 AM
To: dev@jclouds.apache.org
Subject: RE: Contributing Azure support
At this time we have no intention of migrating to REST, that may change, but I'm not able to guarantee it at this time. If the community want to do the REST work then we'll work alongside that. This is a tried and tested mechanisms in Apache projects to allow parallel approaches are explored simultaneously. One approach will win out, either we'll agree REST is the way to go, our we'll agree there are other approaches that work.
We could spend thousands of words on discussing this in the abstract case, but I'd rather get the code out there and discuss it in concrete form.
There are very good reasons for doing it using the SDK, there are very good reasons for insisting on the REST approach. We should address each of these reasons as they come up in code. I don't believe it is helpful to make a blanket statement of x is better than y.
Internally we are evaluating the REST vs SDK approach. Once that evaluation is complete we can engage in productive discussion about the best approach. But, at this time, I don't want to imply it is a given that we will spend our time contributing to a REST implementation. It isn't clear that such an approach is the right thing for JClouds (it's also unclear whether or approach is the right thing in the general case).
What I am committing to is spending time addressing the issues at hand, in the most open and constructive way possible. I hope the JClouds community will be equally open to this approach. Together we will find the optimal approach.
I believe our goal is the same as yours. A truly valuable implementation of JClouds Azure, where value is measured in terms of both code maintenance and usability.
Ross
Sent from my Windows Phone
________________________________
From: Ignasi Barrera<ma...@apache.org>
Sent: 2/6/2015 1:24 AM
To: dev@jclouds.apache.org<ma...@jclouds.apache.org>
Subject: Re: Contributing Azure support
The only thing that worries me about the SDK vs REST api is the uncertainity of "when" that change (required for promotion) will be made.
We have always stated and made efforts to make it clear that providers in labs are not production ready. We can make non-backwards compatible changes there, completely refactor the interfaces, or even drop providers if there is no hope they can be promoted. However, despite all these well-known statements, people still uses providers in labs, and we have to (up to a certain level) care about that. As Andrea said, Apache Brooklyn will use the Azure support and, as other users, it will need it to be "jclouds compliant"; it will expect to provide the same feature set and architecture than other providers.
We can live without this while in labs, but... when would the change/refactor to the rest API happen? This uncertainity is my only worry. As you said, in X months the initial contributors might not be active maintainers (it is ok, we can maintain and evolve it as long as we have a testing account), but if at that point the provider still uses the SDK, we'll be at a difficult position: we might not be able to assume the effort to rework the provider but there might be active users of it. We'll have a provider with very little hope to be promoted, but users depending on it, and would be bad for everyone.
Experience shows that we should work on the tough parts when everyone is full of energy, and that moment is now, when starting the development.
Also there are being active contributions to the existing Azure compute, and Andrea and Bhathiya are working on it now. They're doing a good job and I think we have to take that into account too, so I would propose the following:
* Raise a pull request to add your provider to labs. This way we can see the ComputeService implementation and discuss it.
* Use that implementation as a reference of "what we want", and the API calls that are needed to implement it.
* Then implement those REST api calls in the existing azure compute provider.
* And finally migrate the existing ComputeService implementation to use those rest api calls.
This way I think we can join efforts: Andrea and Bhathiya already have good knowledge of how jclouds works, and them (and I) can start building the rest api calls based on the discussions of the ComputeService implementation. With your knowledge of how Azure works and how it should be used we can build the requirements for the ComputeService implementation, and join forces to build the underlying layer.
Does this sound like a good plan?
On 5 February 2015 at 19:18, Andrew Phillips <ap...@qrmedia.com> wrote:
>> Eduard (who has also joined this list) will work on the initial
>> contribution in the coming days and we can start doing concrete stuff.
>
>
> Great! Looking forward to seeing the code doing the talking ;-)
>
> Regards
>
> ap
RE: Contributing Azure support
Posted by "Ross Gardler (MS OPEN TECH)" <Ro...@microsoft.com>.
At this time we have no intention of migrating to REST, that may change, but I'm not able to guarantee it at this time. If the community want to do the REST work then we'll work alongside that. This is a tried and tested mechanisms in Apache projects to allow parallel approaches are explored simultaneously. One approach will win out, either we'll agree REST is the way to go, our we'll agree there are other approaches that work.
We could spend thousands of words on discussing this in the abstract case, but I'd rather get the code out there and discuss it in concrete form.
There are very good reasons for doing it using the SDK, there are very good reasons for insisting on the REST approach. We should address each of these reasons as they come up in code. I don't believe it is helpful to make a blanket statement of x is better than y.
Internally we are evaluating the REST vs SDK approach. Once that evaluation is complete we can engage in productive discussion about the best approach. But, at this time, I don't want to imply it is a given that we will spend our time contributing to a REST implementation. It isn't clear that such an approach is the right thing for JClouds (it's also unclear whether or approach is the right thing in the general case).
What I am committing to is spending time addressing the issues at hand, in the most open and constructive way possible. I hope the JClouds community will be equally open to this approach. Together we will find the optimal approach.
I believe our goal is the same as yours. A truly valuable implementation of JClouds Azure, where value is measured in terms of both code maintenance and usability.
Ross
Sent from my Windows Phone
________________________________
From: Ignasi Barrera<ma...@apache.org>
Sent: 2/6/2015 1:24 AM
To: dev@jclouds.apache.org<ma...@jclouds.apache.org>
Subject: Re: Contributing Azure support
The only thing that worries me about the SDK vs REST api is the
uncertainity of "when" that change (required for promotion) will be
made.
We have always stated and made efforts to make it clear that providers
in labs are not production ready. We can make non-backwards compatible
changes there, completely refactor the interfaces, or even drop
providers if there is no hope they can be promoted. However, despite
all these well-known statements, people still uses providers in labs,
and we have to (up to a certain level) care about that. As Andrea
said, Apache Brooklyn will use the Azure support and, as other users,
it will need it to be "jclouds compliant"; it will expect to provide
the same feature set and architecture than other providers.
We can live without this while in labs, but... when would the
change/refactor to the rest API happen? This uncertainity is my only
worry. As you said, in X months the initial contributors might not be
active maintainers (it is ok, we can maintain and evolve it as long as
we have a testing account), but if at that point the provider still
uses the SDK, we'll be at a difficult position: we might not be able
to assume the effort to rework the provider but there might be active
users of it. We'll have a provider with very little hope to be
promoted, but users depending on it, and would be bad for everyone.
Experience shows that we should work on the tough parts when everyone
is full of energy, and that moment is now, when starting the
development.
Also there are being active contributions to the existing Azure
compute, and Andrea and Bhathiya are working on it now. They're doing
a good job and I think we have to take that into account too, so I
would propose the following:
* Raise a pull request to add your provider to labs. This way we can
see the ComputeService implementation and discuss it.
* Use that implementation as a reference of "what we want", and the
API calls that are needed to implement it.
* Then implement those REST api calls in the existing azure compute provider.
* And finally migrate the existing ComputeService implementation to
use those rest api calls.
This way I think we can join efforts: Andrea and Bhathiya already have
good knowledge of how jclouds works, and them (and I) can start
building the rest api calls based on the discussions of the
ComputeService implementation. With your knowledge of how Azure works
and how it should be used we can build the requirements for the
ComputeService implementation, and join forces to build the underlying
layer.
Does this sound like a good plan?
On 5 February 2015 at 19:18, Andrew Phillips <ap...@qrmedia.com> wrote:
>> Eduard (who has also joined this list) will work on the initial
>> contribution in the coming days and we can start doing concrete stuff.
>
>
> Great! Looking forward to seeing the code doing the talking ;-)
>
> Regards
>
> ap
Re: Contributing Azure support
Posted by Ignasi Barrera <na...@apache.org>.
The only thing that worries me about the SDK vs REST api is the
uncertainity of "when" that change (required for promotion) will be
made.
We have always stated and made efforts to make it clear that providers
in labs are not production ready. We can make non-backwards compatible
changes there, completely refactor the interfaces, or even drop
providers if there is no hope they can be promoted. However, despite
all these well-known statements, people still uses providers in labs,
and we have to (up to a certain level) care about that. As Andrea
said, Apache Brooklyn will use the Azure support and, as other users,
it will need it to be "jclouds compliant"; it will expect to provide
the same feature set and architecture than other providers.
We can live without this while in labs, but... when would the
change/refactor to the rest API happen? This uncertainity is my only
worry. As you said, in X months the initial contributors might not be
active maintainers (it is ok, we can maintain and evolve it as long as
we have a testing account), but if at that point the provider still
uses the SDK, we'll be at a difficult position: we might not be able
to assume the effort to rework the provider but there might be active
users of it. We'll have a provider with very little hope to be
promoted, but users depending on it, and would be bad for everyone.
Experience shows that we should work on the tough parts when everyone
is full of energy, and that moment is now, when starting the
development.
Also there are being active contributions to the existing Azure
compute, and Andrea and Bhathiya are working on it now. They're doing
a good job and I think we have to take that into account too, so I
would propose the following:
* Raise a pull request to add your provider to labs. This way we can
see the ComputeService implementation and discuss it.
* Use that implementation as a reference of "what we want", and the
API calls that are needed to implement it.
* Then implement those REST api calls in the existing azure compute provider.
* And finally migrate the existing ComputeService implementation to
use those rest api calls.
This way I think we can join efforts: Andrea and Bhathiya already have
good knowledge of how jclouds works, and them (and I) can start
building the rest api calls based on the discussions of the
ComputeService implementation. With your knowledge of how Azure works
and how it should be used we can build the requirements for the
ComputeService implementation, and join forces to build the underlying
layer.
Does this sound like a good plan?
On 5 February 2015 at 19:18, Andrew Phillips <ap...@qrmedia.com> wrote:
>> Eduard (who has also joined this list) will work on the initial
>> contribution in the coming days and we can start doing concrete stuff.
>
>
> Great! Looking forward to seeing the code doing the talking ;-)
>
> Regards
>
> ap
RE: Contributing Azure support
Posted by Andrew Phillips <ap...@qrmedia.com>.
> Eduard (who has also joined this list) will work on the initial
> contribution in the coming days and we can start doing concrete stuff.
Great! Looking forward to seeing the code doing the talking ;-)
Regards
ap
RE: Contributing Azure support
Posted by "Ross Gardler (MS OPEN TECH)" <Ro...@microsoft.com>.
Thanks for the expanded info, very useful.
With respect to SDK vs REST API this is something we'll have to explore during development. We have working code today. It uses the SDK for (potentially) equally good reasons to those for JClouds preferring a standard way of doing things. At this time it is not our intention to move this code over to use the REST API, but we acknowledge that this means it is unlikely to move out of labs. Once the community can get their hands on the code we can see where it goes. If the momentum is such that it moves to the REST API then all good and we'll work with that momentum.
In other words, lets allow the code to do the talking (again acknowledging that the current position of the project is that the current code will not move from labs as a result of the existing policy on REST API).
Eduard (who has also joined this list) will work on the initial contribution in the coming days and we can start doing concrete stuff.
Thanks again for the great summary below.
Ross
Microsoft Open Technologies, Inc.
A subsidiary of Microsoft Corporation
-----Original Message-----
From: Ignasi Barrera [mailto:nacx@apache.org]
Sent: Thursday, February 5, 2015 2:02 AM
To: dev@jclouds.apache.org
Subject: Re: Contributing Azure support
Hi!
Many thanks for reaching us and for willing to help with Azure. We really look forward to having good support for it, so all contributions are very welcome.
As you already summarized, the right place to start is in jclouds-labs. There we can discuss all the implementation details and start testing and using the provider while it is under development, and once it is in a good shape, it will get promoted to the main repo.
To make that happen, the following must happen:
* It is being maintained: This usually means that we need to be able to run live tests against the provider and someone takes care of it.
In the case of Azure this shouldn't be an issue, as Microsoft provides limited free accounts for ASF committers that are enough to run the tests and hack on the provider. I've been using such an account while helping Adrian when he started the Azure compute provider, and it provides all we need, so I'd say this won't be an issue.
* It has an abstraction. This is the important point. We need the ComputeService abstraction implemented in order promote the provider.
jclouds is all about the abstractions and common interfaces, and this is the one that has to be implemented. That means:
- The interface itself has to be implemented, like in other compute providers.
- The AzureComputeServiceLiveTest must be created and should extend the BaseComputeServiceLiveTest [1]. You can look at the DigitalOcean one [2] for an example.
- The AzureTemplateBuilderLiveTest must be created and should extend the BaseTemplateBuilderLiveTest [3]. You can have a look at the HPCloud one [4] for an example.
These test classes are the "ComputeService contract". Once those live tests are passing, we know the abstraction is properly implemented and working as expected.
* Code is up-to-date (tests, style, best practices). This is also important, and is especially relevant the fact that your implementation is currently using the Azure SDK. I completely understand that it can be more convenient to use it, but I have to say that the implementation should use the existing jclouds HTTP layer in order to be promoted. It is not just because "we do it this way"; there are several reasons behind this:
- jclouds provides a driver mechanism that allows users to choose their preferred HTTP driver. Currently we support the Java default HttpUrlConnection, the Apache HTTP Client, and OkHttp. This one allows us to be Android friendly and use PATCH verbs and other stuff that is not supported in the default JRE implementation, so we need to make sure that users still have this choice when using Azure. The driver mechanism must work.
- jclouds also provides a generic way of handling failure at the HTTP layer. It provides a generic fallback mechanism that allows every provider to configure the behavior when failure happens. This fallback delegation is generic and all providers benefit from it, and Azure shouldn't be an exception. We should be able to provide custom fallbacks to customize the behavior upon failure.
- jclouds provides a generic retry policy that allows users to configure how and when failed request are retried. There are several implementations, but users can configure a more convenient one, if needed.
- It also provides a generic way to configure request timeouts "per API method". Users can configure the timeout for each call, if they need to.
These are the most relevant points (there are more but I think there's no need to enumerate everything here) that illustrate why we *need* and *require* providers and apis to use the jclouds built-in HTTP layer. Using the Azure SDK would mean that none of the above would be available, and that is something we can't justify just because "Azure was developed that way".
So, to summarize, the development of the provider should be started in jclouds-labs, and the goals should be to implement the ComputeService interface and have the mentioned tests passing. Once that is working, we can safely say the provider works as expected.
Regarding the usage of the Azure API, I'd say we can live with it in labs, but changing the provider to use the jclouds HTTP mechanisms is a must to have it promoted. I don't really know what is the best way to proceed: have the provider "as-is" now merged into labs and start working from there, or adapt what you already have and contribute to the existing azure provider. Take into account that we are trying to use the "labs" repo just for development, and providers should have a limited TTL there. We should try to avoid partially developed providers from staying there forever, so let's use this thread to come up with a plan to put Azure in the path to promotion
Thanks again for your intereset!
And please, don't take this email as a "no" or "not this way at all".
I've just tried to explain the reasons behind what we require. We really look forward to having Azure in and your intereset in contributing is very much appreciated.
I'll be more than happy to help with the development of the provider, provide guidance, and to align the code with what jclouds needs.
Thanks!
Ignasi
[1] https://github.com/jclouds/jclouds/blob/master/compute/src/test/java/org/jclouds/compute/internal/BaseComputeServiceLiveTest.java
[2] https://github.com/jclouds/jclouds-labs/blob/master/digitalocean/src/test/java/org/jclouds/digitalocean/compute/DigitalOceanComputeServiceLiveTest.java
[3] https://github.com/jclouds/jclouds/blob/master/compute/src/test/java/org/jclouds/compute/internal/BaseTemplateBuilderLiveTest.java
[4] https://github.com/jclouds/jclouds/blob/master/providers/hpcloud-compute/src/test/java/org/jclouds/hpcloud/compute/compute/HPCloudComputeTemplateBuilderLiveTest.java
On 5 February 2015 at 07:29, Ross Gardler (MS OPEN TECH) <Ro...@microsoft.com> wrote:
> Great thanks for the pointer. Even more thanks to Zack who took the
> time to summarize that thread (if only more folks did that in other
> projects I work on, including me!)
>
> Zacks summary, for the purposes of the archives is:
>
> Currently, most of us agree that the following criteria should be met, at a minimum, before an API is promoted to jclouds/jclouds:
> - It is being maintained
> - It has an abstraction
> - Code is up-to-date (tests, style, best practices)
>
> This all seems sensible. I don't want to start that thread again but I
> would point out that the first criteria (it is being maintained) might
> be true on promotion to core but no longer be true in x months time.
> Not a problem today, but one day it is possible there will need to be
> a process for retirement from core too. For the Azure support we'll
> focus on getting it into core though :-)
>
> Looking at other modules that are in core I see there isn't a consistent level of support for compute and storage (our current area of focus). Is there a minimum bar at this point? Perhaps a test suite? No problem if this is still being defined, we can use the Azure support as a use case for defining that minimum bar if it is appropriate.
>
> Ross
>
> -----Original Message-----
> From: Andrew Phillips [mailto:aphillips@qrmedia.com]
> Sent: Wednesday, February 4, 2015 4:31 PM
> To: dev@jclouds.apache.org
> Subject: RE: Contributing Azure support
>
> Hi Ross
>
>> Thanks for the pointer John. Why is Blob in core and Compute in Labs?
>> What is the criteria for leaving labs?
>
> See http://markmail.org/thread/zvsytdvzfeg4w2zi. In short, it's still an ongoing discussion, but support for one of the abstraction layers (which is true for both Azure BlobStore and Compute) is likely to be one of the key ones.
>
> Great to see interest from your side in contributing!
>
> Regards
>
> ap
Re: Contributing Azure support
Posted by Andrea Turli <an...@gmail.com>.
Hi!
+1; will be great to have fully functioning azure support!
With great coincidence of timing, I submitted a PR yesterday [1] to the
jclouds-labs for azurecompute. We need Azure support for the downstream
Apache Brooklyn [2] and Clocker [3] projects. Microsoft have kindly
provided us a sponsored account to help with this.
This PR follows the HTTP-driver approach. This is still work in progress -
it lacks many of the mock + live tests and some minor operations
(reboot/resume/suspendNode and getImage). Bhathiya has also been working on
this [4], and we’re working together to merge the work we’ve each done.
It would be great to work together to complete the jclouds-azurecompute!
Ross, it would be useful to better understand the implementation you
referred to, and to see how we can merge these efforts. Perhaps submitting
a PR to jclouds-labs for the azurecompute would be a good early way to give
visibility of this code?
Best,
Andrea
[1]: https://github.com/jclouds/jclouds-labs/pull/131
[2] https://brooklyn.incubator.apache.org/
[3] http://clocker.io
[4] https://github.com/jclouds/jclouds-labs/pull/118
On Thu Feb 05 2015 at 2:54:22 PM Andrew Phillips <ap...@qrmedia.com>
wrote:
> > I'll be more than happy to help with the development of the provider,
> > provide guidance, and to align the code with what jclouds needs.
>
> Thanks for putting together this comprehensive answer, Ignasi! For
> future reference, I've tried to formulate it more generally here:
>
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=51809189
>
> Regards
>
> ap
>
Re: Contributing Azure support
Posted by Andrew Phillips <ap...@qrmedia.com>.
> I'll be more than happy to help with the development of the provider,
> provide guidance, and to align the code with what jclouds needs.
Thanks for putting together this comprehensive answer, Ignasi! For
future reference, I've tried to formulate it more generally here:
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=51809189
Regards
ap
Re: Contributing Azure support
Posted by Ignasi Barrera <na...@apache.org>.
Hi!
Many thanks for reaching us and for willing to help with Azure. We
really look forward to having good support for it, so all
contributions are very welcome.
As you already summarized, the right place to start is in
jclouds-labs. There we can discuss all the implementation details and
start testing and using the provider while it is under development,
and once it is in a good shape, it will get promoted to the main repo.
To make that happen, the following must happen:
* It is being maintained: This usually means that we need to be able
to run live tests against the provider and someone takes care of it.
In the case of Azure this shouldn't be an issue, as Microsoft provides
limited free accounts for ASF committers that are enough to run the
tests and hack on the provider. I've been using such an account while
helping Adrian when he started the Azure compute provider, and it
provides all we need, so I'd say this won't be an issue.
* It has an abstraction. This is the important point. We need the
ComputeService abstraction implemented in order promote the provider.
jclouds is all about the abstractions and common interfaces, and this
is the one that has to be implemented. That means:
- The interface itself has to be implemented, like in other compute providers.
- The AzureComputeServiceLiveTest must be created and should extend
the BaseComputeServiceLiveTest [1]. You can look at the DigitalOcean
one [2] for an example.
- The AzureTemplateBuilderLiveTest must be created and should extend
the BaseTemplateBuilderLiveTest [3]. You can have a look at the
HPCloud one [4] for an example.
These test classes are the "ComputeService contract". Once those live
tests are passing, we know the abstraction is properly implemented and
working as expected.
* Code is up-to-date (tests, style, best practices). This is also
important, and is especially relevant the fact that your
implementation is currently using the Azure SDK. I completely
understand that it can be more convenient to use it, but I have to say
that the implementation should use the existing jclouds HTTP layer in
order to be promoted. It is not just because "we do it this way";
there are several reasons behind this:
- jclouds provides a driver mechanism that allows users to choose
their preferred HTTP driver. Currently we support the Java default
HttpUrlConnection, the Apache HTTP Client, and OkHttp. This one allows
us to be Android friendly and use PATCH verbs and other stuff that is
not supported in the default JRE implementation, so we need to make
sure that users still have this choice when using Azure. The driver
mechanism must work.
- jclouds also provides a generic way of handling failure at the
HTTP layer. It provides a generic fallback mechanism that allows every
provider to configure the behavior when failure happens. This fallback
delegation is generic and all providers benefit from it, and Azure
shouldn't be an exception. We should be able to provide custom
fallbacks to customize the behavior upon failure.
- jclouds provides a generic retry policy that allows users to
configure how and when failed request are retried. There are several
implementations, but users can configure a more convenient one, if
needed.
- It also provides a generic way to configure request timeouts "per
API method". Users can configure the timeout for each call, if they
need to.
These are the most relevant points (there are more but I think there's
no need to enumerate everything here) that illustrate why we *need*
and *require* providers and apis to use the jclouds built-in HTTP
layer. Using the Azure SDK would mean that none of the above would be
available, and that is something we can't justify just because "Azure
was developed that way".
So, to summarize, the development of the provider should be started in
jclouds-labs, and the goals should be to implement the ComputeService
interface and have the mentioned tests passing. Once that is working,
we can safely say the provider works as expected.
Regarding the usage of the Azure API, I'd say we can live with it in
labs, but changing the provider to use the jclouds HTTP mechanisms is
a must to have it promoted. I don't really know what is the best way
to proceed: have the provider "as-is" now merged into labs and start
working from there, or adapt what you already have and contribute to
the existing azure provider. Take into account that we are trying to
use the "labs" repo just for development, and providers should have a
limited TTL there. We should try to avoid partially developed
providers from staying there forever, so let's use this thread to come
up with a plan to put Azure in the path to promotion
Thanks again for your intereset!
And please, don't take this email as a "no" or "not this way at all".
I've just tried to explain the reasons behind what we require. We
really look forward to having Azure in and your intereset in
contributing is very much appreciated.
I'll be more than happy to help with the development of the provider,
provide guidance, and to align the code with what jclouds needs.
Thanks!
Ignasi
[1] https://github.com/jclouds/jclouds/blob/master/compute/src/test/java/org/jclouds/compute/internal/BaseComputeServiceLiveTest.java
[2] https://github.com/jclouds/jclouds-labs/blob/master/digitalocean/src/test/java/org/jclouds/digitalocean/compute/DigitalOceanComputeServiceLiveTest.java
[3] https://github.com/jclouds/jclouds/blob/master/compute/src/test/java/org/jclouds/compute/internal/BaseTemplateBuilderLiveTest.java
[4] https://github.com/jclouds/jclouds/blob/master/providers/hpcloud-compute/src/test/java/org/jclouds/hpcloud/compute/compute/HPCloudComputeTemplateBuilderLiveTest.java
On 5 February 2015 at 07:29, Ross Gardler (MS OPEN TECH)
<Ro...@microsoft.com> wrote:
> Great thanks for the pointer. Even more thanks to Zack who took the time to summarize that thread (if only more folks did that in other projects I work on, including me!)
>
> Zacks summary, for the purposes of the archives is:
>
> Currently, most of us agree that the following criteria should be met, at a minimum, before an API is promoted to jclouds/jclouds:
> - It is being maintained
> - It has an abstraction
> - Code is up-to-date (tests, style, best practices)
>
> This all seems sensible. I don't want to start that thread again but I would point out that the first criteria (it is being maintained) might be true on promotion to core but no longer be true in x months time. Not a problem today, but one day it is possible there will need to be a process for retirement from core too. For the Azure support we'll focus on getting it into core though :-)
>
> Looking at other modules that are in core I see there isn't a consistent level of support for compute and storage (our current area of focus). Is there a minimum bar at this point? Perhaps a test suite? No problem if this is still being defined, we can use the Azure support as a use case for defining that minimum bar if it is appropriate.
>
> Ross
>
> -----Original Message-----
> From: Andrew Phillips [mailto:aphillips@qrmedia.com]
> Sent: Wednesday, February 4, 2015 4:31 PM
> To: dev@jclouds.apache.org
> Subject: RE: Contributing Azure support
>
> Hi Ross
>
>> Thanks for the pointer John. Why is Blob in core and Compute in
>> Labs? What is the criteria for leaving labs?
>
> See http://markmail.org/thread/zvsytdvzfeg4w2zi. In short, it's still an ongoing discussion, but support for one of the abstraction layers (which is true for both Azure BlobStore and Compute) is likely to be one of the key ones.
>
> Great to see interest from your side in contributing!
>
> Regards
>
> ap
RE: Contributing Azure support
Posted by "Ross Gardler (MS OPEN TECH)" <Ro...@microsoft.com>.
Great thanks for the pointer. Even more thanks to Zack who took the time to summarize that thread (if only more folks did that in other projects I work on, including me!)
Zacks summary, for the purposes of the archives is:
Currently, most of us agree that the following criteria should be met, at a minimum, before an API is promoted to jclouds/jclouds:
- It is being maintained
- It has an abstraction
- Code is up-to-date (tests, style, best practices)
This all seems sensible. I don't want to start that thread again but I would point out that the first criteria (it is being maintained) might be true on promotion to core but no longer be true in x months time. Not a problem today, but one day it is possible there will need to be a process for retirement from core too. For the Azure support we'll focus on getting it into core though :-)
Looking at other modules that are in core I see there isn't a consistent level of support for compute and storage (our current area of focus). Is there a minimum bar at this point? Perhaps a test suite? No problem if this is still being defined, we can use the Azure support as a use case for defining that minimum bar if it is appropriate.
Ross
-----Original Message-----
From: Andrew Phillips [mailto:aphillips@qrmedia.com]
Sent: Wednesday, February 4, 2015 4:31 PM
To: dev@jclouds.apache.org
Subject: RE: Contributing Azure support
Hi Ross
> Thanks for the pointer John. Why is Blob in core and Compute in
> Labs? What is the criteria for leaving labs?
See http://markmail.org/thread/zvsytdvzfeg4w2zi. In short, it's still an ongoing discussion, but support for one of the abstraction layers (which is true for both Azure BlobStore and Compute) is likely to be one of the key ones.
Great to see interest from your side in contributing!
Regards
ap
RE: Contributing Azure support
Posted by Andrew Phillips <ap...@qrmedia.com>.
Hi Ross
> Thanks for the pointer John. Why is Blob in core and Compute in
> Labs? What is the criteria for leaving labs?
See http://markmail.org/thread/zvsytdvzfeg4w2zi. In short, it's still
an ongoing discussion, but support for one of the abstraction layers
(which is true for both Azure BlobStore and Compute) is likely to be
one of the key ones.
Great to see interest from your side in contributing!
Regards
ap
RE: Contributing Azure support
Posted by "Ross Gardler (MS OPEN TECH)" <Ro...@microsoft.com>.
Thanks for the pointer John. Why is Blob in core and Compute in Labs? What is the criteria for leaving labs?
Ross
Microsoft Open Technologies, Inc.
A subsidiary of Microsoft Corporation
-----Original Message-----
From: John Kew [mailto:john.kew@socrata.com]
Sent: Wednesday, February 4, 2015 3:47 PM
To: dev@jclouds.apache.org
Subject: Re: Contributing Azure support
There is also an existing azureblob implementation which we currently use in production:
https://github.com/jclouds/jclouds/tree/master/providers/azureblob
I took Adrian's original implementation and extended it for larger blob types. I don't particularly care about the implementation, and the notion of formal support pleases me (in fact I think I also pushed MS to add JClouds on roadmap since we were on the azure board), but it would be wise to check it out to verify compatibility.
-John
On Wed, Feb 4, 2015 at 3:11 PM, Ross Gardler (MS OPEN TECH) < Ross.Gardler@microsoft.com> wrote:
> Hi JClouds folks,
>
> We'd like to contribute Azure support to JClouds. We already have a
> fairly complete implementation of compute and storage and plan to work
> further on this implementation. However, this implementation is built
> against an the Azure Java SDK rather than the REST API. From
> conversations with some folks familiar with JClouds I understand that
> this is not the usual way of doing things here.
>
> There are good reasons why we've implemented it this way. We can go
> into those if you want, but it will make no real difference since
> that's the working code we have available. It is feasible that we'd
> consider moving this over to a REST API if active community members
> are going to help with the work, but if it's just our ongoing
> contributions then continuing with the work already done is our
> preference. Rather than discussing it endlessly I'd propose a code talks approach.
>
> Having poked around a fair bit it looks to me like your model is to
> have new code worked on in a lab at jclouds-labs. Given the SDK vs
> REST API approach it would seem sensible to start this effort as a lab
> anyway. That is using a lab would give time for both code maturity and
> for the community to evaluate the approach.
>
> I also note that there is some existing work on Azure compute in
> https://github.com/jclouds/jclouds-labs/tree/master/azurecompute it
> would make sense to have our code in the repo so that those working on
> the two different implementations can avoid duplicating effort
> wherever possible and community members have access to both
> implementations in order to provide feedback.
>
> How do you want to proceed? Shall we go ahead and make the
> contribution via that repository? This will provide an early
> opportunity for feedback and code commentary.
>
> (NB both myself and Eduard Koller are ASF committers so the legal side
> of the process is clear, just asking about the community contribution
> process here in JClouds land)
>
> Ross
>
>
Re: Contributing Azure support
Posted by John Kew <jo...@socrata.com>.
There is also an existing azureblob implementation which we currently use
in production:
https://github.com/jclouds/jclouds/tree/master/providers/azureblob
I took Adrian's original implementation and extended it for larger blob
types. I don't particularly care about the implementation, and the notion
of formal support pleases me (in fact I think I also pushed MS to add
JClouds on roadmap since we were on the azure board), but it would be wise
to check it out to verify compatibility.
-John
On Wed, Feb 4, 2015 at 3:11 PM, Ross Gardler (MS OPEN TECH) <
Ross.Gardler@microsoft.com> wrote:
> Hi JClouds folks,
>
> We'd like to contribute Azure support to JClouds. We already have a fairly
> complete implementation of compute and storage and plan to work further on
> this implementation. However, this implementation is built against an the
> Azure Java SDK rather than the REST API. From conversations with some folks
> familiar with JClouds I understand that this is not the usual way of doing
> things here.
>
> There are good reasons why we've implemented it this way. We can go into
> those if you want, but it will make no real difference since that's the
> working code we have available. It is feasible that we'd consider moving
> this over to a REST API if active community members are going to help with
> the work, but if it's just our ongoing contributions then continuing with
> the work already done is our preference. Rather than discussing it
> endlessly I'd propose a code talks approach.
>
> Having poked around a fair bit it looks to me like your model is to have
> new code worked on in a lab at jclouds-labs. Given the SDK vs REST API
> approach it would seem sensible to start this effort as a lab anyway. That
> is using a lab would give time for both code maturity and for the community
> to evaluate the approach.
>
> I also note that there is some existing work on Azure compute in
> https://github.com/jclouds/jclouds-labs/tree/master/azurecompute it would
> make sense to have our code in the repo so that those working on the two
> different implementations can avoid duplicating effort wherever possible
> and community members have access to both implementations in order to
> provide feedback.
>
> How do you want to proceed? Shall we go ahead and make the contribution
> via that repository? This will provide an early opportunity for feedback
> and code commentary.
>
> (NB both myself and Eduard Koller are ASF committers so the legal side of
> the process is clear, just asking about the community contribution process
> here in JClouds land)
>
> Ross
>
>