[DISCUSS] What is the criteria for something to be in the jclouds/jclouds repo?

[Everett Toews <ev...@RACKSPACE.COM>]

What is the criteria for something to be in the jclouds/jclouds repo [1]?

Before we go moving any more code around or removing any more apis/providers we need to step back and have some agreed upon criteria for what belongs in jclouds/jclouds and what doesn’t. If we don’t do that our decisions are effectively arbitrary and the questions of “What to do about X?” will continue to come up.

I took a shot at addressing a topic like this in "Criteria for moving an api/provider from a labs repo to core” [2] but I got zero replies. People seem to be unwilling or uninterested in tackling this question and I think jclouds is suffering because of it. It’s puzzling to me because in email threads or IRC chats most everybody agrees that the labs repos are a failed experiment. So let’s flip it around and discuss what belongs in jclouds/jclouds instead.

I know for a fact that many contributors have an opinion on this. Let’s hear it!

Thanks,
Everett

[1] https://github.com/jclouds/jclouds

Re: [DISCUSS] What is the criteria for something to be in the jclouds/jclouds repo?

[Ignasi Barrera <ig...@gmail.com>]

Chef is not an example for that, as it intergates with the
ComputeService and the portable Compute model. It integrates with the
node bootstrap process, so it makes sense to have it as part of the
jclouds core repo.

I don't think having separate repos for specific providers is a good
idea. Doing that would be pretty much the same than having the labs
repos: we would have repos that would potentially get out of sync from
the main one and we'll be again at the same point.

We have to work on aligning the existing providers with the
abstractions we have, and think about new abstractions that could make
sense. That is what will benefit jclouds, and that is what we should
take into account to promote providers.

On 3 December 2014 at 19:25, Zack Shoylev <za...@rackspace.com> wrote:
> If we want to only keep services with abstraction in jclouds/jclouds, then what about services that will probably never have an abstraction layer, such as chef, for example?
>
> If we are going for separate code repos for jclouds/jclouds and specific providers, then we still have the problems of:
> - releases are more problematic with multiple repos
> - PRs become more complex when they have to happen in multiple repos or depend on other PRs in jclouds/jclouds
>
> To recap, for the criteria, it seems everyone agrees on (as a minimum):
> - Steward
> - Code guidelines
> ________________________________________
> From: Ignasi Barrera [nacx@apache.org]
> Sent: Tuesday, December 02, 2014 2:31 PM
> To: dev@jclouds.apache.org
> Subject: Re: [DISCUSS] What is the criteria for something to be in the jclouds/jclouds repo?
>
> In my opinion there are several requirements to be met, all them mandatory:
>
> 1. New providers MUST have an abstraction (more on this below).
> 2. Must have a Steward, who maintains the provider, takes the issues,
> and runs the live tests periodically (at least before each release).
> 3. It must follow the coding guidelines. As Andrea mentioned,
> technical debt is a *real* blocker when it comes to refactor the core
> to fix structural issues. We can't keep adding it to the main repo.
> Huge providers that are out of date are only going to cause us
> trouble.
>
> Regarding the abstractions. We need to think about what jclouds is.
> jclouds is ALL about the abstractions. It is not an aggregator of any
> api that is labelled "cloud". Without the abstractions, jclouds makes
> no sense. Users don't come to jclouds looking for concrete APIs. They
> come looking for a portable toolkit. If you have a look at the mailing
> list, most questions are about the Blob and Compute abstraction. There
> are also many about the Neutron API, but that is *only* because those
> APIs have failed to create an abstraction. Users are asking for
> concrete APIs just because there is no abstraction for them.
>
> This said, the absence of an abstraction (take network or databases as
> an example) shouldn't be a blocker. Abstractions can be proposed in
> the mailing list, and even better discussed in a PR! We are all
> looking to improve jclouds to make it better for the users, and that
> means making it more portable. Let's work on adding an abstraction
> when that makes sense! But things have to be *real* and that work has
> to be done. Without it, it does not make sense to promote/keep such
> apis/providers.
>
> I.
>
>
> On 2 December 2014 at 18:44, Andrea Turli <an...@gmail.com> wrote:
>> Hi,
>>
>> I personally think we should consider a couple of additional criteria to
>> the good Zack's list.
>> The code should support the latest stable (recommended) version of the API.
>> The implementation should follow all the current guidelines (i. e. Use of
>> Auto Value and Service, MWS, no async API)  so that the new providers don't
>> introduce tech debt, this should be kept during reviews but we need some
>> standard way to "enforce" them.
>> MockTests and LiveTests have to be green.
>> The maintainer has to be able to run the LiveTests for each release.
>>
>> As contributor to labs repos this is my understanding of what it is needed
>> to get the provider out of labs, hope this helps.
>>
>> Andrea
>> Il 02/dic/2014 18:28 "Zack Shoylev" <za...@rackspace.com> ha scritto:
>>
>>> Possible criteria that have been discussed before:
>>>
>>> 1. If it is out of beta
>>> 2. If it has a maintainer
>>> 3. If it has a jclouds abstraction layer
>>> 4. Voted on a case by case basis (current way of doing things)
>>>
>>> Are there any more possible criteria I have left out? In the end it will
>>> be some mix of the above. My thoughts on this is that 2) is sufficient.
>>> -Zack
>>> ________________________________________
>>> From: Dancy, Chris [Chris.Dancy@pega.com]
>>> Sent: Tuesday, December 02, 2014 8:27 AM
>>> To: dev@jclouds.apache.org
>>> Subject: Re: [DISCUSS] What is the criteria for something to be in the
>>> jclouds/jclouds repo?
>>>
>>> As an occasional contributor, and currently coding a Shipyard provider
>>> (JCLOUDS-782), I do have a vested interest in how this ends up. IMO I
>>> think having a clear separation of ³core² projects vs ³baking² projects is
>>> a good idea but it seems that is not working out so great here? Are there
>>> other proposals on how to do this?
>>>
>>> On 12/2/14, 4:09 AM, "Inbar Stolberg" <in...@gmail.com> wrote:
>>>
>>> >I think openstack neutron should be in jcluods master currently it suffers
>>> >from being left out even though it is being updated and runs on openstack
>>> >api v2.
>>> >
>>> >regarding glance I think after
>>> >https://github.com/jclouds/jclouds-labs-openstack/pull/77 is merged we
>>> >should move it.
>>> >
>>> >regarding Ceilometer I think that taking to accout what you said Everett:
>>> >" in email threads or IRC chats most everybody agrees that the labs repos
>>> >are a failed experiment"
>>> >maybe its worth considering porting it to jclouds/jclouds from right after
>>> >the merge of :
>>> >https://github.com/jclouds/jclouds-labs-openstack/pull/166
>>> >
>>> >(which btw I am already going to use in production code...)
>>> >
>>> >my 2 cents
>>> >Regards Inbar
>>> >
>>> >
>>> >On Tue, Dec 2, 2014 at 2:06 AM, Everett Toews
>>> ><ev...@rackspace.com>
>>> >wrote:
>>> >
>>> >> What is the criteria for something to be in the jclouds/jclouds repo
>>> >>[1]?
>>> >>
>>> >> Before we go moving any more code around or removing any more
>>> >> apis/providers we need to step back and have some agreed upon criteria
>>> >>for
>>> >> what belongs in jclouds/jclouds and what doesn¹t. If we don¹t do that
>>> >>our
>>> >> decisions are effectively arbitrary and the questions of ³What to do
>>> >>about
>>> >> X?² will continue to come up.
>>> >>
>>> >> I took a shot at addressing a topic like this in "Criteria for moving an
>>> >> api/provider from a labs repo to core² [2] but I got zero replies.
>>> >>People
>>> >> seem to be unwilling or uninterested in tackling this question and I
>>> >>think
>>> >> jclouds is suffering because of it. It¹s puzzling to me because in email
>>> >> threads or IRC chats most everybody agrees that the labs repos are a
>>> >>failed
>>> >> experiment. So let¹s flip it around and discuss what belongs in
>>> >> jclouds/jclouds instead.
>>> >>
>>> >> I know for a fact that many contributors have an opinion on this. Let¹s
>>> >> hear it!
>>> >>
>>> >> Thanks,
>>> >> Everett
>>> >>
>>> >> [1] https://github.com/jclouds/jclouds
>>>
>>>

RE: [DISCUSS] What is the criteria for something to be in the jclouds/jclouds repo?

[Zack Shoylev <za...@RACKSPACE.COM>]

If we want to only keep services with abstraction in jclouds/jclouds, then what about services that will probably never have an abstraction layer, such as chef, for example?

If we are going for separate code repos for jclouds/jclouds and specific providers, then we still have the problems of:
- releases are more problematic with multiple repos
- PRs become more complex when they have to happen in multiple repos or depend on other PRs in jclouds/jclouds

To recap, for the criteria, it seems everyone agrees on (as a minimum):
- Steward
- Code guidelines
________________________________________
From: Ignasi Barrera [nacx@apache.org]
Sent: Tuesday, December 02, 2014 2:31 PM
To: dev@jclouds.apache.org
Subject: Re: [DISCUSS] What is the criteria for something to be in the jclouds/jclouds repo?

In my opinion there are several requirements to be met, all them mandatory:

1. New providers MUST have an abstraction (more on this below).
2. Must have a Steward, who maintains the provider, takes the issues,
and runs the live tests periodically (at least before each release).
3. It must follow the coding guidelines. As Andrea mentioned,
technical debt is a *real* blocker when it comes to refactor the core
to fix structural issues. We can't keep adding it to the main repo.
Huge providers that are out of date are only going to cause us
trouble.

Regarding the abstractions. We need to think about what jclouds is.
jclouds is ALL about the abstractions. It is not an aggregator of any
api that is labelled "cloud". Without the abstractions, jclouds makes
no sense. Users don't come to jclouds looking for concrete APIs. They
come looking for a portable toolkit. If you have a look at the mailing
list, most questions are about the Blob and Compute abstraction. There
are also many about the Neutron API, but that is *only* because those
APIs have failed to create an abstraction. Users are asking for
concrete APIs just because there is no abstraction for them.

This said, the absence of an abstraction (take network or databases as
an example) shouldn't be a blocker. Abstractions can be proposed in
the mailing list, and even better discussed in a PR! We are all
looking to improve jclouds to make it better for the users, and that
means making it more portable. Let's work on adding an abstraction
when that makes sense! But things have to be *real* and that work has
to be done. Without it, it does not make sense to promote/keep such
apis/providers.

I.


On 2 December 2014 at 18:44, Andrea Turli <an...@gmail.com> wrote:
> Hi,
>
> I personally think we should consider a couple of additional criteria to
> the good Zack's list.
> The code should support the latest stable (recommended) version of the API.
> The implementation should follow all the current guidelines (i. e. Use of
> Auto Value and Service, MWS, no async API)  so that the new providers don't
> introduce tech debt, this should be kept during reviews but we need some
> standard way to "enforce" them.
> MockTests and LiveTests have to be green.
> The maintainer has to be able to run the LiveTests for each release.
>
> As contributor to labs repos this is my understanding of what it is needed
> to get the provider out of labs, hope this helps.
>
> Andrea
> Il 02/dic/2014 18:28 "Zack Shoylev" <za...@rackspace.com> ha scritto:
>
>> Possible criteria that have been discussed before:
>>
>> 1. If it is out of beta
>> 2. If it has a maintainer
>> 3. If it has a jclouds abstraction layer
>> 4. Voted on a case by case basis (current way of doing things)
>>
>> Are there any more possible criteria I have left out? In the end it will
>> be some mix of the above. My thoughts on this is that 2) is sufficient.
>> -Zack
>> ________________________________________
>> From: Dancy, Chris [Chris.Dancy@pega.com]
>> Sent: Tuesday, December 02, 2014 8:27 AM
>> To: dev@jclouds.apache.org
>> Subject: Re: [DISCUSS] What is the criteria for something to be in the
>> jclouds/jclouds repo?
>>
>> As an occasional contributor, and currently coding a Shipyard provider
>> (JCLOUDS-782), I do have a vested interest in how this ends up. IMO I
>> think having a clear separation of ³core² projects vs ³baking² projects is
>> a good idea but it seems that is not working out so great here? Are there
>> other proposals on how to do this?
>>
>> On 12/2/14, 4:09 AM, "Inbar Stolberg" <in...@gmail.com> wrote:
>>
>> >I think openstack neutron should be in jcluods master currently it suffers
>> >from being left out even though it is being updated and runs on openstack
>> >api v2.
>> >
>> >regarding glance I think after
>> >https://github.com/jclouds/jclouds-labs-openstack/pull/77 is merged we
>> >should move it.
>> >
>> >regarding Ceilometer I think that taking to accout what you said Everett:
>> >" in email threads or IRC chats most everybody agrees that the labs repos
>> >are a failed experiment"
>> >maybe its worth considering porting it to jclouds/jclouds from right after
>> >the merge of :
>> >https://github.com/jclouds/jclouds-labs-openstack/pull/166
>> >
>> >(which btw I am already going to use in production code...)
>> >
>> >my 2 cents
>> >Regards Inbar
>> >
>> >
>> >On Tue, Dec 2, 2014 at 2:06 AM, Everett Toews
>> ><ev...@rackspace.com>
>> >wrote:
>> >
>> >> What is the criteria for something to be in the jclouds/jclouds repo
>> >>[1]?
>> >>
>> >> Before we go moving any more code around or removing any more
>> >> apis/providers we need to step back and have some agreed upon criteria
>> >>for
>> >> what belongs in jclouds/jclouds and what doesn¹t. If we don¹t do that
>> >>our
>> >> decisions are effectively arbitrary and the questions of ³What to do
>> >>about
>> >> X?² will continue to come up.
>> >>
>> >> I took a shot at addressing a topic like this in "Criteria for moving an
>> >> api/provider from a labs repo to core² [2] but I got zero replies.
>> >>People
>> >> seem to be unwilling or uninterested in tackling this question and I
>> >>think
>> >> jclouds is suffering because of it. It¹s puzzling to me because in email
>> >> threads or IRC chats most everybody agrees that the labs repos are a
>> >>failed
>> >> experiment. So let¹s flip it around and discuss what belongs in
>> >> jclouds/jclouds instead.
>> >>
>> >> I know for a fact that many contributors have an opinion on this. Let¹s
>> >> hear it!
>> >>
>> >> Thanks,
>> >> Everett
>> >>
>> >> [1] https://github.com/jclouds/jclouds
>>
>>

RE: [DISCUSS] What is the criteria for something to be in the jclouds/jclouds repo?

[Zack Shoylev <za...@RACKSPACE.COM>]

Quick overview:
We are discussing criteria for managing jclouds/jclouds, specifically on handling abstractions and APIs.
Currently, most of us agree that the following criteria should be met, at a minimum, before an API is promoted to jclouds/jclouds:
- It is being maintained
- It has an abstraction
- Code is up-to-date (tests, style, best practices)
________________________________________
From: Ignasi Barrera [nacx@apache.org]
Sent: Thursday, December 18, 2014 2:46 AM
To: dev@jclouds.apache.org
Subject: RE: [DISCUSS] What is the criteria for something to be in the jclouds/jclouds repo?

Right now, this procedure is completely irrelevant for this discussion: do
you know right now how many labs repos would we have? No.

Also that procedure could seem a long one but all those actions are likely
to happen very separated in time. It will take time since an
api/abstraction starts until it is in a good shape to be promoted, so I
wouldn't take that procedure as a step by step one to follow. How often do
we promote apis or providers? Do we really need a such a step by step
procedure? We have done this many times in the past and there was no issue
about how to do it.

I think that the peocedure (which I also consider unnecessary) is
irrelevant for this discussion. Let's focus on the subject of the thread
and elaborate on that.
El 16/12/2014 20:50, "Zack Shoylev" <za...@rackspace.com> escribió:

> Let's work out some examples about promoting APIs and how the repos should
> work.
>
> Example: I have a provider-specific API that extends the compute
> abstraction in a meaningful way.
> Currently I would have to:
> 1. Make a PR for the API in one of the labs
> 2. Make a PR for the abstraction in jclouds/jclouds
> 3. (Same as 1. in a way) Make a PR in other labs repos for other providers
> (as it is likely new abstraction features would have to be supported in
> other providers - some of these might be created by other people, obviously
> - or these features might already be supported)
> 4. Review and merge all of these.
> 5. Make PRs to promote the provider-specific APIs to jclouds/jclouds and
> merge those.
> 6. Make PRs to clean up labs repos.
> Does this make sense? Am I missing something? How can we improve this?
>
> ________________________________________
> From: Ignasi Barrera [nacx@apache.org]
> Sent: Monday, December 15, 2014 3:22 PM
> To: dev@jclouds.apache.org
> Subject: Re: [DISCUSS] What is the criteria for something to be in the
> jclouds/jclouds repo?
>
> I can agree that giving full control over provider specific APIs can
> be a good thing, but always as a secondary goal. As you say,
> abstractions and working on the missing ones should be the high
> priority. Adding provider specific features to complete the
> abstractions is also a common sense thing; and at this concrete point
> in time, keep growing or promoting the APIs that are not related to
> any abstraction don't benefit the project as a whole.
>
> I think the entire point of this discussion is to set up a baseline
> that allows us to "move forward". To do that, we need to have a common
> criteria and a common vision of which should be the next steps to
> evolve the project, and it seems we agree that the high priority
> should be to work on the abstractions and focus on them.
>
> This means that specially the PMC and committers should work to make
> this happen, and focus on this high priority tasks. Setting a rule
> that we need an abstraction to promote the APIs will help us have
> under control what we promote and help us focus on the tasks we have
> defined as high priority. If we keep putting our major efforts in
> things that are not the high priority (apis not related to an
> abstraction), we won't achieve the main purpose of "moving forward".
>
> This is not black or white, and I don't think "we will never promote
> an API that doesn't have an abstraction" is an statement we should do.
> However, at this point in time, we should *really enforce* any
> promoted provider and API to fit in an abstraction. We have limited
> resources, and we need to focus in the high priority tasks. If we
> don't dedicate these few resources to the high priority tasks, we
> won't succeed.
>
>
> So, "will having an abstraction be a mandatory requirement forever and
> ever?" I'd say No.
> But that will only make sense if we work *starting today* together on
> the APIs that actually fit on those abstractions, and start building
> and discussing the ones that could be added. We should prioritise
> that, and make that a *real* thing, leaving the unrelated
> apis/providers aside. Otherwise we will be at the same point and won't
> be able to move forward, regardless of what we promote.
>
>
>
>
>
> On 15 December 2014 at 18:49, Zack Shoylev <za...@rackspace.com>
> wrote:
> >>> Every single API in jclouds must be aligned with the project's
> purpose, and
> > that means *portability*.
> >
> > jclouds also tries to do portability "while giving you full control to
> use cloud-specific features."
> >
> >>>  if someone contributes a patch adding support for Google Drive
> >
> > I was not part of that discussion, but it should be possible to add a
> blobstore abstraction for this, no?
> >
> >>> So, once again, jclouds is *not* a cloud aggregator for any API
> labelled
> > "cloud". It is about portable code, and every single API and provider
> > should meet the project goals.
> >
> > jclouds allows you to write portable applications, but also allows you
> the flexibility to use cloud-specific features. From a user perspective,
> you need to have both. You want to be able to build infrastructure using
> high-level code that works with all the clouds, but occasionally you will
> need the flexibility to do provider-specific things (even just because this
> is how providers differentiate themselves).
> >
> >
> > I wanted to focus on the user perspective for this discussion. In the
> end, users want things to work well, with less code, and with whatever
> providers they choose.
> > Because jclouds is modular, users don't care if jclouds "is not an API
> aggregator of any API that is labelled "cloud"." - users want jclouds to
> work well for whatever providers and clouds they choose.
> > As such, figuring out the criteria of what should be in jclouds/jclouds
> is not really something that affects users directly, but has to be done to
> make development easier, saner, and to reduce current and future technical
> debt.
> > If we had infinite resources, it would make sense, for users, to include
> and release all cloud APIs and their abstractions (where such abstractions
> make sense) as part of jclouds.
> >
> > Part of this is about balance. Currently we have too many cloud-specific
> APIs that could be abstracted, but are not. Clearly some of the
> high-priority work in jclouds is working on abstractions. That's the case
> at the moment. It might be a knee-jerk reaction to limit jclouds just to
> abstractable APIs because of this situation, when in the future we might
> also want to give users the flexibility to also use provider-specific
> features or even provider-specific APIs that might be useful when used
> together with higher-level, more abstract code.
> >
> > This is our current mission statement:
> > "Apache jclouds® is an open source multi-cloud toolkit for the Java
> platform that gives you the freedom to create applications that are
> portable across clouds while giving you full control to use cloud-specific
> features."
> >
> > Should this be morphed into:
> > "Apache jclouds® is an open source multi-cloud toolkit for the Java
> platform that gives you the freedom to create applications that are
> portable across clouds"? Is this better for users?
> >
> > Thanks, and let's keep the discussion going. Figuring things out will
> help both contributors and users.
> >
> > ________________________________________
> > From: Ignasi Barrera [nacx@apache.org]
> > Sent: Wednesday, December 10, 2014 12:59 AM
> > To: dev@jclouds.apache.org
> > Subject: RE: [DISCUSS] What is the criteria for something to be in the
> jclouds/jclouds repo?
> >
> > I'd never do point 2 and would remove those APIs. As I've said several
> > times in the thread, jclouds is not an API aggregator of any API that is
> > labelled "cloud".
> >
> > Every single API in jclouds must be aligned with the project's purpose,
> and
> > that means *portability*. It does not make sense to keep adding APIs that
> > don't fit in the project's purpose. If we do that we would be facing the
> > exact same issues we are facing now (I mentioned them in my previous
> email)
> > regardless of the number of repos we use.
> >
> > To put a concrete example: jclouds is *not* the Rackspace SDK nor is the
> > OpenStack java SDK. Perhaps Rackspace adopts jclouds as its SDK, but
> > jclouds isn't the Rackspace SDK. And this difference is important.
> Perhaps
> > not every single API in Rackspace makes sense in jclouds, as it would
> > hardly be modelled in a portable way. In that case, we shouldn't add it,
> as
> > it does not benefit jclouds as a whole and doesn't serve the jclouds
> > purpose.
> >
> > Let's take another example: if someone contributes a patch adding support
> > for Google Drive (which was discussed in the ML or IRC), providing an API
> > to store documents, should we merge it? That API does not fit in the
> > BlobStore abstraction as it does not manage blobs (raw object data) but
> > documents. The answer is no, because jclouds is not about that (ir is
> just
> > a single, isolated API to store documents out there). Without alignment
> > with what jclouds targets, we shouldn't add it.
> >
> > So, once again, jclouds is *not* a cloud aggregator for any API labelled
> > "cloud". It is about portable code, and every single API and provider
> > should meet the project goals. If not, we should not add them.
> > El 10/12/2014 01:54, "Zack Shoylev" <za...@rackspace.com>
> escribió:
> >
> >> If we take this to its logical conclusion, it sounds like we can
> separate
> >> APIs into two categories:
> >>
> >> 1. Those that have an abstraction or can be abstracted in the future (to
> >> be in jclouds/jclouds). Most APIs would be in here.
> >> 2. Those that provide provider-specific features only, and cannot be
> >> abstracted. There should only be a few here, but this means a few per
> >> provider, so overall this category might contain a large number of APIs.
> >>
> >> If we were to organize the repos using the two categories above, we can
> do
> >> something like (suggestion):
> >>
> >> 1. jclouds/jclouds (Same as now, with more APIs - including beta APIs -
> >> promoted. Regular release schedule)
> >> 2. jclouds/providers (All provider-specific APIs that will never be
> >> abstracted. Release together with 1. This also needs a better name, as
> it
> >> only contains APIs that are not abstractable).
> >> 3. jclouds/labs (very experimental only - will not be released)
> >>
> >> Note that I am strongly in favor of fewer repos for us to manage. Maybe
> we
> >> should not even have 2.
> >>
> >> What would you change in the description above?
> >> Thanks!
> >> -Zack
> >> ________________________________________
> >> From: Ignasi Barrera [ignasi.barrera@gmail.com]
> >> Sent: Monday, December 08, 2014 4:19 PM
> >> To: dev@jclouds.apache.org
> >> Subject: Re: [DISCUSS] What is the criteria for something to be in the
> >> jclouds/jclouds repo?
> >>
> >> 1. So once an api meets the criteria of having a steward, being
> >> modern, and having a proper test suite, it can immediately be
> >> promoted?
> >>
> >> I really think that those APIs need to have an abstraction, or a
> >> potential abstraction that is being developed, but something real,
> >> tangible, that ensures that it is aligned with the essence of what
> >> jclouds is.
> >>
> >> 2. Can you elaborate on this point? Did you have some criteria in mind
> >> for what “makes sense” means?
> >>
> >> There are several features more than one cloud provider offers
> >> (networking, load balancers and databases, are the obvious examples)
> >> that could have an abstraction. Not every feature in every cloud
> >> provider may make sense in others, and thus are not suitable to have
> >> an abstraction. In that case, we should be asking ourselves what value
> >> adding such an "independent" API gives to jclouds. We have provider
> >> specific APIs to *complete* the functionality "offered by the
> >> abstractions", but I really don't think is a good idea to add "entire
> >> independent apis" that won't have an abstraction because they're too
> >> specific to a single cloud provider. When I say "makes sense" I'm
> >> saying to ask ourselves if that API really makes sense in the global
> >> jclouds picture. If it makes sense, let's work on getting those
> >> features available in a portable way to other providers too. If that
> >> doesn't make sense, then that API doesn't make sense to jclouds.
> >>
> >> 3. Are you saying that if there’s an api that would never be a good
> >> fit for abstraction, that it has no chance of being promoted?
> >>
> >> Yes. I really believe we should be doing that. As I said, jclouds is
> >> not an API placeholder to every single API that qualifies itself as
> >> "cloud". jclouds is all about abstractions, and about providing users
> >> a way to write portable code. Of course we have provider specific
> >> features, but as I mentioned before, those should serve the purpose of
> >> completing the abstractions, but never be independent and standalone
> >> providers/apis. That has proven to be difficult to maintain.
> >> considerably increase the codebase complexity and make it more
> >> difficult to evolve jclouds itself.
> >>
> >> 4. Are you saying that if an abstraction doesn’t happen in some amount
> >> of time, that any apis related to that lack-of-abstraction would be
> >> removed?
> >>
> >> Well, this is open source, and "time" is relative. The complexity of
> >> abstractions can be very different: databases seems to be a simple,
> >> but networking can be really complex. What I'm saying is that we
> >> should *work* on those abstractions and *work in a direction to make
> >> those abstractions" something real. But we can't just keep adding more
> >> "satellite" APIs and forget about the rest. We should be thinking
> >> about abstractions and woking on them, and that's what matters, not
> >> time. Take the database APIs as an example. It has been commented more
> >> than once that it would be nice to have an abstraction. What actions
> >> have been taken in that direction? None. Any proposal? Nope. That's
> >> what we have to change.
> >>
> >>
> >>
> >> On 8 December 2014 at 16:01, Everett Toews <everett.toews@rackspace.com
> >
> >> wrote:
> >> > On Dec 2, 2014, at 2:31 PM, Ignasi Barrera <na...@apache.org> wrote:
> >> >
> >> >> This said, the absence of an abstraction (take network or databases
> as
> >> >> an example) shouldn't be a blocker. Abstractions can be proposed in
> >> >> the mailing list, and even better discussed in a PR! We are all
> >> >> looking to improve jclouds to make it better for the users, and that
> >> >> means making it more portable. Let's work on adding an abstraction
> >> >> when that makes sense! But things have to be *real* and that work has
> >> >> to be done. Without it, it does not make sense to promote/keep such
> >> >> apis/providers.
> >> >
> >> > I think having everything in jclouds/jclouds could possibly work but
> we
> >> would need to be more specific about the above. Otherwise I’m concerned
> >> we’d wind up in the same place 6 months down the road.
> >> >
> >> > If this were to be the path we go down, I have some clarifying
> >> questions. These are open questions for anyone to answer.
> >> >
> >> > “the absence of an abstraction shouldn’t be a blocker”
> >> >
> >> > 1. So once an api meets the criteria of having a steward, being
> modern,
> >> and having a proper test suite, it can immediately be promoted?
> >> >
> >> >
> >> > “Let's work on adding an abstraction when that makes sense!”
> >> >
> >> > 2. Can you elaborate on this point? Did you have some criteria in mind
> >> for what “makes sense” means?
> >> >
> >> >
> >> > "it does not make sense to promote/keep such apis/providers.”
> >> >
> >> > These are two different concerns and should be broken out.
> >> >
> >> > "it does not make sense to promote such apis/providers.”
> >> >
> >> > 3. Are you saying that if there’s an api that would never be a good
> fit
> >> for abstraction, that it has no chance of being promoted?
> >> >
> >> > "it does not make sense to keep such apis/providers.”
> >> >
> >> > 4. Are you saying that if an abstraction doesn’t happen in some amount
> >> of time, that any apis related to that lack-of-abstraction would be
> removed?
> >> >
> >> >
> >> > Everett
> >>
>

RE: [DISCUSS] What is the criteria for something to be in the jclouds/jclouds repo?

[Andrew Phillips <ap...@qrmedia.com>]

> I think that the peocedure (which I also consider unnecessary) is
> irrelevant for this discussion. Let's focus on the subject of the thread
> and elaborate on that.

To add to Ignasi's comment: the process for getting stuff into jclouds  
can certainly be improved, but unless and until we determine that *the  
number of PRs* really is the main issue (as opposed to review time,  
lack of examples/best practice guidance, or one of many other  
factors), I'd say that this is not the main thing we need to focus on  
optimizing right now.

To me, the component of this thread that talks about the *content* of  
what we are likely to accept into core seems to be far more critical.  
As Ignasi has said, I think focussing on cloud abstraction is both a  
good way of maintaining coherence for the project overall and giving  
guidance to potential contributors:

1) Does my proposed API/provider match one of the existing  
abstractions? If yes, I will need to implement that abstraction before  
I have a chance of going to core.
2) If not, I will need to start a discussion about what kind of  
abstraction my API/provider represents (including potentially other  
examples) before there is a chance of my API/provider making it to core.

As far as *which* kinds of abstraction jclouds should provide in order  
to remain relevant going forward: *that's* exactly where I think we  
need to move much more to actively (re-)engaging with our user base  
and getting their input.

Regards

ap

RE: [DISCUSS] What is the criteria for something to be in the jclouds/jclouds repo?

[Ignasi Barrera <na...@apache.org>]

Right now, this procedure is completely irrelevant for this discussion: do
you know right now how many labs repos would we have? No.

Also that procedure could seem a long one but all those actions are likely
to happen very separated in time. It will take time since an
api/abstraction starts until it is in a good shape to be promoted, so I
wouldn't take that procedure as a step by step one to follow. How often do
we promote apis or providers? Do we really need a such a step by step
procedure? We have done this many times in the past and there was no issue
about how to do it.

I think that the peocedure (which I also consider unnecessary) is
irrelevant for this discussion. Let's focus on the subject of the thread
and elaborate on that.
El 16/12/2014 20:50, "Zack Shoylev" <za...@rackspace.com> escribió:

> Let's work out some examples about promoting APIs and how the repos should
> work.
>
> Example: I have a provider-specific API that extends the compute
> abstraction in a meaningful way.
> Currently I would have to:
> 1. Make a PR for the API in one of the labs
> 2. Make a PR for the abstraction in jclouds/jclouds
> 3. (Same as 1. in a way) Make a PR in other labs repos for other providers
> (as it is likely new abstraction features would have to be supported in
> other providers - some of these might be created by other people, obviously
> - or these features might already be supported)
> 4. Review and merge all of these.
> 5. Make PRs to promote the provider-specific APIs to jclouds/jclouds and
> merge those.
> 6. Make PRs to clean up labs repos.
> Does this make sense? Am I missing something? How can we improve this?
>
> ________________________________________
> From: Ignasi Barrera [nacx@apache.org]
> Sent: Monday, December 15, 2014 3:22 PM
> To: dev@jclouds.apache.org
> Subject: Re: [DISCUSS] What is the criteria for something to be in the
> jclouds/jclouds repo?
>
> I can agree that giving full control over provider specific APIs can
> be a good thing, but always as a secondary goal. As you say,
> abstractions and working on the missing ones should be the high
> priority. Adding provider specific features to complete the
> abstractions is also a common sense thing; and at this concrete point
> in time, keep growing or promoting the APIs that are not related to
> any abstraction don't benefit the project as a whole.
>
> I think the entire point of this discussion is to set up a baseline
> that allows us to "move forward". To do that, we need to have a common
> criteria and a common vision of which should be the next steps to
> evolve the project, and it seems we agree that the high priority
> should be to work on the abstractions and focus on them.
>
> This means that specially the PMC and committers should work to make
> this happen, and focus on this high priority tasks. Setting a rule
> that we need an abstraction to promote the APIs will help us have
> under control what we promote and help us focus on the tasks we have
> defined as high priority. If we keep putting our major efforts in
> things that are not the high priority (apis not related to an
> abstraction), we won't achieve the main purpose of "moving forward".
>
> This is not black or white, and I don't think "we will never promote
> an API that doesn't have an abstraction" is an statement we should do.
> However, at this point in time, we should *really enforce* any
> promoted provider and API to fit in an abstraction. We have limited
> resources, and we need to focus in the high priority tasks. If we
> don't dedicate these few resources to the high priority tasks, we
> won't succeed.
>
>
> So, "will having an abstraction be a mandatory requirement forever and
> ever?" I'd say No.
> But that will only make sense if we work *starting today* together on
> the APIs that actually fit on those abstractions, and start building
> and discussing the ones that could be added. We should prioritise
> that, and make that a *real* thing, leaving the unrelated
> apis/providers aside. Otherwise we will be at the same point and won't
> be able to move forward, regardless of what we promote.
>
>
>
>
>
> On 15 December 2014 at 18:49, Zack Shoylev <za...@rackspace.com>
> wrote:
> >>> Every single API in jclouds must be aligned with the project's
> purpose, and
> > that means *portability*.
> >
> > jclouds also tries to do portability "while giving you full control to
> use cloud-specific features."
> >
> >>>  if someone contributes a patch adding support for Google Drive
> >
> > I was not part of that discussion, but it should be possible to add a
> blobstore abstraction for this, no?
> >
> >>> So, once again, jclouds is *not* a cloud aggregator for any API
> labelled
> > "cloud". It is about portable code, and every single API and provider
> > should meet the project goals.
> >
> > jclouds allows you to write portable applications, but also allows you
> the flexibility to use cloud-specific features. From a user perspective,
> you need to have both. You want to be able to build infrastructure using
> high-level code that works with all the clouds, but occasionally you will
> need the flexibility to do provider-specific things (even just because this
> is how providers differentiate themselves).
> >
> >
> > I wanted to focus on the user perspective for this discussion. In the
> end, users want things to work well, with less code, and with whatever
> providers they choose.
> > Because jclouds is modular, users don't care if jclouds "is not an API
> aggregator of any API that is labelled "cloud"." - users want jclouds to
> work well for whatever providers and clouds they choose.
> > As such, figuring out the criteria of what should be in jclouds/jclouds
> is not really something that affects users directly, but has to be done to
> make development easier, saner, and to reduce current and future technical
> debt.
> > If we had infinite resources, it would make sense, for users, to include
> and release all cloud APIs and their abstractions (where such abstractions
> make sense) as part of jclouds.
> >
> > Part of this is about balance. Currently we have too many cloud-specific
> APIs that could be abstracted, but are not. Clearly some of the
> high-priority work in jclouds is working on abstractions. That's the case
> at the moment. It might be a knee-jerk reaction to limit jclouds just to
> abstractable APIs because of this situation, when in the future we might
> also want to give users the flexibility to also use provider-specific
> features or even provider-specific APIs that might be useful when used
> together with higher-level, more abstract code.
> >
> > This is our current mission statement:
> > "Apache jclouds® is an open source multi-cloud toolkit for the Java
> platform that gives you the freedom to create applications that are
> portable across clouds while giving you full control to use cloud-specific
> features."
> >
> > Should this be morphed into:
> > "Apache jclouds® is an open source multi-cloud toolkit for the Java
> platform that gives you the freedom to create applications that are
> portable across clouds"? Is this better for users?
> >
> > Thanks, and let's keep the discussion going. Figuring things out will
> help both contributors and users.
> >
> > ________________________________________
> > From: Ignasi Barrera [nacx@apache.org]
> > Sent: Wednesday, December 10, 2014 12:59 AM
> > To: dev@jclouds.apache.org
> > Subject: RE: [DISCUSS] What is the criteria for something to be in the
> jclouds/jclouds repo?
> >
> > I'd never do point 2 and would remove those APIs. As I've said several
> > times in the thread, jclouds is not an API aggregator of any API that is
> > labelled "cloud".
> >
> > Every single API in jclouds must be aligned with the project's purpose,
> and
> > that means *portability*. It does not make sense to keep adding APIs that
> > don't fit in the project's purpose. If we do that we would be facing the
> > exact same issues we are facing now (I mentioned them in my previous
> email)
> > regardless of the number of repos we use.
> >
> > To put a concrete example: jclouds is *not* the Rackspace SDK nor is the
> > OpenStack java SDK. Perhaps Rackspace adopts jclouds as its SDK, but
> > jclouds isn't the Rackspace SDK. And this difference is important.
> Perhaps
> > not every single API in Rackspace makes sense in jclouds, as it would
> > hardly be modelled in a portable way. In that case, we shouldn't add it,
> as
> > it does not benefit jclouds as a whole and doesn't serve the jclouds
> > purpose.
> >
> > Let's take another example: if someone contributes a patch adding support
> > for Google Drive (which was discussed in the ML or IRC), providing an API
> > to store documents, should we merge it? That API does not fit in the
> > BlobStore abstraction as it does not manage blobs (raw object data) but
> > documents. The answer is no, because jclouds is not about that (ir is
> just
> > a single, isolated API to store documents out there). Without alignment
> > with what jclouds targets, we shouldn't add it.
> >
> > So, once again, jclouds is *not* a cloud aggregator for any API labelled
> > "cloud". It is about portable code, and every single API and provider
> > should meet the project goals. If not, we should not add them.
> > El 10/12/2014 01:54, "Zack Shoylev" <za...@rackspace.com>
> escribió:
> >
> >> If we take this to its logical conclusion, it sounds like we can
> separate
> >> APIs into two categories:
> >>
> >> 1. Those that have an abstraction or can be abstracted in the future (to
> >> be in jclouds/jclouds). Most APIs would be in here.
> >> 2. Those that provide provider-specific features only, and cannot be
> >> abstracted. There should only be a few here, but this means a few per
> >> provider, so overall this category might contain a large number of APIs.
> >>
> >> If we were to organize the repos using the two categories above, we can
> do
> >> something like (suggestion):
> >>
> >> 1. jclouds/jclouds (Same as now, with more APIs - including beta APIs -
> >> promoted. Regular release schedule)
> >> 2. jclouds/providers (All provider-specific APIs that will never be
> >> abstracted. Release together with 1. This also needs a better name, as
> it
> >> only contains APIs that are not abstractable).
> >> 3. jclouds/labs (very experimental only - will not be released)
> >>
> >> Note that I am strongly in favor of fewer repos for us to manage. Maybe
> we
> >> should not even have 2.
> >>
> >> What would you change in the description above?
> >> Thanks!
> >> -Zack
> >> ________________________________________
> >> From: Ignasi Barrera [ignasi.barrera@gmail.com]
> >> Sent: Monday, December 08, 2014 4:19 PM
> >> To: dev@jclouds.apache.org
> >> Subject: Re: [DISCUSS] What is the criteria for something to be in the
> >> jclouds/jclouds repo?
> >>
> >> 1. So once an api meets the criteria of having a steward, being
> >> modern, and having a proper test suite, it can immediately be
> >> promoted?
> >>
> >> I really think that those APIs need to have an abstraction, or a
> >> potential abstraction that is being developed, but something real,
> >> tangible, that ensures that it is aligned with the essence of what
> >> jclouds is.
> >>
> >> 2. Can you elaborate on this point? Did you have some criteria in mind
> >> for what “makes sense” means?
> >>
> >> There are several features more than one cloud provider offers
> >> (networking, load balancers and databases, are the obvious examples)
> >> that could have an abstraction. Not every feature in every cloud
> >> provider may make sense in others, and thus are not suitable to have
> >> an abstraction. In that case, we should be asking ourselves what value
> >> adding such an "independent" API gives to jclouds. We have provider
> >> specific APIs to *complete* the functionality "offered by the
> >> abstractions", but I really don't think is a good idea to add "entire
> >> independent apis" that won't have an abstraction because they're too
> >> specific to a single cloud provider. When I say "makes sense" I'm
> >> saying to ask ourselves if that API really makes sense in the global
> >> jclouds picture. If it makes sense, let's work on getting those
> >> features available in a portable way to other providers too. If that
> >> doesn't make sense, then that API doesn't make sense to jclouds.
> >>
> >> 3. Are you saying that if there’s an api that would never be a good
> >> fit for abstraction, that it has no chance of being promoted?
> >>
> >> Yes. I really believe we should be doing that. As I said, jclouds is
> >> not an API placeholder to every single API that qualifies itself as
> >> "cloud". jclouds is all about abstractions, and about providing users
> >> a way to write portable code. Of course we have provider specific
> >> features, but as I mentioned before, those should serve the purpose of
> >> completing the abstractions, but never be independent and standalone
> >> providers/apis. That has proven to be difficult to maintain.
> >> considerably increase the codebase complexity and make it more
> >> difficult to evolve jclouds itself.
> >>
> >> 4. Are you saying that if an abstraction doesn’t happen in some amount
> >> of time, that any apis related to that lack-of-abstraction would be
> >> removed?
> >>
> >> Well, this is open source, and "time" is relative. The complexity of
> >> abstractions can be very different: databases seems to be a simple,
> >> but networking can be really complex. What I'm saying is that we
> >> should *work* on those abstractions and *work in a direction to make
> >> those abstractions" something real. But we can't just keep adding more
> >> "satellite" APIs and forget about the rest. We should be thinking
> >> about abstractions and woking on them, and that's what matters, not
> >> time. Take the database APIs as an example. It has been commented more
> >> than once that it would be nice to have an abstraction. What actions
> >> have been taken in that direction? None. Any proposal? Nope. That's
> >> what we have to change.
> >>
> >>
> >>
> >> On 8 December 2014 at 16:01, Everett Toews <everett.toews@rackspace.com
> >
> >> wrote:
> >> > On Dec 2, 2014, at 2:31 PM, Ignasi Barrera <na...@apache.org> wrote:
> >> >
> >> >> This said, the absence of an abstraction (take network or databases
> as
> >> >> an example) shouldn't be a blocker. Abstractions can be proposed in
> >> >> the mailing list, and even better discussed in a PR! We are all
> >> >> looking to improve jclouds to make it better for the users, and that
> >> >> means making it more portable. Let's work on adding an abstraction
> >> >> when that makes sense! But things have to be *real* and that work has
> >> >> to be done. Without it, it does not make sense to promote/keep such
> >> >> apis/providers.
> >> >
> >> > I think having everything in jclouds/jclouds could possibly work but
> we
> >> would need to be more specific about the above. Otherwise I’m concerned
> >> we’d wind up in the same place 6 months down the road.
> >> >
> >> > If this were to be the path we go down, I have some clarifying
> >> questions. These are open questions for anyone to answer.
> >> >
> >> > “the absence of an abstraction shouldn’t be a blocker”
> >> >
> >> > 1. So once an api meets the criteria of having a steward, being
> modern,
> >> and having a proper test suite, it can immediately be promoted?
> >> >
> >> >
> >> > “Let's work on adding an abstraction when that makes sense!”
> >> >
> >> > 2. Can you elaborate on this point? Did you have some criteria in mind
> >> for what “makes sense” means?
> >> >
> >> >
> >> > "it does not make sense to promote/keep such apis/providers.”
> >> >
> >> > These are two different concerns and should be broken out.
> >> >
> >> > "it does not make sense to promote such apis/providers.”
> >> >
> >> > 3. Are you saying that if there’s an api that would never be a good
> fit
> >> for abstraction, that it has no chance of being promoted?
> >> >
> >> > "it does not make sense to keep such apis/providers.”
> >> >
> >> > 4. Are you saying that if an abstraction doesn’t happen in some amount
> >> of time, that any apis related to that lack-of-abstraction would be
> removed?
> >> >
> >> >
> >> > Everett
> >>
>

RE: [DISCUSS] What is the criteria for something to be in the jclouds/jclouds repo?

[Zack Shoylev <za...@RACKSPACE.COM>]

Let's work out some examples about promoting APIs and how the repos should work.

Example: I have a provider-specific API that extends the compute abstraction in a meaningful way.
Currently I would have to:
1. Make a PR for the API in one of the labs
2. Make a PR for the abstraction in jclouds/jclouds
3. (Same as 1. in a way) Make a PR in other labs repos for other providers (as it is likely new abstraction features would have to be supported in other providers - some of these might be created by other people, obviously - or these features might already be supported)
4. Review and merge all of these.
5. Make PRs to promote the provider-specific APIs to jclouds/jclouds and merge those.
6. Make PRs to clean up labs repos.
Does this make sense? Am I missing something? How can we improve this?
 
________________________________________
From: Ignasi Barrera [nacx@apache.org]
Sent: Monday, December 15, 2014 3:22 PM
To: dev@jclouds.apache.org
Subject: Re: [DISCUSS] What is the criteria for something to be in the jclouds/jclouds repo?

I can agree that giving full control over provider specific APIs can
be a good thing, but always as a secondary goal. As you say,
abstractions and working on the missing ones should be the high
priority. Adding provider specific features to complete the
abstractions is also a common sense thing; and at this concrete point
in time, keep growing or promoting the APIs that are not related to
any abstraction don't benefit the project as a whole.

I think the entire point of this discussion is to set up a baseline
that allows us to "move forward". To do that, we need to have a common
criteria and a common vision of which should be the next steps to
evolve the project, and it seems we agree that the high priority
should be to work on the abstractions and focus on them.

This means that specially the PMC and committers should work to make
this happen, and focus on this high priority tasks. Setting a rule
that we need an abstraction to promote the APIs will help us have
under control what we promote and help us focus on the tasks we have
defined as high priority. If we keep putting our major efforts in
things that are not the high priority (apis not related to an
abstraction), we won't achieve the main purpose of "moving forward".

This is not black or white, and I don't think "we will never promote
an API that doesn't have an abstraction" is an statement we should do.
However, at this point in time, we should *really enforce* any
promoted provider and API to fit in an abstraction. We have limited
resources, and we need to focus in the high priority tasks. If we
don't dedicate these few resources to the high priority tasks, we
won't succeed.


So, "will having an abstraction be a mandatory requirement forever and
ever?" I'd say No.
But that will only make sense if we work *starting today* together on
the APIs that actually fit on those abstractions, and start building
and discussing the ones that could be added. We should prioritise
that, and make that a *real* thing, leaving the unrelated
apis/providers aside. Otherwise we will be at the same point and won't
be able to move forward, regardless of what we promote.





On 15 December 2014 at 18:49, Zack Shoylev <za...@rackspace.com> wrote:
>>> Every single API in jclouds must be aligned with the project's purpose, and
> that means *portability*.
>
> jclouds also tries to do portability "while giving you full control to use cloud-specific features."
>
>>>  if someone contributes a patch adding support for Google Drive
>
> I was not part of that discussion, but it should be possible to add a blobstore abstraction for this, no?
>
>>> So, once again, jclouds is *not* a cloud aggregator for any API labelled
> "cloud". It is about portable code, and every single API and provider
> should meet the project goals.
>
> jclouds allows you to write portable applications, but also allows you the flexibility to use cloud-specific features. From a user perspective, you need to have both. You want to be able to build infrastructure using high-level code that works with all the clouds, but occasionally you will need the flexibility to do provider-specific things (even just because this is how providers differentiate themselves).
>
>
> I wanted to focus on the user perspective for this discussion. In the end, users want things to work well, with less code, and with whatever providers they choose.
> Because jclouds is modular, users don't care if jclouds "is not an API aggregator of any API that is labelled "cloud"." - users want jclouds to work well for whatever providers and clouds they choose.
> As such, figuring out the criteria of what should be in jclouds/jclouds is not really something that affects users directly, but has to be done to make development easier, saner, and to reduce current and future technical debt.
> If we had infinite resources, it would make sense, for users, to include and release all cloud APIs and their abstractions (where such abstractions make sense) as part of jclouds.
>
> Part of this is about balance. Currently we have too many cloud-specific APIs that could be abstracted, but are not. Clearly some of the high-priority work in jclouds is working on abstractions. That's the case at the moment. It might be a knee-jerk reaction to limit jclouds just to abstractable APIs because of this situation, when in the future we might also want to give users the flexibility to also use provider-specific features or even provider-specific APIs that might be useful when used together with higher-level, more abstract code.
>
> This is our current mission statement:
> "Apache jclouds® is an open source multi-cloud toolkit for the Java platform that gives you the freedom to create applications that are portable across clouds while giving you full control to use cloud-specific features."
>
> Should this be morphed into:
> "Apache jclouds® is an open source multi-cloud toolkit for the Java platform that gives you the freedom to create applications that are portable across clouds"? Is this better for users?
>
> Thanks, and let's keep the discussion going. Figuring things out will help both contributors and users.
>
> ________________________________________
> From: Ignasi Barrera [nacx@apache.org]
> Sent: Wednesday, December 10, 2014 12:59 AM
> To: dev@jclouds.apache.org
> Subject: RE: [DISCUSS] What is the criteria for something to be in the jclouds/jclouds repo?
>
> I'd never do point 2 and would remove those APIs. As I've said several
> times in the thread, jclouds is not an API aggregator of any API that is
> labelled "cloud".
>
> Every single API in jclouds must be aligned with the project's purpose, and
> that means *portability*. It does not make sense to keep adding APIs that
> don't fit in the project's purpose. If we do that we would be facing the
> exact same issues we are facing now (I mentioned them in my previous email)
> regardless of the number of repos we use.
>
> To put a concrete example: jclouds is *not* the Rackspace SDK nor is the
> OpenStack java SDK. Perhaps Rackspace adopts jclouds as its SDK, but
> jclouds isn't the Rackspace SDK. And this difference is important. Perhaps
> not every single API in Rackspace makes sense in jclouds, as it would
> hardly be modelled in a portable way. In that case, we shouldn't add it, as
> it does not benefit jclouds as a whole and doesn't serve the jclouds
> purpose.
>
> Let's take another example: if someone contributes a patch adding support
> for Google Drive (which was discussed in the ML or IRC), providing an API
> to store documents, should we merge it? That API does not fit in the
> BlobStore abstraction as it does not manage blobs (raw object data) but
> documents. The answer is no, because jclouds is not about that (ir is just
> a single, isolated API to store documents out there). Without alignment
> with what jclouds targets, we shouldn't add it.
>
> So, once again, jclouds is *not* a cloud aggregator for any API labelled
> "cloud". It is about portable code, and every single API and provider
> should meet the project goals. If not, we should not add them.
> El 10/12/2014 01:54, "Zack Shoylev" <za...@rackspace.com> escribió:
>
>> If we take this to its logical conclusion, it sounds like we can separate
>> APIs into two categories:
>>
>> 1. Those that have an abstraction or can be abstracted in the future (to
>> be in jclouds/jclouds). Most APIs would be in here.
>> 2. Those that provide provider-specific features only, and cannot be
>> abstracted. There should only be a few here, but this means a few per
>> provider, so overall this category might contain a large number of APIs.
>>
>> If we were to organize the repos using the two categories above, we can do
>> something like (suggestion):
>>
>> 1. jclouds/jclouds (Same as now, with more APIs - including beta APIs -
>> promoted. Regular release schedule)
>> 2. jclouds/providers (All provider-specific APIs that will never be
>> abstracted. Release together with 1. This also needs a better name, as it
>> only contains APIs that are not abstractable).
>> 3. jclouds/labs (very experimental only - will not be released)
>>
>> Note that I am strongly in favor of fewer repos for us to manage. Maybe we
>> should not even have 2.
>>
>> What would you change in the description above?
>> Thanks!
>> -Zack
>> ________________________________________
>> From: Ignasi Barrera [ignasi.barrera@gmail.com]
>> Sent: Monday, December 08, 2014 4:19 PM
>> To: dev@jclouds.apache.org
>> Subject: Re: [DISCUSS] What is the criteria for something to be in the
>> jclouds/jclouds repo?
>>
>> 1. So once an api meets the criteria of having a steward, being
>> modern, and having a proper test suite, it can immediately be
>> promoted?
>>
>> I really think that those APIs need to have an abstraction, or a
>> potential abstraction that is being developed, but something real,
>> tangible, that ensures that it is aligned with the essence of what
>> jclouds is.
>>
>> 2. Can you elaborate on this point? Did you have some criteria in mind
>> for what “makes sense” means?
>>
>> There are several features more than one cloud provider offers
>> (networking, load balancers and databases, are the obvious examples)
>> that could have an abstraction. Not every feature in every cloud
>> provider may make sense in others, and thus are not suitable to have
>> an abstraction. In that case, we should be asking ourselves what value
>> adding such an "independent" API gives to jclouds. We have provider
>> specific APIs to *complete* the functionality "offered by the
>> abstractions", but I really don't think is a good idea to add "entire
>> independent apis" that won't have an abstraction because they're too
>> specific to a single cloud provider. When I say "makes sense" I'm
>> saying to ask ourselves if that API really makes sense in the global
>> jclouds picture. If it makes sense, let's work on getting those
>> features available in a portable way to other providers too. If that
>> doesn't make sense, then that API doesn't make sense to jclouds.
>>
>> 3. Are you saying that if there’s an api that would never be a good
>> fit for abstraction, that it has no chance of being promoted?
>>
>> Yes. I really believe we should be doing that. As I said, jclouds is
>> not an API placeholder to every single API that qualifies itself as
>> "cloud". jclouds is all about abstractions, and about providing users
>> a way to write portable code. Of course we have provider specific
>> features, but as I mentioned before, those should serve the purpose of
>> completing the abstractions, but never be independent and standalone
>> providers/apis. That has proven to be difficult to maintain.
>> considerably increase the codebase complexity and make it more
>> difficult to evolve jclouds itself.
>>
>> 4. Are you saying that if an abstraction doesn’t happen in some amount
>> of time, that any apis related to that lack-of-abstraction would be
>> removed?
>>
>> Well, this is open source, and "time" is relative. The complexity of
>> abstractions can be very different: databases seems to be a simple,
>> but networking can be really complex. What I'm saying is that we
>> should *work* on those abstractions and *work in a direction to make
>> those abstractions" something real. But we can't just keep adding more
>> "satellite" APIs and forget about the rest. We should be thinking
>> about abstractions and woking on them, and that's what matters, not
>> time. Take the database APIs as an example. It has been commented more
>> than once that it would be nice to have an abstraction. What actions
>> have been taken in that direction? None. Any proposal? Nope. That's
>> what we have to change.
>>
>>
>>
>> On 8 December 2014 at 16:01, Everett Toews <ev...@rackspace.com>
>> wrote:
>> > On Dec 2, 2014, at 2:31 PM, Ignasi Barrera <na...@apache.org> wrote:
>> >
>> >> This said, the absence of an abstraction (take network or databases as
>> >> an example) shouldn't be a blocker. Abstractions can be proposed in
>> >> the mailing list, and even better discussed in a PR! We are all
>> >> looking to improve jclouds to make it better for the users, and that
>> >> means making it more portable. Let's work on adding an abstraction
>> >> when that makes sense! But things have to be *real* and that work has
>> >> to be done. Without it, it does not make sense to promote/keep such
>> >> apis/providers.
>> >
>> > I think having everything in jclouds/jclouds could possibly work but we
>> would need to be more specific about the above. Otherwise I’m concerned
>> we’d wind up in the same place 6 months down the road.
>> >
>> > If this were to be the path we go down, I have some clarifying
>> questions. These are open questions for anyone to answer.
>> >
>> > “the absence of an abstraction shouldn’t be a blocker”
>> >
>> > 1. So once an api meets the criteria of having a steward, being modern,
>> and having a proper test suite, it can immediately be promoted?
>> >
>> >
>> > “Let's work on adding an abstraction when that makes sense!”
>> >
>> > 2. Can you elaborate on this point? Did you have some criteria in mind
>> for what “makes sense” means?
>> >
>> >
>> > "it does not make sense to promote/keep such apis/providers.”
>> >
>> > These are two different concerns and should be broken out.
>> >
>> > "it does not make sense to promote such apis/providers.”
>> >
>> > 3. Are you saying that if there’s an api that would never be a good fit
>> for abstraction, that it has no chance of being promoted?
>> >
>> > "it does not make sense to keep such apis/providers.”
>> >
>> > 4. Are you saying that if an abstraction doesn’t happen in some amount
>> of time, that any apis related to that lack-of-abstraction would be removed?
>> >
>> >
>> > Everett
>>

Re: [DISCUSS] What is the criteria for something to be in the jclouds/jclouds repo?

[Ignasi Barrera <na...@apache.org>]

I can agree that giving full control over provider specific APIs can
be a good thing, but always as a secondary goal. As you say,
abstractions and working on the missing ones should be the high
priority. Adding provider specific features to complete the
abstractions is also a common sense thing; and at this concrete point
in time, keep growing or promoting the APIs that are not related to
any abstraction don't benefit the project as a whole.

I think the entire point of this discussion is to set up a baseline
that allows us to "move forward". To do that, we need to have a common
criteria and a common vision of which should be the next steps to
evolve the project, and it seems we agree that the high priority
should be to work on the abstractions and focus on them.

This means that specially the PMC and committers should work to make
this happen, and focus on this high priority tasks. Setting a rule
that we need an abstraction to promote the APIs will help us have
under control what we promote and help us focus on the tasks we have
defined as high priority. If we keep putting our major efforts in
things that are not the high priority (apis not related to an
abstraction), we won't achieve the main purpose of "moving forward".

This is not black or white, and I don't think "we will never promote
an API that doesn't have an abstraction" is an statement we should do.
However, at this point in time, we should *really enforce* any
promoted provider and API to fit in an abstraction. We have limited
resources, and we need to focus in the high priority tasks. If we
don't dedicate these few resources to the high priority tasks, we
won't succeed.


So, "will having an abstraction be a mandatory requirement forever and
ever?" I'd say No.
But that will only make sense if we work *starting today* together on
the APIs that actually fit on those abstractions, and start building
and discussing the ones that could be added. We should prioritise
that, and make that a *real* thing, leaving the unrelated
apis/providers aside. Otherwise we will be at the same point and won't
be able to move forward, regardless of what we promote.





On 15 December 2014 at 18:49, Zack Shoylev <za...@rackspace.com> wrote:
>>> Every single API in jclouds must be aligned with the project's purpose, and
> that means *portability*.
>
> jclouds also tries to do portability "while giving you full control to use cloud-specific features."
>
>>>  if someone contributes a patch adding support for Google Drive
>
> I was not part of that discussion, but it should be possible to add a blobstore abstraction for this, no?
>
>>> So, once again, jclouds is *not* a cloud aggregator for any API labelled
> "cloud". It is about portable code, and every single API and provider
> should meet the project goals.
>
> jclouds allows you to write portable applications, but also allows you the flexibility to use cloud-specific features. From a user perspective, you need to have both. You want to be able to build infrastructure using high-level code that works with all the clouds, but occasionally you will need the flexibility to do provider-specific things (even just because this is how providers differentiate themselves).
>
>
> I wanted to focus on the user perspective for this discussion. In the end, users want things to work well, with less code, and with whatever providers they choose.
> Because jclouds is modular, users don't care if jclouds "is not an API aggregator of any API that is labelled "cloud"." - users want jclouds to work well for whatever providers and clouds they choose.
> As such, figuring out the criteria of what should be in jclouds/jclouds is not really something that affects users directly, but has to be done to make development easier, saner, and to reduce current and future technical debt.
> If we had infinite resources, it would make sense, for users, to include and release all cloud APIs and their abstractions (where such abstractions make sense) as part of jclouds.
>
> Part of this is about balance. Currently we have too many cloud-specific APIs that could be abstracted, but are not. Clearly some of the high-priority work in jclouds is working on abstractions. That's the case at the moment. It might be a knee-jerk reaction to limit jclouds just to abstractable APIs because of this situation, when in the future we might also want to give users the flexibility to also use provider-specific features or even provider-specific APIs that might be useful when used together with higher-level, more abstract code.
>
> This is our current mission statement:
> "Apache jclouds® is an open source multi-cloud toolkit for the Java platform that gives you the freedom to create applications that are portable across clouds while giving you full control to use cloud-specific features."
>
> Should this be morphed into:
> "Apache jclouds® is an open source multi-cloud toolkit for the Java platform that gives you the freedom to create applications that are portable across clouds"? Is this better for users?
>
> Thanks, and let's keep the discussion going. Figuring things out will help both contributors and users.
>
> ________________________________________
> From: Ignasi Barrera [nacx@apache.org]
> Sent: Wednesday, December 10, 2014 12:59 AM
> To: dev@jclouds.apache.org
> Subject: RE: [DISCUSS] What is the criteria for something to be in the jclouds/jclouds repo?
>
> I'd never do point 2 and would remove those APIs. As I've said several
> times in the thread, jclouds is not an API aggregator of any API that is
> labelled "cloud".
>
> Every single API in jclouds must be aligned with the project's purpose, and
> that means *portability*. It does not make sense to keep adding APIs that
> don't fit in the project's purpose. If we do that we would be facing the
> exact same issues we are facing now (I mentioned them in my previous email)
> regardless of the number of repos we use.
>
> To put a concrete example: jclouds is *not* the Rackspace SDK nor is the
> OpenStack java SDK. Perhaps Rackspace adopts jclouds as its SDK, but
> jclouds isn't the Rackspace SDK. And this difference is important. Perhaps
> not every single API in Rackspace makes sense in jclouds, as it would
> hardly be modelled in a portable way. In that case, we shouldn't add it, as
> it does not benefit jclouds as a whole and doesn't serve the jclouds
> purpose.
>
> Let's take another example: if someone contributes a patch adding support
> for Google Drive (which was discussed in the ML or IRC), providing an API
> to store documents, should we merge it? That API does not fit in the
> BlobStore abstraction as it does not manage blobs (raw object data) but
> documents. The answer is no, because jclouds is not about that (ir is just
> a single, isolated API to store documents out there). Without alignment
> with what jclouds targets, we shouldn't add it.
>
> So, once again, jclouds is *not* a cloud aggregator for any API labelled
> "cloud". It is about portable code, and every single API and provider
> should meet the project goals. If not, we should not add them.
> El 10/12/2014 01:54, "Zack Shoylev" <za...@rackspace.com> escribió:
>
>> If we take this to its logical conclusion, it sounds like we can separate
>> APIs into two categories:
>>
>> 1. Those that have an abstraction or can be abstracted in the future (to
>> be in jclouds/jclouds). Most APIs would be in here.
>> 2. Those that provide provider-specific features only, and cannot be
>> abstracted. There should only be a few here, but this means a few per
>> provider, so overall this category might contain a large number of APIs.
>>
>> If we were to organize the repos using the two categories above, we can do
>> something like (suggestion):
>>
>> 1. jclouds/jclouds (Same as now, with more APIs - including beta APIs -
>> promoted. Regular release schedule)
>> 2. jclouds/providers (All provider-specific APIs that will never be
>> abstracted. Release together with 1. This also needs a better name, as it
>> only contains APIs that are not abstractable).
>> 3. jclouds/labs (very experimental only - will not be released)
>>
>> Note that I am strongly in favor of fewer repos for us to manage. Maybe we
>> should not even have 2.
>>
>> What would you change in the description above?
>> Thanks!
>> -Zack
>> ________________________________________
>> From: Ignasi Barrera [ignasi.barrera@gmail.com]
>> Sent: Monday, December 08, 2014 4:19 PM
>> To: dev@jclouds.apache.org
>> Subject: Re: [DISCUSS] What is the criteria for something to be in the
>> jclouds/jclouds repo?
>>
>> 1. So once an api meets the criteria of having a steward, being
>> modern, and having a proper test suite, it can immediately be
>> promoted?
>>
>> I really think that those APIs need to have an abstraction, or a
>> potential abstraction that is being developed, but something real,
>> tangible, that ensures that it is aligned with the essence of what
>> jclouds is.
>>
>> 2. Can you elaborate on this point? Did you have some criteria in mind
>> for what “makes sense” means?
>>
>> There are several features more than one cloud provider offers
>> (networking, load balancers and databases, are the obvious examples)
>> that could have an abstraction. Not every feature in every cloud
>> provider may make sense in others, and thus are not suitable to have
>> an abstraction. In that case, we should be asking ourselves what value
>> adding such an "independent" API gives to jclouds. We have provider
>> specific APIs to *complete* the functionality "offered by the
>> abstractions", but I really don't think is a good idea to add "entire
>> independent apis" that won't have an abstraction because they're too
>> specific to a single cloud provider. When I say "makes sense" I'm
>> saying to ask ourselves if that API really makes sense in the global
>> jclouds picture. If it makes sense, let's work on getting those
>> features available in a portable way to other providers too. If that
>> doesn't make sense, then that API doesn't make sense to jclouds.
>>
>> 3. Are you saying that if there’s an api that would never be a good
>> fit for abstraction, that it has no chance of being promoted?
>>
>> Yes. I really believe we should be doing that. As I said, jclouds is
>> not an API placeholder to every single API that qualifies itself as
>> "cloud". jclouds is all about abstractions, and about providing users
>> a way to write portable code. Of course we have provider specific
>> features, but as I mentioned before, those should serve the purpose of
>> completing the abstractions, but never be independent and standalone
>> providers/apis. That has proven to be difficult to maintain.
>> considerably increase the codebase complexity and make it more
>> difficult to evolve jclouds itself.
>>
>> 4. Are you saying that if an abstraction doesn’t happen in some amount
>> of time, that any apis related to that lack-of-abstraction would be
>> removed?
>>
>> Well, this is open source, and "time" is relative. The complexity of
>> abstractions can be very different: databases seems to be a simple,
>> but networking can be really complex. What I'm saying is that we
>> should *work* on those abstractions and *work in a direction to make
>> those abstractions" something real. But we can't just keep adding more
>> "satellite" APIs and forget about the rest. We should be thinking
>> about abstractions and woking on them, and that's what matters, not
>> time. Take the database APIs as an example. It has been commented more
>> than once that it would be nice to have an abstraction. What actions
>> have been taken in that direction? None. Any proposal? Nope. That's
>> what we have to change.
>>
>>
>>
>> On 8 December 2014 at 16:01, Everett Toews <ev...@rackspace.com>
>> wrote:
>> > On Dec 2, 2014, at 2:31 PM, Ignasi Barrera <na...@apache.org> wrote:
>> >
>> >> This said, the absence of an abstraction (take network or databases as
>> >> an example) shouldn't be a blocker. Abstractions can be proposed in
>> >> the mailing list, and even better discussed in a PR! We are all
>> >> looking to improve jclouds to make it better for the users, and that
>> >> means making it more portable. Let's work on adding an abstraction
>> >> when that makes sense! But things have to be *real* and that work has
>> >> to be done. Without it, it does not make sense to promote/keep such
>> >> apis/providers.
>> >
>> > I think having everything in jclouds/jclouds could possibly work but we
>> would need to be more specific about the above. Otherwise I’m concerned
>> we’d wind up in the same place 6 months down the road.
>> >
>> > If this were to be the path we go down, I have some clarifying
>> questions. These are open questions for anyone to answer.
>> >
>> > “the absence of an abstraction shouldn’t be a blocker”
>> >
>> > 1. So once an api meets the criteria of having a steward, being modern,
>> and having a proper test suite, it can immediately be promoted?
>> >
>> >
>> > “Let's work on adding an abstraction when that makes sense!”
>> >
>> > 2. Can you elaborate on this point? Did you have some criteria in mind
>> for what “makes sense” means?
>> >
>> >
>> > "it does not make sense to promote/keep such apis/providers.”
>> >
>> > These are two different concerns and should be broken out.
>> >
>> > "it does not make sense to promote such apis/providers.”
>> >
>> > 3. Are you saying that if there’s an api that would never be a good fit
>> for abstraction, that it has no chance of being promoted?
>> >
>> > "it does not make sense to keep such apis/providers.”
>> >
>> > 4. Are you saying that if an abstraction doesn’t happen in some amount
>> of time, that any apis related to that lack-of-abstraction would be removed?
>> >
>> >
>> > Everett
>>

RE: [DISCUSS] What is the criteria for something to be in the jclouds/jclouds repo?

[Zack Shoylev <za...@RACKSPACE.COM>]

>> Every single API in jclouds must be aligned with the project's purpose, and
that means *portability*. 

jclouds also tries to do portability "while giving you full control to use cloud-specific features."

>>  if someone contributes a patch adding support for Google Drive

I was not part of that discussion, but it should be possible to add a blobstore abstraction for this, no?

>> So, once again, jclouds is *not* a cloud aggregator for any API labelled
"cloud". It is about portable code, and every single API and provider
should meet the project goals. 

jclouds allows you to write portable applications, but also allows you the flexibility to use cloud-specific features. From a user perspective, you need to have both. You want to be able to build infrastructure using high-level code that works with all the clouds, but occasionally you will need the flexibility to do provider-specific things (even just because this is how providers differentiate themselves).


I wanted to focus on the user perspective for this discussion. In the end, users want things to work well, with less code, and with whatever providers they choose.
Because jclouds is modular, users don't care if jclouds "is not an API aggregator of any API that is labelled "cloud"." - users want jclouds to work well for whatever providers and clouds they choose.
As such, figuring out the criteria of what should be in jclouds/jclouds is not really something that affects users directly, but has to be done to make development easier, saner, and to reduce current and future technical debt.
If we had infinite resources, it would make sense, for users, to include and release all cloud APIs and their abstractions (where such abstractions make sense) as part of jclouds.

Part of this is about balance. Currently we have too many cloud-specific APIs that could be abstracted, but are not. Clearly some of the high-priority work in jclouds is working on abstractions. That's the case at the moment. It might be a knee-jerk reaction to limit jclouds just to abstractable APIs because of this situation, when in the future we might also want to give users the flexibility to also use provider-specific features or even provider-specific APIs that might be useful when used together with higher-level, more abstract code.

This is our current mission statement:
"Apache jclouds® is an open source multi-cloud toolkit for the Java platform that gives you the freedom to create applications that are portable across clouds while giving you full control to use cloud-specific features."

Should this be morphed into:
"Apache jclouds® is an open source multi-cloud toolkit for the Java platform that gives you the freedom to create applications that are portable across clouds"? Is this better for users?

Thanks, and let's keep the discussion going. Figuring things out will help both contributors and users.

________________________________________
From: Ignasi Barrera [nacx@apache.org]
Sent: Wednesday, December 10, 2014 12:59 AM
To: dev@jclouds.apache.org
Subject: RE: [DISCUSS] What is the criteria for something to be in the jclouds/jclouds repo?

I'd never do point 2 and would remove those APIs. As I've said several
times in the thread, jclouds is not an API aggregator of any API that is
labelled "cloud".

Every single API in jclouds must be aligned with the project's purpose, and
that means *portability*. It does not make sense to keep adding APIs that
don't fit in the project's purpose. If we do that we would be facing the
exact same issues we are facing now (I mentioned them in my previous email)
regardless of the number of repos we use.

To put a concrete example: jclouds is *not* the Rackspace SDK nor is the
OpenStack java SDK. Perhaps Rackspace adopts jclouds as its SDK, but
jclouds isn't the Rackspace SDK. And this difference is important. Perhaps
not every single API in Rackspace makes sense in jclouds, as it would
hardly be modelled in a portable way. In that case, we shouldn't add it, as
it does not benefit jclouds as a whole and doesn't serve the jclouds
purpose.

Let's take another example: if someone contributes a patch adding support
for Google Drive (which was discussed in the ML or IRC), providing an API
to store documents, should we merge it? That API does not fit in the
BlobStore abstraction as it does not manage blobs (raw object data) but
documents. The answer is no, because jclouds is not about that (ir is just
a single, isolated API to store documents out there). Without alignment
with what jclouds targets, we shouldn't add it.

So, once again, jclouds is *not* a cloud aggregator for any API labelled
"cloud". It is about portable code, and every single API and provider
should meet the project goals. If not, we should not add them.
El 10/12/2014 01:54, "Zack Shoylev" <za...@rackspace.com> escribió:

> If we take this to its logical conclusion, it sounds like we can separate
> APIs into two categories:
>
> 1. Those that have an abstraction or can be abstracted in the future (to
> be in jclouds/jclouds). Most APIs would be in here.
> 2. Those that provide provider-specific features only, and cannot be
> abstracted. There should only be a few here, but this means a few per
> provider, so overall this category might contain a large number of APIs.
>
> If we were to organize the repos using the two categories above, we can do
> something like (suggestion):
>
> 1. jclouds/jclouds (Same as now, with more APIs - including beta APIs -
> promoted. Regular release schedule)
> 2. jclouds/providers (All provider-specific APIs that will never be
> abstracted. Release together with 1. This also needs a better name, as it
> only contains APIs that are not abstractable).
> 3. jclouds/labs (very experimental only - will not be released)
>
> Note that I am strongly in favor of fewer repos for us to manage. Maybe we
> should not even have 2.
>
> What would you change in the description above?
> Thanks!
> -Zack
> ________________________________________
> From: Ignasi Barrera [ignasi.barrera@gmail.com]
> Sent: Monday, December 08, 2014 4:19 PM
> To: dev@jclouds.apache.org
> Subject: Re: [DISCUSS] What is the criteria for something to be in the
> jclouds/jclouds repo?
>
> 1. So once an api meets the criteria of having a steward, being
> modern, and having a proper test suite, it can immediately be
> promoted?
>
> I really think that those APIs need to have an abstraction, or a
> potential abstraction that is being developed, but something real,
> tangible, that ensures that it is aligned with the essence of what
> jclouds is.
>
> 2. Can you elaborate on this point? Did you have some criteria in mind
> for what “makes sense” means?
>
> There are several features more than one cloud provider offers
> (networking, load balancers and databases, are the obvious examples)
> that could have an abstraction. Not every feature in every cloud
> provider may make sense in others, and thus are not suitable to have
> an abstraction. In that case, we should be asking ourselves what value
> adding such an "independent" API gives to jclouds. We have provider
> specific APIs to *complete* the functionality "offered by the
> abstractions", but I really don't think is a good idea to add "entire
> independent apis" that won't have an abstraction because they're too
> specific to a single cloud provider. When I say "makes sense" I'm
> saying to ask ourselves if that API really makes sense in the global
> jclouds picture. If it makes sense, let's work on getting those
> features available in a portable way to other providers too. If that
> doesn't make sense, then that API doesn't make sense to jclouds.
>
> 3. Are you saying that if there’s an api that would never be a good
> fit for abstraction, that it has no chance of being promoted?
>
> Yes. I really believe we should be doing that. As I said, jclouds is
> not an API placeholder to every single API that qualifies itself as
> "cloud". jclouds is all about abstractions, and about providing users
> a way to write portable code. Of course we have provider specific
> features, but as I mentioned before, those should serve the purpose of
> completing the abstractions, but never be independent and standalone
> providers/apis. That has proven to be difficult to maintain.
> considerably increase the codebase complexity and make it more
> difficult to evolve jclouds itself.
>
> 4. Are you saying that if an abstraction doesn’t happen in some amount
> of time, that any apis related to that lack-of-abstraction would be
> removed?
>
> Well, this is open source, and "time" is relative. The complexity of
> abstractions can be very different: databases seems to be a simple,
> but networking can be really complex. What I'm saying is that we
> should *work* on those abstractions and *work in a direction to make
> those abstractions" something real. But we can't just keep adding more
> "satellite" APIs and forget about the rest. We should be thinking
> about abstractions and woking on them, and that's what matters, not
> time. Take the database APIs as an example. It has been commented more
> than once that it would be nice to have an abstraction. What actions
> have been taken in that direction? None. Any proposal? Nope. That's
> what we have to change.
>
>
>
> On 8 December 2014 at 16:01, Everett Toews <ev...@rackspace.com>
> wrote:
> > On Dec 2, 2014, at 2:31 PM, Ignasi Barrera <na...@apache.org> wrote:
> >
> >> This said, the absence of an abstraction (take network or databases as
> >> an example) shouldn't be a blocker. Abstractions can be proposed in
> >> the mailing list, and even better discussed in a PR! We are all
> >> looking to improve jclouds to make it better for the users, and that
> >> means making it more portable. Let's work on adding an abstraction
> >> when that makes sense! But things have to be *real* and that work has
> >> to be done. Without it, it does not make sense to promote/keep such
> >> apis/providers.
> >
> > I think having everything in jclouds/jclouds could possibly work but we
> would need to be more specific about the above. Otherwise I’m concerned
> we’d wind up in the same place 6 months down the road.
> >
> > If this were to be the path we go down, I have some clarifying
> questions. These are open questions for anyone to answer.
> >
> > “the absence of an abstraction shouldn’t be a blocker”
> >
> > 1. So once an api meets the criteria of having a steward, being modern,
> and having a proper test suite, it can immediately be promoted?
> >
> >
> > “Let's work on adding an abstraction when that makes sense!”
> >
> > 2. Can you elaborate on this point? Did you have some criteria in mind
> for what “makes sense” means?
> >
> >
> > "it does not make sense to promote/keep such apis/providers.”
> >
> > These are two different concerns and should be broken out.
> >
> > "it does not make sense to promote such apis/providers.”
> >
> > 3. Are you saying that if there’s an api that would never be a good fit
> for abstraction, that it has no chance of being promoted?
> >
> > "it does not make sense to keep such apis/providers.”
> >
> > 4. Are you saying that if an abstraction doesn’t happen in some amount
> of time, that any apis related to that lack-of-abstraction would be removed?
> >
> >
> > Everett
>

RE: [DISCUSS] What is the criteria for something to be in the jclouds/jclouds repo?

[Andrew Phillips <ap...@qrmedia.com>]

To echo what Ignasi said, I think we need to note that jclouds calls  
itself "the Java multi-cloud toolkit." To find out what that means in  
the face of some pretty big changes in cloud usage, I think we really  
need to ask our users, but more on that in a different thread. To me,  
the "multi-cloud" in that definition means a couple of things:

1. Providing useful, portable cloud abstractions that are relevant to  
our users now and going forward
2. In APIs and providers, focusing on code that allows us to implement  
those abstractions
3. Ensuring that APIs and providers, especially the most popular ones,  
remain up-to-date and functional
4. Allowing jclouds to be used in as many JVM-based environments, with as few
implications for your code, as feasible

By extension, this also means:

5. Pulling in as few dependencies as possible, and in general imposing minimum
constraints on user code
6. Spending less time trying to support non-portable features in APIs  
and providers (whilst being open to updating the portable  
abstractions, or adding new ones, where justified!)
7. Spending less time supporting APIs and providers that are unrelated  
to portable abstractions
8. Prioritizing issues or missing features in providers that are most  
heavily used, rather than those you happen to be familiar with
9. Removing APIs and providers that no longer work

Taking into account the reality of limited resources, my suggestion  
(concretely, for 1.9.0) would be to try to make jclouds as lean, mean  
and easy to contribute to as possible:

* promote all providers and APIs that implement abstractions and are  
in common use to jclouds/jclouds
* complete half-finished abstractions and discuss where existing  
abstractions need to be modernized
* remove all deadwood providers and APIs
* create an "example PR" of how to add a new minimal provider or API,  
using the "latest and greatest" jclouds dev practices (@nacx: would  
the Abiquo re-impl be a good candidate?)

Regards

ap

RE: [DISCUSS] What is the criteria for something to be in the jclouds/jclouds repo?

[Ignasi Barrera <na...@apache.org>]

I'd never do point 2 and would remove those APIs. As I've said several
times in the thread, jclouds is not an API aggregator of any API that is
labelled "cloud".

Every single API in jclouds must be aligned with the project's purpose, and
that means *portability*. It does not make sense to keep adding APIs that
don't fit in the project's purpose. If we do that we would be facing the
exact same issues we are facing now (I mentioned them in my previous email)
regardless of the number of repos we use.

To put a concrete example: jclouds is *not* the Rackspace SDK nor is the
OpenStack java SDK. Perhaps Rackspace adopts jclouds as its SDK, but
jclouds isn't the Rackspace SDK. And this difference is important. Perhaps
not every single API in Rackspace makes sense in jclouds, as it would
hardly be modelled in a portable way. In that case, we shouldn't add it, as
it does not benefit jclouds as a whole and doesn't serve the jclouds
purpose.

Let's take another example: if someone contributes a patch adding support
for Google Drive (which was discussed in the ML or IRC), providing an API
to store documents, should we merge it? That API does not fit in the
BlobStore abstraction as it does not manage blobs (raw object data) but
documents. The answer is no, because jclouds is not about that (ir is just
a single, isolated API to store documents out there). Without alignment
with what jclouds targets, we shouldn't add it.

So, once again, jclouds is *not* a cloud aggregator for any API labelled
"cloud". It is about portable code, and every single API and provider
should meet the project goals. If not, we should not add them.
El 10/12/2014 01:54, "Zack Shoylev" <za...@rackspace.com> escribió:

> If we take this to its logical conclusion, it sounds like we can separate
> APIs into two categories:
>
> 1. Those that have an abstraction or can be abstracted in the future (to
> be in jclouds/jclouds). Most APIs would be in here.
> 2. Those that provide provider-specific features only, and cannot be
> abstracted. There should only be a few here, but this means a few per
> provider, so overall this category might contain a large number of APIs.
>
> If we were to organize the repos using the two categories above, we can do
> something like (suggestion):
>
> 1. jclouds/jclouds (Same as now, with more APIs - including beta APIs -
> promoted. Regular release schedule)
> 2. jclouds/providers (All provider-specific APIs that will never be
> abstracted. Release together with 1. This also needs a better name, as it
> only contains APIs that are not abstractable).
> 3. jclouds/labs (very experimental only - will not be released)
>
> Note that I am strongly in favor of fewer repos for us to manage. Maybe we
> should not even have 2.
>
> What would you change in the description above?
> Thanks!
> -Zack
> ________________________________________
> From: Ignasi Barrera [ignasi.barrera@gmail.com]
> Sent: Monday, December 08, 2014 4:19 PM
> To: dev@jclouds.apache.org
> Subject: Re: [DISCUSS] What is the criteria for something to be in the
> jclouds/jclouds repo?
>
> 1. So once an api meets the criteria of having a steward, being
> modern, and having a proper test suite, it can immediately be
> promoted?
>
> I really think that those APIs need to have an abstraction, or a
> potential abstraction that is being developed, but something real,
> tangible, that ensures that it is aligned with the essence of what
> jclouds is.
>
> 2. Can you elaborate on this point? Did you have some criteria in mind
> for what “makes sense” means?
>
> There are several features more than one cloud provider offers
> (networking, load balancers and databases, are the obvious examples)
> that could have an abstraction. Not every feature in every cloud
> provider may make sense in others, and thus are not suitable to have
> an abstraction. In that case, we should be asking ourselves what value
> adding such an "independent" API gives to jclouds. We have provider
> specific APIs to *complete* the functionality "offered by the
> abstractions", but I really don't think is a good idea to add "entire
> independent apis" that won't have an abstraction because they're too
> specific to a single cloud provider. When I say "makes sense" I'm
> saying to ask ourselves if that API really makes sense in the global
> jclouds picture. If it makes sense, let's work on getting those
> features available in a portable way to other providers too. If that
> doesn't make sense, then that API doesn't make sense to jclouds.
>
> 3. Are you saying that if there’s an api that would never be a good
> fit for abstraction, that it has no chance of being promoted?
>
> Yes. I really believe we should be doing that. As I said, jclouds is
> not an API placeholder to every single API that qualifies itself as
> "cloud". jclouds is all about abstractions, and about providing users
> a way to write portable code. Of course we have provider specific
> features, but as I mentioned before, those should serve the purpose of
> completing the abstractions, but never be independent and standalone
> providers/apis. That has proven to be difficult to maintain.
> considerably increase the codebase complexity and make it more
> difficult to evolve jclouds itself.
>
> 4. Are you saying that if an abstraction doesn’t happen in some amount
> of time, that any apis related to that lack-of-abstraction would be
> removed?
>
> Well, this is open source, and "time" is relative. The complexity of
> abstractions can be very different: databases seems to be a simple,
> but networking can be really complex. What I'm saying is that we
> should *work* on those abstractions and *work in a direction to make
> those abstractions" something real. But we can't just keep adding more
> "satellite" APIs and forget about the rest. We should be thinking
> about abstractions and woking on them, and that's what matters, not
> time. Take the database APIs as an example. It has been commented more
> than once that it would be nice to have an abstraction. What actions
> have been taken in that direction? None. Any proposal? Nope. That's
> what we have to change.
>
>
>
> On 8 December 2014 at 16:01, Everett Toews <ev...@rackspace.com>
> wrote:
> > On Dec 2, 2014, at 2:31 PM, Ignasi Barrera <na...@apache.org> wrote:
> >
> >> This said, the absence of an abstraction (take network or databases as
> >> an example) shouldn't be a blocker. Abstractions can be proposed in
> >> the mailing list, and even better discussed in a PR! We are all
> >> looking to improve jclouds to make it better for the users, and that
> >> means making it more portable. Let's work on adding an abstraction
> >> when that makes sense! But things have to be *real* and that work has
> >> to be done. Without it, it does not make sense to promote/keep such
> >> apis/providers.
> >
> > I think having everything in jclouds/jclouds could possibly work but we
> would need to be more specific about the above. Otherwise I’m concerned
> we’d wind up in the same place 6 months down the road.
> >
> > If this were to be the path we go down, I have some clarifying
> questions. These are open questions for anyone to answer.
> >
> > “the absence of an abstraction shouldn’t be a blocker”
> >
> > 1. So once an api meets the criteria of having a steward, being modern,
> and having a proper test suite, it can immediately be promoted?
> >
> >
> > “Let's work on adding an abstraction when that makes sense!”
> >
> > 2. Can you elaborate on this point? Did you have some criteria in mind
> for what “makes sense” means?
> >
> >
> > "it does not make sense to promote/keep such apis/providers.”
> >
> > These are two different concerns and should be broken out.
> >
> > "it does not make sense to promote such apis/providers.”
> >
> > 3. Are you saying that if there’s an api that would never be a good fit
> for abstraction, that it has no chance of being promoted?
> >
> > "it does not make sense to keep such apis/providers.”
> >
> > 4. Are you saying that if an abstraction doesn’t happen in some amount
> of time, that any apis related to that lack-of-abstraction would be removed?
> >
> >
> > Everett
>

RE: [DISCUSS] What is the criteria for something to be in the jclouds/jclouds repo?

[Zack Shoylev <za...@RACKSPACE.COM>]

If we take this to its logical conclusion, it sounds like we can separate APIs into two categories:

1. Those that have an abstraction or can be abstracted in the future (to be in jclouds/jclouds). Most APIs would be in here.
2. Those that provide provider-specific features only, and cannot be abstracted. There should only be a few here, but this means a few per provider, so overall this category might contain a large number of APIs.

If we were to organize the repos using the two categories above, we can do something like (suggestion):

1. jclouds/jclouds (Same as now, with more APIs - including beta APIs - promoted. Regular release schedule)
2. jclouds/providers (All provider-specific APIs that will never be abstracted. Release together with 1. This also needs a better name, as it only contains APIs that are not abstractable).
3. jclouds/labs (very experimental only - will not be released)

Note that I am strongly in favor of fewer repos for us to manage. Maybe we should not even have 2.

What would you change in the description above?
Thanks!
-Zack
________________________________________
From: Ignasi Barrera [ignasi.barrera@gmail.com]
Sent: Monday, December 08, 2014 4:19 PM
To: dev@jclouds.apache.org
Subject: Re: [DISCUSS] What is the criteria for something to be in the jclouds/jclouds repo?

1. So once an api meets the criteria of having a steward, being
modern, and having a proper test suite, it can immediately be
promoted?

I really think that those APIs need to have an abstraction, or a
potential abstraction that is being developed, but something real,
tangible, that ensures that it is aligned with the essence of what
jclouds is.

2. Can you elaborate on this point? Did you have some criteria in mind
for what “makes sense” means?

There are several features more than one cloud provider offers
(networking, load balancers and databases, are the obvious examples)
that could have an abstraction. Not every feature in every cloud
provider may make sense in others, and thus are not suitable to have
an abstraction. In that case, we should be asking ourselves what value
adding such an "independent" API gives to jclouds. We have provider
specific APIs to *complete* the functionality "offered by the
abstractions", but I really don't think is a good idea to add "entire
independent apis" that won't have an abstraction because they're too
specific to a single cloud provider. When I say "makes sense" I'm
saying to ask ourselves if that API really makes sense in the global
jclouds picture. If it makes sense, let's work on getting those
features available in a portable way to other providers too. If that
doesn't make sense, then that API doesn't make sense to jclouds.

3. Are you saying that if there’s an api that would never be a good
fit for abstraction, that it has no chance of being promoted?

Yes. I really believe we should be doing that. As I said, jclouds is
not an API placeholder to every single API that qualifies itself as
"cloud". jclouds is all about abstractions, and about providing users
a way to write portable code. Of course we have provider specific
features, but as I mentioned before, those should serve the purpose of
completing the abstractions, but never be independent and standalone
providers/apis. That has proven to be difficult to maintain.
considerably increase the codebase complexity and make it more
difficult to evolve jclouds itself.

4. Are you saying that if an abstraction doesn’t happen in some amount
of time, that any apis related to that lack-of-abstraction would be
removed?

Well, this is open source, and "time" is relative. The complexity of
abstractions can be very different: databases seems to be a simple,
but networking can be really complex. What I'm saying is that we
should *work* on those abstractions and *work in a direction to make
those abstractions" something real. But we can't just keep adding more
"satellite" APIs and forget about the rest. We should be thinking
about abstractions and woking on them, and that's what matters, not
time. Take the database APIs as an example. It has been commented more
than once that it would be nice to have an abstraction. What actions
have been taken in that direction? None. Any proposal? Nope. That's
what we have to change.



On 8 December 2014 at 16:01, Everett Toews <ev...@rackspace.com> wrote:
> On Dec 2, 2014, at 2:31 PM, Ignasi Barrera <na...@apache.org> wrote:
>
>> This said, the absence of an abstraction (take network or databases as
>> an example) shouldn't be a blocker. Abstractions can be proposed in
>> the mailing list, and even better discussed in a PR! We are all
>> looking to improve jclouds to make it better for the users, and that
>> means making it more portable. Let's work on adding an abstraction
>> when that makes sense! But things have to be *real* and that work has
>> to be done. Without it, it does not make sense to promote/keep such
>> apis/providers.
>
> I think having everything in jclouds/jclouds could possibly work but we would need to be more specific about the above. Otherwise I’m concerned we’d wind up in the same place 6 months down the road.
>
> If this were to be the path we go down, I have some clarifying questions. These are open questions for anyone to answer.
>
> “the absence of an abstraction shouldn’t be a blocker”
>
> 1. So once an api meets the criteria of having a steward, being modern, and having a proper test suite, it can immediately be promoted?
>
>
> “Let's work on adding an abstraction when that makes sense!”
>
> 2. Can you elaborate on this point? Did you have some criteria in mind for what “makes sense” means?
>
>
> "it does not make sense to promote/keep such apis/providers.”
>
> These are two different concerns and should be broken out.
>
> "it does not make sense to promote such apis/providers.”
>
> 3. Are you saying that if there’s an api that would never be a good fit for abstraction, that it has no chance of being promoted?
>
> "it does not make sense to keep such apis/providers.”
>
> 4. Are you saying that if an abstraction doesn’t happen in some amount of time, that any apis related to that lack-of-abstraction would be removed?
>
>
> Everett

Re: [DISCUSS] What is the criteria for something to be in the jclouds/jclouds repo?

[Andrew Phillips <ap...@qrmedia.com>]

> 1. So once an api meets the criteria of having a steward, being
> modern, and having a proper test suite, it can immediately be
> promoted?

Looks like Ignasi beat me to it - I was going to make points very much  
along those lines.

About stewards: there has been a long and (it seems) heated   
discussion on board@ about such positions of "special responsibility."  
Our suggestion is not quite the same - the board@ discussion talks  
about an additional criterion that the steward would have to approve  
any change to that part of the codebase - but it made me wonder  
whether this is a good idea and really in line with the Apache Way.

My current thinking is that this is *not* something we should be  
doing, as it risks implying:

a) that certain people have "more" responsibility for a part of the  
code than others and
b) in consequence, that "the rest" has *less* responsibility for that  
part of the code

We already have a significant challenge in ensuring widely-used parts  
of the jclouds codebase remain usable (never mind up-to-date!) because  
there are no obvious "stewards" and, more importantly, the rest of the  
committer base (me included) evidently do not consider it "essential  
enough" to put time into fixing things.

To me, this is an absolutely critical issue to address, more so even  
than figuring out when or if to promote a provider or API. *All of  
jclouds is the code and the responsibility of the entire community -  
not only the bits we as individuals find most interesting.*

To that end, I would like to avoid any notion that we nominate certain  
people to take care of parts of the code more than others.

I'd be very interested to hear what others with experience of more  
Apache projects have to say here.

Regards

ap

Re: [DISCUSS] What is the criteria for something to be in the jclouds/jclouds repo?

[Ignasi Barrera <ig...@gmail.com>]

1. So once an api meets the criteria of having a steward, being
modern, and having a proper test suite, it can immediately be
promoted?

I really think that those APIs need to have an abstraction, or a
potential abstraction that is being developed, but something real,
tangible, that ensures that it is aligned with the essence of what
jclouds is.

2. Can you elaborate on this point? Did you have some criteria in mind
for what “makes sense” means?

There are several features more than one cloud provider offers
(networking, load balancers and databases, are the obvious examples)
that could have an abstraction. Not every feature in every cloud
provider may make sense in others, and thus are not suitable to have
an abstraction. In that case, we should be asking ourselves what value
adding such an "independent" API gives to jclouds. We have provider
specific APIs to *complete* the functionality "offered by the
abstractions", but I really don't think is a good idea to add "entire
independent apis" that won't have an abstraction because they're too
specific to a single cloud provider. When I say "makes sense" I'm
saying to ask ourselves if that API really makes sense in the global
jclouds picture. If it makes sense, let's work on getting those
features available in a portable way to other providers too. If that
doesn't make sense, then that API doesn't make sense to jclouds.

3. Are you saying that if there’s an api that would never be a good
fit for abstraction, that it has no chance of being promoted?

Yes. I really believe we should be doing that. As I said, jclouds is
not an API placeholder to every single API that qualifies itself as
"cloud". jclouds is all about abstractions, and about providing users
a way to write portable code. Of course we have provider specific
features, but as I mentioned before, those should serve the purpose of
completing the abstractions, but never be independent and standalone
providers/apis. That has proven to be difficult to maintain.
considerably increase the codebase complexity and make it more
difficult to evolve jclouds itself.

4. Are you saying that if an abstraction doesn’t happen in some amount
of time, that any apis related to that lack-of-abstraction would be
removed?

Well, this is open source, and "time" is relative. The complexity of
abstractions can be very different: databases seems to be a simple,
but networking can be really complex. What I'm saying is that we
should *work* on those abstractions and *work in a direction to make
those abstractions" something real. But we can't just keep adding more
"satellite" APIs and forget about the rest. We should be thinking
about abstractions and woking on them, and that's what matters, not
time. Take the database APIs as an example. It has been commented more
than once that it would be nice to have an abstraction. What actions
have been taken in that direction? None. Any proposal? Nope. That's
what we have to change.



On 8 December 2014 at 16:01, Everett Toews <ev...@rackspace.com> wrote:
> On Dec 2, 2014, at 2:31 PM, Ignasi Barrera <na...@apache.org> wrote:
>
>> This said, the absence of an abstraction (take network or databases as
>> an example) shouldn't be a blocker. Abstractions can be proposed in
>> the mailing list, and even better discussed in a PR! We are all
>> looking to improve jclouds to make it better for the users, and that
>> means making it more portable. Let's work on adding an abstraction
>> when that makes sense! But things have to be *real* and that work has
>> to be done. Without it, it does not make sense to promote/keep such
>> apis/providers.
>
> I think having everything in jclouds/jclouds could possibly work but we would need to be more specific about the above. Otherwise I’m concerned we’d wind up in the same place 6 months down the road.
>
> If this were to be the path we go down, I have some clarifying questions. These are open questions for anyone to answer.
>
> “the absence of an abstraction shouldn’t be a blocker”
>
> 1. So once an api meets the criteria of having a steward, being modern, and having a proper test suite, it can immediately be promoted?
>
>
> “Let's work on adding an abstraction when that makes sense!”
>
> 2. Can you elaborate on this point? Did you have some criteria in mind for what “makes sense” means?
>
>
> "it does not make sense to promote/keep such apis/providers.”
>
> These are two different concerns and should be broken out.
>
> "it does not make sense to promote such apis/providers.”
>
> 3. Are you saying that if there’s an api that would never be a good fit for abstraction, that it has no chance of being promoted?
>
> "it does not make sense to keep such apis/providers.”
>
> 4. Are you saying that if an abstraction doesn’t happen in some amount of time, that any apis related to that lack-of-abstraction would be removed?
>
>
> Everett

Re: [DISCUSS] What is the criteria for something to be in the jclouds/jclouds repo?

[Everett Toews <ev...@RACKSPACE.COM>]

On Dec 2, 2014, at 2:31 PM, Ignasi Barrera <na...@apache.org> wrote:

> This said, the absence of an abstraction (take network or databases as
> an example) shouldn't be a blocker. Abstractions can be proposed in
> the mailing list, and even better discussed in a PR! We are all
> looking to improve jclouds to make it better for the users, and that
> means making it more portable. Let's work on adding an abstraction
> when that makes sense! But things have to be *real* and that work has
> to be done. Without it, it does not make sense to promote/keep such
> apis/providers.

I think having everything in jclouds/jclouds could possibly work but we would need to be more specific about the above. Otherwise I’m concerned we’d wind up in the same place 6 months down the road.

If this were to be the path we go down, I have some clarifying questions. These are open questions for anyone to answer. 

“the absence of an abstraction shouldn’t be a blocker”

1. So once an api meets the criteria of having a steward, being modern, and having a proper test suite, it can immediately be promoted?


“Let's work on adding an abstraction when that makes sense!”

2. Can you elaborate on this point? Did you have some criteria in mind for what “makes sense” means?


"it does not make sense to promote/keep such apis/providers.”

These are two different concerns and should be broken out.

"it does not make sense to promote such apis/providers.”

3. Are you saying that if there’s an api that would never be a good fit for abstraction, that it has no chance of being promoted?

"it does not make sense to keep such apis/providers.”

4. Are you saying that if an abstraction doesn’t happen in some amount of time, that any apis related to that lack-of-abstraction would be removed? 


Everett

Re: [DISCUSS] What is the criteria for something to be in the jclouds/jclouds repo?

[Ignasi Barrera <na...@apache.org>]

In my opinion there are several requirements to be met, all them mandatory:

1. New providers MUST have an abstraction (more on this below).
2. Must have a Steward, who maintains the provider, takes the issues,
and runs the live tests periodically (at least before each release).
3. It must follow the coding guidelines. As Andrea mentioned,
technical debt is a *real* blocker when it comes to refactor the core
to fix structural issues. We can't keep adding it to the main repo.
Huge providers that are out of date are only going to cause us
trouble.

Regarding the abstractions. We need to think about what jclouds is.
jclouds is ALL about the abstractions. It is not an aggregator of any
api that is labelled "cloud". Without the abstractions, jclouds makes
no sense. Users don't come to jclouds looking for concrete APIs. They
come looking for a portable toolkit. If you have a look at the mailing
list, most questions are about the Blob and Compute abstraction. There
are also many about the Neutron API, but that is *only* because those
APIs have failed to create an abstraction. Users are asking for
concrete APIs just because there is no abstraction for them.

This said, the absence of an abstraction (take network or databases as
an example) shouldn't be a blocker. Abstractions can be proposed in
the mailing list, and even better discussed in a PR! We are all
looking to improve jclouds to make it better for the users, and that
means making it more portable. Let's work on adding an abstraction
when that makes sense! But things have to be *real* and that work has
to be done. Without it, it does not make sense to promote/keep such
apis/providers.

I.


On 2 December 2014 at 18:44, Andrea Turli <an...@gmail.com> wrote:
> Hi,
>
> I personally think we should consider a couple of additional criteria to
> the good Zack's list.
> The code should support the latest stable (recommended) version of the API.
> The implementation should follow all the current guidelines (i. e. Use of
> Auto Value and Service, MWS, no async API)  so that the new providers don't
> introduce tech debt, this should be kept during reviews but we need some
> standard way to "enforce" them.
> MockTests and LiveTests have to be green.
> The maintainer has to be able to run the LiveTests for each release.
>
> As contributor to labs repos this is my understanding of what it is needed
> to get the provider out of labs, hope this helps.
>
> Andrea
> Il 02/dic/2014 18:28 "Zack Shoylev" <za...@rackspace.com> ha scritto:
>
>> Possible criteria that have been discussed before:
>>
>> 1. If it is out of beta
>> 2. If it has a maintainer
>> 3. If it has a jclouds abstraction layer
>> 4. Voted on a case by case basis (current way of doing things)
>>
>> Are there any more possible criteria I have left out? In the end it will
>> be some mix of the above. My thoughts on this is that 2) is sufficient.
>> -Zack
>> ________________________________________
>> From: Dancy, Chris [Chris.Dancy@pega.com]
>> Sent: Tuesday, December 02, 2014 8:27 AM
>> To: dev@jclouds.apache.org
>> Subject: Re: [DISCUSS] What is the criteria for something to be in the
>> jclouds/jclouds repo?
>>
>> As an occasional contributor, and currently coding a Shipyard provider
>> (JCLOUDS-782), I do have a vested interest in how this ends up. IMO I
>> think having a clear separation of ³core² projects vs ³baking² projects is
>> a good idea but it seems that is not working out so great here? Are there
>> other proposals on how to do this?
>>
>> On 12/2/14, 4:09 AM, "Inbar Stolberg" <in...@gmail.com> wrote:
>>
>> >I think openstack neutron should be in jcluods master currently it suffers
>> >from being left out even though it is being updated and runs on openstack
>> >api v2.
>> >
>> >regarding glance I think after
>> >https://github.com/jclouds/jclouds-labs-openstack/pull/77 is merged we
>> >should move it.
>> >
>> >regarding Ceilometer I think that taking to accout what you said Everett:
>> >" in email threads or IRC chats most everybody agrees that the labs repos
>> >are a failed experiment"
>> >maybe its worth considering porting it to jclouds/jclouds from right after
>> >the merge of :
>> >https://github.com/jclouds/jclouds-labs-openstack/pull/166
>> >
>> >(which btw I am already going to use in production code...)
>> >
>> >my 2 cents
>> >Regards Inbar
>> >
>> >
>> >On Tue, Dec 2, 2014 at 2:06 AM, Everett Toews
>> ><ev...@rackspace.com>
>> >wrote:
>> >
>> >> What is the criteria for something to be in the jclouds/jclouds repo
>> >>[1]?
>> >>
>> >> Before we go moving any more code around or removing any more
>> >> apis/providers we need to step back and have some agreed upon criteria
>> >>for
>> >> what belongs in jclouds/jclouds and what doesn¹t. If we don¹t do that
>> >>our
>> >> decisions are effectively arbitrary and the questions of ³What to do
>> >>about
>> >> X?² will continue to come up.
>> >>
>> >> I took a shot at addressing a topic like this in "Criteria for moving an
>> >> api/provider from a labs repo to core² [2] but I got zero replies.
>> >>People
>> >> seem to be unwilling or uninterested in tackling this question and I
>> >>think
>> >> jclouds is suffering because of it. It¹s puzzling to me because in email
>> >> threads or IRC chats most everybody agrees that the labs repos are a
>> >>failed
>> >> experiment. So let¹s flip it around and discuss what belongs in
>> >> jclouds/jclouds instead.
>> >>
>> >> I know for a fact that many contributors have an opinion on this. Let¹s
>> >> hear it!
>> >>
>> >> Thanks,
>> >> Everett
>> >>
>> >> [1] https://github.com/jclouds/jclouds
>>
>>

RE: [DISCUSS] What is the criteria for something to be in the jclouds/jclouds repo?

[Andrea Turli <an...@gmail.com>]

Hi,

I personally think we should consider a couple of additional criteria to
the good Zack's list.
The code should support the latest stable (recommended) version of the API.
The implementation should follow all the current guidelines (i. e. Use of
Auto Value and Service, MWS, no async API)  so that the new providers don't
introduce tech debt, this should be kept during reviews but we need some
standard way to "enforce" them.
MockTests and LiveTests have to be green.
The maintainer has to be able to run the LiveTests for each release.

As contributor to labs repos this is my understanding of what it is needed
to get the provider out of labs, hope this helps.

Andrea
Il 02/dic/2014 18:28 "Zack Shoylev" <za...@rackspace.com> ha scritto:

> Possible criteria that have been discussed before:
>
> 1. If it is out of beta
> 2. If it has a maintainer
> 3. If it has a jclouds abstraction layer
> 4. Voted on a case by case basis (current way of doing things)
>
> Are there any more possible criteria I have left out? In the end it will
> be some mix of the above. My thoughts on this is that 2) is sufficient.
> -Zack
> ________________________________________
> From: Dancy, Chris [Chris.Dancy@pega.com]
> Sent: Tuesday, December 02, 2014 8:27 AM
> To: dev@jclouds.apache.org
> Subject: Re: [DISCUSS] What is the criteria for something to be in the
> jclouds/jclouds repo?
>
> As an occasional contributor, and currently coding a Shipyard provider
> (JCLOUDS-782), I do have a vested interest in how this ends up. IMO I
> think having a clear separation of ³core² projects vs ³baking² projects is
> a good idea but it seems that is not working out so great here? Are there
> other proposals on how to do this?
>
> On 12/2/14, 4:09 AM, "Inbar Stolberg" <in...@gmail.com> wrote:
>
> >I think openstack neutron should be in jcluods master currently it suffers
> >from being left out even though it is being updated and runs on openstack
> >api v2.
> >
> >regarding glance I think after
> >https://github.com/jclouds/jclouds-labs-openstack/pull/77 is merged we
> >should move it.
> >
> >regarding Ceilometer I think that taking to accout what you said Everett:
> >" in email threads or IRC chats most everybody agrees that the labs repos
> >are a failed experiment"
> >maybe its worth considering porting it to jclouds/jclouds from right after
> >the merge of :
> >https://github.com/jclouds/jclouds-labs-openstack/pull/166
> >
> >(which btw I am already going to use in production code...)
> >
> >my 2 cents
> >Regards Inbar
> >
> >
> >On Tue, Dec 2, 2014 at 2:06 AM, Everett Toews
> ><ev...@rackspace.com>
> >wrote:
> >
> >> What is the criteria for something to be in the jclouds/jclouds repo
> >>[1]?
> >>
> >> Before we go moving any more code around or removing any more
> >> apis/providers we need to step back and have some agreed upon criteria
> >>for
> >> what belongs in jclouds/jclouds and what doesn¹t. If we don¹t do that
> >>our
> >> decisions are effectively arbitrary and the questions of ³What to do
> >>about
> >> X?² will continue to come up.
> >>
> >> I took a shot at addressing a topic like this in "Criteria for moving an
> >> api/provider from a labs repo to core² [2] but I got zero replies.
> >>People
> >> seem to be unwilling or uninterested in tackling this question and I
> >>think
> >> jclouds is suffering because of it. It¹s puzzling to me because in email
> >> threads or IRC chats most everybody agrees that the labs repos are a
> >>failed
> >> experiment. So let¹s flip it around and discuss what belongs in
> >> jclouds/jclouds instead.
> >>
> >> I know for a fact that many contributors have an opinion on this. Let¹s
> >> hear it!
> >>
> >> Thanks,
> >> Everett
> >>
> >> [1] https://github.com/jclouds/jclouds
>
>

RE: [DISCUSS] What is the criteria for something to be in the jclouds/jclouds repo?

[Zack Shoylev <za...@RACKSPACE.COM>]

Possible criteria that have been discussed before: 

1. If it is out of beta
2. If it has a maintainer
3. If it has a jclouds abstraction layer
4. Voted on a case by case basis (current way of doing things)

Are there any more possible criteria I have left out? In the end it will be some mix of the above. My thoughts on this is that 2) is sufficient.
-Zack
________________________________________
From: Dancy, Chris [Chris.Dancy@pega.com]
Sent: Tuesday, December 02, 2014 8:27 AM
To: dev@jclouds.apache.org
Subject: Re: [DISCUSS] What is the criteria for something to be in the jclouds/jclouds repo?

As an occasional contributor, and currently coding a Shipyard provider
(JCLOUDS-782), I do have a vested interest in how this ends up. IMO I
think having a clear separation of ³core² projects vs ³baking² projects is
a good idea but it seems that is not working out so great here? Are there
other proposals on how to do this?

On 12/2/14, 4:09 AM, "Inbar Stolberg" <in...@gmail.com> wrote:

>I think openstack neutron should be in jcluods master currently it suffers
>from being left out even though it is being updated and runs on openstack
>api v2.
>
>regarding glance I think after
>https://github.com/jclouds/jclouds-labs-openstack/pull/77 is merged we
>should move it.
>
>regarding Ceilometer I think that taking to accout what you said Everett:
>" in email threads or IRC chats most everybody agrees that the labs repos
>are a failed experiment"
>maybe its worth considering porting it to jclouds/jclouds from right after
>the merge of :
>https://github.com/jclouds/jclouds-labs-openstack/pull/166
>
>(which btw I am already going to use in production code...)
>
>my 2 cents
>Regards Inbar
>
>
>On Tue, Dec 2, 2014 at 2:06 AM, Everett Toews
><ev...@rackspace.com>
>wrote:
>
>> What is the criteria for something to be in the jclouds/jclouds repo
>>[1]?
>>
>> Before we go moving any more code around or removing any more
>> apis/providers we need to step back and have some agreed upon criteria
>>for
>> what belongs in jclouds/jclouds and what doesn¹t. If we don¹t do that
>>our
>> decisions are effectively arbitrary and the questions of ³What to do
>>about
>> X?² will continue to come up.
>>
>> I took a shot at addressing a topic like this in "Criteria for moving an
>> api/provider from a labs repo to core² [2] but I got zero replies.
>>People
>> seem to be unwilling or uninterested in tackling this question and I
>>think
>> jclouds is suffering because of it. It¹s puzzling to me because in email
>> threads or IRC chats most everybody agrees that the labs repos are a
>>failed
>> experiment. So let¹s flip it around and discuss what belongs in
>> jclouds/jclouds instead.
>>
>> I know for a fact that many contributors have an opinion on this. Let¹s
>> hear it!
>>
>> Thanks,
>> Everett
>>
>> [1] https://github.com/jclouds/jclouds

Re: [DISCUSS] What is the criteria for something to be in the jclouds/jclouds repo?

["Dancy, Chris" <Ch...@pega.com>]

As an occasional contributor, and currently coding a Shipyard provider
(JCLOUDS-782), I do have a vested interest in how this ends up. IMO I
think having a clear separation of ³core² projects vs ³baking² projects is
a good idea but it seems that is not working out so great here? Are there
other proposals on how to do this?

On 12/2/14, 4:09 AM, "Inbar Stolberg" <in...@gmail.com> wrote:

>I think openstack neutron should be in jcluods master currently it suffers
>from being left out even though it is being updated and runs on openstack
>api v2.
>
>regarding glance I think after
>https://github.com/jclouds/jclouds-labs-openstack/pull/77 is merged we
>should move it.
>
>regarding Ceilometer I think that taking to accout what you said Everett:
>" in email threads or IRC chats most everybody agrees that the labs repos
>are a failed experiment"
>maybe its worth considering porting it to jclouds/jclouds from right after
>the merge of :
>https://github.com/jclouds/jclouds-labs-openstack/pull/166
>
>(which btw I am already going to use in production code...)
>
>my 2 cents
>Regards Inbar
>
>
>On Tue, Dec 2, 2014 at 2:06 AM, Everett Toews
><ev...@rackspace.com>
>wrote:
>
>> What is the criteria for something to be in the jclouds/jclouds repo
>>[1]?
>>
>> Before we go moving any more code around or removing any more
>> apis/providers we need to step back and have some agreed upon criteria
>>for
>> what belongs in jclouds/jclouds and what doesn¹t. If we don¹t do that
>>our
>> decisions are effectively arbitrary and the questions of ³What to do
>>about
>> X?² will continue to come up.
>>
>> I took a shot at addressing a topic like this in "Criteria for moving an
>> api/provider from a labs repo to core² [2] but I got zero replies.
>>People
>> seem to be unwilling or uninterested in tackling this question and I
>>think
>> jclouds is suffering because of it. It¹s puzzling to me because in email
>> threads or IRC chats most everybody agrees that the labs repos are a
>>failed
>> experiment. So let¹s flip it around and discuss what belongs in
>> jclouds/jclouds instead.
>>
>> I know for a fact that many contributors have an opinion on this. Let¹s
>> hear it!
>>
>> Thanks,
>> Everett
>>
>> [1] https://github.com/jclouds/jclouds

Re: [DISCUSS] What is the criteria for something to be in the jclouds/jclouds repo?

[Inbar Stolberg <in...@gmail.com>]

I think openstack neutron should be in jcluods master currently it suffers
from being left out even though it is being updated and runs on openstack
api v2.

regarding glance I think after
https://github.com/jclouds/jclouds-labs-openstack/pull/77 is merged we
should move it.

regarding Ceilometer I think that taking to accout what you said Everett:
" in email threads or IRC chats most everybody agrees that the labs repos
are a failed experiment"
maybe its worth considering porting it to jclouds/jclouds from right after
the merge of :
https://github.com/jclouds/jclouds-labs-openstack/pull/166

(which btw I am already going to use in production code...)

my 2 cents
Regards Inbar


On Tue, Dec 2, 2014 at 2:06 AM, Everett Toews <ev...@rackspace.com>
wrote:

> What is the criteria for something to be in the jclouds/jclouds repo [1]?
>
> Before we go moving any more code around or removing any more
> apis/providers we need to step back and have some agreed upon criteria for
> what belongs in jclouds/jclouds and what doesn’t. If we don’t do that our
> decisions are effectively arbitrary and the questions of “What to do about
> X?” will continue to come up.
>
> I took a shot at addressing a topic like this in "Criteria for moving an
> api/provider from a labs repo to core” [2] but I got zero replies. People
> seem to be unwilling or uninterested in tackling this question and I think
> jclouds is suffering because of it. It’s puzzling to me because in email
> threads or IRC chats most everybody agrees that the labs repos are a failed
> experiment. So let’s flip it around and discuss what belongs in
> jclouds/jclouds instead.
>
> I know for a fact that many contributors have an opinion on this. Let’s
> hear it!
>
> Thanks,
> Everett
>
> [1] https://github.com/jclouds/jclouds