Weight of compute apis

[Adrian Cole <ad...@gmail.com>]

Some trivia, which relates to my earlier vcloud-director note.

According to cloc [1], our heaviest api by far is cloudstack, weighing
in at ~24K lines of java. Even adding openstack keystone (shared by
all openstack) Nova sits at ~15k lines. Interestingly, the old version
of vcloud is ~11k lines; half of the 23k lines of vcloud-director.

Middle-of-the road folks include gogrid, softlayer, and glesys: all in
the 4k range.

Simple (json) compute apis are even smaller. For example, joyent and
digitalocean are both <3k lines.

While only one signal, which discounts cost of maintaining tests or
lack thereof, we can use this 2.5-25k line of code weight to help
decide what to promote.

I'd suggest we look hard at the apis that fall at or above a 5k
watermark and be careful which we choose to promote, as these likely
will have higher maintenance that other apis even if they are stable
from a version standpoint.

vcloud-director (incomplete) ~23k
abiquo ~13k
google-compute-engine (inc shared oauth) ~10k
cloudsigma2 ~7k
virtualbox ~6k
fgcp ~5k

I know that google and cloudsigma2 are currently very healthy, and of
course google is really not optional.

That said, are there any providers above we are ok to stop working
towards promotion (and instead towards deletion)?
-A


[1] Taken from cloc, ignoring tests, whitespace and comments.
$ (for I in */src/main/java; do echo `echo $I|sed
's~/src/main/java~~'` `cloc $I|grep Java|grep -v script|awk '{print
$5}'`; done)|sort -k2 -n

Re: Weight of compute apis

[Andrew Bayer <an...@gmail.com>]

I'll confess, part of my love for CloudStack is the demented sprawl of its
API. IT DOES EVERYTHING. It's bonkers and wonderful. Buuuuut yeah, it's
code sprawl and probably could use a bunch of refactoring to abstract away
more. I'll throw that on my todo list. Well, when I'm not bloating it
further by adding tags support everywhere. =)

A.

On Monday, October 20, 2014, Adrian Cole <ad...@gmail.com> wrote:

> I really agree with refocusing apis on "least api needed" including
> leaving fields unmapped until there's a use case.
>
> We used to map all ops and fields, and for smaller apis, this can
> still make sense. However, with bulkier apis, we've found this just
> leads to a lot of debt to constantly refactor, wondering how often
> that effort was well spent. I can't count how many misspelled fields
> or incorrect fallbacks etc I've found in big apis. That code was never
> used, basically.
>
> I'm going to take this "least api" approach with Azure Compute, for
> example. Since no-one has really touched it in years, we can safely
> assume least is best.
>
> I haven't followed up on vCloud thread in the user group, but again,
> unless there's a very solid plan for sustainable support, I highly
> suggest resisting to create anything past least needed even if someone
> on a payroll offers to do the frontwork. It isn't the frontwork that
> hurts years later when the api is completely different or we want to
> change something inside jclouds.
>
> -A
>
> On Mon, Oct 20, 2014 at 4:22 AM, Ignasi Barrera <nacx@apache.org
> <javascript:;>> wrote:
> > It's been a while, but I'll take care of the Abiquo one:
> >
> > There are a lot of legacy classes there, and even APIs that are out of
> > the scope of jclouds (infrastructure management, hypervisor discovery,
> > etc). Also, the latest Abiquo version out there is 3.0 and the
> > provider supports up to Abiquo 2.4, which is quite old now.
> >
> > My plan is to remove everything but the skeleton of the project, and
> > start writing *only* what is needed to implement the ComputeService,
> > to avoid the feature explosion we have right now. This way I hope
> > we'll have a clean provider that we'll be able to promote and
> > maintain.
> >
> > On 12 October 2014 00:23, Adrian Cole <adrian.f.cole@gmail.com
> <javascript:;>> wrote:
> >> Some trivia, which relates to my earlier vcloud-director note.
> >>
> >> According to cloc [1], our heaviest api by far is cloudstack, weighing
> >> in at ~24K lines of java. Even adding openstack keystone (shared by
> >> all openstack) Nova sits at ~15k lines. Interestingly, the old version
> >> of vcloud is ~11k lines; half of the 23k lines of vcloud-director.
> >>
> >> Middle-of-the road folks include gogrid, softlayer, and glesys: all in
> >> the 4k range.
> >>
> >> Simple (json) compute apis are even smaller. For example, joyent and
> >> digitalocean are both <3k lines.
> >>
> >> While only one signal, which discounts cost of maintaining tests or
> >> lack thereof, we can use this 2.5-25k line of code weight to help
> >> decide what to promote.
> >>
> >> I'd suggest we look hard at the apis that fall at or above a 5k
> >> watermark and be careful which we choose to promote, as these likely
> >> will have higher maintenance that other apis even if they are stable
> >> from a version standpoint.
> >>
> >> vcloud-director (incomplete) ~23k
> >> abiquo ~13k
> >> google-compute-engine (inc shared oauth) ~10k
> >> cloudsigma2 ~7k
> >> virtualbox ~6k
> >> fgcp ~5k
> >>
> >> I know that google and cloudsigma2 are currently very healthy, and of
> >> course google is really not optional.
> >>
> >> That said, are there any providers above we are ok to stop working
> >> towards promotion (and instead towards deletion)?
> >> -A
> >>
> >>
> >> [1] Taken from cloc, ignoring tests, whitespace and comments.
> >> $ (for I in */src/main/java; do echo `echo $I|sed
> >> 's~/src/main/java~~'` `cloc $I|grep Java|grep -v script|awk '{print
> >> $5}'`; done)|sort -k2 -n
>

Re: Weight of compute apis

[Adrian Cole <ad...@gmail.com>]

I really agree with refocusing apis on "least api needed" including
leaving fields unmapped until there's a use case.

We used to map all ops and fields, and for smaller apis, this can
still make sense. However, with bulkier apis, we've found this just
leads to a lot of debt to constantly refactor, wondering how often
that effort was well spent. I can't count how many misspelled fields
or incorrect fallbacks etc I've found in big apis. That code was never
used, basically.

I'm going to take this "least api" approach with Azure Compute, for
example. Since no-one has really touched it in years, we can safely
assume least is best.

I haven't followed up on vCloud thread in the user group, but again,
unless there's a very solid plan for sustainable support, I highly
suggest resisting to create anything past least needed even if someone
on a payroll offers to do the frontwork. It isn't the frontwork that
hurts years later when the api is completely different or we want to
change something inside jclouds.

-A

On Mon, Oct 20, 2014 at 4:22 AM, Ignasi Barrera <na...@apache.org> wrote:
> It's been a while, but I'll take care of the Abiquo one:
>
> There are a lot of legacy classes there, and even APIs that are out of
> the scope of jclouds (infrastructure management, hypervisor discovery,
> etc). Also, the latest Abiquo version out there is 3.0 and the
> provider supports up to Abiquo 2.4, which is quite old now.
>
> My plan is to remove everything but the skeleton of the project, and
> start writing *only* what is needed to implement the ComputeService,
> to avoid the feature explosion we have right now. This way I hope
> we'll have a clean provider that we'll be able to promote and
> maintain.
>
> On 12 October 2014 00:23, Adrian Cole <ad...@gmail.com> wrote:
>> Some trivia, which relates to my earlier vcloud-director note.
>>
>> According to cloc [1], our heaviest api by far is cloudstack, weighing
>> in at ~24K lines of java. Even adding openstack keystone (shared by
>> all openstack) Nova sits at ~15k lines. Interestingly, the old version
>> of vcloud is ~11k lines; half of the 23k lines of vcloud-director.
>>
>> Middle-of-the road folks include gogrid, softlayer, and glesys: all in
>> the 4k range.
>>
>> Simple (json) compute apis are even smaller. For example, joyent and
>> digitalocean are both <3k lines.
>>
>> While only one signal, which discounts cost of maintaining tests or
>> lack thereof, we can use this 2.5-25k line of code weight to help
>> decide what to promote.
>>
>> I'd suggest we look hard at the apis that fall at or above a 5k
>> watermark and be careful which we choose to promote, as these likely
>> will have higher maintenance that other apis even if they are stable
>> from a version standpoint.
>>
>> vcloud-director (incomplete) ~23k
>> abiquo ~13k
>> google-compute-engine (inc shared oauth) ~10k
>> cloudsigma2 ~7k
>> virtualbox ~6k
>> fgcp ~5k
>>
>> I know that google and cloudsigma2 are currently very healthy, and of
>> course google is really not optional.
>>
>> That said, are there any providers above we are ok to stop working
>> towards promotion (and instead towards deletion)?
>> -A
>>
>>
>> [1] Taken from cloc, ignoring tests, whitespace and comments.
>> $ (for I in */src/main/java; do echo `echo $I|sed
>> 's~/src/main/java~~'` `cloc $I|grep Java|grep -v script|awk '{print
>> $5}'`; done)|sort -k2 -n

Re: Weight of compute apis

[Ignasi Barrera <na...@apache.org>]

It's been a while, but I'll take care of the Abiquo one:

There are a lot of legacy classes there, and even APIs that are out of
the scope of jclouds (infrastructure management, hypervisor discovery,
etc). Also, the latest Abiquo version out there is 3.0 and the
provider supports up to Abiquo 2.4, which is quite old now.

My plan is to remove everything but the skeleton of the project, and
start writing *only* what is needed to implement the ComputeService,
to avoid the feature explosion we have right now. This way I hope
we'll have a clean provider that we'll be able to promote and
maintain.

On 12 October 2014 00:23, Adrian Cole <ad...@gmail.com> wrote:
> Some trivia, which relates to my earlier vcloud-director note.
>
> According to cloc [1], our heaviest api by far is cloudstack, weighing
> in at ~24K lines of java. Even adding openstack keystone (shared by
> all openstack) Nova sits at ~15k lines. Interestingly, the old version
> of vcloud is ~11k lines; half of the 23k lines of vcloud-director.
>
> Middle-of-the road folks include gogrid, softlayer, and glesys: all in
> the 4k range.
>
> Simple (json) compute apis are even smaller. For example, joyent and
> digitalocean are both <3k lines.
>
> While only one signal, which discounts cost of maintaining tests or
> lack thereof, we can use this 2.5-25k line of code weight to help
> decide what to promote.
>
> I'd suggest we look hard at the apis that fall at or above a 5k
> watermark and be careful which we choose to promote, as these likely
> will have higher maintenance that other apis even if they are stable
> from a version standpoint.
>
> vcloud-director (incomplete) ~23k
> abiquo ~13k
> google-compute-engine (inc shared oauth) ~10k
> cloudsigma2 ~7k
> virtualbox ~6k
> fgcp ~5k
>
> I know that google and cloudsigma2 are currently very healthy, and of
> course google is really not optional.
>
> That said, are there any providers above we are ok to stop working
> towards promotion (and instead towards deletion)?
> -A
>
>
> [1] Taken from cloc, ignoring tests, whitespace and comments.
> $ (for I in */src/main/java; do echo `echo $I|sed
> 's~/src/main/java~~'` `cloc $I|grep Java|grep -v script|awk '{print
> $5}'`; done)|sort -k2 -n