Where should admin tooling for jclouds go?

[Andrew Phillips <ap...@qrmedia.com>]

With #239 [1], we'll hopefully be able to get an interesting and  
useful benchmarking tool from Maginatics out to the wider jclouds  
audience. Some debate going on in the issue, though, about where the  
right place for this and related administrative tools and utilities  
should be.

I personally would like to keep these out of jclouds itself, since I  
see things in there more as libraries than standalone tools or  
distributions (we haven't used the "distribution" profile [2] for ages  
now, and I think it should probably be removed).

Curious to hear what others think..?

ap

[1] https://github.com/jclouds/jclouds/pull/239
[2] https://github.com/jclouds/jclouds/blob/master/pom.xml#L121

Re: Where should admin tooling for jclouds go?

[Andrew Gaul <ga...@apache.org>]

On Wed, Dec 18, 2013 at 05:15:59PM -0500, David Nalley wrote:
> On Wed, Dec 18, 2013 at 2:18 PM, Andrew Gaul <ga...@apache.org> wrote:
> > To gives some scope, Maginatics hopes to release a series of tools under
> > the jclouds umbrella:
> 
> Thoughts on whether these need to go through IP Clearance? (and if so,
> lets please do them all at once rather than one at a time)

I doubt it; these tools represent a few thousand lines of code over a
handful of files.  I consider them logically part of jclouds; Maginatics
did not immediately contribute these upstream due to questions over
where to host the tools.

-- 
Andrew Gaul
http://gaul.org/

Re: Where should admin tooling for jclouds go?

[David Nalley <da...@gnsa.us>]

On Wed, Dec 18, 2013 at 2:18 PM, Andrew Gaul <ga...@apache.org> wrote:
> On Wed, Dec 18, 2013 at 07:59:48PM +0100, Andrew Phillips wrote:
>> With #239 [1], we'll hopefully be able to get an interesting and useful
>> benchmarking tool from Maginatics out to the wider jclouds audience. Some
>> debate going on in the issue, though, about where the right place for this
>> and related administrative tools and utilities should be.
>>
>> I personally would like to keep these out of jclouds itself, since I see
>> things in there more as libraries than standalone tools or distributions (we
>> haven't used the "distribution" profile [2] for ages now, and I think it
>> should probably be removed).
>>
>> Curious to hear what others think..?
>
> To gives some scope, Maginatics hopes to release a series of tools under
> the jclouds umbrella:
>
> BlobStoreBench will be the first tool, which gathers bandwidth and
> latency statistics.  This allows exploring different naming schemes for
> blobs, e.g., AWS-S3 prefers naming blobs with a random prefix, Atmos
> prefers a limited number of blobs per directory, and Swift prefers
> distributing blobs between containers.
>
> BlobStoreValidator diagnoses blobstore compatibility, usually used with
> S3- or Swift-compatible private blobstores which have varying
> authentication, service paths, and quirks like MD5 support.
>
> BlobStoreCli replaces some uses of the existing karaf-based jclouds-cli
> using really-executable-jar.  This offers a more intuitive user
> experience and slimmer binary size.
>
> Returning to your question, I prefer to put these in the main repository
> since they impose little testing and packaging overhead.  Further the
> explosion of repositories makes developing and releasing jclouds painful
> as we encounter regressions and inconsistencies when making changes to a
> single repository.  We should endeavor to bring more code into the main
> repository, e.g., jclouds-chef.  Finally hosting these in the main
> repository makes these tools more discoverable by our users.
>


Thoughts on whether these need to go through IP Clearance? (and if so,
lets please do them all at once rather than one at a time)

--David

Re: Where should admin tooling for jclouds go?

[Andrew Gaul <ga...@apache.org>]

On Wed, Dec 18, 2013 at 08:53:26PM +0100, Andrew Phillips wrote:
> >BlobStoreCli replaces some uses of the existing karaf-based jclouds-cli
> >using really-executable-jar.  This offers a more intuitive user
> >experience and slimmer binary size.
> 
> This is probably a different discussion, but it sounds like the feeling is
> that Karaf is not sufficiently improvable to make changes to that
> worthwhile?
> 
> Multiple tools that try to do similar things sounds like something to
> approach with caution? Also from the perspective of not confusing users with
> multiple ways to do similar things.
> 
> To be clear, though: I'd obviously have nothing against a new BlobStoreCli
> totally unrelated to karaf or cli if those just can't do what we/our users
> need.

jclouds-cli use of karaf is a big hammer to solve the simple problem of
translating command-line tokens into jclouds methods.  Having spent
significant time improving jclouds-cli, I prefer not continue down this
path.  I believe the BlobStoreCli approach gives a more intuitive user
experience, better configuration and logging, and has better
performance/lower overhead.

-- 
Andrew Gaul
http://gaul.org/

Re: Where should admin tooling for jclouds go?

[Andrew Phillips <ap...@qrmedia.com>]

> BlobStoreCli replaces some uses of the existing karaf-based jclouds-cli
> using really-executable-jar.  This offers a more intuitive user
> experience and slimmer binary size.

This is probably a different discussion, but it sounds like the  
feeling is that Karaf is not sufficiently improvable to make changes  
to that worthwhile?

Multiple tools that try to do similar things sounds like something to  
approach with caution? Also from the perspective of not confusing  
users with multiple ways to do similar things.

To be clear, though: I'd obviously have nothing against a new  
BlobStoreCli totally unrelated to karaf or cli if those just can't do  
what we/our users need.

ap

Re: Where should admin tooling for jclouds go?

[Andrew Gaul <ga...@apache.org>]

On Wed, Dec 18, 2013 at 07:59:48PM +0100, Andrew Phillips wrote:
> With #239 [1], we'll hopefully be able to get an interesting and useful
> benchmarking tool from Maginatics out to the wider jclouds audience. Some
> debate going on in the issue, though, about where the right place for this
> and related administrative tools and utilities should be.
> 
> I personally would like to keep these out of jclouds itself, since I see
> things in there more as libraries than standalone tools or distributions (we
> haven't used the "distribution" profile [2] for ages now, and I think it
> should probably be removed).
> 
> Curious to hear what others think..?

To gives some scope, Maginatics hopes to release a series of tools under
the jclouds umbrella:

BlobStoreBench will be the first tool, which gathers bandwidth and
latency statistics.  This allows exploring different naming schemes for
blobs, e.g., AWS-S3 prefers naming blobs with a random prefix, Atmos
prefers a limited number of blobs per directory, and Swift prefers
distributing blobs between containers.

BlobStoreValidator diagnoses blobstore compatibility, usually used with
S3- or Swift-compatible private blobstores which have varying
authentication, service paths, and quirks like MD5 support.

BlobStoreCli replaces some uses of the existing karaf-based jclouds-cli
using really-executable-jar.  This offers a more intuitive user
experience and slimmer binary size.

Returning to your question, I prefer to put these in the main repository
since they impose little testing and packaging overhead.  Further the
explosion of repositories makes developing and releasing jclouds painful
as we encounter regressions and inconsistencies when making changes to a
single repository.  We should endeavor to bring more code into the main
repository, e.g., jclouds-chef.  Finally hosting these in the main
repository makes these tools more discoverable by our users.

-- 
Andrew Gaul
http://gaul.org/