Boxgrinder is not actively developed anymore

[Bruno Mahé <bm...@apache.org>]

Hi all,

I was told that boxgrinder is not actively developed anymore. Given the 
speed at which distributions and the ruby language evolve, it may even 
break and not work anymore sooner than later.

So I would like to start a conversation to establish a plan for phasing 
out boxgrinder and to replace it with another solution.
 From my understanding, people are directed to use imagefactory 
(http://imgfac.org/).
imagefactory seems to have less features than boxgrinder and to use xml 
files instead of yaml. It would fit the requirements, except that for 
now, the documentation is pretty sparse and/or outdated. I also cannot 
make it work after following the doc and poking around for 15min. So 
until basic image creation works for me, I cannot recommend it.
I will try to spend some more time on it though.

Would anyone has any other tool they would recommend?


But regardless of the tool we pick, we don't have to stop using 
boxgrinder right away and we don't even have to delete our boxgrinder 
recipes either.
My goal would be to establish a consensus here so we can all put our 
efforts in common.

To me, the requirements for such tool are:
* Ability to build images for desktop virtualization
* Ability to push images to cloud providers
* Make it easy to customize images and derive new ones
* No manual steps
* Libre software

Thanks,
Bruno

Re: Boxgrinder is not actively developed anymore

[Bruno Mahé <bm...@apache.org>]

On 03/16/2013 08:04 PM, Sean Mackrory wrote:
> I haven't used VMBuilder or imgfac yet, but I will check them out so I can
> have a more intelligent 2c to share. I agree with Bruno's requirements,
> although I think I can help get us around these two:
>
> * Make it easy to customize images and derive new ones
> * No manual steps
>
> I recently hacked my way around Boxgrinder to add a few features, tweaks
> and fixes, and having done that, I'd be happy to apply the same ideas to
> whatever tool we end up going with (see BIGTOP-871). I suspect that no tool
> will be perfect but that the basic principles are similar enough that we
> could define a configuration and have it interpreted by various "back-ends"
> (which is essentially what I'm doing already, with boxgrinder being the
> only "back-end" right now).
>
>
> On Sat, Mar 16, 2013 at 7:26 PM, Konstantin Boudnik <co...@apache.org> wrote:
>
>> I never been a fun of boxgrinder mostly because of its Ruby'ness and
>> fedora/centos centism - distros that I use for pretty much nothing at all -
>> and I am glad to see it go.
>>
>> That said: why jump on some colledge project with unknown outcome and not
>> to
>> use something mature? How about vmbuilder? It see to do the job almost
>> perfectly, has been around for years and has a huge community.
>>
>> Cos
>>
>> On Sat, Mar 16, 2013 at 04:04PM, Bruno MahИ wrote:
>>> Hi all,
>>>
>>> I was told that boxgrinder is not actively developed anymore. Given
>>> the speed at which distributions and the ruby language evolve, it
>>> may even break and not work anymore sooner than later.
>>>
>>> So I would like to start a conversation to establish a plan for
>>> phasing out boxgrinder and to replace it with another solution.
>>>  From my understanding, people are directed to use imagefactory
>>> (http://imgfac.org/).
>>> imagefactory seems to have less features than boxgrinder and to use
>>> xml files instead of yaml. It would fit the requirements, except
>>> that for now, the documentation is pretty sparse and/or outdated. I
>>> also cannot make it work after following the doc and poking around
>>> for 15min. So until basic image creation works for me, I cannot
>>> recommend it.
>>> I will try to spend some more time on it though.
>>>
>>> Would anyone has any other tool they would recommend?
>>>
>>>
>>> But regardless of the tool we pick, we don't have to stop using
>>> boxgrinder right away and we don't even have to delete our
>>> boxgrinder recipes either.
>>> My goal would be to establish a consensus here so we can all put our
>>> efforts in common.
>>>
>>> To me, the requirements for such tool are:
>>> * Ability to build images for desktop virtualization
>>> * Ability to push images to cloud providers
>>> * Make it easy to customize images and derive new ones
>>> * No manual steps
>>> * Libre software
>>>
>>> Thanks,
>>> Bruno
>>
>


Why don't you describe your ideas/code/efforts? They sound great and 
would be delighted to learn more about them.
The conversation would benefit a lot from it.

Thanks,
Bruno

Re: Boxgrinder is not actively developed anymore

[Sean Mackrory <ma...@gmail.com>]

I haven't used VMBuilder or imgfac yet, but I will check them out so I can
have a more intelligent 2c to share. I agree with Bruno's requirements,
although I think I can help get us around these two:

* Make it easy to customize images and derive new ones
* No manual steps

I recently hacked my way around Boxgrinder to add a few features, tweaks
and fixes, and having done that, I'd be happy to apply the same ideas to
whatever tool we end up going with (see BIGTOP-871). I suspect that no tool
will be perfect but that the basic principles are similar enough that we
could define a configuration and have it interpreted by various "back-ends"
(which is essentially what I'm doing already, with boxgrinder being the
only "back-end" right now).


On Sat, Mar 16, 2013 at 7:26 PM, Konstantin Boudnik <co...@apache.org> wrote:

> I never been a fun of boxgrinder mostly because of its Ruby'ness and
> fedora/centos centism - distros that I use for pretty much nothing at all -
> and I am glad to see it go.
>
> That said: why jump on some colledge project with unknown outcome and not
> to
> use something mature? How about vmbuilder? It see to do the job almost
> perfectly, has been around for years and has a huge community.
>
> Cos
>
> On Sat, Mar 16, 2013 at 04:04PM, Bruno MahИ wrote:
> > Hi all,
> >
> > I was told that boxgrinder is not actively developed anymore. Given
> > the speed at which distributions and the ruby language evolve, it
> > may even break and not work anymore sooner than later.
> >
> > So I would like to start a conversation to establish a plan for
> > phasing out boxgrinder and to replace it with another solution.
> > From my understanding, people are directed to use imagefactory
> > (http://imgfac.org/).
> > imagefactory seems to have less features than boxgrinder and to use
> > xml files instead of yaml. It would fit the requirements, except
> > that for now, the documentation is pretty sparse and/or outdated. I
> > also cannot make it work after following the doc and poking around
> > for 15min. So until basic image creation works for me, I cannot
> > recommend it.
> > I will try to spend some more time on it though.
> >
> > Would anyone has any other tool they would recommend?
> >
> >
> > But regardless of the tool we pick, we don't have to stop using
> > boxgrinder right away and we don't even have to delete our
> > boxgrinder recipes either.
> > My goal would be to establish a consensus here so we can all put our
> > efforts in common.
> >
> > To me, the requirements for such tool are:
> > * Ability to build images for desktop virtualization
> > * Ability to push images to cloud providers
> > * Make it easy to customize images and derive new ones
> > * No manual steps
> > * Libre software
> >
> > Thanks,
> > Bruno
>

Re: Boxgrinder is not actively developed anymore

[Konstantin Boudnik <co...@apache.org>]

Actually, I think you have misread me - I never did such a thing as trashing. I
only stated that I am not using Fedora off-springs. Here's the exact quote:

>>> mostly because of its Ruby'ness and
>>> fedora/centos centism - distros that I use for pretty much nothing at all

Hope it clear the air!
  Cos

On Sun, Mar 17, 2013 at 01:33PM, Bruno Mahé wrote:
> See reply inline.
> 
> On 03/17/2013 11:16 AM, Konstantin Boudnik wrote:
> >Oh Bruno - I am sorry if it felt too personal on you. Please accept my sincere
> >apologies. I will make an effort not to talk about any R/C or F distros in the
> >future!
> >
> 
> I believe you missed my point.
> My point is: if not welcoming, we should have at least a civil and
> courteous community/mailing-list.
> As of my feelings, don't worry, they were not touched :)


> But are people from red hat/fedora/openstack supposed to feel
> welcomed after you just trashed them?
> 
> >However, my comment was made about this line from your original email:
> >>xml files instead of yaml. It would fit the requirements, except that
> >>for now, the documentation is pretty sparse and/or outdated. I also
> >
> >which totally fills like a colledge kids project. If I am mistaken - I take my
> >non-R hat off to these guys.
> >
> 
> Let's wait to have more information to make such judgment.
> 
> 
> >As for
> >>I am not convinced by vmbuilder. Its only community is ubuntu.
> >>As expected it only handles ubuntu/debian distros, which is fine.
> >
> >a simple http://is.gd/AmLkQe shows that not only .deb distros are supported.
> >Pretty much all modern virtualization platforms are supported as well.
> >
> 
> This is pointing to a lonely github repository not updated in a year :/
> 
> >But even this is besides the point. Perhaps, we won't found one-fits-all too
> >for VM creation - it's not a big deal as we can use two or more if needed. I
> >don't see this is a huge problem.
> >
> 
> Agreed. It is not an either or type of choice. But if we can avoid
> duplication of effort, it would be great.
> And we won't know if such one-fits-all exist if we don't look for it :)
> 
> >As for
> >>interesting. I would hate to look into moderation of mailing lists
> >>at the Apache Foundation.
> >I beg you not to do this, because the discussions like this are so much fun!
> >
> 
> +1
> But still, the first recommendation for mailing list on the Apache
> Foundation website is "Respectful and considerate communities are
> one of the pillars of the Apache way. Please aim to provide
> constructive comments and do not denigrate others."
> 
> 
> Thanks,
> Bruno

Re: Boxgrinder is not actively developed anymore

[Sean Mackrory <ma...@gmail.com>]

>> Why don't you describe your ideas/code/efforts? They sound great and
would be delighted to learn more about them. The conversation would benefit
a lot from it.

Certainly! So I created some JSON files that roughly follow the same
structure as Boxgrinder YAML. Each one represents a specific set of
functionality, and resides in a directory with any files it will pull into
the appliance. They can list other JSON files as dependencies, and the
python script will merge dictionaries and concatenate arrays (with
descendants overriding ancestors), then write it out to a single YAML file
and a directory of the other files. It's pretty hacky at this point (having
just been a side-project done as-needed), but by generating the manifest
gives a lot of flexibility to add little hacks here and there, and it was
easier to manage multiple appliances than with Boxgrinder's inheritance
alone (I've forgotten the specific shortcomings, but it wasn't sufficient
for my purposes). I've added code to generate the .mozilla/firefox
directory with a list of bookmarks, too.

|---grind.py
|---hdfs
|   |---appliance.json
|   |---hdfs-site.xml
|
|---hbase (depends on HDFS)
    |---appliance.json
    |---hbase-site.xml

Going forward, I'd like to make it more modular to support different
back-ends (perhaps kickstarter, for Live CDs, etc), and perhaps break down
the JSON files / directories further so that you can define the individual
Bigtop components and combine them however you want into. I'm putting
together a "bigtop" example for my current mechanism and will post it.


On Sun, Mar 17, 2013 at 1:33 PM, Bruno Mahé <bm...@apache.org> wrote:

> See reply inline.
>
>
> On 03/17/2013 11:16 AM, Konstantin Boudnik wrote:
>
>> Oh Bruno - I am sorry if it felt too personal on you. Please accept my
>> sincere
>> apologies. I will make an effort not to talk about any R/C or F distros
>> in the
>> future!
>>
>>
> I believe you missed my point.
> My point is: if not welcoming, we should have at least a civil and
> courteous community/mailing-list.
> As of my feelings, don't worry, they were not touched :)
>
> But are people from red hat/fedora/openstack supposed to feel welcomed
> after you just trashed them?
>
>
>
>  However, my comment was made about this line from your original email:
>>
>>> xml files instead of yaml. It would fit the requirements, except that
>>> for now, the documentation is pretty sparse and/or outdated. I also
>>>
>>
>> which totally fills like a colledge kids project. If I am mistaken - I
>> take my
>> non-R hat off to these guys.
>>
>>
> Let's wait to have more information to make such judgment.
>
>
>
>  As for
>>
>>> I am not convinced by vmbuilder. Its only community is ubuntu.
>>> As expected it only handles ubuntu/debian distros, which is fine.
>>>
>>
>> a simple http://is.gd/AmLkQe shows that not only .deb distros are
>> supported.
>> Pretty much all modern virtualization platforms are supported as well.
>>
>>
> This is pointing to a lonely github repository not updated in a year :/
>
>
>  But even this is besides the point. Perhaps, we won't found one-fits-all
>> too
>> for VM creation - it's not a big deal as we can use two or more if
>> needed. I
>> don't see this is a huge problem.
>>
>>
> Agreed. It is not an either or type of choice. But if we can avoid
> duplication of effort, it would be great.
> And we won't know if such one-fits-all exist if we don't look for it :)
>
>
>  As for
>>
>>> interesting. I would hate to look into moderation of mailing lists
>>> at the Apache Foundation.
>>>
>> I beg you not to do this, because the discussions like this are so much
>> fun!
>>
>>
> +1
> But still, the first recommendation for mailing list on the Apache
> Foundation website is "Respectful and considerate communities are one of
> the pillars of the Apache way. Please aim to provide constructive comments
> and do not denigrate others."
>
>
> Thanks,
> Bruno
>

Re: Boxgrinder is not actively developed anymore

[Bruno Mahé <bm...@apache.org>]

See reply inline.

On 03/17/2013 11:16 AM, Konstantin Boudnik wrote:
> Oh Bruno - I am sorry if it felt too personal on you. Please accept my sincere
> apologies. I will make an effort not to talk about any R/C or F distros in the
> future!
>

I believe you missed my point.
My point is: if not welcoming, we should have at least a civil and 
courteous community/mailing-list.
As of my feelings, don't worry, they were not touched :)

But are people from red hat/fedora/openstack supposed to feel welcomed 
after you just trashed them?


> However, my comment was made about this line from your original email:
>> xml files instead of yaml. It would fit the requirements, except that
>> for now, the documentation is pretty sparse and/or outdated. I also
>
> which totally fills like a colledge kids project. If I am mistaken - I take my
> non-R hat off to these guys.
>

Let's wait to have more information to make such judgment.


> As for
>> I am not convinced by vmbuilder. Its only community is ubuntu.
>> As expected it only handles ubuntu/debian distros, which is fine.
>
> a simple http://is.gd/AmLkQe shows that not only .deb distros are supported.
> Pretty much all modern virtualization platforms are supported as well.
>

This is pointing to a lonely github repository not updated in a year :/

> But even this is besides the point. Perhaps, we won't found one-fits-all too
> for VM creation - it's not a big deal as we can use two or more if needed. I
> don't see this is a huge problem.
>

Agreed. It is not an either or type of choice. But if we can avoid 
duplication of effort, it would be great.
And we won't know if such one-fits-all exist if we don't look for it :)

> As for
>> interesting. I would hate to look into moderation of mailing lists
>> at the Apache Foundation.
> I beg you not to do this, because the discussions like this are so much fun!
>

+1
But still, the first recommendation for mailing list on the Apache 
Foundation website is "Respectful and considerate communities are one of 
the pillars of the Apache way. Please aim to provide constructive 
comments and do not denigrate others."


Thanks,
Bruno

Re: Boxgrinder is not actively developed anymore

[Konstantin Boudnik <co...@apache.org>]

Oh Bruno - I am sorry if it felt too personal on you. Please accept my sincere
apologies. I will make an effort not to talk about any R/C or F distros in the
future!

However, my comment was made about this line from your original email:
> xml files instead of yaml. It would fit the requirements, except that
> for now, the documentation is pretty sparse and/or outdated. I also

which totally fills like a colledge kids project. If I am mistaken - I take my
non-R hat off to these guys.

As for
> I am not convinced by vmbuilder. Its only community is ubuntu.
> As expected it only handles ubuntu/debian distros, which is fine.

a simple http://is.gd/AmLkQe shows that not only .deb distros are supported.
Pretty much all modern virtualization platforms are supported as well.

But even this is besides the point. Perhaps, we won't found one-fits-all too
for VM creation - it's not a big deal as we can use two or more if needed. I
don't see this is a huge problem. 

As for
> interesting. I would hate to look into moderation of mailing lists
> at the Apache Foundation.
I beg you not to do this, because the discussions like this are so much fun! 

Thank you!
  Cos

On Sat, Mar 16, 2013 at 09:34PM, Bruno Mahé wrote:
> See reply inline.
> 
> On 03/16/2013 07:26 PM, Konstantin Boudnik wrote:
> >I never been a fun of boxgrinder mostly because of its Ruby'ness and
> >fedora/centos centism - distros that I use for pretty much nothing at all -
> >and I am glad to see it go.
> >
> 
> Please remain civil and courteous. Spewing nonsense baseless trolls
> is not welcome. So if you feel to send such email in the future,
> just don't even bother.
> 
> 
> >That said: why jump on some colledge project with unknown outcome and not to
> >use something mature?
> 
> Good to know that people at red hat/fedora are not serious people
> doing serious business.
> Again, insulting and trashing people/projects is not fun nor
> interesting. I would hate to look into moderation of mailing lists
> at the Apache Foundation.
> 
> This may surprise you, but imagefactory, which is based on oz, do
> support distributions outside of the red hat ecosystem.
> See https://github.com/clalancette/oz/wiki
> 
> 
> >How about vmbuilder? It see to do the job almost
> >perfectly, has been around for years and has a huge community.
> >
> 
> I am not convinced by vmbuilder. Its only community is ubuntu.
> As expected it only handles ubuntu/debian distros, which is fine.
> Unfortunately, the way you configure the vm does not seem easy nor
> intuitive or convenient. Boxgrinder or imagefactory do much better
> in that regard.
> 
> From
> https://help.ubuntu.com/12.04/serverguide/jeos-and-vmbuilder.html it
> does not even seem to have any config files. Although a much older
> version of ubuntu seems to have that ability (
> https://help.ubuntu.com/community/JeOSVMBuilder [see last few
> paragraphs] )
> 
> It also seems to only support ec2 as cloud providers. All other
> tools support more than that.
> 
> But above all, we tried vmbuilder before using boxgrinder. And the
> resulting configuration and scripts were a mess to maintain.
> So I am not interested in going back to that.
> 
> Thanks,
> Bruno

Re: Boxgrinder is not actively developed anymore

[Bruno Mahé <bm...@apache.org>]

See reply inline.

On 03/16/2013 07:26 PM, Konstantin Boudnik wrote:
> I never been a fun of boxgrinder mostly because of its Ruby'ness and
> fedora/centos centism - distros that I use for pretty much nothing at all -
> and I am glad to see it go.
>

Please remain civil and courteous. Spewing nonsense baseless trolls is 
not welcome. So if you feel to send such email in the future, just don't 
even bother.


> That said: why jump on some colledge project with unknown outcome and not to
> use something mature?

Good to know that people at red hat/fedora are not serious people doing 
serious business.
Again, insulting and trashing people/projects is not fun nor 
interesting. I would hate to look into moderation of mailing lists at 
the Apache Foundation.

This may surprise you, but imagefactory, which is based on oz, do 
support distributions outside of the red hat ecosystem.
See https://github.com/clalancette/oz/wiki


> How about vmbuilder? It see to do the job almost
> perfectly, has been around for years and has a huge community.
>

I am not convinced by vmbuilder. Its only community is ubuntu.
As expected it only handles ubuntu/debian distros, which is fine.
Unfortunately, the way you configure the vm does not seem easy nor 
intuitive or convenient. Boxgrinder or imagefactory do much better in 
that regard.

 From https://help.ubuntu.com/12.04/serverguide/jeos-and-vmbuilder.html 
it does not even seem to have any config files. Although a much older 
version of ubuntu seems to have that ability ( 
https://help.ubuntu.com/community/JeOSVMBuilder [see last few paragraphs] )

It also seems to only support ec2 as cloud providers. All other tools 
support more than that.

But above all, we tried vmbuilder before using boxgrinder. And the 
resulting configuration and scripts were a mess to maintain.
So I am not interested in going back to that.

Thanks,
Bruno

Re: Boxgrinder is not actively developed anymore

[Konstantin Boudnik <co...@apache.org>]

I never been a fun of boxgrinder mostly because of its Ruby'ness and
fedora/centos centism - distros that I use for pretty much nothing at all -
and I am glad to see it go.

That said: why jump on some colledge project with unknown outcome and not to
use something mature? How about vmbuilder? It see to do the job almost
perfectly, has been around for years and has a huge community.

Cos

On Sat, Mar 16, 2013 at 04:04PM, Bruno MahИ wrote:
> Hi all,
> 
> I was told that boxgrinder is not actively developed anymore. Given
> the speed at which distributions and the ruby language evolve, it
> may even break and not work anymore sooner than later.
> 
> So I would like to start a conversation to establish a plan for
> phasing out boxgrinder and to replace it with another solution.
> From my understanding, people are directed to use imagefactory
> (http://imgfac.org/).
> imagefactory seems to have less features than boxgrinder and to use
> xml files instead of yaml. It would fit the requirements, except
> that for now, the documentation is pretty sparse and/or outdated. I
> also cannot make it work after following the doc and poking around
> for 15min. So until basic image creation works for me, I cannot
> recommend it.
> I will try to spend some more time on it though.
> 
> Would anyone has any other tool they would recommend?
> 
> 
> But regardless of the tool we pick, we don't have to stop using
> boxgrinder right away and we don't even have to delete our
> boxgrinder recipes either.
> My goal would be to establish a consensus here so we can all put our
> efforts in common.
> 
> To me, the requirements for such tool are:
> * Ability to build images for desktop virtualization
> * Ability to push images to cloud providers
> * Make it easy to customize images and derive new ones
> * No manual steps
> * Libre software
> 
> Thanks,
> Bruno