You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by Eric Baldeschwieler <er...@hortonworks.com> on 2012/05/01 00:30:57 UTC
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
A question...
The stated goal of BigTop is to package "official releases of the Apache Hadoop-related projects". I believe stated policy is to require that only official releases from Apache projects can be packaged, no patches or development branches. Do I have that right? Has Cloudera contributed Hue to Apache or are you suggesting a change to the charter of BigTop to package vendor hosted projects as well as Apache Projects?
Thanks,
E14
PS - Just looked up the incubating proposal:
http://mail-archives.apache.org/mod_mbox/incubator-general/201106.mbox/%3CBANLkTimriyVS5G5MAKLQviNAUZ9H6S5hWw@mail.gmail.com%3E
> Build, packaging and integration test code that depends upon official
> releases of the Apache Hadoop-related projects (HDFS, MapReduce,
> HBase, Hive, Pig, ZooKeeper, etc...) will be developed and released by
> this project. As bugs and other issues are found we expect these to be
> fixed upstream.
----
> From: Roman Shaposhnik <rv...@apache.org>
> Date: Fri, Apr 27, 2012 at 5:40 PM
> Subject: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
> To: bigtop-dev@incubator.apache.org
>
>
> I'd like to put the stake in the ground and propose
> that we support the following BOM:
> * Hadoop 2.0.0-alpha
> * Zookeeper 3.4.3
> * HBase 0.92.1
> * Pig 0.10.0
> * Hive 0.9.0
> * Oozie 3.2
> * Sqoop 1.4.1
> * Whirr 0.7.1
> * Mahout 0.6.1
> * Flume 1.1.0
> * Hue 2.0.0
>
> The only components on the list that have a chance of
> delaying the release are Oozie and Mahout. I will try
> to deal with those communities separately and figure
> out what can be done. It would be really nice to have
> Hama and HCatalog but I haven't seen any major
> updates on those projects. Please let me know if
> I'm missing something.
>
> As far as the supported platforms go I would like
> to propose the following:
> * Fedora 16
> * Ubuntu 10.04 (LTS)
> * Ubuntu 12.04 (LTS)
> * CentOS (RHEL) 5.7
> * CentOS (RHEL) 6.0
> * SLES 11
>
> Thanks,
> Roman.
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Konstantin Boudnik <co...@apache.org>.
On Fri, May 04, 2012 at 11:02AM, Patrick Hunt wrote:
> On Fri, May 4, 2012 at 9:47 AM, Owen O'Malley <om...@apache.org> wrote:
> > On Fri, May 4, 2012 at 8:58 AM, Patrick Hunt <ph...@apache.org> wrote:
> >> It's not the job of the incubator to create new rules, but rather to
> >> help podlings to graduation while following existing Apache
> >> guidelines.
> >
> > We aren't making new rules. We are trying to help the Bigtop project
> > understand the rules about not releasing non-Apache software. There is
> > a huge difference between depending on an artifact from another
> > project and building and distributing non-Apache rpms in the project's
> > /dist directory.
>
> They are not releasing non-Apache software. They are not forking an
> existing project. Bigtop's release artifact will contain packaging
> code which allows users to compile packages (deb, rpm, etc...) for
> this ASL licensed component, not the source/binaries of the component
> itself.
Thank you Patrick to bringing this back again! It seems that this point keeps
dropping on the floor all the time. BigTop releases are merely a source code
for tools to produce and validate the integrity of software stacks. Let's keep
in mind at the next round of deliberations.
Packages are secondary and can be stored someplace else, because really anyone
can produce them with ease using BigTop. If someone dislike component-A for
one reason or another - it is easy to remote it from a particular release's
BOM file.
Cos
> >> It's very clear from
> >> http://www.apache.org/legal/resolved.html that what has been proposed
> >> is acceptable under existing Apache rules.
> >
> > Can you find a single instance other than the disagreement between
> > Apache Lucene and Apache Commons where one project is distributing
> > another project's rpms? Are there any other non-Apache rpms in /dist?
> > Clearly the answer is a resounding NO. It would be a huge violation of
> > the trust the incubator is putting in me as a mentor if I didn't block
> > Bigtop's plan to do so.
>
> If the component made an objection to being included in Bigtop then I
> could see an argument to be made, that's not the case here. The
> opposite is true from what I've seen -- people want their software to
> be included so that users can more easily consume it. That's why they
> released their software under a less restrictive license in the first
> place.
>
> EOD existing Apache rules/license make no such distinction. "Works
> under the following licenses may be included within Apache products"
> (includes ASL).
>
> Patrick
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Konstantin Boudnik <co...@apache.org>.
On Fri, May 04, 2012 at 11:02AM, Patrick Hunt wrote:
> On Fri, May 4, 2012 at 9:47 AM, Owen O'Malley <om...@apache.org> wrote:
> > On Fri, May 4, 2012 at 8:58 AM, Patrick Hunt <ph...@apache.org> wrote:
> >> It's not the job of the incubator to create new rules, but rather to
> >> help podlings to graduation while following existing Apache
> >> guidelines.
> >
> > We aren't making new rules. We are trying to help the Bigtop project
> > understand the rules about not releasing non-Apache software. There is
> > a huge difference between depending on an artifact from another
> > project and building and distributing non-Apache rpms in the project's
> > /dist directory.
>
> They are not releasing non-Apache software. They are not forking an
> existing project. Bigtop's release artifact will contain packaging
> code which allows users to compile packages (deb, rpm, etc...) for
> this ASL licensed component, not the source/binaries of the component
> itself.
Thank you Patrick to bringing this back again! It seems that this point keeps
dropping on the floor all the time. BigTop releases are merely a source code
for tools to produce and validate the integrity of software stacks. Let's keep
in mind at the next round of deliberations.
Packages are secondary and can be stored someplace else, because really anyone
can produce them with ease using BigTop. If someone dislike component-A for
one reason or another - it is easy to remote it from a particular release's
BOM file.
Cos
> >> It's very clear from
> >> http://www.apache.org/legal/resolved.html that what has been proposed
> >> is acceptable under existing Apache rules.
> >
> > Can you find a single instance other than the disagreement between
> > Apache Lucene and Apache Commons where one project is distributing
> > another project's rpms? Are there any other non-Apache rpms in /dist?
> > Clearly the answer is a resounding NO. It would be a huge violation of
> > the trust the incubator is putting in me as a mentor if I didn't block
> > Bigtop's plan to do so.
>
> If the component made an objection to being included in Bigtop then I
> could see an argument to be made, that's not the case here. The
> opposite is true from what I've seen -- people want their software to
> be included so that users can more easily consume it. That's why they
> released their software under a less restrictive license in the first
> place.
>
> EOD existing Apache rules/license make no such distinction. "Works
> under the following licenses may be included within Apache products"
> (includes ASL).
>
> Patrick
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Patrick Hunt <ph...@apache.org>.
On Fri, May 4, 2012 at 11:54 AM, Greg Stein <gs...@gmail.com> wrote:
> On May 4, 2012 2:03 PM, "Patrick Hunt" <ph...@apache.org> wrote:
>>...
>> EOD existing Apache rules/license make no such distinction. "Works
>> under the following licenses may be included within Apache products"
>> (includes ASL).
>
> Can people please stop using "ASL" or "APL"? No such thing. It is the
> Apache License. AL for short, or even ALv2.
Sorry for the incorrect tla usage. Will do.
Patrick
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Mark Struberg <st...@yahoo.de>.
spreading the world since years, and it's always funny what new combinations come up:
https://twitter.com/#!/struberg/status/197003905027149824
:)
LieGrue,
strub
>________________________________
> From: Greg Stein <gs...@gmail.com>
>To: general@incubator.apache.org
>Cc: bigtop-dev@incubator.apache.org
>Sent: Friday, May 4, 2012 8:54 PM
>Subject: Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
>
>On May 4, 2012 2:03 PM, "Patrick Hunt" <ph...@apache.org> wrote:
>>...
>> EOD existing Apache rules/license make no such distinction. "Works
>> under the following licenses may be included within Apache products"
>> (includes ASL).
>
>Can people please stop using "ASL" or "APL"? No such thing. It is the
>Apache License. AL for short, or even ALv2.
>
>Thanks,
>-g
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Patrick Hunt <ph...@apache.org>.
On Fri, May 4, 2012 at 11:54 AM, Greg Stein <gs...@gmail.com> wrote:
> On May 4, 2012 2:03 PM, "Patrick Hunt" <ph...@apache.org> wrote:
>>...
>> EOD existing Apache rules/license make no such distinction. "Works
>> under the following licenses may be included within Apache products"
>> (includes ASL).
>
> Can people please stop using "ASL" or "APL"? No such thing. It is the
> Apache License. AL for short, or even ALv2.
Sorry for the incorrect tla usage. Will do.
Patrick
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Greg Stein <gs...@gmail.com>.
On May 4, 2012 2:03 PM, "Patrick Hunt" <ph...@apache.org> wrote:
>...
> EOD existing Apache rules/license make no such distinction. "Works
> under the following licenses may be included within Apache products"
> (includes ASL).
Can people please stop using "ASL" or "APL"? No such thing. It is the
Apache License. AL for short, or even ALv2.
Thanks,
-g
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Greg Stein <gs...@gmail.com>.
On May 4, 2012 2:03 PM, "Patrick Hunt" <ph...@apache.org> wrote:
>...
> EOD existing Apache rules/license make no such distinction. "Works
> under the following licenses may be included within Apache products"
> (includes ASL).
Can people please stop using "ASL" or "APL"? No such thing. It is the
Apache License. AL for short, or even ALv2.
Thanks,
-g
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Patrick Hunt <ph...@apache.org>.
On Fri, May 4, 2012 at 9:47 AM, Owen O'Malley <om...@apache.org> wrote:
> On Fri, May 4, 2012 at 8:58 AM, Patrick Hunt <ph...@apache.org> wrote:
>> It's not the job of the incubator to create new rules, but rather to
>> help podlings to graduation while following existing Apache
>> guidelines.
>
> We aren't making new rules. We are trying to help the Bigtop project
> understand the rules about not releasing non-Apache software. There is
> a huge difference between depending on an artifact from another
> project and building and distributing non-Apache rpms in the project's
> /dist directory.
They are not releasing non-Apache software. They are not forking an
existing project. Bigtop's release artifact will contain packaging
code which allows users to compile packages (deb, rpm, etc...) for
this ASL licensed component, not the source/binaries of the component
itself.
>
>> It's very clear from
>> http://www.apache.org/legal/resolved.html that what has been proposed
>> is acceptable under existing Apache rules.
>
> Can you find a single instance other than the disagreement between
> Apache Lucene and Apache Commons where one project is distributing
> another project's rpms? Are there any other non-Apache rpms in /dist?
> Clearly the answer is a resounding NO. It would be a huge violation of
> the trust the incubator is putting in me as a mentor if I didn't block
> Bigtop's plan to do so.
If the component made an objection to being included in Bigtop then I
could see an argument to be made, that's not the case here. The
opposite is true from what I've seen -- people want their software to
be included so that users can more easily consume it. That's why they
released their software under a less restrictive license in the first
place.
EOD existing Apache rules/license make no such distinction. "Works
under the following licenses may be included within Apache products"
(includes ASL).
Patrick
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Patrick Hunt <ph...@apache.org>.
On Fri, May 4, 2012 at 9:47 AM, Owen O'Malley <om...@apache.org> wrote:
> On Fri, May 4, 2012 at 8:58 AM, Patrick Hunt <ph...@apache.org> wrote:
>> It's not the job of the incubator to create new rules, but rather to
>> help podlings to graduation while following existing Apache
>> guidelines.
>
> We aren't making new rules. We are trying to help the Bigtop project
> understand the rules about not releasing non-Apache software. There is
> a huge difference between depending on an artifact from another
> project and building and distributing non-Apache rpms in the project's
> /dist directory.
They are not releasing non-Apache software. They are not forking an
existing project. Bigtop's release artifact will contain packaging
code which allows users to compile packages (deb, rpm, etc...) for
this ASL licensed component, not the source/binaries of the component
itself.
>
>> It's very clear from
>> http://www.apache.org/legal/resolved.html that what has been proposed
>> is acceptable under existing Apache rules.
>
> Can you find a single instance other than the disagreement between
> Apache Lucene and Apache Commons where one project is distributing
> another project's rpms? Are there any other non-Apache rpms in /dist?
> Clearly the answer is a resounding NO. It would be a huge violation of
> the trust the incubator is putting in me as a mentor if I didn't block
> Bigtop's plan to do so.
If the component made an objection to being included in Bigtop then I
could see an argument to be made, that's not the case here. The
opposite is true from what I've seen -- people want their software to
be included so that users can more easily consume it. That's why they
released their software under a less restrictive license in the first
place.
EOD existing Apache rules/license make no such distinction. "Works
under the following licenses may be included within Apache products"
(includes ASL).
Patrick
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Owen O'Malley <om...@apache.org>.
On Fri, May 4, 2012 at 8:58 AM, Patrick Hunt <ph...@apache.org> wrote:
> It's not the job of the incubator to create new rules, but rather to
> help podlings to graduation while following existing Apache
> guidelines.
We aren't making new rules. We are trying to help the Bigtop project
understand the rules about not releasing non-Apache software. There is
a huge difference between depending on an artifact from another
project and building and distributing non-Apache rpms in the project's
/dist directory.
> It's very clear from
> http://www.apache.org/legal/resolved.html that what has been proposed
> is acceptable under existing Apache rules.
Can you find a single instance other than the disagreement between
Apache Lucene and Apache Commons where one project is distributing
another project's rpms? Are there any other non-Apache rpms in /dist?
Clearly the answer is a resounding NO. It would be a huge violation of
the trust the incubator is putting in me as a mentor if I didn't block
Bigtop's plan to do so.
-- Owen
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Owen O'Malley <om...@apache.org>.
On Fri, May 4, 2012 at 8:58 AM, Patrick Hunt <ph...@apache.org> wrote:
> It's not the job of the incubator to create new rules, but rather to
> help podlings to graduation while following existing Apache
> guidelines.
We aren't making new rules. We are trying to help the Bigtop project
understand the rules about not releasing non-Apache software. There is
a huge difference between depending on an artifact from another
project and building and distributing non-Apache rpms in the project's
/dist directory.
> It's very clear from
> http://www.apache.org/legal/resolved.html that what has been proposed
> is acceptable under existing Apache rules.
Can you find a single instance other than the disagreement between
Apache Lucene and Apache Commons where one project is distributing
another project's rpms? Are there any other non-Apache rpms in /dist?
Clearly the answer is a resounding NO. It would be a huge violation of
the trust the incubator is putting in me as a mentor if I didn't block
Bigtop's plan to do so.
-- Owen
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Patrick Hunt <ph...@apache.org>.
It's not the job of the incubator to create new rules, but rather to
help podlings to graduation while following existing Apache
guidelines. It's very clear from
http://www.apache.org/legal/resolved.html that what has been proposed
is acceptable under existing Apache rules. Bigtop is building
on/around ASL licensed software (packaging, iteroperability tests,
utilities, etc...), much like many other Apache projects, much like
other distributions do with Apache's own projects/software. I don't
see any reason to create new rules which limit the podling's ability
to include compatibly licensed software in their releases. (as long as
they follow the licenses of said software)
Patrick
On Thu, May 3, 2012 at 4:30 PM, Bruno Mahé <br...@cloudera.com> wrote:
> Hi,
>
> Please see my reply inline.
>
> On 05/03/2012 04:00 PM, Owen O'Malley wrote:
>> On Thu, May 3, 2012 at 12:01 PM, Bruno Mahé <bm...@apache.org> wrote:
>>>>
>>>> As a mentor of the Bigtop project, I don't see it as acceptable for an
>>>> Apache project to distribute binaries of non-Apache software. If the
>>>> owners of the Hue project decide to donate it to Apache and it had
>>>> been released by Apache, then it would be acceptable. I'm strictly -1
>>>> on releasing any version of Bigtop with Hue or any other non-Apache
>>>> software as part of the release.
>>>>
>>>> -- Owen
>>>
>>>
>>> As part of mentoring Apache Bigtop (incubating) project, it would also
>>> be greatly appreciated if you would explain why this -1.
>>>
>>> Apache Bigtop (incubating) does not and will not include anything that
>>> does not belong to the Apache Foundation.
>>> So I am really confused as to why this strong reaction.
>>
>> The strong reaction is because Roman was proposing a Bigtop release
>> with rpms and debs for non-Apache projects. That is a non-starter.
>> Apache will not distribute non-Apache projects. Saying that Bigtop
>> does not release the projects that it incorporates is not justified
>> given the fact that Bigtop is putting rpms of each of the incorporated
>> projects into /dist/incubator/bigtop. The 2.9GB size of the latest
>> Bigtop release has already caused infrastructure significant
>> headaches.
>>
>
> Seems like you are still (this is not the first time this is explained
> to you) conflating Apache releases on which members vote on, with
> convenience artefacts.
> Apache Bigtop (incubating) releases are not the packages. Apache Bigtop
> (incubating) releases are Apache Bigtop (incubating) source code.
> RPMs and DEBs are convenience artefact.
> If they are not that convenient to the Apache Foundation, I don't see
> the issue with not distributing the ones that are not convenient.
>
> As far as I know, Apache Infra was only asking for heads up. Which we
> will provide and we will pay attention to work more closely with them.
> I also fail to see the relationship between the size of the convenience
> artefacts and the bill of materials for the coming release of Apache
> Bigtop (incubating), which I repeat only contains Apache Bigtop
> (incubating) source code.
>
> So now we have establish that you issue is about the convenience
> artefacts, I don't see any remaining issue with Apache Bigtop
> (incubating) releases.
>
>
>>> The convenience artefact may pull Hue in, but this is in no way
>>> different from Apache Hadoop pulling in Google protocol buffer or Google
>>> guava. So again, how is this different? Is Apache Hadoop going to
>>> avandon Google Protocolbuffer?
>>
>> There is a big difference between referencing external projects that
>> are required for your project's functionality and incorporating
>> non-Apache projects into your project and publishing releases of them
>> using independent artifacts. When the user installs a Hadoop rpm, the
>> protobuf.jar is there under the hood, but is considered an
>> implementation detail that is required for Hadoop to run.
>>
>> I'd complain similarly if Hadoop was downloading protobuf tarballs,
>> making changes to protobuf, making protobuf rpms with those changes,
>> and publishing those rpms on Apache's servers.
>>
>
> In any case, there is still distribution of a non-Apache project's
> artefacts by both projects.
> You either distribute artefacts of it, or you don't. Here the end goal
> is not to provide packages, but a deployable big data stack. Packages
> are just a mean to an end.
> We don't distribute upstream projects, they are dependencies.
>
>
>> However, it goes deeper than than that. If the user installs Bigtop's
>> rpms and hits a bug do they contact Hue or Bigtop? Furthermore, I'm
>> sure the links that are displayed when you run Bigtop's Hue point off
>> to Cloudera's bug and support system. That kind of branding is not ok
>> for an Apache project.
>>
>
> Hue is not even integrated into Apache Bigtop (incubating). So let's
> cross that bridge when we get there. And in any case, this is an issue
> that can be fixed trivially, so I wouldn't make it a blocker.
>
> But beyond that, we don't patch anything. So any product issue would
> come from the product. Any integration issue would be an Apache Bigtop
> (incubating) issue. The same way with Apache Hadoop.
> No matter what you do, educating users will be the most important part.
>
>
>> Even with Apache projects, Bigtop may become problematic. Look at the
>> mess that happened when Lucene made a bugfix release of Apache Commons
>> CSV. Lucene needed a bugfix release of CSV, didn't wait for CSV to
>> release, and instead released it themselves. Needless to say Apache
>> Commons didn't like that result. Bigtop is overriding decisions made
>> by the upstream projects about things like the way the launching
>> scripts operate and where the configuration directory is. When asked
>> to correct it, they complain about compatibility for their users
>> rather than compatibility for Hadoop's users.
>>
>
> How is this related with Hue?
> I am also not sure about the point of complaining about this here on
> general@. This is a direction taken by the Apache Bigtop (incubating)
> community at large. You may not agree, but there is a consensus on that
> part. Unless there is something in the Apache Foundation bylaws that
> would force an upstream project to force decisions on other communities?
>
> And beyond that, this is not new. As it was explained in an other thread
> to Matt on Apache Bigtop (incubating) mailing list:
> Apache Bigtop (incubating) has made the choice from the very beginning
> to be close to what GNU/Linux distributions have been doing and what
> sysadmins have been used to. This can differ with the experience one may
> have with a tarball development of Apache Hadoop.
>
> Keep in mind also that each component has its own way of doing things.
> Each one having its own issues.
> Apache Bigtop (incubating) smooth this up and provide an easy and
> unified experience to users. Furthermore, we cannot satisfy any whim of
> every single upstream components.
>
> In order to use pristine Apache Hadoop, one would have to be familiar
> with its usage and its configuration. Apache Hadoop experience is only
> familiar to people *already* familiar with Apache Hadoop.
>
> So using upstream Apache Hadoop will imply a lot of reading through
> forums, documentation and frustration.
> For instance, Apache Bigtop (incubating) will pre-set the ulimits for
> you, will set up the logging to well-known locations, provide init
> scripts and make a pseudo-configuration available to users.
>
> In a word, Apache Bigtop (incubating) has a different use case than
> upstream Apache Hadoop and therefore will be different in some areas.
> Or put it differently, upstream projects should focus on what they do
> best, which is to work on delivering awesome projects, while in Apache
> Bigtop (incubating) we focus on what we do best, which is making a
> production quality/ready big data stack.
>
> And I don't see the issue of caring about compatibility of Apache Bigtop
> (incubating) users within the Apache Bigtop (incubating) project.
>
> But again, all of this is unrelated to this thread. So you may want to
> move that discussion to another thread.
>
> Thanks,
> Bruno
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Patrick Hunt <ph...@apache.org>.
It's not the job of the incubator to create new rules, but rather to
help podlings to graduation while following existing Apache
guidelines. It's very clear from
http://www.apache.org/legal/resolved.html that what has been proposed
is acceptable under existing Apache rules. Bigtop is building
on/around ASL licensed software (packaging, iteroperability tests,
utilities, etc...), much like many other Apache projects, much like
other distributions do with Apache's own projects/software. I don't
see any reason to create new rules which limit the podling's ability
to include compatibly licensed software in their releases. (as long as
they follow the licenses of said software)
Patrick
On Thu, May 3, 2012 at 4:30 PM, Bruno Mahé <br...@cloudera.com> wrote:
> Hi,
>
> Please see my reply inline.
>
> On 05/03/2012 04:00 PM, Owen O'Malley wrote:
>> On Thu, May 3, 2012 at 12:01 PM, Bruno Mahé <bm...@apache.org> wrote:
>>>>
>>>> As a mentor of the Bigtop project, I don't see it as acceptable for an
>>>> Apache project to distribute binaries of non-Apache software. If the
>>>> owners of the Hue project decide to donate it to Apache and it had
>>>> been released by Apache, then it would be acceptable. I'm strictly -1
>>>> on releasing any version of Bigtop with Hue or any other non-Apache
>>>> software as part of the release.
>>>>
>>>> -- Owen
>>>
>>>
>>> As part of mentoring Apache Bigtop (incubating) project, it would also
>>> be greatly appreciated if you would explain why this -1.
>>>
>>> Apache Bigtop (incubating) does not and will not include anything that
>>> does not belong to the Apache Foundation.
>>> So I am really confused as to why this strong reaction.
>>
>> The strong reaction is because Roman was proposing a Bigtop release
>> with rpms and debs for non-Apache projects. That is a non-starter.
>> Apache will not distribute non-Apache projects. Saying that Bigtop
>> does not release the projects that it incorporates is not justified
>> given the fact that Bigtop is putting rpms of each of the incorporated
>> projects into /dist/incubator/bigtop. The 2.9GB size of the latest
>> Bigtop release has already caused infrastructure significant
>> headaches.
>>
>
> Seems like you are still (this is not the first time this is explained
> to you) conflating Apache releases on which members vote on, with
> convenience artefacts.
> Apache Bigtop (incubating) releases are not the packages. Apache Bigtop
> (incubating) releases are Apache Bigtop (incubating) source code.
> RPMs and DEBs are convenience artefact.
> If they are not that convenient to the Apache Foundation, I don't see
> the issue with not distributing the ones that are not convenient.
>
> As far as I know, Apache Infra was only asking for heads up. Which we
> will provide and we will pay attention to work more closely with them.
> I also fail to see the relationship between the size of the convenience
> artefacts and the bill of materials for the coming release of Apache
> Bigtop (incubating), which I repeat only contains Apache Bigtop
> (incubating) source code.
>
> So now we have establish that you issue is about the convenience
> artefacts, I don't see any remaining issue with Apache Bigtop
> (incubating) releases.
>
>
>>> The convenience artefact may pull Hue in, but this is in no way
>>> different from Apache Hadoop pulling in Google protocol buffer or Google
>>> guava. So again, how is this different? Is Apache Hadoop going to
>>> avandon Google Protocolbuffer?
>>
>> There is a big difference between referencing external projects that
>> are required for your project's functionality and incorporating
>> non-Apache projects into your project and publishing releases of them
>> using independent artifacts. When the user installs a Hadoop rpm, the
>> protobuf.jar is there under the hood, but is considered an
>> implementation detail that is required for Hadoop to run.
>>
>> I'd complain similarly if Hadoop was downloading protobuf tarballs,
>> making changes to protobuf, making protobuf rpms with those changes,
>> and publishing those rpms on Apache's servers.
>>
>
> In any case, there is still distribution of a non-Apache project's
> artefacts by both projects.
> You either distribute artefacts of it, or you don't. Here the end goal
> is not to provide packages, but a deployable big data stack. Packages
> are just a mean to an end.
> We don't distribute upstream projects, they are dependencies.
>
>
>> However, it goes deeper than than that. If the user installs Bigtop's
>> rpms and hits a bug do they contact Hue or Bigtop? Furthermore, I'm
>> sure the links that are displayed when you run Bigtop's Hue point off
>> to Cloudera's bug and support system. That kind of branding is not ok
>> for an Apache project.
>>
>
> Hue is not even integrated into Apache Bigtop (incubating). So let's
> cross that bridge when we get there. And in any case, this is an issue
> that can be fixed trivially, so I wouldn't make it a blocker.
>
> But beyond that, we don't patch anything. So any product issue would
> come from the product. Any integration issue would be an Apache Bigtop
> (incubating) issue. The same way with Apache Hadoop.
> No matter what you do, educating users will be the most important part.
>
>
>> Even with Apache projects, Bigtop may become problematic. Look at the
>> mess that happened when Lucene made a bugfix release of Apache Commons
>> CSV. Lucene needed a bugfix release of CSV, didn't wait for CSV to
>> release, and instead released it themselves. Needless to say Apache
>> Commons didn't like that result. Bigtop is overriding decisions made
>> by the upstream projects about things like the way the launching
>> scripts operate and where the configuration directory is. When asked
>> to correct it, they complain about compatibility for their users
>> rather than compatibility for Hadoop's users.
>>
>
> How is this related with Hue?
> I am also not sure about the point of complaining about this here on
> general@. This is a direction taken by the Apache Bigtop (incubating)
> community at large. You may not agree, but there is a consensus on that
> part. Unless there is something in the Apache Foundation bylaws that
> would force an upstream project to force decisions on other communities?
>
> And beyond that, this is not new. As it was explained in an other thread
> to Matt on Apache Bigtop (incubating) mailing list:
> Apache Bigtop (incubating) has made the choice from the very beginning
> to be close to what GNU/Linux distributions have been doing and what
> sysadmins have been used to. This can differ with the experience one may
> have with a tarball development of Apache Hadoop.
>
> Keep in mind also that each component has its own way of doing things.
> Each one having its own issues.
> Apache Bigtop (incubating) smooth this up and provide an easy and
> unified experience to users. Furthermore, we cannot satisfy any whim of
> every single upstream components.
>
> In order to use pristine Apache Hadoop, one would have to be familiar
> with its usage and its configuration. Apache Hadoop experience is only
> familiar to people *already* familiar with Apache Hadoop.
>
> So using upstream Apache Hadoop will imply a lot of reading through
> forums, documentation and frustration.
> For instance, Apache Bigtop (incubating) will pre-set the ulimits for
> you, will set up the logging to well-known locations, provide init
> scripts and make a pseudo-configuration available to users.
>
> In a word, Apache Bigtop (incubating) has a different use case than
> upstream Apache Hadoop and therefore will be different in some areas.
> Or put it differently, upstream projects should focus on what they do
> best, which is to work on delivering awesome projects, while in Apache
> Bigtop (incubating) we focus on what we do best, which is making a
> production quality/ready big data stack.
>
> And I don't see the issue of caring about compatibility of Apache Bigtop
> (incubating) users within the Apache Bigtop (incubating) project.
>
> But again, all of this is unrelated to this thread. So you may want to
> move that discussion to another thread.
>
> Thanks,
> Bruno
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Bruno Mahé <br...@cloudera.com>.
Hi,
Please see my reply inline.
On 05/03/2012 04:00 PM, Owen O'Malley wrote:
> On Thu, May 3, 2012 at 12:01 PM, Bruno Mahé <bm...@apache.org> wrote:
>>>
>>> As a mentor of the Bigtop project, I don't see it as acceptable for an
>>> Apache project to distribute binaries of non-Apache software. If the
>>> owners of the Hue project decide to donate it to Apache and it had
>>> been released by Apache, then it would be acceptable. I'm strictly -1
>>> on releasing any version of Bigtop with Hue or any other non-Apache
>>> software as part of the release.
>>>
>>> -- Owen
>>
>>
>> As part of mentoring Apache Bigtop (incubating) project, it would also
>> be greatly appreciated if you would explain why this -1.
>>
>> Apache Bigtop (incubating) does not and will not include anything that
>> does not belong to the Apache Foundation.
>> So I am really confused as to why this strong reaction.
>
> The strong reaction is because Roman was proposing a Bigtop release
> with rpms and debs for non-Apache projects. That is a non-starter.
> Apache will not distribute non-Apache projects. Saying that Bigtop
> does not release the projects that it incorporates is not justified
> given the fact that Bigtop is putting rpms of each of the incorporated
> projects into /dist/incubator/bigtop. The 2.9GB size of the latest
> Bigtop release has already caused infrastructure significant
> headaches.
>
Seems like you are still (this is not the first time this is explained
to you) conflating Apache releases on which members vote on, with
convenience artefacts.
Apache Bigtop (incubating) releases are not the packages. Apache Bigtop
(incubating) releases are Apache Bigtop (incubating) source code.
RPMs and DEBs are convenience artefact.
If they are not that convenient to the Apache Foundation, I don't see
the issue with not distributing the ones that are not convenient.
As far as I know, Apache Infra was only asking for heads up. Which we
will provide and we will pay attention to work more closely with them.
I also fail to see the relationship between the size of the convenience
artefacts and the bill of materials for the coming release of Apache
Bigtop (incubating), which I repeat only contains Apache Bigtop
(incubating) source code.
So now we have establish that you issue is about the convenience
artefacts, I don't see any remaining issue with Apache Bigtop
(incubating) releases.
>> The convenience artefact may pull Hue in, but this is in no way
>> different from Apache Hadoop pulling in Google protocol buffer or Google
>> guava. So again, how is this different? Is Apache Hadoop going to
>> avandon Google Protocolbuffer?
>
> There is a big difference between referencing external projects that
> are required for your project's functionality and incorporating
> non-Apache projects into your project and publishing releases of them
> using independent artifacts. When the user installs a Hadoop rpm, the
> protobuf.jar is there under the hood, but is considered an
> implementation detail that is required for Hadoop to run.
>
> I'd complain similarly if Hadoop was downloading protobuf tarballs,
> making changes to protobuf, making protobuf rpms with those changes,
> and publishing those rpms on Apache's servers.
>
In any case, there is still distribution of a non-Apache project's
artefacts by both projects.
You either distribute artefacts of it, or you don't. Here the end goal
is not to provide packages, but a deployable big data stack. Packages
are just a mean to an end.
We don't distribute upstream projects, they are dependencies.
> However, it goes deeper than than that. If the user installs Bigtop's
> rpms and hits a bug do they contact Hue or Bigtop? Furthermore, I'm
> sure the links that are displayed when you run Bigtop's Hue point off
> to Cloudera's bug and support system. That kind of branding is not ok
> for an Apache project.
>
Hue is not even integrated into Apache Bigtop (incubating). So let's
cross that bridge when we get there. And in any case, this is an issue
that can be fixed trivially, so I wouldn't make it a blocker.
But beyond that, we don't patch anything. So any product issue would
come from the product. Any integration issue would be an Apache Bigtop
(incubating) issue. The same way with Apache Hadoop.
No matter what you do, educating users will be the most important part.
> Even with Apache projects, Bigtop may become problematic. Look at the
> mess that happened when Lucene made a bugfix release of Apache Commons
> CSV. Lucene needed a bugfix release of CSV, didn't wait for CSV to
> release, and instead released it themselves. Needless to say Apache
> Commons didn't like that result. Bigtop is overriding decisions made
> by the upstream projects about things like the way the launching
> scripts operate and where the configuration directory is. When asked
> to correct it, they complain about compatibility for their users
> rather than compatibility for Hadoop's users.
>
How is this related with Hue?
I am also not sure about the point of complaining about this here on
general@. This is a direction taken by the Apache Bigtop (incubating)
community at large. You may not agree, but there is a consensus on that
part. Unless there is something in the Apache Foundation bylaws that
would force an upstream project to force decisions on other communities?
And beyond that, this is not new. As it was explained in an other thread
to Matt on Apache Bigtop (incubating) mailing list:
Apache Bigtop (incubating) has made the choice from the very beginning
to be close to what GNU/Linux distributions have been doing and what
sysadmins have been used to. This can differ with the experience one may
have with a tarball development of Apache Hadoop.
Keep in mind also that each component has its own way of doing things.
Each one having its own issues.
Apache Bigtop (incubating) smooth this up and provide an easy and
unified experience to users. Furthermore, we cannot satisfy any whim of
every single upstream components.
In order to use pristine Apache Hadoop, one would have to be familiar
with its usage and its configuration. Apache Hadoop experience is only
familiar to people *already* familiar with Apache Hadoop.
So using upstream Apache Hadoop will imply a lot of reading through
forums, documentation and frustration.
For instance, Apache Bigtop (incubating) will pre-set the ulimits for
you, will set up the logging to well-known locations, provide init
scripts and make a pseudo-configuration available to users.
In a word, Apache Bigtop (incubating) has a different use case than
upstream Apache Hadoop and therefore will be different in some areas.
Or put it differently, upstream projects should focus on what they do
best, which is to work on delivering awesome projects, while in Apache
Bigtop (incubating) we focus on what we do best, which is making a
production quality/ready big data stack.
And I don't see the issue of caring about compatibility of Apache Bigtop
(incubating) users within the Apache Bigtop (incubating) project.
But again, all of this is unrelated to this thread. So you may want to
move that discussion to another thread.
Thanks,
Bruno
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Bruno Mahé <br...@cloudera.com>.
Hi,
Please see my reply inline.
On 05/03/2012 04:00 PM, Owen O'Malley wrote:
> On Thu, May 3, 2012 at 12:01 PM, Bruno Mahé <bm...@apache.org> wrote:
>>>
>>> As a mentor of the Bigtop project, I don't see it as acceptable for an
>>> Apache project to distribute binaries of non-Apache software. If the
>>> owners of the Hue project decide to donate it to Apache and it had
>>> been released by Apache, then it would be acceptable. I'm strictly -1
>>> on releasing any version of Bigtop with Hue or any other non-Apache
>>> software as part of the release.
>>>
>>> -- Owen
>>
>>
>> As part of mentoring Apache Bigtop (incubating) project, it would also
>> be greatly appreciated if you would explain why this -1.
>>
>> Apache Bigtop (incubating) does not and will not include anything that
>> does not belong to the Apache Foundation.
>> So I am really confused as to why this strong reaction.
>
> The strong reaction is because Roman was proposing a Bigtop release
> with rpms and debs for non-Apache projects. That is a non-starter.
> Apache will not distribute non-Apache projects. Saying that Bigtop
> does not release the projects that it incorporates is not justified
> given the fact that Bigtop is putting rpms of each of the incorporated
> projects into /dist/incubator/bigtop. The 2.9GB size of the latest
> Bigtop release has already caused infrastructure significant
> headaches.
>
Seems like you are still (this is not the first time this is explained
to you) conflating Apache releases on which members vote on, with
convenience artefacts.
Apache Bigtop (incubating) releases are not the packages. Apache Bigtop
(incubating) releases are Apache Bigtop (incubating) source code.
RPMs and DEBs are convenience artefact.
If they are not that convenient to the Apache Foundation, I don't see
the issue with not distributing the ones that are not convenient.
As far as I know, Apache Infra was only asking for heads up. Which we
will provide and we will pay attention to work more closely with them.
I also fail to see the relationship between the size of the convenience
artefacts and the bill of materials for the coming release of Apache
Bigtop (incubating), which I repeat only contains Apache Bigtop
(incubating) source code.
So now we have establish that you issue is about the convenience
artefacts, I don't see any remaining issue with Apache Bigtop
(incubating) releases.
>> The convenience artefact may pull Hue in, but this is in no way
>> different from Apache Hadoop pulling in Google protocol buffer or Google
>> guava. So again, how is this different? Is Apache Hadoop going to
>> avandon Google Protocolbuffer?
>
> There is a big difference between referencing external projects that
> are required for your project's functionality and incorporating
> non-Apache projects into your project and publishing releases of them
> using independent artifacts. When the user installs a Hadoop rpm, the
> protobuf.jar is there under the hood, but is considered an
> implementation detail that is required for Hadoop to run.
>
> I'd complain similarly if Hadoop was downloading protobuf tarballs,
> making changes to protobuf, making protobuf rpms with those changes,
> and publishing those rpms on Apache's servers.
>
In any case, there is still distribution of a non-Apache project's
artefacts by both projects.
You either distribute artefacts of it, or you don't. Here the end goal
is not to provide packages, but a deployable big data stack. Packages
are just a mean to an end.
We don't distribute upstream projects, they are dependencies.
> However, it goes deeper than than that. If the user installs Bigtop's
> rpms and hits a bug do they contact Hue or Bigtop? Furthermore, I'm
> sure the links that are displayed when you run Bigtop's Hue point off
> to Cloudera's bug and support system. That kind of branding is not ok
> for an Apache project.
>
Hue is not even integrated into Apache Bigtop (incubating). So let's
cross that bridge when we get there. And in any case, this is an issue
that can be fixed trivially, so I wouldn't make it a blocker.
But beyond that, we don't patch anything. So any product issue would
come from the product. Any integration issue would be an Apache Bigtop
(incubating) issue. The same way with Apache Hadoop.
No matter what you do, educating users will be the most important part.
> Even with Apache projects, Bigtop may become problematic. Look at the
> mess that happened when Lucene made a bugfix release of Apache Commons
> CSV. Lucene needed a bugfix release of CSV, didn't wait for CSV to
> release, and instead released it themselves. Needless to say Apache
> Commons didn't like that result. Bigtop is overriding decisions made
> by the upstream projects about things like the way the launching
> scripts operate and where the configuration directory is. When asked
> to correct it, they complain about compatibility for their users
> rather than compatibility for Hadoop's users.
>
How is this related with Hue?
I am also not sure about the point of complaining about this here on
general@. This is a direction taken by the Apache Bigtop (incubating)
community at large. You may not agree, but there is a consensus on that
part. Unless there is something in the Apache Foundation bylaws that
would force an upstream project to force decisions on other communities?
And beyond that, this is not new. As it was explained in an other thread
to Matt on Apache Bigtop (incubating) mailing list:
Apache Bigtop (incubating) has made the choice from the very beginning
to be close to what GNU/Linux distributions have been doing and what
sysadmins have been used to. This can differ with the experience one may
have with a tarball development of Apache Hadoop.
Keep in mind also that each component has its own way of doing things.
Each one having its own issues.
Apache Bigtop (incubating) smooth this up and provide an easy and
unified experience to users. Furthermore, we cannot satisfy any whim of
every single upstream components.
In order to use pristine Apache Hadoop, one would have to be familiar
with its usage and its configuration. Apache Hadoop experience is only
familiar to people *already* familiar with Apache Hadoop.
So using upstream Apache Hadoop will imply a lot of reading through
forums, documentation and frustration.
For instance, Apache Bigtop (incubating) will pre-set the ulimits for
you, will set up the logging to well-known locations, provide init
scripts and make a pseudo-configuration available to users.
In a word, Apache Bigtop (incubating) has a different use case than
upstream Apache Hadoop and therefore will be different in some areas.
Or put it differently, upstream projects should focus on what they do
best, which is to work on delivering awesome projects, while in Apache
Bigtop (incubating) we focus on what we do best, which is making a
production quality/ready big data stack.
And I don't see the issue of caring about compatibility of Apache Bigtop
(incubating) users within the Apache Bigtop (incubating) project.
But again, all of this is unrelated to this thread. So you may want to
move that discussion to another thread.
Thanks,
Bruno
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Owen O'Malley <om...@apache.org>.
On Thu, May 3, 2012 at 12:01 PM, Bruno Mahé <bm...@apache.org> wrote:
>>
>> As a mentor of the Bigtop project, I don't see it as acceptable for an
>> Apache project to distribute binaries of non-Apache software. If the
>> owners of the Hue project decide to donate it to Apache and it had
>> been released by Apache, then it would be acceptable. I'm strictly -1
>> on releasing any version of Bigtop with Hue or any other non-Apache
>> software as part of the release.
>>
>> -- Owen
>
>
> As part of mentoring Apache Bigtop (incubating) project, it would also
> be greatly appreciated if you would explain why this -1.
>
> Apache Bigtop (incubating) does not and will not include anything that
> does not belong to the Apache Foundation.
> So I am really confused as to why this strong reaction.
The strong reaction is because Roman was proposing a Bigtop release
with rpms and debs for non-Apache projects. That is a non-starter.
Apache will not distribute non-Apache projects. Saying that Bigtop
does not release the projects that it incorporates is not justified
given the fact that Bigtop is putting rpms of each of the incorporated
projects into /dist/incubator/bigtop. The 2.9GB size of the latest
Bigtop release has already caused infrastructure significant
headaches.
> The convenience artefact may pull Hue in, but this is in no way
> different from Apache Hadoop pulling in Google protocol buffer or Google
> guava. So again, how is this different? Is Apache Hadoop going to
> avandon Google Protocolbuffer?
There is a big difference between referencing external projects that
are required for your project's functionality and incorporating
non-Apache projects into your project and publishing releases of them
using independent artifacts. When the user installs a Hadoop rpm, the
protobuf.jar is there under the hood, but is considered an
implementation detail that is required for Hadoop to run.
I'd complain similarly if Hadoop was downloading protobuf tarballs,
making changes to protobuf, making protobuf rpms with those changes,
and publishing those rpms on Apache's servers.
However, it goes deeper than than that. If the user installs Bigtop's
rpms and hits a bug do they contact Hue or Bigtop? Furthermore, I'm
sure the links that are displayed when you run Bigtop's Hue point off
to Cloudera's bug and support system. That kind of branding is not ok
for an Apache project.
Even with Apache projects, Bigtop may become problematic. Look at the
mess that happened when Lucene made a bugfix release of Apache Commons
CSV. Lucene needed a bugfix release of CSV, didn't wait for CSV to
release, and instead released it themselves. Needless to say Apache
Commons didn't like that result. Bigtop is overriding decisions made
by the upstream projects about things like the way the launching
scripts operate and where the configuration directory is. When asked
to correct it, they complain about compatibility for their users
rather than compatibility for Hadoop's users.
-- Owen
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Owen O'Malley <om...@apache.org>.
On Thu, May 3, 2012 at 12:01 PM, Bruno Mahé <bm...@apache.org> wrote:
>>
>> As a mentor of the Bigtop project, I don't see it as acceptable for an
>> Apache project to distribute binaries of non-Apache software. If the
>> owners of the Hue project decide to donate it to Apache and it had
>> been released by Apache, then it would be acceptable. I'm strictly -1
>> on releasing any version of Bigtop with Hue or any other non-Apache
>> software as part of the release.
>>
>> -- Owen
>
>
> As part of mentoring Apache Bigtop (incubating) project, it would also
> be greatly appreciated if you would explain why this -1.
>
> Apache Bigtop (incubating) does not and will not include anything that
> does not belong to the Apache Foundation.
> So I am really confused as to why this strong reaction.
The strong reaction is because Roman was proposing a Bigtop release
with rpms and debs for non-Apache projects. That is a non-starter.
Apache will not distribute non-Apache projects. Saying that Bigtop
does not release the projects that it incorporates is not justified
given the fact that Bigtop is putting rpms of each of the incorporated
projects into /dist/incubator/bigtop. The 2.9GB size of the latest
Bigtop release has already caused infrastructure significant
headaches.
> The convenience artefact may pull Hue in, but this is in no way
> different from Apache Hadoop pulling in Google protocol buffer or Google
> guava. So again, how is this different? Is Apache Hadoop going to
> avandon Google Protocolbuffer?
There is a big difference between referencing external projects that
are required for your project's functionality and incorporating
non-Apache projects into your project and publishing releases of them
using independent artifacts. When the user installs a Hadoop rpm, the
protobuf.jar is there under the hood, but is considered an
implementation detail that is required for Hadoop to run.
I'd complain similarly if Hadoop was downloading protobuf tarballs,
making changes to protobuf, making protobuf rpms with those changes,
and publishing those rpms on Apache's servers.
However, it goes deeper than than that. If the user installs Bigtop's
rpms and hits a bug do they contact Hue or Bigtop? Furthermore, I'm
sure the links that are displayed when you run Bigtop's Hue point off
to Cloudera's bug and support system. That kind of branding is not ok
for an Apache project.
Even with Apache projects, Bigtop may become problematic. Look at the
mess that happened when Lucene made a bugfix release of Apache Commons
CSV. Lucene needed a bugfix release of CSV, didn't wait for CSV to
release, and instead released it themselves. Needless to say Apache
Commons didn't like that result. Bigtop is overriding decisions made
by the upstream projects about things like the way the launching
scripts operate and where the configuration directory is. When asked
to correct it, they complain about compatibility for their users
rather than compatibility for Hadoop's users.
-- Owen
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Bruno Mahé <bm...@apache.org>.
On 05/03/2012 08:06 AM, Owen O'Malley wrote:
> On Mon, Apr 30, 2012 at 9:39 PM, Roman Shaposhnik <rv...@apache.org> wrote:
>> On Mon, Apr 30, 2012 at 9:13 PM, Eric Baldeschwieler
>> <er...@hortonworks.com> wrote:
>>> So you are suggesting expanding the charter to include projects not hosted at Apache?
>>
>> I don't think this is what Bruno suggested. Personally I can't find
>> any reference in Bigtop
>> charter that restricts it to Projects that belong to Apache Software
>> Foundation.
>
> As a mentor of the Bigtop project, I don't see it as acceptable for an
> Apache project to distribute binaries of non-Apache software. If the
> owners of the Hue project decide to donate it to Apache and it had
> been released by Apache, then it would be acceptable. I'm strictly -1
> on releasing any version of Bigtop with Hue or any other non-Apache
> software as part of the release.
>
> -- Owen
As part of mentoring Apache Bigtop (incubating) project, it would also
be greatly appreciated if you would explain why this -1.
Apache Bigtop (incubating) does not and will not include anything that
does not belong to the Apache Foundation.
So I am really confused as to why this strong reaction.
The convenience artefact may pull Hue in, but this is in no way
different from Apache Hadoop pulling in Google protocol buffer or Google
guava. So again, how is this different? Is Apache Hadoop going to
avandon Google Protocolbuffer?
Furthermore as I just stated above, Apache Bigtop (incubating) does not
and will not include any source of Hue.
I am also confused on your definition of release since from my
understanding, only source releases are voted on. And they won't contain
Hue or any code that does not belong to the Apache Foundation. We don't
check in any of our dependency, so I am confused about your statements.
All of this is really no different from Apache Hadoop which pulls some
non-Apache Foundation dependencies such as Google Protocolbuffer.
Thanks,
Bruno
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Bruno Mahé <bm...@apache.org>.
On 05/03/2012 08:06 AM, Owen O'Malley wrote:
> On Mon, Apr 30, 2012 at 9:39 PM, Roman Shaposhnik <rv...@apache.org> wrote:
>> On Mon, Apr 30, 2012 at 9:13 PM, Eric Baldeschwieler
>> <er...@hortonworks.com> wrote:
>>> So you are suggesting expanding the charter to include projects not hosted at Apache?
>>
>> I don't think this is what Bruno suggested. Personally I can't find
>> any reference in Bigtop
>> charter that restricts it to Projects that belong to Apache Software
>> Foundation.
>
> As a mentor of the Bigtop project, I don't see it as acceptable for an
> Apache project to distribute binaries of non-Apache software. If the
> owners of the Hue project decide to donate it to Apache and it had
> been released by Apache, then it would be acceptable. I'm strictly -1
> on releasing any version of Bigtop with Hue or any other non-Apache
> software as part of the release.
>
> -- Owen
As part of mentoring Apache Bigtop (incubating) project, it would also
be greatly appreciated if you would explain why this -1.
Apache Bigtop (incubating) does not and will not include anything that
does not belong to the Apache Foundation.
So I am really confused as to why this strong reaction.
The convenience artefact may pull Hue in, but this is in no way
different from Apache Hadoop pulling in Google protocol buffer or Google
guava. So again, how is this different? Is Apache Hadoop going to
avandon Google Protocolbuffer?
Furthermore as I just stated above, Apache Bigtop (incubating) does not
and will not include any source of Hue.
I am also confused on your definition of release since from my
understanding, only source releases are voted on. And they won't contain
Hue or any code that does not belong to the Apache Foundation. We don't
check in any of our dependency, so I am confused about your statements.
All of this is really no different from Apache Hadoop which pulls some
non-Apache Foundation dependencies such as Google Protocolbuffer.
Thanks,
Bruno
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Alan Gates <ga...@hortonworks.com>.
Roman,
I see your point that many Apache projects include non-Apache code in their binary distributions. But there is a distinction here. In the case of Hadoop and other projects, they bring things such Guava along because they need them, not for the express purpose of distributing those artifacts. Bigtop, by its nature, is different because it provides artifacts for users to download regardless of what other components they need. It is the difference between "we include this because we need it" and "we include this because you might want it".
My concern is that this is a slippery slope. There are lots of other things people use with Hadoop (Ganglia for monitoring, Postgres for their Hive metastore, Cascading, etc.). Would we want Bigtop distributing those? This would consume a lot of Apache resources to host these things on the download servers.
Additionally, we need to think about maintaining Apache's brand. When we redistribute Apache binaries, we know those have gone through an established release process. With non-Apache binaries, even those that are APL, we know nothing of their releases processes, code quality, etc. I do not mean this as a slight to Hue nor any of the projects mentioned above. But if we let one in we will have to let others in. Again this is important because we would be opening ourselves up as a distribution point for those projects independent of their usage in other Apache projects.
By drawing the line at distributing only Apache projects we protect Apache both in terms of server resource usage and in branding.
As Bruno pointed out in the thread on bigtop-dev (http://mail-archives.apache.org/mod_mbox/incubator-bigtop-dev/201205.mbox/%3C4F9F4F6C.4000404%40apache.org%3E ) this does limit Bigtop, so I understand the motivation to do it. But before we take this step it merits discussion in the community.
Finally, a comment on the role of mentors. You were concerned that Owen was vetoing this for non-technical reasons. Your mentors are not here to guide the project just, or even primarily, technically. We are here to help the project learn the Apache way. It is perfectly legitimate, even expected, for a mentor to raise non-techincal concerns such as these.
Alan.
On May 3, 2012, at 8:23 AM, Roman Shaposhnik wrote:
> On Thu, May 3, 2012 at 8:06 AM, Owen O'Malley <om...@apache.org> wrote:
>> On Mon, Apr 30, 2012 at 9:39 PM, Roman Shaposhnik <rv...@apache.org> wrote:
>>> On Mon, Apr 30, 2012 at 9:13 PM, Eric Baldeschwieler
>>> <er...@hortonworks.com> wrote:
>>>> So you are suggesting expanding the charter to include projects not hosted at Apache?
>>>
>>> I don't think this is what Bruno suggested. Personally I can't find
>>> any reference in Bigtop
>>> charter that restricts it to Projects that belong to Apache Software
>>> Foundation.
>>
>> As a mentor of the Bigtop project, I don't see it as acceptable for an
>> Apache project to distribute binaries of non-Apache software.
>
> In that case, I suggest you start with filing upstream JIRAs for pretty much
> all of the Hadoop ecosystem projects kindly asking them to remove
> dependencies on non-ASF (but APL!) projects like Guava libraries. Until
> that happens there's very little Bigtop can do.
>
>> If the owners of the Hue project decide to donate it to Apache and it had
>> been released by Apache, then it would be acceptable. I'm strictly -1
>> on releasing any version of Bigtop with Hue or any other non-Apache
>> software as part of the release.
>
> I would like to point out, that your -1 on Hue is highly inconsistent for the
> reason I mentioned above. I'm not sure what rules ASF has for gratuitous
> -1 votes devoid of clear technical reasoning, but I'm sure as a project
> mentor you can help us find out what that policy is.
>
> Thanks,
> Roman.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Bruno Mahé <bm...@apache.org>.
On 05/03/2012 11:46 AM, Alan Gates wrote:
> Roman,
>
> I see your point that many Apache projects include non-Apache code in their binary distributions. But there is a distinction here. In the case of Hadoop and other projects, they bring things such Guava along because they need them, not for the express purpose of distributing those artifacts. Bigtop, by its nature, is different because it provides artifacts for users to download regardless of what other components they need. It is the difference between "we include this because we need it" and "we include this because you might want it".
>
> My concern is that this is a slippery slope. There are lots of other things people use with Hadoop (Ganglia for monitoring, Postgres for their Hive metastore, Cascading, etc.). Would we want Bigtop distributing those? This would consume a lot of Apache resources to host these things on the download servers.
>
> Additionally, we need to think about maintaining Apache's brand. When we redistribute Apache binaries, we know those have gone through an established release process. With non-Apache binaries, even those that are APL, we know nothing of their releases processes, code quality, etc. I do not mean this as a slight to Hue nor any of the projects mentioned above. But if we let one in we will have to let others in. Again this is important because we would be opening ourselves up as a distribution point for those projects independent of their usage in other Apache projects.
>
> By drawing the line at distributing only Apache projects we protect Apache both in terms of server resource usage and in branding.
>
> As Bruno pointed out in the thread on bigtop-dev (http://mail-archives.apache.org/mod_mbox/incubator-bigtop-dev/201205.mbox/%3C4F9F4F6C.4000404%40apache.org%3E ) this does limit Bigtop, so I understand the motivation to do it. But before we take this step it merits discussion in the community.
>
> Finally, a comment on the role of mentors. You were concerned that Owen was vetoing this for non-technical reasons. Your mentors are not here to guide the project just, or even primarily, technically. We are here to help the project learn the Apache way. It is perfectly legitimate, even expected, for a mentor to raise non-techincal concerns such as these.
>
> Alan.
>
Apache Bigtop (incubating) releases don't distribute any binary. They
only contain Apache Bigtop (incubating) source code.
Convenience binaries are just convenience as far as I understand. So
from what you are saying, your issues have nothing to do with Apache
Bigtop (incubating) releases, but the convenience binaries. Therefore I
don't see the need to suddenly -1 releases of Apache Bigtop (incubating).
Given that we already have to host our own build machines because the
Apache infrastructure is not suited to this sort of work, we could also
host the artefacts on our jenkins instance and save them some burded?
But even though, this is for a discussion with Apache Infrastructure.
And also I wouldn't see any point in distributing Ganglia or Postgres,
even though I see your point.
Or we could restrict the convenience binaries to be just of those
projects belonging to the Apache Foundation. So it would not limit the
scope and abilities of Apache Bigtop (incubating) so much.
Regarding the fact we don't consider these as dependencies, I disagree.
Apache Bigtop (incubating) is an integration point for any project
related to Apache Hadoop. So we produce recipes for a consistent stack.
So of course convenience binaries will include packages, but Apache
Bigtop (incubating) goes way beyond that. Anyone can build their own
stack, with their own version on their favorite GNU/Linux distribution.
There are all sorts of tests (integration, performance, package),
recipes to build your own virtual machine, bootable image and even
deployment recipes for puppet.
So packages are not an end but only a part of this story. The end
product is a deployable stack of Apache Hadoop ecosystem and these
projects related to Apache Hadoop are just used as dependencies.
Thanks,
Bruno
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Roman Shaposhnik <rv...@apache.org>.
Hi Alan!
On Thu, May 3, 2012 at 11:46 AM, Alan Gates <ga...@hortonworks.com> wrote:
> Bigtop, by its nature, is different because it provides artifacts for users to download
> regardless of what other components they need.
> It is the difference between "we include this because we need it" and "we include this because you might want it".
I think what needs to be kept in mind is that at the end of the day,
Apache Foundation Projects release source code. Everything else
is inconsequential binary artifacts.
For some projects binary artifacts hardly matter (e.g. Apache HTTPD
server), for some others they get a lot of attention and consume
a lot of precious Apache INFRA bandwidth (e.g. Apache OpenOffice
[incubating], Apache Bigtop [incubating]).
Yet, I would dare to say that from ASF perspective even OpenOffice
is all about source code releases.
>From that perspective, Bigtop's ultimate goal is to be an ASF project that
provides an integration, validation and deployment framework for
Bigdata management stacks based on Apache Hadoop. In that respect
Bigtop is very close to OpenEmbedded/OpenWRT projects in how
they provide you with a tools to create custom linux distributions.
> My concern is that this is a slippery slope. There are lots of other things people
> use with Hadoop (Ganglia for monitoring, Postgres for their Hive metastore,
> Cascading, etc.). Would we want Bigtop distributing those?
Including support for them into the framework -- sure. Why not? If folks
find it useful to manage all of the above via the Bigtop framework -- where's
harm in that? As long as the license is compatible with APL -- my answer
would be a positive one. In fact, if you look closely you'll see that Bigtop
already supports Kerberos, MySQL and hopefully will soon support
Ganglia via Puppet code.
Now, if you're asking a question of whether we should be including all these
dependencies into the binary convenience artifacts that we publish, then
my answer is predicated on two things:
#1 I would like to steer clear from distributing anything that is
not APL licensed
using Apache infrastructure.
#2 I would like to get an explicit Apache INFRA team blessing that they are
comfortable with the storage and bandwidth requirements.
> Additionally, we need to think about maintaining Apache's brand.
> When we redistribute Apache binaries, we know those have gone
> through an established release process. With non-Apache binaries,
> even those that are APL, we know nothing of their releases processes,
> code quality, etc. I do not mean this as a slight to Hue nor any of the
> projects mentioned above. But if we let one in we will have to let others in.
> Again this is important because we would be opening ourselves up as a
> distribution point for those projects independent of their usage in other
> Apache projects.
I don't think I agree with your line of thought as far as "known
release process"
is concerned. The same argument could be applied to every project that
depends on Guava -- it is something we know nothing about, yet we're perefctly
willing to make it part of the binary artifacts for Hadoop and quite a few other
projects.
I do, however, agree with your concern as far as "distribution point"
goes. Indeed,
it is pretty outlandish to imagine somebody downloading Hadoop binary artifacts
just to extract Guava and use it from there. Thus, by any stretch of
imagination,
Hadoop is not a distribution point for Guava, but if we start
packaging it in Bigtop's
binary convenience artifact -- we will be. This is a valid concern.
Now, personally, I think that this is much more a concern for the
project itself,
rather that Apache, but I'd be curious to hear what other think about it.
> Finally, a comment on the role of mentors. You were concerned that Owen
> was vetoing this for non-technical reasons. Your mentors are not here to
> guide the project just, or even primarily, technically. We are here to help the
> project learn the Apache way. It is perfectly legitimate, even expected, for a
> mentor to raise non-techincal concerns such as these.
Good point. However, as a mentee I expect the level of discourse to be
appropriate.
I have no problem with your reply and careful explanation of the issues at hand.
In fact, I welcome it very much. Owen's reply lacked any of the nuance and felt
as a "my way or the highway" kind of -1. I don't think there's
anything to be learned
from that type of mentoring approach.
Thanks,
Roman.
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Alan Gates <ga...@hortonworks.com>.
Roman,
I see your point that many Apache projects include non-Apache code in their binary distributions. But there is a distinction here. In the case of Hadoop and other projects, they bring things such Guava along because they need them, not for the express purpose of distributing those artifacts. Bigtop, by its nature, is different because it provides artifacts for users to download regardless of what other components they need. It is the difference between "we include this because we need it" and "we include this because you might want it".
My concern is that this is a slippery slope. There are lots of other things people use with Hadoop (Ganglia for monitoring, Postgres for their Hive metastore, Cascading, etc.). Would we want Bigtop distributing those? This would consume a lot of Apache resources to host these things on the download servers.
Additionally, we need to think about maintaining Apache's brand. When we redistribute Apache binaries, we know those have gone through an established release process. With non-Apache binaries, even those that are APL, we know nothing of their releases processes, code quality, etc. I do not mean this as a slight to Hue nor any of the projects mentioned above. But if we let one in we will have to let others in. Again this is important because we would be opening ourselves up as a distribution point for those projects independent of their usage in other Apache projects.
By drawing the line at distributing only Apache projects we protect Apache both in terms of server resource usage and in branding.
As Bruno pointed out in the thread on bigtop-dev (http://mail-archives.apache.org/mod_mbox/incubator-bigtop-dev/201205.mbox/%3C4F9F4F6C.4000404%40apache.org%3E ) this does limit Bigtop, so I understand the motivation to do it. But before we take this step it merits discussion in the community.
Finally, a comment on the role of mentors. You were concerned that Owen was vetoing this for non-technical reasons. Your mentors are not here to guide the project just, or even primarily, technically. We are here to help the project learn the Apache way. It is perfectly legitimate, even expected, for a mentor to raise non-techincal concerns such as these.
Alan.
On May 3, 2012, at 8:23 AM, Roman Shaposhnik wrote:
> On Thu, May 3, 2012 at 8:06 AM, Owen O'Malley <om...@apache.org> wrote:
>> On Mon, Apr 30, 2012 at 9:39 PM, Roman Shaposhnik <rv...@apache.org> wrote:
>>> On Mon, Apr 30, 2012 at 9:13 PM, Eric Baldeschwieler
>>> <er...@hortonworks.com> wrote:
>>>> So you are suggesting expanding the charter to include projects not hosted at Apache?
>>>
>>> I don't think this is what Bruno suggested. Personally I can't find
>>> any reference in Bigtop
>>> charter that restricts it to Projects that belong to Apache Software
>>> Foundation.
>>
>> As a mentor of the Bigtop project, I don't see it as acceptable for an
>> Apache project to distribute binaries of non-Apache software.
>
> In that case, I suggest you start with filing upstream JIRAs for pretty much
> all of the Hadoop ecosystem projects kindly asking them to remove
> dependencies on non-ASF (but APL!) projects like Guava libraries. Until
> that happens there's very little Bigtop can do.
>
>> If the owners of the Hue project decide to donate it to Apache and it had
>> been released by Apache, then it would be acceptable. I'm strictly -1
>> on releasing any version of Bigtop with Hue or any other non-Apache
>> software as part of the release.
>
> I would like to point out, that your -1 on Hue is highly inconsistent for the
> reason I mentioned above. I'm not sure what rules ASF has for gratuitous
> -1 votes devoid of clear technical reasoning, but I'm sure as a project
> mentor you can help us find out what that policy is.
>
> Thanks,
> Roman.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Roman Shaposhnik <rv...@apache.org>.
On Thu, May 3, 2012 at 8:06 AM, Owen O'Malley <om...@apache.org> wrote:
> On Mon, Apr 30, 2012 at 9:39 PM, Roman Shaposhnik <rv...@apache.org> wrote:
>> On Mon, Apr 30, 2012 at 9:13 PM, Eric Baldeschwieler
>> <er...@hortonworks.com> wrote:
>>> So you are suggesting expanding the charter to include projects not hosted at Apache?
>>
>> I don't think this is what Bruno suggested. Personally I can't find
>> any reference in Bigtop
>> charter that restricts it to Projects that belong to Apache Software
>> Foundation.
>
> As a mentor of the Bigtop project, I don't see it as acceptable for an
> Apache project to distribute binaries of non-Apache software.
In that case, I suggest you start with filing upstream JIRAs for pretty much
all of the Hadoop ecosystem projects kindly asking them to remove
dependencies on non-ASF (but APL!) projects like Guava libraries. Until
that happens there's very little Bigtop can do.
> If the owners of the Hue project decide to donate it to Apache and it had
> been released by Apache, then it would be acceptable. I'm strictly -1
> on releasing any version of Bigtop with Hue or any other non-Apache
> software as part of the release.
I would like to point out, that your -1 on Hue is highly inconsistent for the
reason I mentioned above. I'm not sure what rules ASF has for gratuitous
-1 votes devoid of clear technical reasoning, but I'm sure as a project
mentor you can help us find out what that policy is.
Thanks,
Roman.
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Patrick Hunt <ph...@apache.org>.
On Sat, May 5, 2012 at 1:32 AM, Jukka Zitting <ju...@gmail.com> wrote:
> On Fri, May 4, 2012 at 11:00 PM, Roman Shaposhnik <rv...@apache.org> wrote:
>> Perhaps this preso can help a bit:
>> http://people.apache.org/~rvs/apache-bigtop2.pdf
>
> Perfect, thanks!
Roman could you post this on the wiki? (looked but didn't notice it there)
Patrick
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Roman Shaposhnik <rv...@apache.org>.
Hi!
On Sat, May 5, 2012 at 1:32 AM, Jukka Zitting <ju...@gmail.com> wrote:
>>> * If the former, then each subdirectory of [1] falls fairly
>>> conveniently into the traditional concept of convenience binaries
>>> built from the source release. The only extra thing you'd need is a
>>> proper set of license metadata and signatures for the binaries.
>>
>> This is, in fact, the process we've been following with our convenience
>> artifacts for as long as we've had releases. We're signing every single
>> binary and are pretty pedantic about providing LICENSE and NOTICE files.
>
> Sounds good.
>
> I'm just wondering about where do I find for example the licensing
> metadata for example for the files in
> http://www.apache.org/dist/incubator/bigtop/bigtop-0.3.0-incubating/repos/fedora16/hive/
All of them are inside the packages in the location specific to the
given OS. E.g.:
$ rpm -ql -p ~/Downloads/hive-0.8.1-1.fc16.noarch.rpm | egrep
"LICENSE|NOTICE"
/usr/lib/hive/LICENSE
/usr/lib/hive/NOTICE
These are basically rules of Linux distributions:
http://fedoraproject.org/wiki/Packaging:LicensingGuidelines#License_Text
http://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
Please let us know if that's enough from the Apache standpoint or whether
it is desirable to have those files eslewhere (including the repository itself).
Thanks,
Roman.
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Bruno Mahé <bm...@apache.org>.
On 05/05/2012 01:32 AM, Jukka Zitting wrote:
> I'm just wondering about where do I find for example the licensing
> metadata for example for the files in
> http://www.apache.org/dist/incubator/bigtop/bigtop-0.3.0-incubating/repos/fedora16/hive/
>
> Or am I just missing something obvious? Like that you're using RPM
> conventions for those bits, similar to how binary jars typically have
> their licensing metadata in META-INF/{LICENSE,NOTICE}.
>
> BR,
>
> Jukka Zitting
>
RPM meta-data are embedded into it.
Repositories will also embed some sort of meta-data indexing. In your
case, see files in
http://www.apache.org/dist/incubator/bigtop/bigtop-0.3.0-incubating/repos/fedora16/repodata/
If the RPM you want to inspect is already installed, you can use
commandes such as "rpm -i <PKG NAME>" or yum (for CentOS/RedHat/Fedora)
or zypper commands.
Usually packages ship also NOTICE and LICENSE file in the doc dir.
So depending on your system, once a apckage installed, you can find such
a file in /usr/share/doc/<PKG_NAME>/ or
/usr/share/doc/<PKG_NAME>-<PKG_VERSION>/
If you can't find it for a given package, I would recommend to open a
ticket.
Thanks,
Bruno
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
On Fri, May 4, 2012 at 11:00 PM, Roman Shaposhnik <rv...@apache.org> wrote:
> Perhaps this preso can help a bit:
> http://people.apache.org/~rvs/apache-bigtop2.pdf
Perfect, thanks!
>> * If the former, then each subdirectory of [1] falls fairly
>> conveniently into the traditional concept of convenience binaries
>> built from the source release. The only extra thing you'd need is a
>> proper set of license metadata and signatures for the binaries.
>
> This is, in fact, the process we've been following with our convenience
> artifacts for as long as we've had releases. We're signing every single
> binary and are pretty pedantic about providing LICENSE and NOTICE files.
Sounds good.
I'm just wondering about where do I find for example the licensing
metadata for example for the files in
http://www.apache.org/dist/incubator/bigtop/bigtop-0.3.0-incubating/repos/fedora16/hive/
Or am I just missing something obvious? Like that you're using RPM
conventions for those bits, similar to how binary jars typically have
their licensing metadata in META-INF/{LICENSE,NOTICE}.
BR,
Jukka Zitting
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Roman Shaposhnik <rv...@apache.org>.
On Tue, May 8, 2012 at 12:50 AM, Daniel Shahaf <d....@daniel.shahaf.name> wrote:
> Greg Stein wrote on Tue, May 08, 2012 at 03:32:32 -0400:
>> On Tue, May 8, 2012 at 3:03 AM, Bruno Mahé <bm...@apache.org> wrote:
>> > Note also that Infra has already been involved.
>>
>> Yup. Saw.
>
> What's the infra issue here? I see no new mentions of bigtop on infra
> list in the last day
I don't think there's an issue to be resolved, but rather an awareness in
the Bigtop community that given the size of our releases we have to
be very proactive about coordinating them with the Apache Infra team.
Bigtop community and its release managers are well aware of that and
are very sensitive to making INFRA's job easier. We will definitely not
surprise you with 0.4.0.
Thanks,
Roman.
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Greg Stein wrote on Tue, May 08, 2012 at 03:32:32 -0400:
> On Tue, May 8, 2012 at 3:03 AM, Bruno Mahé <bm...@apache.org> wrote:
> > Note also that Infra has already been involved.
>
> Yup. Saw.
What's the infra issue here? I see no new mentions of bigtop on infra
list in the last day
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Bruno Mahé <bm...@apache.org>.
On 05/08/2012 05:50 AM, Owen O'Malley wrote:
> On Mon, May 7, 2012 at 9:05 AM, Roman Shaposhnik <rv...@apache.org> wrote:
>> On Mon, May 7, 2012 at 8:46 AM, Owen O'Malley <om...@apache.org> wrote:
>>> On Fri, May 4, 2012 at 2:00 PM, Roman Shaposhnik <rv...@apache.org> wrote:
>>>
>>>>> I don't understand the BigTop use cases
>>>>
>>>> Perhaps this preso can help a bit:
>>>> http://people.apache.org/~rvs/apache-bigtop2.pdf
>>>
>>> Notes from this presentation:
>>> * 100% Apache Bigdata management distribution
>>> * Bigtop is a distribution for users
>>
>> Owen, I'd appreciate if you refrained from quoting out of context.
>> Please take a look at slide #11 for a complete list of things
>> that Bigtop is trying to accomplish. Saying that it is *just*
>> "a distribution for users" is like saying that Debian GNU Linux
>> is just about the Linux kernel.
>
> I never said "just" a distribution for users, but your presentation
> makes it very clear it is first and foremost a distribution for users.
> Since we now agree about that, how about getting back to whether an
> Apache distro project should be distributing non-Apache software.
>
> -- Owen
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
Again, we don't distribute non-Apache software, as well as we don't
distribute other Apache software either.
Since you keep on doing this mistake, there is surely a
misunderstanding. So it would probably be easier if you could point us
to other Apache (or non-Apache) software being distributed as part as
any Apache Bigtop (incubating) releases on which members have voted on?
And again, Apache releases are what people vote on. not the convenience
artefacts which are there for convenience and are not voted on. So I
still don't see any reason to not withdraw your -1 since your only issue
seems to be related to the dependencies in the convenience artefacts
(which is standard practice, even within Apache Hadoop)
I would really appreciate if you could stop conflating Apache releases
with convenience artefacts. The discussion would be able to go much
further and not just spam general@ members.
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Owen O'Malley <om...@apache.org>.
On Tue, May 8, 2012 at 12:30 AM, Jukka Zitting <ju...@gmail.com> wrote:
> Hi,
>
> On Tue, May 8, 2012 at 8:38 AM, Owen O'Malley <om...@apache.org> wrote:
>> They are proposing adding Hue, which is a Apache licensed project on
>> Github.
>
> Does Hue match the guidelines in http://www.apache.org/legal/resolved.html?
>
> If yes, then it's fine by ASF policy for an Apache project to include it.
>
> Is there some other reason why BigTop shouldn't be adding Hue?
Hue is Apache licensed, so that isn't an issue. The issue is that
Bigtop is planning on distributing non-Apache software for the primary
purpose of providing it to their users. It seems to me pretty
fundamental that Apache projects should only distribute Apache
software except as a dependency. Bigtop is only including Hue to
provide it to their users with no dependency at all.
It would be great if Cloudera were to contribute Hue to Apache. Unless
they do, I really don't think it is appropriate for us to distribute
standalone binary packages of their project from Apache's servers.
-- Owen
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
On Tue, May 8, 2012 at 8:38 AM, Owen O'Malley <om...@apache.org> wrote:
> They are proposing adding Hue, which is a Apache licensed project on
> Github.
Does Hue match the guidelines in http://www.apache.org/legal/resolved.html?
If yes, then it's fine by ASF policy for an Apache project to include it.
Is there some other reason why BigTop shouldn't be adding Hue?
BR,
Jukka Zitting
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Owen O'Malley <om...@apache.org>.
On Mon, May 7, 2012 at 11:08 PM, Greg Stein <gs...@gmail.com> wrote:
> On Mon, May 7, 2012 at 9:48 PM, Bruno Mahé <bm...@apache.org> wrote:
>>...
>
> It seems that we're talking about this location:
> http://www.apache.org/dist/incubator/bigtop/bigtop-0.3.0-incubating/
>
>> Again, we don't distribute non-Apache software,
>
> I didn't find any non-Apache software in the location noted above.
They are proposing adding Hue, which is a Apache licensed project on
Github. I raised the concern when they stated the plan, not after the
release.
-- Owen
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Roman Shaposhnik <rv...@apache.org>.
On Tue, May 8, 2012 at 4:30 AM, Marvin Humphrey <ma...@rectangular.com> wrote:
> On Tue, May 8, 2012 at 12:03 AM, Bruno Mahé <bm...@apache.org> wrote:
>> The output of Apache Bigtop (incubating) can be quite unusual since it
>> is a deployable production quality big data stack.
>
> What does it take to get a product into the Bigtop stack?
Two things are needed for a component to be proposed for
addition to Bigtop:
* The license has to be AL
* There has to be a champion for it
After that there's a vote by the community to include a component
into the next release (whatever that may be). Bigtop follow a rigid
date-drive quarterly release model and we vote on Bills Of Materials
(BOMs) for every new release that we are about to work on.
Finally, the work that is required for inclusion consists of:
* providing packaging for the component on all supported platforms
* providing deployment code for the component
* providing smoke test code for the component
One that patch including all 3 of the above is posted on a JIRA,
+1ed and committed the component is considered to be part of
the Bigtop. We do not have a notion of "maintainers". Once something
is added to Bigtop it is everybody's responsibility to maintain it.
> I don't see any legal problems, but is vendor neutrality an issue here?
At this point I'm slightly confused as to what this thread is actually
trying to accomplish. I thought the original intent was to:
* screen for potential legal issues
* help incubator mentors understand Bigtop
I believe we've accomplished both. To the best of our collective
knowledge there are no legal issues with adding Hue to Bigtop
and it feels like the mission and charter of Bigtop is now well
understood.
Is there anything else left to be discussed?
Thanks,
Roman.
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Marvin Humphrey <ma...@rectangular.com>.
On Tue, May 8, 2012 at 12:03 AM, Bruno Mahé <bm...@apache.org> wrote:
> The output of Apache Bigtop (incubating) can be quite unusual since it
> is a deployable production quality big data stack.
What does it take to get a product into the Bigtop stack?
I don't see any legal problems, but is vendor neutrality an issue here?
Marvin Humphrey
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Greg Stein <gs...@gmail.com>.
On Tue, May 8, 2012 at 3:03 AM, Bruno Mahé <bm...@apache.org> wrote:
> First, thank you very much for taking the time to write a thoughtful reply.
>
>
> On 05/08/2012 02:08 PM, Greg Stein wrote:
>...
>>> as well as we don't
>>> distribute other Apache software either.
>>
>> I found a bunch of Apache software in there.
>>
>> So I'd say you're distributing it.
>>
>> (not that I believe that is a problem, but let's try and get things
>> straight here)
>>
>
> But all these files are convenience artefacts. They are not part of a
> release.
Sure.
> I don't see any difference from the Google protocolbuffer binary
> artefacts found in Apache Hadoop convenience artefacts. Except that
> instead of all of them being hidden into a giant tarball, they are
> organized openly inside a repository.
I don't care what other projects do. We're talking about Bigtop right now.
(I can comment on Hadoop/protobuf, but will not do so in this thread)
>>> Since you keep on doing this mistake, there is surely a
>>> misunderstanding. So it would probably be easier if you could point us
>>> to other Apache (or non-Apache) software being distributed as part as
>>> any Apache Bigtop (incubating) releases on which members have voted on?
>>
>> I can see anybody looking at that URL location believing that you're
>> distributing both Bigtop *AND* other Apache software.
>>
>> If you're using a different definition of "distributing", then we need
>> to resolve that situation. I see lots of ASF software being
>> distributed via that URL.
>
> I don't think I am using a different definition of "distributing. But I
> would like to stress out that I make a clear distinction between what is
> distributed as part of an Apache release and what is distributed as part
> of a convenience artefact.
Agreed. There *is* a distinction.
My point was that placing them both in the same directory (URL;
whatever) makes it *appear* they are combined. And based on that
appearance, people are (empirically demonstrated) getting confused.
But stepping back a little bit. You are distributing two things:
1) Apache Bigtop (incubating)
2) Convenience artifacts
Do you agree with that?
> Here is a quote from Owen: "I'm strictly -1
> on releasing any version of Bigtop with Hue or any other non-Apache
> software as part of the release."
> And we truly do not include anything else then Apache Bigtop
> (incubating) in Apache Bigtop (incubating) releases. And as far as I can
> tell, the issue Owen is having is related to convenience artefacts, not
> releases.
Fine. Relax. So he misspoke in one single sentence. It happens. Move
on, already.
I will offer a piece of advice here: if you ever write an email with
somebody's name in it, then please step back and reconsider. A
technical/procedural/whatever discussion *does not* require people's
names to be productive. Speak to the issue at hand, the problem being
raised, the disconnect that has occurred. Continuing to bring Owen's
name into this discussion is a complete distraction.
In short: omit names. It really helps.
> We use these upstream projects as dependencies. We don't distribute
> directly other software in the convenience artefacts of Apache Bigtop
> (incubating).
You distribute *some* software. I don't know that it matters what that
software is, but simply the fact that you're distributing two things:
Bigtop, and "other stuff". If you agree with that, then let's talk
about people's concerns with the "other stuff" part.
(I don't think anybody has raised any issue with the Apache Bigtop
(incubating) part of the release, so assuming that's true, let's just
please drop that part, stop harping on it, and move on to the relevant
part of the discussion)
>...
>> It is true that votes only apply to Bigtop releases and not those
>> artifacts under bigtop-0.3.0-incubating/repos/.
>>
>> It is also true that (in normal PMC operation) it is not possible to
>> veto a release. I have no strong opinion, but would believe the that a
>> podling can also make a release with three IPMC +1 votes (and, again,
>> vetoes do not apply). Note: IPMC votes, not PPMC votes. If somebody
>> raises a -1, then I suspect getting those IPMC votes might be
>> difficult until the concern is discussed.
>>
>> Regarding the release artifacts: I don't have any strong opinions
>> right now. I can see the confusion with them all in the same location.
>> I can also see that Infra should probably get involved in some way to
>> deal with mirroring issues and to somehow distinguish, verify, and
>> validate these binary artifacts.
>
> What confuses me the most, is that Owen's issue is with the convenience
> artefacts, not with the release itself. So there is no real reason to -1
> releases if they are not the issue.
Whatever. Let's assume he misspoke and restart the conversation
instead of rehashing this over and over. Let's not try and state what
he should or should not do, and whether he has any "real reason" to do
so.
> And if I understand correctly from what you are saying, there is really
> nothing preventing us from continuing, providing the Apache Bigtop
> (incubating) community is in agreement (ie. enough +1 on tickets and
> enough +1 from IPMC members for releases).
Correct. All ASF releases require three +1 votes from PMC members. For
podlings, that means Incubator PMC members. (the PPMC is a construct
of the IPMC; it has no bearing on ASF releases)
> Although I would rather end
> up with everyone agreeing.
That is the best plan. Discord is never a good thing.
> But in the worst case scenario, we can always stop publishing
> convenience artefacts as part of Apache releases, and host these
> convenience artefacts outside of the Apache Infrastructure as
> contributors and bypass all these concerns.
Whatever you would like to do.
It seems the only point of real discussion here is what to do with
hosting artifacts that are NOT ASF products. (eg. HUE)
Is that the real discussion point? Non-ASF products?
Note: many ASF products incorporate third-party source (that match our
licensing policies). This is normal and not a problem. So what is the
differentiator? Is it simply that the ASF has compiled it first? Is
that really a problem?
> Note also that Infra has already been involved.
Yup. Saw.
Cheers,
-g
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Bruno Mahé <bm...@apache.org>.
First, thank you very much for taking the time to write a thoughtful reply.
On 05/08/2012 02:08 PM, Greg Stein wrote:
> On Mon, May 7, 2012 at 9:48 PM, Bruno Mahé <bm...@apache.org> wrote:
>> ...
>
> It seems that we're talking about this location:
> http://www.apache.org/dist/incubator/bigtop/bigtop-0.3.0-incubating/
>
>> Again, we don't distribute non-Apache software,
>
> I didn't find any non-Apache software in the location noted above.
>
>> as well as we don't
>> distribute other Apache software either.
>
> I found a bunch of Apache software in there.
>
> So I'd say you're distributing it.
>
> (not that I believe that is a problem, but let's try and get things
> straight here)
>
But all these files are convenience artefacts. They are not part of a
release.
I don't see any difference from the Google protocolbuffer binary
artefacts found in Apache Hadoop convenience artefacts. Except that
instead of all of them being hidden into a giant tarball, they are
organized openly inside a repository.
>> Since you keep on doing this mistake, there is surely a
>> misunderstanding. So it would probably be easier if you could point us
>> to other Apache (or non-Apache) software being distributed as part as
>> any Apache Bigtop (incubating) releases on which members have voted on?
>
> I can see anybody looking at that URL location believing that you're
> distributing both Bigtop *AND* other Apache software.
>
> If you're using a different definition of "distributing", then we need
> to resolve that situation. I see lots of ASF software being
> distributed via that URL.
>
I don't think I am using a different definition of "distributing. But I
would like to stress out that I make a clear distinction between what is
distributed as part of an Apache release and what is distributed as part
of a convenience artefact.
Here is a quote from Owen: "I'm strictly -1
on releasing any version of Bigtop with Hue or any other non-Apache
software as part of the release."
And we truly do not include anything else then Apache Bigtop
(incubating) in Apache Bigtop (incubating) releases. And as far as I can
tell, the issue Owen is having is related to convenience artefacts, not
releases.
We use these upstream projects as dependencies. We don't distribute
directly other software in the convenience artefacts of Apache Bigtop
(incubating).
The output of Apache Bigtop (incubating) can be quite unusual since it
is a deployable production quality big data stack. So we need to
consider the project as a whole, not just one single part.
The release provides the recipes to enable you to build and/or customize
such a stack. These recipes include the recipes to build packages,
virtual machines, integration, performance and validation tests as well
as recipes (puppet recipes) to deploy such stack.
But we also provide a validated stack as convenience artefacts.
>> And again, Apache releases are what people vote on. not the convenience
>> artefacts which are there for convenience and are not voted on. So I
>> still don't see any reason to not withdraw your -1 since your only issue
>> seems to be related to the dependencies in the convenience artefacts
>> (which is standard practice, even within Apache Hadoop)
>
> It is true that votes only apply to Bigtop releases and not those
> artifacts under bigtop-0.3.0-incubating/repos/.
>
> It is also true that (in normal PMC operation) it is not possible to
> veto a release. I have no strong opinion, but would believe the that a
> podling can also make a release with three IPMC +1 votes (and, again,
> vetoes do not apply). Note: IPMC votes, not PPMC votes. If somebody
> raises a -1, then I suspect getting those IPMC votes might be
> difficult until the concern is discussed.
>
> Regarding the release artifacts: I don't have any strong opinions
> right now. I can see the confusion with them all in the same location.
> I can also see that Infra should probably get involved in some way to
> deal with mirroring issues and to somehow distinguish, verify, and
> validate these binary artifacts.
>
>> ...
>
What confuses me the most, is that Owen's issue is with the convenience
artefacts, not with the release itself. So there is no real reason to -1
releases if they are not the issue.
And if I understand correctly from what you are saying, there is really
nothing preventing us from continuing, providing the Apache Bigtop
(incubating) community is in agreement (ie. enough +1 on tickets and
enough +1 from IPMC members for releases). Although I would rather end
up with everyone agreeing.
But in the worst case scenario, we can always stop publishing
convenience artefacts as part of Apache releases, and host these
convenience artefacts outside of the Apache Infrastructure as
contributors and bypass all these concerns.
Note also that Infra has already been involved.
Thanks,
Bruno
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Greg Stein <gs...@gmail.com>.
On Mon, May 7, 2012 at 9:48 PM, Bruno Mahé <bm...@apache.org> wrote:
>...
It seems that we're talking about this location:
http://www.apache.org/dist/incubator/bigtop/bigtop-0.3.0-incubating/
> Again, we don't distribute non-Apache software,
I didn't find any non-Apache software in the location noted above.
> as well as we don't
> distribute other Apache software either.
I found a bunch of Apache software in there.
So I'd say you're distributing it.
(not that I believe that is a problem, but let's try and get things
straight here)
> Since you keep on doing this mistake, there is surely a
> misunderstanding. So it would probably be easier if you could point us
> to other Apache (or non-Apache) software being distributed as part as
> any Apache Bigtop (incubating) releases on which members have voted on?
I can see anybody looking at that URL location believing that you're
distributing both Bigtop *AND* other Apache software.
If you're using a different definition of "distributing", then we need
to resolve that situation. I see lots of ASF software being
distributed via that URL.
> And again, Apache releases are what people vote on. not the convenience
> artefacts which are there for convenience and are not voted on. So I
> still don't see any reason to not withdraw your -1 since your only issue
> seems to be related to the dependencies in the convenience artefacts
> (which is standard practice, even within Apache Hadoop)
It is true that votes only apply to Bigtop releases and not those
artifacts under bigtop-0.3.0-incubating/repos/.
It is also true that (in normal PMC operation) it is not possible to
veto a release. I have no strong opinion, but would believe the that a
podling can also make a release with three IPMC +1 votes (and, again,
vetoes do not apply). Note: IPMC votes, not PPMC votes. If somebody
raises a -1, then I suspect getting those IPMC votes might be
difficult until the concern is discussed.
Regarding the release artifacts: I don't have any strong opinions
right now. I can see the confusion with them all in the same location.
I can also see that Infra should probably get involved in some way to
deal with mirroring issues and to somehow distinguish, verify, and
validate these binary artifacts.
>...
Cheers,
-g
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Bruno Mahé <bm...@apache.org>.
On 05/08/2012 05:50 AM, Owen O'Malley wrote:
> On Mon, May 7, 2012 at 9:05 AM, Roman Shaposhnik <rv...@apache.org> wrote:
>> On Mon, May 7, 2012 at 8:46 AM, Owen O'Malley <om...@apache.org> wrote:
>>> On Fri, May 4, 2012 at 2:00 PM, Roman Shaposhnik <rv...@apache.org> wrote:
>>>
>>>>> I don't understand the BigTop use cases
>>>>
>>>> Perhaps this preso can help a bit:
>>>> http://people.apache.org/~rvs/apache-bigtop2.pdf
>>>
>>> Notes from this presentation:
>>> * 100% Apache Bigdata management distribution
>>> * Bigtop is a distribution for users
>>
>> Owen, I'd appreciate if you refrained from quoting out of context.
>> Please take a look at slide #11 for a complete list of things
>> that Bigtop is trying to accomplish. Saying that it is *just*
>> "a distribution for users" is like saying that Debian GNU Linux
>> is just about the Linux kernel.
>
> I never said "just" a distribution for users, but your presentation
> makes it very clear it is first and foremost a distribution for users.
> Since we now agree about that, how about getting back to whether an
> Apache distro project should be distributing non-Apache software.
>
> -- Owen
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
Again, we don't distribute non-Apache software, as well as we don't
distribute other Apache software either.
Since you keep on doing this mistake, there is surely a
misunderstanding. So it would probably be easier if you could point us
to other Apache (or non-Apache) software being distributed as part as
any Apache Bigtop (incubating) releases on which members have voted on?
And again, Apache releases are what people vote on. not the convenience
artefacts which are there for convenience and are not voted on. So I
still don't see any reason to not withdraw your -1 since your only issue
seems to be related to the dependencies in the convenience artefacts
(which is standard practice, even within Apache Hadoop)
I would really appreciate if you could stop conflating Apache releases
with convenience artefacts. The discussion would be able to go much
further and not just spam general@ members.
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Owen O'Malley <om...@apache.org>.
On Mon, May 7, 2012 at 9:05 AM, Roman Shaposhnik <rv...@apache.org> wrote:
> On Mon, May 7, 2012 at 8:46 AM, Owen O'Malley <om...@apache.org> wrote:
>> On Fri, May 4, 2012 at 2:00 PM, Roman Shaposhnik <rv...@apache.org> wrote:
>>
>>>> I don't understand the BigTop use cases
>>>
>>> Perhaps this preso can help a bit:
>>> http://people.apache.org/~rvs/apache-bigtop2.pdf
>>
>> Notes from this presentation:
>> * 100% Apache Bigdata management distribution
>> * Bigtop is a distribution for users
>
> Owen, I'd appreciate if you refrained from quoting out of context.
> Please take a look at slide #11 for a complete list of things
> that Bigtop is trying to accomplish. Saying that it is *just*
> "a distribution for users" is like saying that Debian GNU Linux
> is just about the Linux kernel.
I never said "just" a distribution for users, but your presentation
makes it very clear it is first and foremost a distribution for users.
Since we now agree about that, how about getting back to whether an
Apache distro project should be distributing non-Apache software.
-- Owen
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Roman Shaposhnik <rv...@apache.org>.
On Mon, May 7, 2012 at 8:46 AM, Owen O'Malley <om...@apache.org> wrote:
> On Fri, May 4, 2012 at 2:00 PM, Roman Shaposhnik <rv...@apache.org> wrote:
>
>>> I don't understand the BigTop use cases
>>
>> Perhaps this preso can help a bit:
>> http://people.apache.org/~rvs/apache-bigtop2.pdf
>
> Notes from this presentation:
> * 100% Apache Bigdata management distribution
> * Bigtop is a distribution for users
Owen, I'd appreciate if you refrained from quoting out of context.
Please take a look at slide #11 for a complete list of things
that Bigtop is trying to accomplish. Saying that it is *just*
"a distribution for users" is like saying that Debian GNU Linux
is just about the Linux kernel.
Thanks,
Roman.
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Owen O'Malley <om...@apache.org>.
On Fri, May 4, 2012 at 2:00 PM, Roman Shaposhnik <rv...@apache.org> wrote:
>> I don't understand the BigTop use cases
>
> Perhaps this preso can help a bit:
> http://people.apache.org/~rvs/apache-bigtop2.pdf
Notes from this presentation:
* 100% Apache Bigdata management distribution
* Bigtop is a distribution for users
When users install "Bigtop" they get a small rpm of bigtop utilities
and rpms lot of other projects. Apache has never had a Redhat-like
project before, but to claim that publishing non-Apache rpms and debs
for the sole purpose of providing them to users is identical to
project X bundling their required external dependencies is bending the
truth substantially.
-- Owen
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Roman Shaposhnik <rv...@apache.org>.
Hi Jukka!
Thanks a million for chiming in.
On Fri, May 4, 2012 at 12:52 PM, Jukka Zitting <ju...@gmail.com> wrote:
> Hi,
>
> I don't understand the BigTop use cases
Perhaps this preso can help a bit:
http://people.apache.org/~rvs/apache-bigtop2.pdf
> * If the former, then each subdirectory of [1] falls fairly
> conveniently into the traditional concept of convenience binaries
> built from the source release. The only extra thing you'd need is a
> proper set of license metadata and signatures for the binaries.
This is, in fact, the process we've been following with our convenience
artifacts for as long as we've had releases. We're signing every single
binary and are pretty pedantic about providing LICENSE and NOTICE files.
All of the distributed convenience artifacts have Apache License and this
policy will remain. Staring from Bigtop 0.4.0 release we are going to also
take great care in coordinating release of our convenience artifacts with
Apache Software Foundation's Infrastructure team in order to avoid
surprises and reduce strain on the mirroring infrastructure.
Please let me know if, from your point of view, the above alleviates the
concerns that have been expressed on this list.
Thanks,
Roman.
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Steve Loughran <st...@gmail.com>.
On 4 May 2012 16:34, Alan Gates <ga...@hortonworks.com> wrote:
>
> > * In that case there might still be a role for BigTop to provide a
> > central repository for such easily consumable upstream releases. This
> > would be somewhat similar to the discussions that took place a few
> > years ago about whether and how the ASF could host something like the
> > central Maven repository.
>
>
> Do you know what list that discussion took place on and a general time
> frame? Reading through that would be very helpful for my thinking on this
> topic.
>
>
That probably dates back a decade -the outcome was that maven.org became
the place where any JAR could go, irrespective of origin, authentication or
quality of pom.xml metadata. The ASF artifacts only go there after
validation of the release process.
the nice thing about JAR files is that they are platform-independent, so a
single global repository can share stuff without problems; the artifacts
remain valid forever.
Once you get into platform binaries and OS-specific packaging, you get
platform and version problems.
My thoughts
- It's up to the OS teams & vendors to run their own repositories. It's
their business model, and I'm happy to use ubuntu and redhat's repositories
of signed artifacts.
- Similarly, it's up to them to compile native binaries down for their
platforms, qualify them on clusters.
- What they do need is a set of programs that are designed on their
platforms, putting stuff in the right places, working with the OS, rather
than against it (a permanent issue in java-land)
- The metadata to create the OS specific installation bundles -RPM, Deb,
whatever else- are something that helps them, and helps ensure that the
downstream distributor doesn't come up with their own rules and layout that
doesn't work or causes support issues for everyone.
- What is useful for them are better tests to qualify the entire stack,
so a nighly ubuntu or redhat stack can test the stable hadoop stack to
catch OS/JDK regressions, other people can qualify their entire cluster
with more than just terasort, etc, etc. These tests should be designed to
qualify a cluster independently of how it was bundled, installed or
deployed.
- The ASF hasn't ever been in the role of an RPM/Deb source itself, and
never pushing out virtual hard disk images containing entire Linux VMs.
Owen's unhappy that HUE is going in as even though it has a license that
works with Apache products, it's not part of the in-Apache Hadoop community
codebase. It's aim is to provide a front end/management system for the
product, and while some such product is invaluable, the issue that is
arising is this
-should the ASF be dictating which external tool should be managing your
Hadoop cluster and doing so by declaring that the bigtop artifacts depend
upon it -so giving what is effectively one product a defacto seal of
approval?
I don't think it should unless there's some consensus within all the
projects contributing the code that yes, they are happy for the ASF
integration project to do this, and that there is no alternative way to do
it.
The thing is, there is no reason to say apache-bigtop-x.y depends on HUE.
-The ASF RPM/Deb metadata and any sandbox artifacts can include the ASF
code and any dependent artifacts that they have no choice but to rely on.
-Redistributors of any kind are free to make the production artifacts they
choose.
-They are also free to pull any other dependencies they want to. That
includes Hue, ganglia, Nagios, SmartFrog (which does currently do its own
Hadoop RPM and would gladly not do so).
There's a big difference in saying "here is the metadata to create the
apache artifacts" and "here is the MD to create the apache artifacts and
some others we want everyone downstream to take up". That's more of a
decision for the downstream people.
There's a separate issue which is:
Should the ASF be providing its own OS specific repositories of artifacts?
That's a step beyond sticking JAR files up online. I think its good to have
the artifacts on repositories for apt-get and yum, but there's no reason
why the responsibility for production releases should not be pushed out to
Redhat, Canonical, etc. Anything hosted by the ASF should be restricted to
nightly build/snapshot releases so that people testing the distribution
process has access to those releases, but formal releases ought to be
pushed out to whoever wants to take the source code and build and qualify
the packages so generated. Otherwise you can't say "we are just doing the
metadata"
Finally, there's the VMs. I'm never personally fond of downloading them as
they never seem to come with a keyboard setting that works in my locale,
but they do give people a great way to get up and running. At the same time
they not only stick in the binary stack, they also include the linux image
and take up a fairly large amount of bandwidth
I think the tactic there is to work with someone like VirtualBoxImages who
stick the VMs up on SourceForge, such as for all centos VMs
http://virtualboxes.org/images/centos/
They get to build up the VMs with their choices, and they take on the cost
of keeping those images secure, that being the great pain of VM images
-every backup OS image you have ages at the rate of one critical adobe
patch a fortnight. (**)
Do that off-ASF packaging, you get to pull in whatever extra layers on top
you want and infrastructure stop expressing so much concern.
-Steve
(who does now work at Hortonworks, but is actually working on the SmartFrog
source tree this w/end using it to bring up a Hadoop pseudo cluster on the
linux laptop so that I can submit groovy MR jobs to it from my new laptop
and so get my berlin buzzwords talk finished)
(**) The solution here is actually to provide not the VM image, but the
metadata to create the entire VM image, and hand off the work to LinuxCOE,
http://linuxcoe.sourceforge.net/ , which you can see live at
http://www.pro.instalinux.com/cgi-bin/coe_bootimage.cgi
"creating VMs by hand is like statically linking C++ binaries yourself"
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Greg Stein <gs...@gmail.com>.
On May 7, 2012 11:57 AM, "Owen O'Malley" <om...@apache.org> wrote:
>
> On Sat, May 5, 2012 at 2:13 AM, Jukka Zitting <ju...@gmail.com>
wrote:
>
> > Now (again IIUC) the interesting bit is whether it's better for BigTop
> > to be repackaging and -distributing upstream components by itself, or
> > if it would in fact be better for BigTop to simply provide something
> > like "bigtop-x.y.rpm" and "bigtop-x.y.deb" packages that just declare
> > dependencies to specific, integration-tested versions of upstream
> > packages.
>
> It would be much better for Bigtop to work with the Apache dev
> communities that already exist for each of the projects rather than
> releasing incompatible artifacts.
As long as Bigtop is using published releases (rather than trunk, say),
then it is not "incompatible".
Downstream packagers are going to do all kinds of things with Apache
packages. Just something to deal with.
"Better" to work with, but not required. Hadoop can also pull, just like
any other project finding good ideas in the wild and incorporating them
back into their project.
The IPMC and the mentors should be guiding Bigtop in the social aspect:
respect other people and other projects. There aren't technical rules that
are being skipped, that I'm aware of, but maybe some social patterns that
could be improved.
Side note: Apache Subversion is actually discussing *removing* our .rpm and
.deb package specs. Downstream users all have different ones anyway, custom
to their distro. Nobody really uses ours, so they are growing stale. Of
course, Subversion is at a very mature point, relatively, and every distro
carries it. We no longer need to "seed" these specs. But the point I'm
trying to make is that distros are all going to have their own version.
Upstream needs to accept that, and work accordingly.
(if/when downstream starts applying (lots of) source patches, then problems
arise... dunno if that is an issue here)
> It creates a lot of confusion that
> the Bigtop Hadoop rpms are labelled as "hadoop-*.rpm" and yet are
> significantly different from the Hadoop rpms that are produced by the
> Hadoop project. Even more troubling is that the Bigtop distro is
> starting to change the interaction with the user to the projects.
If the users prefer Bigtop RPMs, then maybe Hadoop has something to learn.
But this isn't truly a problem.
> > To do this, BigTop would need to work with the upstream projects to
> > help them produce the appropriate deployment packages as a part of
> > their normal release processes. And BigTop could also team up with
> > Infrastructure to maintain the kind of repository structure and
> > download service expected by deployment tools like yum and apt, a bit
> > like what Maven projects have in https://repository.apache.org/.
Concur. /dist is kind of a weird place for the supporting artifacts, as
that feeds into the mirror system. We don't want to be a central
single-source repository either, for bandwidth purposes. ... something for
Bigtop to work on with Infra.
Cheers,
-g
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Owen O'Malley <om...@apache.org>.
On Sat, May 5, 2012 at 2:13 AM, Jukka Zitting <ju...@gmail.com> wrote:
> Now (again IIUC) the interesting bit is whether it's better for BigTop
> to be repackaging and -distributing upstream components by itself, or
> if it would in fact be better for BigTop to simply provide something
> like "bigtop-x.y.rpm" and "bigtop-x.y.deb" packages that just declare
> dependencies to specific, integration-tested versions of upstream
> packages.
It would be much better for Bigtop to work with the Apache dev
communities that already exist for each of the projects rather than
releasing incompatible artifacts. It creates a lot of confusion that
the Bigtop Hadoop rpms are labelled as "hadoop-*.rpm" and yet are
significantly different from the Hadoop rpms that are produced by the
Hadoop project. Even more troubling is that the Bigtop distro is
starting to change the interaction with the user to the projects.
> To do this, BigTop would need to work with the upstream projects to
> help them produce the appropriate deployment packages as a part of
> their normal release processes. And BigTop could also team up with
> Infrastructure to maintain the kind of repository structure and
> download service expected by deployment tools like yum and apt, a bit
> like what Maven projects have in https://repository.apache.org/.
I asked Roman to do this and he decided not to. Certainly Hadoop is
interested in having the improvements made in their rpms and debs.
-- Owen
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
On Sat, May 5, 2012 at 1:34 AM, Alan Gates <ga...@hortonworks.com> wrote:
> My question here was whether this concept of convenience binaries
> should extend beyond ASF owned packages. I realize that many existing
> convenience binaries contain non-ASF jars, etc. But taking the next
> step of explicitly distributing non-ASF binaries on their own concerns me.
The normal guidelines of what kind of third-party bits can be
redistributed by Apache projects are in
http://www.apache.org/legal/resolved.html. As long as all the
components included in a "BigTop x.y" installation meet those
guidelines or are system dependencies like "Fedora 16" that don't come
form the ASF, then the ASF policy side of things should be fine.
A somewhat similar example from another domain is Apache Tika, which
binds together a lot of upstream libraries from both within and
outside the ASF and makes them available as a single integrated and
tested package. AFAIUI the main difference here is that unlike in
Tika, BigTop doesn't have a programming API for accessing the included
components. Instead, IIUC, the "BigTop API" is a deployment/testing
one with stuff like "yum install", etc.
Now (again IIUC) the interesting bit is whether it's better for BigTop
to be repackaging and -distributing upstream components by itself, or
if it would in fact be better for BigTop to simply provide something
like "bigtop-x.y.rpm" and "bigtop-x.y.deb" packages that just declare
dependencies to specific, integration-tested versions of upstream
packages.
To do this, BigTop would need to work with the upstream projects to
help them produce the appropriate deployment packages as a part of
their normal release processes. And BigTop could also team up with
Infrastructure to maintain the kind of repository structure and
download service expected by deployment tools like yum and apt, a bit
like what Maven projects have in https://repository.apache.org/.
This is in fact a bit like what Tika has recently been considering
(see http://markmail.org/message/wj4xkoax2ojnqlht) with it's upstream
components. Instead of repackaging and -distributing them directly as
a part of a Tika release, we're looking at ways to add the required
extra bits to the upstream releases so that Tika could just consume
them as normal dependencies.
>> * In that case there might still be a role for BigTop to provide a
>> central repository for such easily consumable upstream releases. This
>> would be somewhat similar to the discussions that took place a few
>> years ago about whether and how the ASF could host something like the
>> central Maven repository.
>
> Do you know what list that discussion took place on and a general time frame?
> Reading through that would be very helpful for my thinking on this topic.
See January 2007 on board@ and infrastructure@.
BR,
Jukka Zitting
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Alan Gates <ga...@hortonworks.com>.
Jukka,
Thanks for your response, this is very helpful. I have a couple of questions/clarifications inlined.
On May 4, 2012, at 12:52 PM, Jukka Zitting wrote:
>
>
> * As far as I can tell from the discussion, the BigTop repos directory
> [1] doesn't neatly fit into either of the above categories. I guess
> the key question here is whether the purpose of BigTop is to be a
> particular, tested combination of upstream projects or rather a tool
> for testing and building such combinations. (Or perhaps something else
> entirely?)
>
> * If the former, then each subdirectory of [1] falls fairly
> conveniently into the traditional concept of convenience binaries
> built from the source release. The only extra thing you'd need is a
> proper set of license metadata and signatures for the binaries.
My question here was whether this concept of convenience binaries should extend beyond ASF owned packages. I realize that many existing convenience binaries contain non-ASF jars, etc. But taking the next step of explicitly distributing non-ASF binaries on their own concerns me.
>
> * If the latter, it seems to me that it isn't BigTop that should be
> distributing the packages in [1]. Instead each upstream project should
> using BigTop as a tool to produce such packages as a part of their own
> release processes.
>
> * In that case there might still be a role for BigTop to provide a
> central repository for such easily consumable upstream releases. This
> would be somewhat similar to the discussions that took place a few
> years ago about whether and how the ASF could host something like the
> central Maven repository.
Do you know what list that discussion took place on and a general time frame? Reading through that would be very helpful for my thinking on this topic.
Alan.
>
> [1] http://www.apache.org/dist/incubator/bigtop/bigtop-0.3.0-incubating/repos/
>
> BR,
>
> Jukka Zitting
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
I don't understand the BigTop use cases and release model in too much
detail to have very specific or hard opinions on this, but here's a
few high-level observations that hopefully are useful for this
discussion:
* AFAICT there's no immediate release that's being blocked by this
discussion, so everyone can calm down. An issue was brought up, it's
being discussed and I'm sure we'll soon enough have a solution that
everyone is happy about.
* It sounds like BigTop is doing something that few Apache projects
have done before. Thus it's fine to question whether and to what
extent existing rules apply. However, at the same time it's good to
acknowledge that new rules and consensus on a new interpretation of
existing rules may be needed for something like this. It's natural for
this process to take some time and involve some misunderstandings
along the way.
* The convenience binaries many projects are providing are normally
the result of building the respective source release (together with
any required third party dependencies). As a general rule it should be
possible for anyone to reproduce equivalent binaries by following the
build instructions included in the source release.
* As a recent addition, some projects have started providing also
convenience packages containing such dependencies required by the
source build as described above. In both cases the contents of all
such binary packages should be properly signed and contain appropriate
LICENSE and NOTICE files.
* As far as I can tell from the discussion, the BigTop repos directory
[1] doesn't neatly fit into either of the above categories. I guess
the key question here is whether the purpose of BigTop is to be a
particular, tested combination of upstream projects or rather a tool
for testing and building such combinations. (Or perhaps something else
entirely?)
* If the former, then each subdirectory of [1] falls fairly
conveniently into the traditional concept of convenience binaries
built from the source release. The only extra thing you'd need is a
proper set of license metadata and signatures for the binaries.
* If the latter, it seems to me that it isn't BigTop that should be
distributing the packages in [1]. Instead each upstream project should
using BigTop as a tool to produce such packages as a part of their own
release processes.
* In that case there might still be a role for BigTop to provide a
central repository for such easily consumable upstream releases. This
would be somewhat similar to the discussions that took place a few
years ago about whether and how the ASF could host something like the
central Maven repository.
[1] http://www.apache.org/dist/incubator/bigtop/bigtop-0.3.0-incubating/repos/
BR,
Jukka Zitting
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
I don't understand the BigTop use cases and release model in too much
detail to have very specific or hard opinions on this, but here's a
few high-level observations that hopefully are useful for this
discussion:
* AFAICT there's no immediate release that's being blocked by this
discussion, so everyone can calm down. An issue was brought up, it's
being discussed and I'm sure we'll soon enough have a solution that
everyone is happy about.
* It sounds like BigTop is doing something that few Apache projects
have done before. Thus it's fine to question whether and to what
extent existing rules apply. However, at the same time it's good to
acknowledge that new rules and consensus on a new interpretation of
existing rules may be needed for something like this. It's natural for
this process to take some time and involve some misunderstandings
along the way.
* The convenience binaries many projects are providing are normally
the result of building the respective source release (together with
any required third party dependencies). As a general rule it should be
possible for anyone to reproduce equivalent binaries by following the
build instructions included in the source release.
* As a recent addition, some projects have started providing also
convenience packages containing such dependencies required by the
source build as described above. In both cases the contents of all
such binary packages should be properly signed and contain appropriate
LICENSE and NOTICE files.
* As far as I can tell from the discussion, the BigTop repos directory
[1] doesn't neatly fit into either of the above categories. I guess
the key question here is whether the purpose of BigTop is to be a
particular, tested combination of upstream projects or rather a tool
for testing and building such combinations. (Or perhaps something else
entirely?)
* If the former, then each subdirectory of [1] falls fairly
conveniently into the traditional concept of convenience binaries
built from the source release. The only extra thing you'd need is a
proper set of license metadata and signatures for the binaries.
* If the latter, it seems to me that it isn't BigTop that should be
distributing the packages in [1]. Instead each upstream project should
using BigTop as a tool to produce such packages as a part of their own
release processes.
* In that case there might still be a role for BigTop to provide a
central repository for such easily consumable upstream releases. This
would be somewhat similar to the discussions that took place a few
years ago about whether and how the ASF could host something like the
central Maven repository.
[1] http://www.apache.org/dist/incubator/bigtop/bigtop-0.3.0-incubating/repos/
BR,
Jukka Zitting
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Owen O'Malley <om...@apache.org>.
On Mon, Apr 30, 2012 at 9:39 PM, Roman Shaposhnik <rv...@apache.org> wrote:
> On Mon, Apr 30, 2012 at 9:13 PM, Eric Baldeschwieler
> <er...@hortonworks.com> wrote:
>> So you are suggesting expanding the charter to include projects not hosted at Apache?
>
> I don't think this is what Bruno suggested. Personally I can't find
> any reference in Bigtop
> charter that restricts it to Projects that belong to Apache Software
> Foundation.
As a mentor of the Bigtop project, I don't see it as acceptable for an
Apache project to distribute binaries of non-Apache software. If the
owners of the Hue project decide to donate it to Apache and it had
been released by Apache, then it would be acceptable. I'm strictly -1
on releasing any version of Bigtop with Hue or any other non-Apache
software as part of the release.
-- Owen
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Eric Baldeschwieler <er...@hortonworks.com>.
Hi Roman,
Reasonable people could differ on the interpretation of the paragraph I quoted, but until now my understanding of the aim of BigTop was to package Apache projects and that has been what BigTop has done.
I take it from your response that you do want to extend (or clarify) the charter of BigTop to include packaging of code that has not been contributed to Apache?
It seems like this does merit discussion what do others think?
E14
PS Here is the incubation proposal again:
http://mail-archives.apache.org/mod_mbox/incubator-general/201106.mbox/%3CBANLkTimriyVS5G5MAKLQviNAUZ9H6S5hWw@mail.gmail.com%3E
> Build, packaging and integration test code that depends upon official
> releases of the Apache Hadoop-related projects (HDFS, MapReduce,
> HBase, Hive, Pig, ZooKeeper, etc...) will be developed and released by
> this project. As bugs and other issues are found we expect these to be
> fixed upstream.
On Apr 30, 2012, at 9:39 PM, Roman Shaposhnik wrote:
> On Mon, Apr 30, 2012 at 9:13 PM, Eric Baldeschwieler
> <er...@hortonworks.com> wrote:
>> So you are suggesting expanding the charter to include projects not hosted at Apache?
>
> I don't think this is what Bruno suggested. Personally I can't find
> any reference in Bigtop
> charter that restricts it to Projects that belong to Apache Software
> Foundation. Honestly,
> now that I look at "Apache Hadoop-related projects" statement I
> realize that the way
> I parse is it this "(Apache Hadoop)-related projects". IOW, to my eyes
> "Apache" in that
> statement is not a free floating noun but part of the correct way to
> reference Hadoop project
> in an official document: "Apache Hadoop".
>
> That said, I can see how this can be confusing to somebody not privied
> to the more
> official monikers of Apache projects (I can relate -- I still have to
> make a mental
> effort to not forget adding [incubating] every time I mention Bigtop
> in an official context).
>
> So if you think a clarification is in order -- please suggest the
> wording that might work.
>
>> This is a new precedent and seems like it merits discussion.
>
> Sure. Bigtop is a 100% community driven project and you're more then
> welcome to join
> our community (we actually have a couple of cool projects that we can suggest to
> work on -- we always need extra help especially for improving our
> integration testing
> story).
>
> Now, in order to familiarize yourself with the project and where it is
> going it would be
> helpful to subscribe yourself to the development mailing list. On top
> of that, our wiki,
> while not ideal can, help somewhat.
>
> That's pretty much all there is to it. Follow the list/JIRAs (the Hue
> JIRA has existed
> for a couple of weeks now) and feel free to tap into the community's opinion via
> marking messages as [DISCUSS] or [VOTE].
>
> Hope this helps.
>
> Thanks,
> Roman.
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Owen O'Malley <om...@apache.org>.
On Mon, Apr 30, 2012 at 9:39 PM, Roman Shaposhnik <rv...@apache.org> wrote:
> On Mon, Apr 30, 2012 at 9:13 PM, Eric Baldeschwieler
> <er...@hortonworks.com> wrote:
>> So you are suggesting expanding the charter to include projects not hosted at Apache?
>
> I don't think this is what Bruno suggested. Personally I can't find
> any reference in Bigtop
> charter that restricts it to Projects that belong to Apache Software
> Foundation.
As a mentor of the Bigtop project, I don't see it as acceptable for an
Apache project to distribute binaries of non-Apache software. If the
owners of the Hue project decide to donate it to Apache and it had
been released by Apache, then it would be acceptable. I'm strictly -1
on releasing any version of Bigtop with Hue or any other non-Apache
software as part of the release.
-- Owen
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Roman Shaposhnik <rv...@apache.org>.
On Mon, Apr 30, 2012 at 9:13 PM, Eric Baldeschwieler
<er...@hortonworks.com> wrote:
> So you are suggesting expanding the charter to include projects not hosted at Apache?
I don't think this is what Bruno suggested. Personally I can't find
any reference in Bigtop
charter that restricts it to Projects that belong to Apache Software
Foundation. Honestly,
now that I look at "Apache Hadoop-related projects" statement I
realize that the way
I parse is it this "(Apache Hadoop)-related projects". IOW, to my eyes
"Apache" in that
statement is not a free floating noun but part of the correct way to
reference Hadoop project
in an official document: "Apache Hadoop".
That said, I can see how this can be confusing to somebody not privied
to the more
official monikers of Apache projects (I can relate -- I still have to
make a mental
effort to not forget adding [incubating] every time I mention Bigtop
in an official context).
So if you think a clarification is in order -- please suggest the
wording that might work.
> This is a new precedent and seems like it merits discussion.
Sure. Bigtop is a 100% community driven project and you're more then
welcome to join
our community (we actually have a couple of cool projects that we can suggest to
work on -- we always need extra help especially for improving our
integration testing
story).
Now, in order to familiarize yourself with the project and where it is
going it would be
helpful to subscribe yourself to the development mailing list. On top
of that, our wiki,
while not ideal can, help somewhat.
That's pretty much all there is to it. Follow the list/JIRAs (the Hue
JIRA has existed
for a couple of weeks now) and feel free to tap into the community's opinion via
marking messages as [DISCUSS] or [VOTE].
Hope this helps.
Thanks,
Roman.
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Eric Baldeschwieler <er...@hortonworks.com>.
So you are suggesting expanding the charter to include projects not hosted at Apache?
This is a new precedent and seems like it merits discussion.
On Apr 30, 2012, at 7:50 PM, Bruno Mahé wrote:
> I am not sure to follow.
> The charter hasn't changed at all. Hue is Apache licensed and related
> to Apache Hadoop. So I don't see any issue.
> It is not different from Apache Hadoop depending on other projects not
> being part of the Apache Foundation. These are just build dependencies.
>
> Restricting ourselves to Apache Foundation projects only would really
> limit the opportunities of Apache Bigtop. The same way if Apache Hadoop
> could only depend on dependencies being part of the Apache Foundation.
>
>
>
> On 04/30/2012 03:30 PM, Eric Baldeschwieler wrote:
>> A question...
>>
>> The stated goal of BigTop is to package "official releases of the Apache Hadoop-related projects". I believe stated policy is to require that only official releases from Apache projects can be packaged, no patches or development branches. Do I have that right? Has Cloudera contributed Hue to Apache or are you suggesting a change to the charter of BigTop to package vendor hosted projects as well as Apache Projects?
>>
>> Thanks,
>>
>> E14
>>
>> PS - Just looked up the incubating proposal:
>>
>> http://mail-archives.apache.org/mod_mbox/incubator-general/201106.mbox/%3CBANLkTimriyVS5G5MAKLQviNAUZ9H6S5hWw@mail.gmail.com%3E
>>
>>> Build, packaging and integration test code that depends upon official
>>> releases of the Apache Hadoop-related projects (HDFS, MapReduce,
>>> HBase, Hive, Pig, ZooKeeper, etc...) will be developed and released by
>>> this project. As bugs and other issues are found we expect these to be
>>> fixed upstream.
>>
>
Re: [DISCUSS] BOM and supported platforms for Bigtop 0.4.0
Posted by Bruno Mahé <bm...@apache.org>.
I am not sure to follow.
The charter hasn't changed at all. Hue is Apache licensed and related
to Apache Hadoop. So I don't see any issue.
It is not different from Apache Hadoop depending on other projects not
being part of the Apache Foundation. These are just build dependencies.
Restricting ourselves to Apache Foundation projects only would really
limit the opportunities of Apache Bigtop. The same way if Apache Hadoop
could only depend on dependencies being part of the Apache Foundation.
On 04/30/2012 03:30 PM, Eric Baldeschwieler wrote:
> A question...
>
> The stated goal of BigTop is to package "official releases of the Apache Hadoop-related projects". I believe stated policy is to require that only official releases from Apache projects can be packaged, no patches or development branches. Do I have that right? Has Cloudera contributed Hue to Apache or are you suggesting a change to the charter of BigTop to package vendor hosted projects as well as Apache Projects?
>
> Thanks,
>
> E14
>
> PS - Just looked up the incubating proposal:
>
> http://mail-archives.apache.org/mod_mbox/incubator-general/201106.mbox/%3CBANLkTimriyVS5G5MAKLQviNAUZ9H6S5hWw@mail.gmail.com%3E
>
>> Build, packaging and integration test code that depends upon official
>> releases of the Apache Hadoop-related projects (HDFS, MapReduce,
>> HBase, Hive, Pig, ZooKeeper, etc...) will be developed and released by
>> this project. As bugs and other issues are found we expect these to be
>> fixed upstream.
>