You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@bigtop.apache.org by Daithi O Crualaoich <da...@guardian.co.uk> on 2012/08/22 15:29:36 UTC
bigtop-0.4.0-incubating/repos?
Hello,
I got caught when the link at
http://www.apache.org/dist/incubator/bigtop/stable was moved.
There is a http://www.apache.org/dist/incubator/bigtop/bigtop-0.3.0-incubating/repos
but no http://www.apache.org/dist/incubator/bigtop/bigtop-0.4.0-incubating/repos?
I had a look at using
http://bigtop01.cloudera.org:8080/view/Releases/job/Bigtop-0.4.0/label=precise/lastSuccessfulBuild/artifact/output/bigtop.list
instead but the packages are not signed.
Daithi
Please consider the environment before printing this email.
------------------------------------------------------------------
Visit guardian.co.uk - newspaper of the year
www.guardian.co.uk www.observer.co.uk www.guardiannews.com
On your mobile, visit m.guardian.co.uk or download the Guardian
iPhone app www.guardian.co.uk/iphone and iPad edition www.guardian.co.uk/iPad
Save up to 37% by subscribing to the Guardian and Observer - choose the papers you want and get full digital access.
Visit guardian.co.uk/subscribe
---------------------------------------------------------------------
This e-mail and all attachments are confidential and may also
be privileged. If you are not the named recipient, please notify
the sender and delete the e-mail and all attachments immediately.
Do not disclose the contents to another person. You may not use
the information for any purpose, or store, or copy, it in any way.
Guardian News & Media Limited is not liable for any computer
viruses or other material transmitted with or as part of this
e-mail. You should employ virus checking software.
Guardian News & Media Limited
A member of Guardian Media Group plc
Registered Office
PO Box 68164
Kings Place
90 York Way
London
N1P 2AP
Registered in England Number 908396
Re: bigtop-0.4.0-incubating/repos?
Posted by Daithi O Crualaoich <da...@guardian.co.uk>.
> There was a discussion a couple of month ago that made us reconsider
> publishing binary convenience artifacts on the Apache infrastructure.
> Bigtop 0.4.0 is the first release where we only put the source code out (just
> like any Apache project should). Anybody can build the convenience artifacts
> from the sources or if they so prefer they can use the artifacts hosted on
> the same bigtop01 infrastructure.
Okay. That makes sense.
> I've just pushed the signed repos out and the list/repo pointer files can be
> grabbed from their usual place:
> http://www.apache.org/dist/incubator/bigtop/bigtop-0.4.0-incubating/repos/
> (or from your local mirror once that is updated).
Thanks.
> Either way you want to consume the artifacts -- please let us know your
> experience with Bigtop 0.4.0!
Was very glad to find Bigtop. I haven't had fun installing flume,
zookeeper and some of the other elements in the past so this was
great. The version of Flume isn't so good any more so I might take a
look at FLUME-1404.
Keep up the great work.
Daithi
Please consider the environment before printing this email.
------------------------------------------------------------------
Visit guardian.co.uk - newspaper of the year
www.guardian.co.uk www.observer.co.uk www.guardiannews.com
On your mobile, visit m.guardian.co.uk or download the Guardian
iPhone app www.guardian.co.uk/iphone and iPad edition www.guardian.co.uk/iPad
Save up to 37% by subscribing to the Guardian and Observer - choose the papers you want and get full digital access.
Visit guardian.co.uk/subscribe
---------------------------------------------------------------------
This e-mail and all attachments are confidential and may also
be privileged. If you are not the named recipient, please notify
the sender and delete the e-mail and all attachments immediately.
Do not disclose the contents to another person. You may not use
the information for any purpose, or store, or copy, it in any way.
Guardian News & Media Limited is not liable for any computer
viruses or other material transmitted with or as part of this
e-mail. You should employ virus checking software.
Guardian News & Media Limited
A member of Guardian Media Group plc
Registered Office
PO Box 68164
Kings Place
90 York Way
London
N1P 2AP
Registered in England Number 908396
Re: bigtop-0.4.0-incubating/repos?
Posted by MTG dev <de...@magnatempusgroup.net>.
Sorry, for some email encoding issues.
On Thu, August 23, 2012
11:13 pm, MTG dev wrote:
> Roman,
> we at Magna Tempus Group are working on essentially Bigtop
> distribution of
>
> Hadoop-based data analytics stack. The packages for Fedora
> and Ubuntu are available from our repository here
http://www.magnatempusgroup.net/ftphost/release/MTG-0.3.1/. > More
platforms and components are coming in the near future.
> I was
wondering if the artifacts you have published are an
> Apache only release or other vendors can share their distributions
as
> well using the same entry point? We keep supporting 1.x line of
Hadoop and
> it might be useful for people who are using this version and plan to
stay
> on in for some time.
>
> Please let me know how we can help to share our artifacts
> with the rest of the community using links from the dist.
> With regards,
>
> Alef
> On Thu, August 23, 2012 2:59 pm, Roman
> Shaposhnik wrote:
>
>> Hi Daithi!
>
>>
>
>> On Wed, Aug 22, 2012 at 6:29 AM, Daithi O Crualaoich
>
>> <da...@guardian.co.uk> wrote:
>
>>> Hello,
>
>>>
>
>>> I got caught when the link at
>
>>> http://www.apache.org/dist/incubator/bigtop/stable was
moved.
>
>>
>
>> Yes we did push the Bigtop 0.4.0 out a couple of weeks ago.
>
>> Bigtop 0.4.0 is a the first release of bigtop to be built on
top
>
>> of next generation Hadoop platform (Hadoop 2.0 codeline)
>
>> so depending on your needs you might want to either stay
>
>> with Bigtop 0.3.0 (which is still in place) or embrace the
future
>
>> of YARN ;-)
>
>>
>
>>> There is a
>
>>>
>
http://www.apache.org/dist/incubator/bigtop/bigtop-0.3.0-incubating/repos
>
>>> but no
>
>>>
>
http://www.apache.org/dist/incubator/bigtop/bigtop-0.4.0-incubating/repos?
>
>>>
>
>>> I had a look at using
>
>>>
>
http://bigtop01.cloudera.org:8080/view/Releases/job/Bigtop-0.4.0/label=precise/lastSuccessfulBuild/artifact/output/bigtop.list
>
>>> instead but the packages are not signed.
>
>>
>
>> There was a discussion a couple of month ago that made us
> reconsider
>
>> publishing binary convenience artifacts on the Apache
> infrastructure.
>
>> Bigtop 0.4.0 is the first release where we only put the source
code
> out
>
>> (just
>
>> like any Apache project should). Anybody can build the
convenience
>
>> artifacts
>
>> from the sources or if they so prefer they can use the
artifacts
> hosted on
>
>> the same bigtop01 infrastructure.
>
>>
>
>> I've just pushed the signed repos out and the list/repo pointer
files
> can
>
>> be
>
>> grabbed from their usual place:
>
>>
>
http://www.apache.org/dist/incubator/bigtop/bigtop-0.4.0-incubating/repos/
>
>> (or from your local mirror once that is updated).
>
>>
>
>> Either way you want to consume the artifacts -- please let us
know
> your
>
>> experience with Bigtop 0.4.0!
>
>>
>
>> Thanks,
>
>> Roman.
>
>>
>
Re: bigtop-0.4.0-incubating/repos?
Posted by MTG dev <de...@magnatempusgroup.net>.
Hello Bruno.
I appreciate the detailed explanation. Yes, you right - we have picked 1.0.3
because it was the most advanced 1.x release at the date - I missed that part.
As for the formalities and the procedure - I don't have an educated opinion
on these matters, because I am not a part of ASF in any way. So, I am sure you
are painting a correct picture here. We are going to be offering our support
for BigTop in the form of patches in the future. But I am not sure we can
stick the same release pace or set of comprising components.
Thanks for the information, at any rate.
Good luck!
Alef
Magna Tempus Group
On Wed, Aug 29, 2012 at 11:02PM, Bruno MahИ wrote:
> Hi Alef,
>
> Please see my reply inline.
>
>
> >Bruno,
> >
> >the reason we have put this patch (OOZIE-810) and another one fixing
> >dependencies in the Hadoop (HADOOP-8417) because without these the stack
> >wasn't functional.
> >
> >We could not publish non-working bits - it would completely negate the purpose
> >of it.
> >
>
> I completely understand your need for backporting patches, but Magna
> Tempus Group release contains a different version of Apache Hadoop
> to which HADOOP-8417 only seems to apply to (the ticket only lists
> Apache Hadoop 1.0.3). And Apache Bigtop (incubating) 0.3.0 only
> ships with Apache Hadoop 1.0.1. So this issue may or may not apply.
>
> But I don't recall anyone mentioning on our mailing-lists/ticket
> tracking system any of these issues since the release of Apache
> Bigtop (incubating) [Except on Cos' bom].
> Instead of asking the Apache Bigtop (incubating) community to
> advertise your fork as the convenience artefacts of Apache Bigtop
> (incubating) releases, I would rather recommend to open tickets so
> we can keep track of these issues and help us cut a release. This
> way, we can all help you and you don't have the burden of
> maintaining your own fork.
>
>
>
> >I see your point about not referring a derivatives of BigTop through the
> >Apache dist. Now, if I remember correctly (and I am in no position to lecture
> >people here), Apache project releases consist of source code; binaries are
> >published pretty much as a courtesy of a release manager. If so - and please
> >correct me if I am wrong - then Magna Tempus Group release is equivalent to
> >that of BigTop and only provides convenience artifacts for the consumption of
> >the end users.
> >
>
> Magna Tempus Group releases are not equivalent to that of Apache
> Bigtop (incubating). You are not building from a release of Apache
> Bigtop (incubating), you are building a fork of Apache Bigtop
> (incubating).
> You have components of different versions as well as backported
> patches. Backported patches also do go against Apache Bigtop
> (incubating) practices.
>
>
> >At any rate, it is up to the people in the project to make the decision.
> >
> >Regards,
> > Alef.
> >
>
> This is not really up to the people in the project to make the
> decision since by definition, the convenience artefacts are about
> the release, not random bits.
> See http://www.apache.org/dev/release.html#what :
> "In some cases, binary/bytecode packages are also produced as a
> convenience to users that might not have the appropriate tools to
> build a compiled version of the source. In all such cases, the
> binary/bytecode package must have the same version number as the
> source release and may only add binary/bytecode files that are the
> result of compiling that version of the source code release."
>
>
> Thanks,
> Bruno
Re: bigtop-0.4.0-incubating/repos?
Posted by Bruno Mahé <bm...@apache.org>.
Hi Alef,
Please see my reply inline.
> Bruno,
>
> the reason we have put this patch (OOZIE-810) and another one fixing
> dependencies in the Hadoop (HADOOP-8417) because without these the stack
> wasn't functional.
>
> We could not publish non-working bits - it would completely negate the purpose
> of it.
>
I completely understand your need for backporting patches, but Magna
Tempus Group release contains a different version of Apache Hadoop to
which HADOOP-8417 only seems to apply to (the ticket only lists Apache
Hadoop 1.0.3). And Apache Bigtop (incubating) 0.3.0 only ships with
Apache Hadoop 1.0.1. So this issue may or may not apply.
But I don't recall anyone mentioning on our mailing-lists/ticket
tracking system any of these issues since the release of Apache Bigtop
(incubating) [Except on Cos' bom].
Instead of asking the Apache Bigtop (incubating) community to advertise
your fork as the convenience artefacts of Apache Bigtop (incubating)
releases, I would rather recommend to open tickets so we can keep track
of these issues and help us cut a release. This way, we can all help you
and you don't have the burden of maintaining your own fork.
> I see your point about not referring a derivatives of BigTop through the
> Apache dist. Now, if I remember correctly (and I am in no position to lecture
> people here), Apache project releases consist of source code; binaries are
> published pretty much as a courtesy of a release manager. If so - and please
> correct me if I am wrong - then Magna Tempus Group release is equivalent to
> that of BigTop and only provides convenience artifacts for the consumption of
> the end users.
>
Magna Tempus Group releases are not equivalent to that of Apache Bigtop
(incubating). You are not building from a release of Apache Bigtop
(incubating), you are building a fork of Apache Bigtop (incubating).
You have components of different versions as well as backported patches.
Backported patches also do go against Apache Bigtop (incubating) practices.
> At any rate, it is up to the people in the project to make the decision.
>
> Regards,
> Alef.
>
This is not really up to the people in the project to make the decision
since by definition, the convenience artefacts are about the release,
not random bits.
See http://www.apache.org/dev/release.html#what :
"In some cases, binary/bytecode packages are also produced as a
convenience to users that might not have the appropriate tools to build
a compiled version of the source. In all such cases, the binary/bytecode
package must have the same version number as the source release and may
only add binary/bytecode files that are the result of compiling that
version of the source code release."
Thanks,
Bruno
Re: bigtop-0.4.0-incubating/repos?
Posted by MTG dev <de...@magnatempusgroup.net>.
On Sat, Aug 25, 2012 at 12:22PM, Bruno MahИ wrote:
> On 08/24/2012 05:31 PM, Roman Shaposhnik wrote:
> >Hi!
> >
> >On Thu, Aug 23, 2012 at 11:13 PM, MTG dev <de...@magnatempusgroup.net> wrote:
> >> I was wondering if the artifacts you have published are an
> >>Apache only release or other vendors can share their distributions as
> >>well using the same entry point? We keep supporting 1.x line of Hadoop and
> >>it might be useful for people who are using this version and plan to stay
> >>on in for some time.
> >
> >Personally, I'd be very much in favor of folks stepping in to support
> >that code line.
> >If the upcoming 0.3.1 release could be hosted by you guys I see it as very
> >beneficial to the project.
> >
> >Unless anybody from the community objects there shouldn't be any issue with
> >that as far as I can tell -- you just update the repo/list files and
> >point to your
> >website.
> >
> >Thanks,
> >Roman.
> >
>
> Hi,
>
> I am also all in favor of having and helping community members
> stepping in to help support that branch and even host some
> convenience artefacts.
> But I am a little bit confused by what is exactly being asked.
>
> Is it about having a section on the wiki listing derivatives of
> Apache Bigtop (incubating) or is it about making the dist directory
> pointing to MTG?
>
> The only issue is that it looks like from
> http://www.magnatempusgroup.net/blog/2012/05/25/mtg-release-0-3-1/
> that this distribution is a derivative of Apache Bigtop (incubating)
> and not just convenience artefacts of Apache Bigtop (incubating)
> releases. We cannot advertise derivatives of Apache Bigtop
> (incubating) as convenience artefacts of specific releases of Apache
> Bigtop (incubating).
> http://www.magnatempusgroup.net/ftphost/releases/MTG-0.3.1/fedora16/oozie/oozie-3.1.3-1.fc16.src.rpm
> indicates Oozie is indeed patched.
>
> Derivatives can be listed on the wiki there in any case:
> https://cwiki.apache.org/confluence/display/BIGTOP/Powered+By+Bigtop
> There could also be an entire section dedicated to them if needed.
Bruno,
the reason we have put this patch (OOZIE-810) and another one fixing
dependencies in the Hadoop (HADOOP-8417) because without these the stack
wasn't functional.
We could not publish non-working bits - it would completely negate the purpose
of it.
I see your point about not referring a derivatives of BigTop through the
Apache dist. Now, if I remember correctly (and I am in no position to lecture
people here), Apache project releases consist of source code; binaries are
published pretty much as a courtesy of a release manager. If so - and please
correct me if I am wrong - then Magna Tempus Group release is equivalent to
that of BigTop and only provides convenience artifacts for the consumption of
the end users.
At any rate, it is up to the people in the project to make the decision.
Regards,
Alef.
Re: bigtop-0.4.0-incubating/repos?
Posted by Bruno Mahé <bm...@apache.org>.
On 08/24/2012 05:31 PM, Roman Shaposhnik wrote:
> Hi!
>
> On Thu, Aug 23, 2012 at 11:13 PM, MTG dev <de...@magnatempusgroup.net> wrote:
>> I was wondering if the artifacts you have published are an
>> Apache only release or other vendors can share their distributions as
>> well using the same entry point? We keep supporting 1.x line of Hadoop and
>> it might be useful for people who are using this version and plan to stay
>> on in for some time.
>
> Personally, I'd be very much in favor of folks stepping in to support
> that code line.
> If the upcoming 0.3.1 release could be hosted by you guys I see it as very
> beneficial to the project.
>
> Unless anybody from the community objects there shouldn't be any issue with
> that as far as I can tell -- you just update the repo/list files and
> point to your
> website.
>
> Thanks,
> Roman.
>
Hi,
I am also all in favor of having and helping community members stepping
in to help support that branch and even host some convenience artefacts.
But I am a little bit confused by what is exactly being asked.
Is it about having a section on the wiki listing derivatives of Apache
Bigtop (incubating) or is it about making the dist directory pointing to
MTG?
The only issue is that it looks like from
http://www.magnatempusgroup.net/blog/2012/05/25/mtg-release-0-3-1/ that
this distribution is a derivative of Apache Bigtop (incubating) and not
just convenience artefacts of Apache Bigtop (incubating) releases. We
cannot advertise derivatives of Apache Bigtop (incubating) as
convenience artefacts of specific releases of Apache Bigtop (incubating).
http://www.magnatempusgroup.net/ftphost/releases/MTG-0.3.1/fedora16/oozie/oozie-3.1.3-1.fc16.src.rpm
indicates Oozie is indeed patched.
Derivatives can be listed on the wiki there in any case:
https://cwiki.apache.org/confluence/display/BIGTOP/Powered+By+Bigtop
There could also be an entire section dedicated to them if needed.
Thanks,
Bruno
Re: bigtop-0.4.0-incubating/repos?
Posted by Roman Shaposhnik <rv...@apache.org>.
Hi!
On Thu, Aug 23, 2012 at 11:13 PM, MTG dev <de...@magnatempusgroup.net> wrote:
> I was wondering if the artifacts you have published are an
> Apache only release or other vendors can share their distributions as
> well using the same entry point? We keep supporting 1.x line of Hadoop and
> it might be useful for people who are using this version and plan to stay
> on in for some time.
Personally, I'd be very much in favor of folks stepping in to support
that code line.
If the upcoming 0.3.1 release could be hosted by you guys I see it as very
beneficial to the project.
Unless anybody from the community objects there shouldn't be any issue with
that as far as I can tell -- you just update the repo/list files and
point to your
website.
Thanks,
Roman.
Re: bigtop-0.4.0-incubating/repos?
Posted by MTG dev <de...@magnatempusgroup.net>.
�� Roman,
�� we at Magna Tempus Group are working on essentially Bigtop
distribution of
�� Hadoop-based data analytics stack. The packages for Fedora
and Ubuntu are
�� available from our repository here
�� http://www.magnatempusgroup.net/ftphost/releases/MTG-0.3.1/.
More
�� platforms and components are coming in the near future.
�� I was wondering if the artifacts you have published are an
Apache only
�� release or other vendors can share their distributions as
well using the
�� same entry point? We keep supporting 1.x line of Hadoop and
it might be
�� useful for people who are using this version and plan to stay
on in for
�� some time.
�� Please let me know how we can help to share our artifacts
with the rest of
�� the community using links from the dist.
�� With regards,
���� Alef
On Thu, August 23, 2012 2:59 pm, Roman
Shaposhnik wrote:
> Hi Daithi!
>
> On Wed, Aug 22, 2012 at 6:29 AM, Daithi O Crualaoich
> <da...@guardian.co.uk> wrote:
>> Hello,
>>
>> I got caught when the link at
>> http://www.apache.org/dist/incubator/bigtop/stable was moved.
>
> Yes we did push the Bigtop 0.4.0 out a couple of weeks ago.
> Bigtop 0.4.0 is a the first release of bigtop to be built on top
> of next generation Hadoop platform (Hadoop 2.0 codeline)
> so depending on your needs you might want to either stay
> with Bigtop 0.3.0 (which is still in place) or embrace the future
> of YARN ;-)
>
>> There is a
>>
http://www.apache.org/dist/incubator/bigtop/bigtop-0.3.0-incubating/repos
>> but no
>>
http://www.apache.org/dist/incubator/bigtop/bigtop-0.4.0-incubating/repos?
>>
>> I had a look at using
>>
http://bigtop01.cloudera.org:8080/view/Releases/job/Bigtop-0.4.0/label=precise/lastSuccessfulBuild/artifact/output/bigtop.list
>> instead but the packages are not signed.
>
> There was a discussion a couple of month ago that made us
reconsider
> publishing binary convenience artifacts on the Apache
infrastructure.
> Bigtop 0.4.0 is the first release where we only put the source code
out
> (just
> like any Apache project should). Anybody can build the convenience
> artifacts
> from the sources or if they so prefer they can use the artifacts
hosted on
> the same bigtop01 infrastructure.
>
> I've just pushed the signed repos out and the list/repo pointer files
can
> be
> grabbed from their usual place:
>
http://www.apache.org/dist/incubator/bigtop/bigtop-0.4.0-incubating/repos/
> (or from your local mirror once that is updated).
>
> Either way you want to consume the artifacts -- please let us know
your
> experience with Bigtop 0.4.0!
>
> Thanks,
> Roman.
>
Re: bigtop-0.4.0-incubating/repos?
Posted by Roman Shaposhnik <rv...@apache.org>.
Hi Daithi!
On Wed, Aug 22, 2012 at 6:29 AM, Daithi O Crualaoich
<da...@guardian.co.uk> wrote:
> Hello,
>
> I got caught when the link at
> http://www.apache.org/dist/incubator/bigtop/stable was moved.
Yes we did push the Bigtop 0.4.0 out a couple of weeks ago.
Bigtop 0.4.0 is a the first release of bigtop to be built on top
of next generation Hadoop platform (Hadoop 2.0 codeline)
so depending on your needs you might want to either stay
with Bigtop 0.3.0 (which is still in place) or embrace the future
of YARN ;-)
> There is a http://www.apache.org/dist/incubator/bigtop/bigtop-0.3.0-incubating/repos
> but no http://www.apache.org/dist/incubator/bigtop/bigtop-0.4.0-incubating/repos?
>
> I had a look at using
> http://bigtop01.cloudera.org:8080/view/Releases/job/Bigtop-0.4.0/label=precise/lastSuccessfulBuild/artifact/output/bigtop.list
> instead but the packages are not signed.
There was a discussion a couple of month ago that made us reconsider
publishing binary convenience artifacts on the Apache infrastructure.
Bigtop 0.4.0 is the first release where we only put the source code out (just
like any Apache project should). Anybody can build the convenience artifacts
from the sources or if they so prefer they can use the artifacts hosted on
the same bigtop01 infrastructure.
I've just pushed the signed repos out and the list/repo pointer files can be
grabbed from their usual place:
http://www.apache.org/dist/incubator/bigtop/bigtop-0.4.0-incubating/repos/
(or from your local mirror once that is updated).
Either way you want to consume the artifacts -- please let us know your
experience with Bigtop 0.4.0!
Thanks,
Roman.