You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by Roman Shaposhnik <rv...@apache.org> on 2013/08/25 23:57:21 UTC
[VOTE] Bigtop 0.7.0 and Bigtop 0.8.0 BOMs
Hi!
this is a bit of an unusual vote thread, since it votes
on two BOMs at once. The reasons I decided to run
it that way is simple -- I'd like to get us to a closure
on two recent threads: a Bigtop 0.7.0 discussion thread
and a potential Bigtop singularity release. I do believe
that what boms of both releases can influence the
other one -- hence this double VOTE.
Here's my proposal: we do a Bigtop 0.7.0 pretty much
as Cos has proposed with a few tweaks to the BOM
and release the bits around end of Sep 2013. We then
follow with a regular Bigtop 0.8.0 with a few components
undergoing major version changes and release the bits
at around end of Dec. Essentially Bigtop 0.8.0 becomes
our singularity release -- something that we hopefully won't
have to to for another year or so. The downside of this,
of course, is that Bigtop 0.8.0 *may* end up being less
stable, but at least it'll serve as a forcing function for
the integration concerns of all the Hadoop ecosystem projects.
With that in mind here's a formal BOM for Bigtop 0.7.0:
* Zookeeper 3.4.5 (no change from Bigtop 0.6.0)
* Sqoop 1.99.2 (no change from Bigtop 0.6.0)
* Oozie 3.3.2 (no change from Bigtop 0.6.0)
* Mahout 0.7 (no change from Bigtop 0.6.0)
* Tomcat 6.0.36 (no change from Bigtop 0.6.0)
* Datafu 0.0.6 (no change from Bigtop 0.6.0)
* Giraph 1.0.0 (no change from Bigtop 0.6.0)
* Whirr 0.8.2 (no change from Bigtop 0.6.0)
* Pig 0.11.1 (no change from Bigtop 0.6.0)
* Hadoop 2.0.6-alpha
* HBase 0.94.11
* Crunch 0.7.0
* Hive/HCatalog 0.11.0
* Flume 1.4.0
* Hue 2.5.0
* Solr 4.4.0
And here's what would make Bigtop 0.8.0 a singularity one:
* Hadoop 2.1.x-beta
* HBase 0.96.x
* Oozie 4.0.x
* Hue 3.0.x
* Mahout 0.8
of course, there will be few minor version bumps as well,
but what we have to agree in this vote is that we're all
comfortable with the above 5 changing in such a major way
in lock-step.
Thanks,
Roman.
Re: [VOTE] Bigtop 0.7.0 and Bigtop 0.8.0 BOMs
Posted by Roman Shaposhnik <rv...@apache.org>.
Guys, I'm scraping this vote and starting a new one.
Thanks,
Roman.
On Tue, Aug 27, 2013 at 10:33 AM, Roman Shaposhnik <rv...@apache.org> wrote:
> On Tue, Aug 27, 2013 at 8:38 AM, Konstantin Boudnik <co...@apache.org> wrote:
>> You are right Bruno. However, we can make an exception about non-patching the
>> components and include one specific patch for npath to close this issue once
>> and for all. The patch would be quite simple as I imagine and consists of
>> renaming npath to something that Teradata can't bitch about.
>
> I agree -- we do have quite a few ways of this not being an issue. I think I'll
> include 0.11.1 in the BOM proposal and will note that we'd be ok with
> shipping 0.11.0 if 0.11.1 is not ready/not well tested.
>
> Thanks,
> Roman.
Re: [VOTE] Bigtop 0.7.0 and Bigtop 0.8.0 BOMs
Posted by Roman Shaposhnik <rv...@apache.org>.
On Tue, Aug 27, 2013 at 8:38 AM, Konstantin Boudnik <co...@apache.org> wrote:
> You are right Bruno. However, we can make an exception about non-patching the
> components and include one specific patch for npath to close this issue once
> and for all. The patch would be quite simple as I imagine and consists of
> renaming npath to something that Teradata can't bitch about.
I agree -- we do have quite a few ways of this not being an issue. I think I'll
include 0.11.1 in the BOM proposal and will note that we'd be ok with
shipping 0.11.0 if 0.11.1 is not ready/not well tested.
Thanks,
Roman.
Re: [VOTE] Bigtop 0.7.0 and Bigtop 0.8.0 BOMs
Posted by Konstantin Boudnik <co...@apache.org>.
You are right Bruno. However, we can make an exception about non-patching the
components and include one specific patch for npath to close this issue once
and for all. The patch would be quite simple as I imagine and consists of
renaming npath to something that Teradata can't bitch about.
What do you think?
Cos
On Mon, Aug 26, 2013 at 11:35PM, Bruno Mahé wrote:
> On 08/26/2013 09:08 PM, Konstantin Boudnik wrote:
>> we have chatted with Roman about this and the problem is that we don't know
>> the target date for 0.11.1. It might be the end of September or November. The
>> Teradata issue has like three months time frame to be addressed.
>>
>> On Mon, Aug 26, 2013 at 05:55PM, Mark Grover wrote:
>>> To close the loop on Hive 0.11* in Bigtop 0.7, I resurrected the thread on
>>> Hive-dev mailing list and heard back that the 0.11.1 will address the NPath
>>> UDF concerns.
>>> Reference:
>>> http://markmail.org/search/?q=Proposing%20a%200.11.1#query:Proposing%20a%200.11.1+page:1+mid:n3vi4n4pwpqizdmh+state:results
>>>
>>> Consequently, I'd suggest that we include the to-be-released Hive 0.11.1 in
>>> our Bigtop 0.7 release.
>>>
>>> Thanks!
>>>
>>>
>>>
>>>
>>>
>>> On Mon, Aug 26, 2013 at 3:52 PM, Roman Shaposhnik <rv...@apache.org> wrote:
>>>
>>>> On Sun, Aug 25, 2013 at 4:43 PM, Konstantin Boudnik <co...@apache.org>
>>>> wrote:
>>>>> The upcoming version of Spark is 0.8 - should be ready in a bit and is
>>>> already
>>>>> pretty stable right now.
>>>>
>>>> Ok, I'll restart the VOTE. Sorry about missing it.
>>>>
>>>> Thanks,
>>>> Roman.
>>>>
>
>
> I am not sure to follow.
> If we release Apache Bigtop with Apache Hive 0.11, the issue will still
> be there by the 3 month expiration, wouldn't it? Even more so if 0.11.1
> has not been released by then.
>
> Thanks,
> Bruno
Re: [VOTE] Bigtop 0.7.0 and Bigtop 0.8.0 BOMs
Posted by Bruno Mahé <bm...@apache.org>.
On 08/26/2013 09:08 PM, Konstantin Boudnik wrote:
> we have chatted with Roman about this and the problem is that we don't know
> the target date for 0.11.1. It might be the end of September or November. The
> Teradata issue has like three months time frame to be addressed.
>
> On Mon, Aug 26, 2013 at 05:55PM, Mark Grover wrote:
>> To close the loop on Hive 0.11* in Bigtop 0.7, I resurrected the thread on
>> Hive-dev mailing list and heard back that the 0.11.1 will address the NPath
>> UDF concerns.
>> Reference:
>> http://markmail.org/search/?q=Proposing%20a%200.11.1#query:Proposing%20a%200.11.1+page:1+mid:n3vi4n4pwpqizdmh+state:results
>>
>> Consequently, I'd suggest that we include the to-be-released Hive 0.11.1 in
>> our Bigtop 0.7 release.
>>
>> Thanks!
>>
>>
>>
>>
>>
>> On Mon, Aug 26, 2013 at 3:52 PM, Roman Shaposhnik <rv...@apache.org> wrote:
>>
>>> On Sun, Aug 25, 2013 at 4:43 PM, Konstantin Boudnik <co...@apache.org>
>>> wrote:
>>>> The upcoming version of Spark is 0.8 - should be ready in a bit and is
>>> already
>>>> pretty stable right now.
>>>
>>> Ok, I'll restart the VOTE. Sorry about missing it.
>>>
>>> Thanks,
>>> Roman.
>>>
I am not sure to follow.
If we release Apache Bigtop with Apache Hive 0.11, the issue will still
be there by the 3 month expiration, wouldn't it? Even more so if 0.11.1
has not been released by then.
Thanks,
Bruno
Re: [VOTE] Bigtop 0.7.0 and Bigtop 0.8.0 BOMs
Posted by Mark Grover <ma...@apache.org>.
I see, that's a valid point. As much as I would like Hive 0.11.1 to be a
part of Bigtop 0.7, I would personally be ok with including Hive 0.11 in
Bigtop 0.7 if Hive 0.11.1 is not out by that time.
Would it make sense to mention "Hive 0.11.*" in the Bigtop 0.7 BOM to allow
us this flexibility?
Mark
On Mon, Aug 26, 2013 at 9:08 PM, Konstantin Boudnik <co...@apache.org> wrote:
> we have chatted with Roman about this and the problem is that we don't know
> the target date for 0.11.1. It might be the end of September or November.
> The
> Teradata issue has like three months time frame to be addressed.
>
> On Mon, Aug 26, 2013 at 05:55PM, Mark Grover wrote:
> > To close the loop on Hive 0.11* in Bigtop 0.7, I resurrected the thread
> on
> > Hive-dev mailing list and heard back that the 0.11.1 will address the
> NPath
> > UDF concerns.
> > Reference:
> >
> http://markmail.org/search/?q=Proposing%20a%200.11.1#query:Proposing%20a%200.11.1+page:1+mid:n3vi4n4pwpqizdmh+state:results
> >
> > Consequently, I'd suggest that we include the to-be-released Hive 0.11.1
> in
> > our Bigtop 0.7 release.
> >
> > Thanks!
> >
> >
> >
> >
> >
> > On Mon, Aug 26, 2013 at 3:52 PM, Roman Shaposhnik <rv...@apache.org>
> wrote:
> >
> > > On Sun, Aug 25, 2013 at 4:43 PM, Konstantin Boudnik <co...@apache.org>
> > > wrote:
> > > > The upcoming version of Spark is 0.8 - should be ready in a bit and
> is
> > > already
> > > > pretty stable right now.
> > >
> > > Ok, I'll restart the VOTE. Sorry about missing it.
> > >
> > > Thanks,
> > > Roman.
> > >
>
Re: [VOTE] Bigtop 0.7.0 and Bigtop 0.8.0 BOMs
Posted by Konstantin Boudnik <co...@apache.org>.
we have chatted with Roman about this and the problem is that we don't know
the target date for 0.11.1. It might be the end of September or November. The
Teradata issue has like three months time frame to be addressed.
On Mon, Aug 26, 2013 at 05:55PM, Mark Grover wrote:
> To close the loop on Hive 0.11* in Bigtop 0.7, I resurrected the thread on
> Hive-dev mailing list and heard back that the 0.11.1 will address the NPath
> UDF concerns.
> Reference:
> http://markmail.org/search/?q=Proposing%20a%200.11.1#query:Proposing%20a%200.11.1+page:1+mid:n3vi4n4pwpqizdmh+state:results
>
> Consequently, I'd suggest that we include the to-be-released Hive 0.11.1 in
> our Bigtop 0.7 release.
>
> Thanks!
>
>
>
>
>
> On Mon, Aug 26, 2013 at 3:52 PM, Roman Shaposhnik <rv...@apache.org> wrote:
>
> > On Sun, Aug 25, 2013 at 4:43 PM, Konstantin Boudnik <co...@apache.org>
> > wrote:
> > > The upcoming version of Spark is 0.8 - should be ready in a bit and is
> > already
> > > pretty stable right now.
> >
> > Ok, I'll restart the VOTE. Sorry about missing it.
> >
> > Thanks,
> > Roman.
> >
Re: [VOTE] Bigtop 0.7.0 and Bigtop 0.8.0 BOMs
Posted by Mark Grover <ma...@apache.org>.
To close the loop on Hive 0.11* in Bigtop 0.7, I resurrected the thread on
Hive-dev mailing list and heard back that the 0.11.1 will address the NPath
UDF concerns.
Reference:
http://markmail.org/search/?q=Proposing%20a%200.11.1#query:Proposing%20a%200.11.1+page:1+mid:n3vi4n4pwpqizdmh+state:results
Consequently, I'd suggest that we include the to-be-released Hive 0.11.1 in
our Bigtop 0.7 release.
Thanks!
On Mon, Aug 26, 2013 at 3:52 PM, Roman Shaposhnik <rv...@apache.org> wrote:
> On Sun, Aug 25, 2013 at 4:43 PM, Konstantin Boudnik <co...@apache.org>
> wrote:
> > The upcoming version of Spark is 0.8 - should be ready in a bit and is
> already
> > pretty stable right now.
>
> Ok, I'll restart the VOTE. Sorry about missing it.
>
> Thanks,
> Roman.
>
Re: [VOTE] Bigtop 0.7.0 and Bigtop 0.8.0 BOMs
Posted by Roman Shaposhnik <rv...@apache.org>.
On Sun, Aug 25, 2013 at 4:43 PM, Konstantin Boudnik <co...@apache.org> wrote:
> The upcoming version of Spark is 0.8 - should be ready in a bit and is already
> pretty stable right now.
Ok, I'll restart the VOTE. Sorry about missing it.
Thanks,
Roman.
Re: [VOTE] Bigtop 0.7.0 and Bigtop 0.8.0 BOMs
Posted by Konstantin Boudnik <co...@apache.org>.
On Sun, Aug 25, 2013 at 04:35PM, Roman Shaposhnik wrote:
> On Sun, Aug 25, 2013 at 4:29 PM, Konstantin Boudnik <co...@apache.org> wrote:
> > I think 0.7 BOM is missing Spark component that is already a part of the
> > master.
>
> What version are you targeting? Would you feel comfortable releasing
> what's there in Bigtop 0.7.0?
The upcoming version of Spark is 0.8 - should be ready in a bit and is already
pretty stable right now.
The current packaging bits in 0.7.0 are good to go. We'll commit the smoke
tests in the next a couple of days - just fixing some last minute issues.
Cos
Re: [VOTE] Bigtop 0.7.0 and Bigtop 0.8.0 BOMs
Posted by Roman Shaposhnik <ro...@shaposhnik.org>.
On Sun, Aug 25, 2013 at 4:29 PM, Konstantin Boudnik <co...@apache.org> wrote:
> I think 0.7 BOM is missing Spark component that is already a part of the
> master.
What version are you targeting? Would you feel comfortable releasing
what's there in Bigtop 0.7.0?
Thanks,
Roman.
Re: [VOTE] Bigtop 0.7.0 and Bigtop 0.8.0 BOMs
Posted by Konstantin Boudnik <co...@apache.org>.
I think 0.7 BOM is missing Spark component that is already a part of the
master.
Cos
On Sun, Aug 25, 2013 at 02:57PM, Roman Shaposhnik wrote:
> Hi!
>
> this is a bit of an unusual vote thread, since it votes
> on two BOMs at once. The reasons I decided to run
> it that way is simple -- I'd like to get us to a closure
> on two recent threads: a Bigtop 0.7.0 discussion thread
> and a potential Bigtop singularity release. I do believe
> that what boms of both releases can influence the
> other one -- hence this double VOTE.
>
> Here's my proposal: we do a Bigtop 0.7.0 pretty much
> as Cos has proposed with a few tweaks to the BOM
> and release the bits around end of Sep 2013. We then
> follow with a regular Bigtop 0.8.0 with a few components
> undergoing major version changes and release the bits
> at around end of Dec. Essentially Bigtop 0.8.0 becomes
> our singularity release -- something that we hopefully won't
> have to to for another year or so. The downside of this,
> of course, is that Bigtop 0.8.0 *may* end up being less
> stable, but at least it'll serve as a forcing function for
> the integration concerns of all the Hadoop ecosystem projects.
>
> With that in mind here's a formal BOM for Bigtop 0.7.0:
> * Zookeeper 3.4.5 (no change from Bigtop 0.6.0)
> * Sqoop 1.99.2 (no change from Bigtop 0.6.0)
> * Oozie 3.3.2 (no change from Bigtop 0.6.0)
> * Mahout 0.7 (no change from Bigtop 0.6.0)
> * Tomcat 6.0.36 (no change from Bigtop 0.6.0)
> * Datafu 0.0.6 (no change from Bigtop 0.6.0)
> * Giraph 1.0.0 (no change from Bigtop 0.6.0)
> * Whirr 0.8.2 (no change from Bigtop 0.6.0)
> * Pig 0.11.1 (no change from Bigtop 0.6.0)
> * Hadoop 2.0.6-alpha
> * HBase 0.94.11
> * Crunch 0.7.0
> * Hive/HCatalog 0.11.0
> * Flume 1.4.0
> * Hue 2.5.0
> * Solr 4.4.0
>
> And here's what would make Bigtop 0.8.0 a singularity one:
> * Hadoop 2.1.x-beta
> * HBase 0.96.x
> * Oozie 4.0.x
> * Hue 3.0.x
> * Mahout 0.8
> of course, there will be few minor version bumps as well,
> but what we have to agree in this vote is that we're all
> comfortable with the above 5 changing in such a major way
> in lock-step.
>
> Thanks,
> Roman.
Re: [VOTE] Bigtop 0.7.0 and Bigtop 0.8.0 BOMs
Posted by Roman Shaposhnik <rv...@apache.org>.
On Sun, Aug 25, 2013 at 11:44 PM, Bruno Mahé <bm...@apache.org> wrote:
> I see Apache Hive 0.11.0 in the BOM.
> Does that mean it is now safe to include it?
Well, personally I think it is. But then again -- that's
why I started the vote so that folks with a strong
opinions on this (and other) issues have a chance
of saying yay/nay.
Thanks,
Roman.
Re: [VOTE] Bigtop 0.7.0 and Bigtop 0.8.0 BOMs
Posted by Mark Grover <ma...@apache.org>.
Bruno,
The Hive community started a discussion to release Hive 0.11.1 where I
asked the question if that release would address removal of the NPath UDF
from Hive code but we didn't hear back. I will try to revive that thread.
http://markmail.org/search/?q=Proposing%20a%200.11.1#query:Proposing%20a%200.11.1+page:1+mid:gafrgoa2bym37kq3+state:results
Mark
On Sun, Aug 25, 2013 at 11:44 PM, Bruno Mahé <bm...@apache.org> wrote:
> On 08/25/2013 02:59 PM, Roman Shaposhnik wrote:
>
>> On Sun, Aug 25, 2013 at 2:59 PM, Roman Shaposhnik <rv...@apache.org> wrote:
>>
>>> Trigger happy fingers :-(
>>>
>>> I meant to add that the vote will run till noon on Wed 8/28 PST.
>>>
>>
>> +1 (binding)
>>
>> Thanks,
>> Roman.
>>
>>
> I see Apache Hive 0.11.0 in the BOM.
> Does that mean it is now safe to include it?
>
>
> Thanks,
> Bruno
>
Re: [VOTE] Bigtop 0.7.0 and Bigtop 0.8.0 BOMs
Posted by Bruno Mahé <bm...@apache.org>.
On 08/25/2013 02:59 PM, Roman Shaposhnik wrote:
> On Sun, Aug 25, 2013 at 2:59 PM, Roman Shaposhnik <rv...@apache.org> wrote:
>> Trigger happy fingers :-(
>>
>> I meant to add that the vote will run till noon on Wed 8/28 PST.
>
> +1 (binding)
>
> Thanks,
> Roman.
>
I see Apache Hive 0.11.0 in the BOM.
Does that mean it is now safe to include it?
Thanks,
Bruno
Re: [VOTE] Bigtop 0.7.0 and Bigtop 0.8.0 BOMs
Posted by Roman Shaposhnik <rv...@apache.org>.
On Sun, Aug 25, 2013 at 2:59 PM, Roman Shaposhnik <rv...@apache.org> wrote:
> Trigger happy fingers :-(
>
> I meant to add that the vote will run till noon on Wed 8/28 PST.
+1 (binding)
Thanks,
Roman.
Re: [VOTE] Bigtop 0.7.0 and Bigtop 0.8.0 BOMs
Posted by Roman Shaposhnik <rv...@apache.org>.
Trigger happy fingers :-(
I meant to add that the vote will run till noon on Wed 8/28 PST.
On Sun, Aug 25, 2013 at 2:57 PM, Roman Shaposhnik <rv...@apache.org> wrote:
> Hi!
>
> this is a bit of an unusual vote thread, since it votes
> on two BOMs at once. The reasons I decided to run
> it that way is simple -- I'd like to get us to a closure
> on two recent threads: a Bigtop 0.7.0 discussion thread
> and a potential Bigtop singularity release. I do believe
> that what boms of both releases can influence the
> other one -- hence this double VOTE.
>
> Here's my proposal: we do a Bigtop 0.7.0 pretty much
> as Cos has proposed with a few tweaks to the BOM
> and release the bits around end of Sep 2013. We then
> follow with a regular Bigtop 0.8.0 with a few components
> undergoing major version changes and release the bits
> at around end of Dec. Essentially Bigtop 0.8.0 becomes
> our singularity release -- something that we hopefully won't
> have to to for another year or so. The downside of this,
> of course, is that Bigtop 0.8.0 *may* end up being less
> stable, but at least it'll serve as a forcing function for
> the integration concerns of all the Hadoop ecosystem projects.
>
> With that in mind here's a formal BOM for Bigtop 0.7.0:
> * Zookeeper 3.4.5 (no change from Bigtop 0.6.0)
> * Sqoop 1.99.2 (no change from Bigtop 0.6.0)
> * Oozie 3.3.2 (no change from Bigtop 0.6.0)
> * Mahout 0.7 (no change from Bigtop 0.6.0)
> * Tomcat 6.0.36 (no change from Bigtop 0.6.0)
> * Datafu 0.0.6 (no change from Bigtop 0.6.0)
> * Giraph 1.0.0 (no change from Bigtop 0.6.0)
> * Whirr 0.8.2 (no change from Bigtop 0.6.0)
> * Pig 0.11.1 (no change from Bigtop 0.6.0)
> * Hadoop 2.0.6-alpha
> * HBase 0.94.11
> * Crunch 0.7.0
> * Hive/HCatalog 0.11.0
> * Flume 1.4.0
> * Hue 2.5.0
> * Solr 4.4.0
>
> And here's what would make Bigtop 0.8.0 a singularity one:
> * Hadoop 2.1.x-beta
> * HBase 0.96.x
> * Oozie 4.0.x
> * Hue 3.0.x
> * Mahout 0.8
> of course, there will be few minor version bumps as well,
> but what we have to agree in this vote is that we're all
> comfortable with the above 5 changing in such a major way
> in lock-step.
>
> Thanks,
> Roman.