You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by Konstantin Boudnik <co...@apache.org> on 2013/07/09 05:50:33 UTC
[DISCUSS] BOM for release 0.7.0 of Bigtop
Guys,
I wanna kick-off the discussion on the content of 0.7.0 BOM
Release 0.6.0 was all about stabilization of the stack and I think we got a
great headway on that. The following components/OS were in the frame of the
discussion:
http://is.gd/H52iVe
My personal take that we need to spend this release cycle working on the
improvements in Bigtop itself: we got enough "technical debts" in the
pipeline that have to be addressed. To name a few:
- testability/test coverage
- test framework
- package improvements
- build improvements (including performance)
In order to be able to deliver a solid stack again yet improve all things
Bigtop I'd like to focus on the latter, hence keeping the former at bay and
limiting the component updates to the bugfix releases only (if warranted). E.g.
Hadoop 2.0.5 or later (stabilization branch of Hadoop 2)
HBase 0.94.9 (update from the Bigtop 0.6.0)
HCatalog 0.5.0 (same as 0.6.0, as well as following...)
Zookeeper 3.4.5
Pig 0.11.1
Hive 0.10.0
Sqoop 2
Oozie 3.3.2
Whirr 0.8.1
Mahout 0.7
Flume 1.3.1
Giraph 0.2.0
Hue 2.2.0
Datafu 0.0.6
Solr 4.2.1
Crunch 0.5.0
Tomcat 6.0.36
Spark 0.7.3 (it has been in the queue for a long time)
Also, I'd suggest to keep the same set of OSes as last time:
CentOS/RHEL5
CentOS/RHEL6
SLES11
Ubuntu 12.04 (LTS)
Fedora 18
OpenSUSE 12.3
Ubuntu 12.10
To reiterate, with a known stable version of the stack we can safely focus on
the improvements to the framework and the overall system usability.
Please jump on the discussion Also, I have opened up the following JIRA to
track the BOM update https://issues.apache.org/jira/browse/BIGTOP-1023
Thanks,
Cos
Re: [DISCUSS] BOM for release 0.7.0 of Bigtop
Posted by Konstantin Boudnik <co...@apache.org>.
On that note I would be happy to add OpenJDK7 into the mix.
Debian addition looks reasonable. And I'd look for Peter Linnell's input on
SUSE side.
I am not familiar with the scope of the updates in Hue, Whirr, nor Flume. So,
if someone can chime in on the impact of these updates it will be great!
As for Parquet: this seems like an early stage non-Apache project, hence I
would be hesitant to bring it into the stabilization release. But as always -
this should be a community decision.
Cos
On Tue, Jul 09, 2013 at 09:47AM, Sean Mackrory wrote:
> Without wanting to detract from the spirit of focussing on system
> stability, I'd like to suggest a few changes I think it's time we at least
> discuss seriously:
>
> JDKs: I've seen a lot of people ask about JDK 7. Perhaps time to add
> support for Oracle JDK 7? It's working pretty well in my experience, and
> although it's less tested upstream, the only JDK we officially support is
> officially EOL, so we're not exactly in a good position now IMO.
>
> Debian 7 has also been out for a while, and I think we should do at least
> one release on it. It's likely very little work but I think there's value
> in certifying the stack will work well there. (On the topic of OS's - are
> we specifically talking SP3 of SLES 11?). I don't feel strongly on this,
> but I'm just curious if there's a reason you're suggesting staying with
> 12.10 and not 13.04 - other than wanting less change in this release?
> Again, I hardly have an opinion on that one.
>
> Other components that have recently had releases that I don't consider to
> impact the
> - Hue 2.4.0
> - Whirr 0.8.2
> - Flume 1.4.0
>
> There's also been a ticket to package Avro for a long time and I'd like to
> get to that soon. Perhaps Parquet as well? Although like Phoenix and
> DataFu, I would suggest doing just the libraries for now, not all the CLI
> tools.
>
> Again - I don't mean to take away from the focus on stability, but I also
> don't think we shouldn't stretch to stay up to date either.
>
> +1 to everything else as suggested, however.
>
>
> On Tue, Jul 9, 2013 at 8:59 AM, Andrew Purtell <ap...@apache.org> wrote:
> > Would you be willing to consider Phoenix, only BIGTOP-993? Installing the
> > package produced by 993 only drops a library for HBase into
> > /usr/lib/phoenix, essentially the same relationship between the DataFu
> > package and Pig. There is follow up work that is more ambitious, for
> > example BIGTOP-1007, but that is not required by any means and could come
> > in if/whenever you are comfortable with it.
> >
> >
> > On Mon, Jul 8, 2013 at 8:50 PM, Konstantin Boudnik <co...@apache.org> wrote:
> >
> >> Guys,
> >>
> >> I wanna kick-off the discussion on the content of 0.7.0 BOM
> >> Release 0.6.0 was all about stabilization of the stack and I think we
> got a
> >> great headway on that. The following components/OS were in the frame of
> the
> >> discussion:
> >> http://is.gd/H52iVe
> >>
> >> My personal take that we need to spend this release cycle working on the
> >> improvements in Bigtop itself: we got enough "technical debts" in the
> >> pipeline that have to be addressed. To name a few:
> >> - testability/test coverage
> >> - test framework
> >> - package improvements
> >> - build improvements (including performance)
> >>
> >> In order to be able to deliver a solid stack again yet improve all things
> >> Bigtop I'd like to focus on the latter, hence keeping the former at bay
> and
> >> limiting the component updates to the bugfix releases only (if
> warranted).
> >> E.g.
> >>
> >> Hadoop 2.0.5 or later (stabilization branch of Hadoop 2)
> >> HBase 0.94.9 (update from the Bigtop 0.6.0)
> >> HCatalog 0.5.0 (same as 0.6.0, as well as following...)
> >> Zookeeper 3.4.5
> >> Pig 0.11.1
> >> Hive 0.10.0
> >> Sqoop 2
> >> Oozie 3.3.2
> >> Whirr 0.8.1
> >> Mahout 0.7
> >> Flume 1.3.1
> >> Giraph 0.2.0
> >> Hue 2.2.0
> >> Datafu 0.0.6
> >> Solr 4.2.1
> >> Crunch 0.5.0
> >> Tomcat 6.0.36
> >> Spark 0.7.3 (it has been in the queue for a long time)
> >>
> >> Also, I'd suggest to keep the same set of OSes as last time:
> >>
> >> CentOS/RHEL5
> >> CentOS/RHEL6
> >> SLES11
> >> Ubuntu 12.04 (LTS)
> >> Fedora 18
> >> OpenSUSE 12.3
> >> Ubuntu 12.10
> >>
> >> To reiterate, with a known stable version of the stack we can safely
> focus
> >> on
> >> the improvements to the framework and the overall system usability.
> >>
> >> Please jump on the discussion Also, I have opened up the following JIRA
> to
> >> track the BOM update https://issues.apache.org/jira/browse/BIGTOP-1023
> >>
> >> Thanks,
> >> Cos
> >>
> >>
> >
> >
> > --
> > Best regards,
> >
> > - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
Re: [DISCUSS] BOM for release 0.7.0 of Bigtop
Posted by Bruno Mahé <bm...@apache.org>.
On 07/09/2013 10:40 AM, Konstantin Boudnik wrote:
> Bruno,
>
> just to clarify my stance of 'stability': it is more about stability of the
> Bigtop as a framework than a stability of the stack.
>
> I am not sure we have resources to do maintenance releases at this point. May
> be it is just me.
>
> Cos
>
Just to confirm we are on the same page:
Do you mean "stability of the Bigtop as a framework" as in the stability
of the APIs of its components?
Thanks,
Bruno
Re: [DISCUSS] BOM for release 0.7.0 of Bigtop
Posted by Peter Linnell <pl...@apache.org>.
On Tue, 9 Jul 2013 10:40:43 -0700
Konstantin Boudnik <co...@apache.org> wrote:
> Bruno,
>
> just to clarify my stance of 'stability': it is more about stability
> of the Bigtop as a framework than a stability of the stack.
>
> I am not sure we have resources to do maintenance releases at this
> point. May be it is just me.
>
> Cos
>
> On Tue, Jul 09, 2013 at 10:10AM, Bruno MahИ wrote:
> > On 07/09/2013 09:47 AM, Sean Mackrory wrote:
> >> Without wanting to detract from the spirit of focussing on system
> >> stability, I'd like to suggest a few changes I think it's time we
> >> at least discuss seriously:
> >>
> >> JDKs: I've seen a lot of people ask about JDK 7. Perhaps time to
> >> add support for Oracle JDK 7? It's working pretty well in my
> >> experience, and although it's less tested upstream, the only JDK
> >> we officially support is officially EOL, so we're not exactly in a
> >> good position now IMO.
> >>
> >> Debian 7 has also been out for a while, and I think we should do
> >> at least one release on it. It's likely very little work but I
> >> think there's value in certifying the stack will work well there.
> >> (On the topic of OS's - are we specifically talking SP3 of SLES
> >> 11?). I don't feel strongly on this, but I'm just curious if
> >> there's a reason you're suggesting staying with 12.10 and not
> >> 13.04 - other than wanting less change in this release? Again, I
> >> hardly have an opinion on that one.
> >>
> >> Other components that have recently had releases that I don't
> >> consider to impact the
> >> - Hue 2.4.0
> >> - Whirr 0.8.2
> >> - Flume 1.4.0
> >>
> >> There's also been a ticket to package Avro for a long time and I'd
> >> like to get to that soon. Perhaps Parquet as well? Although like
> >> Phoenix and DataFu, I would suggest doing just the libraries for
> >> now, not all the CLI tools.
> >>
> >> Again - I don't mean to take away from the focus on stability, but
> >> I also don't think we shouldn't stretch to stay up to date either.
> >>
> >> +1 to everything else as suggested, however.
> >>
> >>
> >> On Tue, Jul 9, 2013 at 8:59 AM, Andrew Purtell
> >> <ap...@apache.org> wrote:
> >>> Would you be willing to consider Phoenix, only BIGTOP-993?
> >>> Installing the package produced by 993 only drops a library for
> >>> HBase into /usr/lib/phoenix, essentially the same relationship
> >>> between the DataFu package and Pig. There is follow up work that
> >>> is more ambitious, for example BIGTOP-1007, but that is not
> >>> required by any means and could come in if/whenever you are
> >>> comfortable with it.
> >>>
> >>>
> >>> On Mon, Jul 8, 2013 at 8:50 PM, Konstantin Boudnik
> >>> <co...@apache.org> wrote:
> >>>
> >>>> Guys,
> >>>>
> >>>> I wanna kick-off the discussion on the content of 0.7.0 BOM
> >>>> Release 0.6.0 was all about stabilization of the stack and I
> >>>> think we
> >> got a
> >>>> great headway on that. The following components/OS were in the
> >>>> frame of
> >> the
> >>>> discussion:
> >>>> http://is.gd/H52iVe
> >>>>
> >>>> My personal take that we need to spend this release cycle
> >>>> working on the improvements in Bigtop itself: we got enough
> >>>> "technical debts" in the pipeline that have to be addressed. To
> >>>> name a few:
> >>>> - testability/test coverage
> >>>> - test framework
> >>>> - package improvements
> >>>> - build improvements (including performance)
> >>>>
> >>>> In order to be able to deliver a solid stack again yet improve
> >>>> all things Bigtop I'd like to focus on the latter, hence keeping
> >>>> the former at bay
> >> and
> >>>> limiting the component updates to the bugfix releases only (if
> >> warranted).
> >>>> E.g.
> >>>>
> >>>> Hadoop 2.0.5 or later (stabilization branch of Hadoop 2)
> >>>> HBase 0.94.9 (update from the Bigtop 0.6.0)
> >>>> HCatalog 0.5.0 (same as 0.6.0, as well as following...)
> >>>> Zookeeper 3.4.5
> >>>> Pig 0.11.1
> >>>> Hive 0.10.0
> >>>> Sqoop 2
> >>>> Oozie 3.3.2
> >>>> Whirr 0.8.1
> >>>> Mahout 0.7
> >>>> Flume 1.3.1
> >>>> Giraph 0.2.0
> >>>> Hue 2.2.0
> >>>> Datafu 0.0.6
> >>>> Solr 4.2.1
> >>>> Crunch 0.5.0
> >>>> Tomcat 6.0.36
> >>>> Spark 0.7.3 (it has been in the queue for a long time)
> >>>>
> >>>> Also, I'd suggest to keep the same set of OSes as last time:
> >>>>
> >>>> CentOS/RHEL5
> >>>> CentOS/RHEL6
> >>>> SLES11
> >>>> Ubuntu 12.04 (LTS)
> >>>> Fedora 18
> >>>> OpenSUSE 12.3
> >>>> Ubuntu 12.10
> >>>>
> >>>> To reiterate, with a known stable version of the stack we can
> >>>> safely
> >> focus
> >>>> on
> >>>> the improvements to the framework and the overall system
> >>>> usability.
> >>>>
> >>>> Please jump on the discussion Also, I have opened up the
> >>>> following JIRA
> >> to
> >>>> track the BOM update
> >>>> https://issues.apache.org/jira/browse/BIGTOP-1023
> >>>>
> >>>> Thanks,
> >>>> Cos
> >>>>
> >>>>
> >>>
> >>>
> >>> --
> >>> Best regards,
> >>>
> >>> - Andy
> >>>
> >>> Problems worthy of attack prove their worth by hitting back. -
> >>> Piet Hein (via Tom White)
> >>
> >
> >
> > +1 to what Andrew and Sean said.
> > I would also like to add:
> > * I would also like to keep the door open for Apache Gora to come
> > in. Someone is already working on it
> > * Fedora 19 is out and maybe 20 will be out by the time we release
> > 0.7.0
> >
> > With regard to stability, if included all the changes listed so far
> > seem a little bit too much, what about dedicating a 0.6.1 to
> > stability so 0.7.0 can include all the shiny things listed above? A
> > 0.6.1 would enable us to be more strict with the
> > patches/fixes/updates being included as well as signaling a more
> > stable version to users.
> >
> >
> > Thanks,
> > Bruno
I suggest we should target SLES 11 Sp3 which is just out. SP2 will only
have 6 months of official support from now.
As for openSUSE 12.3, yes, with the caveat we need get a reliable AWS
image for Jenkins. I have started on this, but got sidetracked by real
work TM.
Also, I am in agreement about Java 7. From what I understand the delta
from the Oracle 7 and openJDK 7 is much smaller.
+1 for Phoenix, as well.
Thanks,
Peter
Thanks,
PEter
Re: [DISCUSS] BOM for release 0.7.0 of Bigtop
Posted by Konstantin Boudnik <co...@apache.org>.
Bruno,
just to clarify my stance of 'stability': it is more about stability of the
Bigtop as a framework than a stability of the stack.
I am not sure we have resources to do maintenance releases at this point. May
be it is just me.
Cos
On Tue, Jul 09, 2013 at 10:10AM, Bruno MahИ wrote:
> On 07/09/2013 09:47 AM, Sean Mackrory wrote:
>> Without wanting to detract from the spirit of focussing on system
>> stability, I'd like to suggest a few changes I think it's time we at least
>> discuss seriously:
>>
>> JDKs: I've seen a lot of people ask about JDK 7. Perhaps time to add
>> support for Oracle JDK 7? It's working pretty well in my experience, and
>> although it's less tested upstream, the only JDK we officially support is
>> officially EOL, so we're not exactly in a good position now IMO.
>>
>> Debian 7 has also been out for a while, and I think we should do at least
>> one release on it. It's likely very little work but I think there's value
>> in certifying the stack will work well there. (On the topic of OS's - are
>> we specifically talking SP3 of SLES 11?). I don't feel strongly on this,
>> but I'm just curious if there's a reason you're suggesting staying with
>> 12.10 and not 13.04 - other than wanting less change in this release?
>> Again, I hardly have an opinion on that one.
>>
>> Other components that have recently had releases that I don't consider to
>> impact the
>> - Hue 2.4.0
>> - Whirr 0.8.2
>> - Flume 1.4.0
>>
>> There's also been a ticket to package Avro for a long time and I'd like to
>> get to that soon. Perhaps Parquet as well? Although like Phoenix and
>> DataFu, I would suggest doing just the libraries for now, not all the CLI
>> tools.
>>
>> Again - I don't mean to take away from the focus on stability, but I also
>> don't think we shouldn't stretch to stay up to date either.
>>
>> +1 to everything else as suggested, however.
>>
>>
>> On Tue, Jul 9, 2013 at 8:59 AM, Andrew Purtell <ap...@apache.org> wrote:
>>> Would you be willing to consider Phoenix, only BIGTOP-993? Installing the
>>> package produced by 993 only drops a library for HBase into
>>> /usr/lib/phoenix, essentially the same relationship between the DataFu
>>> package and Pig. There is follow up work that is more ambitious, for
>>> example BIGTOP-1007, but that is not required by any means and could come
>>> in if/whenever you are comfortable with it.
>>>
>>>
>>> On Mon, Jul 8, 2013 at 8:50 PM, Konstantin Boudnik <co...@apache.org> wrote:
>>>
>>>> Guys,
>>>>
>>>> I wanna kick-off the discussion on the content of 0.7.0 BOM
>>>> Release 0.6.0 was all about stabilization of the stack and I think we
>> got a
>>>> great headway on that. The following components/OS were in the frame of
>> the
>>>> discussion:
>>>> http://is.gd/H52iVe
>>>>
>>>> My personal take that we need to spend this release cycle working on the
>>>> improvements in Bigtop itself: we got enough "technical debts" in the
>>>> pipeline that have to be addressed. To name a few:
>>>> - testability/test coverage
>>>> - test framework
>>>> - package improvements
>>>> - build improvements (including performance)
>>>>
>>>> In order to be able to deliver a solid stack again yet improve all things
>>>> Bigtop I'd like to focus on the latter, hence keeping the former at bay
>> and
>>>> limiting the component updates to the bugfix releases only (if
>> warranted).
>>>> E.g.
>>>>
>>>> Hadoop 2.0.5 or later (stabilization branch of Hadoop 2)
>>>> HBase 0.94.9 (update from the Bigtop 0.6.0)
>>>> HCatalog 0.5.0 (same as 0.6.0, as well as following...)
>>>> Zookeeper 3.4.5
>>>> Pig 0.11.1
>>>> Hive 0.10.0
>>>> Sqoop 2
>>>> Oozie 3.3.2
>>>> Whirr 0.8.1
>>>> Mahout 0.7
>>>> Flume 1.3.1
>>>> Giraph 0.2.0
>>>> Hue 2.2.0
>>>> Datafu 0.0.6
>>>> Solr 4.2.1
>>>> Crunch 0.5.0
>>>> Tomcat 6.0.36
>>>> Spark 0.7.3 (it has been in the queue for a long time)
>>>>
>>>> Also, I'd suggest to keep the same set of OSes as last time:
>>>>
>>>> CentOS/RHEL5
>>>> CentOS/RHEL6
>>>> SLES11
>>>> Ubuntu 12.04 (LTS)
>>>> Fedora 18
>>>> OpenSUSE 12.3
>>>> Ubuntu 12.10
>>>>
>>>> To reiterate, with a known stable version of the stack we can safely
>> focus
>>>> on
>>>> the improvements to the framework and the overall system usability.
>>>>
>>>> Please jump on the discussion Also, I have opened up the following JIRA
>> to
>>>> track the BOM update https://issues.apache.org/jira/browse/BIGTOP-1023
>>>>
>>>> Thanks,
>>>> Cos
>>>>
>>>>
>>>
>>>
>>> --
>>> Best regards,
>>>
>>> - Andy
>>>
>>> Problems worthy of attack prove their worth by hitting back. - Piet Hein
>>> (via Tom White)
>>
>
>
> +1 to what Andrew and Sean said.
> I would also like to add:
> * I would also like to keep the door open for Apache Gora to come in.
> Someone is already working on it
> * Fedora 19 is out and maybe 20 will be out by the time we release 0.7.0
>
> With regard to stability, if included all the changes listed so far seem
> a little bit too much, what about dedicating a 0.6.1 to stability so
> 0.7.0 can include all the shiny things listed above? A 0.6.1 would
> enable us to be more strict with the patches/fixes/updates being
> included as well as signaling a more stable version to users.
>
>
> Thanks,
> Bruno
Re: [DISCUSS] BOM for release 0.7.0 of Bigtop
Posted by Bruno Mahé <bm...@apache.org>.
On 07/09/2013 09:47 AM, Sean Mackrory wrote:
> Without wanting to detract from the spirit of focussing on system
> stability, I'd like to suggest a few changes I think it's time we at least
> discuss seriously:
>
> JDKs: I've seen a lot of people ask about JDK 7. Perhaps time to add
> support for Oracle JDK 7? It's working pretty well in my experience, and
> although it's less tested upstream, the only JDK we officially support is
> officially EOL, so we're not exactly in a good position now IMO.
>
> Debian 7 has also been out for a while, and I think we should do at least
> one release on it. It's likely very little work but I think there's value
> in certifying the stack will work well there. (On the topic of OS's - are
> we specifically talking SP3 of SLES 11?). I don't feel strongly on this,
> but I'm just curious if there's a reason you're suggesting staying with
> 12.10 and not 13.04 - other than wanting less change in this release?
> Again, I hardly have an opinion on that one.
>
> Other components that have recently had releases that I don't consider to
> impact the
> - Hue 2.4.0
> - Whirr 0.8.2
> - Flume 1.4.0
>
> There's also been a ticket to package Avro for a long time and I'd like to
> get to that soon. Perhaps Parquet as well? Although like Phoenix and
> DataFu, I would suggest doing just the libraries for now, not all the CLI
> tools.
>
> Again - I don't mean to take away from the focus on stability, but I also
> don't think we shouldn't stretch to stay up to date either.
>
> +1 to everything else as suggested, however.
>
>
> On Tue, Jul 9, 2013 at 8:59 AM, Andrew Purtell <ap...@apache.org> wrote:
>> Would you be willing to consider Phoenix, only BIGTOP-993? Installing the
>> package produced by 993 only drops a library for HBase into
>> /usr/lib/phoenix, essentially the same relationship between the DataFu
>> package and Pig. There is follow up work that is more ambitious, for
>> example BIGTOP-1007, but that is not required by any means and could come
>> in if/whenever you are comfortable with it.
>>
>>
>> On Mon, Jul 8, 2013 at 8:50 PM, Konstantin Boudnik <co...@apache.org> wrote:
>>
>>> Guys,
>>>
>>> I wanna kick-off the discussion on the content of 0.7.0 BOM
>>> Release 0.6.0 was all about stabilization of the stack and I think we
> got a
>>> great headway on that. The following components/OS were in the frame of
> the
>>> discussion:
>>> http://is.gd/H52iVe
>>>
>>> My personal take that we need to spend this release cycle working on the
>>> improvements in Bigtop itself: we got enough "technical debts" in the
>>> pipeline that have to be addressed. To name a few:
>>> - testability/test coverage
>>> - test framework
>>> - package improvements
>>> - build improvements (including performance)
>>>
>>> In order to be able to deliver a solid stack again yet improve all things
>>> Bigtop I'd like to focus on the latter, hence keeping the former at bay
> and
>>> limiting the component updates to the bugfix releases only (if
> warranted).
>>> E.g.
>>>
>>> Hadoop 2.0.5 or later (stabilization branch of Hadoop 2)
>>> HBase 0.94.9 (update from the Bigtop 0.6.0)
>>> HCatalog 0.5.0 (same as 0.6.0, as well as following...)
>>> Zookeeper 3.4.5
>>> Pig 0.11.1
>>> Hive 0.10.0
>>> Sqoop 2
>>> Oozie 3.3.2
>>> Whirr 0.8.1
>>> Mahout 0.7
>>> Flume 1.3.1
>>> Giraph 0.2.0
>>> Hue 2.2.0
>>> Datafu 0.0.6
>>> Solr 4.2.1
>>> Crunch 0.5.0
>>> Tomcat 6.0.36
>>> Spark 0.7.3 (it has been in the queue for a long time)
>>>
>>> Also, I'd suggest to keep the same set of OSes as last time:
>>>
>>> CentOS/RHEL5
>>> CentOS/RHEL6
>>> SLES11
>>> Ubuntu 12.04 (LTS)
>>> Fedora 18
>>> OpenSUSE 12.3
>>> Ubuntu 12.10
>>>
>>> To reiterate, with a known stable version of the stack we can safely
> focus
>>> on
>>> the improvements to the framework and the overall system usability.
>>>
>>> Please jump on the discussion Also, I have opened up the following JIRA
> to
>>> track the BOM update https://issues.apache.org/jira/browse/BIGTOP-1023
>>>
>>> Thanks,
>>> Cos
>>>
>>>
>>
>>
>> --
>> Best regards,
>>
>> - Andy
>>
>> Problems worthy of attack prove their worth by hitting back. - Piet Hein
>> (via Tom White)
>
+1 to what Andrew and Sean said.
I would also like to add:
* I would also like to keep the door open for Apache Gora to come in.
Someone is already working on it
* Fedora 19 is out and maybe 20 will be out by the time we release 0.7.0
With regard to stability, if included all the changes listed so far seem
a little bit too much, what about dedicating a 0.6.1 to stability so
0.7.0 can include all the shiny things listed above? A 0.6.1 would
enable us to be more strict with the patches/fixes/updates being
included as well as signaling a more stable version to users.
Thanks,
Bruno
Re: [DISCUSS] BOM for release 0.7.0 of Bigtop
Posted by Sean Mackrory <ma...@gmail.com>.
Without wanting to detract from the spirit of focussing on system
stability, I'd like to suggest a few changes I think it's time we at least
discuss seriously:
JDKs: I've seen a lot of people ask about JDK 7. Perhaps time to add
support for Oracle JDK 7? It's working pretty well in my experience, and
although it's less tested upstream, the only JDK we officially support is
officially EOL, so we're not exactly in a good position now IMO.
Debian 7 has also been out for a while, and I think we should do at least
one release on it. It's likely very little work but I think there's value
in certifying the stack will work well there. (On the topic of OS's - are
we specifically talking SP3 of SLES 11?). I don't feel strongly on this,
but I'm just curious if there's a reason you're suggesting staying with
12.10 and not 13.04 - other than wanting less change in this release?
Again, I hardly have an opinion on that one.
Other components that have recently had releases that I don't consider to
impact the
- Hue 2.4.0
- Whirr 0.8.2
- Flume 1.4.0
There's also been a ticket to package Avro for a long time and I'd like to
get to that soon. Perhaps Parquet as well? Although like Phoenix and
DataFu, I would suggest doing just the libraries for now, not all the CLI
tools.
Again - I don't mean to take away from the focus on stability, but I also
don't think we shouldn't stretch to stay up to date either.
+1 to everything else as suggested, however.
On Tue, Jul 9, 2013 at 8:59 AM, Andrew Purtell <ap...@apache.org> wrote:
> Would you be willing to consider Phoenix, only BIGTOP-993? Installing the
> package produced by 993 only drops a library for HBase into
> /usr/lib/phoenix, essentially the same relationship between the DataFu
> package and Pig. There is follow up work that is more ambitious, for
> example BIGTOP-1007, but that is not required by any means and could come
> in if/whenever you are comfortable with it.
>
>
> On Mon, Jul 8, 2013 at 8:50 PM, Konstantin Boudnik <co...@apache.org> wrote:
>
>> Guys,
>>
>> I wanna kick-off the discussion on the content of 0.7.0 BOM
>> Release 0.6.0 was all about stabilization of the stack and I think we
got a
>> great headway on that. The following components/OS were in the frame of
the
>> discussion:
>> http://is.gd/H52iVe
>>
>> My personal take that we need to spend this release cycle working on the
>> improvements in Bigtop itself: we got enough "technical debts" in the
>> pipeline that have to be addressed. To name a few:
>> - testability/test coverage
>> - test framework
>> - package improvements
>> - build improvements (including performance)
>>
>> In order to be able to deliver a solid stack again yet improve all things
>> Bigtop I'd like to focus on the latter, hence keeping the former at bay
and
>> limiting the component updates to the bugfix releases only (if
warranted).
>> E.g.
>>
>> Hadoop 2.0.5 or later (stabilization branch of Hadoop 2)
>> HBase 0.94.9 (update from the Bigtop 0.6.0)
>> HCatalog 0.5.0 (same as 0.6.0, as well as following...)
>> Zookeeper 3.4.5
>> Pig 0.11.1
>> Hive 0.10.0
>> Sqoop 2
>> Oozie 3.3.2
>> Whirr 0.8.1
>> Mahout 0.7
>> Flume 1.3.1
>> Giraph 0.2.0
>> Hue 2.2.0
>> Datafu 0.0.6
>> Solr 4.2.1
>> Crunch 0.5.0
>> Tomcat 6.0.36
>> Spark 0.7.3 (it has been in the queue for a long time)
>>
>> Also, I'd suggest to keep the same set of OSes as last time:
>>
>> CentOS/RHEL5
>> CentOS/RHEL6
>> SLES11
>> Ubuntu 12.04 (LTS)
>> Fedora 18
>> OpenSUSE 12.3
>> Ubuntu 12.10
>>
>> To reiterate, with a known stable version of the stack we can safely
focus
>> on
>> the improvements to the framework and the overall system usability.
>>
>> Please jump on the discussion Also, I have opened up the following JIRA
to
>> track the BOM update https://issues.apache.org/jira/browse/BIGTOP-1023
>>
>> Thanks,
>> Cos
>>
>>
>
>
> --
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
Re: [DISCUSS] BOM for release 0.7.0 of Bigtop
Posted by Konstantin Boudnik <co...@apache.org>.
sure, sounds like a good addition to me.
On Tue, Jul 09, 2013 at 08:59AM, Andrew Purtell wrote:
> Would you be willing to consider Phoenix, only BIGTOP-993? Installing the
> package produced by 993 only drops a library for HBase into
> /usr/lib/phoenix, essentially the same relationship between the DataFu
> package and Pig. There is follow up work that is more ambitious, for
> example BIGTOP-1007, but that is not required by any means and could come
> in if/whenever you are comfortable with it.
>
>
> On Mon, Jul 8, 2013 at 8:50 PM, Konstantin Boudnik <co...@apache.org> wrote:
>
> > Guys,
> >
> > I wanna kick-off the discussion on the content of 0.7.0 BOM
> > Release 0.6.0 was all about stabilization of the stack and I think we got a
> > great headway on that. The following components/OS were in the frame of the
> > discussion:
> > http://is.gd/H52iVe
> >
> > My personal take that we need to spend this release cycle working on the
> > improvements in Bigtop itself: we got enough "technical debts" in the
> > pipeline that have to be addressed. To name a few:
> > - testability/test coverage
> > - test framework
> > - package improvements
> > - build improvements (including performance)
> >
> > In order to be able to deliver a solid stack again yet improve all things
> > Bigtop I'd like to focus on the latter, hence keeping the former at bay and
> > limiting the component updates to the bugfix releases only (if warranted).
> > E.g.
> >
> > Hadoop 2.0.5 or later (stabilization branch of Hadoop 2)
> > HBase 0.94.9 (update from the Bigtop 0.6.0)
> > HCatalog 0.5.0 (same as 0.6.0, as well as following...)
> > Zookeeper 3.4.5
> > Pig 0.11.1
> > Hive 0.10.0
> > Sqoop 2
> > Oozie 3.3.2
> > Whirr 0.8.1
> > Mahout 0.7
> > Flume 1.3.1
> > Giraph 0.2.0
> > Hue 2.2.0
> > Datafu 0.0.6
> > Solr 4.2.1
> > Crunch 0.5.0
> > Tomcat 6.0.36
> > Spark 0.7.3 (it has been in the queue for a long time)
> >
> > Also, I'd suggest to keep the same set of OSes as last time:
> >
> > CentOS/RHEL5
> > CentOS/RHEL6
> > SLES11
> > Ubuntu 12.04 (LTS)
> > Fedora 18
> > OpenSUSE 12.3
> > Ubuntu 12.10
> >
> > To reiterate, with a known stable version of the stack we can safely focus
> > on
> > the improvements to the framework and the overall system usability.
> >
> > Please jump on the discussion Also, I have opened up the following JIRA to
> > track the BOM update https://issues.apache.org/jira/browse/BIGTOP-1023
> >
> > Thanks,
> > Cos
> >
> >
>
>
> --
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
Re: [DISCUSS] BOM for release 0.7.0 of Bigtop
Posted by Andrew Purtell <ap...@apache.org>.
Would you be willing to consider Phoenix, only BIGTOP-993? Installing the
package produced by 993 only drops a library for HBase into
/usr/lib/phoenix, essentially the same relationship between the DataFu
package and Pig. There is follow up work that is more ambitious, for
example BIGTOP-1007, but that is not required by any means and could come
in if/whenever you are comfortable with it.
On Mon, Jul 8, 2013 at 8:50 PM, Konstantin Boudnik <co...@apache.org> wrote:
> Guys,
>
> I wanna kick-off the discussion on the content of 0.7.0 BOM
> Release 0.6.0 was all about stabilization of the stack and I think we got a
> great headway on that. The following components/OS were in the frame of the
> discussion:
> http://is.gd/H52iVe
>
> My personal take that we need to spend this release cycle working on the
> improvements in Bigtop itself: we got enough "technical debts" in the
> pipeline that have to be addressed. To name a few:
> - testability/test coverage
> - test framework
> - package improvements
> - build improvements (including performance)
>
> In order to be able to deliver a solid stack again yet improve all things
> Bigtop I'd like to focus on the latter, hence keeping the former at bay and
> limiting the component updates to the bugfix releases only (if warranted).
> E.g.
>
> Hadoop 2.0.5 or later (stabilization branch of Hadoop 2)
> HBase 0.94.9 (update from the Bigtop 0.6.0)
> HCatalog 0.5.0 (same as 0.6.0, as well as following...)
> Zookeeper 3.4.5
> Pig 0.11.1
> Hive 0.10.0
> Sqoop 2
> Oozie 3.3.2
> Whirr 0.8.1
> Mahout 0.7
> Flume 1.3.1
> Giraph 0.2.0
> Hue 2.2.0
> Datafu 0.0.6
> Solr 4.2.1
> Crunch 0.5.0
> Tomcat 6.0.36
> Spark 0.7.3 (it has been in the queue for a long time)
>
> Also, I'd suggest to keep the same set of OSes as last time:
>
> CentOS/RHEL5
> CentOS/RHEL6
> SLES11
> Ubuntu 12.04 (LTS)
> Fedora 18
> OpenSUSE 12.3
> Ubuntu 12.10
>
> To reiterate, with a known stable version of the stack we can safely focus
> on
> the improvements to the framework and the overall system usability.
>
> Please jump on the discussion Also, I have opened up the following JIRA to
> track the BOM update https://issues.apache.org/jira/browse/BIGTOP-1023
>
> Thanks,
> Cos
>
>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: [DISCUSS] BOM for release 0.7.0 of Bigtop
Posted by Konstantin Boudnik <co...@apache.org>.
Well, two rubles would be about $0.07, which is pretty steep.
Don't think 9/22 is a bit too aggressive? Although, it really depends on the
how much we wanna do in this release.
I'd be more comfortable with a bit longer release cycle and focus on the
framework issues as well as updating the stack per BOM.
My 0.6(6) rubles ;)
Cos
On Mon, Aug 05, 2013 at 07:20PM, Roman Shaposhnik wrote:
> Sorry for coming to the party rather late (vacation and everything ;-)).
> Here's my 2 rubles worth of feedback:
>
> On Mon, Jul 8, 2013 at 8:50 PM, Konstantin Boudnik <co...@apache.org> wrote:
> > Guys,
> >
> > I wanna kick-off the discussion on the content of 0.7.0 BOM
> > Release 0.6.0 was all about stabilization of the stack and I think we got a
> > great headway on that. The following components/OS were in the frame of the
> > discussion:
> > http://is.gd/H52iVe
> >
> > My personal take that we need to spend this release cycle working on the
> > improvements in Bigtop itself: we got enough "technical debts" in the
> > pipeline that have to be addressed.
>
> A huge +1 to that! Also, it would be awesome to get back on track
> wrt. our release schedule (quarterly). Hence let me put the stake
> in the ground and propose a release date of 9/22 (we could call
> it a Fall release ;-)).
>
> > To name a few:
> > - testability/test coverage
> > - test framework
> > - package improvements
> > - build improvements (including performance)
> >
> > In order to be able to deliver a solid stack again yet improve all things
> > Bigtop I'd like to focus on the latter, hence keeping the former at bay and
> > limiting the component updates to the bugfix releases only (if warranted). E.g.
> >
> > Hadoop 2.0.5 or later (stabilization branch of Hadoop 2)
> > HBase 0.94.9 (update from the Bigtop 0.6.0)
> > HCatalog 0.5.0 (same as 0.6.0, as well as following...)
> > Zookeeper 3.4.5
> > Pig 0.11.1
> > Hive 0.10.0
> > Sqoop 2
> > Oozie 3.3.2
> > Whirr 0.8.1
> > Mahout 0.7
> > Flume 1.3.1
> > Giraph 0.2.0
> > Hue 2.2.0
> > Datafu 0.0.6
> > Solr 4.2.1
> > Crunch 0.5.0
> > Tomcat 6.0.36
> > Spark 0.7.3 (it has been in the queue for a long time)
> >
> > Also, I'd suggest to keep the same set of OSes as last time:
> >
> > CentOS/RHEL5
> > CentOS/RHEL6
> > SLES11
> > Ubuntu 12.04 (LTS)
> > Fedora 18
> > OpenSUSE 12.3
> > Ubuntu 12.10
> >
> > To reiterate, with a known stable version of the stack we can safely focus on
> > the improvements to the framework and the overall system usability.
> >
> > Please jump on the discussion Also, I have opened up the following JIRA to
> > track the BOM update https://issues.apache.org/jira/browse/BIGTOP-1023
>
> I've provided a few bits of feedback on the JIRA itself. Seems like there's
> not too much opposition to the current plan so lets nail down a few details
> this week and I can file subjiras defining this release.
>
> Thanks,
> Roman.
Re: [DISCUSS] BOM for release 0.7.0 of Bigtop
Posted by Roman Shaposhnik <rv...@apache.org>.
Sorry for coming to the party rather late (vacation and everything ;-)).
Here's my 2 rubles worth of feedback:
On Mon, Jul 8, 2013 at 8:50 PM, Konstantin Boudnik <co...@apache.org> wrote:
> Guys,
>
> I wanna kick-off the discussion on the content of 0.7.0 BOM
> Release 0.6.0 was all about stabilization of the stack and I think we got a
> great headway on that. The following components/OS were in the frame of the
> discussion:
> http://is.gd/H52iVe
>
> My personal take that we need to spend this release cycle working on the
> improvements in Bigtop itself: we got enough "technical debts" in the
> pipeline that have to be addressed.
A huge +1 to that! Also, it would be awesome to get back on track
wrt. our release schedule (quarterly). Hence let me put the stake
in the ground and propose a release date of 9/22 (we could call
it a Fall release ;-)).
> To name a few:
> - testability/test coverage
> - test framework
> - package improvements
> - build improvements (including performance)
>
> In order to be able to deliver a solid stack again yet improve all things
> Bigtop I'd like to focus on the latter, hence keeping the former at bay and
> limiting the component updates to the bugfix releases only (if warranted). E.g.
>
> Hadoop 2.0.5 or later (stabilization branch of Hadoop 2)
> HBase 0.94.9 (update from the Bigtop 0.6.0)
> HCatalog 0.5.0 (same as 0.6.0, as well as following...)
> Zookeeper 3.4.5
> Pig 0.11.1
> Hive 0.10.0
> Sqoop 2
> Oozie 3.3.2
> Whirr 0.8.1
> Mahout 0.7
> Flume 1.3.1
> Giraph 0.2.0
> Hue 2.2.0
> Datafu 0.0.6
> Solr 4.2.1
> Crunch 0.5.0
> Tomcat 6.0.36
> Spark 0.7.3 (it has been in the queue for a long time)
>
> Also, I'd suggest to keep the same set of OSes as last time:
>
> CentOS/RHEL5
> CentOS/RHEL6
> SLES11
> Ubuntu 12.04 (LTS)
> Fedora 18
> OpenSUSE 12.3
> Ubuntu 12.10
>
> To reiterate, with a known stable version of the stack we can safely focus on
> the improvements to the framework and the overall system usability.
>
> Please jump on the discussion Also, I have opened up the following JIRA to
> track the BOM update https://issues.apache.org/jira/browse/BIGTOP-1023
I've provided a few bits of feedback on the JIRA itself. Seems like there's
not too much opposition to the current plan so lets nail down a few details
this week and I can file subjiras defining this release.
Thanks,
Roman.
Re: [DISCUSS] BOM for release 0.7.0 of Bigtop
Posted by Konstantin Boudnik <co...@apache.org>.
Hi all,
I have updated BIGTOP-1023 to reflect and reconcile the discussion on this
thread. Please check and chime in if I missed or misrepresented anything.
Thanks,
Cos
On Mon, Jul 08, 2013 at 08:50PM, Konstantin Boudnik wrote:
> Guys,
>
> I wanna kick-off the discussion on the content of 0.7.0 BOM
> Release 0.6.0 was all about stabilization of the stack and I think we got a
> great headway on that. The following components/OS were in the frame of the
> discussion:
> http://is.gd/H52iVe
>
> My personal take that we need to spend this release cycle working on the
> improvements in Bigtop itself: we got enough "technical debts" in the
> pipeline that have to be addressed. To name a few:
> - testability/test coverage
> - test framework
> - package improvements
> - build improvements (including performance)
>
> In order to be able to deliver a solid stack again yet improve all things
> Bigtop I'd like to focus on the latter, hence keeping the former at bay and
> limiting the component updates to the bugfix releases only (if warranted). E.g.
>
> Hadoop 2.0.5 or later (stabilization branch of Hadoop 2)
> HBase 0.94.9 (update from the Bigtop 0.6.0)
> HCatalog 0.5.0 (same as 0.6.0, as well as following...)
> Zookeeper 3.4.5
> Pig 0.11.1
> Hive 0.10.0
> Sqoop 2
> Oozie 3.3.2
> Whirr 0.8.1
> Mahout 0.7
> Flume 1.3.1
> Giraph 0.2.0
> Hue 2.2.0
> Datafu 0.0.6
> Solr 4.2.1
> Crunch 0.5.0
> Tomcat 6.0.36
> Spark 0.7.3 (it has been in the queue for a long time)
>
> Also, I'd suggest to keep the same set of OSes as last time:
>
> CentOS/RHEL5
> CentOS/RHEL6
> SLES11
> Ubuntu 12.04 (LTS)
> Fedora 18
> OpenSUSE 12.3
> Ubuntu 12.10
>
> To reiterate, with a known stable version of the stack we can safely focus on
> the improvements to the framework and the overall system usability.
>
> Please jump on the discussion Also, I have opened up the following JIRA to
> track the BOM update https://issues.apache.org/jira/browse/BIGTOP-1023
>
> Thanks,
> Cos
>
Re: [DISCUSS] BOM for release 0.7.0 of Bigtop
Posted by Jarek Jarcec Cecho <ja...@apache.org>.
I would like to thank the Bigtop community for the job that you are doing. I personally see a great value in integrating all the various Hadoop ecosystem projects under one roof and making sure that they are working with each other. Thank you!
We had discussions about what Sqoop community would like to see in Bigtop distribution on our last Sqoop user meet up. We've agreed there that Sqoop2 is our primary focus and the future of the project. However for the time being Sqoop1 contains far more features (incremental import, various special connectors, ...) and downstream integrations (Avro, Hive, HBase, HCatalog, ...) in comparison with Sqoop2. Thus we as a Sqoop community would prefer for sake of our users to have both major versions available inside the BigTop distribution. Sqoop1 for users who needs all the advance features and Sqoop2 for pushing the future direction to people, so that we can get early feedback and incorporate it into the project. I strongly believe that we as a community do understand that it's just our wish that might not fit into BigTop philosophy of releasing bleeding edge versions, so please do not take this as that we are forcing Bigtop to include both major versions. I believe that Sqoop community will understand if BigTop community decides to include only Sqoop2 for the 0.7 release.
Jarcec
On Tue, Jul 09, 2013 at 06:03:20PM -0700, Venkat Ranganathan wrote:
> + Jarek Jarcec Cecho who was missed in the last email.
>
> There is a reason why the Sqoop2 release is called 1.99.x (not even the 2.0
> alpha) because it needs some more features to be called Sqoop2. Currently
> there are some customer used Sqoop 1 feature that are not in Sqoop 1.99.x.
>
>
> That said, I am not wedded to this view. I might have misunderstood the
> charter to mean things differently than what you intended.
>
> Venkat
>
>
> On Tue, Jul 9, 2013 at 5:19 PM, Mark Grover <gr...@gmail.com>wrote:
>
> > I am inclined against making Sqoop1 the default version in Bigtop precisely
> > because of the point Andrew raised. Moreover, we had some good reasons when
> > we moved to Sqoop2 that resonated with Bigtop's charter of a cutting edge
> > distribution and helping in the stabilization of Hadoop ecosystem projects.
> > More details at https://issues.apache.org/jira/browse/BIGTOP-805
> >
> > As far as adding back Sqoop1 back to Bigtop is concerned, this is a
> > community led project, so if the community wants it, it will happen:-) The
> > general sentiment when introducing Sqoop2 was that there wasn't a need for
> > having 2 versions of Sqoop. From poking around, I think we did the same for
> > Flume when migrating from Flume OG to Flume NG (
> > https://issues.apache.org/jira/browse/BIGTOP-323).
> >
> > As far as Sqoop2 being preview releases, one could argue that the Hadoop
> > releases bigtop bundles are preview as well. In my personal opinion, the
> > charter of Bigtop, is to be that very cutting edge well tested distribution
> > that helps in stabilizing them along the way. Personally, I feel like
> > Sqoop2 being default falls in line with that. Given the above, I would
> > personally vote for Sqoop2 being present in BOM. And, adding Sqoop1 back in
> > as non-default Sqoop if there is traction in the community.
> >
> > I am open to feedback, though. What do others think?
> >
> > Mark
> >
> > On Tue, Jul 9, 2013 at 4:42 PM, Venkat Ranganathan <
> > vranganathan@hortonworks.com> wrote:
> >
> > > I understand. The discussion we had was around the current distributions
> > > ship with Sqoop 1.x as the default sqoop product (primarily because
> > Sqoop 2
> > > is in preview releases currently. The current focus of the team is to
> > > bring sqoop 2 to fruition quickly but Sqoop 1.x is the release that
> > > customers currently are using and hence the suggestion.
> > >
> > > Venkat
> > >
> > >
> > > On Tue, Jul 9, 2013 at 2:58 PM, Andrew Purtell <ap...@apache.org>
> > > wrote:
> > >
> > > > On Tue, Jul 9, 2013 at 2:52 PM, Venkat Ranganathan <
> > > > vranganathan@hortonworks.com> wrote:
> > > >
> > > > > I would also suggest we revert back to
> > > > > making Sqoop 1 the default sqoop version
> > > > >
> > > >
> > > > Wouldn't that make an upgrade from Bigtop 0.6 to 0.7 a Sqoop downgrade?
> > > >
> > > >
> > > > --
> > > > Best regards,
> > > >
> > > > - Andy
> > > >
> > > > Problems worthy of attack prove their worth by hitting back. - Piet
> > Hein
> > > > (via Tom White)
> > > >
> > >
> >
Re: [DISCUSS] BOM for release 0.7.0 of Bigtop
Posted by Venkat Ranganathan <vr...@hortonworks.com>.
+ Jarek Jarcec Cecho who was missed in the last email.
There is a reason why the Sqoop2 release is called 1.99.x (not even the 2.0
alpha) because it needs some more features to be called Sqoop2. Currently
there are some customer used Sqoop 1 feature that are not in Sqoop 1.99.x.
That said, I am not wedded to this view. I might have misunderstood the
charter to mean things differently than what you intended.
Venkat
On Tue, Jul 9, 2013 at 5:19 PM, Mark Grover <gr...@gmail.com>wrote:
> I am inclined against making Sqoop1 the default version in Bigtop precisely
> because of the point Andrew raised. Moreover, we had some good reasons when
> we moved to Sqoop2 that resonated with Bigtop's charter of a cutting edge
> distribution and helping in the stabilization of Hadoop ecosystem projects.
> More details at https://issues.apache.org/jira/browse/BIGTOP-805
>
> As far as adding back Sqoop1 back to Bigtop is concerned, this is a
> community led project, so if the community wants it, it will happen:-) The
> general sentiment when introducing Sqoop2 was that there wasn't a need for
> having 2 versions of Sqoop. From poking around, I think we did the same for
> Flume when migrating from Flume OG to Flume NG (
> https://issues.apache.org/jira/browse/BIGTOP-323).
>
> As far as Sqoop2 being preview releases, one could argue that the Hadoop
> releases bigtop bundles are preview as well. In my personal opinion, the
> charter of Bigtop, is to be that very cutting edge well tested distribution
> that helps in stabilizing them along the way. Personally, I feel like
> Sqoop2 being default falls in line with that. Given the above, I would
> personally vote for Sqoop2 being present in BOM. And, adding Sqoop1 back in
> as non-default Sqoop if there is traction in the community.
>
> I am open to feedback, though. What do others think?
>
> Mark
>
> On Tue, Jul 9, 2013 at 4:42 PM, Venkat Ranganathan <
> vranganathan@hortonworks.com> wrote:
>
> > I understand. The discussion we had was around the current distributions
> > ship with Sqoop 1.x as the default sqoop product (primarily because
> Sqoop 2
> > is in preview releases currently. The current focus of the team is to
> > bring sqoop 2 to fruition quickly but Sqoop 1.x is the release that
> > customers currently are using and hence the suggestion.
> >
> > Venkat
> >
> >
> > On Tue, Jul 9, 2013 at 2:58 PM, Andrew Purtell <ap...@apache.org>
> > wrote:
> >
> > > On Tue, Jul 9, 2013 at 2:52 PM, Venkat Ranganathan <
> > > vranganathan@hortonworks.com> wrote:
> > >
> > > > I would also suggest we revert back to
> > > > making Sqoop 1 the default sqoop version
> > > >
> > >
> > > Wouldn't that make an upgrade from Bigtop 0.6 to 0.7 a Sqoop downgrade?
> > >
> > >
> > > --
> > > Best regards,
> > >
> > > - Andy
> > >
> > > Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> > > (via Tom White)
> > >
> >
>
Re: [DISCUSS] BOM for release 0.7.0 of Bigtop
Posted by Bruno Mahé <bm...@apache.org>.
As long as someone volunteer to add a sqoop1 package for 0.7.0 or 0.6.1,
I don't have any issue with it.
The only thing I care about is to not go back in versions and to not
break upgrades.
So if commands or path conflicts (sqoop1 vs sqoop2), then sqoop 2 should
take priority.
Thanks,
Bruno
On 07/09/2013 09:10 PM, Konstantin Boudnik wrote:
> Ah, make sense. It looks like having both version might be a mess... Shall we
> stick with the plan of having Sqoop1 restored just in 0.6.1 then ?
>
> On Tue, Jul 09, 2013 at 09:06PM, Sean Mackrory wrote:
>>>> To start with I don't even understand what it means 'make version X a default'
>>
>> I think what they/we mean is which one gets to be called just "sqoop"
>> in the directories and command names, and which one has its version as
>> a suffix (e.g. "sqoop1", "sqoop2"). Alternatively they could both have
>> the suffix. I don't feel that strongly any particular way, just
>> clarifying what I think is meant.
>>
>> On Tue, Jul 9, 2013 at 7:51 PM, Konstantin Boudnik <co...@apache.org> wrote:
>>>
>>> To start with I don't even understand what it means 'make version X a
>>> default'. All components including into BOM are equal, except for Hadoop that
>>> is more equal than others ;)
>>>
>>> At any rate: bleeding edge mantra is just what we like to present Bigtop, it
>>> isn't really a policy of the project. Besides, Bigtop 0.6.0 was used as a
>>> stabilization of the stack based on Hadoop 2.0.x, namely 2.0.5
>>>
>>> Another alternative to having Sqoop 1.x returned is too quickly bake 0.6.1
>>> (along with Bruno's original idea), but with a single change in its BOM, ie
>>> Sqoop 1.x added into it.
>>> The scope of the release would be really limited, e.g. just one JIRA, and we
>>> should be able to get it out in a matter of a couple of days without
>>> disrupting 0.7.0. If this seems like a good way to go - let's separate these
>>> two and keep pn 0.7.0 discussion.
>>>
>>> Thoughts,
>>> Cos
>>>
>>> On Tue, Jul 09, 2013 at 05:19PM, Mark Grover wrote:
>>>> I am inclined against making Sqoop1 the default version in Bigtop precisely
>>>> because of the point Andrew raised. Moreover, we had some good reasons when
>>>> we moved to Sqoop2 that resonated with Bigtop's charter of a cutting edge
>>>> distribution and helping in the stabilization of Hadoop ecosystem projects.
>>>> More details at https://issues.apache.org/jira/browse/BIGTOP-805
>>>>
>>>> As far as adding back Sqoop1 back to Bigtop is concerned, this is a
>>>> community led project, so if the community wants it, it will happen:-) The
>>>> general sentiment when introducing Sqoop2 was that there wasn't a need for
>>>> having 2 versions of Sqoop. From poking around, I think we did the same for
>>>> Flume when migrating from Flume OG to Flume NG (
>>>> https://issues.apache.org/jira/browse/BIGTOP-323).
>>>>
>>>> As far as Sqoop2 being preview releases, one could argue that the Hadoop
>>>> releases bigtop bundles are preview as well. In my personal opinion, the
>>>> charter of Bigtop, is to be that very cutting edge well tested distribution
>>>> that helps in stabilizing them along the way. Personally, I feel like
>>>> Sqoop2 being default falls in line with that. Given the above, I would
>>>> personally vote for Sqoop2 being present in BOM. And, adding Sqoop1 back in
>>>> as non-default Sqoop if there is traction in the community.
>>>>
>>>> I am open to feedback, though. What do others think?
>>>>
>>>> Mark
>>>>
>>>> On Tue, Jul 9, 2013 at 4:42 PM, Venkat Ranganathan <
>>>> vranganathan@hortonworks.com> wrote:
>>>>
>>>>> I understand. The discussion we had was around the current distributions
>>>>> ship with Sqoop 1.x as the default sqoop product (primarily because Sqoop 2
>>>>> is in preview releases currently. The current focus of the team is to
>>>>> bring sqoop 2 to fruition quickly but Sqoop 1.x is the release that
>>>>> customers currently are using and hence the suggestion.
>>>>>
>>>>> Venkat
>>>>>
>>>>>
>>>>> On Tue, Jul 9, 2013 at 2:58 PM, Andrew Purtell <ap...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> On Tue, Jul 9, 2013 at 2:52 PM, Venkat Ranganathan <
>>>>>> vranganathan@hortonworks.com> wrote:
>>>>>>
>>>>>>> I would also suggest we revert back to
>>>>>>> making Sqoop 1 the default sqoop version
>>>>>>>
>>>>>>
>>>>>> Wouldn't that make an upgrade from Bigtop 0.6 to 0.7 a Sqoop downgrade?
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Best regards,
>>>>>>
>>>>>> - Andy
>>>>>>
>>>>>> Problems worthy of attack prove their worth by hitting back. - Piet Hein
>>>>>> (via Tom White)
>>>>>>
>>>>>
Re: [DISCUSS] BOM for release 0.7.0 of Bigtop
Posted by Konstantin Boudnik <co...@apache.org>.
Ah, make sense. It looks like having both version might be a mess... Shall we
stick with the plan of having Sqoop1 restored just in 0.6.1 then ?
On Tue, Jul 09, 2013 at 09:06PM, Sean Mackrory wrote:
> >> To start with I don't even understand what it means 'make version X a default'
>
> I think what they/we mean is which one gets to be called just "sqoop"
> in the directories and command names, and which one has its version as
> a suffix (e.g. "sqoop1", "sqoop2"). Alternatively they could both have
> the suffix. I don't feel that strongly any particular way, just
> clarifying what I think is meant.
>
> On Tue, Jul 9, 2013 at 7:51 PM, Konstantin Boudnik <co...@apache.org> wrote:
> >
> > To start with I don't even understand what it means 'make version X a
> > default'. All components including into BOM are equal, except for Hadoop that
> > is more equal than others ;)
> >
> > At any rate: bleeding edge mantra is just what we like to present Bigtop, it
> > isn't really a policy of the project. Besides, Bigtop 0.6.0 was used as a
> > stabilization of the stack based on Hadoop 2.0.x, namely 2.0.5
> >
> > Another alternative to having Sqoop 1.x returned is too quickly bake 0.6.1
> > (along with Bruno's original idea), but with a single change in its BOM, ie
> > Sqoop 1.x added into it.
> > The scope of the release would be really limited, e.g. just one JIRA, and we
> > should be able to get it out in a matter of a couple of days without
> > disrupting 0.7.0. If this seems like a good way to go - let's separate these
> > two and keep pn 0.7.0 discussion.
> >
> > Thoughts,
> > Cos
> >
> > On Tue, Jul 09, 2013 at 05:19PM, Mark Grover wrote:
> > > I am inclined against making Sqoop1 the default version in Bigtop precisely
> > > because of the point Andrew raised. Moreover, we had some good reasons when
> > > we moved to Sqoop2 that resonated with Bigtop's charter of a cutting edge
> > > distribution and helping in the stabilization of Hadoop ecosystem projects.
> > > More details at https://issues.apache.org/jira/browse/BIGTOP-805
> > >
> > > As far as adding back Sqoop1 back to Bigtop is concerned, this is a
> > > community led project, so if the community wants it, it will happen:-) The
> > > general sentiment when introducing Sqoop2 was that there wasn't a need for
> > > having 2 versions of Sqoop. From poking around, I think we did the same for
> > > Flume when migrating from Flume OG to Flume NG (
> > > https://issues.apache.org/jira/browse/BIGTOP-323).
> > >
> > > As far as Sqoop2 being preview releases, one could argue that the Hadoop
> > > releases bigtop bundles are preview as well. In my personal opinion, the
> > > charter of Bigtop, is to be that very cutting edge well tested distribution
> > > that helps in stabilizing them along the way. Personally, I feel like
> > > Sqoop2 being default falls in line with that. Given the above, I would
> > > personally vote for Sqoop2 being present in BOM. And, adding Sqoop1 back in
> > > as non-default Sqoop if there is traction in the community.
> > >
> > > I am open to feedback, though. What do others think?
> > >
> > > Mark
> > >
> > > On Tue, Jul 9, 2013 at 4:42 PM, Venkat Ranganathan <
> > > vranganathan@hortonworks.com> wrote:
> > >
> > > > I understand. The discussion we had was around the current distributions
> > > > ship with Sqoop 1.x as the default sqoop product (primarily because Sqoop 2
> > > > is in preview releases currently. The current focus of the team is to
> > > > bring sqoop 2 to fruition quickly but Sqoop 1.x is the release that
> > > > customers currently are using and hence the suggestion.
> > > >
> > > > Venkat
> > > >
> > > >
> > > > On Tue, Jul 9, 2013 at 2:58 PM, Andrew Purtell <ap...@apache.org>
> > > > wrote:
> > > >
> > > > > On Tue, Jul 9, 2013 at 2:52 PM, Venkat Ranganathan <
> > > > > vranganathan@hortonworks.com> wrote:
> > > > >
> > > > > > I would also suggest we revert back to
> > > > > > making Sqoop 1 the default sqoop version
> > > > > >
> > > > >
> > > > > Wouldn't that make an upgrade from Bigtop 0.6 to 0.7 a Sqoop downgrade?
> > > > >
> > > > >
> > > > > --
> > > > > Best regards,
> > > > >
> > > > > - Andy
> > > > >
> > > > > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > > > > (via Tom White)
> > > > >
> > > >
Re: [DISCUSS] BOM for release 0.7.0 of Bigtop
Posted by Sean Mackrory <ma...@gmail.com>.
And my apologies for duplicating half my draft :)
On Tue, Jul 9, 2013 at 9:06 PM, Sean Mackrory <ma...@gmail.com> wrote:
>>> To start with I don't even understand what it means 'make version X a default'
>
> I think what they/we mean is which one gets to be called just "sqoop"
> in the directories and command names, and which one has its version as
> a suffix (e.g. "sqoop1", "sqoop2"). Alternatively they could both have
> the suffix. I don't feel that strongly any particular way, just
> clarifying what I think is meant.
>
> command is just called "sqoop", and which one has its version as a
> suffix (i.e. "sqoop1", sqoop2"). Alternately they could both have the
> suffix. I don't feel that strongly any particular way, just clarifying
> what I think is meant.
>
> On Tue, Jul 9, 2013 at 7:51 PM, Konstantin Boudnik <co...@apache.org> wrote:
>>
>> To start with I don't even understand what it means 'make version X a
>> default'. All components including into BOM are equal, except for Hadoop that
>> is more equal than others ;)
>>
>> At any rate: bleeding edge mantra is just what we like to present Bigtop, it
>> isn't really a policy of the project. Besides, Bigtop 0.6.0 was used as a
>> stabilization of the stack based on Hadoop 2.0.x, namely 2.0.5
>>
>> Another alternative to having Sqoop 1.x returned is too quickly bake 0.6.1
>> (along with Bruno's original idea), but with a single change in its BOM, ie
>> Sqoop 1.x added into it.
>> The scope of the release would be really limited, e.g. just one JIRA, and we
>> should be able to get it out in a matter of a couple of days without
>> disrupting 0.7.0. If this seems like a good way to go - let's separate these
>> two and keep pn 0.7.0 discussion.
>>
>> Thoughts,
>> Cos
>>
>> On Tue, Jul 09, 2013 at 05:19PM, Mark Grover wrote:
>> > I am inclined against making Sqoop1 the default version in Bigtop precisely
>> > because of the point Andrew raised. Moreover, we had some good reasons when
>> > we moved to Sqoop2 that resonated with Bigtop's charter of a cutting edge
>> > distribution and helping in the stabilization of Hadoop ecosystem projects.
>> > More details at https://issues.apache.org/jira/browse/BIGTOP-805
>> >
>> > As far as adding back Sqoop1 back to Bigtop is concerned, this is a
>> > community led project, so if the community wants it, it will happen:-) The
>> > general sentiment when introducing Sqoop2 was that there wasn't a need for
>> > having 2 versions of Sqoop. From poking around, I think we did the same for
>> > Flume when migrating from Flume OG to Flume NG (
>> > https://issues.apache.org/jira/browse/BIGTOP-323).
>> >
>> > As far as Sqoop2 being preview releases, one could argue that the Hadoop
>> > releases bigtop bundles are preview as well. In my personal opinion, the
>> > charter of Bigtop, is to be that very cutting edge well tested distribution
>> > that helps in stabilizing them along the way. Personally, I feel like
>> > Sqoop2 being default falls in line with that. Given the above, I would
>> > personally vote for Sqoop2 being present in BOM. And, adding Sqoop1 back in
>> > as non-default Sqoop if there is traction in the community.
>> >
>> > I am open to feedback, though. What do others think?
>> >
>> > Mark
>> >
>> > On Tue, Jul 9, 2013 at 4:42 PM, Venkat Ranganathan <
>> > vranganathan@hortonworks.com> wrote:
>> >
>> > > I understand. The discussion we had was around the current distributions
>> > > ship with Sqoop 1.x as the default sqoop product (primarily because Sqoop 2
>> > > is in preview releases currently. The current focus of the team is to
>> > > bring sqoop 2 to fruition quickly but Sqoop 1.x is the release that
>> > > customers currently are using and hence the suggestion.
>> > >
>> > > Venkat
>> > >
>> > >
>> > > On Tue, Jul 9, 2013 at 2:58 PM, Andrew Purtell <ap...@apache.org>
>> > > wrote:
>> > >
>> > > > On Tue, Jul 9, 2013 at 2:52 PM, Venkat Ranganathan <
>> > > > vranganathan@hortonworks.com> wrote:
>> > > >
>> > > > > I would also suggest we revert back to
>> > > > > making Sqoop 1 the default sqoop version
>> > > > >
>> > > >
>> > > > Wouldn't that make an upgrade from Bigtop 0.6 to 0.7 a Sqoop downgrade?
>> > > >
>> > > >
>> > > > --
>> > > > Best regards,
>> > > >
>> > > > - Andy
>> > > >
>> > > > Problems worthy of attack prove their worth by hitting back. - Piet Hein
>> > > > (via Tom White)
>> > > >
>> > >
Re: [DISCUSS] BOM for release 0.7.0 of Bigtop
Posted by Sean Mackrory <ma...@gmail.com>.
>> To start with I don't even understand what it means 'make version X a default'
I think what they/we mean is which one gets to be called just "sqoop"
in the directories and command names, and which one has its version as
a suffix (e.g. "sqoop1", "sqoop2"). Alternatively they could both have
the suffix. I don't feel that strongly any particular way, just
clarifying what I think is meant.
command is just called "sqoop", and which one has its version as a
suffix (i.e. "sqoop1", sqoop2"). Alternately they could both have the
suffix. I don't feel that strongly any particular way, just clarifying
what I think is meant.
On Tue, Jul 9, 2013 at 7:51 PM, Konstantin Boudnik <co...@apache.org> wrote:
>
> To start with I don't even understand what it means 'make version X a
> default'. All components including into BOM are equal, except for Hadoop that
> is more equal than others ;)
>
> At any rate: bleeding edge mantra is just what we like to present Bigtop, it
> isn't really a policy of the project. Besides, Bigtop 0.6.0 was used as a
> stabilization of the stack based on Hadoop 2.0.x, namely 2.0.5
>
> Another alternative to having Sqoop 1.x returned is too quickly bake 0.6.1
> (along with Bruno's original idea), but with a single change in its BOM, ie
> Sqoop 1.x added into it.
> The scope of the release would be really limited, e.g. just one JIRA, and we
> should be able to get it out in a matter of a couple of days without
> disrupting 0.7.0. If this seems like a good way to go - let's separate these
> two and keep pn 0.7.0 discussion.
>
> Thoughts,
> Cos
>
> On Tue, Jul 09, 2013 at 05:19PM, Mark Grover wrote:
> > I am inclined against making Sqoop1 the default version in Bigtop precisely
> > because of the point Andrew raised. Moreover, we had some good reasons when
> > we moved to Sqoop2 that resonated with Bigtop's charter of a cutting edge
> > distribution and helping in the stabilization of Hadoop ecosystem projects.
> > More details at https://issues.apache.org/jira/browse/BIGTOP-805
> >
> > As far as adding back Sqoop1 back to Bigtop is concerned, this is a
> > community led project, so if the community wants it, it will happen:-) The
> > general sentiment when introducing Sqoop2 was that there wasn't a need for
> > having 2 versions of Sqoop. From poking around, I think we did the same for
> > Flume when migrating from Flume OG to Flume NG (
> > https://issues.apache.org/jira/browse/BIGTOP-323).
> >
> > As far as Sqoop2 being preview releases, one could argue that the Hadoop
> > releases bigtop bundles are preview as well. In my personal opinion, the
> > charter of Bigtop, is to be that very cutting edge well tested distribution
> > that helps in stabilizing them along the way. Personally, I feel like
> > Sqoop2 being default falls in line with that. Given the above, I would
> > personally vote for Sqoop2 being present in BOM. And, adding Sqoop1 back in
> > as non-default Sqoop if there is traction in the community.
> >
> > I am open to feedback, though. What do others think?
> >
> > Mark
> >
> > On Tue, Jul 9, 2013 at 4:42 PM, Venkat Ranganathan <
> > vranganathan@hortonworks.com> wrote:
> >
> > > I understand. The discussion we had was around the current distributions
> > > ship with Sqoop 1.x as the default sqoop product (primarily because Sqoop 2
> > > is in preview releases currently. The current focus of the team is to
> > > bring sqoop 2 to fruition quickly but Sqoop 1.x is the release that
> > > customers currently are using and hence the suggestion.
> > >
> > > Venkat
> > >
> > >
> > > On Tue, Jul 9, 2013 at 2:58 PM, Andrew Purtell <ap...@apache.org>
> > > wrote:
> > >
> > > > On Tue, Jul 9, 2013 at 2:52 PM, Venkat Ranganathan <
> > > > vranganathan@hortonworks.com> wrote:
> > > >
> > > > > I would also suggest we revert back to
> > > > > making Sqoop 1 the default sqoop version
> > > > >
> > > >
> > > > Wouldn't that make an upgrade from Bigtop 0.6 to 0.7 a Sqoop downgrade?
> > > >
> > > >
> > > > --
> > > > Best regards,
> > > >
> > > > - Andy
> > > >
> > > > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > > > (via Tom White)
> > > >
> > >
Re: [DISCUSS] BOM for release 0.7.0 of Bigtop
Posted by Konstantin Boudnik <co...@apache.org>.
To start with I don't even understand what it means 'make version X a
default'. All components including into BOM are equal, except for Hadoop that
is more equal than others ;)
At any rate: bleeding edge mantra is just what we like to present Bigtop, it
isn't really a policy of the project. Besides, Bigtop 0.6.0 was used as a
stabilization of the stack based on Hadoop 2.0.x, namely 2.0.5
Another alternative to having Sqoop 1.x returned is too quickly bake 0.6.1
(along with Bruno's original idea), but with a single change in its BOM, ie
Sqoop 1.x added into it.
The scope of the release would be really limited, e.g. just one JIRA, and we
should be able to get it out in a matter of a couple of days without
disrupting 0.7.0. If this seems like a good way to go - let's separate these
two and keep pn 0.7.0 discussion.
Thoughts,
Cos
On Tue, Jul 09, 2013 at 05:19PM, Mark Grover wrote:
> I am inclined against making Sqoop1 the default version in Bigtop precisely
> because of the point Andrew raised. Moreover, we had some good reasons when
> we moved to Sqoop2 that resonated with Bigtop's charter of a cutting edge
> distribution and helping in the stabilization of Hadoop ecosystem projects.
> More details at https://issues.apache.org/jira/browse/BIGTOP-805
>
> As far as adding back Sqoop1 back to Bigtop is concerned, this is a
> community led project, so if the community wants it, it will happen:-) The
> general sentiment when introducing Sqoop2 was that there wasn't a need for
> having 2 versions of Sqoop. From poking around, I think we did the same for
> Flume when migrating from Flume OG to Flume NG (
> https://issues.apache.org/jira/browse/BIGTOP-323).
>
> As far as Sqoop2 being preview releases, one could argue that the Hadoop
> releases bigtop bundles are preview as well. In my personal opinion, the
> charter of Bigtop, is to be that very cutting edge well tested distribution
> that helps in stabilizing them along the way. Personally, I feel like
> Sqoop2 being default falls in line with that. Given the above, I would
> personally vote for Sqoop2 being present in BOM. And, adding Sqoop1 back in
> as non-default Sqoop if there is traction in the community.
>
> I am open to feedback, though. What do others think?
>
> Mark
>
> On Tue, Jul 9, 2013 at 4:42 PM, Venkat Ranganathan <
> vranganathan@hortonworks.com> wrote:
>
> > I understand. The discussion we had was around the current distributions
> > ship with Sqoop 1.x as the default sqoop product (primarily because Sqoop 2
> > is in preview releases currently. The current focus of the team is to
> > bring sqoop 2 to fruition quickly but Sqoop 1.x is the release that
> > customers currently are using and hence the suggestion.
> >
> > Venkat
> >
> >
> > On Tue, Jul 9, 2013 at 2:58 PM, Andrew Purtell <ap...@apache.org>
> > wrote:
> >
> > > On Tue, Jul 9, 2013 at 2:52 PM, Venkat Ranganathan <
> > > vranganathan@hortonworks.com> wrote:
> > >
> > > > I would also suggest we revert back to
> > > > making Sqoop 1 the default sqoop version
> > > >
> > >
> > > Wouldn't that make an upgrade from Bigtop 0.6 to 0.7 a Sqoop downgrade?
> > >
> > >
> > > --
> > > Best regards,
> > >
> > > - Andy
> > >
> > > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > > (via Tom White)
> > >
> >
Re: [DISCUSS] BOM for release 0.7.0 of Bigtop
Posted by Andrew Purtell <ap...@apache.org>.
On Tue, Jul 9, 2013 at 5:19 PM, Mark Grover <gr...@gmail.com>wrote:
> As far as adding back Sqoop1 back to Bigtop is concerned, this is a
> community led project, so if the community wants it, it will happen:-)
>
That is https://issues.apache.org/jira/browse/BIGTOP-1016.
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: [DISCUSS] BOM for release 0.7.0 of Bigtop
Posted by Mark Grover <gr...@gmail.com>.
I am inclined against making Sqoop1 the default version in Bigtop precisely
because of the point Andrew raised. Moreover, we had some good reasons when
we moved to Sqoop2 that resonated with Bigtop's charter of a cutting edge
distribution and helping in the stabilization of Hadoop ecosystem projects.
More details at https://issues.apache.org/jira/browse/BIGTOP-805
As far as adding back Sqoop1 back to Bigtop is concerned, this is a
community led project, so if the community wants it, it will happen:-) The
general sentiment when introducing Sqoop2 was that there wasn't a need for
having 2 versions of Sqoop. From poking around, I think we did the same for
Flume when migrating from Flume OG to Flume NG (
https://issues.apache.org/jira/browse/BIGTOP-323).
As far as Sqoop2 being preview releases, one could argue that the Hadoop
releases bigtop bundles are preview as well. In my personal opinion, the
charter of Bigtop, is to be that very cutting edge well tested distribution
that helps in stabilizing them along the way. Personally, I feel like
Sqoop2 being default falls in line with that. Given the above, I would
personally vote for Sqoop2 being present in BOM. And, adding Sqoop1 back in
as non-default Sqoop if there is traction in the community.
I am open to feedback, though. What do others think?
Mark
On Tue, Jul 9, 2013 at 4:42 PM, Venkat Ranganathan <
vranganathan@hortonworks.com> wrote:
> I understand. The discussion we had was around the current distributions
> ship with Sqoop 1.x as the default sqoop product (primarily because Sqoop 2
> is in preview releases currently. The current focus of the team is to
> bring sqoop 2 to fruition quickly but Sqoop 1.x is the release that
> customers currently are using and hence the suggestion.
>
> Venkat
>
>
> On Tue, Jul 9, 2013 at 2:58 PM, Andrew Purtell <ap...@apache.org>
> wrote:
>
> > On Tue, Jul 9, 2013 at 2:52 PM, Venkat Ranganathan <
> > vranganathan@hortonworks.com> wrote:
> >
> > > I would also suggest we revert back to
> > > making Sqoop 1 the default sqoop version
> > >
> >
> > Wouldn't that make an upgrade from Bigtop 0.6 to 0.7 a Sqoop downgrade?
> >
> >
> > --
> > Best regards,
> >
> > - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
> >
>
Re: [DISCUSS] BOM for release 0.7.0 of Bigtop
Posted by Venkat Ranganathan <vr...@hortonworks.com>.
I understand. The discussion we had was around the current distributions
ship with Sqoop 1.x as the default sqoop product (primarily because Sqoop 2
is in preview releases currently. The current focus of the team is to
bring sqoop 2 to fruition quickly but Sqoop 1.x is the release that
customers currently are using and hence the suggestion.
Venkat
On Tue, Jul 9, 2013 at 2:58 PM, Andrew Purtell <ap...@apache.org> wrote:
> On Tue, Jul 9, 2013 at 2:52 PM, Venkat Ranganathan <
> vranganathan@hortonworks.com> wrote:
>
> > I would also suggest we revert back to
> > making Sqoop 1 the default sqoop version
> >
>
> Wouldn't that make an upgrade from Bigtop 0.6 to 0.7 a Sqoop downgrade?
>
>
> --
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>
Re: [DISCUSS] BOM for release 0.7.0 of Bigtop
Posted by Andrew Purtell <ap...@apache.org>.
On Tue, Jul 9, 2013 at 2:52 PM, Venkat Ranganathan <
vranganathan@hortonworks.com> wrote:
> I would also suggest we revert back to
> making Sqoop 1 the default sqoop version
>
Wouldn't that make an upgrade from Bigtop 0.6 to 0.7 a Sqoop downgrade?
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: [DISCUSS] BOM for release 0.7.0 of Bigtop
Posted by Venkat Ranganathan <vr...@hortonworks.com>.
Good to see that it is agreed now. I would also suggest we revert back to
making Sqoop 1 the default sqoop version - i think we discussed this in the
Sqoop meetup before
Venkat
On Tue, Jul 9, 2013 at 1:54 PM, Konstantin Boudnik <co...@apache.org> wrote:
> good point, +1 on that.
>
> On Tue, Jul 09, 2013 at 01:39PM, Anatoli Fomenko wrote:
> > I had a quick chat with Jarcec (Sqoop), and, per my understanding, he
> would
> > like to have both Sqoop 1 and Sqoop 2 in Bigtop 0.7, with specific
> versions:
> > Sqoop 1 - 1.4.4 (currently in development)
> > Sqoop 2 - 1.92.3 (should be ready by Sep, current version 1.92.1).
> >
> > Per my understanding, 0.6 had only Sqoop 2, and the tests were converted
> to Sqoop 2. Adding Sqoop 1 would require some work, but should be doable,
> if it fits 0.7' stabilization sentiment.═
> >
> > I CCed Jarcec, in case if we need broader or deeper discussion.
> >
> > Thanks,
> > Anatoli═
> >
> >
> > ________________________________
> > From: Konstantin Boudnik <co...@apache.org>
> > To: dev@bigtop.apache.org
> > Sent: Monday, July 8, 2013 8:50 PM
> > Subject: [DISCUSS] BOM for release 0.7.0 of Bigtop
> >
> >
> > Guys,
> >
> > I wanna kick-off the discussion on the content of 0.7.0 BOM
> > Release 0.6.0 was all about stabilization of the stack and I think we
> got a
> > great headway on that. The following components/OS were in the frame of
> the
> > discussion:
> > ═ ═ http://is.gd/H52iVe
> >
> > My personal take that we need to spend this release cycle working on the
> > improvements in Bigtop itself: we got enough "technical debts" in the
> > pipeline that have to be addressed. To name a few:
> > ═ - testability/test coverage
> > ═ - test framework
> > ═ - package improvements
> > ═ - build improvements (including performance)
> >
> > In order to be able to deliver a solid stack again yet improve all things
> > Bigtop I'd like to focus on the latter, hence keeping the former at bay
> and
> > limiting the component updates to the bugfix releases only (if
> warranted). E.g.
> >
> > ═ ═ Hadoop 2.0.5 or later (stabilization branch of Hadoop 2)
> > ═ ═ HBase 0.94.9 (update from the Bigtop 0.6.0)
> > ═ ═ HCatalog 0.5.0 (same as 0.6.0, as well as following...)
> > ═ ═ Zookeeper 3.4.5
> > ═ ═ Pig 0.11.1
> > ═ ═ Hive 0.10.0
> > ═ ═ Sqoop 2
> > ═ ═ Oozie 3.3.2
> > ═ ═ Whirr 0.8.1
> > ═ ═ Mahout 0.7
> > ═ ═ Flume 1.3.1
> > ═ ═ Giraph 0.2.0
> > ═ ═ Hue 2.2.0
> > ═ ═ Datafu 0.0.6
> > ═ ═ Solr 4.2.1
> > ═ ═ Crunch 0.5.0
> > ═ ═ Tomcat 6.0.36
> > ═ ═ Spark 0.7.3═ ═ (it has been in the queue for a long time)
> >
> > Also, I'd suggest to keep the same set of OSes as last time:
> >
> > ═ ═ CentOS/RHEL5
> > ═ ═ CentOS/RHEL6
> > ═ ═ SLES11
> > ═ ═ Ubuntu 12.04 (LTS)
> > ═ ═ Fedora 18
> > ═ ═ OpenSUSE 12.3
> > ═ ═ Ubuntu 12.10
> >
> > To reiterate, with a known stable version of the stack we can safely
> focus on
> > the improvements to the framework and the overall system usability.
> >
> > Please jump on the discussion═ Also, I have opened up the following JIRA
> to
> > track the BOM update https://issues.apache.org/jira/browse/BIGTOP-1023
> >
> > Thanks,
> > ═ Cos
>
Re: [DISCUSS] BOM for release 0.7.0 of Bigtop
Posted by Konstantin Boudnik <co...@apache.org>.
good point, +1 on that.
On Tue, Jul 09, 2013 at 01:39PM, Anatoli Fomenko wrote:
> I had a quick chat with Jarcec (Sqoop), and, per my understanding, he would
> like to have both Sqoop 1 and Sqoop 2 in Bigtop 0.7, with specific versions:
> Sqoop 1 - 1.4.4 (currently in development)
> Sqoop 2 - 1.92.3 (should be ready by Sep, current version 1.92.1).
>
> Per my understanding, 0.6 had only Sqoop 2, and the tests were converted to Sqoop 2. Adding Sqoop 1 would require some work, but should be doable, if it fits 0.7' stabilization sentiment.═
>
> I CCed Jarcec, in case if we need broader or deeper discussion.
>
> Thanks,
> Anatoli═
>
>
> ________________________________
> From: Konstantin Boudnik <co...@apache.org>
> To: dev@bigtop.apache.org
> Sent: Monday, July 8, 2013 8:50 PM
> Subject: [DISCUSS] BOM for release 0.7.0 of Bigtop
>
>
> Guys,
>
> I wanna kick-off the discussion on the content of 0.7.0 BOM
> Release 0.6.0 was all about stabilization of the stack and I think we got a
> great headway on that. The following components/OS were in the frame of the
> discussion:
> ═ ═ http://is.gd/H52iVe
>
> My personal take that we need to spend this release cycle working on the
> improvements in Bigtop itself: we got enough "technical debts" in the
> pipeline that have to be addressed. To name a few:
> ═ - testability/test coverage
> ═ - test framework
> ═ - package improvements
> ═ - build improvements (including performance)
>
> In order to be able to deliver a solid stack again yet improve all things
> Bigtop I'd like to focus on the latter, hence keeping the former at bay and
> limiting the component updates to the bugfix releases only (if warranted). E.g.
>
> ═ ═ Hadoop 2.0.5 or later (stabilization branch of Hadoop 2)
> ═ ═ HBase 0.94.9 (update from the Bigtop 0.6.0)
> ═ ═ HCatalog 0.5.0 (same as 0.6.0, as well as following...)
> ═ ═ Zookeeper 3.4.5
> ═ ═ Pig 0.11.1
> ═ ═ Hive 0.10.0
> ═ ═ Sqoop 2
> ═ ═ Oozie 3.3.2
> ═ ═ Whirr 0.8.1
> ═ ═ Mahout 0.7
> ═ ═ Flume 1.3.1
> ═ ═ Giraph 0.2.0
> ═ ═ Hue 2.2.0
> ═ ═ Datafu 0.0.6
> ═ ═ Solr 4.2.1
> ═ ═ Crunch 0.5.0
> ═ ═ Tomcat 6.0.36
> ═ ═ Spark 0.7.3═ ═ (it has been in the queue for a long time)
>
> Also, I'd suggest to keep the same set of OSes as last time:
>
> ═ ═ CentOS/RHEL5
> ═ ═ CentOS/RHEL6
> ═ ═ SLES11
> ═ ═ Ubuntu 12.04 (LTS)
> ═ ═ Fedora 18
> ═ ═ OpenSUSE 12.3
> ═ ═ Ubuntu 12.10
>
> To reiterate, with a known stable version of the stack we can safely focus on
> the improvements to the framework and the overall system usability.
>
> Please jump on the discussion═ Also, I have opened up the following JIRA to
> track the BOM update https://issues.apache.org/jira/browse/BIGTOP-1023
>
> Thanks,
> ═ Cos
Re: [DISCUSS] BOM for release 0.7.0 of Bigtop
Posted by Anatoli Fomenko <af...@yahoo.com>.
I had a quick chat with Jarcec (Sqoop), and, per my understanding, he would like to have both Sqoop 1 and Sqoop 2 in Bigtop 0.7, with specific versions:
Sqoop 1 - 1.4.4 (currently in development)
Sqoop 2 - 1.92.3 (should be ready by Sep, current version 1.92.1).
Per my understanding, 0.6 had only Sqoop 2, and the tests were converted to Sqoop 2. Adding Sqoop 1 would require some work, but should be doable, if it fits 0.7' stabilization sentiment.
I CCed Jarcec, in case if we need broader or deeper discussion.
Thanks,
Anatoli
________________________________
From: Konstantin Boudnik <co...@apache.org>
To: dev@bigtop.apache.org
Sent: Monday, July 8, 2013 8:50 PM
Subject: [DISCUSS] BOM for release 0.7.0 of Bigtop
Guys,
I wanna kick-off the discussion on the content of 0.7.0 BOM
Release 0.6.0 was all about stabilization of the stack and I think we got a
great headway on that. The following components/OS were in the frame of the
discussion:
http://is.gd/H52iVe
My personal take that we need to spend this release cycle working on the
improvements in Bigtop itself: we got enough "technical debts" in the
pipeline that have to be addressed. To name a few:
- testability/test coverage
- test framework
- package improvements
- build improvements (including performance)
In order to be able to deliver a solid stack again yet improve all things
Bigtop I'd like to focus on the latter, hence keeping the former at bay and
limiting the component updates to the bugfix releases only (if warranted). E.g.
Hadoop 2.0.5 or later (stabilization branch of Hadoop 2)
HBase 0.94.9 (update from the Bigtop 0.6.0)
HCatalog 0.5.0 (same as 0.6.0, as well as following...)
Zookeeper 3.4.5
Pig 0.11.1
Hive 0.10.0
Sqoop 2
Oozie 3.3.2
Whirr 0.8.1
Mahout 0.7
Flume 1.3.1
Giraph 0.2.0
Hue 2.2.0
Datafu 0.0.6
Solr 4.2.1
Crunch 0.5.0
Tomcat 6.0.36
Spark 0.7.3 (it has been in the queue for a long time)
Also, I'd suggest to keep the same set of OSes as last time:
CentOS/RHEL5
CentOS/RHEL6
SLES11
Ubuntu 12.04 (LTS)
Fedora 18
OpenSUSE 12.3
Ubuntu 12.10
To reiterate, with a known stable version of the stack we can safely focus on
the improvements to the framework and the overall system usability.
Please jump on the discussion Also, I have opened up the following JIRA to
track the BOM update https://issues.apache.org/jira/browse/BIGTOP-1023
Thanks,
Cos