You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by Jun HE <ju...@apache.org> on 2018/08/07 16:11:53 UTC
[DISCUSS] Bigtop 1.3.0 release
Hi, folks,
As you may read from previous thread, I'm proposing to do a new release,
1.3.0, before ApacheCon in Sept.
I'm a +1 for this because:
* quite a few fixes/improvements for building and deployment
* important components upgrade
* x86 and non-x86 platforms are aligned in master
If there are enough votes for a release I'd like to volunteer to be the RM
for 1.3.0.
Evans has initiated a well-defined release BOM (
https://issues.apache.org/jira/browse/BIGTOP-2945), and I did some minor
modifications mainly about components versions in 1.3.0.
And issues might need fix: https://s.apache.org/Ja7e
Your comments/thoughts are mostly welcomed!
Regards,
Jun
Re: [DISCUSS] Bigtop 1.3.0 release
Posted by Olaf Flebbe <of...@oflebbe.de>.
Hi Jun,
We should do a release in order to push out changes which accumulate over the time. So I you have my +1
But bigtop needs some cleanup:
1) How do we handle failing packages?
My vote is to simply remove them:
Candidates are crunch, oozie, pig
I only looked at oozie right now:
oozie seems to be broken by conflicting requirements by log4j and shared libraries between hive and oozie see BIGTOP-2986
IMHO hive2 is way more important than having oozie, so I would say either oozie is fixed or out.
There is another issue related to oozie packaging, though easy fixable: BIGTOP-3051. Looking at that I have to conclude that oozie never worked correctly in 1.2.x
Putting all that together I would recommend to remove oozie from Bigtop for now.
2) I have a bad feeling for staying at hadoop-2.8.1. But since I do not have any productive environment any more I cannot judge why we should not upgrade to 2.8.4
thanks,
olaf
> Am 07.08.2018 um 18:11 schrieb Jun HE <ju...@apache.org>:
>
> Hi, folks,
>
> As you may read from previous thread, I'm proposing to do a new release,
> 1.3.0, before ApacheCon in Sept.
> I'm a +1 for this because:
> * quite a few fixes/improvements for building and deployment
> * important components upgrade
> * x86 and non-x86 platforms are aligned in master
>
> If there are enough votes for a release I'd like to volunteer to be the RM
> for 1.3.0.
>
> Evans has initiated a well-defined release BOM (
> https://issues.apache.org/jira/browse/BIGTOP-2945), and I did some minor
> modifications mainly about components versions in 1.3.0.
> And issues might need fix: https://s.apache.org/Ja7e
>
> Your comments/thoughts are mostly welcomed!
>
> Regards,
>
> Jun
Re: [DISCUSS] Bigtop 1.3.0 release
Posted by Evans Ye <ev...@apache.org>.
BIGTOP-3075 reviewed. Just need a rebase.
For BIGTOP-3079, Kafka 0.10.1.1 is 2 years old. I'm very much like to have
it upgraded.
Besides that, due to rocksdb license ban by ASF, we removed Kafka binary
artifact from by S3, which is not good for user experience.
I'll try to look into it. but not sure how much I can do. The very last
solution is as what you suggested, keep them as is.
sign packages:
Just your code signing key. Your public key is on "KEYS".
upload to S3:
We'll send you the credential.
deploy artifacts:
This requires your ASF account. But I'm not quite sure whether you can
perform the operations. Just let me know when you're blocked.
Jun HE <ry...@gmail.com> 於 2018年9月17日 週一 下午1:05寫道：
> Thanks for checking in, Evans.
>
> So far all components are built successfully on 5 distros+3 archs. You may
> check the CI here:
> https://ci.bigtop.apache.org/job/Bigtop-1.3-packages/
>
> And two JIRAs need to be solved for v1.3: BIGTOP-3075 and BIGTOP3079.
> BIGTOP-3075 needs your folks review and inputs.
> And as mentioned in BIGTOP-3079, flume-1.7 has problem to build with
> kafka-0.10.2.2. These two versions are defined in v1.3 BOM. For now there
> is no solution for this. So I'd like to keep Kafka to stay on current
> version for v1.3. What do you think?
>
> If these two are done, I can start to tag and produce rpm/deb packages.
> After that, these steps (sign packages, upload to S3 and deploy artifacts)
> probably need somebody's help as I may not have the access privilege.
>
> Regards,
>
> Jun
>
>
> Evans Ye <ev...@apache.org> 于2018年9月17日周一 上午12:36写道：
>
> > ApacheCon is around the corner. Let me check in with our RM Jun He:
> > Anything you'd like to call out for help to meet our target release date?
> >
> > Andrew Purtell <ap...@apache.org> 於 2018年8月15日 週三 上午9:32寫道：
> >
> > > Ok, please proceed with the 1.3 version, don't be blocked on us. We
> will
> > > fix this (eventually)
> > >
> > > On Tue, Aug 14, 2018 at 6:23 PM Jun HE <ju...@apache.org> wrote:
> > >
> > > > Thanks for the info, Andrew.
> > > >
> > > > Did a quick check on HBase 1.4.6, build failed with:
> > > >
> > > >
> > >
> >
> -------------------------------------------------------------------------------------------
> > > > [ERROR] Failed to execute goal
> > > > org.apache.maven.plugins:maven-site-plugin:3.4:site (default-site) on
> > > > project hbase: Error generating
> maven-javadoc-plugin:2.10.3:aggregate:
> > > > [ERROR] Exit code: 1 - warning: unknown enum constant When.MAYBE
> > > > [ERROR] reason: class file for javax.annotation.meta.When not found
> > > > [ERROR] warning: unknown enum constant When.ALWAYS
> > > > [ERROR] warning: unknown enum constant When.UNKNOWN
> > > > [ERROR] javadoc: error - class file for
> > > > javax.annotation.concurrent.Immutable not found
> > > >
> > > >
> > >
> >
> -------------------------------------------------------------------------------------------
> > > >
> > > > Might related to this (
> > https://issues.apache.org/jira/browse/HBASE-19663
> > > ).
> > > > If someone can help to fix this, I'm more than happy to include 1.4.6
> > in
> > > > v1.3 release. :)
> > > >
> > > > 2018-08-15 1:30 GMT+08:00 Andrew Purtell <ap...@apache.org>:
> > > >
> > > > > FWIW latest stable release of HBase is 1.4.6, released 14 days ago.
> > > > >
> > > > >
> > > > >
> > > > > On Tue, Aug 14, 2018 at 1:12 AM Jun HE <ry...@gmail.com>
> wrote:
> > > > >
> > > > > > Hi, folks,
> > > > > >
> > > > > > In BIGTOP-2945 I listed the proposed changes, comparing to
> > v1.2.1. A
> > > > > brief
> > > > > > is:
> > > > > >
> > > > > > ambari 2.5.0 => 2.6.1
> > > > > > crunch 0.14.0 => 0.15.0
> > > > > > flink 1.1.3 => 1.4.2
> > > > > > gpdb 5.0.0-alpha.0 => 5.10.0
> > > > > > hadoop 2.7.3 => 2.8.4
> > > > > > hama 0.7.0 => 0.7.1
> > > > > > hbase 1.1.9 => 1.3.2
> > > > > > hive 1.2.1 => 2.3.3
> > > > > > kafka 0.10.1.1 => 0.10.2.2
> > > > > > phoenix 4.9.0-HBase-1.1 => 4.13.1-HBase-1.3
> > > > > > qfs 1.1.4 => 2.0.0
> > > > > > solr 4.10.4 => 6.6.0
> > > > > > spark 2.1.1 => 2.2.1
> > > > > > tez 0.6.2 => 0.9.1
> > > > > > ycsb 0.4.0 => 0.12.0
> > > > > > zeppelin 0.7.2 => 0.7.3
> > > > > >
> > > > > > If there are no more comments on these versions change, I'd like
> to
> > > > > > finalize the BOM, create responding JIRAs and link them to
> > > BIGTOP-3063.
> > > > > >
> > > > > > Thanks & regards,
> > > > > >
> > > > > > Jun
> > > > > >
> > > > > >
> > > > > >
> > > > > > 2018-08-08 19:01 GMT+08:00 Dan Moraru <
> dmoraru@ligo-wa.caltech.edu
> > >:
> > > > > >
> > > > > > > How much more work would be involved in generating a separate
> set
> > > of
> > > > > > > Hadoop 3 packages for those who would like to try out HDFS
> > erasure
> > > > > coding
> > > > > > > (among other things)?
> > > > > > >
> > > > > > > Thanks,
> > > > > > >
> > > > > > > Dan
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On 08/07/2018 10:04 PM, Jun HE wrote:
> > > > > > >
> > > > > > >> Thanks for your inputs.
> > > > > > >>
> > > > > > >> For failing packages I agree with Olaf and Evans: remove them
> if
> > > no
> > > > > one
> > > > > > >> would take over the fix task.
> > > > > > >> Oozie and pig have JIRAs(BIGTOP-2986/BIGTOP-2975). For crunch
> > > there
> > > > > > >> isn't.
> > > > > > >> I'll create one later and do some initial investigation to see
> > if
> > > I
> > > > > can
> > > > > > >> fix
> > > > > > >> it or not.
> > > > > > >> For Hadoop upgrade, I'm fine to upgrade it to 2.8.4. Actually
> > > we've
> > > > > done
> > > > > > >> this in Linaro's ERP-18.06 release. (Yes, we're using Bigtop
> to
> > > > build
> > > > > > >> BigData packages :)) So there is no problem for it.
> > > > > > >>
> > > > > > >> Regards,
> > > > > > >>
> > > > > > >> Jun
> > > > > > >>
> > > > > > >> 2018-08-08 10:31 GMT+08:00 Evans Ye <ev...@apache.org>:
> > > > > > >>
> > > > > > >> Yes. We've features/bugfixes stacked up over time and
> definitely
> > > > need
> > > > > a
> > > > > > >>> release.
> > > > > > >>> I'm +1 to what Olaf said. Unless someone is willing to take
> > over
> > > > the
> > > > > > task
> > > > > > >>> and fix it.
> > > > > > >>> From my perspective, Oozie and Pig's market are declining.
> So
> > I
> > > > > > suggest
> > > > > > >>> let's spend our effort on those major components, and the
> > > > > functionality
> > > > > > >>> that Bigtop provides such as deployment and testing.
> > > > > > >>>
> > > > > > >>>
> > > > > > >>> Naresh Bhat <na...@linaro.org> 於 2018年8月8日 週三
> 上午8:58寫道：
> > > > > > >>>
> > > > > > >>> +1
> > > > > > >>>>
> > > > > > >>>> On 7 August 2018 at 21:41, Jun HE <ju...@apache.org> wrote:
> > > > > > >>>>
> > > > > > >>>> Hi, folks,
> > > > > > >>>>>
> > > > > > >>>>> As you may read from previous thread, I'm proposing to do a
> > new
> > > > > > >>>>>
> > > > > > >>>> release,
> > > > > > >>>
> > > > > > >>>> 1.3.0, before ApacheCon in Sept.
> > > > > > >>>>> I'm a +1 for this because:
> > > > > > >>>>> * quite a few fixes/improvements for building and
> deployment
> > > > > > >>>>> * important components upgrade
> > > > > > >>>>> * x86 and non-x86 platforms are aligned in master
> > > > > > >>>>>
> > > > > > >>>>> If there are enough votes for a release I'd like to
> volunteer
> > > to
> > > > be
> > > > > > the
> > > > > > >>>>>
> > > > > > >>>> RM
> > > > > > >>>>
> > > > > > >>>>> for 1.3.0.
> > > > > > >>>>>
> > > > > > >>>>> Evans has initiated a well-defined release BOM (
> > > > > > >>>>> https://issues.apache.org/jira/browse/BIGTOP-2945), and I
> > did
> > > > some
> > > > > > >>>>>
> > > > > > >>>> minor
> > > > > > >>>
> > > > > > >>>> modifications mainly about components versions in 1.3.0.
> > > > > > >>>>> And issues might need fix: https://s.apache.org/Ja7e
> > > > > > >>>>>
> > > > > > >>>>> Your comments/thoughts are mostly welcomed!
> > > > > > >>>>>
> > > > > > >>>>> Regards,
> > > > > > >>>>>
> > > > > > >>>>> Jun
> > > > > > >>>>>
> > > > > > >>>>>
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Best regards,
> > > > > Andrew
> > > > >
> > > > > Words like orphans lost among the crosstalk, meaning torn from
> > truth's
> > > > > decrepit hands
> > > > > - A23, Crosstalk
> > > > >
> > > >
> > >
> > >
> > > --
> > > Best regards,
> > > Andrew
> > >
> > > Words like orphans lost among the crosstalk, meaning torn from truth's
> > > decrepit hands
> > > - A23, Crosstalk
> > >
> >
>
Re: [DISCUSS] Bigtop 1.3.0 release
Posted by Jun HE <ry...@gmail.com>.
Thanks for checking in, Evans.
So far all components are built successfully on 5 distros+3 archs. You may
check the CI here:
https://ci.bigtop.apache.org/job/Bigtop-1.3-packages/
And two JIRAs need to be solved for v1.3: BIGTOP-3075 and BIGTOP3079.
BIGTOP-3075 needs your folks review and inputs.
And as mentioned in BIGTOP-3079, flume-1.7 has problem to build with
kafka-0.10.2.2. These two versions are defined in v1.3 BOM. For now there
is no solution for this. So I'd like to keep Kafka to stay on current
version for v1.3. What do you think?
If these two are done, I can start to tag and produce rpm/deb packages.
After that, these steps (sign packages, upload to S3 and deploy artifacts)
probably need somebody's help as I may not have the access privilege.
Regards,
Jun
Evans Ye <ev...@apache.org> 于2018年9月17日周一 上午12:36写道：
> ApacheCon is around the corner. Let me check in with our RM Jun He:
> Anything you'd like to call out for help to meet our target release date?
>
> Andrew Purtell <ap...@apache.org> 於 2018年8月15日 週三 上午9:32寫道：
>
> > Ok, please proceed with the 1.3 version, don't be blocked on us. We will
> > fix this (eventually)
> >
> > On Tue, Aug 14, 2018 at 6:23 PM Jun HE <ju...@apache.org> wrote:
> >
> > > Thanks for the info, Andrew.
> > >
> > > Did a quick check on HBase 1.4.6, build failed with:
> > >
> > >
> >
> -------------------------------------------------------------------------------------------
> > > [ERROR] Failed to execute goal
> > > org.apache.maven.plugins:maven-site-plugin:3.4:site (default-site) on
> > > project hbase: Error generating maven-javadoc-plugin:2.10.3:aggregate:
> > > [ERROR] Exit code: 1 - warning: unknown enum constant When.MAYBE
> > > [ERROR] reason: class file for javax.annotation.meta.When not found
> > > [ERROR] warning: unknown enum constant When.ALWAYS
> > > [ERROR] warning: unknown enum constant When.UNKNOWN
> > > [ERROR] javadoc: error - class file for
> > > javax.annotation.concurrent.Immutable not found
> > >
> > >
> >
> -------------------------------------------------------------------------------------------
> > >
> > > Might related to this (
> https://issues.apache.org/jira/browse/HBASE-19663
> > ).
> > > If someone can help to fix this, I'm more than happy to include 1.4.6
> in
> > > v1.3 release. :)
> > >
> > > 2018-08-15 1:30 GMT+08:00 Andrew Purtell <ap...@apache.org>:
> > >
> > > > FWIW latest stable release of HBase is 1.4.6, released 14 days ago.
> > > >
> > > >
> > > >
> > > > On Tue, Aug 14, 2018 at 1:12 AM Jun HE <ry...@gmail.com> wrote:
> > > >
> > > > > Hi, folks,
> > > > >
> > > > > In BIGTOP-2945 I listed the proposed changes, comparing to
> v1.2.1. A
> > > > brief
> > > > > is:
> > > > >
> > > > > ambari 2.5.0 => 2.6.1
> > > > > crunch 0.14.0 => 0.15.0
> > > > > flink 1.1.3 => 1.4.2
> > > > > gpdb 5.0.0-alpha.0 => 5.10.0
> > > > > hadoop 2.7.3 => 2.8.4
> > > > > hama 0.7.0 => 0.7.1
> > > > > hbase 1.1.9 => 1.3.2
> > > > > hive 1.2.1 => 2.3.3
> > > > > kafka 0.10.1.1 => 0.10.2.2
> > > > > phoenix 4.9.0-HBase-1.1 => 4.13.1-HBase-1.3
> > > > > qfs 1.1.4 => 2.0.0
> > > > > solr 4.10.4 => 6.6.0
> > > > > spark 2.1.1 => 2.2.1
> > > > > tez 0.6.2 => 0.9.1
> > > > > ycsb 0.4.0 => 0.12.0
> > > > > zeppelin 0.7.2 => 0.7.3
> > > > >
> > > > > If there are no more comments on these versions change, I'd like to
> > > > > finalize the BOM, create responding JIRAs and link them to
> > BIGTOP-3063.
> > > > >
> > > > > Thanks & regards,
> > > > >
> > > > > Jun
> > > > >
> > > > >
> > > > >
> > > > > 2018-08-08 19:01 GMT+08:00 Dan Moraru <dmoraru@ligo-wa.caltech.edu
> >:
> > > > >
> > > > > > How much more work would be involved in generating a separate set
> > of
> > > > > > Hadoop 3 packages for those who would like to try out HDFS
> erasure
> > > > coding
> > > > > > (among other things)?
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Dan
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > On 08/07/2018 10:04 PM, Jun HE wrote:
> > > > > >
> > > > > >> Thanks for your inputs.
> > > > > >>
> > > > > >> For failing packages I agree with Olaf and Evans: remove them if
> > no
> > > > one
> > > > > >> would take over the fix task.
> > > > > >> Oozie and pig have JIRAs(BIGTOP-2986/BIGTOP-2975). For crunch
> > there
> > > > > >> isn't.
> > > > > >> I'll create one later and do some initial investigation to see
> if
> > I
> > > > can
> > > > > >> fix
> > > > > >> it or not.
> > > > > >> For Hadoop upgrade, I'm fine to upgrade it to 2.8.4. Actually
> > we've
> > > > done
> > > > > >> this in Linaro's ERP-18.06 release. (Yes, we're using Bigtop to
> > > build
> > > > > >> BigData packages :)) So there is no problem for it.
> > > > > >>
> > > > > >> Regards,
> > > > > >>
> > > > > >> Jun
> > > > > >>
> > > > > >> 2018-08-08 10:31 GMT+08:00 Evans Ye <ev...@apache.org>:
> > > > > >>
> > > > > >> Yes. We've features/bugfixes stacked up over time and definitely
> > > need
> > > > a
> > > > > >>> release.
> > > > > >>> I'm +1 to what Olaf said. Unless someone is willing to take
> over
> > > the
> > > > > task
> > > > > >>> and fix it.
> > > > > >>> From my perspective, Oozie and Pig's market are declining. So
> I
> > > > > suggest
> > > > > >>> let's spend our effort on those major components, and the
> > > > functionality
> > > > > >>> that Bigtop provides such as deployment and testing.
> > > > > >>>
> > > > > >>>
> > > > > >>> Naresh Bhat <na...@linaro.org> 於 2018年8月8日 週三 上午8:58寫道：
> > > > > >>>
> > > > > >>> +1
> > > > > >>>>
> > > > > >>>> On 7 August 2018 at 21:41, Jun HE <ju...@apache.org> wrote:
> > > > > >>>>
> > > > > >>>> Hi, folks,
> > > > > >>>>>
> > > > > >>>>> As you may read from previous thread, I'm proposing to do a
> new
> > > > > >>>>>
> > > > > >>>> release,
> > > > > >>>
> > > > > >>>> 1.3.0, before ApacheCon in Sept.
> > > > > >>>>> I'm a +1 for this because:
> > > > > >>>>> * quite a few fixes/improvements for building and deployment
> > > > > >>>>> * important components upgrade
> > > > > >>>>> * x86 and non-x86 platforms are aligned in master
> > > > > >>>>>
> > > > > >>>>> If there are enough votes for a release I'd like to volunteer
> > to
> > > be
> > > > > the
> > > > > >>>>>
> > > > > >>>> RM
> > > > > >>>>
> > > > > >>>>> for 1.3.0.
> > > > > >>>>>
> > > > > >>>>> Evans has initiated a well-defined release BOM (
> > > > > >>>>> https://issues.apache.org/jira/browse/BIGTOP-2945), and I
> did
> > > some
> > > > > >>>>>
> > > > > >>>> minor
> > > > > >>>
> > > > > >>>> modifications mainly about components versions in 1.3.0.
> > > > > >>>>> And issues might need fix: https://s.apache.org/Ja7e
> > > > > >>>>>
> > > > > >>>>> Your comments/thoughts are mostly welcomed!
> > > > > >>>>>
> > > > > >>>>> Regards,
> > > > > >>>>>
> > > > > >>>>> Jun
> > > > > >>>>>
> > > > > >>>>>
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > > Best regards,
> > > > Andrew
> > > >
> > > > Words like orphans lost among the crosstalk, meaning torn from
> truth's
> > > > decrepit hands
> > > > - A23, Crosstalk
> > > >
> > >
> >
> >
> > --
> > Best regards,
> > Andrew
> >
> > Words like orphans lost among the crosstalk, meaning torn from truth's
> > decrepit hands
> > - A23, Crosstalk
> >
>
Re: [DISCUSS] Bigtop 1.3.0 release
Posted by Evans Ye <ev...@apache.org>.
ApacheCon is around the corner. Let me check in with our RM Jun He:
Anything you'd like to call out for help to meet our target release date?
Andrew Purtell <ap...@apache.org> 於 2018年8月15日 週三 上午9:32寫道：
> Ok, please proceed with the 1.3 version, don't be blocked on us. We will
> fix this (eventually)
>
> On Tue, Aug 14, 2018 at 6:23 PM Jun HE <ju...@apache.org> wrote:
>
> > Thanks for the info, Andrew.
> >
> > Did a quick check on HBase 1.4.6, build failed with:
> >
> >
> -------------------------------------------------------------------------------------------
> > [ERROR] Failed to execute goal
> > org.apache.maven.plugins:maven-site-plugin:3.4:site (default-site) on
> > project hbase: Error generating maven-javadoc-plugin:2.10.3:aggregate:
> > [ERROR] Exit code: 1 - warning: unknown enum constant When.MAYBE
> > [ERROR] reason: class file for javax.annotation.meta.When not found
> > [ERROR] warning: unknown enum constant When.ALWAYS
> > [ERROR] warning: unknown enum constant When.UNKNOWN
> > [ERROR] javadoc: error - class file for
> > javax.annotation.concurrent.Immutable not found
> >
> >
> -------------------------------------------------------------------------------------------
> >
> > Might related to this (https://issues.apache.org/jira/browse/HBASE-19663
> ).
> > If someone can help to fix this, I'm more than happy to include 1.4.6 in
> > v1.3 release. :)
> >
> > 2018-08-15 1:30 GMT+08:00 Andrew Purtell <ap...@apache.org>:
> >
> > > FWIW latest stable release of HBase is 1.4.6, released 14 days ago.
> > >
> > >
> > >
> > > On Tue, Aug 14, 2018 at 1:12 AM Jun HE <ry...@gmail.com> wrote:
> > >
> > > > Hi, folks,
> > > >
> > > > In BIGTOP-2945 I listed the proposed changes, comparing to v1.2.1. A
> > > brief
> > > > is:
> > > >
> > > > ambari 2.5.0 => 2.6.1
> > > > crunch 0.14.0 => 0.15.0
> > > > flink 1.1.3 => 1.4.2
> > > > gpdb 5.0.0-alpha.0 => 5.10.0
> > > > hadoop 2.7.3 => 2.8.4
> > > > hama 0.7.0 => 0.7.1
> > > > hbase 1.1.9 => 1.3.2
> > > > hive 1.2.1 => 2.3.3
> > > > kafka 0.10.1.1 => 0.10.2.2
> > > > phoenix 4.9.0-HBase-1.1 => 4.13.1-HBase-1.3
> > > > qfs 1.1.4 => 2.0.0
> > > > solr 4.10.4 => 6.6.0
> > > > spark 2.1.1 => 2.2.1
> > > > tez 0.6.2 => 0.9.1
> > > > ycsb 0.4.0 => 0.12.0
> > > > zeppelin 0.7.2 => 0.7.3
> > > >
> > > > If there are no more comments on these versions change, I'd like to
> > > > finalize the BOM, create responding JIRAs and link them to
> BIGTOP-3063.
> > > >
> > > > Thanks & regards,
> > > >
> > > > Jun
> > > >
> > > >
> > > >
> > > > 2018-08-08 19:01 GMT+08:00 Dan Moraru <dm...@ligo-wa.caltech.edu>:
> > > >
> > > > > How much more work would be involved in generating a separate set
> of
> > > > > Hadoop 3 packages for those who would like to try out HDFS erasure
> > > coding
> > > > > (among other things)?
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Dan
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On 08/07/2018 10:04 PM, Jun HE wrote:
> > > > >
> > > > >> Thanks for your inputs.
> > > > >>
> > > > >> For failing packages I agree with Olaf and Evans: remove them if
> no
> > > one
> > > > >> would take over the fix task.
> > > > >> Oozie and pig have JIRAs(BIGTOP-2986/BIGTOP-2975). For crunch
> there
> > > > >> isn't.
> > > > >> I'll create one later and do some initial investigation to see if
> I
> > > can
> > > > >> fix
> > > > >> it or not.
> > > > >> For Hadoop upgrade, I'm fine to upgrade it to 2.8.4. Actually
> we've
> > > done
> > > > >> this in Linaro's ERP-18.06 release. (Yes, we're using Bigtop to
> > build
> > > > >> BigData packages :)) So there is no problem for it.
> > > > >>
> > > > >> Regards,
> > > > >>
> > > > >> Jun
> > > > >>
> > > > >> 2018-08-08 10:31 GMT+08:00 Evans Ye <ev...@apache.org>:
> > > > >>
> > > > >> Yes. We've features/bugfixes stacked up over time and definitely
> > need
> > > a
> > > > >>> release.
> > > > >>> I'm +1 to what Olaf said. Unless someone is willing to take over
> > the
> > > > task
> > > > >>> and fix it.
> > > > >>> From my perspective, Oozie and Pig's market are declining. So I
> > > > suggest
> > > > >>> let's spend our effort on those major components, and the
> > > functionality
> > > > >>> that Bigtop provides such as deployment and testing.
> > > > >>>
> > > > >>>
> > > > >>> Naresh Bhat <na...@linaro.org> 於 2018年8月8日 週三 上午8:58寫道：
> > > > >>>
> > > > >>> +1
> > > > >>>>
> > > > >>>> On 7 August 2018 at 21:41, Jun HE <ju...@apache.org> wrote:
> > > > >>>>
> > > > >>>> Hi, folks,
> > > > >>>>>
> > > > >>>>> As you may read from previous thread, I'm proposing to do a new
> > > > >>>>>
> > > > >>>> release,
> > > > >>>
> > > > >>>> 1.3.0, before ApacheCon in Sept.
> > > > >>>>> I'm a +1 for this because:
> > > > >>>>> * quite a few fixes/improvements for building and deployment
> > > > >>>>> * important components upgrade
> > > > >>>>> * x86 and non-x86 platforms are aligned in master
> > > > >>>>>
> > > > >>>>> If there are enough votes for a release I'd like to volunteer
> to
> > be
> > > > the
> > > > >>>>>
> > > > >>>> RM
> > > > >>>>
> > > > >>>>> for 1.3.0.
> > > > >>>>>
> > > > >>>>> Evans has initiated a well-defined release BOM (
> > > > >>>>> https://issues.apache.org/jira/browse/BIGTOP-2945), and I did
> > some
> > > > >>>>>
> > > > >>>> minor
> > > > >>>
> > > > >>>> modifications mainly about components versions in 1.3.0.
> > > > >>>>> And issues might need fix: https://s.apache.org/Ja7e
> > > > >>>>>
> > > > >>>>> Your comments/thoughts are mostly welcomed!
> > > > >>>>>
> > > > >>>>> Regards,
> > > > >>>>>
> > > > >>>>> Jun
> > > > >>>>>
> > > > >>>>>
> > > > >
> > > >
> > >
> > >
> > > --
> > > Best regards,
> > > Andrew
> > >
> > > Words like orphans lost among the crosstalk, meaning torn from truth's
> > > decrepit hands
> > > - A23, Crosstalk
> > >
> >
>
>
> --
> Best regards,
> Andrew
>
> Words like orphans lost among the crosstalk, meaning torn from truth's
> decrepit hands
> - A23, Crosstalk
>
Re: [DISCUSS] Bigtop 1.3.0 release
Posted by Andrew Purtell <ap...@apache.org>.
Ok, please proceed with the 1.3 version, don't be blocked on us. We will
fix this (eventually)
On Tue, Aug 14, 2018 at 6:23 PM Jun HE <ju...@apache.org> wrote:
> Thanks for the info, Andrew.
>
> Did a quick check on HBase 1.4.6, build failed with:
>
> -------------------------------------------------------------------------------------------
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-site-plugin:3.4:site (default-site) on
> project hbase: Error generating maven-javadoc-plugin:2.10.3:aggregate:
> [ERROR] Exit code: 1 - warning: unknown enum constant When.MAYBE
> [ERROR] reason: class file for javax.annotation.meta.When not found
> [ERROR] warning: unknown enum constant When.ALWAYS
> [ERROR] warning: unknown enum constant When.UNKNOWN
> [ERROR] javadoc: error - class file for
> javax.annotation.concurrent.Immutable not found
>
> -------------------------------------------------------------------------------------------
>
> Might related to this (https://issues.apache.org/jira/browse/HBASE-19663).
> If someone can help to fix this, I'm more than happy to include 1.4.6 in
> v1.3 release. :)
>
> 2018-08-15 1:30 GMT+08:00 Andrew Purtell <ap...@apache.org>:
>
> > FWIW latest stable release of HBase is 1.4.6, released 14 days ago.
> >
> >
> >
> > On Tue, Aug 14, 2018 at 1:12 AM Jun HE <ry...@gmail.com> wrote:
> >
> > > Hi, folks,
> > >
> > > In BIGTOP-2945 I listed the proposed changes, comparing to v1.2.1. A
> > brief
> > > is:
> > >
> > > ambari 2.5.0 => 2.6.1
> > > crunch 0.14.0 => 0.15.0
> > > flink 1.1.3 => 1.4.2
> > > gpdb 5.0.0-alpha.0 => 5.10.0
> > > hadoop 2.7.3 => 2.8.4
> > > hama 0.7.0 => 0.7.1
> > > hbase 1.1.9 => 1.3.2
> > > hive 1.2.1 => 2.3.3
> > > kafka 0.10.1.1 => 0.10.2.2
> > > phoenix 4.9.0-HBase-1.1 => 4.13.1-HBase-1.3
> > > qfs 1.1.4 => 2.0.0
> > > solr 4.10.4 => 6.6.0
> > > spark 2.1.1 => 2.2.1
> > > tez 0.6.2 => 0.9.1
> > > ycsb 0.4.0 => 0.12.0
> > > zeppelin 0.7.2 => 0.7.3
> > >
> > > If there are no more comments on these versions change, I'd like to
> > > finalize the BOM, create responding JIRAs and link them to BIGTOP-3063.
> > >
> > > Thanks & regards,
> > >
> > > Jun
> > >
> > >
> > >
> > > 2018-08-08 19:01 GMT+08:00 Dan Moraru <dm...@ligo-wa.caltech.edu>:
> > >
> > > > How much more work would be involved in generating a separate set of
> > > > Hadoop 3 packages for those who would like to try out HDFS erasure
> > coding
> > > > (among other things)?
> > > >
> > > > Thanks,
> > > >
> > > > Dan
> > > >
> > > >
> > > >
> > > >
> > > > On 08/07/2018 10:04 PM, Jun HE wrote:
> > > >
> > > >> Thanks for your inputs.
> > > >>
> > > >> For failing packages I agree with Olaf and Evans: remove them if no
> > one
> > > >> would take over the fix task.
> > > >> Oozie and pig have JIRAs(BIGTOP-2986/BIGTOP-2975). For crunch there
> > > >> isn't.
> > > >> I'll create one later and do some initial investigation to see if I
> > can
> > > >> fix
> > > >> it or not.
> > > >> For Hadoop upgrade, I'm fine to upgrade it to 2.8.4. Actually we've
> > done
> > > >> this in Linaro's ERP-18.06 release. (Yes, we're using Bigtop to
> build
> > > >> BigData packages :)) So there is no problem for it.
> > > >>
> > > >> Regards,
> > > >>
> > > >> Jun
> > > >>
> > > >> 2018-08-08 10:31 GMT+08:00 Evans Ye <ev...@apache.org>:
> > > >>
> > > >> Yes. We've features/bugfixes stacked up over time and definitely
> need
> > a
> > > >>> release.
> > > >>> I'm +1 to what Olaf said. Unless someone is willing to take over
> the
> > > task
> > > >>> and fix it.
> > > >>> From my perspective, Oozie and Pig's market are declining. So I
> > > suggest
> > > >>> let's spend our effort on those major components, and the
> > functionality
> > > >>> that Bigtop provides such as deployment and testing.
> > > >>>
> > > >>>
> > > >>> Naresh Bhat <na...@linaro.org> 於 2018年8月8日 週三 上午8:58寫道：
> > > >>>
> > > >>> +1
> > > >>>>
> > > >>>> On 7 August 2018 at 21:41, Jun HE <ju...@apache.org> wrote:
> > > >>>>
> > > >>>> Hi, folks,
> > > >>>>>
> > > >>>>> As you may read from previous thread, I'm proposing to do a new
> > > >>>>>
> > > >>>> release,
> > > >>>
> > > >>>> 1.3.0, before ApacheCon in Sept.
> > > >>>>> I'm a +1 for this because:
> > > >>>>> * quite a few fixes/improvements for building and deployment
> > > >>>>> * important components upgrade
> > > >>>>> * x86 and non-x86 platforms are aligned in master
> > > >>>>>
> > > >>>>> If there are enough votes for a release I'd like to volunteer to
> be
> > > the
> > > >>>>>
> > > >>>> RM
> > > >>>>
> > > >>>>> for 1.3.0.
> > > >>>>>
> > > >>>>> Evans has initiated a well-defined release BOM (
> > > >>>>> https://issues.apache.org/jira/browse/BIGTOP-2945), and I did
> some
> > > >>>>>
> > > >>>> minor
> > > >>>
> > > >>>> modifications mainly about components versions in 1.3.0.
> > > >>>>> And issues might need fix: https://s.apache.org/Ja7e
> > > >>>>>
> > > >>>>> Your comments/thoughts are mostly welcomed!
> > > >>>>>
> > > >>>>> Regards,
> > > >>>>>
> > > >>>>> Jun
> > > >>>>>
> > > >>>>>
> > > >
> > >
> >
> >
> > --
> > Best regards,
> > Andrew
> >
> > Words like orphans lost among the crosstalk, meaning torn from truth's
> > decrepit hands
> > - A23, Crosstalk
> >
>
--
Best regards,
Andrew
Words like orphans lost among the crosstalk, meaning torn from truth's
decrepit hands
- A23, Crosstalk
Re: [DISCUSS] Bigtop 1.3.0 release
Posted by Jun HE <ju...@apache.org>.
Thanks for the info, Andrew.
Did a quick check on HBase 1.4.6, build failed with:
-------------------------------------------------------------------------------------------
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-site-plugin:3.4:site (default-site) on
project hbase: Error generating maven-javadoc-plugin:2.10.3:aggregate:
[ERROR] Exit code: 1 - warning: unknown enum constant When.MAYBE
[ERROR] reason: class file for javax.annotation.meta.When not found
[ERROR] warning: unknown enum constant When.ALWAYS
[ERROR] warning: unknown enum constant When.UNKNOWN
[ERROR] javadoc: error - class file for
javax.annotation.concurrent.Immutable not found
-------------------------------------------------------------------------------------------
Might related to this (https://issues.apache.org/jira/browse/HBASE-19663).
If someone can help to fix this, I'm more than happy to include 1.4.6 in
v1.3 release. :)
2018-08-15 1:30 GMT+08:00 Andrew Purtell <ap...@apache.org>:
> FWIW latest stable release of HBase is 1.4.6, released 14 days ago.
>
>
>
> On Tue, Aug 14, 2018 at 1:12 AM Jun HE <ry...@gmail.com> wrote:
>
> > Hi, folks,
> >
> > In BIGTOP-2945 I listed the proposed changes, comparing to v1.2.1. A
> brief
> > is:
> >
> > ambari 2.5.0 => 2.6.1
> > crunch 0.14.0 => 0.15.0
> > flink 1.1.3 => 1.4.2
> > gpdb 5.0.0-alpha.0 => 5.10.0
> > hadoop 2.7.3 => 2.8.4
> > hama 0.7.0 => 0.7.1
> > hbase 1.1.9 => 1.3.2
> > hive 1.2.1 => 2.3.3
> > kafka 0.10.1.1 => 0.10.2.2
> > phoenix 4.9.0-HBase-1.1 => 4.13.1-HBase-1.3
> > qfs 1.1.4 => 2.0.0
> > solr 4.10.4 => 6.6.0
> > spark 2.1.1 => 2.2.1
> > tez 0.6.2 => 0.9.1
> > ycsb 0.4.0 => 0.12.0
> > zeppelin 0.7.2 => 0.7.3
> >
> > If there are no more comments on these versions change, I'd like to
> > finalize the BOM, create responding JIRAs and link them to BIGTOP-3063.
> >
> > Thanks & regards,
> >
> > Jun
> >
> >
> >
> > 2018-08-08 19:01 GMT+08:00 Dan Moraru <dm...@ligo-wa.caltech.edu>:
> >
> > > How much more work would be involved in generating a separate set of
> > > Hadoop 3 packages for those who would like to try out HDFS erasure
> coding
> > > (among other things)?
> > >
> > > Thanks,
> > >
> > > Dan
> > >
> > >
> > >
> > >
> > > On 08/07/2018 10:04 PM, Jun HE wrote:
> > >
> > >> Thanks for your inputs.
> > >>
> > >> For failing packages I agree with Olaf and Evans: remove them if no
> one
> > >> would take over the fix task.
> > >> Oozie and pig have JIRAs(BIGTOP-2986/BIGTOP-2975). For crunch there
> > >> isn't.
> > >> I'll create one later and do some initial investigation to see if I
> can
> > >> fix
> > >> it or not.
> > >> For Hadoop upgrade, I'm fine to upgrade it to 2.8.4. Actually we've
> done
> > >> this in Linaro's ERP-18.06 release. (Yes, we're using Bigtop to build
> > >> BigData packages :)) So there is no problem for it.
> > >>
> > >> Regards,
> > >>
> > >> Jun
> > >>
> > >> 2018-08-08 10:31 GMT+08:00 Evans Ye <ev...@apache.org>:
> > >>
> > >> Yes. We've features/bugfixes stacked up over time and definitely need
> a
> > >>> release.
> > >>> I'm +1 to what Olaf said. Unless someone is willing to take over the
> > task
> > >>> and fix it.
> > >>> From my perspective, Oozie and Pig's market are declining. So I
> > suggest
> > >>> let's spend our effort on those major components, and the
> functionality
> > >>> that Bigtop provides such as deployment and testing.
> > >>>
> > >>>
> > >>> Naresh Bhat <na...@linaro.org> 於 2018年8月8日 週三 上午8:58寫道：
> > >>>
> > >>> +1
> > >>>>
> > >>>> On 7 August 2018 at 21:41, Jun HE <ju...@apache.org> wrote:
> > >>>>
> > >>>> Hi, folks,
> > >>>>>
> > >>>>> As you may read from previous thread, I'm proposing to do a new
> > >>>>>
> > >>>> release,
> > >>>
> > >>>> 1.3.0, before ApacheCon in Sept.
> > >>>>> I'm a +1 for this because:
> > >>>>> * quite a few fixes/improvements for building and deployment
> > >>>>> * important components upgrade
> > >>>>> * x86 and non-x86 platforms are aligned in master
> > >>>>>
> > >>>>> If there are enough votes for a release I'd like to volunteer to be
> > the
> > >>>>>
> > >>>> RM
> > >>>>
> > >>>>> for 1.3.0.
> > >>>>>
> > >>>>> Evans has initiated a well-defined release BOM (
> > >>>>> https://issues.apache.org/jira/browse/BIGTOP-2945), and I did some
> > >>>>>
> > >>>> minor
> > >>>
> > >>>> modifications mainly about components versions in 1.3.0.
> > >>>>> And issues might need fix: https://s.apache.org/Ja7e
> > >>>>>
> > >>>>> Your comments/thoughts are mostly welcomed!
> > >>>>>
> > >>>>> Regards,
> > >>>>>
> > >>>>> Jun
> > >>>>>
> > >>>>>
> > >
> >
>
>
> --
> Best regards,
> Andrew
>
> Words like orphans lost among the crosstalk, meaning torn from truth's
> decrepit hands
> - A23, Crosstalk
>
Re: [DISCUSS] Bigtop 1.3.0 release
Posted by Andrew Purtell <ap...@apache.org>.
FWIW latest stable release of HBase is 1.4.6, released 14 days ago.
On Tue, Aug 14, 2018 at 1:12 AM Jun HE <ry...@gmail.com> wrote:
> Hi, folks,
>
> In BIGTOP-2945 I listed the proposed changes, comparing to v1.2.1. A brief
> is:
>
> ambari 2.5.0 => 2.6.1
> crunch 0.14.0 => 0.15.0
> flink 1.1.3 => 1.4.2
> gpdb 5.0.0-alpha.0 => 5.10.0
> hadoop 2.7.3 => 2.8.4
> hama 0.7.0 => 0.7.1
> hbase 1.1.9 => 1.3.2
> hive 1.2.1 => 2.3.3
> kafka 0.10.1.1 => 0.10.2.2
> phoenix 4.9.0-HBase-1.1 => 4.13.1-HBase-1.3
> qfs 1.1.4 => 2.0.0
> solr 4.10.4 => 6.6.0
> spark 2.1.1 => 2.2.1
> tez 0.6.2 => 0.9.1
> ycsb 0.4.0 => 0.12.0
> zeppelin 0.7.2 => 0.7.3
>
> If there are no more comments on these versions change, I'd like to
> finalize the BOM, create responding JIRAs and link them to BIGTOP-3063.
>
> Thanks & regards,
>
> Jun
>
>
>
> 2018-08-08 19:01 GMT+08:00 Dan Moraru <dm...@ligo-wa.caltech.edu>:
>
> > How much more work would be involved in generating a separate set of
> > Hadoop 3 packages for those who would like to try out HDFS erasure coding
> > (among other things)?
> >
> > Thanks,
> >
> > Dan
> >
> >
> >
> >
> > On 08/07/2018 10:04 PM, Jun HE wrote:
> >
> >> Thanks for your inputs.
> >>
> >> For failing packages I agree with Olaf and Evans: remove them if no one
> >> would take over the fix task.
> >> Oozie and pig have JIRAs(BIGTOP-2986/BIGTOP-2975). For crunch there
> >> isn't.
> >> I'll create one later and do some initial investigation to see if I can
> >> fix
> >> it or not.
> >> For Hadoop upgrade, I'm fine to upgrade it to 2.8.4. Actually we've done
> >> this in Linaro's ERP-18.06 release. (Yes, we're using Bigtop to build
> >> BigData packages :)) So there is no problem for it.
> >>
> >> Regards,
> >>
> >> Jun
> >>
> >> 2018-08-08 10:31 GMT+08:00 Evans Ye <ev...@apache.org>:
> >>
> >> Yes. We've features/bugfixes stacked up over time and definitely need a
> >>> release.
> >>> I'm +1 to what Olaf said. Unless someone is willing to take over the
> task
> >>> and fix it.
> >>> From my perspective, Oozie and Pig's market are declining. So I
> suggest
> >>> let's spend our effort on those major components, and the functionality
> >>> that Bigtop provides such as deployment and testing.
> >>>
> >>>
> >>> Naresh Bhat <na...@linaro.org> 於 2018年8月8日 週三 上午8:58寫道：
> >>>
> >>> +1
> >>>>
> >>>> On 7 August 2018 at 21:41, Jun HE <ju...@apache.org> wrote:
> >>>>
> >>>> Hi, folks,
> >>>>>
> >>>>> As you may read from previous thread, I'm proposing to do a new
> >>>>>
> >>>> release,
> >>>
> >>>> 1.3.0, before ApacheCon in Sept.
> >>>>> I'm a +1 for this because:
> >>>>> * quite a few fixes/improvements for building and deployment
> >>>>> * important components upgrade
> >>>>> * x86 and non-x86 platforms are aligned in master
> >>>>>
> >>>>> If there are enough votes for a release I'd like to volunteer to be
> the
> >>>>>
> >>>> RM
> >>>>
> >>>>> for 1.3.0.
> >>>>>
> >>>>> Evans has initiated a well-defined release BOM (
> >>>>> https://issues.apache.org/jira/browse/BIGTOP-2945), and I did some
> >>>>>
> >>>> minor
> >>>
> >>>> modifications mainly about components versions in 1.3.0.
> >>>>> And issues might need fix: https://s.apache.org/Ja7e
> >>>>>
> >>>>> Your comments/thoughts are mostly welcomed!
> >>>>>
> >>>>> Regards,
> >>>>>
> >>>>> Jun
> >>>>>
> >>>>>
> >
>
--
Best regards,
Andrew
Words like orphans lost among the crosstalk, meaning torn from truth's
decrepit hands
- A23, Crosstalk
Re: [DISCUSS] Bigtop 1.3.0 release
Posted by Evans Ye <ev...@apache.org>.
The BOM looks good to me.
Shall we add Ubuntu 18.04? Any distro-wise input?
Jun HE <ry...@gmail.com> 於 2018年8月14日 週二 下午4:12寫道：
> Hi, folks,
>
> In BIGTOP-2945 I listed the proposed changes, comparing to v1.2.1. A brief
> is:
>
> ambari 2.5.0 => 2.6.1
> crunch 0.14.0 => 0.15.0
> flink 1.1.3 => 1.4.2
> gpdb 5.0.0-alpha.0 => 5.10.0
> hadoop 2.7.3 => 2.8.4
> hama 0.7.0 => 0.7.1
> hbase 1.1.9 => 1.3.2
> hive 1.2.1 => 2.3.3
> kafka 0.10.1.1 => 0.10.2.2
> phoenix 4.9.0-HBase-1.1 => 4.13.1-HBase-1.3
> qfs 1.1.4 => 2.0.0
> solr 4.10.4 => 6.6.0
> spark 2.1.1 => 2.2.1
> tez 0.6.2 => 0.9.1
> ycsb 0.4.0 => 0.12.0
> zeppelin 0.7.2 => 0.7.3
>
> If there are no more comments on these versions change, I'd like to
> finalize the BOM, create responding JIRAs and link them to BIGTOP-3063.
>
> Thanks & regards,
>
> Jun
>
>
>
> 2018-08-08 19:01 GMT+08:00 Dan Moraru <dm...@ligo-wa.caltech.edu>:
>
> > How much more work would be involved in generating a separate set of
> > Hadoop 3 packages for those who would like to try out HDFS erasure coding
> > (among other things)?
> >
> > Thanks,
> >
> > Dan
> >
> >
> >
> >
> > On 08/07/2018 10:04 PM, Jun HE wrote:
> >
> >> Thanks for your inputs.
> >>
> >> For failing packages I agree with Olaf and Evans: remove them if no one
> >> would take over the fix task.
> >> Oozie and pig have JIRAs(BIGTOP-2986/BIGTOP-2975). For crunch there
> >> isn't.
> >> I'll create one later and do some initial investigation to see if I can
> >> fix
> >> it or not.
> >> For Hadoop upgrade, I'm fine to upgrade it to 2.8.4. Actually we've done
> >> this in Linaro's ERP-18.06 release. (Yes, we're using Bigtop to build
> >> BigData packages :)) So there is no problem for it.
> >>
> >> Regards,
> >>
> >> Jun
> >>
> >> 2018-08-08 10:31 GMT+08:00 Evans Ye <ev...@apache.org>:
> >>
> >> Yes. We've features/bugfixes stacked up over time and definitely need a
> >>> release.
> >>> I'm +1 to what Olaf said. Unless someone is willing to take over the
> task
> >>> and fix it.
> >>> From my perspective, Oozie and Pig's market are declining. So I
> suggest
> >>> let's spend our effort on those major components, and the functionality
> >>> that Bigtop provides such as deployment and testing.
> >>>
> >>>
> >>> Naresh Bhat <na...@linaro.org> 於 2018年8月8日 週三 上午8:58寫道：
> >>>
> >>> +1
> >>>>
> >>>> On 7 August 2018 at 21:41, Jun HE <ju...@apache.org> wrote:
> >>>>
> >>>> Hi, folks,
> >>>>>
> >>>>> As you may read from previous thread, I'm proposing to do a new
> >>>>>
> >>>> release,
> >>>
> >>>> 1.3.0, before ApacheCon in Sept.
> >>>>> I'm a +1 for this because:
> >>>>> * quite a few fixes/improvements for building and deployment
> >>>>> * important components upgrade
> >>>>> * x86 and non-x86 platforms are aligned in master
> >>>>>
> >>>>> If there are enough votes for a release I'd like to volunteer to be
> the
> >>>>>
> >>>> RM
> >>>>
> >>>>> for 1.3.0.
> >>>>>
> >>>>> Evans has initiated a well-defined release BOM (
> >>>>> https://issues.apache.org/jira/browse/BIGTOP-2945), and I did some
> >>>>>
> >>>> minor
> >>>
> >>>> modifications mainly about components versions in 1.3.0.
> >>>>> And issues might need fix: https://s.apache.org/Ja7e
> >>>>>
> >>>>> Your comments/thoughts are mostly welcomed!
> >>>>>
> >>>>> Regards,
> >>>>>
> >>>>> Jun
> >>>>>
> >>>>>
> >
>
Re: [DISCUSS] Bigtop 1.3.0 release
Posted by Jun HE <ry...@gmail.com>.
Hi, folks,
In BIGTOP-2945 I listed the proposed changes, comparing to v1.2.1. A brief
is:
ambari 2.5.0 => 2.6.1
crunch 0.14.0 => 0.15.0
flink 1.1.3 => 1.4.2
gpdb 5.0.0-alpha.0 => 5.10.0
hadoop 2.7.3 => 2.8.4
hama 0.7.0 => 0.7.1
hbase 1.1.9 => 1.3.2
hive 1.2.1 => 2.3.3
kafka 0.10.1.1 => 0.10.2.2
phoenix 4.9.0-HBase-1.1 => 4.13.1-HBase-1.3
qfs 1.1.4 => 2.0.0
solr 4.10.4 => 6.6.0
spark 2.1.1 => 2.2.1
tez 0.6.2 => 0.9.1
ycsb 0.4.0 => 0.12.0
zeppelin 0.7.2 => 0.7.3
If there are no more comments on these versions change, I'd like to
finalize the BOM, create responding JIRAs and link them to BIGTOP-3063.
Thanks & regards,
Jun
2018-08-08 19:01 GMT+08:00 Dan Moraru <dm...@ligo-wa.caltech.edu>:
> How much more work would be involved in generating a separate set of
> Hadoop 3 packages for those who would like to try out HDFS erasure coding
> (among other things)?
>
> Thanks,
>
> Dan
>
>
>
>
> On 08/07/2018 10:04 PM, Jun HE wrote:
>
>> Thanks for your inputs.
>>
>> For failing packages I agree with Olaf and Evans: remove them if no one
>> would take over the fix task.
>> Oozie and pig have JIRAs(BIGTOP-2986/BIGTOP-2975). For crunch there
>> isn't.
>> I'll create one later and do some initial investigation to see if I can
>> fix
>> it or not.
>> For Hadoop upgrade, I'm fine to upgrade it to 2.8.4. Actually we've done
>> this in Linaro's ERP-18.06 release. (Yes, we're using Bigtop to build
>> BigData packages :)) So there is no problem for it.
>>
>> Regards,
>>
>> Jun
>>
>> 2018-08-08 10:31 GMT+08:00 Evans Ye <ev...@apache.org>:
>>
>> Yes. We've features/bugfixes stacked up over time and definitely need a
>>> release.
>>> I'm +1 to what Olaf said. Unless someone is willing to take over the task
>>> and fix it.
>>> From my perspective, Oozie and Pig's market are declining. So I suggest
>>> let's spend our effort on those major components, and the functionality
>>> that Bigtop provides such as deployment and testing.
>>>
>>>
>>> Naresh Bhat <na...@linaro.org> 於 2018年8月8日 週三 上午8:58寫道：
>>>
>>> +1
>>>>
>>>> On 7 August 2018 at 21:41, Jun HE <ju...@apache.org> wrote:
>>>>
>>>> Hi, folks,
>>>>>
>>>>> As you may read from previous thread, I'm proposing to do a new
>>>>>
>>>> release,
>>>
>>>> 1.3.0, before ApacheCon in Sept.
>>>>> I'm a +1 for this because:
>>>>> * quite a few fixes/improvements for building and deployment
>>>>> * important components upgrade
>>>>> * x86 and non-x86 platforms are aligned in master
>>>>>
>>>>> If there are enough votes for a release I'd like to volunteer to be the
>>>>>
>>>> RM
>>>>
>>>>> for 1.3.0.
>>>>>
>>>>> Evans has initiated a well-defined release BOM (
>>>>> https://issues.apache.org/jira/browse/BIGTOP-2945), and I did some
>>>>>
>>>> minor
>>>
>>>> modifications mainly about components versions in 1.3.0.
>>>>> And issues might need fix: https://s.apache.org/Ja7e
>>>>>
>>>>> Your comments/thoughts are mostly welcomed!
>>>>>
>>>>> Regards,
>>>>>
>>>>> Jun
>>>>>
>>>>>
>
Re: [DISCUSS] Bigtop 1.3.0 release
Posted by Dan Moraru <dm...@ligo-wa.caltech.edu>.
How much more work would be involved in generating a separate set of
Hadoop 3 packages for those who would like to try out HDFS erasure
coding (among other things)?
Thanks,
Dan
On 08/07/2018 10:04 PM, Jun HE wrote:
> Thanks for your inputs.
>
> For failing packages I agree with Olaf and Evans: remove them if no one
> would take over the fix task.
> Oozie and pig have JIRAs(BIGTOP-2986/BIGTOP-2975). For crunch there isn't.
> I'll create one later and do some initial investigation to see if I can fix
> it or not.
> For Hadoop upgrade, I'm fine to upgrade it to 2.8.4. Actually we've done
> this in Linaro's ERP-18.06 release. (Yes, we're using Bigtop to build
> BigData packages :)) So there is no problem for it.
>
> Regards,
>
> Jun
>
> 2018-08-08 10:31 GMT+08:00 Evans Ye <ev...@apache.org>:
>
>> Yes. We've features/bugfixes stacked up over time and definitely need a
>> release.
>> I'm +1 to what Olaf said. Unless someone is willing to take over the task
>> and fix it.
>> From my perspective, Oozie and Pig's market are declining. So I suggest
>> let's spend our effort on those major components, and the functionality
>> that Bigtop provides such as deployment and testing.
>>
>>
>> Naresh Bhat <na...@linaro.org> 於 2018年8月8日 週三 上午8:58寫道：
>>
>>> +1
>>>
>>> On 7 August 2018 at 21:41, Jun HE <ju...@apache.org> wrote:
>>>
>>>> Hi, folks,
>>>>
>>>> As you may read from previous thread, I'm proposing to do a new
>> release,
>>>> 1.3.0, before ApacheCon in Sept.
>>>> I'm a +1 for this because:
>>>> * quite a few fixes/improvements for building and deployment
>>>> * important components upgrade
>>>> * x86 and non-x86 platforms are aligned in master
>>>>
>>>> If there are enough votes for a release I'd like to volunteer to be the
>>> RM
>>>> for 1.3.0.
>>>>
>>>> Evans has initiated a well-defined release BOM (
>>>> https://issues.apache.org/jira/browse/BIGTOP-2945), and I did some
>> minor
>>>> modifications mainly about components versions in 1.3.0.
>>>> And issues might need fix: https://s.apache.org/Ja7e
>>>>
>>>> Your comments/thoughts are mostly welcomed!
>>>>
>>>> Regards,
>>>>
>>>> Jun
>>>>
Re: [DISCUSS] Bigtop 1.3.0 release
Posted by Jun HE <ry...@gmail.com>.
Thanks for your inputs.
For failing packages I agree with Olaf and Evans: remove them if no one
would take over the fix task.
Oozie and pig have JIRAs(BIGTOP-2986/BIGTOP-2975). For crunch there isn't.
I'll create one later and do some initial investigation to see if I can fix
it or not.
For Hadoop upgrade, I'm fine to upgrade it to 2.8.4. Actually we've done
this in Linaro's ERP-18.06 release. (Yes, we're using Bigtop to build
BigData packages :)) So there is no problem for it.
Regards,
Jun
2018-08-08 10:31 GMT+08:00 Evans Ye <ev...@apache.org>:
> Yes. We've features/bugfixes stacked up over time and definitely need a
> release.
> I'm +1 to what Olaf said. Unless someone is willing to take over the task
> and fix it.
> From my perspective, Oozie and Pig's market are declining. So I suggest
> let's spend our effort on those major components, and the functionality
> that Bigtop provides such as deployment and testing.
>
>
> Naresh Bhat <na...@linaro.org> 於 2018年8月8日 週三 上午8:58寫道：
>
> > +1
> >
> > On 7 August 2018 at 21:41, Jun HE <ju...@apache.org> wrote:
> >
> > > Hi, folks,
> > >
> > > As you may read from previous thread, I'm proposing to do a new
> release,
> > > 1.3.0, before ApacheCon in Sept.
> > > I'm a +1 for this because:
> > > * quite a few fixes/improvements for building and deployment
> > > * important components upgrade
> > > * x86 and non-x86 platforms are aligned in master
> > >
> > > If there are enough votes for a release I'd like to volunteer to be the
> > RM
> > > for 1.3.0.
> > >
> > > Evans has initiated a well-defined release BOM (
> > > https://issues.apache.org/jira/browse/BIGTOP-2945), and I did some
> minor
> > > modifications mainly about components versions in 1.3.0.
> > > And issues might need fix: https://s.apache.org/Ja7e
> > >
> > > Your comments/thoughts are mostly welcomed!
> > >
> > > Regards,
> > >
> > > Jun
> > >
> >
>
Re: [DISCUSS] Bigtop 1.3.0 release
Posted by Evans Ye <ev...@apache.org>.
Yes. We've features/bugfixes stacked up over time and definitely need a
release.
I'm +1 to what Olaf said. Unless someone is willing to take over the task
and fix it.
From my perspective, Oozie and Pig's market are declining. So I suggest
let's spend our effort on those major components, and the functionality
that Bigtop provides such as deployment and testing.
Naresh Bhat <na...@linaro.org> 於 2018年8月8日 週三 上午8:58寫道：
> +1
>
> On 7 August 2018 at 21:41, Jun HE <ju...@apache.org> wrote:
>
> > Hi, folks,
> >
> > As you may read from previous thread, I'm proposing to do a new release,
> > 1.3.0, before ApacheCon in Sept.
> > I'm a +1 for this because:
> > * quite a few fixes/improvements for building and deployment
> > * important components upgrade
> > * x86 and non-x86 platforms are aligned in master
> >
> > If there are enough votes for a release I'd like to volunteer to be the
> RM
> > for 1.3.0.
> >
> > Evans has initiated a well-defined release BOM (
> > https://issues.apache.org/jira/browse/BIGTOP-2945), and I did some minor
> > modifications mainly about components versions in 1.3.0.
> > And issues might need fix: https://s.apache.org/Ja7e
> >
> > Your comments/thoughts are mostly welcomed!
> >
> > Regards,
> >
> > Jun
> >
>
Re: [DISCUSS] Bigtop 1.3.0 release
Posted by Naresh Bhat <na...@linaro.org>.
+1
On 7 August 2018 at 21:41, Jun HE <ju...@apache.org> wrote:
> Hi, folks,
>
> As you may read from previous thread, I'm proposing to do a new release,
> 1.3.0, before ApacheCon in Sept.
> I'm a +1 for this because:
> * quite a few fixes/improvements for building and deployment
> * important components upgrade
> * x86 and non-x86 platforms are aligned in master
>
> If there are enough votes for a release I'd like to volunteer to be the RM
> for 1.3.0.
>
> Evans has initiated a well-defined release BOM (
> https://issues.apache.org/jira/browse/BIGTOP-2945), and I did some minor
> modifications mainly about components versions in 1.3.0.
> And issues might need fix: https://s.apache.org/Ja7e
>
> Your comments/thoughts are mostly welcomed!
>
> Regards,
>
> Jun
>
Re: [DISCUSS] Bigtop 1.3.0 release
Posted by Ganesh Raju <ga...@linaro.org>.
+1
On Tue, Aug 7, 2018 at 11:12 AM Jun HE <ju...@apache.org> wrote:
> Hi, folks,
>
> As you may read from previous thread, I'm proposing to do a new release,
> 1.3.0, before ApacheCon in Sept.
> I'm a +1 for this because:
> * quite a few fixes/improvements for building and deployment
> * important components upgrade
> * x86 and non-x86 platforms are aligned in master
>
> If there are enough votes for a release I'd like to volunteer to be the RM
> for 1.3.0.
>
> Evans has initiated a well-defined release BOM (
> https://issues.apache.org/jira/browse/BIGTOP-2945), and I did some minor
> modifications mainly about components versions in 1.3.0.
> And issues might need fix: https://s.apache.org/Ja7e
>
> Your comments/thoughts are mostly welcomed!
>
> Regards,
>
> Jun
>
--
IRC: ganeshraju@#linaro on irc.freenode.ne <http://irc.freenode.net/>t
Re: [DISCUSS] Bigtop 1.3.0 release
Posted by Kevin Monroe <ke...@canonical.com>.
+1 on a 1.3.0 release. Thanks for volunteering as the RM!
-Kevin
On Tue, Aug 7, 2018 at 11:12 AM Jun HE <ju...@apache.org> wrote:
> Hi, folks,
>
> As you may read from previous thread, I'm proposing to do a new release,
> 1.3.0, before ApacheCon in Sept.
> I'm a +1 for this because:
> * quite a few fixes/improvements for building and deployment
> * important components upgrade
> * x86 and non-x86 platforms are aligned in master
>
> If there are enough votes for a release I'd like to volunteer to be the RM
> for 1.3.0.
>
> Evans has initiated a well-defined release BOM (
> https://issues.apache.org/jira/browse/BIGTOP-2945), and I did some minor
> modifications mainly about components versions in 1.3.0.
> And issues might need fix: https://s.apache.org/Ja7e
>
> Your comments/thoughts are mostly welcomed!
>
> Regards,
>
> Jun
>