You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by Evans Ye <ev...@apache.org> on 2019/03/04 07:11:17 UTC
[DISCUSS] Jump to 1.4.0 release directly instead of 1.3.1
Hi all,
The overhaul as well as development is almost completed. With several new
features added, I'd like to propose to release Bigtop 1.4.0 directly
instead of doing 1.3.1 release(https://s.apache.org/2a8K). This helps to
promote these new features:
1. Full support to build and test inside docker with one stop seamlessly
integration.
Now just one command to build and test components:
Example:
$ ./gradlew spark-ind repo-ind docker-provisioner -POS=ubuntu-16.04 -Pnexus
-Penable_local_repo -Pstack=spark-standalone -Psmoke-tests=spark
2. Support to build from git commit hash
Example:
$ ./gradlew kafka-pkg-ind -Pgit_repo=https://github.com/apache/kafka.git
-Pgit_ref=1.1 -Pgit_sha1=4dae083af486eaedd27c69c973c74605bffd416b
-Pbase_version=1.1.1
3. Version bumps and tests
Hadoop 2.8.5, Kafka 1.1.1, Spark 2.4.0, Alluxio 1.8.1
New smoke tests: Flink, Giraph (Crunch)
4. Loooooooots of bug fixes!
If agreed, please +1 and I'll still serve as the RM for 1.4 release, if no
preemption ;)
Best,
Evans
Re: [DISCUSS] Jump to 1.4.0 release directly instead of 1.3.1
Posted by "Youngwoo Kim (김영우)" <yw...@apache.org>.
+1 to your idea, Evans.
Thanks,
Youngwoo
On Mon, Mar 4, 2019 at 4:11 PM Evans Ye <ev...@apache.org> wrote:
> Hi all,
>
> The overhaul as well as development is almost completed. With several new
> features added, I'd like to propose to release Bigtop 1.4.0 directly
> instead of doing 1.3.1 release(https://s.apache.org/2a8K). This helps to
> promote these new features:
>
> 1. Full support to build and test inside docker with one stop seamlessly
> integration.
> Now just one command to build and test components:
> Example:
> $ ./gradlew spark-ind repo-ind docker-provisioner -POS=ubuntu-16.04 -Pnexus
> -Penable_local_repo -Pstack=spark-standalone -Psmoke-tests=spark
>
> 2. Support to build from git commit hash
> Example:
> $ ./gradlew kafka-pkg-ind -Pgit_repo=https://github.com/apache/kafka.git
> -Pgit_ref=1.1 <https://github.com/apache/kafka.git-Pgit_ref=1.1>
> -Pgit_sha1=4dae083af486eaedd27c69c973c74605bffd416b
> -Pbase_version=1.1.1
>
> 3. Version bumps and tests
> Hadoop 2.8.5, Kafka 1.1.1, Spark 2.4.0, Alluxio 1.8.1
> New smoke tests: Flink, Giraph (Crunch)
>
> 4. Loooooooots of bug fixes!
>
> If agreed, please +1 and I'll still serve as the RM for 1.4 release, if no
> preemption ;)
>
>
> Best,
> Evans
>
Re: [DISCUSS] Jump to 1.4.0 release directly instead of 1.3.1
Posted by Evans Ye <ev...@apache.org>.
A quick test but got no luck with hive:
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR]
/bigtop/build/hive/rpm/BUILD/apache-hive-2.3.3-src/ql/src/java/org/apache/hadoop/hive/ql/exec/spark/LocalHiveSparkClient.java:[87,26]
cannot find symbol
symbol: method
addListener(org.apache.hadoop.hive.ql.exec.spark.status.impl.JobMetricsListener)
location: class org.apache.spark.scheduler.LiveListenerBus
Though I feel strange why API not compatible with minor version change...
In summary, I've limited cycle and we have pressure on the release date.
So if someone is up for this, then great and I'll add in in 1.4.0.
Otherwise, we'll stick to the plan :)
Виктор Михайлов <vi...@gmail.com> 於 2019年3月13日 週三 下午7:13寫道：
> Yes
> I have successful build with Spark 2.3.1 and unsuccessful with 2.3.2
> With 2.3.3 I didn't try
>
> Victor
>
> ср, 13 мар. 2019 г., 13:43 Evans Ye <ev...@apache.org>:
>
> > Hey Victor,
> >
> > Did you tried?
> > As long as hive, ignite, oozie, zeppelin can built with Spark 2.3, then
> we
> > should be good to go.
> >
> >
> > Виктор Михайлов <vi...@gmail.com> 於 2019年3月13日 週三 下午4:06寫道：
> >
> > > What about Spark 2.3.x?
> > >
> > > Victor
> > >
> > > ср, 13 мар. 2019 г., 5:58 Evans Ye <ev...@apache.org>:
> > >
> > > > Thank you all for the support!
> > > >
> > > > Until now, I was dealing with various compatibility issue for Spark.
> > > > However there're too many incompatible components with spark 2.4.0:
> > > >
> > > >
> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
> > > >
> > > > Therefore I'd like to revoke the 2.4.0 upgrade and just slightly bump
> > it
> > > up
> > > > to 2.2.3.
> > > > The new release date is targeted at the end of March.
> > > >
> > > >
> > > >
> > > > Jun HE <ry...@gmail.com> 於 2019年3月6日 週三 下午9:28寫道：
> > > >
> > > > > Awesome, and huge +1 to this!
> > > > >
> > > > > Konstantin Boudnik <co...@boudnik.org> 于2019年3月6日周三 下午9:11写道：
> > > > >
> > > > > > Oh my, that's amazing! Thank you man!
> > > > > > +1 on moving directly to 1.4
> > > > > > --
> > > > > > With regards,
> > > > > > Konstantin
> > > > > >
> > > > > > On March 5, 2019 12:58:44 AM GMT+03:00, Olaf Flebbe <
> of@oflebbe.de
> > >
> > > > > wrote:
> > > > > > >Hi Evans,
> > > > > > >
> > > > > > >You found and fixed a lot of bugs and made the smoke tests work,
> > so
> > > a
> > > > > > >big +1 .
> > > > > > >
> > > > > > >Olaf
> > > > > > >
> > > > > > >> Am 04.03.2019 um 08:12 schrieb Evans Ye <ev...@apache.org>:
> > > > > > >>
> > > > > > >> -1 welcomed as well !!!
> > > > > > >>
> > > > > > >> Evans Ye <ev...@apache.org> 於 2019年3月4日 週一 下午3:11寫道：
> > > > > > >>
> > > > > > >>> Hi all,
> > > > > > >>>
> > > > > > >>> The overhaul as well as development is almost completed. With
> > > > > > >several new
> > > > > > >>> features added, I'd like to propose to release Bigtop 1.4.0
> > > > directly
> > > > > > >>> instead of doing 1.3.1 release(https://s.apache.org/2a8K).
> > This
> > > > > > >helps to
> > > > > > >>> promote these new features:
> > > > > > >>>
> > > > > > >>> 1. Full support to build and test inside docker with one stop
> > > > > > >seamlessly
> > > > > > >>> integration.
> > > > > > >>> Now just one command to build and test components:
> > > > > > >>> Example:
> > > > > > >>> $ ./gradlew spark-ind repo-ind docker-provisioner
> > > -POS=ubuntu-16.04
> > > > > > >>> -Pnexus -Penable_local_repo -Pstack=spark-standalone
> > > > > > >-Psmoke-tests=spark
> > > > > > >>>
> > > > > > >>> 2. Support to build from git commit hash
> > > > > > >>> Example:
> > > > > > >>> $ ./gradlew kafka-pkg-ind
> > > > > > >-Pgit_repo=https://github.com/apache/kafka.git
> > > > > > >>> -Pgit_ref=1.1
> > -Pgit_sha1=4dae083af486eaedd27c69c973c74605bffd416b
> > > > > > >>> -Pbase_version=1.1.1
> > > > > > >>>
> > > > > > >>> 3. Version bumps and tests
> > > > > > >>> Hadoop 2.8.5, Kafka 1.1.1, Spark 2.4.0, Alluxio 1.8.1
> > > > > > >>> New smoke tests: Flink, Giraph (Crunch)
> > > > > > >>>
> > > > > > >>> 4. Loooooooots of bug fixes!
> > > > > > >>>
> > > > > > >>> If agreed, please +1 and I'll still serve as the RM for 1.4
> > > > release,
> > > > > > >if no
> > > > > > >>> preemption ;)
> > > > > > >>>
> > > > > > >>>
> > > > > > >>> Best,
> > > > > > >>> Evans
> > > > > > >>>
> > > > > > >>>
> > > > > > >>>
> > > > > > >>>
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] Jump to 1.4.0 release directly instead of 1.3.1
Posted by Виктор Михайлов <vi...@gmail.com>.
Yes
I have successful build with Spark 2.3.1 and unsuccessful with 2.3.2
With 2.3.3 I didn't try
Victor
ср, 13 мар. 2019 г., 13:43 Evans Ye <ev...@apache.org>:
> Hey Victor,
>
> Did you tried?
> As long as hive, ignite, oozie, zeppelin can built with Spark 2.3, then we
> should be good to go.
>
>
> Виктор Михайлов <vi...@gmail.com> 於 2019年3月13日 週三 下午4:06寫道：
>
> > What about Spark 2.3.x?
> >
> > Victor
> >
> > ср, 13 мар. 2019 г., 5:58 Evans Ye <ev...@apache.org>:
> >
> > > Thank you all for the support!
> > >
> > > Until now, I was dealing with various compatibility issue for Spark.
> > > However there're too many incompatible components with spark 2.4.0:
> > >
> > > https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
> > >
> > > Therefore I'd like to revoke the 2.4.0 upgrade and just slightly bump
> it
> > up
> > > to 2.2.3.
> > > The new release date is targeted at the end of March.
> > >
> > >
> > >
> > > Jun HE <ry...@gmail.com> 於 2019年3月6日 週三 下午9:28寫道：
> > >
> > > > Awesome, and huge +1 to this!
> > > >
> > > > Konstantin Boudnik <co...@boudnik.org> 于2019年3月6日周三 下午9:11写道：
> > > >
> > > > > Oh my, that's amazing! Thank you man!
> > > > > +1 on moving directly to 1.4
> > > > > --
> > > > > With regards,
> > > > > Konstantin
> > > > >
> > > > > On March 5, 2019 12:58:44 AM GMT+03:00, Olaf Flebbe <of@oflebbe.de
> >
> > > > wrote:
> > > > > >Hi Evans,
> > > > > >
> > > > > >You found and fixed a lot of bugs and made the smoke tests work,
> so
> > a
> > > > > >big +1 .
> > > > > >
> > > > > >Olaf
> > > > > >
> > > > > >> Am 04.03.2019 um 08:12 schrieb Evans Ye <ev...@apache.org>:
> > > > > >>
> > > > > >> -1 welcomed as well !!!
> > > > > >>
> > > > > >> Evans Ye <ev...@apache.org> 於 2019年3月4日 週一 下午3:11寫道：
> > > > > >>
> > > > > >>> Hi all,
> > > > > >>>
> > > > > >>> The overhaul as well as development is almost completed. With
> > > > > >several new
> > > > > >>> features added, I'd like to propose to release Bigtop 1.4.0
> > > directly
> > > > > >>> instead of doing 1.3.1 release(https://s.apache.org/2a8K).
> This
> > > > > >helps to
> > > > > >>> promote these new features:
> > > > > >>>
> > > > > >>> 1. Full support to build and test inside docker with one stop
> > > > > >seamlessly
> > > > > >>> integration.
> > > > > >>> Now just one command to build and test components:
> > > > > >>> Example:
> > > > > >>> $ ./gradlew spark-ind repo-ind docker-provisioner
> > -POS=ubuntu-16.04
> > > > > >>> -Pnexus -Penable_local_repo -Pstack=spark-standalone
> > > > > >-Psmoke-tests=spark
> > > > > >>>
> > > > > >>> 2. Support to build from git commit hash
> > > > > >>> Example:
> > > > > >>> $ ./gradlew kafka-pkg-ind
> > > > > >-Pgit_repo=https://github.com/apache/kafka.git
> > > > > >>> -Pgit_ref=1.1
> -Pgit_sha1=4dae083af486eaedd27c69c973c74605bffd416b
> > > > > >>> -Pbase_version=1.1.1
> > > > > >>>
> > > > > >>> 3. Version bumps and tests
> > > > > >>> Hadoop 2.8.5, Kafka 1.1.1, Spark 2.4.0, Alluxio 1.8.1
> > > > > >>> New smoke tests: Flink, Giraph (Crunch)
> > > > > >>>
> > > > > >>> 4. Loooooooots of bug fixes!
> > > > > >>>
> > > > > >>> If agreed, please +1 and I'll still serve as the RM for 1.4
> > > release,
> > > > > >if no
> > > > > >>> preemption ;)
> > > > > >>>
> > > > > >>>
> > > > > >>> Best,
> > > > > >>> Evans
> > > > > >>>
> > > > > >>>
> > > > > >>>
> > > > > >>>
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] Jump to 1.4.0 release directly instead of 1.3.1
Posted by Evans Ye <ev...@apache.org>.
Hey Victor,
Did you tried?
As long as hive, ignite, oozie, zeppelin can built with Spark 2.3, then we
should be good to go.
Виктор Михайлов <vi...@gmail.com> 於 2019年3月13日 週三 下午4:06寫道：
> What about Spark 2.3.x?
>
> Victor
>
> ср, 13 мар. 2019 г., 5:58 Evans Ye <ev...@apache.org>:
>
> > Thank you all for the support!
> >
> > Until now, I was dealing with various compatibility issue for Spark.
> > However there're too many incompatible components with spark 2.4.0:
> >
> > https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
> >
> > Therefore I'd like to revoke the 2.4.0 upgrade and just slightly bump it
> up
> > to 2.2.3.
> > The new release date is targeted at the end of March.
> >
> >
> >
> > Jun HE <ry...@gmail.com> 於 2019年3月6日 週三 下午9:28寫道：
> >
> > > Awesome, and huge +1 to this!
> > >
> > > Konstantin Boudnik <co...@boudnik.org> 于2019年3月6日周三 下午9:11写道：
> > >
> > > > Oh my, that's amazing! Thank you man!
> > > > +1 on moving directly to 1.4
> > > > --
> > > > With regards,
> > > > Konstantin
> > > >
> > > > On March 5, 2019 12:58:44 AM GMT+03:00, Olaf Flebbe <of...@oflebbe.de>
> > > wrote:
> > > > >Hi Evans,
> > > > >
> > > > >You found and fixed a lot of bugs and made the smoke tests work, so
> a
> > > > >big +1 .
> > > > >
> > > > >Olaf
> > > > >
> > > > >> Am 04.03.2019 um 08:12 schrieb Evans Ye <ev...@apache.org>:
> > > > >>
> > > > >> -1 welcomed as well !!!
> > > > >>
> > > > >> Evans Ye <ev...@apache.org> 於 2019年3月4日 週一 下午3:11寫道：
> > > > >>
> > > > >>> Hi all,
> > > > >>>
> > > > >>> The overhaul as well as development is almost completed. With
> > > > >several new
> > > > >>> features added, I'd like to propose to release Bigtop 1.4.0
> > directly
> > > > >>> instead of doing 1.3.1 release(https://s.apache.org/2a8K). This
> > > > >helps to
> > > > >>> promote these new features:
> > > > >>>
> > > > >>> 1. Full support to build and test inside docker with one stop
> > > > >seamlessly
> > > > >>> integration.
> > > > >>> Now just one command to build and test components:
> > > > >>> Example:
> > > > >>> $ ./gradlew spark-ind repo-ind docker-provisioner
> -POS=ubuntu-16.04
> > > > >>> -Pnexus -Penable_local_repo -Pstack=spark-standalone
> > > > >-Psmoke-tests=spark
> > > > >>>
> > > > >>> 2. Support to build from git commit hash
> > > > >>> Example:
> > > > >>> $ ./gradlew kafka-pkg-ind
> > > > >-Pgit_repo=https://github.com/apache/kafka.git
> > > > >>> -Pgit_ref=1.1 -Pgit_sha1=4dae083af486eaedd27c69c973c74605bffd416b
> > > > >>> -Pbase_version=1.1.1
> > > > >>>
> > > > >>> 3. Version bumps and tests
> > > > >>> Hadoop 2.8.5, Kafka 1.1.1, Spark 2.4.0, Alluxio 1.8.1
> > > > >>> New smoke tests: Flink, Giraph (Crunch)
> > > > >>>
> > > > >>> 4. Loooooooots of bug fixes!
> > > > >>>
> > > > >>> If agreed, please +1 and I'll still serve as the RM for 1.4
> > release,
> > > > >if no
> > > > >>> preemption ;)
> > > > >>>
> > > > >>>
> > > > >>> Best,
> > > > >>> Evans
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>>
> > > >
> > >
> >
>
Re: [DISCUSS] Jump to 1.4.0 release directly instead of 1.3.1
Posted by Виктор Михайлов <vi...@gmail.com>.
What about Spark 2.3.x?
Victor
ср, 13 мар. 2019 г., 5:58 Evans Ye <ev...@apache.org>:
> Thank you all for the support!
>
> Until now, I was dealing with various compatibility issue for Spark.
> However there're too many incompatible components with spark 2.4.0:
>
> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
>
> Therefore I'd like to revoke the 2.4.0 upgrade and just slightly bump it up
> to 2.2.3.
> The new release date is targeted at the end of March.
>
>
>
> Jun HE <ry...@gmail.com> 於 2019年3月6日 週三 下午9:28寫道：
>
> > Awesome, and huge +1 to this!
> >
> > Konstantin Boudnik <co...@boudnik.org> 于2019年3月6日周三 下午9:11写道：
> >
> > > Oh my, that's amazing! Thank you man!
> > > +1 on moving directly to 1.4
> > > --
> > > With regards,
> > > Konstantin
> > >
> > > On March 5, 2019 12:58:44 AM GMT+03:00, Olaf Flebbe <of...@oflebbe.de>
> > wrote:
> > > >Hi Evans,
> > > >
> > > >You found and fixed a lot of bugs and made the smoke tests work, so a
> > > >big +1 .
> > > >
> > > >Olaf
> > > >
> > > >> Am 04.03.2019 um 08:12 schrieb Evans Ye <ev...@apache.org>:
> > > >>
> > > >> -1 welcomed as well !!!
> > > >>
> > > >> Evans Ye <ev...@apache.org> 於 2019年3月4日 週一 下午3:11寫道：
> > > >>
> > > >>> Hi all,
> > > >>>
> > > >>> The overhaul as well as development is almost completed. With
> > > >several new
> > > >>> features added, I'd like to propose to release Bigtop 1.4.0
> directly
> > > >>> instead of doing 1.3.1 release(https://s.apache.org/2a8K). This
> > > >helps to
> > > >>> promote these new features:
> > > >>>
> > > >>> 1. Full support to build and test inside docker with one stop
> > > >seamlessly
> > > >>> integration.
> > > >>> Now just one command to build and test components:
> > > >>> Example:
> > > >>> $ ./gradlew spark-ind repo-ind docker-provisioner -POS=ubuntu-16.04
> > > >>> -Pnexus -Penable_local_repo -Pstack=spark-standalone
> > > >-Psmoke-tests=spark
> > > >>>
> > > >>> 2. Support to build from git commit hash
> > > >>> Example:
> > > >>> $ ./gradlew kafka-pkg-ind
> > > >-Pgit_repo=https://github.com/apache/kafka.git
> > > >>> -Pgit_ref=1.1 -Pgit_sha1=4dae083af486eaedd27c69c973c74605bffd416b
> > > >>> -Pbase_version=1.1.1
> > > >>>
> > > >>> 3. Version bumps and tests
> > > >>> Hadoop 2.8.5, Kafka 1.1.1, Spark 2.4.0, Alluxio 1.8.1
> > > >>> New smoke tests: Flink, Giraph (Crunch)
> > > >>>
> > > >>> 4. Loooooooots of bug fixes!
> > > >>>
> > > >>> If agreed, please +1 and I'll still serve as the RM for 1.4
> release,
> > > >if no
> > > >>> preemption ;)
> > > >>>
> > > >>>
> > > >>> Best,
> > > >>> Evans
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > >
> >
>
Re: [DISCUSS] Jump to 1.4.0 release directly instead of 1.3.1
Posted by Evans Ye <ev...@apache.org>.
Alright. The release is at the corner.
Here's the list of 1.4.0 unsolved JIRA: https://s.apache.org/jVDq
I plan to clean that up and cut the branch of 1.4 around 21th March.
Right now I'm working on to stablize our CI. If you have feature that would
like to catch up the release train, please raise your hand!
Konstantin Boudnik <co...@apache.org> 於 2019年3月18日 週一 上午7:52寫道：
> First draft of the patch is here
> https://issues.apache.org/jira/browse/BIGTOP-3192
> Would appreciate any comments. Thanks!
> --
> With regards,
> Konstantin (Cos) Boudnik
> 2CAC 8312 4870 D885 8616 6115 220F 6980 1F27 E622
>
> Disclaimer: Opinions expressed in this email are those of the author,
> and do not necessarily represent the views of any company the author
> might be affiliated with at the moment of writing.
>
> On Sun, Mar 17, 2019 at 12:27 PM Evans Ye <ev...@apache.org> wrote:
> >
> > That'd be great!
> >
> > Konstantin Boudnik <co...@apache.org> 於 2019年3月17日 週日 下午9:23寫道：
> >
> > > Yup, indeed I do ;) Will give it a try tonight...
> > > --
> > > With regards,
> > > Konstantin (Cos) Boudnik
> > > 2CAC 8312 4870 D885 8616 6115 220F 6980 1F27 E622
> > >
> > > Disclaimer: Opinions expressed in this email are those of the author,
> > > and do not necessarily represent the views of any company the author
> > > might be affiliated with at the moment of writing.
> > >
> > > On Sun, Mar 17, 2019 at 9:15 AM Olaf Flebbe <of...@oflebbe.de> wrote:
> > > >
> > > > Hi Cos,
> > > >
> > > > if you are answering to
> > > > my question regarding removal of spark1: yes please.
> > > >
> > > > Olaf
> > > >
> > > > > Am 17.03.2019 um 13:10 schrieb Konstantin Boudnik <cos@apache.org
> >:
> > > > >
> > > > > Honestly, I think it is long overdue... I don't think we should
> waste
> > > > > any resources on it. I can do a quick patch to get rid of it in
> > > > > master, if there's no objections.
> > > > > --
> > > > > With regards,
> > > > > Konstantin (Cos) Boudnik
> > > > > 2CAC 8312 4870 D885 8616 6115 220F 6980 1F27 E622
> > > > >
> > > > > Disclaimer: Opinions expressed in this email are those of the
> author,
> > > > > and do not necessarily represent the views of any company the
> author
> > > > > might be affiliated with at the moment of writing.
> > > > >
> > > > >> On Wed, Mar 13, 2019 at 1:48 AM Olaf Flebbe <of...@oflebbe.de>
> wrote:
> > > > >>
> > > > >> Hi,
> > > > >>
> > > > >> We learned the hard way spark1 vs spark2 . Can we drop spark1 now?
> > > What do you think?
> > > > >>
> > > > >> olaf
> > > > >>
> > > > >>> Am 13.03.2019 um 06:16 schrieb Konstantin Boudnik <
> cos@apache.org>:
> > > > >>>
> > > > >>> While I am all for the latest and greatest, I learned to be
> cautious
> > > > >>> when it gets to the software. Especially, when we look at
> something
> > > > >>> considering itself on the "top of the stack". Hence I welcome
> your
> > > the
> > > > >>> conservative approach!
> > > > >>> --
> > > > >>> With regards,
> > > > >>> Konstantin (Cos) Boudnik
> > > > >>> 2CAC 8312 4870 D885 8616 6115 220F 6980 1F27 E622
> > > > >>>
> > > > >>> Disclaimer: Opinions expressed in this email are those of the
> author,
> > > > >>> and do not necessarily represent the views of any company the
> author
> > > > >>> might be affiliated with at the moment of writing.
> > > > >>>
> > > > >>>> On Wed, Mar 13, 2019 at 5:58 AM Evans Ye <ev...@apache.org>
> > > wrote:
> > > > >>>>
> > > > >>>> Thank you all for the support!
> > > > >>>>
> > > > >>>> Until now, I was dealing with various compatibility issue for
> Spark.
> > > > >>>> However there're too many incompatible components with spark
> 2.4.0:
> > > > >>>>
> > > > >>>>
> > > https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
> > > > >>>>
> > > > >>>> Therefore I'd like to revoke the 2.4.0 upgrade and just slightly
> > > bump it up
> > > > >>>> to 2.2.3.
> > > > >>>> The new release date is targeted at the end of March.
> > > > >>>>
> > > > >>>>
> > > > >>>>
> > > > >>>> Jun HE <ry...@gmail.com> 於 2019年3月6日 週三 下午9:28寫道：
> > > > >>>>
> > > > >>>>> Awesome, and huge +1 to this!
> > > > >>>>>
> > > > >>>>> Konstantin Boudnik <co...@boudnik.org> 于2019年3月6日周三 下午9:11写道：
> > > > >>>>>
> > > > >>>>>> Oh my, that's amazing! Thank you man!
> > > > >>>>>> +1 on moving directly to 1.4
> > > > >>>>>> --
> > > > >>>>>> With regards,
> > > > >>>>>> Konstantin
> > > > >>>>>>
> > > > >>>>>> On March 5, 2019 12:58:44 AM GMT+03:00, Olaf Flebbe <
> > > of@oflebbe.de>
> > > > >>>>> wrote:
> > > > >>>>>>> Hi Evans,
> > > > >>>>>>>
> > > > >>>>>>> You found and fixed a lot of bugs and made the smoke tests
> work,
> > > so a
> > > > >>>>>>> big +1 .
> > > > >>>>>>>
> > > > >>>>>>> Olaf
> > > > >>>>>>>
> > > > >>>>>>>> Am 04.03.2019 um 08:12 schrieb Evans Ye <evansye@apache.org
> >:
> > > > >>>>>>>>
> > > > >>>>>>>> -1 welcomed as well !!!
> > > > >>>>>>>>
> > > > >>>>>>>> Evans Ye <ev...@apache.org> 於 2019年3月4日 週一 下午3:11寫道：
> > > > >>>>>>>>
> > > > >>>>>>>>> Hi all,
> > > > >>>>>>>>>
> > > > >>>>>>>>> The overhaul as well as development is almost completed.
> With
> > > > >>>>>>> several new
> > > > >>>>>>>>> features added, I'd like to propose to release Bigtop 1.4.0
> > > directly
> > > > >>>>>>>>> instead of doing 1.3.1 release(https://s.apache.org/2a8K).
> > > This
> > > > >>>>>>> helps to
> > > > >>>>>>>>> promote these new features:
> > > > >>>>>>>>>
> > > > >>>>>>>>> 1. Full support to build and test inside docker with one
> stop
> > > > >>>>>>> seamlessly
> > > > >>>>>>>>> integration.
> > > > >>>>>>>>> Now just one command to build and test components:
> > > > >>>>>>>>> Example:
> > > > >>>>>>>>> $ ./gradlew spark-ind repo-ind docker-provisioner
> > > -POS=ubuntu-16.04
> > > > >>>>>>>>> -Pnexus -Penable_local_repo -Pstack=spark-standalone
> > > > >>>>>>> -Psmoke-tests=spark
> > > > >>>>>>>>>
> > > > >>>>>>>>> 2. Support to build from git commit hash
> > > > >>>>>>>>> Example:
> > > > >>>>>>>>> $ ./gradlew kafka-pkg-ind
> > > > >>>>>>> -Pgit_repo=https://github.com/apache/kafka.git
> > > > >>>>>>>>> -Pgit_ref=1.1
> > > -Pgit_sha1=4dae083af486eaedd27c69c973c74605bffd416b
> > > > >>>>>>>>> -Pbase_version=1.1.1
> > > > >>>>>>>>>
> > > > >>>>>>>>> 3. Version bumps and tests
> > > > >>>>>>>>> Hadoop 2.8.5, Kafka 1.1.1, Spark 2.4.0, Alluxio 1.8.1
> > > > >>>>>>>>> New smoke tests: Flink, Giraph (Crunch)
> > > > >>>>>>>>>
> > > > >>>>>>>>> 4. Loooooooots of bug fixes!
> > > > >>>>>>>>>
> > > > >>>>>>>>> If agreed, please +1 and I'll still serve as the RM for 1.4
> > > release,
> > > > >>>>>>> if no
> > > > >>>>>>>>> preemption ;)
> > > > >>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>> Best,
> > > > >>>>>>>>> Evans
> > > > >>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>>>>
> > > > >>>>>>
> > > > >>>>>
> > >
>
Re: [DISCUSS] Jump to 1.4.0 release directly instead of 1.3.1
Posted by Konstantin Boudnik <co...@apache.org>.
First draft of the patch is here
https://issues.apache.org/jira/browse/BIGTOP-3192
Would appreciate any comments. Thanks!
--
With regards,
Konstantin (Cos) Boudnik
2CAC 8312 4870 D885 8616 6115 220F 6980 1F27 E622
Disclaimer: Opinions expressed in this email are those of the author,
and do not necessarily represent the views of any company the author
might be affiliated with at the moment of writing.
On Sun, Mar 17, 2019 at 12:27 PM Evans Ye <ev...@apache.org> wrote:
>
> That'd be great!
>
> Konstantin Boudnik <co...@apache.org> 於 2019年3月17日 週日 下午9:23寫道：
>
> > Yup, indeed I do ;) Will give it a try tonight...
> > --
> > With regards,
> > Konstantin (Cos) Boudnik
> > 2CAC 8312 4870 D885 8616 6115 220F 6980 1F27 E622
> >
> > Disclaimer: Opinions expressed in this email are those of the author,
> > and do not necessarily represent the views of any company the author
> > might be affiliated with at the moment of writing.
> >
> > On Sun, Mar 17, 2019 at 9:15 AM Olaf Flebbe <of...@oflebbe.de> wrote:
> > >
> > > Hi Cos,
> > >
> > > if you are answering to
> > > my question regarding removal of spark1: yes please.
> > >
> > > Olaf
> > >
> > > > Am 17.03.2019 um 13:10 schrieb Konstantin Boudnik <co...@apache.org>:
> > > >
> > > > Honestly, I think it is long overdue... I don't think we should waste
> > > > any resources on it. I can do a quick patch to get rid of it in
> > > > master, if there's no objections.
> > > > --
> > > > With regards,
> > > > Konstantin (Cos) Boudnik
> > > > 2CAC 8312 4870 D885 8616 6115 220F 6980 1F27 E622
> > > >
> > > > Disclaimer: Opinions expressed in this email are those of the author,
> > > > and do not necessarily represent the views of any company the author
> > > > might be affiliated with at the moment of writing.
> > > >
> > > >> On Wed, Mar 13, 2019 at 1:48 AM Olaf Flebbe <of...@oflebbe.de> wrote:
> > > >>
> > > >> Hi,
> > > >>
> > > >> We learned the hard way spark1 vs spark2 . Can we drop spark1 now?
> > What do you think?
> > > >>
> > > >> olaf
> > > >>
> > > >>> Am 13.03.2019 um 06:16 schrieb Konstantin Boudnik <co...@apache.org>:
> > > >>>
> > > >>> While I am all for the latest and greatest, I learned to be cautious
> > > >>> when it gets to the software. Especially, when we look at something
> > > >>> considering itself on the "top of the stack". Hence I welcome your
> > the
> > > >>> conservative approach!
> > > >>> --
> > > >>> With regards,
> > > >>> Konstantin (Cos) Boudnik
> > > >>> 2CAC 8312 4870 D885 8616 6115 220F 6980 1F27 E622
> > > >>>
> > > >>> Disclaimer: Opinions expressed in this email are those of the author,
> > > >>> and do not necessarily represent the views of any company the author
> > > >>> might be affiliated with at the moment of writing.
> > > >>>
> > > >>>> On Wed, Mar 13, 2019 at 5:58 AM Evans Ye <ev...@apache.org>
> > wrote:
> > > >>>>
> > > >>>> Thank you all for the support!
> > > >>>>
> > > >>>> Until now, I was dealing with various compatibility issue for Spark.
> > > >>>> However there're too many incompatible components with spark 2.4.0:
> > > >>>>
> > > >>>>
> > https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
> > > >>>>
> > > >>>> Therefore I'd like to revoke the 2.4.0 upgrade and just slightly
> > bump it up
> > > >>>> to 2.2.3.
> > > >>>> The new release date is targeted at the end of March.
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>> Jun HE <ry...@gmail.com> 於 2019年3月6日 週三 下午9:28寫道：
> > > >>>>
> > > >>>>> Awesome, and huge +1 to this!
> > > >>>>>
> > > >>>>> Konstantin Boudnik <co...@boudnik.org> 于2019年3月6日周三 下午9:11写道：
> > > >>>>>
> > > >>>>>> Oh my, that's amazing! Thank you man!
> > > >>>>>> +1 on moving directly to 1.4
> > > >>>>>> --
> > > >>>>>> With regards,
> > > >>>>>> Konstantin
> > > >>>>>>
> > > >>>>>> On March 5, 2019 12:58:44 AM GMT+03:00, Olaf Flebbe <
> > of@oflebbe.de>
> > > >>>>> wrote:
> > > >>>>>>> Hi Evans,
> > > >>>>>>>
> > > >>>>>>> You found and fixed a lot of bugs and made the smoke tests work,
> > so a
> > > >>>>>>> big +1 .
> > > >>>>>>>
> > > >>>>>>> Olaf
> > > >>>>>>>
> > > >>>>>>>> Am 04.03.2019 um 08:12 schrieb Evans Ye <ev...@apache.org>:
> > > >>>>>>>>
> > > >>>>>>>> -1 welcomed as well !!!
> > > >>>>>>>>
> > > >>>>>>>> Evans Ye <ev...@apache.org> 於 2019年3月4日 週一 下午3:11寫道：
> > > >>>>>>>>
> > > >>>>>>>>> Hi all,
> > > >>>>>>>>>
> > > >>>>>>>>> The overhaul as well as development is almost completed. With
> > > >>>>>>> several new
> > > >>>>>>>>> features added, I'd like to propose to release Bigtop 1.4.0
> > directly
> > > >>>>>>>>> instead of doing 1.3.1 release(https://s.apache.org/2a8K).
> > This
> > > >>>>>>> helps to
> > > >>>>>>>>> promote these new features:
> > > >>>>>>>>>
> > > >>>>>>>>> 1. Full support to build and test inside docker with one stop
> > > >>>>>>> seamlessly
> > > >>>>>>>>> integration.
> > > >>>>>>>>> Now just one command to build and test components:
> > > >>>>>>>>> Example:
> > > >>>>>>>>> $ ./gradlew spark-ind repo-ind docker-provisioner
> > -POS=ubuntu-16.04
> > > >>>>>>>>> -Pnexus -Penable_local_repo -Pstack=spark-standalone
> > > >>>>>>> -Psmoke-tests=spark
> > > >>>>>>>>>
> > > >>>>>>>>> 2. Support to build from git commit hash
> > > >>>>>>>>> Example:
> > > >>>>>>>>> $ ./gradlew kafka-pkg-ind
> > > >>>>>>> -Pgit_repo=https://github.com/apache/kafka.git
> > > >>>>>>>>> -Pgit_ref=1.1
> > -Pgit_sha1=4dae083af486eaedd27c69c973c74605bffd416b
> > > >>>>>>>>> -Pbase_version=1.1.1
> > > >>>>>>>>>
> > > >>>>>>>>> 3. Version bumps and tests
> > > >>>>>>>>> Hadoop 2.8.5, Kafka 1.1.1, Spark 2.4.0, Alluxio 1.8.1
> > > >>>>>>>>> New smoke tests: Flink, Giraph (Crunch)
> > > >>>>>>>>>
> > > >>>>>>>>> 4. Loooooooots of bug fixes!
> > > >>>>>>>>>
> > > >>>>>>>>> If agreed, please +1 and I'll still serve as the RM for 1.4
> > release,
> > > >>>>>>> if no
> > > >>>>>>>>> preemption ;)
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>> Best,
> > > >>>>>>>>> Evans
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>
> > > >>>>>
> >
Re: [DISCUSS] Jump to 1.4.0 release directly instead of 1.3.1
Posted by Evans Ye <ev...@apache.org>.
That'd be great!
Konstantin Boudnik <co...@apache.org> 於 2019年3月17日 週日 下午9:23寫道：
> Yup, indeed I do ;) Will give it a try tonight...
> --
> With regards,
> Konstantin (Cos) Boudnik
> 2CAC 8312 4870 D885 8616 6115 220F 6980 1F27 E622
>
> Disclaimer: Opinions expressed in this email are those of the author,
> and do not necessarily represent the views of any company the author
> might be affiliated with at the moment of writing.
>
> On Sun, Mar 17, 2019 at 9:15 AM Olaf Flebbe <of...@oflebbe.de> wrote:
> >
> > Hi Cos,
> >
> > if you are answering to
> > my question regarding removal of spark1: yes please.
> >
> > Olaf
> >
> > > Am 17.03.2019 um 13:10 schrieb Konstantin Boudnik <co...@apache.org>:
> > >
> > > Honestly, I think it is long overdue... I don't think we should waste
> > > any resources on it. I can do a quick patch to get rid of it in
> > > master, if there's no objections.
> > > --
> > > With regards,
> > > Konstantin (Cos) Boudnik
> > > 2CAC 8312 4870 D885 8616 6115 220F 6980 1F27 E622
> > >
> > > Disclaimer: Opinions expressed in this email are those of the author,
> > > and do not necessarily represent the views of any company the author
> > > might be affiliated with at the moment of writing.
> > >
> > >> On Wed, Mar 13, 2019 at 1:48 AM Olaf Flebbe <of...@oflebbe.de> wrote:
> > >>
> > >> Hi,
> > >>
> > >> We learned the hard way spark1 vs spark2 . Can we drop spark1 now?
> What do you think?
> > >>
> > >> olaf
> > >>
> > >>> Am 13.03.2019 um 06:16 schrieb Konstantin Boudnik <co...@apache.org>:
> > >>>
> > >>> While I am all for the latest and greatest, I learned to be cautious
> > >>> when it gets to the software. Especially, when we look at something
> > >>> considering itself on the "top of the stack". Hence I welcome your
> the
> > >>> conservative approach!
> > >>> --
> > >>> With regards,
> > >>> Konstantin (Cos) Boudnik
> > >>> 2CAC 8312 4870 D885 8616 6115 220F 6980 1F27 E622
> > >>>
> > >>> Disclaimer: Opinions expressed in this email are those of the author,
> > >>> and do not necessarily represent the views of any company the author
> > >>> might be affiliated with at the moment of writing.
> > >>>
> > >>>> On Wed, Mar 13, 2019 at 5:58 AM Evans Ye <ev...@apache.org>
> wrote:
> > >>>>
> > >>>> Thank you all for the support!
> > >>>>
> > >>>> Until now, I was dealing with various compatibility issue for Spark.
> > >>>> However there're too many incompatible components with spark 2.4.0:
> > >>>>
> > >>>>
> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
> > >>>>
> > >>>> Therefore I'd like to revoke the 2.4.0 upgrade and just slightly
> bump it up
> > >>>> to 2.2.3.
> > >>>> The new release date is targeted at the end of March.
> > >>>>
> > >>>>
> > >>>>
> > >>>> Jun HE <ry...@gmail.com> 於 2019年3月6日 週三 下午9:28寫道：
> > >>>>
> > >>>>> Awesome, and huge +1 to this!
> > >>>>>
> > >>>>> Konstantin Boudnik <co...@boudnik.org> 于2019年3月6日周三 下午9:11写道：
> > >>>>>
> > >>>>>> Oh my, that's amazing! Thank you man!
> > >>>>>> +1 on moving directly to 1.4
> > >>>>>> --
> > >>>>>> With regards,
> > >>>>>> Konstantin
> > >>>>>>
> > >>>>>> On March 5, 2019 12:58:44 AM GMT+03:00, Olaf Flebbe <
> of@oflebbe.de>
> > >>>>> wrote:
> > >>>>>>> Hi Evans,
> > >>>>>>>
> > >>>>>>> You found and fixed a lot of bugs and made the smoke tests work,
> so a
> > >>>>>>> big +1 .
> > >>>>>>>
> > >>>>>>> Olaf
> > >>>>>>>
> > >>>>>>>> Am 04.03.2019 um 08:12 schrieb Evans Ye <ev...@apache.org>:
> > >>>>>>>>
> > >>>>>>>> -1 welcomed as well !!!
> > >>>>>>>>
> > >>>>>>>> Evans Ye <ev...@apache.org> 於 2019年3月4日 週一 下午3:11寫道：
> > >>>>>>>>
> > >>>>>>>>> Hi all,
> > >>>>>>>>>
> > >>>>>>>>> The overhaul as well as development is almost completed. With
> > >>>>>>> several new
> > >>>>>>>>> features added, I'd like to propose to release Bigtop 1.4.0
> directly
> > >>>>>>>>> instead of doing 1.3.1 release(https://s.apache.org/2a8K).
> This
> > >>>>>>> helps to
> > >>>>>>>>> promote these new features:
> > >>>>>>>>>
> > >>>>>>>>> 1. Full support to build and test inside docker with one stop
> > >>>>>>> seamlessly
> > >>>>>>>>> integration.
> > >>>>>>>>> Now just one command to build and test components:
> > >>>>>>>>> Example:
> > >>>>>>>>> $ ./gradlew spark-ind repo-ind docker-provisioner
> -POS=ubuntu-16.04
> > >>>>>>>>> -Pnexus -Penable_local_repo -Pstack=spark-standalone
> > >>>>>>> -Psmoke-tests=spark
> > >>>>>>>>>
> > >>>>>>>>> 2. Support to build from git commit hash
> > >>>>>>>>> Example:
> > >>>>>>>>> $ ./gradlew kafka-pkg-ind
> > >>>>>>> -Pgit_repo=https://github.com/apache/kafka.git
> > >>>>>>>>> -Pgit_ref=1.1
> -Pgit_sha1=4dae083af486eaedd27c69c973c74605bffd416b
> > >>>>>>>>> -Pbase_version=1.1.1
> > >>>>>>>>>
> > >>>>>>>>> 3. Version bumps and tests
> > >>>>>>>>> Hadoop 2.8.5, Kafka 1.1.1, Spark 2.4.0, Alluxio 1.8.1
> > >>>>>>>>> New smoke tests: Flink, Giraph (Crunch)
> > >>>>>>>>>
> > >>>>>>>>> 4. Loooooooots of bug fixes!
> > >>>>>>>>>
> > >>>>>>>>> If agreed, please +1 and I'll still serve as the RM for 1.4
> release,
> > >>>>>>> if no
> > >>>>>>>>> preemption ;)
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> Best,
> > >>>>>>>>> Evans
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>
> > >>>>>
>
Re: [DISCUSS] Jump to 1.4.0 release directly instead of 1.3.1
Posted by Konstantin Boudnik <co...@apache.org>.
Yup, indeed I do ;) Will give it a try tonight...
--
With regards,
Konstantin (Cos) Boudnik
2CAC 8312 4870 D885 8616 6115 220F 6980 1F27 E622
Disclaimer: Opinions expressed in this email are those of the author,
and do not necessarily represent the views of any company the author
might be affiliated with at the moment of writing.
On Sun, Mar 17, 2019 at 9:15 AM Olaf Flebbe <of...@oflebbe.de> wrote:
>
> Hi Cos,
>
> if you are answering to
> my question regarding removal of spark1: yes please.
>
> Olaf
>
> > Am 17.03.2019 um 13:10 schrieb Konstantin Boudnik <co...@apache.org>:
> >
> > Honestly, I think it is long overdue... I don't think we should waste
> > any resources on it. I can do a quick patch to get rid of it in
> > master, if there's no objections.
> > --
> > With regards,
> > Konstantin (Cos) Boudnik
> > 2CAC 8312 4870 D885 8616 6115 220F 6980 1F27 E622
> >
> > Disclaimer: Opinions expressed in this email are those of the author,
> > and do not necessarily represent the views of any company the author
> > might be affiliated with at the moment of writing.
> >
> >> On Wed, Mar 13, 2019 at 1:48 AM Olaf Flebbe <of...@oflebbe.de> wrote:
> >>
> >> Hi,
> >>
> >> We learned the hard way spark1 vs spark2 . Can we drop spark1 now? What do you think?
> >>
> >> olaf
> >>
> >>> Am 13.03.2019 um 06:16 schrieb Konstantin Boudnik <co...@apache.org>:
> >>>
> >>> While I am all for the latest and greatest, I learned to be cautious
> >>> when it gets to the software. Especially, when we look at something
> >>> considering itself on the "top of the stack". Hence I welcome your the
> >>> conservative approach!
> >>> --
> >>> With regards,
> >>> Konstantin (Cos) Boudnik
> >>> 2CAC 8312 4870 D885 8616 6115 220F 6980 1F27 E622
> >>>
> >>> Disclaimer: Opinions expressed in this email are those of the author,
> >>> and do not necessarily represent the views of any company the author
> >>> might be affiliated with at the moment of writing.
> >>>
> >>>> On Wed, Mar 13, 2019 at 5:58 AM Evans Ye <ev...@apache.org> wrote:
> >>>>
> >>>> Thank you all for the support!
> >>>>
> >>>> Until now, I was dealing with various compatibility issue for Spark.
> >>>> However there're too many incompatible components with spark 2.4.0:
> >>>>
> >>>> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
> >>>>
> >>>> Therefore I'd like to revoke the 2.4.0 upgrade and just slightly bump it up
> >>>> to 2.2.3.
> >>>> The new release date is targeted at the end of March.
> >>>>
> >>>>
> >>>>
> >>>> Jun HE <ry...@gmail.com> 於 2019年3月6日 週三 下午9:28寫道：
> >>>>
> >>>>> Awesome, and huge +1 to this!
> >>>>>
> >>>>> Konstantin Boudnik <co...@boudnik.org> 于2019年3月6日周三 下午9:11写道：
> >>>>>
> >>>>>> Oh my, that's amazing! Thank you man!
> >>>>>> +1 on moving directly to 1.4
> >>>>>> --
> >>>>>> With regards,
> >>>>>> Konstantin
> >>>>>>
> >>>>>> On March 5, 2019 12:58:44 AM GMT+03:00, Olaf Flebbe <of...@oflebbe.de>
> >>>>> wrote:
> >>>>>>> Hi Evans,
> >>>>>>>
> >>>>>>> You found and fixed a lot of bugs and made the smoke tests work, so a
> >>>>>>> big +1 .
> >>>>>>>
> >>>>>>> Olaf
> >>>>>>>
> >>>>>>>> Am 04.03.2019 um 08:12 schrieb Evans Ye <ev...@apache.org>:
> >>>>>>>>
> >>>>>>>> -1 welcomed as well !!!
> >>>>>>>>
> >>>>>>>> Evans Ye <ev...@apache.org> 於 2019年3月4日 週一 下午3:11寫道：
> >>>>>>>>
> >>>>>>>>> Hi all,
> >>>>>>>>>
> >>>>>>>>> The overhaul as well as development is almost completed. With
> >>>>>>> several new
> >>>>>>>>> features added, I'd like to propose to release Bigtop 1.4.0 directly
> >>>>>>>>> instead of doing 1.3.1 release(https://s.apache.org/2a8K). This
> >>>>>>> helps to
> >>>>>>>>> promote these new features:
> >>>>>>>>>
> >>>>>>>>> 1. Full support to build and test inside docker with one stop
> >>>>>>> seamlessly
> >>>>>>>>> integration.
> >>>>>>>>> Now just one command to build and test components:
> >>>>>>>>> Example:
> >>>>>>>>> $ ./gradlew spark-ind repo-ind docker-provisioner -POS=ubuntu-16.04
> >>>>>>>>> -Pnexus -Penable_local_repo -Pstack=spark-standalone
> >>>>>>> -Psmoke-tests=spark
> >>>>>>>>>
> >>>>>>>>> 2. Support to build from git commit hash
> >>>>>>>>> Example:
> >>>>>>>>> $ ./gradlew kafka-pkg-ind
> >>>>>>> -Pgit_repo=https://github.com/apache/kafka.git
> >>>>>>>>> -Pgit_ref=1.1 -Pgit_sha1=4dae083af486eaedd27c69c973c74605bffd416b
> >>>>>>>>> -Pbase_version=1.1.1
> >>>>>>>>>
> >>>>>>>>> 3. Version bumps and tests
> >>>>>>>>> Hadoop 2.8.5, Kafka 1.1.1, Spark 2.4.0, Alluxio 1.8.1
> >>>>>>>>> New smoke tests: Flink, Giraph (Crunch)
> >>>>>>>>>
> >>>>>>>>> 4. Loooooooots of bug fixes!
> >>>>>>>>>
> >>>>>>>>> If agreed, please +1 and I'll still serve as the RM for 1.4 release,
> >>>>>>> if no
> >>>>>>>>> preemption ;)
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Best,
> >>>>>>>>> Evans
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>
> >>>>>
Re: [DISCUSS] Jump to 1.4.0 release directly instead of 1.3.1
Posted by Olaf Flebbe <of...@oflebbe.de>.
Hi Cos,
if you are answering to
my question regarding removal of spark1: yes please.
Olaf
> Am 17.03.2019 um 13:10 schrieb Konstantin Boudnik <co...@apache.org>:
>
> Honestly, I think it is long overdue... I don't think we should waste
> any resources on it. I can do a quick patch to get rid of it in
> master, if there's no objections.
> --
> With regards,
> Konstantin (Cos) Boudnik
> 2CAC 8312 4870 D885 8616 6115 220F 6980 1F27 E622
>
> Disclaimer: Opinions expressed in this email are those of the author,
> and do not necessarily represent the views of any company the author
> might be affiliated with at the moment of writing.
>
>> On Wed, Mar 13, 2019 at 1:48 AM Olaf Flebbe <of...@oflebbe.de> wrote:
>>
>> Hi,
>>
>> We learned the hard way spark1 vs spark2 . Can we drop spark1 now? What do you think?
>>
>> olaf
>>
>>> Am 13.03.2019 um 06:16 schrieb Konstantin Boudnik <co...@apache.org>:
>>>
>>> While I am all for the latest and greatest, I learned to be cautious
>>> when it gets to the software. Especially, when we look at something
>>> considering itself on the "top of the stack". Hence I welcome your the
>>> conservative approach!
>>> --
>>> With regards,
>>> Konstantin (Cos) Boudnik
>>> 2CAC 8312 4870 D885 8616 6115 220F 6980 1F27 E622
>>>
>>> Disclaimer: Opinions expressed in this email are those of the author,
>>> and do not necessarily represent the views of any company the author
>>> might be affiliated with at the moment of writing.
>>>
>>>> On Wed, Mar 13, 2019 at 5:58 AM Evans Ye <ev...@apache.org> wrote:
>>>>
>>>> Thank you all for the support!
>>>>
>>>> Until now, I was dealing with various compatibility issue for Spark.
>>>> However there're too many incompatible components with spark 2.4.0:
>>>>
>>>> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
>>>>
>>>> Therefore I'd like to revoke the 2.4.0 upgrade and just slightly bump it up
>>>> to 2.2.3.
>>>> The new release date is targeted at the end of March.
>>>>
>>>>
>>>>
>>>> Jun HE <ry...@gmail.com> 於 2019年3月6日 週三 下午9:28寫道：
>>>>
>>>>> Awesome, and huge +1 to this!
>>>>>
>>>>> Konstantin Boudnik <co...@boudnik.org> 于2019年3月6日周三 下午9:11写道：
>>>>>
>>>>>> Oh my, that's amazing! Thank you man!
>>>>>> +1 on moving directly to 1.4
>>>>>> --
>>>>>> With regards,
>>>>>> Konstantin
>>>>>>
>>>>>> On March 5, 2019 12:58:44 AM GMT+03:00, Olaf Flebbe <of...@oflebbe.de>
>>>>> wrote:
>>>>>>> Hi Evans,
>>>>>>>
>>>>>>> You found and fixed a lot of bugs and made the smoke tests work, so a
>>>>>>> big +1 .
>>>>>>>
>>>>>>> Olaf
>>>>>>>
>>>>>>>> Am 04.03.2019 um 08:12 schrieb Evans Ye <ev...@apache.org>:
>>>>>>>>
>>>>>>>> -1 welcomed as well !!!
>>>>>>>>
>>>>>>>> Evans Ye <ev...@apache.org> 於 2019年3月4日 週一 下午3:11寫道：
>>>>>>>>
>>>>>>>>> Hi all,
>>>>>>>>>
>>>>>>>>> The overhaul as well as development is almost completed. With
>>>>>>> several new
>>>>>>>>> features added, I'd like to propose to release Bigtop 1.4.0 directly
>>>>>>>>> instead of doing 1.3.1 release(https://s.apache.org/2a8K). This
>>>>>>> helps to
>>>>>>>>> promote these new features:
>>>>>>>>>
>>>>>>>>> 1. Full support to build and test inside docker with one stop
>>>>>>> seamlessly
>>>>>>>>> integration.
>>>>>>>>> Now just one command to build and test components:
>>>>>>>>> Example:
>>>>>>>>> $ ./gradlew spark-ind repo-ind docker-provisioner -POS=ubuntu-16.04
>>>>>>>>> -Pnexus -Penable_local_repo -Pstack=spark-standalone
>>>>>>> -Psmoke-tests=spark
>>>>>>>>>
>>>>>>>>> 2. Support to build from git commit hash
>>>>>>>>> Example:
>>>>>>>>> $ ./gradlew kafka-pkg-ind
>>>>>>> -Pgit_repo=https://github.com/apache/kafka.git
>>>>>>>>> -Pgit_ref=1.1 -Pgit_sha1=4dae083af486eaedd27c69c973c74605bffd416b
>>>>>>>>> -Pbase_version=1.1.1
>>>>>>>>>
>>>>>>>>> 3. Version bumps and tests
>>>>>>>>> Hadoop 2.8.5, Kafka 1.1.1, Spark 2.4.0, Alluxio 1.8.1
>>>>>>>>> New smoke tests: Flink, Giraph (Crunch)
>>>>>>>>>
>>>>>>>>> 4. Loooooooots of bug fixes!
>>>>>>>>>
>>>>>>>>> If agreed, please +1 and I'll still serve as the RM for 1.4 release,
>>>>>>> if no
>>>>>>>>> preemption ;)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Best,
>>>>>>>>> Evans
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>
>>>>>
Re: [DISCUSS] Jump to 1.4.0 release directly instead of 1.3.1
Posted by Konstantin Boudnik <co...@apache.org>.
Honestly, I think it is long overdue... I don't think we should waste
any resources on it. I can do a quick patch to get rid of it in
master, if there's no objections.
--
With regards,
Konstantin (Cos) Boudnik
2CAC 8312 4870 D885 8616 6115 220F 6980 1F27 E622
Disclaimer: Opinions expressed in this email are those of the author,
and do not necessarily represent the views of any company the author
might be affiliated with at the moment of writing.
On Wed, Mar 13, 2019 at 1:48 AM Olaf Flebbe <of...@oflebbe.de> wrote:
>
> Hi,
>
> We learned the hard way spark1 vs spark2 . Can we drop spark1 now? What do you think?
>
> olaf
>
> > Am 13.03.2019 um 06:16 schrieb Konstantin Boudnik <co...@apache.org>:
> >
> > While I am all for the latest and greatest, I learned to be cautious
> > when it gets to the software. Especially, when we look at something
> > considering itself on the "top of the stack". Hence I welcome your the
> > conservative approach!
> > --
> > With regards,
> > Konstantin (Cos) Boudnik
> > 2CAC 8312 4870 D885 8616 6115 220F 6980 1F27 E622
> >
> > Disclaimer: Opinions expressed in this email are those of the author,
> > and do not necessarily represent the views of any company the author
> > might be affiliated with at the moment of writing.
> >
> >> On Wed, Mar 13, 2019 at 5:58 AM Evans Ye <ev...@apache.org> wrote:
> >>
> >> Thank you all for the support!
> >>
> >> Until now, I was dealing with various compatibility issue for Spark.
> >> However there're too many incompatible components with spark 2.4.0:
> >>
> >> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
> >>
> >> Therefore I'd like to revoke the 2.4.0 upgrade and just slightly bump it up
> >> to 2.2.3.
> >> The new release date is targeted at the end of March.
> >>
> >>
> >>
> >> Jun HE <ry...@gmail.com> 於 2019年3月6日 週三 下午9:28寫道：
> >>
> >>> Awesome, and huge +1 to this!
> >>>
> >>> Konstantin Boudnik <co...@boudnik.org> 于2019年3月6日周三 下午9:11写道：
> >>>
> >>>> Oh my, that's amazing! Thank you man!
> >>>> +1 on moving directly to 1.4
> >>>> --
> >>>> With regards,
> >>>> Konstantin
> >>>>
> >>>> On March 5, 2019 12:58:44 AM GMT+03:00, Olaf Flebbe <of...@oflebbe.de>
> >>> wrote:
> >>>>> Hi Evans,
> >>>>>
> >>>>> You found and fixed a lot of bugs and made the smoke tests work, so a
> >>>>> big +1 .
> >>>>>
> >>>>> Olaf
> >>>>>
> >>>>>> Am 04.03.2019 um 08:12 schrieb Evans Ye <ev...@apache.org>:
> >>>>>>
> >>>>>> -1 welcomed as well !!!
> >>>>>>
> >>>>>> Evans Ye <ev...@apache.org> 於 2019年3月4日 週一 下午3:11寫道：
> >>>>>>
> >>>>>>> Hi all,
> >>>>>>>
> >>>>>>> The overhaul as well as development is almost completed. With
> >>>>> several new
> >>>>>>> features added, I'd like to propose to release Bigtop 1.4.0 directly
> >>>>>>> instead of doing 1.3.1 release(https://s.apache.org/2a8K). This
> >>>>> helps to
> >>>>>>> promote these new features:
> >>>>>>>
> >>>>>>> 1. Full support to build and test inside docker with one stop
> >>>>> seamlessly
> >>>>>>> integration.
> >>>>>>> Now just one command to build and test components:
> >>>>>>> Example:
> >>>>>>> $ ./gradlew spark-ind repo-ind docker-provisioner -POS=ubuntu-16.04
> >>>>>>> -Pnexus -Penable_local_repo -Pstack=spark-standalone
> >>>>> -Psmoke-tests=spark
> >>>>>>>
> >>>>>>> 2. Support to build from git commit hash
> >>>>>>> Example:
> >>>>>>> $ ./gradlew kafka-pkg-ind
> >>>>> -Pgit_repo=https://github.com/apache/kafka.git
> >>>>>>> -Pgit_ref=1.1 -Pgit_sha1=4dae083af486eaedd27c69c973c74605bffd416b
> >>>>>>> -Pbase_version=1.1.1
> >>>>>>>
> >>>>>>> 3. Version bumps and tests
> >>>>>>> Hadoop 2.8.5, Kafka 1.1.1, Spark 2.4.0, Alluxio 1.8.1
> >>>>>>> New smoke tests: Flink, Giraph (Crunch)
> >>>>>>>
> >>>>>>> 4. Loooooooots of bug fixes!
> >>>>>>>
> >>>>>>> If agreed, please +1 and I'll still serve as the RM for 1.4 release,
> >>>>> if no
> >>>>>>> preemption ;)
> >>>>>>>
> >>>>>>>
> >>>>>>> Best,
> >>>>>>> Evans
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>
> >>>
Re: [DISCUSS] Jump to 1.4.0 release directly instead of 1.3.1
Posted by Olaf Flebbe <of...@oflebbe.de>.
Hi,
We learned the hard way spark1 vs spark2 . Can we drop spark1 now? What do you think?
olaf
> Am 13.03.2019 um 06:16 schrieb Konstantin Boudnik <co...@apache.org>:
>
> While I am all for the latest and greatest, I learned to be cautious
> when it gets to the software. Especially, when we look at something
> considering itself on the "top of the stack". Hence I welcome your the
> conservative approach!
> --
> With regards,
> Konstantin (Cos) Boudnik
> 2CAC 8312 4870 D885 8616 6115 220F 6980 1F27 E622
>
> Disclaimer: Opinions expressed in this email are those of the author,
> and do not necessarily represent the views of any company the author
> might be affiliated with at the moment of writing.
>
>> On Wed, Mar 13, 2019 at 5:58 AM Evans Ye <ev...@apache.org> wrote:
>>
>> Thank you all for the support!
>>
>> Until now, I was dealing with various compatibility issue for Spark.
>> However there're too many incompatible components with spark 2.4.0:
>>
>> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
>>
>> Therefore I'd like to revoke the 2.4.0 upgrade and just slightly bump it up
>> to 2.2.3.
>> The new release date is targeted at the end of March.
>>
>>
>>
>> Jun HE <ry...@gmail.com> 於 2019年3月6日 週三 下午9:28寫道：
>>
>>> Awesome, and huge +1 to this!
>>>
>>> Konstantin Boudnik <co...@boudnik.org> 于2019年3月6日周三 下午9:11写道：
>>>
>>>> Oh my, that's amazing! Thank you man!
>>>> +1 on moving directly to 1.4
>>>> --
>>>> With regards,
>>>> Konstantin
>>>>
>>>> On March 5, 2019 12:58:44 AM GMT+03:00, Olaf Flebbe <of...@oflebbe.de>
>>> wrote:
>>>>> Hi Evans,
>>>>>
>>>>> You found and fixed a lot of bugs and made the smoke tests work, so a
>>>>> big +1 .
>>>>>
>>>>> Olaf
>>>>>
>>>>>> Am 04.03.2019 um 08:12 schrieb Evans Ye <ev...@apache.org>:
>>>>>>
>>>>>> -1 welcomed as well !!!
>>>>>>
>>>>>> Evans Ye <ev...@apache.org> 於 2019年3月4日 週一 下午3:11寫道：
>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> The overhaul as well as development is almost completed. With
>>>>> several new
>>>>>>> features added, I'd like to propose to release Bigtop 1.4.0 directly
>>>>>>> instead of doing 1.3.1 release(https://s.apache.org/2a8K). This
>>>>> helps to
>>>>>>> promote these new features:
>>>>>>>
>>>>>>> 1. Full support to build and test inside docker with one stop
>>>>> seamlessly
>>>>>>> integration.
>>>>>>> Now just one command to build and test components:
>>>>>>> Example:
>>>>>>> $ ./gradlew spark-ind repo-ind docker-provisioner -POS=ubuntu-16.04
>>>>>>> -Pnexus -Penable_local_repo -Pstack=spark-standalone
>>>>> -Psmoke-tests=spark
>>>>>>>
>>>>>>> 2. Support to build from git commit hash
>>>>>>> Example:
>>>>>>> $ ./gradlew kafka-pkg-ind
>>>>> -Pgit_repo=https://github.com/apache/kafka.git
>>>>>>> -Pgit_ref=1.1 -Pgit_sha1=4dae083af486eaedd27c69c973c74605bffd416b
>>>>>>> -Pbase_version=1.1.1
>>>>>>>
>>>>>>> 3. Version bumps and tests
>>>>>>> Hadoop 2.8.5, Kafka 1.1.1, Spark 2.4.0, Alluxio 1.8.1
>>>>>>> New smoke tests: Flink, Giraph (Crunch)
>>>>>>>
>>>>>>> 4. Loooooooots of bug fixes!
>>>>>>>
>>>>>>> If agreed, please +1 and I'll still serve as the RM for 1.4 release,
>>>>> if no
>>>>>>> preemption ;)
>>>>>>>
>>>>>>>
>>>>>>> Best,
>>>>>>> Evans
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>
>>>
Re: [DISCUSS] Jump to 1.4.0 release directly instead of 1.3.1
Posted by Konstantin Boudnik <co...@apache.org>.
While I am all for the latest and greatest, I learned to be cautious
when it gets to the software. Especially, when we look at something
considering itself on the "top of the stack". Hence I welcome your the
conservative approach!
--
With regards,
Konstantin (Cos) Boudnik
2CAC 8312 4870 D885 8616 6115 220F 6980 1F27 E622
Disclaimer: Opinions expressed in this email are those of the author,
and do not necessarily represent the views of any company the author
might be affiliated with at the moment of writing.
On Wed, Mar 13, 2019 at 5:58 AM Evans Ye <ev...@apache.org> wrote:
>
> Thank you all for the support!
>
> Until now, I was dealing with various compatibility issue for Spark.
> However there're too many incompatible components with spark 2.4.0:
>
> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
>
> Therefore I'd like to revoke the 2.4.0 upgrade and just slightly bump it up
> to 2.2.3.
> The new release date is targeted at the end of March.
>
>
>
> Jun HE <ry...@gmail.com> 於 2019年3月6日 週三 下午9:28寫道：
>
> > Awesome, and huge +1 to this!
> >
> > Konstantin Boudnik <co...@boudnik.org> 于2019年3月6日周三 下午9:11写道：
> >
> > > Oh my, that's amazing! Thank you man!
> > > +1 on moving directly to 1.4
> > > --
> > > With regards,
> > > Konstantin
> > >
> > > On March 5, 2019 12:58:44 AM GMT+03:00, Olaf Flebbe <of...@oflebbe.de>
> > wrote:
> > > >Hi Evans,
> > > >
> > > >You found and fixed a lot of bugs and made the smoke tests work, so a
> > > >big +1 .
> > > >
> > > >Olaf
> > > >
> > > >> Am 04.03.2019 um 08:12 schrieb Evans Ye <ev...@apache.org>:
> > > >>
> > > >> -1 welcomed as well !!!
> > > >>
> > > >> Evans Ye <ev...@apache.org> 於 2019年3月4日 週一 下午3:11寫道：
> > > >>
> > > >>> Hi all,
> > > >>>
> > > >>> The overhaul as well as development is almost completed. With
> > > >several new
> > > >>> features added, I'd like to propose to release Bigtop 1.4.0 directly
> > > >>> instead of doing 1.3.1 release(https://s.apache.org/2a8K). This
> > > >helps to
> > > >>> promote these new features:
> > > >>>
> > > >>> 1. Full support to build and test inside docker with one stop
> > > >seamlessly
> > > >>> integration.
> > > >>> Now just one command to build and test components:
> > > >>> Example:
> > > >>> $ ./gradlew spark-ind repo-ind docker-provisioner -POS=ubuntu-16.04
> > > >>> -Pnexus -Penable_local_repo -Pstack=spark-standalone
> > > >-Psmoke-tests=spark
> > > >>>
> > > >>> 2. Support to build from git commit hash
> > > >>> Example:
> > > >>> $ ./gradlew kafka-pkg-ind
> > > >-Pgit_repo=https://github.com/apache/kafka.git
> > > >>> -Pgit_ref=1.1 -Pgit_sha1=4dae083af486eaedd27c69c973c74605bffd416b
> > > >>> -Pbase_version=1.1.1
> > > >>>
> > > >>> 3. Version bumps and tests
> > > >>> Hadoop 2.8.5, Kafka 1.1.1, Spark 2.4.0, Alluxio 1.8.1
> > > >>> New smoke tests: Flink, Giraph (Crunch)
> > > >>>
> > > >>> 4. Loooooooots of bug fixes!
> > > >>>
> > > >>> If agreed, please +1 and I'll still serve as the RM for 1.4 release,
> > > >if no
> > > >>> preemption ;)
> > > >>>
> > > >>>
> > > >>> Best,
> > > >>> Evans
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > >
> >
Re: [DISCUSS] Jump to 1.4.0 release directly instead of 1.3.1
Posted by Evans Ye <ev...@apache.org>.
Thank you all for the support!
Until now, I was dealing with various compatibility issue for Spark.
However there're too many incompatible components with spark 2.4.0:
https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/
Therefore I'd like to revoke the 2.4.0 upgrade and just slightly bump it up
to 2.2.3.
The new release date is targeted at the end of March.
Jun HE <ry...@gmail.com> 於 2019年3月6日 週三 下午9:28寫道：
> Awesome, and huge +1 to this!
>
> Konstantin Boudnik <co...@boudnik.org> 于2019年3月6日周三 下午9:11写道：
>
> > Oh my, that's amazing! Thank you man!
> > +1 on moving directly to 1.4
> > --
> > With regards,
> > Konstantin
> >
> > On March 5, 2019 12:58:44 AM GMT+03:00, Olaf Flebbe <of...@oflebbe.de>
> wrote:
> > >Hi Evans,
> > >
> > >You found and fixed a lot of bugs and made the smoke tests work, so a
> > >big +1 .
> > >
> > >Olaf
> > >
> > >> Am 04.03.2019 um 08:12 schrieb Evans Ye <ev...@apache.org>:
> > >>
> > >> -1 welcomed as well !!!
> > >>
> > >> Evans Ye <ev...@apache.org> 於 2019年3月4日 週一 下午3:11寫道：
> > >>
> > >>> Hi all,
> > >>>
> > >>> The overhaul as well as development is almost completed. With
> > >several new
> > >>> features added, I'd like to propose to release Bigtop 1.4.0 directly
> > >>> instead of doing 1.3.1 release(https://s.apache.org/2a8K). This
> > >helps to
> > >>> promote these new features:
> > >>>
> > >>> 1. Full support to build and test inside docker with one stop
> > >seamlessly
> > >>> integration.
> > >>> Now just one command to build and test components:
> > >>> Example:
> > >>> $ ./gradlew spark-ind repo-ind docker-provisioner -POS=ubuntu-16.04
> > >>> -Pnexus -Penable_local_repo -Pstack=spark-standalone
> > >-Psmoke-tests=spark
> > >>>
> > >>> 2. Support to build from git commit hash
> > >>> Example:
> > >>> $ ./gradlew kafka-pkg-ind
> > >-Pgit_repo=https://github.com/apache/kafka.git
> > >>> -Pgit_ref=1.1 -Pgit_sha1=4dae083af486eaedd27c69c973c74605bffd416b
> > >>> -Pbase_version=1.1.1
> > >>>
> > >>> 3. Version bumps and tests
> > >>> Hadoop 2.8.5, Kafka 1.1.1, Spark 2.4.0, Alluxio 1.8.1
> > >>> New smoke tests: Flink, Giraph (Crunch)
> > >>>
> > >>> 4. Loooooooots of bug fixes!
> > >>>
> > >>> If agreed, please +1 and I'll still serve as the RM for 1.4 release,
> > >if no
> > >>> preemption ;)
> > >>>
> > >>>
> > >>> Best,
> > >>> Evans
> > >>>
> > >>>
> > >>>
> > >>>
> >
>
Re: [DISCUSS] Jump to 1.4.0 release directly instead of 1.3.1
Posted by Jun HE <ry...@gmail.com>.
Awesome, and huge +1 to this!
Konstantin Boudnik <co...@boudnik.org> 于2019年3月6日周三 下午9:11写道：
> Oh my, that's amazing! Thank you man!
> +1 on moving directly to 1.4
> --
> With regards,
> Konstantin
>
> On March 5, 2019 12:58:44 AM GMT+03:00, Olaf Flebbe <of...@oflebbe.de> wrote:
> >Hi Evans,
> >
> >You found and fixed a lot of bugs and made the smoke tests work, so a
> >big +1 .
> >
> >Olaf
> >
> >> Am 04.03.2019 um 08:12 schrieb Evans Ye <ev...@apache.org>:
> >>
> >> -1 welcomed as well !!!
> >>
> >> Evans Ye <ev...@apache.org> 於 2019年3月4日 週一 下午3:11寫道：
> >>
> >>> Hi all,
> >>>
> >>> The overhaul as well as development is almost completed. With
> >several new
> >>> features added, I'd like to propose to release Bigtop 1.4.0 directly
> >>> instead of doing 1.3.1 release(https://s.apache.org/2a8K). This
> >helps to
> >>> promote these new features:
> >>>
> >>> 1. Full support to build and test inside docker with one stop
> >seamlessly
> >>> integration.
> >>> Now just one command to build and test components:
> >>> Example:
> >>> $ ./gradlew spark-ind repo-ind docker-provisioner -POS=ubuntu-16.04
> >>> -Pnexus -Penable_local_repo -Pstack=spark-standalone
> >-Psmoke-tests=spark
> >>>
> >>> 2. Support to build from git commit hash
> >>> Example:
> >>> $ ./gradlew kafka-pkg-ind
> >-Pgit_repo=https://github.com/apache/kafka.git
> >>> -Pgit_ref=1.1 -Pgit_sha1=4dae083af486eaedd27c69c973c74605bffd416b
> >>> -Pbase_version=1.1.1
> >>>
> >>> 3. Version bumps and tests
> >>> Hadoop 2.8.5, Kafka 1.1.1, Spark 2.4.0, Alluxio 1.8.1
> >>> New smoke tests: Flink, Giraph (Crunch)
> >>>
> >>> 4. Loooooooots of bug fixes!
> >>>
> >>> If agreed, please +1 and I'll still serve as the RM for 1.4 release,
> >if no
> >>> preemption ;)
> >>>
> >>>
> >>> Best,
> >>> Evans
> >>>
> >>>
> >>>
> >>>
>
Re: [DISCUSS] Jump to 1.4.0 release directly instead of 1.3.1
Posted by Konstantin Boudnik <co...@boudnik.org>.
Oh my, that's amazing! Thank you man!
+1 on moving directly to 1.4
--
With regards,
Konstantin
On March 5, 2019 12:58:44 AM GMT+03:00, Olaf Flebbe <of...@oflebbe.de> wrote:
>Hi Evans,
>
>You found and fixed a lot of bugs and made the smoke tests work, so a
>big +1 .
>
>Olaf
>
>> Am 04.03.2019 um 08:12 schrieb Evans Ye <ev...@apache.org>:
>>
>> -1 welcomed as well !!!
>>
>> Evans Ye <ev...@apache.org> 於 2019年3月4日 週一 下午3:11寫道：
>>
>>> Hi all,
>>>
>>> The overhaul as well as development is almost completed. With
>several new
>>> features added, I'd like to propose to release Bigtop 1.4.0 directly
>>> instead of doing 1.3.1 release(https://s.apache.org/2a8K). This
>helps to
>>> promote these new features:
>>>
>>> 1. Full support to build and test inside docker with one stop
>seamlessly
>>> integration.
>>> Now just one command to build and test components:
>>> Example:
>>> $ ./gradlew spark-ind repo-ind docker-provisioner -POS=ubuntu-16.04
>>> -Pnexus -Penable_local_repo -Pstack=spark-standalone
>-Psmoke-tests=spark
>>>
>>> 2. Support to build from git commit hash
>>> Example:
>>> $ ./gradlew kafka-pkg-ind
>-Pgit_repo=https://github.com/apache/kafka.git
>>> -Pgit_ref=1.1 -Pgit_sha1=4dae083af486eaedd27c69c973c74605bffd416b
>>> -Pbase_version=1.1.1
>>>
>>> 3. Version bumps and tests
>>> Hadoop 2.8.5, Kafka 1.1.1, Spark 2.4.0, Alluxio 1.8.1
>>> New smoke tests: Flink, Giraph (Crunch)
>>>
>>> 4. Loooooooots of bug fixes!
>>>
>>> If agreed, please +1 and I'll still serve as the RM for 1.4 release,
>if no
>>> preemption ;)
>>>
>>>
>>> Best,
>>> Evans
>>>
>>>
>>>
>>>
Re: [DISCUSS] Jump to 1.4.0 release directly instead of 1.3.1
Posted by Olaf Flebbe <of...@oflebbe.de>.
Hi Evans,
You found and fixed a lot of bugs and made the smoke tests work, so a big +1 .
Olaf
> Am 04.03.2019 um 08:12 schrieb Evans Ye <ev...@apache.org>:
>
> -1 welcomed as well !!!
>
> Evans Ye <ev...@apache.org> 於 2019年3月4日 週一 下午3:11寫道：
>
>> Hi all,
>>
>> The overhaul as well as development is almost completed. With several new
>> features added, I'd like to propose to release Bigtop 1.4.0 directly
>> instead of doing 1.3.1 release(https://s.apache.org/2a8K). This helps to
>> promote these new features:
>>
>> 1. Full support to build and test inside docker with one stop seamlessly
>> integration.
>> Now just one command to build and test components:
>> Example:
>> $ ./gradlew spark-ind repo-ind docker-provisioner -POS=ubuntu-16.04
>> -Pnexus -Penable_local_repo -Pstack=spark-standalone -Psmoke-tests=spark
>>
>> 2. Support to build from git commit hash
>> Example:
>> $ ./gradlew kafka-pkg-ind -Pgit_repo=https://github.com/apache/kafka.git
>> -Pgit_ref=1.1 -Pgit_sha1=4dae083af486eaedd27c69c973c74605bffd416b
>> -Pbase_version=1.1.1
>>
>> 3. Version bumps and tests
>> Hadoop 2.8.5, Kafka 1.1.1, Spark 2.4.0, Alluxio 1.8.1
>> New smoke tests: Flink, Giraph (Crunch)
>>
>> 4. Loooooooots of bug fixes!
>>
>> If agreed, please +1 and I'll still serve as the RM for 1.4 release, if no
>> preemption ;)
>>
>>
>> Best,
>> Evans
>>
>>
>>
>>
Re: [DISCUSS] Jump to 1.4.0 release directly instead of 1.3.1
Posted by Evans Ye <ev...@apache.org>.
-1 welcomed as well !!!
Evans Ye <ev...@apache.org> 於 2019年3月4日 週一 下午3:11寫道：
> Hi all,
>
> The overhaul as well as development is almost completed. With several new
> features added, I'd like to propose to release Bigtop 1.4.0 directly
> instead of doing 1.3.1 release(https://s.apache.org/2a8K). This helps to
> promote these new features:
>
> 1. Full support to build and test inside docker with one stop seamlessly
> integration.
> Now just one command to build and test components:
> Example:
> $ ./gradlew spark-ind repo-ind docker-provisioner -POS=ubuntu-16.04
> -Pnexus -Penable_local_repo -Pstack=spark-standalone -Psmoke-tests=spark
>
> 2. Support to build from git commit hash
> Example:
> $ ./gradlew kafka-pkg-ind -Pgit_repo=https://github.com/apache/kafka.git
> -Pgit_ref=1.1 -Pgit_sha1=4dae083af486eaedd27c69c973c74605bffd416b
> -Pbase_version=1.1.1
>
> 3. Version bumps and tests
> Hadoop 2.8.5, Kafka 1.1.1, Spark 2.4.0, Alluxio 1.8.1
> New smoke tests: Flink, Giraph (Crunch)
>
> 4. Loooooooots of bug fixes!
>
> If agreed, please +1 and I'll still serve as the RM for 1.4 release, if no
> preemption ;)
>
>
> Best,
> Evans
>
>
>
>