You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Scott Wegner <sw...@google.com> on 2018/04/30 23:58:48 UTC
Gradle Status: Migrated!
Many many of you have been hacking diligently on the Gradle build, and I'm
happy to announce that we now have a fully-functioning Gradle build!
There's been a ton of progress since our last update [1]:
* Improved nightly snapshot release [2]
* Improve runner quickstarts [5] [11]
* Python post-commit ported to Gradle [3]
* Update performance testing framework for Gradle [4] [12]
* Generate javadocs from Gradle [6]
* Update to latest Gradle version [7] [21]
* Updated documentation [8] [22]
* Tune CI build resource usage for Jenkins [9] [19]
* Improve shading of test jars [10] [13] [14]
* Add 'errorprone' and 'spotless' static analysis [15] [24]
* Improve IntelliJ project generation [16] [17]
* Reduce number of ValidatesRunner tests [18]
* Update release documentation for Gradle [20]
* Update docker build scripts for Gradle [23]
The build process and Jenkins environment have stabilized and we've
resolved migration blockers. The final step is to use Gradle to produce an
official release. The release documentation has been updated for Gradle and
I recommend we use these docs for the 2.5.0 release. Assuming the release
goes well, we can declare the migration fully validated and stop supporting
dual build systems.
During the migration we identified a number of opportunities to improve the
build even further. Feel free to grab one of the items off of the JIRA:
BEAM-4045 [24]
Thanks again to all those that contributed. This has truly been a community
effort!
[1]
https://lists.apache.org/thread.html/5f6bae323acc1b050962e68ec310613e0121b05bc5c42915c536fb59@%3Cdev.beam.apache.org%3E
[2] https://github.com/apache/beam/pull/5142
[3] https://github.com/apache/beam/pull/5146
[4] https://github.com/apache/beam/pull/5003
[5] https://github.com/apache/beam/pull/5151
[6] https://github.com/apache/beam/pull/5121
[7] https://github.com/apache/beam/pull/5104
[8] https://github.com/apache/beam/pull/5183
[9] https://github.com/apache/beam/pull/5171
[10] https://github.com/apache/beam/pull/5117
[11] https://github.com/apache/beam/pull/5200
[12] https://github.com/apache/beam/pull/5051
[13] https://github.com/apache/beam/pull/4740
[14] https://github.com/apache/beam/pull/4702
[15] https://github.com/apache/beam/pull/4701
[16] https://github.com/apache/beam/pull/4626
[17] https://github.com/apache/beam/pull/4625
[18] https://github.com/apache/beam/pull/5193
[19] https://github.com/apache/beam/pull/5222
[20] https://github.com/apache/beam/pull/5187
[21] https://github.com/apache/beam/pull/5217
[22] https://github.com/apache/beam/pull/5115
[23] https://github.com/apache/beam/pull/5252
[24] https://github.com/apache/beam/pull/5161
[25] https://issues.apache.org/jira/browse/BEAM-4045
--
Got feedback? http://go/swegner-feedback
Re: Gradle Status: Migrated!
Posted by Henning Rohde <he...@google.com>.
JB - for your comparison, please also omit cross-compiling all the Go
examples because they are only built using Gradle.
On Tue, May 1, 2018 at 8:59 AM Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
> Thanks for the update Kenn, that makes sense.
>
> I'm checking the artifacts generated by Gradle right now.
>
> Regards
> JB
>
> On 01/05/2018 17:42, Kenneth Knowles wrote:
> > Raw execution time for tasks from clean is not the only thing to test. I
> > would say it is not even important. Try these from clean:
> >
> > - Gradle: ./gradlew :beam-sdks-java-io-mongodb:test && ./gradlew
> > :beam-sdks-java-io-mongodb:test
> > - Maven: mvn -pl sdks/java/io/mongodb test -am && mvn -pl
> > sdks/java/io/mongodb test -am
> >
> > Quick run on my laptop:
> >
> > - Gradle: 66s (65s then 1s)
> > - Maven: 317s (173s then 144s)
> >
> > Of course, the mvn command runs a bunch of useless executions AND it is
> > incorrect because it isn't using built jars. That's part of the point -
> > there is no way to do what you want with mvn. Let's try to make a
> > command that avoids useless work and builds the jars:
> >
> > - Maven: (mvn -pl sdks/java/io/mongodb install -DskipTests -am && mvn
> > -pl sdks/java/io/mongodb test) && (each time)
> >
> > That takes 102s the first time and 64s the second time. And that is
> > about the realistic workflow for someone trying to get something done.
> > Even if we touch a file Gradle finishes in 20s. So the best case for mvn
> > is this head-to-head:
> >
> > - Gradle: 65s + 20s + 20s + 20s + 20s + ...
> > - Maven: 102s + 64s + 64s + 64s + 64s + ...
> >
> > Kenn
> >
> >
> > On Tue, May 1, 2018 at 8:09 AM Jean-Baptiste Onofré <jb@nanthrax.net
> > <ma...@nanthrax.net>> wrote:
> >
> > Thanks, for me, Maven 3.5.2 takes quite the same time than Gradle
> > (using
> > the wrapper). It's maybe related to my environment.
> >
> > Anyway, I'm doing a complete build review both in term of building
> > time,
> > and equivalence (artifacts publishing, test, plugin execution).
> >
> > I will provide an update soon.
> >
> > Regards
> > JB
> >
> > On 01/05/2018 16:57, Reuven Lax wrote:
> > > Luke did gather data which showed that on our Jenkins executors
> the
> > > Gradle build was much faster than the Maven build. Also right now
> we
> > > have incremental builds turned off, but once we're confident
> > enough to
> > > enable them (at least for local development) that will often drop
> > build
> > > times a lot.
> > >
> > > On Tue, May 1, 2018 at 4:01 AM Jean-Baptiste Onofré
> > <jb@nanthrax.net <ma...@nanthrax.net>
> > > <mailto:jb@nanthrax.net <ma...@nanthrax.net>>> wrote:
> > >
> > > By the way, I'm curious: did someone evaluate the build time
> gap
> > > between Maven
> > > and Gradle ? One of the main reason to migrate to Gradle was
> > the inc
> > > build and
> > > build time. The builds I have launched are quite the same in
> > > duration. I will do
> > > deeper tests to evaluate the gap.
> > >
> > > Regards
> > > JB
> > >
> > > On 05/01/2018 12:48 PM, Łukasz Gajowy wrote:
> > > > Hi Scott,
> > > >
> > > > thanks for the update! Just a clarification about IO
> > performance
> > > tests: those
> > > > were fully migrated in Beam and all task necessary for
> running
> > > them are there
> > > > but Jenkins jobs still run mvn commands. This is due the
> > fact that
> > > > PerfkitBenchmarker code (which is invoked by Jenkins and
> > > constructs the commands
> > > > by itself) was not updated yet. This should be finished
> before
> > > fully dropping mvn.
> > > >
> > > > More on that topic here, in
> > > > comments: https://issues.apache.org/jira/browse/BEAM-3942
> > > > PR changing the commands to gradle is waiting for PerfKit
> > devs review
> > > > here:
> > >
> https://github.com/GoogleCloudPlatform/PerfKitBenchmarker/pull/1648
> > > >
> > > > Best regards,
> > > >
> > > > 2018-05-01 9:17 GMT+02:00 Romain Manni-Bucau
> > > <rmannibucau@gmail.com <ma...@gmail.com>
> > <mailto:rmannibucau@gmail.com <ma...@gmail.com>>
> > > > <mailto:rmannibucau@gmail.com
> > <ma...@gmail.com> <mailto:rmannibucau@gmail.com
> > <ma...@gmail.com>>>>:
> > > >
> > > > Hi Scott
> > > >
> > > > While
> > >
> https://issues.apache.org/jira/plugins/servlet/mobile#issue/BEAM-4057
> > > >
> > >
> > <
> https://issues.apache.org/jira/plugins/servlet/mobile#issue/BEAM-4057> is
> > > > open, gradle is a concurrent of maven but maven must
> > stay the
> > > default build
> > > > tool cause gradle breaks users.
> > > >
> > > >
> > > > Le 1 mai 2018 01:59, "Scott Wegner"
> > <swegner@google.com <ma...@google.com>
> > > <mailto:swegner@google.com <ma...@google.com>>
> > > > <mailto:swegner@google.com <ma...@google.com>
> > <mailto:swegner@google.com <ma...@google.com>>>> a
> > > écrit :
> > > >
> > > > Many many of you have been hacking diligently on
> the
> > > Gradle build, and
> > > > I'm happy to announce that we now have a
> > > fully-functioning Gradle build!
> > > > There's been a ton of progress since our last
> > update [1]:
> > > >
> > > > * Improved nightly snapshot release [2]
> > > > * Improve runner quickstarts [5] [11]
> > > > * Python post-commit ported to Gradle [3]
> > > > * Update performance testing framework for Gradle
> > [4] [12]
> > > > * Generate javadocs from Gradle [6]
> > > > * Update to latest Gradle version [7] [21]
> > > > * Updated documentation [8] [22]
> > > > * Tune CI build resource usage for Jenkins [9] [19]
> > > > * Improve shading of test jars [10] [13] [14]
> > > > * Add 'errorprone' and 'spotless' static analysis
> > [15] [24]
> > > > * Improve IntelliJ project generation [16] [17]
> > > > * Reduce number of ValidatesRunner tests [18]
> > > > * Update release documentation for Gradle [20]
> > > > * Update docker build scripts for Gradle [23]
> > > >
> > > > The build process and Jenkins environment have
> > stabilized
> > > and we've
> > > > resolved migration blockers. The final step is to
> use
> > > Gradle to produce
> > > > an official release. The release documentation has
> > been
> > > updated for
> > > > Gradle and I recommend we use these docs for the
> 2.5.0
> > > release. Assuming
> > > > the release goes well, we can declare the
> > migration fully
> > > validated and
> > > > stop supporting dual build systems.
> > > >
> > > > During the migration we identified a number of
> > > opportunities to improve
> > > > the build even further. Feel free to grab one of
> the
> > > items off of the
> > > > JIRA: BEAM-4045 [24]
> > > >
> > > > Thanks again to all those that contributed. This
> has
> > > truly been a
> > > > community effort!
> > > >
> > > > [1]
> > >
> >
> https://lists.apache.org/thread.html/5f6bae323acc1b050962e68ec310613e0121b05bc5c42915c536fb59@%3Cdev.beam.apache.org%3E
> > > >
> > >
> > <
> https://lists.apache.org/thread.html/5f6bae323acc1b050962e68ec310613e0121b05bc5c42915c536fb59@%3Cdev.beam.apache.org%3E
> >
> > > > [2] https://github.com/apache/beam/pull/5142
> > > > <https://github.com/apache/beam/pull/5142>
> > > > [3] https://github.com/apache/beam/pull/5146
> > > > <https://github.com/apache/beam/pull/5146>
> > > > [4] https://github.com/apache/beam/pull/5003
> > > > <https://github.com/apache/beam/pull/5003>
> > > > [5] https://github.com/apache/beam/pull/5151
> > > > <https://github.com/apache/beam/pull/5151>
> > > > [6] https://github.com/apache/beam/pull/5121
> > > > <https://github.com/apache/beam/pull/5121>
> > > > [7] https://github.com/apache/beam/pull/5104
> > > > <https://github.com/apache/beam/pull/5104>
> > > > [8] https://github.com/apache/beam/pull/5183
> > > > <https://github.com/apache/beam/pull/5183>
> > > > [9] https://github.com/apache/beam/pull/5171
> > > > <https://github.com/apache/beam/pull/5171>
> > > > [10] https://github.com/apache/beam/pull/5117
> > > > <https://github.com/apache/beam/pull/5117>
> > > > [11] https://github.com/apache/beam/pull/5200
> > > > <https://github.com/apache/beam/pull/5200>
> > > > [12] https://github.com/apache/beam/pull/5051
> > > > <https://github.com/apache/beam/pull/5051>
> > > > [13] https://github.com/apache/beam/pull/4740
> > > > <https://github.com/apache/beam/pull/4740>
> > > > [14] https://github.com/apache/beam/pull/4702
> > > > <https://github.com/apache/beam/pull/4702>
> > > > [15] https://github.com/apache/beam/pull/4701
> > > > <https://github.com/apache/beam/pull/4701>
> > > > [16] https://github.com/apache/beam/pull/4626
> > > > <https://github.com/apache/beam/pull/4626>
> > > > [17] https://github.com/apache/beam/pull/4625
> > > > <https://github.com/apache/beam/pull/4625>
> > > > [18] https://github.com/apache/beam/pull/5193
> > > > <https://github.com/apache/beam/pull/5193>
> > > > [19] https://github.com/apache/beam/pull/5222
> > > > <https://github.com/apache/beam/pull/5222>
> > > > [20] https://github.com/apache/beam/pull/5187
> > > > <https://github.com/apache/beam/pull/5187>
> > > > [21] https://github.com/apache/beam/pull/5217
> > > > <https://github.com/apache/beam/pull/5217>
> > > > [22] https://github.com/apache/beam/pull/5115
> > > > <https://github.com/apache/beam/pull/5115>
> > > > [23] https://github.com/apache/beam/pull/5252
> > > > <https://github.com/apache/beam/pull/5252>
> > > > [24] https://github.com/apache/beam/pull/5161
> > > > <https://github.com/apache/beam/pull/5161>
> > > > [25]
> https://issues.apache.org/jira/browse/BEAM-4045
> > > > <https://issues.apache.org/jira/browse/BEAM-4045>
> > > > --
> > > >
> > > >
> > > > Got feedback? http://go/swegner-feedback
> > > >
> > > >
> > >
> > > --
> > > Jean-Baptiste Onofré
> > > jbonofre@apache.org <ma...@apache.org>
> > <mailto:jbonofre@apache.org <ma...@apache.org>>
> > > http://blog.nanthrax.net
> > > Talend - http://www.talend.com
> > >
> >
>
Re: Gradle Status: Migrated!
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Thanks for the update Kenn, that makes sense.
I'm checking the artifacts generated by Gradle right now.
Regards
JB
On 01/05/2018 17:42, Kenneth Knowles wrote:
> Raw execution time for tasks from clean is not the only thing to test. I
> would say it is not even important. Try these from clean:
>
> - Gradle: ./gradlew :beam-sdks-java-io-mongodb:test && ./gradlew
> :beam-sdks-java-io-mongodb:test
> - Maven: mvn -pl sdks/java/io/mongodb test -am && mvn -pl
> sdks/java/io/mongodb test -am
>
> Quick run on my laptop:
>
> - Gradle: 66s (65s then 1s)
> - Maven: 317s (173s then 144s)
>
> Of course, the mvn command runs a bunch of useless executions AND it is
> incorrect because it isn't using built jars. That's part of the point -
> there is no way to do what you want with mvn. Let's try to make a
> command that avoids useless work and builds the jars:
>
> - Maven: (mvn -pl sdks/java/io/mongodb install -DskipTests -am && mvn
> -pl sdks/java/io/mongodb test) && (each time)
>
> That takes 102s the first time and 64s the second time. And that is
> about the realistic workflow for someone trying to get something done.
> Even if we touch a file Gradle finishes in 20s. So the best case for mvn
> is this head-to-head:
>
> - Gradle: 65s + 20s + 20s + 20s + 20s + ...
> - Maven: 102s + 64s + 64s + 64s + 64s + ...
>
> Kenn
>
>
> On Tue, May 1, 2018 at 8:09 AM Jean-Baptiste Onofré <jb@nanthrax.net
> <ma...@nanthrax.net>> wrote:
>
> Thanks, for me, Maven 3.5.2 takes quite the same time than Gradle
> (using
> the wrapper). It's maybe related to my environment.
>
> Anyway, I'm doing a complete build review both in term of building
> time,
> and equivalence (artifacts publishing, test, plugin execution).
>
> I will provide an update soon.
>
> Regards
> JB
>
> On 01/05/2018 16:57, Reuven Lax wrote:
> > Luke did gather data which showed that on our Jenkins executors the
> > Gradle build was much faster than the Maven build. Also right now we
> > have incremental builds turned off, but once we're confident
> enough to
> > enable them (at least for local development) that will often drop
> build
> > times a lot.
> >
> > On Tue, May 1, 2018 at 4:01 AM Jean-Baptiste Onofré
> <jb@nanthrax.net <ma...@nanthrax.net>
> > <mailto:jb@nanthrax.net <ma...@nanthrax.net>>> wrote:
> >
> > By the way, I'm curious: did someone evaluate the build time gap
> > between Maven
> > and Gradle ? One of the main reason to migrate to Gradle was
> the inc
> > build and
> > build time. The builds I have launched are quite the same in
> > duration. I will do
> > deeper tests to evaluate the gap.
> >
> > Regards
> > JB
> >
> > On 05/01/2018 12:48 PM, Łukasz Gajowy wrote:
> > > Hi Scott,
> > >
> > > thanks for the update! Just a clarification about IO
> performance
> > tests: those
> > > were fully migrated in Beam and all task necessary for running
> > them are there
> > > but Jenkins jobs still run mvn commands. This is due the
> fact that
> > > PerfkitBenchmarker code (which is invoked by Jenkins and
> > constructs the commands
> > > by itself) was not updated yet. This should be finished before
> > fully dropping mvn.
> > >
> > > More on that topic here, in
> > > comments: https://issues.apache.org/jira/browse/BEAM-3942
> > > PR changing the commands to gradle is waiting for PerfKit
> devs review
> > > here:
> > https://github.com/GoogleCloudPlatform/PerfKitBenchmarker/pull/1648
> > >
> > > Best regards,
> > >
> > > 2018-05-01 9:17 GMT+02:00 Romain Manni-Bucau
> > <rmannibucau@gmail.com <ma...@gmail.com>
> <mailto:rmannibucau@gmail.com <ma...@gmail.com>>
> > > <mailto:rmannibucau@gmail.com
> <ma...@gmail.com> <mailto:rmannibucau@gmail.com
> <ma...@gmail.com>>>>:
> > >
> > > Hi Scott
> > >
> > > While
> > https://issues.apache.org/jira/plugins/servlet/mobile#issue/BEAM-4057
> > >
> >
> <https://issues.apache.org/jira/plugins/servlet/mobile#issue/BEAM-4057> is
> > > open, gradle is a concurrent of maven but maven must
> stay the
> > default build
> > > tool cause gradle breaks users.
> > >
> > >
> > > Le 1 mai 2018 01:59, "Scott Wegner"
> <swegner@google.com <ma...@google.com>
> > <mailto:swegner@google.com <ma...@google.com>>
> > > <mailto:swegner@google.com <ma...@google.com>
> <mailto:swegner@google.com <ma...@google.com>>>> a
> > écrit :
> > >
> > > Many many of you have been hacking diligently on the
> > Gradle build, and
> > > I'm happy to announce that we now have a
> > fully-functioning Gradle build!
> > > There's been a ton of progress since our last
> update [1]:
> > >
> > > * Improved nightly snapshot release [2]
> > > * Improve runner quickstarts [5] [11]
> > > * Python post-commit ported to Gradle [3]
> > > * Update performance testing framework for Gradle
> [4] [12]
> > > * Generate javadocs from Gradle [6]
> > > * Update to latest Gradle version [7] [21]
> > > * Updated documentation [8] [22]
> > > * Tune CI build resource usage for Jenkins [9] [19]
> > > * Improve shading of test jars [10] [13] [14]
> > > * Add 'errorprone' and 'spotless' static analysis
> [15] [24]
> > > * Improve IntelliJ project generation [16] [17]
> > > * Reduce number of ValidatesRunner tests [18]
> > > * Update release documentation for Gradle [20]
> > > * Update docker build scripts for Gradle [23]
> > >
> > > The build process and Jenkins environment have
> stabilized
> > and we've
> > > resolved migration blockers. The final step is to use
> > Gradle to produce
> > > an official release. The release documentation has
> been
> > updated for
> > > Gradle and I recommend we use these docs for the 2.5.0
> > release. Assuming
> > > the release goes well, we can declare the
> migration fully
> > validated and
> > > stop supporting dual build systems.
> > >
> > > During the migration we identified a number of
> > opportunities to improve
> > > the build even further. Feel free to grab one of the
> > items off of the
> > > JIRA: BEAM-4045 [24]
> > >
> > > Thanks again to all those that contributed. This has
> > truly been a
> > > community effort!
> > >
> > > [1]
> >
> https://lists.apache.org/thread.html/5f6bae323acc1b050962e68ec310613e0121b05bc5c42915c536fb59@%3Cdev.beam.apache.org%3E
> > >
> >
> <https://lists.apache.org/thread.html/5f6bae323acc1b050962e68ec310613e0121b05bc5c42915c536fb59@%3Cdev.beam.apache.org%3E>
> > > [2] https://github.com/apache/beam/pull/5142
> > > <https://github.com/apache/beam/pull/5142>
> > > [3] https://github.com/apache/beam/pull/5146
> > > <https://github.com/apache/beam/pull/5146>
> > > [4] https://github.com/apache/beam/pull/5003
> > > <https://github.com/apache/beam/pull/5003>
> > > [5] https://github.com/apache/beam/pull/5151
> > > <https://github.com/apache/beam/pull/5151>
> > > [6] https://github.com/apache/beam/pull/5121
> > > <https://github.com/apache/beam/pull/5121>
> > > [7] https://github.com/apache/beam/pull/5104
> > > <https://github.com/apache/beam/pull/5104>
> > > [8] https://github.com/apache/beam/pull/5183
> > > <https://github.com/apache/beam/pull/5183>
> > > [9] https://github.com/apache/beam/pull/5171
> > > <https://github.com/apache/beam/pull/5171>
> > > [10] https://github.com/apache/beam/pull/5117
> > > <https://github.com/apache/beam/pull/5117>
> > > [11] https://github.com/apache/beam/pull/5200
> > > <https://github.com/apache/beam/pull/5200>
> > > [12] https://github.com/apache/beam/pull/5051
> > > <https://github.com/apache/beam/pull/5051>
> > > [13] https://github.com/apache/beam/pull/4740
> > > <https://github.com/apache/beam/pull/4740>
> > > [14] https://github.com/apache/beam/pull/4702
> > > <https://github.com/apache/beam/pull/4702>
> > > [15] https://github.com/apache/beam/pull/4701
> > > <https://github.com/apache/beam/pull/4701>
> > > [16] https://github.com/apache/beam/pull/4626
> > > <https://github.com/apache/beam/pull/4626>
> > > [17] https://github.com/apache/beam/pull/4625
> > > <https://github.com/apache/beam/pull/4625>
> > > [18] https://github.com/apache/beam/pull/5193
> > > <https://github.com/apache/beam/pull/5193>
> > > [19] https://github.com/apache/beam/pull/5222
> > > <https://github.com/apache/beam/pull/5222>
> > > [20] https://github.com/apache/beam/pull/5187
> > > <https://github.com/apache/beam/pull/5187>
> > > [21] https://github.com/apache/beam/pull/5217
> > > <https://github.com/apache/beam/pull/5217>
> > > [22] https://github.com/apache/beam/pull/5115
> > > <https://github.com/apache/beam/pull/5115>
> > > [23] https://github.com/apache/beam/pull/5252
> > > <https://github.com/apache/beam/pull/5252>
> > > [24] https://github.com/apache/beam/pull/5161
> > > <https://github.com/apache/beam/pull/5161>
> > > [25] https://issues.apache.org/jira/browse/BEAM-4045
> > > <https://issues.apache.org/jira/browse/BEAM-4045>
> > > --
> > >
> > >
> > > Got feedback? http://go/swegner-feedback
> > >
> > >
> >
> > --
> > Jean-Baptiste Onofré
> > jbonofre@apache.org <ma...@apache.org>
> <mailto:jbonofre@apache.org <ma...@apache.org>>
> > http://blog.nanthrax.net
> > Talend - http://www.talend.com
> >
>
Re: Gradle Status: Migrated!
Posted by Kenneth Knowles <kl...@google.com>.
Raw execution time for tasks from clean is not the only thing to test. I
would say it is not even important. Try these from clean:
- Gradle: ./gradlew :beam-sdks-java-io-mongodb:test && ./gradlew :
beam-sdks-java-io-mongodb:test
- Maven: mvn -pl sdks/java/io/mongodb test -am && mvn -pl
sdks/java/io/mongodb test -am
Quick run on my laptop:
- Gradle: 66s (65s then 1s)
- Maven: 317s (173s then 144s)
Of course, the mvn command runs a bunch of useless executions AND it is
incorrect because it isn't using built jars. That's part of the point -
there is no way to do what you want with mvn. Let's try to make a command
that avoids useless work and builds the jars:
- Maven: (mvn -pl sdks/java/io/mongodb install -DskipTests -am && mvn -pl
sdks/java/io/mongodb test) && (each time)
That takes 102s the first time and 64s the second time. And that is about
the realistic workflow for someone trying to get something done. Even if we
touch a file Gradle finishes in 20s. So the best case for mvn is this
head-to-head:
- Gradle: 65s + 20s + 20s + 20s + 20s + ...
- Maven: 102s + 64s + 64s + 64s + 64s + ...
Kenn
On Tue, May 1, 2018 at 8:09 AM Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
> Thanks, for me, Maven 3.5.2 takes quite the same time than Gradle (using
> the wrapper). It's maybe related to my environment.
>
> Anyway, I'm doing a complete build review both in term of building time,
> and equivalence (artifacts publishing, test, plugin execution).
>
> I will provide an update soon.
>
> Regards
> JB
>
> On 01/05/2018 16:57, Reuven Lax wrote:
> > Luke did gather data which showed that on our Jenkins executors the
> > Gradle build was much faster than the Maven build. Also right now we
> > have incremental builds turned off, but once we're confident enough to
> > enable them (at least for local development) that will often drop build
> > times a lot.
> >
> > On Tue, May 1, 2018 at 4:01 AM Jean-Baptiste Onofré <jb@nanthrax.net
> > <ma...@nanthrax.net>> wrote:
> >
> > By the way, I'm curious: did someone evaluate the build time gap
> > between Maven
> > and Gradle ? One of the main reason to migrate to Gradle was the inc
> > build and
> > build time. The builds I have launched are quite the same in
> > duration. I will do
> > deeper tests to evaluate the gap.
> >
> > Regards
> > JB
> >
> > On 05/01/2018 12:48 PM, Łukasz Gajowy wrote:
> > > Hi Scott,
> > >
> > > thanks for the update! Just a clarification about IO performance
> > tests: those
> > > were fully migrated in Beam and all task necessary for running
> > them are there
> > > but Jenkins jobs still run mvn commands. This is due the fact that
> > > PerfkitBenchmarker code (which is invoked by Jenkins and
> > constructs the commands
> > > by itself) was not updated yet. This should be finished before
> > fully dropping mvn.
> > >
> > > More on that topic here, in
> > > comments: https://issues.apache.org/jira/browse/BEAM-3942
> > > PR changing the commands to gradle is waiting for PerfKit devs
> review
> > > here:
> > https://github.com/GoogleCloudPlatform/PerfKitBenchmarker/pull/1648
> > >
> > > Best regards,
> > >
> > > 2018-05-01 9:17 GMT+02:00 Romain Manni-Bucau
> > <rmannibucau@gmail.com <ma...@gmail.com>
> > > <mailto:rmannibucau@gmail.com <ma...@gmail.com>>>:
> > >
> > > Hi Scott
> > >
> > > While
> >
> https://issues.apache.org/jira/plugins/servlet/mobile#issue/BEAM-4057
> > >
> > <
> https://issues.apache.org/jira/plugins/servlet/mobile#issue/BEAM-4057> is
> > > open, gradle is a concurrent of maven but maven must stay the
> > default build
> > > tool cause gradle breaks users.
> > >
> > >
> > > Le 1 mai 2018 01:59, "Scott Wegner" <swegner@google.com
> > <ma...@google.com>
> > > <mailto:swegner@google.com <ma...@google.com>>> a
> > écrit :
> > >
> > > Many many of you have been hacking diligently on the
> > Gradle build, and
> > > I'm happy to announce that we now have a
> > fully-functioning Gradle build!
> > > There's been a ton of progress since our last update [1]:
> > >
> > > * Improved nightly snapshot release [2]
> > > * Improve runner quickstarts [5] [11]
> > > * Python post-commit ported to Gradle [3]
> > > * Update performance testing framework for Gradle [4] [12]
> > > * Generate javadocs from Gradle [6]
> > > * Update to latest Gradle version [7] [21]
> > > * Updated documentation [8] [22]
> > > * Tune CI build resource usage for Jenkins [9] [19]
> > > * Improve shading of test jars [10] [13] [14]
> > > * Add 'errorprone' and 'spotless' static analysis [15]
> [24]
> > > * Improve IntelliJ project generation [16] [17]
> > > * Reduce number of ValidatesRunner tests [18]
> > > * Update release documentation for Gradle [20]
> > > * Update docker build scripts for Gradle [23]
> > >
> > > The build process and Jenkins environment have stabilized
> > and we've
> > > resolved migration blockers. The final step is to use
> > Gradle to produce
> > > an official release. The release documentation has been
> > updated for
> > > Gradle and I recommend we use these docs for the 2.5.0
> > release. Assuming
> > > the release goes well, we can declare the migration fully
> > validated and
> > > stop supporting dual build systems.
> > >
> > > During the migration we identified a number of
> > opportunities to improve
> > > the build even further. Feel free to grab one of the
> > items off of the
> > > JIRA: BEAM-4045 [24]
> > >
> > > Thanks again to all those that contributed. This has
> > truly been a
> > > community effort!
> > >
> > > [1]
> >
> https://lists.apache.org/thread.html/5f6bae323acc1b050962e68ec310613e0121b05bc5c42915c536fb59@%3Cdev.beam.apache.org%3E
> > >
> > <
> https://lists.apache.org/thread.html/5f6bae323acc1b050962e68ec310613e0121b05bc5c42915c536fb59@%3Cdev.beam.apache.org%3E
> >
> > > [2] https://github.com/apache/beam/pull/5142
> > > <https://github.com/apache/beam/pull/5142>
> > > [3] https://github.com/apache/beam/pull/5146
> > > <https://github.com/apache/beam/pull/5146>
> > > [4] https://github.com/apache/beam/pull/5003
> > > <https://github.com/apache/beam/pull/5003>
> > > [5] https://github.com/apache/beam/pull/5151
> > > <https://github.com/apache/beam/pull/5151>
> > > [6] https://github.com/apache/beam/pull/5121
> > > <https://github.com/apache/beam/pull/5121>
> > > [7] https://github.com/apache/beam/pull/5104
> > > <https://github.com/apache/beam/pull/5104>
> > > [8] https://github.com/apache/beam/pull/5183
> > > <https://github.com/apache/beam/pull/5183>
> > > [9] https://github.com/apache/beam/pull/5171
> > > <https://github.com/apache/beam/pull/5171>
> > > [10] https://github.com/apache/beam/pull/5117
> > > <https://github.com/apache/beam/pull/5117>
> > > [11] https://github.com/apache/beam/pull/5200
> > > <https://github.com/apache/beam/pull/5200>
> > > [12] https://github.com/apache/beam/pull/5051
> > > <https://github.com/apache/beam/pull/5051>
> > > [13] https://github.com/apache/beam/pull/4740
> > > <https://github.com/apache/beam/pull/4740>
> > > [14] https://github.com/apache/beam/pull/4702
> > > <https://github.com/apache/beam/pull/4702>
> > > [15] https://github.com/apache/beam/pull/4701
> > > <https://github.com/apache/beam/pull/4701>
> > > [16] https://github.com/apache/beam/pull/4626
> > > <https://github.com/apache/beam/pull/4626>
> > > [17] https://github.com/apache/beam/pull/4625
> > > <https://github.com/apache/beam/pull/4625>
> > > [18] https://github.com/apache/beam/pull/5193
> > > <https://github.com/apache/beam/pull/5193>
> > > [19] https://github.com/apache/beam/pull/5222
> > > <https://github.com/apache/beam/pull/5222>
> > > [20] https://github.com/apache/beam/pull/5187
> > > <https://github.com/apache/beam/pull/5187>
> > > [21] https://github.com/apache/beam/pull/5217
> > > <https://github.com/apache/beam/pull/5217>
> > > [22] https://github.com/apache/beam/pull/5115
> > > <https://github.com/apache/beam/pull/5115>
> > > [23] https://github.com/apache/beam/pull/5252
> > > <https://github.com/apache/beam/pull/5252>
> > > [24] https://github.com/apache/beam/pull/5161
> > > <https://github.com/apache/beam/pull/5161>
> > > [25] https://issues.apache.org/jira/browse/BEAM-4045
> > > <https://issues.apache.org/jira/browse/BEAM-4045>
> > > --
> > >
> > >
> > > Got feedback? http://go/swegner-feedback
> > >
> > >
> >
> > --
> > Jean-Baptiste Onofré
> > jbonofre@apache.org <ma...@apache.org>
> > http://blog.nanthrax.net
> > Talend - http://www.talend.com
> >
>
Re: Gradle Status: Migrated!
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Thanks, for me, Maven 3.5.2 takes quite the same time than Gradle (using
the wrapper). It's maybe related to my environment.
Anyway, I'm doing a complete build review both in term of building time,
and equivalence (artifacts publishing, test, plugin execution).
I will provide an update soon.
Regards
JB
On 01/05/2018 16:57, Reuven Lax wrote:
> Luke did gather data which showed that on our Jenkins executors the
> Gradle build was much faster than the Maven build. Also right now we
> have incremental builds turned off, but once we're confident enough to
> enable them (at least for local development) that will often drop build
> times a lot.
>
> On Tue, May 1, 2018 at 4:01 AM Jean-Baptiste Onofré <jb@nanthrax.net
> <ma...@nanthrax.net>> wrote:
>
> By the way, I'm curious: did someone evaluate the build time gap
> between Maven
> and Gradle ? One of the main reason to migrate to Gradle was the inc
> build and
> build time. The builds I have launched are quite the same in
> duration. I will do
> deeper tests to evaluate the gap.
>
> Regards
> JB
>
> On 05/01/2018 12:48 PM, Łukasz Gajowy wrote:
> > Hi Scott,
> >
> > thanks for the update! Just a clarification about IO performance
> tests: those
> > were fully migrated in Beam and all task necessary for running
> them are there
> > but Jenkins jobs still run mvn commands. This is due the fact that
> > PerfkitBenchmarker code (which is invoked by Jenkins and
> constructs the commands
> > by itself) was not updated yet. This should be finished before
> fully dropping mvn.
> >
> > More on that topic here, in
> > comments: https://issues.apache.org/jira/browse/BEAM-3942
> > PR changing the commands to gradle is waiting for PerfKit devs review
> > here:
> https://github.com/GoogleCloudPlatform/PerfKitBenchmarker/pull/1648
> >
> > Best regards,
> >
> > 2018-05-01 9:17 GMT+02:00 Romain Manni-Bucau
> <rmannibucau@gmail.com <ma...@gmail.com>
> > <mailto:rmannibucau@gmail.com <ma...@gmail.com>>>:
> >
> > Hi Scott
> >
> > While
> https://issues.apache.org/jira/plugins/servlet/mobile#issue/BEAM-4057
> >
> <https://issues.apache.org/jira/plugins/servlet/mobile#issue/BEAM-4057> is
> > open, gradle is a concurrent of maven but maven must stay the
> default build
> > tool cause gradle breaks users.
> >
> >
> > Le 1 mai 2018 01:59, "Scott Wegner" <swegner@google.com
> <ma...@google.com>
> > <mailto:swegner@google.com <ma...@google.com>>> a
> écrit :
> >
> > Many many of you have been hacking diligently on the
> Gradle build, and
> > I'm happy to announce that we now have a
> fully-functioning Gradle build!
> > There's been a ton of progress since our last update [1]:
> >
> > * Improved nightly snapshot release [2]
> > * Improve runner quickstarts [5] [11]
> > * Python post-commit ported to Gradle [3]
> > * Update performance testing framework for Gradle [4] [12]
> > * Generate javadocs from Gradle [6]
> > * Update to latest Gradle version [7] [21]
> > * Updated documentation [8] [22]
> > * Tune CI build resource usage for Jenkins [9] [19]
> > * Improve shading of test jars [10] [13] [14]
> > * Add 'errorprone' and 'spotless' static analysis [15] [24]
> > * Improve IntelliJ project generation [16] [17]
> > * Reduce number of ValidatesRunner tests [18]
> > * Update release documentation for Gradle [20]
> > * Update docker build scripts for Gradle [23]
> >
> > The build process and Jenkins environment have stabilized
> and we've
> > resolved migration blockers. The final step is to use
> Gradle to produce
> > an official release. The release documentation has been
> updated for
> > Gradle and I recommend we use these docs for the 2.5.0
> release. Assuming
> > the release goes well, we can declare the migration fully
> validated and
> > stop supporting dual build systems.
> >
> > During the migration we identified a number of
> opportunities to improve
> > the build even further. Feel free to grab one of the
> items off of the
> > JIRA: BEAM-4045 [24]
> >
> > Thanks again to all those that contributed. This has
> truly been a
> > community effort!
> >
> > [1]
> https://lists.apache.org/thread.html/5f6bae323acc1b050962e68ec310613e0121b05bc5c42915c536fb59@%3Cdev.beam.apache.org%3E
> >
> <https://lists.apache.org/thread.html/5f6bae323acc1b050962e68ec310613e0121b05bc5c42915c536fb59@%3Cdev.beam.apache.org%3E>
> > [2] https://github.com/apache/beam/pull/5142
> > <https://github.com/apache/beam/pull/5142>
> > [3] https://github.com/apache/beam/pull/5146
> > <https://github.com/apache/beam/pull/5146>
> > [4] https://github.com/apache/beam/pull/5003
> > <https://github.com/apache/beam/pull/5003>
> > [5] https://github.com/apache/beam/pull/5151
> > <https://github.com/apache/beam/pull/5151>
> > [6] https://github.com/apache/beam/pull/5121
> > <https://github.com/apache/beam/pull/5121>
> > [7] https://github.com/apache/beam/pull/5104
> > <https://github.com/apache/beam/pull/5104>
> > [8] https://github.com/apache/beam/pull/5183
> > <https://github.com/apache/beam/pull/5183>
> > [9] https://github.com/apache/beam/pull/5171
> > <https://github.com/apache/beam/pull/5171>
> > [10] https://github.com/apache/beam/pull/5117
> > <https://github.com/apache/beam/pull/5117>
> > [11] https://github.com/apache/beam/pull/5200
> > <https://github.com/apache/beam/pull/5200>
> > [12] https://github.com/apache/beam/pull/5051
> > <https://github.com/apache/beam/pull/5051>
> > [13] https://github.com/apache/beam/pull/4740
> > <https://github.com/apache/beam/pull/4740>
> > [14] https://github.com/apache/beam/pull/4702
> > <https://github.com/apache/beam/pull/4702>
> > [15] https://github.com/apache/beam/pull/4701
> > <https://github.com/apache/beam/pull/4701>
> > [16] https://github.com/apache/beam/pull/4626
> > <https://github.com/apache/beam/pull/4626>
> > [17] https://github.com/apache/beam/pull/4625
> > <https://github.com/apache/beam/pull/4625>
> > [18] https://github.com/apache/beam/pull/5193
> > <https://github.com/apache/beam/pull/5193>
> > [19] https://github.com/apache/beam/pull/5222
> > <https://github.com/apache/beam/pull/5222>
> > [20] https://github.com/apache/beam/pull/5187
> > <https://github.com/apache/beam/pull/5187>
> > [21] https://github.com/apache/beam/pull/5217
> > <https://github.com/apache/beam/pull/5217>
> > [22] https://github.com/apache/beam/pull/5115
> > <https://github.com/apache/beam/pull/5115>
> > [23] https://github.com/apache/beam/pull/5252
> > <https://github.com/apache/beam/pull/5252>
> > [24] https://github.com/apache/beam/pull/5161
> > <https://github.com/apache/beam/pull/5161>
> > [25] https://issues.apache.org/jira/browse/BEAM-4045
> > <https://issues.apache.org/jira/browse/BEAM-4045>
> > --
> >
> >
> > Got feedback? http://go/swegner-feedback
> >
> >
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org <ma...@apache.org>
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
Re: Gradle Status: Migrated!
Posted by Reuven Lax <re...@google.com>.
Luke did gather data which showed that on our Jenkins executors the Gradle
build was much faster than the Maven build. Also right now we have
incremental builds turned off, but once we're confident enough to enable
them (at least for local development) that will often drop build times a
lot.
On Tue, May 1, 2018 at 4:01 AM Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
> By the way, I'm curious: did someone evaluate the build time gap between
> Maven
> and Gradle ? One of the main reason to migrate to Gradle was the inc build
> and
> build time. The builds I have launched are quite the same in duration. I
> will do
> deeper tests to evaluate the gap.
>
> Regards
> JB
>
> On 05/01/2018 12:48 PM, Łukasz Gajowy wrote:
> > Hi Scott,
> >
> > thanks for the update! Just a clarification about IO performance tests:
> those
> > were fully migrated in Beam and all task necessary for running them are
> there
> > but Jenkins jobs still run mvn commands. This is due the fact that
> > PerfkitBenchmarker code (which is invoked by Jenkins and constructs the
> commands
> > by itself) was not updated yet. This should be finished before fully
> dropping mvn.
> >
> > More on that topic here, in
> > comments: https://issues.apache.org/jira/browse/BEAM-3942
> > PR changing the commands to gradle is waiting for PerfKit devs review
> > here:
> https://github.com/GoogleCloudPlatform/PerfKitBenchmarker/pull/1648
> >
> > Best regards,
> >
> > 2018-05-01 9:17 GMT+02:00 Romain Manni-Bucau <rmannibucau@gmail.com
> > <ma...@gmail.com>>:
> >
> > Hi Scott
> >
> > While
> https://issues.apache.org/jira/plugins/servlet/mobile#issue/BEAM-4057
> > <
> https://issues.apache.org/jira/plugins/servlet/mobile#issue/BEAM-4057> is
> > open, gradle is a concurrent of maven but maven must stay the
> default build
> > tool cause gradle breaks users.
> >
> >
> > Le 1 mai 2018 01:59, "Scott Wegner" <swegner@google.com
> > <ma...@google.com>> a écrit :
> >
> > Many many of you have been hacking diligently on the Gradle
> build, and
> > I'm happy to announce that we now have a fully-functioning
> Gradle build!
> > There's been a ton of progress since our last update [1]:
> >
> > * Improved nightly snapshot release [2]
> > * Improve runner quickstarts [5] [11]
> > * Python post-commit ported to Gradle [3]
> > * Update performance testing framework for Gradle [4] [12]
> > * Generate javadocs from Gradle [6]
> > * Update to latest Gradle version [7] [21]
> > * Updated documentation [8] [22]
> > * Tune CI build resource usage for Jenkins [9] [19]
> > * Improve shading of test jars [10] [13] [14]
> > * Add 'errorprone' and 'spotless' static analysis [15] [24]
> > * Improve IntelliJ project generation [16] [17]
> > * Reduce number of ValidatesRunner tests [18]
> > * Update release documentation for Gradle [20]
> > * Update docker build scripts for Gradle [23]
> >
> > The build process and Jenkins environment have stabilized and
> we've
> > resolved migration blockers. The final step is to use Gradle to
> produce
> > an official release. The release documentation has been updated
> for
> > Gradle and I recommend we use these docs for the 2.5.0 release.
> Assuming
> > the release goes well, we can declare the migration fully
> validated and
> > stop supporting dual build systems.
> >
> > During the migration we identified a number of opportunities to
> improve
> > the build even further. Feel free to grab one of the items off
> of the
> > JIRA: BEAM-4045 [24]
> >
> > Thanks again to all those that contributed. This has truly been a
> > community effort!
> >
> > [1]
> https://lists.apache.org/thread.html/5f6bae323acc1b050962e68ec310613e0121b05bc5c42915c536fb59@%3Cdev.beam.apache.org%3E
> > <
> https://lists.apache.org/thread.html/5f6bae323acc1b050962e68ec310613e0121b05bc5c42915c536fb59@%3Cdev.beam.apache.org%3E
> >
> > [2] https://github.com/apache/beam/pull/5142
> > <https://github.com/apache/beam/pull/5142>
> > [3] https://github.com/apache/beam/pull/5146
> > <https://github.com/apache/beam/pull/5146>
> > [4] https://github.com/apache/beam/pull/5003
> > <https://github.com/apache/beam/pull/5003>
> > [5] https://github.com/apache/beam/pull/5151
> > <https://github.com/apache/beam/pull/5151>
> > [6] https://github.com/apache/beam/pull/5121
> > <https://github.com/apache/beam/pull/5121>
> > [7] https://github.com/apache/beam/pull/5104
> > <https://github.com/apache/beam/pull/5104>
> > [8] https://github.com/apache/beam/pull/5183
> > <https://github.com/apache/beam/pull/5183>
> > [9] https://github.com/apache/beam/pull/5171
> > <https://github.com/apache/beam/pull/5171>
> > [10] https://github.com/apache/beam/pull/5117
> > <https://github.com/apache/beam/pull/5117>
> > [11] https://github.com/apache/beam/pull/5200
> > <https://github.com/apache/beam/pull/5200>
> > [12] https://github.com/apache/beam/pull/5051
> > <https://github.com/apache/beam/pull/5051>
> > [13] https://github.com/apache/beam/pull/4740
> > <https://github.com/apache/beam/pull/4740>
> > [14] https://github.com/apache/beam/pull/4702
> > <https://github.com/apache/beam/pull/4702>
> > [15] https://github.com/apache/beam/pull/4701
> > <https://github.com/apache/beam/pull/4701>
> > [16] https://github.com/apache/beam/pull/4626
> > <https://github.com/apache/beam/pull/4626>
> > [17] https://github.com/apache/beam/pull/4625
> > <https://github.com/apache/beam/pull/4625>
> > [18] https://github.com/apache/beam/pull/5193
> > <https://github.com/apache/beam/pull/5193>
> > [19] https://github.com/apache/beam/pull/5222
> > <https://github.com/apache/beam/pull/5222>
> > [20] https://github.com/apache/beam/pull/5187
> > <https://github.com/apache/beam/pull/5187>
> > [21] https://github.com/apache/beam/pull/5217
> > <https://github.com/apache/beam/pull/5217>
> > [22] https://github.com/apache/beam/pull/5115
> > <https://github.com/apache/beam/pull/5115>
> > [23] https://github.com/apache/beam/pull/5252
> > <https://github.com/apache/beam/pull/5252>
> > [24] https://github.com/apache/beam/pull/5161
> > <https://github.com/apache/beam/pull/5161>
> > [25] https://issues.apache.org/jira/browse/BEAM-4045
> > <https://issues.apache.org/jira/browse/BEAM-4045>
> > --
> >
> >
> > Got feedback? http://go/swegner-feedback
> >
> >
>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
Re: Gradle Status: Migrated!
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
By the way, I'm curious: did someone evaluate the build time gap between Maven
and Gradle ? One of the main reason to migrate to Gradle was the inc build and
build time. The builds I have launched are quite the same in duration. I will do
deeper tests to evaluate the gap.
Regards
JB
On 05/01/2018 12:48 PM, Łukasz Gajowy wrote:
> Hi Scott,
>
> thanks for the update! Just a clarification about IO performance tests: those
> were fully migrated in Beam and all task necessary for running them are there
> but Jenkins jobs still run mvn commands. This is due the fact that
> PerfkitBenchmarker code (which is invoked by Jenkins and constructs the commands
> by itself) was not updated yet. This should be finished before fully dropping mvn.
>
> More on that topic here, in
> comments: https://issues.apache.org/jira/browse/BEAM-3942
> PR changing the commands to gradle is waiting for PerfKit devs review
> here: https://github.com/GoogleCloudPlatform/PerfKitBenchmarker/pull/1648
>
> Best regards,
>
> 2018-05-01 9:17 GMT+02:00 Romain Manni-Bucau <rmannibucau@gmail.com
> <ma...@gmail.com>>:
>
> Hi Scott
>
> While https://issues.apache.org/jira/plugins/servlet/mobile#issue/BEAM-4057
> <https://issues.apache.org/jira/plugins/servlet/mobile#issue/BEAM-4057> is
> open, gradle is a concurrent of maven but maven must stay the default build
> tool cause gradle breaks users.
>
>
> Le 1 mai 2018 01:59, "Scott Wegner" <swegner@google.com
> <ma...@google.com>> a écrit :
>
> Many many of you have been hacking diligently on the Gradle build, and
> I'm happy to announce that we now have a fully-functioning Gradle build!
> There's been a ton of progress since our last update [1]:
>
> * Improved nightly snapshot release [2]
> * Improve runner quickstarts [5] [11]
> * Python post-commit ported to Gradle [3]
> * Update performance testing framework for Gradle [4] [12]
> * Generate javadocs from Gradle [6]
> * Update to latest Gradle version [7] [21]
> * Updated documentation [8] [22]
> * Tune CI build resource usage for Jenkins [9] [19]
> * Improve shading of test jars [10] [13] [14]
> * Add 'errorprone' and 'spotless' static analysis [15] [24]
> * Improve IntelliJ project generation [16] [17]
> * Reduce number of ValidatesRunner tests [18]
> * Update release documentation for Gradle [20]
> * Update docker build scripts for Gradle [23]
>
> The build process and Jenkins environment have stabilized and we've
> resolved migration blockers. The final step is to use Gradle to produce
> an official release. The release documentation has been updated for
> Gradle and I recommend we use these docs for the 2.5.0 release. Assuming
> the release goes well, we can declare the migration fully validated and
> stop supporting dual build systems.
>
> During the migration we identified a number of opportunities to improve
> the build even further. Feel free to grab one of the items off of the
> JIRA: BEAM-4045 [24]
>
> Thanks again to all those that contributed. This has truly been a
> community effort!
>
> [1] https://lists.apache.org/thread.html/5f6bae323acc1b050962e68ec310613e0121b05bc5c42915c536fb59@%3Cdev.beam.apache.org%3E
> <https://lists.apache.org/thread.html/5f6bae323acc1b050962e68ec310613e0121b05bc5c42915c536fb59@%3Cdev.beam.apache.org%3E>
> [2] https://github.com/apache/beam/pull/5142
> <https://github.com/apache/beam/pull/5142>
> [3] https://github.com/apache/beam/pull/5146
> <https://github.com/apache/beam/pull/5146>
> [4] https://github.com/apache/beam/pull/5003
> <https://github.com/apache/beam/pull/5003>
> [5] https://github.com/apache/beam/pull/5151
> <https://github.com/apache/beam/pull/5151>
> [6] https://github.com/apache/beam/pull/5121
> <https://github.com/apache/beam/pull/5121>
> [7] https://github.com/apache/beam/pull/5104
> <https://github.com/apache/beam/pull/5104>
> [8] https://github.com/apache/beam/pull/5183
> <https://github.com/apache/beam/pull/5183>
> [9] https://github.com/apache/beam/pull/5171
> <https://github.com/apache/beam/pull/5171>
> [10] https://github.com/apache/beam/pull/5117
> <https://github.com/apache/beam/pull/5117>
> [11] https://github.com/apache/beam/pull/5200
> <https://github.com/apache/beam/pull/5200>
> [12] https://github.com/apache/beam/pull/5051
> <https://github.com/apache/beam/pull/5051>
> [13] https://github.com/apache/beam/pull/4740
> <https://github.com/apache/beam/pull/4740>
> [14] https://github.com/apache/beam/pull/4702
> <https://github.com/apache/beam/pull/4702>
> [15] https://github.com/apache/beam/pull/4701
> <https://github.com/apache/beam/pull/4701>
> [16] https://github.com/apache/beam/pull/4626
> <https://github.com/apache/beam/pull/4626>
> [17] https://github.com/apache/beam/pull/4625
> <https://github.com/apache/beam/pull/4625>
> [18] https://github.com/apache/beam/pull/5193
> <https://github.com/apache/beam/pull/5193>
> [19] https://github.com/apache/beam/pull/5222
> <https://github.com/apache/beam/pull/5222>
> [20] https://github.com/apache/beam/pull/5187
> <https://github.com/apache/beam/pull/5187>
> [21] https://github.com/apache/beam/pull/5217
> <https://github.com/apache/beam/pull/5217>
> [22] https://github.com/apache/beam/pull/5115
> <https://github.com/apache/beam/pull/5115>
> [23] https://github.com/apache/beam/pull/5252
> <https://github.com/apache/beam/pull/5252>
> [24] https://github.com/apache/beam/pull/5161
> <https://github.com/apache/beam/pull/5161>
> [25] https://issues.apache.org/jira/browse/BEAM-4045
> <https://issues.apache.org/jira/browse/BEAM-4045>
> --
>
>
> Got feedback? http://go/swegner-feedback
>
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: Gradle Status: Migrated!
Posted by Łukasz Gajowy <lu...@gmail.com>.
Hi Scott,
thanks for the update! Just a clarification about IO performance tests:
those were fully migrated in Beam and all task necessary for running them
are there but Jenkins jobs still run mvn commands. This is due the fact
that PerfkitBenchmarker code (which is invoked by Jenkins and constructs
the commands by itself) was not updated yet. This should be finished before
fully dropping mvn.
More on that topic here, in comments:
https://issues.apache.org/jira/browse/BEAM-3942
PR changing the commands to gradle is waiting for PerfKit devs review here:
https://github.com/GoogleCloudPlatform/PerfKitBenchmarker/pull/1648
Best regards,
2018-05-01 9:17 GMT+02:00 Romain Manni-Bucau <rm...@gmail.com>:
> Hi Scott
>
> While https://issues.apache.org/jira/plugins/servlet/
> mobile#issue/BEAM-4057 is open, gradle is a concurrent of maven but maven
> must stay the default build tool cause gradle breaks users.
>
>
> Le 1 mai 2018 01:59, "Scott Wegner" <sw...@google.com> a écrit :
>
>> Many many of you have been hacking diligently on the Gradle build, and
>> I'm happy to announce that we now have a fully-functioning Gradle build!
>> There's been a ton of progress since our last update [1]:
>>
>> * Improved nightly snapshot release [2]
>> * Improve runner quickstarts [5] [11]
>> * Python post-commit ported to Gradle [3]
>> * Update performance testing framework for Gradle [4] [12]
>> * Generate javadocs from Gradle [6]
>> * Update to latest Gradle version [7] [21]
>> * Updated documentation [8] [22]
>> * Tune CI build resource usage for Jenkins [9] [19]
>> * Improve shading of test jars [10] [13] [14]
>> * Add 'errorprone' and 'spotless' static analysis [15] [24]
>> * Improve IntelliJ project generation [16] [17]
>> * Reduce number of ValidatesRunner tests [18]
>> * Update release documentation for Gradle [20]
>> * Update docker build scripts for Gradle [23]
>>
>> The build process and Jenkins environment have stabilized and we've
>> resolved migration blockers. The final step is to use Gradle to produce an
>> official release. The release documentation has been updated for Gradle and
>> I recommend we use these docs for the 2.5.0 release. Assuming the release
>> goes well, we can declare the migration fully validated and stop supporting
>> dual build systems.
>>
>> During the migration we identified a number of opportunities to improve
>> the build even further. Feel free to grab one of the items off of the JIRA:
>> BEAM-4045 [24]
>>
>> Thanks again to all those that contributed. This has truly been a
>> community effort!
>>
>> [1] https://lists.apache.org/thread.html/5f6bae323acc1b05096
>> 2e68ec310613e0121b05bc5c42915c536fb59@%3Cdev.beam.apache.org%3E
>> [2] https://github.com/apache/beam/pull/5142
>> [3] https://github.com/apache/beam/pull/5146
>> [4] https://github.com/apache/beam/pull/5003
>> [5] https://github.com/apache/beam/pull/5151
>> [6] https://github.com/apache/beam/pull/5121
>> [7] https://github.com/apache/beam/pull/5104
>> [8] https://github.com/apache/beam/pull/5183
>> [9] https://github.com/apache/beam/pull/5171
>> [10] https://github.com/apache/beam/pull/5117
>> [11] https://github.com/apache/beam/pull/5200
>> [12] https://github.com/apache/beam/pull/5051
>> [13] https://github.com/apache/beam/pull/4740
>> [14] https://github.com/apache/beam/pull/4702
>> [15] https://github.com/apache/beam/pull/4701
>> [16] https://github.com/apache/beam/pull/4626
>> [17] https://github.com/apache/beam/pull/4625
>> [18] https://github.com/apache/beam/pull/5193
>> [19] https://github.com/apache/beam/pull/5222
>> [20] https://github.com/apache/beam/pull/5187
>> [21] https://github.com/apache/beam/pull/5217
>> [22] https://github.com/apache/beam/pull/5115
>> [23] https://github.com/apache/beam/pull/5252
>> [24] https://github.com/apache/beam/pull/5161
>> [25] https://issues.apache.org/jira/browse/BEAM-4045
>> --
>>
>>
>> Got feedback? http://go/swegner-feedback
>>
>
Re: Gradle Status: Migrated!
Posted by Romain Manni-Bucau <rm...@gmail.com>.
Hi Scott
While https://issues.apache.org/jira/plugins/servlet/mobile#issue/BEAM-4057
is open, gradle is a concurrent of maven but maven must stay the default
build tool cause gradle breaks users.
Le 1 mai 2018 01:59, "Scott Wegner" <sw...@google.com> a écrit :
> Many many of you have been hacking diligently on the Gradle build, and I'm
> happy to announce that we now have a fully-functioning Gradle build!
> There's been a ton of progress since our last update [1]:
>
> * Improved nightly snapshot release [2]
> * Improve runner quickstarts [5] [11]
> * Python post-commit ported to Gradle [3]
> * Update performance testing framework for Gradle [4] [12]
> * Generate javadocs from Gradle [6]
> * Update to latest Gradle version [7] [21]
> * Updated documentation [8] [22]
> * Tune CI build resource usage for Jenkins [9] [19]
> * Improve shading of test jars [10] [13] [14]
> * Add 'errorprone' and 'spotless' static analysis [15] [24]
> * Improve IntelliJ project generation [16] [17]
> * Reduce number of ValidatesRunner tests [18]
> * Update release documentation for Gradle [20]
> * Update docker build scripts for Gradle [23]
>
> The build process and Jenkins environment have stabilized and we've
> resolved migration blockers. The final step is to use Gradle to produce an
> official release. The release documentation has been updated for Gradle and
> I recommend we use these docs for the 2.5.0 release. Assuming the release
> goes well, we can declare the migration fully validated and stop supporting
> dual build systems.
>
> During the migration we identified a number of opportunities to improve
> the build even further. Feel free to grab one of the items off of the JIRA:
> BEAM-4045 [24]
>
> Thanks again to all those that contributed. This has truly been a
> community effort!
>
> [1] https://lists.apache.org/thread.html/5f6bae323acc1b050962e68ec31061
> 3e0121b05bc5c42915c536fb59@%3Cdev.beam.apache.org%3E
> [2] https://github.com/apache/beam/pull/5142
> [3] https://github.com/apache/beam/pull/5146
> [4] https://github.com/apache/beam/pull/5003
> [5] https://github.com/apache/beam/pull/5151
> [6] https://github.com/apache/beam/pull/5121
> [7] https://github.com/apache/beam/pull/5104
> [8] https://github.com/apache/beam/pull/5183
> [9] https://github.com/apache/beam/pull/5171
> [10] https://github.com/apache/beam/pull/5117
> [11] https://github.com/apache/beam/pull/5200
> [12] https://github.com/apache/beam/pull/5051
> [13] https://github.com/apache/beam/pull/4740
> [14] https://github.com/apache/beam/pull/4702
> [15] https://github.com/apache/beam/pull/4701
> [16] https://github.com/apache/beam/pull/4626
> [17] https://github.com/apache/beam/pull/4625
> [18] https://github.com/apache/beam/pull/5193
> [19] https://github.com/apache/beam/pull/5222
> [20] https://github.com/apache/beam/pull/5187
> [21] https://github.com/apache/beam/pull/5217
> [22] https://github.com/apache/beam/pull/5115
> [23] https://github.com/apache/beam/pull/5252
> [24] https://github.com/apache/beam/pull/5161
> [25] https://issues.apache.org/jira/browse/BEAM-4045
> --
>
>
> Got feedback? http://go/swegner-feedback
>