You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@netbeans.apache.org by Alessandro <al...@gmail.com> on 2020/10/26 08:43:33 UTC
Maven Daemon
Hi all,
I remember some time ago a discussion on improving Maven projects build
speed in NetBeans.
Stumbled upon this yesterday: https://github.com/mvndaemon/mvnd . It tries
to port to Maven the same concept behind the Gradle daemon.
Could an integration with the above daemon be worth pursuing to improve
Maven build speed?
Regards
*--*
*Alessandro Falappa*
Re: Maven Daemon
Posted by Jaroslav Tulach <ja...@gmail.com>.
I have always wanted to investigate usage of Maven Daemon, but it is hard to
find enough time...
>
> I don't think the PRs will conflict, but I think there's a bunch of
> stuff in Maven execution that might need some rethinking, particularly
> with Maven 4 on the horizon? mvnw is in Maven 4, and from a recent
> thread on legal@ I assume mvnd might be too?
This is the info I got from Maven guys:
> Until Olivier has finished the donation, you need to check
> https://github.com/mvndaemon/mvnd
and
> ... he is guiding the donation.
> I haven't looked at the code yet, so I don't know if it will be
> compatible with the build/consumer POM mechanism introduced in Maven 4
> in its current state. Otherwise we'll need to make it compatible in one
> of the alpha releases.
> Assuming the donation succeeds, you can expect that the Maven project
> has a daemon solution compatible with the first GA release of Maven 4.
-jt
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: Maven Daemon
Posted by Michael Bien <mb...@gmail.com>.
ah, i thought it was still related to this thread.
there is an mvnd issue to exclude some modules from running in parallel:
https://github.com/apache/maven-mvnd/issues/492
-mbien
On 10.01.22 20:02, antonio wrote:
> Building _in_ the IDE != building _the_ IDE.
>
> Tim refers to building Mavenized NetBeans platform apps, such as
> https://github.com/timboudreau/ANTLR4-Plugins-for-NetBeans/blob/master/pom.xml
>
> Cheers,
> Antonio
>
> El 10/1/22 a las 19:14, Michael Bien escribió:
>> I am not sure how this could help the NB build itself since its all
>> ANT (with some minor exceptions).
>
> [...]
>
>> On 10.01.22 18:46, Tim Boudreau wrote:
>>> [...]
>>> those, it might provide a nice speed boost to building in the IDE as
>>> well.
>>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
> For additional commands, e-mail: dev-help@netbeans.apache.org
>
> For further information about the NetBeans mailing lists, visit:
> https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: Maven Daemon
Posted by antonio <an...@vieiro.net>.
Building _in_ the IDE != building _the_ IDE.
Tim refers to building Mavenized NetBeans platform apps, such as
https://github.com/timboudreau/ANTLR4-Plugins-for-NetBeans/blob/master/pom.xml
Cheers,
Antonio
El 10/1/22 a las 19:14, Michael Bien escribió:
> I am not sure how this could help the NB build itself since its all ANT
> (with some minor exceptions).
[...]
> On 10.01.22 18:46, Tim Boudreau wrote:
>> [...]
>> those, it might provide a nice speed boost to building in the IDE as
>> well.
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: Maven Daemon
Posted by Eric Bresie <eb...@gmail.com>.
On Mon, Jan 10, 2022 at 12:15 PM Michael Bien <mb...@gmail.com> wrote:
> mvnd actually runs in parallel by default. You would have to set -T1 to
> get more or less regular maven behavior out of the daemon.
>
> I am not sure how this could help the NB build itself since its all ANT
> (with some minor exceptions).
So any future plan to migrate the Netbeans project from ant to maven?
That might help with some of the dependency handling..
Eric
>
> --
Eric Bresie
ebresie@gmail.com
Re: Maven Daemon
Posted by Michael Bien <mb...@gmail.com>.
mvnd actually runs in parallel by default. You would have to set -T1 to
get more or less regular maven behavior out of the daemon.
I am not sure how this could help the NB build itself since its all ANT
(with some minor exceptions).
-mbien
On 10.01.22 18:46, Tim Boudreau wrote:
> For multi-module builds, where it works (not everywhere), `mvn -T 8`
> (really you want `numCores * 2`) buys much faster builds. I'm not sure
> exactly how it falls down in topologically sorting the graph of what to
> build, but it occasionally does (for example, with my Antlr suite of 50+
> modules, it works well until it gets to building the test netbeans
> application project, and then tries to build that before some dependencies
> are built), but if we could detect the cases where it might fail and avoid
> those, it might provide a nice speed boost to building in the IDE as well.
>
> -Tim
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: Maven Daemon
Posted by Tim Boudreau <ni...@gmail.com>.
For multi-module builds, where it works (not everywhere), `mvn -T 8`
(really you want `numCores * 2`) buys much faster builds. I'm not sure
exactly how it falls down in topologically sorting the graph of what to
build, but it occasionally does (for example, with my Antlr suite of 50+
modules, it works well until it gets to building the test netbeans
application project, and then tries to build that before some dependencies
are built), but if we could detect the cases where it might fail and avoid
those, it might provide a nice speed boost to building in the IDE as well.
-Tim
Re: Maven Daemon
Posted by Michael Bien <mb...@gmail.com>.
merged :)
On 09.01.22 22:47, Geertjan Wielenga wrote:
> I’d say yes.
>
> Gj
>
> On Sun, 9 Jan 2022 at 22:31, Michael Bien <mb...@gmail.com> wrote:
>
>> so what shall we do with this one. The PR is basically ready since
>> October but integrating it to 12.6 felt a bit rushed.
>>
>> mvnd 0.7.1 has nicer console output which is very similar to normal
>> maven. The project also moved to apache recently.
>>
>> I just rebased to latest master and tests are still green. Should we
>> merge it?
>>
>>
>> -> https://github.com/apache/netbeans/pull/3210
>>
>> -michael
>>
>>
>>
>> On 05.10.21 14:42, Neil C Smith wrote:
>>> On Tue, 5 Oct 2021 at 13:23, Michael Bien <mb...@gmail.com> wrote:
>>>> interesting, would be nice if mvnd would make it into maven itself (as
>>>> option).
>>> ...
>>>> i don't know anything about mvnw. But judging from the name it is
>>>> probably a launcher which downloads maven before it runs it?
>>> Yes, and downloading and using the precise version specified.
>>>
>>> For current (Maven 3) wrapper see
>> https://github.com/takari/maven-wrapper
>>> For Maven 4 wrapper see
>> https://maven.apache.org/plugins/maven-wrapper-plugin/
>>> I'm intrigued how these two might interact in future. ie. use the
>>> wrapper, get the daemon automatically???
>>>
>>> Incidentally, I noticed while implementing this that we already have
>>> our own experimental support for downloading alternative Maven
>>> versions - should probably look to migrate that to the wrapper too.
>>>
>>> I'm also planning to look at a UI to configure the wrapper in a
>>> project at some point soon, assuming existing PR gets reviewed and
>>> merged.
>>>
>>> Best wishes,
>>>
>>> Neil
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
>> For additional commands, e-mail: dev-help@netbeans.apache.org
>>
>> For further information about the NetBeans mailing lists, visit:
>> https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
>>
>>
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: Maven Daemon
Posted by Geertjan Wielenga <ge...@googlemail.com.INVALID>.
I’d say yes.
Gj
On Sun, 9 Jan 2022 at 22:31, Michael Bien <mb...@gmail.com> wrote:
> so what shall we do with this one. The PR is basically ready since
> October but integrating it to 12.6 felt a bit rushed.
>
> mvnd 0.7.1 has nicer console output which is very similar to normal
> maven. The project also moved to apache recently.
>
> I just rebased to latest master and tests are still green. Should we
> merge it?
>
>
> -> https://github.com/apache/netbeans/pull/3210
>
> -michael
>
>
>
> On 05.10.21 14:42, Neil C Smith wrote:
> > On Tue, 5 Oct 2021 at 13:23, Michael Bien <mb...@gmail.com> wrote:
> >> interesting, would be nice if mvnd would make it into maven itself (as
> >> option).
> > ...
> >> i don't know anything about mvnw. But judging from the name it is
> >> probably a launcher which downloads maven before it runs it?
> > Yes, and downloading and using the precise version specified.
> >
> > For current (Maven 3) wrapper see
> https://github.com/takari/maven-wrapper
> >
> > For Maven 4 wrapper see
> https://maven.apache.org/plugins/maven-wrapper-plugin/
> >
> > I'm intrigued how these two might interact in future. ie. use the
> > wrapper, get the daemon automatically???
> >
> > Incidentally, I noticed while implementing this that we already have
> > our own experimental support for downloading alternative Maven
> > versions - should probably look to migrate that to the wrapper too.
> >
> > I'm also planning to look at a UI to configure the wrapper in a
> > project at some point soon, assuming existing PR gets reviewed and
> > merged.
> >
> > Best wishes,
> >
> > Neil
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
> For additional commands, e-mail: dev-help@netbeans.apache.org
>
> For further information about the NetBeans mailing lists, visit:
> https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
>
>
>
>
Re: Maven Daemon
Posted by Michael Bien <mb...@gmail.com>.
so what shall we do with this one. The PR is basically ready since
October but integrating it to 12.6 felt a bit rushed.
mvnd 0.7.1 has nicer console output which is very similar to normal
maven. The project also moved to apache recently.
I just rebased to latest master and tests are still green. Should we
merge it?
-> https://github.com/apache/netbeans/pull/3210
-michael
On 05.10.21 14:42, Neil C Smith wrote:
> On Tue, 5 Oct 2021 at 13:23, Michael Bien <mb...@gmail.com> wrote:
>> interesting, would be nice if mvnd would make it into maven itself (as
>> option).
> ...
>> i don't know anything about mvnw. But judging from the name it is
>> probably a launcher which downloads maven before it runs it?
> Yes, and downloading and using the precise version specified.
>
> For current (Maven 3) wrapper see https://github.com/takari/maven-wrapper
>
> For Maven 4 wrapper see https://maven.apache.org/plugins/maven-wrapper-plugin/
>
> I'm intrigued how these two might interact in future. ie. use the
> wrapper, get the daemon automatically???
>
> Incidentally, I noticed while implementing this that we already have
> our own experimental support for downloading alternative Maven
> versions - should probably look to migrate that to the wrapper too.
>
> I'm also planning to look at a UI to configure the wrapper in a
> project at some point soon, assuming existing PR gets reviewed and
> merged.
>
> Best wishes,
>
> Neil
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: Maven Daemon
Posted by Neil C Smith <ne...@apache.org>.
On Tue, 5 Oct 2021 at 13:23, Michael Bien <mb...@gmail.com> wrote:
> interesting, would be nice if mvnd would make it into maven itself (as
> option).
...
> i don't know anything about mvnw. But judging from the name it is
> probably a launcher which downloads maven before it runs it?
Yes, and downloading and using the precise version specified.
For current (Maven 3) wrapper see https://github.com/takari/maven-wrapper
For Maven 4 wrapper see https://maven.apache.org/plugins/maven-wrapper-plugin/
I'm intrigued how these two might interact in future. ie. use the
wrapper, get the daemon automatically???
Incidentally, I noticed while implementing this that we already have
our own experimental support for downloading alternative Maven
versions - should probably look to migrate that to the wrapper too.
I'm also planning to look at a UI to configure the wrapper in a
project at some point soon, assuming existing PR gets reviewed and
merged.
Best wishes,
Neil
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: Maven Daemon
Posted by Michael Bien <mb...@gmail.com>.
On 05.10.21 10:26, Neil C Smith wrote:
> On Tue, 5 Oct 2021 at 00:17, Michael Bien <mb...@gmail.com> wrote:
>> On 27.10.20 16:27, Jaroslav Tulach wrote:
>>> po 26. 10. 2020 v 9:43 odesílatel Alessandro <al...@gmail.com>
>>> napsal:
>>>
>>>> Hi all,
>>>> I remember some time ago a discussion on improving Maven projects build
>>>> speed in NetBeans.
>>>> Stumbled upon this yesterday: https://github.com/mvndaemon/mvnd . It tries
>>>> to port to Maven the same concept behind the Gradle daemon.
>>>> Could an integration with the above daemon be worth pursuing to improve
>>>> Maven build speed?
>>>>
>>> Yes, very likely it could. Noted in
>>> https://issues.apache.org/jira/browse/NETBEANS-4746 - no we just need one
>>> thing: somebody to do it ;-)
>>> -jt
>>>
>> quickly hacked something together to try it out (ignore the failed
>> builds please, overlooked a non-jdk8 method call)
>>
>> https://github.com/apache/netbeans/pull/3210
> Interesting! Last week I added a PR for Maven Wrapper support (mvnw)
> - https://github.com/apache/netbeans/pull/3198
>
> I don't think the PRs will conflict, but I think there's a bunch of
> stuff in Maven execution that might need some rethinking, particularly
> with Maven 4 on the horizon? mvnw is in Maven 4, and from a recent
> thread on legal@ I assume mvnd might be too?
interesting, would be nice if mvnd would make it into maven itself (as
option).
The code might still work if Maven 4 does not get rid of mvn and mvnd
entirely. Before changing anything in NB I symlinked a few things to
make it work without changes. Assuming mvnd will be in mvn_home/bin/ and
all jars will be still in mvn_home/lib it would just use the old code
path for the version check and still work with bin/mvnd (thats where i
had the symlinks).
It would only need a "don't use daemon" option in the UI, since it would
pick it all the time due to the fact that the homes are not separate
anymore.
i don't know anything about mvnw. But judging from the name it is
probably a launcher which downloads maven before it runs it?
-mbien
>
> Best wishes,
>
> Neil
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
> For additional commands, e-mail: dev-help@netbeans.apache.org
>
> For further information about the NetBeans mailing lists, visit:
> https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: Maven Daemon
Posted by Neil C Smith <ne...@apache.org>.
On Tue, 5 Oct 2021 at 00:17, Michael Bien <mb...@gmail.com> wrote:
>
> On 27.10.20 16:27, Jaroslav Tulach wrote:
> > po 26. 10. 2020 v 9:43 odesílatel Alessandro <al...@gmail.com>
> > napsal:
> >
> >> Hi all,
> >> I remember some time ago a discussion on improving Maven projects build
> >> speed in NetBeans.
> >> Stumbled upon this yesterday: https://github.com/mvndaemon/mvnd . It tries
> >> to port to Maven the same concept behind the Gradle daemon.
> >> Could an integration with the above daemon be worth pursuing to improve
> >> Maven build speed?
> >>
> > Yes, very likely it could. Noted in
> > https://issues.apache.org/jira/browse/NETBEANS-4746 - no we just need one
> > thing: somebody to do it ;-)
> > -jt
> >
> quickly hacked something together to try it out (ignore the failed
> builds please, overlooked a non-jdk8 method call)
>
> https://github.com/apache/netbeans/pull/3210
Interesting! Last week I added a PR for Maven Wrapper support (mvnw)
- https://github.com/apache/netbeans/pull/3198
I don't think the PRs will conflict, but I think there's a bunch of
stuff in Maven execution that might need some rethinking, particularly
with Maven 4 on the horizon? mvnw is in Maven 4, and from a recent
thread on legal@ I assume mvnd might be too?
Best wishes,
Neil
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: Maven Daemon
Posted by Michael Bien <mb...@gmail.com>.
On 27.10.20 16:27, Jaroslav Tulach wrote:
> po 26. 10. 2020 v 9:43 odesílatel Alessandro <al...@gmail.com>
> napsal:
>
>> Hi all,
>> I remember some time ago a discussion on improving Maven projects build
>> speed in NetBeans.
>> Stumbled upon this yesterday: https://github.com/mvndaemon/mvnd . It tries
>> to port to Maven the same concept behind the Gradle daemon.
>> Could an integration with the above daemon be worth pursuing to improve
>> Maven build speed?
>>
> Yes, very likely it could. Noted in
> https://issues.apache.org/jira/browse/NETBEANS-4746 - no we just need one
> thing: somebody to do it ;-)
> -jt
>
quickly hacked something together to try it out (ignore the failed
builds please, overlooked a non-jdk8 method call)
https://github.com/apache/netbeans/pull/3210
having a warmed up JVM in the background feels a bit faster for sure.
-michael
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@netbeans.apache.org
For additional commands, e-mail: dev-help@netbeans.apache.org
For further information about the NetBeans mailing lists, visit:
https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
Re: Maven Daemon
Posted by Eric Bresie <eb...@gmail.com>.
Saw this article (1) and was thinking about this past and similar maven
daemon speed threads and thought I would pass it on.
(1) https://www.infoq.com/news/2020/12/mvnd-mavens-speed-daemon/
Eric
On Tue, Oct 27, 2020 at 10:28 AM Jaroslav Tulach <ja...@gmail.com>
wrote:
> po 26. 10. 2020 v 9:43 odesílatel Alessandro <al...@gmail.com>
> napsal:
>
> > Hi all,
> > I remember some time ago a discussion on improving Maven projects build
> > speed in NetBeans.
> > Stumbled upon this yesterday: https://github.com/mvndaemon/mvnd . It
> tries
> > to port to Maven the same concept behind the Gradle daemon.
> > Could an integration with the above daemon be worth pursuing to improve
> > Maven build speed?
> >
>
> Yes, very likely it could. Noted in
> https://issues.apache.org/jira/browse/NETBEANS-4746 - no we just need one
> thing: somebody to do it ;-)
> -jt
>
--
Eric Bresie
ebresie@gmail.com
Re: Maven Daemon
Posted by Jaroslav Tulach <ja...@gmail.com>.
po 26. 10. 2020 v 9:43 odesílatel Alessandro <al...@gmail.com>
napsal:
> Hi all,
> I remember some time ago a discussion on improving Maven projects build
> speed in NetBeans.
> Stumbled upon this yesterday: https://github.com/mvndaemon/mvnd . It tries
> to port to Maven the same concept behind the Gradle daemon.
> Could an integration with the above daemon be worth pursuing to improve
> Maven build speed?
>
Yes, very likely it could. Noted in
https://issues.apache.org/jira/browse/NETBEANS-4746 - no we just need one
thing: somebody to do it ;-)
-jt