You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@camel.apache.org by Peter Palaga <pp...@redhat.com> on 2023/06/22 22:59:49 UTC
Newcomer friendly default build, faster fast build and better reproducibility
Hi,
In PR https://github.com/apache/camel/pull/10476 I tried to make some
changes to improve ergonomics and reproducibility.
The first idea is to auto-fix formatting issues instead of outputting
error messages. I.e. rather than using Checkstyle plugin that is not
able to fix even basic violations of its policies, maven-license-plugin,
maven-formatter-plugin and maven-impsort-plugin are enabled by default
to format the sources in the desired way without contributors having to
configure their IDEs or anything else manually. There are some more
details in https://issues.apache.org/jira/browse/CAMEL-19461
My second goal was to make the fast build (done from a state that passed
the CI) as fast as possible by removing the unessential Maven mojos from
the Maven execution plan altogether. We use this since longer in Camel
Quarkus and it is explained in detail in this blog:
https://peter.palaga.org/2020/10/29/skipping-maven-mojos-properly.html
The new mechanism is slightly different from how our original
`fastinstall` profile worked. `fastinstall` was disabling mojos by
defining their skip properties and it was activated simply by passing
`-Pfastinstall`. The new fast build does something different: it
disables the default `full` profile by passing a property. Because the
invocation had to change from -P to -D I dared to use the same property
name as we use in Camel Quarkus: `-Dquickly` I hope that's fine for
everybody. See also https://issues.apache.org/jira/browse/CAMEL-19466
On my machine with 32 virtual threads, invoking `mvnd clean install
-Dquickly` takes
* 1st build: 1m 21s
* 2nd build: 1m 10s
The subsequent builds are slower. I have not investigated why, it is
perhaps because of garbage collection or some plugin might be leaking
memory.
The third thing is not having the Apache Snapshots Maven repository
enabled by default for the sake of reproducibility locally and on CI.
It is still present in the `apache-snapshots` profile.
There are more details in https://issues.apache.org/jira/browse/CAMEL-19465
Any feedback is welcome.
Thanks,
-- Peter
Re: Newcomer friendly default build, faster fast build and better reproducibility
Posted by Otavio Rodolfo Piske <an...@gmail.com>.
Hi, yes we should.
Some days ago I created this task on the Camel Website, which is related to
reorganizing the contribution documentation:
https://github.com/apache/camel-website/issues/1017.
There should be some PRs this week reorganizing this.
Kind regards
On Tue, Jun 27, 2023 at 9:00 AM Claus Ibsen <cl...@gmail.com> wrote:
> Hi
>
> Thanks Peter for the quick build.
>
> I wonder if the building page should be updated (or have a section about
> quickly)
> https://camel.apache.org/manual/building.html
>
> Also if the auto formatting, source check etc are removed etc then this
> page may need to be updated
>
> On Fri, Jun 23, 2023 at 1:00 AM Peter Palaga <pp...@redhat.com> wrote:
>
> > Hi,
> >
> > In PR https://github.com/apache/camel/pull/10476 I tried to make some
> > changes to improve ergonomics and reproducibility.
> >
> > The first idea is to auto-fix formatting issues instead of outputting
> > error messages. I.e. rather than using Checkstyle plugin that is not
> > able to fix even basic violations of its policies, maven-license-plugin,
> > maven-formatter-plugin and maven-impsort-plugin are enabled by default
> > to format the sources in the desired way without contributors having to
> > configure their IDEs or anything else manually. There are some more
> > details in https://issues.apache.org/jira/browse/CAMEL-19461
> >
> > My second goal was to make the fast build (done from a state that passed
> > the CI) as fast as possible by removing the unessential Maven mojos from
> > the Maven execution plan altogether. We use this since longer in Camel
> > Quarkus and it is explained in detail in this blog:
> > https://peter.palaga.org/2020/10/29/skipping-maven-mojos-properly.html
> > The new mechanism is slightly different from how our original
> > `fastinstall` profile worked. `fastinstall` was disabling mojos by
> > defining their skip properties and it was activated simply by passing
> > `-Pfastinstall`. The new fast build does something different: it
> > disables the default `full` profile by passing a property. Because the
> > invocation had to change from -P to -D I dared to use the same property
> > name as we use in Camel Quarkus: `-Dquickly` I hope that's fine for
> > everybody. See also https://issues.apache.org/jira/browse/CAMEL-19466
> >
> > On my machine with 32 virtual threads, invoking `mvnd clean install
> > -Dquickly` takes
> >
> > * 1st build: 1m 21s
> > * 2nd build: 1m 10s
> >
> > The subsequent builds are slower. I have not investigated why, it is
> > perhaps because of garbage collection or some plugin might be leaking
> > memory.
> >
> > The third thing is not having the Apache Snapshots Maven repository
> > enabled by default for the sake of reproducibility locally and on CI.
> > It is still present in the `apache-snapshots` profile.
> > There are more details in
> > https://issues.apache.org/jira/browse/CAMEL-19465
> >
> > Any feedback is welcome.
> >
> > Thanks,
> >
> > -- Peter
> >
> >
>
> --
> Claus Ibsen
> -----------------
> @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2
>
--
Otavio R. Piske
http://orpiske.net
Re: Newcomer friendly default build, faster fast build and better reproducibility
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
Thanks Peter for the quick build.
I wonder if the building page should be updated (or have a section about
quickly)
https://camel.apache.org/manual/building.html
Also if the auto formatting, source check etc are removed etc then this
page may need to be updated
On Fri, Jun 23, 2023 at 1:00 AM Peter Palaga <pp...@redhat.com> wrote:
> Hi,
>
> In PR https://github.com/apache/camel/pull/10476 I tried to make some
> changes to improve ergonomics and reproducibility.
>
> The first idea is to auto-fix formatting issues instead of outputting
> error messages. I.e. rather than using Checkstyle plugin that is not
> able to fix even basic violations of its policies, maven-license-plugin,
> maven-formatter-plugin and maven-impsort-plugin are enabled by default
> to format the sources in the desired way without contributors having to
> configure their IDEs or anything else manually. There are some more
> details in https://issues.apache.org/jira/browse/CAMEL-19461
>
> My second goal was to make the fast build (done from a state that passed
> the CI) as fast as possible by removing the unessential Maven mojos from
> the Maven execution plan altogether. We use this since longer in Camel
> Quarkus and it is explained in detail in this blog:
> https://peter.palaga.org/2020/10/29/skipping-maven-mojos-properly.html
> The new mechanism is slightly different from how our original
> `fastinstall` profile worked. `fastinstall` was disabling mojos by
> defining their skip properties and it was activated simply by passing
> `-Pfastinstall`. The new fast build does something different: it
> disables the default `full` profile by passing a property. Because the
> invocation had to change from -P to -D I dared to use the same property
> name as we use in Camel Quarkus: `-Dquickly` I hope that's fine for
> everybody. See also https://issues.apache.org/jira/browse/CAMEL-19466
>
> On my machine with 32 virtual threads, invoking `mvnd clean install
> -Dquickly` takes
>
> * 1st build: 1m 21s
> * 2nd build: 1m 10s
>
> The subsequent builds are slower. I have not investigated why, it is
> perhaps because of garbage collection or some plugin might be leaking
> memory.
>
> The third thing is not having the Apache Snapshots Maven repository
> enabled by default for the sake of reproducibility locally and on CI.
> It is still present in the `apache-snapshots` profile.
> There are more details in
> https://issues.apache.org/jira/browse/CAMEL-19465
>
> Any feedback is welcome.
>
> Thanks,
>
> -- Peter
>
>
--
Claus Ibsen
-----------------
@davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
Re: Newcomer friendly default build, faster fast build and better reproducibility
Posted by Otavio Rodolfo Piske <an...@gmail.com>.
This patch provided a very nice improvement in terms of build speed for
Camel! I really like it.
That said, I believe we still have some areas where we could improve and
make our build even faster. I have just logged the issue CAMEL-19505 which
is related to a problem of our build compiling certain modules twice (due
to code generators the plugins being tied to a post-compilation phase).
This seems to be consuming precious seconds on the build.
https://issues.apache.org/jira/browse/CAMEL-19504
If anyone knowledgeable about Maven plugins wants to look at it, I think it
could provide another great productivity improvement for the whole
community.
Kind regards
On Fri, Jun 23, 2023 at 6:08 PM Babak Vahdat
<ba...@swissonline.ch.invalid> wrote:
> Thanks Peter!
>
> --
> Babak
>
> > Am 23.06.2023 um 16:55 schrieb Peter Palaga <pp...@redhat.com>:
> >
> > Thanks for the feedback, Alex and Babak!
> >
> > I have found a way how -Pfastinstall could be handled as an alias of
> -Dquickly: https://github.com/apache/camel/pull/10483
> > Hope that solution is good enough to meet your needs?
> >
> > Thanks again,
> >
> > -- Peter
> >
> > On 23/06/2023 14:48, Babak Vahdat wrote:
> >>>> On 23 Jun 2023, at 14:27, Alexandre Gallice <al...@gmail.com>
> wrote:
> >>>
> >>> Beyond bisect scripts, other developers could be impacted... it's a bit
> >>> like deprecating a profile.
> >>> For the sake of smooth transition, I wonder if it would be possible to
> have
> >>> something like the fastinstall profile setting -Dquickly and printing a
> >>> message that fastinstall might be possibly removed in the future.
> >>>
> >>> Alex
> >> Hi
> >> Yeah as that Maven profile has been exposed to the users as well since
> ages, so removing it (and asking them to adjust their settings.xml to
> achieve the same) doesn’t sound much “Newcomer friendly”:
> >>
> https://camel.apache.org/manual/building.html#_a_quick_build_without_running_tests
> <
> https://camel.apache.org/manual/building.html#_a_quick_build_without_running_tests
> >
> >> And thanks Peter a lot for his awesome work on his PR!
> >> --
> >> Babak
> >>>
> >>> On Fri, Jun 23, 2023 at 11:47 AM Otavio Rodolfo Piske <
> angusyoung@gmail.com>
> >>> wrote:
> >>>
> >>>> So, the primary reason I'd like to keep the fastinstall profile is
> because
> >>>> I bisect Camel often to find which commit introduced a certain bug,
> test
> >>>> failure or behavior. If we remove this profile, then I would have to
> adjust
> >>>> my bisect script so that in newer versions it uses "mvn -Dquickly"
> and in
> >>>> versions before this patch it would use "mvn -Pfastinstall" ... and I
> would
> >>>> have to, somehow, detect from when to start using one or the other (I
> think
> >>>> it would be difficult).
> >>>>
> >>>> I haven't thought about adding the profile to my user's settings.xml
> file,
> >>>> but maybe that would do the trick ... I'll give it a try.
> >>>>
> >>>> Regardless of that, the lack of fastinstall profile is not a blocker
> from
> >>>> my end. I think this is a good improvement.
> >>>>
> >>>> Kind regards
> >>>>
> >>>> On Fri, Jun 23, 2023 at 11:30 AM Peter Palaga <pp...@redhat.com>
> wrote:
> >>>>
> >>>>> On 23/06/2023 09:38, Otavio Rodolfo Piske wrote:
> >>>>>> +1 from me.
> >>>>>>
> >>>>>> All in all I think this is a good thing. Our build takes _way_ too
> >>>> long,
> >>>>> so
> >>>>>> any effort that can reduce this time is welcome. Also, I like that
> this
> >>>>>> would make the build similar to CEQ. Ideally, in the future, we
> could
> >>>>> also
> >>>>>> apply the same pattern to both CSB and CKC, so that we have an
> uniform
> >>>>>> build process across the board.
> >>>>>>
> >>>>>> That said, I have one (hopefully) simple request* in all of this:
> for
> >>>> us
> >>>>> to
> >>>>>> retain the fastinstall profile. The reason for me asking this is
> >>>> because
> >>>>> it
> >>>>>> makes my bisect scripts much easier.
> >>>>>>
> >>>>>> * Obs.: as far as I can tell, the PR does not remove the fastinstall
> >>>>>> profile, so this is more like "just to be sure".
> >>>>>
> >>>>> Thanks for the feedback, Otavio! I actually wanted to remove the
> >>>>> fastinstall profile altogether but now I see there are some
> leftovers.
> >>>>> You mean you want to keep the fastinstall profile as it was with all
> >>>>> those skip properties? I am open to that, but still, I wonder whether
> >>>>> defining fastinstall in your settings.xml would not be a good enough
> >>>>> solution too?
> >>>>>
> >>>>> -- P
> >>>>>
> >>>>>> Great work, Peter ... I'm hoping to see these changes merged.
> >>>>>>
> >>>>>> Kind regards
> >>>>>>
> >>>>>> On Fri, Jun 23, 2023 at 1:00 AM Peter Palaga <pp...@redhat.com>
> >>>> wrote:
> >>>>>>
> >>>>>>> Hi,
> >>>>>>>
> >>>>>>> In PR https://github.com/apache/camel/pull/10476 I tried to make
> some
> >>>>>>> changes to improve ergonomics and reproducibility.
> >>>>>>>
> >>>>>>> The first idea is to auto-fix formatting issues instead of
> outputting
> >>>>>>> error messages. I.e. rather than using Checkstyle plugin that is
> not
> >>>>>>> able to fix even basic violations of its policies,
> >>>> maven-license-plugin,
> >>>>>>> maven-formatter-plugin and maven-impsort-plugin are enabled by
> default
> >>>>>>> to format the sources in the desired way without contributors
> having
> >>>> to
> >>>>>>> configure their IDEs or anything else manually. There are some more
> >>>>>>> details in https://issues.apache.org/jira/browse/CAMEL-19461
> >>>>>>>
> >>>>>>> My second goal was to make the fast build (done from a state that
> >>>> passed
> >>>>>>> the CI) as fast as possible by removing the unessential Maven mojos
> >>>> from
> >>>>>>> the Maven execution plan altogether. We use this since longer in
> Camel
> >>>>>>> Quarkus and it is explained in detail in this blog:
> >>>>>>>
> >>>>
> https://peter.palaga.org/2020/10/29/skipping-maven-mojos-properly.html
> >>>>>>> The new mechanism is slightly different from how our original
> >>>>>>> `fastinstall` profile worked. `fastinstall` was disabling mojos by
> >>>>>>> defining their skip properties and it was activated simply by
> passing
> >>>>>>> `-Pfastinstall`. The new fast build does something different: it
> >>>>>>> disables the default `full` profile by passing a property. Because
> the
> >>>>>>> invocation had to change from -P to -D I dared to use the same
> >>>> property
> >>>>>>> name as we use in Camel Quarkus: `-Dquickly` I hope that's fine for
> >>>>>>> everybody. See also
> https://issues.apache.org/jira/browse/CAMEL-19466
> >>>>>>>
> >>>>>>> On my machine with 32 virtual threads, invoking `mvnd clean install
> >>>>>>> -Dquickly` takes
> >>>>>>>
> >>>>>>> * 1st build: 1m 21s
> >>>>>>> * 2nd build: 1m 10s
> >>>>>>>
> >>>>>>> The subsequent builds are slower. I have not investigated why, it
> is
> >>>>>>> perhaps because of garbage collection or some plugin might be
> leaking
> >>>>>>> memory.
> >>>>>>>
> >>>>>>> The third thing is not having the Apache Snapshots Maven repository
> >>>>>>> enabled by default for the sake of reproducibility locally and on
> CI.
> >>>>>>> It is still present in the `apache-snapshots` profile.
> >>>>>>> There are more details in
> >>>>>>> https://issues.apache.org/jira/browse/CAMEL-19465
> >>>>>>>
> >>>>>>> Any feedback is welcome.
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>>
> >>>>>>> -- Peter
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>> --
> >>>> Otavio R. Piske
> >>>> http://orpiske.net
> >>>>
> >
>
>
--
Otavio R. Piske
http://orpiske.net
Re: Newcomer friendly default build, faster fast build and better reproducibility
Posted by Babak Vahdat <ba...@swissonline.ch.INVALID>.
Thanks Peter!
--
Babak
> Am 23.06.2023 um 16:55 schrieb Peter Palaga <pp...@redhat.com>:
>
> Thanks for the feedback, Alex and Babak!
>
> I have found a way how -Pfastinstall could be handled as an alias of -Dquickly: https://github.com/apache/camel/pull/10483
> Hope that solution is good enough to meet your needs?
>
> Thanks again,
>
> -- Peter
>
> On 23/06/2023 14:48, Babak Vahdat wrote:
>>>> On 23 Jun 2023, at 14:27, Alexandre Gallice <al...@gmail.com> wrote:
>>>
>>> Beyond bisect scripts, other developers could be impacted... it's a bit
>>> like deprecating a profile.
>>> For the sake of smooth transition, I wonder if it would be possible to have
>>> something like the fastinstall profile setting -Dquickly and printing a
>>> message that fastinstall might be possibly removed in the future.
>>>
>>> Alex
>> Hi
>> Yeah as that Maven profile has been exposed to the users as well since ages, so removing it (and asking them to adjust their settings.xml to achieve the same) doesn’t sound much “Newcomer friendly”:
>> https://camel.apache.org/manual/building.html#_a_quick_build_without_running_tests <https://camel.apache.org/manual/building.html#_a_quick_build_without_running_tests>
>> And thanks Peter a lot for his awesome work on his PR!
>> --
>> Babak
>>>
>>> On Fri, Jun 23, 2023 at 11:47 AM Otavio Rodolfo Piske <an...@gmail.com>
>>> wrote:
>>>
>>>> So, the primary reason I'd like to keep the fastinstall profile is because
>>>> I bisect Camel often to find which commit introduced a certain bug, test
>>>> failure or behavior. If we remove this profile, then I would have to adjust
>>>> my bisect script so that in newer versions it uses "mvn -Dquickly" and in
>>>> versions before this patch it would use "mvn -Pfastinstall" ... and I would
>>>> have to, somehow, detect from when to start using one or the other (I think
>>>> it would be difficult).
>>>>
>>>> I haven't thought about adding the profile to my user's settings.xml file,
>>>> but maybe that would do the trick ... I'll give it a try.
>>>>
>>>> Regardless of that, the lack of fastinstall profile is not a blocker from
>>>> my end. I think this is a good improvement.
>>>>
>>>> Kind regards
>>>>
>>>> On Fri, Jun 23, 2023 at 11:30 AM Peter Palaga <pp...@redhat.com> wrote:
>>>>
>>>>> On 23/06/2023 09:38, Otavio Rodolfo Piske wrote:
>>>>>> +1 from me.
>>>>>>
>>>>>> All in all I think this is a good thing. Our build takes _way_ too
>>>> long,
>>>>> so
>>>>>> any effort that can reduce this time is welcome. Also, I like that this
>>>>>> would make the build similar to CEQ. Ideally, in the future, we could
>>>>> also
>>>>>> apply the same pattern to both CSB and CKC, so that we have an uniform
>>>>>> build process across the board.
>>>>>>
>>>>>> That said, I have one (hopefully) simple request* in all of this: for
>>>> us
>>>>> to
>>>>>> retain the fastinstall profile. The reason for me asking this is
>>>> because
>>>>> it
>>>>>> makes my bisect scripts much easier.
>>>>>>
>>>>>> * Obs.: as far as I can tell, the PR does not remove the fastinstall
>>>>>> profile, so this is more like "just to be sure".
>>>>>
>>>>> Thanks for the feedback, Otavio! I actually wanted to remove the
>>>>> fastinstall profile altogether but now I see there are some leftovers.
>>>>> You mean you want to keep the fastinstall profile as it was with all
>>>>> those skip properties? I am open to that, but still, I wonder whether
>>>>> defining fastinstall in your settings.xml would not be a good enough
>>>>> solution too?
>>>>>
>>>>> -- P
>>>>>
>>>>>> Great work, Peter ... I'm hoping to see these changes merged.
>>>>>>
>>>>>> Kind regards
>>>>>>
>>>>>> On Fri, Jun 23, 2023 at 1:00 AM Peter Palaga <pp...@redhat.com>
>>>> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> In PR https://github.com/apache/camel/pull/10476 I tried to make some
>>>>>>> changes to improve ergonomics and reproducibility.
>>>>>>>
>>>>>>> The first idea is to auto-fix formatting issues instead of outputting
>>>>>>> error messages. I.e. rather than using Checkstyle plugin that is not
>>>>>>> able to fix even basic violations of its policies,
>>>> maven-license-plugin,
>>>>>>> maven-formatter-plugin and maven-impsort-plugin are enabled by default
>>>>>>> to format the sources in the desired way without contributors having
>>>> to
>>>>>>> configure their IDEs or anything else manually. There are some more
>>>>>>> details in https://issues.apache.org/jira/browse/CAMEL-19461
>>>>>>>
>>>>>>> My second goal was to make the fast build (done from a state that
>>>> passed
>>>>>>> the CI) as fast as possible by removing the unessential Maven mojos
>>>> from
>>>>>>> the Maven execution plan altogether. We use this since longer in Camel
>>>>>>> Quarkus and it is explained in detail in this blog:
>>>>>>>
>>>> https://peter.palaga.org/2020/10/29/skipping-maven-mojos-properly.html
>>>>>>> The new mechanism is slightly different from how our original
>>>>>>> `fastinstall` profile worked. `fastinstall` was disabling mojos by
>>>>>>> defining their skip properties and it was activated simply by passing
>>>>>>> `-Pfastinstall`. The new fast build does something different: it
>>>>>>> disables the default `full` profile by passing a property. Because the
>>>>>>> invocation had to change from -P to -D I dared to use the same
>>>> property
>>>>>>> name as we use in Camel Quarkus: `-Dquickly` I hope that's fine for
>>>>>>> everybody. See also https://issues.apache.org/jira/browse/CAMEL-19466
>>>>>>>
>>>>>>> On my machine with 32 virtual threads, invoking `mvnd clean install
>>>>>>> -Dquickly` takes
>>>>>>>
>>>>>>> * 1st build: 1m 21s
>>>>>>> * 2nd build: 1m 10s
>>>>>>>
>>>>>>> The subsequent builds are slower. I have not investigated why, it is
>>>>>>> perhaps because of garbage collection or some plugin might be leaking
>>>>>>> memory.
>>>>>>>
>>>>>>> The third thing is not having the Apache Snapshots Maven repository
>>>>>>> enabled by default for the sake of reproducibility locally and on CI.
>>>>>>> It is still present in the `apache-snapshots` profile.
>>>>>>> There are more details in
>>>>>>> https://issues.apache.org/jira/browse/CAMEL-19465
>>>>>>>
>>>>>>> Any feedback is welcome.
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> -- Peter
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>> --
>>>> Otavio R. Piske
>>>> http://orpiske.net
>>>>
>
Re: Newcomer friendly default build, faster fast build and better reproducibility
Posted by Alexandre Gallice <al...@gmail.com>.
Well done Peter. There are spies even in maven those days... And here it
helps well :)
On Fri, Jun 23, 2023 at 4:55 PM Peter Palaga <pp...@redhat.com> wrote:
> Thanks for the feedback, Alex and Babak!
>
> I have found a way how -Pfastinstall could be handled as an alias of
> -Dquickly: https://github.com/apache/camel/pull/10483
> Hope that solution is good enough to meet your needs?
>
> Thanks again,
>
> -- Peter
>
> On 23/06/2023 14:48, Babak Vahdat wrote:
> >
> >
> >> On 23 Jun 2023, at 14:27, Alexandre Gallice <al...@gmail.com>
> wrote:
> >>
> >> Beyond bisect scripts, other developers could be impacted... it's a bit
> >> like deprecating a profile.
> >> For the sake of smooth transition, I wonder if it would be possible to
> have
> >> something like the fastinstall profile setting -Dquickly and printing a
> >> message that fastinstall might be possibly removed in the future.
> >>
> >> Alex
> >
> > Hi
> >
> > Yeah as that Maven profile has been exposed to the users as well since
> ages, so removing it (and asking them to adjust their settings.xml to
> achieve the same) doesn’t sound much “Newcomer friendly”:
> >
> >
> https://camel.apache.org/manual/building.html#_a_quick_build_without_running_tests
> <
> https://camel.apache.org/manual/building.html#_a_quick_build_without_running_tests
> >
> >
> > And thanks Peter a lot for his awesome work on his PR!
> >
> > --
> > Babak
> >
> >>
> >> On Fri, Jun 23, 2023 at 11:47 AM Otavio Rodolfo Piske <
> angusyoung@gmail.com>
> >> wrote:
> >>
> >>> So, the primary reason I'd like to keep the fastinstall profile is
> because
> >>> I bisect Camel often to find which commit introduced a certain bug,
> test
> >>> failure or behavior. If we remove this profile, then I would have to
> adjust
> >>> my bisect script so that in newer versions it uses "mvn -Dquickly" and
> in
> >>> versions before this patch it would use "mvn -Pfastinstall" ... and I
> would
> >>> have to, somehow, detect from when to start using one or the other (I
> think
> >>> it would be difficult).
> >>>
> >>> I haven't thought about adding the profile to my user's settings.xml
> file,
> >>> but maybe that would do the trick ... I'll give it a try.
> >>>
> >>> Regardless of that, the lack of fastinstall profile is not a blocker
> from
> >>> my end. I think this is a good improvement.
> >>>
> >>> Kind regards
> >>>
> >>> On Fri, Jun 23, 2023 at 11:30 AM Peter Palaga <pp...@redhat.com>
> wrote:
> >>>
> >>>> On 23/06/2023 09:38, Otavio Rodolfo Piske wrote:
> >>>>> +1 from me.
> >>>>>
> >>>>> All in all I think this is a good thing. Our build takes _way_ too
> >>> long,
> >>>> so
> >>>>> any effort that can reduce this time is welcome. Also, I like that
> this
> >>>>> would make the build similar to CEQ. Ideally, in the future, we could
> >>>> also
> >>>>> apply the same pattern to both CSB and CKC, so that we have an
> uniform
> >>>>> build process across the board.
> >>>>>
> >>>>> That said, I have one (hopefully) simple request* in all of this: for
> >>> us
> >>>> to
> >>>>> retain the fastinstall profile. The reason for me asking this is
> >>> because
> >>>> it
> >>>>> makes my bisect scripts much easier.
> >>>>>
> >>>>> * Obs.: as far as I can tell, the PR does not remove the fastinstall
> >>>>> profile, so this is more like "just to be sure".
> >>>>
> >>>> Thanks for the feedback, Otavio! I actually wanted to remove the
> >>>> fastinstall profile altogether but now I see there are some leftovers.
> >>>> You mean you want to keep the fastinstall profile as it was with all
> >>>> those skip properties? I am open to that, but still, I wonder whether
> >>>> defining fastinstall in your settings.xml would not be a good enough
> >>>> solution too?
> >>>>
> >>>> -- P
> >>>>
> >>>>> Great work, Peter ... I'm hoping to see these changes merged.
> >>>>>
> >>>>> Kind regards
> >>>>>
> >>>>> On Fri, Jun 23, 2023 at 1:00 AM Peter Palaga <pp...@redhat.com>
> >>> wrote:
> >>>>>
> >>>>>> Hi,
> >>>>>>
> >>>>>> In PR https://github.com/apache/camel/pull/10476 I tried to make
> some
> >>>>>> changes to improve ergonomics and reproducibility.
> >>>>>>
> >>>>>> The first idea is to auto-fix formatting issues instead of
> outputting
> >>>>>> error messages. I.e. rather than using Checkstyle plugin that is not
> >>>>>> able to fix even basic violations of its policies,
> >>> maven-license-plugin,
> >>>>>> maven-formatter-plugin and maven-impsort-plugin are enabled by
> default
> >>>>>> to format the sources in the desired way without contributors having
> >>> to
> >>>>>> configure their IDEs or anything else manually. There are some more
> >>>>>> details in https://issues.apache.org/jira/browse/CAMEL-19461
> >>>>>>
> >>>>>> My second goal was to make the fast build (done from a state that
> >>> passed
> >>>>>> the CI) as fast as possible by removing the unessential Maven mojos
> >>> from
> >>>>>> the Maven execution plan altogether. We use this since longer in
> Camel
> >>>>>> Quarkus and it is explained in detail in this blog:
> >>>>>>
> >>> https://peter.palaga.org/2020/10/29/skipping-maven-mojos-properly.html
> >>>>>> The new mechanism is slightly different from how our original
> >>>>>> `fastinstall` profile worked. `fastinstall` was disabling mojos by
> >>>>>> defining their skip properties and it was activated simply by
> passing
> >>>>>> `-Pfastinstall`. The new fast build does something different: it
> >>>>>> disables the default `full` profile by passing a property. Because
> the
> >>>>>> invocation had to change from -P to -D I dared to use the same
> >>> property
> >>>>>> name as we use in Camel Quarkus: `-Dquickly` I hope that's fine for
> >>>>>> everybody. See also
> https://issues.apache.org/jira/browse/CAMEL-19466
> >>>>>>
> >>>>>> On my machine with 32 virtual threads, invoking `mvnd clean install
> >>>>>> -Dquickly` takes
> >>>>>>
> >>>>>> * 1st build: 1m 21s
> >>>>>> * 2nd build: 1m 10s
> >>>>>>
> >>>>>> The subsequent builds are slower. I have not investigated why, it is
> >>>>>> perhaps because of garbage collection or some plugin might be
> leaking
> >>>>>> memory.
> >>>>>>
> >>>>>> The third thing is not having the Apache Snapshots Maven repository
> >>>>>> enabled by default for the sake of reproducibility locally and on
> CI.
> >>>>>> It is still present in the `apache-snapshots` profile.
> >>>>>> There are more details in
> >>>>>> https://issues.apache.org/jira/browse/CAMEL-19465
> >>>>>>
> >>>>>> Any feedback is welcome.
> >>>>>>
> >>>>>> Thanks,
> >>>>>>
> >>>>>> -- Peter
> >>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>>
> >>>
> >>> --
> >>> Otavio R. Piske
> >>> http://orpiske.net
> >>>
> >
> >
>
>
Re: Newcomer friendly default build, faster fast build and better reproducibility
Posted by Peter Palaga <pp...@redhat.com>.
Thanks for the feedback, Alex and Babak!
I have found a way how -Pfastinstall could be handled as an alias of
-Dquickly: https://github.com/apache/camel/pull/10483
Hope that solution is good enough to meet your needs?
Thanks again,
-- Peter
On 23/06/2023 14:48, Babak Vahdat wrote:
>
>
>> On 23 Jun 2023, at 14:27, Alexandre Gallice <al...@gmail.com> wrote:
>>
>> Beyond bisect scripts, other developers could be impacted... it's a bit
>> like deprecating a profile.
>> For the sake of smooth transition, I wonder if it would be possible to have
>> something like the fastinstall profile setting -Dquickly and printing a
>> message that fastinstall might be possibly removed in the future.
>>
>> Alex
>
> Hi
>
> Yeah as that Maven profile has been exposed to the users as well since ages, so removing it (and asking them to adjust their settings.xml to achieve the same) doesn’t sound much “Newcomer friendly”:
>
> https://camel.apache.org/manual/building.html#_a_quick_build_without_running_tests <https://camel.apache.org/manual/building.html#_a_quick_build_without_running_tests>
>
> And thanks Peter a lot for his awesome work on his PR!
>
> --
> Babak
>
>>
>> On Fri, Jun 23, 2023 at 11:47 AM Otavio Rodolfo Piske <an...@gmail.com>
>> wrote:
>>
>>> So, the primary reason I'd like to keep the fastinstall profile is because
>>> I bisect Camel often to find which commit introduced a certain bug, test
>>> failure or behavior. If we remove this profile, then I would have to adjust
>>> my bisect script so that in newer versions it uses "mvn -Dquickly" and in
>>> versions before this patch it would use "mvn -Pfastinstall" ... and I would
>>> have to, somehow, detect from when to start using one or the other (I think
>>> it would be difficult).
>>>
>>> I haven't thought about adding the profile to my user's settings.xml file,
>>> but maybe that would do the trick ... I'll give it a try.
>>>
>>> Regardless of that, the lack of fastinstall profile is not a blocker from
>>> my end. I think this is a good improvement.
>>>
>>> Kind regards
>>>
>>> On Fri, Jun 23, 2023 at 11:30 AM Peter Palaga <pp...@redhat.com> wrote:
>>>
>>>> On 23/06/2023 09:38, Otavio Rodolfo Piske wrote:
>>>>> +1 from me.
>>>>>
>>>>> All in all I think this is a good thing. Our build takes _way_ too
>>> long,
>>>> so
>>>>> any effort that can reduce this time is welcome. Also, I like that this
>>>>> would make the build similar to CEQ. Ideally, in the future, we could
>>>> also
>>>>> apply the same pattern to both CSB and CKC, so that we have an uniform
>>>>> build process across the board.
>>>>>
>>>>> That said, I have one (hopefully) simple request* in all of this: for
>>> us
>>>> to
>>>>> retain the fastinstall profile. The reason for me asking this is
>>> because
>>>> it
>>>>> makes my bisect scripts much easier.
>>>>>
>>>>> * Obs.: as far as I can tell, the PR does not remove the fastinstall
>>>>> profile, so this is more like "just to be sure".
>>>>
>>>> Thanks for the feedback, Otavio! I actually wanted to remove the
>>>> fastinstall profile altogether but now I see there are some leftovers.
>>>> You mean you want to keep the fastinstall profile as it was with all
>>>> those skip properties? I am open to that, but still, I wonder whether
>>>> defining fastinstall in your settings.xml would not be a good enough
>>>> solution too?
>>>>
>>>> -- P
>>>>
>>>>> Great work, Peter ... I'm hoping to see these changes merged.
>>>>>
>>>>> Kind regards
>>>>>
>>>>> On Fri, Jun 23, 2023 at 1:00 AM Peter Palaga <pp...@redhat.com>
>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> In PR https://github.com/apache/camel/pull/10476 I tried to make some
>>>>>> changes to improve ergonomics and reproducibility.
>>>>>>
>>>>>> The first idea is to auto-fix formatting issues instead of outputting
>>>>>> error messages. I.e. rather than using Checkstyle plugin that is not
>>>>>> able to fix even basic violations of its policies,
>>> maven-license-plugin,
>>>>>> maven-formatter-plugin and maven-impsort-plugin are enabled by default
>>>>>> to format the sources in the desired way without contributors having
>>> to
>>>>>> configure their IDEs or anything else manually. There are some more
>>>>>> details in https://issues.apache.org/jira/browse/CAMEL-19461
>>>>>>
>>>>>> My second goal was to make the fast build (done from a state that
>>> passed
>>>>>> the CI) as fast as possible by removing the unessential Maven mojos
>>> from
>>>>>> the Maven execution plan altogether. We use this since longer in Camel
>>>>>> Quarkus and it is explained in detail in this blog:
>>>>>>
>>> https://peter.palaga.org/2020/10/29/skipping-maven-mojos-properly.html
>>>>>> The new mechanism is slightly different from how our original
>>>>>> `fastinstall` profile worked. `fastinstall` was disabling mojos by
>>>>>> defining their skip properties and it was activated simply by passing
>>>>>> `-Pfastinstall`. The new fast build does something different: it
>>>>>> disables the default `full` profile by passing a property. Because the
>>>>>> invocation had to change from -P to -D I dared to use the same
>>> property
>>>>>> name as we use in Camel Quarkus: `-Dquickly` I hope that's fine for
>>>>>> everybody. See also https://issues.apache.org/jira/browse/CAMEL-19466
>>>>>>
>>>>>> On my machine with 32 virtual threads, invoking `mvnd clean install
>>>>>> -Dquickly` takes
>>>>>>
>>>>>> * 1st build: 1m 21s
>>>>>> * 2nd build: 1m 10s
>>>>>>
>>>>>> The subsequent builds are slower. I have not investigated why, it is
>>>>>> perhaps because of garbage collection or some plugin might be leaking
>>>>>> memory.
>>>>>>
>>>>>> The third thing is not having the Apache Snapshots Maven repository
>>>>>> enabled by default for the sake of reproducibility locally and on CI.
>>>>>> It is still present in the `apache-snapshots` profile.
>>>>>> There are more details in
>>>>>> https://issues.apache.org/jira/browse/CAMEL-19465
>>>>>>
>>>>>> Any feedback is welcome.
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> -- Peter
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>
>>> --
>>> Otavio R. Piske
>>> http://orpiske.net
>>>
>
>
Re: Newcomer friendly default build, faster fast build and better reproducibility
Posted by Babak Vahdat <ba...@swissonline.ch.INVALID>.
> On 23 Jun 2023, at 14:27, Alexandre Gallice <al...@gmail.com> wrote:
>
> Beyond bisect scripts, other developers could be impacted... it's a bit
> like deprecating a profile.
> For the sake of smooth transition, I wonder if it would be possible to have
> something like the fastinstall profile setting -Dquickly and printing a
> message that fastinstall might be possibly removed in the future.
>
> Alex
Hi
Yeah as that Maven profile has been exposed to the users as well since ages, so removing it (and asking them to adjust their settings.xml to achieve the same) doesn’t sound much “Newcomer friendly”:
https://camel.apache.org/manual/building.html#_a_quick_build_without_running_tests <https://camel.apache.org/manual/building.html#_a_quick_build_without_running_tests>
And thanks Peter a lot for his awesome work on his PR!
--
Babak
>
> On Fri, Jun 23, 2023 at 11:47 AM Otavio Rodolfo Piske <an...@gmail.com>
> wrote:
>
>> So, the primary reason I'd like to keep the fastinstall profile is because
>> I bisect Camel often to find which commit introduced a certain bug, test
>> failure or behavior. If we remove this profile, then I would have to adjust
>> my bisect script so that in newer versions it uses "mvn -Dquickly" and in
>> versions before this patch it would use "mvn -Pfastinstall" ... and I would
>> have to, somehow, detect from when to start using one or the other (I think
>> it would be difficult).
>>
>> I haven't thought about adding the profile to my user's settings.xml file,
>> but maybe that would do the trick ... I'll give it a try.
>>
>> Regardless of that, the lack of fastinstall profile is not a blocker from
>> my end. I think this is a good improvement.
>>
>> Kind regards
>>
>> On Fri, Jun 23, 2023 at 11:30 AM Peter Palaga <pp...@redhat.com> wrote:
>>
>>> On 23/06/2023 09:38, Otavio Rodolfo Piske wrote:
>>>> +1 from me.
>>>>
>>>> All in all I think this is a good thing. Our build takes _way_ too
>> long,
>>> so
>>>> any effort that can reduce this time is welcome. Also, I like that this
>>>> would make the build similar to CEQ. Ideally, in the future, we could
>>> also
>>>> apply the same pattern to both CSB and CKC, so that we have an uniform
>>>> build process across the board.
>>>>
>>>> That said, I have one (hopefully) simple request* in all of this: for
>> us
>>> to
>>>> retain the fastinstall profile. The reason for me asking this is
>> because
>>> it
>>>> makes my bisect scripts much easier.
>>>>
>>>> * Obs.: as far as I can tell, the PR does not remove the fastinstall
>>>> profile, so this is more like "just to be sure".
>>>
>>> Thanks for the feedback, Otavio! I actually wanted to remove the
>>> fastinstall profile altogether but now I see there are some leftovers.
>>> You mean you want to keep the fastinstall profile as it was with all
>>> those skip properties? I am open to that, but still, I wonder whether
>>> defining fastinstall in your settings.xml would not be a good enough
>>> solution too?
>>>
>>> -- P
>>>
>>>> Great work, Peter ... I'm hoping to see these changes merged.
>>>>
>>>> Kind regards
>>>>
>>>> On Fri, Jun 23, 2023 at 1:00 AM Peter Palaga <pp...@redhat.com>
>> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> In PR https://github.com/apache/camel/pull/10476 I tried to make some
>>>>> changes to improve ergonomics and reproducibility.
>>>>>
>>>>> The first idea is to auto-fix formatting issues instead of outputting
>>>>> error messages. I.e. rather than using Checkstyle plugin that is not
>>>>> able to fix even basic violations of its policies,
>> maven-license-plugin,
>>>>> maven-formatter-plugin and maven-impsort-plugin are enabled by default
>>>>> to format the sources in the desired way without contributors having
>> to
>>>>> configure their IDEs or anything else manually. There are some more
>>>>> details in https://issues.apache.org/jira/browse/CAMEL-19461
>>>>>
>>>>> My second goal was to make the fast build (done from a state that
>> passed
>>>>> the CI) as fast as possible by removing the unessential Maven mojos
>> from
>>>>> the Maven execution plan altogether. We use this since longer in Camel
>>>>> Quarkus and it is explained in detail in this blog:
>>>>>
>> https://peter.palaga.org/2020/10/29/skipping-maven-mojos-properly.html
>>>>> The new mechanism is slightly different from how our original
>>>>> `fastinstall` profile worked. `fastinstall` was disabling mojos by
>>>>> defining their skip properties and it was activated simply by passing
>>>>> `-Pfastinstall`. The new fast build does something different: it
>>>>> disables the default `full` profile by passing a property. Because the
>>>>> invocation had to change from -P to -D I dared to use the same
>> property
>>>>> name as we use in Camel Quarkus: `-Dquickly` I hope that's fine for
>>>>> everybody. See also https://issues.apache.org/jira/browse/CAMEL-19466
>>>>>
>>>>> On my machine with 32 virtual threads, invoking `mvnd clean install
>>>>> -Dquickly` takes
>>>>>
>>>>> * 1st build: 1m 21s
>>>>> * 2nd build: 1m 10s
>>>>>
>>>>> The subsequent builds are slower. I have not investigated why, it is
>>>>> perhaps because of garbage collection or some plugin might be leaking
>>>>> memory.
>>>>>
>>>>> The third thing is not having the Apache Snapshots Maven repository
>>>>> enabled by default for the sake of reproducibility locally and on CI.
>>>>> It is still present in the `apache-snapshots` profile.
>>>>> There are more details in
>>>>> https://issues.apache.org/jira/browse/CAMEL-19465
>>>>>
>>>>> Any feedback is welcome.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> -- Peter
>>>>>
>>>>>
>>>>
>>>
>>>
>>
>> --
>> Otavio R. Piske
>> http://orpiske.net
>>
Re: Newcomer friendly default build, faster fast build and better reproducibility
Posted by Alexandre Gallice <al...@gmail.com>.
Beyond bisect scripts, other developers could be impacted... it's a bit
like deprecating a profile.
For the sake of smooth transition, I wonder if it would be possible to have
something like the fastinstall profile setting -Dquickly and printing a
message that fastinstall might be possibly removed in the future.
Alex
On Fri, Jun 23, 2023 at 11:47 AM Otavio Rodolfo Piske <an...@gmail.com>
wrote:
> So, the primary reason I'd like to keep the fastinstall profile is because
> I bisect Camel often to find which commit introduced a certain bug, test
> failure or behavior. If we remove this profile, then I would have to adjust
> my bisect script so that in newer versions it uses "mvn -Dquickly" and in
> versions before this patch it would use "mvn -Pfastinstall" ... and I would
> have to, somehow, detect from when to start using one or the other (I think
> it would be difficult).
>
> I haven't thought about adding the profile to my user's settings.xml file,
> but maybe that would do the trick ... I'll give it a try.
>
> Regardless of that, the lack of fastinstall profile is not a blocker from
> my end. I think this is a good improvement.
>
> Kind regards
>
> On Fri, Jun 23, 2023 at 11:30 AM Peter Palaga <pp...@redhat.com> wrote:
>
> > On 23/06/2023 09:38, Otavio Rodolfo Piske wrote:
> > > +1 from me.
> > >
> > > All in all I think this is a good thing. Our build takes _way_ too
> long,
> > so
> > > any effort that can reduce this time is welcome. Also, I like that this
> > > would make the build similar to CEQ. Ideally, in the future, we could
> > also
> > > apply the same pattern to both CSB and CKC, so that we have an uniform
> > > build process across the board.
> > >
> > > That said, I have one (hopefully) simple request* in all of this: for
> us
> > to
> > > retain the fastinstall profile. The reason for me asking this is
> because
> > it
> > > makes my bisect scripts much easier.
> > >
> > > * Obs.: as far as I can tell, the PR does not remove the fastinstall
> > > profile, so this is more like "just to be sure".
> >
> > Thanks for the feedback, Otavio! I actually wanted to remove the
> > fastinstall profile altogether but now I see there are some leftovers.
> > You mean you want to keep the fastinstall profile as it was with all
> > those skip properties? I am open to that, but still, I wonder whether
> > defining fastinstall in your settings.xml would not be a good enough
> > solution too?
> >
> > -- P
> >
> > > Great work, Peter ... I'm hoping to see these changes merged.
> > >
> > > Kind regards
> > >
> > > On Fri, Jun 23, 2023 at 1:00 AM Peter Palaga <pp...@redhat.com>
> wrote:
> > >
> > >> Hi,
> > >>
> > >> In PR https://github.com/apache/camel/pull/10476 I tried to make some
> > >> changes to improve ergonomics and reproducibility.
> > >>
> > >> The first idea is to auto-fix formatting issues instead of outputting
> > >> error messages. I.e. rather than using Checkstyle plugin that is not
> > >> able to fix even basic violations of its policies,
> maven-license-plugin,
> > >> maven-formatter-plugin and maven-impsort-plugin are enabled by default
> > >> to format the sources in the desired way without contributors having
> to
> > >> configure their IDEs or anything else manually. There are some more
> > >> details in https://issues.apache.org/jira/browse/CAMEL-19461
> > >>
> > >> My second goal was to make the fast build (done from a state that
> passed
> > >> the CI) as fast as possible by removing the unessential Maven mojos
> from
> > >> the Maven execution plan altogether. We use this since longer in Camel
> > >> Quarkus and it is explained in detail in this blog:
> > >>
> https://peter.palaga.org/2020/10/29/skipping-maven-mojos-properly.html
> > >> The new mechanism is slightly different from how our original
> > >> `fastinstall` profile worked. `fastinstall` was disabling mojos by
> > >> defining their skip properties and it was activated simply by passing
> > >> `-Pfastinstall`. The new fast build does something different: it
> > >> disables the default `full` profile by passing a property. Because the
> > >> invocation had to change from -P to -D I dared to use the same
> property
> > >> name as we use in Camel Quarkus: `-Dquickly` I hope that's fine for
> > >> everybody. See also https://issues.apache.org/jira/browse/CAMEL-19466
> > >>
> > >> On my machine with 32 virtual threads, invoking `mvnd clean install
> > >> -Dquickly` takes
> > >>
> > >> * 1st build: 1m 21s
> > >> * 2nd build: 1m 10s
> > >>
> > >> The subsequent builds are slower. I have not investigated why, it is
> > >> perhaps because of garbage collection or some plugin might be leaking
> > >> memory.
> > >>
> > >> The third thing is not having the Apache Snapshots Maven repository
> > >> enabled by default for the sake of reproducibility locally and on CI.
> > >> It is still present in the `apache-snapshots` profile.
> > >> There are more details in
> > >> https://issues.apache.org/jira/browse/CAMEL-19465
> > >>
> > >> Any feedback is welcome.
> > >>
> > >> Thanks,
> > >>
> > >> -- Peter
> > >>
> > >>
> > >
> >
> >
>
> --
> Otavio R. Piske
> http://orpiske.net
>
Re: Newcomer friendly default build, faster fast build and better reproducibility
Posted by Otavio Rodolfo Piske <an...@gmail.com>.
So, the primary reason I'd like to keep the fastinstall profile is because
I bisect Camel often to find which commit introduced a certain bug, test
failure or behavior. If we remove this profile, then I would have to adjust
my bisect script so that in newer versions it uses "mvn -Dquickly" and in
versions before this patch it would use "mvn -Pfastinstall" ... and I would
have to, somehow, detect from when to start using one or the other (I think
it would be difficult).
I haven't thought about adding the profile to my user's settings.xml file,
but maybe that would do the trick ... I'll give it a try.
Regardless of that, the lack of fastinstall profile is not a blocker from
my end. I think this is a good improvement.
Kind regards
On Fri, Jun 23, 2023 at 11:30 AM Peter Palaga <pp...@redhat.com> wrote:
> On 23/06/2023 09:38, Otavio Rodolfo Piske wrote:
> > +1 from me.
> >
> > All in all I think this is a good thing. Our build takes _way_ too long,
> so
> > any effort that can reduce this time is welcome. Also, I like that this
> > would make the build similar to CEQ. Ideally, in the future, we could
> also
> > apply the same pattern to both CSB and CKC, so that we have an uniform
> > build process across the board.
> >
> > That said, I have one (hopefully) simple request* in all of this: for us
> to
> > retain the fastinstall profile. The reason for me asking this is because
> it
> > makes my bisect scripts much easier.
> >
> > * Obs.: as far as I can tell, the PR does not remove the fastinstall
> > profile, so this is more like "just to be sure".
>
> Thanks for the feedback, Otavio! I actually wanted to remove the
> fastinstall profile altogether but now I see there are some leftovers.
> You mean you want to keep the fastinstall profile as it was with all
> those skip properties? I am open to that, but still, I wonder whether
> defining fastinstall in your settings.xml would not be a good enough
> solution too?
>
> -- P
>
> > Great work, Peter ... I'm hoping to see these changes merged.
> >
> > Kind regards
> >
> > On Fri, Jun 23, 2023 at 1:00 AM Peter Palaga <pp...@redhat.com> wrote:
> >
> >> Hi,
> >>
> >> In PR https://github.com/apache/camel/pull/10476 I tried to make some
> >> changes to improve ergonomics and reproducibility.
> >>
> >> The first idea is to auto-fix formatting issues instead of outputting
> >> error messages. I.e. rather than using Checkstyle plugin that is not
> >> able to fix even basic violations of its policies, maven-license-plugin,
> >> maven-formatter-plugin and maven-impsort-plugin are enabled by default
> >> to format the sources in the desired way without contributors having to
> >> configure their IDEs or anything else manually. There are some more
> >> details in https://issues.apache.org/jira/browse/CAMEL-19461
> >>
> >> My second goal was to make the fast build (done from a state that passed
> >> the CI) as fast as possible by removing the unessential Maven mojos from
> >> the Maven execution plan altogether. We use this since longer in Camel
> >> Quarkus and it is explained in detail in this blog:
> >> https://peter.palaga.org/2020/10/29/skipping-maven-mojos-properly.html
> >> The new mechanism is slightly different from how our original
> >> `fastinstall` profile worked. `fastinstall` was disabling mojos by
> >> defining their skip properties and it was activated simply by passing
> >> `-Pfastinstall`. The new fast build does something different: it
> >> disables the default `full` profile by passing a property. Because the
> >> invocation had to change from -P to -D I dared to use the same property
> >> name as we use in Camel Quarkus: `-Dquickly` I hope that's fine for
> >> everybody. See also https://issues.apache.org/jira/browse/CAMEL-19466
> >>
> >> On my machine with 32 virtual threads, invoking `mvnd clean install
> >> -Dquickly` takes
> >>
> >> * 1st build: 1m 21s
> >> * 2nd build: 1m 10s
> >>
> >> The subsequent builds are slower. I have not investigated why, it is
> >> perhaps because of garbage collection or some plugin might be leaking
> >> memory.
> >>
> >> The third thing is not having the Apache Snapshots Maven repository
> >> enabled by default for the sake of reproducibility locally and on CI.
> >> It is still present in the `apache-snapshots` profile.
> >> There are more details in
> >> https://issues.apache.org/jira/browse/CAMEL-19465
> >>
> >> Any feedback is welcome.
> >>
> >> Thanks,
> >>
> >> -- Peter
> >>
> >>
> >
>
>
--
Otavio R. Piske
http://orpiske.net
Re: Newcomer friendly default build, faster fast build and better reproducibility
Posted by Peter Palaga <pp...@redhat.com>.
On 23/06/2023 09:38, Otavio Rodolfo Piske wrote:
> +1 from me.
>
> All in all I think this is a good thing. Our build takes _way_ too long, so
> any effort that can reduce this time is welcome. Also, I like that this
> would make the build similar to CEQ. Ideally, in the future, we could also
> apply the same pattern to both CSB and CKC, so that we have an uniform
> build process across the board.
>
> That said, I have one (hopefully) simple request* in all of this: for us to
> retain the fastinstall profile. The reason for me asking this is because it
> makes my bisect scripts much easier.
>
> * Obs.: as far as I can tell, the PR does not remove the fastinstall
> profile, so this is more like "just to be sure".
Thanks for the feedback, Otavio! I actually wanted to remove the
fastinstall profile altogether but now I see there are some leftovers.
You mean you want to keep the fastinstall profile as it was with all
those skip properties? I am open to that, but still, I wonder whether
defining fastinstall in your settings.xml would not be a good enough
solution too?
-- P
> Great work, Peter ... I'm hoping to see these changes merged.
>
> Kind regards
>
> On Fri, Jun 23, 2023 at 1:00 AM Peter Palaga <pp...@redhat.com> wrote:
>
>> Hi,
>>
>> In PR https://github.com/apache/camel/pull/10476 I tried to make some
>> changes to improve ergonomics and reproducibility.
>>
>> The first idea is to auto-fix formatting issues instead of outputting
>> error messages. I.e. rather than using Checkstyle plugin that is not
>> able to fix even basic violations of its policies, maven-license-plugin,
>> maven-formatter-plugin and maven-impsort-plugin are enabled by default
>> to format the sources in the desired way without contributors having to
>> configure their IDEs or anything else manually. There are some more
>> details in https://issues.apache.org/jira/browse/CAMEL-19461
>>
>> My second goal was to make the fast build (done from a state that passed
>> the CI) as fast as possible by removing the unessential Maven mojos from
>> the Maven execution plan altogether. We use this since longer in Camel
>> Quarkus and it is explained in detail in this blog:
>> https://peter.palaga.org/2020/10/29/skipping-maven-mojos-properly.html
>> The new mechanism is slightly different from how our original
>> `fastinstall` profile worked. `fastinstall` was disabling mojos by
>> defining their skip properties and it was activated simply by passing
>> `-Pfastinstall`. The new fast build does something different: it
>> disables the default `full` profile by passing a property. Because the
>> invocation had to change from -P to -D I dared to use the same property
>> name as we use in Camel Quarkus: `-Dquickly` I hope that's fine for
>> everybody. See also https://issues.apache.org/jira/browse/CAMEL-19466
>>
>> On my machine with 32 virtual threads, invoking `mvnd clean install
>> -Dquickly` takes
>>
>> * 1st build: 1m 21s
>> * 2nd build: 1m 10s
>>
>> The subsequent builds are slower. I have not investigated why, it is
>> perhaps because of garbage collection or some plugin might be leaking
>> memory.
>>
>> The third thing is not having the Apache Snapshots Maven repository
>> enabled by default for the sake of reproducibility locally and on CI.
>> It is still present in the `apache-snapshots` profile.
>> There are more details in
>> https://issues.apache.org/jira/browse/CAMEL-19465
>>
>> Any feedback is welcome.
>>
>> Thanks,
>>
>> -- Peter
>>
>>
>
Re: Newcomer friendly default build, faster fast build and better reproducibility
Posted by Otavio Rodolfo Piske <an...@gmail.com>.
+1 from me.
All in all I think this is a good thing. Our build takes _way_ too long, so
any effort that can reduce this time is welcome. Also, I like that this
would make the build similar to CEQ. Ideally, in the future, we could also
apply the same pattern to both CSB and CKC, so that we have an uniform
build process across the board.
That said, I have one (hopefully) simple request* in all of this: for us to
retain the fastinstall profile. The reason for me asking this is because it
makes my bisect scripts much easier.
* Obs.: as far as I can tell, the PR does not remove the fastinstall
profile, so this is more like "just to be sure".
Great work, Peter ... I'm hoping to see these changes merged.
Kind regards
On Fri, Jun 23, 2023 at 1:00 AM Peter Palaga <pp...@redhat.com> wrote:
> Hi,
>
> In PR https://github.com/apache/camel/pull/10476 I tried to make some
> changes to improve ergonomics and reproducibility.
>
> The first idea is to auto-fix formatting issues instead of outputting
> error messages. I.e. rather than using Checkstyle plugin that is not
> able to fix even basic violations of its policies, maven-license-plugin,
> maven-formatter-plugin and maven-impsort-plugin are enabled by default
> to format the sources in the desired way without contributors having to
> configure their IDEs or anything else manually. There are some more
> details in https://issues.apache.org/jira/browse/CAMEL-19461
>
> My second goal was to make the fast build (done from a state that passed
> the CI) as fast as possible by removing the unessential Maven mojos from
> the Maven execution plan altogether. We use this since longer in Camel
> Quarkus and it is explained in detail in this blog:
> https://peter.palaga.org/2020/10/29/skipping-maven-mojos-properly.html
> The new mechanism is slightly different from how our original
> `fastinstall` profile worked. `fastinstall` was disabling mojos by
> defining their skip properties and it was activated simply by passing
> `-Pfastinstall`. The new fast build does something different: it
> disables the default `full` profile by passing a property. Because the
> invocation had to change from -P to -D I dared to use the same property
> name as we use in Camel Quarkus: `-Dquickly` I hope that's fine for
> everybody. See also https://issues.apache.org/jira/browse/CAMEL-19466
>
> On my machine with 32 virtual threads, invoking `mvnd clean install
> -Dquickly` takes
>
> * 1st build: 1m 21s
> * 2nd build: 1m 10s
>
> The subsequent builds are slower. I have not investigated why, it is
> perhaps because of garbage collection or some plugin might be leaking
> memory.
>
> The third thing is not having the Apache Snapshots Maven repository
> enabled by default for the sake of reproducibility locally and on CI.
> It is still present in the `apache-snapshots` profile.
> There are more details in
> https://issues.apache.org/jira/browse/CAMEL-19465
>
> Any feedback is welcome.
>
> Thanks,
>
> -- Peter
>
>
--
Otavio R. Piske
http://orpiske.net