Gradle cache utilization & build speedup

[Arnold Galovics <ar...@apache.org>]

Hi everyone,

I think all of us are annoyed sometimes with the time it takes to build
fineract especially if you have integration tests that need to be debugged
and fixed.
A single integration test run (without starting up the fineract server)
involved rebuilding the fineract-client module and regenerating all the
clients with it, then rebuilding the integration-tests module which took a
lot of time; and this was true even though you haven't changed anything.
The reason was that Gradle's incremental build is not utilized properly and
the up-to-date checks were broken.

Yesterday I managed to fix a lot of these issues, hence the fineract-client
will not get regenerated and the integration-tests will not be recompiled
every time you try to run it. The changes are already available on develop.

My feeling is that these changes mean around an 80-85% decrease in the time
needed to run integration tests from IntelliJ, but I don't have exact
measurements.

If you encounter issues with the recent changes, let me know.

Best,
Arnold

Re: Gradle cache utilization & build speedup

[Aleksandar Vidakovic <ch...@monkeysintown.com>]

... maybe that brings down the time for my Testcontainers executions...

On Sun, Apr 10, 2022 at 9:26 AM Petri Tuomola <pe...@gmail.com>
wrote:

> Thanks Arnold - that’s great!
>
> I also spent a lot of time fixing the same issue a few months ago, and
> back then managed to get the up-to-date checks / incremental builds working
> correctly. However it seems like they were then again broken by some of the
> more recent changes - so let’s hope they remain fixed going forward, and
> keep an eye on this.
>
> One problematic area seems to be definitely the OpenAPI generation /
> fineract.yaml file. The fineract.yaml file is both a source file (for the
> client generation) as well as an output (from fineract-provider build) and
> also something that is included in the different distributions. This
> doesn’t fit nicely with the “one-directional” sources -> outputs ->
> distributions model assumed by Gradle, and seemed to get Gradle confused
> enough to trigger too many rebuilds in some scenarios. But good to hear
> you’ve found a way to solve these - I’ll take a look later today to learn
> how it’s done!
>
> Great work again!
>
> Regards
> Petri
>
>
> > On 10 Apr 2022, at 15:06, Arnold Galovics <ar...@apache.org> wrote:
> >
> > Hi everyone,
> >
> > I think all of us are annoyed sometimes with the time it takes to build
> fineract especially if you have integration tests that need to be debugged
> and fixed.
> > A single integration test run (without starting up the fineract server)
> involved rebuilding the fineract-client module and regenerating all the
> clients with it, then rebuilding the integration-tests module which took a
> lot of time; and this was true even though you haven't changed anything.
> The reason was that Gradle's incremental build is not utilized properly and
> the up-to-date checks were broken.
> >
> > Yesterday I managed to fix a lot of these issues, hence the
> fineract-client will not get regenerated and the integration-tests will not
> be recompiled every time you try to run it. The changes are already
> available on develop.
> >
> > My feeling is that these changes mean around an 80-85% decrease in the
> time needed to run integration tests from IntelliJ, but I don't have exact
> measurements.
> >
> > If you encounter issues with the recent changes, let me know.
> >
> > Best,
> > Arnold
>
>

Re: Gradle cache utilization & build speedup

[Petri Tuomola <pe...@gmail.com>]

Thanks Arnold - that’s great!

I also spent a lot of time fixing the same issue a few months ago, and back then managed to get the up-to-date checks / incremental builds working correctly. However it seems like they were then again broken by some of the more recent changes - so let’s hope they remain fixed going forward, and keep an eye on this.

One problematic area seems to be definitely the OpenAPI generation / fineract.yaml file. The fineract.yaml file is both a source file (for the client generation) as well as an output (from fineract-provider build) and also something that is included in the different distributions. This doesn’t fit nicely with the “one-directional” sources -> outputs -> distributions model assumed by Gradle, and seemed to get Gradle confused enough to trigger too many rebuilds in some scenarios. But good to hear you’ve found a way to solve these - I’ll take a look later today to learn how it’s done!

Great work again! 
 
Regards 
Petri


> On 10 Apr 2022, at 15:06, Arnold Galovics <ar...@apache.org> wrote:
> 
> Hi everyone,
> 
> I think all of us are annoyed sometimes with the time it takes to build fineract especially if you have integration tests that need to be debugged and fixed. 
> A single integration test run (without starting up the fineract server) involved rebuilding the fineract-client module and regenerating all the clients with it, then rebuilding the integration-tests module which took a lot of time; and this was true even though you haven't changed anything. The reason was that Gradle's incremental build is not utilized properly and the up-to-date checks were broken.
> 
> Yesterday I managed to fix a lot of these issues, hence the fineract-client will not get regenerated and the integration-tests will not be recompiled every time you try to run it. The changes are already available on develop.
> 
> My feeling is that these changes mean around an 80-85% decrease in the time needed to run integration tests from IntelliJ, but I don't have exact measurements.
> 
> If you encounter issues with the recent changes, let me know.
> 
> Best,
> Arnold