ivy cache, changing patterns and time to live

[Geißler, Daniel <da...@salt-solutions.de>]

Hi,

we are investigating some serious pain with our build times and identified ivy:resolve as the main time consumer.
While having similar dependency trees in most of the components in the multi module build, all artifacts seem to be searched in the remote repository all the time.

We know that dynamic and SNAPSHOT versions are one reason for that, but still there must be a way to keep the results at least for one build fresh.
This is where we found the defaultTTL or ttl aspects of caches (http://ant.apache.org/ivy/history/latest-milestone/settings/caches/ttl.html).

With some rough testing we figured out that enabling the checkmodified flag on a resolver forces every artifact to be resolved (be it a SNAPSHOT dependency or not, all artifacts are listed as searched by the resolve task).
But when we set checkmodified="false" and a short defaultTTL on the cache, repeated calls will not trigger a search until the time expires.

Here's the resolver configuration for the repository containing SNAPSHOT and released artifacts:

<ibiblio name="all-development" m2compatible="true" latest="my-strategy" useMavenMetadata="false"
            root="${ivy.repository.root}/${ivy.repository.name}"
            checkmodified="false" changingMatcher="regexp" changingPattern=".*-SNAPSHOT.*" />


and this is how our cache is configured:

<cache name="my-cache" useOrigin="true" basedir="${ivy.cache.repository.dir}"
            ivyPattern="${ivy.cache.ivy-pattern}" artifactPattern="${ivy.cache.artifact-pattern}"
            lockStrategy="artifact-lock-nio" />

Anyone can give brief explanation when to use what and which sideeffects to expect? The documentation is not exactly clear about how to handle changing artifacts in a effective way.

any help would be appreciated
Daniel Geißler
_____________________________________

SALT Solutions AG

www.salt-solutions.de<http://www.salt-solutions.de>

Vorstand: Dieter Heyde (Vors.), Dr. Bernhard Blüthner, Dr. Hans Christoph Dönges, Claudia Lang, Frank Reinecke
Vorsitzender des Aufsichtsrats: Dr. Michael Fuchs
Sitz: München, AG München, HRB 228545

Re: ivy cache, changing patterns and time to live

[Jaikiran Pai <ja...@gmail.com>]

To be honest, I did not fully understand what the issue/question is. Is 
it possible for you to create a simple application which reproduces and 
explains what the issue is?

P.S: AFAIK, this mailing list doesn't support attachments, so if you do 
have that simple application, you might want it included within the 
email as text.

-Jaikiran


On 01/02/18 2:33 AM, Geißler, Daniel wrote:
> Hi,
>
> we are investigating some serious pain with our build times and identified ivy:resolve as the main time consumer.
> While having similar dependency trees in most of the components in the multi module build, all artifacts seem to be searched in the remote repository all the time.
>
> We know that dynamic and SNAPSHOT versions are one reason for that, but still there must be a way to keep the results at least for one build fresh.
> This is where we found the defaultTTL or ttl aspects of caches (http://ant.apache.org/ivy/history/latest-milestone/settings/caches/ttl.html).
>
> With some rough testing we figured out that enabling the checkmodified flag on a resolver forces every artifact to be resolved (be it a SNAPSHOT dependency or not, all artifacts are listed as searched by the resolve task).
> But when we set checkmodified="false" and a short defaultTTL on the cache, repeated calls will not trigger a search until the time expires.
>
> Here's the resolver configuration for the repository containing SNAPSHOT and released artifacts:
>
> <ibiblio name="all-development" m2compatible="true" latest="my-strategy" useMavenMetadata="false"
>              root="${ivy.repository.root}/${ivy.repository.name}"
>              checkmodified="false" changingMatcher="regexp" changingPattern=".*-SNAPSHOT.*" />
>
>
> and this is how our cache is configured:
>
> <cache name="my-cache" useOrigin="true" basedir="${ivy.cache.repository.dir}"
>              ivyPattern="${ivy.cache.ivy-pattern}" artifactPattern="${ivy.cache.artifact-pattern}"
>              lockStrategy="artifact-lock-nio" />
>
> Anyone can give brief explanation when to use what and which sideeffects to expect? The documentation is not exactly clear about how to handle changing artifacts in a effective way.
>
> any help would be appreciated
> Daniel Geißler
> _____________________________________
>
> SALT Solutions AG
>
> www.salt-solutions.de<http://www.salt-solutions.de>
>
> Vorstand: Dieter Heyde (Vors.), Dr. Bernhard Blüthner, Dr. Hans Christoph Dönges, Claudia Lang, Frank Reinecke
> Vorsitzender des Aufsichtsrats: Dr. Michael Fuchs
> Sitz: München, AG München, HRB 228545
>