You are viewing a plain text version of this content. The canonical link for it is here.
Posted to ivy-user@ant.apache.org by Hans Dockter <ma...@dockter.biz> on 2009/09/02 12:10:09 UTC

Ivy Performance

Hi,

I'm trying to improve the resolve performance of Gradle's usage of Ivy.

As a test I have an ivy module with three configurations (testRuntime  
extends testCompile extends compile).

As a comparison I use the following ant script:

<?xml version="1.0"?>
<project xmlns:ivy="antlib:org.apache.ivy.ant" name="sample"  
default="resolve1">
     <target name="resolve1">
         <ivy:resolve conf="compile" haltonfailure="false"/>
     </target>

     <target name="resolve11" depends="resolve1">
         <ivy:resolve conf="compile" haltonfailure="false"/>
     </target>

     <target name="resolve2" depends="resolve1">
         <ivy:resolve conf="testCompile" haltonfailure="false"/>
     </target>

     <target name="resolve3" depends="resolve2">
         <ivy:resolve conf="testRuntime" haltonfailure="false"/>
     </target>

     <target name="resolve4">
         <ivy:resolve conf="compile,testCompile,testRuntime"  
haltonfailure="false"/>
     </target>
</project>

The Ant resolve seems to be smart in that it remembers earlier  
resolves. For example when executing resolve11 the second resolve is  
much faster than the first resolve. This is also true for resolves of  
extending configuration (e.g. resolve2 and resolve3).

The difference between resolve3 and the aggregate resolve4 is only 200  
ms in my example (1800ms vs 1600ms).

With Gradle the resolve performance of successive resolves seems not  
to benefit from earlier resolves. Unfortunately I was not successful  
yet to improve this behaviour. We use the same Ivy instance for  
successive resolves.

Where in Ivy does the optimization takes place? What can I do to  
switch this on?

Any help is very much appreciated

- Hans

--
Hans Dockter
Gradle Project Manager
http://www.gradle.org

Re: Ivy Performance

Posted by Hans Dockter <ma...@dockter.biz>.
Hi Gilles,

On Sep 3, 2009, at 9:54 PM, Gilles Scokart wrote:

> You may have a look to https://issues.apache.org/jira/browse/IVY-872  
> where
> some of the performance optimization have been implemented.
>

Thanks a lot. That is exactly the information I was looking for. For  
some reason some of Gradle's special custom resolvers disable this  
optimization. But that should be easy to fix.

- Hans

--
Hans Dockter
Gradle Project Manager
http://www.gradle.org

Re: Ivy Performance

Posted by Gilles Scokart <gs...@gmail.com>.
You may have a look to https://issues.apache.org/jira/browse/IVY-872 where
some of the performance optimization have been implemented.


Gilles Scokart


2009/9/3 Hans Dockter <ma...@dockter.biz>

>
> On Sep 3, 2009, at 4:40 PM, Carlton Brown wrote:
>
>  The resolve task actually fetches artifacts into the local cache, so you
>> would expect subsequent resolves of the same configuration to be much
>> faster (likewise with extended configurations, if they are not too
>> disjoint from the base configuration).
>>
>> If you observe that subsequent resolves don't benefit from previous
>> ones, it sounds as if you're somehow circumventing the cache
>> mechanism... perhaps defeating its effectiveness by setting the
>> useOrigin attribute, using a TTL, cleaning the cache between each
>> resolve, or using a distinct cache for each resolve.   None of these
>> behaviors are enabled by default.
>>
>> I'd check the state of the cache after each resolve... if it isn't
>> getting populated with artifacts, or if it's getting wiped entirely,
>> there's your problem.
>>
>
> Thanks for your reply. I should have mentioned that all artifacts are in
> the cache and are retrieved from the cache. But a resolve from a cache also
> easily takes more than a second. In the case of a second Ant Ivy resolve
> this number shrinks in our examples to something like 200ms. Therefore I was
> wondering what optimization is taking place here and how to use this for our
> build system.
>
>
> - Hans
>
> --
> Hans Dockter
> Gradle Project Manager
> http://www.gradle.org
>
>
>

Re: Ivy Performance

Posted by Hans Dockter <ma...@dockter.biz>.
On Sep 3, 2009, at 4:40 PM, Carlton Brown wrote:

> The resolve task actually fetches artifacts into the local cache, so  
> you
> would expect subsequent resolves of the same configuration to be much
> faster (likewise with extended configurations, if they are not too
> disjoint from the base configuration).
>
> If you observe that subsequent resolves don't benefit from previous
> ones, it sounds as if you're somehow circumventing the cache
> mechanism... perhaps defeating its effectiveness by setting the
> useOrigin attribute, using a TTL, cleaning the cache between each
> resolve, or using a distinct cache for each resolve.   None of these
> behaviors are enabled by default.
>
> I'd check the state of the cache after each resolve... if it isn't
> getting populated with artifacts, or if it's getting wiped entirely,
> there's your problem.

Thanks for your reply. I should have mentioned that all artifacts are  
in the cache and are retrieved from the cache. But a resolve from a  
cache also easily takes more than a second. In the case of a second  
Ant Ivy resolve this number shrinks in our examples to something like  
200ms. Therefore I was wondering what optimization is taking place  
here and how to use this for our build system.

- Hans

--
Hans Dockter
Gradle Project Manager
http://www.gradle.org

RE: Ivy Performance

Posted by Carlton Brown <cb...@silverpop.com>.
The resolve task actually fetches artifacts into the local cache, so you
would expect subsequent resolves of the same configuration to be much
faster (likewise with extended configurations, if they are not too
disjoint from the base configuration).

If you observe that subsequent resolves don't benefit from previous
ones, it sounds as if you're somehow circumventing the cache
mechanism... perhaps defeating its effectiveness by setting the
useOrigin attribute, using a TTL, cleaning the cache between each
resolve, or using a distinct cache for each resolve.   None of these
behaviors are enabled by default.

I'd check the state of the cache after each resolve... if it isn't
getting populated with artifacts, or if it's getting wiped entirely,
there's your problem.

-----Original Message-----
From: Hans Dockter [mailto:mail@dockter.biz] 
Sent: Wednesday, September 02, 2009 6:10 AM
To: ivy-user@ant.apache.org
Subject: Ivy Performance

Hi,

I'm trying to improve the resolve performance of Gradle's usage of Ivy.

As a test I have an ivy module with three configurations (testRuntime  
extends testCompile extends compile).

As a comparison I use the following ant script:

<?xml version="1.0"?>
<project xmlns:ivy="antlib:org.apache.ivy.ant" name="sample"  
default="resolve1">
     <target name="resolve1">
         <ivy:resolve conf="compile" haltonfailure="false"/>
     </target>

     <target name="resolve11" depends="resolve1">
         <ivy:resolve conf="compile" haltonfailure="false"/>
     </target>

     <target name="resolve2" depends="resolve1">
         <ivy:resolve conf="testCompile" haltonfailure="false"/>
     </target>

     <target name="resolve3" depends="resolve2">
         <ivy:resolve conf="testRuntime" haltonfailure="false"/>
     </target>

     <target name="resolve4">
         <ivy:resolve conf="compile,testCompile,testRuntime"  
haltonfailure="false"/>
     </target>
</project>

The Ant resolve seems to be smart in that it remembers earlier  
resolves. For example when executing resolve11 the second resolve is  
much faster than the first resolve. This is also true for resolves of  
extending configuration (e.g. resolve2 and resolve3).

The difference between resolve3 and the aggregate resolve4 is only 200  
ms in my example (1800ms vs 1600ms).

With Gradle the resolve performance of successive resolves seems not  
to benefit from earlier resolves. Unfortunately I was not successful  
yet to improve this behaviour. We use the same Ivy instance for  
successive resolves.

Where in Ivy does the optimization takes place? What can I do to  
switch this on?

Any help is very much appreciated

- Hans

--
Hans Dockter
Gradle Project Manager
http://www.gradle.org


***CONFIDENTIALITY NOTICE and DISCLAIMER*** 
This message and any attachment are confidential and may be
privileged or otherwise protected from disclosure and solely for
the use of the person(s) or entity to whom it is intended. If you
have received this message in error and are not the intended
recipient, please notify the sender immediately and delete this
message and any attachment from your system. If you are not the
intended recipient, be advised that any use of this message is
prohibited and may be unlawful, and you must not copy this
message or attachment or disclose the contents to any other person.