You are viewing a plain text version of this content. The canonical link for it is here.
Posted to ivy-user@ant.apache.org by Geißler,
Daniel <da...@salt-solutions.de> on 2014/11/05 12:27:05 UTC
ivy:report changes between Ivy 2.2.0 and 2.4.0-rc1
Hi,
today i was investigating the build times of different versions of our projects. What catched my eye is that the older version of a pretty large multi module project has significantly faster build times than the newer one. The older builds in 20 minutes while the newer takes about 1 hour, this is not explainable by new code introduced.
What I found is that the ivy:report task takes extraordinary amounts of time. It was never very fast but on some projects the report build time changed like this:
Ivy version
2.2.0
2.3.0-rc1
Diff
Module 1
ivy:report time
00:00:21
00:06:18
+1800%
html report file size
42.19 MB
867.92 MB
+2057%
graphml report
513.55 KB
573.38 KB
+11%
dot report
83.60 KB
94.11 KB
+12%
Module 2
ivy:report time
00:01:29
00:20:43
+1396%
html report file size
129.58 MB
2.18 GB
+1682%
graphml report
370.96 KB
457.42 KB
+23%
dot report
61.17 KB
75.73 KB
+23%
The call looks like this:
<ivy:report conf="${fetch.configurations},build"
xml="true"
dot="true"
todir="${project.dir.build.ivy-reports}" />
We kept the reports building just in case something strange happens with the dependencies and with Jenkins it never used to be a problem to wait some extra minutes, but with these build times it s totaly over the top. Not to mention that I don't know any browser that can display a html page of 2 GB in a reasonable amount of time.
Are there some new features I'd miss in the release notes or should I open a JIRA ticket for this?
kind regards
Daniel
_____________________________________
www.salt-solutions.de<http://www.salt-solutions.de>
Geschäftsführer: Dr. Bernhard Blüthner, Dieter Heyde, Markus Honold
Sitz: München, AG München, HRB 146081
Re: ivy:report changes between Ivy 2.2.0 and 2.4.0-rc1
Posted by Marc De Boeck <md...@gmail.com>.
Hi Daniel,
I did exactly the same thing. In our case, the reports were so big, that
they couldn't be loaded in a browser.
Here is what I changed in the xsl file:
Original:
<xsl:for-each
select="$modules/revision/caller[(@organisation=$organisation and
@name=$module) and @callerrev=$rev]">
<xsl:call-template name="called">
<xsl:with-param name="callstack" select="concat($callstack,
string('#'), $organisation, string('/'), $module)"/>
<xsl:with-param name="indent" select="concat($indent,
string('---'))"/>
<xsl:with-param name="revision" select=".."/>
</xsl:call-template>
</xsl:for-each>
My version:
<xsl:if test="not(contains($indent, '-->' ))">
<xsl:for-each
select="$modules/revision/caller[(@organisation=$organisation and
@name=$module) and @callerrev=$rev]">
<xsl:call-template name="called">
<xsl:with-param name="callstack" select="concat($callstack,
string('#'), $organisation, string('/'), $module)"/>
<xsl:with-param name="indent" select="concat($indent,
string('-->'))"/>
<xsl:with-param name="revision" select=".."/>
</xsl:call-template>
</xsl:for-each>
</xsl:if>
Note that I also changed the indentation marks "---" into "-->" , because i
found it looked better that way.
Regards,
Marc
2014-11-07 14:19 GMT+01:00 Geißler, Daniel <
daniel.geissler@salt-solutions.de>:
> Hi,
>
> Ii found an answer myself. As the ivy-report.xsl didn't change (at least
> not when looking at the git history) I guess my dependency tree grew a way
> too much to get a useful html report out of it.
>
> So the thing I changed was cutting of the dependency trees in the html
> report (see the recursion on the end of template "called").
> This was cutting down my builds to 10-20% of their previous duration.
>
> It seems that there is some kind of exponential growth with the xsl
> describing the html report. I'd suggest to review it, as it is a very
> useful tool but costs too many resources.
> One idea is to cut off all depdendency trees like I did, or maybe show the
> one overall tree but change the module detail dependencies to show only
> direct dependencies.
>
> Anyway thanks for reading
> Daniel
> _____________________________________
>
> www.salt-solutions.de
>
> Geschäftsführer: Dr. Bernhard Blüthner, Dieter Heyde, Markus Honold
> Sitz: München, AG München, HRB 146081
> -----Ursprüngliche Nachricht-----
> Von: Geißler, Daniel
> Gesendet: Mi, 5. November 2014 12:27
> An: ivy-user@ant.apache.org
> Betreff: ivy:report changes between Ivy 2.2.0 and 2.4.0-rc1
>
> Hi,
>
> today i was investigating the build times of different versions of our
> projects. What catched my eye is that the older version of a pretty large
> multi module project has significantly faster build times than the newer
> one. The older builds in 20 minutes while the newer takes about 1 hour,
> this is not explainable by new code introduced.
>
> What I found is that the ivy:report task takes extraordinary amounts of
> time. It was never very fast but on some projects the report build time
> changed like this:
>
> Ivy version
>
> 2.2.0
>
> 2.3.0-rc1
>
> Diff
>
> Module 1
>
>
>
>
>
>
>
> ivy:report time
>
> 00:00:21
>
> 00:06:18
>
> +1800%
>
> html report file size
>
> 42.19 MB
>
> 867.92 MB
>
> +2057%
>
> graphml report
>
> 513.55 KB
>
> 573.38 KB
>
> +11%
>
> dot report
>
> 83.60 KB
>
> 94.11 KB
>
> +12%
>
> Module 2
>
>
>
>
>
>
>
> ivy:report time
>
> 00:01:29
>
> 00:20:43
>
> +1396%
>
> html report file size
>
> 129.58 MB
>
> 2.18 GB
>
> +1682%
>
> graphml report
>
> 370.96 KB
>
> 457.42 KB
>
> +23%
>
> dot report
>
> 61.17 KB
>
> 75.73 KB
>
> +23%
>
>
> The call looks like this:
>
> <ivy:report conf="${fetch.configurations},build"
> xml="true"
> dot="true"
> todir="${project.dir.build.ivy-reports}" />
>
> We kept the reports building just in case something strange happens with
> the dependencies and with Jenkins it never used to be a problem to wait
> some extra minutes, but with these build times it s totaly over the top.
> Not to mention that I don't know any browser that can display a html page
> of 2 GB in a reasonable amount of time.
>
> Are there some new features I'd miss in the release notes or should I open
> a JIRA ticket for this?
>
> kind regards
> Daniel
>
> _____________________________________
> www.salt-solutions.de<http://www.salt-solutions.de>
>
> Geschäftsführer: Dr. Bernhard Blüthner, Dieter Heyde, Markus Honold
> Sitz: München, AG München, HRB 146081
>
AW: ivy:report changes between Ivy 2.2.0 and 2.4.0-rc1
Posted by Geißler,
Daniel <da...@salt-solutions.de>.
Hi,
Ii found an answer myself. As the ivy-report.xsl didn't change (at least not when looking at the git history) I guess my dependency tree grew a way too much to get a useful html report out of it.
So the thing I changed was cutting of the dependency trees in the html report (see the recursion on the end of template "called").
This was cutting down my builds to 10-20% of their previous duration.
It seems that there is some kind of exponential growth with the xsl describing the html report. I'd suggest to review it, as it is a very useful tool but costs too many resources.
One idea is to cut off all depdendency trees like I did, or maybe show the one overall tree but change the module detail dependencies to show only direct dependencies.
Anyway thanks for reading
Daniel
_____________________________________
www.salt-solutions.de
Geschäftsführer: Dr. Bernhard Blüthner, Dieter Heyde, Markus Honold
Sitz: München, AG München, HRB 146081
-----Ursprüngliche Nachricht-----
Von: Geißler, Daniel
Gesendet: Mi, 5. November 2014 12:27
An: ivy-user@ant.apache.org
Betreff: ivy:report changes between Ivy 2.2.0 and 2.4.0-rc1
Hi,
today i was investigating the build times of different versions of our projects. What catched my eye is that the older version of a pretty large multi module project has significantly faster build times than the newer one. The older builds in 20 minutes while the newer takes about 1 hour, this is not explainable by new code introduced.
What I found is that the ivy:report task takes extraordinary amounts of time. It was never very fast but on some projects the report build time changed like this:
Ivy version
2.2.0
2.3.0-rc1
Diff
Module 1
ivy:report time
00:00:21
00:06:18
+1800%
html report file size
42.19 MB
867.92 MB
+2057%
graphml report
513.55 KB
573.38 KB
+11%
dot report
83.60 KB
94.11 KB
+12%
Module 2
ivy:report time
00:01:29
00:20:43
+1396%
html report file size
129.58 MB
2.18 GB
+1682%
graphml report
370.96 KB
457.42 KB
+23%
dot report
61.17 KB
75.73 KB
+23%
The call looks like this:
<ivy:report conf="${fetch.configurations},build"
xml="true"
dot="true"
todir="${project.dir.build.ivy-reports}" />
We kept the reports building just in case something strange happens with the dependencies and with Jenkins it never used to be a problem to wait some extra minutes, but with these build times it s totaly over the top. Not to mention that I don't know any browser that can display a html page of 2 GB in a reasonable amount of time.
Are there some new features I'd miss in the release notes or should I open a JIRA ticket for this?
kind regards
Daniel
_____________________________________
www.salt-solutions.de<http://www.salt-solutions.de>
Geschäftsführer: Dr. Bernhard Blüthner, Dieter Heyde, Markus Honold
Sitz: München, AG München, HRB 146081