You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ant.apache.org by Laura Dean <lg...@cyrusinnovation.com> on 2010/08/02 04:27:08 UTC
Re: junit tests running slower in 1.8 than in 1.7
On Wed, Jul 28, 2010 at 6:44 AM, Stefan Bodewig <bo...@apache.org> wrote:
>
...
> OK, I'll look into this when I get access to my Windows machine again
> next week. It didn't take more than the small test case you provided to
> see the difference, right?
Right. The small test case was enough to see the difference, on the 2
or 3 machines I tried, though not enough to make it very striking.
(The numbers in my first message are from the small test case, to give
you some idea.) Thanks,
Laura
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@ant.apache.org
For additional commands, e-mail: user-help@ant.apache.org
Re: junit tests running slower in 1.8 than in 1.7
Posted by Stefan Bodewig <bo...@apache.org>.
On 2010-08-08, Michael Ludwig wrote:
> Stefan Bodewig schrieb am 08.08.2010 um 22:12 (+0200):
>> Finally I found the reason - the enhancement that is Bugzilla Issue
>> 31885. Ant writes output synchronously in order to support advanced
>> test UIs.
> https://issues.apache.org/bugzilla/show_bug.cgi?id=31885
> Great you found the cause! I might be wrong, but I think this kind
> of IO is simply referred to as "unbuffered IO", whereas "blocking",
> "non-blocking" and "asynchronous IO" would involve device readiness.
Neither term is what I meant (and yes, I am aware of the differences, my
language was just sloppy 8-).
Ant's test runner buffers all output of all tests and flushes it to the
test formatters once a testsuite finishes (I'm glossing over a few
details here), the "testevents" are sent to System.out immediately
rather than being buffered up - that's what I refered to as
synchronously since output is synchronous to the tests rather than
delayed.
Stefan
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@ant.apache.org
For additional commands, e-mail: user-help@ant.apache.org
Re: junit tests running slower in 1.8 than in 1.7
Posted by Michael Ludwig <mi...@gmx.de>.
Stefan Bodewig schrieb am 08.08.2010 um 22:12 (+0200):
> Finally I found the reason - the enhancement that is Bugzilla Issue
> 31885. Ant writes output synchronously in order to support advanced
> test UIs.
https://issues.apache.org/bugzilla/show_bug.cgi?id=31885
Great you found the cause! I might be wrong, but I think this kind
of IO is simply referred to as "unbuffered IO", whereas "blocking",
"non-blocking" and "asynchronous IO" would involve device readiness.
--
Michael Ludwig
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@ant.apache.org
For additional commands, e-mail: user-help@ant.apache.org
Re: junit tests running slower in 1.8 than in 1.7
Posted by Laura Dean <lg...@cyrusinnovation.com>.
We tried it out, and our tests ran much faster. We liked it so much
that we took the Ant 1.8.1 source and applied your change to it, so we
could use it for the time being. We put the source here:
http://github.com/cyrusinnovation/Apache-Ant
Please note that we have no intention of maintaining this fork at all;
we just put it there for our convenience, and I mention it now it in
case it saves anyone else a little work.
Thanks again!
Laura
On Thu, Aug 19, 2010 at 11:10 AM, Laura Dean <lg...@cyrusinnovation.com> wrote:
> On Thu, Aug 19, 2010 at 8:54 AM, Stefan Bodewig <bo...@apache.org> wrote:
>> The change has been implemented in svn trunk[1] and may become part of
>> the next Ant release. Using svn trunk my tests run as fast (or even
>> faster) as they do using Ant 1.7.1.
>
> Thank you very much! We'll definitely try it out.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@ant.apache.org
For additional commands, e-mail: user-help@ant.apache.org
Re: junit tests running slower in 1.8 than in 1.7
Posted by Laura Dean <lg...@cyrusinnovation.com>.
On Thu, Aug 19, 2010 at 8:54 AM, Stefan Bodewig <bo...@apache.org> wrote:
> The change has been implemented in svn trunk[1] and may become part of
> the next Ant release. Using svn trunk my tests run as fast (or even
> faster) as they do using Ant 1.7.1.
Thank you very much! We'll definitely try it out.
> To be clear, the change only affects forked tests, is mostly invisible
> on anything but Windows and really only affects tests that finish in far
> less than 100ms. If your average test takes several seconds, there
> won't be any difference between Ant 1.7.1 and 1.8.1. Yes, tests run
> slower with Ant 1.8.x but generally not a lot.
That fits our experience exactly. Our unit tests slowed down
dramatically, but our integration tests didn't. I wish I had noticed
that difference at the time, but I was so focused on the unit-test
slowdown that I didn't really think about it. (If anything, I
attributed the difference to the fact that we have so many *more* unit
tests.)
Laura
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@ant.apache.org
For additional commands, e-mail: user-help@ant.apache.org
Re: junit tests running slower in 1.8 than in 1.7
Posted by Stefan Bodewig <bo...@apache.org>.
On 2010-08-08, Stefan Bodewig wrote:
> I'll suggest to add a configuration option that will allow people to
> disbale those requests on the dev list, but right now I can't do more
> than confirm that forked JUnit tasks are slower with Ant 1.8.[01] than
> with 1.7.1.
The change has been implemented in svn trunk[1] and may become part of
the next Ant release. Using svn trunk my tests run as fast (or even
faster) as they do using Ant 1.7.1.
To be clear, the change only affects forked tests, is mostly invisible
on anything but Windows and really only affects tests that finish in far
less than 100ms. If your average test takes several seconds, there
won't be any difference between Ant 1.7.1 and 1.8.1. Yes, tests run
slower with Ant 1.8.x but generally not a lot.
Ant's own testsuite finishes in a bit more that 15 minutes on my Windows
test system and has been sped up by less than six seconds with the
change - and this difference may very well be due to other influences.
Stefan
[1] http://svn.apache.org/viewvc?view=revision&revision=987139
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@ant.apache.org
For additional commands, e-mail: user-help@ant.apache.org
Re: junit tests running slower in 1.8 than in 1.7
Posted by Stefan Bodewig <bo...@apache.org>.
On 2010-08-02, Stefan Bodewig wrote:
> Now I can confirm I see some differences on the Windows machine as well.
> There is almost no difference for the tests that get run first (which
> take longer than the code-identical subsequent tests), so the warmup
> time hasn't changed much.
> I'll try to find the reason for this.
Found it, and there won't be a simple fix for it.
The reason really is Ant's changed I/O handling of forked processes and
that's why you see the impact on Windows. I've taken the svn tag
matching Ant 1.7.1 and replaced two classes (StreamPumper and
PumpStreamHandler) with their Ant 1.8.0 cousins and see the performance
numbers of Ant 1.8.0.
With Ant 1.8.0 I/O handling has been changed on Windows to ensure Ant
doesn't hang indefinitely in the presence of grandchild-processes, so
without that change Ant simply doesn't wirk correctly and a loss in
performance is the price we have to pay.
What was puzzling me for a long time was why the changes affect the
forked VM at all when your testcases don't create any output - and even
if they did, output would be sent batched to the listeners at the end of
each testsuite.
Finally I found the reason - the enhancement that is Bugzilla Issue
31885. Ant writes output synchronously in order to support advanced
test UIs. Here you pay for a feature you don't want to use and in fact
if I disable these events, I get better performance than with stock Ant
1.7.1.
I'll suggest to add a configuration option that will allow people to
disbale those requests on the dev list, but right now I can't do more
than confirm that forked JUnit tasks are slower with Ant 1.8.[01] than
with 1.7.1.
Stefan
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@ant.apache.org
For additional commands, e-mail: user-help@ant.apache.org
Re: junit tests running slower in 1.8 than in 1.7
Posted by Stefan Bodewig <bo...@apache.org>.
On 2010-08-02, Laura Dean wrote:
> On Wed, Jul 28, 2010 at 6:44 AM, Stefan Bodewig <bo...@apache.org> wrote:
> ...
>> OK, I'll look into this when I get access to my Windows machine again
>> next week. It didn't take more than the small test case you provided to
>> see the difference, right?
> Right. The small test case was enough to see the difference, on the 2
> or 3 machines I tried, though not enough to make it very striking.
Now I can confirm I see some differences on the Windows machine as well.
There is almost no difference for the tests that get run first (which
take longer than the code-identical subsequent tests), so the warmup
time hasn't changed much.
I'll try to find the reason for this.
Stefan
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@ant.apache.org
For additional commands, e-mail: user-help@ant.apache.org