You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-dev@db.apache.org by Daniel John Debrunner <dj...@apache.org> on 2007/04/24 06:17:40 UTC
0.05% of test fixtures contribute 53% of the total running time!!
In a test run (junit-all) of 5,816 fixtures that took 4124 seconds
(~68mins) *three* fixtures took 53% of the total time.
testPositionAgressive 1564.684 seconds
testNetworkServerSecurityMechanism 497.280 seconds
testTypesInActionStatement 128.928 seconds
I think testNetworkServerSecurityMechanism is being worked on but is
anyone looking at reducing the time for testPositionAgressive?
I'll take a look at the testTypesInActionStatement since I added that test.
Dan.
Re: 0.05% of test fixtures contribute 53% of the total running time!!
Posted by Daniel John Debrunner <dj...@apache.org>.
Øystein Grøvlen wrote:
> Daniel John Debrunner wrote:
>> In a test run (junit-all) of 5,816 fixtures that took 4124 seconds
>> (~68mins) *three* fixtures took 53% of the total time.
[snip]
> Do you have some tips on how to measure the duration of individual test
> cases.
Running tests through ant gives the times at the fixture level, though I
don't think we have a target to execute an arbitrary test. I think if
you run the tests using the junitreport target the report will include
the times. I'm running the tests using junit-all via CruiseControl which
automatically takes the xml files from the test runs and produces a
report including times. Though it doesn't provide any ordering ability
on the times.
Dan.
Re: 0.05% of test fixtures contribute 53% of the total running time!!
Posted by Knut Anders Hatlen <Kn...@Sun.COM>.
Øystein Grøvlen <Oy...@Sun.COM> writes:
> Knut Anders Hatlen wrote:
>
>> I once wrote my own test runner which printed the time in milliseconds
>> for each test case, see the attached TimeRunner.java.
>
> Thanks, this is useful.
>
>>
>> Note that testPositionAgressive doesn't always take that much time. I
>> think that test and the other testPosition tests use random offset and
>> length and therefore the variation is high.
>
> While this is true for several of the test cases in BlobClob4BlobTest,
> by looking at the code, I am not able to find anything random in the
> execution of testPositionAgressive. I will try to run them and see
> what I get.
I think you are correct, there's nothing random in that fixture. It
seems like it started taking a lot more time at revision 530085. I had
run the tests at an earlier revision.
--
Knut Anders
Re: 0.05% of test fixtures contribute 53% of the total running time!!
Posted by Øystein Grøvlen <Oy...@Sun.COM>.
Knut Anders Hatlen wrote:
> I once wrote my own test runner which printed the time in milliseconds
> for each test case, see the attached TimeRunner.java.
Thanks, this is useful.
>
> Note that testPositionAgressive doesn't always take that much time. I
> think that test and the other testPosition tests use random offset and
> length and therefore the variation is high.
While this is true for several of the test cases in BlobClob4BlobTest,
by looking at the code, I am not able to find anything random in the
execution of testPositionAgressive. I will try to run them and see what
I get.
--
Øystein
Re: 0.05% of test fixtures contribute 53% of the total running time!!
Posted by Knut Anders Hatlen <Kn...@Sun.COM>.
Øystein Grøvlen <Oy...@Sun.COM> writes:
> Daniel John Debrunner wrote:
>> In a test run (junit-all) of 5,816 fixtures that took 4124 seconds
>> (~68mins) *three* fixtures took 53% of the total time.
>>
>> testPositionAgressive 1564.684 seconds
>> testNetworkServerSecurityMechanism 497.280 seconds
>> testTypesInActionStatement 128.928 seconds
>>
>> I think testNetworkServerSecurityMechanism is being worked on but is
>> anyone looking at reducing the time for testPositionAgressive?
>>
>> I'll take a look at the testTypesInActionStatement since I added that test.
>
> I will take a look at testPosisitionAgressive since I am currently
> working with the BlobClob4BlobTest. It suprises me a bit that it
> takes so long time since I do not think I have experience so long run
> times for the entire BlobClob4BlobTest.
>
> Do you have some tips on how to measure the duration of individual
> test cases.
I once wrote my own test runner which printed the time in milliseconds
for each test case, see the attached TimeRunner.java.
Note that testPositionAgressive doesn't always take that much time. I
think that test and the other testPosition tests use random offset and
length and therefore the variation is high. Last time I timed the tests
(one week ago), these tests were top ten:
testNetworkServerSecurityMechanism 363.902 seconds
derbyStress 192.423 seconds
testParameterMapping 172.574 seconds
derbyStress 117.360 seconds
testResultSetGarbageCollection 91.856 seconds
testResultSetGarbageCollection 91.237 seconds
SecureServerTest 62.783 seconds
testDeleteRowWithDeleteTrigger 60.398 seconds
testDeleteRowWithDeleteTrigger 60.201 seconds
SetQueryTimeoutTest 41.896 seconds
(the duplicates on the list are because the tests ran in both embedded
and client/server)
In this run, testPositionAgressive used 4.233 seconds and 2.947 seconds.
--
Knut Anders
Re: 0.05% of test fixtures contribute 53% of the total running time!!
Posted by Øystein Grøvlen <Oy...@Sun.COM>.
Daniel John Debrunner wrote:
> In a test run (junit-all) of 5,816 fixtures that took 4124 seconds
> (~68mins) *three* fixtures took 53% of the total time.
>
> testPositionAgressive 1564.684 seconds
> testNetworkServerSecurityMechanism 497.280 seconds
> testTypesInActionStatement 128.928 seconds
>
> I think testNetworkServerSecurityMechanism is being worked on but is
> anyone looking at reducing the time for testPositionAgressive?
>
> I'll take a look at the testTypesInActionStatement since I added that test.
I will take a look at testPosisitionAgressive since I am currently
working with the BlobClob4BlobTest. It suprises me a bit that it takes
so long time since I do not think I have experience so long run times
for the entire BlobClob4BlobTest.
Do you have some tips on how to measure the duration of individual test
cases.
--
Øystein
Re: 0.05% of test fixtures contribute 53% of the total running time!!
Posted by Sunitha Kambhampati <ks...@gmail.com>.
Oystein Grovlen - Sun Norway wrote:
>
> I have created DERBY-2586 for the testPositionAgressive problem.
>
DERBY-2586 is fixed and the fix makes the BlobClob4BlobTest go faster.
The changes for DERBY-2346 (r530085) slowed it down in the first place
and there has not been significant changes to the test itself.
Here are the running times for BlobClob4BlobTest on my linux box.
(before change for derby-2346), revision r530070 - test took 200s
(after change for derby-2346),revision r530085 - test took ~3000s
With fix for derby2586 - test is back to ~ 200s
Is anyone looking at this perf regression with Derby-2346 changes..
Thanks,
Sunitha.
Re: 0.05% of test fixtures contribute 53% of the total running time!!
Posted by Oystein Grovlen - Sun Norway <Oy...@Sun.COM>.
Daniel John Debrunner wrote:
>
> I think testNetworkServerSecurityMechanism is being worked on but is
> anyone looking at reducing the time for testPositionAgressive?
I have created DERBY-2596 for the testPositionAgressive problem.
--
Øystein