You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@harmony.apache.org by Stepan Mishura <st...@gmail.com> on 2007/09/27 11:37:42 UTC
[general][M3] Functional test suite status (Was: Re: [general] M3 - code frozen)
On 9/27/07, Pavlenko, Andrey A <an...@googlemail.com> wrote:
> I've investigated the errors of the func tests. Please see my comments
> below.
>
> >
> functional.org.apache.harmony.test.func.api.java.security.F_KeyFactoryTest_01
> HARMONY-4857
> > functional.org.apache.harmony.test.func.api.javax.swing.AbstractButton
> Intermittent. Always passes in a single mode.
>
> The following tests fail because they are too heavy and sometimes they are
> not finishing in 300 sec (default timeout). For example, some tests perform
> several millions of iterations... I'm not sure if it's required to do so
> many iterations, but if it's so the timeout should be increased.
I'm afraid that increasing timeout may not help. Sometimes I observe
hanged java processes forked by the suite run - they eat 99% of CPU
and I had to kill them manually.
Of cause we can try to increase timeout so what timeout should be set
to make these test pass?
Thanks,
Stepan.
Also these
> tests pass if I decrease concurrency to 2. By default the tests are running
> in 4 threads.
> > functional.org.apache.harmony.test.func.api.java.io.DataInputStream
> Timeout
> > functional.org.apache.harmony.test.func.jit.HLO.abcd.Test1
> Timeout
> > functional.org.apache.harmony.test.func.jit.HLO.lazyexc.ExcObjectUse2
> Timeout
> > functional.org.apache.harmony.test.func.jit.HLO.peel.ExceptionLoop
> Timeout
> > functional.org.apache.harmony.test.func.jit.HLO.peel.LoopVar2
> Timeout
> > functional.org.apache.harmony.test.func.jit.HLO.peel.TryCatch4
> Timeout
> > functional.org.apache.harmony.test.func.reg.vm.btest1355
> Timeout
> > functional.org.apache.harmony.test.func.jit.HLO.uce.unreachathrow
> Timeout. Too many iterations - 20000000. the test pass in ~45s if I
> decrease the number of iterations to 500000
>
> So, it looks like there is 1 regression - HARMONY-4857.
>
<SNIP>
Re: [general][M3] Functional test suite status (Was: Re: [general] M3 - code frozen)
Posted by Alexey Varlamov <al...@gmail.com>.
> On 9/27/07, Pavlenko, Andrey A <an...@googlemail.com> wrote:
> > I've investigated the errors of the func tests. Please see my comments
> > below.
> >
> > >
> > functional.org.apache.harmony.test.func.api.java.security.F_KeyFactoryTest_01
> > HARMONY-4857
Upon evaluation, I've reopened HARMONY-4184 "default Policy fails to
grant permissions to extensions classes", and we better fix this one
in M3.
The HARMONY-4857 itself IMO may be postponed after M3.
<SNIP>
Re: [general][M3] Functional test suite status (Was: Re: [general] M3 - code frozen)
Posted by Pavel Ozhdikhin <pa...@gmail.com>.
Probably the test author chose such a big number of iterations to reliably
test recompiled code in the server mode. You may select a better value by
running with '-Xem:server -Xverbose:em' options and make sure your value is
sufficient to unreach.test() is recompiled by SD2_OPT JIT.
In the long term I think we should get rid of such long-running test and
rely on the special testing modes in JIT. For example, when conversion of
the 'server_static' mode to 'opt' one is complete, most of the server
optimizations could be tested without such a trick for switching on
recompilation by loop iterations.
Thanks,
Pavel
On 9/28/07, Pavlenko, Andrey A <an...@googlemail.com> wrote:
>
> Stepan,
>
> Frankly speaking I think that timeout in 300 sec should be more than
> enough
> for functional tests. So, IMO we should make the tests faster rather than
> increasing timeout. For example, looking at the test
> functional.org.apache.harmony.test.func.jit.HLO.uce.unreachathrow:
>
> private int invoker() {
> UnreachATHROW unreach = new UnreachATHROW();
> for(int i=0; i<20000000; i++) {
> try {
> unreach.test();
> return fail("FAILED: exception should be thrown");
> }
> catch(Exception e) {
> //ignore
> }
> }
> return pass();
> }
>
>
> It's simple, but it performs 20000000 iterations! I'm not sure if it's
> required to perform so much iterations. May be JIT gurus could say
> something?
> There are also other similar examples.
>
> On 9/27/07, Stepan Mishura <st...@gmail.com> wrote:
> >
> > On 9/27/07, Pavlenko, Andrey A <an...@googlemail.com> wrote:
> > > I've investigated the errors of the func tests. Please see my comments
> > > below.
> > >
> > > >
> > >
> >
> functional.org.apache.harmony.test.func.api.java.security.F_KeyFactoryTest_01
> > > HARMONY-4857
> > > >
> functional.org.apache.harmony.test.func.api.javax.swing.AbstractButton
> > > Intermittent. Always passes in a single mode.
> > >
> > > The following tests fail because they are too heavy and sometimes they
> > are
> > > not finishing in 300 sec (default timeout). For example, some tests
> > perform
> > > several millions of iterations... I'm not sure if it's required to do
> so
> > > many iterations, but if it's so the timeout should be increased.
> >
> > I'm afraid that increasing timeout may not help. Sometimes I observe
> > hanged java processes forked by the suite run - they eat 99% of CPU
> > and I had to kill them manually.
> >
> > Of cause we can try to increase timeout so what timeout should be set
> > to make these test pass?
> >
> > Thanks,
> > Stepan.
> >
> > Also these
> > > tests pass if I decrease concurrency to 2. By default the tests are
> > running
> > > in 4 threads.
> > > > functional.org.apache.harmony.test.func.api.java.io.DataInputStream
> > > Timeout
> > > > functional.org.apache.harmony.test.func.jit.HLO.abcd.Test1
> > > Timeout
> > > >
> functional.org.apache.harmony.test.func.jit.HLO.lazyexc.ExcObjectUse2
> > > Timeout
> > > > functional.org.apache.harmony.test.func.jit.HLO.peel.ExceptionLoop
> > > Timeout
> > > > functional.org.apache.harmony.test.func.jit.HLO.peel.LoopVar2
> > > Timeout
> > > > functional.org.apache.harmony.test.func.jit.HLO.peel.TryCatch4
> > > Timeout
> > > > functional.org.apache.harmony.test.func.reg.vm.btest1355
> > > Timeout
> > > > functional.org.apache.harmony.test.func.jit.HLO.uce.unreachathrow
> > > Timeout. Too many iterations - 20000000. the test pass in ~45s if I
> > > decrease the number of iterations to 500000
> > >
> > > So, it looks like there is 1 regression - HARMONY-4857.
> > >
> >
> > <SNIP>
> >
>
Re: [general][M3] Functional test suite status (Was: Re: [general] M3 - code frozen)
Posted by "Pavlenko, Andrey A" <an...@googlemail.com>.
Stepan,
Frankly speaking I think that timeout in 300 sec should be more than enough
for functional tests. So, IMO we should make the tests faster rather than
increasing timeout. For example, looking at the test
functional.org.apache.harmony.test.func.jit.HLO.uce.unreachathrow:
private int invoker() {
UnreachATHROW unreach = new UnreachATHROW();
for(int i=0; i<20000000; i++) {
try {
unreach.test();
return fail("FAILED: exception should be thrown");
}
catch(Exception e) {
//ignore
}
}
return pass();
}
It's simple, but it performs 20000000 iterations! I'm not sure if it's
required to perform so much iterations. May be JIT gurus could say
something?
There are also other similar examples.
On 9/27/07, Stepan Mishura <st...@gmail.com> wrote:
>
> On 9/27/07, Pavlenko, Andrey A <an...@googlemail.com> wrote:
> > I've investigated the errors of the func tests. Please see my comments
> > below.
> >
> > >
> >
> functional.org.apache.harmony.test.func.api.java.security.F_KeyFactoryTest_01
> > HARMONY-4857
> > > functional.org.apache.harmony.test.func.api.javax.swing.AbstractButton
> > Intermittent. Always passes in a single mode.
> >
> > The following tests fail because they are too heavy and sometimes they
> are
> > not finishing in 300 sec (default timeout). For example, some tests
> perform
> > several millions of iterations... I'm not sure if it's required to do so
> > many iterations, but if it's so the timeout should be increased.
>
> I'm afraid that increasing timeout may not help. Sometimes I observe
> hanged java processes forked by the suite run - they eat 99% of CPU
> and I had to kill them manually.
>
> Of cause we can try to increase timeout so what timeout should be set
> to make these test pass?
>
> Thanks,
> Stepan.
>
> Also these
> > tests pass if I decrease concurrency to 2. By default the tests are
> running
> > in 4 threads.
> > > functional.org.apache.harmony.test.func.api.java.io.DataInputStream
> > Timeout
> > > functional.org.apache.harmony.test.func.jit.HLO.abcd.Test1
> > Timeout
> > > functional.org.apache.harmony.test.func.jit.HLO.lazyexc.ExcObjectUse2
> > Timeout
> > > functional.org.apache.harmony.test.func.jit.HLO.peel.ExceptionLoop
> > Timeout
> > > functional.org.apache.harmony.test.func.jit.HLO.peel.LoopVar2
> > Timeout
> > > functional.org.apache.harmony.test.func.jit.HLO.peel.TryCatch4
> > Timeout
> > > functional.org.apache.harmony.test.func.reg.vm.btest1355
> > Timeout
> > > functional.org.apache.harmony.test.func.jit.HLO.uce.unreachathrow
> > Timeout. Too many iterations - 20000000. the test pass in ~45s if I
> > decrease the number of iterations to 500000
> >
> > So, it looks like there is 1 regression - HARMONY-4857.
> >
>
> <SNIP>
>