You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@harmony.apache.org by "Volynets, Vera" <ve...@intel.com> on 2006/10/25 18:29:02 UTC
[drlvm][test]Exclude some tests from "build test" target, make 'build test' pass
Geir
Some tests launched by command "build test" fail.
The idea of "build test" is to run it before each commit. In this way you can catch regressions.
In order to effectively catch regressions, i.e. tests that started to fail after some change,
it's necessary to have 'build test' pass in a stable way. One of the ways to achieve stable state
is to exclude failing tests or tests which show unstable behavior.
So I analysed statistics of test runs on win ia32 platform and made up a list of tests to be excluded:
1) smoke
*** gc.LOS fails always on jitrino and interpreter, debug and release
2) kernel
*** java.lang.ClasssGenericsTest and
*** java.lang.ClassGenericsTest4 fail because of timeout, so I increase timeout in kernel.test.xml
*** java.lang.ObjectTest fail on interpreter with following message:
Testsuite: java.lang.ObjectTest
Tests run: 18, Failures: 1, Errors: 0, Time elapsed: 6.109 sec
Testcase: testWait1(java.lang.ObjectTest): FAILED
An InterruptedException must be thrown in test thread!
junit.framework.AssertionFailedError: An InterruptedException must be thrown in test thread!
See HARNONY-1966 issue with attached patch.
Vera Volynets
Intel Middleware Products Division
Re: [drlvm][test]Exclude some tests from "build test" target, make 'build test' pass
Posted by Pavel Ozhdikhin <pa...@gmail.com>.
On 10/26/06, Gregory Shimansky <gs...@gmail.com> wrote:
>
> On Wednesday 25 October 2006 21:05 Rana Dasgupta wrote:
> > The ideal way would be for acceptance tests like "build test" to always
> > pass and to catch and roll back the patch that breaks this invariant,
> > rather than to disable the tests. But I agree with Vera, it is important
> to
> > keep a running set up as acceptance tests, so disabling the well known
> > failures may be the only way until we fix the problems.
> >
> > I don't know that any of the tests are "unstable". These are
> implementation
> > bugs. gc.LOS is a bug in thread yielding by the apr Windows
> functionality.
> > The java.lang.ObjectTest also looks like an interpreter implementation
> > error.
>
> I wonder about gc.LOS, I remember there was a discussion, and I think
> there
> were patches too. But I cannot find any right now. Were there any fixes to
> the problem which made gc.LOS to work on WinXP?
This is Harmony-1933 <https://issues.apache.org/jira/browse/HARMONY-1933> issue
and it still lacks a patch.
Thanks,
Pavel
Hmm and java.lang.ObjectTest works for me both on windows and linux...
>
> --
> Gregory Shimansky, Intel Middleware Products Division
>
Re: [drlvm][test]Exclude some tests from "build test" target, make 'build test' pass
Posted by Gregory Shimansky <gs...@gmail.com>.
On Wednesday 25 October 2006 21:05 Rana Dasgupta wrote:
> The ideal way would be for acceptance tests like "build test" to always
> pass and to catch and roll back the patch that breaks this invariant,
> rather than to disable the tests. But I agree with Vera, it is important to
> keep a running set up as acceptance tests, so disabling the well known
> failures may be the only way until we fix the problems.
>
> I don't know that any of the tests are "unstable". These are implementation
> bugs. gc.LOS is a bug in thread yielding by the apr Windows functionality.
> The java.lang.ObjectTest also looks like an interpreter implementation
> error.
I wonder about gc.LOS, I remember there was a discussion, and I think there
were patches too. But I cannot find any right now. Were there any fixes to
the problem which made gc.LOS to work on WinXP?
Hmm and java.lang.ObjectTest works for me both on windows and linux...
--
Gregory Shimansky, Intel Middleware Products Division
Re: [drlvm][test]Exclude some tests from "build test" target, make
'build test' pass
Posted by Gregory Shimansky <gs...@gmail.com>.
Elena Semukhina wrote:
> On 11/17/06, Gregory Shimansky <gs...@gmail.com> wrote:
>>
>> Elena Semukhina wrote:
>> > On 11/16/06, Gregory Shimansky <gs...@gmail.com> wrote:
>> >>
>> >> Elena Semukhina wrote:
>> >> > As Gregory mentioned ThreadTest.testJoinlongint() failure he
>> observed
>> >> > yesterday, I filed a
>> http://issues.apache.org/jira/browse/HARMONY-2204
>> >> > issue
>> >> > for that. I saw this failure quite long ago but cannot reproduce it
>> >> today
>> >> > trying different platforms/EMs. When the test failed, it reported
>> that
>> >> > actual wait time is less than required.
>> >> >
>> >> > The spec for join() reads: "Waits at most millis milliseconds plus
>> >> > nanosnanoseconds for this thread to die. " There is an obvious
>> >> > misprint here:
>> >> > there should be "at least" but not "at most" and this was mentioned
>> in
>> >> >
>> >> > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4132653
>> >> > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5068368
>> >> >
>> >> > On the other hand, in classlib ThreadTest I saw that they agree with
>> >> > Thread.sleep(1000) duration >= 800 and join(100, 999999)
>> duration <=
>> >> 300.
>> >> >
>> >> > Can the current DRLVM guarantee that it does not exit join() earlier
>> >> than
>> >> > expected?
>> >> > If not, we should make the test more tolerable to timing.
>> >>
>> >> There is also an unstable ObjectTest.testWaitlongint which fails once
>> in
>> >> a while (about once in 50 - 100 runs). It also seems to wait a bit
>> less
>> >> than required. I think there is also a bug in the test, nanos
>> should be
>> >> 500000, not 500. But fixing this doesn't help, test still fails on
>> >> windows.
>> >
>> >
>> > Gregory, why do you think this is a bug? nanos can be any value between
>> 0
>> > and 999999. Actually there is a bug in thread_native_condvar.c which
>>
>> Yes it can. But I think that comparison at the end of the test that
>> "finish - start + 1 > millis" mean that it should be 1 more millisecond
>> of waiting. To do this it was intended to wait for 500 and a half =
>> 500.5 milliseconds, which means that nanos should be 500000, not 500.
>>
>> Correct my if I am wrong about the original test intension.
>
>
> I'm not an author of the test but I think we should wait one more
> millisecond even if we ask to wait 500.0005 ms because a millisecond is a
> minimal available time unit. I expect that the time is rounded up rather
> than towards the nearest vlaue. Am I wrong?
Well it appears that on windows assertion in the test fails once in a
while. The time of waiting finish - start + 1 = 500, so it is not
strictly greater than millis = 500.
I didn't try to run the test multiple times after your patch to APR.
I'll let you know if I get the failures again.
--
Gregory
Re: [drlvm][test]Exclude some tests from "build test" target, make 'build test' pass
Posted by Elena Semukhina <el...@gmail.com>.
On 11/17/06, Gregory Shimansky <gs...@gmail.com> wrote:
>
> Elena Semukhina wrote:
> > On 11/16/06, Gregory Shimansky <gs...@gmail.com> wrote:
> >>
> >> Elena Semukhina wrote:
> >> > As Gregory mentioned ThreadTest.testJoinlongint() failure he observed
> >> > yesterday, I filed a
> http://issues.apache.org/jira/browse/HARMONY-2204
> >> > issue
> >> > for that. I saw this failure quite long ago but cannot reproduce it
> >> today
> >> > trying different platforms/EMs. When the test failed, it reported
> that
> >> > actual wait time is less than required.
> >> >
> >> > The spec for join() reads: "Waits at most millis milliseconds plus
> >> > nanosnanoseconds for this thread to die. " There is an obvious
> >> > misprint here:
> >> > there should be "at least" but not "at most" and this was mentioned
> in
> >> >
> >> > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4132653
> >> > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5068368
> >> >
> >> > On the other hand, in classlib ThreadTest I saw that they agree with
> >> > Thread.sleep(1000) duration >= 800 and join(100, 999999) duration <=
> >> 300.
> >> >
> >> > Can the current DRLVM guarantee that it does not exit join() earlier
> >> than
> >> > expected?
> >> > If not, we should make the test more tolerable to timing.
> >>
> >> There is also an unstable ObjectTest.testWaitlongint which fails once
> in
> >> a while (about once in 50 - 100 runs). It also seems to wait a bit less
> >> than required. I think there is also a bug in the test, nanos should be
> >> 500000, not 500. But fixing this doesn't help, test still fails on
> >> windows.
> >
> >
> > Gregory, why do you think this is a bug? nanos can be any value between
> 0
> > and 999999. Actually there is a bug in thread_native_condvar.c which
>
> Yes it can. But I think that comparison at the end of the test that
> "finish - start + 1 > millis" mean that it should be 1 more millisecond
> of waiting. To do this it was intended to wait for 500 and a half =
> 500.5 milliseconds, which means that nanos should be 500000, not 500.
>
> Correct my if I am wrong about the original test intension.
I'm not an author of the test but I think we should wait one more
millisecond even if we ask to wait 500.0005 ms because a millisecond is a
minimal available time unit. I expect that the time is rounded up rather
than towards the nearest vlaue. Am I wrong?
> ignores
> > any nanos < 1000 when converting them to microseconds.
> >
> > The spec for Object.wait(long) reads that waiting lasts until:
> > ...
> > The specified amount of real time has elapsed, more or less.
> >
> > It does not say neither "at most" nor "at least". I think we can fix
> both
> > tests so that they could wait a little bit less than it is now expected
> and
> > print the values at error messages. I attached a patch to HARMONY-2204.
> > Please take a look and commit if it is OK. Then we can see if the tests
> > still fail.
>
> --
> Gregory
>
>
--
Thanks,
Elena
Re: [drlvm][test]Exclude some tests from "build test" target, make
'build test' pass
Posted by Gregory Shimansky <gs...@gmail.com>.
Elena Semukhina wrote:
> On 11/16/06, Gregory Shimansky <gs...@gmail.com> wrote:
>>
>> Elena Semukhina wrote:
>> > As Gregory mentioned ThreadTest.testJoinlongint() failure he observed
>> > yesterday, I filed a http://issues.apache.org/jira/browse/HARMONY-2204
>> > issue
>> > for that. I saw this failure quite long ago but cannot reproduce it
>> today
>> > trying different platforms/EMs. When the test failed, it reported that
>> > actual wait time is less than required.
>> >
>> > The spec for join() reads: "Waits at most millis milliseconds plus
>> > nanosnanoseconds for this thread to die. " There is an obvious
>> > misprint here:
>> > there should be "at least" but not "at most" and this was mentioned in
>> >
>> > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4132653
>> > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5068368
>> >
>> > On the other hand, in classlib ThreadTest I saw that they agree with
>> > Thread.sleep(1000) duration >= 800 and join(100, 999999) duration <=
>> 300.
>> >
>> > Can the current DRLVM guarantee that it does not exit join() earlier
>> than
>> > expected?
>> > If not, we should make the test more tolerable to timing.
>>
>> There is also an unstable ObjectTest.testWaitlongint which fails once in
>> a while (about once in 50 - 100 runs). It also seems to wait a bit less
>> than required. I think there is also a bug in the test, nanos should be
>> 500000, not 500. But fixing this doesn't help, test still fails on
>> windows.
>
>
> Gregory, why do you think this is a bug? nanos can be any value between 0
> and 999999. Actually there is a bug in thread_native_condvar.c which
Yes it can. But I think that comparison at the end of the test that
"finish - start + 1 > millis" mean that it should be 1 more millisecond
of waiting. To do this it was intended to wait for 500 and a half =
500.5 milliseconds, which means that nanos should be 500000, not 500.
Correct my if I am wrong about the original test intension.
> ignores
> any nanos < 1000 when converting them to microseconds.
>
> The spec for Object.wait(long) reads that waiting lasts until:
> ...
> The specified amount of real time has elapsed, more or less.
>
> It does not say neither "at most" nor "at least". I think we can fix both
> tests so that they could wait a little bit less than it is now expected and
> print the values at error messages. I attached a patch to HARMONY-2204.
> Please take a look and commit if it is OK. Then we can see if the tests
> still fail.
--
Gregory
Re: [drlvm][test]Exclude some tests from "build test" target, make 'build test' pass
Posted by Elena Semukhina <el...@gmail.com>.
On 11/16/06, Gregory Shimansky <gs...@gmail.com> wrote:
>
> Elena Semukhina wrote:
> > As Gregory mentioned ThreadTest.testJoinlongint() failure he observed
> > yesterday, I filed a http://issues.apache.org/jira/browse/HARMONY-2204
> > issue
> > for that. I saw this failure quite long ago but cannot reproduce it
> today
> > trying different platforms/EMs. When the test failed, it reported that
> > actual wait time is less than required.
> >
> > The spec for join() reads: "Waits at most millis milliseconds plus
> > nanosnanoseconds for this thread to die. " There is an obvious
> > misprint here:
> > there should be "at least" but not "at most" and this was mentioned in
> >
> > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4132653
> > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5068368
> >
> > On the other hand, in classlib ThreadTest I saw that they agree with
> > Thread.sleep(1000) duration >= 800 and join(100, 999999) duration <=
> 300.
> >
> > Can the current DRLVM guarantee that it does not exit join() earlier
> than
> > expected?
> > If not, we should make the test more tolerable to timing.
>
> There is also an unstable ObjectTest.testWaitlongint which fails once in
> a while (about once in 50 - 100 runs). It also seems to wait a bit less
> than required. I think there is also a bug in the test, nanos should be
> 500000, not 500. But fixing this doesn't help, test still fails on
> windows.
Gregory, why do you think this is a bug? nanos can be any value between 0
and 999999. Actually there is a bug in thread_native_condvar.c which ignores
any nanos < 1000 when converting them to microseconds.
The spec for Object.wait(long) reads that waiting lasts until:
...
The specified amount of real time has elapsed, more or less.
It does not say neither "at most" nor "at least". I think we can fix both
tests so that they could wait a little bit less than it is now expected and
print the values at error messages. I attached a patch to HARMONY-2204.
Please take a look and commit if it is OK. Then we can see if the tests
still fail.
I checked the APR function and underlying win API functions, the
> System.currentTimeMillis is measured with microseconds accuracy. The
> Thread.join and Thread.wait end up with WaitForSingleObject function
> which accepts an argument in milliseconds. So I cannot understand why
> the wait/join time is less than required.
>
> --
> Gregory
>
>
--
Thanks,
Elena
Re: [drlvm][test]Exclude some tests from "build test" target, make
'build test' pass
Posted by Gregory Shimansky <gs...@gmail.com>.
Elena Semukhina wrote:
> As Gregory mentioned ThreadTest.testJoinlongint() failure he observed
> yesterday, I filed a http://issues.apache.org/jira/browse/HARMONY-2204
> issue
> for that. I saw this failure quite long ago but cannot reproduce it today
> trying different platforms/EMs. When the test failed, it reported that
> actual wait time is less than required.
>
> The spec for join() reads: "Waits at most millis milliseconds plus
> nanosnanoseconds for this thread to die. " There is an obvious
> misprint here:
> there should be "at least" but not "at most" and this was mentioned in
>
> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4132653
> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5068368
>
> On the other hand, in classlib ThreadTest I saw that they agree with
> Thread.sleep(1000) duration >= 800 and join(100, 999999) duration <= 300.
>
> Can the current DRLVM guarantee that it does not exit join() earlier than
> expected?
> If not, we should make the test more tolerable to timing.
There is also an unstable ObjectTest.testWaitlongint which fails once in
a while (about once in 50 - 100 runs). It also seems to wait a bit less
than required. I think there is also a bug in the test, nanos should be
500000, not 500. But fixing this doesn't help, test still fails on windows.
I checked the APR function and underlying win API functions, the
System.currentTimeMillis is measured with microseconds accuracy. The
Thread.join and Thread.wait end up with WaitForSingleObject function
which accepts an argument in milliseconds. So I cannot understand why
the wait/join time is less than required.
--
Gregory
Re: [drlvm][test]Exclude some tests from "build test" target, make 'build test' pass
Posted by Elena Semukhina <el...@gmail.com>.
As Gregory mentioned ThreadTest.testJoinlongint() failure he observed
yesterday, I filed a http://issues.apache.org/jira/browse/HARMONY-2204 issue
for that. I saw this failure quite long ago but cannot reproduce it today
trying different platforms/EMs. When the test failed, it reported that
actual wait time is less than required.
The spec for join() reads: "Waits at most millis milliseconds plus
nanosnanoseconds for this thread to die. " There is an obvious
misprint here:
there should be "at least" but not "at most" and this was mentioned in
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4132653
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5068368
On the other hand, in classlib ThreadTest I saw that they agree with
Thread.sleep(1000) duration >= 800 and join(100, 999999) duration <= 300.
Can the current DRLVM guarantee that it does not exit join() earlier than
expected?
If not, we should make the test more tolerable to timing.
On 11/16/06, Elena Semukhina <el...@gmail.com> wrote:
>
> Thank you, Gregory!
>
> On 11/15/06, Gregory Shimansky <gs...@gmail.com> wrote:
> >
> > Elena Semukhina wrote:
> > > I've found one more issue in the kernel ThreadTest. Filed
> > > https://issues.apache.org/jira/browse/HARMONY-2193.
> > >
> > > Alexey, please take a look!
> >
> > I am not an Alexey, but I assigned this JIRA to myself. The testcase
> > seems to be working both on linux and windows, I committed the patch.
> >
> > > On 11/14/06, Alexey Varlamov <al...@gmail.com> wrote:
> > >>
> > >> 2006/11/14, Gregory Shimansky < gshimansky@gmail.com>:
> > >> > On Monday 13 November 2006 15:51 Elena Semukhina wrote:
> > >> > > On 10/26/06, Elena Semukhina <elena.semukhina@gmail.com > wrote:
> > >> > > > After H-1823 has been committed, I still see intermittent
> > failures
> > >> of
> > >> > > > drlvm kernel ThreadTest. Normally this test passes but today I
> > got
> > >> > > > ThreadTest.testInterrupt_Sleeping() and testGetStateBlocked()
> > >> failures.
> > >> > > > There is H-1789 for the first issue but it is not reprodiced
> > with
> > >> the
> > >> > > > attached test anymore. For the second test I filed H-1876 to
> > make
> > >> the
> > >> > > > test more stable. The patch has been committed recently but the
> > >> test
> > >> > > > still may fail. I'd like someone to review the above mentioned
> > >> tests
> > >> to
> > >> > > > be sure they are correct otherwise we need to investigate
> > failures
> > >> and
> > >> > > > file JIRA's before the tests are exclued.
> > >> > >
> > >> > > I looked at ThreadTest again and found one more incorrectness in
> > >> > > testGetStateBlocked(): it does not wait for the thread's run()
> > method
> > >> > > actually starts. I filed
> > >> https://issues.apache.org/jira/browse/HARMONY-2166
> > >> > > and attached the patch. I ran the test more than 100 times and
> > did
> > >> not
> > >> see
> > >> > > a failure.
> > >> >
> > >> > Cool! I'd like to see this patch applied (in case it really helps)
> > >> ASAP.
> > >> > Alexey V, please don't hold it for long. I would really like to see
> > all
> > >> > acceptance to pass again on all platforms and then we'll maintain
> > no
> > >> > regression state.
> > >>
> > >> Verified and applied (at r474672). BTW, seems we are very close to
> > >> 100% pass rate for classlib tests on DRLVM. The wiki status page [1]
> > >> lists just few pending issues, hopefully we'll fix them all in a
> > >> couple of days.
> > >>
> > >> [1] http://wiki.apache.org/harmony/Unit_Tests_Pass_on_DRLVM
> > >>
> > >
> > >
> > >
> >
> >
> > --
> > Gregory
> >
> >
>
>
> --
> Thanks,
> Elena
--
Thanks,
Elena
Re: [drlvm][test]Exclude some tests from "build test" target, make 'build test' pass
Posted by Elena Semukhina <el...@gmail.com>.
Thank you, Gregory!
On 11/15/06, Gregory Shimansky <gs...@gmail.com> wrote:
>
> Elena Semukhina wrote:
> > I've found one more issue in the kernel ThreadTest. Filed
> > https://issues.apache.org/jira/browse/HARMONY-2193.
> >
> > Alexey, please take a look!
>
> I am not an Alexey, but I assigned this JIRA to myself. The testcase
> seems to be working both on linux and windows, I committed the patch.
>
> > On 11/14/06, Alexey Varlamov <al...@gmail.com> wrote:
> >>
> >> 2006/11/14, Gregory Shimansky <gs...@gmail.com>:
> >> > On Monday 13 November 2006 15:51 Elena Semukhina wrote:
> >> > > On 10/26/06, Elena Semukhina <el...@gmail.com> wrote:
> >> > > > After H-1823 has been committed, I still see intermittent
> failures
> >> of
> >> > > > drlvm kernel ThreadTest. Normally this test passes but today I
> got
> >> > > > ThreadTest.testInterrupt_Sleeping() and testGetStateBlocked()
> >> failures.
> >> > > > There is H-1789 for the first issue but it is not reprodiced with
> >> the
> >> > > > attached test anymore. For the second test I filed H-1876 to make
> >> the
> >> > > > test more stable. The patch has been committed recently but the
> >> test
> >> > > > still may fail. I'd like someone to review the above mentioned
> >> tests
> >> to
> >> > > > be sure they are correct otherwise we need to investigate
> failures
> >> and
> >> > > > file JIRA's before the tests are exclued.
> >> > >
> >> > > I looked at ThreadTest again and found one more incorrectness in
> >> > > testGetStateBlocked(): it does not wait for the thread's run()
> method
> >> > > actually starts. I filed
> >> https://issues.apache.org/jira/browse/HARMONY-2166
> >> > > and attached the patch. I ran the test more than 100 times and did
> >> not
> >> see
> >> > > a failure.
> >> >
> >> > Cool! I'd like to see this patch applied (in case it really helps)
> >> ASAP.
> >> > Alexey V, please don't hold it for long. I would really like to see
> all
> >> > acceptance to pass again on all platforms and then we'll maintain no
> >> > regression state.
> >>
> >> Verified and applied (at r474672). BTW, seems we are very close to
> >> 100% pass rate for classlib tests on DRLVM. The wiki status page [1]
> >> lists just few pending issues, hopefully we'll fix them all in a
> >> couple of days.
> >>
> >> [1] http://wiki.apache.org/harmony/Unit_Tests_Pass_on_DRLVM
> >>
> >
> >
> >
>
>
> --
> Gregory
>
>
--
Thanks,
Elena
Re: [drlvm][test]Exclude some tests from "build test" target, make
'build test' pass
Posted by Gregory Shimansky <gs...@gmail.com>.
Elena Semukhina wrote:
> I've found one more issue in the kernel ThreadTest. Filed
> https://issues.apache.org/jira/browse/HARMONY-2193.
>
> Alexey, please take a look!
I am not an Alexey, but I assigned this JIRA to myself. The testcase
seems to be working both on linux and windows, I committed the patch.
> On 11/14/06, Alexey Varlamov <al...@gmail.com> wrote:
>>
>> 2006/11/14, Gregory Shimansky <gs...@gmail.com>:
>> > On Monday 13 November 2006 15:51 Elena Semukhina wrote:
>> > > On 10/26/06, Elena Semukhina <el...@gmail.com> wrote:
>> > > > After H-1823 has been committed, I still see intermittent failures
>> of
>> > > > drlvm kernel ThreadTest. Normally this test passes but today I got
>> > > > ThreadTest.testInterrupt_Sleeping() and testGetStateBlocked()
>> failures.
>> > > > There is H-1789 for the first issue but it is not reprodiced with
>> the
>> > > > attached test anymore. For the second test I filed H-1876 to make
>> the
>> > > > test more stable. The patch has been committed recently but the
>> test
>> > > > still may fail. I'd like someone to review the above mentioned
>> tests
>> to
>> > > > be sure they are correct otherwise we need to investigate failures
>> and
>> > > > file JIRA's before the tests are exclued.
>> > >
>> > > I looked at ThreadTest again and found one more incorrectness in
>> > > testGetStateBlocked(): it does not wait for the thread's run() method
>> > > actually starts. I filed
>> https://issues.apache.org/jira/browse/HARMONY-2166
>> > > and attached the patch. I ran the test more than 100 times and did
>> not
>> see
>> > > a failure.
>> >
>> > Cool! I'd like to see this patch applied (in case it really helps)
>> ASAP.
>> > Alexey V, please don't hold it for long. I would really like to see all
>> > acceptance to pass again on all platforms and then we'll maintain no
>> > regression state.
>>
>> Verified and applied (at r474672). BTW, seems we are very close to
>> 100% pass rate for classlib tests on DRLVM. The wiki status page [1]
>> lists just few pending issues, hopefully we'll fix them all in a
>> couple of days.
>>
>> [1] http://wiki.apache.org/harmony/Unit_Tests_Pass_on_DRLVM
>>
>
>
>
--
Gregory
Re: [drlvm][test]Exclude some tests from "build test" target, make 'build test' pass
Posted by Elena Semukhina <el...@gmail.com>.
I've found one more issue in the kernel ThreadTest. Filed
https://issues.apache.org/jira/browse/HARMONY-2193.
Alexey, please take a look!
On 11/14/06, Alexey Varlamov <al...@gmail.com> wrote:
>
> 2006/11/14, Gregory Shimansky <gs...@gmail.com>:
> > On Monday 13 November 2006 15:51 Elena Semukhina wrote:
> > > On 10/26/06, Elena Semukhina <el...@gmail.com> wrote:
> > > > After H-1823 has been committed, I still see intermittent failures
> of
> > > > drlvm kernel ThreadTest. Normally this test passes but today I got
> > > > ThreadTest.testInterrupt_Sleeping() and testGetStateBlocked()
> failures.
> > > > There is H-1789 for the first issue but it is not reprodiced with
> the
> > > > attached test anymore. For the second test I filed H-1876 to make
> the
> > > > test more stable. The patch has been committed recently but the test
> > > > still may fail. I'd like someone to review the above mentioned tests
> to
> > > > be sure they are correct otherwise we need to investigate failures
> and
> > > > file JIRA's before the tests are exclued.
> > >
> > > I looked at ThreadTest again and found one more incorrectness in
> > > testGetStateBlocked(): it does not wait for the thread's run() method
> > > actually starts. I filed
> https://issues.apache.org/jira/browse/HARMONY-2166
> > > and attached the patch. I ran the test more than 100 times and did not
> see
> > > a failure.
> >
> > Cool! I'd like to see this patch applied (in case it really helps) ASAP.
> > Alexey V, please don't hold it for long. I would really like to see all
> > acceptance to pass again on all platforms and then we'll maintain no
> > regression state.
>
> Verified and applied (at r474672). BTW, seems we are very close to
> 100% pass rate for classlib tests on DRLVM. The wiki status page [1]
> lists just few pending issues, hopefully we'll fix them all in a
> couple of days.
>
> [1] http://wiki.apache.org/harmony/Unit_Tests_Pass_on_DRLVM
>
--
Thanks,
Elena
Re: [drlvm][test]Exclude some tests from "build test" target, make 'build test' pass
Posted by Alexey Varlamov <al...@gmail.com>.
2006/11/14, Gregory Shimansky <gs...@gmail.com>:
> On Monday 13 November 2006 15:51 Elena Semukhina wrote:
> > On 10/26/06, Elena Semukhina <el...@gmail.com> wrote:
> > > After H-1823 has been committed, I still see intermittent failures of
> > > drlvm kernel ThreadTest. Normally this test passes but today I got
> > > ThreadTest.testInterrupt_Sleeping() and testGetStateBlocked() failures.
> > > There is H-1789 for the first issue but it is not reprodiced with the
> > > attached test anymore. For the second test I filed H-1876 to make the
> > > test more stable. The patch has been committed recently but the test
> > > still may fail. I'd like someone to review the above mentioned tests to
> > > be sure they are correct otherwise we need to investigate failures and
> > > file JIRA's before the tests are exclued.
> >
> > I looked at ThreadTest again and found one more incorrectness in
> > testGetStateBlocked(): it does not wait for the thread's run() method
> > actually starts. I filed https://issues.apache.org/jira/browse/HARMONY-2166
> > and attached the patch. I ran the test more than 100 times and did not see
> > a failure.
>
> Cool! I'd like to see this patch applied (in case it really helps) ASAP.
> Alexey V, please don't hold it for long. I would really like to see all
> acceptance to pass again on all platforms and then we'll maintain no
> regression state.
Verified and applied (at r474672). BTW, seems we are very close to
100% pass rate for classlib tests on DRLVM. The wiki status page [1]
lists just few pending issues, hopefully we'll fix them all in a
couple of days.
[1] http://wiki.apache.org/harmony/Unit_Tests_Pass_on_DRLVM
Re: [drlvm][test]Exclude some tests from "build test" target, make 'build test' pass
Posted by Gregory Shimansky <gs...@gmail.com>.
On Monday 13 November 2006 15:51 Elena Semukhina wrote:
> On 10/26/06, Elena Semukhina <el...@gmail.com> wrote:
> > After H-1823 has been committed, I still see intermittent failures of
> > drlvm kernel ThreadTest. Normally this test passes but today I got
> > ThreadTest.testInterrupt_Sleeping() and testGetStateBlocked() failures.
> > There is H-1789 for the first issue but it is not reprodiced with the
> > attached test anymore. For the second test I filed H-1876 to make the
> > test more stable. The patch has been committed recently but the test
> > still may fail. I'd like someone to review the above mentioned tests to
> > be sure they are correct otherwise we need to investigate failures and
> > file JIRA's before the tests are exclued.
>
> I looked at ThreadTest again and found one more incorrectness in
> testGetStateBlocked(): it does not wait for the thread's run() method
> actually starts. I filed https://issues.apache.org/jira/browse/HARMONY-2166
> and attached the patch. I ran the test more than 100 times and did not see
> a failure.
Cool! I'd like to see this patch applied (in case it really helps) ASAP.
Alexey V, please don't hold it for long. I would really like to see all
acceptance to pass again on all platforms and then we'll maintain no
regression state.
--
Gregory Shimansky, Intel Middleware Products Division
Re: [drlvm][test]Exclude some tests from "build test" target, make 'build test' pass
Posted by Elena Semukhina <el...@gmail.com>.
On 10/26/06, Elena Semukhina <el...@gmail.com> wrote:
>
> After H-1823 has been committed, I still see intermittent failures of
> drlvm kernel ThreadTest. Normally this test passes but today I got
> ThreadTest.testInterrupt_Sleeping() and testGetStateBlocked() failures.
> There is H-1789 for the first issue but it is not reprodiced with the
> attached test anymore. For the second test I filed H-1876 to make the test
> more stable. The patch has been committed recently but the test still may
> fail. I'd like someone to review the above mentioned tests to be sure they
> are correct otherwise we need to investigate failures and file JIRA's before
> the tests are exclued.
>
I looked at ThreadTest again and found one more incorrectness in
testGetStateBlocked(): it does not wait for the thread's run() method
actually starts. I filed https://issues.apache.org/jira/browse/HARMONY-2166 and
attached the patch. I ran the test more than 100 times and did not see a
failure.
> Vera, could you please review the tests?
>
>
> On 10/26/06, Pavel Ozhdikhin <pa...@gmail.com> wrote:
> >
> > +1 to exclude failing tests for now and require that all remaining tests
> > must pass. Assuming some tests fail anyway cause people don't treat the
> > failures seriously. As soon as the bug causing the failure is fixed the
> > tests need to be unexcluded.
> >
> > Thanks,
> > Pavel
> >
> >
> > On 10/26/06, Rana Dasgupta <rdasgupt@gmail.com > wrote:
> > >
> > > The ideal way would be for acceptance tests like "build test" to
> > always
> > > pass
> > > and to catch and roll back the patch that breaks this invariant,
> > rather
> > > than
> > > to disable the tests. But I agree with Vera, it is important to keep a
> > > running set up as acceptance tests, so disabling the well known
> > failures
> > > may
> > > be the only way until we fix the problems.
> > >
> > > I don't know that any of the tests are "unstable". These are
> > > implementation
> > > bugs. gc.LOS is a bug in thread yielding by the apr Windows
> > functionality.
> > > The java.lang.ObjectTest also looks like an interpreter implementation
> > > error.
> > >
> > >
> > >
> > >
> > >
> > > > On 10/25/06, Volynets, Vera <ve...@intel.com> wrote:
> > > > >
> > > > > Geir
> > > > > Some tests launched by command "build test" fail.
> > > > > The idea of "build test" is to run it before each commit. In this
> > way
> > > > > you can catch regressions.
> > > > > In order to effectively catch regressions, i.e. tests that started
> > to
> > > > > fail after some change,
> > > > > it's necessary to have 'build test' pass in a stable way. One of
> > the
> > > > > ways to achieve stable state
> > > > > is to exclude failing tests or tests which show unstable behavior.
> > > > > So I analysed statistics of test runs on win ia32 platform and
> > made up
> > > a
> > > > > list of tests to be excluded:
> > > > > 1) smoke
> > > > > *** gc.LOS fails always on jitrino and interpreter, debug and
> > release
> > > > > 2) kernel
> > > > > *** java.lang.ClasssGenericsTest and
> > > > > *** java.lang.ClassGenericsTest4 fail because of timeout, so
> > > I increase
> > > > > timeout in kernel.test.xml
> > > > > *** java.lang.ObjectTest fail on interpreter with following
> > message:
> > > > > Testsuite: java.lang.ObjectTest
> > > > > Tests run: 18, Failures: 1, Errors: 0, Time elapsed: 6.109 sec
> > > > > Testcase: testWait1(java.lang.ObjectTest): FAILED
> > > > > An InterruptedException must be thrown in test thread!
> > > > > junit.framework.AssertionFailedError: An InterruptedException
> > must
> > > be
> > > > > thrown in test thread!
> > > > >
> > > > > See HARNONY-1966 issue with attached patch.
> > > > >
> > > > >
> > > > > Vera Volynets
> > > > > Intel Middleware ProductsDivision
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>
> --
> Thanks,
> Elena
--
Thanks,
Elena
Re: [drlvm][test]Exclude some tests from "build test" target, make
'build test' pass
Posted by "Geir Magnusson Jr." <ge...@pobox.com>.
Great - thanks!
Note that Salikh has offered an alternative - can you take a look?
Evgueni Brevnov wrote:
> The reason of gc.Free failure was identefied. Thanks to Salikh and
> Ivan! Here is the patch
> http://issues.apache.org/jira/browse/HARMONY-1978
>
> Thanks
> Evgueni
>
> On 10/26/06, Elena Semukhina <el...@gmail.com> wrote:
>> On 10/26/06, Gregory Shimansky <gs...@gmail.com> wrote:
>> >
>> > Elena Semukhina wrote:
>> > > After H-1823 has been committed, I still see intermittent failures of
>> > drlvm
>> > > kernel ThreadTest. Normally this test passes but today I got
>> > > ThreadTest.testInterrupt_Sleeping() and testGetStateBlocked()
>> failures.
>> > > There is H-1789 for the first issue but it is not reprodiced with the
>> > > attached test anymore. For the second test I filed H-1876 to make the
>> > test
>> > > more stable. The patch has been committed recently but the test still
>> > may
>> > > fail. I'd like someone to review the above mentioned tests to be sure
>> > they
>> > > are correct otherwise we need to investigate failures and file JIRA's
>> > > before
>> > > the tests are exclued.
>> >
>> > For me ThreadTest (testJoinlongint) always fails on win32 2003 server
>> > SP1 on P4 with HT, works ok on win32 XP and on different Linuxes. I
>> just
>> > checked and patch from 1876 is applied, but it fixes another test case,
>> > so it is not relevant.
>> >
>> > BTW I've encountered timeout on ClassGenericsTest and
>> ClassGenericsTest4
>> > tests on some slower machines, so it looks like they do require
>> > increased timeout. Now I agree completely with patch which Vera
>> attached
>> > in HARMONY-1966.
>>
>>
>> I don't agree that we should exclude the whole ObjectTest (or
>> ThreadTest).
>> They contain dozens of test cases and if we have one test case stably or
>> intermittently failing we should file JIRA first and then comment out
>> this
>> test case so that other test cases could run. In JIRA comment we need to
>> note that the test was commented out and uncomment it at the same time
>> when
>> JIRA is fixed.
>>
>> --
>> > Gregory Shimansky, Intel Middleware Products Division
>> >
>> >
>>
>>
>> --
>> Thanks,
>> Elena
>>
>>
>
Re: [drlvm][test]Exclude some tests from "build test" target, make 'build test' pass
Posted by Evgueni Brevnov <ev...@gmail.com>.
The reason of gc.Free failure was identefied. Thanks to Salikh and
Ivan! Here is the patch
http://issues.apache.org/jira/browse/HARMONY-1978
Thanks
Evgueni
On 10/26/06, Elena Semukhina <el...@gmail.com> wrote:
> On 10/26/06, Gregory Shimansky <gs...@gmail.com> wrote:
> >
> > Elena Semukhina wrote:
> > > After H-1823 has been committed, I still see intermittent failures of
> > drlvm
> > > kernel ThreadTest. Normally this test passes but today I got
> > > ThreadTest.testInterrupt_Sleeping() and testGetStateBlocked() failures.
> > > There is H-1789 for the first issue but it is not reprodiced with the
> > > attached test anymore. For the second test I filed H-1876 to make the
> > test
> > > more stable. The patch has been committed recently but the test still
> > may
> > > fail. I'd like someone to review the above mentioned tests to be sure
> > they
> > > are correct otherwise we need to investigate failures and file JIRA's
> > > before
> > > the tests are exclued.
> >
> > For me ThreadTest (testJoinlongint) always fails on win32 2003 server
> > SP1 on P4 with HT, works ok on win32 XP and on different Linuxes. I just
> > checked and patch from 1876 is applied, but it fixes another test case,
> > so it is not relevant.
> >
> > BTW I've encountered timeout on ClassGenericsTest and ClassGenericsTest4
> > tests on some slower machines, so it looks like they do require
> > increased timeout. Now I agree completely with patch which Vera attached
> > in HARMONY-1966.
>
>
> I don't agree that we should exclude the whole ObjectTest (or ThreadTest).
> They contain dozens of test cases and if we have one test case stably or
> intermittently failing we should file JIRA first and then comment out this
> test case so that other test cases could run. In JIRA comment we need to
> note that the test was commented out and uncomment it at the same time when
> JIRA is fixed.
>
> --
> > Gregory Shimansky, Intel Middleware Products Division
> >
> >
>
>
> --
> Thanks,
> Elena
>
>
Re: [drlvm][test]Exclude some tests from "build test" target, make 'build test' pass
Posted by Elena Semukhina <el...@gmail.com>.
On 10/26/06, Gregory Shimansky <gs...@gmail.com> wrote:
>
> Elena Semukhina wrote:
> > After H-1823 has been committed, I still see intermittent failures of
> drlvm
> > kernel ThreadTest. Normally this test passes but today I got
> > ThreadTest.testInterrupt_Sleeping() and testGetStateBlocked() failures.
> > There is H-1789 for the first issue but it is not reprodiced with the
> > attached test anymore. For the second test I filed H-1876 to make the
> test
> > more stable. The patch has been committed recently but the test still
> may
> > fail. I'd like someone to review the above mentioned tests to be sure
> they
> > are correct otherwise we need to investigate failures and file JIRA's
> > before
> > the tests are exclued.
>
> For me ThreadTest (testJoinlongint) always fails on win32 2003 server
> SP1 on P4 with HT, works ok on win32 XP and on different Linuxes. I just
> checked and patch from 1876 is applied, but it fixes another test case,
> so it is not relevant.
>
> BTW I've encountered timeout on ClassGenericsTest and ClassGenericsTest4
> tests on some slower machines, so it looks like they do require
> increased timeout. Now I agree completely with patch which Vera attached
> in HARMONY-1966.
I don't agree that we should exclude the whole ObjectTest (or ThreadTest).
They contain dozens of test cases and if we have one test case stably or
intermittently failing we should file JIRA first and then comment out this
test case so that other test cases could run. In JIRA comment we need to
note that the test was commented out and uncomment it at the same time when
JIRA is fixed.
--
> Gregory Shimansky, Intel Middleware Products Division
>
>
--
Thanks,
Elena
Re: [drlvm][test]Exclude some tests from "build test" target, make
'build test' pass
Posted by Gregory Shimansky <gs...@gmail.com>.
Elena Semukhina wrote:
> After H-1823 has been committed, I still see intermittent failures of drlvm
> kernel ThreadTest. Normally this test passes but today I got
> ThreadTest.testInterrupt_Sleeping() and testGetStateBlocked() failures.
> There is H-1789 for the first issue but it is not reprodiced with the
> attached test anymore. For the second test I filed H-1876 to make the test
> more stable. The patch has been committed recently but the test still may
> fail. I'd like someone to review the above mentioned tests to be sure they
> are correct otherwise we need to investigate failures and file JIRA's
> before
> the tests are exclued.
For me ThreadTest (testJoinlongint) always fails on win32 2003 server
SP1 on P4 with HT, works ok on win32 XP and on different Linuxes. I just
checked and patch from 1876 is applied, but it fixes another test case,
so it is not relevant.
BTW I've encountered timeout on ClassGenericsTest and ClassGenericsTest4
tests on some slower machines, so it looks like they do require
increased timeout. Now I agree completely with patch which Vera attached
in HARMONY-1966.
--
Gregory Shimansky, Intel Middleware Products Division
Re: [drlvm][test]Exclude some tests from "build test" target, make 'build test' pass
Posted by Elena Semukhina <el...@gmail.com>.
After H-1823 has been committed, I still see intermittent failures of drlvm
kernel ThreadTest. Normally this test passes but today I got
ThreadTest.testInterrupt_Sleeping() and testGetStateBlocked() failures.
There is H-1789 for the first issue but it is not reprodiced with the
attached test anymore. For the second test I filed H-1876 to make the test
more stable. The patch has been committed recently but the test still may
fail. I'd like someone to review the above mentioned tests to be sure they
are correct otherwise we need to investigate failures and file JIRA's before
the tests are exclued.
Vera, could you please review the tests?
On 10/26/06, Pavel Ozhdikhin <pa...@gmail.com> wrote:
>
> +1 to exclude failing tests for now and require that all remaining tests
> must pass. Assuming some tests fail anyway cause people don't treat the
> failures seriously. As soon as the bug causing the failure is fixed the
> tests need to be unexcluded.
>
> Thanks,
> Pavel
>
>
> On 10/26/06, Rana Dasgupta <rd...@gmail.com> wrote:
> >
> > The ideal way would be for acceptance tests like "build test" to always
> > pass
> > and to catch and roll back the patch that breaks this invariant, rather
> > than
> > to disable the tests. But I agree with Vera, it is important to keep a
> > running set up as acceptance tests, so disabling the well known failures
> > may
> > be the only way until we fix the problems.
> >
> > I don't know that any of the tests are "unstable". These are
> > implementation
> > bugs. gc.LOS is a bug in thread yielding by the apr Windows
> functionality.
> > The java.lang.ObjectTest also looks like an interpreter implementation
> > error.
> >
> >
> >
> >
> >
> > > On 10/25/06, Volynets, Vera <ve...@intel.com> wrote:
> > > >
> > > > Geir
> > > > Some tests launched by command "build test" fail.
> > > > The idea of "build test" is to run it before each commit. In this
> way
> > > > you can catch regressions.
> > > > In order to effectively catch regressions, i.e. tests that started
> to
> > > > fail after some change,
> > > > it's necessary to have 'build test' pass in a stable way. One of the
> > > > ways to achieve stable state
> > > > is to exclude failing tests or tests which show unstable behavior.
> > > > So I analysed statistics of test runs on win ia32 platform and made
> up
> > a
> > > > list of tests to be excluded:
> > > > 1) smoke
> > > > *** gc.LOS fails always on jitrino and interpreter, debug and
> release
> > > > 2) kernel
> > > > *** java.lang.ClasssGenericsTest and
> > > > *** java.lang.ClassGenericsTest4 fail because of timeout, so
> > I increase
> > > > timeout in kernel.test.xml
> > > > *** java.lang.ObjectTest fail on interpreter with following message:
> > > > Testsuite: java.lang.ObjectTest
> > > > Tests run: 18, Failures: 1, Errors: 0, Time elapsed: 6.109 sec
> > > > Testcase: testWait1(java.lang.ObjectTest): FAILED
> > > > An InterruptedException must be thrown in test thread!
> > > > junit.framework.AssertionFailedError: An InterruptedException
> must
> > be
> > > > thrown in test thread!
> > > >
> > > > See HARNONY-1966 issue with attached patch.
> > > >
> > > >
> > > > Vera Volynets
> > > > Intel Middleware ProductsDivision
> > > >
> > >
> > >
> >
> >
>
>
--
Thanks,
Elena
Re: [drlvm][test]Exclude some tests from "build test" target, make 'build test' pass
Posted by Pavel Ozhdikhin <pa...@gmail.com>.
+1 to exclude failing tests for now and require that all remaining tests
must pass. Assuming some tests fail anyway cause people don't treat the
failures seriously. As soon as the bug causing the failure is fixed the
tests need to be unexcluded.
Thanks,
Pavel
On 10/26/06, Rana Dasgupta <rd...@gmail.com> wrote:
>
> The ideal way would be for acceptance tests like "build test" to always
> pass
> and to catch and roll back the patch that breaks this invariant, rather
> than
> to disable the tests. But I agree with Vera, it is important to keep a
> running set up as acceptance tests, so disabling the well known failures
> may
> be the only way until we fix the problems.
>
> I don't know that any of the tests are "unstable". These are
> implementation
> bugs. gc.LOS is a bug in thread yielding by the apr Windows functionality.
> The java.lang.ObjectTest also looks like an interpreter implementation
> error.
>
>
>
>
>
> > On 10/25/06, Volynets, Vera <ve...@intel.com> wrote:
> > >
> > > Geir
> > > Some tests launched by command "build test" fail.
> > > The idea of "build test" is to run it before each commit. In this way
> > > you can catch regressions.
> > > In order to effectively catch regressions, i.e. tests that started to
> > > fail after some change,
> > > it's necessary to have 'build test' pass in a stable way. One of the
> > > ways to achieve stable state
> > > is to exclude failing tests or tests which show unstable behavior.
> > > So I analysed statistics of test runs on win ia32 platform and made up
> a
> > > list of tests to be excluded:
> > > 1) smoke
> > > *** gc.LOS fails always on jitrino and interpreter, debug and release
> > > 2) kernel
> > > *** java.lang.ClasssGenericsTest and
> > > *** java.lang.ClassGenericsTest4 fail because of timeout, so
> I increase
> > > timeout in kernel.test.xml
> > > *** java.lang.ObjectTest fail on interpreter with following message:
> > > Testsuite: java.lang.ObjectTest
> > > Tests run: 18, Failures: 1, Errors: 0, Time elapsed: 6.109 sec
> > > Testcase: testWait1(java.lang.ObjectTest): FAILED
> > > An InterruptedException must be thrown in test thread!
> > > junit.framework.AssertionFailedError: An InterruptedException must
> be
> > > thrown in test thread!
> > >
> > > See HARNONY-1966 issue with attached patch.
> > >
> > >
> > > Vera Volynets
> > > Intel Middleware ProductsDivision
> > >
> >
> >
>
>
Re: [drlvm][test]Exclude some tests from "build test" target, make
'build test' pass
Posted by "Geir Magnusson Jr." <ge...@pobox.com>.
No argument from me. But we've never passed all, and would love to do so.
Much recent work has been trying to get to that point, and we did accept
the disruption of TM and Invocation API patches, with the idea to decide
to commit and then fix after to make progress.
geir
Rana Dasgupta wrote:
> The ideal way would be for acceptance tests like "build test" to always
> pass
> and to catch and roll back the patch that breaks this invariant, rather
> than
> to disable the tests. But I agree with Vera, it is important to keep a
> running set up as acceptance tests, so disabling the well known failures
> may
> be the only way until we fix the problems.
>
> I don't know that any of the tests are "unstable". These are implementation
> bugs. gc.LOS is a bug in thread yielding by the apr Windows functionality.
> The java.lang.ObjectTest also looks like an interpreter implementation
> error.
>
>
>
>
>
>> On 10/25/06, Volynets, Vera <ve...@intel.com> wrote:
>> >
>> > Geir
>> > Some tests launched by command "build test" fail.
>> > The idea of "build test" is to run it before each commit. In this way
>> > you can catch regressions.
>> > In order to effectively catch regressions, i.e. tests that started to
>> > fail after some change,
>> > it's necessary to have 'build test' pass in a stable way. One of the
>> > ways to achieve stable state
>> > is to exclude failing tests or tests which show unstable behavior.
>> > So I analysed statistics of test runs on win ia32 platform and made
>> up a
>> > list of tests to be excluded:
>> > 1) smoke
>> > *** gc.LOS fails always on jitrino and interpreter, debug and release
>> > 2) kernel
>> > *** java.lang.ClasssGenericsTest and
>> > *** java.lang.ClassGenericsTest4 fail because of timeout, so I
>> increase
>> > timeout in kernel.test.xml
>> > *** java.lang.ObjectTest fail on interpreter with following message:
>> > Testsuite: java.lang.ObjectTest
>> > Tests run: 18, Failures: 1, Errors: 0, Time elapsed: 6.109 sec
>> > Testcase: testWait1(java.lang.ObjectTest): FAILED
>> > An InterruptedException must be thrown in test thread!
>> > junit.framework.AssertionFailedError: An InterruptedException
>> must be
>> > thrown in test thread!
>> >
>> > See HARNONY-1966 issue with attached patch.
>> >
>> >
>> > Vera Volynets
>> > Intel Middleware ProductsDivision
>> >
>>
>>
>
Re: [drlvm][test]Exclude some tests from "build test" target, make 'build test' pass
Posted by Rana Dasgupta <rd...@gmail.com>.
The ideal way would be for acceptance tests like "build test" to always pass
and to catch and roll back the patch that breaks this invariant, rather than
to disable the tests. But I agree with Vera, it is important to keep a
running set up as acceptance tests, so disabling the well known failures may
be the only way until we fix the problems.
I don't know that any of the tests are "unstable". These are implementation
bugs. gc.LOS is a bug in thread yielding by the apr Windows functionality.
The java.lang.ObjectTest also looks like an interpreter implementation
error.
> On 10/25/06, Volynets, Vera <ve...@intel.com> wrote:
> >
> > Geir
> > Some tests launched by command "build test" fail.
> > The idea of "build test" is to run it before each commit. In this way
> > you can catch regressions.
> > In order to effectively catch regressions, i.e. tests that started to
> > fail after some change,
> > it's necessary to have 'build test' pass in a stable way. One of the
> > ways to achieve stable state
> > is to exclude failing tests or tests which show unstable behavior.
> > So I analysed statistics of test runs on win ia32 platform and made up a
> > list of tests to be excluded:
> > 1) smoke
> > *** gc.LOS fails always on jitrino and interpreter, debug and release
> > 2) kernel
> > *** java.lang.ClasssGenericsTest and
> > *** java.lang.ClassGenericsTest4 fail because of timeout, so I increase
> > timeout in kernel.test.xml
> > *** java.lang.ObjectTest fail on interpreter with following message:
> > Testsuite: java.lang.ObjectTest
> > Tests run: 18, Failures: 1, Errors: 0, Time elapsed: 6.109 sec
> > Testcase: testWait1(java.lang.ObjectTest): FAILED
> > An InterruptedException must be thrown in test thread!
> > junit.framework.AssertionFailedError: An InterruptedException must be
> > thrown in test thread!
> >
> > See HARNONY-1966 issue with attached patch.
> >
> >
> > Vera Volynets
> > Intel Middleware ProductsDivision
> >
>
>
Re: [drlvm][test]Exclude some tests from "build test" target, make 'build test' pass
Posted by Evgueni Brevnov <ev...@gmail.com>.
HI,
Do we have strong reasons to run all kernel tests in three different
modes before commit? Probably, it makes sense to choose two or three
most powerful tests and include them to smoke?
Evgueni
On 11/12/06, Pavel Pervov <pm...@gmail.com> wrote:
> In addition to excluding some tests, I would suggest removing
> smoke/classloader/ClassTestGetDeclaringClass as it is exact copy of kernel
> test with the same name.
> Also, smoke/classloader/ExceptionInInitializerTest must be returned to
> acceptance runs.
>
> Pavel.
>
> On 10/25/06, Volynets, Vera <ve...@intel.com> wrote:
> >
> > Geir
> > Some tests launched by command "build test" fail.
> > The idea of "build test" is to run it before each commit. In this way you
> > can catch regressions.
> > In order to effectively catch regressions, i.e. tests that started to fail
> > after some change,
> > it's necessary to have 'build test' pass in a stable way. One of the ways
> > to achieve stable state
> > is to exclude failing tests or tests which show unstable behavior.
> > So I analysed statistics of test runs on win ia32 platform and made up a
> > list of tests to be excluded:
> > 1) smoke
> > *** gc.LOS fails always on jitrino and interpreter, debug and release
> > 2) kernel
> > *** java.lang.ClasssGenericsTest and
> > *** java.lang.ClassGenericsTest4 fail because of timeout, so I increase
> > timeout in kernel.test.xml
> > *** java.lang.ObjectTest fail on interpreter with following message:
> > Testsuite: java.lang.ObjectTest
> > Tests run: 18, Failures: 1, Errors: 0, Time elapsed: 6.109 sec
> > Testcase: testWait1(java.lang.ObjectTest): FAILED
> > An InterruptedException must be thrown in test thread!
> > junit.framework.AssertionFailedError: An InterruptedException must be
> > thrown in test thread!
> >
> > See HARNONY-1966 issue with attached patch.
> >
> >
> > Vera Volynets
> > Intel Middleware ProductsDivision
> >
>
>
>
> --
> Pavel Pervov,
> Intel Enterprise Solutions Software Division
>
>
Re: [drlvm][test]Exclude some tests from "build test" target, make 'build test' pass
Posted by Pavel Pervov <pm...@gmail.com>.
In addition to excluding some tests, I would suggest removing
smoke/classloader/ClassTestGetDeclaringClass as it is exact copy of kernel
test with the same name.
Also, smoke/classloader/ExceptionInInitializerTest must be returned to
acceptance runs.
Pavel.
On 10/25/06, Volynets, Vera <ve...@intel.com> wrote:
>
> Geir
> Some tests launched by command "build test" fail.
> The idea of "build test" is to run it before each commit. In this way you
> can catch regressions.
> In order to effectively catch regressions, i.e. tests that started to fail
> after some change,
> it's necessary to have 'build test' pass in a stable way. One of the ways
> to achieve stable state
> is to exclude failing tests or tests which show unstable behavior.
> So I analysed statistics of test runs on win ia32 platform and made up a
> list of tests to be excluded:
> 1) smoke
> *** gc.LOS fails always on jitrino and interpreter, debug and release
> 2) kernel
> *** java.lang.ClasssGenericsTest and
> *** java.lang.ClassGenericsTest4 fail because of timeout, so I increase
> timeout in kernel.test.xml
> *** java.lang.ObjectTest fail on interpreter with following message:
> Testsuite: java.lang.ObjectTest
> Tests run: 18, Failures: 1, Errors: 0, Time elapsed: 6.109 sec
> Testcase: testWait1(java.lang.ObjectTest): FAILED
> An InterruptedException must be thrown in test thread!
> junit.framework.AssertionFailedError: An InterruptedException must be
> thrown in test thread!
>
> See HARNONY-1966 issue with attached patch.
>
>
> Vera Volynets
> Intel Middleware ProductsDivision
>
--
Pavel Pervov,
Intel Enterprise Solutions Software Division
Re: [drlvm][test]Exclude some tests from "build test" target, make
'build test' pass
Posted by "Geir Magnusson Jr." <ge...@pobox.com>.
Volynets, Vera wrote:
> Geir
> Some tests launched by command "build test" fail.
> The idea of "build test" is to run it before each commit. In this way you can catch regressions.
Yes, that is what I do
> In order to effectively catch regressions, i.e. tests that started to fail after some change,
> it's necessary to have 'build test' pass in a stable way. One of the ways to achieve stable state
> is to exclude failing tests or tests which show unstable behavior.
> So I analysed statistics of test runs on win ia32 platform and made up a list of tests to be excluded:
> 1) smoke
> *** gc.LOS fails always on jitrino and interpreter, debug and release
Not for me on linux.
> 2) kernel
> *** java.lang.ClasssGenericsTest and
> *** java.lang.ClassGenericsTest4 fail because of timeout, so I increase timeout in kernel.test.xml
> *** java.lang.ObjectTest fail on interpreter with following message:
> Testsuite: java.lang.ObjectTest
> Tests run: 18, Failures: 1, Errors: 0, Time elapsed: 6.109 sec
> Testcase: testWait1(java.lang.ObjectTest): FAILED
> An InterruptedException must be thrown in test thread!
> junit.framework.AssertionFailedError: An InterruptedException must be thrown in test thread!
>
> See HARNONY-1966 issue with attached patch.
Yay!
>
>
> Vera Volynets
> Intel Middleware Products Division
>
Re: [drlvm][test]Exclude some tests from "build test" target, make 'build test' pass
Posted by Rana Dasgupta <rd...@gmail.com>.
I'll check on 1830. Somehow the JIRA dropped my comments! I was debugging on
Windows since that was the original bug report.
An exception may be OK, since this is a resource intensive test, but it
should not crash when handling the exception. That part should have been
fixed. Also, we need to see RI behaviour on Linux. Anyway, I will recheck.
Thanks for closing the other two.
On 11/3/06, Geir Magnusson Jr. <ge...@pobox.com> wrote:
>
>
>
> Rana Dasgupta wrote:
> > Hi,
> > I did not want to start a new thread for this. Could someone please
> close
> > JIRA issues 1109, 1662, 1830? These have been fixed by recent changes in
> > the
> > stack and exception handling. In any case, none of them repro. Comments
> on
> > teh JIRA.
> >
>
> I closed 1109 and 1162, but I have a different problem with 1830 on
> linux - after 160 iterations or so of the test program, I get a
> j.n.SocketException - too many open files...
>
> geir
>
>
> > Thanks,
> > Rana
> >
> >
> > On 10/25/06, Volynets, Vera <ve...@intel.com> wrote:
> >>
> >> Geir
> >> Some tests launched by command "build test" fail.
> >> The idea of "build test" is to run it before each commit. In this way
> >> you
> >> can catch regressions.
> >> In order to effectively catch regressions, i.e. tests that started to
> >> fail
> >> after some change,
> >> it's necessary to have 'build test' pass in a stable way. One of the
> ways
> >> to achieve stable state
> >> is to exclude failing tests or tests which show unstable behavior.
> >> So I analysed statistics of test runs on win ia32 platform and made up
> a
> >> list of tests to be excluded:
> >> 1) smoke
> >> *** gc.LOS fails always on jitrino and interpreter, debug and release
> >> 2) kernel
> >> *** java.lang.ClasssGenericsTest and
> >> *** java.lang.ClassGenericsTest4 fail because of timeout, so
> I increase
> >> timeout in kernel.test.xml
> >> *** java.lang.ObjectTest fail on interpreter with following message:
> >> Testsuite: java.lang.ObjectTest
> >> Tests run: 18, Failures: 1, Errors: 0, Time elapsed: 6.109 sec
> >> Testcase: testWait1(java.lang.ObjectTest): FAILED
> >> An InterruptedException must be thrown in test thread!
> >> junit.framework.AssertionFailedError: An InterruptedException must
> be
> >> thrown in test thread!
> >>
> >> See HARNONY-1966 issue with attached patch.
> >>
> >>
> >> Vera Volynets
> >> Intel Middleware ProductsDivision
> >>
> >
>
Re: [drlvm][test]Exclude some tests from "build test" target, make
'build test' pass
Posted by "Geir Magnusson Jr." <ge...@pobox.com>.
Rana Dasgupta wrote:
> Hi,
> I did not want to start a new thread for this. Could someone please close
> JIRA issues 1109, 1662, 1830? These have been fixed by recent changes in
> the
> stack and exception handling. In any case, none of them repro. Comments on
> teh JIRA.
>
I closed 1109 and 1162, but I have a different problem with 1830 on
linux - after 160 iterations or so of the test program, I get a
j.n.SocketException - too many open files...
geir
> Thanks,
> Rana
>
>
> On 10/25/06, Volynets, Vera <ve...@intel.com> wrote:
>>
>> Geir
>> Some tests launched by command "build test" fail.
>> The idea of "build test" is to run it before each commit. In this way
>> you
>> can catch regressions.
>> In order to effectively catch regressions, i.e. tests that started to
>> fail
>> after some change,
>> it's necessary to have 'build test' pass in a stable way. One of the ways
>> to achieve stable state
>> is to exclude failing tests or tests which show unstable behavior.
>> So I analysed statistics of test runs on win ia32 platform and made up a
>> list of tests to be excluded:
>> 1) smoke
>> *** gc.LOS fails always on jitrino and interpreter, debug and release
>> 2) kernel
>> *** java.lang.ClasssGenericsTest and
>> *** java.lang.ClassGenericsTest4 fail because of timeout, so I increase
>> timeout in kernel.test.xml
>> *** java.lang.ObjectTest fail on interpreter with following message:
>> Testsuite: java.lang.ObjectTest
>> Tests run: 18, Failures: 1, Errors: 0, Time elapsed: 6.109 sec
>> Testcase: testWait1(java.lang.ObjectTest): FAILED
>> An InterruptedException must be thrown in test thread!
>> junit.framework.AssertionFailedError: An InterruptedException must be
>> thrown in test thread!
>>
>> See HARNONY-1966 issue with attached patch.
>>
>>
>> Vera Volynets
>> Intel Middleware ProductsDivision
>>
>
Re: [drlvm][test]Exclude some tests from "build test" target, make 'build test' pass
Posted by Rana Dasgupta <rd...@gmail.com>.
Hi,
I did not want to start a new thread for this. Could someone please close
JIRA issues 1109, 1662, 1830? These have been fixed by recent changes in the
stack and exception handling. In any case, none of them repro. Comments on
teh JIRA.
Thanks,
Rana
On 10/25/06, Volynets, Vera <ve...@intel.com> wrote:
>
> Geir
> Some tests launched by command "build test" fail.
> The idea of "build test" is to run it before each commit. In this way you
> can catch regressions.
> In order to effectively catch regressions, i.e. tests that started to fail
> after some change,
> it's necessary to have 'build test' pass in a stable way. One of the ways
> to achieve stable state
> is to exclude failing tests or tests which show unstable behavior.
> So I analysed statistics of test runs on win ia32 platform and made up a
> list of tests to be excluded:
> 1) smoke
> *** gc.LOS fails always on jitrino and interpreter, debug and release
> 2) kernel
> *** java.lang.ClasssGenericsTest and
> *** java.lang.ClassGenericsTest4 fail because of timeout, so I increase
> timeout in kernel.test.xml
> *** java.lang.ObjectTest fail on interpreter with following message:
> Testsuite: java.lang.ObjectTest
> Tests run: 18, Failures: 1, Errors: 0, Time elapsed: 6.109 sec
> Testcase: testWait1(java.lang.ObjectTest): FAILED
> An InterruptedException must be thrown in test thread!
> junit.framework.AssertionFailedError: An InterruptedException must be
> thrown in test thread!
>
> See HARNONY-1966 issue with attached patch.
>
>
> Vera Volynets
> Intel Middleware ProductsDivision
>
Re: [drlvm][test]Exclude some tests from "build test" target, make 'build test' pass
Posted by Alexey Varlamov <al...@gmail.com>.
27.10.06, Salikh Zakirov<Sa...@intel.com> написал(а):
> Gregory Shimansky wrote:
> >> But on Linux these tests fail with
> >> lost of different exceptions:
> >>
> >> For java.lang.ClassGenericsTest:
> >>
> >> java.lang.ClassNotFoundException: java.lang.ClassGenericsTest$Mc009^Z^Z^Z
> >> java.lang.NoClassDefFoundError: java/lang/ClassGenericsTest$Mc007^Z
> >> java.lang.TypeNotPresentException: Type
> >> java.lang.ClassGenericsTest$Mc009\0d5\0b6\0db\080\0db\0b1 not present
> >>
> >> For java.lang.ClassGenericsTest4:
> >>
> >> Some NPEs from unobvious sources. I've attached two test reports to
> >> this email. Anyways, it looks like timeout is not the case for these
> >> tests failures.
> >
> > Ok I think I've found the cause of problems with these two tests on
> > Linux which I had yesterday. On Gentoo I have russian locale set
> > "ru_RU.KOI8-R" while on all other Linux installations it appears there
> > are various combinations of *.UTF-8 locales (usually "en_US.UTF-8", but
> > I found "en_AU.UTF-8" on oner server).
> >
> > I experimented a bit with this stuff and found out that these tests work
> > ok when locale is some sort of UTF-8 one, including ru_RU.UTF-8 one.
> >
> > The result of test run is actually affected by locale set at compilation
> > time, not the one set at the subsequent test runs. When locale at
> > compile time is set to *.UTF-8, then test run is successful on any
> > locale set after it. If locale is set to something simple like "C" (!)
> > or "ru_RU.KOI8-R" at compile time then test run fails no matter what
> > locale is set when tests are running.
> >
> > I wonder what we should blame here, compiler, VM, classlib or the tests?
>
> IMHO, the test is not correct for non-unicode configurations,
> as some unicode-named classfiles cannot be saved with filename in locale-specified character set.
>
> I think that fixing \uXXXX characters in these tests to some ASCII characters should
> fix the problem (I haven't checked myself).
>
> However, I do not know the original intention why those characters were put to
> the class names in the first place, does anyone know an explanation?
>
I suppose the intention was to test generics parser's ability to work
with supplementary unicode characters, and the most straightforward
way is to let Java compiler compose signatures from given types, with
class files created as a side effect. Probably such tests should be
unit tests of the parser itself, with hardcoded string parameters to
avoid extra dependency on environment. At least until VM is
unicode-ready or there is a bunch of unicode tests for the whole
stack...
Re: [drlvm][test]Exclude some tests from "build test" target, make
'build test' pass
Posted by Salikh Zakirov <Sa...@Intel.com>.
Gregory Shimansky wrote:
>> But on Linux these tests fail with
>> lost of different exceptions:
>>
>> For java.lang.ClassGenericsTest:
>>
>> java.lang.ClassNotFoundException: java.lang.ClassGenericsTest$Mc009^Z^Z^Z
>> java.lang.NoClassDefFoundError: java/lang/ClassGenericsTest$Mc007^Z
>> java.lang.TypeNotPresentException: Type
>> java.lang.ClassGenericsTest$Mc009\0d5\0b6\0db\080\0db\0b1 not present
>>
>> For java.lang.ClassGenericsTest4:
>>
>> Some NPEs from unobvious sources. I've attached two test reports to
>> this email. Anyways, it looks like timeout is not the case for these
>> tests failures.
>
> Ok I think I've found the cause of problems with these two tests on
> Linux which I had yesterday. On Gentoo I have russian locale set
> "ru_RU.KOI8-R" while on all other Linux installations it appears there
> are various combinations of *.UTF-8 locales (usually "en_US.UTF-8", but
> I found "en_AU.UTF-8" on oner server).
>
> I experimented a bit with this stuff and found out that these tests work
> ok when locale is some sort of UTF-8 one, including ru_RU.UTF-8 one.
>
> The result of test run is actually affected by locale set at compilation
> time, not the one set at the subsequent test runs. When locale at
> compile time is set to *.UTF-8, then test run is successful on any
> locale set after it. If locale is set to something simple like "C" (!)
> or "ru_RU.KOI8-R" at compile time then test run fails no matter what
> locale is set when tests are running.
>
> I wonder what we should blame here, compiler, VM, classlib or the tests?
IMHO, the test is not correct for non-unicode configurations,
as some unicode-named classfiles cannot be saved with filename in locale-specified character set.
I think that fixing \uXXXX characters in these tests to some ASCII characters should
fix the problem (I haven't checked myself).
However, I do not know the original intention why those characters were put to
the class names in the first place, does anyone know an explanation?
Re: [drlvm][test]Exclude some tests from "build test" target, make
'build test' pass
Posted by "Geir Magnusson Jr." <ge...@pobox.com>.
Gregory Shimansky wrote:
> Ok I think I've found the cause of problems with these two tests on
> Linux which I had yesterday. On Gentoo I have russian locale set
> "ru_RU.KOI8-R" while on all other Linux installations it appears there
> are various combinations of *.UTF-8 locales (usually "en_US.UTF-8", but
> I found "en_AU.UTF-8" on oner server).
>
> I experimented a bit with this stuff and found out that these tests work
> ok when locale is some sort of UTF-8 one, including ru_RU.UTF-8 one.
>
> The result of test run is actually affected by locale set at compilation
> time, not the one set at the subsequent test runs. When locale at
> compile time is set to *.UTF-8, then test run is successful on any
> locale set after it. If locale is set to something simple like "C" (!)
> or "ru_RU.KOI8-R" at compile time then test run fails no matter what
> locale is set when tests are running.
>
> I wonder what we should blame here, compiler, VM, classlib or the tests?
>
/me runs the other way...
(nice work :)
geir
Re: [drlvm][test]Exclude some tests from "build test" target, make
'build test' pass
Posted by Gregory Shimansky <gs...@gmail.com>.
Gregory Shimansky wrote:
>> 2) kernel
>> *** java.lang.ClasssGenericsTest and
>> *** java.lang.ClassGenericsTest4 fail because of timeout, so I increase
>> timeout in kernel.test.xml
>
> These tests work for me on Windows. But on Linux these tests fail with lost of
> different exceptions:
>
> For java.lang.ClassGenericsTest:
>
> java.lang.ClassNotFoundException: java.lang.ClassGenericsTest$Mc009^Z^Z^Z
> java.lang.NoClassDefFoundError: java/lang/ClassGenericsTest$Mc007^Z
> java.lang.TypeNotPresentException: Type
> java.lang.ClassGenericsTest$Mc009\0d5\0b6\0db\080\0db\0b1 not present
>
> For java.lang.ClassGenericsTest4:
>
> Some NPEs from unobvious sources. I've attached two test reports to this
> email. Anyways, it looks like timeout is not the case for these tests
> failures.
Ok I think I've found the cause of problems with these two tests on
Linux which I had yesterday. On Gentoo I have russian locale set
"ru_RU.KOI8-R" while on all other Linux installations it appears there
are various combinations of *.UTF-8 locales (usually "en_US.UTF-8", but
I found "en_AU.UTF-8" on oner server).
I experimented a bit with this stuff and found out that these tests work
ok when locale is some sort of UTF-8 one, including ru_RU.UTF-8 one.
The result of test run is actually affected by locale set at compilation
time, not the one set at the subsequent test runs. When locale at
compile time is set to *.UTF-8, then test run is successful on any
locale set after it. If locale is set to something simple like "C" (!)
or "ru_RU.KOI8-R" at compile time then test run fails no matter what
locale is set when tests are running.
I wonder what we should blame here, compiler, VM, classlib or the tests?
--
Gregory Shimansky, Intel Middleware Products Division
Re: [drlvm][test]Exclude some tests from "build test" target, make 'build test' pass
Posted by Gregory Shimansky <gs...@gmail.com>.
On Wednesday 25 October 2006 20:29 Volynets, Vera wrote:
> Geir
> Some tests launched by command "build test" fail.
> The idea of "build test" is to run it before each commit. In this way you
> can catch regressions. In order to effectively catch regressions, i.e.
> tests that started to fail after some change, it's necessary to have 'build
> test' pass in a stable way. One of the ways to achieve stable state is to
> exclude failing tests or tests which show unstable behavior. So I analysed
> statistics of test runs on win ia32 platform and made up a list of tests to
> be excluded: 1) smoke
> *** gc.LOS fails always on jitrino and interpreter, debug and release
On windows XP. Can we exclude it only for win32 and leave it on Linux?
> 2) kernel
> *** java.lang.ClasssGenericsTest and
> *** java.lang.ClassGenericsTest4 fail because of timeout, so I increase
> timeout in kernel.test.xml
These tests work for me on Windows. But on Linux these tests fail with lost of
different exceptions:
For java.lang.ClassGenericsTest:
java.lang.ClassNotFoundException: java.lang.ClassGenericsTest$Mc009^Z^Z^Z
java.lang.NoClassDefFoundError: java/lang/ClassGenericsTest$Mc007^Z
java.lang.TypeNotPresentException: Type
java.lang.ClassGenericsTest$Mc009\0d5\0b6\0db\080\0db\0b1 not present
For java.lang.ClassGenericsTest4:
Some NPEs from unobvious sources. I've attached two test reports to this
email. Anyways, it looks like timeout is not the case for these tests
failures.
> *** java.lang.ObjectTest fail on interpreter
> with following message: Testsuite: java.lang.ObjectTest
> Tests run: 18, Failures: 1, Errors: 0, Time elapsed: 6.109 sec
> Testcase: testWait1(java.lang.ObjectTest): FAILED
> An InterruptedException must be thrown in test thread!
> junit.framework.AssertionFailedError: An InterruptedException must be
> thrown in test thread!
This test works ok for me, probably after HARMONY-1823 was applied. Are you
sure you run the latest sources?
--
Gregory Shimansky, Intel Middleware Products Division