You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@harmony.apache.org by Alexei Zakharov <al...@gmail.com> on 2007/03/23 15:30:08 UTC
[classlib][testing] intermittent tests
Hi all,
In HARMONY-1979 we (me, Tatyana and friends) faced the problem that
some tests fail very rarely. Say, one failure per 50 tries. So the
question to all is: should this be treated as an intermittent failure?
Thoughts?
Thanks,
2007/3/22, tatyana doubtsova (JIRA) <ji...@apache.org>:
>
> [ https://issues.apache.org/jira/browse/HARMONY-1979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12483130 ]
>
> tatyana doubtsova commented on HARMONY-1979:
> --------------------------------------------
>
> Alexei, the behavior of awt module tests is unstable on linux ia32, ia 64. New runs show new failures.
>
> lnx ia32 (gcc, debug, classlib over drlvm) intermittent failures:
>
> 1. java.awt.WindowTest.testSetLocationRelativeTo fails 6 times out of 50
> expected:<java.awt.Point[x=50,y=50]> but was:<java.awt.Point[x=461,y=334]>
> junit.framework.AssertionFailedError: expected:<java.awt.Point[x=50,y=50]> but was:<java.awt.Point[x=461,y=334]> at java.awt.WindowTest.testSetLocationRelativeTo(WindowTest.java:89) at java.lang.reflect.VMReflection.invokeMethod(VMReflection.java)
>
> 2. java.awt.event.InvocationEventTest - possible crash 1 out of 50
>
> 3. org.apache.harmony.awt.gl.MultiRectAreaTest.testUnion - failes 2 out of 50
> org.apache.harmony.awt.gl.MultiRectAreaTest.testSubtract - failes 2 out of 50
> java.lang.OutOfMemoryError <no stack trace available>
>
> 4. java.awt.font.LineBreakMeasurerTest - possible crash 1 out of 50
>
> 5. java.awt.font.TextHitInfoTest - possible crash 1 out of 50
>
> lnx ia64 (gcc, debug, classlib over drlvm) intermittent failures:
>
> 1. java.awt.EventDispatchThreadRTest - possible crash 4 out of 30
> 2. java.awt.AccessibleAWTWindowTest - possible crash 9 out of 30
> 3. java.awt.event.InvocationEventTest - possible crash 1 out of 30
> 4. java.awt.geom.FlatteningPathIteratorTest 1 - possible crash 1 out of 30
> 5. java.awt.BorderLayoutRTest - possible crash 1 out of 30
> 6. java.awt.WindowTest.testSetLocationRelativeTo fails 8 times out of 30
> expected:<java.awt.Point[x=50,y=50]> but was:<java.awt.Point[x=461,y=334]>
> junit.framework.AssertionFailedError: expected:<java.awt.Point[x=50,y=50]> but was:<java.awt.Point[x=461,y=334]> at java.awt.WindowTest.testSetLocationRelativeTo(WindowTest.java:89) at java.lang.reflect.VMReflection.invokeMethod(VMReflection.java)
> 7. java.awt.KeyboardFocusManagerTest.testAddPropertyChangeListenerString failes 1 time out of 30
> junit.framework.AssertionFailedError at java.awt.KeyboardFocusManagerTest.testAddPropertyChangeListenerString(KeyboardFocusManagerTest.java:152) at java.lang.reflect.VMReflection.invokeMethod(VMReflection.java)
>
>
>
> > [classlib][awt][test] Many excluded AWT tests pass
> > --------------------------------------------------
> >
> > Key: HARMONY-1979
> > URL: https://issues.apache.org/jira/browse/HARMONY-1979
> > Project: Harmony
> > Issue Type: Bug
> > Components: build - test - ci
> > Reporter: Alexey A. Ivanov
> > Assigned To: Alexei Zakharov
> > Attachments: H1979-awt-unexclude-tests.patch
> >
> >
> > Many excluded AWT tests pass successfully without any modification. I tried both Windows and Linux.
>
> --
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
--
Alexei Zakharov,
Intel ESSD
Re: [classlib][testing] intermittent tests
Posted by Mikhail Loenko <ml...@gmail.com>.
2007/3/27, Alexei Zakharov <al...@gmail.com>:
> > Do you think it is a permanent failure? :)
>
> No :)
> But my intention was at least to wait until CC sends failure alerts
> about these tests. My initial question was about a formal criteria to
> be used when placing a test to intermittent exclude list. The current
> criteria is periodical CC failure alerts, right?
I don't remember any agreement on a formal criteria.
I think that we exclude [using common sense] those tests
that may produce false alarms of CCs.
I think that the threshold (whether it fails often enough to exclude it)
should depend on the number of configurations running regularly:
if we have just on CC project that runs once a week then it's OK to
have it intermittently failing for example every 10 runs. Currently we
have dozens of CC projects and it would be good to have a false alarm
not often than once per week. Basing on the desired false alarm
frequency, number of CC projects, number of intermittently failing tests
we can derive "target" failure frequency that is enough for inclusion
to the "intermittent" exclude list :)
So...
Once per million runs is probably too rare for exclusion, but once
per 50 times is probably enough
Thanks,
Mikhail
>
> Thanks,
>
> 2007/3/27, Mikhail Loenko <ml...@gmail.com>:
> > 2007/3/23, Alexei Zakharov <al...@gmail.com>:
> > > Hi all,
> > >
> > > In HARMONY-1979 we (me, Tatyana and friends) faced the problem that
> > > some tests fail very rarely. Say, one failure per 50 tries. So the
> > > question to all is: should this be treated as an intermittent failure?
> > > Thoughts?
> >
> > Do you think it is a permanent failure? :)
> >
> > if it's still a failure, we should not run the tests under CCs to avoid
> > false alarms
> >
> > Thanks,
> > Mikhail
> >
> >
> > >
> > > Thanks,
> > >
> > > 2007/3/22, tatyana doubtsova (JIRA) <ji...@apache.org>:
> > > >
> > > > [ https://issues.apache.org/jira/browse/HARMONY-1979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12483130 ]
> > > >
> > > > tatyana doubtsova commented on HARMONY-1979:
> > > > --------------------------------------------
> > > >
> > > > Alexei, the behavior of awt module tests is unstable on linux ia32, ia 64. New runs show new failures.
> > > >
> > > > lnx ia32 (gcc, debug, classlib over drlvm) intermittent failures:
> > > >
> > > > 1. java.awt.WindowTest.testSetLocationRelativeTo fails 6 times out of 50
> > > > expected:<java.awt.Point[x=50,y=50]> but was:<java.awt.Point[x=461,y=334]>
> > > > junit.framework.AssertionFailedError: expected:<java.awt.Point[x=50,y=50]> but was:<java.awt.Point[x=461,y=334]> at java.awt.WindowTest.testSetLocationRelativeTo(WindowTest.java:89) at java.lang.reflect.VMReflection.invokeMethod(VMReflection.java)
> > > >
> > > > 2. java.awt.event.InvocationEventTest - possible crash 1 out of 50
> > > >
> > > > 3. org.apache.harmony.awt.gl.MultiRectAreaTest.testUnion - failes 2 out of 50
> > > > org.apache.harmony.awt.gl.MultiRectAreaTest.testSubtract - failes 2 out of 50
> > > > java.lang.OutOfMemoryError <no stack trace available>
> > > >
> > > > 4. java.awt.font.LineBreakMeasurerTest - possible crash 1 out of 50
> > > >
> > > > 5. java.awt.font.TextHitInfoTest - possible crash 1 out of 50
> > > >
> > > > lnx ia64 (gcc, debug, classlib over drlvm) intermittent failures:
> > > >
> > > > 1. java.awt.EventDispatchThreadRTest - possible crash 4 out of 30
> > > > 2. java.awt.AccessibleAWTWindowTest - possible crash 9 out of 30
> > > > 3. java.awt.event.InvocationEventTest - possible crash 1 out of 30
> > > > 4. java.awt.geom.FlatteningPathIteratorTest 1 - possible crash 1 out of 30
> > > > 5. java.awt.BorderLayoutRTest - possible crash 1 out of 30
> > > > 6. java.awt.WindowTest.testSetLocationRelativeTo fails 8 times out of 30
> > > > expected:<java.awt.Point[x=50,y=50]> but was:<java.awt.Point[x=461,y=334]>
> > > > junit.framework.AssertionFailedError: expected:<java.awt.Point[x=50,y=50]> but was:<java.awt.Point[x=461,y=334]> at java.awt.WindowTest.testSetLocationRelativeTo(WindowTest.java:89) at java.lang.reflect.VMReflection.invokeMethod(VMReflection.java)
> > > > 7. java.awt.KeyboardFocusManagerTest.testAddPropertyChangeListenerString failes 1 time out of 30
> > > > junit.framework.AssertionFailedError at java.awt.KeyboardFocusManagerTest.testAddPropertyChangeListenerString(KeyboardFocusManagerTest.java:152) at java.lang.reflect.VMReflection.invokeMethod(VMReflection.java)
> > > >
> > > >
> > > >
> > > > > [classlib][awt][test] Many excluded AWT tests pass
> > > > > --------------------------------------------------
> > > > >
> > > > > Key: HARMONY-1979
> > > > > URL: https://issues.apache.org/jira/browse/HARMONY-1979
> > > > > Project: Harmony
> > > > > Issue Type: Bug
> > > > > Components: build - test - ci
> > > > > Reporter: Alexey A. Ivanov
> > > > > Assigned To: Alexei Zakharov
> > > > > Attachments: H1979-awt-unexclude-tests.patch
> > > > >
> > > > >
> > > > > Many excluded AWT tests pass successfully without any modification. I tried both Windows and Linux.
> > > >
> > > > --
> > > > This message is automatically generated by JIRA.
> > > > -
> > > > You can reply to this email to add a comment to the issue online.
>
>
> --
> Alexei Zakharov,
> Intel ESSD
>
Re: [classlib][testing] intermittent tests
Posted by Alexei Zakharov <al...@gmail.com>.
> Do you think it is a permanent failure? :)
No :)
But my intention was at least to wait until CC sends failure alerts
about these tests. My initial question was about a formal criteria to
be used when placing a test to intermittent exclude list. The current
criteria is periodical CC failure alerts, right?
Thanks,
2007/3/27, Mikhail Loenko <ml...@gmail.com>:
> 2007/3/23, Alexei Zakharov <al...@gmail.com>:
> > Hi all,
> >
> > In HARMONY-1979 we (me, Tatyana and friends) faced the problem that
> > some tests fail very rarely. Say, one failure per 50 tries. So the
> > question to all is: should this be treated as an intermittent failure?
> > Thoughts?
>
> Do you think it is a permanent failure? :)
>
> if it's still a failure, we should not run the tests under CCs to avoid
> false alarms
>
> Thanks,
> Mikhail
>
>
> >
> > Thanks,
> >
> > 2007/3/22, tatyana doubtsova (JIRA) <ji...@apache.org>:
> > >
> > > [ https://issues.apache.org/jira/browse/HARMONY-1979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12483130 ]
> > >
> > > tatyana doubtsova commented on HARMONY-1979:
> > > --------------------------------------------
> > >
> > > Alexei, the behavior of awt module tests is unstable on linux ia32, ia 64. New runs show new failures.
> > >
> > > lnx ia32 (gcc, debug, classlib over drlvm) intermittent failures:
> > >
> > > 1. java.awt.WindowTest.testSetLocationRelativeTo fails 6 times out of 50
> > > expected:<java.awt.Point[x=50,y=50]> but was:<java.awt.Point[x=461,y=334]>
> > > junit.framework.AssertionFailedError: expected:<java.awt.Point[x=50,y=50]> but was:<java.awt.Point[x=461,y=334]> at java.awt.WindowTest.testSetLocationRelativeTo(WindowTest.java:89) at java.lang.reflect.VMReflection.invokeMethod(VMReflection.java)
> > >
> > > 2. java.awt.event.InvocationEventTest - possible crash 1 out of 50
> > >
> > > 3. org.apache.harmony.awt.gl.MultiRectAreaTest.testUnion - failes 2 out of 50
> > > org.apache.harmony.awt.gl.MultiRectAreaTest.testSubtract - failes 2 out of 50
> > > java.lang.OutOfMemoryError <no stack trace available>
> > >
> > > 4. java.awt.font.LineBreakMeasurerTest - possible crash 1 out of 50
> > >
> > > 5. java.awt.font.TextHitInfoTest - possible crash 1 out of 50
> > >
> > > lnx ia64 (gcc, debug, classlib over drlvm) intermittent failures:
> > >
> > > 1. java.awt.EventDispatchThreadRTest - possible crash 4 out of 30
> > > 2. java.awt.AccessibleAWTWindowTest - possible crash 9 out of 30
> > > 3. java.awt.event.InvocationEventTest - possible crash 1 out of 30
> > > 4. java.awt.geom.FlatteningPathIteratorTest 1 - possible crash 1 out of 30
> > > 5. java.awt.BorderLayoutRTest - possible crash 1 out of 30
> > > 6. java.awt.WindowTest.testSetLocationRelativeTo fails 8 times out of 30
> > > expected:<java.awt.Point[x=50,y=50]> but was:<java.awt.Point[x=461,y=334]>
> > > junit.framework.AssertionFailedError: expected:<java.awt.Point[x=50,y=50]> but was:<java.awt.Point[x=461,y=334]> at java.awt.WindowTest.testSetLocationRelativeTo(WindowTest.java:89) at java.lang.reflect.VMReflection.invokeMethod(VMReflection.java)
> > > 7. java.awt.KeyboardFocusManagerTest.testAddPropertyChangeListenerString failes 1 time out of 30
> > > junit.framework.AssertionFailedError at java.awt.KeyboardFocusManagerTest.testAddPropertyChangeListenerString(KeyboardFocusManagerTest.java:152) at java.lang.reflect.VMReflection.invokeMethod(VMReflection.java)
> > >
> > >
> > >
> > > > [classlib][awt][test] Many excluded AWT tests pass
> > > > --------------------------------------------------
> > > >
> > > > Key: HARMONY-1979
> > > > URL: https://issues.apache.org/jira/browse/HARMONY-1979
> > > > Project: Harmony
> > > > Issue Type: Bug
> > > > Components: build - test - ci
> > > > Reporter: Alexey A. Ivanov
> > > > Assigned To: Alexei Zakharov
> > > > Attachments: H1979-awt-unexclude-tests.patch
> > > >
> > > >
> > > > Many excluded AWT tests pass successfully without any modification. I tried both Windows and Linux.
> > >
> > > --
> > > This message is automatically generated by JIRA.
> > > -
> > > You can reply to this email to add a comment to the issue online.
--
Alexei Zakharov,
Intel ESSD
Re: [classlib][testing] intermittent tests
Posted by Alexei Zakharov <al...@gmail.com>.
Well, the difference is that intermittently failing tests are not
excluded by default and it is expected that they will be used for
pre-commit testing. However, CC excludes intermittent tests from its
test runs.
Regards,
2007/3/27, Tony Wu <wu...@gmail.com>:
> IMO we have to deal with them no matter they are intermittent failures
> or permanent failures. all of them should be excluded. we mark a
> failure as intermittent for someone who will try to fix them in the
> future, he will not be confused on getting passed results when running
> a few times.
>
> On 3/27/07, Mikhail Loenko <ml...@gmail.com> wrote:
> > 2007/3/23, Alexei Zakharov <al...@gmail.com>:
> > > Hi all,
> > >
> > > In HARMONY-1979 we (me, Tatyana and friends) faced the problem that
> > > some tests fail very rarely. Say, one failure per 50 tries. So the
> > > question to all is: should this be treated as an intermittent failure?
> > > Thoughts?
> >
> > Do you think it is a permanent failure? :)
> >
> > if it's still a failure, we should not run the tests under CCs to avoid
> > false alarms
> >
> > Thanks,
> > Mikhail
> >
> >
> > >
> > > Thanks,
> > >
> > > 2007/3/22, tatyana doubtsova (JIRA) <ji...@apache.org>:
> > > >
> > > > [ https://issues.apache.org/jira/browse/HARMONY-1979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12483130 ]
> > > >
> > > > tatyana doubtsova commented on HARMONY-1979:
> > > > --------------------------------------------
> > > >
> > > > Alexei, the behavior of awt module tests is unstable on linux ia32, ia 64. New runs show new failures.
> > > >
> > > > lnx ia32 (gcc, debug, classlib over drlvm) intermittent failures:
> > > >
> > > > 1. java.awt.WindowTest.testSetLocationRelativeTo fails 6 times out of 50
> > > > expected:<java.awt.Point[x=50,y=50]> but was:<java.awt.Point[x=461,y=334]>
> > > > junit.framework.AssertionFailedError: expected:<java.awt.Point[x=50,y=50]> but was:<java.awt.Point[x=461,y=334]> at java.awt.WindowTest.testSetLocationRelativeTo(WindowTest.java:89) at java.lang.reflect.VMReflection.invokeMethod(VMReflection.java)
> > > >
> > > > 2. java.awt.event.InvocationEventTest - possible crash 1 out of 50
> > > >
> > > > 3. org.apache.harmony.awt.gl.MultiRectAreaTest.testUnion - failes 2 out of 50
> > > > org.apache.harmony.awt.gl.MultiRectAreaTest.testSubtract - failes 2 out of 50
> > > > java.lang.OutOfMemoryError <no stack trace available>
> > > >
> > > > 4. java.awt.font.LineBreakMeasurerTest - possible crash 1 out of 50
> > > >
> > > > 5. java.awt.font.TextHitInfoTest - possible crash 1 out of 50
> > > >
> > > > lnx ia64 (gcc, debug, classlib over drlvm) intermittent failures:
> > > >
> > > > 1. java.awt.EventDispatchThreadRTest - possible crash 4 out of 30
> > > > 2. java.awt.AccessibleAWTWindowTest - possible crash 9 out of 30
> > > > 3. java.awt.event.InvocationEventTest - possible crash 1 out of 30
> > > > 4. java.awt.geom.FlatteningPathIteratorTest 1 - possible crash 1 out of 30
> > > > 5. java.awt.BorderLayoutRTest - possible crash 1 out of 30
> > > > 6. java.awt.WindowTest.testSetLocationRelativeTo fails 8 times out of 30
> > > > expected:<java.awt.Point[x=50,y=50]> but was:<java.awt.Point[x=461,y=334]>
> > > > junit.framework.AssertionFailedError: expected:<java.awt.Point[x=50,y=50]> but was:<java.awt.Point[x=461,y=334]> at java.awt.WindowTest.testSetLocationRelativeTo(WindowTest.java:89) at java.lang.reflect.VMReflection.invokeMethod(VMReflection.java)
> > > > 7. java.awt.KeyboardFocusManagerTest.testAddPropertyChangeListenerString failes 1 time out of 30
> > > > junit.framework.AssertionFailedError at java.awt.KeyboardFocusManagerTest.testAddPropertyChangeListenerString(KeyboardFocusManagerTest.java:152) at java.lang.reflect.VMReflection.invokeMethod(VMReflection.java)
> > > >
> > > >
> > > >
> > > > > [classlib][awt][test] Many excluded AWT tests pass
> > > > > --------------------------------------------------
> > > > >
> > > > > Key: HARMONY-1979
> > > > > URL: https://issues.apache.org/jira/browse/HARMONY-1979
> > > > > Project: Harmony
> > > > > Issue Type: Bug
> > > > > Components: build - test - ci
> > > > > Reporter: Alexey A. Ivanov
> > > > > Assigned To: Alexei Zakharov
> > > > > Attachments: H1979-awt-unexclude-tests.patch
> > > > >
> > > > >
> > > > > Many excluded AWT tests pass successfully without any modification. I tried both Windows and Linux.
> > > >
> > > > --
> > > > This message is automatically generated by JIRA.
> > > > -
> > > > You can reply to this email to add a comment to the issue online.
--
Alexei Zakharov,
Intel ESSD
Re: [classlib][testing] intermittent tests
Posted by Tony Wu <wu...@gmail.com>.
IMO we have to deal with them no matter they are intermittent failures
or permanent failures. all of them should be excluded. we mark a
failure as intermittent for someone who will try to fix them in the
future, he will not be confused on getting passed results when running
a few times.
On 3/27/07, Mikhail Loenko <ml...@gmail.com> wrote:
> 2007/3/23, Alexei Zakharov <al...@gmail.com>:
> > Hi all,
> >
> > In HARMONY-1979 we (me, Tatyana and friends) faced the problem that
> > some tests fail very rarely. Say, one failure per 50 tries. So the
> > question to all is: should this be treated as an intermittent failure?
> > Thoughts?
>
> Do you think it is a permanent failure? :)
>
> if it's still a failure, we should not run the tests under CCs to avoid
> false alarms
>
> Thanks,
> Mikhail
>
>
> >
> > Thanks,
> >
> > 2007/3/22, tatyana doubtsova (JIRA) <ji...@apache.org>:
> > >
> > > [ https://issues.apache.org/jira/browse/HARMONY-1979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12483130 ]
> > >
> > > tatyana doubtsova commented on HARMONY-1979:
> > > --------------------------------------------
> > >
> > > Alexei, the behavior of awt module tests is unstable on linux ia32, ia 64. New runs show new failures.
> > >
> > > lnx ia32 (gcc, debug, classlib over drlvm) intermittent failures:
> > >
> > > 1. java.awt.WindowTest.testSetLocationRelativeTo fails 6 times out of 50
> > > expected:<java.awt.Point[x=50,y=50]> but was:<java.awt.Point[x=461,y=334]>
> > > junit.framework.AssertionFailedError: expected:<java.awt.Point[x=50,y=50]> but was:<java.awt.Point[x=461,y=334]> at java.awt.WindowTest.testSetLocationRelativeTo(WindowTest.java:89) at java.lang.reflect.VMReflection.invokeMethod(VMReflection.java)
> > >
> > > 2. java.awt.event.InvocationEventTest - possible crash 1 out of 50
> > >
> > > 3. org.apache.harmony.awt.gl.MultiRectAreaTest.testUnion - failes 2 out of 50
> > > org.apache.harmony.awt.gl.MultiRectAreaTest.testSubtract - failes 2 out of 50
> > > java.lang.OutOfMemoryError <no stack trace available>
> > >
> > > 4. java.awt.font.LineBreakMeasurerTest - possible crash 1 out of 50
> > >
> > > 5. java.awt.font.TextHitInfoTest - possible crash 1 out of 50
> > >
> > > lnx ia64 (gcc, debug, classlib over drlvm) intermittent failures:
> > >
> > > 1. java.awt.EventDispatchThreadRTest - possible crash 4 out of 30
> > > 2. java.awt.AccessibleAWTWindowTest - possible crash 9 out of 30
> > > 3. java.awt.event.InvocationEventTest - possible crash 1 out of 30
> > > 4. java.awt.geom.FlatteningPathIteratorTest 1 - possible crash 1 out of 30
> > > 5. java.awt.BorderLayoutRTest - possible crash 1 out of 30
> > > 6. java.awt.WindowTest.testSetLocationRelativeTo fails 8 times out of 30
> > > expected:<java.awt.Point[x=50,y=50]> but was:<java.awt.Point[x=461,y=334]>
> > > junit.framework.AssertionFailedError: expected:<java.awt.Point[x=50,y=50]> but was:<java.awt.Point[x=461,y=334]> at java.awt.WindowTest.testSetLocationRelativeTo(WindowTest.java:89) at java.lang.reflect.VMReflection.invokeMethod(VMReflection.java)
> > > 7. java.awt.KeyboardFocusManagerTest.testAddPropertyChangeListenerString failes 1 time out of 30
> > > junit.framework.AssertionFailedError at java.awt.KeyboardFocusManagerTest.testAddPropertyChangeListenerString(KeyboardFocusManagerTest.java:152) at java.lang.reflect.VMReflection.invokeMethod(VMReflection.java)
> > >
> > >
> > >
> > > > [classlib][awt][test] Many excluded AWT tests pass
> > > > --------------------------------------------------
> > > >
> > > > Key: HARMONY-1979
> > > > URL: https://issues.apache.org/jira/browse/HARMONY-1979
> > > > Project: Harmony
> > > > Issue Type: Bug
> > > > Components: build - test - ci
> > > > Reporter: Alexey A. Ivanov
> > > > Assigned To: Alexei Zakharov
> > > > Attachments: H1979-awt-unexclude-tests.patch
> > > >
> > > >
> > > > Many excluded AWT tests pass successfully without any modification. I tried both Windows and Linux.
> > >
> > > --
> > > This message is automatically generated by JIRA.
> > > -
> > > You can reply to this email to add a comment to the issue online.
> >
> >
> > --
> > Alexei Zakharov,
> > Intel ESSD
> >
>
--
Tony Wu
China Software Development Lab, IBM
Re: [classlib][testing] intermittent tests
Posted by Mikhail Loenko <ml...@gmail.com>.
2007/3/23, Alexei Zakharov <al...@gmail.com>:
> Hi all,
>
> In HARMONY-1979 we (me, Tatyana and friends) faced the problem that
> some tests fail very rarely. Say, one failure per 50 tries. So the
> question to all is: should this be treated as an intermittent failure?
> Thoughts?
Do you think it is a permanent failure? :)
if it's still a failure, we should not run the tests under CCs to avoid
false alarms
Thanks,
Mikhail
>
> Thanks,
>
> 2007/3/22, tatyana doubtsova (JIRA) <ji...@apache.org>:
> >
> > [ https://issues.apache.org/jira/browse/HARMONY-1979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12483130 ]
> >
> > tatyana doubtsova commented on HARMONY-1979:
> > --------------------------------------------
> >
> > Alexei, the behavior of awt module tests is unstable on linux ia32, ia 64. New runs show new failures.
> >
> > lnx ia32 (gcc, debug, classlib over drlvm) intermittent failures:
> >
> > 1. java.awt.WindowTest.testSetLocationRelativeTo fails 6 times out of 50
> > expected:<java.awt.Point[x=50,y=50]> but was:<java.awt.Point[x=461,y=334]>
> > junit.framework.AssertionFailedError: expected:<java.awt.Point[x=50,y=50]> but was:<java.awt.Point[x=461,y=334]> at java.awt.WindowTest.testSetLocationRelativeTo(WindowTest.java:89) at java.lang.reflect.VMReflection.invokeMethod(VMReflection.java)
> >
> > 2. java.awt.event.InvocationEventTest - possible crash 1 out of 50
> >
> > 3. org.apache.harmony.awt.gl.MultiRectAreaTest.testUnion - failes 2 out of 50
> > org.apache.harmony.awt.gl.MultiRectAreaTest.testSubtract - failes 2 out of 50
> > java.lang.OutOfMemoryError <no stack trace available>
> >
> > 4. java.awt.font.LineBreakMeasurerTest - possible crash 1 out of 50
> >
> > 5. java.awt.font.TextHitInfoTest - possible crash 1 out of 50
> >
> > lnx ia64 (gcc, debug, classlib over drlvm) intermittent failures:
> >
> > 1. java.awt.EventDispatchThreadRTest - possible crash 4 out of 30
> > 2. java.awt.AccessibleAWTWindowTest - possible crash 9 out of 30
> > 3. java.awt.event.InvocationEventTest - possible crash 1 out of 30
> > 4. java.awt.geom.FlatteningPathIteratorTest 1 - possible crash 1 out of 30
> > 5. java.awt.BorderLayoutRTest - possible crash 1 out of 30
> > 6. java.awt.WindowTest.testSetLocationRelativeTo fails 8 times out of 30
> > expected:<java.awt.Point[x=50,y=50]> but was:<java.awt.Point[x=461,y=334]>
> > junit.framework.AssertionFailedError: expected:<java.awt.Point[x=50,y=50]> but was:<java.awt.Point[x=461,y=334]> at java.awt.WindowTest.testSetLocationRelativeTo(WindowTest.java:89) at java.lang.reflect.VMReflection.invokeMethod(VMReflection.java)
> > 7. java.awt.KeyboardFocusManagerTest.testAddPropertyChangeListenerString failes 1 time out of 30
> > junit.framework.AssertionFailedError at java.awt.KeyboardFocusManagerTest.testAddPropertyChangeListenerString(KeyboardFocusManagerTest.java:152) at java.lang.reflect.VMReflection.invokeMethod(VMReflection.java)
> >
> >
> >
> > > [classlib][awt][test] Many excluded AWT tests pass
> > > --------------------------------------------------
> > >
> > > Key: HARMONY-1979
> > > URL: https://issues.apache.org/jira/browse/HARMONY-1979
> > > Project: Harmony
> > > Issue Type: Bug
> > > Components: build - test - ci
> > > Reporter: Alexey A. Ivanov
> > > Assigned To: Alexei Zakharov
> > > Attachments: H1979-awt-unexclude-tests.patch
> > >
> > >
> > > Many excluded AWT tests pass successfully without any modification. I tried both Windows and Linux.
> >
> > --
> > This message is automatically generated by JIRA.
> > -
> > You can reply to this email to add a comment to the issue online.
>
>
> --
> Alexei Zakharov,
> Intel ESSD
>