You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@harmony.apache.org by chunrong lai <ch...@gmail.com> on 2008/08/11 19:26:21 UTC

[testing] first M7 candidate (r681495) testing status

Hi, all:

  Here is r681495 snapshot testing status:
http://people.apache.org/~chunrong/snapshots/r681495/index.html. I am using
two platforms (Linux x86, windows x86_64) at the moment. Hopefully we will
have other two platforms in future for M8. Well, although we are testing
only two platforms for M7, my experience is that the status for another two
platforms should be not worse or just include some extra intermittent errors
which can be investigated in some later stages.

  The following suites passed on Linux x86/Windows x86_64 platforms: Ant
Scenario (or self-hosting), Axis application, Dacapo, DRLVM regression
tests, Geronimo Unit Tests, Scimark, Tomcat scenario, VTS VM Test Suite.

  Most of the failures are known issues (for M6). Although we can observe
15~20 new issues, those issues happen only in 1 platform and they look more
like the intermittent/timeout issues (less reproducible) to me. Overall I
myself would like to think r681495 is more stable than M6.

  Please add your comments and clarifications (please also see M6 testing
results http://people.apache.org/~smishura/r653525/,
http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805070304l38845ee0se01fb93fbfc05586@mail.gmail.com%3eand
integrity testing results
http://people.apache.org/~chunrong/harmony-integrity/ as a comparison).

 1) Classlib:
    1.1) Since r644719 (which committed patch for HARMONY-5684)

        org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest

org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest

         failed in both platforms

    1.2) Two failures

        org.apache.harmony.luni.tests.java.net.MulticastSocketTest (Failed
in windows_x86 running of M6)
        tests.api.java.security.PermissionCollectionTest           (Failed
in M6)

         are observed in Linux x86.

 2) DRLVM tests:
    2.1) One failure

         java.lang.ClassGenericsTest.test_2

         is observed in Linux x86 snapshot testing.
         I can see some old discussion in the mailing list about that but I
am not sure the expected status here.
         They should be intemittent errors since the integrity testing just
run well mostly.

 3) EUTs:

    3.1) Linux x86: 99.36%
         A recorded JIRA for this suite is HARMONY-2914 which wastes file
handlers and makes system unstable.

 4) Functional:
    4.1) Old regressions on both platforms:
         api.java.text.MessageFormat (HARMONY-5430)
         api.java.util.jar.Manifest  (HARMONY-5473)
         api.java.beans.beancontext.BeanContextTest (also in M6, recorded
as  regression caused by changes in locale data)
         api.java.beans.persistence.EncoderTest  (also in M6, recorded as
regression in the beans module)
         api.java.beans.persistence.EncoderDecoderTest (also in M6,
regression in the beans module)
         reg.vm.btest5625 (also in M6, recorded as intermittent and not
reproducible manually)

    4.2) Old regressions on 1 platform
         api.java.rmi.basicexception (ERROR in Linux x86, HARMONY-5823)
         api.java.rmi.basicregistry.RemoteServerTest (ERROR in Linux x86,
HARMONY-5823)
         jpda.jdwp.scenario.ST07.ST07Test (ERROR in windows x86_64, in M6 it
is recorded as regression since M4)
         java.math.F_BigIntegerMatrixMultiplyTest_01 (ERROR on Linux x86,
recorded as Timeout, not reproducible
 in M6)
         reg.vm.btest5717 (ERROR on Windows X86_64, recorded as "timeout,
the test is too heavy" in M6)
         jit.HLO.inline.ControlFlow.IfElse.IfElse1.IfElseTest1 (FAILED in
windows x86_64, recorded as "looks like
issue in test" in M6)
         jit.HLO.devirt.Runtime.RuntimeExtend1 (FAILED on windows x86_64, in
M6 it is recorded as not regression and started to fail on M5)
         reg.vm.btest6353.Btest6353 (ERROR on Windows x86_64, recorded also
failed on M3 and might be issue with the test)

    4.3) New regressions on 1 platform (looks intermittent)
         reg.jit.btest8029.Btest8029 (FAILED in Linux x86)
         func.reg.jit.btest5710.Btest5710 (FAILED in Linux x86)
         api.java.security.cert.F_CertPathTest_06.F_CertPathTest_06 (ERROR
in Linux x86)
         api.java.security.cert.F_CertPathTest_05.F_CertPathTest_05 (ERROR
in Linux x86)

 5) JDKTools Tests:
    Several timeouts are observed in Linux x86 snapshot running. They are:

org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch001

org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch002

org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch003

org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch004

org.apache.harmony.jpda.tests.jdwp.Events.CombinedEventsTest.testCombinedEvents_04

org.apache.harmony.jpda.tests.jdwp.MultiSession.AttachConnectorTest.testAttachConnector001

org.apache.harmony.jpda.tests.jdwp.MultiSession.MethodEntryExitTest.testMethodEvent001

org.apache.harmony.jpda.tests.jdwp.MultiSession.ResumeTest.testResume

org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadEndTest.testThreadEnd001

org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadStartTest.testThreadStart001
    The Linux-only timeouts are also observed in the integrity testing
results.
    JIRA HARMONY-5833 has been filed for one of them.

 6) JettyScenario:
    The Linux x86 running failed because of the unresolved HARMONY-5219.

 7) Reliability:
    Several failures are observed in windows x86_64 running.
    7.1) Old regressions
         api.net.DatagramTest (HARMONY-5531)
         api.text.DecimalFormat_Locales - (in M6 it is recorded as also
intermittent in M5)

    7.2) New/intemittent regressions
         api.kernel.thread.ThreadLocalTest.ThreadLocalTest
         api.kernel.exec.RunExec

 8) Stress
    Different test cases failed on different platforms.
    8.1) Timeouts on Linux x86.

stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD003.ThreadTest003

stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD007.ThreadTest007

stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD009.ThreadTest009

stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD011.ThreadTest011

    8.2) Failed cases on Windows x86_64 with unknown reasons.

stress.org.apache.harmony.test.stress.classloader.MixThreads.TreeClasses.testTreeClasses2

stress.org.apache.harmony.test.stress.classloader.NotSynchThreads.TreeClasses.testTreeClasses

    I have not found records for them.

 9) Strut_test
    Broken with same error report as M6.

 10) Eclipse Hello World Application.
    Although the testing framework just reports EUT-API status as "PASSED".
A fresh workspace running just failed in configuration stage (
http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805120106o387a49f1rfb2c33d1042d2f41@mail.gmail.com%3e)
since SVN commit 641928 (which committed patch for HARMONY-4569).

thanks,
chunrong

Re: [testing] first M7 candidate (r681495) testing status

Posted by chunrong lai <ch...@gmail.com>.
hi, xiaofeng:
    Sorry for the confusion. The reason that I think it is stabler is
because more M6 old issues (~30 on Linux x86 and Windows X86_64 of last
report) are not observed now, although I think some of them are intermittent
errors (they are still there but just not observed).


On 8/12/08, Xiao-Feng Li <xi...@gmail.com> wrote:
>
> Chunrong, thanks for the good work!
>
> You mentioned the M7 candidate has some new issues, does it have less
> old issues? I wonder why you think it is stabler.
>
> Thanks,
> xiaofeng
>
> On Tue, Aug 12, 2008 at 1:26 AM, chunrong lai <ch...@gmail.com>
> wrote:
> > Hi, all:
> >
> >  Here is r681495 snapshot testing status:
> > http://people.apache.org/~chunrong/snapshots/r681495/index.html. I am
> using
> > two platforms (Linux x86, windows x86_64) at the moment. Hopefully we
> will
> > have other two platforms in future for M8. Well, although we are testing
> > only two platforms for M7, my experience is that the status for another
> two
> > platforms should be not worse or just include some extra intermittent
> errors
> > which can be investigated in some later stages.
> >
> >  The following suites passed on Linux x86/Windows x86_64 platforms: Ant
> > Scenario (or self-hosting), Axis application, Dacapo, DRLVM regression
> > tests, Geronimo Unit Tests, Scimark, Tomcat scenario, VTS VM Test Suite.
> >
> >  Most of the failures are known issues (for M6). Although we can observe
> > 15~20 new issues, those issues happen only in 1 platform and they look
> more
> > like the intermittent/timeout issues (less reproducible) to me. Overall I
> > myself would like to think r681495 is more stable than M6.
> >
> >  Please add your comments and clarifications (please also see M6 testing
> > results http://people.apache.org/~smishura/r653525/,
> >
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805070304l38845ee0se01fb93fbfc05586@mail.gmail.com%3eand
> > integrity testing results
> > http://people.apache.org/~chunrong/harmony-integrity/ as a comparison).
> >
> >  1) Classlib:
> >    1.1) Since r644719 (which committed patch for HARMONY-5684)
> >
> >        org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest
> >
> >
> org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
> >
> >         failed in both platforms
> >
> >    1.2) Two failures
> >
> >        org.apache.harmony.luni.tests.java.net.MulticastSocketTest (Failed
> > in windows_x86 running of M6)
> >        tests.api.java.security.PermissionCollectionTest           (Failed
> > in M6)
> >
> >         are observed in Linux x86.
> >
> >  2) DRLVM tests:
> >    2.1) One failure
> >
> >         java.lang.ClassGenericsTest.test_2
> >
> >         is observed in Linux x86 snapshot testing.
> >         I can see some old discussion in the mailing list about that but
> I
> > am not sure the expected status here.
> >         They should be intemittent errors since the integrity testing
> just
> > run well mostly.
> >
> >  3) EUTs:
> >
> >    3.1) Linux x86: 99.36%
> >         A recorded JIRA for this suite is HARMONY-2914 which wastes file
> > handlers and makes system unstable.
> >
> >  4) Functional:
> >    4.1) Old regressions on both platforms:
> >         api.java.text.MessageFormat (HARMONY-5430)
> >         api.java.util.jar.Manifest  (HARMONY-5473)
> >         api.java.beans.beancontext.BeanContextTest (also in M6, recorded
> > as  regression caused by changes in locale data)
> >         api.java.beans.persistence.EncoderTest  (also in M6, recorded as
> > regression in the beans module)
> >         api.java.beans.persistence.EncoderDecoderTest (also in M6,
> > regression in the beans module)
> >         reg.vm.btest5625 (also in M6, recorded as intermittent and not
> > reproducible manually)
> >
> >    4.2) Old regressions on 1 platform
> >         api.java.rmi.basicexception (ERROR in Linux x86, HARMONY-5823)
> >         api.java.rmi.basicregistry.RemoteServerTest (ERROR in Linux x86,
> > HARMONY-5823)
> >         jpda.jdwp.scenario.ST07.ST07Test (ERROR in windows x86_64, in M6
> it
> > is recorded as regression since M4)
> >         java.math.F_BigIntegerMatrixMultiplyTest_01 (ERROR on Linux x86,
> > recorded as Timeout, not reproducible
> >  in M6)
> >         reg.vm.btest5717 (ERROR on Windows X86_64, recorded as "timeout,
> > the test is too heavy" in M6)
> >         jit.HLO.inline.ControlFlow.IfElse.IfElse1.IfElseTest1 (FAILED in
> > windows x86_64, recorded as "looks like
> > issue in test" in M6)
> >         jit.HLO.devirt.Runtime.RuntimeExtend1 (FAILED on windows x86_64,
> in
> > M6 it is recorded as not regression and started to fail on M5)
> >         reg.vm.btest6353.Btest6353 (ERROR on Windows x86_64, recorded
> also
> > failed on M3 and might be issue with the test)
> >
> >    4.3) New regressions on 1 platform (looks intermittent)
> >         reg.jit.btest8029.Btest8029 (FAILED in Linux x86)
> >         func.reg.jit.btest5710.Btest5710 (FAILED in Linux x86)
> >         api.java.security.cert.F_CertPathTest_06.F_CertPathTest_06 (ERROR
> > in Linux x86)
> >         api.java.security.cert.F_CertPathTest_05.F_CertPathTest_05 (ERROR
> > in Linux x86)
> >
> >  5) JDKTools Tests:
> >    Several timeouts are observed in Linux x86 snapshot running. They are:
> >
> >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch001
> >
> >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch002
> >
> >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch003
> >
> >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch004
> >
> >
> org.apache.harmony.jpda.tests.jdwp.Events.CombinedEventsTest.testCombinedEvents_04
> >
> >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.AttachConnectorTest.testAttachConnector001
> >
> >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.MethodEntryExitTest.testMethodEvent001
> >
> > org.apache.harmony.jpda.tests.jdwp.MultiSession.ResumeTest.testResume
> >
> >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadEndTest.testThreadEnd001
> >
> >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadStartTest.testThreadStart001
> >    The Linux-only timeouts are also observed in the integrity testing
> > results.
> >    JIRA HARMONY-5833 has been filed for one of them.
> >
> >  6) JettyScenario:
> >    The Linux x86 running failed because of the unresolved HARMONY-5219.
> >
> >  7) Reliability:
> >    Several failures are observed in windows x86_64 running.
> >    7.1) Old regressions
> >         api.net.DatagramTest (HARMONY-5531)
> >         api.text.DecimalFormat_Locales - (in M6 it is recorded as also
> > intermittent in M5)
> >
> >    7.2) New/intemittent regressions
> >         api.kernel.thread.ThreadLocalTest.ThreadLocalTest
> >         api.kernel.exec.RunExec
> >
> >  8) Stress
> >    Different test cases failed on different platforms.
> >    8.1) Timeouts on Linux x86.
> >
> >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD003.ThreadTest003
> >
> >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD007.ThreadTest007
> >
> >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD009.ThreadTest009
> >
> >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD011.ThreadTest011
> >
> >    8.2) Failed cases on Windows x86_64 with unknown reasons.
> >
> >
> stress.org.apache.harmony.test.stress.classloader.MixThreads.TreeClasses.testTreeClasses2
> >
> >
> stress.org.apache.harmony.test.stress.classloader.NotSynchThreads.TreeClasses.testTreeClasses
> >
> >    I have not found records for them.
> >
> >  9) Strut_test
> >    Broken with same error report as M6.
> >
> >  10) Eclipse Hello World Application.
> >    Although the testing framework just reports EUT-API status as
> "PASSED".
> > A fresh workspace running just failed in configuration stage (
> >
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805120106o387a49f1rfb2c33d1042d2f41@mail.gmail.com%3e
> )
> > since SVN commit 641928 (which committed patch for HARMONY-4569).
> >
> > thanks,
> > chunrong
> >
>
>
>
> --
> http://xiao-feng.blogspot.com
>

Re: [testing] first M7 candidate (r681495) testing status

Posted by Xiao-Feng Li <xi...@gmail.com>.
Chunrong, thanks for the good work!

You mentioned the M7 candidate has some new issues, does it have less
old issues? I wonder why you think it is stabler.

Thanks,
xiaofeng

On Tue, Aug 12, 2008 at 1:26 AM, chunrong lai <ch...@gmail.com> wrote:
> Hi, all:
>
>  Here is r681495 snapshot testing status:
> http://people.apache.org/~chunrong/snapshots/r681495/index.html. I am using
> two platforms (Linux x86, windows x86_64) at the moment. Hopefully we will
> have other two platforms in future for M8. Well, although we are testing
> only two platforms for M7, my experience is that the status for another two
> platforms should be not worse or just include some extra intermittent errors
> which can be investigated in some later stages.
>
>  The following suites passed on Linux x86/Windows x86_64 platforms: Ant
> Scenario (or self-hosting), Axis application, Dacapo, DRLVM regression
> tests, Geronimo Unit Tests, Scimark, Tomcat scenario, VTS VM Test Suite.
>
>  Most of the failures are known issues (for M6). Although we can observe
> 15~20 new issues, those issues happen only in 1 platform and they look more
> like the intermittent/timeout issues (less reproducible) to me. Overall I
> myself would like to think r681495 is more stable than M6.
>
>  Please add your comments and clarifications (please also see M6 testing
> results http://people.apache.org/~smishura/r653525/,
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805070304l38845ee0se01fb93fbfc05586@mail.gmail.com%3eand
> integrity testing results
> http://people.apache.org/~chunrong/harmony-integrity/ as a comparison).
>
>  1) Classlib:
>    1.1) Since r644719 (which committed patch for HARMONY-5684)
>
>        org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest
>
> org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
>
>         failed in both platforms
>
>    1.2) Two failures
>
>        org.apache.harmony.luni.tests.java.net.MulticastSocketTest (Failed
> in windows_x86 running of M6)
>        tests.api.java.security.PermissionCollectionTest           (Failed
> in M6)
>
>         are observed in Linux x86.
>
>  2) DRLVM tests:
>    2.1) One failure
>
>         java.lang.ClassGenericsTest.test_2
>
>         is observed in Linux x86 snapshot testing.
>         I can see some old discussion in the mailing list about that but I
> am not sure the expected status here.
>         They should be intemittent errors since the integrity testing just
> run well mostly.
>
>  3) EUTs:
>
>    3.1) Linux x86: 99.36%
>         A recorded JIRA for this suite is HARMONY-2914 which wastes file
> handlers and makes system unstable.
>
>  4) Functional:
>    4.1) Old regressions on both platforms:
>         api.java.text.MessageFormat (HARMONY-5430)
>         api.java.util.jar.Manifest  (HARMONY-5473)
>         api.java.beans.beancontext.BeanContextTest (also in M6, recorded
> as  regression caused by changes in locale data)
>         api.java.beans.persistence.EncoderTest  (also in M6, recorded as
> regression in the beans module)
>         api.java.beans.persistence.EncoderDecoderTest (also in M6,
> regression in the beans module)
>         reg.vm.btest5625 (also in M6, recorded as intermittent and not
> reproducible manually)
>
>    4.2) Old regressions on 1 platform
>         api.java.rmi.basicexception (ERROR in Linux x86, HARMONY-5823)
>         api.java.rmi.basicregistry.RemoteServerTest (ERROR in Linux x86,
> HARMONY-5823)
>         jpda.jdwp.scenario.ST07.ST07Test (ERROR in windows x86_64, in M6 it
> is recorded as regression since M4)
>         java.math.F_BigIntegerMatrixMultiplyTest_01 (ERROR on Linux x86,
> recorded as Timeout, not reproducible
>  in M6)
>         reg.vm.btest5717 (ERROR on Windows X86_64, recorded as "timeout,
> the test is too heavy" in M6)
>         jit.HLO.inline.ControlFlow.IfElse.IfElse1.IfElseTest1 (FAILED in
> windows x86_64, recorded as "looks like
> issue in test" in M6)
>         jit.HLO.devirt.Runtime.RuntimeExtend1 (FAILED on windows x86_64, in
> M6 it is recorded as not regression and started to fail on M5)
>         reg.vm.btest6353.Btest6353 (ERROR on Windows x86_64, recorded also
> failed on M3 and might be issue with the test)
>
>    4.3) New regressions on 1 platform (looks intermittent)
>         reg.jit.btest8029.Btest8029 (FAILED in Linux x86)
>         func.reg.jit.btest5710.Btest5710 (FAILED in Linux x86)
>         api.java.security.cert.F_CertPathTest_06.F_CertPathTest_06 (ERROR
> in Linux x86)
>         api.java.security.cert.F_CertPathTest_05.F_CertPathTest_05 (ERROR
> in Linux x86)
>
>  5) JDKTools Tests:
>    Several timeouts are observed in Linux x86 snapshot running. They are:
>
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch001
>
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch002
>
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch003
>
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch004
>
> org.apache.harmony.jpda.tests.jdwp.Events.CombinedEventsTest.testCombinedEvents_04
>
> org.apache.harmony.jpda.tests.jdwp.MultiSession.AttachConnectorTest.testAttachConnector001
>
> org.apache.harmony.jpda.tests.jdwp.MultiSession.MethodEntryExitTest.testMethodEvent001
>
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ResumeTest.testResume
>
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadEndTest.testThreadEnd001
>
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadStartTest.testThreadStart001
>    The Linux-only timeouts are also observed in the integrity testing
> results.
>    JIRA HARMONY-5833 has been filed for one of them.
>
>  6) JettyScenario:
>    The Linux x86 running failed because of the unresolved HARMONY-5219.
>
>  7) Reliability:
>    Several failures are observed in windows x86_64 running.
>    7.1) Old regressions
>         api.net.DatagramTest (HARMONY-5531)
>         api.text.DecimalFormat_Locales - (in M6 it is recorded as also
> intermittent in M5)
>
>    7.2) New/intemittent regressions
>         api.kernel.thread.ThreadLocalTest.ThreadLocalTest
>         api.kernel.exec.RunExec
>
>  8) Stress
>    Different test cases failed on different platforms.
>    8.1) Timeouts on Linux x86.
>
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD003.ThreadTest003
>
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD007.ThreadTest007
>
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD009.ThreadTest009
>
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD011.ThreadTest011
>
>    8.2) Failed cases on Windows x86_64 with unknown reasons.
>
> stress.org.apache.harmony.test.stress.classloader.MixThreads.TreeClasses.testTreeClasses2
>
> stress.org.apache.harmony.test.stress.classloader.NotSynchThreads.TreeClasses.testTreeClasses
>
>    I have not found records for them.
>
>  9) Strut_test
>    Broken with same error report as M6.
>
>  10) Eclipse Hello World Application.
>    Although the testing framework just reports EUT-API status as "PASSED".
> A fresh workspace running just failed in configuration stage (
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805120106o387a49f1rfb2c33d1042d2f41@mail.gmail.com%3e)
> since SVN commit 641928 (which committed patch for HARMONY-4569).
>
> thanks,
> chunrong
>



-- 
http://xiao-feng.blogspot.com

Re: [testing] first M7 candidate (r681495) testing status

Posted by Gregory Shimansky <gs...@apache.org>.
On 23 August 2008 Nathan Beyer wrote:
> Hmm. I guess I'm not familiar enough with what's be tested.
>
> Given your final statement, should we add some checking to the build
> scripts that test the build system-level locale and and error out if
> it's not support. If not erroring out, then skipping these tests with
> a big warning message.

I think at least it makes sense to add this to the requirements on the site. 
If you know how to determine that system local is truly unicode, probably it 
would be good to add such check to the test scripts.

> On Fri, Aug 22, 2008 at 7:48 AM, Gregory Shimansky
>
> <gs...@apache.org> wrote:
> > Nathan Beyer said the following on 21.08.2008 17:51:
> >> Can we tweak the Locale during the test case, so that the Locale
> >> matches the fixture data?
> >
> > The locale during test case is not enough. It is locale during build time
> > is what is important. Test produce lots of classes with international
> > characters in their names. These classes have to be somehow written to
> > the file system when they are compiled.
> >
> > If system locale is set to some value that doesn't support some
> > international characters, then compiled class files contain "?" symbols
> > in their names. Obviously class files with such names cannot be found
> > later when they are searched by class loader.
> >
> > I don't know in which international characters are used in the tests, but
> > it looks like they weren't picked up from a particular language but were
> > selected randomly from the whole possible set of the unicode symbols.
> >
> > So only with locales that are truly unicode, not some subset of it (e.g
> > "C", "ASCII", "POSIX", "ru_RU.KOI8-R, zh_CN.GB2312 are all subsets for
> > some particular language and don't support any other languages), it is
> > possibly to successfully build and execute these tests.
> >
> >> On Thu, Aug 21, 2008 at 8:36 AM, Gregory Shimansky
> >>
> >> <gs...@apache.org> wrote:
> >>> chunrong lai said the following on 21.08.2008 6:51:
> >>>> hi,:
> >>>>  I got an update about the failure in
> >>>> java.lang.ClassGenericsTest.test_2 in
> >>>> DRLVM_test.
> >>>>  I see the test case is using some harded-UTF-encoded class-name
> >>>> during the
> >>>> test. So the test only passes if the locale is set to "en_US.UTF-8" or
> >>>> "zh_CN.utf8". If the locale is set to "C" or "zh_CN.GB2312" it just
> >>>> fails.
> >>>> RI also can not pass the test case in such locales. So it looks like a
> >>>> test
> >>>> case (or environment setting) problem.
> >>>
> >>> Yes, I investigated this test's failure two years ago :) See [1] and
> >>> replies
> >>> from Salikh, Alexey and Rana.
> >>>
> >>> [1] http://markmail.org/message/ku3juqhnyhysy5gb
> >>>
> >>> --
> >>> Gregory
> >
> > --
> > Gregory



-- 
Gregory

Re: [testing] first M7 candidate (r681495) testing status

Posted by Nathan Beyer <nd...@apache.org>.
Hmm. I guess I'm not familiar enough with what's be tested.

Given your final statement, should we add some checking to the build
scripts that test the build system-level locale and and error out if
it's not support. If not erroring out, then skipping these tests with
a big warning message.

-Nathan

On Fri, Aug 22, 2008 at 7:48 AM, Gregory Shimansky
<gs...@apache.org> wrote:
> Nathan Beyer said the following on 21.08.2008 17:51:
>>
>> Can we tweak the Locale during the test case, so that the Locale
>> matches the fixture data?
>
> The locale during test case is not enough. It is locale during build time is
> what is important. Test produce lots of classes with international
> characters in their names. These classes have to be somehow written to the
> file system when they are compiled.
>
> If system locale is set to some value that doesn't support some
> international characters, then compiled class files contain "?" symbols in
> their names. Obviously class files with such names cannot be found later
> when they are searched by class loader.
>
> I don't know in which international characters are used in the tests, but it
> looks like they weren't picked up from a particular language but were
> selected randomly from the whole possible set of the unicode symbols.
>
> So only with locales that are truly unicode, not some subset of it (e.g "C",
> "ASCII", "POSIX", "ru_RU.KOI8-R, zh_CN.GB2312 are all subsets for some
> particular language and don't support any other languages), it is possibly
> to successfully build and execute these tests.
>
>> On Thu, Aug 21, 2008 at 8:36 AM, Gregory Shimansky
>> <gs...@apache.org> wrote:
>>>
>>> chunrong lai said the following on 21.08.2008 6:51:
>>>>
>>>> hi,:
>>>>  I got an update about the failure in java.lang.ClassGenericsTest.test_2
>>>> in
>>>> DRLVM_test.
>>>>  I see the test case is using some harded-UTF-encoded class-name during
>>>> the
>>>> test. So the test only passes if the locale is set to "en_US.UTF-8" or
>>>> "zh_CN.utf8". If the locale is set to "C" or "zh_CN.GB2312" it just
>>>> fails.
>>>> RI also can not pass the test case in such locales. So it looks like a
>>>> test
>>>> case (or environment setting) problem.
>>>
>>> Yes, I investigated this test's failure two years ago :) See [1] and
>>> replies
>>> from Salikh, Alexey and Rana.
>>>
>>> [1] http://markmail.org/message/ku3juqhnyhysy5gb
>>>
>>> --
>>> Gregory
>>>
>>>
>
>
> --
> Gregory
>
>

Re: [testing] first M7 candidate (r681495) testing status

Posted by Gregory Shimansky <gs...@apache.org>.
Nathan Beyer said the following on 21.08.2008 17:51:
> Can we tweak the Locale during the test case, so that the Locale
> matches the fixture data?

The locale during test case is not enough. It is locale during build 
time is what is important. Test produce lots of classes with 
international characters in their names. These classes have to be 
somehow written to the file system when they are compiled.

If system locale is set to some value that doesn't support some 
international characters, then compiled class files contain "?" symbols 
in their names. Obviously class files with such names cannot be found 
later when they are searched by class loader.

I don't know in which international characters are used in the tests, 
but it looks like they weren't picked up from a particular language but 
were selected randomly from the whole possible set of the unicode symbols.

So only with locales that are truly unicode, not some subset of it (e.g 
"C", "ASCII", "POSIX", "ru_RU.KOI8-R, zh_CN.GB2312 are all subsets for 
some particular language and don't support any other languages), it is 
possibly to successfully build and execute these tests.

> On Thu, Aug 21, 2008 at 8:36 AM, Gregory Shimansky
> <gs...@apache.org> wrote:
>> chunrong lai said the following on 21.08.2008 6:51:
>>> hi,:
>>>  I got an update about the failure in java.lang.ClassGenericsTest.test_2
>>> in
>>> DRLVM_test.
>>>  I see the test case is using some harded-UTF-encoded class-name during
>>> the
>>> test. So the test only passes if the locale is set to "en_US.UTF-8" or
>>> "zh_CN.utf8". If the locale is set to "C" or "zh_CN.GB2312" it just fails.
>>> RI also can not pass the test case in such locales. So it looks like a
>>> test
>>> case (or environment setting) problem.
>> Yes, I investigated this test's failure two years ago :) See [1] and replies
>> from Salikh, Alexey and Rana.
>>
>> [1] http://markmail.org/message/ku3juqhnyhysy5gb
>>
>> --
>> Gregory
>>
>>


-- 
Gregory


Re: [testing] first M7 candidate (r681495) testing status

Posted by Nathan Beyer <nd...@apache.org>.
Can we tweak the Locale during the test case, so that the Locale
matches the fixture data?

-Nathan

On Thu, Aug 21, 2008 at 8:36 AM, Gregory Shimansky
<gs...@apache.org> wrote:
> chunrong lai said the following on 21.08.2008 6:51:
>>
>> hi,:
>>  I got an update about the failure in java.lang.ClassGenericsTest.test_2
>> in
>> DRLVM_test.
>>  I see the test case is using some harded-UTF-encoded class-name during
>> the
>> test. So the test only passes if the locale is set to "en_US.UTF-8" or
>> "zh_CN.utf8". If the locale is set to "C" or "zh_CN.GB2312" it just fails.
>> RI also can not pass the test case in such locales. So it looks like a
>> test
>> case (or environment setting) problem.
>
> Yes, I investigated this test's failure two years ago :) See [1] and replies
> from Salikh, Alexey and Rana.
>
> [1] http://markmail.org/message/ku3juqhnyhysy5gb
>
> --
> Gregory
>
>

Re: [testing] first M7 candidate (r681495) testing status

Posted by Gregory Shimansky <gs...@apache.org>.
chunrong lai said the following on 21.08.2008 6:51:
> hi,:
>  I got an update about the failure in java.lang.ClassGenericsTest.test_2 in
> DRLVM_test.
>  I see the test case is using some harded-UTF-encoded class-name during the
> test. So the test only passes if the locale is set to "en_US.UTF-8" or
> "zh_CN.utf8". If the locale is set to "C" or "zh_CN.GB2312" it just fails.
> RI also can not pass the test case in such locales. So it looks like a test
> case (or environment setting) problem.

Yes, I investigated this test's failure two years ago :) See [1] and 
replies from Salikh, Alexey and Rana.

[1] http://markmail.org/message/ku3juqhnyhysy5gb

-- 
Gregory


Re: [testing] first M7 candidate (r681495) testing status

Posted by Xiao-Feng Li <xi...@gmail.com>.
Good finding and interesting!  Thanks, -xiaofeng

On Thu, Aug 21, 2008 at 10:51 AM, chunrong lai <ch...@gmail.com> wrote:
> hi,:
>  I got an update about the failure in java.lang.ClassGenericsTest.test_2 in
> DRLVM_test.
>  I see the test case is using some harded-UTF-encoded class-name during the
> test. So the test only passes if the locale is set to "en_US.UTF-8" or
> "zh_CN.utf8". If the locale is set to "C" or "zh_CN.GB2312" it just fails.
> RI also can not pass the test case in such locales. So it looks like a test
> case (or environment setting) problem.
>
>  Thanks.
>
> On 8/12/08, chunrong lai <ch...@gmail.com> wrote:
>>
>>
>>
>> Hi, all:
>>
>>   Here is r681495 snapshot testing status:
>> http://people.apache.org/~chunrong/snapshots/r681495/index.html. I am
>> using two platforms (Linux x86, windows x86_64) at the moment. Hopefully we
>> will have other two platforms in future for M8. Well, although we are
>> testing only two platforms for M7, my experience is that the status for
>> another two platforms should be not worse or just include some extra
>> intermittent errors which can be investigated in some later stages.
>>
>>   The following suites passed on Linux x86/Windows x86_64 platforms: Ant
>> Scenario (or self-hosting), Axis application, Dacapo, DRLVM regression
>> tests, Geronimo Unit Tests, Scimark, Tomcat scenario, VTS VM Test Suite.
>>
>>   Most of the failures are known issues (for M6). Although we can observe
>> 15~20 new issues, those issues happen only in 1 platform and they look more
>> like the intermittent/timeout issues (less reproducible) to me. Overall I
>> myself would like to think r681495 is more stable than M6.
>>
>>   Please add your comments and clarifications (please also see M6 testing
>> results http://people.apache.org/~smishura/r653525/,
>> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805070304l38845ee0se01fb93fbfc05586@mail.gmail.com%3eand integrity testing results
>> http://people.apache.org/~chunrong/harmony-integrity/ as a comparison).
>>
>>  1) Classlib:
>>     1.1) Since r644719 (which committed patch for HARMONY-5684)
>>
>>         org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest
>>
>> org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
>>
>>          failed in both platforms
>>
>>     1.2) Two failures
>>
>>         org.apache.harmony.luni.tests.java.net.MulticastSocketTest (Failed
>> in windows_x86 running of M6)
>>         tests.api.java.security.PermissionCollectionTest           (Failed
>> in M6)
>>
>>          are observed in Linux x86.
>>
>>  2) DRLVM tests:
>>     2.1) One failure
>>
>>          java.lang.ClassGenericsTest.test_2
>>
>>          is observed in Linux x86 snapshot testing.
>>          I can see some old discussion in the mailing list about that but I
>> am not sure the expected status here.
>>          They should be intemittent errors since the integrity testing just
>> run well mostly.
>>
>>  3) EUTs:
>>
>>     3.1) Linux x86: 99.36%
>>          A recorded JIRA for this suite is HARMONY-2914 which wastes file
>> handlers and makes system unstable.
>>
>>  4) Functional:
>>     4.1) Old regressions on both platforms:
>>          api.java.text.MessageFormat (HARMONY-5430)
>>          api.java.util.jar.Manifest  (HARMONY-5473)
>>          api.java.beans.beancontext.BeanContextTest (also in M6, recorded
>> as  regression caused by changes in locale data)
>>          api.java.beans.persistence.EncoderTest  (also in M6, recorded as
>> regression in the beans module)
>>          api.java.beans.persistence.EncoderDecoderTest (also in M6,
>> regression in the beans module)
>>          reg.vm.btest5625 (also in M6, recorded as intermittent and not
>> reproducible manually)
>>
>>     4.2) Old regressions on 1 platform
>>          api.java.rmi.basicexception (ERROR in Linux x86, HARMONY-5823)
>>          api.java.rmi.basicregistry.RemoteServerTest (ERROR in Linux x86,
>> HARMONY-5823)
>>          jpda.jdwp.scenario.ST07.ST07Test (ERROR in windows x86_64, in M6
>> it is recorded as regression since M4)
>>          java.math.F_BigIntegerMatrixMultiplyTest_01 (ERROR on Linux x86,
>> recorded as Timeout, not reproducible
>>  in M6)
>>          reg.vm.btest5717 (ERROR on Windows X86_64, recorded as "timeout,
>> the test is too heavy" in M6)
>>          jit.HLO.inline.ControlFlow.IfElse.IfElse1.IfElseTest1 (FAILED in
>> windows x86_64, recorded as "looks like
>> issue in test" in M6)
>>          jit.HLO.devirt.Runtime.RuntimeExtend1 (FAILED on windows x86_64,
>> in M6 it is recorded as not regression and started to fail on M5)
>>          reg.vm.btest6353.Btest6353 (ERROR on Windows x86_64, recorded also
>> failed on M3 and might be issue with the test)
>>
>>     4.3) New regressions on 1 platform (looks intermittent)
>>          reg.jit.btest8029.Btest8029 (FAILED in Linux x86)
>>          func.reg.jit.btest5710.Btest5710 (FAILED in Linux x86)
>>          api.java.security.cert.F_CertPathTest_06.F_CertPathTest_06 (ERROR
>> in Linux x86)
>>          api.java.security.cert.F_CertPathTest_05.F_CertPathTest_05 (ERROR
>> in Linux x86)
>>
>>  5) JDKTools Tests:
>>     Several timeouts are observed in Linux x86 snapshot running. They are:
>>
>> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch001
>>
>> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch002
>>
>> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch003
>>
>> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch004
>>
>> org.apache.harmony.jpda.tests.jdwp.Events.CombinedEventsTest.testCombinedEvents_04
>>
>> org.apache.harmony.jpda.tests.jdwp.MultiSession.AttachConnectorTest.testAttachConnector001
>>
>> org.apache.harmony.jpda.tests.jdwp.MultiSession.MethodEntryExitTest.testMethodEvent001
>>
>> org.apache.harmony.jpda.tests.jdwp.MultiSession.ResumeTest.testResume
>>
>> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadEndTest.testThreadEnd001
>>
>> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadStartTest.testThreadStart001
>>     The Linux-only timeouts are also observed in the integrity testing
>> results.
>>     JIRA HARMONY-5833 has been filed for one of them.
>>
>>  6) JettyScenario:
>>     The Linux x86 running failed because of the unresolved HARMONY-5219.
>>
>>  7) Reliability:
>>     Several failures are observed in windows x86_64 running.
>>     7.1) Old regressions
>>          api.net.DatagramTest (HARMONY-5531)
>>          api.text.DecimalFormat_Locales - (in M6 it is recorded as also
>> intermittent in M5)
>>
>>     7.2) New/intemittent regressions
>>          api.kernel.thread.ThreadLocalTest.ThreadLocalTest
>>          api.kernel.exec.RunExec
>>
>>  8) Stress
>>     Different test cases failed on different platforms.
>>     8.1) Timeouts on Linux x86.
>>
>> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD003.ThreadTest003
>>
>> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD007.ThreadTest007
>>
>> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD009.ThreadTest009
>>
>> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD011.ThreadTest011
>>
>>     8.2) Failed cases on Windows x86_64 with unknown reasons.
>>
>> stress.org.apache.harmony.test.stress.classloader.MixThreads.TreeClasses.testTreeClasses2
>>
>> stress.org.apache.harmony.test.stress.classloader.NotSynchThreads.TreeClasses.testTreeClasses
>>
>>     I have not found records for them.
>>
>>  9) Strut_test
>>     Broken with same error report as M6.
>>
>>  10) Eclipse Hello World Application.
>>     Although the testing framework just reports EUT-API status as "PASSED".
>> A fresh workspace running just failed in configuration stage (
>> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805120106o387a49f1rfb2c33d1042d2f41@mail.gmail.com%3e)
>> since SVN commit 641928 (which committed patch for HARMONY-4569).
>>
>> thanks,
>> chunrong
>>
>



-- 
http://xiao-feng.blogspot.com

Re: [testing] first M7 candidate (r681495) testing status

Posted by chunrong lai <ch...@gmail.com>.
hi,:
 I got an update about the failure in java.lang.ClassGenericsTest.test_2 in
DRLVM_test.
 I see the test case is using some harded-UTF-encoded class-name during the
test. So the test only passes if the locale is set to "en_US.UTF-8" or
"zh_CN.utf8". If the locale is set to "C" or "zh_CN.GB2312" it just fails.
RI also can not pass the test case in such locales. So it looks like a test
case (or environment setting) problem.

 Thanks.

On 8/12/08, chunrong lai <ch...@gmail.com> wrote:
>
>
>
> Hi, all:
>
>   Here is r681495 snapshot testing status:
> http://people.apache.org/~chunrong/snapshots/r681495/index.html. I am
> using two platforms (Linux x86, windows x86_64) at the moment. Hopefully we
> will have other two platforms in future for M8. Well, although we are
> testing only two platforms for M7, my experience is that the status for
> another two platforms should be not worse or just include some extra
> intermittent errors which can be investigated in some later stages.
>
>   The following suites passed on Linux x86/Windows x86_64 platforms: Ant
> Scenario (or self-hosting), Axis application, Dacapo, DRLVM regression
> tests, Geronimo Unit Tests, Scimark, Tomcat scenario, VTS VM Test Suite.
>
>   Most of the failures are known issues (for M6). Although we can observe
> 15~20 new issues, those issues happen only in 1 platform and they look more
> like the intermittent/timeout issues (less reproducible) to me. Overall I
> myself would like to think r681495 is more stable than M6.
>
>   Please add your comments and clarifications (please also see M6 testing
> results http://people.apache.org/~smishura/r653525/,
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805070304l38845ee0se01fb93fbfc05586@mail.gmail.com%3eand integrity testing results
> http://people.apache.org/~chunrong/harmony-integrity/ as a comparison).
>
>  1) Classlib:
>     1.1) Since r644719 (which committed patch for HARMONY-5684)
>
>         org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest
>
> org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
>
>          failed in both platforms
>
>     1.2) Two failures
>
>         org.apache.harmony.luni.tests.java.net.MulticastSocketTest (Failed
> in windows_x86 running of M6)
>         tests.api.java.security.PermissionCollectionTest           (Failed
> in M6)
>
>          are observed in Linux x86.
>
>  2) DRLVM tests:
>     2.1) One failure
>
>          java.lang.ClassGenericsTest.test_2
>
>          is observed in Linux x86 snapshot testing.
>          I can see some old discussion in the mailing list about that but I
> am not sure the expected status here.
>          They should be intemittent errors since the integrity testing just
> run well mostly.
>
>  3) EUTs:
>
>     3.1) Linux x86: 99.36%
>          A recorded JIRA for this suite is HARMONY-2914 which wastes file
> handlers and makes system unstable.
>
>  4) Functional:
>     4.1) Old regressions on both platforms:
>          api.java.text.MessageFormat (HARMONY-5430)
>          api.java.util.jar.Manifest  (HARMONY-5473)
>          api.java.beans.beancontext.BeanContextTest (also in M6, recorded
> as  regression caused by changes in locale data)
>          api.java.beans.persistence.EncoderTest  (also in M6, recorded as
> regression in the beans module)
>          api.java.beans.persistence.EncoderDecoderTest (also in M6,
> regression in the beans module)
>          reg.vm.btest5625 (also in M6, recorded as intermittent and not
> reproducible manually)
>
>     4.2) Old regressions on 1 platform
>          api.java.rmi.basicexception (ERROR in Linux x86, HARMONY-5823)
>          api.java.rmi.basicregistry.RemoteServerTest (ERROR in Linux x86,
> HARMONY-5823)
>          jpda.jdwp.scenario.ST07.ST07Test (ERROR in windows x86_64, in M6
> it is recorded as regression since M4)
>          java.math.F_BigIntegerMatrixMultiplyTest_01 (ERROR on Linux x86,
> recorded as Timeout, not reproducible
>  in M6)
>          reg.vm.btest5717 (ERROR on Windows X86_64, recorded as "timeout,
> the test is too heavy" in M6)
>          jit.HLO.inline.ControlFlow.IfElse.IfElse1.IfElseTest1 (FAILED in
> windows x86_64, recorded as "looks like
> issue in test" in M6)
>          jit.HLO.devirt.Runtime.RuntimeExtend1 (FAILED on windows x86_64,
> in M6 it is recorded as not regression and started to fail on M5)
>          reg.vm.btest6353.Btest6353 (ERROR on Windows x86_64, recorded also
> failed on M3 and might be issue with the test)
>
>     4.3) New regressions on 1 platform (looks intermittent)
>          reg.jit.btest8029.Btest8029 (FAILED in Linux x86)
>          func.reg.jit.btest5710.Btest5710 (FAILED in Linux x86)
>          api.java.security.cert.F_CertPathTest_06.F_CertPathTest_06 (ERROR
> in Linux x86)
>          api.java.security.cert.F_CertPathTest_05.F_CertPathTest_05 (ERROR
> in Linux x86)
>
>  5) JDKTools Tests:
>     Several timeouts are observed in Linux x86 snapshot running. They are:
>
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch001
>
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch002
>
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch003
>
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch004
>
> org.apache.harmony.jpda.tests.jdwp.Events.CombinedEventsTest.testCombinedEvents_04
>
> org.apache.harmony.jpda.tests.jdwp.MultiSession.AttachConnectorTest.testAttachConnector001
>
> org.apache.harmony.jpda.tests.jdwp.MultiSession.MethodEntryExitTest.testMethodEvent001
>
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ResumeTest.testResume
>
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadEndTest.testThreadEnd001
>
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadStartTest.testThreadStart001
>     The Linux-only timeouts are also observed in the integrity testing
> results.
>     JIRA HARMONY-5833 has been filed for one of them.
>
>  6) JettyScenario:
>     The Linux x86 running failed because of the unresolved HARMONY-5219.
>
>  7) Reliability:
>     Several failures are observed in windows x86_64 running.
>     7.1) Old regressions
>          api.net.DatagramTest (HARMONY-5531)
>          api.text.DecimalFormat_Locales - (in M6 it is recorded as also
> intermittent in M5)
>
>     7.2) New/intemittent regressions
>          api.kernel.thread.ThreadLocalTest.ThreadLocalTest
>          api.kernel.exec.RunExec
>
>  8) Stress
>     Different test cases failed on different platforms.
>     8.1) Timeouts on Linux x86.
>
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD003.ThreadTest003
>
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD007.ThreadTest007
>
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD009.ThreadTest009
>
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD011.ThreadTest011
>
>     8.2) Failed cases on Windows x86_64 with unknown reasons.
>
> stress.org.apache.harmony.test.stress.classloader.MixThreads.TreeClasses.testTreeClasses2
>
> stress.org.apache.harmony.test.stress.classloader.NotSynchThreads.TreeClasses.testTreeClasses
>
>     I have not found records for them.
>
>  9) Strut_test
>     Broken with same error report as M6.
>
>  10) Eclipse Hello World Application.
>     Although the testing framework just reports EUT-API status as "PASSED".
> A fresh workspace running just failed in configuration stage (
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805120106o387a49f1rfb2c33d1042d2f41@mail.gmail.com%3e)
> since SVN commit 641928 (which committed patch for HARMONY-4569).
>
> thanks,
> chunrong
>

Re: [testing] first M7 candidate (r681495) testing status

Posted by Nathan Beyer <nd...@apache.org>.
We should get this integrated into buildtest directly and documented.
This after all, is the whole point of buildtest.

-Nathan

On Tue, Aug 12, 2008 at 9:40 PM, chunrong lai <ch...@gmail.com> wrote:
> Thanks.
> (1) Setting for e-mail notifications are in framework.local.properties as
> http://harmony.apache.org/subcomponents/buildtest/howto.html.
>
> # Setup for mail notifications
> framework.parameters.usemail=...
> framework.parameters.usemail.to=alerts@harmony.apache.org
> framework.parameters.usemail.from=nomail@intel.com
>
> One may need to copy mail.jar and activation.jar from
> infra/build/libs/cruisecontrol-bin-2.6.2/lib/ to $ANT_HOME/lib.
>
> (2) After running integrity testing the direcory set in
> framework.local.properties
>
> # Directory to store archived results
> framework.parameters.optional.upload.dir=*<results_dir>*
>
> will take effect thus an archived results will be stored there. The file
> name is like
>
> [laichunrong@MRObj-lnx32-03 infra]$ ls ~/ccresult/build/to_upload/
> autouploading                        linux_x86.drlvm-test.status.zip
> linux_x86.jdktools-test.status.zip
> linux_x86.classlib-test.status.zip   linux_x86.ehwa-api.status.zip
> linux_x86.make-snapshot.status.zip
> linux_x86.drlvm-reg-test.status.zip  linux_x86.hdk.status.zip
> uploadresults.sh
>
> (3)
> http://svn.apache.org/repos/asf/harmony/enhanced/buildtest/trunk/tools/integrity/status-table/
>
> include the table and perl scripts to generate the integrity testing
> result page from above results.
>
> One needs to specify the result file/web page location and can configure the
> included platforms/test suites there to enable it.
>
> So having a web page to look at every evening can be achieved by
> uploading/extracting the results files and running the perl scripts
> everyday. I see all required settings have been documented.
> (4) We also added local contents to cc-exec-adaptor.xml in BTI for the
> auto-uploading/web-page generating. It has not been committed/documented.
>
>             update="true" />
>
> +        <available property="autouploads"
> file="${upload.dir}/autouploading"/>
> +        <antcall target="-auto-upload"/>
>    </target>
>
> +    <target name="-auto-upload" if="autouploads">
> +        <exec executable="${upload.dir}/uploadresults.sh"
> dir="${upload.dir}">
> +        </exec>
> +    </target>
> </project>
> The scripts use scp to (auto)upload results and ssh to (auto)run above perl
> scripts remotely (required keys setting).
>
> Thanks.
>
> On 8/13/08, Nathan Beyer <nb...@gmail.com> wrote:
>>
>> On Mon, Aug 11, 2008 at 10:16 PM, chunrong lai <chunronglai@gmail.com
>> >wrote:
>>
>> >   Thanks.
>> >   I think the standard scripts produce that site although I always added
>> > publish/e-mail notification setup before install/run the test suites.
>>
>>
>> Where is it at, I'm not able to find it and I would really like to recreate
>> this? How do you get it published to people.apache.org? I don't want an
>> email, I just want a web page to look at every evening.
>>
>> Can you document how you set it up? We need to have this integrity thing
>> documented, so anyone can recreate it for the next milestone.
>>
>> -Nathan
>>
>>
>> >
>> >   It is understandable that some tests in classlib-test and drlvm-test
>> get
>> > failing report. We just need to understand the failures.
>> >   As reported in M6,
>> > org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest,
>> >
>> >
>> org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
>> > etc fail classlib-test.
>> >   Also we observed that java.lang.ClassGenericsTest.test_2 sometimes fail
>> > drlvm-test.
>> >   Another timeout issue of drlvm-test comes from
>> > thread.SmallStackThreadTest_jit, as discussed in HARMONY-4601, the test
>> > case
>> > may run very slow when system resource is tight and finally introduce a
>> > timeout. Re-running the suite generally fixes the timeout.
>> >
>> >
>> >
>> >
>> >
>> > On 8/12/08, Nathan Beyer <nd...@apache.org> wrote:
>> > >
>> > > It's the tests that are failing - both classlib-test and drlvm-test.
>> > >
>> > > What's the specific integrity setup that's being used for that site?
>> Does
>> > > the standard out-of-the-box buildtest scripts produce that site?
>> > >
>> > > -Nathan
>> > >
>> > > On Mon, Aug 11, 2008 at 9:05 PM, chunrong lai <ch...@gmail.com>
>> > > wrote:
>> > >
>> > > > hi, Nathan:
>> > > >     Steps to run testing cycle are listed in
>> > > > http://harmony.apache.org/subcomponents/buildtest/index.html. One
>> > > > integrity
>> > > > testing example is
>> > > >
>> http://harmony.apache.org/subcomponents/buildtest/howto.html#Extended.
>> > > >      What is the error message you met (in what step)? What is the
>> > > content
>> > > > of the framework.local.properties file? I think that will help to
>> > figure
>> > > > out
>> > > > the problem.
>> > > >
>> > > >
>> > > > On 8/12/08, Nathan Beyer <nd...@apache.org> wrote:
>> > > > >
>> > > > > What's being used to produce the integrity tests? I have an x86_64
>> > > Linux
>> > > > > box
>> > > > > that I can dedicate to testing. I've tried just using buildtest
>> with
>> > > > > 'classlib,drlvm,classib-test,drlvm-test', but I haven't gotten a
>> > clean
>> > > > pass
>> > > > > yet.
>> > > > >
>> > > > > -Nathan
>> > > > >
>> > > > > On Mon, Aug 11, 2008 at 12:26 PM, chunrong lai <
>> > chunronglai@gmail.com
>> > > > > >wrote:
>> > > > >
>> > > > > > Hi, all:
>> > > > > >
>> > > > > >  Here is r681495 snapshot testing status:
>> > > > > > http://people.apache.org/~chunrong/snapshots/r681495/index.html<
>> http://people.apache.org/%7Echunrong/snapshots/r681495/index.html>
>> > <
>> > > http://people.apache.org/%7Echunrong/snapshots/r681495/index.html>
>> > > > <
>> > > > > http://people.apache.org/%7Echunrong/snapshots/r681495/index.html
>> >.
>> > > > > > I am using
>> > > > > > two platforms (Linux x86, windows x86_64) at the moment.
>> Hopefully
>> > we
>> > > > > will
>> > > > > > have other two platforms in future for M8. Well, although we are
>> > > > testing
>> > > > > > only two platforms for M7, my experience is that the status for
>> > > another
>> > > > > two
>> > > > > > platforms should be not worse or just include some extra
>> > intermittent
>> > > > > > errors
>> > > > > > which can be investigated in some later stages.
>> > > > > >
>> > > > > >  The following suites passed on Linux x86/Windows x86_64
>> platforms:
>> > > Ant
>> > > > > > Scenario (or self-hosting), Axis application, Dacapo, DRLVM
>> > > regression
>> > > > > > tests, Geronimo Unit Tests, Scimark, Tomcat scenario, VTS VM Test
>> > > > Suite.
>> > > > > >
>> > > > > >  Most of the failures are known issues (for M6). Although we can
>> > > > observe
>> > > > > > 15~20 new issues, those issues happen only in 1 platform and they
>> > > look
>> > > > > more
>> > > > > > like the intermittent/timeout issues (less reproducible) to me.
>> > > Overall
>> > > > I
>> > > > > > myself would like to think r681495 is more stable than M6.
>> > > > > >
>> > > > > >  Please add your comments and clarifications (please also see M6
>> > > > testing
>> > > > > > results http://people.apache.org/~smishura/r653525/<
>> http://people.apache.org/%7Esmishura/r653525/>
>> > <
>> > > http://people.apache.org/%7Esmishura/r653525/>
>> > > > <
>> > > > > http://people.apache.org/%7Esmishura/r653525/>
>> > > > > > ,
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805070304l38845ee0se01fb93fbfc05586@mail.gmail.com%3eand
>> > > > > > integrity testing results
>> > > > > > http://people.apache.org/~chunrong/harmony-integrity/<
>> http://people.apache.org/%7Echunrong/harmony-integrity/>
>> > <
>> > > http://people.apache.org/%7Echunrong/harmony-integrity/>
>> > > > <
>> > > > > http://people.apache.org/%7Echunrong/harmony-integrity/>as a
>> > > > comparison).
>> > > > > >
>> > > > > >  1) Classlib:
>> > > > > >    1.1) Since r644719 (which committed patch for HARMONY-5684)
>> > > > > >
>> > > > >
>> >        org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
>> > > > > >
>> > > > > >         failed in both platforms
>> > > > > >
>> > > > > >    1.2) Two failures
>> > > > > >
>> > > > > >        org.apache.harmony.luni.tests.java.net.MulticastSocketTest
>> > > > (Failed
>> > > > > > in windows_x86 running of M6)
>> > > > > >        tests.api.java.security.PermissionCollectionTest
>> > > > (Failed
>> > > > > > in M6)
>> > > > > >
>> > > > > >         are observed in Linux x86.
>> > > > > >
>> > > > > >  2) DRLVM tests:
>> > > > > >    2.1) One failure
>> > > > > >
>> > > > > >         java.lang.ClassGenericsTest.test_2
>> > > > > >
>> > > > > >         is observed in Linux x86 snapshot testing.
>> > > > > >         I can see some old discussion in the mailing list about
>> > that
>> > > > but
>> > > > > I
>> > > > > > am not sure the expected status here.
>> > > > > >         They should be intemittent errors since the integrity
>> > testing
>> > > > > just
>> > > > > > run well mostly.
>> > > > > >
>> > > > > >  3) EUTs:
>> > > > > >
>> > > > > >    3.1) Linux x86: 99.36%
>> > > > > >         A recorded JIRA for this suite is HARMONY-2914 which
>> wastes
>> > > > file
>> > > > > > handlers and makes system unstable.
>> > > > > >
>> > > > > >  4) Functional:
>> > > > > >    4.1) Old regressions on both platforms:
>> > > > > >         api.java.text.MessageFormat (HARMONY-5430)
>> > > > > >         api.java.util.jar.Manifest  (HARMONY-5473)
>> > > > > >         api.java.beans.beancontext.BeanContextTest (also in M6,
>> > > > recorded
>> > > > > > as  regression caused by changes in locale data)
>> > > > > >         api.java.beans.persistence.EncoderTest  (also in M6,
>> > recorded
>> > > > as
>> > > > > > regression in the beans module)
>> > > > > >         api.java.beans.persistence.EncoderDecoderTest (also in
>> M6,
>> > > > > > regression in the beans module)
>> > > > > >         reg.vm.btest5625 (also in M6, recorded as intermittent
>> and
>> > > not
>> > > > > > reproducible manually)
>> > > > > >
>> > > > > >    4.2) Old regressions on 1 platform
>> > > > > >         api.java.rmi.basicexception (ERROR in Linux x86,
>> > > HARMONY-5823)
>> > > > > >         api.java.rmi.basicregistry.RemoteServerTest (ERROR in
>> Linux
>> > > > x86,
>> > > > > > HARMONY-5823)
>> > > > > >         jpda.jdwp.scenario.ST07.ST07Test (ERROR in windows
>> x86_64,
>> > in
>> > > > M6
>> > > > > it
>> > > > > > is recorded as regression since M4)
>> > > > > >         java.math.F_BigIntegerMatrixMultiplyTest_01 (ERROR on
>> Linux
>> > > > x86,
>> > > > > > recorded as Timeout, not reproducible
>> > > > > >  in M6)
>> > > > > >         reg.vm.btest5717 (ERROR on Windows X86_64, recorded as
>> > > > "timeout,
>> > > > > > the test is too heavy" in M6)
>> > > > > >         jit.HLO.inline.ControlFlow.IfElse.IfElse1.IfElseTest1
>> > (FAILED
>> > > > in
>> > > > > > windows x86_64, recorded as "looks like
>> > > > > > issue in test" in M6)
>> > > > > >         jit.HLO.devirt.Runtime.RuntimeExtend1 (FAILED on windows
>> > > > x86_64,
>> > > > > in
>> > > > > > M6 it is recorded as not regression and started to fail on M5)
>> > > > > >         reg.vm.btest6353.Btest6353 (ERROR on Windows x86_64,
>> > recorded
>> > > > > also
>> > > > > > failed on M3 and might be issue with the test)
>> > > > > >
>> > > > > >    4.3) New regressions on 1 platform (looks intermittent)
>> > > > > >         reg.jit.btest8029.Btest8029 (FAILED in Linux x86)
>> > > > > >         func.reg.jit.btest5710.Btest5710 (FAILED in Linux x86)
>> > > > > >
>> api.java.security.cert.F_CertPathTest_06.F_CertPathTest_06
>> > > > (ERROR
>> > > > > > in Linux x86)
>> > > > > >
>> api.java.security.cert.F_CertPathTest_05.F_CertPathTest_05
>> > > > (ERROR
>> > > > > > in Linux x86)
>> > > > > >
>> > > > > >  5) JDKTools Tests:
>> > > > > >    Several timeouts are observed in Linux x86 snapshot running.
>> > They
>> > > > are:
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch001
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch002
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch003
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch004
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.apache.harmony.jpda.tests.jdwp.Events.CombinedEventsTest.testCombinedEvents_04
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.apache.harmony.jpda.tests.jdwp.MultiSession.AttachConnectorTest.testAttachConnector001
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.apache.harmony.jpda.tests.jdwp.MultiSession.MethodEntryExitTest.testMethodEvent001
>> > > > > >
>> > > > > >
>> > org.apache.harmony.jpda.tests.jdwp.MultiSession.ResumeTest.testResume
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadEndTest.testThreadEnd001
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadStartTest.testThreadStart001
>> > > > > >    The Linux-only timeouts are also observed in the integrity
>> > testing
>> > > > > > results.
>> > > > > >    JIRA HARMONY-5833 has been filed for one of them.
>> > > > > >
>> > > > > >  6) JettyScenario:
>> > > > > >    The Linux x86 running failed because of the unresolved
>> > > HARMONY-5219.
>> > > > > >
>> > > > > >  7) Reliability:
>> > > > > >    Several failures are observed in windows x86_64 running.
>> > > > > >    7.1) Old regressions
>> > > > > >         api.net.DatagramTest (HARMONY-5531)
>> > > > > >         api.text.DecimalFormat_Locales - (in M6 it is recorded as
>> > > also
>> > > > > > intermittent in M5)
>> > > > > >
>> > > > > >    7.2) New/intemittent regressions
>> > > > > >         api.kernel.thread.ThreadLocalTest.ThreadLocalTest
>> > > > > >         api.kernel.exec.RunExec
>> > > > > >
>> > > > > >  8) Stress
>> > > > > >    Different test cases failed on different platforms.
>> > > > > >    8.1) Timeouts on Linux x86.
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD003.ThreadTest003
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD007.ThreadTest007
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD009.ThreadTest009
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD011.ThreadTest011
>> > > > > >
>> > > > > >    8.2) Failed cases on Windows x86_64 with unknown reasons.
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> stress.org.apache.harmony.test.stress.classloader.MixThreads.TreeClasses.testTreeClasses2
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> stress.org.apache.harmony.test.stress.classloader.NotSynchThreads.TreeClasses.testTreeClasses
>> > > > > >
>> > > > > >    I have not found records for them.
>> > > > > >
>> > > > > >  9) Strut_test
>> > > > > >    Broken with same error report as M6.
>> > > > > >
>> > > > > >  10) Eclipse Hello World Application.
>> > > > > >    Although the testing framework just reports EUT-API status as
>> > > > > "PASSED".
>> > > > > > A fresh workspace running just failed in configuration stage (
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805120106o387a49f1rfb2c33d1042d2f41@mail.gmail.com%3e
>> > > > > > )
>> > > > > > since SVN commit 641928 (which committed patch for HARMONY-4569).
>> > > > > >
>> > > > > > thanks,
>> > > > > > chunrong
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>

Re: [testing] first M7 candidate (r681495) testing status

Posted by chunrong lai <ch...@gmail.com>.
Thanks.
(1) Setting for e-mail notifications are in framework.local.properties as
http://harmony.apache.org/subcomponents/buildtest/howto.html.

# Setup for mail notifications
framework.parameters.usemail=...
framework.parameters.usemail.to=alerts@harmony.apache.org
framework.parameters.usemail.from=nomail@intel.com

One may need to copy mail.jar and activation.jar from
infra/build/libs/cruisecontrol-bin-2.6.2/lib/ to $ANT_HOME/lib.

(2) After running integrity testing the direcory set in
framework.local.properties

# Directory to store archived results
framework.parameters.optional.upload.dir=*<results_dir>*

will take effect thus an archived results will be stored there. The file
name is like

[laichunrong@MRObj-lnx32-03 infra]$ ls ~/ccresult/build/to_upload/
autouploading                        linux_x86.drlvm-test.status.zip
linux_x86.jdktools-test.status.zip
linux_x86.classlib-test.status.zip   linux_x86.ehwa-api.status.zip
linux_x86.make-snapshot.status.zip
linux_x86.drlvm-reg-test.status.zip  linux_x86.hdk.status.zip
uploadresults.sh

(3)
http://svn.apache.org/repos/asf/harmony/enhanced/buildtest/trunk/tools/integrity/status-table/

include the table and perl scripts to generate the integrity testing
result page from above results.

One needs to specify the result file/web page location and can configure the
included platforms/test suites there to enable it.

So having a web page to look at every evening can be achieved by
uploading/extracting the results files and running the perl scripts
everyday. I see all required settings have been documented.
(4) We also added local contents to cc-exec-adaptor.xml in BTI for the
auto-uploading/web-page generating. It has not been committed/documented.

             update="true" />

+        <available property="autouploads"
file="${upload.dir}/autouploading"/>
+        <antcall target="-auto-upload"/>
    </target>

+    <target name="-auto-upload" if="autouploads">
+        <exec executable="${upload.dir}/uploadresults.sh"
dir="${upload.dir}">
+        </exec>
+    </target>
</project>
The scripts use scp to (auto)upload results and ssh to (auto)run above perl
scripts remotely (required keys setting).

Thanks.

On 8/13/08, Nathan Beyer <nb...@gmail.com> wrote:
>
> On Mon, Aug 11, 2008 at 10:16 PM, chunrong lai <chunronglai@gmail.com
> >wrote:
>
> >   Thanks.
> >   I think the standard scripts produce that site although I always added
> > publish/e-mail notification setup before install/run the test suites.
>
>
> Where is it at, I'm not able to find it and I would really like to recreate
> this? How do you get it published to people.apache.org? I don't want an
> email, I just want a web page to look at every evening.
>
> Can you document how you set it up? We need to have this integrity thing
> documented, so anyone can recreate it for the next milestone.
>
> -Nathan
>
>
> >
> >   It is understandable that some tests in classlib-test and drlvm-test
> get
> > failing report. We just need to understand the failures.
> >   As reported in M6,
> > org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest,
> >
> >
> org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
> > etc fail classlib-test.
> >   Also we observed that java.lang.ClassGenericsTest.test_2 sometimes fail
> > drlvm-test.
> >   Another timeout issue of drlvm-test comes from
> > thread.SmallStackThreadTest_jit, as discussed in HARMONY-4601, the test
> > case
> > may run very slow when system resource is tight and finally introduce a
> > timeout. Re-running the suite generally fixes the timeout.
> >
> >
> >
> >
> >
> > On 8/12/08, Nathan Beyer <nd...@apache.org> wrote:
> > >
> > > It's the tests that are failing - both classlib-test and drlvm-test.
> > >
> > > What's the specific integrity setup that's being used for that site?
> Does
> > > the standard out-of-the-box buildtest scripts produce that site?
> > >
> > > -Nathan
> > >
> > > On Mon, Aug 11, 2008 at 9:05 PM, chunrong lai <ch...@gmail.com>
> > > wrote:
> > >
> > > > hi, Nathan:
> > > >     Steps to run testing cycle are listed in
> > > > http://harmony.apache.org/subcomponents/buildtest/index.html. One
> > > > integrity
> > > > testing example is
> > > >
> http://harmony.apache.org/subcomponents/buildtest/howto.html#Extended.
> > > >      What is the error message you met (in what step)? What is the
> > > content
> > > > of the framework.local.properties file? I think that will help to
> > figure
> > > > out
> > > > the problem.
> > > >
> > > >
> > > > On 8/12/08, Nathan Beyer <nd...@apache.org> wrote:
> > > > >
> > > > > What's being used to produce the integrity tests? I have an x86_64
> > > Linux
> > > > > box
> > > > > that I can dedicate to testing. I've tried just using buildtest
> with
> > > > > 'classlib,drlvm,classib-test,drlvm-test', but I haven't gotten a
> > clean
> > > > pass
> > > > > yet.
> > > > >
> > > > > -Nathan
> > > > >
> > > > > On Mon, Aug 11, 2008 at 12:26 PM, chunrong lai <
> > chunronglai@gmail.com
> > > > > >wrote:
> > > > >
> > > > > > Hi, all:
> > > > > >
> > > > > >  Here is r681495 snapshot testing status:
> > > > > > http://people.apache.org/~chunrong/snapshots/r681495/index.html<
> http://people.apache.org/%7Echunrong/snapshots/r681495/index.html>
> > <
> > > http://people.apache.org/%7Echunrong/snapshots/r681495/index.html>
> > > > <
> > > > > http://people.apache.org/%7Echunrong/snapshots/r681495/index.html
> >.
> > > > > > I am using
> > > > > > two platforms (Linux x86, windows x86_64) at the moment.
> Hopefully
> > we
> > > > > will
> > > > > > have other two platforms in future for M8. Well, although we are
> > > > testing
> > > > > > only two platforms for M7, my experience is that the status for
> > > another
> > > > > two
> > > > > > platforms should be not worse or just include some extra
> > intermittent
> > > > > > errors
> > > > > > which can be investigated in some later stages.
> > > > > >
> > > > > >  The following suites passed on Linux x86/Windows x86_64
> platforms:
> > > Ant
> > > > > > Scenario (or self-hosting), Axis application, Dacapo, DRLVM
> > > regression
> > > > > > tests, Geronimo Unit Tests, Scimark, Tomcat scenario, VTS VM Test
> > > > Suite.
> > > > > >
> > > > > >  Most of the failures are known issues (for M6). Although we can
> > > > observe
> > > > > > 15~20 new issues, those issues happen only in 1 platform and they
> > > look
> > > > > more
> > > > > > like the intermittent/timeout issues (less reproducible) to me.
> > > Overall
> > > > I
> > > > > > myself would like to think r681495 is more stable than M6.
> > > > > >
> > > > > >  Please add your comments and clarifications (please also see M6
> > > > testing
> > > > > > results http://people.apache.org/~smishura/r653525/<
> http://people.apache.org/%7Esmishura/r653525/>
> > <
> > > http://people.apache.org/%7Esmishura/r653525/>
> > > > <
> > > > > http://people.apache.org/%7Esmishura/r653525/>
> > > > > > ,
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805070304l38845ee0se01fb93fbfc05586@mail.gmail.com%3eand
> > > > > > integrity testing results
> > > > > > http://people.apache.org/~chunrong/harmony-integrity/<
> http://people.apache.org/%7Echunrong/harmony-integrity/>
> > <
> > > http://people.apache.org/%7Echunrong/harmony-integrity/>
> > > > <
> > > > > http://people.apache.org/%7Echunrong/harmony-integrity/>as a
> > > > comparison).
> > > > > >
> > > > > >  1) Classlib:
> > > > > >    1.1) Since r644719 (which committed patch for HARMONY-5684)
> > > > > >
> > > > >
> >        org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
> > > > > >
> > > > > >         failed in both platforms
> > > > > >
> > > > > >    1.2) Two failures
> > > > > >
> > > > > >        org.apache.harmony.luni.tests.java.net.MulticastSocketTest
> > > > (Failed
> > > > > > in windows_x86 running of M6)
> > > > > >        tests.api.java.security.PermissionCollectionTest
> > > > (Failed
> > > > > > in M6)
> > > > > >
> > > > > >         are observed in Linux x86.
> > > > > >
> > > > > >  2) DRLVM tests:
> > > > > >    2.1) One failure
> > > > > >
> > > > > >         java.lang.ClassGenericsTest.test_2
> > > > > >
> > > > > >         is observed in Linux x86 snapshot testing.
> > > > > >         I can see some old discussion in the mailing list about
> > that
> > > > but
> > > > > I
> > > > > > am not sure the expected status here.
> > > > > >         They should be intemittent errors since the integrity
> > testing
> > > > > just
> > > > > > run well mostly.
> > > > > >
> > > > > >  3) EUTs:
> > > > > >
> > > > > >    3.1) Linux x86: 99.36%
> > > > > >         A recorded JIRA for this suite is HARMONY-2914 which
> wastes
> > > > file
> > > > > > handlers and makes system unstable.
> > > > > >
> > > > > >  4) Functional:
> > > > > >    4.1) Old regressions on both platforms:
> > > > > >         api.java.text.MessageFormat (HARMONY-5430)
> > > > > >         api.java.util.jar.Manifest  (HARMONY-5473)
> > > > > >         api.java.beans.beancontext.BeanContextTest (also in M6,
> > > > recorded
> > > > > > as  regression caused by changes in locale data)
> > > > > >         api.java.beans.persistence.EncoderTest  (also in M6,
> > recorded
> > > > as
> > > > > > regression in the beans module)
> > > > > >         api.java.beans.persistence.EncoderDecoderTest (also in
> M6,
> > > > > > regression in the beans module)
> > > > > >         reg.vm.btest5625 (also in M6, recorded as intermittent
> and
> > > not
> > > > > > reproducible manually)
> > > > > >
> > > > > >    4.2) Old regressions on 1 platform
> > > > > >         api.java.rmi.basicexception (ERROR in Linux x86,
> > > HARMONY-5823)
> > > > > >         api.java.rmi.basicregistry.RemoteServerTest (ERROR in
> Linux
> > > > x86,
> > > > > > HARMONY-5823)
> > > > > >         jpda.jdwp.scenario.ST07.ST07Test (ERROR in windows
> x86_64,
> > in
> > > > M6
> > > > > it
> > > > > > is recorded as regression since M4)
> > > > > >         java.math.F_BigIntegerMatrixMultiplyTest_01 (ERROR on
> Linux
> > > > x86,
> > > > > > recorded as Timeout, not reproducible
> > > > > >  in M6)
> > > > > >         reg.vm.btest5717 (ERROR on Windows X86_64, recorded as
> > > > "timeout,
> > > > > > the test is too heavy" in M6)
> > > > > >         jit.HLO.inline.ControlFlow.IfElse.IfElse1.IfElseTest1
> > (FAILED
> > > > in
> > > > > > windows x86_64, recorded as "looks like
> > > > > > issue in test" in M6)
> > > > > >         jit.HLO.devirt.Runtime.RuntimeExtend1 (FAILED on windows
> > > > x86_64,
> > > > > in
> > > > > > M6 it is recorded as not regression and started to fail on M5)
> > > > > >         reg.vm.btest6353.Btest6353 (ERROR on Windows x86_64,
> > recorded
> > > > > also
> > > > > > failed on M3 and might be issue with the test)
> > > > > >
> > > > > >    4.3) New regressions on 1 platform (looks intermittent)
> > > > > >         reg.jit.btest8029.Btest8029 (FAILED in Linux x86)
> > > > > >         func.reg.jit.btest5710.Btest5710 (FAILED in Linux x86)
> > > > > >
> api.java.security.cert.F_CertPathTest_06.F_CertPathTest_06
> > > > (ERROR
> > > > > > in Linux x86)
> > > > > >
> api.java.security.cert.F_CertPathTest_05.F_CertPathTest_05
> > > > (ERROR
> > > > > > in Linux x86)
> > > > > >
> > > > > >  5) JDKTools Tests:
> > > > > >    Several timeouts are observed in Linux x86 snapshot running.
> > They
> > > > are:
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch001
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch002
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch003
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch004
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.harmony.jpda.tests.jdwp.Events.CombinedEventsTest.testCombinedEvents_04
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.AttachConnectorTest.testAttachConnector001
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.MethodEntryExitTest.testMethodEvent001
> > > > > >
> > > > > >
> > org.apache.harmony.jpda.tests.jdwp.MultiSession.ResumeTest.testResume
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadEndTest.testThreadEnd001
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadStartTest.testThreadStart001
> > > > > >    The Linux-only timeouts are also observed in the integrity
> > testing
> > > > > > results.
> > > > > >    JIRA HARMONY-5833 has been filed for one of them.
> > > > > >
> > > > > >  6) JettyScenario:
> > > > > >    The Linux x86 running failed because of the unresolved
> > > HARMONY-5219.
> > > > > >
> > > > > >  7) Reliability:
> > > > > >    Several failures are observed in windows x86_64 running.
> > > > > >    7.1) Old regressions
> > > > > >         api.net.DatagramTest (HARMONY-5531)
> > > > > >         api.text.DecimalFormat_Locales - (in M6 it is recorded as
> > > also
> > > > > > intermittent in M5)
> > > > > >
> > > > > >    7.2) New/intemittent regressions
> > > > > >         api.kernel.thread.ThreadLocalTest.ThreadLocalTest
> > > > > >         api.kernel.exec.RunExec
> > > > > >
> > > > > >  8) Stress
> > > > > >    Different test cases failed on different platforms.
> > > > > >    8.1) Timeouts on Linux x86.
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD003.ThreadTest003
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD007.ThreadTest007
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD009.ThreadTest009
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD011.ThreadTest011
> > > > > >
> > > > > >    8.2) Failed cases on Windows x86_64 with unknown reasons.
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> stress.org.apache.harmony.test.stress.classloader.MixThreads.TreeClasses.testTreeClasses2
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> stress.org.apache.harmony.test.stress.classloader.NotSynchThreads.TreeClasses.testTreeClasses
> > > > > >
> > > > > >    I have not found records for them.
> > > > > >
> > > > > >  9) Strut_test
> > > > > >    Broken with same error report as M6.
> > > > > >
> > > > > >  10) Eclipse Hello World Application.
> > > > > >    Although the testing framework just reports EUT-API status as
> > > > > "PASSED".
> > > > > > A fresh workspace running just failed in configuration stage (
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805120106o387a49f1rfb2c33d1042d2f41@mail.gmail.com%3e
> > > > > > )
> > > > > > since SVN commit 641928 (which committed patch for HARMONY-4569).
> > > > > >
> > > > > > thanks,
> > > > > > chunrong
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [testing] first M7 candidate (r681495) testing status

Posted by Nathan Beyer <nb...@gmail.com>.
On Mon, Aug 11, 2008 at 10:16 PM, chunrong lai <ch...@gmail.com>wrote:

>   Thanks.
>   I think the standard scripts produce that site although I always added
> publish/e-mail notification setup before install/run the test suites.


Where is it at, I'm not able to find it and I would really like to recreate
this? How do you get it published to people.apache.org? I don't want an
email, I just want a web page to look at every evening.

Can you document how you set it up? We need to have this integrity thing
documented, so anyone can recreate it for the next milestone.

-Nathan


>
>   It is understandable that some tests in classlib-test and drlvm-test get
> failing report. We just need to understand the failures.
>   As reported in M6,
> org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest,
>
> org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
> etc fail classlib-test.
>   Also we observed that java.lang.ClassGenericsTest.test_2 sometimes fail
> drlvm-test.
>   Another timeout issue of drlvm-test comes from
> thread.SmallStackThreadTest_jit, as discussed in HARMONY-4601, the test
> case
> may run very slow when system resource is tight and finally introduce a
> timeout. Re-running the suite generally fixes the timeout.
>
>
>
>
>
> On 8/12/08, Nathan Beyer <nd...@apache.org> wrote:
> >
> > It's the tests that are failing - both classlib-test and drlvm-test.
> >
> > What's the specific integrity setup that's being used for that site? Does
> > the standard out-of-the-box buildtest scripts produce that site?
> >
> > -Nathan
> >
> > On Mon, Aug 11, 2008 at 9:05 PM, chunrong lai <ch...@gmail.com>
> > wrote:
> >
> > > hi, Nathan:
> > >     Steps to run testing cycle are listed in
> > > http://harmony.apache.org/subcomponents/buildtest/index.html. One
> > > integrity
> > > testing example is
> > > http://harmony.apache.org/subcomponents/buildtest/howto.html#Extended.
> > >      What is the error message you met (in what step)? What is the
> > content
> > > of the framework.local.properties file? I think that will help to
> figure
> > > out
> > > the problem.
> > >
> > >
> > > On 8/12/08, Nathan Beyer <nd...@apache.org> wrote:
> > > >
> > > > What's being used to produce the integrity tests? I have an x86_64
> > Linux
> > > > box
> > > > that I can dedicate to testing. I've tried just using buildtest with
> > > > 'classlib,drlvm,classib-test,drlvm-test', but I haven't gotten a
> clean
> > > pass
> > > > yet.
> > > >
> > > > -Nathan
> > > >
> > > > On Mon, Aug 11, 2008 at 12:26 PM, chunrong lai <
> chunronglai@gmail.com
> > > > >wrote:
> > > >
> > > > > Hi, all:
> > > > >
> > > > >  Here is r681495 snapshot testing status:
> > > > > http://people.apache.org/~chunrong/snapshots/r681495/index.html<http://people.apache.org/%7Echunrong/snapshots/r681495/index.html>
> <
> > http://people.apache.org/%7Echunrong/snapshots/r681495/index.html>
> > > <
> > > > http://people.apache.org/%7Echunrong/snapshots/r681495/index.html>.
> > > > > I am using
> > > > > two platforms (Linux x86, windows x86_64) at the moment. Hopefully
> we
> > > > will
> > > > > have other two platforms in future for M8. Well, although we are
> > > testing
> > > > > only two platforms for M7, my experience is that the status for
> > another
> > > > two
> > > > > platforms should be not worse or just include some extra
> intermittent
> > > > > errors
> > > > > which can be investigated in some later stages.
> > > > >
> > > > >  The following suites passed on Linux x86/Windows x86_64 platforms:
> > Ant
> > > > > Scenario (or self-hosting), Axis application, Dacapo, DRLVM
> > regression
> > > > > tests, Geronimo Unit Tests, Scimark, Tomcat scenario, VTS VM Test
> > > Suite.
> > > > >
> > > > >  Most of the failures are known issues (for M6). Although we can
> > > observe
> > > > > 15~20 new issues, those issues happen only in 1 platform and they
> > look
> > > > more
> > > > > like the intermittent/timeout issues (less reproducible) to me.
> > Overall
> > > I
> > > > > myself would like to think r681495 is more stable than M6.
> > > > >
> > > > >  Please add your comments and clarifications (please also see M6
> > > testing
> > > > > results http://people.apache.org/~smishura/r653525/<http://people.apache.org/%7Esmishura/r653525/>
> <
> > http://people.apache.org/%7Esmishura/r653525/>
> > > <
> > > > http://people.apache.org/%7Esmishura/r653525/>
> > > > > ,
> > > > >
> > > > >
> > > >
> > >
> >
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805070304l38845ee0se01fb93fbfc05586@mail.gmail.com%3eand
> > > > > integrity testing results
> > > > > http://people.apache.org/~chunrong/harmony-integrity/<http://people.apache.org/%7Echunrong/harmony-integrity/>
> <
> > http://people.apache.org/%7Echunrong/harmony-integrity/>
> > > <
> > > > http://people.apache.org/%7Echunrong/harmony-integrity/>as a
> > > comparison).
> > > > >
> > > > >  1) Classlib:
> > > > >    1.1) Since r644719 (which committed patch for HARMONY-5684)
> > > > >
> > > > >        org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
> > > > >
> > > > >         failed in both platforms
> > > > >
> > > > >    1.2) Two failures
> > > > >
> > > > >        org.apache.harmony.luni.tests.java.net.MulticastSocketTest
> > > (Failed
> > > > > in windows_x86 running of M6)
> > > > >        tests.api.java.security.PermissionCollectionTest
> > > (Failed
> > > > > in M6)
> > > > >
> > > > >         are observed in Linux x86.
> > > > >
> > > > >  2) DRLVM tests:
> > > > >    2.1) One failure
> > > > >
> > > > >         java.lang.ClassGenericsTest.test_2
> > > > >
> > > > >         is observed in Linux x86 snapshot testing.
> > > > >         I can see some old discussion in the mailing list about
> that
> > > but
> > > > I
> > > > > am not sure the expected status here.
> > > > >         They should be intemittent errors since the integrity
> testing
> > > > just
> > > > > run well mostly.
> > > > >
> > > > >  3) EUTs:
> > > > >
> > > > >    3.1) Linux x86: 99.36%
> > > > >         A recorded JIRA for this suite is HARMONY-2914 which wastes
> > > file
> > > > > handlers and makes system unstable.
> > > > >
> > > > >  4) Functional:
> > > > >    4.1) Old regressions on both platforms:
> > > > >         api.java.text.MessageFormat (HARMONY-5430)
> > > > >         api.java.util.jar.Manifest  (HARMONY-5473)
> > > > >         api.java.beans.beancontext.BeanContextTest (also in M6,
> > > recorded
> > > > > as  regression caused by changes in locale data)
> > > > >         api.java.beans.persistence.EncoderTest  (also in M6,
> recorded
> > > as
> > > > > regression in the beans module)
> > > > >         api.java.beans.persistence.EncoderDecoderTest (also in M6,
> > > > > regression in the beans module)
> > > > >         reg.vm.btest5625 (also in M6, recorded as intermittent and
> > not
> > > > > reproducible manually)
> > > > >
> > > > >    4.2) Old regressions on 1 platform
> > > > >         api.java.rmi.basicexception (ERROR in Linux x86,
> > HARMONY-5823)
> > > > >         api.java.rmi.basicregistry.RemoteServerTest (ERROR in Linux
> > > x86,
> > > > > HARMONY-5823)
> > > > >         jpda.jdwp.scenario.ST07.ST07Test (ERROR in windows x86_64,
> in
> > > M6
> > > > it
> > > > > is recorded as regression since M4)
> > > > >         java.math.F_BigIntegerMatrixMultiplyTest_01 (ERROR on Linux
> > > x86,
> > > > > recorded as Timeout, not reproducible
> > > > >  in M6)
> > > > >         reg.vm.btest5717 (ERROR on Windows X86_64, recorded as
> > > "timeout,
> > > > > the test is too heavy" in M6)
> > > > >         jit.HLO.inline.ControlFlow.IfElse.IfElse1.IfElseTest1
> (FAILED
> > > in
> > > > > windows x86_64, recorded as "looks like
> > > > > issue in test" in M6)
> > > > >         jit.HLO.devirt.Runtime.RuntimeExtend1 (FAILED on windows
> > > x86_64,
> > > > in
> > > > > M6 it is recorded as not regression and started to fail on M5)
> > > > >         reg.vm.btest6353.Btest6353 (ERROR on Windows x86_64,
> recorded
> > > > also
> > > > > failed on M3 and might be issue with the test)
> > > > >
> > > > >    4.3) New regressions on 1 platform (looks intermittent)
> > > > >         reg.jit.btest8029.Btest8029 (FAILED in Linux x86)
> > > > >         func.reg.jit.btest5710.Btest5710 (FAILED in Linux x86)
> > > > >         api.java.security.cert.F_CertPathTest_06.F_CertPathTest_06
> > > (ERROR
> > > > > in Linux x86)
> > > > >         api.java.security.cert.F_CertPathTest_05.F_CertPathTest_05
> > > (ERROR
> > > > > in Linux x86)
> > > > >
> > > > >  5) JDKTools Tests:
> > > > >    Several timeouts are observed in Linux x86 snapshot running.
> They
> > > are:
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch001
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch002
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch003
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch004
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> org.apache.harmony.jpda.tests.jdwp.Events.CombinedEventsTest.testCombinedEvents_04
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.AttachConnectorTest.testAttachConnector001
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.MethodEntryExitTest.testMethodEvent001
> > > > >
> > > > >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ResumeTest.testResume
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadEndTest.testThreadEnd001
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadStartTest.testThreadStart001
> > > > >    The Linux-only timeouts are also observed in the integrity
> testing
> > > > > results.
> > > > >    JIRA HARMONY-5833 has been filed for one of them.
> > > > >
> > > > >  6) JettyScenario:
> > > > >    The Linux x86 running failed because of the unresolved
> > HARMONY-5219.
> > > > >
> > > > >  7) Reliability:
> > > > >    Several failures are observed in windows x86_64 running.
> > > > >    7.1) Old regressions
> > > > >         api.net.DatagramTest (HARMONY-5531)
> > > > >         api.text.DecimalFormat_Locales - (in M6 it is recorded as
> > also
> > > > > intermittent in M5)
> > > > >
> > > > >    7.2) New/intemittent regressions
> > > > >         api.kernel.thread.ThreadLocalTest.ThreadLocalTest
> > > > >         api.kernel.exec.RunExec
> > > > >
> > > > >  8) Stress
> > > > >    Different test cases failed on different platforms.
> > > > >    8.1) Timeouts on Linux x86.
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD003.ThreadTest003
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD007.ThreadTest007
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD009.ThreadTest009
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD011.ThreadTest011
> > > > >
> > > > >    8.2) Failed cases on Windows x86_64 with unknown reasons.
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> stress.org.apache.harmony.test.stress.classloader.MixThreads.TreeClasses.testTreeClasses2
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> stress.org.apache.harmony.test.stress.classloader.NotSynchThreads.TreeClasses.testTreeClasses
> > > > >
> > > > >    I have not found records for them.
> > > > >
> > > > >  9) Strut_test
> > > > >    Broken with same error report as M6.
> > > > >
> > > > >  10) Eclipse Hello World Application.
> > > > >    Although the testing framework just reports EUT-API status as
> > > > "PASSED".
> > > > > A fresh workspace running just failed in configuration stage (
> > > > >
> > > > >
> > > >
> > >
> >
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805120106o387a49f1rfb2c33d1042d2f41@mail.gmail.com%3e
> > > > > )
> > > > > since SVN commit 641928 (which committed patch for HARMONY-4569).
> > > > >
> > > > > thanks,
> > > > > chunrong
> > > > >
> > > >
> > >
> >
>

Re: [testing] first M7 candidate (r681495) testing status

Posted by chunrong lai <ch...@gmail.com>.
 Just a note that I did not observe the failures with commands below:

C:\harmony.latest\trunk\working_classlib>ant
-Dtest.jre.home=c:\harmony.latest\t
runk\working_vm\build\windows_x86_msvc_debug\deploy\jdk\jre\
-Dtest.case=org.apa
che.harmony.rmi.server.ServerCloneExceptionTest test

C:\harmony.latest\trunk\working_classlib>ant
-Dtest.jre.home=c:\harmony.latest\t
runk\working_vm\build\windows_x86_msvc_debug\deploy\jdk\jre\
-Dtest.case=org.apa
che.harmony.luni.tests.java.net.URLConnectionTest test

On 8/12/08, Sian January <si...@googlemail.com> wrote:
>
> I am not seeing the management failures on my local (Windows) machine,
> but I'm seeing an rmi failure:
>
>
> org.apache.harmony.rmi.server.ServerCloneExceptionTest.test_Constructor_String
> Cause already initialized
> java.lang.IllegalStateException: Cause already initialized
> at java.lang.Throwable.initCause(Throwable.java:293)
> at
> org.apache.harmony.rmi.server.ServerCloneExceptionTest.test_Constructor_String(ServerCloneExceptionTest.java:70)
> at java.lang.reflect.AccessibleObject.invokeV(AccessibleObject.java:25)
>
> and an intermittent failure in java.net:
>
>
> org.apache.harmony.luni.tests.java.net.URLConnectionTest.test_getInputStream
>
> Error receiving content coded data:
> junit.framework.AssertionFailedError: Error receiving content coded data:
> at tests.support.Support_HttpTests.runTests(Support_HttpTests.java:15)
> at
> org.apache.harmony.luni.tests.java.net.URLConnectionTest.test_getInputStream(URLConnectionTest.java:632)
> at java.lang.reflect.AccessibleObject.invokeV(AccessibleObject.java:25)
>
> Is anyone else seeing these?
>
> Sian
>
> On 12/08/2008, chunrong lai <ch...@gmail.com> wrote:
> >   Thanks.
> >   I think the standard scripts produce that site although I always added
> > publish/e-mail notification setup before install/run the test suites.
> >   It is understandable that some tests in classlib-test and drlvm-test
> get
> > failing report. We just need to understand the failures.
> >   As reported in M6,
> > org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest,
> >
> org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
> > etc fail classlib-test.
> >   Also we observed that java.lang.ClassGenericsTest.test_2 sometimes fail
> > drlvm-test.
> >   Another timeout issue of drlvm-test comes from
> > thread.SmallStackThreadTest_jit, as discussed in HARMONY-4601, the test
> case
> > may run very slow when system resource is tight and finally introduce a
> > timeout. Re-running the suite generally fixes the timeout.
> >
> >
> >
> >
> >
> > On 8/12/08, Nathan Beyer <nd...@apache.org> wrote:
> > >
> > > It's the tests that are failing - both classlib-test and drlvm-test.
> > >
> > > What's the specific integrity setup that's being used for that site?
> Does
> > > the standard out-of-the-box buildtest scripts produce that site?
> > >
> > > -Nathan
> > >
> > > On Mon, Aug 11, 2008 at 9:05 PM, chunrong lai <ch...@gmail.com>
> > > wrote:
> > >
> > > > hi, Nathan:
> > > >     Steps to run testing cycle are listed in
> > > > http://harmony.apache.org/subcomponents/buildtest/index.html. One
> > > > integrity
> > > > testing example is
> > > >
> http://harmony.apache.org/subcomponents/buildtest/howto.html#Extended.
> > > >      What is the error message you met (in what step)? What is the
> > > content
> > > > of the framework.local.properties file? I think that will help to
> figure
> > > > out
> > > > the problem.
> > > >
> > > >
> > > > On 8/12/08, Nathan Beyer <nd...@apache.org> wrote:
> > > > >
> > > > > What's being used to produce the integrity tests? I have an x86_64
> > > Linux
> > > > > box
> > > > > that I can dedicate to testing. I've tried just using buildtest
> with
> > > > > 'classlib,drlvm,classib-test,drlvm-test', but I haven't gotten a
> clean
> > > > pass
> > > > > yet.
> > > > >
> > > > > -Nathan
> > > > >
> > > > > On Mon, Aug 11, 2008 at 12:26 PM, chunrong lai <
> chunronglai@gmail.com
> > > > > >wrote:
> > > > >
> > > > > > Hi, all:
> > > > > >
> > > > > >  Here is r681495 snapshot testing status:
> > > > > > http://people.apache.org/~chunrong/snapshots/r681495/index.html<
> > > http://people.apache.org/%7Echunrong/snapshots/r681495/index.html>
> > > > <
> > > > > http://people.apache.org/%7Echunrong/snapshots/r681495/index.html
> >.
> > > > > > I am using
> > > > > > two platforms (Linux x86, windows x86_64) at the moment.
> Hopefully we
> > > > > will
> > > > > > have other two platforms in future for M8. Well, although we are
> > > > testing
> > > > > > only two platforms for M7, my experience is that the status for
> > > another
> > > > > two
> > > > > > platforms should be not worse or just include some extra
> intermittent
> > > > > > errors
> > > > > > which can be investigated in some later stages.
> > > > > >
> > > > > >  The following suites passed on Linux x86/Windows x86_64
> platforms:
> > > Ant
> > > > > > Scenario (or self-hosting), Axis application, Dacapo, DRLVM
> > > regression
> > > > > > tests, Geronimo Unit Tests, Scimark, Tomcat scenario, VTS VM Test
> > > > Suite.
> > > > > >
> > > > > >  Most of the failures are known issues (for M6). Although we can
> > > > observe
> > > > > > 15~20 new issues, those issues happen only in 1 platform and they
> > > look
> > > > > more
> > > > > > like the intermittent/timeout issues (less reproducible) to me.
> > > Overall
> > > > I
> > > > > > myself would like to think r681495 is more stable than M6.
> > > > > >
> > > > > >  Please add your comments and clarifications (please also see M6
> > > > testing
> > > > > > results http://people.apache.org/~smishura/r653525/<
> > > http://people.apache.org/%7Esmishura/r653525/>
> > > > <
> > > > > http://people.apache.org/%7Esmishura/r653525/>
> > > > > > ,
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805070304l38845ee0se01fb93fbfc05586@mail.gmail.com%3eand
> > > > > > integrity testing results
> > > > > > http://people.apache.org/~chunrong/harmony-integrity/<
> > > http://people.apache.org/%7Echunrong/harmony-integrity/>
> > > > <
> > > > > http://people.apache.org/%7Echunrong/harmony-integrity/>as a
> > > > comparison).
> > > > > >
> > > > > >  1) Classlib:
> > > > > >    1.1) Since r644719 (which committed patch for HARMONY-5684)
> > > > > >
> > > > >
> >        org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
> > > > > >
> > > > > >         failed in both platforms
> > > > > >
> > > > > >    1.2) Two failures
> > > > > >
> > > > > >        org.apache.harmony.luni.tests.java.net.MulticastSocketTest
> > > > (Failed
> > > > > > in windows_x86 running of M6)
> > > > > >        tests.api.java.security.PermissionCollectionTest
> > > > (Failed
> > > > > > in M6)
> > > > > >
> > > > > >         are observed in Linux x86.
> > > > > >
> > > > > >  2) DRLVM tests:
> > > > > >    2.1) One failure
> > > > > >
> > > > > >         java.lang.ClassGenericsTest.test_2
> > > > > >
> > > > > >         is observed in Linux x86 snapshot testing.
> > > > > >         I can see some old discussion in the mailing list about
> that
> > > > but
> > > > > I
> > > > > > am not sure the expected status here.
> > > > > >         They should be intemittent errors since the integrity
> testing
> > > > > just
> > > > > > run well mostly.
> > > > > >
> > > > > >  3) EUTs:
> > > > > >
> > > > > >    3.1) Linux x86: 99.36%
> > > > > >         A recorded JIRA for this suite is HARMONY-2914 which
> wastes
> > > > file
> > > > > > handlers and makes system unstable.
> > > > > >
> > > > > >  4) Functional:
> > > > > >    4.1) Old regressions on both platforms:
> > > > > >         api.java.text.MessageFormat (HARMONY-5430)
> > > > > >         api.java.util.jar.Manifest  (HARMONY-5473)
> > > > > >         api.java.beans.beancontext.BeanContextTest (also in M6,
> > > > recorded
> > > > > > as  regression caused by changes in locale data)
> > > > > >         api.java.beans.persistence.EncoderTest  (also in M6,
> recorded
> > > > as
> > > > > > regression in the beans module)
> > > > > >         api.java.beans.persistence.EncoderDecoderTest (also in
> M6,
> > > > > > regression in the beans module)
> > > > > >         reg.vm.btest5625 (also in M6, recorded as intermittent
> and
> > > not
> > > > > > reproducible manually)
> > > > > >
> > > > > >    4.2) Old regressions on 1 platform
> > > > > >         api.java.rmi.basicexception (ERROR in Linux x86,
> > > HARMONY-5823)
> > > > > >         api.java.rmi.basicregistry.RemoteServerTest (ERROR in
> Linux
> > > > x86,
> > > > > > HARMONY-5823)
> > > > > >         jpda.jdwp.scenario.ST07.ST07Test (ERROR in windows
> x86_64, in
> > > > M6
> > > > > it
> > > > > > is recorded as regression since M4)
> > > > > >         java.math.F_BigIntegerMatrixMultiplyTest_01 (ERROR on
> Linux
> > > > x86,
> > > > > > recorded as Timeout, not reproducible
> > > > > >  in M6)
> > > > > >         reg.vm.btest5717 (ERROR on Windows X86_64, recorded as
> > > > "timeout,
> > > > > > the test is too heavy" in M6)
> > > > > >         jit.HLO.inline.ControlFlow.IfElse.IfElse1.IfElseTest1
> (FAILED
> > > > in
> > > > > > windows x86_64, recorded as "looks like
> > > > > > issue in test" in M6)
> > > > > >         jit.HLO.devirt.Runtime.RuntimeExtend1 (FAILED on windows
> > > > x86_64,
> > > > > in
> > > > > > M6 it is recorded as not regression and started to fail on M5)
> > > > > >         reg.vm.btest6353.Btest6353 (ERROR on Windows x86_64,
> recorded
> > > > > also
> > > > > > failed on M3 and might be issue with the test)
> > > > > >
> > > > > >    4.3) New regressions on 1 platform (looks intermittent)
> > > > > >         reg.jit.btest8029.Btest8029 (FAILED in Linux x86)
> > > > > >         func.reg.jit.btest5710.Btest5710 (FAILED in Linux x86)
> > > > > >
> api.java.security.cert.F_CertPathTest_06.F_CertPathTest_06
> > > > (ERROR
> > > > > > in Linux x86)
> > > > > >
> api.java.security.cert.F_CertPathTest_05.F_CertPathTest_05
> > > > (ERROR
> > > > > > in Linux x86)
> > > > > >
> > > > > >  5) JDKTools Tests:
> > > > > >    Several timeouts are observed in Linux x86 snapshot running.
> They
> > > > are:
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch001
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch002
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch003
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch004
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> org.apache.harmony.jpda.tests.jdwp.Events.CombinedEventsTest.testCombinedEvents_04
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.AttachConnectorTest.testAttachConnector001
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.MethodEntryExitTest.testMethodEvent001
> > > > > >
> > > > > >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ResumeTest.testResume
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadEndTest.testThreadEnd001
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadStartTest.testThreadStart001
> > > > > >    The Linux-only timeouts are also observed in the integrity
> testing
> > > > > > results.
> > > > > >    JIRA HARMONY-5833 has been filed for one of them.
> > > > > >
> > > > > >  6) JettyScenario:
> > > > > >    The Linux x86 running failed because of the unresolved
> > > HARMONY-5219.
> > > > > >
> > > > > >  7) Reliability:
> > > > > >    Several failures are observed in windows x86_64 running.
> > > > > >    7.1) Old regressions
> > > > > >         api.net.DatagramTest (HARMONY-5531)
> > > > > >         api.text.DecimalFormat_Locales - (in M6 it is recorded as
> > > also
> > > > > > intermittent in M5)
> > > > > >
> > > > > >    7.2) New/intemittent regressions
> > > > > >         api.kernel.thread.ThreadLocalTest.ThreadLocalTest
> > > > > >         api.kernel.exec.RunExec
> > > > > >
> > > > > >  8) Stress
> > > > > >    Different test cases failed on different platforms.
> > > > > >    8.1) Timeouts on Linux x86.
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD003.ThreadTest003
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD007.ThreadTest007
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD009.ThreadTest009
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD011.ThreadTest011
> > > > > >
> > > > > >    8.2) Failed cases on Windows x86_64 with unknown reasons.
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> stress.org.apache.harmony.test.stress.classloader.MixThreads.TreeClasses.testTreeClasses2
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> stress.org.apache.harmony.test.stress.classloader.NotSynchThreads.TreeClasses.testTreeClasses
> > > > > >
> > > > > >    I have not found records for them.
> > > > > >
> > > > > >  9) Strut_test
> > > > > >    Broken with same error report as M6.
> > > > > >
> > > > > >  10) Eclipse Hello World Application.
> > > > > >    Although the testing framework just reports EUT-API status as
> > > > > "PASSED".
> > > > > > A fresh workspace running just failed in configuration stage (
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805120106o387a49f1rfb2c33d1042d2f41@mail.gmail.com%3e
> > > > > > )
> > > > > > since SVN commit 641928 (which committed patch for HARMONY-4569).
> > > > > >
> > > > > > thanks,
> > > > > > chunrong
> > > > > >
> > > > >
> > > >
> > >
> >
>
>
>
> --
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>

Re: [testing] first M7 candidate (r681495) testing status

Posted by Oliver Deakin <ol...@googlemail.com>.
I agree - this can probably be left until after the M7 code freeze.
As Sian says, the DRLVM Throwable implementation of initCause() can 
probably just be taken from the luni-kernel version in classlib, as this 
appears to match the spec.

Regards,
Oliver

Sian January wrote:
> Oliver has helped me track this down this morning, and it's occuring
> because there are differences between the versions of
> Throwable.initCause(..) in luni and in DRLVM.  According to the spec,
> I think the test should be changed to check for IllegalStateException
> instead of IllegalArgumentException and then I think the version of
> Throwable in DRLVM should be changed to behave the same as the one in
> luni.  However I don't think this is a big deal for M7 because I don't
> think it's very serious and it's not a regression since M6.  What does
> anyone else think?
>
> Sian
>
> On 13/08/2008, Sian January <si...@googlemail.com> wrote:
>   
>> I think the code is right and the test is wrong here, because the
>> Javadoc for ServerCloneException says that "Invoking the method
>> Throwable.initCause(Throwable) on an instance of ServerCloneException
>> always throws IllegalStateException."
>>
>> What is really confusing me is how this can be passing for anyone
>> else... The exclude list hasn't changed recently and I don't see how
>> it could be VM dependent either.
>>
>>
>> On 13/08/2008, Nathan Beyer <nd...@apache.org> wrote:
>>     
>>> On Tue, Aug 12, 2008 at 10:18 AM, Sian January
>>> <si...@googlemail.com>wrote:
>>>
>>>       
>>>> I am not seeing the management failures on my local (Windows) machine,
>>>> but I'm seeing an rmi failure:
>>>>
>>>>
>>>> org.apache.harmony.rmi.server.ServerCloneExceptionTest.test_Constructor_String
>>>> Cause already initialized
>>>> java.lang.IllegalStateException: Cause already initialized
>>>> at java.lang.Throwable.initCause(Throwable.java:293)
>>>> at
>>>> org.apache.harmony.rmi.server.ServerCloneExceptionTest.test_Constructor_String(ServerCloneExceptionTest.java:70)
>>>> at java.lang.reflect.AccessibleObject.invokeV(AccessibleObject.java:25)
>>>>         
>>> This test [1] is creating an exception and then passing itself into
>>> 'initCause', which should throw IllegalArgumentException. Looking at the
>>> code though, it does look like there's a bug, as 'null' is being passed to
>>> the super() constructor, which will mess up Throwable, since it initializes
>>> 'cause' to 'this', so Throwable will think the cause is already set.
>>>
>>> Did this test recently get removed from an exclusion list?
>>>
>>>
>>> [1]
>>> http://svn.apache.org/viewvc/harmony/enhanced/classlib/trunk/modules/rmi/src/test/api/java/org/apache/harmony/rmi/server/ServerCloneExceptionTest.java?view=annotate
>>>
>>>       
>>>> and an intermittent failure in java.net:
>>>>
>>>>
>>>> org.apache.harmony.luni.tests.java.net.URLConnectionTest.test_getInputStream
>>>>
>>>> Error receiving content coded data:
>>>> junit.framework.AssertionFailedError: Error receiving content coded data:
>>>> at tests.support.Support_HttpTests.runTests(Support_HttpTests.java:15)
>>>> at
>>>> org.apache.harmony.luni.tests.java.net.URLConnectionTest.test_getInputStream(URLConnectionTest.java:632)
>>>> at java.lang.reflect.AccessibleObject.invokeV(AccessibleObject.java:25)
>>>>
>>>> Is anyone else seeing these?
>>>>
>>>> Sian
>>>>
>>>> On 12/08/2008, chunrong lai <ch...@gmail.com> wrote:
>>>>         
>>>>>   Thanks.
>>>>>   I think the standard scripts produce that site although I always added
>>>>> publish/e-mail notification setup before install/run the test suites.
>>>>>   It is understandable that some tests in classlib-test and drlvm-test
>>>>>           
>>>> get
>>>>         
>>>>> failing report. We just need to understand the failures.
>>>>>   As reported in M6,
>>>>> org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest,
>>>>>
>>>>>           
>>>> org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
>>>>         
>>>>> etc fail classlib-test.
>>>>>   Also we observed that java.lang.ClassGenericsTest.test_2 sometimes fail
>>>>> drlvm-test.
>>>>>   Another timeout issue of drlvm-test comes from
>>>>> thread.SmallStackThreadTest_jit, as discussed in HARMONY-4601, the test
>>>>>           
>>>> case
>>>>         
>>>>> may run very slow when system resource is tight and finally introduce a
>>>>> timeout. Re-running the suite generally fixes the timeout.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On 8/12/08, Nathan Beyer <nd...@apache.org> wrote:
>>>>>           
>>>>>> It's the tests that are failing - both classlib-test and drlvm-test.
>>>>>>
>>>>>> What's the specific integrity setup that's being used for that site?
>>>>>>             
>>>> Does
>>>>         
>>>>>> the standard out-of-the-box buildtest scripts produce that site?
>>>>>>
>>>>>> -Nathan
>>>>>>
>>>>>> On Mon, Aug 11, 2008 at 9:05 PM, chunrong lai <ch...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>             
>>>>>>> hi, Nathan:
>>>>>>>     Steps to run testing cycle are listed in
>>>>>>> http://harmony.apache.org/subcomponents/buildtest/index.html. One
>>>>>>> integrity
>>>>>>> testing example is
>>>>>>>
>>>>>>>               
>>>> http://harmony.apache.org/subcomponents/buildtest/howto.html#Extended.
>>>>         
>>>>>>>      What is the error message you met (in what step)? What is the
>>>>>>>               
>>>>>> content
>>>>>>             
>>>>>>> of the framework.local.properties file? I think that will help to
>>>>>>>               
>>>> figure
>>>>         
>>>>>>> out
>>>>>>> the problem.
>>>>>>>
>>>>>>>
>>>>>>> On 8/12/08, Nathan Beyer <nd...@apache.org> wrote:
>>>>>>>               
>>>>>>>> What's being used to produce the integrity tests? I have an x86_64
>>>>>>>>                 
>>>>>> Linux
>>>>>>             
>>>>>>>> box
>>>>>>>> that I can dedicate to testing. I've tried just using buildtest
>>>>>>>>                 
>>>> with
>>>>         
>>>>>>>> 'classlib,drlvm,classib-test,drlvm-test', but I haven't gotten a
>>>>>>>>                 
>>>> clean
>>>>         
>>>>>>> pass
>>>>>>>               
>>>>>>>> yet.
>>>>>>>>
>>>>>>>> -Nathan
>>>>>>>>
>>>>>>>> On Mon, Aug 11, 2008 at 12:26 PM, chunrong lai <
>>>>>>>>                 
>>>> chunronglai@gmail.com
>>>>         
>>>>>>>>> wrote:
>>>>>>>>>                   
>>>>>>>>> Hi, all:
>>>>>>>>>
>>>>>>>>>  Here is r681495 snapshot testing status:
>>>>>>>>> http://people.apache.org/~chunrong/snapshots/r681495/index.html<http://people.apache.org/%7Echunrong/snapshots/r681495/index.html>
>>>>>>>>>                   
>>>> <
>>>>         
>>>>>> http://people.apache.org/%7Echunrong/snapshots/r681495/index.html>
>>>>>>             
>>>>>>> <
>>>>>>>               
>>>>>>>> http://people.apache.org/%7Echunrong/snapshots/r681495/index.html
>>>>>>>>                 
>>>>> .
>>>>>           
>>>>>>>>> I am using
>>>>>>>>> two platforms (Linux x86, windows x86_64) at the moment.
>>>>>>>>>                   
>>>> Hopefully we
>>>>         
>>>>>>>> will
>>>>>>>>                 
>>>>>>>>> have other two platforms in future for M8. Well, although we are
>>>>>>>>>                   
>>>>>>> testing
>>>>>>>               
>>>>>>>>> only two platforms for M7, my experience is that the status for
>>>>>>>>>                   
>>>>>> another
>>>>>>             
>>>>>>>> two
>>>>>>>>                 
>>>>>>>>> platforms should be not worse or just include some extra
>>>>>>>>>                   
>>>> intermittent
>>>>         
>>>>>>>>> errors
>>>>>>>>> which can be investigated in some later stages.
>>>>>>>>>
>>>>>>>>>  The following suites passed on Linux x86/Windows x86_64
>>>>>>>>>                   
>>>> platforms:
>>>>         
>>>>>> Ant
>>>>>>             
>>>>>>>>> Scenario (or self-hosting), Axis application, Dacapo, DRLVM
>>>>>>>>>                   
>>>>>> regression
>>>>>>             
>>>>>>>>> tests, Geronimo Unit Tests, Scimark, Tomcat scenario, VTS VM Test
>>>>>>>>>                   
>>>>>>> Suite.
>>>>>>>               
>>>>>>>>>  Most of the failures are known issues (for M6). Although we can
>>>>>>>>>                   
>>>>>>> observe
>>>>>>>               
>>>>>>>>> 15~20 new issues, those issues happen only in 1 platform and they
>>>>>>>>>                   
>>>>>> look
>>>>>>             
>>>>>>>> more
>>>>>>>>                 
>>>>>>>>> like the intermittent/timeout issues (less reproducible) to me.
>>>>>>>>>                   
>>>>>> Overall
>>>>>>             
>>>>>>> I
>>>>>>>               
>>>>>>>>> myself would like to think r681495 is more stable than M6.
>>>>>>>>>
>>>>>>>>>  Please add your comments and clarifications (please also see M6
>>>>>>>>>                   
>>>>>>> testing
>>>>>>>               
>>>>>>>>> results http://people.apache.org/~smishura/r653525/<http://people.apache.org/%7Esmishura/r653525/>
>>>>>>>>>                   
>>>> <
>>>>         
>>>>>> http://people.apache.org/%7Esmishura/r653525/>
>>>>>>             
>>>>>>> <
>>>>>>>               
>>>>>>>> http://people.apache.org/%7Esmishura/r653525/>
>>>>>>>>                 
>>>>>>>>> ,
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805070304l38845ee0se01fb93fbfc05586@mail.gmail.com%3eand
>>>>         
>>>>>>>>> integrity testing results
>>>>>>>>> http://people.apache.org/~chunrong/harmony-integrity/<http://people.apache.org/%7Echunrong/harmony-integrity/>
>>>>>>>>>                   
>>>> <
>>>>         
>>>>>> http://people.apache.org/%7Echunrong/harmony-integrity/>
>>>>>>             
>>>>>>> <
>>>>>>>               
>>>>>>>> http://people.apache.org/%7Echunrong/harmony-integrity/>as a
>>>>>>>>                 
>>>>>>> comparison).
>>>>>>>               
>>>>>>>>>  1) Classlib:
>>>>>>>>>    1.1) Since r644719 (which committed patch for HARMONY-5684)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>  org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest
>>>>         
>>>>>>>>>
>>>>>>>>>                   
>>>> org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
>>>>         
>>>>>>>>>         failed in both platforms
>>>>>>>>>
>>>>>>>>>    1.2) Two failures
>>>>>>>>>
>>>>>>>>>        org.apache.harmony.luni.tests.java.net.MulticastSocketTest
>>>>>>>>>                   
>>>>>>> (Failed
>>>>>>>               
>>>>>>>>> in windows_x86 running of M6)
>>>>>>>>>        tests.api.java.security.PermissionCollectionTest
>>>>>>>>>                   
>>>>>>> (Failed
>>>>>>>               
>>>>>>>>> in M6)
>>>>>>>>>
>>>>>>>>>         are observed in Linux x86.
>>>>>>>>>
>>>>>>>>>  2) DRLVM tests:
>>>>>>>>>    2.1) One failure
>>>>>>>>>
>>>>>>>>>         java.lang.ClassGenericsTest.test_2
>>>>>>>>>
>>>>>>>>>         is observed in Linux x86 snapshot testing.
>>>>>>>>>         I can see some old discussion in the mailing list about
>>>>>>>>>                   
>>>> that
>>>>         
>>>>>>> but
>>>>>>>               
>>>>>>>> I
>>>>>>>>                 
>>>>>>>>> am not sure the expected status here.
>>>>>>>>>         They should be intemittent errors since the integrity
>>>>>>>>>                   
>>>> testing
>>>>         
>>>>>>>> just
>>>>>>>>                 
>>>>>>>>> run well mostly.
>>>>>>>>>
>>>>>>>>>  3) EUTs:
>>>>>>>>>
>>>>>>>>>    3.1) Linux x86: 99.36%
>>>>>>>>>         A recorded JIRA for this suite is HARMONY-2914 which
>>>>>>>>>                   
>>>> wastes
>>>>         
>>>>>>> file
>>>>>>>               
>>>>>>>>> handlers and makes system unstable.
>>>>>>>>>
>>>>>>>>>  4) Functional:
>>>>>>>>>    4.1) Old regressions on both platforms:
>>>>>>>>>         api.java.text.MessageFormat (HARMONY-5430)
>>>>>>>>>         api.java.util.jar.Manifest  (HARMONY-5473)
>>>>>>>>>         api.java.beans.beancontext.BeanContextTest (also in M6,
>>>>>>>>>                   
>>>>>>> recorded
>>>>>>>               
>>>>>>>>> as  regression caused by changes in locale data)
>>>>>>>>>         api.java.beans.persistence.EncoderTest  (also in M6,
>>>>>>>>>                   
>>>> recorded
>>>>         
>>>>>>> as
>>>>>>>               
>>>>>>>>> regression in the beans module)
>>>>>>>>>         api.java.beans.persistence.EncoderDecoderTest (also in
>>>>>>>>>                   
>>>> M6,
>>>>         
>>>>>>>>> regression in the beans module)
>>>>>>>>>         reg.vm.btest5625 (also in M6, recorded as intermittent
>>>>>>>>>                   
>>>> and
>>>>         
>>>>>> not
>>>>>>             
>>>>>>>>> reproducible manually)
>>>>>>>>>
>>>>>>>>>    4.2) Old regressions on 1 platform
>>>>>>>>>         api.java.rmi.basicexception (ERROR in Linux x86,
>>>>>>>>>                   
>>>>>> HARMONY-5823)
>>>>>>             
>>>>>>>>>         api.java.rmi.basicregistry.RemoteServerTest (ERROR in
>>>>>>>>>                   
>>>> Linux
>>>>         
>>>>>>> x86,
>>>>>>>               
>>>>>>>>> HARMONY-5823)
>>>>>>>>>         jpda.jdwp.scenario.ST07.ST07Test (ERROR in windows
>>>>>>>>>                   
>>>> x86_64, in
>>>>         
>>>>>>> M6
>>>>>>>               
>>>>>>>> it
>>>>>>>>                 
>>>>>>>>> is recorded as regression since M4)
>>>>>>>>>         java.math.F_BigIntegerMatrixMultiplyTest_01 (ERROR on
>>>>>>>>>                   
>>>> Linux
>>>>         
>>>>>>> x86,
>>>>>>>               
>>>>>>>>> recorded as Timeout, not reproducible
>>>>>>>>>  in M6)
>>>>>>>>>         reg.vm.btest5717 (ERROR on Windows X86_64, recorded as
>>>>>>>>>                   
>>>>>>> "timeout,
>>>>>>>               
>>>>>>>>> the test is too heavy" in M6)
>>>>>>>>>         jit.HLO.inline.ControlFlow.IfElse.IfElse1.IfElseTest1
>>>>>>>>>                   
>>>> (FAILED
>>>>         
>>>>>>> in
>>>>>>>               
>>>>>>>>> windows x86_64, recorded as "looks like
>>>>>>>>> issue in test" in M6)
>>>>>>>>>         jit.HLO.devirt.Runtime.RuntimeExtend1 (FAILED on windows
>>>>>>>>>                   
>>>>>>> x86_64,
>>>>>>>               
>>>>>>>> in
>>>>>>>>                 
>>>>>>>>> M6 it is recorded as not regression and started to fail on M5)
>>>>>>>>>         reg.vm.btest6353.Btest6353 (ERROR on Windows x86_64,
>>>>>>>>>                   
>>>> recorded
>>>>         
>>>>>>>> also
>>>>>>>>                 
>>>>>>>>> failed on M3 and might be issue with the test)
>>>>>>>>>
>>>>>>>>>    4.3) New regressions on 1 platform (looks intermittent)
>>>>>>>>>         reg.jit.btest8029.Btest8029 (FAILED in Linux x86)
>>>>>>>>>         func.reg.jit.btest5710.Btest5710 (FAILED in Linux x86)
>>>>>>>>>
>>>>>>>>>                   
>>>> api.java.security.cert.F_CertPathTest_06.F_CertPathTest_06
>>>>         
>>>>>>> (ERROR
>>>>>>>               
>>>>>>>>> in Linux x86)
>>>>>>>>>
>>>>>>>>>                   
>>>> api.java.security.cert.F_CertPathTest_05.F_CertPathTest_05
>>>>         
>>>>>>> (ERROR
>>>>>>>               
>>>>>>>>> in Linux x86)
>>>>>>>>>
>>>>>>>>>  5) JDKTools Tests:
>>>>>>>>>    Several timeouts are observed in Linux x86 snapshot running.
>>>>>>>>>                   
>>>> They
>>>>         
>>>>>>> are:
>>>>>>>               
>>>>>>>>>
>>>>>>>>>                   
>>>> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch001
>>>>         
>>>>>>>>>
>>>>>>>>>                   
>>>> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch002
>>>>         
>>>>>>>>>
>>>>>>>>>                   
>>>> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch003
>>>>         
>>>>>>>>>
>>>>>>>>>                   
>>>> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch004
>>>>         
>>>>>>>>>
>>>>>>>>>                   
>>>> org.apache.harmony.jpda.tests.jdwp.Events.CombinedEventsTest.testCombinedEvents_04
>>>>         
>>>>>>>>>
>>>>>>>>>                   
>>>> org.apache.harmony.jpda.tests.jdwp.MultiSession.AttachConnectorTest.testAttachConnector001
>>>>         
>>>>>>>>>
>>>>>>>>>                   
>>>> org.apache.harmony.jpda.tests.jdwp.MultiSession.MethodEntryExitTest.testMethodEvent001
>>>>         
>>>>>>>>>                   
>>>> org.apache.harmony.jpda.tests.jdwp.MultiSession.ResumeTest.testResume
>>>>         
>>>>>>>>>
>>>>>>>>>                   
>>>> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadEndTest.testThreadEnd001
>>>>         
>>>>>>>>>
>>>>>>>>>                   
>>>> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadStartTest.testThreadStart001
>>>>         
>>>>>>>>>    The Linux-only timeouts are also observed in the integrity
>>>>>>>>>                   
>>>> testing
>>>>         
>>>>>>>>> results.
>>>>>>>>>    JIRA HARMONY-5833 has been filed for one of them.
>>>>>>>>>
>>>>>>>>>  6) JettyScenario:
>>>>>>>>>    The Linux x86 running failed because of the unresolved
>>>>>>>>>                   
>>>>>> HARMONY-5219.
>>>>>>             
>>>>>>>>>  7) Reliability:
>>>>>>>>>    Several failures are observed in windows x86_64 running.
>>>>>>>>>    7.1) Old regressions
>>>>>>>>>         api.net.DatagramTest (HARMONY-5531)
>>>>>>>>>         api.text.DecimalFormat_Locales - (in M6 it is recorded as
>>>>>>>>>                   
>>>>>> also
>>>>>>             
>>>>>>>>> intermittent in M5)
>>>>>>>>>
>>>>>>>>>    7.2) New/intemittent regressions
>>>>>>>>>         api.kernel.thread.ThreadLocalTest.ThreadLocalTest
>>>>>>>>>         api.kernel.exec.RunExec
>>>>>>>>>
>>>>>>>>>  8) Stress
>>>>>>>>>    Different test cases failed on different platforms.
>>>>>>>>>    8.1) Timeouts on Linux x86.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD003.ThreadTest003
>>>>         
>>>>>>>>>
>>>>>>>>>                   
>>>> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD007.ThreadTest007
>>>>         
>>>>>>>>>
>>>>>>>>>                   
>>>> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD009.ThreadTest009
>>>>         
>>>>>>>>>
>>>>>>>>>                   
>>>> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD011.ThreadTest011
>>>>         
>>>>>>>>>    8.2) Failed cases on Windows x86_64 with unknown reasons.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>> stress.org.apache.harmony.test.stress.classloader.MixThreads.TreeClasses.testTreeClasses2
>>>>         
>>>>>>>>>
>>>>>>>>>                   
>>>> stress.org.apache.harmony.test.stress.classloader.NotSynchThreads.TreeClasses.testTreeClasses
>>>>         
>>>>>>>>>    I have not found records for them.
>>>>>>>>>
>>>>>>>>>  9) Strut_test
>>>>>>>>>    Broken with same error report as M6.
>>>>>>>>>
>>>>>>>>>  10) Eclipse Hello World Application.
>>>>>>>>>    Although the testing framework just reports EUT-API status as
>>>>>>>>>                   
>>>>>>>> "PASSED".
>>>>>>>>                 
>>>>>>>>> A fresh workspace running just failed in configuration stage (
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805120106o387a49f1rfb2c33d1042d2f41@mail.gmail.com%3e
>>>>         
>>>>>>>>> )
>>>>>>>>> since SVN commit 641928 (which committed patch for HARMONY-4569).
>>>>>>>>>
>>>>>>>>> thanks,
>>>>>>>>> chunrong
>>>>>>>>>
>>>>>>>>>                   
>>>>
>>>> --
>>>> Unless stated otherwise above:
>>>> IBM United Kingdom Limited - Registered in England and Wales with number
>>>> 741598.
>>>> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>>>>
>>>>         
>> --
>> Unless stated otherwise above:
>> IBM United Kingdom Limited - Registered in England and Wales with number 741598.
>> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>>
>>     
>
>
>   

-- 
Oliver Deakin
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU


Re: [testing] first M7 candidate (r681495) testing status

Posted by Sian January <si...@googlemail.com>.
Thanks Ligang - I meant to get to this at some point before M8 but
you've saved me a job now :-)



On 08/09/2008, Ligang Wang <wa...@gmail.com> wrote:
> Sian, I looked at the Throwable.initCause() problem and agree with you.
>
> My observation on this issue is as follows.
>
> DRLVM implementation of Throwable.initCause():
>       public Throwable initCause(Throwable initialCause) {
>              if (initialCause == this) {
>                     throw new IllegalArgumentException("A throwable cannot
> be its own cause.");
>              }
>              // second call of initCause(Throwable)
>              if (cause != this) {
>                     throw new IllegalStateException("A cause can be set at
> most once." +
>                    " Illegal attempt to re-set the cause of " + this);
>              }
>              cause = initialCause;
>              return this;
>       }
> Firstly "this" is assigned to the private member "cause".
>
> CLASSLIB implementation of Throwable.initCause():
> public synchronized Throwable initCause(Throwable throwable) {
>        if (cause == this) {
>            if (throwable != this) {
>                cause = throwable;
>                return this;
>            }
>            throw new IllegalArgumentException("Cause cannot be the
> receiver");
>        }
>        throw new IllegalStateException("Cause already initialized");
> }
> Firstly "this" is assigned to the private member "cause".
>
> The description of of JavaDoc of Throwable.initCause():
> public Throwable<file:///C:/Program%20Files/Java/jdk-1.5.0-docs/api/java/lang/Throwable.html>
> *initCause*(Throwable<file:///C:/Program%20Files/Java/jdk-1.5.0-docs/api/java/lang/Throwable.html>
>  cause)
> Initializes the *cause* of this throwable to the specified value. This
> method can be called at most once.
> *Throws:*
> IllegalArgumentException<file:///C:/Program%20Files/Java/jdk-1.5.0-docs/api/java/lang/IllegalArgumentException.html>-
> if
> cause is this throwable. (A throwable cannot be its own cause.)
> IllegalStateException<file:///C:/Program%20Files/Java/jdk-1.5.0-docs/api/java/lang/IllegalStateException.html>-
> if this throwable was created with
> Throwable(Throwable)<file:///C:/Program%20Files/Java/jdk-1.5.0-docs/api/java/lang/Throwable.html#Throwable(java.lang.Throwable)>or
> Throwable(String,Throwable)<file:///C:/Program%20Files/Java/jdk-1.5.0-docs/api/java/lang/Throwable.html#Throwable(java.lang.String,
> java.lang.Throwable)>, or this method has already been called on this
> throwable.
>
> The first exception is thrown when "cause" parameter is "this", and the
> second exception is thrown when "causce", the instance member, is set for
> the second time. From the description of JavaDoc, we can see there is an
> overlap between the two exceptions. If initCause is called for the second
> time and "cause" parameter is "this", which exception will be thrown?
> JavaDoc doesn't have clear description on this. The DRLVM implementation
> throws the first exception, while the CLASSLIB implementation throws the
> second. We can not tell which is better.
>
> But as Sian said, JavaDoc for ServerCloneException says that "Invoking the
> method Throwable.initCause(Throwable)<file:///C:/Program%20Files/Java/jdk-1.5.0-docs/api/java/lang/Throwable.html#initCause(java.lang.Throwable)>on
> an instance of
> ServerCloneException always throws
> IllegalStateException<file:///C:/Program%20Files/Java/jdk-1.5.0-docs/api/java/lang/IllegalStateException.html>".
> With this statement, we may see the CLASSLIB implementation is better. I
> have modified ServerCloneExceptionTest.test_Constructor_String() and
> Throwable.initCause() accordingly. The related JIRA issue is
> https://issues.apache.org/jira/browse/HARMONY-5971.
>
> Thanks,
> Ligang
>
>
> On Wed, Aug 13, 2008 at 6:50 PM, Sian January <si...@googlemail.com>wrote:
>
> > Oliver has helped me track this down this morning, and it's occuring
> > because there are differences between the versions of
> > Throwable.initCause(..) in luni and in DRLVM.  According to the spec,
> > I think the test should be changed to check for IllegalStateException
> > instead of IllegalArgumentException and then I think the version of
> > Throwable in DRLVM should be changed to behave the same as the one in
> > luni.  However I don't think this is a big deal for M7 because I don't
> > think it's very serious and it's not a regression since M6.  What does
> > anyone else think?
> >
> > Sian
> >
> > On 13/08/2008, Sian January <si...@googlemail.com> wrote:
> > > I think the code is right and the test is wrong here, because the
> > > Javadoc for ServerCloneException says that "Invoking the method
> > > Throwable.initCause(Throwable) on an instance of ServerCloneException
> > > always throws IllegalStateException."
> > >
> > > What is really confusing me is how this can be passing for anyone
> > > else... The exclude list hasn't changed recently and I don't see how
> > > it could be VM dependent either.
> > >
> > >
> > > On 13/08/2008, Nathan Beyer <nd...@apache.org> wrote:
> > > > On Tue, Aug 12, 2008 at 10:18 AM, Sian January
> > > > <si...@googlemail.com>wrote:
> > > >
> > > > > I am not seeing the management failures on my local (Windows)
> > machine,
> > > > > but I'm seeing an rmi failure:
> > > > >
> > > > >
> > > > >
> > org.apache.harmony.rmi.server.ServerCloneExceptionTest.test_Constructor_String
> > > > > Cause already initialized
> > > > > java.lang.IllegalStateException: Cause already initialized
> > > > > at java.lang.Throwable.initCause(Throwable.java:293)
> > > > > at
> > > > >
> > org.apache.harmony.rmi.server.ServerCloneExceptionTest.test_Constructor_String(ServerCloneExceptionTest.java:70)
> > > > > at
> > java.lang.reflect.AccessibleObject.invokeV(AccessibleObject.java:25)
> > > >
> > > >
> > > > This test [1] is creating an exception and then passing itself into
> > > > 'initCause', which should throw IllegalArgumentException. Looking at
> > the
> > > > code though, it does look like there's a bug, as 'null' is being passed
> > to
> > > > the super() constructor, which will mess up Throwable, since it
> > initializes
> > > > 'cause' to 'this', so Throwable will think the cause is already set.
> > > >
> > > > Did this test recently get removed from an exclusion list?
> > > >
> > > >
> > > > [1]
> > > >
> > http://svn.apache.org/viewvc/harmony/enhanced/classlib/trunk/modules/rmi/src/test/api/java/org/apache/harmony/rmi/server/ServerCloneExceptionTest.java?view=annotate
> > > >
> > > > >
> > > > >
> > > > > and an intermittent failure in java.net:
> > > > >
> > > > >
> > > > >
> > org.apache.harmony.luni.tests.java.net.URLConnectionTest.test_getInputStream
> > > > >
> > > > > Error receiving content coded data:
> > > > > junit.framework.AssertionFailedError: Error receiving content coded
> > data:
> > > > > at
> > tests.support.Support_HttpTests.runTests(Support_HttpTests.java:15)
> > > > > at
> > > > >
> > org.apache.harmony.luni.tests.java.net.URLConnectionTest.test_getInputStream(URLConnectionTest.java:632)
> > > > > at
> > java.lang.reflect.AccessibleObject.invokeV(AccessibleObject.java:25)
> > > > >
> > > > > Is anyone else seeing these?
> > > > >
> > > > > Sian
> > > > >
> > > > > On 12/08/2008, chunrong lai <ch...@gmail.com> wrote:
> > > > > >   Thanks.
> > > > > >   I think the standard scripts produce that site although I always
> > added
> > > > > > publish/e-mail notification setup before install/run the test
> > suites.
> > > > > >   It is understandable that some tests in classlib-test and
> > drlvm-test
> > > > > get
> > > > > > failing report. We just need to understand the failures.
> > > > > >   As reported in M6,
> > > > > > org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest,
> > > > > >
> > > > >
> > org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
> > > > > > etc fail classlib-test.
> > > > > >   Also we observed that java.lang.ClassGenericsTest.test_2
> > sometimes fail
> > > > > > drlvm-test.
> > > > > >   Another timeout issue of drlvm-test comes from
> > > > > > thread.SmallStackThreadTest_jit, as discussed in HARMONY-4601, the
> > test
> > > > > case
> > > > > > may run very slow when system resource is tight and finally
> > introduce a
> > > > > > timeout. Re-running the suite generally fixes the timeout.
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > On 8/12/08, Nathan Beyer <nd...@apache.org> wrote:
> > > > > > >
> > > > > > > It's the tests that are failing - both classlib-test and
> > drlvm-test.
> > > > > > >
> > > > > > > What's the specific integrity setup that's being used for that
> > site?
> > > > > Does
> > > > > > > the standard out-of-the-box buildtest scripts produce that site?
> > > > > > >
> > > > > > > -Nathan
> > > > > > >
> > > > > > > On Mon, Aug 11, 2008 at 9:05 PM, chunrong lai <
> > chunronglai@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > hi, Nathan:
> > > > > > > >     Steps to run testing cycle are listed in
> > > > > > > > http://harmony.apache.org/subcomponents/buildtest/index.html.
> > One
> > > > > > > > integrity
> > > > > > > > testing example is
> > > > > > > >
> > > > >
> > http://harmony.apache.org/subcomponents/buildtest/howto.html#Extended.
> > > > > > > >      What is the error message you met (in what step)? What is
> > the
> > > > > > > content
> > > > > > > > of the framework.local.properties file? I think that will help
> > to
> > > > > figure
> > > > > > > > out
> > > > > > > > the problem.
> > > > > > > >
> > > > > > > >
> > > > > > > > On 8/12/08, Nathan Beyer <nd...@apache.org> wrote:
> > > > > > > > >
> > > > > > > > > What's being used to produce the integrity tests? I have an
> > x86_64
> > > > > > > Linux
> > > > > > > > > box
> > > > > > > > > that I can dedicate to testing. I've tried just using
> > buildtest
> > > > > with
> > > > > > > > > 'classlib,drlvm,classib-test,drlvm-test', but I haven't
> > gotten a
> > > > > clean
> > > > > > > > pass
> > > > > > > > > yet.
> > > > > > > > >
> > > > > > > > > -Nathan
> > > > > > > > >
> > > > > > > > > On Mon, Aug 11, 2008 at 12:26 PM, chunrong lai <
> > > > > chunronglai@gmail.com
> > > > > > > > > >wrote:
> > > > > > > > >
> > > > > > > > > > Hi, all:
> > > > > > > > > >
> > > > > > > > > >  Here is r681495 snapshot testing status:
> > > > > > > > > >
> > http://people.apache.org/~chunrong/snapshots/r681495/index.html<
> > http://people.apache.org/%7Echunrong/snapshots/r681495/index.html>
> > > > > <
> > > > > > >
> > http://people.apache.org/%7Echunrong/snapshots/r681495/index.html>
> > > > > > > > <
> > > > > > > > >
> > http://people.apache.org/%7Echunrong/snapshots/r681495/index.html
> > > > > >.
> > > > > > > > > > I am using
> > > > > > > > > > two platforms (Linux x86, windows x86_64) at the moment.
> > > > > Hopefully we
> > > > > > > > > will
> > > > > > > > > > have other two platforms in future for M8. Well, although
> > we are
> > > > > > > > testing
> > > > > > > > > > only two platforms for M7, my experience is that the status
> > for
> > > > > > > another
> > > > > > > > > two
> > > > > > > > > > platforms should be not worse or just include some extra
> > > > > intermittent
> > > > > > > > > > errors
> > > > > > > > > > which can be investigated in some later stages.
> > > > > > > > > >
> > > > > > > > > >  The following suites passed on Linux x86/Windows x86_64
> > > > > platforms:
> > > > > > > Ant
> > > > > > > > > > Scenario (or self-hosting), Axis application, Dacapo, DRLVM
> > > > > > > regression
> > > > > > > > > > tests, Geronimo Unit Tests, Scimark, Tomcat scenario, VTS
> > VM Test
> > > > > > > > Suite.
> > > > > > > > > >
> > > > > > > > > >  Most of the failures are known issues (for M6). Although
> > we can
> > > > > > > > observe
> > > > > > > > > > 15~20 new issues, those issues happen only in 1 platform
> > and they
> > > > > > > look
> > > > > > > > > more
> > > > > > > > > > like the intermittent/timeout issues (less reproducible) to
> > me.
> > > > > > > Overall
> > > > > > > > I
> > > > > > > > > > myself would like to think r681495 is more stable than M6.
> > > > > > > > > >
> > > > > > > > > >  Please add your comments and clarifications (please also
> > see M6
> > > > > > > > testing
> > > > > > > > > > results http://people.apache.org/~smishura/r653525/<
> > http://people.apache.org/%7Esmishura/r653525/>
> > > > > <
> > > > > > > http://people.apache.org/%7Esmishura/r653525/>
> > > > > > > > <
> > > > > > > > > http://people.apache.org/%7Esmishura/r653525/>
> > > > > > > > > > ,
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805070304l38845ee0se01fb93fbfc05586@mail.gmail.com%3eand
> > > > > > > > > > integrity testing results
> > > > > > > > > > http://people.apache.org/~chunrong/harmony-integrity/<
> > http://people.apache.org/%7Echunrong/harmony-integrity/>
> > > > > <
> > > > > > > http://people.apache.org/%7Echunrong/harmony-integrity/>
> > > > > > > > <
> > > > > > > > > http://people.apache.org/%7Echunrong/harmony-integrity/>as a
> > > > > > > > comparison).
> > > > > > > > > >
> > > > > > > > > >  1) Classlib:
> > > > > > > > > >    1.1) Since r644719 (which committed patch for
> > HARMONY-5684)
> > > > > > > > > >
> > > > > > > > > >
> > > > >  org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
> > > > > > > > > >
> > > > > > > > > >         failed in both platforms
> > > > > > > > > >
> > > > > > > > > >    1.2) Two failures
> > > > > > > > > >
> > > > > > > > > >
> >  org.apache.harmony.luni.tests.java.net.MulticastSocketTest
> > > > > > > > (Failed
> > > > > > > > > > in windows_x86 running of M6)
> > > > > > > > > >        tests.api.java.security.PermissionCollectionTest
> > > > > > > > (Failed
> > > > > > > > > > in M6)
> > > > > > > > > >
> > > > > > > > > >         are observed in Linux x86.
> > > > > > > > > >
> > > > > > > > > >  2) DRLVM tests:
> > > > > > > > > >    2.1) One failure
> > > > > > > > > >
> > > > > > > > > >         java.lang.ClassGenericsTest.test_2
> > > > > > > > > >
> > > > > > > > > >         is observed in Linux x86 snapshot testing.
> > > > > > > > > >         I can see some old discussion in the mailing list
> > about
> > > > > that
> > > > > > > > but
> > > > > > > > > I
> > > > > > > > > > am not sure the expected status here.
> > > > > > > > > >         They should be intemittent errors since the
> > integrity
> > > > > testing
> > > > > > > > > just
> > > > > > > > > > run well mostly.
> > > > > > > > > >
> > > > > > > > > >  3) EUTs:
> > > > > > > > > >
> > > > > > > > > >    3.1) Linux x86: 99.36%
> > > > > > > > > >         A recorded JIRA for this suite is HARMONY-2914
> > which
> > > > > wastes
> > > > > > > > file
> > > > > > > > > > handlers and makes system unstable.
> > > > > > > > > >
> > > > > > > > > >  4) Functional:
> > > > > > > > > >    4.1) Old regressions on both platforms:
> > > > > > > > > >         api.java.text.MessageFormat (HARMONY-5430)
> > > > > > > > > >         api.java.util.jar.Manifest  (HARMONY-5473)
> > > > > > > > > >         api.java.beans.beancontext.BeanContextTest (also in
> > M6,
> > > > > > > > recorded
> > > > > > > > > > as  regression caused by changes in locale data)
> > > > > > > > > >         api.java.beans.persistence.EncoderTest  (also in
> > M6,
> > > > > recorded
> > > > > > > > as
> > > > > > > > > > regression in the beans module)
> > > > > > > > > >         api.java.beans.persistence.EncoderDecoderTest (also
> > in
> > > > > M6,
> > > > > > > > > > regression in the beans module)
> > > > > > > > > >         reg.vm.btest5625 (also in M6, recorded as
> > intermittent
> > > > > and
> > > > > > > not
> > > > > > > > > > reproducible manually)
> > > > > > > > > >
> > > > > > > > > >    4.2) Old regressions on 1 platform
> > > > > > > > > >         api.java.rmi.basicexception (ERROR in Linux x86,
> > > > > > > HARMONY-5823)
> > > > > > > > > >         api.java.rmi.basicregistry.RemoteServerTest (ERROR
> > in
> > > > > Linux
> > > > > > > > x86,
> > > > > > > > > > HARMONY-5823)
> > > > > > > > > >         jpda.jdwp.scenario.ST07.ST07Test (ERROR in windows
> > > > > x86_64, in
> > > > > > > > M6
> > > > > > > > > it
> > > > > > > > > > is recorded as regression since M4)
> > > > > > > > > >         java.math.F_BigIntegerMatrixMultiplyTest_01 (ERROR
> > on
> > > > > Linux
> > > > > > > > x86,
> > > > > > > > > > recorded as Timeout, not reproducible
> > > > > > > > > >  in M6)
> > > > > > > > > >         reg.vm.btest5717 (ERROR on Windows X86_64, recorded
> > as
> > > > > > > > "timeout,
> > > > > > > > > > the test is too heavy" in M6)
> > > > > > > > > >
> > jit.HLO.inline.ControlFlow.IfElse.IfElse1.IfElseTest1
> > > > > (FAILED
> > > > > > > > in
> > > > > > > > > > windows x86_64, recorded as "looks like
> > > > > > > > > > issue in test" in M6)
> > > > > > > > > >         jit.HLO.devirt.Runtime.RuntimeExtend1 (FAILED on
> > windows
> > > > > > > > x86_64,
> > > > > > > > > in
> > > > > > > > > > M6 it is recorded as not regression and started to fail on
> > M5)
> > > > > > > > > >         reg.vm.btest6353.Btest6353 (ERROR on Windows
> > x86_64,
> > > > > recorded
> > > > > > > > > also
> > > > > > > > > > failed on M3 and might be issue with the test)
> > > > > > > > > >
> > > > > > > > > >    4.3) New regressions on 1 platform (looks intermittent)
> > > > > > > > > >         reg.jit.btest8029.Btest8029 (FAILED in Linux x86)
> > > > > > > > > >         func.reg.jit.btest5710.Btest5710 (FAILED in Linux
> > x86)
> > > > > > > > > >
> > > > > api.java.security.cert.F_CertPathTest_06.F_CertPathTest_06
> > > > > > > > (ERROR
> > > > > > > > > > in Linux x86)
> > > > > > > > > >
> > > > > api.java.security.cert.F_CertPathTest_05.F_CertPathTest_05
> > > > > > > > (ERROR
> > > > > > > > > > in Linux x86)
> > > > > > > > > >
> > > > > > > > > >  5) JDKTools Tests:
> > > > > > > > > >    Several timeouts are observed in Linux x86 snapshot
> > running.
> > > > > They
> > > > > > > > are:
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch001
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch002
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch003
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch004
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > org.apache.harmony.jpda.tests.jdwp.Events.CombinedEventsTest.testCombinedEvents_04
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > org.apache.harmony.jpda.tests.jdwp.MultiSession.AttachConnectorTest.testAttachConnector001
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > org.apache.harmony.jpda.tests.jdwp.MultiSession.MethodEntryExitTest.testMethodEvent001
> > > > > > > > > >
> > > > > > > > > >
> > > > > org.apache.harmony.jpda.tests.jdwp.MultiSession.ResumeTest.testResume
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadEndTest.testThreadEnd001
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadStartTest.testThreadStart001
> > > > > > > > > >    The Linux-only timeouts are also observed in the
> > integrity
> > > > > testing
> > > > > > > > > > results.
> > > > > > > > > >    JIRA HARMONY-5833 has been filed for one of them.
> > > > > > > > > >
> > > > > > > > > >  6) JettyScenario:
> > > > > > > > > >    The Linux x86 running failed because of the unresolved
> > > > > > > HARMONY-5219.
> > > > > > > > > >
> > > > > > > > > >  7) Reliability:
> > > > > > > > > >    Several failures are observed in windows x86_64 running.
> > > > > > > > > >    7.1) Old regressions
> > > > > > > > > >         api.net.DatagramTest (HARMONY-5531)
> > > > > > > > > >         api.text.DecimalFormat_Locales - (in M6 it is
> > recorded as
> > > > > > > also
> > > > > > > > > > intermittent in M5)
> > > > > > > > > >
> > > > > > > > > >    7.2) New/intemittent regressions
> > > > > > > > > >         api.kernel.thread.ThreadLocalTest.ThreadLocalTest
> > > > > > > > > >         api.kernel.exec.RunExec
> > > > > > > > > >
> > > > > > > > > >  8) Stress
> > > > > > > > > >    Different test cases failed on different platforms.
> > > > > > > > > >    8.1) Timeouts on Linux x86.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD003.ThreadTest003
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD007.ThreadTest007
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD009.ThreadTest009
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD011.ThreadTest011
> > > > > > > > > >
> > > > > > > > > >    8.2) Failed cases on Windows x86_64 with unknown
> > reasons.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > stress.org.apache.harmony.test.stress.classloader.MixThreads.TreeClasses.testTreeClasses2
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > stress.org.apache.harmony.test.stress.classloader.NotSynchThreads.TreeClasses.testTreeClasses
> > > > > > > > > >
> > > > > > > > > >    I have not found records for them.
> > > > > > > > > >
> > > > > > > > > >  9) Strut_test
> > > > > > > > > >    Broken with same error report as M6.
> > > > > > > > > >
> > > > > > > > > >  10) Eclipse Hello World Application.
> > > > > > > > > >    Although the testing framework just reports EUT-API
> > status as
> > > > > > > > > "PASSED".
> > > > > > > > > > A fresh workspace running just failed in configuration
> > stage (
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805120106o387a49f1rfb2c33d1042d2f41@mail.gmail.com%3e
> > > > > > > > > > )
> > > > > > > > > > since SVN commit 641928 (which committed patch for
> > HARMONY-4569).
> > > > > > > > > >
> > > > > > > > > > thanks,
> > > > > > > > > > chunrong
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Unless stated otherwise above:
> > > > > IBM United Kingdom Limited - Registered in England and Wales with
> > number
> > > > > 741598.
> > > > > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire
> > PO6 3AU
> > > > >
> > > >
> > >
> > >
> > > --
> > > Unless stated otherwise above:
> > > IBM United Kingdom Limited - Registered in England and Wales with number
> > 741598.
> > > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
> > 3AU
> > >
> >
> >
> > --
> >  Unless stated otherwise above:
> > IBM United Kingdom Limited - Registered in England and Wales with number
> > 741598.
> > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
> >
>


-- 
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

Re: [testing] first M7 candidate (r681495) testing status

Posted by Ligang Wang <wa...@gmail.com>.
Sian, I looked at the Throwable.initCause() problem and agree with you.

My observation on this issue is as follows.

DRLVM implementation of Throwable.initCause():
       public Throwable initCause(Throwable initialCause) {
              if (initialCause == this) {
                     throw new IllegalArgumentException("A throwable cannot
be its own cause.");
              }
              // second call of initCause(Throwable)
              if (cause != this) {
                     throw new IllegalStateException("A cause can be set at
most once." +
                    " Illegal attempt to re-set the cause of " + this);
              }
              cause = initialCause;
              return this;
       }
Firstly "this" is assigned to the private member "cause".

CLASSLIB implementation of Throwable.initCause():
public synchronized Throwable initCause(Throwable throwable) {
        if (cause == this) {
            if (throwable != this) {
                cause = throwable;
                return this;
            }
            throw new IllegalArgumentException("Cause cannot be the
receiver");
        }
        throw new IllegalStateException("Cause already initialized");
}
Firstly "this" is assigned to the private member "cause".

The description of of JavaDoc of Throwable.initCause():
public Throwable<file:///C:/Program%20Files/Java/jdk-1.5.0-docs/api/java/lang/Throwable.html>
*initCause*(Throwable<file:///C:/Program%20Files/Java/jdk-1.5.0-docs/api/java/lang/Throwable.html>
 cause)
Initializes the *cause* of this throwable to the specified value. This
method can be called at most once.
*Throws:*
IllegalArgumentException<file:///C:/Program%20Files/Java/jdk-1.5.0-docs/api/java/lang/IllegalArgumentException.html>-
if
cause is this throwable. (A throwable cannot be its own cause.)
IllegalStateException<file:///C:/Program%20Files/Java/jdk-1.5.0-docs/api/java/lang/IllegalStateException.html>-
if this throwable was created with
Throwable(Throwable)<file:///C:/Program%20Files/Java/jdk-1.5.0-docs/api/java/lang/Throwable.html#Throwable(java.lang.Throwable)>or
Throwable(String,Throwable)<file:///C:/Program%20Files/Java/jdk-1.5.0-docs/api/java/lang/Throwable.html#Throwable(java.lang.String,
java.lang.Throwable)>, or this method has already been called on this
throwable.

The first exception is thrown when "cause" parameter is "this", and the
second exception is thrown when "causce", the instance member, is set for
the second time. From the description of JavaDoc, we can see there is an
overlap between the two exceptions. If initCause is called for the second
time and "cause" parameter is "this", which exception will be thrown?
JavaDoc doesn't have clear description on this. The DRLVM implementation
throws the first exception, while the CLASSLIB implementation throws the
second. We can not tell which is better.

But as Sian said, JavaDoc for ServerCloneException says that "Invoking the
method Throwable.initCause(Throwable)<file:///C:/Program%20Files/Java/jdk-1.5.0-docs/api/java/lang/Throwable.html#initCause(java.lang.Throwable)>on
an instance of
ServerCloneException always throws
IllegalStateException<file:///C:/Program%20Files/Java/jdk-1.5.0-docs/api/java/lang/IllegalStateException.html>".
With this statement, we may see the CLASSLIB implementation is better. I
have modified ServerCloneExceptionTest.test_Constructor_String() and
Throwable.initCause() accordingly. The related JIRA issue is
https://issues.apache.org/jira/browse/HARMONY-5971.

Thanks,
Ligang


On Wed, Aug 13, 2008 at 6:50 PM, Sian January <si...@googlemail.com>wrote:

> Oliver has helped me track this down this morning, and it's occuring
> because there are differences between the versions of
> Throwable.initCause(..) in luni and in DRLVM.  According to the spec,
> I think the test should be changed to check for IllegalStateException
> instead of IllegalArgumentException and then I think the version of
> Throwable in DRLVM should be changed to behave the same as the one in
> luni.  However I don't think this is a big deal for M7 because I don't
> think it's very serious and it's not a regression since M6.  What does
> anyone else think?
>
> Sian
>
> On 13/08/2008, Sian January <si...@googlemail.com> wrote:
> > I think the code is right and the test is wrong here, because the
> > Javadoc for ServerCloneException says that "Invoking the method
> > Throwable.initCause(Throwable) on an instance of ServerCloneException
> > always throws IllegalStateException."
> >
> > What is really confusing me is how this can be passing for anyone
> > else... The exclude list hasn't changed recently and I don't see how
> > it could be VM dependent either.
> >
> >
> > On 13/08/2008, Nathan Beyer <nd...@apache.org> wrote:
> > > On Tue, Aug 12, 2008 at 10:18 AM, Sian January
> > > <si...@googlemail.com>wrote:
> > >
> > > > I am not seeing the management failures on my local (Windows)
> machine,
> > > > but I'm seeing an rmi failure:
> > > >
> > > >
> > > >
> org.apache.harmony.rmi.server.ServerCloneExceptionTest.test_Constructor_String
> > > > Cause already initialized
> > > > java.lang.IllegalStateException: Cause already initialized
> > > > at java.lang.Throwable.initCause(Throwable.java:293)
> > > > at
> > > >
> org.apache.harmony.rmi.server.ServerCloneExceptionTest.test_Constructor_String(ServerCloneExceptionTest.java:70)
> > > > at
> java.lang.reflect.AccessibleObject.invokeV(AccessibleObject.java:25)
> > >
> > >
> > > This test [1] is creating an exception and then passing itself into
> > > 'initCause', which should throw IllegalArgumentException. Looking at
> the
> > > code though, it does look like there's a bug, as 'null' is being passed
> to
> > > the super() constructor, which will mess up Throwable, since it
> initializes
> > > 'cause' to 'this', so Throwable will think the cause is already set.
> > >
> > > Did this test recently get removed from an exclusion list?
> > >
> > >
> > > [1]
> > >
> http://svn.apache.org/viewvc/harmony/enhanced/classlib/trunk/modules/rmi/src/test/api/java/org/apache/harmony/rmi/server/ServerCloneExceptionTest.java?view=annotate
> > >
> > > >
> > > >
> > > > and an intermittent failure in java.net:
> > > >
> > > >
> > > >
> org.apache.harmony.luni.tests.java.net.URLConnectionTest.test_getInputStream
> > > >
> > > > Error receiving content coded data:
> > > > junit.framework.AssertionFailedError: Error receiving content coded
> data:
> > > > at
> tests.support.Support_HttpTests.runTests(Support_HttpTests.java:15)
> > > > at
> > > >
> org.apache.harmony.luni.tests.java.net.URLConnectionTest.test_getInputStream(URLConnectionTest.java:632)
> > > > at
> java.lang.reflect.AccessibleObject.invokeV(AccessibleObject.java:25)
> > > >
> > > > Is anyone else seeing these?
> > > >
> > > > Sian
> > > >
> > > > On 12/08/2008, chunrong lai <ch...@gmail.com> wrote:
> > > > >   Thanks.
> > > > >   I think the standard scripts produce that site although I always
> added
> > > > > publish/e-mail notification setup before install/run the test
> suites.
> > > > >   It is understandable that some tests in classlib-test and
> drlvm-test
> > > > get
> > > > > failing report. We just need to understand the failures.
> > > > >   As reported in M6,
> > > > > org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest,
> > > > >
> > > >
> org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
> > > > > etc fail classlib-test.
> > > > >   Also we observed that java.lang.ClassGenericsTest.test_2
> sometimes fail
> > > > > drlvm-test.
> > > > >   Another timeout issue of drlvm-test comes from
> > > > > thread.SmallStackThreadTest_jit, as discussed in HARMONY-4601, the
> test
> > > > case
> > > > > may run very slow when system resource is tight and finally
> introduce a
> > > > > timeout. Re-running the suite generally fixes the timeout.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On 8/12/08, Nathan Beyer <nd...@apache.org> wrote:
> > > > > >
> > > > > > It's the tests that are failing - both classlib-test and
> drlvm-test.
> > > > > >
> > > > > > What's the specific integrity setup that's being used for that
> site?
> > > > Does
> > > > > > the standard out-of-the-box buildtest scripts produce that site?
> > > > > >
> > > > > > -Nathan
> > > > > >
> > > > > > On Mon, Aug 11, 2008 at 9:05 PM, chunrong lai <
> chunronglai@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > hi, Nathan:
> > > > > > >     Steps to run testing cycle are listed in
> > > > > > > http://harmony.apache.org/subcomponents/buildtest/index.html.
> One
> > > > > > > integrity
> > > > > > > testing example is
> > > > > > >
> > > >
> http://harmony.apache.org/subcomponents/buildtest/howto.html#Extended.
> > > > > > >      What is the error message you met (in what step)? What is
> the
> > > > > > content
> > > > > > > of the framework.local.properties file? I think that will help
> to
> > > > figure
> > > > > > > out
> > > > > > > the problem.
> > > > > > >
> > > > > > >
> > > > > > > On 8/12/08, Nathan Beyer <nd...@apache.org> wrote:
> > > > > > > >
> > > > > > > > What's being used to produce the integrity tests? I have an
> x86_64
> > > > > > Linux
> > > > > > > > box
> > > > > > > > that I can dedicate to testing. I've tried just using
> buildtest
> > > > with
> > > > > > > > 'classlib,drlvm,classib-test,drlvm-test', but I haven't
> gotten a
> > > > clean
> > > > > > > pass
> > > > > > > > yet.
> > > > > > > >
> > > > > > > > -Nathan
> > > > > > > >
> > > > > > > > On Mon, Aug 11, 2008 at 12:26 PM, chunrong lai <
> > > > chunronglai@gmail.com
> > > > > > > > >wrote:
> > > > > > > >
> > > > > > > > > Hi, all:
> > > > > > > > >
> > > > > > > > >  Here is r681495 snapshot testing status:
> > > > > > > > >
> http://people.apache.org/~chunrong/snapshots/r681495/index.html<
> http://people.apache.org/%7Echunrong/snapshots/r681495/index.html>
> > > > <
> > > > > >
> http://people.apache.org/%7Echunrong/snapshots/r681495/index.html>
> > > > > > > <
> > > > > > > >
> http://people.apache.org/%7Echunrong/snapshots/r681495/index.html
> > > > >.
> > > > > > > > > I am using
> > > > > > > > > two platforms (Linux x86, windows x86_64) at the moment.
> > > > Hopefully we
> > > > > > > > will
> > > > > > > > > have other two platforms in future for M8. Well, although
> we are
> > > > > > > testing
> > > > > > > > > only two platforms for M7, my experience is that the status
> for
> > > > > > another
> > > > > > > > two
> > > > > > > > > platforms should be not worse or just include some extra
> > > > intermittent
> > > > > > > > > errors
> > > > > > > > > which can be investigated in some later stages.
> > > > > > > > >
> > > > > > > > >  The following suites passed on Linux x86/Windows x86_64
> > > > platforms:
> > > > > > Ant
> > > > > > > > > Scenario (or self-hosting), Axis application, Dacapo, DRLVM
> > > > > > regression
> > > > > > > > > tests, Geronimo Unit Tests, Scimark, Tomcat scenario, VTS
> VM Test
> > > > > > > Suite.
> > > > > > > > >
> > > > > > > > >  Most of the failures are known issues (for M6). Although
> we can
> > > > > > > observe
> > > > > > > > > 15~20 new issues, those issues happen only in 1 platform
> and they
> > > > > > look
> > > > > > > > more
> > > > > > > > > like the intermittent/timeout issues (less reproducible) to
> me.
> > > > > > Overall
> > > > > > > I
> > > > > > > > > myself would like to think r681495 is more stable than M6.
> > > > > > > > >
> > > > > > > > >  Please add your comments and clarifications (please also
> see M6
> > > > > > > testing
> > > > > > > > > results http://people.apache.org/~smishura/r653525/<
> http://people.apache.org/%7Esmishura/r653525/>
> > > > <
> > > > > > http://people.apache.org/%7Esmishura/r653525/>
> > > > > > > <
> > > > > > > > http://people.apache.org/%7Esmishura/r653525/>
> > > > > > > > > ,
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > >
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805070304l38845ee0se01fb93fbfc05586@mail.gmail.com%3eand
> > > > > > > > > integrity testing results
> > > > > > > > > http://people.apache.org/~chunrong/harmony-integrity/<
> http://people.apache.org/%7Echunrong/harmony-integrity/>
> > > > <
> > > > > > http://people.apache.org/%7Echunrong/harmony-integrity/>
> > > > > > > <
> > > > > > > > http://people.apache.org/%7Echunrong/harmony-integrity/>as a
> > > > > > > comparison).
> > > > > > > > >
> > > > > > > > >  1) Classlib:
> > > > > > > > >    1.1) Since r644719 (which committed patch for
> HARMONY-5684)
> > > > > > > > >
> > > > > > > > >
> > > >  org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > >
> org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
> > > > > > > > >
> > > > > > > > >         failed in both platforms
> > > > > > > > >
> > > > > > > > >    1.2) Two failures
> > > > > > > > >
> > > > > > > > >
>  org.apache.harmony.luni.tests.java.net.MulticastSocketTest
> > > > > > > (Failed
> > > > > > > > > in windows_x86 running of M6)
> > > > > > > > >        tests.api.java.security.PermissionCollectionTest
> > > > > > > (Failed
> > > > > > > > > in M6)
> > > > > > > > >
> > > > > > > > >         are observed in Linux x86.
> > > > > > > > >
> > > > > > > > >  2) DRLVM tests:
> > > > > > > > >    2.1) One failure
> > > > > > > > >
> > > > > > > > >         java.lang.ClassGenericsTest.test_2
> > > > > > > > >
> > > > > > > > >         is observed in Linux x86 snapshot testing.
> > > > > > > > >         I can see some old discussion in the mailing list
> about
> > > > that
> > > > > > > but
> > > > > > > > I
> > > > > > > > > am not sure the expected status here.
> > > > > > > > >         They should be intemittent errors since the
> integrity
> > > > testing
> > > > > > > > just
> > > > > > > > > run well mostly.
> > > > > > > > >
> > > > > > > > >  3) EUTs:
> > > > > > > > >
> > > > > > > > >    3.1) Linux x86: 99.36%
> > > > > > > > >         A recorded JIRA for this suite is HARMONY-2914
> which
> > > > wastes
> > > > > > > file
> > > > > > > > > handlers and makes system unstable.
> > > > > > > > >
> > > > > > > > >  4) Functional:
> > > > > > > > >    4.1) Old regressions on both platforms:
> > > > > > > > >         api.java.text.MessageFormat (HARMONY-5430)
> > > > > > > > >         api.java.util.jar.Manifest  (HARMONY-5473)
> > > > > > > > >         api.java.beans.beancontext.BeanContextTest (also in
> M6,
> > > > > > > recorded
> > > > > > > > > as  regression caused by changes in locale data)
> > > > > > > > >         api.java.beans.persistence.EncoderTest  (also in
> M6,
> > > > recorded
> > > > > > > as
> > > > > > > > > regression in the beans module)
> > > > > > > > >         api.java.beans.persistence.EncoderDecoderTest (also
> in
> > > > M6,
> > > > > > > > > regression in the beans module)
> > > > > > > > >         reg.vm.btest5625 (also in M6, recorded as
> intermittent
> > > > and
> > > > > > not
> > > > > > > > > reproducible manually)
> > > > > > > > >
> > > > > > > > >    4.2) Old regressions on 1 platform
> > > > > > > > >         api.java.rmi.basicexception (ERROR in Linux x86,
> > > > > > HARMONY-5823)
> > > > > > > > >         api.java.rmi.basicregistry.RemoteServerTest (ERROR
> in
> > > > Linux
> > > > > > > x86,
> > > > > > > > > HARMONY-5823)
> > > > > > > > >         jpda.jdwp.scenario.ST07.ST07Test (ERROR in windows
> > > > x86_64, in
> > > > > > > M6
> > > > > > > > it
> > > > > > > > > is recorded as regression since M4)
> > > > > > > > >         java.math.F_BigIntegerMatrixMultiplyTest_01 (ERROR
> on
> > > > Linux
> > > > > > > x86,
> > > > > > > > > recorded as Timeout, not reproducible
> > > > > > > > >  in M6)
> > > > > > > > >         reg.vm.btest5717 (ERROR on Windows X86_64, recorded
> as
> > > > > > > "timeout,
> > > > > > > > > the test is too heavy" in M6)
> > > > > > > > >
> jit.HLO.inline.ControlFlow.IfElse.IfElse1.IfElseTest1
> > > > (FAILED
> > > > > > > in
> > > > > > > > > windows x86_64, recorded as "looks like
> > > > > > > > > issue in test" in M6)
> > > > > > > > >         jit.HLO.devirt.Runtime.RuntimeExtend1 (FAILED on
> windows
> > > > > > > x86_64,
> > > > > > > > in
> > > > > > > > > M6 it is recorded as not regression and started to fail on
> M5)
> > > > > > > > >         reg.vm.btest6353.Btest6353 (ERROR on Windows
> x86_64,
> > > > recorded
> > > > > > > > also
> > > > > > > > > failed on M3 and might be issue with the test)
> > > > > > > > >
> > > > > > > > >    4.3) New regressions on 1 platform (looks intermittent)
> > > > > > > > >         reg.jit.btest8029.Btest8029 (FAILED in Linux x86)
> > > > > > > > >         func.reg.jit.btest5710.Btest5710 (FAILED in Linux
> x86)
> > > > > > > > >
> > > > api.java.security.cert.F_CertPathTest_06.F_CertPathTest_06
> > > > > > > (ERROR
> > > > > > > > > in Linux x86)
> > > > > > > > >
> > > > api.java.security.cert.F_CertPathTest_05.F_CertPathTest_05
> > > > > > > (ERROR
> > > > > > > > > in Linux x86)
> > > > > > > > >
> > > > > > > > >  5) JDKTools Tests:
> > > > > > > > >    Several timeouts are observed in Linux x86 snapshot
> running.
> > > > They
> > > > > > > are:
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch001
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch002
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch003
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch004
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > >
> org.apache.harmony.jpda.tests.jdwp.Events.CombinedEventsTest.testCombinedEvents_04
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.AttachConnectorTest.testAttachConnector001
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.MethodEntryExitTest.testMethodEvent001
> > > > > > > > >
> > > > > > > > >
> > > > org.apache.harmony.jpda.tests.jdwp.MultiSession.ResumeTest.testResume
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadEndTest.testThreadEnd001
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadStartTest.testThreadStart001
> > > > > > > > >    The Linux-only timeouts are also observed in the
> integrity
> > > > testing
> > > > > > > > > results.
> > > > > > > > >    JIRA HARMONY-5833 has been filed for one of them.
> > > > > > > > >
> > > > > > > > >  6) JettyScenario:
> > > > > > > > >    The Linux x86 running failed because of the unresolved
> > > > > > HARMONY-5219.
> > > > > > > > >
> > > > > > > > >  7) Reliability:
> > > > > > > > >    Several failures are observed in windows x86_64 running.
> > > > > > > > >    7.1) Old regressions
> > > > > > > > >         api.net.DatagramTest (HARMONY-5531)
> > > > > > > > >         api.text.DecimalFormat_Locales - (in M6 it is
> recorded as
> > > > > > also
> > > > > > > > > intermittent in M5)
> > > > > > > > >
> > > > > > > > >    7.2) New/intemittent regressions
> > > > > > > > >         api.kernel.thread.ThreadLocalTest.ThreadLocalTest
> > > > > > > > >         api.kernel.exec.RunExec
> > > > > > > > >
> > > > > > > > >  8) Stress
> > > > > > > > >    Different test cases failed on different platforms.
> > > > > > > > >    8.1) Timeouts on Linux x86.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD003.ThreadTest003
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD007.ThreadTest007
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD009.ThreadTest009
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD011.ThreadTest011
> > > > > > > > >
> > > > > > > > >    8.2) Failed cases on Windows x86_64 with unknown
> reasons.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > >
> stress.org.apache.harmony.test.stress.classloader.MixThreads.TreeClasses.testTreeClasses2
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > >
> stress.org.apache.harmony.test.stress.classloader.NotSynchThreads.TreeClasses.testTreeClasses
> > > > > > > > >
> > > > > > > > >    I have not found records for them.
> > > > > > > > >
> > > > > > > > >  9) Strut_test
> > > > > > > > >    Broken with same error report as M6.
> > > > > > > > >
> > > > > > > > >  10) Eclipse Hello World Application.
> > > > > > > > >    Although the testing framework just reports EUT-API
> status as
> > > > > > > > "PASSED".
> > > > > > > > > A fresh workspace running just failed in configuration
> stage (
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > >
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805120106o387a49f1rfb2c33d1042d2f41@mail.gmail.com%3e
> > > > > > > > > )
> > > > > > > > > since SVN commit 641928 (which committed patch for
> HARMONY-4569).
> > > > > > > > >
> > > > > > > > > thanks,
> > > > > > > > > chunrong
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Unless stated otherwise above:
> > > > IBM United Kingdom Limited - Registered in England and Wales with
> number
> > > > 741598.
> > > > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire
> PO6 3AU
> > > >
> > >
> >
> >
> > --
> > Unless stated otherwise above:
> > IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
> 3AU
> >
>
>
> --
>  Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>

Re: [testing] first M7 candidate (r681495) testing status

Posted by Sian January <si...@googlemail.com>.
Oliver has helped me track this down this morning, and it's occuring
because there are differences between the versions of
Throwable.initCause(..) in luni and in DRLVM.  According to the spec,
I think the test should be changed to check for IllegalStateException
instead of IllegalArgumentException and then I think the version of
Throwable in DRLVM should be changed to behave the same as the one in
luni.  However I don't think this is a big deal for M7 because I don't
think it's very serious and it's not a regression since M6.  What does
anyone else think?

Sian

On 13/08/2008, Sian January <si...@googlemail.com> wrote:
> I think the code is right and the test is wrong here, because the
> Javadoc for ServerCloneException says that "Invoking the method
> Throwable.initCause(Throwable) on an instance of ServerCloneException
> always throws IllegalStateException."
>
> What is really confusing me is how this can be passing for anyone
> else... The exclude list hasn't changed recently and I don't see how
> it could be VM dependent either.
>
>
> On 13/08/2008, Nathan Beyer <nd...@apache.org> wrote:
> > On Tue, Aug 12, 2008 at 10:18 AM, Sian January
> > <si...@googlemail.com>wrote:
> >
> > > I am not seeing the management failures on my local (Windows) machine,
> > > but I'm seeing an rmi failure:
> > >
> > >
> > > org.apache.harmony.rmi.server.ServerCloneExceptionTest.test_Constructor_String
> > > Cause already initialized
> > > java.lang.IllegalStateException: Cause already initialized
> > > at java.lang.Throwable.initCause(Throwable.java:293)
> > > at
> > > org.apache.harmony.rmi.server.ServerCloneExceptionTest.test_Constructor_String(ServerCloneExceptionTest.java:70)
> > > at java.lang.reflect.AccessibleObject.invokeV(AccessibleObject.java:25)
> >
> >
> > This test [1] is creating an exception and then passing itself into
> > 'initCause', which should throw IllegalArgumentException. Looking at the
> > code though, it does look like there's a bug, as 'null' is being passed to
> > the super() constructor, which will mess up Throwable, since it initializes
> > 'cause' to 'this', so Throwable will think the cause is already set.
> >
> > Did this test recently get removed from an exclusion list?
> >
> >
> > [1]
> > http://svn.apache.org/viewvc/harmony/enhanced/classlib/trunk/modules/rmi/src/test/api/java/org/apache/harmony/rmi/server/ServerCloneExceptionTest.java?view=annotate
> >
> > >
> > >
> > > and an intermittent failure in java.net:
> > >
> > >
> > > org.apache.harmony.luni.tests.java.net.URLConnectionTest.test_getInputStream
> > >
> > > Error receiving content coded data:
> > > junit.framework.AssertionFailedError: Error receiving content coded data:
> > > at tests.support.Support_HttpTests.runTests(Support_HttpTests.java:15)
> > > at
> > > org.apache.harmony.luni.tests.java.net.URLConnectionTest.test_getInputStream(URLConnectionTest.java:632)
> > > at java.lang.reflect.AccessibleObject.invokeV(AccessibleObject.java:25)
> > >
> > > Is anyone else seeing these?
> > >
> > > Sian
> > >
> > > On 12/08/2008, chunrong lai <ch...@gmail.com> wrote:
> > > >   Thanks.
> > > >   I think the standard scripts produce that site although I always added
> > > > publish/e-mail notification setup before install/run the test suites.
> > > >   It is understandable that some tests in classlib-test and drlvm-test
> > > get
> > > > failing report. We just need to understand the failures.
> > > >   As reported in M6,
> > > > org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest,
> > > >
> > > org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
> > > > etc fail classlib-test.
> > > >   Also we observed that java.lang.ClassGenericsTest.test_2 sometimes fail
> > > > drlvm-test.
> > > >   Another timeout issue of drlvm-test comes from
> > > > thread.SmallStackThreadTest_jit, as discussed in HARMONY-4601, the test
> > > case
> > > > may run very slow when system resource is tight and finally introduce a
> > > > timeout. Re-running the suite generally fixes the timeout.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On 8/12/08, Nathan Beyer <nd...@apache.org> wrote:
> > > > >
> > > > > It's the tests that are failing - both classlib-test and drlvm-test.
> > > > >
> > > > > What's the specific integrity setup that's being used for that site?
> > > Does
> > > > > the standard out-of-the-box buildtest scripts produce that site?
> > > > >
> > > > > -Nathan
> > > > >
> > > > > On Mon, Aug 11, 2008 at 9:05 PM, chunrong lai <ch...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > hi, Nathan:
> > > > > >     Steps to run testing cycle are listed in
> > > > > > http://harmony.apache.org/subcomponents/buildtest/index.html. One
> > > > > > integrity
> > > > > > testing example is
> > > > > >
> > > http://harmony.apache.org/subcomponents/buildtest/howto.html#Extended.
> > > > > >      What is the error message you met (in what step)? What is the
> > > > > content
> > > > > > of the framework.local.properties file? I think that will help to
> > > figure
> > > > > > out
> > > > > > the problem.
> > > > > >
> > > > > >
> > > > > > On 8/12/08, Nathan Beyer <nd...@apache.org> wrote:
> > > > > > >
> > > > > > > What's being used to produce the integrity tests? I have an x86_64
> > > > > Linux
> > > > > > > box
> > > > > > > that I can dedicate to testing. I've tried just using buildtest
> > > with
> > > > > > > 'classlib,drlvm,classib-test,drlvm-test', but I haven't gotten a
> > > clean
> > > > > > pass
> > > > > > > yet.
> > > > > > >
> > > > > > > -Nathan
> > > > > > >
> > > > > > > On Mon, Aug 11, 2008 at 12:26 PM, chunrong lai <
> > > chunronglai@gmail.com
> > > > > > > >wrote:
> > > > > > >
> > > > > > > > Hi, all:
> > > > > > > >
> > > > > > > >  Here is r681495 snapshot testing status:
> > > > > > > > http://people.apache.org/~chunrong/snapshots/r681495/index.html<http://people.apache.org/%7Echunrong/snapshots/r681495/index.html>
> > > <
> > > > > http://people.apache.org/%7Echunrong/snapshots/r681495/index.html>
> > > > > > <
> > > > > > > http://people.apache.org/%7Echunrong/snapshots/r681495/index.html
> > > >.
> > > > > > > > I am using
> > > > > > > > two platforms (Linux x86, windows x86_64) at the moment.
> > > Hopefully we
> > > > > > > will
> > > > > > > > have other two platforms in future for M8. Well, although we are
> > > > > > testing
> > > > > > > > only two platforms for M7, my experience is that the status for
> > > > > another
> > > > > > > two
> > > > > > > > platforms should be not worse or just include some extra
> > > intermittent
> > > > > > > > errors
> > > > > > > > which can be investigated in some later stages.
> > > > > > > >
> > > > > > > >  The following suites passed on Linux x86/Windows x86_64
> > > platforms:
> > > > > Ant
> > > > > > > > Scenario (or self-hosting), Axis application, Dacapo, DRLVM
> > > > > regression
> > > > > > > > tests, Geronimo Unit Tests, Scimark, Tomcat scenario, VTS VM Test
> > > > > > Suite.
> > > > > > > >
> > > > > > > >  Most of the failures are known issues (for M6). Although we can
> > > > > > observe
> > > > > > > > 15~20 new issues, those issues happen only in 1 platform and they
> > > > > look
> > > > > > > more
> > > > > > > > like the intermittent/timeout issues (less reproducible) to me.
> > > > > Overall
> > > > > > I
> > > > > > > > myself would like to think r681495 is more stable than M6.
> > > > > > > >
> > > > > > > >  Please add your comments and clarifications (please also see M6
> > > > > > testing
> > > > > > > > results http://people.apache.org/~smishura/r653525/<http://people.apache.org/%7Esmishura/r653525/>
> > > <
> > > > > http://people.apache.org/%7Esmishura/r653525/>
> > > > > > <
> > > > > > > http://people.apache.org/%7Esmishura/r653525/>
> > > > > > > > ,
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805070304l38845ee0se01fb93fbfc05586@mail.gmail.com%3eand
> > > > > > > > integrity testing results
> > > > > > > > http://people.apache.org/~chunrong/harmony-integrity/<http://people.apache.org/%7Echunrong/harmony-integrity/>
> > > <
> > > > > http://people.apache.org/%7Echunrong/harmony-integrity/>
> > > > > > <
> > > > > > > http://people.apache.org/%7Echunrong/harmony-integrity/>as a
> > > > > > comparison).
> > > > > > > >
> > > > > > > >  1) Classlib:
> > > > > > > >    1.1) Since r644719 (which committed patch for HARMONY-5684)
> > > > > > > >
> > > > > > > >
> > >  org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
> > > > > > > >
> > > > > > > >         failed in both platforms
> > > > > > > >
> > > > > > > >    1.2) Two failures
> > > > > > > >
> > > > > > > >        org.apache.harmony.luni.tests.java.net.MulticastSocketTest
> > > > > > (Failed
> > > > > > > > in windows_x86 running of M6)
> > > > > > > >        tests.api.java.security.PermissionCollectionTest
> > > > > > (Failed
> > > > > > > > in M6)
> > > > > > > >
> > > > > > > >         are observed in Linux x86.
> > > > > > > >
> > > > > > > >  2) DRLVM tests:
> > > > > > > >    2.1) One failure
> > > > > > > >
> > > > > > > >         java.lang.ClassGenericsTest.test_2
> > > > > > > >
> > > > > > > >         is observed in Linux x86 snapshot testing.
> > > > > > > >         I can see some old discussion in the mailing list about
> > > that
> > > > > > but
> > > > > > > I
> > > > > > > > am not sure the expected status here.
> > > > > > > >         They should be intemittent errors since the integrity
> > > testing
> > > > > > > just
> > > > > > > > run well mostly.
> > > > > > > >
> > > > > > > >  3) EUTs:
> > > > > > > >
> > > > > > > >    3.1) Linux x86: 99.36%
> > > > > > > >         A recorded JIRA for this suite is HARMONY-2914 which
> > > wastes
> > > > > > file
> > > > > > > > handlers and makes system unstable.
> > > > > > > >
> > > > > > > >  4) Functional:
> > > > > > > >    4.1) Old regressions on both platforms:
> > > > > > > >         api.java.text.MessageFormat (HARMONY-5430)
> > > > > > > >         api.java.util.jar.Manifest  (HARMONY-5473)
> > > > > > > >         api.java.beans.beancontext.BeanContextTest (also in M6,
> > > > > > recorded
> > > > > > > > as  regression caused by changes in locale data)
> > > > > > > >         api.java.beans.persistence.EncoderTest  (also in M6,
> > > recorded
> > > > > > as
> > > > > > > > regression in the beans module)
> > > > > > > >         api.java.beans.persistence.EncoderDecoderTest (also in
> > > M6,
> > > > > > > > regression in the beans module)
> > > > > > > >         reg.vm.btest5625 (also in M6, recorded as intermittent
> > > and
> > > > > not
> > > > > > > > reproducible manually)
> > > > > > > >
> > > > > > > >    4.2) Old regressions on 1 platform
> > > > > > > >         api.java.rmi.basicexception (ERROR in Linux x86,
> > > > > HARMONY-5823)
> > > > > > > >         api.java.rmi.basicregistry.RemoteServerTest (ERROR in
> > > Linux
> > > > > > x86,
> > > > > > > > HARMONY-5823)
> > > > > > > >         jpda.jdwp.scenario.ST07.ST07Test (ERROR in windows
> > > x86_64, in
> > > > > > M6
> > > > > > > it
> > > > > > > > is recorded as regression since M4)
> > > > > > > >         java.math.F_BigIntegerMatrixMultiplyTest_01 (ERROR on
> > > Linux
> > > > > > x86,
> > > > > > > > recorded as Timeout, not reproducible
> > > > > > > >  in M6)
> > > > > > > >         reg.vm.btest5717 (ERROR on Windows X86_64, recorded as
> > > > > > "timeout,
> > > > > > > > the test is too heavy" in M6)
> > > > > > > >         jit.HLO.inline.ControlFlow.IfElse.IfElse1.IfElseTest1
> > > (FAILED
> > > > > > in
> > > > > > > > windows x86_64, recorded as "looks like
> > > > > > > > issue in test" in M6)
> > > > > > > >         jit.HLO.devirt.Runtime.RuntimeExtend1 (FAILED on windows
> > > > > > x86_64,
> > > > > > > in
> > > > > > > > M6 it is recorded as not regression and started to fail on M5)
> > > > > > > >         reg.vm.btest6353.Btest6353 (ERROR on Windows x86_64,
> > > recorded
> > > > > > > also
> > > > > > > > failed on M3 and might be issue with the test)
> > > > > > > >
> > > > > > > >    4.3) New regressions on 1 platform (looks intermittent)
> > > > > > > >         reg.jit.btest8029.Btest8029 (FAILED in Linux x86)
> > > > > > > >         func.reg.jit.btest5710.Btest5710 (FAILED in Linux x86)
> > > > > > > >
> > > api.java.security.cert.F_CertPathTest_06.F_CertPathTest_06
> > > > > > (ERROR
> > > > > > > > in Linux x86)
> > > > > > > >
> > > api.java.security.cert.F_CertPathTest_05.F_CertPathTest_05
> > > > > > (ERROR
> > > > > > > > in Linux x86)
> > > > > > > >
> > > > > > > >  5) JDKTools Tests:
> > > > > > > >    Several timeouts are observed in Linux x86 snapshot running.
> > > They
> > > > > > are:
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch001
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch002
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch003
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch004
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > org.apache.harmony.jpda.tests.jdwp.Events.CombinedEventsTest.testCombinedEvents_04
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > org.apache.harmony.jpda.tests.jdwp.MultiSession.AttachConnectorTest.testAttachConnector001
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > org.apache.harmony.jpda.tests.jdwp.MultiSession.MethodEntryExitTest.testMethodEvent001
> > > > > > > >
> > > > > > > >
> > > org.apache.harmony.jpda.tests.jdwp.MultiSession.ResumeTest.testResume
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadEndTest.testThreadEnd001
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadStartTest.testThreadStart001
> > > > > > > >    The Linux-only timeouts are also observed in the integrity
> > > testing
> > > > > > > > results.
> > > > > > > >    JIRA HARMONY-5833 has been filed for one of them.
> > > > > > > >
> > > > > > > >  6) JettyScenario:
> > > > > > > >    The Linux x86 running failed because of the unresolved
> > > > > HARMONY-5219.
> > > > > > > >
> > > > > > > >  7) Reliability:
> > > > > > > >    Several failures are observed in windows x86_64 running.
> > > > > > > >    7.1) Old regressions
> > > > > > > >         api.net.DatagramTest (HARMONY-5531)
> > > > > > > >         api.text.DecimalFormat_Locales - (in M6 it is recorded as
> > > > > also
> > > > > > > > intermittent in M5)
> > > > > > > >
> > > > > > > >    7.2) New/intemittent regressions
> > > > > > > >         api.kernel.thread.ThreadLocalTest.ThreadLocalTest
> > > > > > > >         api.kernel.exec.RunExec
> > > > > > > >
> > > > > > > >  8) Stress
> > > > > > > >    Different test cases failed on different platforms.
> > > > > > > >    8.1) Timeouts on Linux x86.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD003.ThreadTest003
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD007.ThreadTest007
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD009.ThreadTest009
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD011.ThreadTest011
> > > > > > > >
> > > > > > > >    8.2) Failed cases on Windows x86_64 with unknown reasons.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > stress.org.apache.harmony.test.stress.classloader.MixThreads.TreeClasses.testTreeClasses2
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > stress.org.apache.harmony.test.stress.classloader.NotSynchThreads.TreeClasses.testTreeClasses
> > > > > > > >
> > > > > > > >    I have not found records for them.
> > > > > > > >
> > > > > > > >  9) Strut_test
> > > > > > > >    Broken with same error report as M6.
> > > > > > > >
> > > > > > > >  10) Eclipse Hello World Application.
> > > > > > > >    Although the testing framework just reports EUT-API status as
> > > > > > > "PASSED".
> > > > > > > > A fresh workspace running just failed in configuration stage (
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805120106o387a49f1rfb2c33d1042d2f41@mail.gmail.com%3e
> > > > > > > > )
> > > > > > > > since SVN commit 641928 (which committed patch for HARMONY-4569).
> > > > > > > >
> > > > > > > > thanks,
> > > > > > > > chunrong
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Unless stated otherwise above:
> > > IBM United Kingdom Limited - Registered in England and Wales with number
> > > 741598.
> > > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
> > >
> >
>
>
> --
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>


-- 
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

Re: [testing] first M7 candidate (r681495) testing status

Posted by Sian January <si...@googlemail.com>.
I think the code is right and the test is wrong here, because the
Javadoc for ServerCloneException says that "Invoking the method
Throwable.initCause(Throwable) on an instance of ServerCloneException
always throws IllegalStateException."

What is really confusing me is how this can be passing for anyone
else... The exclude list hasn't changed recently and I don't see how
it could be VM dependent either.


On 13/08/2008, Nathan Beyer <nd...@apache.org> wrote:
> On Tue, Aug 12, 2008 at 10:18 AM, Sian January
> <si...@googlemail.com>wrote:
>
> > I am not seeing the management failures on my local (Windows) machine,
> > but I'm seeing an rmi failure:
> >
> >
> > org.apache.harmony.rmi.server.ServerCloneExceptionTest.test_Constructor_String
> > Cause already initialized
> > java.lang.IllegalStateException: Cause already initialized
> > at java.lang.Throwable.initCause(Throwable.java:293)
> > at
> > org.apache.harmony.rmi.server.ServerCloneExceptionTest.test_Constructor_String(ServerCloneExceptionTest.java:70)
> > at java.lang.reflect.AccessibleObject.invokeV(AccessibleObject.java:25)
>
>
> This test [1] is creating an exception and then passing itself into
> 'initCause', which should throw IllegalArgumentException. Looking at the
> code though, it does look like there's a bug, as 'null' is being passed to
> the super() constructor, which will mess up Throwable, since it initializes
> 'cause' to 'this', so Throwable will think the cause is already set.
>
> Did this test recently get removed from an exclusion list?
>
>
> [1]
> http://svn.apache.org/viewvc/harmony/enhanced/classlib/trunk/modules/rmi/src/test/api/java/org/apache/harmony/rmi/server/ServerCloneExceptionTest.java?view=annotate
>
> >
> >
> > and an intermittent failure in java.net:
> >
> >
> > org.apache.harmony.luni.tests.java.net.URLConnectionTest.test_getInputStream
> >
> > Error receiving content coded data:
> > junit.framework.AssertionFailedError: Error receiving content coded data:
> > at tests.support.Support_HttpTests.runTests(Support_HttpTests.java:15)
> > at
> > org.apache.harmony.luni.tests.java.net.URLConnectionTest.test_getInputStream(URLConnectionTest.java:632)
> > at java.lang.reflect.AccessibleObject.invokeV(AccessibleObject.java:25)
> >
> > Is anyone else seeing these?
> >
> > Sian
> >
> > On 12/08/2008, chunrong lai <ch...@gmail.com> wrote:
> > >   Thanks.
> > >   I think the standard scripts produce that site although I always added
> > > publish/e-mail notification setup before install/run the test suites.
> > >   It is understandable that some tests in classlib-test and drlvm-test
> > get
> > > failing report. We just need to understand the failures.
> > >   As reported in M6,
> > > org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest,
> > >
> > org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
> > > etc fail classlib-test.
> > >   Also we observed that java.lang.ClassGenericsTest.test_2 sometimes fail
> > > drlvm-test.
> > >   Another timeout issue of drlvm-test comes from
> > > thread.SmallStackThreadTest_jit, as discussed in HARMONY-4601, the test
> > case
> > > may run very slow when system resource is tight and finally introduce a
> > > timeout. Re-running the suite generally fixes the timeout.
> > >
> > >
> > >
> > >
> > >
> > > On 8/12/08, Nathan Beyer <nd...@apache.org> wrote:
> > > >
> > > > It's the tests that are failing - both classlib-test and drlvm-test.
> > > >
> > > > What's the specific integrity setup that's being used for that site?
> > Does
> > > > the standard out-of-the-box buildtest scripts produce that site?
> > > >
> > > > -Nathan
> > > >
> > > > On Mon, Aug 11, 2008 at 9:05 PM, chunrong lai <ch...@gmail.com>
> > > > wrote:
> > > >
> > > > > hi, Nathan:
> > > > >     Steps to run testing cycle are listed in
> > > > > http://harmony.apache.org/subcomponents/buildtest/index.html. One
> > > > > integrity
> > > > > testing example is
> > > > >
> > http://harmony.apache.org/subcomponents/buildtest/howto.html#Extended.
> > > > >      What is the error message you met (in what step)? What is the
> > > > content
> > > > > of the framework.local.properties file? I think that will help to
> > figure
> > > > > out
> > > > > the problem.
> > > > >
> > > > >
> > > > > On 8/12/08, Nathan Beyer <nd...@apache.org> wrote:
> > > > > >
> > > > > > What's being used to produce the integrity tests? I have an x86_64
> > > > Linux
> > > > > > box
> > > > > > that I can dedicate to testing. I've tried just using buildtest
> > with
> > > > > > 'classlib,drlvm,classib-test,drlvm-test', but I haven't gotten a
> > clean
> > > > > pass
> > > > > > yet.
> > > > > >
> > > > > > -Nathan
> > > > > >
> > > > > > On Mon, Aug 11, 2008 at 12:26 PM, chunrong lai <
> > chunronglai@gmail.com
> > > > > > >wrote:
> > > > > >
> > > > > > > Hi, all:
> > > > > > >
> > > > > > >  Here is r681495 snapshot testing status:
> > > > > > > http://people.apache.org/~chunrong/snapshots/r681495/index.html<http://people.apache.org/%7Echunrong/snapshots/r681495/index.html>
> > <
> > > > http://people.apache.org/%7Echunrong/snapshots/r681495/index.html>
> > > > > <
> > > > > > http://people.apache.org/%7Echunrong/snapshots/r681495/index.html
> > >.
> > > > > > > I am using
> > > > > > > two platforms (Linux x86, windows x86_64) at the moment.
> > Hopefully we
> > > > > > will
> > > > > > > have other two platforms in future for M8. Well, although we are
> > > > > testing
> > > > > > > only two platforms for M7, my experience is that the status for
> > > > another
> > > > > > two
> > > > > > > platforms should be not worse or just include some extra
> > intermittent
> > > > > > > errors
> > > > > > > which can be investigated in some later stages.
> > > > > > >
> > > > > > >  The following suites passed on Linux x86/Windows x86_64
> > platforms:
> > > > Ant
> > > > > > > Scenario (or self-hosting), Axis application, Dacapo, DRLVM
> > > > regression
> > > > > > > tests, Geronimo Unit Tests, Scimark, Tomcat scenario, VTS VM Test
> > > > > Suite.
> > > > > > >
> > > > > > >  Most of the failures are known issues (for M6). Although we can
> > > > > observe
> > > > > > > 15~20 new issues, those issues happen only in 1 platform and they
> > > > look
> > > > > > more
> > > > > > > like the intermittent/timeout issues (less reproducible) to me.
> > > > Overall
> > > > > I
> > > > > > > myself would like to think r681495 is more stable than M6.
> > > > > > >
> > > > > > >  Please add your comments and clarifications (please also see M6
> > > > > testing
> > > > > > > results http://people.apache.org/~smishura/r653525/<http://people.apache.org/%7Esmishura/r653525/>
> > <
> > > > http://people.apache.org/%7Esmishura/r653525/>
> > > > > <
> > > > > > http://people.apache.org/%7Esmishura/r653525/>
> > > > > > > ,
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805070304l38845ee0se01fb93fbfc05586@mail.gmail.com%3eand
> > > > > > > integrity testing results
> > > > > > > http://people.apache.org/~chunrong/harmony-integrity/<http://people.apache.org/%7Echunrong/harmony-integrity/>
> > <
> > > > http://people.apache.org/%7Echunrong/harmony-integrity/>
> > > > > <
> > > > > > http://people.apache.org/%7Echunrong/harmony-integrity/>as a
> > > > > comparison).
> > > > > > >
> > > > > > >  1) Classlib:
> > > > > > >    1.1) Since r644719 (which committed patch for HARMONY-5684)
> > > > > > >
> > > > > > >
> >  org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
> > > > > > >
> > > > > > >         failed in both platforms
> > > > > > >
> > > > > > >    1.2) Two failures
> > > > > > >
> > > > > > >        org.apache.harmony.luni.tests.java.net.MulticastSocketTest
> > > > > (Failed
> > > > > > > in windows_x86 running of M6)
> > > > > > >        tests.api.java.security.PermissionCollectionTest
> > > > > (Failed
> > > > > > > in M6)
> > > > > > >
> > > > > > >         are observed in Linux x86.
> > > > > > >
> > > > > > >  2) DRLVM tests:
> > > > > > >    2.1) One failure
> > > > > > >
> > > > > > >         java.lang.ClassGenericsTest.test_2
> > > > > > >
> > > > > > >         is observed in Linux x86 snapshot testing.
> > > > > > >         I can see some old discussion in the mailing list about
> > that
> > > > > but
> > > > > > I
> > > > > > > am not sure the expected status here.
> > > > > > >         They should be intemittent errors since the integrity
> > testing
> > > > > > just
> > > > > > > run well mostly.
> > > > > > >
> > > > > > >  3) EUTs:
> > > > > > >
> > > > > > >    3.1) Linux x86: 99.36%
> > > > > > >         A recorded JIRA for this suite is HARMONY-2914 which
> > wastes
> > > > > file
> > > > > > > handlers and makes system unstable.
> > > > > > >
> > > > > > >  4) Functional:
> > > > > > >    4.1) Old regressions on both platforms:
> > > > > > >         api.java.text.MessageFormat (HARMONY-5430)
> > > > > > >         api.java.util.jar.Manifest  (HARMONY-5473)
> > > > > > >         api.java.beans.beancontext.BeanContextTest (also in M6,
> > > > > recorded
> > > > > > > as  regression caused by changes in locale data)
> > > > > > >         api.java.beans.persistence.EncoderTest  (also in M6,
> > recorded
> > > > > as
> > > > > > > regression in the beans module)
> > > > > > >         api.java.beans.persistence.EncoderDecoderTest (also in
> > M6,
> > > > > > > regression in the beans module)
> > > > > > >         reg.vm.btest5625 (also in M6, recorded as intermittent
> > and
> > > > not
> > > > > > > reproducible manually)
> > > > > > >
> > > > > > >    4.2) Old regressions on 1 platform
> > > > > > >         api.java.rmi.basicexception (ERROR in Linux x86,
> > > > HARMONY-5823)
> > > > > > >         api.java.rmi.basicregistry.RemoteServerTest (ERROR in
> > Linux
> > > > > x86,
> > > > > > > HARMONY-5823)
> > > > > > >         jpda.jdwp.scenario.ST07.ST07Test (ERROR in windows
> > x86_64, in
> > > > > M6
> > > > > > it
> > > > > > > is recorded as regression since M4)
> > > > > > >         java.math.F_BigIntegerMatrixMultiplyTest_01 (ERROR on
> > Linux
> > > > > x86,
> > > > > > > recorded as Timeout, not reproducible
> > > > > > >  in M6)
> > > > > > >         reg.vm.btest5717 (ERROR on Windows X86_64, recorded as
> > > > > "timeout,
> > > > > > > the test is too heavy" in M6)
> > > > > > >         jit.HLO.inline.ControlFlow.IfElse.IfElse1.IfElseTest1
> > (FAILED
> > > > > in
> > > > > > > windows x86_64, recorded as "looks like
> > > > > > > issue in test" in M6)
> > > > > > >         jit.HLO.devirt.Runtime.RuntimeExtend1 (FAILED on windows
> > > > > x86_64,
> > > > > > in
> > > > > > > M6 it is recorded as not regression and started to fail on M5)
> > > > > > >         reg.vm.btest6353.Btest6353 (ERROR on Windows x86_64,
> > recorded
> > > > > > also
> > > > > > > failed on M3 and might be issue with the test)
> > > > > > >
> > > > > > >    4.3) New regressions on 1 platform (looks intermittent)
> > > > > > >         reg.jit.btest8029.Btest8029 (FAILED in Linux x86)
> > > > > > >         func.reg.jit.btest5710.Btest5710 (FAILED in Linux x86)
> > > > > > >
> > api.java.security.cert.F_CertPathTest_06.F_CertPathTest_06
> > > > > (ERROR
> > > > > > > in Linux x86)
> > > > > > >
> > api.java.security.cert.F_CertPathTest_05.F_CertPathTest_05
> > > > > (ERROR
> > > > > > > in Linux x86)
> > > > > > >
> > > > > > >  5) JDKTools Tests:
> > > > > > >    Several timeouts are observed in Linux x86 snapshot running.
> > They
> > > > > are:
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch001
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch002
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch003
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch004
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > org.apache.harmony.jpda.tests.jdwp.Events.CombinedEventsTest.testCombinedEvents_04
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > org.apache.harmony.jpda.tests.jdwp.MultiSession.AttachConnectorTest.testAttachConnector001
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > org.apache.harmony.jpda.tests.jdwp.MultiSession.MethodEntryExitTest.testMethodEvent001
> > > > > > >
> > > > > > >
> > org.apache.harmony.jpda.tests.jdwp.MultiSession.ResumeTest.testResume
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadEndTest.testThreadEnd001
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadStartTest.testThreadStart001
> > > > > > >    The Linux-only timeouts are also observed in the integrity
> > testing
> > > > > > > results.
> > > > > > >    JIRA HARMONY-5833 has been filed for one of them.
> > > > > > >
> > > > > > >  6) JettyScenario:
> > > > > > >    The Linux x86 running failed because of the unresolved
> > > > HARMONY-5219.
> > > > > > >
> > > > > > >  7) Reliability:
> > > > > > >    Several failures are observed in windows x86_64 running.
> > > > > > >    7.1) Old regressions
> > > > > > >         api.net.DatagramTest (HARMONY-5531)
> > > > > > >         api.text.DecimalFormat_Locales - (in M6 it is recorded as
> > > > also
> > > > > > > intermittent in M5)
> > > > > > >
> > > > > > >    7.2) New/intemittent regressions
> > > > > > >         api.kernel.thread.ThreadLocalTest.ThreadLocalTest
> > > > > > >         api.kernel.exec.RunExec
> > > > > > >
> > > > > > >  8) Stress
> > > > > > >    Different test cases failed on different platforms.
> > > > > > >    8.1) Timeouts on Linux x86.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD003.ThreadTest003
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD007.ThreadTest007
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD009.ThreadTest009
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD011.ThreadTest011
> > > > > > >
> > > > > > >    8.2) Failed cases on Windows x86_64 with unknown reasons.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > stress.org.apache.harmony.test.stress.classloader.MixThreads.TreeClasses.testTreeClasses2
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > stress.org.apache.harmony.test.stress.classloader.NotSynchThreads.TreeClasses.testTreeClasses
> > > > > > >
> > > > > > >    I have not found records for them.
> > > > > > >
> > > > > > >  9) Strut_test
> > > > > > >    Broken with same error report as M6.
> > > > > > >
> > > > > > >  10) Eclipse Hello World Application.
> > > > > > >    Although the testing framework just reports EUT-API status as
> > > > > > "PASSED".
> > > > > > > A fresh workspace running just failed in configuration stage (
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805120106o387a49f1rfb2c33d1042d2f41@mail.gmail.com%3e
> > > > > > > )
> > > > > > > since SVN commit 641928 (which committed patch for HARMONY-4569).
> > > > > > >
> > > > > > > thanks,
> > > > > > > chunrong
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> > Unless stated otherwise above:
> > IBM United Kingdom Limited - Registered in England and Wales with number
> > 741598.
> > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
> >
>


-- 
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

Re: [testing] first M7 candidate (r681495) testing status

Posted by Nathan Beyer <nd...@apache.org>.
On Tue, Aug 12, 2008 at 10:18 AM, Sian January
<si...@googlemail.com>wrote:

> I am not seeing the management failures on my local (Windows) machine,
> but I'm seeing an rmi failure:
>
>
> org.apache.harmony.rmi.server.ServerCloneExceptionTest.test_Constructor_String
> Cause already initialized
> java.lang.IllegalStateException: Cause already initialized
> at java.lang.Throwable.initCause(Throwable.java:293)
> at
> org.apache.harmony.rmi.server.ServerCloneExceptionTest.test_Constructor_String(ServerCloneExceptionTest.java:70)
> at java.lang.reflect.AccessibleObject.invokeV(AccessibleObject.java:25)


This test [1] is creating an exception and then passing itself into
'initCause', which should throw IllegalArgumentException. Looking at the
code though, it does look like there's a bug, as 'null' is being passed to
the super() constructor, which will mess up Throwable, since it initializes
'cause' to 'this', so Throwable will think the cause is already set.

Did this test recently get removed from an exclusion list?


[1]
http://svn.apache.org/viewvc/harmony/enhanced/classlib/trunk/modules/rmi/src/test/api/java/org/apache/harmony/rmi/server/ServerCloneExceptionTest.java?view=annotate

>
>
> and an intermittent failure in java.net:
>
>
> org.apache.harmony.luni.tests.java.net.URLConnectionTest.test_getInputStream
>
> Error receiving content coded data:
> junit.framework.AssertionFailedError: Error receiving content coded data:
> at tests.support.Support_HttpTests.runTests(Support_HttpTests.java:15)
> at
> org.apache.harmony.luni.tests.java.net.URLConnectionTest.test_getInputStream(URLConnectionTest.java:632)
> at java.lang.reflect.AccessibleObject.invokeV(AccessibleObject.java:25)
>
> Is anyone else seeing these?
>
> Sian
>
> On 12/08/2008, chunrong lai <ch...@gmail.com> wrote:
> >   Thanks.
> >   I think the standard scripts produce that site although I always added
> > publish/e-mail notification setup before install/run the test suites.
> >   It is understandable that some tests in classlib-test and drlvm-test
> get
> > failing report. We just need to understand the failures.
> >   As reported in M6,
> > org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest,
> >
> org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
> > etc fail classlib-test.
> >   Also we observed that java.lang.ClassGenericsTest.test_2 sometimes fail
> > drlvm-test.
> >   Another timeout issue of drlvm-test comes from
> > thread.SmallStackThreadTest_jit, as discussed in HARMONY-4601, the test
> case
> > may run very slow when system resource is tight and finally introduce a
> > timeout. Re-running the suite generally fixes the timeout.
> >
> >
> >
> >
> >
> > On 8/12/08, Nathan Beyer <nd...@apache.org> wrote:
> > >
> > > It's the tests that are failing - both classlib-test and drlvm-test.
> > >
> > > What's the specific integrity setup that's being used for that site?
> Does
> > > the standard out-of-the-box buildtest scripts produce that site?
> > >
> > > -Nathan
> > >
> > > On Mon, Aug 11, 2008 at 9:05 PM, chunrong lai <ch...@gmail.com>
> > > wrote:
> > >
> > > > hi, Nathan:
> > > >     Steps to run testing cycle are listed in
> > > > http://harmony.apache.org/subcomponents/buildtest/index.html. One
> > > > integrity
> > > > testing example is
> > > >
> http://harmony.apache.org/subcomponents/buildtest/howto.html#Extended.
> > > >      What is the error message you met (in what step)? What is the
> > > content
> > > > of the framework.local.properties file? I think that will help to
> figure
> > > > out
> > > > the problem.
> > > >
> > > >
> > > > On 8/12/08, Nathan Beyer <nd...@apache.org> wrote:
> > > > >
> > > > > What's being used to produce the integrity tests? I have an x86_64
> > > Linux
> > > > > box
> > > > > that I can dedicate to testing. I've tried just using buildtest
> with
> > > > > 'classlib,drlvm,classib-test,drlvm-test', but I haven't gotten a
> clean
> > > > pass
> > > > > yet.
> > > > >
> > > > > -Nathan
> > > > >
> > > > > On Mon, Aug 11, 2008 at 12:26 PM, chunrong lai <
> chunronglai@gmail.com
> > > > > >wrote:
> > > > >
> > > > > > Hi, all:
> > > > > >
> > > > > >  Here is r681495 snapshot testing status:
> > > > > > http://people.apache.org/~chunrong/snapshots/r681495/index.html<http://people.apache.org/%7Echunrong/snapshots/r681495/index.html>
> <
> > > http://people.apache.org/%7Echunrong/snapshots/r681495/index.html>
> > > > <
> > > > > http://people.apache.org/%7Echunrong/snapshots/r681495/index.html
> >.
> > > > > > I am using
> > > > > > two platforms (Linux x86, windows x86_64) at the moment.
> Hopefully we
> > > > > will
> > > > > > have other two platforms in future for M8. Well, although we are
> > > > testing
> > > > > > only two platforms for M7, my experience is that the status for
> > > another
> > > > > two
> > > > > > platforms should be not worse or just include some extra
> intermittent
> > > > > > errors
> > > > > > which can be investigated in some later stages.
> > > > > >
> > > > > >  The following suites passed on Linux x86/Windows x86_64
> platforms:
> > > Ant
> > > > > > Scenario (or self-hosting), Axis application, Dacapo, DRLVM
> > > regression
> > > > > > tests, Geronimo Unit Tests, Scimark, Tomcat scenario, VTS VM Test
> > > > Suite.
> > > > > >
> > > > > >  Most of the failures are known issues (for M6). Although we can
> > > > observe
> > > > > > 15~20 new issues, those issues happen only in 1 platform and they
> > > look
> > > > > more
> > > > > > like the intermittent/timeout issues (less reproducible) to me.
> > > Overall
> > > > I
> > > > > > myself would like to think r681495 is more stable than M6.
> > > > > >
> > > > > >  Please add your comments and clarifications (please also see M6
> > > > testing
> > > > > > results http://people.apache.org/~smishura/r653525/<http://people.apache.org/%7Esmishura/r653525/>
> <
> > > http://people.apache.org/%7Esmishura/r653525/>
> > > > <
> > > > > http://people.apache.org/%7Esmishura/r653525/>
> > > > > > ,
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805070304l38845ee0se01fb93fbfc05586@mail.gmail.com%3eand
> > > > > > integrity testing results
> > > > > > http://people.apache.org/~chunrong/harmony-integrity/<http://people.apache.org/%7Echunrong/harmony-integrity/>
> <
> > > http://people.apache.org/%7Echunrong/harmony-integrity/>
> > > > <
> > > > > http://people.apache.org/%7Echunrong/harmony-integrity/>as a
> > > > comparison).
> > > > > >
> > > > > >  1) Classlib:
> > > > > >    1.1) Since r644719 (which committed patch for HARMONY-5684)
> > > > > >
> > > > > >
>  org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
> > > > > >
> > > > > >         failed in both platforms
> > > > > >
> > > > > >    1.2) Two failures
> > > > > >
> > > > > >        org.apache.harmony.luni.tests.java.net.MulticastSocketTest
> > > > (Failed
> > > > > > in windows_x86 running of M6)
> > > > > >        tests.api.java.security.PermissionCollectionTest
> > > > (Failed
> > > > > > in M6)
> > > > > >
> > > > > >         are observed in Linux x86.
> > > > > >
> > > > > >  2) DRLVM tests:
> > > > > >    2.1) One failure
> > > > > >
> > > > > >         java.lang.ClassGenericsTest.test_2
> > > > > >
> > > > > >         is observed in Linux x86 snapshot testing.
> > > > > >         I can see some old discussion in the mailing list about
> that
> > > > but
> > > > > I
> > > > > > am not sure the expected status here.
> > > > > >         They should be intemittent errors since the integrity
> testing
> > > > > just
> > > > > > run well mostly.
> > > > > >
> > > > > >  3) EUTs:
> > > > > >
> > > > > >    3.1) Linux x86: 99.36%
> > > > > >         A recorded JIRA for this suite is HARMONY-2914 which
> wastes
> > > > file
> > > > > > handlers and makes system unstable.
> > > > > >
> > > > > >  4) Functional:
> > > > > >    4.1) Old regressions on both platforms:
> > > > > >         api.java.text.MessageFormat (HARMONY-5430)
> > > > > >         api.java.util.jar.Manifest  (HARMONY-5473)
> > > > > >         api.java.beans.beancontext.BeanContextTest (also in M6,
> > > > recorded
> > > > > > as  regression caused by changes in locale data)
> > > > > >         api.java.beans.persistence.EncoderTest  (also in M6,
> recorded
> > > > as
> > > > > > regression in the beans module)
> > > > > >         api.java.beans.persistence.EncoderDecoderTest (also in
> M6,
> > > > > > regression in the beans module)
> > > > > >         reg.vm.btest5625 (also in M6, recorded as intermittent
> and
> > > not
> > > > > > reproducible manually)
> > > > > >
> > > > > >    4.2) Old regressions on 1 platform
> > > > > >         api.java.rmi.basicexception (ERROR in Linux x86,
> > > HARMONY-5823)
> > > > > >         api.java.rmi.basicregistry.RemoteServerTest (ERROR in
> Linux
> > > > x86,
> > > > > > HARMONY-5823)
> > > > > >         jpda.jdwp.scenario.ST07.ST07Test (ERROR in windows
> x86_64, in
> > > > M6
> > > > > it
> > > > > > is recorded as regression since M4)
> > > > > >         java.math.F_BigIntegerMatrixMultiplyTest_01 (ERROR on
> Linux
> > > > x86,
> > > > > > recorded as Timeout, not reproducible
> > > > > >  in M6)
> > > > > >         reg.vm.btest5717 (ERROR on Windows X86_64, recorded as
> > > > "timeout,
> > > > > > the test is too heavy" in M6)
> > > > > >         jit.HLO.inline.ControlFlow.IfElse.IfElse1.IfElseTest1
> (FAILED
> > > > in
> > > > > > windows x86_64, recorded as "looks like
> > > > > > issue in test" in M6)
> > > > > >         jit.HLO.devirt.Runtime.RuntimeExtend1 (FAILED on windows
> > > > x86_64,
> > > > > in
> > > > > > M6 it is recorded as not regression and started to fail on M5)
> > > > > >         reg.vm.btest6353.Btest6353 (ERROR on Windows x86_64,
> recorded
> > > > > also
> > > > > > failed on M3 and might be issue with the test)
> > > > > >
> > > > > >    4.3) New regressions on 1 platform (looks intermittent)
> > > > > >         reg.jit.btest8029.Btest8029 (FAILED in Linux x86)
> > > > > >         func.reg.jit.btest5710.Btest5710 (FAILED in Linux x86)
> > > > > >
> api.java.security.cert.F_CertPathTest_06.F_CertPathTest_06
> > > > (ERROR
> > > > > > in Linux x86)
> > > > > >
> api.java.security.cert.F_CertPathTest_05.F_CertPathTest_05
> > > > (ERROR
> > > > > > in Linux x86)
> > > > > >
> > > > > >  5) JDKTools Tests:
> > > > > >    Several timeouts are observed in Linux x86 snapshot running.
> They
> > > > are:
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch001
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch002
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch003
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch004
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> org.apache.harmony.jpda.tests.jdwp.Events.CombinedEventsTest.testCombinedEvents_04
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.AttachConnectorTest.testAttachConnector001
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.MethodEntryExitTest.testMethodEvent001
> > > > > >
> > > > > >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ResumeTest.testResume
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadEndTest.testThreadEnd001
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadStartTest.testThreadStart001
> > > > > >    The Linux-only timeouts are also observed in the integrity
> testing
> > > > > > results.
> > > > > >    JIRA HARMONY-5833 has been filed for one of them.
> > > > > >
> > > > > >  6) JettyScenario:
> > > > > >    The Linux x86 running failed because of the unresolved
> > > HARMONY-5219.
> > > > > >
> > > > > >  7) Reliability:
> > > > > >    Several failures are observed in windows x86_64 running.
> > > > > >    7.1) Old regressions
> > > > > >         api.net.DatagramTest (HARMONY-5531)
> > > > > >         api.text.DecimalFormat_Locales - (in M6 it is recorded as
> > > also
> > > > > > intermittent in M5)
> > > > > >
> > > > > >    7.2) New/intemittent regressions
> > > > > >         api.kernel.thread.ThreadLocalTest.ThreadLocalTest
> > > > > >         api.kernel.exec.RunExec
> > > > > >
> > > > > >  8) Stress
> > > > > >    Different test cases failed on different platforms.
> > > > > >    8.1) Timeouts on Linux x86.
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD003.ThreadTest003
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD007.ThreadTest007
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD009.ThreadTest009
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD011.ThreadTest011
> > > > > >
> > > > > >    8.2) Failed cases on Windows x86_64 with unknown reasons.
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> stress.org.apache.harmony.test.stress.classloader.MixThreads.TreeClasses.testTreeClasses2
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> stress.org.apache.harmony.test.stress.classloader.NotSynchThreads.TreeClasses.testTreeClasses
> > > > > >
> > > > > >    I have not found records for them.
> > > > > >
> > > > > >  9) Strut_test
> > > > > >    Broken with same error report as M6.
> > > > > >
> > > > > >  10) Eclipse Hello World Application.
> > > > > >    Although the testing framework just reports EUT-API status as
> > > > > "PASSED".
> > > > > > A fresh workspace running just failed in configuration stage (
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805120106o387a49f1rfb2c33d1042d2f41@mail.gmail.com%3e
> > > > > > )
> > > > > > since SVN commit 641928 (which committed patch for HARMONY-4569).
> > > > > >
> > > > > > thanks,
> > > > > > chunrong
> > > > > >
> > > > >
> > > >
> > >
> >
>
>
>
> --
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>

Re: [testing] first M7 candidate (r681495) testing status

Posted by Sian January <si...@googlemail.com>.
I am not seeing the management failures on my local (Windows) machine,
but I'm seeing an rmi failure:

org.apache.harmony.rmi.server.ServerCloneExceptionTest.test_Constructor_String
Cause already initialized
java.lang.IllegalStateException: Cause already initialized
at java.lang.Throwable.initCause(Throwable.java:293)
at org.apache.harmony.rmi.server.ServerCloneExceptionTest.test_Constructor_String(ServerCloneExceptionTest.java:70)
at java.lang.reflect.AccessibleObject.invokeV(AccessibleObject.java:25)

and an intermittent failure in java.net:

org.apache.harmony.luni.tests.java.net.URLConnectionTest.test_getInputStream

Error receiving content coded data:
junit.framework.AssertionFailedError: Error receiving content coded data:
at tests.support.Support_HttpTests.runTests(Support_HttpTests.java:15)
at org.apache.harmony.luni.tests.java.net.URLConnectionTest.test_getInputStream(URLConnectionTest.java:632)
at java.lang.reflect.AccessibleObject.invokeV(AccessibleObject.java:25)

Is anyone else seeing these?

Sian

On 12/08/2008, chunrong lai <ch...@gmail.com> wrote:
>   Thanks.
>   I think the standard scripts produce that site although I always added
> publish/e-mail notification setup before install/run the test suites.
>   It is understandable that some tests in classlib-test and drlvm-test get
> failing report. We just need to understand the failures.
>   As reported in M6,
> org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest,
> org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
> etc fail classlib-test.
>   Also we observed that java.lang.ClassGenericsTest.test_2 sometimes fail
> drlvm-test.
>   Another timeout issue of drlvm-test comes from
> thread.SmallStackThreadTest_jit, as discussed in HARMONY-4601, the test case
> may run very slow when system resource is tight and finally introduce a
> timeout. Re-running the suite generally fixes the timeout.
>
>
>
>
>
> On 8/12/08, Nathan Beyer <nd...@apache.org> wrote:
> >
> > It's the tests that are failing - both classlib-test and drlvm-test.
> >
> > What's the specific integrity setup that's being used for that site? Does
> > the standard out-of-the-box buildtest scripts produce that site?
> >
> > -Nathan
> >
> > On Mon, Aug 11, 2008 at 9:05 PM, chunrong lai <ch...@gmail.com>
> > wrote:
> >
> > > hi, Nathan:
> > >     Steps to run testing cycle are listed in
> > > http://harmony.apache.org/subcomponents/buildtest/index.html. One
> > > integrity
> > > testing example is
> > > http://harmony.apache.org/subcomponents/buildtest/howto.html#Extended.
> > >      What is the error message you met (in what step)? What is the
> > content
> > > of the framework.local.properties file? I think that will help to figure
> > > out
> > > the problem.
> > >
> > >
> > > On 8/12/08, Nathan Beyer <nd...@apache.org> wrote:
> > > >
> > > > What's being used to produce the integrity tests? I have an x86_64
> > Linux
> > > > box
> > > > that I can dedicate to testing. I've tried just using buildtest with
> > > > 'classlib,drlvm,classib-test,drlvm-test', but I haven't gotten a clean
> > > pass
> > > > yet.
> > > >
> > > > -Nathan
> > > >
> > > > On Mon, Aug 11, 2008 at 12:26 PM, chunrong lai <chunronglai@gmail.com
> > > > >wrote:
> > > >
> > > > > Hi, all:
> > > > >
> > > > >  Here is r681495 snapshot testing status:
> > > > > http://people.apache.org/~chunrong/snapshots/r681495/index.html<
> > http://people.apache.org/%7Echunrong/snapshots/r681495/index.html>
> > > <
> > > > http://people.apache.org/%7Echunrong/snapshots/r681495/index.html>.
> > > > > I am using
> > > > > two platforms (Linux x86, windows x86_64) at the moment. Hopefully we
> > > > will
> > > > > have other two platforms in future for M8. Well, although we are
> > > testing
> > > > > only two platforms for M7, my experience is that the status for
> > another
> > > > two
> > > > > platforms should be not worse or just include some extra intermittent
> > > > > errors
> > > > > which can be investigated in some later stages.
> > > > >
> > > > >  The following suites passed on Linux x86/Windows x86_64 platforms:
> > Ant
> > > > > Scenario (or self-hosting), Axis application, Dacapo, DRLVM
> > regression
> > > > > tests, Geronimo Unit Tests, Scimark, Tomcat scenario, VTS VM Test
> > > Suite.
> > > > >
> > > > >  Most of the failures are known issues (for M6). Although we can
> > > observe
> > > > > 15~20 new issues, those issues happen only in 1 platform and they
> > look
> > > > more
> > > > > like the intermittent/timeout issues (less reproducible) to me.
> > Overall
> > > I
> > > > > myself would like to think r681495 is more stable than M6.
> > > > >
> > > > >  Please add your comments and clarifications (please also see M6
> > > testing
> > > > > results http://people.apache.org/~smishura/r653525/<
> > http://people.apache.org/%7Esmishura/r653525/>
> > > <
> > > > http://people.apache.org/%7Esmishura/r653525/>
> > > > > ,
> > > > >
> > > > >
> > > >
> > >
> > http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805070304l38845ee0se01fb93fbfc05586@mail.gmail.com%3eand
> > > > > integrity testing results
> > > > > http://people.apache.org/~chunrong/harmony-integrity/<
> > http://people.apache.org/%7Echunrong/harmony-integrity/>
> > > <
> > > > http://people.apache.org/%7Echunrong/harmony-integrity/>as a
> > > comparison).
> > > > >
> > > > >  1) Classlib:
> > > > >    1.1) Since r644719 (which committed patch for HARMONY-5684)
> > > > >
> > > > >        org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest
> > > > >
> > > > >
> > > > >
> > > >
> > >
> > org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
> > > > >
> > > > >         failed in both platforms
> > > > >
> > > > >    1.2) Two failures
> > > > >
> > > > >        org.apache.harmony.luni.tests.java.net.MulticastSocketTest
> > > (Failed
> > > > > in windows_x86 running of M6)
> > > > >        tests.api.java.security.PermissionCollectionTest
> > > (Failed
> > > > > in M6)
> > > > >
> > > > >         are observed in Linux x86.
> > > > >
> > > > >  2) DRLVM tests:
> > > > >    2.1) One failure
> > > > >
> > > > >         java.lang.ClassGenericsTest.test_2
> > > > >
> > > > >         is observed in Linux x86 snapshot testing.
> > > > >         I can see some old discussion in the mailing list about that
> > > but
> > > > I
> > > > > am not sure the expected status here.
> > > > >         They should be intemittent errors since the integrity testing
> > > > just
> > > > > run well mostly.
> > > > >
> > > > >  3) EUTs:
> > > > >
> > > > >    3.1) Linux x86: 99.36%
> > > > >         A recorded JIRA for this suite is HARMONY-2914 which wastes
> > > file
> > > > > handlers and makes system unstable.
> > > > >
> > > > >  4) Functional:
> > > > >    4.1) Old regressions on both platforms:
> > > > >         api.java.text.MessageFormat (HARMONY-5430)
> > > > >         api.java.util.jar.Manifest  (HARMONY-5473)
> > > > >         api.java.beans.beancontext.BeanContextTest (also in M6,
> > > recorded
> > > > > as  regression caused by changes in locale data)
> > > > >         api.java.beans.persistence.EncoderTest  (also in M6, recorded
> > > as
> > > > > regression in the beans module)
> > > > >         api.java.beans.persistence.EncoderDecoderTest (also in M6,
> > > > > regression in the beans module)
> > > > >         reg.vm.btest5625 (also in M6, recorded as intermittent and
> > not
> > > > > reproducible manually)
> > > > >
> > > > >    4.2) Old regressions on 1 platform
> > > > >         api.java.rmi.basicexception (ERROR in Linux x86,
> > HARMONY-5823)
> > > > >         api.java.rmi.basicregistry.RemoteServerTest (ERROR in Linux
> > > x86,
> > > > > HARMONY-5823)
> > > > >         jpda.jdwp.scenario.ST07.ST07Test (ERROR in windows x86_64, in
> > > M6
> > > > it
> > > > > is recorded as regression since M4)
> > > > >         java.math.F_BigIntegerMatrixMultiplyTest_01 (ERROR on Linux
> > > x86,
> > > > > recorded as Timeout, not reproducible
> > > > >  in M6)
> > > > >         reg.vm.btest5717 (ERROR on Windows X86_64, recorded as
> > > "timeout,
> > > > > the test is too heavy" in M6)
> > > > >         jit.HLO.inline.ControlFlow.IfElse.IfElse1.IfElseTest1 (FAILED
> > > in
> > > > > windows x86_64, recorded as "looks like
> > > > > issue in test" in M6)
> > > > >         jit.HLO.devirt.Runtime.RuntimeExtend1 (FAILED on windows
> > > x86_64,
> > > > in
> > > > > M6 it is recorded as not regression and started to fail on M5)
> > > > >         reg.vm.btest6353.Btest6353 (ERROR on Windows x86_64, recorded
> > > > also
> > > > > failed on M3 and might be issue with the test)
> > > > >
> > > > >    4.3) New regressions on 1 platform (looks intermittent)
> > > > >         reg.jit.btest8029.Btest8029 (FAILED in Linux x86)
> > > > >         func.reg.jit.btest5710.Btest5710 (FAILED in Linux x86)
> > > > >         api.java.security.cert.F_CertPathTest_06.F_CertPathTest_06
> > > (ERROR
> > > > > in Linux x86)
> > > > >         api.java.security.cert.F_CertPathTest_05.F_CertPathTest_05
> > > (ERROR
> > > > > in Linux x86)
> > > > >
> > > > >  5) JDKTools Tests:
> > > > >    Several timeouts are observed in Linux x86 snapshot running. They
> > > are:
> > > > >
> > > > >
> > > > >
> > > >
> > >
> > org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch001
> > > > >
> > > > >
> > > > >
> > > >
> > >
> > org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch002
> > > > >
> > > > >
> > > > >
> > > >
> > >
> > org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch003
> > > > >
> > > > >
> > > > >
> > > >
> > >
> > org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch004
> > > > >
> > > > >
> > > > >
> > > >
> > >
> > org.apache.harmony.jpda.tests.jdwp.Events.CombinedEventsTest.testCombinedEvents_04
> > > > >
> > > > >
> > > > >
> > > >
> > >
> > org.apache.harmony.jpda.tests.jdwp.MultiSession.AttachConnectorTest.testAttachConnector001
> > > > >
> > > > >
> > > > >
> > > >
> > >
> > org.apache.harmony.jpda.tests.jdwp.MultiSession.MethodEntryExitTest.testMethodEvent001
> > > > >
> > > > > org.apache.harmony.jpda.tests.jdwp.MultiSession.ResumeTest.testResume
> > > > >
> > > > >
> > > > >
> > > >
> > >
> > org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadEndTest.testThreadEnd001
> > > > >
> > > > >
> > > > >
> > > >
> > >
> > org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadStartTest.testThreadStart001
> > > > >    The Linux-only timeouts are also observed in the integrity testing
> > > > > results.
> > > > >    JIRA HARMONY-5833 has been filed for one of them.
> > > > >
> > > > >  6) JettyScenario:
> > > > >    The Linux x86 running failed because of the unresolved
> > HARMONY-5219.
> > > > >
> > > > >  7) Reliability:
> > > > >    Several failures are observed in windows x86_64 running.
> > > > >    7.1) Old regressions
> > > > >         api.net.DatagramTest (HARMONY-5531)
> > > > >         api.text.DecimalFormat_Locales - (in M6 it is recorded as
> > also
> > > > > intermittent in M5)
> > > > >
> > > > >    7.2) New/intemittent regressions
> > > > >         api.kernel.thread.ThreadLocalTest.ThreadLocalTest
> > > > >         api.kernel.exec.RunExec
> > > > >
> > > > >  8) Stress
> > > > >    Different test cases failed on different platforms.
> > > > >    8.1) Timeouts on Linux x86.
> > > > >
> > > > >
> > > > >
> > > >
> > >
> > stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD003.ThreadTest003
> > > > >
> > > > >
> > > > >
> > > >
> > >
> > stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD007.ThreadTest007
> > > > >
> > > > >
> > > > >
> > > >
> > >
> > stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD009.ThreadTest009
> > > > >
> > > > >
> > > > >
> > > >
> > >
> > stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD011.ThreadTest011
> > > > >
> > > > >    8.2) Failed cases on Windows x86_64 with unknown reasons.
> > > > >
> > > > >
> > > > >
> > > >
> > >
> > stress.org.apache.harmony.test.stress.classloader.MixThreads.TreeClasses.testTreeClasses2
> > > > >
> > > > >
> > > > >
> > > >
> > >
> > stress.org.apache.harmony.test.stress.classloader.NotSynchThreads.TreeClasses.testTreeClasses
> > > > >
> > > > >    I have not found records for them.
> > > > >
> > > > >  9) Strut_test
> > > > >    Broken with same error report as M6.
> > > > >
> > > > >  10) Eclipse Hello World Application.
> > > > >    Although the testing framework just reports EUT-API status as
> > > > "PASSED".
> > > > > A fresh workspace running just failed in configuration stage (
> > > > >
> > > > >
> > > >
> > >
> > http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805120106o387a49f1rfb2c33d1042d2f41@mail.gmail.com%3e
> > > > > )
> > > > > since SVN commit 641928 (which committed patch for HARMONY-4569).
> > > > >
> > > > > thanks,
> > > > > chunrong
> > > > >
> > > >
> > >
> >
>



-- 
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

Re: [testing] first M7 candidate (r681495) testing status

Posted by chunrong lai <ch...@gmail.com>.
   Thanks.
   I think the standard scripts produce that site although I always added
publish/e-mail notification setup before install/run the test suites.
   It is understandable that some tests in classlib-test and drlvm-test get
failing report. We just need to understand the failures.
   As reported in M6,
org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest,
org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
etc fail classlib-test.
   Also we observed that java.lang.ClassGenericsTest.test_2 sometimes fail
drlvm-test.
   Another timeout issue of drlvm-test comes from
thread.SmallStackThreadTest_jit, as discussed in HARMONY-4601, the test case
may run very slow when system resource is tight and finally introduce a
timeout. Re-running the suite generally fixes the timeout.





On 8/12/08, Nathan Beyer <nd...@apache.org> wrote:
>
> It's the tests that are failing - both classlib-test and drlvm-test.
>
> What's the specific integrity setup that's being used for that site? Does
> the standard out-of-the-box buildtest scripts produce that site?
>
> -Nathan
>
> On Mon, Aug 11, 2008 at 9:05 PM, chunrong lai <ch...@gmail.com>
> wrote:
>
> > hi, Nathan:
> >     Steps to run testing cycle are listed in
> > http://harmony.apache.org/subcomponents/buildtest/index.html. One
> > integrity
> > testing example is
> > http://harmony.apache.org/subcomponents/buildtest/howto.html#Extended.
> >      What is the error message you met (in what step)? What is the
> content
> > of the framework.local.properties file? I think that will help to figure
> > out
> > the problem.
> >
> >
> > On 8/12/08, Nathan Beyer <nd...@apache.org> wrote:
> > >
> > > What's being used to produce the integrity tests? I have an x86_64
> Linux
> > > box
> > > that I can dedicate to testing. I've tried just using buildtest with
> > > 'classlib,drlvm,classib-test,drlvm-test', but I haven't gotten a clean
> > pass
> > > yet.
> > >
> > > -Nathan
> > >
> > > On Mon, Aug 11, 2008 at 12:26 PM, chunrong lai <chunronglai@gmail.com
> > > >wrote:
> > >
> > > > Hi, all:
> > > >
> > > >  Here is r681495 snapshot testing status:
> > > > http://people.apache.org/~chunrong/snapshots/r681495/index.html<
> http://people.apache.org/%7Echunrong/snapshots/r681495/index.html>
> > <
> > > http://people.apache.org/%7Echunrong/snapshots/r681495/index.html>.
> > > > I am using
> > > > two platforms (Linux x86, windows x86_64) at the moment. Hopefully we
> > > will
> > > > have other two platforms in future for M8. Well, although we are
> > testing
> > > > only two platforms for M7, my experience is that the status for
> another
> > > two
> > > > platforms should be not worse or just include some extra intermittent
> > > > errors
> > > > which can be investigated in some later stages.
> > > >
> > > >  The following suites passed on Linux x86/Windows x86_64 platforms:
> Ant
> > > > Scenario (or self-hosting), Axis application, Dacapo, DRLVM
> regression
> > > > tests, Geronimo Unit Tests, Scimark, Tomcat scenario, VTS VM Test
> > Suite.
> > > >
> > > >  Most of the failures are known issues (for M6). Although we can
> > observe
> > > > 15~20 new issues, those issues happen only in 1 platform and they
> look
> > > more
> > > > like the intermittent/timeout issues (less reproducible) to me.
> Overall
> > I
> > > > myself would like to think r681495 is more stable than M6.
> > > >
> > > >  Please add your comments and clarifications (please also see M6
> > testing
> > > > results http://people.apache.org/~smishura/r653525/<
> http://people.apache.org/%7Esmishura/r653525/>
> > <
> > > http://people.apache.org/%7Esmishura/r653525/>
> > > > ,
> > > >
> > > >
> > >
> >
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805070304l38845ee0se01fb93fbfc05586@mail.gmail.com%3eand
> > > > integrity testing results
> > > > http://people.apache.org/~chunrong/harmony-integrity/<
> http://people.apache.org/%7Echunrong/harmony-integrity/>
> > <
> > > http://people.apache.org/%7Echunrong/harmony-integrity/>as a
> > comparison).
> > > >
> > > >  1) Classlib:
> > > >    1.1) Since r644719 (which committed patch for HARMONY-5684)
> > > >
> > > >        org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest
> > > >
> > > >
> > > >
> > >
> >
> org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
> > > >
> > > >         failed in both platforms
> > > >
> > > >    1.2) Two failures
> > > >
> > > >        org.apache.harmony.luni.tests.java.net.MulticastSocketTest
> > (Failed
> > > > in windows_x86 running of M6)
> > > >        tests.api.java.security.PermissionCollectionTest
> > (Failed
> > > > in M6)
> > > >
> > > >         are observed in Linux x86.
> > > >
> > > >  2) DRLVM tests:
> > > >    2.1) One failure
> > > >
> > > >         java.lang.ClassGenericsTest.test_2
> > > >
> > > >         is observed in Linux x86 snapshot testing.
> > > >         I can see some old discussion in the mailing list about that
> > but
> > > I
> > > > am not sure the expected status here.
> > > >         They should be intemittent errors since the integrity testing
> > > just
> > > > run well mostly.
> > > >
> > > >  3) EUTs:
> > > >
> > > >    3.1) Linux x86: 99.36%
> > > >         A recorded JIRA for this suite is HARMONY-2914 which wastes
> > file
> > > > handlers and makes system unstable.
> > > >
> > > >  4) Functional:
> > > >    4.1) Old regressions on both platforms:
> > > >         api.java.text.MessageFormat (HARMONY-5430)
> > > >         api.java.util.jar.Manifest  (HARMONY-5473)
> > > >         api.java.beans.beancontext.BeanContextTest (also in M6,
> > recorded
> > > > as  regression caused by changes in locale data)
> > > >         api.java.beans.persistence.EncoderTest  (also in M6, recorded
> > as
> > > > regression in the beans module)
> > > >         api.java.beans.persistence.EncoderDecoderTest (also in M6,
> > > > regression in the beans module)
> > > >         reg.vm.btest5625 (also in M6, recorded as intermittent and
> not
> > > > reproducible manually)
> > > >
> > > >    4.2) Old regressions on 1 platform
> > > >         api.java.rmi.basicexception (ERROR in Linux x86,
> HARMONY-5823)
> > > >         api.java.rmi.basicregistry.RemoteServerTest (ERROR in Linux
> > x86,
> > > > HARMONY-5823)
> > > >         jpda.jdwp.scenario.ST07.ST07Test (ERROR in windows x86_64, in
> > M6
> > > it
> > > > is recorded as regression since M4)
> > > >         java.math.F_BigIntegerMatrixMultiplyTest_01 (ERROR on Linux
> > x86,
> > > > recorded as Timeout, not reproducible
> > > >  in M6)
> > > >         reg.vm.btest5717 (ERROR on Windows X86_64, recorded as
> > "timeout,
> > > > the test is too heavy" in M6)
> > > >         jit.HLO.inline.ControlFlow.IfElse.IfElse1.IfElseTest1 (FAILED
> > in
> > > > windows x86_64, recorded as "looks like
> > > > issue in test" in M6)
> > > >         jit.HLO.devirt.Runtime.RuntimeExtend1 (FAILED on windows
> > x86_64,
> > > in
> > > > M6 it is recorded as not regression and started to fail on M5)
> > > >         reg.vm.btest6353.Btest6353 (ERROR on Windows x86_64, recorded
> > > also
> > > > failed on M3 and might be issue with the test)
> > > >
> > > >    4.3) New regressions on 1 platform (looks intermittent)
> > > >         reg.jit.btest8029.Btest8029 (FAILED in Linux x86)
> > > >         func.reg.jit.btest5710.Btest5710 (FAILED in Linux x86)
> > > >         api.java.security.cert.F_CertPathTest_06.F_CertPathTest_06
> > (ERROR
> > > > in Linux x86)
> > > >         api.java.security.cert.F_CertPathTest_05.F_CertPathTest_05
> > (ERROR
> > > > in Linux x86)
> > > >
> > > >  5) JDKTools Tests:
> > > >    Several timeouts are observed in Linux x86 snapshot running. They
> > are:
> > > >
> > > >
> > > >
> > >
> >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch001
> > > >
> > > >
> > > >
> > >
> >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch002
> > > >
> > > >
> > > >
> > >
> >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch003
> > > >
> > > >
> > > >
> > >
> >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch004
> > > >
> > > >
> > > >
> > >
> >
> org.apache.harmony.jpda.tests.jdwp.Events.CombinedEventsTest.testCombinedEvents_04
> > > >
> > > >
> > > >
> > >
> >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.AttachConnectorTest.testAttachConnector001
> > > >
> > > >
> > > >
> > >
> >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.MethodEntryExitTest.testMethodEvent001
> > > >
> > > > org.apache.harmony.jpda.tests.jdwp.MultiSession.ResumeTest.testResume
> > > >
> > > >
> > > >
> > >
> >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadEndTest.testThreadEnd001
> > > >
> > > >
> > > >
> > >
> >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadStartTest.testThreadStart001
> > > >    The Linux-only timeouts are also observed in the integrity testing
> > > > results.
> > > >    JIRA HARMONY-5833 has been filed for one of them.
> > > >
> > > >  6) JettyScenario:
> > > >    The Linux x86 running failed because of the unresolved
> HARMONY-5219.
> > > >
> > > >  7) Reliability:
> > > >    Several failures are observed in windows x86_64 running.
> > > >    7.1) Old regressions
> > > >         api.net.DatagramTest (HARMONY-5531)
> > > >         api.text.DecimalFormat_Locales - (in M6 it is recorded as
> also
> > > > intermittent in M5)
> > > >
> > > >    7.2) New/intemittent regressions
> > > >         api.kernel.thread.ThreadLocalTest.ThreadLocalTest
> > > >         api.kernel.exec.RunExec
> > > >
> > > >  8) Stress
> > > >    Different test cases failed on different platforms.
> > > >    8.1) Timeouts on Linux x86.
> > > >
> > > >
> > > >
> > >
> >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD003.ThreadTest003
> > > >
> > > >
> > > >
> > >
> >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD007.ThreadTest007
> > > >
> > > >
> > > >
> > >
> >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD009.ThreadTest009
> > > >
> > > >
> > > >
> > >
> >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD011.ThreadTest011
> > > >
> > > >    8.2) Failed cases on Windows x86_64 with unknown reasons.
> > > >
> > > >
> > > >
> > >
> >
> stress.org.apache.harmony.test.stress.classloader.MixThreads.TreeClasses.testTreeClasses2
> > > >
> > > >
> > > >
> > >
> >
> stress.org.apache.harmony.test.stress.classloader.NotSynchThreads.TreeClasses.testTreeClasses
> > > >
> > > >    I have not found records for them.
> > > >
> > > >  9) Strut_test
> > > >    Broken with same error report as M6.
> > > >
> > > >  10) Eclipse Hello World Application.
> > > >    Although the testing framework just reports EUT-API status as
> > > "PASSED".
> > > > A fresh workspace running just failed in configuration stage (
> > > >
> > > >
> > >
> >
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805120106o387a49f1rfb2c33d1042d2f41@mail.gmail.com%3e
> > > > )
> > > > since SVN commit 641928 (which committed patch for HARMONY-4569).
> > > >
> > > > thanks,
> > > > chunrong
> > > >
> > >
> >
>

Re: [testing] first M7 candidate (r681495) testing status

Posted by Nathan Beyer <nd...@apache.org>.
It's the tests that are failing - both classlib-test and drlvm-test.

What's the specific integrity setup that's being used for that site? Does
the standard out-of-the-box buildtest scripts produce that site?

-Nathan

On Mon, Aug 11, 2008 at 9:05 PM, chunrong lai <ch...@gmail.com> wrote:

> hi, Nathan:
>     Steps to run testing cycle are listed in
> http://harmony.apache.org/subcomponents/buildtest/index.html. One
> integrity
> testing example is
> http://harmony.apache.org/subcomponents/buildtest/howto.html#Extended.
>      What is the error message you met (in what step)? What is the content
> of the framework.local.properties file? I think that will help to figure
> out
> the problem.
>
>
> On 8/12/08, Nathan Beyer <nd...@apache.org> wrote:
> >
> > What's being used to produce the integrity tests? I have an x86_64 Linux
> > box
> > that I can dedicate to testing. I've tried just using buildtest with
> > 'classlib,drlvm,classib-test,drlvm-test', but I haven't gotten a clean
> pass
> > yet.
> >
> > -Nathan
> >
> > On Mon, Aug 11, 2008 at 12:26 PM, chunrong lai <chunronglai@gmail.com
> > >wrote:
> >
> > > Hi, all:
> > >
> > >  Here is r681495 snapshot testing status:
> > > http://people.apache.org/~chunrong/snapshots/r681495/index.html<http://people.apache.org/%7Echunrong/snapshots/r681495/index.html>
> <
> > http://people.apache.org/%7Echunrong/snapshots/r681495/index.html>.
> > > I am using
> > > two platforms (Linux x86, windows x86_64) at the moment. Hopefully we
> > will
> > > have other two platforms in future for M8. Well, although we are
> testing
> > > only two platforms for M7, my experience is that the status for another
> > two
> > > platforms should be not worse or just include some extra intermittent
> > > errors
> > > which can be investigated in some later stages.
> > >
> > >  The following suites passed on Linux x86/Windows x86_64 platforms: Ant
> > > Scenario (or self-hosting), Axis application, Dacapo, DRLVM regression
> > > tests, Geronimo Unit Tests, Scimark, Tomcat scenario, VTS VM Test
> Suite.
> > >
> > >  Most of the failures are known issues (for M6). Although we can
> observe
> > > 15~20 new issues, those issues happen only in 1 platform and they look
> > more
> > > like the intermittent/timeout issues (less reproducible) to me. Overall
> I
> > > myself would like to think r681495 is more stable than M6.
> > >
> > >  Please add your comments and clarifications (please also see M6
> testing
> > > results http://people.apache.org/~smishura/r653525/<http://people.apache.org/%7Esmishura/r653525/>
> <
> > http://people.apache.org/%7Esmishura/r653525/>
> > > ,
> > >
> > >
> >
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805070304l38845ee0se01fb93fbfc05586@mail.gmail.com%3eand
> > > integrity testing results
> > > http://people.apache.org/~chunrong/harmony-integrity/<http://people.apache.org/%7Echunrong/harmony-integrity/>
> <
> > http://people.apache.org/%7Echunrong/harmony-integrity/>as a
> comparison).
> > >
> > >  1) Classlib:
> > >    1.1) Since r644719 (which committed patch for HARMONY-5684)
> > >
> > >        org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest
> > >
> > >
> > >
> >
> org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
> > >
> > >         failed in both platforms
> > >
> > >    1.2) Two failures
> > >
> > >        org.apache.harmony.luni.tests.java.net.MulticastSocketTest
> (Failed
> > > in windows_x86 running of M6)
> > >        tests.api.java.security.PermissionCollectionTest
> (Failed
> > > in M6)
> > >
> > >         are observed in Linux x86.
> > >
> > >  2) DRLVM tests:
> > >    2.1) One failure
> > >
> > >         java.lang.ClassGenericsTest.test_2
> > >
> > >         is observed in Linux x86 snapshot testing.
> > >         I can see some old discussion in the mailing list about that
> but
> > I
> > > am not sure the expected status here.
> > >         They should be intemittent errors since the integrity testing
> > just
> > > run well mostly.
> > >
> > >  3) EUTs:
> > >
> > >    3.1) Linux x86: 99.36%
> > >         A recorded JIRA for this suite is HARMONY-2914 which wastes
> file
> > > handlers and makes system unstable.
> > >
> > >  4) Functional:
> > >    4.1) Old regressions on both platforms:
> > >         api.java.text.MessageFormat (HARMONY-5430)
> > >         api.java.util.jar.Manifest  (HARMONY-5473)
> > >         api.java.beans.beancontext.BeanContextTest (also in M6,
> recorded
> > > as  regression caused by changes in locale data)
> > >         api.java.beans.persistence.EncoderTest  (also in M6, recorded
> as
> > > regression in the beans module)
> > >         api.java.beans.persistence.EncoderDecoderTest (also in M6,
> > > regression in the beans module)
> > >         reg.vm.btest5625 (also in M6, recorded as intermittent and not
> > > reproducible manually)
> > >
> > >    4.2) Old regressions on 1 platform
> > >         api.java.rmi.basicexception (ERROR in Linux x86, HARMONY-5823)
> > >         api.java.rmi.basicregistry.RemoteServerTest (ERROR in Linux
> x86,
> > > HARMONY-5823)
> > >         jpda.jdwp.scenario.ST07.ST07Test (ERROR in windows x86_64, in
> M6
> > it
> > > is recorded as regression since M4)
> > >         java.math.F_BigIntegerMatrixMultiplyTest_01 (ERROR on Linux
> x86,
> > > recorded as Timeout, not reproducible
> > >  in M6)
> > >         reg.vm.btest5717 (ERROR on Windows X86_64, recorded as
> "timeout,
> > > the test is too heavy" in M6)
> > >         jit.HLO.inline.ControlFlow.IfElse.IfElse1.IfElseTest1 (FAILED
> in
> > > windows x86_64, recorded as "looks like
> > > issue in test" in M6)
> > >         jit.HLO.devirt.Runtime.RuntimeExtend1 (FAILED on windows
> x86_64,
> > in
> > > M6 it is recorded as not regression and started to fail on M5)
> > >         reg.vm.btest6353.Btest6353 (ERROR on Windows x86_64, recorded
> > also
> > > failed on M3 and might be issue with the test)
> > >
> > >    4.3) New regressions on 1 platform (looks intermittent)
> > >         reg.jit.btest8029.Btest8029 (FAILED in Linux x86)
> > >         func.reg.jit.btest5710.Btest5710 (FAILED in Linux x86)
> > >         api.java.security.cert.F_CertPathTest_06.F_CertPathTest_06
> (ERROR
> > > in Linux x86)
> > >         api.java.security.cert.F_CertPathTest_05.F_CertPathTest_05
> (ERROR
> > > in Linux x86)
> > >
> > >  5) JDKTools Tests:
> > >    Several timeouts are observed in Linux x86 snapshot running. They
> are:
> > >
> > >
> > >
> >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch001
> > >
> > >
> > >
> >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch002
> > >
> > >
> > >
> >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch003
> > >
> > >
> > >
> >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch004
> > >
> > >
> > >
> >
> org.apache.harmony.jpda.tests.jdwp.Events.CombinedEventsTest.testCombinedEvents_04
> > >
> > >
> > >
> >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.AttachConnectorTest.testAttachConnector001
> > >
> > >
> > >
> >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.MethodEntryExitTest.testMethodEvent001
> > >
> > > org.apache.harmony.jpda.tests.jdwp.MultiSession.ResumeTest.testResume
> > >
> > >
> > >
> >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadEndTest.testThreadEnd001
> > >
> > >
> > >
> >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadStartTest.testThreadStart001
> > >    The Linux-only timeouts are also observed in the integrity testing
> > > results.
> > >    JIRA HARMONY-5833 has been filed for one of them.
> > >
> > >  6) JettyScenario:
> > >    The Linux x86 running failed because of the unresolved HARMONY-5219.
> > >
> > >  7) Reliability:
> > >    Several failures are observed in windows x86_64 running.
> > >    7.1) Old regressions
> > >         api.net.DatagramTest (HARMONY-5531)
> > >         api.text.DecimalFormat_Locales - (in M6 it is recorded as also
> > > intermittent in M5)
> > >
> > >    7.2) New/intemittent regressions
> > >         api.kernel.thread.ThreadLocalTest.ThreadLocalTest
> > >         api.kernel.exec.RunExec
> > >
> > >  8) Stress
> > >    Different test cases failed on different platforms.
> > >    8.1) Timeouts on Linux x86.
> > >
> > >
> > >
> >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD003.ThreadTest003
> > >
> > >
> > >
> >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD007.ThreadTest007
> > >
> > >
> > >
> >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD009.ThreadTest009
> > >
> > >
> > >
> >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD011.ThreadTest011
> > >
> > >    8.2) Failed cases on Windows x86_64 with unknown reasons.
> > >
> > >
> > >
> >
> stress.org.apache.harmony.test.stress.classloader.MixThreads.TreeClasses.testTreeClasses2
> > >
> > >
> > >
> >
> stress.org.apache.harmony.test.stress.classloader.NotSynchThreads.TreeClasses.testTreeClasses
> > >
> > >    I have not found records for them.
> > >
> > >  9) Strut_test
> > >    Broken with same error report as M6.
> > >
> > >  10) Eclipse Hello World Application.
> > >    Although the testing framework just reports EUT-API status as
> > "PASSED".
> > > A fresh workspace running just failed in configuration stage (
> > >
> > >
> >
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805120106o387a49f1rfb2c33d1042d2f41@mail.gmail.com%3e
> > > )
> > > since SVN commit 641928 (which committed patch for HARMONY-4569).
> > >
> > > thanks,
> > > chunrong
> > >
> >
>

Re: [testing] first M7 candidate (r681495) testing status

Posted by chunrong lai <ch...@gmail.com>.
hi, Nathan:
     Steps to run testing cycle are listed in
http://harmony.apache.org/subcomponents/buildtest/index.html. One integrity
testing example is
http://harmony.apache.org/subcomponents/buildtest/howto.html#Extended.
      What is the error message you met (in what step)? What is the content
of the framework.local.properties file? I think that will help to figure out
the problem.


On 8/12/08, Nathan Beyer <nd...@apache.org> wrote:
>
> What's being used to produce the integrity tests? I have an x86_64 Linux
> box
> that I can dedicate to testing. I've tried just using buildtest with
> 'classlib,drlvm,classib-test,drlvm-test', but I haven't gotten a clean pass
> yet.
>
> -Nathan
>
> On Mon, Aug 11, 2008 at 12:26 PM, chunrong lai <chunronglai@gmail.com
> >wrote:
>
> > Hi, all:
> >
> >  Here is r681495 snapshot testing status:
> > http://people.apache.org/~chunrong/snapshots/r681495/index.html<
> http://people.apache.org/%7Echunrong/snapshots/r681495/index.html>.
> > I am using
> > two platforms (Linux x86, windows x86_64) at the moment. Hopefully we
> will
> > have other two platforms in future for M8. Well, although we are testing
> > only two platforms for M7, my experience is that the status for another
> two
> > platforms should be not worse or just include some extra intermittent
> > errors
> > which can be investigated in some later stages.
> >
> >  The following suites passed on Linux x86/Windows x86_64 platforms: Ant
> > Scenario (or self-hosting), Axis application, Dacapo, DRLVM regression
> > tests, Geronimo Unit Tests, Scimark, Tomcat scenario, VTS VM Test Suite.
> >
> >  Most of the failures are known issues (for M6). Although we can observe
> > 15~20 new issues, those issues happen only in 1 platform and they look
> more
> > like the intermittent/timeout issues (less reproducible) to me. Overall I
> > myself would like to think r681495 is more stable than M6.
> >
> >  Please add your comments and clarifications (please also see M6 testing
> > results http://people.apache.org/~smishura/r653525/<
> http://people.apache.org/%7Esmishura/r653525/>
> > ,
> >
> >
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805070304l38845ee0se01fb93fbfc05586@mail.gmail.com%3eand
> > integrity testing results
> > http://people.apache.org/~chunrong/harmony-integrity/<
> http://people.apache.org/%7Echunrong/harmony-integrity/>as a comparison).
> >
> >  1) Classlib:
> >    1.1) Since r644719 (which committed patch for HARMONY-5684)
> >
> >        org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest
> >
> >
> >
> org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
> >
> >         failed in both platforms
> >
> >    1.2) Two failures
> >
> >        org.apache.harmony.luni.tests.java.net.MulticastSocketTest (Failed
> > in windows_x86 running of M6)
> >        tests.api.java.security.PermissionCollectionTest           (Failed
> > in M6)
> >
> >         are observed in Linux x86.
> >
> >  2) DRLVM tests:
> >    2.1) One failure
> >
> >         java.lang.ClassGenericsTest.test_2
> >
> >         is observed in Linux x86 snapshot testing.
> >         I can see some old discussion in the mailing list about that but
> I
> > am not sure the expected status here.
> >         They should be intemittent errors since the integrity testing
> just
> > run well mostly.
> >
> >  3) EUTs:
> >
> >    3.1) Linux x86: 99.36%
> >         A recorded JIRA for this suite is HARMONY-2914 which wastes file
> > handlers and makes system unstable.
> >
> >  4) Functional:
> >    4.1) Old regressions on both platforms:
> >         api.java.text.MessageFormat (HARMONY-5430)
> >         api.java.util.jar.Manifest  (HARMONY-5473)
> >         api.java.beans.beancontext.BeanContextTest (also in M6, recorded
> > as  regression caused by changes in locale data)
> >         api.java.beans.persistence.EncoderTest  (also in M6, recorded as
> > regression in the beans module)
> >         api.java.beans.persistence.EncoderDecoderTest (also in M6,
> > regression in the beans module)
> >         reg.vm.btest5625 (also in M6, recorded as intermittent and not
> > reproducible manually)
> >
> >    4.2) Old regressions on 1 platform
> >         api.java.rmi.basicexception (ERROR in Linux x86, HARMONY-5823)
> >         api.java.rmi.basicregistry.RemoteServerTest (ERROR in Linux x86,
> > HARMONY-5823)
> >         jpda.jdwp.scenario.ST07.ST07Test (ERROR in windows x86_64, in M6
> it
> > is recorded as regression since M4)
> >         java.math.F_BigIntegerMatrixMultiplyTest_01 (ERROR on Linux x86,
> > recorded as Timeout, not reproducible
> >  in M6)
> >         reg.vm.btest5717 (ERROR on Windows X86_64, recorded as "timeout,
> > the test is too heavy" in M6)
> >         jit.HLO.inline.ControlFlow.IfElse.IfElse1.IfElseTest1 (FAILED in
> > windows x86_64, recorded as "looks like
> > issue in test" in M6)
> >         jit.HLO.devirt.Runtime.RuntimeExtend1 (FAILED on windows x86_64,
> in
> > M6 it is recorded as not regression and started to fail on M5)
> >         reg.vm.btest6353.Btest6353 (ERROR on Windows x86_64, recorded
> also
> > failed on M3 and might be issue with the test)
> >
> >    4.3) New regressions on 1 platform (looks intermittent)
> >         reg.jit.btest8029.Btest8029 (FAILED in Linux x86)
> >         func.reg.jit.btest5710.Btest5710 (FAILED in Linux x86)
> >         api.java.security.cert.F_CertPathTest_06.F_CertPathTest_06 (ERROR
> > in Linux x86)
> >         api.java.security.cert.F_CertPathTest_05.F_CertPathTest_05 (ERROR
> > in Linux x86)
> >
> >  5) JDKTools Tests:
> >    Several timeouts are observed in Linux x86 snapshot running. They are:
> >
> >
> >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch001
> >
> >
> >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch002
> >
> >
> >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch003
> >
> >
> >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch004
> >
> >
> >
> org.apache.harmony.jpda.tests.jdwp.Events.CombinedEventsTest.testCombinedEvents_04
> >
> >
> >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.AttachConnectorTest.testAttachConnector001
> >
> >
> >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.MethodEntryExitTest.testMethodEvent001
> >
> > org.apache.harmony.jpda.tests.jdwp.MultiSession.ResumeTest.testResume
> >
> >
> >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadEndTest.testThreadEnd001
> >
> >
> >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadStartTest.testThreadStart001
> >    The Linux-only timeouts are also observed in the integrity testing
> > results.
> >    JIRA HARMONY-5833 has been filed for one of them.
> >
> >  6) JettyScenario:
> >    The Linux x86 running failed because of the unresolved HARMONY-5219.
> >
> >  7) Reliability:
> >    Several failures are observed in windows x86_64 running.
> >    7.1) Old regressions
> >         api.net.DatagramTest (HARMONY-5531)
> >         api.text.DecimalFormat_Locales - (in M6 it is recorded as also
> > intermittent in M5)
> >
> >    7.2) New/intemittent regressions
> >         api.kernel.thread.ThreadLocalTest.ThreadLocalTest
> >         api.kernel.exec.RunExec
> >
> >  8) Stress
> >    Different test cases failed on different platforms.
> >    8.1) Timeouts on Linux x86.
> >
> >
> >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD003.ThreadTest003
> >
> >
> >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD007.ThreadTest007
> >
> >
> >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD009.ThreadTest009
> >
> >
> >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD011.ThreadTest011
> >
> >    8.2) Failed cases on Windows x86_64 with unknown reasons.
> >
> >
> >
> stress.org.apache.harmony.test.stress.classloader.MixThreads.TreeClasses.testTreeClasses2
> >
> >
> >
> stress.org.apache.harmony.test.stress.classloader.NotSynchThreads.TreeClasses.testTreeClasses
> >
> >    I have not found records for them.
> >
> >  9) Strut_test
> >    Broken with same error report as M6.
> >
> >  10) Eclipse Hello World Application.
> >    Although the testing framework just reports EUT-API status as
> "PASSED".
> > A fresh workspace running just failed in configuration stage (
> >
> >
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805120106o387a49f1rfb2c33d1042d2f41@mail.gmail.com%3e
> > )
> > since SVN commit 641928 (which committed patch for HARMONY-4569).
> >
> > thanks,
> > chunrong
> >
>

Re: [testing] first M7 candidate (r681495) testing status

Posted by chunrong lai <ch...@gmail.com>.
Thanks.
I run it from a xterm in a VNC client (a VNC server is running in the
machine). Then I just invoke cruise control and let it go.


On 8/14/08, Nathan Beyer <nb...@gmail.com> wrote:
>
> I understand the concept, but running CruiseControl (a headless daemon
> process) and a local X session are a bit at odds, aren't they?
>
> I'm asking, in practical terms, how are you doing it? Start a local X
> session, start an xterm, invoke cruise control and let it go? Is there
> anyway to do this remotely? Direct access to a machine isn't always an
> option and generally a remote client can't be permanently left up. Is
> there anyway to do a 'nohup' with an X session?
>
> -Nathan
>
> On Wed, Aug 13, 2008 at 9:18 PM, chunrong lai <ch...@gmail.com>
> wrote:
> >  I understand that CruiseControl does trigger the test suite running. But
> > some test suites themselves (classlib, dacapo etc) do require GUI
> support.
> > So in Linux one needs to run cruisecontrol ("ant run-cc") in an
> environment
> > with X-connection (for example, in X, vnc-client) instead of text-mode
> > client (putty, ssh, term etc).
> >
> >
> >
> > On 8/14/08, Nathan Beyer <nd...@apache.org> wrote:
> >>
> >> On Wed, Aug 13, 2008 at 8:58 PM, chunrong lai <ch...@gmail.com>
> >> wrote:
> >> > (1) M7 status is got in Fedora distros and M6/M5 status were got in
> SuSe
> >> > Linux distros.
> >> > (2) According to the stacktrace
> >> > ( java.awt.Toolkit.init,  java.awt.SystemColor.<init>) of the
> >> > java.beans.Encoder error (I have not observed such error reports in
> >> previous
> >> > logs),  I guess the problem is from GUI system initialization. Is
> >> > X-connection available in your testing? In my understanding that is
> >> required
> >> > to run classlib test.
> >>
> >> It's running under CruiseControl, like the howto suggested, how do you
> >> do an X-connection with that?
> >>
> >> >
> >> >
> >> > On 8/14/08, Nathan Beyer <nd...@apache.org> wrote:
> >> >>
> >> >> What Linux distro is it? I'm running Ubuntu and I'm seeing some
> >> >> different issues.
> >> >>
> >> >> I'm getting this failure -
> >> >>
> >> >> java.lang.ExceptionInInitializerError
> >> >> at java.lang.reflect.VMField.getObject(VMField.java)
> >> >> at java.lang.reflect.Field.get(Field.java:163)
> >> >> at
> >> >>
> >>
> java.beans.StaticFieldPersistenceDelegate.init(StaticFieldPersistenceDelegate.java:36)
> >> >> at
> >> >>
> >>
> java.beans.AwtSystemColorPersistenceDelegate.<clinit>(AwtSystemColorPersistenceDelegate.java:28)
> >> >> at java.beans.Encoder.<clinit>(Encoder.java:124)
> >> >> at
> >> >>
> >>
> java.beans.ArrayPersistenceDelegateTest.testInstantiate_Normal(ArrayPersistenceDelegateTest.java:130)
> >> >> at java.lang.reflect.VMReflection.invokeMethod(VMReflection.java)
> >> >> Caused by: java.lang.NullPointerException
> >> >> at java.awt.Toolkit.init(Toolkit.java:566)
> >> >> at java.awt.Toolkit.<init>(Toolkit.java:540)
> >> >> at java.awt.ToolkitImpl.<init>(ToolkitImpl.java:40)
> >> >> at java.awt.Toolkit.getDefaultToolkit(Toolkit.java:472)
> >> >> at java.awt.SystemColor.<init>(SystemColor.java:143)
> >> >> at java.awt.SystemColor.<clinit>(SystemColor.java:91)
> >> >> at java.lang.reflect.VMField.getObject(VMField.java)
> >> >> ... 18 more
> >> >>
> >> >> Then a bunch of NoClassDefFoundErrors for java.beans.Encoder.
> >> >>
> >> >>
> >> >> On Wed, Aug 13, 2008 at 3:21 AM, chunrong lai <chunronglai@gmail.com
> >
> >> >> wrote:
> >> >> >
> >> >> > hi, Nathan:
> >> >> >     It is also noted that currently we do setup Linux x86-64
> integrity
> >> >> > testing, which eventually test every SVN revision, with results
> >> reported
> >> >> in
> >> >> > http://people.apache.org/~chunrong/harmony-integrity/. Your
> mentioned
> >> >> test
> >> >> > suites classib-test,drlvm-test are included there. (classlib+drlvm
> >> >> > are almost same as the HDK build reported in the page).
> >> >> >     We only use two platforms currently for the snapshot testing
> >> cycle,
> >> >> > (table is generated by
> >> >> >
> >> >>
> >>
> http://svn.apache.org/viewvc/harmony/enhanced/buildtest/trunk/tools/snapshot/snapshots-table/
> >> >> )
> >> >> > , to test more test suites with longer time for every snapshot. In
> my
> >> >> > observation adding two other platforms here just does not
> necessarily
> >> add
> >> >> > enough value in current stage of Harmony.
> >> >> >     I do think more testings/discussions help to improve Harmony
> >> quality.
> >> >> > Thanks for the dedication. If there are more I can help please let
> me
> >> >> know.
> >> >> >
> >> >> > On 8/12/08, Nathan Beyer <nd...@apache.org> wrote:
> >> >> > >
> >> >> > > What's being used to produce the integrity tests? I have an
> x86_64
> >> >> Linux
> >> >> > > box
> >> >> > > that I can dedicate to testing. I've tried just using buildtest
> with
> >> >> > > 'classlib,drlvm,classib-test,drlvm-test', but I haven't gotten a
> >> clean
> >> >> pass
> >> >> > > yet.
> >> >> > >
> >> >> > > -Nathan
> >> >> > >
> >> >> > > On Mon, Aug 11, 2008 at 12:26 PM, chunrong lai <
> >> chunronglai@gmail.com
> >> >> > > >wrote:
> >> >> > >
> >> >> > > > Hi, all:
> >> >> > > >
> >> >> > > >  Here is r681495 snapshot testing status:
> >> >> > > >
> http://people.apache.org/~chunrong/snapshots/r681495/index.html<
> >> >> > >
> http://people.apache.org/%7Echunrong/snapshots/r681495/index.html>.
> >> >> > > > I am using
> >> >> > > > two platforms (Linux x86, windows x86_64) at the moment.
> Hopefully
> >> we
> >> >> > > will
> >> >> > > > have other two platforms in future for M8. Well, although we
> are
> >> >> testing
> >> >> > > > only two platforms for M7, my experience is that the status for
> >> >> another
> >> >> > > two
> >> >> > > > platforms should be not worse or just include some extra
> >> intermittent
> >> >> > > > errors
> >> >> > > > which can be investigated in some later stages.
> >> >> > > >
> >> >> > > >  The following suites passed on Linux x86/Windows x86_64
> >> platforms:
> >> >> Ant
> >> >> > > > Scenario (or self-hosting), Axis application, Dacapo, DRLVM
> >> >> regression
> >> >> > > > tests, Geronimo Unit Tests, Scimark, Tomcat scenario, VTS VM
> Test
> >> >> Suite.
> >> >> > > >
> >> >> > > >  Most of the failures are known issues (for M6). Although we
> can
> >> >> observe
> >> >> > > > 15~20 new issues, those issues happen only in 1 platform and
> they
> >> >> look
> >> >> > > more
> >> >> > > > like the intermittent/timeout issues (less reproducible) to me.
> >> >> Overall I
> >> >> > > > myself would like to think r681495 is more stable than M6.
> >> >> > > >
> >> >> > > >  Please add your comments and clarifications (please also see
> M6
> >> >> testing
> >> >> > > > results http://people.apache.org/~smishura/r653525/<
> >> >> > > http://people.apache.org/%7Esmishura/r653525/>
> >> >> > > > ,
> >> >> > > >
> >> >> > > >
> >> >> > >
> >> >>
> >>
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805070304l38845ee0se01fb93fbfc05586@mail.gmail.com%3eand
> >> >> > > > integrity testing results
> >> >> > > > http://people.apache.org/~chunrong/harmony-integrity/<
> >> >> > > http://people.apache.org/%7Echunrong/harmony-integrity/>as a
> >> >> comparison).
> >> >> > > >
> >> >> > > >  1) Classlib:
> >> >> > > >    1.1) Since r644719 (which committed patch for HARMONY-5684)
> >> >> > > >
> >> >> > >
> >        org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > >
> >> >>
> >>
> org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
> >> >> > > >
> >> >> > > >         failed in both platforms
> >> >> > > >
> >> >> > > >    1.2) Two failures
> >> >> > > >
> >> >> > >
> >        org.apache.harmony.luni.tests.java.net.MulticastSocketTest
> >> >> (Failed
> >> >> > > > in windows_x86 running of M6)
> >> >> > > >        tests.api.java.security.PermissionCollectionTest
> >> >> (Failed
> >> >> > > > in M6)
> >> >> > > >
> >> >> > > >         are observed in Linux x86.
> >> >> > > >
> >> >> > > >  2) DRLVM tests:
> >> >> > > >    2.1) One failure
> >> >> > > >
> >> >> > > >         java.lang.ClassGenericsTest.test_2
> >> >> > > >
> >> >> > > >         is observed in Linux x86 snapshot testing.
> >> >> > > >         I can see some old discussion in the mailing list about
> >> that
> >> >> but
> >> >> > > I
> >> >> > > > am not sure the expected status here.
> >> >> > > >         They should be intemittent errors since the integrity
> >> testing
> >> >> > > just
> >> >> > > > run well mostly.
> >> >> > > >
> >> >> > > >  3) EUTs:
> >> >> > > >
> >> >> > > >    3.1) Linux x86: 99.36%
> >> >> > > >         A recorded JIRA for this suite is HARMONY-2914 which
> >> wastes
> >> >> file
> >> >> > > > handlers and makes system unstable.
> >> >> > > >
> >> >> > > >  4) Functional:
> >> >> > > >    4.1) Old regressions on both platforms:
> >> >> > > >         api.java.text.MessageFormat (HARMONY-5430)
> >> >> > > >         api.java.util.jar.Manifest  (HARMONY-5473)
> >> >> > > >         api.java.beans.beancontext.BeanContextTest (also in M6,
> >> >> recorded
> >> >> > > > as  regression caused by changes in locale data)
> >> >> > > >         api.java.beans.persistence.EncoderTest  (also in M6,
> >> recorded
> >> >> as
> >> >> > > > regression in the beans module)
> >> >> > > >         api.java.beans.persistence.EncoderDecoderTest (also in
> M6,
> >> >> > > > regression in the beans module)
> >> >> > > >         reg.vm.btest5625 (also in M6, recorded as intermittent
> and
> >> >> not
> >> >> > > > reproducible manually)
> >> >> > > >
> >> >> > > >    4.2) Old regressions on 1 platform
> >> >> > > >         api.java.rmi.basicexception (ERROR in Linux x86,
> >> >> HARMONY-5823)
> >> >> > > >         api.java.rmi.basicregistry.RemoteServerTest (ERROR in
> >> Linux
> >> >> x86,
> >> >> > > > HARMONY-5823)
> >> >> > > >         jpda.jdwp.scenario.ST07.ST07Test (ERROR in windows
> x86_64,
> >> in
> >> >> M6
> >> >> > > it
> >> >> > > > is recorded as regression since M4)
> >> >> > > >         java.math.F_BigIntegerMatrixMultiplyTest_01 (ERROR on
> >> Linux
> >> >> x86,
> >> >> > > > recorded as Timeout, not reproducible
> >> >> > > >  in M6)
> >> >> > > >         reg.vm.btest5717 (ERROR on Windows X86_64, recorded as
> >> >> "timeout,
> >> >> > > > the test is too heavy" in M6)
> >> >> > > >         jit.HLO.inline.ControlFlow.IfElse.IfElse1.IfElseTest1
> >> (FAILED
> >> >> in
> >> >> > > > windows x86_64, recorded as "looks like
> >> >> > > > issue in test" in M6)
> >> >> > > >         jit.HLO.devirt.Runtime.RuntimeExtend1 (FAILED on
> windows
> >> >> x86_64,
> >> >> > > in
> >> >> > > > M6 it is recorded as not regression and started to fail on M5)
> >> >> > > >         reg.vm.btest6353.Btest6353 (ERROR on Windows x86_64,
> >> recorded
> >> >> > > also
> >> >> > > > failed on M3 and might be issue with the test)
> >> >> > > >
> >> >> > > >    4.3) New regressions on 1 platform (looks intermittent)
> >> >> > > >         reg.jit.btest8029.Btest8029 (FAILED in Linux x86)
> >> >> > > >         func.reg.jit.btest5710.Btest5710 (FAILED in Linux x86)
> >> >> > > >
> api.java.security.cert.F_CertPathTest_06.F_CertPathTest_06
> >> >> (ERROR
> >> >> > > > in Linux x86)
> >> >> > > >
> api.java.security.cert.F_CertPathTest_05.F_CertPathTest_05
> >> >> (ERROR
> >> >> > > > in Linux x86)
> >> >> > > >
> >> >> > > >  5) JDKTools Tests:
> >> >> > > >    Several timeouts are observed in Linux x86 snapshot running.
> >> They
> >> >> are:
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > >
> >> >>
> >>
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch001
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > >
> >> >>
> >>
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch002
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > >
> >> >>
> >>
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch003
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > >
> >> >>
> >>
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch004
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > >
> >> >>
> >>
> org.apache.harmony.jpda.tests.jdwp.Events.CombinedEventsTest.testCombinedEvents_04
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > >
> >> >>
> >>
> org.apache.harmony.jpda.tests.jdwp.MultiSession.AttachConnectorTest.testAttachConnector001
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > >
> >> >>
> >>
> org.apache.harmony.jpda.tests.jdwp.MultiSession.MethodEntryExitTest.testMethodEvent001
> >> >> > > >
> >> >> > > >
> >> org.apache.harmony.jpda.tests.jdwp.MultiSession.ResumeTest.testResume
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > >
> >> >>
> >>
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadEndTest.testThreadEnd001
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > >
> >> >>
> >>
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadStartTest.testThreadStart001
> >> >> > > >    The Linux-only timeouts are also observed in the integrity
> >> testing
> >> >> > > > results.
> >> >> > > >    JIRA HARMONY-5833 has been filed for one of them.
> >> >> > > >
> >> >> > > >  6) JettyScenario:
> >> >> > > >    The Linux x86 running failed because of the unresolved
> >> >> HARMONY-5219.
> >> >> > > >
> >> >> > > >  7) Reliability:
> >> >> > > >    Several failures are observed in windows x86_64 running.
> >> >> > > >    7.1) Old regressions
> >> >> > > >         api.net.DatagramTest (HARMONY-5531)
> >> >> > > >         api.text.DecimalFormat_Locales - (in M6 it is recorded
> as
> >> >> also
> >> >> > > > intermittent in M5)
> >> >> > > >
> >> >> > > >    7.2) New/intemittent regressions
> >> >> > > >         api.kernel.thread.ThreadLocalTest.ThreadLocalTest
> >> >> > > >         api.kernel.exec.RunExec
> >> >> > > >
> >> >> > > >  8) Stress
> >> >> > > >    Different test cases failed on different platforms.
> >> >> > > >    8.1) Timeouts on Linux x86.
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > >
> >> >>
> >>
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD003.ThreadTest003
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > >
> >> >>
> >>
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD007.ThreadTest007
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > >
> >> >>
> >>
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD009.ThreadTest009
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > >
> >> >>
> >>
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD011.ThreadTest011
> >> >> > > >
> >> >> > > >    8.2) Failed cases on Windows x86_64 with unknown reasons.
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > >
> >> >>
> >>
> stress.org.apache.harmony.test.stress.classloader.MixThreads.TreeClasses.testTreeClasses2
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > >
> >> >>
> >>
> stress.org.apache.harmony.test.stress.classloader.NotSynchThreads.TreeClasses.testTreeClasses
> >> >> > > >
> >> >> > > >    I have not found records for them.
> >> >> > > >
> >> >> > > >  9) Strut_test
> >> >> > > >    Broken with same error report as M6.
> >> >> > > >
> >> >> > > >  10) Eclipse Hello World Application.
> >> >> > > >    Although the testing framework just reports EUT-API status
> as
> >> >> > > "PASSED".
> >> >> > > > A fresh workspace running just failed in configuration stage (
> >> >> > > >
> >> >> > > >
> >> >> > >
> >> >>
> >>
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805120106o387a49f1rfb2c33d1042d2f41@mail.gmail.com%3e
> >> >> > > > )
> >> >> > > > since SVN commit 641928 (which committed patch for
> HARMONY-4569).
> >> >> > > >
> >> >> > > > thanks,
> >> >> > > > chunrong
> >> >> > > >
> >> >> > >
> >> >>
> >> >
> >>
> >
>

Re: [testing] first M7 candidate (r681495) testing status

Posted by Nathan Beyer <nb...@gmail.com>.
I understand the concept, but running CruiseControl (a headless daemon
process) and a local X session are a bit at odds, aren't they?

I'm asking, in practical terms, how are you doing it? Start a local X
session, start an xterm, invoke cruise control and let it go? Is there
anyway to do this remotely? Direct access to a machine isn't always an
option and generally a remote client can't be permanently left up. Is
there anyway to do a 'nohup' with an X session?

-Nathan

On Wed, Aug 13, 2008 at 9:18 PM, chunrong lai <ch...@gmail.com> wrote:
>  I understand that CruiseControl does trigger the test suite running. But
> some test suites themselves (classlib, dacapo etc) do require GUI support.
> So in Linux one needs to run cruisecontrol ("ant run-cc") in an environment
> with X-connection (for example, in X, vnc-client) instead of text-mode
> client (putty, ssh, term etc).
>
>
>
> On 8/14/08, Nathan Beyer <nd...@apache.org> wrote:
>>
>> On Wed, Aug 13, 2008 at 8:58 PM, chunrong lai <ch...@gmail.com>
>> wrote:
>> > (1) M7 status is got in Fedora distros and M6/M5 status were got in SuSe
>> > Linux distros.
>> > (2) According to the stacktrace
>> > ( java.awt.Toolkit.init,  java.awt.SystemColor.<init>) of the
>> > java.beans.Encoder error (I have not observed such error reports in
>> previous
>> > logs),  I guess the problem is from GUI system initialization. Is
>> > X-connection available in your testing? In my understanding that is
>> required
>> > to run classlib test.
>>
>> It's running under CruiseControl, like the howto suggested, how do you
>> do an X-connection with that?
>>
>> >
>> >
>> > On 8/14/08, Nathan Beyer <nd...@apache.org> wrote:
>> >>
>> >> What Linux distro is it? I'm running Ubuntu and I'm seeing some
>> >> different issues.
>> >>
>> >> I'm getting this failure -
>> >>
>> >> java.lang.ExceptionInInitializerError
>> >> at java.lang.reflect.VMField.getObject(VMField.java)
>> >> at java.lang.reflect.Field.get(Field.java:163)
>> >> at
>> >>
>> java.beans.StaticFieldPersistenceDelegate.init(StaticFieldPersistenceDelegate.java:36)
>> >> at
>> >>
>> java.beans.AwtSystemColorPersistenceDelegate.<clinit>(AwtSystemColorPersistenceDelegate.java:28)
>> >> at java.beans.Encoder.<clinit>(Encoder.java:124)
>> >> at
>> >>
>> java.beans.ArrayPersistenceDelegateTest.testInstantiate_Normal(ArrayPersistenceDelegateTest.java:130)
>> >> at java.lang.reflect.VMReflection.invokeMethod(VMReflection.java)
>> >> Caused by: java.lang.NullPointerException
>> >> at java.awt.Toolkit.init(Toolkit.java:566)
>> >> at java.awt.Toolkit.<init>(Toolkit.java:540)
>> >> at java.awt.ToolkitImpl.<init>(ToolkitImpl.java:40)
>> >> at java.awt.Toolkit.getDefaultToolkit(Toolkit.java:472)
>> >> at java.awt.SystemColor.<init>(SystemColor.java:143)
>> >> at java.awt.SystemColor.<clinit>(SystemColor.java:91)
>> >> at java.lang.reflect.VMField.getObject(VMField.java)
>> >> ... 18 more
>> >>
>> >> Then a bunch of NoClassDefFoundErrors for java.beans.Encoder.
>> >>
>> >>
>> >> On Wed, Aug 13, 2008 at 3:21 AM, chunrong lai <ch...@gmail.com>
>> >> wrote:
>> >> >
>> >> > hi, Nathan:
>> >> >     It is also noted that currently we do setup Linux x86-64 integrity
>> >> > testing, which eventually test every SVN revision, with results
>> reported
>> >> in
>> >> > http://people.apache.org/~chunrong/harmony-integrity/. Your mentioned
>> >> test
>> >> > suites classib-test,drlvm-test are included there. (classlib+drlvm
>> >> > are almost same as the HDK build reported in the page).
>> >> >     We only use two platforms currently for the snapshot testing
>> cycle,
>> >> > (table is generated by
>> >> >
>> >>
>> http://svn.apache.org/viewvc/harmony/enhanced/buildtest/trunk/tools/snapshot/snapshots-table/
>> >> )
>> >> > , to test more test suites with longer time for every snapshot. In my
>> >> > observation adding two other platforms here just does not necessarily
>> add
>> >> > enough value in current stage of Harmony.
>> >> >     I do think more testings/discussions help to improve Harmony
>> quality.
>> >> > Thanks for the dedication. If there are more I can help please let me
>> >> know.
>> >> >
>> >> > On 8/12/08, Nathan Beyer <nd...@apache.org> wrote:
>> >> > >
>> >> > > What's being used to produce the integrity tests? I have an x86_64
>> >> Linux
>> >> > > box
>> >> > > that I can dedicate to testing. I've tried just using buildtest with
>> >> > > 'classlib,drlvm,classib-test,drlvm-test', but I haven't gotten a
>> clean
>> >> pass
>> >> > > yet.
>> >> > >
>> >> > > -Nathan
>> >> > >
>> >> > > On Mon, Aug 11, 2008 at 12:26 PM, chunrong lai <
>> chunronglai@gmail.com
>> >> > > >wrote:
>> >> > >
>> >> > > > Hi, all:
>> >> > > >
>> >> > > >  Here is r681495 snapshot testing status:
>> >> > > > http://people.apache.org/~chunrong/snapshots/r681495/index.html<
>> >> > > http://people.apache.org/%7Echunrong/snapshots/r681495/index.html>.
>> >> > > > I am using
>> >> > > > two platforms (Linux x86, windows x86_64) at the moment. Hopefully
>> we
>> >> > > will
>> >> > > > have other two platforms in future for M8. Well, although we are
>> >> testing
>> >> > > > only two platforms for M7, my experience is that the status for
>> >> another
>> >> > > two
>> >> > > > platforms should be not worse or just include some extra
>> intermittent
>> >> > > > errors
>> >> > > > which can be investigated in some later stages.
>> >> > > >
>> >> > > >  The following suites passed on Linux x86/Windows x86_64
>> platforms:
>> >> Ant
>> >> > > > Scenario (or self-hosting), Axis application, Dacapo, DRLVM
>> >> regression
>> >> > > > tests, Geronimo Unit Tests, Scimark, Tomcat scenario, VTS VM Test
>> >> Suite.
>> >> > > >
>> >> > > >  Most of the failures are known issues (for M6). Although we can
>> >> observe
>> >> > > > 15~20 new issues, those issues happen only in 1 platform and they
>> >> look
>> >> > > more
>> >> > > > like the intermittent/timeout issues (less reproducible) to me.
>> >> Overall I
>> >> > > > myself would like to think r681495 is more stable than M6.
>> >> > > >
>> >> > > >  Please add your comments and clarifications (please also see M6
>> >> testing
>> >> > > > results http://people.apache.org/~smishura/r653525/<
>> >> > > http://people.apache.org/%7Esmishura/r653525/>
>> >> > > > ,
>> >> > > >
>> >> > > >
>> >> > >
>> >>
>> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805070304l38845ee0se01fb93fbfc05586@mail.gmail.com%3eand
>> >> > > > integrity testing results
>> >> > > > http://people.apache.org/~chunrong/harmony-integrity/<
>> >> > > http://people.apache.org/%7Echunrong/harmony-integrity/>as a
>> >> comparison).
>> >> > > >
>> >> > > >  1) Classlib:
>> >> > > >    1.1) Since r644719 (which committed patch for HARMONY-5684)
>> >> > > >
>> >> > > >        org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > >
>> >>
>> org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
>> >> > > >
>> >> > > >         failed in both platforms
>> >> > > >
>> >> > > >    1.2) Two failures
>> >> > > >
>> >> > > >        org.apache.harmony.luni.tests.java.net.MulticastSocketTest
>> >> (Failed
>> >> > > > in windows_x86 running of M6)
>> >> > > >        tests.api.java.security.PermissionCollectionTest
>> >> (Failed
>> >> > > > in M6)
>> >> > > >
>> >> > > >         are observed in Linux x86.
>> >> > > >
>> >> > > >  2) DRLVM tests:
>> >> > > >    2.1) One failure
>> >> > > >
>> >> > > >         java.lang.ClassGenericsTest.test_2
>> >> > > >
>> >> > > >         is observed in Linux x86 snapshot testing.
>> >> > > >         I can see some old discussion in the mailing list about
>> that
>> >> but
>> >> > > I
>> >> > > > am not sure the expected status here.
>> >> > > >         They should be intemittent errors since the integrity
>> testing
>> >> > > just
>> >> > > > run well mostly.
>> >> > > >
>> >> > > >  3) EUTs:
>> >> > > >
>> >> > > >    3.1) Linux x86: 99.36%
>> >> > > >         A recorded JIRA for this suite is HARMONY-2914 which
>> wastes
>> >> file
>> >> > > > handlers and makes system unstable.
>> >> > > >
>> >> > > >  4) Functional:
>> >> > > >    4.1) Old regressions on both platforms:
>> >> > > >         api.java.text.MessageFormat (HARMONY-5430)
>> >> > > >         api.java.util.jar.Manifest  (HARMONY-5473)
>> >> > > >         api.java.beans.beancontext.BeanContextTest (also in M6,
>> >> recorded
>> >> > > > as  regression caused by changes in locale data)
>> >> > > >         api.java.beans.persistence.EncoderTest  (also in M6,
>> recorded
>> >> as
>> >> > > > regression in the beans module)
>> >> > > >         api.java.beans.persistence.EncoderDecoderTest (also in M6,
>> >> > > > regression in the beans module)
>> >> > > >         reg.vm.btest5625 (also in M6, recorded as intermittent and
>> >> not
>> >> > > > reproducible manually)
>> >> > > >
>> >> > > >    4.2) Old regressions on 1 platform
>> >> > > >         api.java.rmi.basicexception (ERROR in Linux x86,
>> >> HARMONY-5823)
>> >> > > >         api.java.rmi.basicregistry.RemoteServerTest (ERROR in
>> Linux
>> >> x86,
>> >> > > > HARMONY-5823)
>> >> > > >         jpda.jdwp.scenario.ST07.ST07Test (ERROR in windows x86_64,
>> in
>> >> M6
>> >> > > it
>> >> > > > is recorded as regression since M4)
>> >> > > >         java.math.F_BigIntegerMatrixMultiplyTest_01 (ERROR on
>> Linux
>> >> x86,
>> >> > > > recorded as Timeout, not reproducible
>> >> > > >  in M6)
>> >> > > >         reg.vm.btest5717 (ERROR on Windows X86_64, recorded as
>> >> "timeout,
>> >> > > > the test is too heavy" in M6)
>> >> > > >         jit.HLO.inline.ControlFlow.IfElse.IfElse1.IfElseTest1
>> (FAILED
>> >> in
>> >> > > > windows x86_64, recorded as "looks like
>> >> > > > issue in test" in M6)
>> >> > > >         jit.HLO.devirt.Runtime.RuntimeExtend1 (FAILED on windows
>> >> x86_64,
>> >> > > in
>> >> > > > M6 it is recorded as not regression and started to fail on M5)
>> >> > > >         reg.vm.btest6353.Btest6353 (ERROR on Windows x86_64,
>> recorded
>> >> > > also
>> >> > > > failed on M3 and might be issue with the test)
>> >> > > >
>> >> > > >    4.3) New regressions on 1 platform (looks intermittent)
>> >> > > >         reg.jit.btest8029.Btest8029 (FAILED in Linux x86)
>> >> > > >         func.reg.jit.btest5710.Btest5710 (FAILED in Linux x86)
>> >> > > >         api.java.security.cert.F_CertPathTest_06.F_CertPathTest_06
>> >> (ERROR
>> >> > > > in Linux x86)
>> >> > > >         api.java.security.cert.F_CertPathTest_05.F_CertPathTest_05
>> >> (ERROR
>> >> > > > in Linux x86)
>> >> > > >
>> >> > > >  5) JDKTools Tests:
>> >> > > >    Several timeouts are observed in Linux x86 snapshot running.
>> They
>> >> are:
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > >
>> >>
>> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch001
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > >
>> >>
>> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch002
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > >
>> >>
>> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch003
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > >
>> >>
>> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch004
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > >
>> >>
>> org.apache.harmony.jpda.tests.jdwp.Events.CombinedEventsTest.testCombinedEvents_04
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > >
>> >>
>> org.apache.harmony.jpda.tests.jdwp.MultiSession.AttachConnectorTest.testAttachConnector001
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > >
>> >>
>> org.apache.harmony.jpda.tests.jdwp.MultiSession.MethodEntryExitTest.testMethodEvent001
>> >> > > >
>> >> > > >
>> org.apache.harmony.jpda.tests.jdwp.MultiSession.ResumeTest.testResume
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > >
>> >>
>> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadEndTest.testThreadEnd001
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > >
>> >>
>> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadStartTest.testThreadStart001
>> >> > > >    The Linux-only timeouts are also observed in the integrity
>> testing
>> >> > > > results.
>> >> > > >    JIRA HARMONY-5833 has been filed for one of them.
>> >> > > >
>> >> > > >  6) JettyScenario:
>> >> > > >    The Linux x86 running failed because of the unresolved
>> >> HARMONY-5219.
>> >> > > >
>> >> > > >  7) Reliability:
>> >> > > >    Several failures are observed in windows x86_64 running.
>> >> > > >    7.1) Old regressions
>> >> > > >         api.net.DatagramTest (HARMONY-5531)
>> >> > > >         api.text.DecimalFormat_Locales - (in M6 it is recorded as
>> >> also
>> >> > > > intermittent in M5)
>> >> > > >
>> >> > > >    7.2) New/intemittent regressions
>> >> > > >         api.kernel.thread.ThreadLocalTest.ThreadLocalTest
>> >> > > >         api.kernel.exec.RunExec
>> >> > > >
>> >> > > >  8) Stress
>> >> > > >    Different test cases failed on different platforms.
>> >> > > >    8.1) Timeouts on Linux x86.
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > >
>> >>
>> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD003.ThreadTest003
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > >
>> >>
>> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD007.ThreadTest007
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > >
>> >>
>> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD009.ThreadTest009
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > >
>> >>
>> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD011.ThreadTest011
>> >> > > >
>> >> > > >    8.2) Failed cases on Windows x86_64 with unknown reasons.
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > >
>> >>
>> stress.org.apache.harmony.test.stress.classloader.MixThreads.TreeClasses.testTreeClasses2
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > >
>> >>
>> stress.org.apache.harmony.test.stress.classloader.NotSynchThreads.TreeClasses.testTreeClasses
>> >> > > >
>> >> > > >    I have not found records for them.
>> >> > > >
>> >> > > >  9) Strut_test
>> >> > > >    Broken with same error report as M6.
>> >> > > >
>> >> > > >  10) Eclipse Hello World Application.
>> >> > > >    Although the testing framework just reports EUT-API status as
>> >> > > "PASSED".
>> >> > > > A fresh workspace running just failed in configuration stage (
>> >> > > >
>> >> > > >
>> >> > >
>> >>
>> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805120106o387a49f1rfb2c33d1042d2f41@mail.gmail.com%3e
>> >> > > > )
>> >> > > > since SVN commit 641928 (which committed patch for HARMONY-4569).
>> >> > > >
>> >> > > > thanks,
>> >> > > > chunrong
>> >> > > >
>> >> > >
>> >>
>> >
>>
>

Re: [testing] first M7 candidate (r681495) testing status

Posted by chunrong lai <ch...@gmail.com>.
 I understand that CruiseControl does trigger the test suite running. But
some test suites themselves (classlib, dacapo etc) do require GUI support.
So in Linux one needs to run cruisecontrol ("ant run-cc") in an environment
with X-connection (for example, in X, vnc-client) instead of text-mode
client (putty, ssh, term etc).



On 8/14/08, Nathan Beyer <nd...@apache.org> wrote:
>
> On Wed, Aug 13, 2008 at 8:58 PM, chunrong lai <ch...@gmail.com>
> wrote:
> > (1) M7 status is got in Fedora distros and M6/M5 status were got in SuSe
> > Linux distros.
> > (2) According to the stacktrace
> > ( java.awt.Toolkit.init,  java.awt.SystemColor.<init>) of the
> > java.beans.Encoder error (I have not observed such error reports in
> previous
> > logs),  I guess the problem is from GUI system initialization. Is
> > X-connection available in your testing? In my understanding that is
> required
> > to run classlib test.
>
> It's running under CruiseControl, like the howto suggested, how do you
> do an X-connection with that?
>
> >
> >
> > On 8/14/08, Nathan Beyer <nd...@apache.org> wrote:
> >>
> >> What Linux distro is it? I'm running Ubuntu and I'm seeing some
> >> different issues.
> >>
> >> I'm getting this failure -
> >>
> >> java.lang.ExceptionInInitializerError
> >> at java.lang.reflect.VMField.getObject(VMField.java)
> >> at java.lang.reflect.Field.get(Field.java:163)
> >> at
> >>
> java.beans.StaticFieldPersistenceDelegate.init(StaticFieldPersistenceDelegate.java:36)
> >> at
> >>
> java.beans.AwtSystemColorPersistenceDelegate.<clinit>(AwtSystemColorPersistenceDelegate.java:28)
> >> at java.beans.Encoder.<clinit>(Encoder.java:124)
> >> at
> >>
> java.beans.ArrayPersistenceDelegateTest.testInstantiate_Normal(ArrayPersistenceDelegateTest.java:130)
> >> at java.lang.reflect.VMReflection.invokeMethod(VMReflection.java)
> >> Caused by: java.lang.NullPointerException
> >> at java.awt.Toolkit.init(Toolkit.java:566)
> >> at java.awt.Toolkit.<init>(Toolkit.java:540)
> >> at java.awt.ToolkitImpl.<init>(ToolkitImpl.java:40)
> >> at java.awt.Toolkit.getDefaultToolkit(Toolkit.java:472)
> >> at java.awt.SystemColor.<init>(SystemColor.java:143)
> >> at java.awt.SystemColor.<clinit>(SystemColor.java:91)
> >> at java.lang.reflect.VMField.getObject(VMField.java)
> >> ... 18 more
> >>
> >> Then a bunch of NoClassDefFoundErrors for java.beans.Encoder.
> >>
> >>
> >> On Wed, Aug 13, 2008 at 3:21 AM, chunrong lai <ch...@gmail.com>
> >> wrote:
> >> >
> >> > hi, Nathan:
> >> >     It is also noted that currently we do setup Linux x86-64 integrity
> >> > testing, which eventually test every SVN revision, with results
> reported
> >> in
> >> > http://people.apache.org/~chunrong/harmony-integrity/. Your mentioned
> >> test
> >> > suites classib-test,drlvm-test are included there. (classlib+drlvm
> >> > are almost same as the HDK build reported in the page).
> >> >     We only use two platforms currently for the snapshot testing
> cycle,
> >> > (table is generated by
> >> >
> >>
> http://svn.apache.org/viewvc/harmony/enhanced/buildtest/trunk/tools/snapshot/snapshots-table/
> >> )
> >> > , to test more test suites with longer time for every snapshot. In my
> >> > observation adding two other platforms here just does not necessarily
> add
> >> > enough value in current stage of Harmony.
> >> >     I do think more testings/discussions help to improve Harmony
> quality.
> >> > Thanks for the dedication. If there are more I can help please let me
> >> know.
> >> >
> >> > On 8/12/08, Nathan Beyer <nd...@apache.org> wrote:
> >> > >
> >> > > What's being used to produce the integrity tests? I have an x86_64
> >> Linux
> >> > > box
> >> > > that I can dedicate to testing. I've tried just using buildtest with
> >> > > 'classlib,drlvm,classib-test,drlvm-test', but I haven't gotten a
> clean
> >> pass
> >> > > yet.
> >> > >
> >> > > -Nathan
> >> > >
> >> > > On Mon, Aug 11, 2008 at 12:26 PM, chunrong lai <
> chunronglai@gmail.com
> >> > > >wrote:
> >> > >
> >> > > > Hi, all:
> >> > > >
> >> > > >  Here is r681495 snapshot testing status:
> >> > > > http://people.apache.org/~chunrong/snapshots/r681495/index.html<
> >> > > http://people.apache.org/%7Echunrong/snapshots/r681495/index.html>.
> >> > > > I am using
> >> > > > two platforms (Linux x86, windows x86_64) at the moment. Hopefully
> we
> >> > > will
> >> > > > have other two platforms in future for M8. Well, although we are
> >> testing
> >> > > > only two platforms for M7, my experience is that the status for
> >> another
> >> > > two
> >> > > > platforms should be not worse or just include some extra
> intermittent
> >> > > > errors
> >> > > > which can be investigated in some later stages.
> >> > > >
> >> > > >  The following suites passed on Linux x86/Windows x86_64
> platforms:
> >> Ant
> >> > > > Scenario (or self-hosting), Axis application, Dacapo, DRLVM
> >> regression
> >> > > > tests, Geronimo Unit Tests, Scimark, Tomcat scenario, VTS VM Test
> >> Suite.
> >> > > >
> >> > > >  Most of the failures are known issues (for M6). Although we can
> >> observe
> >> > > > 15~20 new issues, those issues happen only in 1 platform and they
> >> look
> >> > > more
> >> > > > like the intermittent/timeout issues (less reproducible) to me.
> >> Overall I
> >> > > > myself would like to think r681495 is more stable than M6.
> >> > > >
> >> > > >  Please add your comments and clarifications (please also see M6
> >> testing
> >> > > > results http://people.apache.org/~smishura/r653525/<
> >> > > http://people.apache.org/%7Esmishura/r653525/>
> >> > > > ,
> >> > > >
> >> > > >
> >> > >
> >>
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805070304l38845ee0se01fb93fbfc05586@mail.gmail.com%3eand
> >> > > > integrity testing results
> >> > > > http://people.apache.org/~chunrong/harmony-integrity/<
> >> > > http://people.apache.org/%7Echunrong/harmony-integrity/>as a
> >> comparison).
> >> > > >
> >> > > >  1) Classlib:
> >> > > >    1.1) Since r644719 (which committed patch for HARMONY-5684)
> >> > > >
> >> > > >        org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >>
> org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
> >> > > >
> >> > > >         failed in both platforms
> >> > > >
> >> > > >    1.2) Two failures
> >> > > >
> >> > > >        org.apache.harmony.luni.tests.java.net.MulticastSocketTest
> >> (Failed
> >> > > > in windows_x86 running of M6)
> >> > > >        tests.api.java.security.PermissionCollectionTest
> >> (Failed
> >> > > > in M6)
> >> > > >
> >> > > >         are observed in Linux x86.
> >> > > >
> >> > > >  2) DRLVM tests:
> >> > > >    2.1) One failure
> >> > > >
> >> > > >         java.lang.ClassGenericsTest.test_2
> >> > > >
> >> > > >         is observed in Linux x86 snapshot testing.
> >> > > >         I can see some old discussion in the mailing list about
> that
> >> but
> >> > > I
> >> > > > am not sure the expected status here.
> >> > > >         They should be intemittent errors since the integrity
> testing
> >> > > just
> >> > > > run well mostly.
> >> > > >
> >> > > >  3) EUTs:
> >> > > >
> >> > > >    3.1) Linux x86: 99.36%
> >> > > >         A recorded JIRA for this suite is HARMONY-2914 which
> wastes
> >> file
> >> > > > handlers and makes system unstable.
> >> > > >
> >> > > >  4) Functional:
> >> > > >    4.1) Old regressions on both platforms:
> >> > > >         api.java.text.MessageFormat (HARMONY-5430)
> >> > > >         api.java.util.jar.Manifest  (HARMONY-5473)
> >> > > >         api.java.beans.beancontext.BeanContextTest (also in M6,
> >> recorded
> >> > > > as  regression caused by changes in locale data)
> >> > > >         api.java.beans.persistence.EncoderTest  (also in M6,
> recorded
> >> as
> >> > > > regression in the beans module)
> >> > > >         api.java.beans.persistence.EncoderDecoderTest (also in M6,
> >> > > > regression in the beans module)
> >> > > >         reg.vm.btest5625 (also in M6, recorded as intermittent and
> >> not
> >> > > > reproducible manually)
> >> > > >
> >> > > >    4.2) Old regressions on 1 platform
> >> > > >         api.java.rmi.basicexception (ERROR in Linux x86,
> >> HARMONY-5823)
> >> > > >         api.java.rmi.basicregistry.RemoteServerTest (ERROR in
> Linux
> >> x86,
> >> > > > HARMONY-5823)
> >> > > >         jpda.jdwp.scenario.ST07.ST07Test (ERROR in windows x86_64,
> in
> >> M6
> >> > > it
> >> > > > is recorded as regression since M4)
> >> > > >         java.math.F_BigIntegerMatrixMultiplyTest_01 (ERROR on
> Linux
> >> x86,
> >> > > > recorded as Timeout, not reproducible
> >> > > >  in M6)
> >> > > >         reg.vm.btest5717 (ERROR on Windows X86_64, recorded as
> >> "timeout,
> >> > > > the test is too heavy" in M6)
> >> > > >         jit.HLO.inline.ControlFlow.IfElse.IfElse1.IfElseTest1
> (FAILED
> >> in
> >> > > > windows x86_64, recorded as "looks like
> >> > > > issue in test" in M6)
> >> > > >         jit.HLO.devirt.Runtime.RuntimeExtend1 (FAILED on windows
> >> x86_64,
> >> > > in
> >> > > > M6 it is recorded as not regression and started to fail on M5)
> >> > > >         reg.vm.btest6353.Btest6353 (ERROR on Windows x86_64,
> recorded
> >> > > also
> >> > > > failed on M3 and might be issue with the test)
> >> > > >
> >> > > >    4.3) New regressions on 1 platform (looks intermittent)
> >> > > >         reg.jit.btest8029.Btest8029 (FAILED in Linux x86)
> >> > > >         func.reg.jit.btest5710.Btest5710 (FAILED in Linux x86)
> >> > > >         api.java.security.cert.F_CertPathTest_06.F_CertPathTest_06
> >> (ERROR
> >> > > > in Linux x86)
> >> > > >         api.java.security.cert.F_CertPathTest_05.F_CertPathTest_05
> >> (ERROR
> >> > > > in Linux x86)
> >> > > >
> >> > > >  5) JDKTools Tests:
> >> > > >    Several timeouts are observed in Linux x86 snapshot running.
> They
> >> are:
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >>
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch001
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >>
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch002
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >>
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch003
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >>
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch004
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >>
> org.apache.harmony.jpda.tests.jdwp.Events.CombinedEventsTest.testCombinedEvents_04
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >>
> org.apache.harmony.jpda.tests.jdwp.MultiSession.AttachConnectorTest.testAttachConnector001
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >>
> org.apache.harmony.jpda.tests.jdwp.MultiSession.MethodEntryExitTest.testMethodEvent001
> >> > > >
> >> > > >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ResumeTest.testResume
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >>
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadEndTest.testThreadEnd001
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >>
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadStartTest.testThreadStart001
> >> > > >    The Linux-only timeouts are also observed in the integrity
> testing
> >> > > > results.
> >> > > >    JIRA HARMONY-5833 has been filed for one of them.
> >> > > >
> >> > > >  6) JettyScenario:
> >> > > >    The Linux x86 running failed because of the unresolved
> >> HARMONY-5219.
> >> > > >
> >> > > >  7) Reliability:
> >> > > >    Several failures are observed in windows x86_64 running.
> >> > > >    7.1) Old regressions
> >> > > >         api.net.DatagramTest (HARMONY-5531)
> >> > > >         api.text.DecimalFormat_Locales - (in M6 it is recorded as
> >> also
> >> > > > intermittent in M5)
> >> > > >
> >> > > >    7.2) New/intemittent regressions
> >> > > >         api.kernel.thread.ThreadLocalTest.ThreadLocalTest
> >> > > >         api.kernel.exec.RunExec
> >> > > >
> >> > > >  8) Stress
> >> > > >    Different test cases failed on different platforms.
> >> > > >    8.1) Timeouts on Linux x86.
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >>
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD003.ThreadTest003
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >>
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD007.ThreadTest007
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >>
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD009.ThreadTest009
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >>
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD011.ThreadTest011
> >> > > >
> >> > > >    8.2) Failed cases on Windows x86_64 with unknown reasons.
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >>
> stress.org.apache.harmony.test.stress.classloader.MixThreads.TreeClasses.testTreeClasses2
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >>
> stress.org.apache.harmony.test.stress.classloader.NotSynchThreads.TreeClasses.testTreeClasses
> >> > > >
> >> > > >    I have not found records for them.
> >> > > >
> >> > > >  9) Strut_test
> >> > > >    Broken with same error report as M6.
> >> > > >
> >> > > >  10) Eclipse Hello World Application.
> >> > > >    Although the testing framework just reports EUT-API status as
> >> > > "PASSED".
> >> > > > A fresh workspace running just failed in configuration stage (
> >> > > >
> >> > > >
> >> > >
> >>
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805120106o387a49f1rfb2c33d1042d2f41@mail.gmail.com%3e
> >> > > > )
> >> > > > since SVN commit 641928 (which committed patch for HARMONY-4569).
> >> > > >
> >> > > > thanks,
> >> > > > chunrong
> >> > > >
> >> > >
> >>
> >
>

Re: [testing] first M7 candidate (r681495) testing status

Posted by Nathan Beyer <nd...@apache.org>.
Brilliant! That does help, wonderfully.

On Thu, Aug 14, 2008 at 2:23 AM, Mark Hindess
<ma...@googlemail.com> wrote:
>
> In message <3b...@mail.gmail.com>,
> "Nathan Beyer" writes:
>>
>> It's running under CruiseControl, like the howto suggested, how do you
>> do an X-connection with that?
>
> Nathan, I tend to use the X virtual frame buffer server when testing.
> So something like:
>
>  # pick a display that isn't in use :0 or :1 if :0 is already active
>  export DISPLAY=:0
>  Xvfb $DISPLAY &
>  ant run-cc
>
> HTH,
> -Mark
>
>

Re: [testing] first M7 candidate (r681495) testing status

Posted by Mark Hindess <ma...@googlemail.com>.
In message <3b...@mail.gmail.com>,
"Nathan Beyer" writes:
>
> It's running under CruiseControl, like the howto suggested, how do you
> do an X-connection with that?

Nathan, I tend to use the X virtual frame buffer server when testing.
So something like:

  # pick a display that isn't in use :0 or :1 if :0 is already active
  export DISPLAY=:0
  Xvfb $DISPLAY &
  ant run-cc

HTH,
-Mark


Re: [testing] first M7 candidate (r681495) testing status

Posted by Nathan Beyer <nd...@apache.org>.
On Wed, Aug 13, 2008 at 8:58 PM, chunrong lai <ch...@gmail.com> wrote:
> (1) M7 status is got in Fedora distros and M6/M5 status were got in SuSe
> Linux distros.
> (2) According to the stacktrace
> ( java.awt.Toolkit.init,  java.awt.SystemColor.<init>) of the
> java.beans.Encoder error (I have not observed such error reports in previous
> logs),  I guess the problem is from GUI system initialization. Is
> X-connection available in your testing? In my understanding that is required
> to run classlib test.

It's running under CruiseControl, like the howto suggested, how do you
do an X-connection with that?

>
>
> On 8/14/08, Nathan Beyer <nd...@apache.org> wrote:
>>
>> What Linux distro is it? I'm running Ubuntu and I'm seeing some
>> different issues.
>>
>> I'm getting this failure -
>>
>> java.lang.ExceptionInInitializerError
>> at java.lang.reflect.VMField.getObject(VMField.java)
>> at java.lang.reflect.Field.get(Field.java:163)
>> at
>> java.beans.StaticFieldPersistenceDelegate.init(StaticFieldPersistenceDelegate.java:36)
>> at
>> java.beans.AwtSystemColorPersistenceDelegate.<clinit>(AwtSystemColorPersistenceDelegate.java:28)
>> at java.beans.Encoder.<clinit>(Encoder.java:124)
>> at
>> java.beans.ArrayPersistenceDelegateTest.testInstantiate_Normal(ArrayPersistenceDelegateTest.java:130)
>> at java.lang.reflect.VMReflection.invokeMethod(VMReflection.java)
>> Caused by: java.lang.NullPointerException
>> at java.awt.Toolkit.init(Toolkit.java:566)
>> at java.awt.Toolkit.<init>(Toolkit.java:540)
>> at java.awt.ToolkitImpl.<init>(ToolkitImpl.java:40)
>> at java.awt.Toolkit.getDefaultToolkit(Toolkit.java:472)
>> at java.awt.SystemColor.<init>(SystemColor.java:143)
>> at java.awt.SystemColor.<clinit>(SystemColor.java:91)
>> at java.lang.reflect.VMField.getObject(VMField.java)
>> ... 18 more
>>
>> Then a bunch of NoClassDefFoundErrors for java.beans.Encoder.
>>
>>
>> On Wed, Aug 13, 2008 at 3:21 AM, chunrong lai <ch...@gmail.com>
>> wrote:
>> >
>> > hi, Nathan:
>> >     It is also noted that currently we do setup Linux x86-64 integrity
>> > testing, which eventually test every SVN revision, with results reported
>> in
>> > http://people.apache.org/~chunrong/harmony-integrity/. Your mentioned
>> test
>> > suites classib-test,drlvm-test are included there. (classlib+drlvm
>> > are almost same as the HDK build reported in the page).
>> >     We only use two platforms currently for the snapshot testing cycle,
>> > (table is generated by
>> >
>> http://svn.apache.org/viewvc/harmony/enhanced/buildtest/trunk/tools/snapshot/snapshots-table/
>> )
>> > , to test more test suites with longer time for every snapshot. In my
>> > observation adding two other platforms here just does not necessarily add
>> > enough value in current stage of Harmony.
>> >     I do think more testings/discussions help to improve Harmony quality.
>> > Thanks for the dedication. If there are more I can help please let me
>> know.
>> >
>> > On 8/12/08, Nathan Beyer <nd...@apache.org> wrote:
>> > >
>> > > What's being used to produce the integrity tests? I have an x86_64
>> Linux
>> > > box
>> > > that I can dedicate to testing. I've tried just using buildtest with
>> > > 'classlib,drlvm,classib-test,drlvm-test', but I haven't gotten a clean
>> pass
>> > > yet.
>> > >
>> > > -Nathan
>> > >
>> > > On Mon, Aug 11, 2008 at 12:26 PM, chunrong lai <chunronglai@gmail.com
>> > > >wrote:
>> > >
>> > > > Hi, all:
>> > > >
>> > > >  Here is r681495 snapshot testing status:
>> > > > http://people.apache.org/~chunrong/snapshots/r681495/index.html<
>> > > http://people.apache.org/%7Echunrong/snapshots/r681495/index.html>.
>> > > > I am using
>> > > > two platforms (Linux x86, windows x86_64) at the moment. Hopefully we
>> > > will
>> > > > have other two platforms in future for M8. Well, although we are
>> testing
>> > > > only two platforms for M7, my experience is that the status for
>> another
>> > > two
>> > > > platforms should be not worse or just include some extra intermittent
>> > > > errors
>> > > > which can be investigated in some later stages.
>> > > >
>> > > >  The following suites passed on Linux x86/Windows x86_64 platforms:
>> Ant
>> > > > Scenario (or self-hosting), Axis application, Dacapo, DRLVM
>> regression
>> > > > tests, Geronimo Unit Tests, Scimark, Tomcat scenario, VTS VM Test
>> Suite.
>> > > >
>> > > >  Most of the failures are known issues (for M6). Although we can
>> observe
>> > > > 15~20 new issues, those issues happen only in 1 platform and they
>> look
>> > > more
>> > > > like the intermittent/timeout issues (less reproducible) to me.
>> Overall I
>> > > > myself would like to think r681495 is more stable than M6.
>> > > >
>> > > >  Please add your comments and clarifications (please also see M6
>> testing
>> > > > results http://people.apache.org/~smishura/r653525/<
>> > > http://people.apache.org/%7Esmishura/r653525/>
>> > > > ,
>> > > >
>> > > >
>> > >
>> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805070304l38845ee0se01fb93fbfc05586@mail.gmail.com%3eand
>> > > > integrity testing results
>> > > > http://people.apache.org/~chunrong/harmony-integrity/<
>> > > http://people.apache.org/%7Echunrong/harmony-integrity/>as a
>> comparison).
>> > > >
>> > > >  1) Classlib:
>> > > >    1.1) Since r644719 (which committed patch for HARMONY-5684)
>> > > >
>> > > >        org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest
>> > > >
>> > > >
>> > > >
>> > >
>> org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
>> > > >
>> > > >         failed in both platforms
>> > > >
>> > > >    1.2) Two failures
>> > > >
>> > > >        org.apache.harmony.luni.tests.java.net.MulticastSocketTest
>> (Failed
>> > > > in windows_x86 running of M6)
>> > > >        tests.api.java.security.PermissionCollectionTest
>> (Failed
>> > > > in M6)
>> > > >
>> > > >         are observed in Linux x86.
>> > > >
>> > > >  2) DRLVM tests:
>> > > >    2.1) One failure
>> > > >
>> > > >         java.lang.ClassGenericsTest.test_2
>> > > >
>> > > >         is observed in Linux x86 snapshot testing.
>> > > >         I can see some old discussion in the mailing list about that
>> but
>> > > I
>> > > > am not sure the expected status here.
>> > > >         They should be intemittent errors since the integrity testing
>> > > just
>> > > > run well mostly.
>> > > >
>> > > >  3) EUTs:
>> > > >
>> > > >    3.1) Linux x86: 99.36%
>> > > >         A recorded JIRA for this suite is HARMONY-2914 which wastes
>> file
>> > > > handlers and makes system unstable.
>> > > >
>> > > >  4) Functional:
>> > > >    4.1) Old regressions on both platforms:
>> > > >         api.java.text.MessageFormat (HARMONY-5430)
>> > > >         api.java.util.jar.Manifest  (HARMONY-5473)
>> > > >         api.java.beans.beancontext.BeanContextTest (also in M6,
>> recorded
>> > > > as  regression caused by changes in locale data)
>> > > >         api.java.beans.persistence.EncoderTest  (also in M6, recorded
>> as
>> > > > regression in the beans module)
>> > > >         api.java.beans.persistence.EncoderDecoderTest (also in M6,
>> > > > regression in the beans module)
>> > > >         reg.vm.btest5625 (also in M6, recorded as intermittent and
>> not
>> > > > reproducible manually)
>> > > >
>> > > >    4.2) Old regressions on 1 platform
>> > > >         api.java.rmi.basicexception (ERROR in Linux x86,
>> HARMONY-5823)
>> > > >         api.java.rmi.basicregistry.RemoteServerTest (ERROR in Linux
>> x86,
>> > > > HARMONY-5823)
>> > > >         jpda.jdwp.scenario.ST07.ST07Test (ERROR in windows x86_64, in
>> M6
>> > > it
>> > > > is recorded as regression since M4)
>> > > >         java.math.F_BigIntegerMatrixMultiplyTest_01 (ERROR on Linux
>> x86,
>> > > > recorded as Timeout, not reproducible
>> > > >  in M6)
>> > > >         reg.vm.btest5717 (ERROR on Windows X86_64, recorded as
>> "timeout,
>> > > > the test is too heavy" in M6)
>> > > >         jit.HLO.inline.ControlFlow.IfElse.IfElse1.IfElseTest1 (FAILED
>> in
>> > > > windows x86_64, recorded as "looks like
>> > > > issue in test" in M6)
>> > > >         jit.HLO.devirt.Runtime.RuntimeExtend1 (FAILED on windows
>> x86_64,
>> > > in
>> > > > M6 it is recorded as not regression and started to fail on M5)
>> > > >         reg.vm.btest6353.Btest6353 (ERROR on Windows x86_64, recorded
>> > > also
>> > > > failed on M3 and might be issue with the test)
>> > > >
>> > > >    4.3) New regressions on 1 platform (looks intermittent)
>> > > >         reg.jit.btest8029.Btest8029 (FAILED in Linux x86)
>> > > >         func.reg.jit.btest5710.Btest5710 (FAILED in Linux x86)
>> > > >         api.java.security.cert.F_CertPathTest_06.F_CertPathTest_06
>> (ERROR
>> > > > in Linux x86)
>> > > >         api.java.security.cert.F_CertPathTest_05.F_CertPathTest_05
>> (ERROR
>> > > > in Linux x86)
>> > > >
>> > > >  5) JDKTools Tests:
>> > > >    Several timeouts are observed in Linux x86 snapshot running. They
>> are:
>> > > >
>> > > >
>> > > >
>> > >
>> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch001
>> > > >
>> > > >
>> > > >
>> > >
>> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch002
>> > > >
>> > > >
>> > > >
>> > >
>> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch003
>> > > >
>> > > >
>> > > >
>> > >
>> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch004
>> > > >
>> > > >
>> > > >
>> > >
>> org.apache.harmony.jpda.tests.jdwp.Events.CombinedEventsTest.testCombinedEvents_04
>> > > >
>> > > >
>> > > >
>> > >
>> org.apache.harmony.jpda.tests.jdwp.MultiSession.AttachConnectorTest.testAttachConnector001
>> > > >
>> > > >
>> > > >
>> > >
>> org.apache.harmony.jpda.tests.jdwp.MultiSession.MethodEntryExitTest.testMethodEvent001
>> > > >
>> > > > org.apache.harmony.jpda.tests.jdwp.MultiSession.ResumeTest.testResume
>> > > >
>> > > >
>> > > >
>> > >
>> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadEndTest.testThreadEnd001
>> > > >
>> > > >
>> > > >
>> > >
>> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadStartTest.testThreadStart001
>> > > >    The Linux-only timeouts are also observed in the integrity testing
>> > > > results.
>> > > >    JIRA HARMONY-5833 has been filed for one of them.
>> > > >
>> > > >  6) JettyScenario:
>> > > >    The Linux x86 running failed because of the unresolved
>> HARMONY-5219.
>> > > >
>> > > >  7) Reliability:
>> > > >    Several failures are observed in windows x86_64 running.
>> > > >    7.1) Old regressions
>> > > >         api.net.DatagramTest (HARMONY-5531)
>> > > >         api.text.DecimalFormat_Locales - (in M6 it is recorded as
>> also
>> > > > intermittent in M5)
>> > > >
>> > > >    7.2) New/intemittent regressions
>> > > >         api.kernel.thread.ThreadLocalTest.ThreadLocalTest
>> > > >         api.kernel.exec.RunExec
>> > > >
>> > > >  8) Stress
>> > > >    Different test cases failed on different platforms.
>> > > >    8.1) Timeouts on Linux x86.
>> > > >
>> > > >
>> > > >
>> > >
>> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD003.ThreadTest003
>> > > >
>> > > >
>> > > >
>> > >
>> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD007.ThreadTest007
>> > > >
>> > > >
>> > > >
>> > >
>> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD009.ThreadTest009
>> > > >
>> > > >
>> > > >
>> > >
>> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD011.ThreadTest011
>> > > >
>> > > >    8.2) Failed cases on Windows x86_64 with unknown reasons.
>> > > >
>> > > >
>> > > >
>> > >
>> stress.org.apache.harmony.test.stress.classloader.MixThreads.TreeClasses.testTreeClasses2
>> > > >
>> > > >
>> > > >
>> > >
>> stress.org.apache.harmony.test.stress.classloader.NotSynchThreads.TreeClasses.testTreeClasses
>> > > >
>> > > >    I have not found records for them.
>> > > >
>> > > >  9) Strut_test
>> > > >    Broken with same error report as M6.
>> > > >
>> > > >  10) Eclipse Hello World Application.
>> > > >    Although the testing framework just reports EUT-API status as
>> > > "PASSED".
>> > > > A fresh workspace running just failed in configuration stage (
>> > > >
>> > > >
>> > >
>> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805120106o387a49f1rfb2c33d1042d2f41@mail.gmail.com%3e
>> > > > )
>> > > > since SVN commit 641928 (which committed patch for HARMONY-4569).
>> > > >
>> > > > thanks,
>> > > > chunrong
>> > > >
>> > >
>>
>

Re: [testing] first M7 candidate (r681495) testing status

Posted by chunrong lai <ch...@gmail.com>.
(1) M7 status is got in Fedora distros and M6/M5 status were got in SuSe
Linux distros.
(2) According to the stacktrace
( java.awt.Toolkit.init,  java.awt.SystemColor.<init>) of the
java.beans.Encoder error (I have not observed such error reports in previous
logs),  I guess the problem is from GUI system initialization. Is
X-connection available in your testing? In my understanding that is required
to run classlib test.


On 8/14/08, Nathan Beyer <nd...@apache.org> wrote:
>
> What Linux distro is it? I'm running Ubuntu and I'm seeing some
> different issues.
>
> I'm getting this failure -
>
> java.lang.ExceptionInInitializerError
> at java.lang.reflect.VMField.getObject(VMField.java)
> at java.lang.reflect.Field.get(Field.java:163)
> at
> java.beans.StaticFieldPersistenceDelegate.init(StaticFieldPersistenceDelegate.java:36)
> at
> java.beans.AwtSystemColorPersistenceDelegate.<clinit>(AwtSystemColorPersistenceDelegate.java:28)
> at java.beans.Encoder.<clinit>(Encoder.java:124)
> at
> java.beans.ArrayPersistenceDelegateTest.testInstantiate_Normal(ArrayPersistenceDelegateTest.java:130)
> at java.lang.reflect.VMReflection.invokeMethod(VMReflection.java)
> Caused by: java.lang.NullPointerException
> at java.awt.Toolkit.init(Toolkit.java:566)
> at java.awt.Toolkit.<init>(Toolkit.java:540)
> at java.awt.ToolkitImpl.<init>(ToolkitImpl.java:40)
> at java.awt.Toolkit.getDefaultToolkit(Toolkit.java:472)
> at java.awt.SystemColor.<init>(SystemColor.java:143)
> at java.awt.SystemColor.<clinit>(SystemColor.java:91)
> at java.lang.reflect.VMField.getObject(VMField.java)
> ... 18 more
>
> Then a bunch of NoClassDefFoundErrors for java.beans.Encoder.
>
>
> On Wed, Aug 13, 2008 at 3:21 AM, chunrong lai <ch...@gmail.com>
> wrote:
> >
> > hi, Nathan:
> >     It is also noted that currently we do setup Linux x86-64 integrity
> > testing, which eventually test every SVN revision, with results reported
> in
> > http://people.apache.org/~chunrong/harmony-integrity/. Your mentioned
> test
> > suites classib-test,drlvm-test are included there. (classlib+drlvm
> > are almost same as the HDK build reported in the page).
> >     We only use two platforms currently for the snapshot testing cycle,
> > (table is generated by
> >
> http://svn.apache.org/viewvc/harmony/enhanced/buildtest/trunk/tools/snapshot/snapshots-table/
> )
> > , to test more test suites with longer time for every snapshot. In my
> > observation adding two other platforms here just does not necessarily add
> > enough value in current stage of Harmony.
> >     I do think more testings/discussions help to improve Harmony quality.
> > Thanks for the dedication. If there are more I can help please let me
> know.
> >
> > On 8/12/08, Nathan Beyer <nd...@apache.org> wrote:
> > >
> > > What's being used to produce the integrity tests? I have an x86_64
> Linux
> > > box
> > > that I can dedicate to testing. I've tried just using buildtest with
> > > 'classlib,drlvm,classib-test,drlvm-test', but I haven't gotten a clean
> pass
> > > yet.
> > >
> > > -Nathan
> > >
> > > On Mon, Aug 11, 2008 at 12:26 PM, chunrong lai <chunronglai@gmail.com
> > > >wrote:
> > >
> > > > Hi, all:
> > > >
> > > >  Here is r681495 snapshot testing status:
> > > > http://people.apache.org/~chunrong/snapshots/r681495/index.html<
> > > http://people.apache.org/%7Echunrong/snapshots/r681495/index.html>.
> > > > I am using
> > > > two platforms (Linux x86, windows x86_64) at the moment. Hopefully we
> > > will
> > > > have other two platforms in future for M8. Well, although we are
> testing
> > > > only two platforms for M7, my experience is that the status for
> another
> > > two
> > > > platforms should be not worse or just include some extra intermittent
> > > > errors
> > > > which can be investigated in some later stages.
> > > >
> > > >  The following suites passed on Linux x86/Windows x86_64 platforms:
> Ant
> > > > Scenario (or self-hosting), Axis application, Dacapo, DRLVM
> regression
> > > > tests, Geronimo Unit Tests, Scimark, Tomcat scenario, VTS VM Test
> Suite.
> > > >
> > > >  Most of the failures are known issues (for M6). Although we can
> observe
> > > > 15~20 new issues, those issues happen only in 1 platform and they
> look
> > > more
> > > > like the intermittent/timeout issues (less reproducible) to me.
> Overall I
> > > > myself would like to think r681495 is more stable than M6.
> > > >
> > > >  Please add your comments and clarifications (please also see M6
> testing
> > > > results http://people.apache.org/~smishura/r653525/<
> > > http://people.apache.org/%7Esmishura/r653525/>
> > > > ,
> > > >
> > > >
> > >
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805070304l38845ee0se01fb93fbfc05586@mail.gmail.com%3eand
> > > > integrity testing results
> > > > http://people.apache.org/~chunrong/harmony-integrity/<
> > > http://people.apache.org/%7Echunrong/harmony-integrity/>as a
> comparison).
> > > >
> > > >  1) Classlib:
> > > >    1.1) Since r644719 (which committed patch for HARMONY-5684)
> > > >
> > > >        org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest
> > > >
> > > >
> > > >
> > >
> org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
> > > >
> > > >         failed in both platforms
> > > >
> > > >    1.2) Two failures
> > > >
> > > >        org.apache.harmony.luni.tests.java.net.MulticastSocketTest
> (Failed
> > > > in windows_x86 running of M6)
> > > >        tests.api.java.security.PermissionCollectionTest
> (Failed
> > > > in M6)
> > > >
> > > >         are observed in Linux x86.
> > > >
> > > >  2) DRLVM tests:
> > > >    2.1) One failure
> > > >
> > > >         java.lang.ClassGenericsTest.test_2
> > > >
> > > >         is observed in Linux x86 snapshot testing.
> > > >         I can see some old discussion in the mailing list about that
> but
> > > I
> > > > am not sure the expected status here.
> > > >         They should be intemittent errors since the integrity testing
> > > just
> > > > run well mostly.
> > > >
> > > >  3) EUTs:
> > > >
> > > >    3.1) Linux x86: 99.36%
> > > >         A recorded JIRA for this suite is HARMONY-2914 which wastes
> file
> > > > handlers and makes system unstable.
> > > >
> > > >  4) Functional:
> > > >    4.1) Old regressions on both platforms:
> > > >         api.java.text.MessageFormat (HARMONY-5430)
> > > >         api.java.util.jar.Manifest  (HARMONY-5473)
> > > >         api.java.beans.beancontext.BeanContextTest (also in M6,
> recorded
> > > > as  regression caused by changes in locale data)
> > > >         api.java.beans.persistence.EncoderTest  (also in M6, recorded
> as
> > > > regression in the beans module)
> > > >         api.java.beans.persistence.EncoderDecoderTest (also in M6,
> > > > regression in the beans module)
> > > >         reg.vm.btest5625 (also in M6, recorded as intermittent and
> not
> > > > reproducible manually)
> > > >
> > > >    4.2) Old regressions on 1 platform
> > > >         api.java.rmi.basicexception (ERROR in Linux x86,
> HARMONY-5823)
> > > >         api.java.rmi.basicregistry.RemoteServerTest (ERROR in Linux
> x86,
> > > > HARMONY-5823)
> > > >         jpda.jdwp.scenario.ST07.ST07Test (ERROR in windows x86_64, in
> M6
> > > it
> > > > is recorded as regression since M4)
> > > >         java.math.F_BigIntegerMatrixMultiplyTest_01 (ERROR on Linux
> x86,
> > > > recorded as Timeout, not reproducible
> > > >  in M6)
> > > >         reg.vm.btest5717 (ERROR on Windows X86_64, recorded as
> "timeout,
> > > > the test is too heavy" in M6)
> > > >         jit.HLO.inline.ControlFlow.IfElse.IfElse1.IfElseTest1 (FAILED
> in
> > > > windows x86_64, recorded as "looks like
> > > > issue in test" in M6)
> > > >         jit.HLO.devirt.Runtime.RuntimeExtend1 (FAILED on windows
> x86_64,
> > > in
> > > > M6 it is recorded as not regression and started to fail on M5)
> > > >         reg.vm.btest6353.Btest6353 (ERROR on Windows x86_64, recorded
> > > also
> > > > failed on M3 and might be issue with the test)
> > > >
> > > >    4.3) New regressions on 1 platform (looks intermittent)
> > > >         reg.jit.btest8029.Btest8029 (FAILED in Linux x86)
> > > >         func.reg.jit.btest5710.Btest5710 (FAILED in Linux x86)
> > > >         api.java.security.cert.F_CertPathTest_06.F_CertPathTest_06
> (ERROR
> > > > in Linux x86)
> > > >         api.java.security.cert.F_CertPathTest_05.F_CertPathTest_05
> (ERROR
> > > > in Linux x86)
> > > >
> > > >  5) JDKTools Tests:
> > > >    Several timeouts are observed in Linux x86 snapshot running. They
> are:
> > > >
> > > >
> > > >
> > >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch001
> > > >
> > > >
> > > >
> > >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch002
> > > >
> > > >
> > > >
> > >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch003
> > > >
> > > >
> > > >
> > >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch004
> > > >
> > > >
> > > >
> > >
> org.apache.harmony.jpda.tests.jdwp.Events.CombinedEventsTest.testCombinedEvents_04
> > > >
> > > >
> > > >
> > >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.AttachConnectorTest.testAttachConnector001
> > > >
> > > >
> > > >
> > >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.MethodEntryExitTest.testMethodEvent001
> > > >
> > > > org.apache.harmony.jpda.tests.jdwp.MultiSession.ResumeTest.testResume
> > > >
> > > >
> > > >
> > >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadEndTest.testThreadEnd001
> > > >
> > > >
> > > >
> > >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadStartTest.testThreadStart001
> > > >    The Linux-only timeouts are also observed in the integrity testing
> > > > results.
> > > >    JIRA HARMONY-5833 has been filed for one of them.
> > > >
> > > >  6) JettyScenario:
> > > >    The Linux x86 running failed because of the unresolved
> HARMONY-5219.
> > > >
> > > >  7) Reliability:
> > > >    Several failures are observed in windows x86_64 running.
> > > >    7.1) Old regressions
> > > >         api.net.DatagramTest (HARMONY-5531)
> > > >         api.text.DecimalFormat_Locales - (in M6 it is recorded as
> also
> > > > intermittent in M5)
> > > >
> > > >    7.2) New/intemittent regressions
> > > >         api.kernel.thread.ThreadLocalTest.ThreadLocalTest
> > > >         api.kernel.exec.RunExec
> > > >
> > > >  8) Stress
> > > >    Different test cases failed on different platforms.
> > > >    8.1) Timeouts on Linux x86.
> > > >
> > > >
> > > >
> > >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD003.ThreadTest003
> > > >
> > > >
> > > >
> > >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD007.ThreadTest007
> > > >
> > > >
> > > >
> > >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD009.ThreadTest009
> > > >
> > > >
> > > >
> > >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD011.ThreadTest011
> > > >
> > > >    8.2) Failed cases on Windows x86_64 with unknown reasons.
> > > >
> > > >
> > > >
> > >
> stress.org.apache.harmony.test.stress.classloader.MixThreads.TreeClasses.testTreeClasses2
> > > >
> > > >
> > > >
> > >
> stress.org.apache.harmony.test.stress.classloader.NotSynchThreads.TreeClasses.testTreeClasses
> > > >
> > > >    I have not found records for them.
> > > >
> > > >  9) Strut_test
> > > >    Broken with same error report as M6.
> > > >
> > > >  10) Eclipse Hello World Application.
> > > >    Although the testing framework just reports EUT-API status as
> > > "PASSED".
> > > > A fresh workspace running just failed in configuration stage (
> > > >
> > > >
> > >
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805120106o387a49f1rfb2c33d1042d2f41@mail.gmail.com%3e
> > > > )
> > > > since SVN commit 641928 (which committed patch for HARMONY-4569).
> > > >
> > > > thanks,
> > > > chunrong
> > > >
> > >
>

Re: [testing] first M7 candidate (r681495) testing status

Posted by Nathan Beyer <nd...@apache.org>.
What Linux distro is it? I'm running Ubuntu and I'm seeing some
different issues.

I'm getting this failure -

java.lang.ExceptionInInitializerError
at java.lang.reflect.VMField.getObject(VMField.java)
at java.lang.reflect.Field.get(Field.java:163)
at java.beans.StaticFieldPersistenceDelegate.init(StaticFieldPersistenceDelegate.java:36)
at java.beans.AwtSystemColorPersistenceDelegate.<clinit>(AwtSystemColorPersistenceDelegate.java:28)
at java.beans.Encoder.<clinit>(Encoder.java:124)
at java.beans.ArrayPersistenceDelegateTest.testInstantiate_Normal(ArrayPersistenceDelegateTest.java:130)
at java.lang.reflect.VMReflection.invokeMethod(VMReflection.java)
Caused by: java.lang.NullPointerException
at java.awt.Toolkit.init(Toolkit.java:566)
at java.awt.Toolkit.<init>(Toolkit.java:540)
at java.awt.ToolkitImpl.<init>(ToolkitImpl.java:40)
at java.awt.Toolkit.getDefaultToolkit(Toolkit.java:472)
at java.awt.SystemColor.<init>(SystemColor.java:143)
at java.awt.SystemColor.<clinit>(SystemColor.java:91)
at java.lang.reflect.VMField.getObject(VMField.java)
... 18 more

Then a bunch of NoClassDefFoundErrors for java.beans.Encoder.


On Wed, Aug 13, 2008 at 3:21 AM, chunrong lai <ch...@gmail.com> wrote:
>
> hi, Nathan:
>     It is also noted that currently we do setup Linux x86-64 integrity
> testing, which eventually test every SVN revision, with results reported in
> http://people.apache.org/~chunrong/harmony-integrity/. Your mentioned test
> suites classib-test,drlvm-test are included there. (classlib+drlvm
> are almost same as the HDK build reported in the page).
>     We only use two platforms currently for the snapshot testing cycle,
> (table is generated by
> http://svn.apache.org/viewvc/harmony/enhanced/buildtest/trunk/tools/snapshot/snapshots-table/)
> , to test more test suites with longer time for every snapshot. In my
> observation adding two other platforms here just does not necessarily add
> enough value in current stage of Harmony.
>     I do think more testings/discussions help to improve Harmony quality.
> Thanks for the dedication. If there are more I can help please let me know.
>
> On 8/12/08, Nathan Beyer <nd...@apache.org> wrote:
> >
> > What's being used to produce the integrity tests? I have an x86_64 Linux
> > box
> > that I can dedicate to testing. I've tried just using buildtest with
> > 'classlib,drlvm,classib-test,drlvm-test', but I haven't gotten a clean pass
> > yet.
> >
> > -Nathan
> >
> > On Mon, Aug 11, 2008 at 12:26 PM, chunrong lai <chunronglai@gmail.com
> > >wrote:
> >
> > > Hi, all:
> > >
> > >  Here is r681495 snapshot testing status:
> > > http://people.apache.org/~chunrong/snapshots/r681495/index.html<
> > http://people.apache.org/%7Echunrong/snapshots/r681495/index.html>.
> > > I am using
> > > two platforms (Linux x86, windows x86_64) at the moment. Hopefully we
> > will
> > > have other two platforms in future for M8. Well, although we are testing
> > > only two platforms for M7, my experience is that the status for another
> > two
> > > platforms should be not worse or just include some extra intermittent
> > > errors
> > > which can be investigated in some later stages.
> > >
> > >  The following suites passed on Linux x86/Windows x86_64 platforms: Ant
> > > Scenario (or self-hosting), Axis application, Dacapo, DRLVM regression
> > > tests, Geronimo Unit Tests, Scimark, Tomcat scenario, VTS VM Test Suite.
> > >
> > >  Most of the failures are known issues (for M6). Although we can observe
> > > 15~20 new issues, those issues happen only in 1 platform and they look
> > more
> > > like the intermittent/timeout issues (less reproducible) to me. Overall I
> > > myself would like to think r681495 is more stable than M6.
> > >
> > >  Please add your comments and clarifications (please also see M6 testing
> > > results http://people.apache.org/~smishura/r653525/<
> > http://people.apache.org/%7Esmishura/r653525/>
> > > ,
> > >
> > >
> > http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805070304l38845ee0se01fb93fbfc05586@mail.gmail.com%3eand
> > > integrity testing results
> > > http://people.apache.org/~chunrong/harmony-integrity/<
> > http://people.apache.org/%7Echunrong/harmony-integrity/>as a comparison).
> > >
> > >  1) Classlib:
> > >    1.1) Since r644719 (which committed patch for HARMONY-5684)
> > >
> > >        org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest
> > >
> > >
> > >
> > org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
> > >
> > >         failed in both platforms
> > >
> > >    1.2) Two failures
> > >
> > >        org.apache.harmony.luni.tests.java.net.MulticastSocketTest (Failed
> > > in windows_x86 running of M6)
> > >        tests.api.java.security.PermissionCollectionTest           (Failed
> > > in M6)
> > >
> > >         are observed in Linux x86.
> > >
> > >  2) DRLVM tests:
> > >    2.1) One failure
> > >
> > >         java.lang.ClassGenericsTest.test_2
> > >
> > >         is observed in Linux x86 snapshot testing.
> > >         I can see some old discussion in the mailing list about that but
> > I
> > > am not sure the expected status here.
> > >         They should be intemittent errors since the integrity testing
> > just
> > > run well mostly.
> > >
> > >  3) EUTs:
> > >
> > >    3.1) Linux x86: 99.36%
> > >         A recorded JIRA for this suite is HARMONY-2914 which wastes file
> > > handlers and makes system unstable.
> > >
> > >  4) Functional:
> > >    4.1) Old regressions on both platforms:
> > >         api.java.text.MessageFormat (HARMONY-5430)
> > >         api.java.util.jar.Manifest  (HARMONY-5473)
> > >         api.java.beans.beancontext.BeanContextTest (also in M6, recorded
> > > as  regression caused by changes in locale data)
> > >         api.java.beans.persistence.EncoderTest  (also in M6, recorded as
> > > regression in the beans module)
> > >         api.java.beans.persistence.EncoderDecoderTest (also in M6,
> > > regression in the beans module)
> > >         reg.vm.btest5625 (also in M6, recorded as intermittent and not
> > > reproducible manually)
> > >
> > >    4.2) Old regressions on 1 platform
> > >         api.java.rmi.basicexception (ERROR in Linux x86, HARMONY-5823)
> > >         api.java.rmi.basicregistry.RemoteServerTest (ERROR in Linux x86,
> > > HARMONY-5823)
> > >         jpda.jdwp.scenario.ST07.ST07Test (ERROR in windows x86_64, in M6
> > it
> > > is recorded as regression since M4)
> > >         java.math.F_BigIntegerMatrixMultiplyTest_01 (ERROR on Linux x86,
> > > recorded as Timeout, not reproducible
> > >  in M6)
> > >         reg.vm.btest5717 (ERROR on Windows X86_64, recorded as "timeout,
> > > the test is too heavy" in M6)
> > >         jit.HLO.inline.ControlFlow.IfElse.IfElse1.IfElseTest1 (FAILED in
> > > windows x86_64, recorded as "looks like
> > > issue in test" in M6)
> > >         jit.HLO.devirt.Runtime.RuntimeExtend1 (FAILED on windows x86_64,
> > in
> > > M6 it is recorded as not regression and started to fail on M5)
> > >         reg.vm.btest6353.Btest6353 (ERROR on Windows x86_64, recorded
> > also
> > > failed on M3 and might be issue with the test)
> > >
> > >    4.3) New regressions on 1 platform (looks intermittent)
> > >         reg.jit.btest8029.Btest8029 (FAILED in Linux x86)
> > >         func.reg.jit.btest5710.Btest5710 (FAILED in Linux x86)
> > >         api.java.security.cert.F_CertPathTest_06.F_CertPathTest_06 (ERROR
> > > in Linux x86)
> > >         api.java.security.cert.F_CertPathTest_05.F_CertPathTest_05 (ERROR
> > > in Linux x86)
> > >
> > >  5) JDKTools Tests:
> > >    Several timeouts are observed in Linux x86 snapshot running. They are:
> > >
> > >
> > >
> > org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch001
> > >
> > >
> > >
> > org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch002
> > >
> > >
> > >
> > org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch003
> > >
> > >
> > >
> > org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch004
> > >
> > >
> > >
> > org.apache.harmony.jpda.tests.jdwp.Events.CombinedEventsTest.testCombinedEvents_04
> > >
> > >
> > >
> > org.apache.harmony.jpda.tests.jdwp.MultiSession.AttachConnectorTest.testAttachConnector001
> > >
> > >
> > >
> > org.apache.harmony.jpda.tests.jdwp.MultiSession.MethodEntryExitTest.testMethodEvent001
> > >
> > > org.apache.harmony.jpda.tests.jdwp.MultiSession.ResumeTest.testResume
> > >
> > >
> > >
> > org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadEndTest.testThreadEnd001
> > >
> > >
> > >
> > org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadStartTest.testThreadStart001
> > >    The Linux-only timeouts are also observed in the integrity testing
> > > results.
> > >    JIRA HARMONY-5833 has been filed for one of them.
> > >
> > >  6) JettyScenario:
> > >    The Linux x86 running failed because of the unresolved HARMONY-5219.
> > >
> > >  7) Reliability:
> > >    Several failures are observed in windows x86_64 running.
> > >    7.1) Old regressions
> > >         api.net.DatagramTest (HARMONY-5531)
> > >         api.text.DecimalFormat_Locales - (in M6 it is recorded as also
> > > intermittent in M5)
> > >
> > >    7.2) New/intemittent regressions
> > >         api.kernel.thread.ThreadLocalTest.ThreadLocalTest
> > >         api.kernel.exec.RunExec
> > >
> > >  8) Stress
> > >    Different test cases failed on different platforms.
> > >    8.1) Timeouts on Linux x86.
> > >
> > >
> > >
> > stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD003.ThreadTest003
> > >
> > >
> > >
> > stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD007.ThreadTest007
> > >
> > >
> > >
> > stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD009.ThreadTest009
> > >
> > >
> > >
> > stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD011.ThreadTest011
> > >
> > >    8.2) Failed cases on Windows x86_64 with unknown reasons.
> > >
> > >
> > >
> > stress.org.apache.harmony.test.stress.classloader.MixThreads.TreeClasses.testTreeClasses2
> > >
> > >
> > >
> > stress.org.apache.harmony.test.stress.classloader.NotSynchThreads.TreeClasses.testTreeClasses
> > >
> > >    I have not found records for them.
> > >
> > >  9) Strut_test
> > >    Broken with same error report as M6.
> > >
> > >  10) Eclipse Hello World Application.
> > >    Although the testing framework just reports EUT-API status as
> > "PASSED".
> > > A fresh workspace running just failed in configuration stage (
> > >
> > >
> > http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805120106o387a49f1rfb2c33d1042d2f41@mail.gmail.com%3e
> > > )
> > > since SVN commit 641928 (which committed patch for HARMONY-4569).
> > >
> > > thanks,
> > > chunrong
> > >
> >

Re: [testing] first M7 candidate (r681495) testing status

Posted by chunrong lai <ch...@gmail.com>.
hi, Nathan:
     It is also noted that currently we do setup Linux x86-64 integrity
testing, which eventually test every SVN revision, with results reported in
http://people.apache.org/~chunrong/harmony-integrity/. Your mentioned test
suites classib-test,drlvm-test are included there. (classlib+drlvm
are almost same as the HDK build reported in the page).
     We only use two platforms currently for the snapshot testing cycle,
(table is generated by
http://svn.apache.org/viewvc/harmony/enhanced/buildtest/trunk/tools/snapshot/snapshots-table/)
, to test more test suites with longer time for every snapshot. In my
observation adding two other platforms here just does not necessarily add
enough value in current stage of Harmony.
     I do think more testings/discussions help to improve Harmony quality.
Thanks for the dedication. If there are more I can help please let me know.

On 8/12/08, Nathan Beyer <nd...@apache.org> wrote:
>
> What's being used to produce the integrity tests? I have an x86_64 Linux
> box
> that I can dedicate to testing. I've tried just using buildtest with
> 'classlib,drlvm,classib-test,drlvm-test', but I haven't gotten a clean pass
> yet.
>
> -Nathan
>
> On Mon, Aug 11, 2008 at 12:26 PM, chunrong lai <chunronglai@gmail.com
> >wrote:
>
> > Hi, all:
> >
> >  Here is r681495 snapshot testing status:
> > http://people.apache.org/~chunrong/snapshots/r681495/index.html<
> http://people.apache.org/%7Echunrong/snapshots/r681495/index.html>.
> > I am using
> > two platforms (Linux x86, windows x86_64) at the moment. Hopefully we
> will
> > have other two platforms in future for M8. Well, although we are testing
> > only two platforms for M7, my experience is that the status for another
> two
> > platforms should be not worse or just include some extra intermittent
> > errors
> > which can be investigated in some later stages.
> >
> >  The following suites passed on Linux x86/Windows x86_64 platforms: Ant
> > Scenario (or self-hosting), Axis application, Dacapo, DRLVM regression
> > tests, Geronimo Unit Tests, Scimark, Tomcat scenario, VTS VM Test Suite.
> >
> >  Most of the failures are known issues (for M6). Although we can observe
> > 15~20 new issues, those issues happen only in 1 platform and they look
> more
> > like the intermittent/timeout issues (less reproducible) to me. Overall I
> > myself would like to think r681495 is more stable than M6.
> >
> >  Please add your comments and clarifications (please also see M6 testing
> > results http://people.apache.org/~smishura/r653525/<
> http://people.apache.org/%7Esmishura/r653525/>
> > ,
> >
> >
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805070304l38845ee0se01fb93fbfc05586@mail.gmail.com%3eand
> > integrity testing results
> > http://people.apache.org/~chunrong/harmony-integrity/<
> http://people.apache.org/%7Echunrong/harmony-integrity/>as a comparison).
> >
> >  1) Classlib:
> >    1.1) Since r644719 (which committed patch for HARMONY-5684)
> >
> >        org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest
> >
> >
> >
> org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
> >
> >         failed in both platforms
> >
> >    1.2) Two failures
> >
> >        org.apache.harmony.luni.tests.java.net.MulticastSocketTest (Failed
> > in windows_x86 running of M6)
> >        tests.api.java.security.PermissionCollectionTest           (Failed
> > in M6)
> >
> >         are observed in Linux x86.
> >
> >  2) DRLVM tests:
> >    2.1) One failure
> >
> >         java.lang.ClassGenericsTest.test_2
> >
> >         is observed in Linux x86 snapshot testing.
> >         I can see some old discussion in the mailing list about that but
> I
> > am not sure the expected status here.
> >         They should be intemittent errors since the integrity testing
> just
> > run well mostly.
> >
> >  3) EUTs:
> >
> >    3.1) Linux x86: 99.36%
> >         A recorded JIRA for this suite is HARMONY-2914 which wastes file
> > handlers and makes system unstable.
> >
> >  4) Functional:
> >    4.1) Old regressions on both platforms:
> >         api.java.text.MessageFormat (HARMONY-5430)
> >         api.java.util.jar.Manifest  (HARMONY-5473)
> >         api.java.beans.beancontext.BeanContextTest (also in M6, recorded
> > as  regression caused by changes in locale data)
> >         api.java.beans.persistence.EncoderTest  (also in M6, recorded as
> > regression in the beans module)
> >         api.java.beans.persistence.EncoderDecoderTest (also in M6,
> > regression in the beans module)
> >         reg.vm.btest5625 (also in M6, recorded as intermittent and not
> > reproducible manually)
> >
> >    4.2) Old regressions on 1 platform
> >         api.java.rmi.basicexception (ERROR in Linux x86, HARMONY-5823)
> >         api.java.rmi.basicregistry.RemoteServerTest (ERROR in Linux x86,
> > HARMONY-5823)
> >         jpda.jdwp.scenario.ST07.ST07Test (ERROR in windows x86_64, in M6
> it
> > is recorded as regression since M4)
> >         java.math.F_BigIntegerMatrixMultiplyTest_01 (ERROR on Linux x86,
> > recorded as Timeout, not reproducible
> >  in M6)
> >         reg.vm.btest5717 (ERROR on Windows X86_64, recorded as "timeout,
> > the test is too heavy" in M6)
> >         jit.HLO.inline.ControlFlow.IfElse.IfElse1.IfElseTest1 (FAILED in
> > windows x86_64, recorded as "looks like
> > issue in test" in M6)
> >         jit.HLO.devirt.Runtime.RuntimeExtend1 (FAILED on windows x86_64,
> in
> > M6 it is recorded as not regression and started to fail on M5)
> >         reg.vm.btest6353.Btest6353 (ERROR on Windows x86_64, recorded
> also
> > failed on M3 and might be issue with the test)
> >
> >    4.3) New regressions on 1 platform (looks intermittent)
> >         reg.jit.btest8029.Btest8029 (FAILED in Linux x86)
> >         func.reg.jit.btest5710.Btest5710 (FAILED in Linux x86)
> >         api.java.security.cert.F_CertPathTest_06.F_CertPathTest_06 (ERROR
> > in Linux x86)
> >         api.java.security.cert.F_CertPathTest_05.F_CertPathTest_05 (ERROR
> > in Linux x86)
> >
> >  5) JDKTools Tests:
> >    Several timeouts are observed in Linux x86 snapshot running. They are:
> >
> >
> >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch001
> >
> >
> >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch002
> >
> >
> >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch003
> >
> >
> >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch004
> >
> >
> >
> org.apache.harmony.jpda.tests.jdwp.Events.CombinedEventsTest.testCombinedEvents_04
> >
> >
> >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.AttachConnectorTest.testAttachConnector001
> >
> >
> >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.MethodEntryExitTest.testMethodEvent001
> >
> > org.apache.harmony.jpda.tests.jdwp.MultiSession.ResumeTest.testResume
> >
> >
> >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadEndTest.testThreadEnd001
> >
> >
> >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadStartTest.testThreadStart001
> >    The Linux-only timeouts are also observed in the integrity testing
> > results.
> >    JIRA HARMONY-5833 has been filed for one of them.
> >
> >  6) JettyScenario:
> >    The Linux x86 running failed because of the unresolved HARMONY-5219.
> >
> >  7) Reliability:
> >    Several failures are observed in windows x86_64 running.
> >    7.1) Old regressions
> >         api.net.DatagramTest (HARMONY-5531)
> >         api.text.DecimalFormat_Locales - (in M6 it is recorded as also
> > intermittent in M5)
> >
> >    7.2) New/intemittent regressions
> >         api.kernel.thread.ThreadLocalTest.ThreadLocalTest
> >         api.kernel.exec.RunExec
> >
> >  8) Stress
> >    Different test cases failed on different platforms.
> >    8.1) Timeouts on Linux x86.
> >
> >
> >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD003.ThreadTest003
> >
> >
> >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD007.ThreadTest007
> >
> >
> >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD009.ThreadTest009
> >
> >
> >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD011.ThreadTest011
> >
> >    8.2) Failed cases on Windows x86_64 with unknown reasons.
> >
> >
> >
> stress.org.apache.harmony.test.stress.classloader.MixThreads.TreeClasses.testTreeClasses2
> >
> >
> >
> stress.org.apache.harmony.test.stress.classloader.NotSynchThreads.TreeClasses.testTreeClasses
> >
> >    I have not found records for them.
> >
> >  9) Strut_test
> >    Broken with same error report as M6.
> >
> >  10) Eclipse Hello World Application.
> >    Although the testing framework just reports EUT-API status as
> "PASSED".
> > A fresh workspace running just failed in configuration stage (
> >
> >
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805120106o387a49f1rfb2c33d1042d2f41@mail.gmail.com%3e
> > )
> > since SVN commit 641928 (which committed patch for HARMONY-4569).
> >
> > thanks,
> > chunrong
> >
>

Re: [testing] first M7 candidate (r681495) testing status

Posted by Nathan Beyer <nd...@apache.org>.
What's being used to produce the integrity tests? I have an x86_64 Linux box
that I can dedicate to testing. I've tried just using buildtest with
'classlib,drlvm,classib-test,drlvm-test', but I haven't gotten a clean pass
yet.

-Nathan

On Mon, Aug 11, 2008 at 12:26 PM, chunrong lai <ch...@gmail.com>wrote:

> Hi, all:
>
>  Here is r681495 snapshot testing status:
> http://people.apache.org/~chunrong/snapshots/r681495/index.html<http://people.apache.org/%7Echunrong/snapshots/r681495/index.html>.
> I am using
> two platforms (Linux x86, windows x86_64) at the moment. Hopefully we will
> have other two platforms in future for M8. Well, although we are testing
> only two platforms for M7, my experience is that the status for another two
> platforms should be not worse or just include some extra intermittent
> errors
> which can be investigated in some later stages.
>
>  The following suites passed on Linux x86/Windows x86_64 platforms: Ant
> Scenario (or self-hosting), Axis application, Dacapo, DRLVM regression
> tests, Geronimo Unit Tests, Scimark, Tomcat scenario, VTS VM Test Suite.
>
>  Most of the failures are known issues (for M6). Although we can observe
> 15~20 new issues, those issues happen only in 1 platform and they look more
> like the intermittent/timeout issues (less reproducible) to me. Overall I
> myself would like to think r681495 is more stable than M6.
>
>  Please add your comments and clarifications (please also see M6 testing
> results http://people.apache.org/~smishura/r653525/<http://people.apache.org/%7Esmishura/r653525/>
> ,
>
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805070304l38845ee0se01fb93fbfc05586@mail.gmail.com%3eand
> integrity testing results
> http://people.apache.org/~chunrong/harmony-integrity/<http://people.apache.org/%7Echunrong/harmony-integrity/>as a comparison).
>
>  1) Classlib:
>    1.1) Since r644719 (which committed patch for HARMONY-5684)
>
>        org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest
>
>
> org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
>
>         failed in both platforms
>
>    1.2) Two failures
>
>        org.apache.harmony.luni.tests.java.net.MulticastSocketTest (Failed
> in windows_x86 running of M6)
>        tests.api.java.security.PermissionCollectionTest           (Failed
> in M6)
>
>         are observed in Linux x86.
>
>  2) DRLVM tests:
>    2.1) One failure
>
>         java.lang.ClassGenericsTest.test_2
>
>         is observed in Linux x86 snapshot testing.
>         I can see some old discussion in the mailing list about that but I
> am not sure the expected status here.
>         They should be intemittent errors since the integrity testing just
> run well mostly.
>
>  3) EUTs:
>
>    3.1) Linux x86: 99.36%
>         A recorded JIRA for this suite is HARMONY-2914 which wastes file
> handlers and makes system unstable.
>
>  4) Functional:
>    4.1) Old regressions on both platforms:
>         api.java.text.MessageFormat (HARMONY-5430)
>         api.java.util.jar.Manifest  (HARMONY-5473)
>         api.java.beans.beancontext.BeanContextTest (also in M6, recorded
> as  regression caused by changes in locale data)
>         api.java.beans.persistence.EncoderTest  (also in M6, recorded as
> regression in the beans module)
>         api.java.beans.persistence.EncoderDecoderTest (also in M6,
> regression in the beans module)
>         reg.vm.btest5625 (also in M6, recorded as intermittent and not
> reproducible manually)
>
>    4.2) Old regressions on 1 platform
>         api.java.rmi.basicexception (ERROR in Linux x86, HARMONY-5823)
>         api.java.rmi.basicregistry.RemoteServerTest (ERROR in Linux x86,
> HARMONY-5823)
>         jpda.jdwp.scenario.ST07.ST07Test (ERROR in windows x86_64, in M6 it
> is recorded as regression since M4)
>         java.math.F_BigIntegerMatrixMultiplyTest_01 (ERROR on Linux x86,
> recorded as Timeout, not reproducible
>  in M6)
>         reg.vm.btest5717 (ERROR on Windows X86_64, recorded as "timeout,
> the test is too heavy" in M6)
>         jit.HLO.inline.ControlFlow.IfElse.IfElse1.IfElseTest1 (FAILED in
> windows x86_64, recorded as "looks like
> issue in test" in M6)
>         jit.HLO.devirt.Runtime.RuntimeExtend1 (FAILED on windows x86_64, in
> M6 it is recorded as not regression and started to fail on M5)
>         reg.vm.btest6353.Btest6353 (ERROR on Windows x86_64, recorded also
> failed on M3 and might be issue with the test)
>
>    4.3) New regressions on 1 platform (looks intermittent)
>         reg.jit.btest8029.Btest8029 (FAILED in Linux x86)
>         func.reg.jit.btest5710.Btest5710 (FAILED in Linux x86)
>         api.java.security.cert.F_CertPathTest_06.F_CertPathTest_06 (ERROR
> in Linux x86)
>         api.java.security.cert.F_CertPathTest_05.F_CertPathTest_05 (ERROR
> in Linux x86)
>
>  5) JDKTools Tests:
>    Several timeouts are observed in Linux x86 snapshot running. They are:
>
>
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch001
>
>
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch002
>
>
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch003
>
>
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch004
>
>
> org.apache.harmony.jpda.tests.jdwp.Events.CombinedEventsTest.testCombinedEvents_04
>
>
> org.apache.harmony.jpda.tests.jdwp.MultiSession.AttachConnectorTest.testAttachConnector001
>
>
> org.apache.harmony.jpda.tests.jdwp.MultiSession.MethodEntryExitTest.testMethodEvent001
>
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ResumeTest.testResume
>
>
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadEndTest.testThreadEnd001
>
>
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadStartTest.testThreadStart001
>    The Linux-only timeouts are also observed in the integrity testing
> results.
>    JIRA HARMONY-5833 has been filed for one of them.
>
>  6) JettyScenario:
>    The Linux x86 running failed because of the unresolved HARMONY-5219.
>
>  7) Reliability:
>    Several failures are observed in windows x86_64 running.
>    7.1) Old regressions
>         api.net.DatagramTest (HARMONY-5531)
>         api.text.DecimalFormat_Locales - (in M6 it is recorded as also
> intermittent in M5)
>
>    7.2) New/intemittent regressions
>         api.kernel.thread.ThreadLocalTest.ThreadLocalTest
>         api.kernel.exec.RunExec
>
>  8) Stress
>    Different test cases failed on different platforms.
>    8.1) Timeouts on Linux x86.
>
>
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD003.ThreadTest003
>
>
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD007.ThreadTest007
>
>
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD009.ThreadTest009
>
>
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD011.ThreadTest011
>
>    8.2) Failed cases on Windows x86_64 with unknown reasons.
>
>
> stress.org.apache.harmony.test.stress.classloader.MixThreads.TreeClasses.testTreeClasses2
>
>
> stress.org.apache.harmony.test.stress.classloader.NotSynchThreads.TreeClasses.testTreeClasses
>
>    I have not found records for them.
>
>  9) Strut_test
>    Broken with same error report as M6.
>
>  10) Eclipse Hello World Application.
>    Although the testing framework just reports EUT-API status as "PASSED".
> A fresh workspace running just failed in configuration stage (
>
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805120106o387a49f1rfb2c33d1042d2f41@mail.gmail.com%3e
> )
> since SVN commit 641928 (which committed patch for HARMONY-4569).
>
> thanks,
> chunrong
>

Re: [testing] first M7 candidate (r681495) testing status

Posted by chunrong lai <ch...@gmail.com>.
  It is noted that I filed HARMONY-5954 for the EHWA-API setup issue. I
think that it is not a same issue as HARMONY-5747 (but both from r641928)
since I does not see missing Manifest-Version from the Manifests in the test
suite. Anyway I think that it is an issue needs to be fixed since RI is OK
here.

 Thanks.

On Fri, Aug 15, 2008 at 9:15 PM, Sian January <si...@googlemail.com>wrote:

> On 15/08/2008, chunrong lai <ch...@gmail.com> wrote:
>
> >
> >  It is also noted that (as the comment in HARMONY-5747) the error in 10)
> is
> > a new issue due to an old classlib checkin r641928 which can be stably
> > reproduced . Maybe we need to have a look in it?
> >
> Investigations in HARMONY-5747 found that the other problem was an
> Eclipse bug.  Eclipse definitely still runs with Harmony, so although
> I do think we should investigate this I don't think it's a blocker for
> M7.
>
> --
>  Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>

Re: [testing] first M7 candidate (r681495) testing status

Posted by Sian January <si...@googlemail.com>.
On 15/08/2008, chunrong lai <ch...@gmail.com> wrote:
>  I think none of these are blocker for M7 because they are basically old
> issues or intermettent errors. I also think that we are doing better than
> M6.

Ok - thanks.  I'm going to kick off a vote now unless anyone objects
in the next few minutes.  I think 2 and a half weeks should be long
enough for everyone to have done the testing they intended to do by
now.

>
>  It is also noted that (as the comment in HARMONY-5747) the error in 10) is
> a new issue due to an old classlib checkin r641928 which can be stably
> reproduced . Maybe we need to have a look in it?
>

Investigations in HARMONY-5747 found that the other problem was an
Eclipse bug.  Eclipse definitely still runs with Harmony, so although
I do think we should investigate this I don't think it's a blocker for
M7.

> On 8/14/08, Sian January <si...@googlemail.com> wrote:
> >
> > Does anyone have a feel for whether any of these failures should be
> > blockers for M7?  In terms of the Harmony test suite I think the pass
> > rate is fine because we're doing better than M6, and EUTs look ok too,
> > but I'm not really sure if we need to be concerned about any of:
> >
> > 4) Functional
> > 5) JDKTools Tests
> > 6) JettyScenario
> > 7) Reliability
> > 8) Stress
> >
> > or
> >
> > 10) Eclipse Hello World Application.
> >
> > Any thoughts?
> >
> > If none of these are blockers then it would be good to start a vote in
> > the next day or two because the code has been frozen for nearly 3
> > weeks now.
> >
> > Thanks,
> >
> > Sian
> >
> >
> > On 11/08/2008, chunrong lai <ch...@gmail.com> wrote:
> > > Hi, all:
> > >
> > >  Here is r681495 snapshot testing status:
> > > http://people.apache.org/~chunrong/snapshots/r681495/index.html. I am
> > using
> > > two platforms (Linux x86, windows x86_64) at the moment. Hopefully we
> > will
> > > have other two platforms in future for M8. Well, although we are testing
> > > only two platforms for M7, my experience is that the status for another
> > two
> > > platforms should be not worse or just include some extra intermittent
> > errors
> > > which can be investigated in some later stages.
> > >
> > >  The following suites passed on Linux x86/Windows x86_64 platforms: Ant
> > > Scenario (or self-hosting), Axis application, Dacapo, DRLVM regression
> > > tests, Geronimo Unit Tests, Scimark, Tomcat scenario, VTS VM Test Suite.
> > >
> > >  Most of the failures are known issues (for M6). Although we can observe
> > > 15~20 new issues, those issues happen only in 1 platform and they look
> > more
> > > like the intermittent/timeout issues (less reproducible) to me. Overall I
> > > myself would like to think r681495 is more stable than M6.
> > >
> > >  Please add your comments and clarifications (please also see M6 testing
> > > results http://people.apache.org/~smishura/r653525/,
> > >
> > http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805070304l38845ee0se01fb93fbfc05586@mail.gmail.com%3eand
> > > integrity testing results
> > > http://people.apache.org/~chunrong/harmony-integrity/ as a comparison).
> > >
> > >  1) Classlib:
> > >    1.1) Since r644719 (which committed patch for HARMONY-5684)
> > >
> > >        org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest
> > >
> > >
> > org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
> > >
> > >         failed in both platforms
> > >
> > >    1.2) Two failures
> > >
> > >        org.apache.harmony.luni.tests.java.net.MulticastSocketTest (Failed
> > > in windows_x86 running of M6)
> > >        tests.api.java.security.PermissionCollectionTest           (Failed
> > > in M6)
> > >
> > >         are observed in Linux x86.
> > >
> > >  2) DRLVM tests:
> > >    2.1) One failure
> > >
> > >         java.lang.ClassGenericsTest.test_2
> > >
> > >         is observed in Linux x86 snapshot testing.
> > >         I can see some old discussion in the mailing list about that but
> > I
> > > am not sure the expected status here.
> > >         They should be intemittent errors since the integrity testing
> > just
> > > run well mostly.
> > >
> > >  3) EUTs:
> > >
> > >    3.1) Linux x86: 99.36%
> > >         A recorded JIRA for this suite is HARMONY-2914 which wastes file
> > > handlers and makes system unstable.
> > >
> > >  4) Functional:
> > >    4.1) Old regressions on both platforms:
> > >         api.java.text.MessageFormat (HARMONY-5430)
> > >         api.java.util.jar.Manifest  (HARMONY-5473)
> > >         api.java.beans.beancontext.BeanContextTest (also in M6, recorded
> > > as  regression caused by changes in locale data)
> > >         api.java.beans.persistence.EncoderTest  (also in M6, recorded as
> > > regression in the beans module)
> > >         api.java.beans.persistence.EncoderDecoderTest (also in M6,
> > > regression in the beans module)
> > >         reg.vm.btest5625 (also in M6, recorded as intermittent and not
> > > reproducible manually)
> > >
> > >    4.2) Old regressions on 1 platform
> > >         api.java.rmi.basicexception (ERROR in Linux x86, HARMONY-5823)
> > >         api.java.rmi.basicregistry.RemoteServerTest (ERROR in Linux x86,
> > > HARMONY-5823)
> > >         jpda.jdwp.scenario.ST07.ST07Test (ERROR in windows x86_64, in M6
> > it
> > > is recorded as regression since M4)
> > >         java.math.F_BigIntegerMatrixMultiplyTest_01 (ERROR on Linux x86,
> > > recorded as Timeout, not reproducible
> > >  in M6)
> > >         reg.vm.btest5717 (ERROR on Windows X86_64, recorded as "timeout,
> > > the test is too heavy" in M6)
> > >         jit.HLO.inline.ControlFlow.IfElse.IfElse1.IfElseTest1 (FAILED in
> > > windows x86_64, recorded as "looks like
> > > issue in test" in M6)
> > >         jit.HLO.devirt.Runtime.RuntimeExtend1 (FAILED on windows x86_64,
> > in
> > > M6 it is recorded as not regression and started to fail on M5)
> > >         reg.vm.btest6353.Btest6353 (ERROR on Windows x86_64, recorded
> > also
> > > failed on M3 and might be issue with the test)
> > >
> > >    4.3) New regressions on 1 platform (looks intermittent)
> > >         reg.jit.btest8029.Btest8029 (FAILED in Linux x86)
> > >         func.reg.jit.btest5710.Btest5710 (FAILED in Linux x86)
> > >         api.java.security.cert.F_CertPathTest_06.F_CertPathTest_06 (ERROR
> > > in Linux x86)
> > >         api.java.security.cert.F_CertPathTest_05.F_CertPathTest_05 (ERROR
> > > in Linux x86)
> > >
> > >  5) JDKTools Tests:
> > >    Several timeouts are observed in Linux x86 snapshot running. They are:
> > >
> > >
> > org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch001
> > >
> > >
> > org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch002
> > >
> > >
> > org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch003
> > >
> > >
> > org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch004
> > >
> > >
> > org.apache.harmony.jpda.tests.jdwp.Events.CombinedEventsTest.testCombinedEvents_04
> > >
> > >
> > org.apache.harmony.jpda.tests.jdwp.MultiSession.AttachConnectorTest.testAttachConnector001
> > >
> > >
> > org.apache.harmony.jpda.tests.jdwp.MultiSession.MethodEntryExitTest.testMethodEvent001
> > >
> > > org.apache.harmony.jpda.tests.jdwp.MultiSession.ResumeTest.testResume
> > >
> > >
> > org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadEndTest.testThreadEnd001
> > >
> > >
> > org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadStartTest.testThreadStart001
> > >    The Linux-only timeouts are also observed in the integrity testing
> > > results.
> > >    JIRA HARMONY-5833 has been filed for one of them.
> > >
> > >  6) JettyScenario:
> > >    The Linux x86 running failed because of the unresolved HARMONY-5219.
> > >
> > >  7) Reliability:
> > >    Several failures are observed in windows x86_64 running.
> > >    7.1) Old regressions
> > >         api.net.DatagramTest (HARMONY-5531)
> > >         api.text.DecimalFormat_Locales - (in M6 it is recorded as also
> > > intermittent in M5)
> > >
> > >    7.2) New/intemittent regressions
> > >         api.kernel.thread.ThreadLocalTest.ThreadLocalTest
> > >         api.kernel.exec.RunExec
> > >
> > >  8) Stress
> > >    Different test cases failed on different platforms.
> > >    8.1) Timeouts on Linux x86.
> > >
> > >
> > stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD003.ThreadTest003
> > >
> > >
> > stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD007.ThreadTest007
> > >
> > >
> > stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD009.ThreadTest009
> > >
> > >
> > stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD011.ThreadTest011
> > >
> > >    8.2) Failed cases on Windows x86_64 with unknown reasons.
> > >
> > >
> > stress.org.apache.harmony.test.stress.classloader.MixThreads.TreeClasses.testTreeClasses2
> > >
> > >
> > stress.org.apache.harmony.test.stress.classloader.NotSynchThreads.TreeClasses.testTreeClasses
> > >
> > >    I have not found records for them.
> > >
> > >  9) Strut_test
> > >    Broken with same error report as M6.
> > >
> > >  10) Eclipse Hello World Application.
> > >    Although the testing framework just reports EUT-API status as
> > "PASSED".
> > > A fresh workspace running just failed in configuration stage (
> > >
> > http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805120106o387a49f1rfb2c33d1042d2f41@mail.gmail.com%3e
> > )
> > > since SVN commit 641928 (which committed patch for HARMONY-4569).
> > >
> > > thanks,
> > > chunrong
> > >
> >
> >
> > --
> > Unless stated otherwise above:
> > IBM United Kingdom Limited - Registered in England and Wales with number
> > 741598.
> > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
> >
>


-- 
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

Re: [testing] first M7 candidate (r681495) testing status

Posted by chunrong lai <ch...@gmail.com>.
  I think none of these are blocker for M7 because they are basically old
issues or intermettent errors. I also think that we are doing better than
M6.

  It is also noted that (as the comment in HARMONY-5747) the error in 10) is
a new issue due to an old classlib checkin r641928 which can be stably
reproduced . Maybe we need to have a look in it?

On 8/14/08, Sian January <si...@googlemail.com> wrote:
>
> Does anyone have a feel for whether any of these failures should be
> blockers for M7?  In terms of the Harmony test suite I think the pass
> rate is fine because we're doing better than M6, and EUTs look ok too,
> but I'm not really sure if we need to be concerned about any of:
>
> 4) Functional
> 5) JDKTools Tests
> 6) JettyScenario
> 7) Reliability
> 8) Stress
>
> or
>
> 10) Eclipse Hello World Application.
>
> Any thoughts?
>
> If none of these are blockers then it would be good to start a vote in
> the next day or two because the code has been frozen for nearly 3
> weeks now.
>
> Thanks,
>
> Sian
>
>
> On 11/08/2008, chunrong lai <ch...@gmail.com> wrote:
> > Hi, all:
> >
> >  Here is r681495 snapshot testing status:
> > http://people.apache.org/~chunrong/snapshots/r681495/index.html. I am
> using
> > two platforms (Linux x86, windows x86_64) at the moment. Hopefully we
> will
> > have other two platforms in future for M8. Well, although we are testing
> > only two platforms for M7, my experience is that the status for another
> two
> > platforms should be not worse or just include some extra intermittent
> errors
> > which can be investigated in some later stages.
> >
> >  The following suites passed on Linux x86/Windows x86_64 platforms: Ant
> > Scenario (or self-hosting), Axis application, Dacapo, DRLVM regression
> > tests, Geronimo Unit Tests, Scimark, Tomcat scenario, VTS VM Test Suite.
> >
> >  Most of the failures are known issues (for M6). Although we can observe
> > 15~20 new issues, those issues happen only in 1 platform and they look
> more
> > like the intermittent/timeout issues (less reproducible) to me. Overall I
> > myself would like to think r681495 is more stable than M6.
> >
> >  Please add your comments and clarifications (please also see M6 testing
> > results http://people.apache.org/~smishura/r653525/,
> >
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805070304l38845ee0se01fb93fbfc05586@mail.gmail.com%3eand
> > integrity testing results
> > http://people.apache.org/~chunrong/harmony-integrity/ as a comparison).
> >
> >  1) Classlib:
> >    1.1) Since r644719 (which committed patch for HARMONY-5684)
> >
> >        org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest
> >
> >
> org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
> >
> >         failed in both platforms
> >
> >    1.2) Two failures
> >
> >        org.apache.harmony.luni.tests.java.net.MulticastSocketTest (Failed
> > in windows_x86 running of M6)
> >        tests.api.java.security.PermissionCollectionTest           (Failed
> > in M6)
> >
> >         are observed in Linux x86.
> >
> >  2) DRLVM tests:
> >    2.1) One failure
> >
> >         java.lang.ClassGenericsTest.test_2
> >
> >         is observed in Linux x86 snapshot testing.
> >         I can see some old discussion in the mailing list about that but
> I
> > am not sure the expected status here.
> >         They should be intemittent errors since the integrity testing
> just
> > run well mostly.
> >
> >  3) EUTs:
> >
> >    3.1) Linux x86: 99.36%
> >         A recorded JIRA for this suite is HARMONY-2914 which wastes file
> > handlers and makes system unstable.
> >
> >  4) Functional:
> >    4.1) Old regressions on both platforms:
> >         api.java.text.MessageFormat (HARMONY-5430)
> >         api.java.util.jar.Manifest  (HARMONY-5473)
> >         api.java.beans.beancontext.BeanContextTest (also in M6, recorded
> > as  regression caused by changes in locale data)
> >         api.java.beans.persistence.EncoderTest  (also in M6, recorded as
> > regression in the beans module)
> >         api.java.beans.persistence.EncoderDecoderTest (also in M6,
> > regression in the beans module)
> >         reg.vm.btest5625 (also in M6, recorded as intermittent and not
> > reproducible manually)
> >
> >    4.2) Old regressions on 1 platform
> >         api.java.rmi.basicexception (ERROR in Linux x86, HARMONY-5823)
> >         api.java.rmi.basicregistry.RemoteServerTest (ERROR in Linux x86,
> > HARMONY-5823)
> >         jpda.jdwp.scenario.ST07.ST07Test (ERROR in windows x86_64, in M6
> it
> > is recorded as regression since M4)
> >         java.math.F_BigIntegerMatrixMultiplyTest_01 (ERROR on Linux x86,
> > recorded as Timeout, not reproducible
> >  in M6)
> >         reg.vm.btest5717 (ERROR on Windows X86_64, recorded as "timeout,
> > the test is too heavy" in M6)
> >         jit.HLO.inline.ControlFlow.IfElse.IfElse1.IfElseTest1 (FAILED in
> > windows x86_64, recorded as "looks like
> > issue in test" in M6)
> >         jit.HLO.devirt.Runtime.RuntimeExtend1 (FAILED on windows x86_64,
> in
> > M6 it is recorded as not regression and started to fail on M5)
> >         reg.vm.btest6353.Btest6353 (ERROR on Windows x86_64, recorded
> also
> > failed on M3 and might be issue with the test)
> >
> >    4.3) New regressions on 1 platform (looks intermittent)
> >         reg.jit.btest8029.Btest8029 (FAILED in Linux x86)
> >         func.reg.jit.btest5710.Btest5710 (FAILED in Linux x86)
> >         api.java.security.cert.F_CertPathTest_06.F_CertPathTest_06 (ERROR
> > in Linux x86)
> >         api.java.security.cert.F_CertPathTest_05.F_CertPathTest_05 (ERROR
> > in Linux x86)
> >
> >  5) JDKTools Tests:
> >    Several timeouts are observed in Linux x86 snapshot running. They are:
> >
> >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch001
> >
> >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch002
> >
> >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch003
> >
> >
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch004
> >
> >
> org.apache.harmony.jpda.tests.jdwp.Events.CombinedEventsTest.testCombinedEvents_04
> >
> >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.AttachConnectorTest.testAttachConnector001
> >
> >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.MethodEntryExitTest.testMethodEvent001
> >
> > org.apache.harmony.jpda.tests.jdwp.MultiSession.ResumeTest.testResume
> >
> >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadEndTest.testThreadEnd001
> >
> >
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadStartTest.testThreadStart001
> >    The Linux-only timeouts are also observed in the integrity testing
> > results.
> >    JIRA HARMONY-5833 has been filed for one of them.
> >
> >  6) JettyScenario:
> >    The Linux x86 running failed because of the unresolved HARMONY-5219.
> >
> >  7) Reliability:
> >    Several failures are observed in windows x86_64 running.
> >    7.1) Old regressions
> >         api.net.DatagramTest (HARMONY-5531)
> >         api.text.DecimalFormat_Locales - (in M6 it is recorded as also
> > intermittent in M5)
> >
> >    7.2) New/intemittent regressions
> >         api.kernel.thread.ThreadLocalTest.ThreadLocalTest
> >         api.kernel.exec.RunExec
> >
> >  8) Stress
> >    Different test cases failed on different platforms.
> >    8.1) Timeouts on Linux x86.
> >
> >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD003.ThreadTest003
> >
> >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD007.ThreadTest007
> >
> >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD009.ThreadTest009
> >
> >
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD011.ThreadTest011
> >
> >    8.2) Failed cases on Windows x86_64 with unknown reasons.
> >
> >
> stress.org.apache.harmony.test.stress.classloader.MixThreads.TreeClasses.testTreeClasses2
> >
> >
> stress.org.apache.harmony.test.stress.classloader.NotSynchThreads.TreeClasses.testTreeClasses
> >
> >    I have not found records for them.
> >
> >  9) Strut_test
> >    Broken with same error report as M6.
> >
> >  10) Eclipse Hello World Application.
> >    Although the testing framework just reports EUT-API status as
> "PASSED".
> > A fresh workspace running just failed in configuration stage (
> >
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805120106o387a49f1rfb2c33d1042d2f41@mail.gmail.com%3e
> )
> > since SVN commit 641928 (which committed patch for HARMONY-4569).
> >
> > thanks,
> > chunrong
> >
>
>
> --
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
>

Re: [testing] first M7 candidate (r681495) testing status

Posted by Sian January <si...@googlemail.com>.
Does anyone have a feel for whether any of these failures should be
blockers for M7?  In terms of the Harmony test suite I think the pass
rate is fine because we're doing better than M6, and EUTs look ok too,
but I'm not really sure if we need to be concerned about any of:

4) Functional
5) JDKTools Tests
6) JettyScenario
7) Reliability
8) Stress

or

10) Eclipse Hello World Application.

Any thoughts?

If none of these are blockers then it would be good to start a vote in
the next day or two because the code has been frozen for nearly 3
weeks now.

Thanks,

Sian


On 11/08/2008, chunrong lai <ch...@gmail.com> wrote:
> Hi, all:
>
>  Here is r681495 snapshot testing status:
> http://people.apache.org/~chunrong/snapshots/r681495/index.html. I am using
> two platforms (Linux x86, windows x86_64) at the moment. Hopefully we will
> have other two platforms in future for M8. Well, although we are testing
> only two platforms for M7, my experience is that the status for another two
> platforms should be not worse or just include some extra intermittent errors
> which can be investigated in some later stages.
>
>  The following suites passed on Linux x86/Windows x86_64 platforms: Ant
> Scenario (or self-hosting), Axis application, Dacapo, DRLVM regression
> tests, Geronimo Unit Tests, Scimark, Tomcat scenario, VTS VM Test Suite.
>
>  Most of the failures are known issues (for M6). Although we can observe
> 15~20 new issues, those issues happen only in 1 platform and they look more
> like the intermittent/timeout issues (less reproducible) to me. Overall I
> myself would like to think r681495 is more stable than M6.
>
>  Please add your comments and clarifications (please also see M6 testing
> results http://people.apache.org/~smishura/r653525/,
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805070304l38845ee0se01fb93fbfc05586@mail.gmail.com%3eand
> integrity testing results
> http://people.apache.org/~chunrong/harmony-integrity/ as a comparison).
>
>  1) Classlib:
>    1.1) Since r644719 (which committed patch for HARMONY-5684)
>
>        org.apache.harmony.lang.management.MemoryPoolMXBeanImplTest
>
> org.apache.harmony.lang.management.tests.java.lang.management.MemoryPoolMXBeanTest
>
>         failed in both platforms
>
>    1.2) Two failures
>
>        org.apache.harmony.luni.tests.java.net.MulticastSocketTest (Failed
> in windows_x86 running of M6)
>        tests.api.java.security.PermissionCollectionTest           (Failed
> in M6)
>
>         are observed in Linux x86.
>
>  2) DRLVM tests:
>    2.1) One failure
>
>         java.lang.ClassGenericsTest.test_2
>
>         is observed in Linux x86 snapshot testing.
>         I can see some old discussion in the mailing list about that but I
> am not sure the expected status here.
>         They should be intemittent errors since the integrity testing just
> run well mostly.
>
>  3) EUTs:
>
>    3.1) Linux x86: 99.36%
>         A recorded JIRA for this suite is HARMONY-2914 which wastes file
> handlers and makes system unstable.
>
>  4) Functional:
>    4.1) Old regressions on both platforms:
>         api.java.text.MessageFormat (HARMONY-5430)
>         api.java.util.jar.Manifest  (HARMONY-5473)
>         api.java.beans.beancontext.BeanContextTest (also in M6, recorded
> as  regression caused by changes in locale data)
>         api.java.beans.persistence.EncoderTest  (also in M6, recorded as
> regression in the beans module)
>         api.java.beans.persistence.EncoderDecoderTest (also in M6,
> regression in the beans module)
>         reg.vm.btest5625 (also in M6, recorded as intermittent and not
> reproducible manually)
>
>    4.2) Old regressions on 1 platform
>         api.java.rmi.basicexception (ERROR in Linux x86, HARMONY-5823)
>         api.java.rmi.basicregistry.RemoteServerTest (ERROR in Linux x86,
> HARMONY-5823)
>         jpda.jdwp.scenario.ST07.ST07Test (ERROR in windows x86_64, in M6 it
> is recorded as regression since M4)
>         java.math.F_BigIntegerMatrixMultiplyTest_01 (ERROR on Linux x86,
> recorded as Timeout, not reproducible
>  in M6)
>         reg.vm.btest5717 (ERROR on Windows X86_64, recorded as "timeout,
> the test is too heavy" in M6)
>         jit.HLO.inline.ControlFlow.IfElse.IfElse1.IfElseTest1 (FAILED in
> windows x86_64, recorded as "looks like
> issue in test" in M6)
>         jit.HLO.devirt.Runtime.RuntimeExtend1 (FAILED on windows x86_64, in
> M6 it is recorded as not regression and started to fail on M5)
>         reg.vm.btest6353.Btest6353 (ERROR on Windows x86_64, recorded also
> failed on M3 and might be issue with the test)
>
>    4.3) New regressions on 1 platform (looks intermittent)
>         reg.jit.btest8029.Btest8029 (FAILED in Linux x86)
>         func.reg.jit.btest5710.Btest5710 (FAILED in Linux x86)
>         api.java.security.cert.F_CertPathTest_06.F_CertPathTest_06 (ERROR
> in Linux x86)
>         api.java.security.cert.F_CertPathTest_05.F_CertPathTest_05 (ERROR
> in Linux x86)
>
>  5) JDKTools Tests:
>    Several timeouts are observed in Linux x86 snapshot running. They are:
>
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch001
>
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch002
>
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch003
>
> org.apache.harmony.jpda.tests.jdwp.DebuggerOnDemand.OnthrowDebuggerLaunchTest.testDebuggerLaunch004
>
> org.apache.harmony.jpda.tests.jdwp.Events.CombinedEventsTest.testCombinedEvents_04
>
> org.apache.harmony.jpda.tests.jdwp.MultiSession.AttachConnectorTest.testAttachConnector001
>
> org.apache.harmony.jpda.tests.jdwp.MultiSession.MethodEntryExitTest.testMethodEvent001
>
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ResumeTest.testResume
>
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadEndTest.testThreadEnd001
>
> org.apache.harmony.jpda.tests.jdwp.MultiSession.ThreadStartTest.testThreadStart001
>    The Linux-only timeouts are also observed in the integrity testing
> results.
>    JIRA HARMONY-5833 has been filed for one of them.
>
>  6) JettyScenario:
>    The Linux x86 running failed because of the unresolved HARMONY-5219.
>
>  7) Reliability:
>    Several failures are observed in windows x86_64 running.
>    7.1) Old regressions
>         api.net.DatagramTest (HARMONY-5531)
>         api.text.DecimalFormat_Locales - (in M6 it is recorded as also
> intermittent in M5)
>
>    7.2) New/intemittent regressions
>         api.kernel.thread.ThreadLocalTest.ThreadLocalTest
>         api.kernel.exec.RunExec
>
>  8) Stress
>    Different test cases failed on different platforms.
>    8.1) Timeouts on Linux x86.
>
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD003.ThreadTest003
>
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD007.ThreadTest007
>
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD009.ThreadTest009
>
> stress.org.apache.harmony.test.stress.jpda.jdwp.scenario.THREAD011.ThreadTest011
>
>    8.2) Failed cases on Windows x86_64 with unknown reasons.
>
> stress.org.apache.harmony.test.stress.classloader.MixThreads.TreeClasses.testTreeClasses2
>
> stress.org.apache.harmony.test.stress.classloader.NotSynchThreads.TreeClasses.testTreeClasses
>
>    I have not found records for them.
>
>  9) Strut_test
>    Broken with same error report as M6.
>
>  10) Eclipse Hello World Application.
>    Although the testing framework just reports EUT-API status as "PASSED".
> A fresh workspace running just failed in configuration stage (
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200805.mbox/%3c6e47b64f0805120106o387a49f1rfb2c33d1042d2f41@mail.gmail.com%3e)
> since SVN commit 641928 (which committed patch for HARMONY-4569).
>
> thanks,
> chunrong
>


-- 
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU