You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@harmony.apache.org by Pavel Pervov <pm...@gmail.com> on 2007/09/26 17:15:27 UTC
[vtsvm] no regressions since M2 Re: [general] M3 - code frozen
Linux x86-64 does not count as there were no results for VTSVM for M2 on
Linux x86-64.
I suggest adding currently failing VTSVM tests to exclude list on Linux
x86-64 (~450 tests).
WBR,
Pavel.
On 9/26/07, Andrey Yakushev <an...@gmail.com> wrote:
> > Thanks for your report. Waiting for real issues analysis.
>
> Analysis details:
>
> 9 failures on Windows:
> api.kernel.hooks.AddRmPropertiesHooksTest timeout
> api.kernel.thread.ThreadArrayTest.ThreadArrayTest
> OutOfMemoryError - regression
> api.kernel.threadgroup.EnumerateTest
> OutOfMemoryError - regression
> api.net.DatagramTest timeout
> api.net.HttpConnectionTest configuration
> problem
> api.net.SingleConnectTest timeout
> api.nio.buffers.ByteBufferallocateTest
> IllegalArgumentException - regression
> api.serialization.SerializableClassesTest timeout
> vm.gc.WeakRefsTest timeout
>
> 13 failed on Linux:
>
> api.kernel.exec.RunExec too many files
> opened
> api.kernel.management.SystemResourceOrientedTest vm crash - SIGSEGV
> api.kernel.thread.ThreadArrayTest.ThreadArrayTest OutOfMemoryError
> api.net.DatagramTest timeout
> api.net.HttpConnectionTest configuration
> problem
> api.net.SingleConnectTest address is not
> available – busy socket(?)
> api.nio.buffers.ByteBufferallocateTest
> IllegalArgumentException
> api.nio.channels.filechannel.FileChannelMapTest timeout
> api.nio.channels.filechannel.MappedByteBufferForceTest vm crash - SIGSEGV
> api.serialization.SerializableClassesTest timeout
> vm.finalization.FinalizeThrowRestoreTest timeout
> vm.gc.WeakRefsTest timeout
> vm.stack.StackUnwindTest SIGABRT in VM code
>
> So it seems we have 3 regressions on Windows. I will check if we have
> all these failures submitted as JIRAs.
>
>
> Thanks,
> Andrey
>
>
>
> On 9/25/07, Stepan Mishura <st...@gmail.com> wrote:
> > On 9/25/07, Andrey Yakushev <an...@gmail.com> wrote:
> > > I also made some analysis for reliability test suite.
> > > I've found that several tests didn't fail but rather were killed by
> > > guarding timeout.
> > >
> > > They are:
> > > api.nio.channels.filechannel.FileChannelMapTest
> > > api.net.DatagramTest
> > > vm.finalization.FinalizeThrowRestoreTest
> > > vm.gc.WeakRefsTest
> > > api.serialization.SerializableClassesTest
> > >
> > > I think it is due to slowing down of the test host after repeated
> > > tests iteration. I suggest increasing the killing timeout. I've opened
> > > HARMONY-4851 for this.
> >
> > I'm OK with increasing timeout.
> >
> > >
> > > I've run excluded reliability tests several times on Windows and Linux
> > > and I don't see the failures in the following excluded tests now:
> > >
> > > api.kernel.thread.Calculation.CalcTest
> > > api.kernel.thread.StackTraceTest.StackTraceTest
> > > api.nio.charset.ContainsTest
> > > api.text.DecimalFormat_Locales
> > > api.zip.ZlibTest
> > >
> > > I suggest removing them from the exclude list. I've opened
> > > HARMONY-4850 for this.
> > >
> >
> > I'm OK with the exclude list clean up.
> >
> > > api.net.HttpConnectionTest fails due to unavailability of
> > > harmony.apache.org site from the test host network. It works fine if
> > > site is accessible.
> > >
> >
> > Why not to fix the test to be run against local http server (for
> > example, jetty)?
> >
> > > Other failures seem the real reliability issues. These tests fail only
> > > after several successful iterations.
> > >
> >
> > Thanks for your report. Waiting for real issues analysis.
> >
> > Thanks,
> > Stepan.
> >
> > > Thanks,
> > > Andrey
> > >
> > >
> > >
> > >
> > > On 9/25/07, Tim Ellison <t....@gmail.com> wrote:
> > > > I'll be looking at some of the functional, reliability, and stress
> tests
> > > > failures and seeing if there is anything I can help with. I'll post
> to
> > > > the list for any areas that I pick up.
> > > >
> > > > Regards,
> > > > Tim
> > > >
> > > > Stepan Mishura wrote:
> > > > > On 9/24/07, Stepan Mishura <st...@gmail.com> wrote:
> > > > >> On 9/22/07, Stepan Mishura wrote:
> > > > >>> Hi,
> > > > >>>
> > > > >>> According to the plan - Sept. 22 is code freeze date for M3 so
> let's
> > > > >>> follow policy:
> > > > >>> "no more commits please without agreement from two committers on
> the dev list."
> > > > >>>
> > > > >>> Let's do more testing and analyze if there any critical/blocker
> issues
> > > > >>> for consideration.
> > > > >>> Please raise here issues that you think MUST be fixed for M3.
> > > > >> Here is [1] the status of the last snapshot (r578410) that
> includes
> > > > >> last classlib updates. Unfortunately, not everything is green
> there.
> > > > >> I'm going to inspect all results to make sure that there are no
> > > > >> failures caused by CC configuration issues.
> > > > >>
> > > > >
> > > > > I've compared testing status for the last snapshot r578410 [1]
> with M2 [2][3].
> > > > >
> > > > > We have the following short status for failed suites:
> > > > >
> > > > > Windows_x86:
> > > > > * classlib: 2 tests from pack200 module fail on snapshot (but
> pass
> > > > > on debug build)
> > > > > * Eclipse unit tests 3.2: there is no tests report like for M3.
> The
> > > > > pass rate was improved from 99.32%[3] to 99.7%[1]
> > > > > * Eclipse unit tests 3.3 are new and the pass rate is 99.77%. I
> > > > > think is acceptable
> > > > > * EGA x48 hours scenario fails. According to [4] it passed on
> M2.
> > > > > * Functional: need more analysis, currently I see that 2 tests
> were
> > > > > enabled and new 15 regressions.
> > > > > * Geronimo: 2 regressions
> > > > > * JDK tools: 1 test failed. It might be intermediate failure -
> the
> > > > > test failed due to timeout
> > > > > * Reliability: 65 tests passed for M2 and 64 for M3.
> Investigation
> > > > > is required.
> > > > > * Stress: 190 tests passed for M2 and 189 for M3. Investigation
> is required.
> > > > >
> > > > > Linux_x86:
> > > > > * classlib: 2 tests from pack200 (as for Windows), 1 luni tests
> > > > > failed and 1 crash of security test
> > > > > * Eclipse unit tests 3.2: 2 suites crashed so pass rate is
> 69.60%. I
> > > > > assume this may be CC host configuration issue. Going to
> investigate.
> > > > > * Eclipse unit tests 3.3 are new and the pass rate is 96.47%. I
> > > > > think is acceptable
> > > > > * EGA x48 hours scenario: the same as for Windows (scenario
> fails on M3)
> > > > > * Functional: need more analysis, similar to Windows - some test
> are
> > > > > passing now but there are new failures.
> > > > > * Geronimo: 2 regressions (the same as for Windows)
> > > > > * Reliability: need more analysis
> > > > > * Stress: need more analysis
> > > > >
> > > > > As I remember Sean took pack200 tests. And Alexei Zakharov took
> > > > > security test crash.
> > > > > I'm going to sort things out with Eclipse unit tests 3.2 crash on
> > > > > Linux. And to look info failed jdktools test.
> > > > >
> > > > > So volunteers are required for: EGAx48, Geronimo, functional,
> > > > > reliability and stress suites.
> > > > >
> > > > > Also we have 2 JIRAs to be resolved for M3 under milstone
> unmblella[5]:
> > > > > https://issues.apache.org/jira/browse/HARMONY-4844
> > > > > https://issues.apache.org/jira/browse/HARMONY-4810
> > > > >
> > > > > [1]
> http://people.apache.org/~mloenko/snapshot_testing/script/r578410/index.html
> > > > > [2]
> http://people.apache.org/~mloenko/snapshot_testing/script/r551077/index.html
> > > > > [3] http://wiki.apache.org/harmony/milestones/M2
> > > > > [4]
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200706.mbox/%3c678b3f320706290508l554079dau6af1a82b17d62b54@mail.gmail.com%3e
> > > > > [5] https://issues.apache.org/jira/browse/HARMONY-4843
> > > > >
> > > > > Thanks,
> > > > > Stepan.
> > > > >
> > > > >
> > > > >> Please feel free to pick up any issue for investigation. For
> example,
> > > > >> 2 pack200 classlib test failed. Also there is a crash of
> > > > >>
> org.apache.harmony.security.tests.java.security.cert.serialization.CertificateTest
> > > > >> on Linux x86
> > > > >>
> > > > >> BTW, should we consider BTI's workspace (that we use for M3
> testing)
> > > > >> frozen too?
> > > > >>
> > > > >> [1]
> http://people.apache.org/~mloenko/snapshot_testing/script/r578410/index.html
> > > > >>
> > > > >> Thanks,
> > > > >> Stepan.
> > > > >
> > > >
> > >
> > >
> > > --
> > > Thanks,
> > > Andrey
> >
>
>
> --
> Thanks,
> Andrey
>
--
Pavel Pervov,
Intel Enterprise Solutions Software Division
Re: [vtsvm] no regressions since M2 Re: [general] M3 - code frozen
Posted by Stepan Mishura <st...@gmail.com>.
On 9/27/07, Pavel Pervov <pm...@gmail.com> wrote:
> Stepan,
>
> most of the tests which fail on Linux x86-64 are JVMTI tests.
>
> So, the root cause is that JVMTI is not fully supported and tested on x86-64
> platforms yet.
>
OK. I see.
> I suggest adding currently failing VTSVM tests to exclude list on Linux
x86-64 (~450 tests).
Well, but I see only common exclude list for all platforms so they
won't be run on all platforms. IMO this is not good.
Thanks,
Stepan.
> WBR,
> Pavel.
> On 9/27/07, Stepan Mishura <st...@gmail.com> wrote:
>
> > On 9/26/07, Pavel Pervov <pm...@gmail.com> wrote:
> > > Linux x86-64 does not count as there were no results for VTSVM for M2 on
> > > Linux x86-64.
> > >
> > > I suggest adding currently failing VTSVM tests to exclude list on Linux
> > > x86-64 (~450 tests).
> > >
> >
> > Hi Pavel,
> >
> > Could you give us more info please - why so many tests fail on Linux
> > 64bit? Have you find a root cause?
> >
> > Thanks,
> > Stepan.
> >
> > <SNIP>
> >
>
>
>
> --
> Pavel Pervov,
> Intel Enterprise Solutions Software Division
>
Re: [vtsvm] no regressions since M2 Re: [general] M3 - code frozen
Posted by Pavel Pervov <pm...@gmail.com>.
Stepan,
most of the tests which fail on Linux x86-64 are JVMTI tests.
So, the root cause is that JVMTI is not fully supported and tested on x86-64
platforms yet.
WBR,
Pavel.
On 9/27/07, Stepan Mishura <st...@gmail.com> wrote:
> On 9/26/07, Pavel Pervov <pm...@gmail.com> wrote:
> > Linux x86-64 does not count as there were no results for VTSVM for M2 on
> > Linux x86-64.
> >
> > I suggest adding currently failing VTSVM tests to exclude list on Linux
> > x86-64 (~450 tests).
> >
>
> Hi Pavel,
>
> Could you give us more info please - why so many tests fail on Linux
> 64bit? Have you find a root cause?
>
> Thanks,
> Stepan.
>
> <SNIP>
>
--
Pavel Pervov,
Intel Enterprise Solutions Software Division
Re: [vtsvm] no regressions since M2 Re: [general] M3 - code frozen
Posted by Stepan Mishura <st...@gmail.com>.
On 9/26/07, Pavel Pervov <pm...@gmail.com> wrote:
> Linux x86-64 does not count as there were no results for VTSVM for M2 on
> Linux x86-64.
>
> I suggest adding currently failing VTSVM tests to exclude list on Linux
> x86-64 (~450 tests).
>
Hi Pavel,
Could you give us more info please - why so many tests fail on Linux
64bit? Have you find a root cause?
Thanks,
Stepan.
<SNIP>