[general][M3] Reliability test suite status (Was: Re: [general] M3 - code frozen)

[Stepan Mishura <st...@gmail.com>]

On 9/25/07, Andrey Yakushev <an...@gmail.com> wrote:
> I also made some analysis for reliability test suite.
> I've found that several tests didn't fail but rather were killed by
> guarding timeout.

Hi Andrey,

I've applied suggested changes but it looks like this didn't improve
the suite status - there are still a lot of failures on the new
snapshot. Could you look into them:
http://people.apache.org/~smishura/r579330/Windows_x86/reliability/
http://people.apache.org/~smishura/r579330/Linux_x86/reliability/

Thanks,
Stepan.

>
> They are:
> api.nio.channels.filechannel.FileChannelMapTest
> api.net.DatagramTest
> vm.finalization.FinalizeThrowRestoreTest
> vm.gc.WeakRefsTest
> api.serialization.SerializableClassesTest
>
> I think it is due to slowing down of the test host after repeated
> tests iteration. I suggest increasing the killing timeout. I've opened
> HARMONY-4851 for this.
>
> I've run excluded reliability tests several times on Windows and Linux
> and I don't see the failures in the following excluded tests now:
>
> api.kernel.thread.Calculation.CalcTest
> api.kernel.thread.StackTraceTest.StackTraceTest
> api.nio.charset.ContainsTest
> api.text.DecimalFormat_Locales
> api.zip.ZlibTest
>
> I suggest removing them from the exclude list. I've opened
> HARMONY-4850 for this.
>
> api.net.HttpConnectionTest fails due to unavailability of
> harmony.apache.org site from the test host network. It works fine if
> site is accessible.
>
> Other failures seem the real reliability issues. These tests fail only
> after several successful iterations.
>
> Thanks,
> Andrey
>
<SNIP>