You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@harmony.apache.org by Apache Wiki <wi...@apache.org> on 2008/03/04 14:59:04 UTC
[Harmony Wiki] Update of "Reliability Tests on DRLVM" by Vladimir Beliaev
Dear Wiki user,
You have subscribed to a wiki page or wiki category on "Harmony Wiki" for change notification.
The following page has been changed by Vladimir Beliaev:
http://wiki.apache.org/harmony/Reliability_Tests_on_DRLVM
The comment on the change is:
Reformatted, added comments...
------------------------------------------------------------------------------
[wiki:Self:DRLVMTestTracking Back to DRLVM Test Tracking]
- '''Reliability Tests suite''' contans 100 tests, 13 of which are considered as invalid currently.
+ [[Anchor(Summary)]]
+ === Summary ===
- '''Here are the results of failures analysis'''
+ ~+Reliability Tests Suite+~ contans 100 tests including 13 that are considered as invalid currently. This suite is critical for testing Harmony readiness for running long large workloads such as Eclipse in the development mode. {*}
+ __~+Pass rates+~ according to
- '''Pass rates''' according to [http://people.apache.org/~mloenko/snapshot_testing/script/r629320/index.html cruise control results]:
+ [http://people.apache.org/~mloenko/snapshot_testing/script/r629320/index.html Automated Testing results]:__
+ ||<rowbgcolor="#a0f0a0">Platform||Valid||Failed||Excluded||Failed+excluded||Pass rate||Covered by JIRA||
+ ||Windows x86 ||87 || 2 ||13 || 15|| 83%|| 15||
+ ||Linux x86 ||87 || 3 ||13 || 16|| 82%|| 16||
+ ||Linux x86_64 ||87 || 2 ||28 || 30|| 65%|| 30||
+ __~+Regressions+~ since M4 release to be resolved ASAP: __
- ||Platform ||Valid||Failed||Excluded||Failed+excluded||Pass rate||Covered by JIRA||
- ||Windows x86 ||87 || 2 ||13 || 15|| 83%|| 15||
- ||Linux x86 ||87 || 3 ||13 || 16|| 82%|| 16||
- ||Linux x86_64||87 || 2 ||28 || 30|| 65%|| 30||
+ ||<rowbgcolor="#f0a0a0">'''Test name'''||'''JIRA'''||'''Summary'''||
- '''Regressions'''
-
- ||'''Excluded test name'''||'''Comments'''||
- ||vm.classloading.ClassCastTest|| [https://issues.apache.org/jira/browse/HARMONY-5395 HARMONY-5395 ]||
+ ||vm.classloading.ClassCastTest|| [https://issues.apache.org/jira/browse/HARMONY-5395 HARMONY-5395 ]||[drlvm][Linux64 gcc bug] vm.classloading.ClassCastTest hangs||
- ||api.serialization.SerializableClassesTest||[https://issues.apache.org/jira/browse/HARMONY-5532 HARMONY-5532]||
+ ||api.serialization.SerializableClassesTest||[https://issues.apache.org/jira/browse/HARMONY-5532 HARMONY-5532]||[classlib] SerializableClassesTest failure from the reliability test suite||
- '''Latest Windows x86 failures'''
- ||'''Test name'''||'''Comments'''||
+ [[Anchor(Latest)]]
+ === Latest failures under evaluation ===
+
+ {i} This section contains the failures happens during several recent runs. These failures are being evaluated currently - based on results of evaluation each failure can be marked as a regression and moved to the ~+Regressions+~ list above or it can be marked as non critical and related test can be just excluded.
+
+ ||||<rowbgcolor="#c0c0c0">'''Latest Windows x86 failures'''||
+ ||<rowbgcolor="#f0f0f0">'''''Test name'''''||'''''Comments'''''||
||api.net.DatagramTest||[https://issues.apache.org/jira/browse/HARMONY-5531 HARMONY-5531] ||
||api.text.getSentenceInstance||intermittent VM crash, requires investigation||
-
- '''Latest Linux x86 failures'''
- ||'''Test name'''||'''Comments'''||
+ ||||||
+ ||||||
+ ||||<rowbgcolor="#c0c0c0">'''Latest Linux x86 failures'''||
+ ||<rowbgcolor="#f0f0f0">'''''Test name'''''||'''''Comments'''''||
||api.net.SingleConnectTest||[https://issues.apache.org/jira/browse/HARMONY-5266 HARMONY-5266 ] problem similar to HttpConnectionTest||
||api.serialization.SerializableClassesTest|| [https://issues.apache.org/jira/browse/HARMONY-5532 HARMONY-5532]||
||api.util.ConcurrentTest||[https://issues.apache.org/jira/browse/HARMONY-5403 HARMONY-5403] regression ||
@@ -35, +42 @@
||api.kernel.management.SystemResourceOrientedTest|| [https://issues.apache.org/jira/browse/HARMONY-5533 HARMONY-5533] ||
||api.kernel.thread.RecursiveThreadTest.RecursiveTest|| [https://issues.apache.org/jira/browse/HARMONY-5534 HARMONY-5534] ||
||api.text.DecimalFormat_Locales|| random vm crash, requires investigation||
-
- '''Latest Linux x86_64 failures'''
-
- ||'''Test name'''||'''Comments'''||
+ ||||||
+ ||||||
+ ||||<rowbgcolor="#c0c0c0">'''Latest Linux x86_64 failures'''||
+ ||<rowbgcolor="#f0f0f0">'''''Test name'''''||'''''Comments'''''||
||api.util.ConcurrentTest|| [https://issues.apache.org/jira/browse/HARMONY-5403 HARMONY-5403 ] regression ||
||api.kernel.thread.ThreadLocalTest.ThreadLocalTest|| not reproducible||
+ ||||||
+ ||||||
+ ||||<rowbgcolor="#c0c0c0">'''Latest failures for all platforms'''||
+ ||<rowbgcolor="#f0f0f0">'''''Test name'''''||'''''Comments'''''||
+ ||DecimalFormat_Locales||This failure is not reproducible locally during 10+ run of 20 minutes each||
+ ||ThreadLocalTest||This failure is not reproducible locally during 10+ run of 20 minutes each||
- '''Common excluded test status'''
- Failed tests
+ [[Anchor(Excluded)]]
+ === List of excluded tests with status ===
- ||'''Excluded test name'''||'''Comments'''||
+ {X} This section lists the tests that are currently excluded. The cause of each failure is given for the moment the failure was evaluated, so this cause information may be a bit out of date now. In general a new failure analysis is required for most of the failures below to find the root cause of issue and you are welcome to do it to increase the overall reliability of Harmony B)
+
+ ||||<rowbgcolor="#c0c0c0">'''Valid tests that failed'''||
+ ||<rowbgcolor="#f0f0f0">'''''Excluded test name'''''||'''''Comments'''''||
+ ||||||
||api.kernel.pckg.PackageGetTest|| [https://issues.apache.org/jira/browse/HARMONY-5512 HARMONY-5512]||
||api.kernel.thread.ExcptHandlerTest.ExcptHandlerTest|| [https://issues.apache.org/jira/browse/HARMONY-5402 HARMONY-5402 ], mostly reproduced on release build||
||api.kernel.thread.LifeSimulationTest.Life|| [https://issues.apache.org/jira/browse/HARMONY-5334 HARMONY-5334 ], memory leak||
@@ -59, +76 @@
||api.kernel.thread.Calculation.CalcTest|| [https://issues.apache.org/jira/browse/HARMONY-5334 HARMONY-5334 ]increased memory leak||
||api.kernel.thread.ThreadArrayTest.ThreadArrayTest|| [https://issues.apache.org/jira/browse/HARMONY-5334 HARMONY-5334 ]increased memory leak||
||api.kernel.object.ObjectFinalizeTest|| [https://issues.apache.org/jira/browse/HARMONY-5334 HARMONY-5334 ]increased memory leak||
-
- Invalid tests
-
- ||'''Excluded test name'''||'''Comments'''||
+ ||||||
+ ||||||
+ ||||||
+ ||||||
+ ||||<rowbgcolor="#c0c0c0">'''Linux x86_64 excluded test status'''||
+ ||<rowbgcolor="#f0f0f0">'''''Excluded test name'''''||'''''Comments'''''||
+ ||||||
+ ||api.kernel.thread.VolatileVariableTest.DekkerTest||[https://issues.apache.org/jira/browse/HARMONY-5295 HARMONY-5295]; intermittent hang||
+ ||api.net.DatagramTest|| [https://issues.apache.org/jira/browse/HARMONY-5531 HARMONY-5531]; intermittent hang||
+ ||api.nio.channels.filechannel.MappedByteBufferForceTest|| [https://issues.apache.org/jira/browse/HARMONY-5553 HARMONY-5553]||
+ ||api.serialization.SerializableClassesTest|| [https://issues.apache.org/jira/browse/HARMONY-5532 HARMONY-5532]||
+ ||vm.classloading.DelegationModelTest||[https://issues.apache.org/jira/browse/HARMONY-5567 HARMONY-5567]||
+ ||||||
+ ||||||
+ ||||||
+ ||||||
+ ||||<rowbgcolor="#c0c0c0">'''Invalid tests''' ||
+ ||<rowbgcolor="#f0f0f0">'''''Excluded test name'''''||'''''Comments'''''||
+ ||||||
||api.kernel.thread.ThreadKill.ThreadKillTest|| [https://issues.apache.org/jira/browse/HARMONY-4220 HARMONY-4220 ], HARMONY-4232, the issue could be in the test||
||api.net.MultiConnectTest|| [https://issues.apache.org/jira/browse/HARMONY-4812 HARMONY-4812 ], Test issue||
||api.nio.charset.CharsetSyncCacheTest|| [https://issues.apache.org/jira/browse/HARMONY-5400 HARMONY-5400 ], reproducible on release and debug; memory leak, may be ICU bug||
@@ -77, +109 @@
||vm.stack.StackFramesRestoreTest|| [https://issues.apache.org/jira/browse/HARMONY-2039 HARMONY-2039 ], no class unloading, memory leak||
||api.kernel.thread.VolatileVariableTest.DekkerTest|| [https://issues.apache.org/jira/browse/HARMONY-5295 HARMONY-5295 ], intermittent hang, also sometimes failed on SUN and BEA; probably test issue||
- '''Linux x86_64 excluded test status'''
+ [#Summary Back to Summary]
+ [wiki:Self:DRLVMTestTracking Back to DRLVM Test Tracking]
- ||'''Excluded test name'''||'''Comment'''||
- ||api.kernel.thread.VolatileVariableTest.DekkerTest||[https://issues.apache.org/jira/browse/HARMONY-5295 HARMONY-5295]; intermittent hang||
- ||api.net.DatagramTest|| [https://issues.apache.org/jira/browse/HARMONY-5531 HARMONY-5531]; intermittent hang||
- ||api.nio.channels.filechannel.MappedByteBufferForceTest|| [https://issues.apache.org/jira/browse/HARMONY-5553 HARMONY-5553]||
- ||api.serialization.SerializableClassesTest|| [https://issues.apache.org/jira/browse/HARMONY-5532 HARMONY-5532]||
- ||vm.classloading.DelegationModelTest||[https://issues.apache.org/jira/browse/HARMONY-5567 HARMONY-5567]||
-
- '''List of failures occured on cruise control hosts, which was not reproduced on local run after at least 10 test run20 minutes each:'''
-
- [http://people.apache.org/~smishura/r613602/Linux_x86_64/reliability/api/text/42_DecimalFormat_Locales.html DecimalFormat_Locales]
-
- [http://people.apache.org/~smishura/r620734/Linux_x86_64/reliability/api/kernel/thread/ThreadLocalTest/16_ThreadLocalTest.html ThreadLocalTest]
-