You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Enis Soztutar (JIRA)" <ji...@apache.org> on 2013/05/24 01:21:20 UTC
[jira] [Commented] (HBASE-8018) Add "Flaky Testcase Detector" tool
into dev-tools
[ https://issues.apache.org/jira/browse/HBASE-8018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13665822#comment-13665822 ]
Enis Soztutar commented on HBASE-8018:
--------------------------------------
This file: dev-support/jenkins-tools/pom.xml needs a copyright header.
Pre-requirement(run under folder jenkins-tools) => Is this still needed?
Rethinking about following, maybe we should not do this, and have it so that the user has to run mvn package in buildstats before calling the .sh script.
{code}
JAR_URL=http://people.apache.org/~enis/buildstats.jar
{code}
> Add "Flaky Testcase Detector" tool into dev-tools
> -------------------------------------------------
>
> Key: HBASE-8018
> URL: https://issues.apache.org/jira/browse/HBASE-8018
> Project: HBase
> Issue Type: Bug
> Components: util
> Reporter: Jeffrey Zhong
> Assignee: Jeffrey Zhong
> Fix For: 0.98.0
>
> Attachments: hbase-8018.patch, hbase-8018-v1.patch
>
>
> Recently I'm working on some hbase test case failures and I think it's useful if we can see a report of all failed test cases from most recent runs. So we can easily see how flaky a test case is. I wrote a tool some time back and below are some reports against different branches from today's run. You can get the tools' source at https://github.com/jeffreyz88/jenkins-tools. If we can run it daily and send out an email then we can quickly notice what may break after recent check-ins.
> Notes:
> 1) " 873 874 875 876 877 878 879 880 881" are builds which has failed test cases in current or previous runs
> 2) 1 means "PASSED", 0 means "NOT RUN AT ALL", -1 means "FAILED"
> HBase-0.95 (from last 10 runs configurable)
> Failed Test Cases 21 22 23 24 25 27
> ========================================================
> org.apache.hadoop.hbase.catalog.testmetamigrationconvertingtopb.org.apache.hadoop.hbase.catalog.testmetamigrationconvertingtopb 0 0 0 0 -1 -1
> org.apache.hadoop.hbase.coprocessor.example.testbulkdeleteprotocol.testbulkdeletecolumn -1 0 0 0 0 0
> org.apache.hadoop.hbase.coprocessor.example.testrowcountendpoint.org.apache.hadoop.hbase.coprocessor.example.testrowcountendpoint -1 0 0 0 0 0
> org.apache.hadoop.hbase.coprocessor.example.testzookeeperscanpolicyobserver.org.apache.hadoop.hbase.coprocessor.example.testzookeeperscanpolicyobserver -1 0 0 0 0 0
> org.apache.hadoop.hbase.master.testmasterfailover.testmasterfailoverwithmockedritondeadrs 1 -1 -1 0 1 1
> org.apache.hadoop.hbase.regionserver.testsplittransactiononcluster.testshouldthrowioexceptionifstorefilesizeisemptyandshouldsuccessfullyexecuterollback 1 1 1 1 1 -1
> org.apache.hadoop.hbase.regionserver.testsplittransactiononcluster.testshutdownfixupwhendaughterhassplit 1 1 1 -1 -1 -1
> org.apache.hadoop.hbase.regionserver.wal.testhlog.testlogcleaning 0 1 1 1 -1 0
> org.apache.hadoop.hbase.replication.testmasterreplication.testcyclicreplication 1 1 1 1 1 -1
> org.apache.hadoop.hbase.replication.testreplicationqueuefailover.queuefailover 1 -1 0 1 -1 0
> org.apache.hadoop.hbase.replication.testreplicationqueuefailovercompressed.queuefailover 0 1 -1 0 -1 0
> org.apache.hadoop.hbase.security.access.testaccesscontroller.org.apache.hadoop.hbase.security.access.testaccesscontroller 0 -1 0 0 0 0
> As you can see we have a few test cases don't run successfully at all or recently.
> HBase-TRUNK (from last 10 runs)
> Failed Test Cases 3908 3909 3910 3912 3913 3914 3915 3916
> ========================================================
> org.apache.hadoop.hbase.catalog.testmetamigrationconvertingtopb.org.apache.hadoop.hbase.catalog.testmetamigrationconvertingtopb 0 -1 0 0 0 0 -1 -1
> org.apache.hadoop.hbase.client.testadmin.testcloseregionwhenservernameisempty 1 1 1 1 1 1 -1 0
> org.apache.hadoop.hbase.client.testscannertimeout.test3686a 1 1 -1 0 1 1 1 1
> org.apache.hadoop.hbase.client.testsnapshotcloneindependence.testofflinesnapshotregionoperationsindependent 0 1 -1 0 1 1 1 1
> org.apache.hadoop.hbase.client.testsnapshotcloneindependence.testonlinesnapshotregionoperationsindependent 1 1 -1 0 1 1 1 1
> org.apache.hadoop.hbase.master.testassignmentmanageroncluster.testmoveregion 1 1 -1 0 1 1 1 1
> org.apache.hadoop.hbase.master.testdistributedlogsplitting.testdelayeddeleteonfailure 1 1 -1 0 1 1 1 1
> org.apache.hadoop.hbase.master.testmasterfailover.testmasterfailoverwithmockedritondeadrs 1 1 1 1 1 -1 0 1
> org.apache.hadoop.hbase.master.testtablelockmanager.testdelete 1 1 -1 0 1 1 1 1
> org.apache.hadoop.hbase.procedure.testzkprocedurecontrollers.testsimplezkcohortmembercontroller 1 1 1 1 1 1 -1 0
> org.apache.hadoop.hbase.regionserver.testsplittransactiononcluster.testexistingznodeblockssplitandwerollback 1 1 1 1 1 -1 0 1
> org.apache.hadoop.hbase.regionserver.testsplittransactiononcluster.testshouldthrowioexceptionifstorefilesizeisemptyandshouldsuccessfullyexecuterollback 1 1 -1 0 1 1 1 1
> org.apache.hadoop.hbase.regionserver.testsplittransactiononcluster.testshutdownfixupwhendaughterhassplit -1 -1 -1 0 -1 0 -1 0
> org.apache.hadoop.hbase.regionserver.wal.testhlog.testlogcleaning 0 -1 -1 0 1 1 1 1
> org.apache.hadoop.hbase.replication.testmultislavereplication.testmultislavereplication 1 1 -1 0 1 1 1 1
> org.apache.hadoop.hbase.replication.testreplicationqueuefailover.queuefailover 1 1 1 -1 0 -1 0 1
> org.apache.hadoop.hbase.replication.testreplicationqueuefailovercompressed.queuefailover 1 1 1 -1 0 -1 0 1
> org.apache.hadoop.hbase.security.access.testaccesscontrolfilter.testqualifieraccess -1 0 1 -1 0 1 1 1
> org.apache.hadoop.hbase.util.testhbasefsck.testdegenerateregions 1 -1 0 1 1 1 1 1
> org.apache.hadoop.hbase.util.testhbasefsck.testfixassignmentsandnohdfschecking 1 -1 0 1 1 1 1 1
> org.apache.hadoop.hbase.util.testhbasefsck.testfixbytable 1 -1 0 1 1 1 1 1
> org.apache.hadoop.hbase.util.testhbasefsck.testhbckthreadpooling 1 -1 0 1 1 1 1 1
> org.apache.hadoop.hbase.util.testhbasefsck.testlingeringreferencefile 1 -1 0 1 1 1 1 1
> org.apache.hadoop.hbase.util.testhbasefsck.testmissingfirstregion 1 -1 0 1 1 1 1 1
> org.apache.hadoop.hbase.util.testhbasefsck.testmissinglastregion 1 -1 0 1 1 1 1 1
> org.apache.hadoop.hbase.util.testhbasefsck.testnotinhdfs 1 -1 0 1 1 1 1 1
> org.apache.hadoop.hbase.util.testhbasefsck.testnotinmetaordeployedhole 1 -1 0 1 1 1 1 1
> org.apache.hadoop.hbase.util.testhbasefsck.testoverlapandorphan 1 -1 0 1 1 1 1 1
> org.apache.hadoop.hbase.util.testhbasefsck.testregionhole 1 -1 0 1 1 1 1 1
> org.apache.hadoop.hbase.util.testhbasefsck.testregionshouldnotbedeployed 1 -1 0 1 1 1 1 1
> org.apache.hadoop.hbase.util.testhbasefsck.testsidelineoverlapregion 1 -1 0 1 1 1 1 1
> org.apache.hadoop.hbase.util.testhbasefsck.testsplitdaughtersnotinmeta 1 -1 0 1 1 1 1 1
> More test cases are not in good shape.
> HBase-0.94(from last 10 runs)
> Failed Test Cases 873 874 875 876 877 878 879 880 881
> ========================================================
> org.apache.hadoop.hbase.backup.testhfilearchiving.testcleaningrace -1 0 1 1 1 1 1 1 1
> org.apache.hadoop.hbase.catalog.testmetareadereditor.testgetregion 1 1 -1 0 1 1 1 1 1
> org.apache.hadoop.hbase.catalog.testmetareadereditor.testretrying 1 1 -1 0 1 1 1 1 1
> org.apache.hadoop.hbase.catalog.testmetareadereditor.testscanmetafortable 1 1 -1 0 1 1 1 1 1
> org.apache.hadoop.hbase.catalog.testmetareadereditor.testtableexists 1 1 -1 0 1 1 1 1 1
> org.apache.hadoop.hbase.client.testmetamigrationremovinghtd.org.apache.hadoop.hbase.client.testmetamigrationremovinghtd 0 -1 0 0 0 0 0 0 0
> org.apache.hadoop.hbase.client.testmultiparallel.testflushcommitsnoabort 1 1 1 1 1 -1 0 1 1
> org.apache.hadoop.hbase.client.testmultiparallel.testflushcommitswithabort 1 1 1 1 1 -1 0 1 1
> org.apache.hadoop.hbase.client.testshell.testrunshelltests 1 1 1 1 1 1 -1 0 1
> org.apache.hadoop.hbase.master.testmasterfailover.testmasterfailoverwithmockedritondeadrs 1 1 -1 -1 0 1 1 1 1
> org.apache.hadoop.hbase.regionserver.testatomicoperation.testmultirowmutationmultithreads 1 1 1 -1 0 1 1 1 1
> org.apache.hadoop.hbase.regionserver.testsplittransactiononcluster.testshutdownfixupwhendaughterhassplit 1 -1 0 1 1 1 1 1 1
> As you can see 0.94 is stable and testmetamigrationremovinghtd hasn't been run for a while either being removed or due to other failures. Some test cases aren't stable but not too bad.
> Thanks,
> -Jeffrey
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira