[jira] [Comment Edited] (HBASE-14372) test-patch doesn't properly account for flakey tests

["Sean Busbey (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-14372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14734700#comment-14734700 ] 

Sean Busbey edited comment on HBASE-14372 at 9/8/15 12:00 PM:
--------------------------------------------------------------

yeah, I think that'd be reasonable. I mostly just noted this here so I would remember to check post-Yetus. :)

Also looks like we get the same empty test failures when the retry doesn't succeed:


{code}

     {color:red}-1 core tests{color}.  The patch failed these unit tests:
     

    
{code}

{code}
Results :

Failed tests: 
org.apache.hadoop.hbase.client.TestFastFail.testFastFail(org.apache.hadoop.hbase.client.TestFastFail)
  Run 1: TestFastFail.testFastFail:274 There should be atleast one PreemptiveFastFail exception, otherwise, the test makes little sense.numPreemptiveFastFailExceptions: 0
  Run 2: TestFastFail.testFastFail:274 There should be atleast one PreemptiveFastFail exception, otherwise, the test makes little sense.numPreemptiveFastFailExceptions: 0




Tests run: 2719, Failures: 1, Errors: 0, Skipped: 19
{code}

test saved: https://builds.apache.org/job/PreCommit-HBASE-Build/15455/


was (Author: busbey):
yeah, I think that'd be reasonable. I mostly just noted this here so I would remember to check post-Yetus. :)

Also looks like we get the same empty test failures when the retry doesn't succeed:


{code}

     {color:red}-1 core tests{color}.  The patch failed these unit tests:
     

    
{Code}

{code}
Results :

Failed tests: 
org.apache.hadoop.hbase.client.TestFastFail.testFastFail(org.apache.hadoop.hbase.client.TestFastFail)
  Run 1: TestFastFail.testFastFail:274 There should be atleast one PreemptiveFastFail exception, otherwise, the test makes little sense.numPreemptiveFastFailExceptions: 0
  Run 2: TestFastFail.testFastFail:274 There should be atleast one PreemptiveFastFail exception, otherwise, the test makes little sense.numPreemptiveFastFailExceptions: 0




Tests run: 2719, Failures: 1, Errors: 0, Skipped: 19
{code}

test saved: https://builds.apache.org/job/PreCommit-HBASE-Build/15455/

> test-patch doesn't properly account for flakey tests
> ----------------------------------------------------
>
>                 Key: HBASE-14372
>                 URL: https://issues.apache.org/jira/browse/HBASE-14372
>             Project: HBase
>          Issue Type: Bug
>          Components: build
>            Reporter: Sean Busbey
>
> we use the surefire plugin to rerun failing tests to check for flakeys. if a flakey test fails once and then succeeds we get a vote that looks like this:
> {code}
>      {color:red}-1 core tests{color}.  The patch failed these unit tests:
>      
> {code}
> (note the lack of any listed tests)
> looking back at the actual run we can see:
> {code}
> Flaked tests: 
> org.apache.hadoop.hbase.util.TestHBaseFsck.testSplitDaughtersNotInMeta(org.apache.hadoop.hbase.util.TestHBaseFsck)
>   Run 1: TestHBaseFsck.testSplitDaughtersNotInMeta:1771 null
>   Run 2: PASS
> Tests run: 2642, Failures: 0, Errors: 0, Skipped: 17, Flakes: 1
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)