[jira] [Commented] (HBASE-25598) TestFromClientSide5.testScanMetrics is flaky

["Hudson (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-25598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17290056#comment-17290056 ] 

Hudson commented on HBASE-25598:
--------------------------------

Results for branch branch-2.2
	[build #173 on builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/173/]: (x) *{color:red}-1 overall{color}*
----
details (if available):

(/) {color:green}+1 general checks{color}
-- For more information [see general report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/173//General_Nightly_Build_Report/]




(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/173//JDK8_Nightly_Build_Report_(Hadoop2)/]


(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/173//JDK8_Nightly_Build_Report_(Hadoop3)/]


(/) {color:green}+1 source release artifact{color}
-- See build output for details.


(x) {color:red}-1 client integration test{color}
--Failed when running client tests on top of Hadoop 2. [see log for details|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.2/173//artifact/output-integration/hadoop-2.log]. (note that this means we didn't run on Hadoop 3)


> TestFromClientSide5.testScanMetrics is flaky
> --------------------------------------------
>
>                 Key: HBASE-25598
>                 URL: https://issues.apache.org/jira/browse/HBASE-25598
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 3.0.0-alpha-1, 2.3.4, 2.4.1
>            Reporter: Sun Xin
>            Assignee: Sun Xin
>            Priority: Major
>             Fix For: 3.0.0-alpha-1, 2.2.7, 2.3.5, 2.4.2
>
>
> In some PRs, I got the following errors in UT results.
> {code:java}
> [ERROR] Errors: 
> [ERROR] org.apache.hadoop.hbase.client.TestFromClientSide5.testScanMetrics[0]
> [ERROR]   Run 1: TestFromClientSide5.testScanMetrics:1018 Did not count the result bytes expected:<60> but was:<120>
> [ERROR]   Run 2: TestFromClientSide5.testScanMetrics:1036 Did not count the result bytes expected:<60> but was:<180>
> [ERROR]   Run 3: TestFromClientSide5.testScanMetrics:951 » MasterRegistryFetch Exception making...
> [INFO] 
> [ERROR] org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor5.testScanMetrics[1]
> [ERROR]   Run 1: TestFromClientSideWithCoprocessor5>TestFromClientSide5.testScanMetrics:1036 Did not count the result bytes expected:<60> but was:<120>
> [ERROR]   Run 2: TestFromClientSideWithCoprocessor5>TestFromClientSide5.testScanMetrics:951 » IO
> [ERROR]   Run 3: TestFromClientSideWithCoprocessor5>TestFromClientSide5.testScanMetrics:951 » IO
> [INFO] 
> {code}
> I read the code further and found that this UT is flaky.
> {code:java}
> // check byte counters
> scan2 = new Scan();
> scan2.setScanMetricsEnabled(true);
> scan2.setCaching(1);
> try (ResultScanner scanner = ht.getScanner(scan2)) {
>   int numBytes = 0;
>   for (Result result : scanner.next(1)) {
>     for (Cell cell : result.listCells()) {
>       numBytes += PrivateCellUtil.estimatedSerializedSizeOf(cell);
>     }
>   }
>   scanner.close();
>   ScanMetrics scanMetrics = scanner.getScanMetrics();
>   assertEquals("Did not count the result bytes", numBytes,
>           scanMetrics.countOfBytesInResults.get());
> }
> {code}
> In the code above, it is to check scanMetrics.countOfBytesInResults, but just get only ONE row by scanner.next(1) . A total of 3 rows are inserted into the table, and scanner prefetch from server in advance until maxCacheSize is exceeded, see [here|https://github.com/apache/hbase/blob/5fa15cfde3d77e77ffb1f09d60dce4db264f3831/hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncTableResultScanner.java#L94].
> So if scanner prefetch more than one row before closing scanner, the UT fails. we can reproduce this problem steadily by sleeping before scanner.close().



--
This message was sent by Atlassian Jira
(v8.3.4#803005)