[jira] [Created] (HBASE-30351) ROWS_SCANNED scan metric double-counts rows passing the filter when the scan uses the joined heap (essential column family filtering)

["Jeongmin Kim (Jira)" <ji...@apache.org>]

Jeongmin Kim created HBASE-30351:
------------------------------------

             Summary: ROWS_SCANNED scan metric double-counts rows passing the filter when the scan uses the joined heap (essential column family filtering)
                 Key: HBASE-30351
                 URL: https://issues.apache.org/jira/browse/HBASE-30351
             Project: HBase
          Issue Type: Bug
          Components: metrics, Scanners
    Affects Versions: 2.6.7, 3.0.0, 4.0.0-alpha-1
            Reporter: Jeongmin Kim


When a scan enables {{Scan#setLoadColumnFamiliesOnDemand(true)}} and its filter marks some column families as non-essential ({{{}Filter#isFamilyEssential{}}}, introduced by HBASE-5416), {{RegionScannerImpl}} populates a row in two steps: the essential families through the store heap and — only if the row passes the filter — the remaining families through the joined heap.

Both steps go through {{{}RegionScannerImpl#populateResult{}}}, which increments the ROWS_SCANNED metric ({{{}ServerSideScanMetrics.COUNT_OF_ROWS_SCANNED_KEY_METRIC_NAME{}}}, introduced by HBASE-5980) each time the given heap finishes the current row:


{code:java}
nextKv = heap.peek();
moreCellsInRow = moreCellsInRow(nextKv, currentRowCell);
if (!moreCellsInRow) {
incrementCountOfRowsScannedMetric(scannerContext);
} {code}
There is no guard against the second call on the joined heap, so every row that passes the filter and has data in a non-essential family is counted twice — once when the store heap finishes the row and once more when the joined heap finishes it. Rows rejected by the filter are counted once. The reported value effectively becomes "rows scanned + rows returned", which breaks the metric for its usual purposes (filter selectivity / scan efficiency analysis, the MapReduce {{ROWS_SCANNED}} HBase counter, comparisons against {{{}ROWS_FILTERED{}}}).

Reproduction with stock filters only:
 # Create a table with two column families, e.g. {{essential}} and {{{}joined{}}}, and put a cell in both families for every row.
 # Scan with {{{}setScanMetricsEnabled(true){}}}, {{setLoadColumnFamiliesOnDemand(true)}} and a {{SingleColumnValueFilter}} on family {{essential}} with {{setFilterIfMissing(true)}} (this makes the other family non-essential via {{{}SingleColumnValueFilter#isFamilyEssential{}}}).
 # With 10 rows of which 5 match the filter, ROWS_SCANNED reports 15 instead of 10, while ROWS_FILTERED correctly reports 5. Without {{setLoadColumnFamiliesOnDemand(true)}} the same scan reports 10.

The double counting is invisible in the existing {{TestServerSideScanMetricsFromClientSide}} because its table has a single column family, so the joined heap is never exercised.

Proposed fix: count a completed row in {{populateResult}} only when populating from the store heap. The joined heap only re-populates rows that already passed the filter, so the store heap completion is the single canonical "row scanned" event (rows filtered by row key keep their separate increment in {{{}nextInternal{}}}). A PR with the fix and a unit test in {{TestServerSideScanMetricsFromClientSide}} follows.

Verified on branch-2, branch-2.6, branch-3 and master (same code in {{populateResult}} / {{{}populateFromJoinedHeap{}}}).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)