[jira] [Commented] (HBASE-30246) MOB compaction does not close MobCell after resolving reference cells

["Hudson (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-30246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18092846#comment-18092846 ] 

Hudson commented on HBASE-30246:
--------------------------------

Results for branch branch-2.6
	[build #63 on builds.a.o|https://ci-hbase.apache.org/job/HBase-Backwards-Compatibility-Test/job/branch-2.6/63/]: (x) *{color:red}-1 overall{color}*
----
Backwards compatibility checks:
(/) {color:green}+1 jdk17 hadoop 3.3.5 backward compatibility checks{color}
-- For more information [see jdk17 report|https://ci-hbase.apache.org/job/HBase-Backwards-Compatibility-Test/job/branch-2.6/63/console]


(/) {color:green}+1 jdk17 hadoop 3.3.6 backward compatibility checks{color}
-- For more information [see jdk17 report|https://ci-hbase.apache.org/job/HBase-Backwards-Compatibility-Test/job/branch-2.6/63/console]


(/) {color:green}+1 jdk17 hadoop 3.4.0 backward compatibility checks{color}
-- For more information [see jdk17 report|https://ci-hbase.apache.org/job/HBase-Backwards-Compatibility-Test/job/branch-2.6/63/console]


(x) {color:red}-1 jdk17 hadoop 3.4.1 backward compatibility checks{color}
-- For more information [see jdk17 report|https://ci-hbase.apache.org/job/HBase-Backwards-Compatibility-Test/job/branch-2.6/63/console]


(/) {color:green}+1 jdk17 hadoop 3.4.2 backward compatibility checks{color}
-- For more information [see jdk17 report|https://ci-hbase.apache.org/job/HBase-Backwards-Compatibility-Test/job/branch-2.6/63/console]


> MOB compaction does not close MobCell after resolving reference cells
> ---------------------------------------------------------------------
>
>                 Key: HBASE-30246
>                 URL: https://issues.apache.org/jira/browse/HBASE-30246
>             Project: HBase
>          Issue Type: Bug
>          Components: Compaction, mob
>            Reporter: Xiao Liu
>            Assignee: Xiao Liu
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 2.7.0, 3.0.0-beta-2, 2.5.16, 2.6.7
>
>
> During MOB compaction, {{DefaultMobStoreCompactor}} resolves each MOB reference cell via:
> {code:java}
> mobCell = mobStore.resolve(c, cacheMobBlocksOnCompaction, false).getCell();
> {code}
> {{HMobStore#resolve(...)}} returns a {{Closeable}} {{MobCell}} that owns the
> {{StoreFileScanner}} opened against the MOB file (and the off-heap ByteBuffers backing
> the cell). The code calls {{getCell()}} and drops the {{MobCell}}, so the scanner/buffers
> are never released — once per reference cell, for the whole compaction. On the off-heap
> read path this pressures the {{ByteBuffAllocator}} pool and can lead to buffer exhaustion
> during large MOB compactions. The bug is long-standing (it predates HBASE-30177, which only
> changed the cacheBlocks argument) and was spotted during the HBASE-30177 review.
> h3. Fix
> Close the {{MobCell}} via try-with-resources right after resolving. The cell must be copied
> to the heap *before* close, because closing releases/recycles the buffers backing it, while
> HFile writers/encoders retain references to appended cells (lastCell, firstCellInBlock,
> encoder prevCell) until {{beforeShipped()}}. Closing without copying would turn the leak into
> silent HFile corruption.
> {code:java}
> protected ExtendedCell resolveMobCell(ExtendedCell reference) throws IOException {
>   try (MobCell mobCell = mobStore.resolve(reference, cacheMobBlocksOnCompaction, false)) {
>     return KeyValueUtil.copyToNewKeyValue(mobCell.getCell());
>   }
> }
> {code}
> h3. Testing
> * New {{TestDefaultMobStoreCompactor#testResolveMobCellClosesMobCellAndReturnsIndependentCopy}}
>   asserts {{close()}} is called and an independent heap copy is returned.
> * {{TestHMobStore}} updated to close resolved {{MobCell}} s.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)