You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Jingcheng Du (JIRA)" <ji...@apache.org> on 2016/10/12 08:50:20 UTC
[jira] [Updated] (HBASE-16578) Mob data loss after mob compaction
and normal compcation
[ https://issues.apache.org/jira/browse/HBASE-16578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jingcheng Du updated HBASE-16578:
---------------------------------
Description:
StoreFileScanners on MOB cells rely on the scannerOrder to find the latest cells after mob compaction. The value of scannerOrder is assigned by the order of maxSeqId of StoreFile, and this maxSeqId is valued only after the reader of the StoreFile is created.
In {{Compactor.compact}}, the compacted store files are cloned and their readers are not created. And in {{StoreFileScanner.getScannersForStoreFiles}} the StoreFiles are sorted before the readers are created and at that time the maxSeqId for each file is -1 (the default value). This will lead to a chaos in scanners in the following compaction. Some older cells might be chosen during the compaction.
We need to create readers either before the sorting in the method {{StoreFileScanner.getScannersForStoreFiles}}, or create readers just after the store files are cloned in {{Compactor.compact}}.
was:
StoreFileScanners on MOB cells rely on the scannerOrder to find the latest cells after compaction. The value of scannerOrder is assigned by the order of maxSeqId of StoreFile, and this maxSeqId is valued only after the reader of the StoreFile is created.
In {{StoreFileScanner.getScannersForStoreFiles}}, the StoreFiles are sorted before the readers are created and at that time the maxSeqId for each file is -1 (the default value). This will lead to a chaos in scanners on MOB cells. We need to create readers just before the sorting in the method {{StoreFileScanner.getScannersForStoreFiles}}.
> Mob data loss after mob compaction and normal compcation
> --------------------------------------------------------
>
> Key: HBASE-16578
> URL: https://issues.apache.org/jira/browse/HBASE-16578
> Project: HBase
> Issue Type: Bug
> Components: mob
> Affects Versions: 2.0.0
> Reporter: huaxiang sun
> Assignee: Jingcheng Du
> Attachments: TestMobCompaction.java, TestMobCompaction.java
>
>
> StoreFileScanners on MOB cells rely on the scannerOrder to find the latest cells after mob compaction. The value of scannerOrder is assigned by the order of maxSeqId of StoreFile, and this maxSeqId is valued only after the reader of the StoreFile is created.
> In {{Compactor.compact}}, the compacted store files are cloned and their readers are not created. And in {{StoreFileScanner.getScannersForStoreFiles}} the StoreFiles are sorted before the readers are created and at that time the maxSeqId for each file is -1 (the default value). This will lead to a chaos in scanners in the following compaction. Some older cells might be chosen during the compaction.
> We need to create readers either before the sorting in the method {{StoreFileScanner.getScannersForStoreFiles}}, or create readers just after the store files are cloned in {{Compactor.compact}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)