[jira] [Commented] (HBASE-4485) Eliminate window of missing Data

["Jonathan Hsieh (Commented) (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-4485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127164#comment-13127164 ] 

Jonathan Hsieh commented on HBASE-4485:
---------------------------------------

@Amitanand

I've applied HBASE-2856's from (https://reviews.apache.org/r/2224/diff/#index_header) onto trunk (with minor tweak) and then applied HBASE-4485 but have a compile failure.  Specificially matcher.ignoreNewerKVs() seems to be missing.  Is there another commit that I'm missing?  
                
> Eliminate window of missing Data
> --------------------------------
>
>                 Key: HBASE-4485
>                 URL: https://issues.apache.org/jira/browse/HBASE-4485
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Amitanand Aiyer
>            Assignee: Amitanand Aiyer
>             Fix For: 0.94.0
>
>         Attachments: 4485-v1.diff, 4485-v2.diff, 4485-v3.diff, 4485-v4.diff, repro_bug-4485.diff
>
>
> After incorporating v11 of the 2856 fix, we discovered that we are still having some ACID violations.
> This time, however, the problem is not about including "newer" updates; but, about missing older updates
> that should be including. 
> Here is what seems to be happening.
> There is a race condition in the StoreScanner.getScanners()
>   private List<KeyValueScanner> getScanners(Scan scan,
>       final NavigableSet<byte[]> columns) throws IOException {
>     // First the store file scanners
>     List<StoreFileScanner> sfScanners = StoreFileScanner
>       .getScannersForStoreFiles(store.getStorefiles(), cacheBlocks,
>                                 isGet, false);
>     List<KeyValueScanner> scanners =
>       new ArrayList<KeyValueScanner>(sfScanners.size()+1);
>     // include only those scan files which pass all filters
>     for (StoreFileScanner sfs : sfScanners) {
>       if (sfs.shouldSeek(scan, columns)) {
>         scanners.add(sfs);
>       }
>     }
>     // Then the memstore scanners
>     if (this.store.memstore.shouldSeek(scan)) {
>       scanners.addAll(this.store.memstore.getScanners());
>     }
>     return scanners;
>   }
> If for example there is a call to Store.updateStorefiles() that happens between
> the store.getStorefiles() and this.store.memstore.getScanners(); then
> it is possible that there was a new HFile created, that is not seen by the
> StoreScanner, and the data is not present in the Memstore.snapshot either.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira