[jira] [Commented] (HBASE-21070) SnapshotFileCache won't update for snapshots stored in S3

["Zach York (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-21070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16605070#comment-16605070 ] 

Zach York commented on HBASE-21070:
-----------------------------------

Sorry for being a little absent on this over the past couple days. I found a bug in the original test code [https://github.com/apache/hbase/blob/master/hbase-server/src/test/java/org/apache/hadoop/hbase/master/snapshot/TestSnapshotFileCache.java#L261] is calling contains for a set of FileStatus's and trying to find a String which will never return true... I've been trying to wrestle to get the tests to perform as expected, but it has taken more time. I should probably fix it in a follow-up JIRA.

> SnapshotFileCache won't update for snapshots stored in S3
> ---------------------------------------------------------
>
>                 Key: HBASE-21070
>                 URL: https://issues.apache.org/jira/browse/HBASE-21070
>             Project: HBase
>          Issue Type: Bug
>          Components: snapshots
>    Affects Versions: 3.0.0, 2.1.1, 1.4.7
>            Reporter: Zach York
>            Assignee: Zach York
>            Priority: Critical
>              Labels: FSRedo
>         Attachments: HBASE-21070.master.001.patch, HBASE-21070.master.002.patch
>
>
> The SnapshotFileCache depends on last modified time to determine whether to update the Snapshot HFile cache. However, in S3, real 'folders' don't exist. S3 filesystems create a dummy file in place of a folder, but the dummy file last modified time is not updated when files are changed 'under' it. This means that the SnapshotFileCache doesn't pick up new snapshot HFiles and these files aren't removed from the HFileCleaner and can be eligible for deletion.
>  
> My patch removes the lastmodified assumption.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)