[jira] [Commented] (HBASE-19275) TestSnapshotFileCache never worked properly

["Xu Cang (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-19275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16552249#comment-16552249 ] 

Xu Cang commented on HBASE-19275:
---------------------------------

[~apurtell] I put some time today to debug this, below is my findings.

#createAndTestSnapshot tries to verify file existence before and after "refreshCache", but this won't work because #getUnreferencedFiles is called, inside this method, it calls #refreshCache here:  [https://github.com/apache/hbase/blob/c3b4f788b16ac4e0e8cfd319f495308ba4d158f5/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotFileCache.java#L190] 

Also, #refreshCache will be called periodically from RefreshCacheTask.  

So, I think it's good and safe to remove this check as shown below:

 
{quote}fs.delete(builder.getSnapshotsDir(), true);
FSUtils.logFileSystemState(fs, rootDir, LOG);

- // The files should be in cache until next refresh
- for (Path filePath: files) {
- Iterable<FileStatus> nonSnapshotFiles = getNonSnapshotFiles(cache, filePath);
- assertFalse("Cache didn't find " + filePath.getName(), Iterables.contains(nonSnapshotFiles,
- filePath.getName()));
- }
{quote}
I am uploading a new patch for review. please let me know if I understood something wrong. 

> TestSnapshotFileCache never worked properly
> -------------------------------------------
>
>                 Key: HBASE-19275
>                 URL: https://issues.apache.org/jira/browse/HBASE-19275
>             Project: HBase
>          Issue Type: Sub-task
>    Affects Versions: 3.0.0, 1.4.0, 1.5.0, 2.0.0
>            Reporter: Andrew Purtell
>            Priority: Major
>         Attachments: HBASE-19275-branch-1.patch
>
>
> Error-prone noticed we were asking Iterables.contains() questions with the wrong type in TestSnapshotFileCache. I've attached a fixed version of the test. The results suggest the cache is not evicting entries properly. 
> {noformat}
> java.lang.AssertionError: Cache found 'hdfs://localhost:52867/user/apurtell/test-data/8ce04c85-ce4b-4844-b454-5303482ade95/data/default/snapshot1/9e49edd0ab41657fb0c6ebb4d9dfad15/cf/f132e5b06f66443f8003363ed1535aac', but it shouldn't have.
> 	at org.junit.Assert.fail(Assert.java:88)
> 	at org.junit.Assert.assertTrue(Assert.java:41)
> 	at org.junit.Assert.assertFalse(Assert.java:64)
> 	at org.apache.hadoop.hbase.master.snapshot.TestSnapshotFileCache.createAndTestSnapshot(TestSnapshotFileCache.java:260)
> 	at org.apache.hadoop.hbase.master.snapshot.TestSnapshotFileCache.createAndTestSnapshotV1(TestSnapshotFileCache.java:206)
> 	at org.apache.hadoop.hbase.master.snapshot.TestSnapshotFileCache.testReloadModifiedDirectory(TestSnapshotFileCache.java:102)
> {noformat}
> {noformat}
> java.lang.AssertionError: Cache found 'hdfs://localhost:52867/user/apurtell/test-data/8ce04c85-ce4b-4844-b454-5303482ade95/data/default/snapshot1a/2e81adb9212c98cff970eafa006fc40b/cf/a2ec478d850e4e348359699c53b732c4', but it shouldn't have.
> 	at org.junit.Assert.fail(Assert.java:88)
> 	at org.junit.Assert.assertTrue(Assert.java:41)
> 	at org.junit.Assert.assertFalse(Assert.java:64)
> 	at org.apache.hadoop.hbase.master.snapshot.TestSnapshotFileCache.createAndTestSnapshot(TestSnapshotFileCache.java:260)
> 	at org.apache.hadoop.hbase.master.snapshot.TestSnapshotFileCache.createAndTestSnapshotV1(TestSnapshotFileCache.java:206)
> 	at org.apache.hadoop.hbase.master.snapshot.TestSnapshotFileCache.testLoadAndDelete(TestSnapshotFileCache.java:88)
> {noformat}
> These changes are part of HBASE-19239
> I've disabled the offending test cases with @Ignore in that patch, but they should be reenabled and fixed. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)