[jira] [Commented] (HBASE-21568) Disable use of BlockCache for LoadIncrementalHFiles

["Josh Elser (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-21568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16713284#comment-16713284 ] 

Josh Elser commented on HBASE-21568:
------------------------------------

Two caveats:
 # if the bulk load is happening into a table that is being created automagically, we do read the input files twice. First we read to get the splits to create the table, and then we read them again to group and bulkload the files. It looks like this was left as a follow-on improvement to eliminate.
 # If we have to re-write an HFile on disk due to region boundaries and the HFile's boundaries not aligning, we'll have to read the HFile twice.

I think both of these are minor and not the usual case. I don't expect a problem removing the config when we could keep these in-memory client-side.

> Disable use of BlockCache for LoadIncrementalHFiles
> ---------------------------------------------------
>
>                 Key: HBASE-21568
>                 URL: https://issues.apache.org/jira/browse/HBASE-21568
>             Project: HBase
>          Issue Type: Bug
>          Components: Client
>            Reporter: Josh Elser
>            Assignee: Josh Elser
>            Priority: Major
>             Fix For: 2.2.0, 2.1.2, 2.0.4
>
>
> [~vrodionov] added some API to {{CacheConfig}} via HBASE-17151 to allow callers to specify that they do not want to use a block cache when reading an HFile.
> If the BucketCache is set up to use the FileSystem, we can have a situation where the client tries to instantiate the BucketCache and is disallowed due to filesystem permissions:
> {code:java}
> 2018-12-03 16:22:03,032 ERROR [LoadIncrementalHFiles-0] bucket.FileIOEngine: Failed allocating cache on /mnt/hbase/cache.data
> java.io.FileNotFoundException: /mnt/hbase/cache.data (Permission denied)
>   at java.io.RandomAccessFile.open0(Native Method)
>   at java.io.RandomAccessFile.open(RandomAccessFile.java:316)
>   at java.io.RandomAccessFile.<init>(RandomAccessFile.java:243)
>   at java.io.RandomAccessFile.<init>(RandomAccessFile.java:124)
>   at org.apache.hadoop.hbase.io.hfile.bucket.FileIOEngine.<init>(FileIOEngine.java:81)
>   at org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.getIOEngineFromName(BucketCache.java:382)
>   at org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.<init>(BucketCache.java:262)
>   at org.apache.hadoop.hbase.io.hfile.CacheConfig.getBucketCache(CacheConfig.java:633)
>   at org.apache.hadoop.hbase.io.hfile.CacheConfig.instantiateBlockCache(CacheConfig.java:663)
>   at org.apache.hadoop.hbase.io.hfile.CacheConfig.<init>(CacheConfig.java:250)
>   at org.apache.hadoop.hbase.tool.LoadIncrementalHFiles.groupOrSplit(LoadIncrementalHFiles.java:713)
>   at org.apache.hadoop.hbase.tool.LoadIncrementalHFiles$3.call(LoadIncrementalHFiles.java:621)
>   at org.apache.hadoop.hbase.tool.LoadIncrementalHFiles$3.call(LoadIncrementalHFiles.java:617)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>   at java.lang.Thread.run(Thread.java:748)
> {code}
> LoadIncrementalHfiles should provide the {{CacheConfig.DISABLE}}.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)