[jira] Created: (HBASE-2175) Investigate .META. slowdowns when more than 1 store files

["Jean-Daniel Cryans (JIRA)" <ji...@apache.org>]

Investigate .META. slowdowns when more than 1 store files
---------------------------------------------------------

                 Key: HBASE-2175
                 URL: https://issues.apache.org/jira/browse/HBASE-2175
             Project: Hadoop HBase
          Issue Type: Bug
            Reporter: Jean-Daniel Cryans
             Fix For: 0.21.0


I'm currently testing Hadoop 0.21 with HBase trunk + HBASE-2066 by importing our main data set. After some time, probably because of log rolls which force flushes and a cluster restart, the .META. region begins to accumulate store files. I'm refreshing the master web UI a lot to see our insert speed and saw that 1) it was getting slower to refresh and 2) the import speed went down at the same time.

Having already seen something like that previously with 0.20, I forced a major compaction on .META. and immediately the refresh speed got 10 times better and the import throughput went 2x (tasks went from 20 min to 10 min).

Why is scanning and doing random reads from the client that slow when .META. has more than 1 store file? If it's a more fondamental speed issue, could we at least force major compactions on .META. when it grows so that the rest of the cluster doesn't get super slow? By the way, that operation takes less than 1 second since that region is so small.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.