[jira] [Resolved] (HBASE-3106) a temporary solution of multi-thread compaction for 0.20.6

["Andrew Purtell (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HBASE-3106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Andrew Purtell resolved HBASE-3106.
-----------------------------------

    Resolution: Not a Problem

> a temporary solution of multi-thread compaction for 0.20.6
> ----------------------------------------------------------
>
>                 Key: HBASE-3106
>                 URL: https://issues.apache.org/jira/browse/HBASE-3106
>             Project: HBase
>          Issue Type: Improvement
>          Components: regionserver
>    Affects Versions: 0.20.6
>         Environment: 1master, 7 region servers, 4 * 7 clients(all clients run on region server host), sequential put 
> configuration of each host is:
> memory: 16G (region server uses 10G)
> disk: 12T
> cpu: 8 core
>            Reporter: andychen
>         Attachments: multithread-compaction-0.20.6-v2.patch, multithread-compaction-0.20.6.patch
>
>
> Each region's memtable size is 64M, global memtable limit is 4G on one region server
> In heavy write scenario, especially disable WAL, sequential put  fills memtable to full very quickly, and will generates too many storefiles in a short time.
> Then there are more and more compaction tasks put into compactionQueue, single thread excutes compaction task too slowly, finally forms a vicious cycle.
> This will block some region's data importing for a long time
> We simplely using ThreadPoolExecutor as multi-thread compaction sulution
> (As we know, 0.90 or 0.92 will optimize memflush, compaction, open, close...etc using multi-thread, but this magnificent version hasn't bee released yet)



--
This message was sent by Atlassian JIRA
(v6.2#6252)