performance problems during bulk load because of triggered compaction?

[Serega Sheypak <se...@gmail.com>]

Hi, I have lowcost hardware, 2 HDD, 10 nodes with HBase 0.98 CDH 5.2.1
i have several apps that read/write to HBase using Java api.
Sometimes I see that response time raises from normal 30-40 ms to 1000-2000
ms or even more.
There are no running MapReduce at that time. But there is a bulk load each
hour.
I see that response degradation and bulk load process happen sometimes.

Table size is 17GB on hdfs and has 84 regions. Most of regions are
150-200MB size.
it has single column family:
{NAME => 'd', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROWCOL',
REPLICATION_SCOPE => '0', COMPRESSION => 'SNAPPY', VERSIONS => '1', TTL =>
'691200 SECONDS (8 DAYS)', MIN_VERSIONS => '0', KEEP_DELETED_CELLS =>
'false', BLOCKSIZE => '65536', IN_MEMORY => 'true', BLOCKCACHE => 'true'}

When bulkload happens, it just updates existing cell value, it brings 0.01%
of new rows.
I keep serialized objects in d:q, where d is column family and q is column
qualifier

How can I get the root cause of performance degradation and minimize it?