Whether turning off assignSeqIds using hbase bulk load will cause adverse impact or not

[沈国权 <sh...@gmail.com>]

Hi community:

    Currently our environment will import about 200G into HBase each day.
These datas are cross six tables which table has about 100 Regions. We use
HBase bulk tools completed these import
tasks. We investigate HBase source code found when bulk loading task
happen, HRegionServer will at firstly flush memstore according to each
HRegion, We want to turn off assignSeqIds avoid each HRegion to force flush
memstore. But We don't know these adverse impact if we do. Please help me
to point out these impacts. Thanks

Re:Whether turning off assignSeqIds using hbase bulk load will cause adverse impact or not

[Allan Yang <al...@163.com>]

Please refer to https://issues.apache.org/jira/browse/HBASE-10958
Assigning seqids to bulk loaded files is to avoid data loss when replay log.
If you are 1. all data are imported by bulkload other than insert or
 2. seqids in the loaded file are guaranteed small than the data in the memory
you can get rid of the assignSeqId process.


Allan Yang







At 2017-03-01 16:18:02, "沈国权" <sh...@gmail.com> wrote:
>Hi community:
>
>    Currently our environment will import about 200G into HBase each day.
>These datas are cross six tables which table has about 100 Regions. We use
>HBase bulk tools completed these import
>tasks. We investigate HBase source code found when bulk loading task
>happen, HRegionServer will at firstly flush memstore according to each
>HRegion, We want to turn off assignSeqIds avoid each HRegion to force flush
>memstore. But We don't know these adverse impact if we do. Please help me
>to point out these impacts. Thanks

RE: Whether turning off assignSeqIds using hbase bulk load will cause adverse impact or not

[Frank Luo <jl...@merkleinc.com>]

Flushing memstore could happen at any time and should not have visible performance impact. Why you want to avoid doing that?

-----Original Message-----
From: 沈国权 [mailto:shenguoquan1986@gmail.com]
Sent: Wednesday, March 01, 2017 2:18 AM
To: user@hbase.apache.org
Subject: Whether turning off assignSeqIds using hbase bulk load will cause adverse impact or not

Hi community:

    Currently our environment will import about 200G into HBase each day.
These datas are cross six tables which table has about 100 Regions. We use HBase bulk tools completed these import tasks. We investigate HBase source code found when bulk loading task happen, HRegionServer will at firstly flush memstore according to each HRegion, We want to turn off assignSeqIds avoid each HRegion to force flush memstore. But We don't know these adverse impact if we do. Please help me to point out these impacts. Thanks
Access Merkle’s award-winning Digital Marketing Report for the latest trends and benchmarks in digital marketing<http://www2.merkleinc.com/l/47252/2017-01-25/4k525x>

Which brand's Super Bowl LI advertisement was best amplified on digital media? Merkle's 2017 Digital Bowl Report has the results!<http://www2.merkleinc.com/l/47252/2017-02-03/4kcz6l>
This email and any attachments transmitted with it are intended for use by the intended recipient(s) only. If you have received this email in error, please notify the sender immediately and then delete it. If you are not the intended recipient, you must not keep, use, disclose, copy or distribute this email without the author’s prior permission. We take precautions to minimize the risk of transmitting software viruses, but we advise you to perform your own virus checks on any attachment to this message. We cannot accept liability for any loss or damage caused by software viruses. The information contained in this communication may be confidential and may be subject to the attorney-client privilege.