Fwd: Fine tuning HBase for bulkload

[Poonam Ligade <po...@gmail.com>]

---------- Forwarded message ----------
From: Poonam Ligade <po...@gmail.com>
Date: Wed, Sep 17, 2014 at 9:00 PM
Subject: Re: Fine tuning HBase for bulkload
To: user@phoenix.apache.org


How to disable WAL using configuration property??

Instead of that i changed deferred log interval to
hbase.regionserver.optionallogflushinterval 10000ms.

Regards,
Poonam

On Wed, Sep 17, 2014 at 8:40 PM, Poonam Ligade <po...@gmail.com>
wrote:

> Hi Gabriel,
>
> I am using phoenix psql.py utility to load this data.
>
> Thanks Yeshwant I have already done that.
>
> Regards,
> Poonam.
>
>
>
> On Wed, Sep 17, 2014 at 8:11 PM, Gabriel Reid <ga...@gmail.com>
> wrote:
>
>> Hi Poonam,
>>
>> Could you say a bit about how you're doing this bulk load? Are you
>> loading from a CSV file, or making JDBC calls yourself (or something
>> else)?
>>
>> Are you aware of the CSV bulk loader
>> (http://phoenix.apache.org/bulk_dataload.html)? That will probably
>> give you the best performance for a large load of data.
>>
>> - Gabriel
>>
>>
>> On Wed, Sep 17, 2014 at 3:44 PM, Poonam Ligade
>> <po...@gmail.com> wrote:
>> > Hi,
>> >
>> > I am trying to load around 15GB of data in hbase
>> > Files are split in around 1.3GB. I have 2 regionservers and 1 master
>> running
>> > on seperate machines with 4 GB physical RAM each.
>> > Heapsize For Hmaster and regionserver is 3 GB each
>> >
>> > How can I configure HBase to handle these writes faster.
>> >
>> > Currently I have below settings
>> >
>> > hbase.regionserver.handler.count                         60
>> > hbase.client.write.buffer                                      8388608
>> > hbase.hregion.memstore.flush.size                     134217728
>> > hbase.regionserver.global.memstore.lowerLimit    0.95
>> >
>> > hbase.regionserver.global.memstore.upperLimit    0.4
>> >
>> > hbase.hregion.max.filesize                                 3294967296
>> >
>> >
>> >
>> > Regards,
>> > Poonam
>>
>
>