You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by ashish singhi <as...@huawei.com> on 2014/09/04 06:37:01 UTC
Writer for a hfile is rolled though it has not reached max size
Hi all.
When a writer for a column family exceeds the hregion.max.filesize then we are rolling writers of all the column families though the writers of the other column family has not exceeded the max size.
Below is the code snippet from HFileOutputFormat2 class.
{code}
// If any of the HFiles for the column families has reached
// maxsize, we need to roll all the writers
if (wl != null && wl.written + length >= maxsize) {
this.rollRequested = true;
}
// This can only happen once a row is finished though
if (rollRequested && Bytes.compareTo(this.previousRow, rowKey) != 0) {
rollWriters();
}
{code}
Is it a bug or I am missing something ?
Regards,
Ashish Singhi
RE: Writer for a hfile is rolled though it has not reached max size
Posted by ashish singhi <as...@huawei.com>.
Thanks Ted. I will surely look into it.
-----Original Message-----
From: Ted Yu [mailto:yuzhihong@gmail.com]
Sent: 04 September 2014 10:32
To: dev@hbase.apache.org
Subject: Re: Writer for a hfile is rolled though it has not reached max size
See the conversation between Todd and Nicolas (on IRC) on HBASE-1861
Cheers
On Wed, Sep 3, 2014 at 9:37 PM, ashish singhi <as...@huawei.com>
wrote:
> Hi all.
>
> When a writer for a column family exceeds the hregion.max.filesize
> then we are rolling writers of all the column families though the
> writers of the other column family has not exceeded the max size.
> Below is the code snippet from HFileOutputFormat2 class.
> {code}
> // If any of the HFiles for the column families has reached
> // maxsize, we need to roll all the writers
> if (wl != null && wl.written + length >= maxsize) {
> this.rollRequested = true;
> }
>
> // This can only happen once a row is finished though
> if (rollRequested && Bytes.compareTo(this.previousRow, rowKey)
> !=
> 0) {
> rollWriters();
> }
> {code}
>
> Is it a bug or I am missing something ?
>
> Regards,
> Ashish Singhi
>
Re: Writer for a hfile is rolled though it has not reached max size
Posted by Ted Yu <yu...@gmail.com>.
See the conversation between Todd and Nicolas (on IRC) on HBASE-1861
Cheers
On Wed, Sep 3, 2014 at 9:37 PM, ashish singhi <as...@huawei.com>
wrote:
> Hi all.
>
> When a writer for a column family exceeds the hregion.max.filesize then
> we are rolling writers of all the column families though the writers of the
> other column family has not exceeded the max size.
> Below is the code snippet from HFileOutputFormat2 class.
> {code}
> // If any of the HFiles for the column families has reached
> // maxsize, we need to roll all the writers
> if (wl != null && wl.written + length >= maxsize) {
> this.rollRequested = true;
> }
>
> // This can only happen once a row is finished though
> if (rollRequested && Bytes.compareTo(this.previousRow, rowKey) !=
> 0) {
> rollWriters();
> }
> {code}
>
> Is it a bug or I am missing something ?
>
> Regards,
> Ashish Singhi
>