You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Varun Sharma <va...@pinterest.com> on 2014/02/24 18:51:16 UTC
Cell values larger than Column Family Block Size
Hi,
What happens if my block size is 32K while the cells are 50K. Do Hfile
blocks round up to 50K or are values split across blocks ? Also how does
this play with the block cache ?
Thanks
Varun
Re: Cell values larger than Column Family Block Size
Posted by Ted Yu <yu...@gmail.com>.
The effective block size would be >= 50K for your schema.
Cheers
On Mon, Feb 24, 2014 at 12:38 PM, Varun Sharma <va...@pinterest.com> wrote:
> Hi Ted,
>
> I am not concerned about wide rows here. My schema has only 1 column in it
> but it has a "value" of 50-100K bytes. The block size is configured to be
> 32K bytes. How does that work in practice - does it mean that the effective
> block size is upwards of 50K ?
>
> Varun
>
>
> On Mon, Feb 24, 2014 at 10:07 AM, Ted Yu <yu...@gmail.com> wrote:
>
> > Cycling old bits:
> >
> >
> >
> http://search-hadoop.com/m/DHED4v7stT1/larger+HFile+block+size+for+very+wide+row&subj=larger+HFile+block+size+for+very+wide+row+
> >
> >
> > On Mon, Feb 24, 2014 at 11:51 AM, Varun Sharma <va...@pinterest.com>
> > wrote:
> >
> > > Hi,
> > >
> > > What happens if my block size is 32K while the cells are 50K. Do Hfile
> > > blocks round up to 50K or are values split across blocks ? Also how
> does
> > > this play with the block cache ?
> > >
> > > Thanks
> > > Varun
> > >
> >
>
Re: Cell values larger than Column Family Block Size
Posted by Varun Sharma <va...@pinterest.com>.
Hi Ted,
I am not concerned about wide rows here. My schema has only 1 column in it
but it has a "value" of 50-100K bytes. The block size is configured to be
32K bytes. How does that work in practice - does it mean that the effective
block size is upwards of 50K ?
Varun
On Mon, Feb 24, 2014 at 10:07 AM, Ted Yu <yu...@gmail.com> wrote:
> Cycling old bits:
>
>
> http://search-hadoop.com/m/DHED4v7stT1/larger+HFile+block+size+for+very+wide+row&subj=larger+HFile+block+size+for+very+wide+row+
>
>
> On Mon, Feb 24, 2014 at 11:51 AM, Varun Sharma <va...@pinterest.com>
> wrote:
>
> > Hi,
> >
> > What happens if my block size is 32K while the cells are 50K. Do Hfile
> > blocks round up to 50K or are values split across blocks ? Also how does
> > this play with the block cache ?
> >
> > Thanks
> > Varun
> >
>
Re: Cell values larger than Column Family Block Size
Posted by Ted Yu <yu...@gmail.com>.
Cycling old bits:
http://search-hadoop.com/m/DHED4v7stT1/larger+HFile+block+size+for+very+wide+row&subj=larger+HFile+block+size+for+very+wide+row+
On Mon, Feb 24, 2014 at 11:51 AM, Varun Sharma <va...@pinterest.com> wrote:
> Hi,
>
> What happens if my block size is 32K while the cells are 50K. Do Hfile
> blocks round up to 50K or are values split across blocks ? Also how does
> this play with the block cache ?
>
> Thanks
> Varun
>
Re: Cell values larger than Column Family Block Size
Posted by Stack <st...@duboce.net>.
On Mon, Feb 24, 2014 at 9:51 AM, Varun Sharma <va...@pinterest.com> wrote:
> Hi,
>
> What happens if my block size is 32K while the cells are 50K. Do Hfile
> blocks round up to 50K or are values split across blocks ? Also how does
> this play with the block cache ?
>
Yes. We round up to the size of the actual value; we do not split cells
across hbase blocks.
Regards block cache, we cache at the block granularity, whatever their size
(in your cache, this 50k value will be pulled in and will float in the BC
using 50k).
St.Ack