You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Dejan Menges <de...@gmail.com> on 2015/06/01 16:04:40 UTC
hfile.bucket.BucketAllocatorException: Allocation too big size
Hi,
I'm getting messages like:
015-06-01 14:02:29,529 WARN
org.apache.hadoop.hbase.io.hfile.bucket.BucketCache: Failed allocating for
block ce18012f4dfa424db88e92de29e76a9b_25809098330
org.apache.hadoop.hbase.io.hfile.bucket.BucketAllocatorException:
Allocation too big size=750465
at
org.apache.hadoop.hbase.io.hfile.bucket.BucketAllocator.allocateBlock(BucketAllocator.java:400)
at
org.apache.hadoop.hbase.io.hfile.bucket.BucketCache$RAMQueueEntry.writeToCache(BucketCache.java:1153)
at
org.apache.hadoop.hbase.io.hfile.bucket.BucketCache$WriterThread.doDrain(BucketCache.java:703)
at
org.apache.hadoop.hbase.io.hfile.bucket.BucketCache$WriterThread.run(BucketCache.java:675)
at java.lang.Thread.run(Thread.java:745)
However, not sure why is this. If I understood it correctly (and probably I
didn't :/) this should fit in one of those:
<property>
<name>hbase.bucketcache.sizes</name>
<value>65536,131072,196608,262144,327680,393216,655360,1310720</value>
</property>
In the same time, hbase,bucketcache.size is 24G. Not sure what I did
(again) wrong?
Re: hfile.bucket.BucketAllocatorException: Allocation too big size
Posted by Anoop John <an...@gmail.com>.
Yes Ted is right. "hbase.bucketcache.bucket.sizes" is the correct config
name... I think wrong name was added to hbase-default.xml.. There was
bug already raised for this? Some thing related to bucket cache was already
there.. Am not sure.. We need fix in xml.
-Anoop-
On Mon, Jun 1, 2015 at 7:49 PM, Ted Yu <yu...@gmail.com> wrote:
> Which hbase release are you using ?
>
> I seem to recall that hbase.bucketcache.bucket.sizes was the key.
>
> Cheers
>
> On Mon, Jun 1, 2015 at 7:04 AM, Dejan Menges <de...@gmail.com>
> wrote:
>
> > Hi,
> >
> > I'm getting messages like:
> >
> > 015-06-01 14:02:29,529 WARN
> > org.apache.hadoop.hbase.io.hfile.bucket.BucketCache: Failed allocating
> for
> > block ce18012f4dfa424db88e92de29e76a9b_25809098330
> >
> > org.apache.hadoop.hbase.io.hfile.bucket.BucketAllocatorException:
> > Allocation too big size=750465
> >
> > at
> >
> >
> org.apache.hadoop.hbase.io.hfile.bucket.BucketAllocator.allocateBlock(BucketAllocator.java:400)
> >
> > at
> >
> >
> org.apache.hadoop.hbase.io.hfile.bucket.BucketCache$RAMQueueEntry.writeToCache(BucketCache.java:1153)
> >
> > at
> >
> >
> org.apache.hadoop.hbase.io.hfile.bucket.BucketCache$WriterThread.doDrain(BucketCache.java:703)
> >
> > at
> >
> >
> org.apache.hadoop.hbase.io.hfile.bucket.BucketCache$WriterThread.run(BucketCache.java:675)
> >
> > at java.lang.Thread.run(Thread.java:745)
> >
> >
> > However, not sure why is this. If I understood it correctly (and
> probably I
> > didn't :/) this should fit in one of those:
> >
> > <property>
> >
> > <name>hbase.bucketcache.sizes</name>
> >
> > <value>65536,131072,196608,262144,327680,393216,655360,1310720</value>
> >
> > </property>
> >
> > In the same time, hbase,bucketcache.size is 24G. Not sure what I did
> > (again) wrong?
> >
>
Re: hfile.bucket.BucketAllocatorException: Allocation too big size
Posted by Dejan Menges <de...@gmail.com>.
Oh, cool, something that will push us to upgrade sooner than later :)
Just for my information - what limit was used than in 2.1 as maximum cache
block size (or whatever name it was)? Size of the block, or something else?
On Mon, Jun 1, 2015 at 5:00 PM Ted Yu <yu...@gmail.com> wrote:
> Dejan:
> "hbase.bucketcache.bucket.sizes" was introduced by:
> HBASE-10641 Configurable Bucket Sizes in bucketCache
>
> which was integrated to 0.98.4
>
> HDP 2.2 has the fix while HDP 2.1 didn't.
>
> FYI
>
> On Mon, Jun 1, 2015 at 7:23 AM, Dejan Menges <de...@gmail.com>
> wrote:
>
> > Hi Ted,
> >
> > It's 0.98.0 with bunch of patches (from Hortonworks).
> >
> > Let me try with that key, on my way :)
> >
> > On Mon, Jun 1, 2015 at 4:19 PM Ted Yu <yu...@gmail.com> wrote:
> >
> > > Which hbase release are you using ?
> > >
> > > I seem to recall that hbase.bucketcache.bucket.sizes was the key.
> > >
> > > Cheers
> > >
> > > On Mon, Jun 1, 2015 at 7:04 AM, Dejan Menges <de...@gmail.com>
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > I'm getting messages like:
> > > >
> > > > 015-06-01 14:02:29,529 WARN
> > > > org.apache.hadoop.hbase.io.hfile.bucket.BucketCache: Failed
> allocating
> > > for
> > > > block ce18012f4dfa424db88e92de29e76a9b_25809098330
> > > >
> > > > org.apache.hadoop.hbase.io.hfile.bucket.BucketAllocatorException:
> > > > Allocation too big size=750465
> > > >
> > > > at
> > > >
> > > > org.apache.hadoop.hbase.io
> > > .hfile.bucket.BucketAllocator.allocateBlock(BucketAllocator.java:400)
> > > >
> > > > at
> > > >
> > > > org.apache.hadoop.hbase.io
> > >
> >
> .hfile.bucket.BucketCache$RAMQueueEntry.writeToCache(BucketCache.java:1153)
> > > >
> > > > at
> > > >
> > > > org.apache.hadoop.hbase.io
> > > .hfile.bucket.BucketCache$WriterThread.doDrain(BucketCache.java:703)
> > > >
> > > > at
> > > >
> > > > org.apache.hadoop.hbase.io
> > > .hfile.bucket.BucketCache$WriterThread.run(BucketCache.java:675)
> > > >
> > > > at java.lang.Thread.run(Thread.java:745)
> > > >
> > > >
> > > > However, not sure why is this. If I understood it correctly (and
> > > probably I
> > > > didn't :/) this should fit in one of those:
> > > >
> > > > <property>
> > > >
> > > > <name>hbase.bucketcache.sizes</name>
> > > >
> > > >
> <value>65536,131072,196608,262144,327680,393216,655360,1310720</value>
> > > >
> > > > </property>
> > > >
> > > > In the same time, hbase,bucketcache.size is 24G. Not sure what I did
> > > > (again) wrong?
> > > >
> > >
> >
>
Re: hfile.bucket.BucketAllocatorException: Allocation too big size
Posted by Ted Yu <yu...@gmail.com>.
Dejan:
"hbase.bucketcache.bucket.sizes" was introduced by:
HBASE-10641 Configurable Bucket Sizes in bucketCache
which was integrated to 0.98.4
HDP 2.2 has the fix while HDP 2.1 didn't.
FYI
On Mon, Jun 1, 2015 at 7:23 AM, Dejan Menges <de...@gmail.com> wrote:
> Hi Ted,
>
> It's 0.98.0 with bunch of patches (from Hortonworks).
>
> Let me try with that key, on my way :)
>
> On Mon, Jun 1, 2015 at 4:19 PM Ted Yu <yu...@gmail.com> wrote:
>
> > Which hbase release are you using ?
> >
> > I seem to recall that hbase.bucketcache.bucket.sizes was the key.
> >
> > Cheers
> >
> > On Mon, Jun 1, 2015 at 7:04 AM, Dejan Menges <de...@gmail.com>
> > wrote:
> >
> > > Hi,
> > >
> > > I'm getting messages like:
> > >
> > > 015-06-01 14:02:29,529 WARN
> > > org.apache.hadoop.hbase.io.hfile.bucket.BucketCache: Failed allocating
> > for
> > > block ce18012f4dfa424db88e92de29e76a9b_25809098330
> > >
> > > org.apache.hadoop.hbase.io.hfile.bucket.BucketAllocatorException:
> > > Allocation too big size=750465
> > >
> > > at
> > >
> > > org.apache.hadoop.hbase.io
> > .hfile.bucket.BucketAllocator.allocateBlock(BucketAllocator.java:400)
> > >
> > > at
> > >
> > > org.apache.hadoop.hbase.io
> >
> .hfile.bucket.BucketCache$RAMQueueEntry.writeToCache(BucketCache.java:1153)
> > >
> > > at
> > >
> > > org.apache.hadoop.hbase.io
> > .hfile.bucket.BucketCache$WriterThread.doDrain(BucketCache.java:703)
> > >
> > > at
> > >
> > > org.apache.hadoop.hbase.io
> > .hfile.bucket.BucketCache$WriterThread.run(BucketCache.java:675)
> > >
> > > at java.lang.Thread.run(Thread.java:745)
> > >
> > >
> > > However, not sure why is this. If I understood it correctly (and
> > probably I
> > > didn't :/) this should fit in one of those:
> > >
> > > <property>
> > >
> > > <name>hbase.bucketcache.sizes</name>
> > >
> > > <value>65536,131072,196608,262144,327680,393216,655360,1310720</value>
> > >
> > > </property>
> > >
> > > In the same time, hbase,bucketcache.size is 24G. Not sure what I did
> > > (again) wrong?
> > >
> >
>
Re: hfile.bucket.BucketAllocatorException: Allocation too big size
Posted by Dejan Menges <de...@gmail.com>.
Hi Ted,
It's 0.98.0 with bunch of patches (from Hortonworks).
Let me try with that key, on my way :)
On Mon, Jun 1, 2015 at 4:19 PM Ted Yu <yu...@gmail.com> wrote:
> Which hbase release are you using ?
>
> I seem to recall that hbase.bucketcache.bucket.sizes was the key.
>
> Cheers
>
> On Mon, Jun 1, 2015 at 7:04 AM, Dejan Menges <de...@gmail.com>
> wrote:
>
> > Hi,
> >
> > I'm getting messages like:
> >
> > 015-06-01 14:02:29,529 WARN
> > org.apache.hadoop.hbase.io.hfile.bucket.BucketCache: Failed allocating
> for
> > block ce18012f4dfa424db88e92de29e76a9b_25809098330
> >
> > org.apache.hadoop.hbase.io.hfile.bucket.BucketAllocatorException:
> > Allocation too big size=750465
> >
> > at
> >
> > org.apache.hadoop.hbase.io
> .hfile.bucket.BucketAllocator.allocateBlock(BucketAllocator.java:400)
> >
> > at
> >
> > org.apache.hadoop.hbase.io
> .hfile.bucket.BucketCache$RAMQueueEntry.writeToCache(BucketCache.java:1153)
> >
> > at
> >
> > org.apache.hadoop.hbase.io
> .hfile.bucket.BucketCache$WriterThread.doDrain(BucketCache.java:703)
> >
> > at
> >
> > org.apache.hadoop.hbase.io
> .hfile.bucket.BucketCache$WriterThread.run(BucketCache.java:675)
> >
> > at java.lang.Thread.run(Thread.java:745)
> >
> >
> > However, not sure why is this. If I understood it correctly (and
> probably I
> > didn't :/) this should fit in one of those:
> >
> > <property>
> >
> > <name>hbase.bucketcache.sizes</name>
> >
> > <value>65536,131072,196608,262144,327680,393216,655360,1310720</value>
> >
> > </property>
> >
> > In the same time, hbase,bucketcache.size is 24G. Not sure what I did
> > (again) wrong?
> >
>
Re: hfile.bucket.BucketAllocatorException: Allocation too big size
Posted by Ted Yu <yu...@gmail.com>.
Which hbase release are you using ?
I seem to recall that hbase.bucketcache.bucket.sizes was the key.
Cheers
On Mon, Jun 1, 2015 at 7:04 AM, Dejan Menges <de...@gmail.com> wrote:
> Hi,
>
> I'm getting messages like:
>
> 015-06-01 14:02:29,529 WARN
> org.apache.hadoop.hbase.io.hfile.bucket.BucketCache: Failed allocating for
> block ce18012f4dfa424db88e92de29e76a9b_25809098330
>
> org.apache.hadoop.hbase.io.hfile.bucket.BucketAllocatorException:
> Allocation too big size=750465
>
> at
>
> org.apache.hadoop.hbase.io.hfile.bucket.BucketAllocator.allocateBlock(BucketAllocator.java:400)
>
> at
>
> org.apache.hadoop.hbase.io.hfile.bucket.BucketCache$RAMQueueEntry.writeToCache(BucketCache.java:1153)
>
> at
>
> org.apache.hadoop.hbase.io.hfile.bucket.BucketCache$WriterThread.doDrain(BucketCache.java:703)
>
> at
>
> org.apache.hadoop.hbase.io.hfile.bucket.BucketCache$WriterThread.run(BucketCache.java:675)
>
> at java.lang.Thread.run(Thread.java:745)
>
>
> However, not sure why is this. If I understood it correctly (and probably I
> didn't :/) this should fit in one of those:
>
> <property>
>
> <name>hbase.bucketcache.sizes</name>
>
> <value>65536,131072,196608,262144,327680,393216,655360,1310720</value>
>
> </property>
>
> In the same time, hbase,bucketcache.size is 24G. Not sure what I did
> (again) wrong?
>