You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Zheng Hu (JIRA)" <ji...@apache.org> on 2019/05/30 13:41:00 UTC
[jira] [Updated] (HBASE-22504) Optimize the
MultiByteBuff#get(ByteBuffer, offset, len)
[ https://issues.apache.org/jira/browse/HBASE-22504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Hu updated HBASE-22504:
-----------------------------
Description:
In HBASE-22483, we saw that the BucketCacheWriter thread was quite busy [^BucketCacheWriter-is-busy.png], the flame graph also indicated that the ByteBufferArray#internalTransfer cost ~6% CPU (see [async-prof-pid-25042-cpu-1.svg|https://issues.apache.org/jira/secure/attachment/12970294/async-prof-pid-25042-cpu-1.svg]). because we used the hbase.ipc.server.allocator.buffer.size=64KB, each HFileBlock will be backend by a MultiByteBuff: one 64KB offheap ByteBuffer and one small heap ByteBuffer.
The path is depending on the MultiByteBuff#get(ByteBuffer, offset, len) now:
{code:java}
RAMQueueEntry#writeToCache
|--> ByteBufferIOEngine#write
|--> ByteBufferArray#internalTransfer
|--> ByteBufferArray$WRITER
|--> MultiByteBuff#get(ByteBuffer, offset, len)
{code}
While the MultiByteBuff#get impl is simple and crude now, can optimze this implementation:
{code:java}
@Override
public void get(ByteBuffer out, int sourceOffset,
int length) {
checkRefCount();
// Not used from real read path actually. So not going with
// optimization
for (int i = 0; i < length; ++i) {
out.put(this.get(sourceOffset + i));
}
}
{code}
was:
In HBASE-22483, we saw that the BucketCacheWriter thread was quite busy [^BucketCacheWriter-is-busy.png], because we used the hbase.ipc.server.allocator.buffer.size=64KB, each HFileBlock will be backend by a MultiByteBuff: one 64KB offheap ByteBuffer and one small heap ByteBuffer.
The path is depending on the MultiByteBuff#get(ByteBuffer, offset, len) now:
{code:java}
RAMQueueEntry#writeToCache
|--> ByteBufferIOEngine#write
|--> ByteBufferArray#internalTransfer
|--> ByteBufferArray$WRITER
|--> MultiByteBuff#get(ByteBuffer, offset, len)
{code}
While the MultiByteBuff#get impl is simple and crude now, can optimze this implementation:
{code:java}
@Override
public void get(ByteBuffer out, int sourceOffset,
int length) {
checkRefCount();
// Not used from real read path actually. So not going with
// optimization
for (int i = 0; i < length; ++i) {
out.put(this.get(sourceOffset + i));
}
}
{code}
> Optimize the MultiByteBuff#get(ByteBuffer, offset, len)
> -------------------------------------------------------
>
> Key: HBASE-22504
> URL: https://issues.apache.org/jira/browse/HBASE-22504
> Project: HBase
> Issue Type: Sub-task
> Reporter: Zheng Hu
> Assignee: Zheng Hu
> Priority: Major
>
> In HBASE-22483, we saw that the BucketCacheWriter thread was quite busy [^BucketCacheWriter-is-busy.png], the flame graph also indicated that the ByteBufferArray#internalTransfer cost ~6% CPU (see [async-prof-pid-25042-cpu-1.svg|https://issues.apache.org/jira/secure/attachment/12970294/async-prof-pid-25042-cpu-1.svg]). because we used the hbase.ipc.server.allocator.buffer.size=64KB, each HFileBlock will be backend by a MultiByteBuff: one 64KB offheap ByteBuffer and one small heap ByteBuffer.
> The path is depending on the MultiByteBuff#get(ByteBuffer, offset, len) now:
> {code:java}
> RAMQueueEntry#writeToCache
> |--> ByteBufferIOEngine#write
> |--> ByteBufferArray#internalTransfer
> |--> ByteBufferArray$WRITER
> |--> MultiByteBuff#get(ByteBuffer, offset, len)
> {code}
> While the MultiByteBuff#get impl is simple and crude now, can optimze this implementation:
> {code:java}
> @Override
> public void get(ByteBuffer out, int sourceOffset,
> int length) {
> checkRefCount();
> // Not used from real read path actually. So not going with
> // optimization
> for (int i = 0; i < length; ++i) {
> out.put(this.get(sourceOffset + i));
> }
> }
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)