You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Zephyr Guo (JIRA)" <ji...@apache.org> on 2016/04/06 16:54:25 UTC
[jira] [Commented] (HBASE-14490) [RpcServer] reuse request read
buffer
[ https://issues.apache.org/jira/browse/HBASE-14490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15228396#comment-15228396 ]
Zephyr Guo commented on HBASE-14490:
------------------------------------
Thanks for everyone. I'm sorry that I didn't pay a attention to this issue for a long time.I have some new idea in here now.
The protocol of hbase can be regarded as two parts.
| head + params | cells |
For first one, we can use a fixed DirectBB in each {{Connection}}.They can reuse to reading first part of protocol.Just ensure
MaxSize(head + params) less than our fixed size.
For second one, we can use a DirectBB pool.Each DBB is same size that equal to NIO_BUFFER_LIMIT.We take the number of DBBs exactly and link them in a special {{CellScanner}}.The scanner would move pointer to read cells ,and give back DBBs if the DBB before the pointer.
For example:
There is a CellScanner consist of 3 DBBs.The pointer moved to the second DDB.So we can give back first DBB now.
-------
DBB <-- no longer use, give back
DBB <-- pointer
DBB
-------
Maybe another issue improve the RpcServer, and I just share my idea.
> [RpcServer] reuse request read buffer
> -------------------------------------
>
> Key: HBASE-14490
> URL: https://issues.apache.org/jira/browse/HBASE-14490
> Project: HBase
> Issue Type: Improvement
> Components: IPC/RPC
> Affects Versions: 2.0.0, 1.0.2
> Reporter: Zephyr Guo
> Assignee: Zephyr Guo
> Labels: performance
> Fix For: 2.0.0, 1.0.2
>
> Attachments: 14490.hack.to.1.2.patch, ByteBufferPool.java, HBASE-14490-v1.patch, HBASE-14490-v10.patch, HBASE-14490-v11.patch, HBASE-14490-v12.patch, HBASE-14490-v2.patch, HBASE-14490-v3.patch, HBASE-14490-v4.patch, HBASE-14490-v5.patch, HBASE-14490-v6.patch, HBASE-14490-v7.patch, HBASE-14490-v8.patch, HBASE-14490-v9.patch, gc.png, hits.png, test-v12-patch
>
>
> Reusing buffer to read request.It's not necessary to every request free buffer.The idea of optimization is to reduce the times that allocate ByteBuffer.
> *Modification*
> 1. {{saslReadAndProcess}} ,{{processOneRpc}}, {{processUnwrappedData}}, {{processConnectionHeader}} accept a ByteBuffer instead of byte[].They can move {{ByteBuffer.position}} correctly when we have read the data.
> 2. {{processUnwrappedData}} no longer use any extra memory.
> 3. Maintaining a buffer pool in each {{Connection}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)