You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Todd Lipcon (JIRA)" <ji...@apache.org> on 2012/05/05 03:17:50 UTC
[jira] [Created] (HBASE-5945) Reduce buffer copies in IPC server
response path
Todd Lipcon created HBASE-5945:
----------------------------------
Summary: Reduce buffer copies in IPC server response path
Key: HBASE-5945
URL: https://issues.apache.org/jira/browse/HBASE-5945
Project: HBase
Issue Type: Improvement
Components: ipc
Affects Versions: 0.96.0
Reporter: Todd Lipcon
Assignee: Todd Lipcon
Priority: Minor
The new PB code is sloppy with buffers and makes several needless copies. This increases GC time a lot. A few simple changes can cut this back down.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5945) Reduce buffer copies in IPC server
response path
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon updated HBASE-5945:
-------------------------------
Attachment: even-fewer-copies.txt
New rev gets rid of some more. This seems to make a noticeable difference in my oprofile output and YCSB results. Would appreciate if other folks could verify
(yes, patch still needs more work, please don't review for style/licenses/etc)
> Reduce buffer copies in IPC server response path
> ------------------------------------------------
>
> Key: HBASE-5945
> URL: https://issues.apache.org/jira/browse/HBASE-5945
> Project: HBase
> Issue Type: Improvement
> Components: ipc
> Affects Versions: 0.96.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Minor
> Attachments: buffer-copies.txt, even-fewer-copies.txt
>
>
> The new PB code is sloppy with buffers and makes several needless copies. This increases GC time a lot. A few simple changes can cut this back down.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5945) Reduce buffer copies in IPC server
response path
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13269128#comment-13269128 ]
stack commented on HBASE-5945:
------------------------------
Looks great Todd.
Could we keep this DataOutputBuffer for reuse?
{code}
+ DataOutputBuffer buf = new DataOutputBuffer(size);
{code}
> Reduce buffer copies in IPC server response path
> ------------------------------------------------
>
> Key: HBASE-5945
> URL: https://issues.apache.org/jira/browse/HBASE-5945
> Project: HBase
> Issue Type: Improvement
> Components: ipc
> Affects Versions: 0.96.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Critical
> Attachments: buffer-copies.txt, even-fewer-copies.txt
>
>
> The new PB code is sloppy with buffers and makes several needless copies. This increases GC time a lot. A few simple changes can cut this back down.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5945) Reduce buffer copies in IPC server
response path
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon updated HBASE-5945:
-------------------------------
Attachment: buffer-copies.txt
Here's a first pass. I don't have great benchmark results, but it looked from my "jstat -gcutil" output like this cut the amount of allocation by a factor of 2.5x or so for a benchmark that returned large responses.
Can probably get rid of at least one more copy in there with a bit more trickery.
> Reduce buffer copies in IPC server response path
> ------------------------------------------------
>
> Key: HBASE-5945
> URL: https://issues.apache.org/jira/browse/HBASE-5945
> Project: HBase
> Issue Type: Improvement
> Components: ipc
> Affects Versions: 0.96.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Minor
> Attachments: buffer-copies.txt
>
>
> The new PB code is sloppy with buffers and makes several needless copies. This increases GC time a lot. A few simple changes can cut this back down.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5945) Reduce buffer copies in IPC server
response path
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13269722#comment-13269722 ]
stack commented on HBASE-5945:
------------------------------
/me hearts todd
> Reduce buffer copies in IPC server response path
> ------------------------------------------------
>
> Key: HBASE-5945
> URL: https://issues.apache.org/jira/browse/HBASE-5945
> Project: HBase
> Issue Type: Improvement
> Components: ipc
> Affects Versions: 0.96.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Critical
> Attachments: buffer-copies.txt, even-fewer-copies.txt, hbase-5495.txt
>
>
> The new PB code is sloppy with buffers and makes several needless copies. This increases GC time a lot. A few simple changes can cut this back down.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5945) Reduce buffer copies in IPC server
response path
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13277639#comment-13277639 ]
Hadoop QA commented on HBASE-5945:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12525785/hbase-5495.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 2 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1908//console
This message is automatically generated.
> Reduce buffer copies in IPC server response path
> ------------------------------------------------
>
> Key: HBASE-5945
> URL: https://issues.apache.org/jira/browse/HBASE-5945
> Project: HBase
> Issue Type: Improvement
> Components: ipc
> Affects Versions: 0.96.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Critical
> Attachments: buffer-copies.txt, even-fewer-copies.txt, hbase-5495.txt
>
>
> The new PB code is sloppy with buffers and makes several needless copies. This increases GC time a lot. A few simple changes can cut this back down.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5945) Reduce buffer copies in IPC server
response path
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-5945:
-------------------------
Priority: Critical (was: Minor)
Making critical so we don't overlook this work.
> Reduce buffer copies in IPC server response path
> ------------------------------------------------
>
> Key: HBASE-5945
> URL: https://issues.apache.org/jira/browse/HBASE-5945
> Project: HBase
> Issue Type: Improvement
> Components: ipc
> Affects Versions: 0.96.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Critical
> Attachments: buffer-copies.txt, even-fewer-copies.txt
>
>
> The new PB code is sloppy with buffers and makes several needless copies. This increases GC time a lot. A few simple changes can cut this back down.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5945) Reduce buffer copies in IPC server
response path
Posted by "Zhihong Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268863#comment-13268863 ]
Zhihong Yu commented on HBASE-5945:
-----------------------------------
Interesting finding.
Can I assume that the number of occurrences of Message's is higher than that of Result's ?
If so, the following check can be lifted above that for Result.class:
{code}
+ if (instance instanceof Message) {
{code}
> Reduce buffer copies in IPC server response path
> ------------------------------------------------
>
> Key: HBASE-5945
> URL: https://issues.apache.org/jira/browse/HBASE-5945
> Project: HBase
> Issue Type: Improvement
> Components: ipc
> Affects Versions: 0.96.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Minor
> Attachments: buffer-copies.txt
>
>
> The new PB code is sloppy with buffers and makes several needless copies. This increases GC time a lot. A few simple changes can cut this back down.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5945) Reduce buffer copies in IPC server
response path
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon updated HBASE-5945:
-------------------------------
Attachment: hbase-5495.txt
attached does the optimization on the client side as well, and fixes a bug with serializing exception responses (forgot to length-prefix it)
> Reduce buffer copies in IPC server response path
> ------------------------------------------------
>
> Key: HBASE-5945
> URL: https://issues.apache.org/jira/browse/HBASE-5945
> Project: HBase
> Issue Type: Improvement
> Components: ipc
> Affects Versions: 0.96.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Critical
> Attachments: buffer-copies.txt, even-fewer-copies.txt, hbase-5495.txt
>
>
> The new PB code is sloppy with buffers and makes several needless copies. This increases GC time a lot. A few simple changes can cut this back down.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5945) Reduce buffer copies in IPC server
response path
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-5945:
--------------------------
Priority: Blocker (was: Critical)
> Reduce buffer copies in IPC server response path
> ------------------------------------------------
>
> Key: HBASE-5945
> URL: https://issues.apache.org/jira/browse/HBASE-5945
> Project: HBase
> Issue Type: Improvement
> Components: ipc
> Affects Versions: 0.96.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Blocker
> Attachments: buffer-copies.txt, even-fewer-copies.txt, hbase-5495.txt
>
>
> The new PB code is sloppy with buffers and makes several needless copies. This increases GC time a lot. A few simple changes can cut this back down.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5945) Reduce buffer copies in IPC server
response path
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13269213#comment-13269213 ]
Todd Lipcon commented on HBASE-5945:
------------------------------------
Yep, I think the next improvement is to use thread-local or connection-local buffers - perhaps even direct buffers. There's still one more copy from the heap byte buffer to a direct buffer within the SocketChannel.write() which would be avoided if we serialized directly into a DirectByteBuffer. Have to benchmark whether that's worth it.
We should also do the same hacks on the client side of the connection - should get similar gains.
> Reduce buffer copies in IPC server response path
> ------------------------------------------------
>
> Key: HBASE-5945
> URL: https://issues.apache.org/jira/browse/HBASE-5945
> Project: HBase
> Issue Type: Improvement
> Components: ipc
> Affects Versions: 0.96.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Critical
> Attachments: buffer-copies.txt, even-fewer-copies.txt
>
>
> The new PB code is sloppy with buffers and makes several needless copies. This increases GC time a lot. A few simple changes can cut this back down.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5945) Reduce buffer copies in IPC server
response path
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Lipcon updated HBASE-5945:
-------------------------------
Status: Patch Available (was: Open)
> Reduce buffer copies in IPC server response path
> ------------------------------------------------
>
> Key: HBASE-5945
> URL: https://issues.apache.org/jira/browse/HBASE-5945
> Project: HBase
> Issue Type: Improvement
> Components: ipc
> Affects Versions: 0.96.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Critical
> Attachments: buffer-copies.txt, even-fewer-copies.txt, hbase-5495.txt
>
>
> The new PB code is sloppy with buffers and makes several needless copies. This increases GC time a lot. A few simple changes can cut this back down.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira