You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@thrift.apache.org by "Bryan Duxbury (JIRA)" <ji...@apache.org> on 2010/01/22 00:08:54 UTC
[jira] Created: (THRIFT-685) Direct buffer access to improve
deserialization performance
Direct buffer access to improve deserialization performance
-----------------------------------------------------------
Key: THRIFT-685
URL: https://issues.apache.org/jira/browse/THRIFT-685
Project: Thrift
Issue Type: Improvement
Components: Library (Java)
Reporter: Bryan Duxbury
Assignee: Bryan Duxbury
Priority: Minor
Fix For: 0.3
After poking around a bit and comparing how Thrift performs versus Protocol Buffers, I think we should change our transports and protocols to support optional direct buffer access behavior.
Basically, the way this works is that if the transport is backed by a buffer, then it can give access to that buffer to the protocol. The protocol can then do things like read a byte without instantiating a new one-byte array or decode a string without an intermediate byte[] copy.
In my initial testing, we can reduce the amount of time it takes to deserialize a struct by at least 25%. There are probably further gains to be had as well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-685) Direct buffer access to improve
deserialization performance
Posted by "Bryan Duxbury (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bryan Duxbury updated THRIFT-685:
---------------------------------
Patch Info: [Patch Available]
> Direct buffer access to improve deserialization performance
> -----------------------------------------------------------
>
> Key: THRIFT-685
> URL: https://issues.apache.org/jira/browse/THRIFT-685
> Project: Thrift
> Issue Type: Improvement
> Components: Library (Java)
> Reporter: Bryan Duxbury
> Assignee: Bryan Duxbury
> Priority: Minor
> Fix For: 0.3
>
> Attachments: thrift-685.patch
>
>
> After poking around a bit and comparing how Thrift performs versus Protocol Buffers, I think we should change our transports and protocols to support optional direct buffer access behavior.
> Basically, the way this works is that if the transport is backed by a buffer, then it can give access to that buffer to the protocol. The protocol can then do things like read a byte without instantiating a new one-byte array or decode a string without an intermediate byte[] copy.
> In my initial testing, we can reduce the amount of time it takes to deserialize a struct by at least 25%. There are probably further gains to be had as well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-685) Direct buffer access to improve
deserialization performance
Posted by "Bryan Duxbury (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bryan Duxbury updated THRIFT-685:
---------------------------------
Attachment: thrift-685.patch
Here's my initial effort. This produces a pretty noticeable speedup in deserialization time when using the compact protocol and TDeserializer. This patch notably does not yet address the binary protocol implementation.
I'd love some people to review it and give me feedback. Ideally, we'd commit what I have here as a starting point and then continue to improve other parts.
> Direct buffer access to improve deserialization performance
> -----------------------------------------------------------
>
> Key: THRIFT-685
> URL: https://issues.apache.org/jira/browse/THRIFT-685
> Project: Thrift
> Issue Type: Improvement
> Components: Library (Java)
> Reporter: Bryan Duxbury
> Assignee: Bryan Duxbury
> Priority: Minor
> Fix For: 0.3
>
> Attachments: thrift-685.patch
>
>
> After poking around a bit and comparing how Thrift performs versus Protocol Buffers, I think we should change our transports and protocols to support optional direct buffer access behavior.
> Basically, the way this works is that if the transport is backed by a buffer, then it can give access to that buffer to the protocol. The protocol can then do things like read a byte without instantiating a new one-byte array or decode a string without an intermediate byte[] copy.
> In my initial testing, we can reduce the amount of time it takes to deserialize a struct by at least 25%. There are probably further gains to be had as well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Closed: (THRIFT-685) Direct buffer access to improve
deserialization performance
Posted by "Bryan Duxbury (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bryan Duxbury closed THRIFT-685.
--------------------------------
Resolution: Fixed
I've committed this patch. I'll open follow-on tickets to improve other parts of the codebase.
> Direct buffer access to improve deserialization performance
> -----------------------------------------------------------
>
> Key: THRIFT-685
> URL: https://issues.apache.org/jira/browse/THRIFT-685
> Project: Thrift
> Issue Type: Improvement
> Components: Library (Java)
> Reporter: Bryan Duxbury
> Assignee: Bryan Duxbury
> Priority: Minor
> Fix For: 0.3
>
> Attachments: thrift-685.patch
>
>
> After poking around a bit and comparing how Thrift performs versus Protocol Buffers, I think we should change our transports and protocols to support optional direct buffer access behavior.
> Basically, the way this works is that if the transport is backed by a buffer, then it can give access to that buffer to the protocol. The protocol can then do things like read a byte without instantiating a new one-byte array or decode a string without an intermediate byte[] copy.
> In my initial testing, we can reduce the amount of time it takes to deserialize a struct by at least 25%. There are probably further gains to be had as well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (THRIFT-685) Direct buffer access to improve
deserialization performance
Posted by "David Reiss (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12803530#action_12803530 ]
David Reiss commented on THRIFT-685:
------------------------------------
+1. The template code that Chad and I worked on for C\+\+ (almost ready for trunk) gets the complier to do something similar for us, and we saw a really significant boost.
> Direct buffer access to improve deserialization performance
> -----------------------------------------------------------
>
> Key: THRIFT-685
> URL: https://issues.apache.org/jira/browse/THRIFT-685
> Project: Thrift
> Issue Type: Improvement
> Components: Library (Java)
> Reporter: Bryan Duxbury
> Assignee: Bryan Duxbury
> Priority: Minor
> Fix For: 0.3
>
>
> After poking around a bit and comparing how Thrift performs versus Protocol Buffers, I think we should change our transports and protocols to support optional direct buffer access behavior.
> Basically, the way this works is that if the transport is backed by a buffer, then it can give access to that buffer to the protocol. The protocol can then do things like read a byte without instantiating a new one-byte array or decode a string without an intermediate byte[] copy.
> In my initial testing, we can reduce the amount of time it takes to deserialize a struct by at least 25%. There are probably further gains to be had as well.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.