You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@thrift.apache.org by "Bryan Duxbury (JIRA)" <ji...@apache.org> on 2010/04/04 22:25:27 UTC
[jira] Created: (THRIFT-754) Improvements to varint reading in
Compact Protocol
Improvements to varint reading in Compact Protocol
--------------------------------------------------
Key: THRIFT-754
URL: https://issues.apache.org/jira/browse/THRIFT-754
Project: Thrift
Issue Type: Improvement
Components: Library (Java)
Affects Versions: 0.3
Reporter: Bryan Duxbury
Assignee: Bryan Duxbury
Fix For: 0.3
I've found a way to make better use of direct buffer access in the Compact Protocol that makes reading varints faster. When there's at least 5 or 10 bytes available in the buffer, we can try to do the whole varint deserialization from the buffer rather than byte by byte, and ultimately only consume the bytes we used. This lets us avoid the cost of going back and forth from the buffer 1-10 times.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (THRIFT-754) Improvements to varint reading in
Compact Protocol
Posted by "Bryan Duxbury (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bryan Duxbury updated THRIFT-754:
---------------------------------
Attachment: thrift-754.patch
This patch makes readVarint32 about 30% faster and readVarint64 about 25% faster.
> Improvements to varint reading in Compact Protocol
> --------------------------------------------------
>
> Key: THRIFT-754
> URL: https://issues.apache.org/jira/browse/THRIFT-754
> Project: Thrift
> Issue Type: Improvement
> Components: Library (Java)
> Affects Versions: 0.3
> Reporter: Bryan Duxbury
> Assignee: Bryan Duxbury
> Fix For: 0.3
>
> Attachments: thrift-754.patch
>
>
> I've found a way to make better use of direct buffer access in the Compact Protocol that makes reading varints faster. When there's at least 5 or 10 bytes available in the buffer, we can try to do the whole varint deserialization from the buffer rather than byte by byte, and ultimately only consume the bytes we used. This lets us avoid the cost of going back and forth from the buffer 1-10 times.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Closed: (THRIFT-754) Improvements to varint reading in
Compact Protocol
Posted by "Bryan Duxbury (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bryan Duxbury closed THRIFT-754.
--------------------------------
Resolution: Fixed
I just committed this.
> Improvements to varint reading in Compact Protocol
> --------------------------------------------------
>
> Key: THRIFT-754
> URL: https://issues.apache.org/jira/browse/THRIFT-754
> Project: Thrift
> Issue Type: Improvement
> Components: Library (Java)
> Affects Versions: 0.3
> Reporter: Bryan Duxbury
> Assignee: Bryan Duxbury
> Fix For: 0.3
>
> Attachments: thrift-754.patch
>
>
> I've found a way to make better use of direct buffer access in the Compact Protocol that makes reading varints faster. When there's at least 5 or 10 bytes available in the buffer, we can try to do the whole varint deserialization from the buffer rather than byte by byte, and ultimately only consume the bytes we used. This lets us avoid the cost of going back and forth from the buffer 1-10 times.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (THRIFT-754) Improvements to varint reading in
Compact Protocol
Posted by "Bryan Duxbury (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12853277#action_12853277 ]
Bryan Duxbury commented on THRIFT-754:
--------------------------------------
It's a worthwhile point. Up to this point, I think Thrift in general has been assuming that the data is good in all cases. I could change it to a for loop, but I'm not sure what performance impact that would have.
> Improvements to varint reading in Compact Protocol
> --------------------------------------------------
>
> Key: THRIFT-754
> URL: https://issues.apache.org/jira/browse/THRIFT-754
> Project: Thrift
> Issue Type: Improvement
> Components: Library (Java)
> Affects Versions: 0.3
> Reporter: Bryan Duxbury
> Assignee: Bryan Duxbury
> Fix For: 0.3
>
> Attachments: thrift-754.patch
>
>
> I've found a way to make better use of direct buffer access in the Compact Protocol that makes reading varints faster. When there's at least 5 or 10 bytes available in the buffer, we can try to do the whole varint deserialization from the buffer rather than byte by byte, and ultimately only consume the bytes we used. This lets us avoid the cost of going back and forth from the buffer 1-10 times.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (THRIFT-754) Improvements to varint reading in
Compact Protocol
Posted by "Mathias Herberts (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12853275#action_12853275 ]
Mathias Herberts commented on THRIFT-754:
-----------------------------------------
I'm never too comfortable knowing there's a while(true) loop whose exit clause could potentially never become true due to either corrupted data or malicious input. Am I being too paranoid on this one?
> Improvements to varint reading in Compact Protocol
> --------------------------------------------------
>
> Key: THRIFT-754
> URL: https://issues.apache.org/jira/browse/THRIFT-754
> Project: Thrift
> Issue Type: Improvement
> Components: Library (Java)
> Affects Versions: 0.3
> Reporter: Bryan Duxbury
> Assignee: Bryan Duxbury
> Fix For: 0.3
>
> Attachments: thrift-754.patch
>
>
> I've found a way to make better use of direct buffer access in the Compact Protocol that makes reading varints faster. When there's at least 5 or 10 bytes available in the buffer, we can try to do the whole varint deserialization from the buffer rather than byte by byte, and ultimately only consume the bytes we used. This lets us avoid the cost of going back and forth from the buffer 1-10 times.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.