You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Doug Cutting (Created) (JIRA)" <ji...@apache.org> on 2011/10/28 19:01:35 UTC
[jira] [Created] (AVRO-947) improve performance of resolver cache
for multi-threaded applications
improve performance of resolver cache for multi-threaded applications
---------------------------------------------------------------------
Key: AVRO-947
URL: https://issues.apache.org/jira/browse/AVRO-947
Project: Avro
Issue Type: Improvement
Components: java
Reporter: Doug Cutting
The DatumReader API is meant to be thread-safe, in that many threads should be able to simultaneously use a single DatumReader instance. When we first switched GenericDatumReader to use a resolver thread-safety was broken. AVRO-650 repaired this with a thread-local resolver cache, but performance suffered. AVRO-743 improved performance for single-threaded applications, but multi-threaded performance is still poor.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-947) improve performance of resolver cache
for multi-threaded applications
Posted by "Doug Cutting (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13138548#comment-13138548 ]
Doug Cutting commented on AVRO-947:
-----------------------------------
Apache Commons MultiKeyMap (http://s.apache.org/c1J) might be useful here. The cache could then be efficiently indexed by thread, the reader schema and the writer schema, using identity hashing, so that lookups should be fast. However this may still require synchronization.
> improve performance of resolver cache for multi-threaded applications
> ---------------------------------------------------------------------
>
> Key: AVRO-947
> URL: https://issues.apache.org/jira/browse/AVRO-947
> Project: Avro
> Issue Type: Improvement
> Components: java
> Reporter: Doug Cutting
>
> The DatumReader API is meant to be thread-safe, in that many threads should be able to simultaneously use a single DatumReader instance. When we first switched GenericDatumReader to use a resolver thread-safety was broken. AVRO-650 repaired this with a thread-local resolver cache, but performance suffered. AVRO-743 improved performance for single-threaded applications, but multi-threaded performance is still poor.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira