[jira] [Commented] (AVRO-1760) Thread scalability problem with the use of SynchronizedMap

["Mulugeta Mammo (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AVRO-1760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15032691#comment-15032691 ] 

Mulugeta Mammo commented on AVRO-1760:
--------------------------------------

[~rdblue]

I checked WeakIdentityConcurrentMap and it doesn't solve the issue. Threads block at org.apache.avro.util.WeakIdentityMap.reap() (see attached). ConcurrentHashMap experiences no blocking.

Please let me know if you would like to fix the issue or go with ConcurrentHashMap (with no copying). Also, do we have data on the performance of a weak hashmap over strong hashmap?

Best,

> Thread scalability problem with the use of SynchronizedMap
> ----------------------------------------------------------
>
>                 Key: AVRO-1760
>                 URL: https://issues.apache.org/jira/browse/AVRO-1760
>             Project: Avro
>          Issue Type: Improvement
>          Components: java
>    Affects Versions: 1.7.4, 1.7.5, 1.7.6, 1.7.7, 1.8.0, 1.8.1, 1.9.0
>            Reporter: Mulugeta Mammo
>            Priority: Critical
>              Labels: patch, performance
>             Fix For: 1.7.4, 1.7.5, 1.7.6, 1.7.7, 1.7.8, 1.8.0, 1.8.1, 1.9.0
>
>         Attachments: AVRO-1760.patch, AVRO-1760.pdf
>
>
> While running Adam Genomics (which uses Avro) on Apache Spark, we discovered that threads (tasks in Spark Context) block in Avro while executing the getDefaultValue(Field field) method in https://github.com/apache/avro/blob/trunk/lang/java/avro/src/main/java/org/apache/avro/generic/GenericData.java



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)