[jira] [Commented] (TEZ-1526) LoadingCache for TezTaskID slow for large jobs

["Rohini Palaniswamy (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-1526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15280861#comment-15280861 ] 

Rohini Palaniswamy commented on TEZ-1526:
-----------------------------------------

http://static.javadoc.io/com.google.guava/guava/r03/com/google/common/collect/MapMaker.html#weakValues() can probably be used instead of our own implementation

> LoadingCache for TezTaskID slow for large jobs
> ----------------------------------------------
>
>                 Key: TEZ-1526
>                 URL: https://issues.apache.org/jira/browse/TEZ-1526
>             Project: Apache Tez
>          Issue Type: Improvement
>            Reporter: Jonathan Eagles
>            Assignee: Jonathan Eagles
>              Labels: performance
>         Attachments: 100000-TezTaskIDs.patch, TEZ-1526-v1.patch, TEZ-1526-v2.patch, TEZ-1526.3.patch, TEZ-1526.4.patch, TEZ-1526.5.patch
>
>
> Using the LoadingCache with default builder settings. 100,000 TezTaskIDs are created in 10 seconds on my setup. With a LoadingCache initialCapacity of 10,000 they are created in 300 ms. With no LoadingCache, they are created in 10 ms. A test case in attached to illustrate the condition I would like to be sped up.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)