[jira] [Commented] (TEZ-1526) LoadingCache for TezTaskID slow for large jobs

["Jonathan Eagles (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-1526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14117048#comment-14117048 ] 

Jonathan Eagles commented on TEZ-1526:
--------------------------------------

One question I have is what solution the LoadingCache is trying to solve for the TezID classes? My understanding is the TezID classes immutable and therefore thread-safe by nature. Are there reasons other than concurrency that I need to be think about? I'd be happy to help implement the solution if this can be done with guidance. Otherwise, feel free for someone with experience to pick this up.

> LoadingCache for TezTaskID slow for large jobs
> ----------------------------------------------
>
>                 Key: TEZ-1526
>                 URL: https://issues.apache.org/jira/browse/TEZ-1526
>             Project: Apache Tez
>          Issue Type: Improvement
>            Reporter: Jonathan Eagles
>              Labels: performance
>         Attachments: 100000-TezTaskIDs.patch
>
>
> Using the LoadingCache with default builder settings. 100,000 TezTaskIDs are created in 10 seconds on my setup. With a LoadingCache initialCapacity of 10,000 they are created in 300 ms. With no LoadingCache, they are created in 10 ms. A test case in attached to illustrate the condition I would like to be sped up.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)