[jira] [Commented] (HUDI-104) Replace ThreadLocal with Resource pool when caching resource type classes

["Taher Koitawala (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HUDI-104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16924263#comment-16924263 ] 

Taher Koitawala commented on HUDI-104:
--------------------------------------

Hi [~vbalaji] I am not very much aware of what really the issue is in-depth. However from what I saw from the above comments. How about using at BalancedPool of PersistentActor? That will give a lot of isolation and will get your rid of all the synchronisation.

> Replace ThreadLocal with Resource pool when caching resource type classes
> -------------------------------------------------------------------------
>
>                 Key: HUDI-104
>                 URL: https://issues.apache.org/jira/browse/HUDI-104
>             Project: Apache Hudi (incubating)
>          Issue Type: Improvement
>          Components: Common Core, Performance
>            Reporter: BALAJI VARADARAJAN
>            Priority: Major
>
> This is another comment that stemmed from Timeline Server code-review.
>  
> Notes from PR comments:
> '''RandomAccessFile is not threadsafe. Hence, we would have to create separate RandomAccessFileObjects for concurrent access. Agree, the current approach of opening a new file per get() call is slow. I have made changes to use ThreadLocal objects. This should reduce the open() calls greatly but for a long running timeline service, there could be open handles getting accumulated per thread. A better approach would be to use Resource Pool but needs to be config tuned and tested. There is another place (ThreadLocal where we might require similar treatment. Will open a ticket to address it. If current perf testing points to the direction of Resource pool, will address it as part of this PR. Otherwise, we can take this in subsequent PR'''



--
This message was sent by Atlassian Jira
(v8.3.2#803003)