You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by "Zhong Yanghong (JIRA)" <ji...@apache.org> on 2017/08/02 08:32:00 UTC
[jira] [Commented] (KYLIN-2716) Using non-thread-safe WeakHashMap
leading to server high cpu
[ https://issues.apache.org/jira/browse/KYLIN-2716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16110528#comment-16110528 ]
Zhong Yanghong commented on KYLIN-2716:
---------------------------------------
Thanks [~liyang.gmt8@gmail.com]'s help. The patch has been refined according your suggestion.
> Using non-thread-safe WeakHashMap leading to server high cpu
> ------------------------------------------------------------
>
> Key: KYLIN-2716
> URL: https://issues.apache.org/jira/browse/KYLIN-2716
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: all
> Reporter: Zhong Yanghong
> Assignee: Zhong Yanghong
> Fix For: v2.1.0
>
> Attachments: APACHE-KYLIN-2716.patch
>
>
> Multiple threads invoke WeakHashMap.get() simultaneously may leading to a dead loop in *WeakHashMap.get() -> getTable() -> expungeStaleEntries()*, which finally resulting in server high cpu. There're two places using WeakHashMap.
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> One is used in the method *ClassUtil.forName()*.
> We made an inner test by invoking the method *ClassUtil.forName()* 1M times, the result is as follows:
> * With cache: 20ms;
> * Without cache: less than 2s.
> By invoking the method *ClassUtil.forName()* with *newInstance()* 1M times, the result is as follows:
> * With cache: around 2s
> * Without cache: around 3s.
> Considering *ClassUtil.forName()* is always invoked with *newInstance()*, there's no much downgrade without cache. Thus the fix is just to remove the cache.
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> Another is used in the method *CubeService.getHTableInfo()*. We changed the WeakHashMap to Guava Cache with introducing size & time limitation.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)