You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2017/04/11 11:54:41 UTC
[jira] [Commented] (FLINK-6295) use LoadingCache instead of
WeakHashMap to lower latency
[ https://issues.apache.org/jira/browse/FLINK-6295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15964253#comment-15964253 ]
ASF GitHub Bot commented on FLINK-6295:
---------------------------------------
GitHub user WangTaoTheTonic opened a pull request:
https://github.com/apache/flink/pull/3709
[FLINK-6295]use LoadingCache instead of WeakHashMap to lower latency
Now in ExecutionGraphHolder, which is used in many handlers, we use a WeakHashMap to cache ExecutionGraph(s), which is only sensitive to garbage collection.
The latency is too high when JVM do GC rarely, which will make status of jobs or its tasks unmatched with the real ones. (WE once observed that the web still shows tasks cancelled/failed, after the actual states of tasks coming back to normal for **30+ mins,** until a gc happened)
LoadingCache is a common used cache implementation from guava lib, we can use its time based eviction to lower latency of status update.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/WangTaoTheTonic/flink FLINK-6295
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/3709.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3709
----
commit d76ced06242623d150f9ad09205e2b92f910c1a1
Author: WangTaoTheTonic <wa...@huawei.com>
Date: 2017-04-11T11:48:52Z
use LoadingCache instead of WeakHashMap to lower latency
----
> use LoadingCache instead of WeakHashMap to lower latency
> --------------------------------------------------------
>
> Key: FLINK-6295
> URL: https://issues.apache.org/jira/browse/FLINK-6295
> Project: Flink
> Issue Type: Bug
> Components: Webfrontend
> Reporter: Tao Wang
> Assignee: Tao Wang
>
> Now in ExecutionGraphHolder, which is used in many handlers, we use a WeakHashMap to cache ExecutionGraph(s), which is only sensitive to garbage collection.
> The latency is too high when JVM do GC rarely, which will make status of jobs or its tasks unmatched with the real ones.
> LoadingCache is a common used cache implementation from guava lib, we can use its time based eviction to lower latency of status update.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)