You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Yuan Luo (Jira)" <ji...@apache.org> on 2024/03/15 02:52:00 UTC
[jira] [Comment Edited] (YARN-11663) Router cache expansion issue
[ https://issues.apache.org/jira/browse/YARN-11663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17827323#comment-17827323 ]
Yuan Luo edited comment on YARN-11663 at 3/15/24 2:51 AM:
----------------------------------------------------------
[~slfan1989] Thanks for your reply.
Our cache setting is 60 seconds, and memory recycling is triggered by full gc. The number of cached objects keeps growing.
!image-2024-03-15-10-50-32-860.png!
I test ehcache or guava cache will not clean up expired keys immediately, it will only be triggered when accessed. After the job ends, the cache key will never be accessed again.
was (Author: luoyuan):
Our cache setting is 60 seconds, and memory recycling is triggered by full gc. The number of cached objects keeps growing.
!image-2024-03-15-10-50-32-860.png!
I test ehcache or guava cache will not clean up expired keys immediately, it will only be triggered when accessed. After the job ends, the cache key will never be accessed again.
> Router cache expansion issue
> ----------------------------
>
> Key: YARN-11663
> URL: https://issues.apache.org/jira/browse/YARN-11663
> Project: Hadoop YARN
> Issue Type: Bug
> Components: federation, yarn
> Affects Versions: 3.4.0
> Reporter: Yuan Luo
> Priority: Major
> Attachments: image-2024-03-14-18-12-28-426.png, image-2024-03-14-18-12-49-950.png, image-2024-03-15-10-50-32-860.png
>
>
> !image-2024-03-14-18-12-28-426.png!
> !image-2024-03-14-18-12-49-950.png!
> hi [~slfan1989] After apply this feature to our prod env, I found the memory of the router keeps growing over time. This is because after jobs finished, we won't access the expired key to trigger cleanup mechanism. Is it better to add cache maximum number limit？
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org