You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2025/11/06 00:24:00 UTC
[jira] [Commented] (HADOOP-18389) Limit stacked call of one connection in client to avoid possible oom in server
[ https://issues.apache.org/jira/browse/HADOOP-18389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18035712#comment-18035712 ]
ASF GitHub Bot commented on HADOOP-18389:
-----------------------------------------
github-actions[bot] commented on PR #4691:
URL: https://github.com/apache/hadoop/pull/4691#issuecomment-3494234029
We're closing this stale PR because it has been open for 100 days with no activity. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you feel like this was a mistake, or you would like to continue working on it, please feel free to re-open it and ask for a committer to remove the stale tag and review again.
Thanks all for your contribution.
> Limit stacked call of one connection in client to avoid possible oom in server
> ------------------------------------------------------------------------------
>
> Key: HADOOP-18389
> URL: https://issues.apache.org/jira/browse/HADOOP-18389
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: ZanderXu
> Assignee: ZanderXu
> Priority: Major
> Labels: pull-request-available
> Attachments: image-2022-08-04-00-22-28-865.png, image-2022-08-04-00-23-18-427.png
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> In our prod environment, we encountered an accident that JN OOM because Server#Connection#responseQueue used 97% memory.
> After analyzed the memory of JN and found that there are 2w+ called stacked in one Server#Connection#responseQueue, because the network between NN and JN jitters with some tcp packet loss.
> !image-2022-08-04-00-22-28-865.png|width=561,height=254!
> !image-2022-08-04-00-23-18-427.png|width=559,height=356!
>
> In this case, I think Client.java should support limit the stacked calls of one connection to avoid the possible OOM in Server. When the number of stacked calls is more than the limit size, we can just throw one IOException to the method caller.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org