You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-dev@hadoop.apache.org by "Tarun Parimi (JIRA)" <ji...@apache.org> on 2019/02/28 12:12:00 UTC
[jira] [Created] (YARN-9336) JobHistoryServer leaks CLOSE_WAIT tcp
connections when using LogAggregationIndexedFileController
Tarun Parimi created YARN-9336:
----------------------------------
Summary: JobHistoryServer leaks CLOSE_WAIT tcp connections when using LogAggregationIndexedFileController
Key: YARN-9336
URL: https://issues.apache.org/jira/browse/YARN-9336
Project: Hadoop YARN
Issue Type: Bug
Components: log-aggregation
Affects Versions: 2.9.0
Reporter: Tarun Parimi
Assignee: Tarun Parimi
The JobHistoryServer is leaking CLOSE_WAIT connections whenever viewing a huge log file in JobhistoryServer. This happens only when the below is configured.
{code:java}
yarn.log-aggregation.file-formats=IndexedFormat
yarn.log-aggregation.file-controller.IndexedFormat.class=org.apache.hadoop.yarn.logaggregation.filecontroller.ifile.LogAggregationIndexedFileController
yarn.log.server.url=http://jobhistory-host:19888/jobhistory/logs
{code}
On investigation, I found that the FSDataInputStream is not closed in org.apache.hadoop.yarn.logaggregation.filecontroller.ifile.IndexedFileAggregatedLogsBlock . Since this block is called every time the Josbhistory page displays the logs CLOSE_WAIT connections to datanode keep on increasing in JobHistoryServer.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org