You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2022/06/24 07:40:00 UTC
[jira] [Work logged] (HDFS-16642) [HDFS] Moving selecting inputstream from JN in EditlogTailer out of FSNLock
[ https://issues.apache.org/jira/browse/HDFS-16642?focusedWorklogId=784441&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-784441 ]
ASF GitHub Bot logged work on HDFS-16642:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 24/Jun/22 07:39
Start Date: 24/Jun/22 07:39
Worklog Time Spent: 10m
Work Description: ZanderXu opened a new pull request, #4497:
URL: https://github.com/apache/hadoop/pull/4497
### Description of PR
In our prod environment, 8020 Handler of Observer NameNode also blocked by EditlogTailer.
And EditlogTailer cost a long time for selecting InputStreams from Journalnode with holding the FSNLock.
During this period, 8020 handlers of Observer NameNode will be blocked by the FSN Lock.
In theory, selecting inputstreams from JournalNode does not involve changing memory information in NameNode, so we can move the selecting out of the FSN Lock, and it can improve the throughput of Observer NameNode.
After reading the relevant code in depth, I found that it is feasible to move the selecting inputStream out of the FSNLock.
Issue Time Tracking
-------------------
Worklog Id: (was: 784441)
Remaining Estimate: 0h
Time Spent: 10m
> [HDFS] Moving selecting inputstream from JN in EditlogTailer out of FSNLock
> ---------------------------------------------------------------------------
>
> Key: HDFS-16642
> URL: https://issues.apache.org/jira/browse/HDFS-16642
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: ZanderXu
> Assignee: ZanderXu
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> EditlogTailer cost a long time for selecting InputStreams from Journalnode while holding the write lock of FSNLock.
> During this period, 8020 handlers of Observer NameNode will be blocked by the FSN Lock.
> In theory, selecting inputstreams from JournalNode does not involve changing memory information in NameNode, so we can move the selecting out of the FSN Lock, and it can improve the throughput of Observer NameNode.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org