You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2022/07/24 02:56:00 UTC
[jira] [Work logged] (HDFS-16642) [SBN read] Moving selecting inputstream from JN in EditlogTailer out of FSNLock
[ https://issues.apache.org/jira/browse/HDFS-16642?focusedWorklogId=794587&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-794587 ]
ASF GitHub Bot logged work on HDFS-16642:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 24/Jul/22 02:55
Start Date: 24/Jul/22 02:55
Worklog Time Spent: 10m
Work Description: ZanderXu commented on PR #4497:
URL: https://github.com/apache/hadoop/pull/4497#issuecomment-1193235376
> Looks thread-safe to me. Has this patch been tested in a production environment?
@jojochuang Thanks for your review and sorry sorry for replaying so late. We have release it in our prod environment and it works well.
After checking some jstacks of ObserverNamenodes, I found that there are no handlers blocked at `Selecting EditInputStreams`, so the performance should be improved.
@ferhui can you help me review this patch?
Issue Time Tracking
-------------------
Worklog Id: (was: 794587)
Time Spent: 40m (was: 0.5h)
> [SBN read] Moving selecting inputstream from JN in EditlogTailer out of FSNLock
> -------------------------------------------------------------------------------
>
> Key: HDFS-16642
> URL: https://issues.apache.org/jira/browse/HDFS-16642
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: ZanderXu
> Assignee: ZanderXu
> Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> EditlogTailer cost a long time for selecting InputStreams from Journalnode while holding the write lock of FSNLock.
> During this period, 8020 handlers of Observer NameNode will be blocked by the FSN Lock.
> In theory, selecting inputstreams from JournalNode does not involve changing memory information in NameNode, so we can move the selecting out of the FSN Lock, and it can improve the throughput of Observer NameNode.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org