You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Weihao Zheng (Jira)" <ji...@apache.org> on 2025/01/21 07:08:00 UTC
[jira] [Commented] (YARN-10213) Using curator-leader-elector, rm will always be in standby state at some times.
[ https://issues.apache.org/jira/browse/YARN-10213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17914869#comment-17914869 ]
Weihao Zheng commented on YARN-10213:
-------------------------------------
Thanks [~Sen Zhao] , this case can be easily reproduced when creating new YARN cluster with yarn.resourcemanager.ha.curator-leader-elector.enabled=true and starting all RMs at the same time. All RMs keep in standby mode until the current leader RM is restarted manually.
We can find this log pattern in the standby RM (with smallest sequence znode in latch path), which shows that the RM has already transitioned to active after elected but transitions to standby wrongly and does not call rejoinElection.
{quote}2025-01-21 14:28:38,799 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2025-01-21 14:28:38,800 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 8032: starting
2025-01-21 14:28:38,851 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Transitioned to active state
2025-01-21 14:28:38,851 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=hadoop OPERATION=transitionToActive TARGET=RM RESULT=SUCCESS
2025-01-21 14:28:38,851 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Transitioning to standby state
2025-01-21 14:28:38,852 WARN org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher: org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher$LauncherThread interrupted. Returning.
{quote}
Hi [~bender] [~rkanter] cloud you take over this issue, since you guys are interesting in Make CuratorBasedElectorService the default HA elector for YARN ResourceManager.
> Using curator-leader-elector, rm will always be in standby state at some times.
> -------------------------------------------------------------------------------
>
> Key: YARN-10213
> URL: https://issues.apache.org/jira/browse/YARN-10213
> Project: Hadoop YARN
> Issue Type: Bug
> Affects Versions: 3.2.1, 3.1.3
> Reporter: Sen Zhao
> Assignee: Sen Zhao
> Priority: Major
> Attachments: YARN-10213.001.patch, YARN-10213.002.patch
>
>
> When we use cursor-leader-elector, *CuratorBasedElectorService.initAndStartLeaderLatch()*
> will initialize and start *leaderLatch* to elect the leader.
> Now：
> 1.ResourceManager will call *CuratorBasedElectorService.initAndStartLeaderLatch()* in *serviceInit()*.
> 2.ResourceManager will transition to standby in *serviceStart()*.
> If *leaderLatch* succeeds in electing leader before rm transitions to standby. ResourceManager will always be in standby state.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org