You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Jason Lowe (JIRA)" <ji...@apache.org> on 2016/08/18 20:00:24 UTC
[jira] [Updated] (YARN-5049) Extend NMStateStore to save queued
container information
[ https://issues.apache.org/jira/browse/YARN-5049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Lowe updated YARN-5049:
-----------------------------
Hadoop Flags: Incompatible change
Sorry for getting here late, as I just ran into this by accident. This breaks rolling upgrades because it changes the major version of the NM state store schema. Therefore when a new NM comes up on an old state store it crashes like this:
{noformat}
2016-08-18 19:37:26,713 INFO [main] service.AbstractService (AbstractService.java:noteFailure(272)) - Service NodeManager failed in state INITED; cause: org.apache.hadoop.service.ServiceStateException: java.io.IOException: Incompatible version for NM state: expecting NM state version 2.0, but loading version 1.0
org.apache.hadoop.service.ServiceStateException: java.io.IOException: Incompatible version for NM state: expecting NM state version 2.0, but loading version 1.0
at org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:59)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:172)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartRecoveryStore(NodeManager.java:236)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:300)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:762)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:823)
Caused by: java.io.IOException: Incompatible version for NM state: expecting NM state version 2.0, but loading version 1.0
at org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService.checkVersion(NMLeveldbStateStoreService.java:1226)
at org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService.initStorage(NMLeveldbStateStoreService.java:1085)
at org.apache.hadoop.yarn.server.nodemanager.recovery.NMStateStoreService.serviceInit(NMStateStoreService.java:249)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
... 5 more
{noformat}
Breaking the compatibility, while annoying, is not terrible if this just went into trunk. However this was also put into branch-2, therefore this breaks rolling upgrades from 2.8->2.9. As such this should be reverted from branch-2 until there's a migration path from schema 1 to schema 2 that doesn't break the NM startup.
> Extend NMStateStore to save queued container information
> --------------------------------------------------------
>
> Key: YARN-5049
> URL: https://issues.apache.org/jira/browse/YARN-5049
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager, resourcemanager
> Reporter: Konstantinos Karanasos
> Assignee: Konstantinos Karanasos
> Fix For: 2.9.0
>
> Attachments: YARN-5049.001.patch, YARN-5049.002.patch, YARN-5049.003.patch
>
>
> This JIRA is about extending the NMStateStore to save queued container information whenever a new container is added to the NM queue.
> It also removes the information from the state store when the queued container starts its execution.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org