You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Jason Lowe (JIRA)" <ji...@apache.org> on 2016/09/09 12:43:21 UTC
[jira] [Commented] (YARN-5630) NM fails to start after downgrade
from 2.8 to 2.7
[ https://issues.apache.org/jira/browse/YARN-5630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15476980#comment-15476980 ]
Jason Lowe commented on YARN-5630:
----------------------------------
This was introduced by YARN-5221. Sample stacktrace:
{noformat}
2016-09-06 17:24:19,258 [main] INFO service.AbstractService: Service org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl failed in state INITED; cause: java.io.IOException: Unexpected container state key: ContainerManager/containers/container_e44_1472715025911_0001_01_000002/version
java.io.IOException: Unexpected container state key: ContainerManager/containers/container_e44_1472715025911_0001_01_000002/version
at org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService.loadContainerState(NMLeveldbStateStoreService.java:243)
at org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService.loadContainersState(NMLeveldbStateStoreService.java:182)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.recover(ContainerManagerImpl.java:267)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceInit(ContainerManagerImpl.java:251)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:263)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:507)
at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:555)
{noformat}
Ideally we should not store a version key unless we are using the feature that requires it. In other words, if the container version is 0 then we don't store the key and instead infer that if the version key is missing then the container version must be zero. I assume we already do this when upgrading from 2.7 to 2.8. This would preserve the ability to downgrade as long as nothing uses the increase container resource feature that needs the new version key. If something uses the increase container resource capability only then do we lose the ability to downgrade.
> NM fails to start after downgrade from 2.8 to 2.7
> -------------------------------------------------
>
> Key: YARN-5630
> URL: https://issues.apache.org/jira/browse/YARN-5630
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
> Reporter: Jason Lowe
> Priority: Blocker
>
> A downgrade from 2.8 to 2.7 causes nodemanagers to fail to start due to an unrecognized "version" container key on startup. This breaks downgrades from 2.8 to 2.7.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org