You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Harsh J (JIRA)" <ji...@apache.org> on 2012/06/17 08:43:43 UTC
[jira] [Created] (MAPREDUCE-4343) ZK recovery support for
ResourceManager
Harsh J created MAPREDUCE-4343:
----------------------------------
Summary: ZK recovery support for ResourceManager
Key: MAPREDUCE-4343
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4343
Project: Hadoop Map/Reduce
Issue Type: Improvement
Reporter: Harsh J
MAPREDUCE-279 included bits and pieces of possible ZK integration for YARN's RM, but looks like it failed to complete it (for scalability reasons? etc?) and there seems to be no JIRA tracking this feature that has been already claimed publicly as a good part about YARN.
If it did complete it, we should document how to use it. Setting the following only yields:
{code}
<property>
<name>yarn.resourcemanager.store.class</name>
<value>org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKStore</value>
</property>
<property>
<name>yarn.resourcemanager.zookeeper-store.address</name>
<value>test.vm:2181/yarn-recovery-store</value>
</property>
{code}
{code}
Error starting ResourceManager
java.lang.RuntimeException: java.lang.NoSuchMethodException: org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKStore.<init>()
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:128)
at org.apache.hadoop.yarn.server.resourcemanager.recovery.StoreFactory.getStore(StoreFactory.java:32)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:621)
Caused by: java.lang.NoSuchMethodException: org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKStore.<init>()
at java.lang.Class.getConstructor0(Class.java:2706)
at java.lang.Class.getDeclaredConstructor(Class.java:1985)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:122)
... 2 more
{code}
This JIRA is hence filed to track the addition/completion of recovery via ZK.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4343) ZK recovery support for
ResourceManager
Posted by "Tsuyoshi OZAWA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13396063#comment-13396063 ]
Tsuyoshi OZAWA commented on MAPREDUCE-4343:
-------------------------------------------
Sharad,
Bikas marked MAPREDUCE-2713 as a duplicated task.
> ZK recovery support for ResourceManager
> ---------------------------------------
>
> Key: MAPREDUCE-4343
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4343
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Reporter: Harsh J
> Attachments: MR-4343.1.patch
>
>
> MAPREDUCE-279 included bits and pieces of possible ZK integration for YARN's RM, but looks like it failed to complete it (for scalability reasons? etc?) and there seems to be no JIRA tracking this feature that has been already claimed publicly as a good part about YARN.
> If it did complete it, we should document how to use it. Setting the following only yields:
> {code}
> <property>
> <name>yarn.resourcemanager.store.class</name>
> <value>org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKStore</value>
> </property>
> <property>
> <name>yarn.resourcemanager.zookeeper-store.address</name>
> <value>test.vm:2181/yarn-recovery-store</value>
> </property>
> {code}
> {code}
> Error starting ResourceManager
> java.lang.RuntimeException: java.lang.NoSuchMethodException: org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKStore.<init>()
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:128)
> at org.apache.hadoop.yarn.server.resourcemanager.recovery.StoreFactory.getStore(StoreFactory.java:32)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:621)
> Caused by: java.lang.NoSuchMethodException: org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKStore.<init>()
> at java.lang.Class.getConstructor0(Class.java:2706)
> at java.lang.Class.getDeclaredConstructor(Class.java:1985)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:122)
> ... 2 more
> {code}
> This JIRA is hence filed to track the addition/completion of recovery via ZK.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4343) ZK recovery support for
ResourceManager
Posted by "Tsuyoshi OZAWA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsuyoshi OZAWA updated MAPREDUCE-4343:
--------------------------------------
Attachment: MR-4343.1.patch
Harsh,
The attached patch allows ResourceManager to use ZKStore.
> ZK recovery support for ResourceManager
> ---------------------------------------
>
> Key: MAPREDUCE-4343
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4343
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Reporter: Harsh J
> Attachments: MR-4343.1.patch
>
>
> MAPREDUCE-279 included bits and pieces of possible ZK integration for YARN's RM, but looks like it failed to complete it (for scalability reasons? etc?) and there seems to be no JIRA tracking this feature that has been already claimed publicly as a good part about YARN.
> If it did complete it, we should document how to use it. Setting the following only yields:
> {code}
> <property>
> <name>yarn.resourcemanager.store.class</name>
> <value>org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKStore</value>
> </property>
> <property>
> <name>yarn.resourcemanager.zookeeper-store.address</name>
> <value>test.vm:2181/yarn-recovery-store</value>
> </property>
> {code}
> {code}
> Error starting ResourceManager
> java.lang.RuntimeException: java.lang.NoSuchMethodException: org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKStore.<init>()
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:128)
> at org.apache.hadoop.yarn.server.resourcemanager.recovery.StoreFactory.getStore(StoreFactory.java:32)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:621)
> Caused by: java.lang.NoSuchMethodException: org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKStore.<init>()
> at java.lang.Class.getConstructor0(Class.java:2706)
> at java.lang.Class.getDeclaredConstructor(Class.java:1985)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:122)
> ... 2 more
> {code}
> This JIRA is hence filed to track the addition/completion of recovery via ZK.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4343) ZK recovery support for
ResourceManager
Posted by "Sharad Agarwal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13395869#comment-13395869 ]
Sharad Agarwal commented on MAPREDUCE-4343:
-------------------------------------------
There is already MAPREDUCE-2713 for this. Some ZK code may be lying around but it is not implemented as yet.
can this be marked as duplicate ?
> ZK recovery support for ResourceManager
> ---------------------------------------
>
> Key: MAPREDUCE-4343
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4343
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Reporter: Harsh J
> Attachments: MR-4343.1.patch
>
>
> MAPREDUCE-279 included bits and pieces of possible ZK integration for YARN's RM, but looks like it failed to complete it (for scalability reasons? etc?) and there seems to be no JIRA tracking this feature that has been already claimed publicly as a good part about YARN.
> If it did complete it, we should document how to use it. Setting the following only yields:
> {code}
> <property>
> <name>yarn.resourcemanager.store.class</name>
> <value>org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKStore</value>
> </property>
> <property>
> <name>yarn.resourcemanager.zookeeper-store.address</name>
> <value>test.vm:2181/yarn-recovery-store</value>
> </property>
> {code}
> {code}
> Error starting ResourceManager
> java.lang.RuntimeException: java.lang.NoSuchMethodException: org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKStore.<init>()
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:128)
> at org.apache.hadoop.yarn.server.resourcemanager.recovery.StoreFactory.getStore(StoreFactory.java:32)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:621)
> Caused by: java.lang.NoSuchMethodException: org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKStore.<init>()
> at java.lang.Class.getConstructor0(Class.java:2706)
> at java.lang.Class.getDeclaredConstructor(Class.java:1985)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:122)
> ... 2 more
> {code}
> This JIRA is hence filed to track the addition/completion of recovery via ZK.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4343) ZK recovery support for
ResourceManager
Posted by "Harsh J (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13393514#comment-13393514 ]
Harsh J commented on MAPREDUCE-4343:
------------------------------------
Hi,
With (1) done in your local build, are you able to successfully validate that (2) is possible today already? In case it is (that the RM recovers its state properly, once ZKStore is fixed), we can just do (1) here and add some docs on how to use ZKStore (the configs) and resolve the ticket.
> ZK recovery support for ResourceManager
> ---------------------------------------
>
> Key: MAPREDUCE-4343
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4343
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Reporter: Harsh J
>
> MAPREDUCE-279 included bits and pieces of possible ZK integration for YARN's RM, but looks like it failed to complete it (for scalability reasons? etc?) and there seems to be no JIRA tracking this feature that has been already claimed publicly as a good part about YARN.
> If it did complete it, we should document how to use it. Setting the following only yields:
> {code}
> <property>
> <name>yarn.resourcemanager.store.class</name>
> <value>org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKStore</value>
> </property>
> <property>
> <name>yarn.resourcemanager.zookeeper-store.address</name>
> <value>test.vm:2181/yarn-recovery-store</value>
> </property>
> {code}
> {code}
> Error starting ResourceManager
> java.lang.RuntimeException: java.lang.NoSuchMethodException: org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKStore.<init>()
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:128)
> at org.apache.hadoop.yarn.server.resourcemanager.recovery.StoreFactory.getStore(StoreFactory.java:32)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:621)
> Caused by: java.lang.NoSuchMethodException: org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKStore.<init>()
> at java.lang.Class.getConstructor0(Class.java:2706)
> at java.lang.Class.getDeclaredConstructor(Class.java:1985)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:122)
> ... 2 more
> {code}
> This JIRA is hence filed to track the addition/completion of recovery via ZK.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4343) ZK recovery support for
ResourceManager
Posted by "Tsuyoshi OZAWA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13393519#comment-13393519 ]
Tsuyoshi OZAWA commented on MAPREDUCE-4343:
-------------------------------------------
I'm trying to solve the 2nd problem now, and I've not estimated how large the work is.
I'll report the status before I go to sleep.
> ZK recovery support for ResourceManager
> ---------------------------------------
>
> Key: MAPREDUCE-4343
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4343
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Reporter: Harsh J
>
> MAPREDUCE-279 included bits and pieces of possible ZK integration for YARN's RM, but looks like it failed to complete it (for scalability reasons? etc?) and there seems to be no JIRA tracking this feature that has been already claimed publicly as a good part about YARN.
> If it did complete it, we should document how to use it. Setting the following only yields:
> {code}
> <property>
> <name>yarn.resourcemanager.store.class</name>
> <value>org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKStore</value>
> </property>
> <property>
> <name>yarn.resourcemanager.zookeeper-store.address</name>
> <value>test.vm:2181/yarn-recovery-store</value>
> </property>
> {code}
> {code}
> Error starting ResourceManager
> java.lang.RuntimeException: java.lang.NoSuchMethodException: org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKStore.<init>()
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:128)
> at org.apache.hadoop.yarn.server.resourcemanager.recovery.StoreFactory.getStore(StoreFactory.java:32)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:621)
> Caused by: java.lang.NoSuchMethodException: org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKStore.<init>()
> at java.lang.Class.getConstructor0(Class.java:2706)
> at java.lang.Class.getDeclaredConstructor(Class.java:1985)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:122)
> ... 2 more
> {code}
> This JIRA is hence filed to track the addition/completion of recovery via ZK.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4343) ZK recovery support for
ResourceManager
Posted by "Tsuyoshi OZAWA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13393567#comment-13393567 ]
Tsuyoshi OZAWA commented on MAPREDUCE-4343:
-------------------------------------------
It's very course patch but works on my environment, so could you review it?
> ZK recovery support for ResourceManager
> ---------------------------------------
>
> Key: MAPREDUCE-4343
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4343
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Reporter: Harsh J
> Attachments: MR-4343.1.patch
>
>
> MAPREDUCE-279 included bits and pieces of possible ZK integration for YARN's RM, but looks like it failed to complete it (for scalability reasons? etc?) and there seems to be no JIRA tracking this feature that has been already claimed publicly as a good part about YARN.
> If it did complete it, we should document how to use it. Setting the following only yields:
> {code}
> <property>
> <name>yarn.resourcemanager.store.class</name>
> <value>org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKStore</value>
> </property>
> <property>
> <name>yarn.resourcemanager.zookeeper-store.address</name>
> <value>test.vm:2181/yarn-recovery-store</value>
> </property>
> {code}
> {code}
> Error starting ResourceManager
> java.lang.RuntimeException: java.lang.NoSuchMethodException: org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKStore.<init>()
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:128)
> at org.apache.hadoop.yarn.server.resourcemanager.recovery.StoreFactory.getStore(StoreFactory.java:32)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:621)
> Caused by: java.lang.NoSuchMethodException: org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKStore.<init>()
> at java.lang.Class.getConstructor0(Class.java:2706)
> at java.lang.Class.getDeclaredConstructor(Class.java:1985)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:122)
> ... 2 more
> {code}
> This JIRA is hence filed to track the addition/completion of recovery via ZK.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-4343) ZK recovery support for
ResourceManager
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy resolved MAPREDUCE-4343.
--------------------------------------
Resolution: Duplicate
Duplicate of MAPREDUCE-4326.
> ZK recovery support for ResourceManager
> ---------------------------------------
>
> Key: MAPREDUCE-4343
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4343
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Reporter: Harsh J
> Attachments: MR-4343.1.patch
>
>
> MAPREDUCE-279 included bits and pieces of possible ZK integration for YARN's RM, but looks like it failed to complete it (for scalability reasons? etc?) and there seems to be no JIRA tracking this feature that has been already claimed publicly as a good part about YARN.
> If it did complete it, we should document how to use it. Setting the following only yields:
> {code}
> <property>
> <name>yarn.resourcemanager.store.class</name>
> <value>org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKStore</value>
> </property>
> <property>
> <name>yarn.resourcemanager.zookeeper-store.address</name>
> <value>test.vm:2181/yarn-recovery-store</value>
> </property>
> {code}
> {code}
> Error starting ResourceManager
> java.lang.RuntimeException: java.lang.NoSuchMethodException: org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKStore.<init>()
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:128)
> at org.apache.hadoop.yarn.server.resourcemanager.recovery.StoreFactory.getStore(StoreFactory.java:32)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:621)
> Caused by: java.lang.NoSuchMethodException: org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKStore.<init>()
> at java.lang.Class.getConstructor0(Class.java:2706)
> at java.lang.Class.getDeclaredConstructor(Class.java:1985)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:122)
> ... 2 more
> {code}
> This JIRA is hence filed to track the addition/completion of recovery via ZK.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4343) ZK recovery support for
ResourceManager
Posted by "Tsuyoshi OZAWA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13393569#comment-13393569 ]
Tsuyoshi OZAWA commented on MAPREDUCE-4343:
-------------------------------------------
s/course/coarse/
> ZK recovery support for ResourceManager
> ---------------------------------------
>
> Key: MAPREDUCE-4343
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4343
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Reporter: Harsh J
> Attachments: MR-4343.1.patch
>
>
> MAPREDUCE-279 included bits and pieces of possible ZK integration for YARN's RM, but looks like it failed to complete it (for scalability reasons? etc?) and there seems to be no JIRA tracking this feature that has been already claimed publicly as a good part about YARN.
> If it did complete it, we should document how to use it. Setting the following only yields:
> {code}
> <property>
> <name>yarn.resourcemanager.store.class</name>
> <value>org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKStore</value>
> </property>
> <property>
> <name>yarn.resourcemanager.zookeeper-store.address</name>
> <value>test.vm:2181/yarn-recovery-store</value>
> </property>
> {code}
> {code}
> Error starting ResourceManager
> java.lang.RuntimeException: java.lang.NoSuchMethodException: org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKStore.<init>()
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:128)
> at org.apache.hadoop.yarn.server.resourcemanager.recovery.StoreFactory.getStore(StoreFactory.java:32)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:621)
> Caused by: java.lang.NoSuchMethodException: org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKStore.<init>()
> at java.lang.Class.getConstructor0(Class.java:2706)
> at java.lang.Class.getDeclaredConstructor(Class.java:1985)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:122)
> ... 2 more
> {code}
> This JIRA is hence filed to track the addition/completion of recovery via ZK.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4343) ZK recovery support for
ResourceManager
Posted by "Harsh J (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13396611#comment-13396611 ]
Harsh J commented on MAPREDUCE-4343:
------------------------------------
Thanks Sharad and Arun. I missed out MAPREDUCE-2713 in my search (apparently missed searching for 'recover' (instead searched for availability)). Sorry for the noise.
Thanks much for the immediate patch Tsuyoshi. It may be considered now in MAPREDUCE-4326 unless the whole thing is being revamped.
> ZK recovery support for ResourceManager
> ---------------------------------------
>
> Key: MAPREDUCE-4343
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4343
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Reporter: Harsh J
> Attachments: MR-4343.1.patch
>
>
> MAPREDUCE-279 included bits and pieces of possible ZK integration for YARN's RM, but looks like it failed to complete it (for scalability reasons? etc?) and there seems to be no JIRA tracking this feature that has been already claimed publicly as a good part about YARN.
> If it did complete it, we should document how to use it. Setting the following only yields:
> {code}
> <property>
> <name>yarn.resourcemanager.store.class</name>
> <value>org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKStore</value>
> </property>
> <property>
> <name>yarn.resourcemanager.zookeeper-store.address</name>
> <value>test.vm:2181/yarn-recovery-store</value>
> </property>
> {code}
> {code}
> Error starting ResourceManager
> java.lang.RuntimeException: java.lang.NoSuchMethodException: org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKStore.<init>()
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:128)
> at org.apache.hadoop.yarn.server.resourcemanager.recovery.StoreFactory.getStore(StoreFactory.java:32)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:621)
> Caused by: java.lang.NoSuchMethodException: org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKStore.<init>()
> at java.lang.Class.getConstructor0(Class.java:2706)
> at java.lang.Class.getDeclaredConstructor(Class.java:1985)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:122)
> ... 2 more
> {code}
> This JIRA is hence filed to track the addition/completion of recovery via ZK.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4343) ZK recovery support for
ResourceManager
Posted by "Tsuyoshi OZAWA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13393510#comment-13393510 ]
Tsuyoshi OZAWA commented on MAPREDUCE-4343:
-------------------------------------------
Harsh,
I think this problem should be split into 2 tickets as follows:
1. Fix the runtime error of ZKStore by adding default constructor.
2. Add ZK recovery support to the resource manager.
In fact, I've already created the patch for fixing the 1st problem.
Should I attach the file here or on the new ticket?
> ZK recovery support for ResourceManager
> ---------------------------------------
>
> Key: MAPREDUCE-4343
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4343
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Reporter: Harsh J
>
> MAPREDUCE-279 included bits and pieces of possible ZK integration for YARN's RM, but looks like it failed to complete it (for scalability reasons? etc?) and there seems to be no JIRA tracking this feature that has been already claimed publicly as a good part about YARN.
> If it did complete it, we should document how to use it. Setting the following only yields:
> {code}
> <property>
> <name>yarn.resourcemanager.store.class</name>
> <value>org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKStore</value>
> </property>
> <property>
> <name>yarn.resourcemanager.zookeeper-store.address</name>
> <value>test.vm:2181/yarn-recovery-store</value>
> </property>
> {code}
> {code}
> Error starting ResourceManager
> java.lang.RuntimeException: java.lang.NoSuchMethodException: org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKStore.<init>()
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:128)
> at org.apache.hadoop.yarn.server.resourcemanager.recovery.StoreFactory.getStore(StoreFactory.java:32)
> at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:621)
> Caused by: java.lang.NoSuchMethodException: org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKStore.<init>()
> at java.lang.Class.getConstructor0(Class.java:2706)
> at java.lang.Class.getDeclaredConstructor(Class.java:1985)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:122)
> ... 2 more
> {code}
> This JIRA is hence filed to track the addition/completion of recovery via ZK.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira