You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@bigtop.apache.org by "Konstantin Boudnik (JIRA)" <ji...@apache.org> on 2016/05/20 22:33:12 UTC

[jira] [Commented] (BIGTOP-2442) resourcemanager role should not require hdfs_init

    [ https://issues.apache.org/jira/browse/BIGTOP-2442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294407#comment-15294407 ] 

Konstantin Boudnik commented on BIGTOP-2442:
--------------------------------------------

I think getting rid of the call to the init_hdfs is the most appropriate. Quick look into the code shows that the same problem exists for HistoryServer class. Would be great to have this fixed, indeed!

> resourcemanager role should not require hdfs_init
> -------------------------------------------------
>
>                 Key: BIGTOP-2442
>                 URL: https://issues.apache.org/jira/browse/BIGTOP-2442
>             Project: Bigtop
>          Issue Type: Bug
>          Components: deployment
>    Affects Versions: 1.1.0
>            Reporter: Kevin W Monroe
>            Priority: Minor
>
> When deploying a standalone resourcemanager, puppet apply fails because hadoop::init_hdfs is not included in the resourcemanager role.  This isn't a problem when deploying both a namenode and resource manager on the same unit (as is typically done on the head node), but it is problematic when you try to deploy RM by itself with no hdfs components.
> Reproduce with the following in ./bigtop-deploy/puppet/hieradata/site.yaml:
> 'bigtop::roles_enabled': true
> 'bigtop::roles': resourcemanager
> Then run puppet apply from ./bigtop-1.1.0:
> puppet apply -d --modulepath=bigtop-deploy/puppet/modules:/etc/puppet/modules bigtop-deploy/puppet/manifests/site.pp
> ...
> Error: Could not find resource 'Class[Hadoop::Init_hdfs]' for relationship on 'Class[Hadoop::Resourcemanager]' on node <xyz>
> ...
> This is fixable in a couple different ways.  First, just "include hadoop::init_hdfs" in the resourcemanager role in init.pp.  That feels a bit heavy handed as I don't think the RM really needs to handle "init_hdfs".
> The second solution is to remove the two Init_hdfs references from the resourcemanger role.  This feels safe to me, and is what we're using in the bigtop juju charms to deploy a standalone resourcemanager.
> I'll attach a patch shortly, but I'm interested in thoughts about this approach.  Presumably, if you're deploying a standalone RM, you've got a namenode somewhere and you've already done the init_hdfs steps.  Are there any scenarios where we really need the RM to do this?
> I was thinking about the history server storing info in hdfs, so perhaps it could be a problem if somehow you tried to start RM with a historyserver hdfs location before hdfs was available... but I dont know why someone would do that.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)