[jira] [Created] (BIGTOP-2442) resourcemanager role should not require hdfs_init

["Kevin W Monroe (JIRA)" <ji...@apache.org>]

Kevin W Monroe created BIGTOP-2442:
--------------------------------------

             Summary: resourcemanager role should not require hdfs_init
                 Key: BIGTOP-2442
                 URL: https://issues.apache.org/jira/browse/BIGTOP-2442
             Project: Bigtop
          Issue Type: Bug
          Components: deployment
    Affects Versions: 1.1.0
            Reporter: Kevin W Monroe
            Priority: Minor


When deploying a standalone resourcemanager, puppet apply fails because hadoop::init_hdfs is not included in the resourcemanager role.  This isn't a problem when deploying both a namenode and resource manager on the same unit (as is typically done on the head node), but it is problematic when you try to deploy RM by itself with no hdfs components.

Reproduce with the following in ./bigtop-deploy/puppet/hieradata/site.yaml:

'bigtop::roles_enabled': true
'bigtop::roles': resourcemanager

Then run puppet apply from ./bigtop-1.1.0:

puppet apply -d --modulepath=bigtop-deploy/puppet/modules:/etc/puppet/modules bigtop-deploy/puppet/manifests/site.pp
...
Error: Could not find resource 'Class[Hadoop::Init_hdfs]' for relationship on 'Class[Hadoop::Resourcemanager]' on node <xyz>
...

This is fixable in a couple different ways.  First, just "include hadoop::init_hdfs" in the resourcemanager role in init.pp.  That feels a bit heavy handed as I don't think the RM really needs to handle "init_hdfs".

The second solution is to remove the two Init_hdfs references from the resourcemanger role.  This feels safe to me, and is what we're using in the bigtop juju charms to deploy a standalone resourcemanager.

I'll attach a patch shortly, but I'm interested in thoughts about this approach.  Presumably, if you're deploying a standalone RM, you've got a namenode somewhere and you've already done the init_hdfs steps.  Are there any scenarios where we really need the RM to do this?

I was thinking about the history server storing info in hdfs, so perhaps it could be a problem if somehow you tried to start RM with a historyserver hdfs location before hdfs was available... but I dont know why someone would do that.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)