[jira] [Commented] (BIGTOP-2453) nodemanager role requires common_mapred_app

["Konstantin Boudnik (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BIGTOP-2453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301459#comment-15301459 ] 

Konstantin Boudnik commented on BIGTOP-2453:
--------------------------------------------

Good catch, and totally makes sense! Will commit it shorly1

> nodemanager role requires common_mapred_app
> -------------------------------------------
>
>                 Key: BIGTOP-2453
>                 URL: https://issues.apache.org/jira/browse/BIGTOP-2453
>             Project: Bigtop
>          Issue Type: Bug
>          Components: deployment
>    Affects Versions: 1.1.0
>            Reporter: Kevin W Monroe
>            Assignee: Kevin W Monroe
>            Priority: Minor
>         Attachments: BIGTOP-2453.patch
>
>
> When deploying a standalone nodemanager, puppet apply fails in a couple different ways:
> First, /etc/hadoop/conf/core-site.xml is not created by the nodemanager class (or any of its prereqs). When trying to install a standalone nodemanger with the following in site.yaml:
> 'bigtop::roles_enabled': true
> 'bigtop::roles': nodemanager
> The puppet apply will fail like this:
> ...
> Error: Could not find dependency File[/etc/hadoop/conf/core-site.xml] for Service[hadoop-yarn-nodemanager] at /home/ubuntu/bigtop.release/bigtop-1.1.0/bigtop-deploy/puppet/modules/hadoop/manifests/init.pp:841
> ...
> At first glance, it may seem like we just need to add common_hdfs (the creator of core-site.xml) to the nodemanager class.  However, I tried that and ran up against a second problem (longer trace at http://paste.ubuntu.com/16637532/):
> ...
> java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.mapred.ShuffleHandler not found
> ...
> The problem here is that yarn-site.xml is specifying a missing mapred shuffle class, which prevents nodemanager from starting.  The solution for me was to include common_mapred_app in the nodemanager class (patch coming shortly).
> This solves both problems with one line since common_mapred_app installs the missing mapred shuffle class and includes common_hdfs to create core-site.xml.
> This may not be a common problem since I suspect most people are combining multiple slave roles (datanode, mapred-app, nodemanager) onto a single machine, but it would bite anyone that wanted to isolate the nodemanager role by itself.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)