[jira] [Created] (BIGTOP-2603) NN/RM charm should include a spark user/group

["Kevin W Monroe (JIRA)" <ji...@apache.org>]

Kevin W Monroe created BIGTOP-2603:
--------------------------------------

             Summary: NN/RM charm should include a spark user/group
                 Key: BIGTOP-2603
                 URL: https://issues.apache.org/jira/browse/BIGTOP-2603
             Project: Bigtop
          Issue Type: Bug
          Components: deployment
    Affects Versions: 1.1.0
            Reporter: Kevin W Monroe
            Assignee: Kevin W Monroe
            Priority: Minor


When in 'yarn-client' mode, the spark charm creates a directory in hdfs with 'spark' user/group permissions.

If spark is not colocated with the NN and RM, this user/group does not exist on those machines.  This causes warnings in the hdfs log like this:

WARN org.apache.hadoop.security.UserGroupInformation: No groups available for user spark

It also prevents the RM history server from accessing spark job history files.

To combat this, always include a spark user/group in the NN/RM charms. This is very similar to how we always create a 'mapred' user on the NN to prevent warnings about mapred being unknown to the namenode.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)