hdfs over http error

["John Beaulaurier -X (jbeaulau - ADVANCED NETWORK INFORMATION INC at Cisco)" <jb...@cisco.com>]

Hello,



Apache Hadoop 0.20.203.0



A colleague is using a SPARK shell on a remote host using HDFS protocol attempting to run a job

on our Hadoop cluster, but the job errors out before finishing with the following noted in the namenode log.



2014-06-11 16:13:24,958 WARN org.apache.hadoop.security.ShellBasedUnixGroupsMapping: got exception trying to get groups for user webuser org.apache.hadoop.util.Shell$ExitCodeException: id: webuser: No such user



The username submitting the job does not exists on the localhost running as namenode, but a three layered directory tree is created in HDFS owned by

the username submitting the job, but stops at a certain point with the error noted above, and the last directory empty.



hadoop fs -ls  /existing_dir/sparkcounts3/_temporary/_attempt_201406101047_0000_m_000000_5



I found online that the default for web access is user webuser  group webgroup, neither are configured on the namenode host OS. If I do the following

should it resolve the issue.



1.        create a local group named webgroup

2.       Create local user with the same name as is submitting the job,

3.       Add that local user to the group webgroup

4.       Add the following property to hdfs-site.xml



<property>

  <name>dfs.web.ugi</name>

  <value> webgroup</value>

</property>



Is a restart of dfs, or mapred, or both necessary after the adding the property?



Thanks

-John