You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-user@hadoop.apache.org by Saurabh Jain <Sa...@symantec.com> on 2014/12/15 09:38:43 UTC

Permission issues with HDFS NFS gateway

Hi,

I am running a single node hadoop cluster and also the HDFS NFS gateway on the same node. HDFS processes are running under the context of user “hadoop” and NFS gateway is running under the context of user “nfsserver”. I mounted the NFS export on the same machine as the root user. Now whenever I try to write to NFS mount as the root user I get “permission denied” error. Also I constantly see this error message on the terminal, where I mounted the NFS export -
INFO security.ShellBasedIdMapping: Can't map group supergroup. Use its string hashcode:-1710818332

HDFS namenode/datanode NFS config -

core-site.xml -
<property>
<name>hadoop.proxyuser.nfsserver.groups</name>
  <value>*</value>
</property>
<property>
<name>hadoop.proxyuser.nfsserver.hosts</name>
<value>*</value>
</property>

hdfs-site.xml -
<property>
    <name>nfs.dump.dir</name>
    <value>/tmp/.hdfs-nfs</value>
</property>
<property>
<name>nfs.exports.allowed.hosts</name>
  <value>* rw</value>
</property>

HDFS NFS gateway config -
core-site.xml -
<property>
<name>hadoop.proxyuser.nfsserver.groups</name>
  <value>*</value>
</property>
<property>
  <name>hadoop.proxyuser.nfsserver.hosts</name>
  <value>* rw</value>
</property>

hdfs-site.xml -
<property>
    <name>nfs.dump.dir</name>
    <value>/tmp/.hdfs-nfs</value>
</property>
<property>
<name>nfs.exports.allowed.hosts</name>
<value>* rw</value>
</property>

When I try to do IO on NFS mount as “hadoop” user I don’t face any permission issue. Why am I getting permission denied error while trying to perform writes as root user?

Thanks
Saurabh

RE: Permission issues with HDFS NFS gateway

Posted by johny casanova <pc...@outlook.com>.

I know sometimes you need to be hdfs user to be able to complete certain tasks.

From: Saurabh_Jain@symantec.com
To: user@hadoop.apache.org
Date: Mon, 15 Dec 2014 00:38:43 -0800
Subject: Permission issues with HDFS NFS gateway

Hi,

I am running a single node hadoop cluster and also the HDFS NFS gateway on the same node. HDFS processes are running under the context of user “hadoop” and NFS gateway is running under the context of user “nfsserver”. I mounted the NFS export on the same machine as the root user. Now whenever I try to write to NFS mount as the root user I get “permission denied” error. Also I constantly see this error message on the terminal, where I mounted the NFS export - 
INFO security.ShellBasedIdMapping: Can't map group supergroup. Use its string hashcode:-1710818332 

HDFS namenode/datanode NFS config -

core-site.xml -
<property>
	<name>hadoop.proxyuser.nfsserver.groups</name>
  	<value>*</value>

</property>
<property>
	<name>hadoop.proxyuser.nfsserver.hosts</name>
	<value>*</value>
</property>

hdfs-site.xml -

<property>
    	<name>nfs.dump.dir</name>
    	<value>/tmp/.hdfs-nfs</value>
</property>
<property>
	<name>nfs.exports.allowed.hosts</name>
  	<value>* rw</value>
</property>

HDFS NFS gateway config -

core-site.xml -
<property>
	<name>hadoop.proxyuser.nfsserver.groups</name>
  	<value>*</value>
</property>
<property>
  	<name>hadoop.proxyuser.nfsserver.hosts</name>
  	<value>* rw</value>
</property>

hdfs-site.xml -

<property>
    	<name>nfs.dump.dir</name>
    	<value>/tmp/.hdfs-nfs</value>
</property>
<property>
	<name>nfs.exports.allowed.hosts</name>
	<value>* rw</value>
</property>

When I try to do IO on NFS mount as “hadoop” user I don’t face any permission issue. Why am I getting permission denied error while trying to perform writes as root user?

Thanks
Saurabh

RE: Permission issues with HDFS NFS gateway

Posted by johny casanova <pc...@outlook.com>.

I know sometimes you need to be hdfs user to be able to complete certain tasks.

From: Saurabh_Jain@symantec.com
To: user@hadoop.apache.org
Date: Mon, 15 Dec 2014 00:38:43 -0800
Subject: Permission issues with HDFS NFS gateway

Hi,

I am running a single node hadoop cluster and also the HDFS NFS gateway on the same node. HDFS processes are running under the context of user “hadoop” and NFS gateway is running under the context of user “nfsserver”. I mounted the NFS export on the same machine as the root user. Now whenever I try to write to NFS mount as the root user I get “permission denied” error. Also I constantly see this error message on the terminal, where I mounted the NFS export - 
INFO security.ShellBasedIdMapping: Can't map group supergroup. Use its string hashcode:-1710818332 

HDFS namenode/datanode NFS config -

core-site.xml -
<property>
	<name>hadoop.proxyuser.nfsserver.groups</name>
  	<value>*</value>

</property>
<property>
	<name>hadoop.proxyuser.nfsserver.hosts</name>
	<value>*</value>
</property>

hdfs-site.xml -

<property>
    	<name>nfs.dump.dir</name>
    	<value>/tmp/.hdfs-nfs</value>
</property>
<property>
	<name>nfs.exports.allowed.hosts</name>
  	<value>* rw</value>
</property>

HDFS NFS gateway config -

core-site.xml -
<property>
	<name>hadoop.proxyuser.nfsserver.groups</name>
  	<value>*</value>
</property>
<property>
  	<name>hadoop.proxyuser.nfsserver.hosts</name>
  	<value>* rw</value>
</property>

hdfs-site.xml -

<property>
    	<name>nfs.dump.dir</name>
    	<value>/tmp/.hdfs-nfs</value>
</property>
<property>
	<name>nfs.exports.allowed.hosts</name>
	<value>* rw</value>
</property>

When I try to do IO on NFS mount as “hadoop” user I don’t face any permission issue. Why am I getting permission denied error while trying to perform writes as root user?

Thanks
Saurabh

RE: Permission issues with HDFS NFS gateway

Posted by johny casanova <pc...@outlook.com>.

I know sometimes you need to be hdfs user to be able to complete certain tasks.

From: Saurabh_Jain@symantec.com
To: user@hadoop.apache.org
Date: Mon, 15 Dec 2014 00:38:43 -0800
Subject: Permission issues with HDFS NFS gateway

Hi,

I am running a single node hadoop cluster and also the HDFS NFS gateway on the same node. HDFS processes are running under the context of user “hadoop” and NFS gateway is running under the context of user “nfsserver”. I mounted the NFS export on the same machine as the root user. Now whenever I try to write to NFS mount as the root user I get “permission denied” error. Also I constantly see this error message on the terminal, where I mounted the NFS export - 
INFO security.ShellBasedIdMapping: Can't map group supergroup. Use its string hashcode:-1710818332 

HDFS namenode/datanode NFS config -

core-site.xml -
<property>
	<name>hadoop.proxyuser.nfsserver.groups</name>
  	<value>*</value>

</property>
<property>
	<name>hadoop.proxyuser.nfsserver.hosts</name>
	<value>*</value>
</property>

hdfs-site.xml -

<property>
    	<name>nfs.dump.dir</name>
    	<value>/tmp/.hdfs-nfs</value>
</property>
<property>
	<name>nfs.exports.allowed.hosts</name>
  	<value>* rw</value>
</property>

HDFS NFS gateway config -

core-site.xml -
<property>
	<name>hadoop.proxyuser.nfsserver.groups</name>
  	<value>*</value>
</property>
<property>
  	<name>hadoop.proxyuser.nfsserver.hosts</name>
  	<value>* rw</value>
</property>

hdfs-site.xml -

<property>
    	<name>nfs.dump.dir</name>
    	<value>/tmp/.hdfs-nfs</value>
</property>
<property>
	<name>nfs.exports.allowed.hosts</name>
	<value>* rw</value>
</property>

When I try to do IO on NFS mount as “hadoop” user I don’t face any permission issue. Why am I getting permission denied error while trying to perform writes as root user?

Thanks
Saurabh

RE: Permission issues with HDFS NFS gateway

Posted by johny casanova <pc...@outlook.com>.

I know sometimes you need to be hdfs user to be able to complete certain tasks.

From: Saurabh_Jain@symantec.com
To: user@hadoop.apache.org
Date: Mon, 15 Dec 2014 00:38:43 -0800
Subject: Permission issues with HDFS NFS gateway

Hi,

I am running a single node hadoop cluster and also the HDFS NFS gateway on the same node. HDFS processes are running under the context of user “hadoop” and NFS gateway is running under the context of user “nfsserver”. I mounted the NFS export on the same machine as the root user. Now whenever I try to write to NFS mount as the root user I get “permission denied” error. Also I constantly see this error message on the terminal, where I mounted the NFS export - 
INFO security.ShellBasedIdMapping: Can't map group supergroup. Use its string hashcode:-1710818332 

HDFS namenode/datanode NFS config -

core-site.xml -
<property>
	<name>hadoop.proxyuser.nfsserver.groups</name>
  	<value>*</value>

</property>
<property>
	<name>hadoop.proxyuser.nfsserver.hosts</name>
	<value>*</value>
</property>

hdfs-site.xml -

<property>
    	<name>nfs.dump.dir</name>
    	<value>/tmp/.hdfs-nfs</value>
</property>
<property>
	<name>nfs.exports.allowed.hosts</name>
  	<value>* rw</value>
</property>

HDFS NFS gateway config -

core-site.xml -
<property>
	<name>hadoop.proxyuser.nfsserver.groups</name>
  	<value>*</value>
</property>
<property>
  	<name>hadoop.proxyuser.nfsserver.hosts</name>
  	<value>* rw</value>
</property>

hdfs-site.xml -

<property>
    	<name>nfs.dump.dir</name>
    	<value>/tmp/.hdfs-nfs</value>
</property>
<property>
	<name>nfs.exports.allowed.hosts</name>
	<value>* rw</value>
</property>

When I try to do IO on NFS mount as “hadoop” user I don’t face any permission issue. Why am I getting permission denied error while trying to perform writes as root user?

Thanks
Saurabh