You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Ian Jackson <Ia...@trilliumsoftware.com> on 2013/11/08 20:20:04 UTC
HDFS-NFS Windows
Is it possible to export using HDFS-NFS on Windows 7?
Re: HDFS-NFS Windows
Posted by Brandon Li <br...@hortonworks.com>.
A workaround has been committed to trunk/branch-2/branch2.2.1.
https://issues.apache.org/jira/browse/HDFS-5469
Now users can define any HDFS sub-directory to share. We can just add the
following property in hdfs-site.xml.
<property>
<name>dfs.nfs3.export.point</name>
<value>/somedirectory</value>
</property>
Another thing with mounting NFS on Windows is the user mapping between
Windows accounts and Linux accounts.
Not that, if the users don't care about the files ownership, they don't
even need to map Window user to Linux user. NFS gateway maps all unknown
user/group to "nobody"/"nobody", which have the Linux (UID, GID) as (-2,
-2).
In this case, to let Windows user access HDFS through NFS, all we need to
do is to create a directory with access mode 777 in HDFS. Then export that
directory or its parent directory(as long as it's not root "/"). After
mount it as a drive on Windows, any Windows user can access it.
Brandon
On Mon, Nov 11, 2013 at 7:04 AM, Ian Jackson <
Ian_Jackson@trilliumsoftware.com> wrote:
> A slight syntax error in my net use command, you must have a colon after
> the drive letter. For example (be sure to replace the 10.70.245.49 with
> your host name or IP address):
>
> net use Z: \\10.70.245.49\!
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: HDFS-NFS Windows
Posted by Brandon Li <br...@hortonworks.com>.
A workaround has been committed to trunk/branch-2/branch2.2.1.
https://issues.apache.org/jira/browse/HDFS-5469
Now users can define any HDFS sub-directory to share. We can just add the
following property in hdfs-site.xml.
<property>
<name>dfs.nfs3.export.point</name>
<value>/somedirectory</value>
</property>
Another thing with mounting NFS on Windows is the user mapping between
Windows accounts and Linux accounts.
Not that, if the users don't care about the files ownership, they don't
even need to map Window user to Linux user. NFS gateway maps all unknown
user/group to "nobody"/"nobody", which have the Linux (UID, GID) as (-2,
-2).
In this case, to let Windows user access HDFS through NFS, all we need to
do is to create a directory with access mode 777 in HDFS. Then export that
directory or its parent directory(as long as it's not root "/"). After
mount it as a drive on Windows, any Windows user can access it.
Brandon
On Mon, Nov 11, 2013 at 7:04 AM, Ian Jackson <
Ian_Jackson@trilliumsoftware.com> wrote:
> A slight syntax error in my net use command, you must have a colon after
> the drive letter. For example (be sure to replace the 10.70.245.49 with
> your host name or IP address):
>
> net use Z: \\10.70.245.49\!
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: HDFS-NFS Windows
Posted by Brandon Li <br...@hortonworks.com>.
A workaround has been committed to trunk/branch-2/branch2.2.1.
https://issues.apache.org/jira/browse/HDFS-5469
Now users can define any HDFS sub-directory to share. We can just add the
following property in hdfs-site.xml.
<property>
<name>dfs.nfs3.export.point</name>
<value>/somedirectory</value>
</property>
Another thing with mounting NFS on Windows is the user mapping between
Windows accounts and Linux accounts.
Not that, if the users don't care about the files ownership, they don't
even need to map Window user to Linux user. NFS gateway maps all unknown
user/group to "nobody"/"nobody", which have the Linux (UID, GID) as (-2,
-2).
In this case, to let Windows user access HDFS through NFS, all we need to
do is to create a directory with access mode 777 in HDFS. Then export that
directory or its parent directory(as long as it's not root "/"). After
mount it as a drive on Windows, any Windows user can access it.
Brandon
On Mon, Nov 11, 2013 at 7:04 AM, Ian Jackson <
Ian_Jackson@trilliumsoftware.com> wrote:
> A slight syntax error in my net use command, you must have a colon after
> the drive letter. For example (be sure to replace the 10.70.245.49 with
> your host name or IP address):
>
> net use Z: \\10.70.245.49\!
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: HDFS-NFS Windows
Posted by Brandon Li <br...@hortonworks.com>.
A workaround has been committed to trunk/branch-2/branch2.2.1.
https://issues.apache.org/jira/browse/HDFS-5469
Now users can define any HDFS sub-directory to share. We can just add the
following property in hdfs-site.xml.
<property>
<name>dfs.nfs3.export.point</name>
<value>/somedirectory</value>
</property>
Another thing with mounting NFS on Windows is the user mapping between
Windows accounts and Linux accounts.
Not that, if the users don't care about the files ownership, they don't
even need to map Window user to Linux user. NFS gateway maps all unknown
user/group to "nobody"/"nobody", which have the Linux (UID, GID) as (-2,
-2).
In this case, to let Windows user access HDFS through NFS, all we need to
do is to create a directory with access mode 777 in HDFS. Then export that
directory or its parent directory(as long as it's not root "/"). After
mount it as a drive on Windows, any Windows user can access it.
Brandon
On Mon, Nov 11, 2013 at 7:04 AM, Ian Jackson <
Ian_Jackson@trilliumsoftware.com> wrote:
> A slight syntax error in my net use command, you must have a colon after
> the drive letter. For example (be sure to replace the 10.70.245.49 with
> your host name or IP address):
>
> net use Z: \\10.70.245.49\!
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
RE: HDFS-NFS Windows
Posted by Ian Jackson <Ia...@trilliumsoftware.com>.
A slight syntax error in my net use command, you must have a colon after the drive letter. For example (be sure to replace the 10.70.245.49 with your host name or IP address):
net use Z: \\10.70.245.49\!
RE: HDFS-NFS Windows
Posted by Ian Jackson <Ia...@trilliumsoftware.com>.
A slight syntax error in my net use command, you must have a colon after the drive letter. For example (be sure to replace the 10.70.245.49 with your host name or IP address):
net use Z: \\10.70.245.49\!
RE: HDFS-NFS Windows
Posted by Ian Jackson <Ia...@trilliumsoftware.com>.
A slight syntax error in my net use command, you must have a colon after the drive letter. For example (be sure to replace the 10.70.245.49 with your host name or IP address):
net use Z: \\10.70.245.49\!
RE: HDFS-NFS Windows
Posted by Ian Jackson <Ia...@trilliumsoftware.com>.
A slight syntax error in my net use command, you must have a colon after the drive letter. For example (be sure to replace the 10.70.245.49 with your host name or IP address):
net use Z: \\10.70.245.49\!
RE: HDFS-NFS Windows
Posted by Ian Jackson <Ia...@trilliumsoftware.com>.
Quite right. Three minor issues occurred. The Windows syntax for mounting the root directory is odd. You must use the exclamation point (!). For me if the server systems was Windows 7 the "use" command was required, for example "net use Z \\10.70.245.49\<file:///\\10.70.245.49\>!" where Z is the name of the drive that is used for the mount point. The last problem required the old saying "Use the Source." Neither portmap nor nf3 were included in the hdfs.cmd file. So one either needs to use Cygwin or edit the hdfs.cmd to add the following lines before ":make_command_agrguments"
@REM ADDED portmap and nfs 3
:portmap
set CLASS=org.apache.hadoop.portmap.Portmap
goto :eof
:nfs3
set CLASS=org.apache.hadoop.hdfs.nfs.nfs3.Nfs3
goto :eof
From: Olivier Renault [mailto:orenault@hortonworks.com]
Sent: Monday, November 11, 2013 4:12 AM
To: user@hadoop.apache.org
Subject: Re: HDFS-NFS Windows
You can use the NFS gateway to present HDFS. From Windows, you can then mount the NFS share.
Olivier
On 8 Nov 2013 19:20, "Ian Jackson" <Ia...@trilliumsoftware.com>> wrote:
Is it possible to export using HDFS-NFS on Windows 7?
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
RE: HDFS-NFS Windows
Posted by Ian Jackson <Ia...@trilliumsoftware.com>.
Quite right. Three minor issues occurred. The Windows syntax for mounting the root directory is odd. You must use the exclamation point (!). For me if the server systems was Windows 7 the "use" command was required, for example "net use Z \\10.70.245.49\<file:///\\10.70.245.49\>!" where Z is the name of the drive that is used for the mount point. The last problem required the old saying "Use the Source." Neither portmap nor nf3 were included in the hdfs.cmd file. So one either needs to use Cygwin or edit the hdfs.cmd to add the following lines before ":make_command_agrguments"
@REM ADDED portmap and nfs 3
:portmap
set CLASS=org.apache.hadoop.portmap.Portmap
goto :eof
:nfs3
set CLASS=org.apache.hadoop.hdfs.nfs.nfs3.Nfs3
goto :eof
From: Olivier Renault [mailto:orenault@hortonworks.com]
Sent: Monday, November 11, 2013 4:12 AM
To: user@hadoop.apache.org
Subject: Re: HDFS-NFS Windows
You can use the NFS gateway to present HDFS. From Windows, you can then mount the NFS share.
Olivier
On 8 Nov 2013 19:20, "Ian Jackson" <Ia...@trilliumsoftware.com>> wrote:
Is it possible to export using HDFS-NFS on Windows 7?
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
RE: HDFS-NFS Windows
Posted by Ian Jackson <Ia...@trilliumsoftware.com>.
Quite right. Three minor issues occurred. The Windows syntax for mounting the root directory is odd. You must use the exclamation point (!). For me if the server systems was Windows 7 the "use" command was required, for example "net use Z \\10.70.245.49\<file:///\\10.70.245.49\>!" where Z is the name of the drive that is used for the mount point. The last problem required the old saying "Use the Source." Neither portmap nor nf3 were included in the hdfs.cmd file. So one either needs to use Cygwin or edit the hdfs.cmd to add the following lines before ":make_command_agrguments"
@REM ADDED portmap and nfs 3
:portmap
set CLASS=org.apache.hadoop.portmap.Portmap
goto :eof
:nfs3
set CLASS=org.apache.hadoop.hdfs.nfs.nfs3.Nfs3
goto :eof
From: Olivier Renault [mailto:orenault@hortonworks.com]
Sent: Monday, November 11, 2013 4:12 AM
To: user@hadoop.apache.org
Subject: Re: HDFS-NFS Windows
You can use the NFS gateway to present HDFS. From Windows, you can then mount the NFS share.
Olivier
On 8 Nov 2013 19:20, "Ian Jackson" <Ia...@trilliumsoftware.com>> wrote:
Is it possible to export using HDFS-NFS on Windows 7?
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
RE: HDFS-NFS Windows
Posted by Ian Jackson <Ia...@trilliumsoftware.com>.
Quite right. Three minor issues occurred. The Windows syntax for mounting the root directory is odd. You must use the exclamation point (!). For me if the server systems was Windows 7 the "use" command was required, for example "net use Z \\10.70.245.49\<file:///\\10.70.245.49\>!" where Z is the name of the drive that is used for the mount point. The last problem required the old saying "Use the Source." Neither portmap nor nf3 were included in the hdfs.cmd file. So one either needs to use Cygwin or edit the hdfs.cmd to add the following lines before ":make_command_agrguments"
@REM ADDED portmap and nfs 3
:portmap
set CLASS=org.apache.hadoop.portmap.Portmap
goto :eof
:nfs3
set CLASS=org.apache.hadoop.hdfs.nfs.nfs3.Nfs3
goto :eof
From: Olivier Renault [mailto:orenault@hortonworks.com]
Sent: Monday, November 11, 2013 4:12 AM
To: user@hadoop.apache.org
Subject: Re: HDFS-NFS Windows
You can use the NFS gateway to present HDFS. From Windows, you can then mount the NFS share.
Olivier
On 8 Nov 2013 19:20, "Ian Jackson" <Ia...@trilliumsoftware.com>> wrote:
Is it possible to export using HDFS-NFS on Windows 7?
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: HDFS-NFS Windows
Posted by Olivier Renault <or...@hortonworks.com>.
You can use the NFS gateway to present HDFS. From Windows, you can then
mount the NFS share.
Olivier
On 8 Nov 2013 19:20, "Ian Jackson" <Ia...@trilliumsoftware.com> wrote:
> Is it possible to export using HDFS-NFS on Windows 7?
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: HDFS-NFS Windows
Posted by Olivier Renault <or...@hortonworks.com>.
You can use the NFS gateway to present HDFS. From Windows, you can then
mount the NFS share.
Olivier
On 8 Nov 2013 19:20, "Ian Jackson" <Ia...@trilliumsoftware.com> wrote:
> Is it possible to export using HDFS-NFS on Windows 7?
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: HDFS-NFS Windows
Posted by Olivier Renault <or...@hortonworks.com>.
You can use the NFS gateway to present HDFS. From Windows, you can then
mount the NFS share.
Olivier
On 8 Nov 2013 19:20, "Ian Jackson" <Ia...@trilliumsoftware.com> wrote:
> Is it possible to export using HDFS-NFS on Windows 7?
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: HDFS-NFS Windows
Posted by Olivier Renault <or...@hortonworks.com>.
You can use the NFS gateway to present HDFS. From Windows, you can then
mount the NFS share.
Olivier
On 8 Nov 2013 19:20, "Ian Jackson" <Ia...@trilliumsoftware.com> wrote:
> Is it possible to export using HDFS-NFS on Windows 7?
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.