You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Hari Sekhon (JIRA)" <ji...@apache.org> on 2014/12/08 17:56:12 UTC
[jira] [Created] (HDFS-7488) HDFS Windows CIFS Gateway
Hari Sekhon created HDFS-7488:
---------------------------------
Summary: HDFS Windows CIFS Gateway
Key: HDFS-7488
URL: https://issues.apache.org/jira/browse/HDFS-7488
Project: Hadoop HDFS
Issue Type: New Feature
Affects Versions: 2.4.0
Environment: HDP 2.1
Reporter: Hari Sekhon
Stakeholders are pressuring for native Windows file share access to our Hadoop clusters.
I've used NFS gateway several times and while it's theoretically viable for users now UID mapping is implemented in 2.5... insecure NFS makes our fully Kerberized clusters security pointless.
We really need CIFS gateway access to enforce authentication which NFSv3 doesn't (NFSv4?).
I've even tried Samba over NFS gateway loopback mount point (don't laugh - they want it that badly), and enabled hdfs atime precision to an hour to prevent FSNamesystem.setTimes() java exceptions in gw logs, but the NFS server still doesn't like the Windows CIFS client actions:
{code}2014-12-08 16:31:38,053 ERROR nfs3.RpcProgramNfs3 (RpcProgramNfs3.java:setattr(346)) - Setting file size is not supported when setattr, fileId: 25597
2014-12-08 16:31:38,065 INFO nfs3.WriteManager (WriteManager.java:handleWrite(136)) - No opened stream for fileId:25597
2014-12-08 16:31:38,122 INFO nfs3.OpenFileCtx (OpenFileCtx.java:receivedNewWriteInternal(624)) - Have to change stable write to unstable write:FILE_SYNC
{code}
A debug of the Samba server shows it's trying to set metadata timestamps which hangs indefinitely, resulting in the creation of a zero byte file when trying to copy a file in to HDFS /tmp via the Windows mapped drive.
{code}
...
smb_set_file_time: setting utimes to modified values.
file_ntime: actime: Thu Jan 1 01:00:00 1970
file_ntime: modtime: Mon Dec 8 16:31:38 2014
file_ntime: ctime: Thu Jan 1 01:00:00 1970
file_ntime: createtime: Thu Jan 1 01:00:00 1970
{code}
This is the traceback from NFS gw log when hdfs precision was set to 0:
{code}org.apache.hadoop.ipc.RemoteException(java.io.IOException): Access time for hdfs is not configured. Please set dfs.namenode.accesstime.precision configuration parameter.
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.setTimes(FSNamesystem.java:1960)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.setTimes(NameNodeRpcServer.java:950)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.setTimes(ClientNamenodeProtocolServerSideTranslatorPB.java:833)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:928)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2013)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2009)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2007)
...
{code}
Regards,
Hari Sekhon
http://www.linkedin.com/in/harisekhon
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)