[jira] [Created] (SLIDER-243) slider client should use cluster configuration for dfs.replication

["Jonathan Maron (JIRA)" <ji...@apache.org>]

Jonathan Maron created SLIDER-243:
-------------------------------------

             Summary: slider client should use cluster configuration for dfs.replication
                 Key: SLIDER-243
                 URL: https://issues.apache.org/jira/browse/SLIDER-243
             Project: Slider
          Issue Type: Bug
            Reporter: Jonathan Maron


In the slider client code, SliderClient's getConfig() call (inherited from AbstractService) appears to return a default value of "3" for the "dfs.replication".  However, if the cluster is not configured for "3" then attempts to create a cluster/application result in an exception:

org.apache.hadoop.ipc.RemoteException(java.io.IOException): file /user/nn/.slider/cluster/cl1/writelock on client 240.0.0.10.
Requested replication 3 exceeds maximum 1
	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.verifyReplication(BlockManager.java:938)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInt(FSNamesystem.java:2218)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:2190)
	at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.create(NameNodeRpcServer.java:526)
	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.create(ClientNamenodeProtocolServerSideTranslatorPB.java:354)
	at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)
	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:928)
	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2013)
	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2009)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:396)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1557)
	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2007)

There needs to be a mechanism in place that will set the value from the client to match the value set for the given hadoop cluster.



--
This message was sent by Atlassian JIRA
(v6.2#6252)