[jira] [Commented] (ORC-237) OrcFile.mergeFiles Specified block size is less than configured minimum value

["EbCead (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ORC-237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16164209#comment-16164209 ] 

EbCead commented on ORC-237:
----------------------------

17/09/12 19:28:11 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
17/09/12 19:28:11 INFO security.UserGroupInformation: Login successful for user xx using keytab file /Users/pipe/xx.keytab
17/09/12 19:28:12 WARN shortcircuit.DomainSocketFactory: The short-circuit local reads feature cannot be used because libhadoop cannot be loaded.
17/09/12 19:28:12 INFO orc.OrcMainTest: Input file list:===:hdfs://a1.dm.ambari:8020/dw/ods/order_orc/success/dt=2017-06-28/part-m-00000.orc
17/09/12 19:28:12 INFO orc.OrcMainTest: Input file size:===:9304992
17/09/12 19:28:12 INFO orc.OrcMainTest: Input file list:===:hdfs://a1.dm.ambari:8020/dw/ods/order_orc/success/dt=2017-06-28/part-m-00001.orc
17/09/12 19:28:12 INFO orc.OrcMainTest: Input file size:===:9616657
17/09/12 19:28:12 INFO orc.OrcMainTest: Input file list:===:hdfs://a1.dm.ambari:8020/dw/ods/order_orc/success/dt=2017-06-28/part-m-00002.orc
17/09/12 19:28:12 INFO orc.OrcMainTest: Input file size:===:3188876
17/09/12 19:28:12 INFO impl.OrcCodecPool: Got brand-new codec ZLIB
17/09/12 19:28:12 INFO impl.PhysicalFsWriter: ORC writer created for path: /dw/ods/order_orc/success/dt=2017-06-28_tmp/part-m-00000.orc with stripeSize: 67108864 blockSize: 131072 compression: ZLIB bufferSize: 131072

org.apache.hadoop.ipc.RemoteException(java.io.IOException): Specified block size is less than configured minimum value (dfs.namenode.fs-limits.min-block-size): 131072 < 1048576
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInt(FSNamesystem.java:2600)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:2555)
	at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.create(NameNodeRpcServer.java:735)
	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.create(ClientNamenodeProtocolServerSideTranslatorPB.java:408)
	at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640)
	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2351)
	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2347)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2345)


	at org.apache.hadoop.ipc.Client.call(Client.java:1475)
	at org.apache.hadoop.ipc.Client.call(Client.java:1412)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
	at com.sun.proxy.$Proxy15.create(Unknown Source)
	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.create(ClientNamenodeProtocolTranslatorPB.java:296)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
	at com.sun.proxy.$Proxy16.create(Unknown Source)
	at org.apache.hadoop.hdfs.DFSOutputStream.newStreamForCreate(DFSOutputStream.java:1648)
	at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1689)
	at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1624)
	at org.apache.hadoop.hdfs.DistributedFileSystem$7.doCall(DistributedFileSystem.java:448)
	at org.apache.hadoop.hdfs.DistributedFileSystem$7.doCall(DistributedFileSystem.java:444)
	at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
	at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:459)
	at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:387)
	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:911)
	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:892)
	at org.apache.orc.impl.PhysicalFsWriter.<init>(PhysicalFsWriter.java:91)
	at org.apache.orc.impl.WriterImpl.<init>(WriterImpl.java:184)
	at org.apache.orc.OrcFile.createWriter(OrcFile.java:685)
	at org.apache.orc.OrcFile.mergeFiles(OrcFile.java:830)


> OrcFile.mergeFiles Specified block size is less than configured minimum value
> -----------------------------------------------------------------------------
>
>                 Key: ORC-237
>                 URL: https://issues.apache.org/jira/browse/ORC-237
>             Project: ORC
>          Issue Type: Bug
>            Reporter: EbCead
>
> impl.PhysicalFsWriter: ORC writer created for path: /dw/ods/order_orc/success/dt=2017-06-28_tmp/part-m-00000.orc with stripeSize: 67108864 blockSize: 131072 compression: ZLIB bufferSize: 131072
> Specified block size is less than configured minimum value (dfs.namenode.fs-limits.min-block-size): 131072 < 1048576



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)