You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@orc.apache.org by "EbCead (JIRA)" <ji...@apache.org> on 2017/09/13 06:39:00 UTC
[jira] [Comment Edited] (ORC-237) OrcFile.mergeFiles Specified
block size is less than configured minimum value
[ https://issues.apache.org/jira/browse/ORC-237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16164212#comment-16164212 ]
EbCead edited comment on ORC-237 at 9/13/17 6:38 AM:
-----------------------------------------------------
{code:java}
public static void doMerge(Configuration configuration, String inputPath, String outputPath, Log LOG) throws Exception {
Path oPath = new Path(outputPath);
FileSystem fileSystem = FileSystem.get(configuration);
if (fileSystem.exists(oPath)) {
fileSystem.delete(oPath, true);
}
OrcFile.WriterOptions writerOptions = OrcFile.writerOptions(configuration);
List<Path> pathList = FileSystemUtil.getFiles(fileSystem, inputPath, LOG);
OrcFile.mergeFiles(oPath, writerOptions, pathList);
}
{code}
was (Author: pipe):
{code:java}
public static void doMerge(Configuration configuration, String inputPath, String outputPath, Log LOG) throws Exception {
Path oPath = new Path(outputPath);
FileSystem fileSystem = FileSystem.get(configuration);
if (fileSystem.exists(oPath)) {
fileSystem.delete(oPath, true);
}
OrcFile.WriterOptions writerOptions = OrcFile.writerOptions(configuration);
List<Path> pathList = FileSystemUtil.getFiles(fileSystem, inputPath, LOG);
OrcFile.mergeFiles(oPath, writerOptions, pathList);
{code}
> OrcFile.mergeFiles Specified block size is less than configured minimum value
> -----------------------------------------------------------------------------
>
> Key: ORC-237
> URL: https://issues.apache.org/jira/browse/ORC-237
> Project: ORC
> Issue Type: Bug
> Reporter: EbCead
>
> impl.PhysicalFsWriter: ORC writer created for path: /dw/ods/order_orc/success/dt=2017-06-28_tmp/part-m-00000.orc with stripeSize: 67108864 blockSize: 131072 compression: ZLIB bufferSize: 131072
> Specified block size is less than configured minimum value (dfs.namenode.fs-limits.min-block-size): 131072 < 1048576
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)