You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@orc.apache.org by "Owen O'Malley (JIRA)" <ji...@apache.org> on 2019/08/02 19:16:00 UTC
[jira] [Resolved] (ORC-536) "expectedEntries should be > 0" error
when bloom filters enabled but rowIndexStride set to 0
[ https://issues.apache.org/jira/browse/ORC-536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley resolved ORC-536.
-------------------------------
Resolution: Fixed
Assignee: Jason Dere
Fix Version/s: 1.6.0
1.5.7
I just committed this. Thanks, Jason!
> "expectedEntries should be > 0" error when bloom filters enabled but rowIndexStride set to 0
> --------------------------------------------------------------------------------------------
>
> Key: ORC-536
> URL: https://issues.apache.org/jira/browse/ORC-536
> Project: ORC
> Issue Type: Bug
> Components: Java
> Reporter: Jason Dere
> Assignee: Jason Dere
> Priority: Major
> Fix For: 1.5.7, 1.6.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {noformat}
> java.lang.IllegalArgumentException: expectedEntries should be > 0
> at shadeorcorg.apache.orc.util.BloomFilter.checkArgument(BloomFilter.java:54)
> at shadeorcorg.apache.orc.util.BloomFilter.<init>(BloomFilter.java:63)
> at shadeorcorg.apache.orc.util.BloomFilterUtf8.<init>(BloomFilterUtf8.java:30)
> at shadeorcorg.apache.orc.impl.writer.TreeWriterBase.<init>(TreeWriterBase.java:116)
> at shadeorcorg.apache.orc.impl.writer.StringBaseTreeWriter.<init>(StringBaseTreeWriter.java:64)
> at shadeorcorg.apache.orc.impl.writer.StringTreeWriter.<init>(StringTreeWriter.java:33)
> at shadeorcorg.apache.orc.impl.writer.TreeWriter$Factory.create(TreeWriter.java:129)
> at shadeorcorg.apache.orc.impl.writer.StructTreeWriter.<init>(StructTreeWriter.java:41)
> at shadeorcorg.apache.orc.impl.writer.TreeWriter$Factory.create(TreeWriter.java:155)
> at shadeorcorg.apache.orc.impl.writer.StructTreeWriter.<init>(StructTreeWriter.java:41)
> at shadeorcorg.apache.orc.impl.writer.TreeWriter$Factory.create(TreeWriter.java:155)
> at shadeorcorg.apache.orc.impl.WriterImpl.<init>(WriterImpl.java:180)
> at shadehive.org.apache.hadoop.hive.ql.io.orc.WriterImpl.<init>(WriterImpl.java:94)
> at shadehive.org.apache.hadoop.hive.ql.io.orc.OrcFile.createWriter(OrcFile.java:378)
> at shadehive.org.apache.hadoop.hive.ql.io.orc.OrcRecordUpdater.initWriter(OrcRecordUpdater.java:603)
> at shadehive.org.apache.hadoop.hive.ql.io.orc.OrcRecordUpdater.addSimpleEvent(OrcRecordUpdater.java:424)
> at shadehive.org.apache.hadoop.hive.ql.io.orc.OrcRecordUpdater.addSplitUpdateEvent(OrcRecordUpdater.java:433)
> at shadehive.org.apache.hadoop.hive.ql.io.orc.OrcRecordUpdater.insert(OrcRecordUpdater.java:485)
> at org.apache.hive.streaming.AbstractRecordWriter.write(AbstractRecordWriter.java:439)
> at org.apache.hive.streaming.HiveStreamingConnection$TransactionBatch.write(HiveStreamingConnection.java:794)
> {noformat}
> Pretty easy to repro with TestWriterImpl:
> {code}
> public void testNoBFIfNoIndex() throws Exception {
> // overriding the flag should result in a successful write (no exception)
> conf.set(OrcConf.OVERWRITE_OUTPUT_FILE.getAttribute(), "true");
> // Enable bloomfilter, but disable index
> conf.set(OrcConf.ROW_INDEX_STRIDE.getAttribute(), "0");
> conf.set(OrcConf.BLOOM_FILTER_COLUMNS.getAttribute(), "*");
> Writer w = OrcFile.createWriter(testFilePath, OrcFile.writerOptions(conf).setSchema(schema));
> w.close();
> }
> {code}
> cc [~prasanth_j]
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)