You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@drill.apache.org by "Timothy Farkas (JIRA)" <ji...@apache.org> on 2018/04/25 08:31:00 UTC
[jira] [Comment Edited] (DRILL-6352) Investigate
IndexOutOfBoundsException in TestBsonRecordReader
[ https://issues.apache.org/jira/browse/DRILL-6352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16451865#comment-16451865 ]
Timothy Farkas edited comment on DRILL-6352 at 4/25/18 8:30 AM:
----------------------------------------------------------------
The SingleMapWriter should check if the allocation suceeded
{code}
FieldWriter writer = fields.get(name.toLowerCase());
if(writer == null) {
ValueVector vector;
ValueVector currentVector = container.getChild(name);
if (unionEnabled){
NullableVarCharVector v = container.addOrGet(name, VARCHAR_TYPE, NullableVarCharVector.class);
writer = new PromotableWriter(v, container);
vector = v;
} else {
NullableVarCharVector v = container.addOrGet(name, VARCHAR_TYPE, NullableVarCharVector.class);
writer = new NullableVarCharWriterImpl(v, this);
vector = v;
}
if (currentVector == null || currentVector != vector) {
vector.allocateNewSafe();
}
writer.setPosition(idx());
fields.put(name.toLowerCase(), writer);
}
return writer;
{code}
was (Author: timothyfarkas):
The single map writer should check if the allocation suceeded
{code}
FieldWriter writer = fields.get(name.toLowerCase());
if(writer == null) {
ValueVector vector;
ValueVector currentVector = container.getChild(name);
if (unionEnabled){
NullableVarCharVector v = container.addOrGet(name, VARCHAR_TYPE, NullableVarCharVector.class);
writer = new PromotableWriter(v, container);
vector = v;
} else {
NullableVarCharVector v = container.addOrGet(name, VARCHAR_TYPE, NullableVarCharVector.class);
writer = new NullableVarCharWriterImpl(v, this);
vector = v;
}
if (currentVector == null || currentVector != vector) {
vector.allocateNewSafe();
}
writer.setPosition(idx());
fields.put(name.toLowerCase(), writer);
}
return writer;
{code}
> Investigate IndexOutOfBoundsException in TestBsonRecordReader
> -------------------------------------------------------------
>
> Key: DRILL-6352
> URL: https://issues.apache.org/jira/browse/DRILL-6352
> Project: Apache Drill
> Issue Type: Bug
> Reporter: Timothy Farkas
> Assignee: Timothy Farkas
> Priority: Major
>
> TestBsonRecordReader requires 400kb on the allocator in order to run all tests successfully. Reducing the memory below that to 300kb will cause an IOB
> {code}
> objc[92518]: Class JavaLaunchHelper is implemented in both /Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/bin/java (0x10e8fe4c0) and /Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home/jre/lib/libinstrument.dylib (0x10e9824e0). One of the two will be used. Which one is undefined.
> java.lang.IndexOutOfBoundsException: DrillBuf[7], udle: [1 0..0], index: 0, length: 4 (expected: range(0, 0))
> DrillBuf[7], udle: [1 0..0]
> at org.apache.drill.exec.memory.BoundsChecking.checkIndex(BoundsChecking.java:80)
> at org.apache.drill.exec.memory.BoundsChecking.lengthCheck(BoundsChecking.java:86)
> at io.netty.buffer.DrillBuf.chk(DrillBuf.java:114)
> at io.netty.buffer.DrillBuf.getInt(DrillBuf.java:484)
> at org.apache.drill.exec.vector.VarCharVector$Mutator.setSafe(VarCharVector.java:696)
> at org.apache.drill.exec.vector.NullableVarCharVector$Mutator.setSafe(NullableVarCharVector.java:609)
> at org.apache.drill.exec.vector.complex.impl.NullableVarCharWriterImpl.write(NullableVarCharWriterImpl.java:110)
> at org.apache.drill.exec.store.bson.BsonRecordReader.writeString(BsonRecordReader.java:276)
> at org.apache.drill.exec.store.bson.BsonRecordReader.writeToListOrMap(BsonRecordReader.java:167)
> at org.apache.drill.exec.store.bson.BsonRecordReader.writeToListOrMap(BsonRecordReader.java:139)
> at org.apache.drill.exec.store.bson.BsonRecordReader.write(BsonRecordReader.java:75)
> at org.apache.drill.exec.store.bson.TestBsonRecordReader.testRecursiveDocuments(TestBsonRecordReader.java:193)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
> at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
> at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
> at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)