You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by "Deneche A. Hakim (JIRA)" <ji...@apache.org> on 2015/02/03 22:50:35 UTC
[jira] [Created] (DRILL-2151) VariableLengthVector.copyFromSafe()
unnecessary sets the offsetVector
Deneche A. Hakim created DRILL-2151:
---------------------------------------
Summary: VariableLengthVector.copyFromSafe() unnecessary sets the offsetVector
Key: DRILL-2151
URL: https://issues.apache.org/jira/browse/DRILL-2151
Project: Apache Drill
Issue Type: Bug
Components: Execution - Data Types
Reporter: Deneche A. Hakim
Assignee: Deneche A. Hakim
Priority: Critical
The following method defined in {{VariableLengthVectors.java}}:
{code}
public boolean copyFromSafe(int fromIndex, int thisIndex, ${minor.class}Vector from){
int start = from.offsetVector.getAccessor().get(fromIndex);
int end = from.offsetVector.getAccessor().get(fromIndex+1);
int len = end - start;
int outputStart = offsetVector.data.get${(minor.javaType!type.javaType)?cap_first}(thisIndex * ${type.width});
if(data.capacity() < outputStart + len) {
reAlloc();
}
offsetVector.getMutator().setSafe(thisIndex + 1, outputStart + len);
from.data.getBytes(start, data, outputStart, len);
offsetVector.getMutator().setSafe( (thisIndex+1) * ${type.width}, outputStart + len);
return true;
}
{code}
at the very end, it multiplies {{thisIndex+1}} by the type width, but the mutator will also multiply the index by the type width. This line should be removed because the offset is already set correctly in the same method.
One possible query where this actually causes problem is the one defined in DRILL-1588 (on TPCH SF 1):
{noformat}
// set slice target to 1 to ensure exchanges are used
0: jdbc:drill:zk=local> alter session set `planner.slice_target` = 1;
+------------+------------+
| ok | summary |
+------------+------------+
| true | planner.slice_target updated. |
+------------+------------+
1 row selected (0.1 seconds)
0: jdbc:drill:zk=local> select * from lineitem l left outer join orders o on (l.l_orderkey = o.o_orderkey) left outer join customer c on (l.l_orderkey = c.c_custkey) left outer join nation n on (l.l_partkey = n.n_nationkey) left outer join region r on (l.l_suppkey = r.r_regionkey) limit 10;
{noformat}
Here is a copy of the execution in the logs:
{noformat}
java.lang.IndexOutOfBoundsException: index: 60496, length: 4 (expected: range(0, 32768))
at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:156) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
at io.netty.buffer.DrillBuf.chk(DrillBuf.java:178) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
at io.netty.buffer.DrillBuf.setInt(DrillBuf.java:473) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
at org.apache.drill.exec.vector.UInt4Vector$Mutator.set(UInt4Vector.java:359) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at org.apache.drill.exec.vector.UInt4Vector$Mutator.setSafe(UInt4Vector.java:366) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at org.apache.drill.exec.vector.VarBinaryVector.copyFromSafe(VarBinaryVector.java:259) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at org.apache.drill.exec.vector.NullableVarBinaryVector.copyFromSafe(NullableVarBinaryVector.java:301) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at org.apache.drill.exec.test.generated.HashJoinProbeGen28.projectBuildRecord(HashJoinProbeTemplate.java:718) ~[na:na]
at org.apache.drill.exec.test.generated.HashJoinProbeGen28.executeProbePhase(HashJoinProbeTemplate.java:173) ~[na:na]
at org.apache.drill.exec.test.generated.HashJoinProbeGen28.probeAndProject(HashJoinProbeTemplate.java:223) ~[na:na]
at org.apache.drill.exec.physical.impl.join.HashJoinBatch.innerNext(HashJoinBatch.java:227) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:118) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:67) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at org.apache.drill.exec.physical.impl.partitionsender.PartitionSenderRootExec.innerNext(PartitionSenderRootExec.java:133) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:57) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:110) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:254) [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_71]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_71]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)