You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@drill.apache.org by "Sudheesh Katkam (JIRA)" <ji...@apache.org> on 2015/04/10 19:12:12 UTC
[jira] [Comment Edited] (DRILL-2630) Merge join over inputs with
complex type hit run-time code compiler error
[ https://issues.apache.org/jira/browse/DRILL-2630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14489942#comment-14489942 ]
Sudheesh Katkam edited comment on DRILL-2630 at 4/10/15 5:11 PM:
-----------------------------------------------------------------
Snippet of code generated:
{code}
...
MapVector vv36;
NullableTinyIntVector vv39;
...
public void doSetup(FragmentContext context, JoinStatus status, VectorContainer outgoing)
throws SchemaChangeException
{
{
...
vv36 = ((MapVector) tmp38);
int[] fieldIds40 = new int[ 1 ] ;
fieldIds40 [ 0 ] = 3;
Object tmp41 = (outgoing).getValueAccessorById(NullableTinyIntVector.class, fieldIds40).getValueVector();
if (tmp41 == null) {
throw new SchemaChangeException("Failure while loading vector vv39 with id: TypedFieldId [fieldIds=[3], remainder=null].");
}
vv39 = ((NullableTinyIntVector) tmp41);
}
}
public void doCopyRight(int rightIndex, int outIndex)
throws SchemaChangeException
{
{
vv27 .copyFromSafe((rightIndex), (outIndex), vv24);
vv33 .copyFromSafe((rightIndex), (outIndex), vv30);
vv39 .copyFromSafe((rightIndex), (outIndex), vv36);
}
}
{code}
And this line in doCopyRight() throws the error (copy from MapVector into NullableTinyIntVector):
{code}
vv39.copyFromSafe((rightIndex), (outIndex), vv36);
{code}
I am guessing this has to do with how complex types are handled in "generate copyRight()" section of generateNewWorker() function in MergeJoinBatch.java
was (Author: sudheeshkatkam):
Snippets of code generated:
{code}
...
MapVector vv36;
NullableTinyIntVector vv39;
...
public void doSetup(FragmentContext context, JoinStatus status, VectorContainer outgoing)
throws SchemaChangeException
{
{
...
vv36 = ((MapVector) tmp38);
int[] fieldIds40 = new int[ 1 ] ;
fieldIds40 [ 0 ] = 3;
Object tmp41 = (outgoing).getValueAccessorById(NullableTinyIntVector.class, fieldIds40).getValueVector();
if (tmp41 == null) {
throw new SchemaChangeException("Failure while loading vector vv39 with id: TypedFieldId [fieldIds=[3], remainder=null].");
}
vv39 = ((NullableTinyIntVector) tmp41);
}
}
public void doCopyRight(int rightIndex, int outIndex)
throws SchemaChangeException
{
{
vv27 .copyFromSafe((rightIndex), (outIndex), vv24);
vv33 .copyFromSafe((rightIndex), (outIndex), vv30);
vv39 .copyFromSafe((rightIndex), (outIndex), vv36);
}
}
{code}
And this line in doCopyRight() throws the error (copy from MapVector into NullableTinyIntVector):
{code}
vv39.copyFromSafe((rightIndex), (outIndex), vv36);
{code}
I am guessing this has to do with how complex types are handled in "generate copyRight()" section of generateNewWorker() function in MergeJoinBatch.java
> Merge join over inputs with complex type hit run-time code compiler error
> -------------------------------------------------------------------------
>
> Key: DRILL-2630
> URL: https://issues.apache.org/jira/browse/DRILL-2630
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Relational Operators
> Affects Versions: 0.8.0
> Reporter: Jinfeng Ni
> Assignee: Jason Altekruse
> Fix For: 1.0.0
>
> Attachments: complex_1.json
>
>
> Hit run-time code complier error, if we have a merge join whose inputs contain complex type.
> {code}
> select * from sys.version;
> +------------+----------------+-------------+-------------+------------+
> | commit_id | commit_message | commit_time | build_email | build_time |
> +------------+----------------+-------------+-------------+------------+
> | 0fbcddba14405ec94d51b0ba3512925168efb433 | DRILL-2375: implement reader reset mechanism and reset reader before accessing it during projection | 30.03.2015 @ 10:27:02 PDT | jni@maprtech.com | 30.03.2015 @ 16:50:01 PDT |
> +------------+----------------+-------------+-------------+------------+
> {code}
> {code}
> alter session set `planner.enable_hashjoin` = false;
> {code}
> {code}
> select a.id, b.oooi.oa.oab.oabc oabc, b.ooof.oa.oab oab from dfs.`/tmp/complex_1.json` a left outer join cp.`/tmp/complex_1.json` b on a.id=b.id order by a.id;
> {code}
> {code}
> +------------+------------+------------+
> | id | oabc | oab |
> +------------+------------+------------+
> Query failed: Query stopped., Line 49, Column 32: No applicable constructor/method found for actual parameters "int, int, org.apache.drill.exec.vector.complex.MapVector"; candidates are: "public void org.apache.drill.exec.vector.NullableTinyIntVector.copyFromSafe(int, int, org.apache.drill.exec.vector.NullableTinyIntVector)", "public void org.apache.drill.exec.vector.NullableTinyIntVector.copyFromSafe(int, int, org.apache.drill.exec.vector.TinyIntVector)" [ e5905a74-98d0-46d4-8090-bcf0cc710e8a on 10.250.0.8:31010 ]
> {code}
> If I switch to hash join, then, the query works fine. Therefore, looks like Merge Join operator has some bug in handling complex type.
> {code}
> alter session set `planner.enable_hashjoin` = true;
> +------------+------------+
> | ok | summary |
> +------------+------------+
> | true | planner.enable_hashjoin updated. |
> +------------+------------+
> 1 row selected (0.058 seconds)
> 0: jdbc:drill:zk=local> select a.id, b.oooi.oa.oab.oabc oabc, b.ooof.oa.oab oab from dfs.`/tmp/complex_1.json` a left outer join dfs.`/tmp/complex_1.json` b on a.id=b.id order by a.id;
> +------------+------------+------------+
> | id | oabc | oab |
> +------------+------------+------------+
> | 1 | 1 | {"oabc":1.5678} |
> | 2 | 2 | {"oabc":2.5678} |
> +------------+------------+------------+
> 2 rows selected (0.73 seconds)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)