[jira] [Comment Edited] (DRILL-4175) calcite parse sql error

["Jinfeng Ni (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/DRILL-4175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15378845#comment-15378845 ] 

Jinfeng Ni edited comment on DRILL-4175 at 7/15/16 8:07 PM:
------------------------------------------------------------

Agreed that we should rebase Drill's fork onto the latest Calcite release. 

According to the list I complied [1], we have pushed majority of patches in Drill's fork to Calcite master branch, except for the last 4 patches. That's why I said we are not completely ready.

The painful thing about this IOBE is it happens intermittently under concurrent queries, and it could impact any query invoking a rule which uses RelProgramBuilder. That makes me inclined to port CALCITE-1009 ASAP, in stead of waiting for rebasing.   

[1] https://docs.google.com/spreadsheets/d/1Z0J5aMCBfqq_9SEl-LXsi_B8Ahaosygqke4cH6pPwmo/edit#gid=0


was (Author: jni):
Agreed that we should rebase Drill's fork onto the latest Calcite release. 

According to the list I complied [1], we have pushed majority of patches in Drill's fork to Calcite master branch, except for the last 4 patches. That's why I said we are not completely ready.

The painful thing about this IOBE is it happens intermittently under concurrent queries, and it could impact any query invoking a rule which uses RelProgramBuilder. That makes me inclined to port CALCITE-1099 ASAP, in stead of waiting for rebasing.   

[1] https://docs.google.com/spreadsheets/d/1Z0J5aMCBfqq_9SEl-LXsi_B8Ahaosygqke4cH6pPwmo/edit#gid=0

> calcite parse sql error
> -----------------------
>
>                 Key: DRILL-4175
>                 URL: https://issues.apache.org/jira/browse/DRILL-4175
>             Project: Apache Drill
>          Issue Type: Bug
>         Environment: distribution
>            Reporter: huntersjm
>
> I queryed a sql just like `selelct v from table limit 1`,I get a error:
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: IndexOutOfBoundsException: Index: 68, Size: 67
> After debug, I found there is a bug in calcite parse:
> first we look line 72 in org.apache.calcite.rex.RexProgramBuilder
> {noformat}
>    registerInternal(RexInputRef.of(i, fields), false);
> {noformat}
> there we get RexInputRef from RexInputRef.of, and it has a method named createName(int idex), here NAMES is SelfPopulatingList.class. SelfPopulatingList.class describe  as Thread-safe list, but in fact it is Thread-unsafe. when NAMES.get(index) is called distributed, it gets a error. We hope SelfPopulatingList.class to be {$0 $1 $2 ....$n}, but when it called distributed, it may be {$0,$1...$29,$30...$59,$30,$31...$59...}.
> We see method registerInternal
> {noformat}
> private RexLocalRef registerInternal(RexNode expr, boolean force) {
> expr = simplify(expr);
>     RexLocalRef ref;
>     final Pair<String, String> key;
>     if (expr instanceof RexLocalRef) {
>       key = null;
>       ref = (RexLocalRef) expr;
>     } else {
>       key = RexUtil.makeKey(expr);
>       ref = exprMap.get(key);
>     }
>     if (ref == null) {
>       if (validating) {
>         validate(
>             expr,
>             exprList.size());
>       }
> {noformat}
> Here makeKey(expr) hope to get different key, however it get same key, so addExpr(expr) called less, in this method
> {noformat}
> RexLocalRef ref;
>     final int index = exprList.size();
>     exprList.add(expr);
>     ref =
>         new RexLocalRef(
>             index,
>             expr.getType());
>     localRefList.add(ref);
>     return ref;
> {noformat}
> localRefList get error size, so in line 939,
> {noformat}
> final RexLocalRef ref = localRefList.get(index);
> {noformat}
> throw IndexOutOfBoundsException
> bugfix:
> We can't change origin code of calcite before they fix this bug, so we can init NAMEs in RexLocalRef on start. Just add 
> {noformat}
> RexInputRef.createName(2048);
> {noformat}
> on Bootstrap.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)