You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@drill.apache.org by "Jinfeng Ni (JIRA)" <ji...@apache.org> on 2016/07/19 00:11:20 UTC

[jira] [Commented] (DRILL-4175) IOBE may occur in Calcite RexProgramBuilder when queries are submitted concurrently

    [ https://issues.apache.org/jira/browse/DRILL-4175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15383346#comment-15383346 ] 

Jinfeng Ni commented on DRILL-4175:
-----------------------------------

My bad. The commit message should be "DRILL-4175", not "DRILL-4715".  I mistyped the JIRA number and did not notice it.



> IOBE may occur in Calcite RexProgramBuilder when queries are submitted concurrently
> -----------------------------------------------------------------------------------
>
>                 Key: DRILL-4175
>                 URL: https://issues.apache.org/jira/browse/DRILL-4175
>             Project: Apache Drill
>          Issue Type: Bug
>         Environment: distribution
>            Reporter: huntersjm
>             Fix For: 1.8.0
>
>
> I queryed a sql just like `selelct v from table limit 1`,I get a error:
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: IndexOutOfBoundsException: Index: 68, Size: 67
> After debug, I found there is a bug in calcite parse:
> first we look line 72 in org.apache.calcite.rex.RexProgramBuilder
> {noformat}
>    registerInternal(RexInputRef.of(i, fields), false);
> {noformat}
> there we get RexInputRef from RexInputRef.of, and it has a method named createName(int idex), here NAMES is SelfPopulatingList.class. SelfPopulatingList.class describe  as Thread-safe list, but in fact it is Thread-unsafe. when NAMES.get(index) is called distributed, it gets a error. We hope SelfPopulatingList.class to be {$0 $1 $2 ....$n}, but when it called distributed, it may be {$0,$1...$29,$30...$59,$30,$31...$59...}.
> We see method registerInternal
> {noformat}
> private RexLocalRef registerInternal(RexNode expr, boolean force) {
> expr = simplify(expr);
>     RexLocalRef ref;
>     final Pair<String, String> key;
>     if (expr instanceof RexLocalRef) {
>       key = null;
>       ref = (RexLocalRef) expr;
>     } else {
>       key = RexUtil.makeKey(expr);
>       ref = exprMap.get(key);
>     }
>     if (ref == null) {
>       if (validating) {
>         validate(
>             expr,
>             exprList.size());
>       }
> {noformat}
> Here makeKey(expr) hope to get different key, however it get same key, so addExpr(expr) called less, in this method
> {noformat}
> RexLocalRef ref;
>     final int index = exprList.size();
>     exprList.add(expr);
>     ref =
>         new RexLocalRef(
>             index,
>             expr.getType());
>     localRefList.add(ref);
>     return ref;
> {noformat}
> localRefList get error size, so in line 939,
> {noformat}
> final RexLocalRef ref = localRefList.get(index);
> {noformat}
> throw IndexOutOfBoundsException
> bugfix:
> We can't change origin code of calcite before they fix this bug, so we can init NAMEs in RexLocalRef on start. Just add 
> {noformat}
> RexInputRef.createName(2048);
> {noformat}
> on Bootstrap.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)