You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by "pengfei.zhan (Jira)" <ji...@apache.org> on 2024/03/25 13:27:00 UTC

[jira] [Comment Edited] (KYLIN-5756) Concat string and number with '+' gives unexpected result

    [ https://issues.apache.org/jira/browse/KYLIN-5756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17830479#comment-17830479 ] 

pengfei.zhan edited comment on KYLIN-5756 at 3/25/24 1:26 PM:
--------------------------------------------------------------

h1. Fix design

# When generating java code in calcite, to determine whether the + operator needs to be converted to plus() or whether to directly use addition in java, add a condition that when the parameters on both sides of the plus operator are of string type or numeric type, plus() is used directly. i.e. fix the '1' + 3 + '3' in the previous sql, the java code is plus('1' + 3) + '3'.
After the fix it is plus(plus('1' + 3), '3')
# Changed the return value of the custom plus function in calcite from Double to bigDecimal. The reasons are as follows:
  * Constant folding by calcite to generate java code derives whether it is isNullable, if the RexCall with its nullable is false, it will automatic unboxing;
  * The expression 'a' + 3 is considered to be non-nullable  by calcite, because the two arguments are constants and neither of them is null, so the whole thing is considered non-null;
  * In spark, 'a' + 3 results in null, so our implementation of the plus method gives the same result
  * In summary, when expression `plus(string, number)` returns Double, the java code for `'a' + 3 + '3' ` is actually `plus(plus('a' + 3).doubleValue(), '3')`, which then throws an NPE when the calculation is performed.

This behavior change of isNullable involves a wide range, and it is dangerous to modify it directly, so we changed the return value of plus operator to BigDecimal to avoid unboxing.


was (Author: JIRAUSER294653):
h1. Fix design

# When generating java code in calcite, to determine whether the + operator needs to be converted to plus() or whether to directly use addition in java, add a condition that when the parameters on both sides of the plus operator are of string type or numeric type, plus() is used directly.

i.e. fix the '1' + 3 + '3' in the previous sql, the java code is plus('1' + 3) + '3'.
After the fix it is plus(plus('1' + 3), '3')

# Changed the return value of the custom plus function in calcite from Double to bigDecimal. The reasons are as follows:
* Constant folding by calcite to generate java code derives whether it is isNullable, if the RexCall with its nullable is false, it will automatic unboxing;
* The expression 'a' + 3 is considered to be non-nullable  by calcite, because the two arguments are constants and neither of them is null, so the whole thing is considered non-null;
* In spark, 'a' + 3 results in null, so our implementation of the plus method gives the same result
* In summary, when expression `plus(string, number)` returns Double, the java code for `'a' + 3 + '3' ` is actually `plus(plus('a' + 3).doubleValue(), '3')`, which then throws an NPE when the calculation is performed.

This behavior change of isNullable involves a wide range, and it is dangerous to modify it directly, so we changed the return value of plus operator to BigDecimal to avoid unboxing.

> Concat string and number with '+' gives unexpected result
> ---------------------------------------------------------
>
>                 Key: KYLIN-5756
>                 URL: https://issues.apache.org/jira/browse/KYLIN-5756
>             Project: Kylin
>          Issue Type: Improvement
>          Components: Query Engine
>    Affects Versions: 5.0-beta
>            Reporter: pengfei.zhan
>            Assignee: pengfei.zhan
>            Priority: Major
>             Fix For: 5.0-beta
>
>
> Concat string and number gives unexpected result.
> For example:
> 1. concat constant with plus operator, gives unexpected result
> {code:java}
> 1' + 3 + 3  // 7 (correct)
> '1' + 3 + '3'  // 4.03 (wrong result)
> '1' + '3' + 'a' // error
> {code}
> 2. concat string and number, the first '+' produces null
> {code:java}
> 'q' + 1 + 1 // error
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)