[jira] [Updated] (HIVE-2249) When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double

["Zhiqiu Kong (Updated) (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Zhiqiu Kong updated HIVE-2249:
------------------------------

    Attachment: HIVE-2249.2.patch.txt

This patch adds support to smartly infer the constant's type when encountering the query like "column CMP constant", where CMP could be any of the comparators supported by Hive. This aims to improving the performance by moving the type conversion from runtime stage to compiling stage.

To be more detailed, the smart type inference will happen when the type of the column is on e of the following:

* TINYINT
* SMALLINT
* INT
* BIGINT
* FLOAT
* DOUBLE

If the type of the columns fits any of the above, the constant on the other hand side will be converted firstly to the column's type. When failing, the constant will then be converted to DOUBLE. If both tries fail, the constant will be left as what type it is.

One exception is when the column is STRING while the constant is BIGINT. In this case, we do nothing. Otherwise, the constant will be converted to DOUBLE.

Other improvements include returning false immediately for the query like "int_col = not_convertable_double_constant", such as "uid = 1.5".


NOTE:

~130 unit test cases need to be updated due to this diff. All updates are limited to convert to the plan like "col = 10" to "col = 10.0", and are carefully checked individually. 

TWO test cases failed during the unit testing:

* testCliDriver_insert2_overwrite_partitions
* testCliDriver_ppr_pushdown

When looking into the query as well as the output, the plans generated were found to be the same while the query results changed. As the queries in these two cases are simple select queries, maybe the default sorting criteria was changed unintentionally by this diff or other diffs. 


                
> When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-2249
>                 URL: https://issues.apache.org/jira/browse/HIVE-2249
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Siying Dong
>            Assignee: Joseph Barillari
>         Attachments: HIVE-2249.1.patch.txt, HIVE-2249.2.patch.txt
>
>
> The current code to build constant expression for numbers, here is the code:
>      try {
>         v = Double.valueOf(expr.getText());
>         v = Long.valueOf(expr.getText());
>         v = Integer.valueOf(expr.getText());
>       } catch (NumberFormatException e) {
>         // do nothing here, we will throw an exception in the following block
>       }
>       if (v == null) {
>         throw new SemanticException(ErrorMsg.INVALID_NUMERICAL_CONSTANT
>             .getMsg(expr));
>       }
>       return new ExprNodeConstantDesc(v);
> The for the case that "WHERE <BIG_INT_COLUMN> = 0", or "WHERE <DOUBLE_COLUMN> = 0", we always have to do a type conversion when comparing, which is unnecessary if it is slightly smarter to choose type when creating the constant expression. We can simply walk one level up the tree, find another comparison party and use the same type with that one if it is possible. For user's wrong query like '<INT_COLUMN>=1.1', we can even do more.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira