You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Siying Dong (JIRA)" <ji...@apache.org> on 2011/07/01 01:45:30 UTC
[jira] [Created] (HIVE-2249) When creating constant expression for
numbers, try to infer type from another comparison operand, instead of
trying to use integer first, and then long and double
When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double
------------------------------------------------------------------------------------------------------------------------------------------------------------------
Key: HIVE-2249
URL: https://issues.apache.org/jira/browse/HIVE-2249
Project: Hive
Issue Type: Improvement
Reporter: Siying Dong
The current code to build constant expression for numbers, here is the code:
try {
v = Double.valueOf(expr.getText());
v = Long.valueOf(expr.getText());
v = Integer.valueOf(expr.getText());
} catch (NumberFormatException e) {
// do nothing here, we will throw an exception in the following block
}
if (v == null) {
throw new SemanticException(ErrorMsg.INVALID_NUMERICAL_CONSTANT
.getMsg(expr));
}
return new ExprNodeConstantDesc(v);
The for the case that "WHERE <BIG_INT_COLUMN> = 0", or "WHERE <DOUBLE_COLUMN> = 0", we always have to do a type conversion when comparing, which is unnecessary if it is slightly smarter to choose type when creating the constant expression. We can simply walk one level up the tree, find another comparison party and use the same type with that one if it is possible. For user's wrong query like '<INT_COLUMN>=1.1', we can even do more.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-2249) When creating constant expression for
numbers, try to infer type from another comparison operand, instead of
trying to use integer first, and then long and double
Posted by "Siying Dong (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siying Dong reassigned HIVE-2249:
---------------------------------
Assignee: Joseph Barillari
> When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2249
> URL: https://issues.apache.org/jira/browse/HIVE-2249
> Project: Hive
> Issue Type: Improvement
> Reporter: Siying Dong
> Assignee: Joseph Barillari
> Attachments: HIVE-2249.1.patch.txt
>
>
> The current code to build constant expression for numbers, here is the code:
> try {
> v = Double.valueOf(expr.getText());
> v = Long.valueOf(expr.getText());
> v = Integer.valueOf(expr.getText());
> } catch (NumberFormatException e) {
> // do nothing here, we will throw an exception in the following block
> }
> if (v == null) {
> throw new SemanticException(ErrorMsg.INVALID_NUMERICAL_CONSTANT
> .getMsg(expr));
> }
> return new ExprNodeConstantDesc(v);
> The for the case that "WHERE <BIG_INT_COLUMN> = 0", or "WHERE <DOUBLE_COLUMN> = 0", we always have to do a type conversion when comparing, which is unnecessary if it is slightly smarter to choose type when creating the constant expression. We can simply walk one level up the tree, find another comparison party and use the same type with that one if it is possible. For user's wrong query like '<INT_COLUMN>=1.1', we can even do more.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2249) When creating constant expression
for numbers, try to infer type from another comparison operand, instead of
trying to use integer first, and then long and double
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13194986#comment-13194986 ]
Phabricator commented on HIVE-2249:
-----------------------------------
stuart983 has commented on the revision "HIVE-2249 [jira] When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double".
INLINE COMMENTS
ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java:735 It's not what I mean. Instead of translating the long value to double, you should keep it as long, and leave the next step to deal with it, either showing a warning or failing the query.
REVISION DETAIL
https://reviews.facebook.net/D1383
> When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2249
> URL: https://issues.apache.org/jira/browse/HIVE-2249
> Project: Hive
> Issue Type: Improvement
> Reporter: Siying Dong
> Assignee: Joseph Barillari
> Attachments: HIVE-2249.1.patch.txt, HIVE-2249.2.patch.txt, HIVE-2249.D1383.1.patch, HIVE-2249.D1383.2.patch, HIVE-2249.D1383.3.patch
>
>
> The current code to build constant expression for numbers, here is the code:
> try {
> v = Double.valueOf(expr.getText());
> v = Long.valueOf(expr.getText());
> v = Integer.valueOf(expr.getText());
> } catch (NumberFormatException e) {
> // do nothing here, we will throw an exception in the following block
> }
> if (v == null) {
> throw new SemanticException(ErrorMsg.INVALID_NUMERICAL_CONSTANT
> .getMsg(expr));
> }
> return new ExprNodeConstantDesc(v);
> The for the case that "WHERE <BIG_INT_COLUMN> = 0", or "WHERE <DOUBLE_COLUMN> = 0", we always have to do a type conversion when comparing, which is unnecessary if it is slightly smarter to choose type when creating the constant expression. We can simply walk one level up the tree, find another comparison party and use the same type with that one if it is possible. For user's wrong query like '<INT_COLUMN>=1.1', we can even do more.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2249) When creating constant expression
for numbers, try to infer type from another comparison operand, instead of
trying to use integer first, and then long and double
Posted by "Siying Dong (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13196740#comment-13196740 ]
Siying Dong commented on HIVE-2249:
-----------------------------------
committed. Thanks Zhiqiu!
> When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2249
> URL: https://issues.apache.org/jira/browse/HIVE-2249
> Project: Hive
> Issue Type: Improvement
> Reporter: Siying Dong
> Assignee: Zhiqiu Kong
> Attachments: HIVE-2249.1.patch.txt, HIVE-2249.2.patch.txt, HIVE-2249.D1383.1.patch, HIVE-2249.D1383.2.patch, HIVE-2249.D1383.3.patch, HIVE-2249.D1383.4.patch
>
>
> The current code to build constant expression for numbers, here is the code:
> try {
> v = Double.valueOf(expr.getText());
> v = Long.valueOf(expr.getText());
> v = Integer.valueOf(expr.getText());
> } catch (NumberFormatException e) {
> // do nothing here, we will throw an exception in the following block
> }
> if (v == null) {
> throw new SemanticException(ErrorMsg.INVALID_NUMERICAL_CONSTANT
> .getMsg(expr));
> }
> return new ExprNodeConstantDesc(v);
> The for the case that "WHERE <BIG_INT_COLUMN> = 0", or "WHERE <DOUBLE_COLUMN> = 0", we always have to do a type conversion when comparing, which is unnecessary if it is slightly smarter to choose type when creating the constant expression. We can simply walk one level up the tree, find another comparison party and use the same type with that one if it is possible. For user's wrong query like '<INT_COLUMN>=1.1', we can even do more.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2249) When creating constant expression
for numbers, try to infer type from another comparison operand, instead of
trying to use integer first, and then long and double
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13194488#comment-13194488 ]
Phabricator commented on HIVE-2249:
-----------------------------------
stuart983 has commented on the revision "HIVE-2249 [jira] When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double".
INLINE COMMENTS
ql/src/test/queries/clientpositive/infer_const_type.q:1 I think it's better to drop the table first.
ql/src/test/queries/clientpositive/infer_const_type.q:34 It's better to not only explain it, but also execute it and see the results are right.
REVISION DETAIL
https://reviews.facebook.net/D1383
> When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2249
> URL: https://issues.apache.org/jira/browse/HIVE-2249
> Project: Hive
> Issue Type: Improvement
> Reporter: Siying Dong
> Assignee: Joseph Barillari
> Attachments: HIVE-2249.1.patch.txt, HIVE-2249.2.patch.txt, HIVE-2249.D1383.1.patch, HIVE-2249.D1383.2.patch
>
>
> The current code to build constant expression for numbers, here is the code:
> try {
> v = Double.valueOf(expr.getText());
> v = Long.valueOf(expr.getText());
> v = Integer.valueOf(expr.getText());
> } catch (NumberFormatException e) {
> // do nothing here, we will throw an exception in the following block
> }
> if (v == null) {
> throw new SemanticException(ErrorMsg.INVALID_NUMERICAL_CONSTANT
> .getMsg(expr));
> }
> return new ExprNodeConstantDesc(v);
> The for the case that "WHERE <BIG_INT_COLUMN> = 0", or "WHERE <DOUBLE_COLUMN> = 0", we always have to do a type conversion when comparing, which is unnecessary if it is slightly smarter to choose type when creating the constant expression. We can simply walk one level up the tree, find another comparison party and use the same type with that one if it is possible. For user's wrong query like '<INT_COLUMN>=1.1', we can even do more.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2249) When creating constant expression
for numbers, try to infer type from another comparison operand, instead of
trying to use integer first, and then long and double
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13194485#comment-13194485 ]
Phabricator commented on HIVE-2249:
-----------------------------------
stuart983 has commented on the revision "HIVE-2249 [jira] When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double".
INLINE COMMENTS
ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java:739 Instead of passing false, I think we should leave the operation tree as what it is to keep absolute compatible.
REVISION DETAIL
https://reviews.facebook.net/D1383
> When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2249
> URL: https://issues.apache.org/jira/browse/HIVE-2249
> Project: Hive
> Issue Type: Improvement
> Reporter: Siying Dong
> Assignee: Joseph Barillari
> Attachments: HIVE-2249.1.patch.txt, HIVE-2249.2.patch.txt, HIVE-2249.D1383.1.patch, HIVE-2249.D1383.2.patch
>
>
> The current code to build constant expression for numbers, here is the code:
> try {
> v = Double.valueOf(expr.getText());
> v = Long.valueOf(expr.getText());
> v = Integer.valueOf(expr.getText());
> } catch (NumberFormatException e) {
> // do nothing here, we will throw an exception in the following block
> }
> if (v == null) {
> throw new SemanticException(ErrorMsg.INVALID_NUMERICAL_CONSTANT
> .getMsg(expr));
> }
> return new ExprNodeConstantDesc(v);
> The for the case that "WHERE <BIG_INT_COLUMN> = 0", or "WHERE <DOUBLE_COLUMN> = 0", we always have to do a type conversion when comparing, which is unnecessary if it is slightly smarter to choose type when creating the constant expression. We can simply walk one level up the tree, find another comparison party and use the same type with that one if it is possible. For user's wrong query like '<INT_COLUMN>=1.1', we can even do more.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2249) When creating constant expression
for numbers, try to infer type from another comparison operand, instead of
trying to use integer first, and then long and double
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13193387#comment-13193387 ]
Phabricator commented on HIVE-2249:
-----------------------------------
zhiqiu has planned changes to the revision "HIVE-2249 [jira] When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double".
* Remove unused lines (the set doubleTypes)
* Fix the test cases by adding "sort by" clause to the query: 1. insert1_overwrite_partitions.q 2. insert2_overwrite_partitions.q 3. ppr_pushdown
* Add test case for this new change on smart type inference
REVISION DETAIL
https://reviews.facebook.net/D1383
> When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2249
> URL: https://issues.apache.org/jira/browse/HIVE-2249
> Project: Hive
> Issue Type: Improvement
> Reporter: Siying Dong
> Assignee: Joseph Barillari
> Attachments: HIVE-2249.1.patch.txt, HIVE-2249.2.patch.txt, HIVE-2249.D1383.1.patch
>
>
> The current code to build constant expression for numbers, here is the code:
> try {
> v = Double.valueOf(expr.getText());
> v = Long.valueOf(expr.getText());
> v = Integer.valueOf(expr.getText());
> } catch (NumberFormatException e) {
> // do nothing here, we will throw an exception in the following block
> }
> if (v == null) {
> throw new SemanticException(ErrorMsg.INVALID_NUMERICAL_CONSTANT
> .getMsg(expr));
> }
> return new ExprNodeConstantDesc(v);
> The for the case that "WHERE <BIG_INT_COLUMN> = 0", or "WHERE <DOUBLE_COLUMN> = 0", we always have to do a type conversion when comparing, which is unnecessary if it is slightly smarter to choose type when creating the constant expression. We can simply walk one level up the tree, find another comparison party and use the same type with that one if it is possible. For user's wrong query like '<INT_COLUMN>=1.1', we can even do more.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2249) When creating constant expression
for numbers, try to infer type from another comparison operand, instead of
trying to use integer first, and then long and double
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13196844#comment-13196844 ]
Hudson commented on HIVE-2249:
------------------------------
Integrated in Hive-trunk-h0.21 #1233 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1233/])
HIVE-2249 When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double (Zhiqiu Kong via Siying Dong)
sdong : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1238175
Files :
* /hive/trunk/contrib/src/test/results/clientpositive/dboutput.q.out
* /hive/trunk/contrib/src/test/results/clientpositive/serde_typedbytes4.q.out
* /hive/trunk/data/files/infer_const_type.txt
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java
* /hive/trunk/ql/src/test/queries/clientpositive/infer_const_type.q
* /hive/trunk/ql/src/test/queries/clientpositive/insert1_overwrite_partitions.q
* /hive/trunk/ql/src/test/queries/clientpositive/insert2_overwrite_partitions.q
* /hive/trunk/ql/src/test/queries/clientpositive/ppr_pushdown.q
* /hive/trunk/ql/src/test/results/clientpositive/auto_join0.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_join11.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_join12.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_join13.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_join14.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_join16.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_join20.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_join21.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_join23.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_join27.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_join28.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_join29.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_join4.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_join5.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_join6.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_join7.q.out
* /hive/trunk/ql/src/test/results/clientpositive/auto_join8.q.out
* /hive/trunk/ql/src/test/results/clientpositive/cast1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/cluster.q.out
* /hive/trunk/ql/src/test/results/clientpositive/create_view.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby_multi_single_reducer.q.out
* /hive/trunk/ql/src/test/results/clientpositive/having.q.out
* /hive/trunk/ql/src/test/results/clientpositive/index_auto.q.out
* /hive/trunk/ql/src/test/results/clientpositive/index_auto_empty.q.out
* /hive/trunk/ql/src/test/results/clientpositive/index_auto_file_format.q.out
* /hive/trunk/ql/src/test/results/clientpositive/index_auto_mult_tables.q.out
* /hive/trunk/ql/src/test/results/clientpositive/index_auto_mult_tables_compact.q.out
* /hive/trunk/ql/src/test/results/clientpositive/index_auto_multiple.q.out
* /hive/trunk/ql/src/test/results/clientpositive/index_auto_partitioned.q.out
* /hive/trunk/ql/src/test/results/clientpositive/index_auto_self_join.q.out
* /hive/trunk/ql/src/test/results/clientpositive/index_auto_unused.q.out
* /hive/trunk/ql/src/test/results/clientpositive/index_auto_update.q.out
* /hive/trunk/ql/src/test/results/clientpositive/index_bitmap3.q.out
* /hive/trunk/ql/src/test/results/clientpositive/index_bitmap_auto.q.out
* /hive/trunk/ql/src/test/results/clientpositive/index_bitmap_auto_partitioned.q.out
* /hive/trunk/ql/src/test/results/clientpositive/index_bitmap_compression.q.out
* /hive/trunk/ql/src/test/results/clientpositive/index_compression.q.out
* /hive/trunk/ql/src/test/results/clientpositive/index_stale.q.out
* /hive/trunk/ql/src/test/results/clientpositive/index_stale_partitioned.q.out
* /hive/trunk/ql/src/test/results/clientpositive/infer_const_type.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input11.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input11_limit.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input12.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input13.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input14.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input14_limit.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input18.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input1_limit.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input2_limit.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input42.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input_part1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input_part2.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input_part5.q.out
* /hive/trunk/ql/src/test/results/clientpositive/input_part7.q.out
* /hive/trunk/ql/src/test/results/clientpositive/insert1_overwrite_partitions.q.out
* /hive/trunk/ql/src/test/results/clientpositive/insert2_overwrite_partitions.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join0.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join11.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join12.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join13.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join14.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join16.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join20.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join21.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join23.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join34.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join35.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join38.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join39.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join4.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join40.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join5.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join6.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join7.q.out
* /hive/trunk/ql/src/test/results/clientpositive/join8.q.out
* /hive/trunk/ql/src/test/results/clientpositive/load_dyn_part13.q.out
* /hive/trunk/ql/src/test/results/clientpositive/louter_join_ppr.q.out
* /hive/trunk/ql/src/test/results/clientpositive/multi_insert.q.out
* /hive/trunk/ql/src/test/results/clientpositive/no_hooks.q.out
* /hive/trunk/ql/src/test/results/clientpositive/noalias_subq1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/notable_alias1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/notable_alias2.q.out
* /hive/trunk/ql/src/test/results/clientpositive/nullgroup.q.out
* /hive/trunk/ql/src/test/results/clientpositive/nullgroup2.q.out
* /hive/trunk/ql/src/test/results/clientpositive/nullgroup4.q.out
* /hive/trunk/ql/src/test/results/clientpositive/nullgroup4_multi_distinct.q.out
* /hive/trunk/ql/src/test/results/clientpositive/order2.q.out
* /hive/trunk/ql/src/test/results/clientpositive/outer_join_ppr.q.out
* /hive/trunk/ql/src/test/results/clientpositive/pcr.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ppd_clusterby.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ppd_multi_insert.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ppd_outer_join1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ppd_transform.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ppd_udf_col.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ppr_pushdown.q.out
* /hive/trunk/ql/src/test/results/clientpositive/ppr_pushdown3.q.out
* /hive/trunk/ql/src/test/results/clientpositive/quote1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/rand_partitionpruner3.q.out
* /hive/trunk/ql/src/test/results/clientpositive/rcfile_null_value.q.out
* /hive/trunk/ql/src/test/results/clientpositive/regex_col.q.out
* /hive/trunk/ql/src/test/results/clientpositive/regexp_extract.q.out
* /hive/trunk/ql/src/test/results/clientpositive/router_join_ppr.q.out
* /hive/trunk/ql/src/test/results/clientpositive/semijoin.q.out
* /hive/trunk/ql/src/test/results/clientpositive/set_processor_namespaces.q.out
* /hive/trunk/ql/src/test/results/clientpositive/skewjoin.q.out
* /hive/trunk/ql/src/test/results/clientpositive/subq.q.out
* /hive/trunk/ql/src/test/results/clientpositive/subq2.q.out
* /hive/trunk/ql/src/test/results/clientpositive/transform_ppr1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/transform_ppr2.q.out
* /hive/trunk/ql/src/test/results/clientpositive/udf1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/udf9.q.out
* /hive/trunk/ql/src/test/results/clientpositive/udf_10_trims.q.out
* /hive/trunk/ql/src/test/results/clientpositive/udf_hour.q.out
* /hive/trunk/ql/src/test/results/clientpositive/udf_like.q.out
* /hive/trunk/ql/src/test/results/clientpositive/udf_lower.q.out
* /hive/trunk/ql/src/test/results/clientpositive/udf_minute.q.out
* /hive/trunk/ql/src/test/results/clientpositive/udf_parse_url.q.out
* /hive/trunk/ql/src/test/results/clientpositive/udf_second.q.out
* /hive/trunk/ql/src/test/results/clientpositive/udf_union.q.out
* /hive/trunk/ql/src/test/results/clientpositive/union.q.out
* /hive/trunk/ql/src/test/results/clientpositive/union20.q.out
* /hive/trunk/ql/src/test/results/clientpositive/union22.q.out
* /hive/trunk/ql/src/test/results/clientpositive/union24.q.out
* /hive/trunk/ql/src/test/results/clientpositive/union_ppr.q.out
* /hive/trunk/ql/src/test/results/compiler/plan/cast1.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/input1.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/input2.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/input3.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/input4.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/input_part1.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/join4.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/join5.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/join6.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/join7.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/join8.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/subq.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/udf1.q.xml
* /hive/trunk/ql/src/test/results/compiler/plan/union.q.xml
> When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2249
> URL: https://issues.apache.org/jira/browse/HIVE-2249
> Project: Hive
> Issue Type: Improvement
> Reporter: Siying Dong
> Assignee: Zhiqiu Kong
> Attachments: HIVE-2249.1.patch.txt, HIVE-2249.2.patch.txt, HIVE-2249.D1383.1.patch, HIVE-2249.D1383.2.patch, HIVE-2249.D1383.3.patch, HIVE-2249.D1383.4.patch
>
>
> The current code to build constant expression for numbers, here is the code:
> try {
> v = Double.valueOf(expr.getText());
> v = Long.valueOf(expr.getText());
> v = Integer.valueOf(expr.getText());
> } catch (NumberFormatException e) {
> // do nothing here, we will throw an exception in the following block
> }
> if (v == null) {
> throw new SemanticException(ErrorMsg.INVALID_NUMERICAL_CONSTANT
> .getMsg(expr));
> }
> return new ExprNodeConstantDesc(v);
> The for the case that "WHERE <BIG_INT_COLUMN> = 0", or "WHERE <DOUBLE_COLUMN> = 0", we always have to do a type conversion when comparing, which is unnecessary if it is slightly smarter to choose type when creating the constant expression. We can simply walk one level up the tree, find another comparison party and use the same type with that one if it is possible. For user's wrong query like '<INT_COLUMN>=1.1', we can even do more.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2249) When creating constant expression for
numbers, try to infer type from another comparison operand, instead of
trying to use integer first, and then long and double
Posted by "Joseph Barillari (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joseph Barillari updated HIVE-2249:
-----------------------------------
Attachment: HIVE-2249.1.patch.txt
As explained in the comments in TypeCheckProcFactory, this patch implements this fix. It also optimizes impossible comparisons (int column and double constant) to FALSE.
It also includes a new unit test, type_matching.q. Note that this test does (and should) produce slightly different output between the trunk version and the patched version, since it runs EXPLAIN SELECT on several queries that this patch will modify, so the string outputs (e.g., "22" vs "22.0" are different.)
> When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2249
> URL: https://issues.apache.org/jira/browse/HIVE-2249
> Project: Hive
> Issue Type: Improvement
> Reporter: Siying Dong
> Attachments: HIVE-2249.1.patch.txt
>
>
> The current code to build constant expression for numbers, here is the code:
> try {
> v = Double.valueOf(expr.getText());
> v = Long.valueOf(expr.getText());
> v = Integer.valueOf(expr.getText());
> } catch (NumberFormatException e) {
> // do nothing here, we will throw an exception in the following block
> }
> if (v == null) {
> throw new SemanticException(ErrorMsg.INVALID_NUMERICAL_CONSTANT
> .getMsg(expr));
> }
> return new ExprNodeConstantDesc(v);
> The for the case that "WHERE <BIG_INT_COLUMN> = 0", or "WHERE <DOUBLE_COLUMN> = 0", we always have to do a type conversion when comparing, which is unnecessary if it is slightly smarter to choose type when creating the constant expression. We can simply walk one level up the tree, find another comparison party and use the same type with that one if it is possible. For user's wrong query like '<INT_COLUMN>=1.1', we can even do more.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2249) When creating constant expression for
numbers, try to infer type from another comparison operand, instead of
trying to use integer first, and then long and double
Posted by "Ashutosh Chauhan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Chauhan updated HIVE-2249:
-----------------------------------
Fix Version/s: 0.9.0
> When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2249
> URL: https://issues.apache.org/jira/browse/HIVE-2249
> Project: Hive
> Issue Type: Improvement
> Reporter: Siying Dong
> Assignee: Zhiqiu Kong
> Fix For: 0.9.0
>
> Attachments: HIVE-2249.1.patch.txt, HIVE-2249.2.patch.txt, HIVE-2249.D1383.1.patch, HIVE-2249.D1383.2.patch, HIVE-2249.D1383.3.patch, HIVE-2249.D1383.4.patch
>
>
> The current code to build constant expression for numbers, here is the code:
> try {
> v = Double.valueOf(expr.getText());
> v = Long.valueOf(expr.getText());
> v = Integer.valueOf(expr.getText());
> } catch (NumberFormatException e) {
> // do nothing here, we will throw an exception in the following block
> }
> if (v == null) {
> throw new SemanticException(ErrorMsg.INVALID_NUMERICAL_CONSTANT
> .getMsg(expr));
> }
> return new ExprNodeConstantDesc(v);
> The for the case that "WHERE <BIG_INT_COLUMN> = 0", or "WHERE <DOUBLE_COLUMN> = 0", we always have to do a type conversion when comparing, which is unnecessary if it is slightly smarter to choose type when creating the constant expression. We can simply walk one level up the tree, find another comparison party and use the same type with that one if it is possible. For user's wrong query like '<INT_COLUMN>=1.1', we can even do more.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HIVE-2249) When creating constant expression for
numbers, try to infer type from another comparison operand, instead of
trying to use integer first, and then long and double
Posted by "Siying Dong (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siying Dong resolved HIVE-2249.
-------------------------------
Resolution: Fixed
> When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2249
> URL: https://issues.apache.org/jira/browse/HIVE-2249
> Project: Hive
> Issue Type: Improvement
> Reporter: Siying Dong
> Assignee: Zhiqiu Kong
> Attachments: HIVE-2249.1.patch.txt, HIVE-2249.2.patch.txt, HIVE-2249.D1383.1.patch, HIVE-2249.D1383.2.patch, HIVE-2249.D1383.3.patch, HIVE-2249.D1383.4.patch
>
>
> The current code to build constant expression for numbers, here is the code:
> try {
> v = Double.valueOf(expr.getText());
> v = Long.valueOf(expr.getText());
> v = Integer.valueOf(expr.getText());
> } catch (NumberFormatException e) {
> // do nothing here, we will throw an exception in the following block
> }
> if (v == null) {
> throw new SemanticException(ErrorMsg.INVALID_NUMERICAL_CONSTANT
> .getMsg(expr));
> }
> return new ExprNodeConstantDesc(v);
> The for the case that "WHERE <BIG_INT_COLUMN> = 0", or "WHERE <DOUBLE_COLUMN> = 0", we always have to do a type conversion when comparing, which is unnecessary if it is slightly smarter to choose type when creating the constant expression. We can simply walk one level up the tree, find another comparison party and use the same type with that one if it is possible. For user's wrong query like '<INT_COLUMN>=1.1', we can even do more.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2249) When creating constant expression
for numbers, try to infer type from another comparison operand, instead of
trying to use integer first, and then long and double
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13193245#comment-13193245 ]
Phabricator commented on HIVE-2249:
-----------------------------------
zhiqiu has commented on the revision "HIVE-2249 [jira] When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double".
Thanks a lot, Kevin! I'll remove the un-used lines.
Just tried to update insert2_overwrite_partitions.q and insert1_overwrite_partitions.q to add "order by" clause.
For ppr_pushdown, I think you analyses also apply. It failed on my side b/c the query like this:
select * from ppr_test where ds = '1234'
The results on my machine are:
1234 1234
abcd 1234
while the "expected" outputs are:
abcd 1234
1234 1234
So I think it is good to also add "order by" to this case, right?
REVISION DETAIL
https://reviews.facebook.net/D1383
> When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2249
> URL: https://issues.apache.org/jira/browse/HIVE-2249
> Project: Hive
> Issue Type: Improvement
> Reporter: Siying Dong
> Assignee: Joseph Barillari
> Attachments: HIVE-2249.1.patch.txt, HIVE-2249.2.patch.txt, HIVE-2249.D1383.1.patch
>
>
> The current code to build constant expression for numbers, here is the code:
> try {
> v = Double.valueOf(expr.getText());
> v = Long.valueOf(expr.getText());
> v = Integer.valueOf(expr.getText());
> } catch (NumberFormatException e) {
> // do nothing here, we will throw an exception in the following block
> }
> if (v == null) {
> throw new SemanticException(ErrorMsg.INVALID_NUMERICAL_CONSTANT
> .getMsg(expr));
> }
> return new ExprNodeConstantDesc(v);
> The for the case that "WHERE <BIG_INT_COLUMN> = 0", or "WHERE <DOUBLE_COLUMN> = 0", we always have to do a type conversion when comparing, which is unnecessary if it is slightly smarter to choose type when creating the constant expression. We can simply walk one level up the tree, find another comparison party and use the same type with that one if it is possible. For user's wrong query like '<INT_COLUMN>=1.1', we can even do more.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2249) When creating constant expression for
numbers, try to infer type from another comparison operand, instead of
trying to use integer first, and then long and double
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2249:
------------------------------
Attachment: HIVE-2249.D1383.2.patch
zhiqiu updated the revision "HIVE-2249 [jira] When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double".
Reviewers: njain, kevinwilfong, heyongqiang, JIRA, stuart983
* Address Kevin's comments: fix the failed test cases (insert1_overwrite_partitions.q, 2. insert2_overwrite_partitions.q 3. ppr_pushdown)
* Add a unit test to verify the smart type inference works (infer_const_type.q)
* Remove un-used lines and fix a bug that str_col = long_const does not return false
REVISION DETAIL
https://reviews.facebook.net/D1383
AFFECTED FILES
contrib/src/test/results/clientpositive/dboutput.q.out
contrib/src/test/results/clientpositive/serde_typedbytes4.q.out
ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java
ql/src/test/queries/clientpositive/infer_const_type.q
ql/src/test/queries/clientpositive/insert1_overwrite_partitions.q
ql/src/test/queries/clientpositive/insert2_overwrite_partitions.q
ql/src/test/queries/clientpositive/ppr_pushdown.q
ql/src/test/results/clientpositive/auto_join0.q.out
ql/src/test/results/clientpositive/auto_join11.q.out
ql/src/test/results/clientpositive/auto_join12.q.out
ql/src/test/results/clientpositive/auto_join13.q.out
ql/src/test/results/clientpositive/auto_join14.q.out
ql/src/test/results/clientpositive/auto_join16.q.out
ql/src/test/results/clientpositive/auto_join20.q.out
ql/src/test/results/clientpositive/auto_join21.q.out
ql/src/test/results/clientpositive/auto_join23.q.out
ql/src/test/results/clientpositive/auto_join27.q.out
ql/src/test/results/clientpositive/auto_join28.q.out
ql/src/test/results/clientpositive/auto_join29.q.out
ql/src/test/results/clientpositive/auto_join4.q.out
ql/src/test/results/clientpositive/auto_join5.q.out
ql/src/test/results/clientpositive/auto_join6.q.out
ql/src/test/results/clientpositive/auto_join7.q.out
ql/src/test/results/clientpositive/auto_join8.q.out
ql/src/test/results/clientpositive/cast1.q.out
ql/src/test/results/clientpositive/cluster.q.out
ql/src/test/results/clientpositive/create_view.q.out
ql/src/test/results/clientpositive/groupby_multi_single_reducer.q.out
ql/src/test/results/clientpositive/having.q.out
ql/src/test/results/clientpositive/index_auto.q.out
ql/src/test/results/clientpositive/index_auto_empty.q.out
ql/src/test/results/clientpositive/index_auto_file_format.q.out
ql/src/test/results/clientpositive/index_auto_mult_tables.q.out
ql/src/test/results/clientpositive/index_auto_mult_tables_compact.q.out
ql/src/test/results/clientpositive/index_auto_multiple.q.out
ql/src/test/results/clientpositive/index_auto_partitioned.q.out
ql/src/test/results/clientpositive/index_auto_self_join.q.out
ql/src/test/results/clientpositive/index_auto_unused.q.out
ql/src/test/results/clientpositive/index_auto_update.q.out
ql/src/test/results/clientpositive/index_bitmap3.q.out
ql/src/test/results/clientpositive/index_bitmap_auto.q.out
ql/src/test/results/clientpositive/index_bitmap_auto_partitioned.q.out
ql/src/test/results/clientpositive/index_bitmap_compression.q.out
ql/src/test/results/clientpositive/index_compression.q.out
ql/src/test/results/clientpositive/index_stale.q.out
ql/src/test/results/clientpositive/index_stale_partitioned.q.out
ql/src/test/results/clientpositive/infer_const_type.q.out
ql/src/test/results/clientpositive/input11.q.out
ql/src/test/results/clientpositive/input11_limit.q.out
ql/src/test/results/clientpositive/input12.q.out
ql/src/test/results/clientpositive/input13.q.out
ql/src/test/results/clientpositive/input14.q.out
ql/src/test/results/clientpositive/input14_limit.q.out
ql/src/test/results/clientpositive/input18.q.out
ql/src/test/results/clientpositive/input1_limit.q.out
ql/src/test/results/clientpositive/input2_limit.q.out
ql/src/test/results/clientpositive/input42.q.out
ql/src/test/results/clientpositive/input_part1.q.out
ql/src/test/results/clientpositive/input_part2.q.out
ql/src/test/results/clientpositive/input_part5.q.out
ql/src/test/results/clientpositive/input_part7.q.out
ql/src/test/results/clientpositive/insert1_overwrite_partitions.q.out
ql/src/test/results/clientpositive/insert2_overwrite_partitions.q.out
ql/src/test/results/clientpositive/join0.q.out
ql/src/test/results/clientpositive/join11.q.out
ql/src/test/results/clientpositive/join12.q.out
ql/src/test/results/clientpositive/join13.q.out
ql/src/test/results/clientpositive/join14.q.out
ql/src/test/results/clientpositive/join16.q.out
ql/src/test/results/clientpositive/join20.q.out
ql/src/test/results/clientpositive/join21.q.out
ql/src/test/results/clientpositive/join23.q.out
ql/src/test/results/clientpositive/join34.q.out
ql/src/test/results/clientpositive/join35.q.out
ql/src/test/results/clientpositive/join38.q.out
ql/src/test/results/clientpositive/join39.q.out
ql/src/test/results/clientpositive/join4.q.out
ql/src/test/results/clientpositive/join40.q.out
ql/src/test/results/clientpositive/join5.q.out
ql/src/test/results/clientpositive/join6.q.out
ql/src/test/results/clientpositive/join7.q.out
ql/src/test/results/clientpositive/join8.q.out
ql/src/test/results/clientpositive/load_dyn_part13.q.out
ql/src/test/results/clientpositive/louter_join_ppr.q.out
ql/src/test/results/clientpositive/multi_insert.q.out
ql/src/test/results/clientpositive/no_hooks.q.out
ql/src/test/results/clientpositive/noalias_subq1.q.out
ql/src/test/results/clientpositive/notable_alias1.q.out
ql/src/test/results/clientpositive/notable_alias2.q.out
ql/src/test/results/clientpositive/nullgroup.q.out
ql/src/test/results/clientpositive/nullgroup2.q.out
ql/src/test/results/clientpositive/nullgroup4.q.out
ql/src/test/results/clientpositive/nullgroup4_multi_distinct.q.out
ql/src/test/results/clientpositive/order2.q.out
ql/src/test/results/clientpositive/outer_join_ppr.q.out
ql/src/test/results/clientpositive/pcr.q.out
ql/src/test/results/clientpositive/ppd_clusterby.q.out
ql/src/test/results/clientpositive/ppd_multi_insert.q.out
ql/src/test/results/clientpositive/ppd_outer_join1.q.out
ql/src/test/results/clientpositive/ppd_transform.q.out
ql/src/test/results/clientpositive/ppd_udf_col.q.out
ql/src/test/results/clientpositive/ppr_pushdown.q.out
ql/src/test/results/clientpositive/ppr_pushdown3.q.out
ql/src/test/results/clientpositive/quote1.q.out
ql/src/test/results/clientpositive/rand_partitionpruner3.q.out
ql/src/test/results/clientpositive/rcfile_null_value.q.out
ql/src/test/results/clientpositive/regex_col.q.out
ql/src/test/results/clientpositive/regexp_extract.q.out
ql/src/test/results/clientpositive/router_join_ppr.q.out
ql/src/test/results/clientpositive/semijoin.q.out
ql/src/test/results/clientpositive/set_processor_namespaces.q.out
ql/src/test/results/clientpositive/skewjoin.q.out
ql/src/test/results/clientpositive/subq.q.out
ql/src/test/results/clientpositive/subq2.q.out
ql/src/test/results/clientpositive/transform_ppr1.q.out
ql/src/test/results/clientpositive/transform_ppr2.q.out
ql/src/test/results/clientpositive/udf1.q.out
ql/src/test/results/clientpositive/udf9.q.out
ql/src/test/results/clientpositive/udf_10_trims.q.out
ql/src/test/results/clientpositive/udf_hour.q.out
ql/src/test/results/clientpositive/udf_like.q.out
ql/src/test/results/clientpositive/udf_lower.q.out
ql/src/test/results/clientpositive/udf_minute.q.out
ql/src/test/results/clientpositive/udf_parse_url.q.out
ql/src/test/results/clientpositive/udf_second.q.out
ql/src/test/results/clientpositive/udf_union.q.out
ql/src/test/results/clientpositive/union.q.out
ql/src/test/results/clientpositive/union20.q.out
ql/src/test/results/clientpositive/union22.q.out
ql/src/test/results/clientpositive/union24.q.out
ql/src/test/results/clientpositive/union_ppr.q.out
ql/src/test/results/compiler/plan/cast1.q.xml
ql/src/test/results/compiler/plan/input1.q.xml
ql/src/test/results/compiler/plan/input2.q.xml
ql/src/test/results/compiler/plan/input3.q.xml
ql/src/test/results/compiler/plan/input4.q.xml
ql/src/test/results/compiler/plan/input_part1.q.xml
ql/src/test/results/compiler/plan/join4.q.xml
ql/src/test/results/compiler/plan/join5.q.xml
ql/src/test/results/compiler/plan/join6.q.xml
ql/src/test/results/compiler/plan/join7.q.xml
ql/src/test/results/compiler/plan/join8.q.xml
ql/src/test/results/compiler/plan/subq.q.xml
ql/src/test/results/compiler/plan/udf1.q.xml
ql/src/test/results/compiler/plan/union.q.xml
> When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2249
> URL: https://issues.apache.org/jira/browse/HIVE-2249
> Project: Hive
> Issue Type: Improvement
> Reporter: Siying Dong
> Assignee: Joseph Barillari
> Attachments: HIVE-2249.1.patch.txt, HIVE-2249.2.patch.txt, HIVE-2249.D1383.1.patch, HIVE-2249.D1383.2.patch
>
>
> The current code to build constant expression for numbers, here is the code:
> try {
> v = Double.valueOf(expr.getText());
> v = Long.valueOf(expr.getText());
> v = Integer.valueOf(expr.getText());
> } catch (NumberFormatException e) {
> // do nothing here, we will throw an exception in the following block
> }
> if (v == null) {
> throw new SemanticException(ErrorMsg.INVALID_NUMERICAL_CONSTANT
> .getMsg(expr));
> }
> return new ExprNodeConstantDesc(v);
> The for the case that "WHERE <BIG_INT_COLUMN> = 0", or "WHERE <DOUBLE_COLUMN> = 0", we always have to do a type conversion when comparing, which is unnecessary if it is slightly smarter to choose type when creating the constant expression. We can simply walk one level up the tree, find another comparison party and use the same type with that one if it is possible. For user's wrong query like '<INT_COLUMN>=1.1', we can even do more.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2249) When creating constant expression for
numbers, try to infer type from another comparison operand, instead of
trying to use integer first, and then long and double
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2249:
------------------------------
Attachment: HIVE-2249.D1383.1.patch
zhiqiu requested code review of "HIVE-2249 [jira] When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double".
Reviewers: njain, kevinwilfong, heyongqiang, JIRA
[jira] [HIVE-2249] Smart type inference of constants
This patch adds support to smartly infer the constant's type when
encountering the query like "column CMP constant", where CMP could be
any of the comparators supported by Hive. This aims to improving the
performance by moving the type conversion from runtime stage to
compiling stage.
To be more detailed, the smart type inference will happen when the type
of the column is one of the followings:
* TINYINT
* SMALLINT
* INT
* BIGINT
* FLOAT
* DOUBLE
If the type of the columns fits any of the above, the constant on the
other hand side will be converted firstly to the column's type.
* TINYINT => Byte
* SMALLINT => Short
* INT => Integer
* BIGINT => Long
* FLOAT => Float
* DOUBLE => Double
If failing, the constant will then be converted to DOUBLE. If both tries
fail, the constant will be left as what type it is.
One exception is when the column is STRING while the constant is BIGINT.
In this case, we do nothing. Otherwise, the constant will be converted
to DOUBLE.
Other improvements include returning false immediately for the query
like "int_col = not_convertable_double_constant", such as "uid = 1.5".
NOTE:
~130 unit test cases need to be updated due to this diff. All updates
are limited to convert to the plan like "col = 10" to "col = 10.0", and
are carefully checked individually.
TWO test cases failed during the unit testing:
* testCliDriver_insert2_overwrite_partitions
* testCliDriver_ppr_pushdown
When looking into the query as well as the output, the plans generated
were found to be the same while the query results changed. As the
queries in these two cases are simple select queries, maybe the default
sorting criteria was changed unintentionally by this diff or other
diffs.
Task ID: #620808
Blame Rev:
The current code to build constant expression for numbers, here is the code:
try {
v = Double.valueOf(expr.getText());
v = Long.valueOf(expr.getText());
v = Integer.valueOf(expr.getText());
} catch (NumberFormatException e) {
// do nothing here, we will throw an exception in the following block
}
if (v == null) {
throw new SemanticException(ErrorMsg.INVALID_NUMERICAL_CONSTANT
.getMsg(expr));
}
return new ExprNodeConstantDesc(v);
The for the case that "WHERE <BIG_INT_COLUMN> = 0", or "WHERE <DOUBLE_COLUMN> = 0", we always have to do a type conversion when comparing, which is unnecessary if it is slightly smarter to choose type when creating the constant expression. We can simply walk one level up the tree, find another comparison party and use the same type with that one if it is possible. For user's wrong query like '<INT_COLUMN>=1.1', we can even do more.
TEST PLAN
Run unit tests.
TWO test cases failed during the unit testing:
* testCliDriver_insert2_overwrite_partitions
* testCliDriver_ppr_pushdown
When looking into the query as well as the output, the plans generated
were found to be the same while the query results changed. As the
queries in these two cases are simple select queries, maybe the default
sorting criteria was changed unintentionally by this diff or other
diffs.
Revert Plan:
Tags:
REVISION DETAIL
https://reviews.facebook.net/D1383
AFFECTED FILES
contrib/src/test/results/clientpositive/dboutput.q.out
contrib/src/test/results/clientpositive/serde_typedbytes4.q.out
ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java
ql/src/test/results/clientpositive/auto_join0.q.out
ql/src/test/results/clientpositive/auto_join11.q.out
ql/src/test/results/clientpositive/auto_join12.q.out
ql/src/test/results/clientpositive/auto_join13.q.out
ql/src/test/results/clientpositive/auto_join14.q.out
ql/src/test/results/clientpositive/auto_join16.q.out
ql/src/test/results/clientpositive/auto_join20.q.out
ql/src/test/results/clientpositive/auto_join21.q.out
ql/src/test/results/clientpositive/auto_join23.q.out
ql/src/test/results/clientpositive/auto_join27.q.out
ql/src/test/results/clientpositive/auto_join28.q.out
ql/src/test/results/clientpositive/auto_join29.q.out
ql/src/test/results/clientpositive/auto_join4.q.out
ql/src/test/results/clientpositive/auto_join5.q.out
ql/src/test/results/clientpositive/auto_join6.q.out
ql/src/test/results/clientpositive/auto_join7.q.out
ql/src/test/results/clientpositive/auto_join8.q.out
ql/src/test/results/clientpositive/cast1.q.out
ql/src/test/results/clientpositive/cluster.q.out
ql/src/test/results/clientpositive/create_view.q.out
ql/src/test/results/clientpositive/groupby_multi_single_reducer.q.out
ql/src/test/results/clientpositive/having.q.out
ql/src/test/results/clientpositive/index_auto.q.out
ql/src/test/results/clientpositive/index_auto_empty.q.out
ql/src/test/results/clientpositive/index_auto_file_format.q.out
ql/src/test/results/clientpositive/index_auto_mult_tables.q.out
ql/src/test/results/clientpositive/index_auto_mult_tables_compact.q.out
ql/src/test/results/clientpositive/index_auto_multiple.q.out
ql/src/test/results/clientpositive/index_auto_partitioned.q.out
ql/src/test/results/clientpositive/index_auto_self_join.q.out
ql/src/test/results/clientpositive/index_auto_unused.q.out
ql/src/test/results/clientpositive/index_auto_update.q.out
ql/src/test/results/clientpositive/index_bitmap3.q.out
ql/src/test/results/clientpositive/index_bitmap_auto.q.out
ql/src/test/results/clientpositive/index_bitmap_auto_partitioned.q.out
ql/src/test/results/clientpositive/index_bitmap_compression.q.out
ql/src/test/results/clientpositive/index_compression.q.out
ql/src/test/results/clientpositive/index_stale.q.out
ql/src/test/results/clientpositive/index_stale_partitioned.q.out
ql/src/test/results/clientpositive/input11.q.out
ql/src/test/results/clientpositive/input11_limit.q.out
ql/src/test/results/clientpositive/input12.q.out
ql/src/test/results/clientpositive/input13.q.out
ql/src/test/results/clientpositive/input14.q.out
ql/src/test/results/clientpositive/input14_limit.q.out
ql/src/test/results/clientpositive/input18.q.out
ql/src/test/results/clientpositive/input1_limit.q.out
ql/src/test/results/clientpositive/input2_limit.q.out
ql/src/test/results/clientpositive/input42.q.out
ql/src/test/results/clientpositive/input_part1.q.out
ql/src/test/results/clientpositive/input_part2.q.out
ql/src/test/results/clientpositive/input_part5.q.out
ql/src/test/results/clientpositive/input_part7.q.out
ql/src/test/results/clientpositive/join0.q.out
ql/src/test/results/clientpositive/join11.q.out
ql/src/test/results/clientpositive/join12.q.out
ql/src/test/results/clientpositive/join13.q.out
ql/src/test/results/clientpositive/join14.q.out
ql/src/test/results/clientpositive/join16.q.out
ql/src/test/results/clientpositive/join20.q.out
ql/src/test/results/clientpositive/join21.q.out
ql/src/test/results/clientpositive/join23.q.out
ql/src/test/results/clientpositive/join34.q.out
ql/src/test/results/clientpositive/join35.q.out
ql/src/test/results/clientpositive/join38.q.out
ql/src/test/results/clientpositive/join39.q.out
ql/src/test/results/clientpositive/join4.q.out
ql/src/test/results/clientpositive/join40.q.out
ql/src/test/results/clientpositive/join5.q.out
ql/src/test/results/clientpositive/join6.q.out
ql/src/test/results/clientpositive/join7.q.out
ql/src/test/results/clientpositive/join8.q.out
ql/src/test/results/clientpositive/load_dyn_part13.q.out
ql/src/test/results/clientpositive/louter_join_ppr.q.out
ql/src/test/results/clientpositive/multi_insert.q.out
ql/src/test/results/clientpositive/no_hooks.q.out
ql/src/test/results/clientpositive/noalias_subq1.q.out
ql/src/test/results/clientpositive/notable_alias1.q.out
ql/src/test/results/clientpositive/notable_alias2.q.out
ql/src/test/results/clientpositive/nullgroup.q.out
ql/src/test/results/clientpositive/nullgroup2.q.out
ql/src/test/results/clientpositive/nullgroup4.q.out
ql/src/test/results/clientpositive/nullgroup4_multi_distinct.q.out
ql/src/test/results/clientpositive/order2.q.out
ql/src/test/results/clientpositive/outer_join_ppr.q.out
ql/src/test/results/clientpositive/pcr.q.out
ql/src/test/results/clientpositive/ppd_clusterby.q.out
ql/src/test/results/clientpositive/ppd_multi_insert.q.out
ql/src/test/results/clientpositive/ppd_outer_join1.q.out
ql/src/test/results/clientpositive/ppd_transform.q.out
ql/src/test/results/clientpositive/ppd_udf_col.q.out
ql/src/test/results/clientpositive/ppr_pushdown3.q.out
ql/src/test/results/clientpositive/quote1.q.out
ql/src/test/results/clientpositive/rand_partitionpruner3.q.out
ql/src/test/results/clientpositive/rcfile_null_value.q.out
ql/src/test/results/clientpositive/regex_col.q.out
ql/src/test/results/clientpositive/regexp_extract.q.out
ql/src/test/results/clientpositive/router_join_ppr.q.out
ql/src/test/results/clientpositive/semijoin.q.out
ql/src/test/results/clientpositive/set_processor_namespaces.q.out
ql/src/test/results/clientpositive/skewjoin.q.out
ql/src/test/results/clientpositive/subq.q.out
ql/src/test/results/clientpositive/subq2.q.out
ql/src/test/results/clientpositive/transform_ppr1.q.out
ql/src/test/results/clientpositive/transform_ppr2.q.out
ql/src/test/results/clientpositive/udf1.q.out
ql/src/test/results/clientpositive/udf9.q.out
ql/src/test/results/clientpositive/udf_10_trims.q.out
ql/src/test/results/clientpositive/udf_hour.q.out
ql/src/test/results/clientpositive/udf_like.q.out
ql/src/test/results/clientpositive/udf_lower.q.out
ql/src/test/results/clientpositive/udf_minute.q.out
ql/src/test/results/clientpositive/udf_parse_url.q.out
ql/src/test/results/clientpositive/udf_second.q.out
ql/src/test/results/clientpositive/udf_union.q.out
ql/src/test/results/clientpositive/union.q.out
ql/src/test/results/clientpositive/union20.q.out
ql/src/test/results/clientpositive/union22.q.out
ql/src/test/results/clientpositive/union24.q.out
ql/src/test/results/clientpositive/union_ppr.q.out
ql/src/test/results/compiler/plan/cast1.q.xml
ql/src/test/results/compiler/plan/input1.q.xml
ql/src/test/results/compiler/plan/input2.q.xml
ql/src/test/results/compiler/plan/input3.q.xml
ql/src/test/results/compiler/plan/input4.q.xml
ql/src/test/results/compiler/plan/input_part1.q.xml
ql/src/test/results/compiler/plan/join4.q.xml
ql/src/test/results/compiler/plan/join5.q.xml
ql/src/test/results/compiler/plan/join6.q.xml
ql/src/test/results/compiler/plan/join7.q.xml
ql/src/test/results/compiler/plan/join8.q.xml
ql/src/test/results/compiler/plan/subq.q.xml
ql/src/test/results/compiler/plan/udf1.q.xml
ql/src/test/results/compiler/plan/union.q.xml
MANAGE HERALD DIFFERENTIAL RULES
https://reviews.facebook.net/herald/view/differential/
WHY DID I GET THIS EMAIL?
https://reviews.facebook.net/herald/transcript/2901/
Tip: use the X-Herald-Rules header to filter Herald messages in your client.
> When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2249
> URL: https://issues.apache.org/jira/browse/HIVE-2249
> Project: Hive
> Issue Type: Improvement
> Reporter: Siying Dong
> Assignee: Joseph Barillari
> Attachments: HIVE-2249.1.patch.txt, HIVE-2249.2.patch.txt, HIVE-2249.D1383.1.patch
>
>
> The current code to build constant expression for numbers, here is the code:
> try {
> v = Double.valueOf(expr.getText());
> v = Long.valueOf(expr.getText());
> v = Integer.valueOf(expr.getText());
> } catch (NumberFormatException e) {
> // do nothing here, we will throw an exception in the following block
> }
> if (v == null) {
> throw new SemanticException(ErrorMsg.INVALID_NUMERICAL_CONSTANT
> .getMsg(expr));
> }
> return new ExprNodeConstantDesc(v);
> The for the case that "WHERE <BIG_INT_COLUMN> = 0", or "WHERE <DOUBLE_COLUMN> = 0", we always have to do a type conversion when comparing, which is unnecessary if it is slightly smarter to choose type when creating the constant expression. We can simply walk one level up the tree, find another comparison party and use the same type with that one if it is possible. For user's wrong query like '<INT_COLUMN>=1.1', we can even do more.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-2249) When creating constant expression for
numbers, try to infer type from another comparison operand, instead of
trying to use integer first, and then long and double
Posted by "Siying Dong (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siying Dong reassigned HIVE-2249:
---------------------------------
Assignee: Zhiqiu Kong (was: Joseph Barillari)
> When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2249
> URL: https://issues.apache.org/jira/browse/HIVE-2249
> Project: Hive
> Issue Type: Improvement
> Reporter: Siying Dong
> Assignee: Zhiqiu Kong
> Attachments: HIVE-2249.1.patch.txt, HIVE-2249.2.patch.txt, HIVE-2249.D1383.1.patch, HIVE-2249.D1383.2.patch, HIVE-2249.D1383.3.patch
>
>
> The current code to build constant expression for numbers, here is the code:
> try {
> v = Double.valueOf(expr.getText());
> v = Long.valueOf(expr.getText());
> v = Integer.valueOf(expr.getText());
> } catch (NumberFormatException e) {
> // do nothing here, we will throw an exception in the following block
> }
> if (v == null) {
> throw new SemanticException(ErrorMsg.INVALID_NUMERICAL_CONSTANT
> .getMsg(expr));
> }
> return new ExprNodeConstantDesc(v);
> The for the case that "WHERE <BIG_INT_COLUMN> = 0", or "WHERE <DOUBLE_COLUMN> = 0", we always have to do a type conversion when comparing, which is unnecessary if it is slightly smarter to choose type when creating the constant expression. We can simply walk one level up the tree, find another comparison party and use the same type with that one if it is possible. For user's wrong query like '<INT_COLUMN>=1.1', we can even do more.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2249) When creating constant expression for
numbers, try to infer type from another comparison operand, instead of
trying to use integer first, and then long and double
Posted by "Zhiqiu Kong (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhiqiu Kong updated HIVE-2249:
------------------------------
Attachment: HIVE-2249.2.patch.txt
This patch adds support to smartly infer the constant's type when encountering the query like "column CMP constant", where CMP could be any of the comparators supported by Hive. This aims to improving the performance by moving the type conversion from runtime stage to compiling stage.
To be more detailed, the smart type inference will happen when the type of the column is on e of the following:
* TINYINT
* SMALLINT
* INT
* BIGINT
* FLOAT
* DOUBLE
If the type of the columns fits any of the above, the constant on the other hand side will be converted firstly to the column's type. When failing, the constant will then be converted to DOUBLE. If both tries fail, the constant will be left as what type it is.
One exception is when the column is STRING while the constant is BIGINT. In this case, we do nothing. Otherwise, the constant will be converted to DOUBLE.
Other improvements include returning false immediately for the query like "int_col = not_convertable_double_constant", such as "uid = 1.5".
NOTE:
~130 unit test cases need to be updated due to this diff. All updates are limited to convert to the plan like "col = 10" to "col = 10.0", and are carefully checked individually.
TWO test cases failed during the unit testing:
* testCliDriver_insert2_overwrite_partitions
* testCliDriver_ppr_pushdown
When looking into the query as well as the output, the plans generated were found to be the same while the query results changed. As the queries in these two cases are simple select queries, maybe the default sorting criteria was changed unintentionally by this diff or other diffs.
> When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2249
> URL: https://issues.apache.org/jira/browse/HIVE-2249
> Project: Hive
> Issue Type: Improvement
> Reporter: Siying Dong
> Assignee: Joseph Barillari
> Attachments: HIVE-2249.1.patch.txt, HIVE-2249.2.patch.txt
>
>
> The current code to build constant expression for numbers, here is the code:
> try {
> v = Double.valueOf(expr.getText());
> v = Long.valueOf(expr.getText());
> v = Integer.valueOf(expr.getText());
> } catch (NumberFormatException e) {
> // do nothing here, we will throw an exception in the following block
> }
> if (v == null) {
> throw new SemanticException(ErrorMsg.INVALID_NUMERICAL_CONSTANT
> .getMsg(expr));
> }
> return new ExprNodeConstantDesc(v);
> The for the case that "WHERE <BIG_INT_COLUMN> = 0", or "WHERE <DOUBLE_COLUMN> = 0", we always have to do a type conversion when comparing, which is unnecessary if it is slightly smarter to choose type when creating the constant expression. We can simply walk one level up the tree, find another comparison party and use the same type with that one if it is possible. For user's wrong query like '<INT_COLUMN>=1.1', we can even do more.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2249) When creating constant expression for
numbers, try to infer type from another comparison operand, instead of
trying to use integer first, and then long and double
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2249:
------------------------------
Attachment: HIVE-2249.D1383.3.patch
zhiqiu updated the revision "HIVE-2249 [jira] When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double".
Reviewers: njain, kevinwilfong, heyongqiang, JIRA, stuart983
Address Siying's and Kevin's comments
* For string_col = long_const, no smart type inference is done
* Add query to test case to verify the results are correct
REVISION DETAIL
https://reviews.facebook.net/D1383
AFFECTED FILES
contrib/src/test/results/clientpositive/dboutput.q.out
contrib/src/test/results/clientpositive/serde_typedbytes4.q.out
data/files/infer_const_type.txt
ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java
ql/src/test/queries/clientpositive/infer_const_type.q
ql/src/test/queries/clientpositive/insert1_overwrite_partitions.q
ql/src/test/queries/clientpositive/insert2_overwrite_partitions.q
ql/src/test/queries/clientpositive/ppr_pushdown.q
ql/src/test/results/clientpositive/auto_join0.q.out
ql/src/test/results/clientpositive/auto_join11.q.out
ql/src/test/results/clientpositive/auto_join12.q.out
ql/src/test/results/clientpositive/auto_join13.q.out
ql/src/test/results/clientpositive/auto_join14.q.out
ql/src/test/results/clientpositive/auto_join16.q.out
ql/src/test/results/clientpositive/auto_join20.q.out
ql/src/test/results/clientpositive/auto_join21.q.out
ql/src/test/results/clientpositive/auto_join23.q.out
ql/src/test/results/clientpositive/auto_join27.q.out
ql/src/test/results/clientpositive/auto_join28.q.out
ql/src/test/results/clientpositive/auto_join29.q.out
ql/src/test/results/clientpositive/auto_join4.q.out
ql/src/test/results/clientpositive/auto_join5.q.out
ql/src/test/results/clientpositive/auto_join6.q.out
ql/src/test/results/clientpositive/auto_join7.q.out
ql/src/test/results/clientpositive/auto_join8.q.out
ql/src/test/results/clientpositive/cast1.q.out
ql/src/test/results/clientpositive/cluster.q.out
ql/src/test/results/clientpositive/create_view.q.out
ql/src/test/results/clientpositive/groupby_multi_single_reducer.q.out
ql/src/test/results/clientpositive/having.q.out
ql/src/test/results/clientpositive/index_auto.q.out
ql/src/test/results/clientpositive/index_auto_empty.q.out
ql/src/test/results/clientpositive/index_auto_file_format.q.out
ql/src/test/results/clientpositive/index_auto_mult_tables.q.out
ql/src/test/results/clientpositive/index_auto_mult_tables_compact.q.out
ql/src/test/results/clientpositive/index_auto_multiple.q.out
ql/src/test/results/clientpositive/index_auto_partitioned.q.out
ql/src/test/results/clientpositive/index_auto_self_join.q.out
ql/src/test/results/clientpositive/index_auto_unused.q.out
ql/src/test/results/clientpositive/index_auto_update.q.out
ql/src/test/results/clientpositive/index_bitmap3.q.out
ql/src/test/results/clientpositive/index_bitmap_auto.q.out
ql/src/test/results/clientpositive/index_bitmap_auto_partitioned.q.out
ql/src/test/results/clientpositive/index_bitmap_compression.q.out
ql/src/test/results/clientpositive/index_compression.q.out
ql/src/test/results/clientpositive/index_stale.q.out
ql/src/test/results/clientpositive/index_stale_partitioned.q.out
ql/src/test/results/clientpositive/infer_const_type.q.out
ql/src/test/results/clientpositive/input11.q.out
ql/src/test/results/clientpositive/input11_limit.q.out
ql/src/test/results/clientpositive/input12.q.out
ql/src/test/results/clientpositive/input13.q.out
ql/src/test/results/clientpositive/input14.q.out
ql/src/test/results/clientpositive/input14_limit.q.out
ql/src/test/results/clientpositive/input18.q.out
ql/src/test/results/clientpositive/input1_limit.q.out
ql/src/test/results/clientpositive/input2_limit.q.out
ql/src/test/results/clientpositive/input42.q.out
ql/src/test/results/clientpositive/input_part1.q.out
ql/src/test/results/clientpositive/input_part2.q.out
ql/src/test/results/clientpositive/input_part5.q.out
ql/src/test/results/clientpositive/input_part7.q.out
ql/src/test/results/clientpositive/insert1_overwrite_partitions.q.out
ql/src/test/results/clientpositive/insert2_overwrite_partitions.q.out
ql/src/test/results/clientpositive/join0.q.out
ql/src/test/results/clientpositive/join11.q.out
ql/src/test/results/clientpositive/join12.q.out
ql/src/test/results/clientpositive/join13.q.out
ql/src/test/results/clientpositive/join14.q.out
ql/src/test/results/clientpositive/join16.q.out
ql/src/test/results/clientpositive/join20.q.out
ql/src/test/results/clientpositive/join21.q.out
ql/src/test/results/clientpositive/join23.q.out
ql/src/test/results/clientpositive/join34.q.out
ql/src/test/results/clientpositive/join35.q.out
ql/src/test/results/clientpositive/join38.q.out
ql/src/test/results/clientpositive/join39.q.out
ql/src/test/results/clientpositive/join4.q.out
ql/src/test/results/clientpositive/join40.q.out
ql/src/test/results/clientpositive/join5.q.out
ql/src/test/results/clientpositive/join6.q.out
ql/src/test/results/clientpositive/join7.q.out
ql/src/test/results/clientpositive/join8.q.out
ql/src/test/results/clientpositive/load_dyn_part13.q.out
ql/src/test/results/clientpositive/louter_join_ppr.q.out
ql/src/test/results/clientpositive/multi_insert.q.out
ql/src/test/results/clientpositive/no_hooks.q.out
ql/src/test/results/clientpositive/noalias_subq1.q.out
ql/src/test/results/clientpositive/notable_alias1.q.out
ql/src/test/results/clientpositive/notable_alias2.q.out
ql/src/test/results/clientpositive/nullgroup.q.out
ql/src/test/results/clientpositive/nullgroup2.q.out
ql/src/test/results/clientpositive/nullgroup4.q.out
ql/src/test/results/clientpositive/nullgroup4_multi_distinct.q.out
ql/src/test/results/clientpositive/order2.q.out
ql/src/test/results/clientpositive/outer_join_ppr.q.out
ql/src/test/results/clientpositive/pcr.q.out
ql/src/test/results/clientpositive/ppd_clusterby.q.out
ql/src/test/results/clientpositive/ppd_multi_insert.q.out
ql/src/test/results/clientpositive/ppd_outer_join1.q.out
ql/src/test/results/clientpositive/ppd_transform.q.out
ql/src/test/results/clientpositive/ppd_udf_col.q.out
ql/src/test/results/clientpositive/ppr_pushdown.q.out
ql/src/test/results/clientpositive/ppr_pushdown3.q.out
ql/src/test/results/clientpositive/quote1.q.out
ql/src/test/results/clientpositive/rand_partitionpruner3.q.out
ql/src/test/results/clientpositive/rcfile_null_value.q.out
ql/src/test/results/clientpositive/regex_col.q.out
ql/src/test/results/clientpositive/regexp_extract.q.out
ql/src/test/results/clientpositive/router_join_ppr.q.out
ql/src/test/results/clientpositive/semijoin.q.out
ql/src/test/results/clientpositive/set_processor_namespaces.q.out
ql/src/test/results/clientpositive/skewjoin.q.out
ql/src/test/results/clientpositive/subq.q.out
ql/src/test/results/clientpositive/subq2.q.out
ql/src/test/results/clientpositive/transform_ppr1.q.out
ql/src/test/results/clientpositive/transform_ppr2.q.out
ql/src/test/results/clientpositive/udf1.q.out
ql/src/test/results/clientpositive/udf9.q.out
ql/src/test/results/clientpositive/udf_10_trims.q.out
ql/src/test/results/clientpositive/udf_hour.q.out
ql/src/test/results/clientpositive/udf_like.q.out
ql/src/test/results/clientpositive/udf_lower.q.out
ql/src/test/results/clientpositive/udf_minute.q.out
ql/src/test/results/clientpositive/udf_parse_url.q.out
ql/src/test/results/clientpositive/udf_second.q.out
ql/src/test/results/clientpositive/udf_union.q.out
ql/src/test/results/clientpositive/union.q.out
ql/src/test/results/clientpositive/union20.q.out
ql/src/test/results/clientpositive/union22.q.out
ql/src/test/results/clientpositive/union24.q.out
ql/src/test/results/clientpositive/union_ppr.q.out
ql/src/test/results/compiler/plan/cast1.q.xml
ql/src/test/results/compiler/plan/input1.q.xml
ql/src/test/results/compiler/plan/input2.q.xml
ql/src/test/results/compiler/plan/input3.q.xml
ql/src/test/results/compiler/plan/input4.q.xml
ql/src/test/results/compiler/plan/input_part1.q.xml
ql/src/test/results/compiler/plan/join4.q.xml
ql/src/test/results/compiler/plan/join5.q.xml
ql/src/test/results/compiler/plan/join6.q.xml
ql/src/test/results/compiler/plan/join7.q.xml
ql/src/test/results/compiler/plan/join8.q.xml
ql/src/test/results/compiler/plan/subq.q.xml
ql/src/test/results/compiler/plan/udf1.q.xml
ql/src/test/results/compiler/plan/union.q.xml
> When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2249
> URL: https://issues.apache.org/jira/browse/HIVE-2249
> Project: Hive
> Issue Type: Improvement
> Reporter: Siying Dong
> Assignee: Joseph Barillari
> Attachments: HIVE-2249.1.patch.txt, HIVE-2249.2.patch.txt, HIVE-2249.D1383.1.patch, HIVE-2249.D1383.2.patch, HIVE-2249.D1383.3.patch
>
>
> The current code to build constant expression for numbers, here is the code:
> try {
> v = Double.valueOf(expr.getText());
> v = Long.valueOf(expr.getText());
> v = Integer.valueOf(expr.getText());
> } catch (NumberFormatException e) {
> // do nothing here, we will throw an exception in the following block
> }
> if (v == null) {
> throw new SemanticException(ErrorMsg.INVALID_NUMERICAL_CONSTANT
> .getMsg(expr));
> }
> return new ExprNodeConstantDesc(v);
> The for the case that "WHERE <BIG_INT_COLUMN> = 0", or "WHERE <DOUBLE_COLUMN> = 0", we always have to do a type conversion when comparing, which is unnecessary if it is slightly smarter to choose type when creating the constant expression. We can simply walk one level up the tree, find another comparison party and use the same type with that one if it is possible. For user's wrong query like '<INT_COLUMN>=1.1', we can even do more.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2249) When creating constant expression
for numbers, try to infer type from another comparison operand, instead of
trying to use integer first, and then long and double
Posted by "Siying Dong (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13070811#comment-13070811 ]
Siying Dong commented on HIVE-2249:
-----------------------------------
Joseph, can you handle the string case too?
> When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2249
> URL: https://issues.apache.org/jira/browse/HIVE-2249
> Project: Hive
> Issue Type: Improvement
> Reporter: Siying Dong
> Assignee: Joseph Barillari
> Attachments: HIVE-2249.1.patch.txt
>
>
> The current code to build constant expression for numbers, here is the code:
> try {
> v = Double.valueOf(expr.getText());
> v = Long.valueOf(expr.getText());
> v = Integer.valueOf(expr.getText());
> } catch (NumberFormatException e) {
> // do nothing here, we will throw an exception in the following block
> }
> if (v == null) {
> throw new SemanticException(ErrorMsg.INVALID_NUMERICAL_CONSTANT
> .getMsg(expr));
> }
> return new ExprNodeConstantDesc(v);
> The for the case that "WHERE <BIG_INT_COLUMN> = 0", or "WHERE <DOUBLE_COLUMN> = 0", we always have to do a type conversion when comparing, which is unnecessary if it is slightly smarter to choose type when creating the constant expression. We can simply walk one level up the tree, find another comparison party and use the same type with that one if it is possible. For user's wrong query like '<INT_COLUMN>=1.1', we can even do more.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2249) When creating constant expression
for numbers, try to infer type from another comparison operand, instead of
trying to use integer first, and then long and double
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13194968#comment-13194968 ]
Phabricator commented on HIVE-2249:
-----------------------------------
kevinwilfong has commented on the revision "HIVE-2249 [jira] When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double".
INLINE COMMENTS
ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java:739 I agree. Looking at the implementation of ExprNodeGenericFuncDesc.newInstance, comparing a string and a bigint will either throw an exception, or display a warning depending on configuration settings, but that functionality will be lost for the case of comparing a column and a constant if false is returned.
REVISION DETAIL
https://reviews.facebook.net/D1383
> When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2249
> URL: https://issues.apache.org/jira/browse/HIVE-2249
> Project: Hive
> Issue Type: Improvement
> Reporter: Siying Dong
> Assignee: Joseph Barillari
> Attachments: HIVE-2249.1.patch.txt, HIVE-2249.2.patch.txt, HIVE-2249.D1383.1.patch, HIVE-2249.D1383.2.patch
>
>
> The current code to build constant expression for numbers, here is the code:
> try {
> v = Double.valueOf(expr.getText());
> v = Long.valueOf(expr.getText());
> v = Integer.valueOf(expr.getText());
> } catch (NumberFormatException e) {
> // do nothing here, we will throw an exception in the following block
> }
> if (v == null) {
> throw new SemanticException(ErrorMsg.INVALID_NUMERICAL_CONSTANT
> .getMsg(expr));
> }
> return new ExprNodeConstantDesc(v);
> The for the case that "WHERE <BIG_INT_COLUMN> = 0", or "WHERE <DOUBLE_COLUMN> = 0", we always have to do a type conversion when comparing, which is unnecessary if it is slightly smarter to choose type when creating the constant expression. We can simply walk one level up the tree, find another comparison party and use the same type with that one if it is possible. For user's wrong query like '<INT_COLUMN>=1.1', we can even do more.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2249) When creating constant expression for
numbers, try to infer type from another comparison operand, instead of
trying to use integer first, and then long and double
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2249:
------------------------------
Attachment: HIVE-2249.D1383.4.patch
zhiqiu updated the revision "HIVE-2249 [jira] When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double".
Reviewers: njain, kevinwilfong, heyongqiang, JIRA, stuart983
Fix insert1_overwrite_partitions.q and insert2_overwrite_partitions.q thus the results are deterministic
REVISION DETAIL
https://reviews.facebook.net/D1383
AFFECTED FILES
contrib/src/test/results/clientpositive/dboutput.q.out
contrib/src/test/results/clientpositive/serde_typedbytes4.q.out
data/files/infer_const_type.txt
ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java
ql/src/test/queries/clientpositive/infer_const_type.q
ql/src/test/queries/clientpositive/insert1_overwrite_partitions.q
ql/src/test/queries/clientpositive/insert2_overwrite_partitions.q
ql/src/test/queries/clientpositive/ppr_pushdown.q
ql/src/test/results/clientpositive/auto_join0.q.out
ql/src/test/results/clientpositive/auto_join11.q.out
ql/src/test/results/clientpositive/auto_join12.q.out
ql/src/test/results/clientpositive/auto_join13.q.out
ql/src/test/results/clientpositive/auto_join14.q.out
ql/src/test/results/clientpositive/auto_join16.q.out
ql/src/test/results/clientpositive/auto_join20.q.out
ql/src/test/results/clientpositive/auto_join21.q.out
ql/src/test/results/clientpositive/auto_join23.q.out
ql/src/test/results/clientpositive/auto_join27.q.out
ql/src/test/results/clientpositive/auto_join28.q.out
ql/src/test/results/clientpositive/auto_join29.q.out
ql/src/test/results/clientpositive/auto_join4.q.out
ql/src/test/results/clientpositive/auto_join5.q.out
ql/src/test/results/clientpositive/auto_join6.q.out
ql/src/test/results/clientpositive/auto_join7.q.out
ql/src/test/results/clientpositive/auto_join8.q.out
ql/src/test/results/clientpositive/cast1.q.out
ql/src/test/results/clientpositive/cluster.q.out
ql/src/test/results/clientpositive/create_view.q.out
ql/src/test/results/clientpositive/groupby_multi_single_reducer.q.out
ql/src/test/results/clientpositive/having.q.out
ql/src/test/results/clientpositive/index_auto.q.out
ql/src/test/results/clientpositive/index_auto_empty.q.out
ql/src/test/results/clientpositive/index_auto_file_format.q.out
ql/src/test/results/clientpositive/index_auto_mult_tables.q.out
ql/src/test/results/clientpositive/index_auto_mult_tables_compact.q.out
ql/src/test/results/clientpositive/index_auto_multiple.q.out
ql/src/test/results/clientpositive/index_auto_partitioned.q.out
ql/src/test/results/clientpositive/index_auto_self_join.q.out
ql/src/test/results/clientpositive/index_auto_unused.q.out
ql/src/test/results/clientpositive/index_auto_update.q.out
ql/src/test/results/clientpositive/index_bitmap3.q.out
ql/src/test/results/clientpositive/index_bitmap_auto.q.out
ql/src/test/results/clientpositive/index_bitmap_auto_partitioned.q.out
ql/src/test/results/clientpositive/index_bitmap_compression.q.out
ql/src/test/results/clientpositive/index_compression.q.out
ql/src/test/results/clientpositive/index_stale.q.out
ql/src/test/results/clientpositive/index_stale_partitioned.q.out
ql/src/test/results/clientpositive/infer_const_type.q.out
ql/src/test/results/clientpositive/input11.q.out
ql/src/test/results/clientpositive/input11_limit.q.out
ql/src/test/results/clientpositive/input12.q.out
ql/src/test/results/clientpositive/input13.q.out
ql/src/test/results/clientpositive/input14.q.out
ql/src/test/results/clientpositive/input14_limit.q.out
ql/src/test/results/clientpositive/input18.q.out
ql/src/test/results/clientpositive/input1_limit.q.out
ql/src/test/results/clientpositive/input2_limit.q.out
ql/src/test/results/clientpositive/input42.q.out
ql/src/test/results/clientpositive/input_part1.q.out
ql/src/test/results/clientpositive/input_part2.q.out
ql/src/test/results/clientpositive/input_part5.q.out
ql/src/test/results/clientpositive/input_part7.q.out
ql/src/test/results/clientpositive/insert1_overwrite_partitions.q.out
ql/src/test/results/clientpositive/insert2_overwrite_partitions.q.out
ql/src/test/results/clientpositive/join0.q.out
ql/src/test/results/clientpositive/join11.q.out
ql/src/test/results/clientpositive/join12.q.out
ql/src/test/results/clientpositive/join13.q.out
ql/src/test/results/clientpositive/join14.q.out
ql/src/test/results/clientpositive/join16.q.out
ql/src/test/results/clientpositive/join20.q.out
ql/src/test/results/clientpositive/join21.q.out
ql/src/test/results/clientpositive/join23.q.out
ql/src/test/results/clientpositive/join34.q.out
ql/src/test/results/clientpositive/join35.q.out
ql/src/test/results/clientpositive/join38.q.out
ql/src/test/results/clientpositive/join39.q.out
ql/src/test/results/clientpositive/join4.q.out
ql/src/test/results/clientpositive/join40.q.out
ql/src/test/results/clientpositive/join5.q.out
ql/src/test/results/clientpositive/join6.q.out
ql/src/test/results/clientpositive/join7.q.out
ql/src/test/results/clientpositive/join8.q.out
ql/src/test/results/clientpositive/load_dyn_part13.q.out
ql/src/test/results/clientpositive/louter_join_ppr.q.out
ql/src/test/results/clientpositive/multi_insert.q.out
ql/src/test/results/clientpositive/no_hooks.q.out
ql/src/test/results/clientpositive/noalias_subq1.q.out
ql/src/test/results/clientpositive/notable_alias1.q.out
ql/src/test/results/clientpositive/notable_alias2.q.out
ql/src/test/results/clientpositive/nullgroup.q.out
ql/src/test/results/clientpositive/nullgroup2.q.out
ql/src/test/results/clientpositive/nullgroup4.q.out
ql/src/test/results/clientpositive/nullgroup4_multi_distinct.q.out
ql/src/test/results/clientpositive/order2.q.out
ql/src/test/results/clientpositive/outer_join_ppr.q.out
ql/src/test/results/clientpositive/pcr.q.out
ql/src/test/results/clientpositive/ppd_clusterby.q.out
ql/src/test/results/clientpositive/ppd_multi_insert.q.out
ql/src/test/results/clientpositive/ppd_outer_join1.q.out
ql/src/test/results/clientpositive/ppd_transform.q.out
ql/src/test/results/clientpositive/ppd_udf_col.q.out
ql/src/test/results/clientpositive/ppr_pushdown.q.out
ql/src/test/results/clientpositive/ppr_pushdown3.q.out
ql/src/test/results/clientpositive/quote1.q.out
ql/src/test/results/clientpositive/rand_partitionpruner3.q.out
ql/src/test/results/clientpositive/rcfile_null_value.q.out
ql/src/test/results/clientpositive/regex_col.q.out
ql/src/test/results/clientpositive/regexp_extract.q.out
ql/src/test/results/clientpositive/router_join_ppr.q.out
ql/src/test/results/clientpositive/semijoin.q.out
ql/src/test/results/clientpositive/set_processor_namespaces.q.out
ql/src/test/results/clientpositive/skewjoin.q.out
ql/src/test/results/clientpositive/subq.q.out
ql/src/test/results/clientpositive/subq2.q.out
ql/src/test/results/clientpositive/transform_ppr1.q.out
ql/src/test/results/clientpositive/transform_ppr2.q.out
ql/src/test/results/clientpositive/udf1.q.out
ql/src/test/results/clientpositive/udf9.q.out
ql/src/test/results/clientpositive/udf_10_trims.q.out
ql/src/test/results/clientpositive/udf_hour.q.out
ql/src/test/results/clientpositive/udf_like.q.out
ql/src/test/results/clientpositive/udf_lower.q.out
ql/src/test/results/clientpositive/udf_minute.q.out
ql/src/test/results/clientpositive/udf_parse_url.q.out
ql/src/test/results/clientpositive/udf_second.q.out
ql/src/test/results/clientpositive/udf_union.q.out
ql/src/test/results/clientpositive/union.q.out
ql/src/test/results/clientpositive/union20.q.out
ql/src/test/results/clientpositive/union22.q.out
ql/src/test/results/clientpositive/union24.q.out
ql/src/test/results/clientpositive/union_ppr.q.out
ql/src/test/results/compiler/plan/cast1.q.xml
ql/src/test/results/compiler/plan/input1.q.xml
ql/src/test/results/compiler/plan/input2.q.xml
ql/src/test/results/compiler/plan/input3.q.xml
ql/src/test/results/compiler/plan/input4.q.xml
ql/src/test/results/compiler/plan/input_part1.q.xml
ql/src/test/results/compiler/plan/join4.q.xml
ql/src/test/results/compiler/plan/join5.q.xml
ql/src/test/results/compiler/plan/join6.q.xml
ql/src/test/results/compiler/plan/join7.q.xml
ql/src/test/results/compiler/plan/join8.q.xml
ql/src/test/results/compiler/plan/subq.q.xml
ql/src/test/results/compiler/plan/udf1.q.xml
ql/src/test/results/compiler/plan/union.q.xml
> When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2249
> URL: https://issues.apache.org/jira/browse/HIVE-2249
> Project: Hive
> Issue Type: Improvement
> Reporter: Siying Dong
> Assignee: Zhiqiu Kong
> Attachments: HIVE-2249.1.patch.txt, HIVE-2249.2.patch.txt, HIVE-2249.D1383.1.patch, HIVE-2249.D1383.2.patch, HIVE-2249.D1383.3.patch, HIVE-2249.D1383.4.patch
>
>
> The current code to build constant expression for numbers, here is the code:
> try {
> v = Double.valueOf(expr.getText());
> v = Long.valueOf(expr.getText());
> v = Integer.valueOf(expr.getText());
> } catch (NumberFormatException e) {
> // do nothing here, we will throw an exception in the following block
> }
> if (v == null) {
> throw new SemanticException(ErrorMsg.INVALID_NUMERICAL_CONSTANT
> .getMsg(expr));
> }
> return new ExprNodeConstantDesc(v);
> The for the case that "WHERE <BIG_INT_COLUMN> = 0", or "WHERE <DOUBLE_COLUMN> = 0", we always have to do a type conversion when comparing, which is unnecessary if it is slightly smarter to choose type when creating the constant expression. We can simply walk one level up the tree, find another comparison party and use the same type with that one if it is possible. For user's wrong query like '<INT_COLUMN>=1.1', we can even do more.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2249) When creating constant expression
for numbers, try to infer type from another comparison operand, instead of
trying to use integer first, and then long and double
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13196535#comment-13196535 ]
Phabricator commented on HIVE-2249:
-----------------------------------
stuart983 has accepted the revision "HIVE-2249 [jira] When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double".
REVISION DETAIL
https://reviews.facebook.net/D1383
> When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2249
> URL: https://issues.apache.org/jira/browse/HIVE-2249
> Project: Hive
> Issue Type: Improvement
> Reporter: Siying Dong
> Assignee: Zhiqiu Kong
> Attachments: HIVE-2249.1.patch.txt, HIVE-2249.2.patch.txt, HIVE-2249.D1383.1.patch, HIVE-2249.D1383.2.patch, HIVE-2249.D1383.3.patch
>
>
> The current code to build constant expression for numbers, here is the code:
> try {
> v = Double.valueOf(expr.getText());
> v = Long.valueOf(expr.getText());
> v = Integer.valueOf(expr.getText());
> } catch (NumberFormatException e) {
> // do nothing here, we will throw an exception in the following block
> }
> if (v == null) {
> throw new SemanticException(ErrorMsg.INVALID_NUMERICAL_CONSTANT
> .getMsg(expr));
> }
> return new ExprNodeConstantDesc(v);
> The for the case that "WHERE <BIG_INT_COLUMN> = 0", or "WHERE <DOUBLE_COLUMN> = 0", we always have to do a type conversion when comparing, which is unnecessary if it is slightly smarter to choose type when creating the constant expression. We can simply walk one level up the tree, find another comparison party and use the same type with that one if it is possible. For user's wrong query like '<INT_COLUMN>=1.1', we can even do more.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2249) When creating constant expression
for numbers, try to infer type from another comparison operand, instead of
trying to use integer first, and then long and double
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13193227#comment-13193227 ]
Phabricator commented on HIVE-2249:
-----------------------------------
kevinwilfong has commented on the revision "HIVE-2249 [jira] When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double".
I looked at the tests that were failing:
insert2_overwrite_partitions.q orders by the column one, but since there are several rows with the same value in column one, this does not guarantee a the rows will be returned in the same order, and hence, with the limit 5, you are not guaranteed to get the same rows. I'm not sure why your change would cause the order to differ, but it shouldn't be an issue.
It looks like the order by was added to try and make the test deterministic. Since your code can't be committed if tests are failing, could you change the order by to be on one, two. This should guarantee you get the same rows every time.
Also, it would probably be worth while to do this in insert1_overwrite_partitions.q as well, as it does a similar order by.
I haven't been able to reproduce a failure in ppr_pushdown.q so far.
INLINE COMMENTS
ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java:709-712 This is never used.
REVISION DETAIL
https://reviews.facebook.net/D1383
> When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2249
> URL: https://issues.apache.org/jira/browse/HIVE-2249
> Project: Hive
> Issue Type: Improvement
> Reporter: Siying Dong
> Assignee: Joseph Barillari
> Attachments: HIVE-2249.1.patch.txt, HIVE-2249.2.patch.txt, HIVE-2249.D1383.1.patch
>
>
> The current code to build constant expression for numbers, here is the code:
> try {
> v = Double.valueOf(expr.getText());
> v = Long.valueOf(expr.getText());
> v = Integer.valueOf(expr.getText());
> } catch (NumberFormatException e) {
> // do nothing here, we will throw an exception in the following block
> }
> if (v == null) {
> throw new SemanticException(ErrorMsg.INVALID_NUMERICAL_CONSTANT
> .getMsg(expr));
> }
> return new ExprNodeConstantDesc(v);
> The for the case that "WHERE <BIG_INT_COLUMN> = 0", or "WHERE <DOUBLE_COLUMN> = 0", we always have to do a type conversion when comparing, which is unnecessary if it is slightly smarter to choose type when creating the constant expression. We can simply walk one level up the tree, find another comparison party and use the same type with that one if it is possible. For user's wrong query like '<INT_COLUMN>=1.1', we can even do more.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2249) When creating constant expression
for numbers, try to infer type from another comparison operand, instead of
trying to use integer first, and then long and double
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13195025#comment-13195025 ]
Phabricator commented on HIVE-2249:
-----------------------------------
zhiqiu has commented on the revision "HIVE-2249 [jira] When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double".
INLINE COMMENTS
ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java:735 Yes. And that's why I use !constType.equalsIgnoreCase(Constants.BIGINT_TYPE_NAME).
REVISION DETAIL
https://reviews.facebook.net/D1383
> When creating constant expression for numbers, try to infer type from another comparison operand, instead of trying to use integer first, and then long and double
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-2249
> URL: https://issues.apache.org/jira/browse/HIVE-2249
> Project: Hive
> Issue Type: Improvement
> Reporter: Siying Dong
> Assignee: Joseph Barillari
> Attachments: HIVE-2249.1.patch.txt, HIVE-2249.2.patch.txt, HIVE-2249.D1383.1.patch, HIVE-2249.D1383.2.patch, HIVE-2249.D1383.3.patch
>
>
> The current code to build constant expression for numbers, here is the code:
> try {
> v = Double.valueOf(expr.getText());
> v = Long.valueOf(expr.getText());
> v = Integer.valueOf(expr.getText());
> } catch (NumberFormatException e) {
> // do nothing here, we will throw an exception in the following block
> }
> if (v == null) {
> throw new SemanticException(ErrorMsg.INVALID_NUMERICAL_CONSTANT
> .getMsg(expr));
> }
> return new ExprNodeConstantDesc(v);
> The for the case that "WHERE <BIG_INT_COLUMN> = 0", or "WHERE <DOUBLE_COLUMN> = 0", we always have to do a type conversion when comparing, which is unnecessary if it is slightly smarter to choose type when creating the constant expression. We can simply walk one level up the tree, find another comparison party and use the same type with that one if it is possible. For user's wrong query like '<INT_COLUMN>=1.1', we can even do more.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira