You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Zoltan Haindrich (JIRA)" <ji...@apache.org> on 2019/01/22 08:31:00 UTC
[jira] [Comment Edited] (HIVE-21001) Upgrade to calcite-1.18
[ https://issues.apache.org/jira/browse/HIVE-21001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16748478#comment-16748478 ]
Zoltan Haindrich edited comment on HIVE-21001 at 1/22/19 8:30 AM:
------------------------------------------------------------------
[~jcamachorodriguez],[~bslim],[~ashutoshc]:
* {{ReduceExpressionsRule}} had a visitor pattern which have entered every {{RexNode}} and run simplify on them...and IIRC it have run into unknownAsX troubles...
* since Simplify also has a visitor pattern the above was became somewhat disabled.
* as a result arguments of BETWEEN are not simplified right now (because it's not handled at all in RexSimplify)
* Druid tests are failing, because during BETWEEN translation it's arguments are expected to be Literals; however since simplifyCast() have not run on them; some type changing {{CAST}} might be still there
Example:
{code}
BETWEEN(false, FLOOR_HOUR($0, FLAG(HOUR)), CAST(_UTF-16LE'2010-01-01 00:00:00'):TIMESTAMP_WITH_LOCAL_TIME_ZONE(15), CAST(_UTF-16LE'2014-01-01 00:00:00'):TIMESTAMP_WITH_LOCAL_TIME_ZONE(15))
{code}
to fix this I can think of the following:
# open between calls during hive2calcite translation; and closing them back from {{PointLookupOptimizer}}
# support between in {{RexSimplify}} (and wait for the next calcite release)
# make druid between translation handle the cast (and wait for the next calcite release)
I think that opening {{BETWEEN}} could also help in making stronger simplifcations...so I think (1) would be the best.
was (Author: kgyrtkirk):
[~jcamachorodriguez],[~bslim]:
* {{ReduceExpressionsRule}} had a visitor pattern which have entered every {{RexNode}} and run simplify on them...and IIRC it have run into unknownAsX troubles...
* since Simplify also has a visitor pattern the above was became somewhat disabled.
* as a result arguments of BETWEEN are not simplified right now (because it's not handled at all in RexSimplify)
* Druid tests are failing, because during BETWEEN translation it's arguments are expected to be Literals; however since simplifyCast() have not run on them; some type changing {{CAST}} might be still there
Example:
{code}
BETWEEN(false, FLOOR_HOUR($0, FLAG(HOUR)), CAST(_UTF-16LE'2010-01-01 00:00:00'):TIMESTAMP_WITH_LOCAL_TIME_ZONE(15), CAST(_UTF-16LE'2014-01-01 00:00:00'):TIMESTAMP_WITH_LOCAL_TIME_ZONE(15))
{code}
to fix this I can think of the following:
# open between calls during hive2calcite translation; and closing them back from {{PointLookupOptimizer}}
# support between in {{RexSimplify}} (and wait for the next calcite release)
# make druid between translation handle the cast (and wait for the next calcite release)
I think that opening {{BETWEEN}} could also help in making stronger simplifcations...so I think (1) would be the best.
> Upgrade to calcite-1.18
> -----------------------
>
> Key: HIVE-21001
> URL: https://issues.apache.org/jira/browse/HIVE-21001
> Project: Hive
> Issue Type: Improvement
> Reporter: Zoltan Haindrich
> Assignee: Zoltan Haindrich
> Priority: Major
> Attachments: HIVE-21001.01.patch, HIVE-21001.01.patch, HIVE-21001.02.patch, HIVE-21001.03.patch, HIVE-21001.04.patch, HIVE-21001.05.patch, HIVE-21001.06.patch, HIVE-21001.06.patch, HIVE-21001.07.patch, HIVE-21001.08.patch, HIVE-21001.08.patch, HIVE-21001.08.patch, HIVE-21001.09.patch, HIVE-21001.09.patch, HIVE-21001.09.patch, HIVE-21001.10.patch, HIVE-21001.11.patch, HIVE-21001.12.patch
>
>
> CLEAR LIBRARY CACHE
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)