You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by "weibin0516 (Jira)" <ji...@apache.org> on 2020/01/20 00:48:00 UTC
[jira] [Commented] (KYLIN-4350) Pushdown improperly rewrites the
query causing it to fail
[ https://issues.apache.org/jira/browse/KYLIN-4350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17019161#comment-17019161 ]
weibin0516 commented on KYLIN-4350:
-----------------------------------
Hi, [~seva_ostapenko], not all databases support use databse, such as postgresql
> Pushdown improperly rewrites the query causing it to fail
> ---------------------------------------------------------
>
> Key: KYLIN-4350
> URL: https://issues.apache.org/jira/browse/KYLIN-4350
> Project: Kylin
> Issue Type: Bug
> Components: Query Engine
> Affects Versions: v2.6.4
> Environment: HDP 2.6.5, Kylin 2.6.4, CentOS 7.6
> Reporter: Vsevolod Ostapenko
> Priority: Major
>
> A query that uses WITH clause and is subject for pushdown to Hive (or Impala) for execution is incorrectly rewritten before being submitted to the execution engine. Table aliases are attributed with database name, with makes query invalid.
> Sample log excerpts are below:
>
> {quote}2020-01-17 12:12:21,997 INFO [Query e844b846-c589-4729-5a04-483f6d73c834-31163] service.QueryService:404 : The original query: with
> t as
> (
> SELECT ZETTICSDW.A_VL_HOURLY_V.IMSIID "ZETTICSDW_A_VL_HOURLY_V_IMSIID",
> ZETTICSDW.A_VL_HOURLY_V.MEDIA_GAP_CALL_ID "ZETTICSDW_A_VL_HOURLY_V_MEDIA_GAP_CALL_ID",
> count(*) cnt
> FROM ZETTICSDW.A_VL_HOURLY_V
> WHERE ((ZETTICSDW.A_VL_HOURLY_V.THEDATE = '20200117')
> AND ((ZETTICSDW.A_VL_HOURLY_V.THEHOUR >= '10')
> AND (ZETTICSDW.A_VL_HOURLY_V.THEHOUR <= '10')))
> GROUP BY ZETTICSDW.A_VL_HOURLY_V.IMSIID, ZETTICSDW.A_VL_HOURLY_V.MEDIA_GAP_CALL_ID
> )
> select t.ZETTICSDW_A_VL_HOURLY_V_IMSIID,
> count(*) "vl_aggs_model___CD_MEDIA_GAP_CALL_ID"
> *from t*
> group by t.ZETTICSDW_A_VL_HOURLY_V_IMSIID
> ORDER BY "vl_aggs_model___CD_MEDIA_GAP_CALL_ID" desc
> LIMIT 500
> ....
> 2020-01-17 12:12:22,073 INFO [Query e844b846-c589-4729-5a04-483f6d73c834-31163] adhocquery.AbstractPushdownRunner:37 : the query is converted to with
> t as
> (
> SELECT ZETTICSDW.A_VL_HOURLY_V.IMSIID `ZETTICSDW_A_VL_HOURLY_V_IMSIID`,
> ZETTICSDW.A_VL_HOURLY_V.MEDIA_GAP_CALL_ID `ZETTICSDW_A_VL_HOURLY_V_MEDIA_GAP_CALL_ID`,
> count(*) cnt
> FROM ZETTICSDW.A_VL_HOURLY_V
> WHERE ((ZETTICSDW.A_VL_HOURLY_V.THEDATE = '20200117')
> AND ((ZETTICSDW.A_VL_HOURLY_V.THEHOUR >= '10')
> AND (ZETTICSDW.A_VL_HOURLY_V.THEHOUR <= '10')))
> GROUP BY ZETTICSDW.A_VL_HOURLY_V.IMSIID, ZETTICSDW.A_VL_HOURLY_V.MEDIA_GAP_CALL_ID
> )
> select t.ZETTICSDW_A_VL_HOURLY_V_IMSIID,
> count(*) `vl_aggs_model___CD_MEDIA_GAP_CALL_ID`
> *{color:#FF0000}from ZETTICSDW.t{color}*
> group by t.ZETTICSDW_A_VL_HOURLY_V_IMSIID
> ORDER BY `vl_aggs_model___CD_MEDIA_GAP_CALL_ID` desc
> LIMIT 500 after applying converter org.apache.kylin.source.adhocquery.HivePushDownConverter
> 2020-01-17 12:12:22,108 ERROR [Query e844b846-c589-4729-5a04-483f6d73c834-31163] service.QueryService:989 : pushdown engine failed current query too
> org.apache.hive.service.cli.HiveSQLException: AnalysisException: Could not resolve table reference: '*zetticsdw.t*'
> {quote}
> Pushdown query should be submitted into query engine as written by the user.
> As the best effort Kylin push down executor should issue "use <database>" over the same JDBC connection right before submitting the query.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)