You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by "Vsevolod Ostapenko (Jira)" <ji...@apache.org> on 2020/01/17 17:29:00 UTC
[jira] [Created] (KYLIN-4350) Pushdown improperly rewrites the
query causing it to fail
Vsevolod Ostapenko created KYLIN-4350:
-----------------------------------------
Summary: Pushdown improperly rewrites the query causing it to fail
Key: KYLIN-4350
URL: https://issues.apache.org/jira/browse/KYLIN-4350
Project: Kylin
Issue Type: Bug
Components: Query Engine
Affects Versions: v2.6.4
Environment: HDP 2.6.5, Kylin 2.6.4, CentOS 7.6
Reporter: Vsevolod Ostapenko
A query that uses WITH clause and is subject for pushdown to Hive (or Impala) for execution is incorrectly rewritten before being submitted to the execution engine. Table aliases are attributed with database name, with makes query invalid.
Sample log excerpts are below:
{quote}2020-01-17 12:12:21,997 INFO [Query e844b846-c589-4729-5a04-483f6d73c834-31163] service.QueryService:404 : The original query: with
t as
(
SELECT ZETTICSDW.A_VL_HOURLY_V.IMSIID "ZETTICSDW_A_VL_HOURLY_V_IMSIID",
ZETTICSDW.A_VL_HOURLY_V.MEDIA_GAP_CALL_ID "ZETTICSDW_A_VL_HOURLY_V_MEDIA_GAP_CALL_ID",
count(*) cnt
FROM ZETTICSDW.A_VL_HOURLY_V
WHERE ((ZETTICSDW.A_VL_HOURLY_V.THEDATE = '20200117')
AND ((ZETTICSDW.A_VL_HOURLY_V.THEHOUR >= '10')
AND (ZETTICSDW.A_VL_HOURLY_V.THEHOUR <= '10')))
GROUP BY ZETTICSDW.A_VL_HOURLY_V.IMSIID, ZETTICSDW.A_VL_HOURLY_V.MEDIA_GAP_CALL_ID
)
select t.ZETTICSDW_A_VL_HOURLY_V_IMSIID,
count(*) "vl_aggs_model___CD_MEDIA_GAP_CALL_ID"
*from t*
group by t.ZETTICSDW_A_VL_HOURLY_V_IMSIID
ORDER BY "vl_aggs_model___CD_MEDIA_GAP_CALL_ID" desc
LIMIT 500
....
2020-01-17 12:12:22,073 INFO [Query e844b846-c589-4729-5a04-483f6d73c834-31163] adhocquery.AbstractPushdownRunner:37 : the query is converted to with
t as
(
SELECT ZETTICSDW.A_VL_HOURLY_V.IMSIID `ZETTICSDW_A_VL_HOURLY_V_IMSIID`,
ZETTICSDW.A_VL_HOURLY_V.MEDIA_GAP_CALL_ID `ZETTICSDW_A_VL_HOURLY_V_MEDIA_GAP_CALL_ID`,
count(*) cnt
FROM ZETTICSDW.A_VL_HOURLY_V
WHERE ((ZETTICSDW.A_VL_HOURLY_V.THEDATE = '20200117')
AND ((ZETTICSDW.A_VL_HOURLY_V.THEHOUR >= '10')
AND (ZETTICSDW.A_VL_HOURLY_V.THEHOUR <= '10')))
GROUP BY ZETTICSDW.A_VL_HOURLY_V.IMSIID, ZETTICSDW.A_VL_HOURLY_V.MEDIA_GAP_CALL_ID
)
select t.ZETTICSDW_A_VL_HOURLY_V_IMSIID,
count(*) `vl_aggs_model___CD_MEDIA_GAP_CALL_ID`
*{color:#FF0000}from ZETTICSDW.t{color}*
group by t.ZETTICSDW_A_VL_HOURLY_V_IMSIID
ORDER BY `vl_aggs_model___CD_MEDIA_GAP_CALL_ID` desc
LIMIT 500 after applying converter org.apache.kylin.source.adhocquery.HivePushDownConverter
2020-01-17 12:12:22,108 ERROR [Query e844b846-c589-4729-5a04-483f6d73c834-31163] service.QueryService:989 : pushdown engine failed current query too
org.apache.hive.service.cli.HiveSQLException: AnalysisException: Could not resolve table reference: '*zetticsdw.t*'
{quote}
Pushdown query should be submitted into query engine as written by the user.
As the best effort Kylin push down executor should issue "use <database>" over the same JDBC connection right before submitting the query.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)