You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Steve Carlin (Jira)" <ji...@apache.org> on 2026/04/20 16:06:00 UTC
[jira] [Updated] (IMPALA-14328) Calcite Planner: Produce column lineage graph when Calcite is the planner
[ https://issues.apache.org/jira/browse/IMPALA-14328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steve Carlin updated IMPALA-14328:
----------------------------------
Parent: (was: IMPALA-12871)
Issue Type: Improvement (was: Sub-task)
> Calcite Planner: Produce column lineage graph when Calcite is the planner
> -------------------------------------------------------------------------
>
> Key: IMPALA-14328
> URL: https://issues.apache.org/jira/browse/IMPALA-14328
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend
> Reporter: Fang-Yu Rao
> Assignee: Fang-Yu Rao
> Priority: Major
>
> We found that Impala could not produce a structurally same column lineage graph when Calcite is the planner. For instance, consider the following query in [lineage.test|https://github.com/apache/impala/blob/master/testdata/workloads/functional-query/queries/QueryTest/lineage.test].
> {code:java}
> select * from (
> select tinyint_col + int_col x from functional.alltypes
> union all
> select sum(bigint_col) y from (select bigint_col from functional.alltypes) v1) v2
> {code}
> We expect Impala to produce a graph with 4 vertices and 1 edge. However, we only get one vertice when Calcite is the planner.
> {code:java}
> {
> "edges": [
> {
> "edgeType": "PROJECTION",
> "sources": [],
> "targets": [
> 0
> ]
> }
> ],
> "endTime": 1755630445,
> "hash": "3968bd65781e9e856eaca799f4501513",
> "queryId": "fb443702ac817ecc:c432854600000000",
> "queryText": "select * from ( select tinyint_col + int_col x from functional.alltypes union all select sum(bigint_col) y from (select bigint_col from functional.alltypes)
> "timestamp": 1755630437,
> "user": "fangyurao",
> "vertices": [
> {
> "id": 0,
> "vertexId": "X",
> "vertexType": "COLUMN"
> }
> ]
> }
> {code}
>
> To reproduce this issue, we could perform the following steps.
> - Execute "{{{}export USE_CALCITE_PLANNER=true{}}}" on the command line.
> - Start the Impala service from the command line using the following.
> {code:java}
> $IMPALA_HOME/bin/start-impala-cluster.py \
> '--impalad_args=--lineage_event_log_dir=/tmp/impala_test_lineage_U_j964 --use_local_catalog=false' \
> '--catalogd_args=--catalog_topic_mode=full'
> {code}
> - Execute "{{{}impala-shell{}}}" on the command enter the Impala shell.
> - Execute "{{{}set use_calcite_planner=1{}}}" in the Impala shell.
> - Submit the query to Impala server in the Impala shell.
> - Collect the lineage file from under the folder "{{{}/tmp/impala_test_lineage_U_j964{}}}" as specified when we started the Impala service after we submit the query.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org