You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2023/04/13 03:19:00 UTC
[jira] [Commented] (IMPALA-12058) Impala create table as select throws exception for Kudu tables with non unique primary key
[ https://issues.apache.org/jira/browse/IMPALA-12058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17711655#comment-17711655 ]
ASF subversion and git services commented on IMPALA-12058:
----------------------------------------------------------
Commit 7bfb1225f3dbf46f424b350c14142c8359af3046 in impala's branch refs/heads/master from wzhou-code
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=7bfb1225f ]
IMPALA-12058: Impala throw exception for CTAS with non unique primary key
The auto-incrementing column is added to target table by Impala frontend
for CTAS statement when creating Kudu table with non unique primary key
so that the Impala internal table has same layout as the table created
by Kudu engine. This is required for insertion operation of CTAS.
But auto-incrementing column is not in expression. It causes Planner
throws exception when computing Lineage graph for the column.
This patch fixes the issue by omiting auto-incrementing column for Kudu
table when adding target column labels for computing Lineage graph.
Testing:
- Added end-to-end test case for creating Kudu tables with Lineage
enabled.
- Passed core tests.
Change-Id: Ic87b7fac8eb857e9dc5c59bdb448270b23f6dd5a
Reviewed-on: http://gerrit.cloudera.org:8080/19725
Reviewed-by: Kurt Deschler <kd...@cloudera.com>
Reviewed-by: Abhishek Chennaka <ac...@cloudera.com>
Reviewed-by: Riza Suminto <ri...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Impala create table as select throws exception for Kudu tables with non unique primary key
> ------------------------------------------------------------------------------------------
>
> Key: IMPALA-12058
> URL: https://issues.apache.org/jira/browse/IMPALA-12058
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Reporter: Wenzhe Zhou
> Assignee: Wenzhe Zhou
> Priority: Major
> Fix For: Impala 4.3.0
>
>
> When Lineage is enabled, Impala throw exception when running CTAS statement to create Kudu table with non unique primary key.
> {code:java}
> Here is the log message:
> I0411 23:55:10.371549 267300 Frontend.java:1917] 9140a74de5d97dec:fad9408c00000000] Analyzing query: CREATE TABLE tbl9 NON UNIQUE PRIMARY KEY(i) PARTITION BY HASH (i) PARTITIONS 3 STORED as KUDU AS SELECT i, s FROM tbl7 db: default
> I0411 23:55:10.371635 267300 Frontend.java:1926] 9140a74de5d97dec:fad9408c00000000] The original executor group sets from executor membership snapshot: [TExecutorGroupSet(curr_num_executors:3, expected_num_executors:20, exec_group_name_prefix:)]
> I0411 23:55:10.371747 267300 Frontend.java:1947] 9140a74de5d97dec:fad9408c00000000] A total of 1 executor group sets to be considered for auto-scaling: [TExecutorGroupSet(curr_num_executors:3, expected_num_executors:20, exec_group_name_prefix:, max_mem_limit:9223372036854775807)]
> I0411 23:55:10.371855 267300 Frontend.java:1962] 9140a74de5d97dec:fad9408c00000000] Consider executor group set: TExecutorGroupSet(curr_num_executors:3, expected_num_executors:20, exec_group_name_prefix:, max_mem_limit:9223372036854775807)
> W0411 23:55:10.463629 267300 HiveConf.java:6520] 9140a74de5d97dec:fad9408c00000000] HiveConf of name hive.metastore.runworker.in does not exist
> I0411 23:55:10.467761 267300 AnalysisContext.java:484] 9140a74de5d97dec:fad9408c00000000] Analysis took 60 ms
> I0411 23:55:10.469448 267300 BaseAuthorizationChecker.java:113] 9140a74de5d97dec:fad9408c00000000] Authorization check took 1 ms
> I0411 23:55:10.469631 267300 Frontend.java:2062] 9140a74de5d97dec:fad9408c00000000] Analysis and authorization finished.
> I0411 23:55:10.488152 267300 jni-util.cc:288] 9140a74de5d97dec:fad9408c00000000] java.lang.IllegalStateException
> at com.google.common.base.Preconditions.checkState(Preconditions.java:492)
> at org.apache.impala.analysis.ColumnLineageGraph.computeProjectionDependencies(ColumnLineageGraph.java:569)
> at org.apache.impala.analysis.ColumnLineageGraph.computeLineageGraph(ColumnLineageGraph.java:542)
> at org.apache.impala.planner.Planner.createPlanFragments(Planner.java:235)
> at org.apache.impala.planner.Planner.createPlans(Planner.java:249)
> at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1732)
> at org.apache.impala.service.Frontend.getPlannedExecRequest(Frontend.java:2344)
> at org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:2181)
> at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1967)
> at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1788)
> at org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:164)
> I0411 23:55:10.488279 267300 status.cc:129] 9140a74de5d97dec:fad9408c00000000] IllegalStateException: null
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org