You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Quanlong Huang (Jira)" <ji...@apache.org> on 2022/06/24 09:35:00 UTC
[jira] [Updated] (IMPALA-11307) Loading snapshot testdata failed by cannot dropping tables used by MV
[ https://issues.apache.org/jira/browse/IMPALA-11307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Quanlong Huang updated IMPALA-11307:
------------------------------------
Component/s: Infrastructure
Epic Link: IMPALA-3446
> Loading snapshot testdata failed by cannot dropping tables used by MV
> ---------------------------------------------------------------------
>
> Key: IMPALA-11307
> URL: https://issues.apache.org/jira/browse/IMPALA-11307
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
> Reporter: Quanlong Huang
> Assignee: Quanlong Huang
> Priority: Critical
>
> I tried to load a snapshot of testdata and encountered this failure:
> {code}
> Loading TPC-H data (logging to /data/jenkins/workspace/impala-private-parameterized/repos/Impala/logs/data_loading/load-tpch.log)...
> Loading TPC-DS data (logging to /data/jenkins/workspace/impala-private-parameterized/repos/Impala/logs/data_loading/load-tpcds.log)...
> FAILED (Took: 4 min 52 sec)
> 'load-data functional-query exhaustive' failed. Tail of log:
> 21:30:39 load-functional-query-exhaustive-hive-generated-seq-bzip-record.sql
> 21:30:39 load-functional-query-exhaustive-hive-generated-seq-snap-block.sql
> 21:30:39 load-functional-query-exhaustive-hive-generated-seq-snap-record.sql
> ...
> 21:32:22 Error executing impala SQL: /data/jenkins/workspace/impala-private-parameterized/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-seq-snap-record.sql See: /data/jenkins/workspace/impala-private-parameterized/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-seq-snap-record.sql.log
> 21:32:22 Beginning execution of impala SQL on localhost: /data/jenkins/workspace/impala-private-parameterized/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-rc-gzip-block.sql
> Background task Loading functional-query data (pid 8863) failed.
> Loading workload 'tpch' using exploration strategy 'core' OK (Took: 6 min 25 sec)
> Loading workload 'tpcds' using exploration strategy 'core' OK (Took: 11 min 42 sec)
> ERROR in /data/jenkins/workspace/impala-private-parameterized/repos/Impala/testdata/bin/create-load-data.sh at line 85: fi
> Generated: /data/jenkins/workspace/impala-private-parameterized/repos/Impala/logs/extra_junit_xml_logs/generate_junitxml.buildall.create-load-data.20220520_04_39_12.xml
> {code}
> Looking into the error log file, the failure is
> {code}
> ERROR: DROP TABLE IF EXISTS functional_seq_record_snap.insert_only_transactional_table
> Traceback (most recent call last):
> File "/data/jenkins/workspace/impala-private-parameterized/repos/Impala/bin/load-data.py", line 188, in exec_impala_query_from_file
> result = impala_client.execute(query)
> File "/data/jenkins/workspace/impala-private-parameterized/repos/Impala/tests/beeswax/impala_beeswax.py", line 189, in execute
> handle = self.__execute_query(query_string.strip(), user=user)
> File "/data/jenkins/workspace/impala-private-parameterized/repos/Impala/tests/beeswax/impala_beeswax.py", line 367, in __execute_query
> self.wait_for_finished(handle)
> File "/data/jenkins/workspace/impala-private-parameterized/repos/Impala/tests/beeswax/impala_beeswax.py", line 388, in wait_for_finished
> raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
> ImpalaBeeswaxException: ImpalaBeeswaxException:
> Query aborted:ImpalaRuntimeException: Error making 'dropTable' RPC to Hive Metastore:
> CAUSED BY: MetaException: Cannot drop table as it is used in the following materialized views [functional_seq_record_snap.materialized_view]
> {code}
> The DROP TABLE statement is executed before each CREATE TABLE statment, e.g.
> {code:sql}
> DROP TABLE IF EXISTS functional_seq_record_snap.insert_only_transactional_table;
> CREATE TABLE IF NOT EXISTS functional_seq_record_snap.insert_only_transactional_table (
> col1 int
> )
> STORED AS SEQUENCEFILE
> LOCATION '/test-warehouse/managed/functional_seq_record_snap.db/insert_only_transactional_table_seq_record_snap'
> TBLPROPERTIES (
> 'transactional_properties' = 'insert_only',
> 'transactional' = 'true'
> );
> {code}
> The MV is supposed to be created later in load-functional-query-exhaustive-hive-generated-seq-snap-record.sql:
> {code:sql}
> DROP TABLE IF EXISTS functional_seq_record_snap.materialized_view;
> -- The create materialized view command is moved down so that the database's
> -- managed directory has been created. Otherwise the command would fail. This
> -- is a bug in Hive.
> CREATE MATERIALIZED VIEW IF NOT EXISTS functional_seq_record_snap.materialized_view
> AS SELECT * FROM functional_seq_record_snap.insert_only_transactional_table;
> {code}
> As I'm loading the snapshot which is done by loading HMS snapshot and then loading the HDFS snapshot, I think those CREATE/DROP statements should be skipped.
> CC [~amansinha]
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org