You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2023/05/02 16:12:00 UTC
[jira] [Commented] (IMPALA-12084) INVALIDATE METADATA under unloaded db hit DCHECK in LocalCatalog mode
[ https://issues.apache.org/jira/browse/IMPALA-12084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17718635#comment-17718635 ]
ASF subversion and git services commented on IMPALA-12084:
----------------------------------------------------------
Commit ff503e4a4b4f5fcace6fea8659a881ccff075b5c in impala's branch refs/heads/master from stiga-huang
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=ff503e4a4 ]
IMPALA-12084: IM under unloaded db should not send full db updates in catalog-v2
INVALIDATE METADATA under unloaded db will also add the newly added db
along with the newly added table in the catalog update. In LocalCatalog
mode, we should only send back invalidation of the db instead of the
full db object. This is a missing logic we should have added in
IMPALA-9936.
Tests:
- Add e2e test on LocalCatalog mode
Change-Id: I38a44f69cca20331ab31acc270c2921a7bfedae5
Reviewed-on: http://gerrit.cloudera.org:8080/19787
Reviewed-by: Daniel Becker <da...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
Reviewed-by: Michael Smith <mi...@cloudera.com>
> INVALIDATE METADATA under unloaded db hit DCHECK in LocalCatalog mode
> ---------------------------------------------------------------------
>
> Key: IMPALA-12084
> URL: https://issues.apache.org/jira/browse/IMPALA-12084
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Reporter: Quanlong Huang
> Assignee: Quanlong Huang
> Priority: Critical
>
> When running in LocalCatalog mode, INVALIDATE METADATA under unloaded db will hit the following DCHECK:
> {code}
> F0423 17:12:50.566385 7565 catalog-op-executor.cc:88] 9c4c9d5f5d7974bc:b97e6dae00000000] Check failed: !obj.db.__isset.metastore_db Minimal database TCatalogObject should have empty metastore_db
> *** Check failure stack trace: ***
> @ 0x37ce3ed google::LogMessage::Fail()
> @ 0x37d0324 google::LogMessage::SendToLog()
> @ 0x37cddcc google::LogMessage::Flush()
> @ 0x37d0849 google::LogMessageFatal::~LogMessageFatal()
> @ 0x195389d VerifyMinimalResponse()
> @ 0x195809c impala::CatalogOpExecutor::Exec()
> @ 0x16ce391 impala::ClientRequestState::ExecDdlRequestImpl()
> @ 0x16d713c boost::detail::function::void_function_obj_invoker0<>::invoke()
> @ 0x18cdb52 impala::Thread::SuperviseThread()
> @ 0x18ce95b boost::detail::thread_data<>::run()
> @ 0x23cd927 thread_proxy
> @ 0x7f196963f6db start_thread
> @ 0x7f19663b761f clone{code}
> This is a bug introduced by IMPALA-9936 in which we added the DCHECK. This use case is still sending back the full DB object to the coordinator.
> For reproducing the issue:
> {code:sql}
> # Start Impala cluster with --hms_event_polling_interval_s=0
> bin/start-impala-cluster.py --catalogd_args="--catalog_topic_mode=minimal --hms_event_polling_interval_s=0" --impalad_args=--use_local_catalog
> # Create new db and new table in Hive
> beeline -u "jdbc:hive2://localhost:11050"
> > create database mydb;
> > create table mydb.mytbl (i int);
> # Run IM in Impala
> impala-shell.sh
> > invalidate metadata mydb.mytbl;
> # The coordinator will crash at hitting the DCHECK
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org