You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2022/09/09 05:22:00 UTC
[jira] [Commented] (IMPALA-11540) Add warning logs for slow ALTER_TABLE event processing
[ https://issues.apache.org/jira/browse/IMPALA-11540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17602123#comment-17602123 ]
ASF subversion and git services commented on IMPALA-11540:
----------------------------------------------------------
Commit 5aff120c40475de35bf45703b3cb953b9ab970f1 in impala's branch refs/heads/master from stiga-huang
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=5aff120c4 ]
IMPALA-11540: Add logs for ALTER_TABLE events that trigger slow metadata reload
ALTER_TABLE event could trigger full refresh on the table, which is
time consuming and can lead to event-processor lagging behind. We need
logs to show the difference between the before and after hmsTables so we
can further distinguish the cases that actully don't need a full table
refresh, e.g. changing the owner, changing tblproperties, etc.
Tests:
- Attach the process using IntelliJ to pause the metadata loading for a
while. So the metadata reloading of ALTER_TABLE event can exceed the
threshold. Manually verified the logs.
Change-Id: Ibf344e6b423f88c9635ca8d61d53385b88ba4dce
Reviewed-on: http://gerrit.cloudera.org:8080/18926
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Quanlong Huang <hu...@gmail.com>
> Add warning logs for slow ALTER_TABLE event processing
> ------------------------------------------------------
>
> Key: IMPALA-11540
> URL: https://issues.apache.org/jira/browse/IMPALA-11540
> Project: IMPALA
> Issue Type: Improvement
> Components: Catalog
> Reporter: Quanlong Huang
> Assignee: Quanlong Huang
> Priority: Major
>
> ALTER_TABLE event could trigger full refresh on the table, which is expansive and can lead to event-processor lagging behind. We need logs to show the difference between the before and after hmsTables so we can further distinguish the cases that actully don't need a full table refresh. E.g. altering the owner don't need to reload the file metadata.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org