You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2023/04/21 23:06:00 UTC
[jira] [Commented] (IMPALA-11822) Optimize the Refresh/Invalidate event processing by skipping unnecessary events
[ https://issues.apache.org/jira/browse/IMPALA-11822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17715189#comment-17715189 ]
ASF subversion and git services commented on IMPALA-11822:
----------------------------------------------------------
Commit 963de82e53767075caf7ee16ca9fd3caafc53c18 in impala's branch refs/heads/master from Sai Hemanth Gantasala
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=963de82e5 ]
IMPALA-11822: (Addendum) Optimize the Refresh/Invalidate event
processing by skipping unnecessary events
IMPALA-11822 introduced a new argument 'lastRefreshEventId' to the
HdfsPartition constructor and this is causing Hive builds to fail.
This addendum patch addresses this issue by keeping the old constructor
in the HdfsPartition class and call the new constructor with -1L as
default value for 'lastRefreshEventId'.
Testing: Verified that the Hive builds are passing with this patch.
Change-Id: I803c13b58e3de047c12f75a584232819c7dc74c1
Reviewed-on: http://gerrit.cloudera.org:8080/19624
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Optimize the Refresh/Invalidate event processing by skipping unnecessary events
> -------------------------------------------------------------------------------
>
> Key: IMPALA-11822
> URL: https://issues.apache.org/jira/browse/IMPALA-11822
> Project: IMPALA
> Issue Type: Improvement
> Components: Catalog
> Reporter: Sai Hemanth Gantasala
> Assignee: Sai Hemanth Gantasala
> Priority: Critical
>
> Optimize the Refresh/Invalidate event processing by skipping unnecessary events.
> Currently, we process every event as a new event. Consider there are 5 refresh events of the same table in the event processor queue. We can process the first refresh event and skip the remaining 4 events by comparing the timestamp of the refreshed table and event time. This way we can greatly improve the performance by skipping unnecessary events.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org