You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2023/01/06 01:16:00 UTC
[jira] [Commented] (IMPALA-11822) Optimize the Refresh/Invalidate event processing by skipping unnecessary events
[ https://issues.apache.org/jira/browse/IMPALA-11822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17655193#comment-17655193 ]
ASF subversion and git services commented on IMPALA-11822:
----------------------------------------------------------
Commit 5925b194b48dbe78ea1871d8aebc8f66aa72d01f in impala's branch refs/heads/master from Sai Hemanth Gantasala
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=5925b194b ]
IMPALA-11808: Add support for reload event in catalogD
This patch supports a new event called reload event in catalogD.
This event is used to update table/file metadata for a refresh
or invalidate command in other replicas of catalogDs when one
of the replicas issued refresh/invalidate command. HIVE-26838 is
the Hive jira that adds support for this reload event. This
feature is disabled by default using a config
enable_reload_events. To use this feature set this config to
true and impala will be able to fire reload events. The
processing of reload events in the event processor is always
enabled. There is an end-to-end test added for this feature
which currently checks firing/creation of the reload event and
self-event check in the event processor. TODO: end-to-end test
should also test this reload event in the event processor.
There is also a follow up jira IMPALA-11822 to track the
optimization patch for this feature.
Change-Id: Ic62d58837d356dc2113f3c0904228ac9de484136
Reviewed-on: http://gerrit.cloudera.org:8080/19378
Tested-by: Aman Sinha <am...@cloudera.com>
Reviewed-by: Andrew Sherman <as...@cloudera.com>
Reviewed-by: Aman Sinha <am...@cloudera.com>
> Optimize the Refresh/Invalidate event processing by skipping unnecessary events
> -------------------------------------------------------------------------------
>
> Key: IMPALA-11822
> URL: https://issues.apache.org/jira/browse/IMPALA-11822
> Project: IMPALA
> Issue Type: Sub-task
> Components: Catalog
> Reporter: Sai Hemanth Gantasala
> Assignee: Sai Hemanth Gantasala
> Priority: Major
>
> Optimize the Refresh/Invalidate event processing by skipping unnecessary events.
> Currently, we process every event as a new event. Consider there are 5 refresh events of the same table in the event processor queue. We can process the first refresh event and skip the remaining 4 events by comparing the timestamp of the refreshed table and event time. This way we can greatly improve the performance by skipping unnecessary events.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org