You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Quanlong Huang (Code Review)" <ge...@cloudera.org> on 2024/02/02 12:30:11 UTC
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Quanlong Huang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/20986
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
IMPALA-12782: Show info of the event processing in /events webUI
The /events page of catalogd shows the metrics and status of the
event-processor. This patch adds more info in this page, including
- lag info
- current event batch that's being processing
See the screenshot attached in the JIRA for how it looks like.
Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
---
M be/src/catalog/catalog-server.cc
M be/src/util/pretty-printer.h
M common/thrift/JniCatalog.thrift
M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java
M www/events.tmpl
5 files changed, 183 insertions(+), 9 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/86/20986/1
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/20986 )
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
Patch Set 8:
PS8 adds a new test to verify the error message for exceptions.
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 8
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 Mar 2024 13:29:04 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20986 )
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
Patch Set 6:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/15455/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 6
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Fri, 08 Mar 2024 01:39:36 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/20986 )
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
Patch Set 8:
(5 comments)
http://gerrit.cloudera.org:8080/#/c/20986/8//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/20986/8//COMMIT_MSG@12
PS8, Line 12: processing
nit: processed
http://gerrit.cloudera.org:8080/#/c/20986/8/be/src/catalog/catalog-server.cc
File be/src/catalog/catalog-server.cc:
http://gerrit.cloudera.org:8080/#/c/20986/8/be/src/catalog/catalog-server.cc@980
PS8, Line 980: last_synced_event_id
idea to reduce the chattiness of the code:
a struct like ObjWrapper could be created with a template function AddMember(const char* name, const T& val), specialized for std::string.
This would allow skipping creating local Value variables and passing the allocator in each AddMember().
So this could look like:
ObjWrapper progress_info_obj(allocator);
progress_info_obj.AddMember("last_synced_event_id", progress_info.last_synced_event_id)
...
document->AddMember("progress-info", progress_info_obj.value, allocator);
Note that RapidJson uses move semanctics, so after Value::AddMember("s", val, allocator), 'val' is set to null, so there is no need to keep it on the stack.
http://gerrit.cloudera.org:8080/#/c/20986/3/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java
File fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java:
http://gerrit.cloudera.org:8080/#/c/20986/3/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java@1192
PS3, Line 1192: * @throws MetastoreNotificationException
> I remove calling it in finally since it will reset currentEvent_ for except
One member when this seems problematic is currentEventIndex_ - if there is an exception then it won't be reset, so after restarting the event processor currentEventIndex_ won't start from zero in the first batch.
What do you think about calling resetProgress() in finally, but in case of an exception, saving the the current event to a member like lastEventDuringException_? This could be returned instead of currentFilteredEvent_ in case currentFilteredEvent_ and the status is error. It could be also useful do display the last problematic event in webui, even if the event processor was restarted.
http://gerrit.cloudera.org:8080/#/c/20986/8/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java
File fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java:
http://gerrit.cloudera.org:8080/#/c/20986/8/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java@1146
PS8, Line 1146: in case they are replaced
: // when being used below
"in case they are replaced concurrently"?
It could be also mentioned that the consistency of the members in progressInfo is not guaranteed in case of parallel modifications.
http://gerrit.cloudera.org:8080/#/c/20986/8/tests/custom_cluster/test_web_pages.py
File tests/custom_cluster/test_web_pages.py:
http://gerrit.cloudera.org:8080/#/c/20986/8/tests/custom_cluster/test_web_pages.py@444
PS8, Line 444: catalogd_event_processing_delay
Probably out of the scope of this patch, but a potential alternative would be to use table properties to inject debug actions to the event processor, so that the debug action would be triggered only for events from the specific table. This would allow these tests to run without using a custom cluster.
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 8
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 Mar 2024 16:02:54 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/20986 )
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
Patch Set 8:
(5 comments)
http://gerrit.cloudera.org:8080/#/c/20986/8//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/20986/8//COMMIT_MSG@12
PS8, Line 12: processing
> nit: processed
Done
http://gerrit.cloudera.org:8080/#/c/20986/8/be/src/catalog/catalog-server.cc
File be/src/catalog/catalog-server.cc:
http://gerrit.cloudera.org:8080/#/c/20986/8/be/src/catalog/catalog-server.cc@980
PS8, Line 980: last_synced_event_id
> idea to reduce the chattiness of the code:
Good point! Added such a wrapper in be/src/util/json-util.h
http://gerrit.cloudera.org:8080/#/c/20986/3/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java
File fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java:
http://gerrit.cloudera.org:8080/#/c/20986/3/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java@1192
PS3, Line 1192: * @throws MetastoreNotificationException
> One member when this seems problematic is currentEventIndex_ - if there is
Starts from PS6, resetProgressInfo() is called in start() so I think the metrics are good when the event processor was restarted.
I also thought about adding a new field to track the last failed event and always show it in the WebUI. But it's a bit confusing when the event processor is actually active (after restart). The current section of "Error Message" is moved to the top of the /events page to indicate the status of the event processor. So I thought it would be better to add a new section about the event processing history. In that section we can show the top-10 expensive events, top-10 expensive tables, and the recent 10 failures, etc. Do you think it's ok to do it in a new JIRA?
http://gerrit.cloudera.org:8080/#/c/20986/8/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java
File fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java:
http://gerrit.cloudera.org:8080/#/c/20986/8/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java@1146
PS8, Line 1146: in case they are replaced
: // when being used below
> "in case they are replaced concurrently"?
Done
http://gerrit.cloudera.org:8080/#/c/20986/8/tests/custom_cluster/test_web_pages.py
File tests/custom_cluster/test_web_pages.py:
http://gerrit.cloudera.org:8080/#/c/20986/8/tests/custom_cluster/test_web_pages.py@444
PS8, Line 444: catalogd_event_processing_delay
> Probably out of the scope of this patch, but a potential alternative would
Good point! Filed IMPALA-12901 for this.
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 8
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Thu, 14 Mar 2024 03:54:40 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Hello k.venureddy2103@gmail.com, Sai Hemanth Gantasala, Csaba Ringhofer, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/20986
to look at the new patch set (#9).
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
IMPALA-12782: Show info of the event processing in /events webUI
The /events page of catalogd shows the metrics and status of the
event-processor. This patch adds more info in this page, including
- lag info
- current event batch that's being processed
See the screenshot attached in the JIRA for how it looks like.
Also moves the error message to the top to highlight the error status.
Adds a debug action, catalogd_event_processing_delay, to inject a sleep
while processing an event. So the web page can be captured more easily.
Also adds a missing test for showing the error message of
event-processing in the /events page.
Tests:
- Add e2e test to verify the content of the page.
Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
---
M be/src/catalog/catalog-server.cc
M be/src/util/json-util.h
M common/thrift/JniCatalog.thrift
M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java
M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java
M fe/src/main/java/org/apache/impala/util/DebugUtils.java
M tests/custom_cluster/test_web_pages.py
M www/events.tmpl
8 files changed, 286 insertions(+), 18 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/86/20986/9
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 9
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20986 )
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
Patch Set 5:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/10290/ DRY_RUN=true
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 5
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Thu, 22 Feb 2024 05:42:44 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20986 )
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
Patch Set 3:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/15255/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Tue, 20 Feb 2024 12:37:16 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/20986 )
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
Patch Set 10:
(2 comments)
PS10 addressed comments and fixes three more issues:
- currentEvent_ is cleared unintentionally when EP is in non-active state but being rescheduled again.
- eventProcessorErrorMsg_ is not cleared after EP is restarted (by global INVALIDATE METADATA)
- latest event id is not updated if EP is stopped.
http://gerrit.cloudera.org:8080/#/c/20986/3/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java
File fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java:
http://gerrit.cloudera.org:8080/#/c/20986/3/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java@1192
PS3, Line 1192: * @param currentEventId Current event id on metastore
> I am ok with merging the current state, but I find it pretty confusing. Doi
Based on the offline discussion, let's add these in IMPALA-12912.
http://gerrit.cloudera.org:8080/#/c/20986/9/www/events.tmpl
File www/events.tmpl:
http://gerrit.cloudera.org:8080/#/c/20986/9/www/events.tmpl@57
PS9, Line 57: proces
> Maybe processed would be clearer?
Done
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 10
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Fri, 15 Mar 2024 10:39:45 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Hello k.venureddy2103@gmail.com, Sai Hemanth Gantasala, Csaba Ringhofer, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/20986
to look at the new patch set (#11).
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
IMPALA-12782: Show info of the event processing in /events webUI
The /events page of catalogd shows the metrics and status of the
event-processor. This patch adds more info in this page, including
- lag info
- current event batch that's being processed
See the screenshot attached in the JIRA for how it looks like.
Also moves the error message to the top to highlight the error status.
Fixes the issue of not updating latest event id when event processor is
stopped. Also fixes the issue of error message not cleared after global
INVALIDATE METADATA.
Adds a debug action, catalogd_event_processing_delay, to inject a sleep
while processing an event. So the web page can be captured more easily.
Also adds a missing test for showing the error message of
event-processing in the /events page.
Tests:
- Add e2e test to verify the content of the page.
Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
---
M be/src/catalog/catalog-server.cc
M be/src/util/json-util.h
M common/thrift/JniCatalog.thrift
M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java
M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java
M fe/src/main/java/org/apache/impala/util/DebugUtils.java
M tests/custom_cluster/test_web_pages.py
M www/events.tmpl
8 files changed, 292 insertions(+), 20 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/86/20986/11
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 11
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Hello k.venureddy2103@gmail.com, Sai Hemanth Gantasala, Csaba Ringhofer, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/20986
to look at the new patch set (#10).
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
IMPALA-12782: Show info of the event processing in /events webUI
The /events page of catalogd shows the metrics and status of the
event-processor. This patch adds more info in this page, including
- lag info
- current event batch that's being processed
See the screenshot attached in the JIRA for how it looks like.
Also moves the error message to the top to highlight the error status.
Fixes the issue of not updating latest event id when event processor is
stopped.
Adds a debug action, catalogd_event_processing_delay, to inject a sleep
while processing an event. So the web page can be captured more easily.
Also adds a missing test for showing the error message of
event-processing in the /events page.
Tests:
- Add e2e test to verify the content of the page.
Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
---
M be/src/catalog/catalog-server.cc
M be/src/util/json-util.h
M common/thrift/JniCatalog.thrift
M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java
M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java
M fe/src/main/java/org/apache/impala/util/DebugUtils.java
M tests/custom_cluster/test_web_pages.py
M www/events.tmpl
8 files changed, 289 insertions(+), 20 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/86/20986/10
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 10
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20986 )
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
Patch Set 10:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/15527/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 10
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Fri, 15 Mar 2024 10:52:02 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20986 )
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
Patch Set 9:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/20986/9/be/src/catalog/catalog-server.cc
File be/src/catalog/catalog-server.cc:
http://gerrit.cloudera.org:8080/#/c/20986/9/be/src/catalog/catalog-server.cc@982
PS9, Line 982: progress_info_obj.AddMember("last_synced_event_time_s", progress_info.last_synced_event_time_s);
line too long (98 > 90)
http://gerrit.cloudera.org:8080/#/c/20986/9/be/src/catalog/catalog-server.cc@1008
PS9, Line 1008: progress_info_obj.AddMember("min_event_time", ToStringFromUnix(progress_info.min_event_time_s));
line too long (100 > 90)
http://gerrit.cloudera.org:8080/#/c/20986/9/be/src/catalog/catalog-server.cc@1009
PS9, Line 1009: progress_info_obj.AddMember("max_event_time", ToStringFromUnix(progress_info.max_event_time_s));
line too long (100 > 90)
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 9
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Thu, 14 Mar 2024 03:55:36 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Hello k.venureddy2103@gmail.com, Sai Hemanth Gantasala, Csaba Ringhofer, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/20986
to look at the new patch set (#8).
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
IMPALA-12782: Show info of the event processing in /events webUI
The /events page of catalogd shows the metrics and status of the
event-processor. This patch adds more info in this page, including
- lag info
- current event batch that's being processing
See the screenshot attached in the JIRA for how it looks like.
Also moves the error message to the top to highlight the error status.
Adds a debug action, catalogd_event_processing_delay, to inject a sleep
while processing an event. So the web page can be captured more easily.
Also adds a missing test for showing the error message of
event-processing in the /events page.
Tests:
- Add e2e test to verify the content of the page.
Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
---
M be/src/catalog/catalog-server.cc
M common/thrift/JniCatalog.thrift
M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java
M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java
M fe/src/main/java/org/apache/impala/util/DebugUtils.java
M tests/custom_cluster/test_web_pages.py
M www/events.tmpl
7 files changed, 296 insertions(+), 18 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/86/20986/8
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 8
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
k.venureddy2103@gmail.com has posted comments on this change. ( http://gerrit.cloudera.org:8080/20986 )
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
Patch Set 2:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/20986/2/common/thrift/JniCatalog.thrift
File common/thrift/JniCatalog.thrift:
http://gerrit.cloudera.org:8080/#/c/20986/2/common/thrift/JniCatalog.thrift@1066
PS2, Line 1066: 1: required bool is_current_event_batched
is_current_event_batched doesn't seem to be used ?
http://gerrit.cloudera.org:8080/#/c/20986/2/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java
File fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java:
http://gerrit.cloudera.org:8080/#/c/20986/2/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java@1130
PS2, Line 1130: progressInfo.current_batch_start_time_ms = currentBatchStartTimeMs_;
currentEventBatch_, currentFilteredEvents_, currentBatchStartTimeMs_, currentEventStartTimeMs_ and currentEventIndex_ values may not always have the state of current batch. I mean, when some variables of current batch are updated but others are not updated by the time of this execution. Lets say currentEventBatch_ alone is updated or may be currentEventBatch_ and currentBatchStartTimeMs_ are updated or currentEventBatch_, currentBatchStartTimeMs_ and currentFilteredEvents_ are updated and so on. We will have previous batch state for remaining variables.
So we may see some inconsistent values. For example, num_synced_events can be seen more than num_hms_events when the previous batch has more events than current batch and currentEventBatch_ variable alone is updated by the time of this execution Similarly other parameters such as synced_percent etc can also be inconsistent.
http://gerrit.cloudera.org:8080/#/c/20986/2/www/events.tmpl
File www/events.tmpl:
http://gerrit.cloudera.org:8080/#/c/20986/2/www/events.tmpl@59
PS2, Line 59: <th>EventId</th>
nit: Can we use the same style in both headers ? Event ID or EventId, Event Time or EventTime etc ?
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Wed, 14 Feb 2024 12:17:40 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/20986 )
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
Patch Set 9: Code-Review+2
(2 comments)
http://gerrit.cloudera.org:8080/#/c/20986/3/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java
File fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java:
http://gerrit.cloudera.org:8080/#/c/20986/3/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java@1192
PS3, Line 1192: * @param events List of NotificationEvents
I am ok with merging the current state, but I find it pretty confusing. Doing this in a synchronized way may not be really necessary but would make to logic easier to understand.
> and the recent 10 failures
I think that storing at least 1 failure is the most critical as generally there should be 0 failures.
A few more thoughts for storing failures:
- after IMPALA-12832 this part of the code would not see table level failures that were solved with invalidating the table. It would be nice to store those too
- besides storing the event the call stack would be also useful
http://gerrit.cloudera.org:8080/#/c/20986/9/www/events.tmpl
File www/events.tmpl:
http://gerrit.cloudera.org:8080/#/c/20986/9/www/events.tmpl@57
PS9, Line 57: synced
Maybe processed would be clearer?
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 9
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Thu, 14 Mar 2024 08:09:32 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20986 )
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
Patch Set 11:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/15528/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 11
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Fri, 15 Mar 2024 11:27:56 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Sai Hemanth Gantasala (Code Review)" <ge...@cloudera.org>.
Sai Hemanth Gantasala has posted comments on this change. ( http://gerrit.cloudera.org:8080/20986 )
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
Patch Set 12: Code-Review+1
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 12
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Mon, 01 Apr 2024 03:46:20 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20986 )
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
Patch Set 4: Verified-1
Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/10287/
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 4
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Wed, 21 Feb 2024 14:10:30 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/20986
to look at the new patch set (#2).
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
IMPALA-12782: Show info of the event processing in /events webUI
The /events page of catalogd shows the metrics and status of the
event-processor. This patch adds more info in this page, including
- lag info
- current event batch that's being processing
See the screenshot attached in the JIRA for how it looks like.
Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
---
M be/src/catalog/catalog-server.cc
M be/src/util/pretty-printer.h
M common/thrift/JniCatalog.thrift
M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java
M www/events.tmpl
5 files changed, 219 insertions(+), 9 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/86/20986/2
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20986 )
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
Patch Set 1:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/15147/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Fri, 02 Feb 2024 12:56:37 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20986 )
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
Patch Set 6: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 6
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Fri, 08 Mar 2024 06:49:29 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20986 )
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
Patch Set 13: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 13
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Tue, 02 Apr 2024 13:35:36 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/20986 )
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
IMPALA-12782: Show info of the event processing in /events webUI
The /events page of catalogd shows the metrics and status of the
event-processor. This patch adds more info in this page, including
- lag info
- current event batch that's being processed
See the screenshot attached in the JIRA for how it looks like.
Also moves the error message to the top to highlight the error status.
Fixes the issue of not updating latest event id when event processor is
stopped. Also fixes the issue of error message not cleared after global
INVALIDATE METADATA.
Adds a debug action, catalogd_event_processing_delay, to inject a sleep
while processing an event. So the web page can be captured more easily.
Also adds a missing test for showing the error message of
event-processing in the /events page.
Tests:
- Add e2e test to verify the content of the page.
Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Reviewed-on: http://gerrit.cloudera.org:8080/20986
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M be/src/catalog/catalog-server.cc
M be/src/util/json-util.h
M common/thrift/JniCatalog.thrift
M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java
M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java
M fe/src/main/java/org/apache/impala/util/DebugUtils.java
M tests/custom_cluster/test_web_pages.py
M www/events.tmpl
8 files changed, 311 insertions(+), 20 deletions(-)
Approvals:
Impala Public Jenkins: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 14
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20986 )
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
Patch Set 13: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 13
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Tue, 02 Apr 2024 18:40:25 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/20986 )
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
Patch Set 12: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 12
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Tue, 02 Apr 2024 13:26:40 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20986 )
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
Patch Set 1:
(31 comments)
http://gerrit.cloudera.org:8080/#/c/20986/1/be/src/catalog/catalog-server.cc
File be/src/catalog/catalog-server.cc:
http://gerrit.cloudera.org:8080/#/c/20986/1/be/src/catalog/catalog-server.cc@936
PS1, Line 936: const TEventBatchProgressInfo& progress_info = event_processor_summary_response.progress;
line too long (91 > 90)
http://gerrit.cloudera.org:8080/#/c/20986/1/be/src/catalog/catalog-server.cc@943
PS1, Line 943: progress_info_obj.AddMember("last_synced_event_id", last_synced_event_id, document->GetAllocator());
line too long (102 > 90)
http://gerrit.cloudera.org:8080/#/c/20986/1/be/src/catalog/catalog-server.cc@944
PS1, Line 944: progress_info_obj.AddMember("last_synced_event_time_s", last_synced_event_time_s, document->GetAllocator());
line too long (110 > 90)
http://gerrit.cloudera.org:8080/#/c/20986/1/be/src/catalog/catalog-server.cc@945
PS1, Line 945: progress_info_obj.AddMember("latest_event_id", latest_event_id, document->GetAllocator());
line too long (92 > 90)
http://gerrit.cloudera.org:8080/#/c/20986/1/be/src/catalog/catalog-server.cc@946
PS1, Line 946: progress_info_obj.AddMember("latest_event_time_s", latest_event_time_s, document->GetAllocator());
line too long (100 > 90)
http://gerrit.cloudera.org:8080/#/c/20986/1/be/src/catalog/catalog-server.cc@947
PS1, Line 947: int64_t lag_time = progress_info.latest_event_time_s - progress_info.last_synced_event_time_s;
line too long (96 > 90)
http://gerrit.cloudera.org:8080/#/c/20986/1/be/src/catalog/catalog-server.cc@948
PS1, Line 948: Value lag_time_str(PrettyPrinter::Print(lag_time, TUnit::TIME_S).c_str(), document->GetAllocator());
line too long (102 > 90)
http://gerrit.cloudera.org:8080/#/c/20986/1/be/src/catalog/catalog-server.cc@950
PS1, Line 950: Value last_synced_event_time(ToStringFromUnix(progress_info.last_synced_event_time_s).c_str(), document->GetAllocator());
line too long (123 > 90)
http://gerrit.cloudera.org:8080/#/c/20986/1/be/src/catalog/catalog-server.cc@951
PS1, Line 951: Value latest_event_time(ToStringFromUnix(progress_info.latest_event_time_s).c_str(), document->GetAllocator());
line too long (113 > 90)
http://gerrit.cloudera.org:8080/#/c/20986/1/be/src/catalog/catalog-server.cc@952
PS1, Line 952: progress_info_obj.AddMember("last_synced_event_time", last_synced_event_time, document->GetAllocator());
line too long (106 > 90)
http://gerrit.cloudera.org:8080/#/c/20986/1/be/src/catalog/catalog-server.cc@953
PS1, Line 953: progress_info_obj.AddMember("latest_event_time", latest_event_time, document->GetAllocator());
line too long (96 > 90)
http://gerrit.cloudera.org:8080/#/c/20986/1/be/src/catalog/catalog-server.cc@959
PS1, Line 959: Value synced_percent(100 * progress_info.current_event_index / progress_info.num_filtered_events);
line too long (102 > 90)
http://gerrit.cloudera.org:8080/#/c/20986/1/be/src/catalog/catalog-server.cc@962
PS1, Line 962: Value min_event_time(ToStringFromUnix(progress_info.min_event_time_s).c_str(), document->GetAllocator());
line too long (109 > 90)
http://gerrit.cloudera.org:8080/#/c/20986/1/be/src/catalog/catalog-server.cc@963
PS1, Line 963: Value max_event_time(ToStringFromUnix(progress_info.max_event_time_s).c_str(), document->GetAllocator());
line too long (109 > 90)
http://gerrit.cloudera.org:8080/#/c/20986/1/be/src/catalog/catalog-server.cc@964
PS1, Line 964: Value start_time(ToStringFromUnixMillis(progress_info.current_batch_start_time_ms).c_str(), document->GetAllocator());
line too long (122 > 90)
http://gerrit.cloudera.org:8080/#/c/20986/1/be/src/catalog/catalog-server.cc@965
PS1, Line 965: Value start_time_of_event(ToStringFromUnixMillis(progress_info.current_event_start_time_ms).c_str(), document->GetAllocator());
line too long (131 > 90)
http://gerrit.cloudera.org:8080/#/c/20986/1/be/src/catalog/catalog-server.cc@967
PS1, Line 967: Value elapsed_time(PrettyPrinter::Print(now_ms - progress_info.current_batch_start_time_ms, TUnit::TIME_MS).c_str(), document->GetAllocator());
line too long (147 > 90)
http://gerrit.cloudera.org:8080/#/c/20986/1/be/src/catalog/catalog-server.cc@968
PS1, Line 968: Value elapsed_time_current_event(PrettyPrinter::Print(now_ms - progress_info.current_event_start_time_ms, TUnit::TIME_MS).c_str(), document->GetAllocator());
line too long (161 > 90)
http://gerrit.cloudera.org:8080/#/c/20986/1/be/src/catalog/catalog-server.cc@969
PS1, Line 969: progress_info_obj.AddMember("num_hms_events", num_hms_events, document->GetAllocator());
line too long (92 > 90)
http://gerrit.cloudera.org:8080/#/c/20986/1/be/src/catalog/catalog-server.cc@970
PS1, Line 970: progress_info_obj.AddMember("num_filtered_events", num_filtered_events, document->GetAllocator());
line too long (102 > 90)
http://gerrit.cloudera.org:8080/#/c/20986/1/be/src/catalog/catalog-server.cc@971
PS1, Line 971: progress_info_obj.AddMember("num_synced_events", num_synced_events, document->GetAllocator());
line too long (98 > 90)
http://gerrit.cloudera.org:8080/#/c/20986/1/be/src/catalog/catalog-server.cc@972
PS1, Line 972: progress_info_obj.AddMember("synced_percent", synced_percent, document->GetAllocator());
line too long (92 > 90)
http://gerrit.cloudera.org:8080/#/c/20986/1/be/src/catalog/catalog-server.cc@975
PS1, Line 975: progress_info_obj.AddMember("min_event_time", min_event_time, document->GetAllocator());
line too long (92 > 90)
http://gerrit.cloudera.org:8080/#/c/20986/1/be/src/catalog/catalog-server.cc@976
PS1, Line 976: progress_info_obj.AddMember("max_event_time", max_event_time, document->GetAllocator());
line too long (92 > 90)
http://gerrit.cloudera.org:8080/#/c/20986/1/be/src/catalog/catalog-server.cc@979
PS1, Line 979: progress_info_obj.AddMember("start_time_of_event", start_time_of_event, document->GetAllocator());
line too long (102 > 90)
http://gerrit.cloudera.org:8080/#/c/20986/1/be/src/catalog/catalog-server.cc@980
PS1, Line 980: progress_info_obj.AddMember("elapsed_time_current_event", elapsed_time_current_event, document->GetAllocator());
line too long (116 > 90)
http://gerrit.cloudera.org:8080/#/c/20986/1/be/src/catalog/catalog-server.cc@984
PS1, Line 984: Value event_time(ToStringFromUnix(progress_info.current_event.eventTime).c_str(), document->GetAllocator());
line too long (114 > 90)
http://gerrit.cloudera.org:8080/#/c/20986/1/be/src/catalog/catalog-server.cc@985
PS1, Line 985: Value event_type(progress_info.current_event.eventType.c_str(), document->GetAllocator());
line too long (96 > 90)
http://gerrit.cloudera.org:8080/#/c/20986/1/be/src/catalog/catalog-server.cc@986
PS1, Line 986: Value cat_name(progress_info.current_event.catName.c_str(), document->GetAllocator());
line too long (92 > 90)
http://gerrit.cloudera.org:8080/#/c/20986/1/be/src/catalog/catalog-server.cc@988
PS1, Line 988: Value tbl_name(progress_info.current_event.tableName.c_str(), document->GetAllocator());
line too long (94 > 90)
http://gerrit.cloudera.org:8080/#/c/20986/1/be/src/catalog/catalog-server.cc@995
PS1, Line 995: progress_info_obj.AddMember("current_event", current_event, document->GetAllocator());
line too long (92 > 90)
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Fri, 02 Feb 2024 12:31:14 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Sai Hemanth Gantasala (Code Review)" <ge...@cloudera.org>.
Sai Hemanth Gantasala has posted comments on this change. ( http://gerrit.cloudera.org:8080/20986 )
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
Patch Set 6: Code-Review+1
(3 comments)
http://gerrit.cloudera.org:8080/#/c/20986/6/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java
File fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java:
http://gerrit.cloudera.org:8080/#/c/20986/6/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java@1234
PS6, Line 1234: logEventMetrics(eventProcessingTime, elapsedNs);
Nit: we could probably remove L#1195 and L#1226 and add that here.
http://gerrit.cloudera.org:8080/#/c/20986/6/www/events.tmpl
File www/events.tmpl:
http://gerrit.cloudera.org:8080/#/c/20986/6/www/events.tmpl@24
PS6, Line 24: <pre>{{event_processor_error_msg}}</pre>
nit: will we have event-id and decompressed event message in the event message.
http://gerrit.cloudera.org:8080/#/c/20986/6/www/events.tmpl@43
PS6, Line 43: <td>Latest Event</td>
nit: should we say "Latest Event in Metastore"?
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 6
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 Mar 2024 00:27:23 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20986 )
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
Patch Set 7:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/15497/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 7
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 Mar 2024 12:02:27 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20986 )
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
Patch Set 10:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/20986/10/be/src/catalog/catalog-server.cc
File be/src/catalog/catalog-server.cc:
http://gerrit.cloudera.org:8080/#/c/20986/10/be/src/catalog/catalog-server.cc@982
PS10, Line 982: progress_info_obj.AddMember("last_synced_event_time_s", progress_info.last_synced_event_time_s);
line too long (98 > 90)
http://gerrit.cloudera.org:8080/#/c/20986/10/be/src/catalog/catalog-server.cc@1008
PS10, Line 1008: progress_info_obj.AddMember("min_event_time", ToStringFromUnix(progress_info.min_event_time_s));
line too long (100 > 90)
http://gerrit.cloudera.org:8080/#/c/20986/10/be/src/catalog/catalog-server.cc@1009
PS10, Line 1009: progress_info_obj.AddMember("max_event_time", ToStringFromUnix(progress_info.max_event_time_s));
line too long (100 > 90)
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 10
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Fri, 15 Mar 2024 10:30:02 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Sai Hemanth Gantasala (Code Review)" <ge...@cloudera.org>.
Sai Hemanth Gantasala has posted comments on this change. ( http://gerrit.cloudera.org:8080/20986 )
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
Patch Set 2: Code-Review+1
(2 comments)
http://gerrit.cloudera.org:8080/#/c/20986/2/www/events.tmpl
File www/events.tmpl:
http://gerrit.cloudera.org:8080/#/c/20986/2/www/events.tmpl@53
PS2, Line 53: Start processing the current batch at {{progress-info.start_time}} ({{progress-info.elapsed_time}} ago).</br>
Nit: does 'Started processing' sound good?
http://gerrit.cloudera.org:8080/#/c/20986/2/www/events.tmpl@54
PS2, Line 54: Start processing the current event at {{progress-info.start_time_of_event}} ({{progress-info.elapsed_time_current_event}} ago).
Nit: Should we also show the current event ID here?
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Tue, 13 Feb 2024 23:38:12 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
k.venureddy2103@gmail.com has posted comments on this change. ( http://gerrit.cloudera.org:8080/20986 )
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
Patch Set 3:
(5 comments)
http://gerrit.cloudera.org:8080/#/c/20986/3/be/src/catalog/catalog-server.cc
File be/src/catalog/catalog-server.cc:
http://gerrit.cloudera.org:8080/#/c/20986/3/be/src/catalog/catalog-server.cc@964
PS3, Line 964: 100 * progress_info.current_event_index / progress_info.num_filtered_events);
There is a possibility of progress_info.num_filtered_events being 0.
http://gerrit.cloudera.org:8080/#/c/20986/3/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java
File fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java:
http://gerrit.cloudera.org:8080/#/c/20986/3/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java@678
PS3, Line 678: throw new RuntimeException(e);
nit: What is your opinion about just logging at this InterruptedException instead of rethrowing RuntimeException?
http://gerrit.cloudera.org:8080/#/c/20986/3/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java
File fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java:
http://gerrit.cloudera.org:8080/#/c/20986/3/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java@1137
PS3, Line 1137: progressInfo.current_event_batch_size = currentFilteredEvent_.getNumberOfEvents();
There is a possibility of currentFilteredEvent_ being null. Recommend a check.
http://gerrit.cloudera.org:8080/#/c/20986/3/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java@1192
PS3, Line 1192: return;
Recommend resetProgress() here as well.
http://gerrit.cloudera.org:8080/#/c/20986/3/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java@1194
PS3, Line 1194: currentEventIndex_ = 0;
nit: Need not set currentEventIndex_ explicitly to 0 now.
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Wed, 21 Feb 2024 06:25:38 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/20986 )
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
Patch Set 4:
(6 comments)
http://gerrit.cloudera.org:8080/#/c/20986/3/be/src/catalog/catalog-server.cc
File be/src/catalog/catalog-server.cc:
http://gerrit.cloudera.org:8080/#/c/20986/3/be/src/catalog/catalog-server.cc@964
PS3, Line 964: if (progress_info.num_filtered_events > 0) {
> There is a possibility of progress_info.num_filtered_events being 0.
Nice catch!
http://gerrit.cloudera.org:8080/#/c/20986/2/common/thrift/JniCatalog.thrift
File common/thrift/JniCatalog.thrift:
http://gerrit.cloudera.org:8080/#/c/20986/2/common/thrift/JniCatalog.thrift@1066
PS2, Line 1066: }
> Nice catch! I planned to show the MetastoreEvent at the beginning but decid
In patch set 3, I finally replace this field with 'current_event_batch_size' and show it in the page.
http://gerrit.cloudera.org:8080/#/c/20986/3/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java
File fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java:
http://gerrit.cloudera.org:8080/#/c/20986/3/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java@678
PS3, Line 678: LOG.info("Ignored exception", e);
> nit: What is your opinion about just logging at this InterruptedException i
Removed this.
http://gerrit.cloudera.org:8080/#/c/20986/3/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java
File fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java:
http://gerrit.cloudera.org:8080/#/c/20986/3/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java@1137
PS3, Line 1137: progressInfo.current_event_batch_size = currentFilteredEvent_ != null ?
> There is a possibility of currentFilteredEvent_ being null. Recommend a che
Nice catch!
http://gerrit.cloudera.org:8080/#/c/20986/3/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java@1192
PS3, Line 1192: lastSyncedEventTimeSecs_.set(e.getEventTime());
> Recommend resetProgress() here as well.
Nice catch!
http://gerrit.cloudera.org:8080/#/c/20986/3/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java@1194
PS3, Line 1194: return;
> nit: Need not set currentEventIndex_ explicitly to 0 now.
Done
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 4
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Wed, 21 Feb 2024 08:41:14 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Hello k.venureddy2103@gmail.com, Sai Hemanth Gantasala, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/20986
to look at the new patch set (#6).
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
IMPALA-12782: Show info of the event processing in /events webUI
The /events page of catalogd shows the metrics and status of the
event-processor. This patch adds more info in this page, including
- lag info
- current event batch that's being processing
See the screenshot attached in the JIRA for how it looks like.
Also moves the error message to the top to highlight the error status.
Adds a debug action, catalogd_event_processing_delay, to inject a sleep
while processing an event. So the web page can be captured more easily.
Tests:
- Add e2e test to verify the content of the page.
Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
---
M be/src/catalog/catalog-server.cc
M common/thrift/JniCatalog.thrift
M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java
M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java
M fe/src/main/java/org/apache/impala/util/DebugUtils.java
M tests/custom_cluster/test_web_pages.py
M www/events.tmpl
7 files changed, 279 insertions(+), 18 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/86/20986/6
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 6
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20986 )
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
Patch Set 6:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/10365/ DRY_RUN=true
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 6
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Fri, 08 Mar 2024 02:10:40 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/20986 )
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
Patch Set 3:
(5 comments)
http://gerrit.cloudera.org:8080/#/c/20986/2/common/thrift/JniCatalog.thrift
File common/thrift/JniCatalog.thrift:
http://gerrit.cloudera.org:8080/#/c/20986/2/common/thrift/JniCatalog.thrift@1066
PS2, Line 1066: }
> is_current_event_batched doesn't seem to be used ?
Nice catch! I planned to show the MetastoreEvent at the beginning but decided to leave it as a follow-up task. So currently only the original HMS event is shown. We can delete this field.
http://gerrit.cloudera.org:8080/#/c/20986/2/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java
File fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java:
http://gerrit.cloudera.org:8080/#/c/20986/2/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java@1130
PS2, Line 1130: progressInfo.max_event_time_s = lastEvent.getEventTime();
> currentEventBatch_, currentFilteredEvents_, currentBatchStartTimeMs_, curre
Yeah, I should have mentioned this somewhere. There are costs to provide consistency in metrics. E.g. if we use a mutex (synchronized) to protect these fields, the web requests could block the event processor. We already have inconsistent metrics, e.g. lastSyncedEventId_ and lastSyncedEventTimeSecs_ could be inconsistent since the updates on them are not atomic.
So currently we are best effort to provide consistency on metrics but cannot garuantee it. E.g. we garuantee the consistency between num_hms_events, min_event_*, max_event* by using the snapshot of currentEventBatch_, But they could be inconsistent with other metrics like num_synced_events.
I added some codes to reset these fields to reduce the possibility of seeing inconsistent metrics. Another way to fix this is wrapping these metrics into a snapshot class and create new instance whenever any metric changes. I think it doesn't worth adding this complexity since the web page is mainly used by admins when event processing is slow. In such situation, the metrics are also updated slowly.
http://gerrit.cloudera.org:8080/#/c/20986/2/www/events.tmpl
File www/events.tmpl:
http://gerrit.cloudera.org:8080/#/c/20986/2/www/events.tmpl@53
PS2, Line 53: Started processing the current batch at {{progress-info.start_time}} ({{progress-info.elapsed_time}} ago).</br>
> Nit: does 'Started processing' sound good?
Done
http://gerrit.cloudera.org:8080/#/c/20986/2/www/events.tmpl@54
PS2, Line 54: Started processing the current event at {{progress-info.start_time_of_event}} ({{progress-info.elapsed_time_current_event}} ago).
> Nit: Should we also show the current event ID here?
It's shown in the table below. Maybe it's ok to skip it here.
http://gerrit.cloudera.org:8080/#/c/20986/2/www/events.tmpl@59
PS2, Line 59: {{/progress-info.current_event_batch_size}}
> nit: Can we use the same style in both headers ? Event ID or EventId, Event
Done
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Tue, 20 Feb 2024 12:14:54 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20986 )
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
Patch Set 4:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/15261/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 4
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Wed, 21 Feb 2024 09:07:17 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20986 )
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
Patch Set 5: Verified-1
Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/10290/
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 5
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Thu, 22 Feb 2024 10:14:46 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20986 )
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
Patch Set 5:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/15270/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 5
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Thu, 22 Feb 2024 00:16:33 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20986 )
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
Patch Set 8:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/15498/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 8
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 Mar 2024 13:51:11 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
k.venureddy2103@gmail.com has posted comments on this change. ( http://gerrit.cloudera.org:8080/20986 )
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
Patch Set 6: Code-Review+1
(1 comment)
Looks good to me. Just have one comment.
http://gerrit.cloudera.org:8080/#/c/20986/3/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java
File fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java:
http://gerrit.cloudera.org:8080/#/c/20986/3/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java@1192
PS3, Line 1192: NotificationEvent e = events.get(events.size() - 1);
> Nice catch!
Sorry i didn't realise that resetProgress() was called in finally. It is not required to call resetProgress() here.
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 6
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 Mar 2024 04:57:17 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20986 )
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
Patch Set 9:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/15507/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 9
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Thu, 14 Mar 2024 04:17:45 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20986 )
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
Patch Set 12:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/15549/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 12
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Sun, 17 Mar 2024 08:23:30 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/20986 )
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
Patch Set 11:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/20986/10/be/src/catalog/catalog-server.cc
File be/src/catalog/catalog-server.cc:
http://gerrit.cloudera.org:8080/#/c/20986/10/be/src/catalog/catalog-server.cc@982
PS10, Line 982: progress_info_obj.AddMember("last_synced_event_time_s",
> line too long (98 > 90)
Done
http://gerrit.cloudera.org:8080/#/c/20986/10/be/src/catalog/catalog-server.cc@1008
PS10, Line 1008: progress_info_obj.AddMember("max_event_id", progress_info.max_event_id);
> line too long (100 > 90)
Done
http://gerrit.cloudera.org:8080/#/c/20986/10/be/src/catalog/catalog-server.cc@1009
PS10, Line 1009: progress_info_obj.AddMember("min_event_time",
> line too long (100 > 90)
Done
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 11
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Fri, 15 Mar 2024 11:04:54 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Hello k.venureddy2103@gmail.com, Sai Hemanth Gantasala, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/20986
to look at the new patch set (#5).
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
IMPALA-12782: Show info of the event processing in /events webUI
The /events page of catalogd shows the metrics and status of the
event-processor. This patch adds more info in this page, including
- lag info
- current event batch that's being processing
See the screenshot attached in the JIRA for how it looks like.
Also moves the error message to the top to highlight the error status.
Adds a debug flag, event_processing_delay_s, to inject a sleep while
processing an event. So the web page can be captured more easily.
Tests:
- Add e2e test to verify the content of the page.
Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
---
M be/src/catalog/catalog-server.cc
M be/src/common/global-flags.cc
M be/src/util/backend-gflag-util.cc
M common/thrift/BackendGflags.thrift
M common/thrift/JniCatalog.thrift
M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java
M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
M tests/custom_cluster/test_web_pages.py
M www/events.tmpl
10 files changed, 291 insertions(+), 19 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/86/20986/5
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 5
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Hello k.venureddy2103@gmail.com, Sai Hemanth Gantasala, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/20986
to look at the new patch set (#3).
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
IMPALA-12782: Show info of the event processing in /events webUI
The /events page of catalogd shows the metrics and status of the
event-processor. This patch adds more info in this page, including
- lag info
- current event batch that's being processing
See the screenshot attached in the JIRA for how it looks like.
Adds a debug flag, event_processing_delay_s, to inject a sleep while
processing an event. So the web page can be captured more easily.
Tests:
- Add e2e test to verify the content of the page.
Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
---
M be/src/catalog/catalog-server.cc
M be/src/common/global-flags.cc
M be/src/util/backend-gflag-util.cc
M be/src/util/pretty-printer.h
M common/thrift/BackendGflags.thrift
M common/thrift/JniCatalog.thrift
M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java
M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
M tests/custom_cluster/test_web_pages.py
M www/events.tmpl
11 files changed, 284 insertions(+), 19 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/86/20986/3
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20986 )
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
Patch Set 4:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/10287/ DRY_RUN=true
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 4
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Wed, 21 Feb 2024 09:38:01 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/20986 )
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
Patch Set 5:
> Patch Set 4: Verified-1
>
> Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/10287/
Reverted changes in pretty-printer.h to fix the test failures. We can improve the timestamp string in a separate patch.
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 5
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Wed, 21 Feb 2024 23:51:24 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Hello k.venureddy2103@gmail.com, Sai Hemanth Gantasala, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/20986
to look at the new patch set (#4).
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
IMPALA-12782: Show info of the event processing in /events webUI
The /events page of catalogd shows the metrics and status of the
event-processor. This patch adds more info in this page, including
- lag info
- current event batch that's being processing
See the screenshot attached in the JIRA for how it looks like.
Also moves the error message to the top to highlight the error status.
Adds a debug flag, event_processing_delay_s, to inject a sleep while
processing an event. So the web page can be captured more easily.
Tests:
- Add e2e test to verify the content of the page.
Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
---
M be/src/catalog/catalog-server.cc
M be/src/common/global-flags.cc
M be/src/util/backend-gflag-util.cc
M be/src/util/pretty-printer.h
M common/thrift/BackendGflags.thrift
M common/thrift/JniCatalog.thrift
M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java
M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
M tests/custom_cluster/test_web_pages.py
M www/events.tmpl
11 files changed, 294 insertions(+), 22 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/86/20986/4
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 4
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20986 )
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
Patch Set 2:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/15148/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Fri, 02 Feb 2024 13:08:11 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Hello k.venureddy2103@gmail.com, Sai Hemanth Gantasala, Csaba Ringhofer, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/20986
to look at the new patch set (#12).
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
IMPALA-12782: Show info of the event processing in /events webUI
The /events page of catalogd shows the metrics and status of the
event-processor. This patch adds more info in this page, including
- lag info
- current event batch that's being processed
See the screenshot attached in the JIRA for how it looks like.
Also moves the error message to the top to highlight the error status.
Fixes the issue of not updating latest event id when event processor is
stopped. Also fixes the issue of error message not cleared after global
INVALIDATE METADATA.
Adds a debug action, catalogd_event_processing_delay, to inject a sleep
while processing an event. So the web page can be captured more easily.
Also adds a missing test for showing the error message of
event-processing in the /events page.
Tests:
- Add e2e test to verify the content of the page.
Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
---
M be/src/catalog/catalog-server.cc
M be/src/util/json-util.h
M common/thrift/JniCatalog.thrift
M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java
M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java
M fe/src/main/java/org/apache/impala/util/DebugUtils.java
M tests/custom_cluster/test_web_pages.py
M www/events.tmpl
8 files changed, 311 insertions(+), 20 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/86/20986/12
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 12
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Hello k.venureddy2103@gmail.com, Sai Hemanth Gantasala, Csaba Ringhofer, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/20986
to look at the new patch set (#7).
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
IMPALA-12782: Show info of the event processing in /events webUI
The /events page of catalogd shows the metrics and status of the
event-processor. This patch adds more info in this page, including
- lag info
- current event batch that's being processing
See the screenshot attached in the JIRA for how it looks like.
Also moves the error message to the top to highlight the error status.
Adds a debug action, catalogd_event_processing_delay, to inject a sleep
while processing an event. So the web page can be captured more easily.
Tests:
- Add e2e test to verify the content of the page.
Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
---
M be/src/catalog/catalog-server.cc
M common/thrift/JniCatalog.thrift
M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java
M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java
M fe/src/main/java/org/apache/impala/util/DebugUtils.java
M tests/custom_cluster/test_web_pages.py
M www/events.tmpl
7 files changed, 279 insertions(+), 18 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/86/20986/7
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 7
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/20986 )
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
Patch Set 7:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/20986/3/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java
File fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java:
http://gerrit.cloudera.org:8080/#/c/20986/3/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java@1192
PS3, Line 1192: * @throws MetastoreNotificationException
> Sorry i didn't realise that resetProgress() was called in finally. It is no
I remove calling it in finally since it will reset currentEvent_ for exceptions which causes no event details shown in the error message. So we still need this here.
http://gerrit.cloudera.org:8080/#/c/20986/6/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java
File fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java:
http://gerrit.cloudera.org:8080/#/c/20986/6/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java@1234
PS6, Line 1234: String desc = String.format("Processing %s on %s, eventId=%d",
> Nit: we could probably remove L#1195 and L#1226 and add that here.
It's intended to not resetting the progress info when hitting exceptions. So currentEvent_ won't be null and can be used in the error message at L1001.
http://gerrit.cloudera.org:8080/#/c/20986/6/www/events.tmpl
File www/events.tmpl:
http://gerrit.cloudera.org:8080/#/c/20986/6/www/events.tmpl@24
PS6, Line 24: <pre>{{event_processor_error_msg}}</pre>
> nit: will we have event-id and decompressed event message in the event mess
Yes but with compressed event message. There is an example in IMPALA-12053. This is an existing feature added by IMPALA-12053. Just move it to the top to highlight the error.
http://gerrit.cloudera.org:8080/#/c/20986/6/www/events.tmpl@43
PS6, Line 43: <td>Latest Event in Metastore</td>
> nit: should we say "Latest Event in Metastore"?
Done
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 7
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 Mar 2024 11:39:32 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12782: Show info of the event processing in /events webUI
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20986 )
Change subject: IMPALA-12782: Show info of the event processing in /events webUI
......................................................................
Patch Set 13:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/10480/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/20986
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I2e7d4952c7fd04ae89b6751204499bf9dd99f57c
Gerrit-Change-Number: 20986
Gerrit-PatchSet: 13
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Tue, 02 Apr 2024 13:35:37 +0000
Gerrit-HasComments: No