You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Sai Hemanth Gantasala (Code Review)" <ge...@cloudera.org> on 2025/11/19 01:03:21 UTC
[Impala-ASF-CR] IMPALA-14562: Enable Hierarchical event processing by default
Sai Hemanth Gantasala has uploaded this change for review. ( http://gerrit.cloudera.org:8080/23687
Change subject: IMPALA-14562: Enable Hierarchical event processing by default
......................................................................
IMPALA-14562: Enable Hierarchical event processing by default
Change-Id: Ie9a28f863ef17456817e0a335215450e514b1f5b
---
M be/src/catalog/catalog-server.cc
1 file changed, 1 insertion(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/87/23687/1
--
To view, visit http://gerrit.cloudera.org:8080/23687
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ie9a28f863ef17456817e0a335215450e514b1f5b
Gerrit-Change-Number: 23687
Gerrit-PatchSet: 1
Gerrit-Owner: Sai Hemanth Gantasala <sa...@cloudera.com>
[Impala-ASF-CR] IMPALA-14562: Enable Hierarchical event processing by default
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/23687 )
Change subject: IMPALA-14562: Enable Hierarchical event processing by default
......................................................................
Patch Set 3: Code-Review+1
--
To view, visit http://gerrit.cloudera.org:8080/23687
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie9a28f863ef17456817e0a335215450e514b1f5b
Gerrit-Change-Number: 23687
Gerrit-PatchSet: 3
Gerrit-Owner: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Tue, 02 Dec 2025 03:08:38 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14562: Enable Hierarchical event processing by default
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23687 )
Change subject: IMPALA-14562: Enable Hierarchical event processing by default
......................................................................
Patch Set 4: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/23687
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie9a28f863ef17456817e0a335215450e514b1f5b
Gerrit-Change-Number: 23687
Gerrit-PatchSet: 4
Gerrit-Owner: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Thu, 04 Dec 2025 07:02:16 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14562: Enable Hierarchical event processing by default
Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
k.venureddy2103@gmail.com has posted comments on this change. ( http://gerrit.cloudera.org:8080/23687 )
Change subject: IMPALA-14562: Enable Hierarchical event processing by default
......................................................................
Patch Set 4: Code-Review+1
--
To view, visit http://gerrit.cloudera.org:8080/23687
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie9a28f863ef17456817e0a335215450e514b1f5b
Gerrit-Change-Number: 23687
Gerrit-PatchSet: 4
Gerrit-Owner: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Wed, 03 Dec 2025 17:41:16 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14562: Enable Hierarchical event processing by default
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/23687 )
Change subject: IMPALA-14562: Enable Hierarchical event processing by default
......................................................................
IMPALA-14562: Enable Hierarchical event processing by default
IMPALA-12709 Added support for hierarchical metastore event processing.
This commit enables hierarchical event processing by default.
hms_event_polling_interval_s can now be set to decimal value (eg: 0.5)
to support millisecond precision interval. Along with that others
configs can be fine tuned, such as:
num_db_event_executors: To set the number of database level event
executors.
num_table_event_executors_per_db_event_executor: To set the number of
table level event executors within a database event executor.
min_event_processor_idle_ms: To set the minimum time to retain idle db
processors and table processors.
max_outstanding_events_on_executors: To set the limit of maximum
outstanding events to process on event executors.
Testing:
- All the testing required to enable this flag is done in IMPALA-12709
and IMPALA-13801.
Change-Id: Ie9a28f863ef17456817e0a335215450e514b1f5b
Reviewed-on: http://gerrit.cloudera.org:8080/23687
Reviewed-by: <k....@gmail.com>
Reviewed-by: Quanlong Huang <hu...@gmail.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M be/src/catalog/catalog-server.cc
M be/src/common/global-flags.cc
2 files changed, 2 insertions(+), 2 deletions(-)
Approvals:
k.venureddy2103@gmail.com: Looks good to me, but someone else must approve
Quanlong Huang: Looks good to me, approved
Impala Public Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/23687
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ie9a28f863ef17456817e0a335215450e514b1f5b
Gerrit-Change-Number: 23687
Gerrit-PatchSet: 5
Gerrit-Owner: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
[Impala-ASF-CR] IMPALA-14562: Enable Hierarchical event processing by default
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23687 )
Change subject: IMPALA-14562: Enable Hierarchical event processing by default
......................................................................
Patch Set 3:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/21061/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/23687
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie9a28f863ef17456817e0a335215450e514b1f5b
Gerrit-Change-Number: 23687
Gerrit-PatchSet: 3
Gerrit-Owner: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Tue, 02 Dec 2025 00:11:39 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14562: Enable Hierarchical event processing by default
Posted by "Sai Hemanth Gantasala (Code Review)" <ge...@cloudera.org>.
Hello Quanlong Huang, Fang-Yu Rao, k.venureddy2103@gmail.com, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/23687
to look at the new patch set (#4).
Change subject: IMPALA-14562: Enable Hierarchical event processing by default
......................................................................
IMPALA-14562: Enable Hierarchical event processing by default
IMPALA-12709 Added support for hierarchical metastore event processing.
This commit enables hierarchical event processing by default.
hms_event_polling_interval_s can now be set to decimal value (eg: 0.5)
to support millisecond precision interval. Along with that others
configs can be fine tuned, such as:
num_db_event_executors: To set the number of database level event
executors.
num_table_event_executors_per_db_event_executor: To set the number of
table level event executors within a database event executor.
min_event_processor_idle_ms: To set the minimum time to retain idle db
processors and table processors.
max_outstanding_events_on_executors: To set the limit of maximum
outstanding events to process on event executors.
Testing:
- All the testing required to enable this flag is done in IMPALA-12709
and IMPALA-13801.
Change-Id: Ie9a28f863ef17456817e0a335215450e514b1f5b
---
M be/src/catalog/catalog-server.cc
M be/src/common/global-flags.cc
2 files changed, 2 insertions(+), 2 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/87/23687/4
--
To view, visit http://gerrit.cloudera.org:8080/23687
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie9a28f863ef17456817e0a335215450e514b1f5b
Gerrit-Change-Number: 23687
Gerrit-PatchSet: 4
Gerrit-Owner: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
[Impala-ASF-CR] IMPALA-14562: Enable Hierarchical event processing by default
Posted by "Sai Hemanth Gantasala (Code Review)" <ge...@cloudera.org>.
Hello Quanlong Huang, Fang-Yu Rao, k.venureddy2103@gmail.com, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/23687
to look at the new patch set (#3).
Change subject: IMPALA-14562: Enable Hierarchical event processing by default
......................................................................
IMPALA-14562: Enable Hierarchical event processing by default
IMPALA-12709 Added support for hierarchical metastore event processing.
This commit enables hierarchical event processing by default.
hms_event_polling_interval_s can now be set to decimal value (eg: 0.2)
to support millisecond precision interval. Along with that others
configs can be fine tuned, such as:
num_db_event_executors: To set the number of database level event
executors.
num_table_event_executors_per_db_event_executor: To set the number of
table level event executors within a database event executor.
min_event_processor_idle_ms: To set the minimum time to retain idle db
processors and table processors.
max_outstanding_events_on_executors: To set the limit of maximum
outstanding events to process on event executors.
Testing:
- All the testing required to enable this flag is done in IMPALA-12709
and IMPALA-13801.
Change-Id: Ie9a28f863ef17456817e0a335215450e514b1f5b
---
M be/src/catalog/catalog-server.cc
M be/src/common/global-flags.cc
2 files changed, 2 insertions(+), 2 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/87/23687/3
--
To view, visit http://gerrit.cloudera.org:8080/23687
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie9a28f863ef17456817e0a335215450e514b1f5b
Gerrit-Change-Number: 23687
Gerrit-PatchSet: 3
Gerrit-Owner: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
[Impala-ASF-CR] IMPALA-14562: Enable Hierarchical event processing by default
Posted by "Sai Hemanth Gantasala (Code Review)" <ge...@cloudera.org>.
Sai Hemanth Gantasala has posted comments on this change. ( http://gerrit.cloudera.org:8080/23687 )
Change subject: IMPALA-14562: Enable Hierarchical event processing by default
......................................................................
Patch Set 3:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/23687/2/be/src/catalog/catalog-server.cc
File be/src/catalog/catalog-server.cc:
http://gerrit.cloudera.org:8080/#/c/23687/2/be/src/catalog/catalog-server.cc@261
PS2, Line 261: DEFINE_bool(enable_hierarchical_event_processing, true,
> ok. We can calibrate later.
Ack
--
To view, visit http://gerrit.cloudera.org:8080/23687
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie9a28f863ef17456817e0a335215450e514b1f5b
Gerrit-Change-Number: 23687
Gerrit-PatchSet: 3
Gerrit-Owner: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Mon, 01 Dec 2025 23:46:01 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-14562: Enable Hierarchical event processing by default
Posted by "Sai Hemanth Gantasala (Code Review)" <ge...@cloudera.org>.
Hello Quanlong Huang, Fang-Yu Rao, k.venureddy2103@gmail.com, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/23687
to look at the new patch set (#2).
Change subject: IMPALA-14562: Enable Hierarchical event processing by default
......................................................................
IMPALA-14562: Enable Hierarchical event processing by default
IMPALA-12709 Added support for hierarchical metastore event processing.
This commit enables hierarchical event processing by default.
hms_event_polling_interval_s can now be set to decimal value (eg: 0.2)
to support millisecond precision interval. Along with that others
configs can be fine tuned, such as:
num_db_event_executors: To set the number of database level event
executors.
num_table_event_executors_per_db_event_executor: To set the number of
table level event executors within a database event executor.
min_event_processor_idle_ms: To set the minimum time to retain idle db
processors and table processors.
max_outstanding_events_on_executors: To set the limit of maximum
outstanding events to process on event executors.
Testing:
- All the testing required to enable this flag is done in IMPALA-12709
and IMPALA-13801.
Change-Id: Ie9a28f863ef17456817e0a335215450e514b1f5b
---
M be/src/catalog/catalog-server.cc
1 file changed, 1 insertion(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/87/23687/2
--
To view, visit http://gerrit.cloudera.org:8080/23687
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie9a28f863ef17456817e0a335215450e514b1f5b
Gerrit-Change-Number: 23687
Gerrit-PatchSet: 2
Gerrit-Owner: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
[Impala-ASF-CR] IMPALA-14562: Enable Hierarchical event processing by default
Posted by "Sai Hemanth Gantasala (Code Review)" <ge...@cloudera.org>.
Sai Hemanth Gantasala has posted comments on this change. ( http://gerrit.cloudera.org:8080/23687 )
Change subject: IMPALA-14562: Enable Hierarchical event processing by default
......................................................................
Patch Set 4:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/23687/3//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/23687/3//COMMIT_MSG@12
PS3, Line 12: hms_event_polling_interval_s can now be set to decimal value (eg: 0.5)
> You would want to change this value in commit message from 0.2 to 0.5 ?
Ack
--
To view, visit http://gerrit.cloudera.org:8080/23687
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie9a28f863ef17456817e0a335215450e514b1f5b
Gerrit-Change-Number: 23687
Gerrit-PatchSet: 4
Gerrit-Owner: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Wed, 03 Dec 2025 17:40:02 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-14562: Enable Hierarchical event processing by default
Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
k.venureddy2103@gmail.com has posted comments on this change. ( http://gerrit.cloudera.org:8080/23687 )
Change subject: IMPALA-14562: Enable Hierarchical event processing by default
......................................................................
Patch Set 3: Code-Review+1
(1 comment)
http://gerrit.cloudera.org:8080/#/c/23687/3//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/23687/3//COMMIT_MSG@12
PS3, Line 12: hms_event_polling_interval_s can now be set to decimal value (eg: 0.2)
You would want to change this value in commit message from 0.2 to 0.5 ?
--
To view, visit http://gerrit.cloudera.org:8080/23687
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie9a28f863ef17456817e0a335215450e514b1f5b
Gerrit-Change-Number: 23687
Gerrit-PatchSet: 3
Gerrit-Owner: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Wed, 03 Dec 2025 05:39:45 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-14562: Enable Hierarchical event processing by default
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/23687 )
Change subject: IMPALA-14562: Enable Hierarchical event processing by default
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/23687/1//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/23687/1//COMMIT_MSG@8
PS1, Line 8:
Please add some commit message like https://gerrit.cloudera.org/c/17612/
E.g. what configs should be changed together, what tests have been done.
Should we set --hms_event_polling_interval_s=0.2 like we do in the test?
https://github.com/apache/impala/blob/54c0074b33b3273d23c2426730dafbcea1b5b49f/tests/custom_cluster/test_events_custom_configs.py#L567-L568
--
To view, visit http://gerrit.cloudera.org:8080/23687
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie9a28f863ef17456817e0a335215450e514b1f5b
Gerrit-Change-Number: 23687
Gerrit-PatchSet: 1
Gerrit-Owner: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Wed, 19 Nov 2025 01:33:16 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-14562: Enable Hierarchical event processing by default
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23687 )
Change subject: IMPALA-14562: Enable Hierarchical event processing by default
......................................................................
Patch Set 4:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/21082/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/23687
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie9a28f863ef17456817e0a335215450e514b1f5b
Gerrit-Change-Number: 23687
Gerrit-PatchSet: 4
Gerrit-Owner: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Wed, 03 Dec 2025 17:58:34 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14562: Enable Hierarchical event processing by default
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/23687 )
Change subject: IMPALA-14562: Enable Hierarchical event processing by default
......................................................................
Patch Set 4: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/23687
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie9a28f863ef17456817e0a335215450e514b1f5b
Gerrit-Change-Number: 23687
Gerrit-PatchSet: 4
Gerrit-Owner: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Thu, 04 Dec 2025 01:59:53 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14562: Enable Hierarchical event processing by default
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23687 )
Change subject: IMPALA-14562: Enable Hierarchical event processing by default
......................................................................
Patch Set 4:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/12744/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/23687
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie9a28f863ef17456817e0a335215450e514b1f5b
Gerrit-Change-Number: 23687
Gerrit-PatchSet: 4
Gerrit-Owner: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Thu, 04 Dec 2025 02:00:22 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14562: Enable Hierarchical event processing by default
Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
k.venureddy2103@gmail.com has posted comments on this change. ( http://gerrit.cloudera.org:8080/23687 )
Change subject: IMPALA-14562: Enable Hierarchical event processing by default
......................................................................
Patch Set 2:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/23687/2/be/src/catalog/catalog-server.cc
File be/src/catalog/catalog-server.cc:
http://gerrit.cloudera.org:8080/#/c/23687/2/be/src/catalog/catalog-server.cc@261
PS2, Line 261: DEFINE_bool(enable_hierarchical_event_processing, true,
> If we set hms_event_polling_interval_s to 0.2s, then we would hit metastore
ok. We can calibrate later.
--
To view, visit http://gerrit.cloudera.org:8080/23687
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie9a28f863ef17456817e0a335215450e514b1f5b
Gerrit-Change-Number: 23687
Gerrit-PatchSet: 2
Gerrit-Owner: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Tue, 25 Nov 2025 10:49:46 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-14562: Enable Hierarchical event processing by default
Posted by "Sai Hemanth Gantasala (Code Review)" <ge...@cloudera.org>.
Sai Hemanth Gantasala has posted comments on this change. ( http://gerrit.cloudera.org:8080/23687 )
Change subject: IMPALA-14562: Enable Hierarchical event processing by default
......................................................................
Patch Set 2:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/23687/2/be/src/catalog/catalog-server.cc
File be/src/catalog/catalog-server.cc:
http://gerrit.cloudera.org:8080/#/c/23687/2/be/src/catalog/catalog-server.cc@261
PS2, Line 261: DEFINE_bool(enable_hierarchical_event_processing, true,
> We need to reduce hms_event_polling_interval_s to < 1sec. Since multiple th
If we set hms_event_polling_interval_s to 0.2s, then we would hit metastore 5 times every 1 second, compared to 1 time per second currently (In the Ideal case), which puts additional stress on metastore on an already busy cluster.
So, I feel we can start at 0.5 seconds (making 2 calls per second) and observe the performance. What do you think?
--
To view, visit http://gerrit.cloudera.org:8080/23687
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie9a28f863ef17456817e0a335215450e514b1f5b
Gerrit-Change-Number: 23687
Gerrit-PatchSet: 2
Gerrit-Owner: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Thu, 20 Nov 2025 18:56:29 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-14562: Enable Hierarchical event processing by default
Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
k.venureddy2103@gmail.com has posted comments on this change. ( http://gerrit.cloudera.org:8080/23687 )
Change subject: IMPALA-14562: Enable Hierarchical event processing by default
......................................................................
Patch Set 2:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/23687/2/be/src/catalog/catalog-server.cc
File be/src/catalog/catalog-server.cc:
http://gerrit.cloudera.org:8080/#/c/23687/2/be/src/catalog/catalog-server.cc@261
PS2, Line 261: DEFINE_bool(enable_hierarchical_event_processing, true,
We need to reduce hms_event_polling_interval_s to < 1sec. Since multiple threads are processing events, polling frequency can be increased to use the full potential of HEP. We can keep 0.2sec or lower. Because we have max_outstanding_events_on_executors to limit the max outstanding events at any point of time. So if outstanding events hit the limit, we don’t pull any more events from HMS(on subsequent processEvents() thread schedules) until it comes down below this.
hms_event_polling_interval_s can be configured to a lower possible value that ensures HMS don't get overloaded with too frequent calls.
--
To view, visit http://gerrit.cloudera.org:8080/23687
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie9a28f863ef17456817e0a335215450e514b1f5b
Gerrit-Change-Number: 23687
Gerrit-PatchSet: 2
Gerrit-Owner: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Thu, 20 Nov 2025 11:36:33 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-14562: Enable Hierarchical event processing by default
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/23687 )
Change subject: IMPALA-14562: Enable Hierarchical event processing by default
......................................................................
Patch Set 1:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/20974/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/23687
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ie9a28f863ef17456817e0a335215450e514b1f5b
Gerrit-Change-Number: 23687
Gerrit-PatchSet: 1
Gerrit-Owner: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <k....@gmail.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Sai Hemanth Gantasala <sa...@cloudera.com>
Gerrit-Comment-Date: Wed, 19 Nov 2025 01:27:33 +0000
Gerrit-HasComments: No