You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2023/03/02 07:01:00 UTC
[jira] [Commented] (IMPALA-11948) Don't log getCatalogServerMetrics requests in catalogd
[ https://issues.apache.org/jira/browse/IMPALA-11948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17695503#comment-17695503 ]
ASF subversion and git services commented on IMPALA-11948:
----------------------------------------------------------
Commit eaf71bca0a3538056765dbbce13c0f503045b38b in impala's branch refs/heads/master from Peter Rozsa
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=eaf71bca0 ]
IMPALA-11948: Remove operation logging from getCatalogServerMetrics
IMPALA-11478 added operation-level logging to every catalog method in
JniCatalog, but getCatalogServerMetrics operation is called many times
by catalogd itself. Start and finish logs for this operation are useless,
therefore only the warnings about size and elapsed time should remain.
Tests:
- manually validated that warning-level logs are still generated
- manually validated that start/finish log entries are not generated
Change-Id: Ifd01e9f42ccc6a4adce41caf8884ea3c2c781452
Reviewed-on: http://gerrit.cloudera.org:8080/19557
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Don't log getCatalogServerMetrics requests in catalogd
> ------------------------------------------------------
>
> Key: IMPALA-11948
> URL: https://issues.apache.org/jira/browse/IMPALA-11948
> Project: IMPALA
> Issue Type: Task
> Components: Frontend
> Reporter: Quanlong Huang
> Assignee: Peter Rozsa
> Priority: Major
>
> After IMPALA-11478, The log file of catalogd is full of the following logs even when the server is idle:
> {noformat}
> I0226 16:55:27.111258 2816 JniUtil.java:161] Finished getCatalogServerMetrics request: Get catalog server metrics. Time spent: 2ms
> I0226 16:55:28.111685 2816 JniUtil.java:157] getCatalogServerMetrics request: Get catalog server metrics
> I0226 16:55:28.111811 2816 JniUtil.java:161] Finished getCatalogServerMetrics request: Get catalog server metrics. Time spent:
> I0226 16:55:29.112185 2816 JniUtil.java:157] getCatalogServerMetrics request: Get catalog server metrics
> I0226 16:55:29.112324 2816 JniUtil.java:161] Finished getCatalogServerMetrics request: Get catalog server metrics. Time spent: 1ms
> I0226 16:55:30.112716 2816 JniUtil.java:157] getCatalogServerMetrics request: Get catalog server metrics
> I0226 16:55:30.112885 2816 JniUtil.java:161] Finished getCatalogServerMetrics request: Get catalog server metrics. Time spent:
> I0226 16:55:31.113215 2816 JniUtil.java:157] getCatalogServerMetrics request: Get catalog server metrics
> I0226 16:55:31.113399 2816 JniUtil.java:161] Finished getCatalogServerMetrics request: Get catalog server metrics. Time spent:
> I0226 16:55:32.113715 2816 JniUtil.java:157] getCatalogServerMetrics request: Get catalog server metrics
> I0226 16:55:32.113834 2816 JniUtil.java:161] Finished getCatalogServerMetrics request: Get catalog server metrics. Time spent:
> I0226 16:55:33.114199 2816 JniUtil.java:157] getCatalogServerMetrics request: Get catalog server metrics
> I0226 16:55:33.114336 2816 JniUtil.java:161] Finished getCatalogServerMetrics request: Get catalog server metrics. Time spent: {noformat}
> getCatalogServerMetrics() is invoked every second by catalogd itself:
> {code:cpp}
> [[noreturn]] void CatalogServer::RefreshMetrics() {
> while (true) {
> SleepForMs(REFRESH_METRICS_INTERVAL_MS);
> TGetCatalogServerMetricsResponse response;
> Status status = catalog_->GetCatalogServerMetrics(&response);
> if (!status.ok()) {
> LOG(ERROR) << "Error refreshing catalog metrics: " << status.GetDetail();
> continue;
> }
> partial_fetch_rpc_queue_len_metric_->SetValue(
> response.catalog_partial_fetch_rpc_queue_len);
> TEventProcessorMetrics eventProcessorMetrics = response.event_metrics;
> MetastoreEventMetrics::refresh(&eventProcessorMetrics);
> }
> }
> {code}
> https://github.com/apache/impala/blob/d0592c0dbf4d634cc26b2e9872a9f62af5249cdd/be/src/catalog/catalog-server.cc#L464-L478
> I think it's useless to log these requests. CC [~prozsa] [~csringhofer]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org