You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Anonymous Coward (Code Review)" <ge...@cloudera.org> on 2023/08/30 02:26:44 UTC
[Impala-ASF-CR] IMPALA-12402: Add some configurations for CatalogdMetaProvider's cache
cclive1601@gmail.com has uploaded this change for review. ( http://gerrit.cloudera.org:8080/20435
Change subject: IMPALA-12402: Add some configurations for CatalogdMetaProvider's cache_
......................................................................
IMPALA-12402: Add some configurations for CatalogdMetaProvider's cache_
The local CatalogMetaProvider's cache_ need to doing some loading process with the default
Concurrency level of 4 , when the table number is very big ,the loading process will need very much
time, in that case the restart process will cost much time too, so we make this parameter configurable.
Change-Id: Idd89e612ce0cc98b0244b7784816e2d7d6dcde22
---
M be/src/runtime/exec-env.cc
M be/src/util/backend-gflag-util.cc
M common/thrift/BackendGflags.thrift
M fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java
4 files changed, 10 insertions(+), 2 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/35/20435/1
--
To view, visit http://gerrit.cloudera.org:8080/20435
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Idd89e612ce0cc98b0244b7784816e2d7d6dcde22
Gerrit-Change-Number: 20435
Gerrit-PatchSet: 1
Gerrit-Owner: Anonymous Coward <cc...@gmail.com>
[Impala-ASF-CR] IMPALA-12402: Add some configurations for CatalogdMetaProvider's cache
Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has abandoned this change. ( http://gerrit.cloudera.org:8080/20435 )
Change subject: IMPALA-12402: Add some configurations for CatalogdMetaProvider's cache_
......................................................................
Abandoned
20443 has a newer version.
--
To view, visit http://gerrit.cloudera.org:8080/20435
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: abandon
Gerrit-Change-Id: Idd89e612ce0cc98b0244b7784816e2d7d6dcde22
Gerrit-Change-Number: 20435
Gerrit-PatchSet: 2
Gerrit-Owner: Anonymous Coward <cc...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] IMPALA-12402: Add some configurations for CatalogdMetaProvider's cache
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20435 )
Change subject: IMPALA-12402: Add some configurations for CatalogdMetaProvider's cache_
......................................................................
Patch Set 2:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/13892/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/20435
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idd89e612ce0cc98b0244b7784816e2d7d6dcde22
Gerrit-Change-Number: 20435
Gerrit-PatchSet: 2
Gerrit-Owner: Anonymous Coward <cc...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Thu, 31 Aug 2023 07:11:21 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12402: Add some configurations for CatalogdMetaProvider's cache
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20435 )
Change subject: IMPALA-12402: Add some configurations for CatalogdMetaProvider's cache_
......................................................................
Patch Set 2:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/20435/2/fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java
File fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java:
http://gerrit.cloudera.org:8080/#/c/20435/2/fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java@368
PS2, Line 368: "consistencyLevel={}", cacheSizeBytes/1024/1024, expirationSecs, consistencyLevel);
line too long (91 > 90)
--
To view, visit http://gerrit.cloudera.org:8080/20435
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idd89e612ce0cc98b0244b7784816e2d7d6dcde22
Gerrit-Change-Number: 20435
Gerrit-PatchSet: 2
Gerrit-Owner: Anonymous Coward <cc...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Thu, 31 Aug 2023 06:46:58 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-12402: Add some configurations for CatalogdMetaProvider's cache
Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/20435
to look at the new patch set (#2).
Change subject: IMPALA-12402: Add some configurations for CatalogdMetaProvider's cache_
......................................................................
IMPALA-12402: Add some configurations for CatalogdMetaProvider's cache_
The local CatalogMetaProvider's cache_ need to doing some loading process with the default
Concurrency level of 4 , when the table number is very big ,the loading process will need very much
time, in that case the restart process will cost much time too, so we make this parameter configurable.
Change-Id: Idd89e612ce0cc98b0244b7784816e2d7d6dcde22
---
M be/src/runtime/exec-env.cc
M be/src/util/backend-gflag-util.cc
M common/thrift/BackendGflags.thrift
M fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java
4 files changed, 11 insertions(+), 2 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/35/20435/2
--
To view, visit http://gerrit.cloudera.org:8080/20435
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Idd89e612ce0cc98b0244b7784816e2d7d6dcde22
Gerrit-Change-Number: 20435
Gerrit-PatchSet: 2
Gerrit-Owner: Anonymous Coward <cc...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] IMPALA-12402: Add some configurations for CatalogdMetaProvider's cache
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20435 )
Change subject: IMPALA-12402: Add some configurations for CatalogdMetaProvider's cache_
......................................................................
Patch Set 1:
Build Failed
https://jenkins.impala.io/job/gerrit-code-review-checks/13879/ : Initial code review checks failed. See linked job for details on the failure.
--
To view, visit http://gerrit.cloudera.org:8080/20435
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idd89e612ce0cc98b0244b7784816e2d7d6dcde22
Gerrit-Change-Number: 20435
Gerrit-PatchSet: 1
Gerrit-Owner: Anonymous Coward <cc...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Wed, 30 Aug 2023 02:42:53 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-12402: Add some configurations for CatalogdMetaProvider's cache
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/20435 )
Change subject: IMPALA-12402: Add some configurations for CatalogdMetaProvider's cache_
......................................................................
Patch Set 1:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/20435/1/be/src/runtime/exec-env.cc
File be/src/runtime/exec-env.cc:
http://gerrit.cloudera.org:8080/#/c/20435/1/be/src/runtime/exec-env.cc@128
PS1, Line 128: "to avoid lock contention , the default value 4 is consistent with the default value of the original cache.");
line too long (114 > 90)
http://gerrit.cloudera.org:8080/#/c/20435/1/fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java
File fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java:
http://gerrit.cloudera.org:8080/#/c/20435/1/fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java@367
PS1, Line 367: LOG.info("Metadata cache configuration: capacity={} MB, expiration={} sec, consistencyLevel={}",
line too long (100 > 90)
--
To view, visit http://gerrit.cloudera.org:8080/20435
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idd89e612ce0cc98b0244b7784816e2d7d6dcde22
Gerrit-Change-Number: 20435
Gerrit-PatchSet: 1
Gerrit-Owner: Anonymous Coward <cc...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Wed, 30 Aug 2023 02:36:49 +0000
Gerrit-HasComments: Yes