You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Riza Suminto (Code Review)" <ge...@cloudera.org> on 2023/02/24 00:21:30 UTC
[Impala-ASF-CR] IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
Riza Suminto has uploaded this change for review. ( http://gerrit.cloudera.org:8080/19530
Change subject: IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
......................................................................
IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
IMPALA-11658 implement Iceberg manifest caching config for Impala. This
patch add documentation for that settings.
Testing:
- Built docs locally
Change-Id: Idd761a81f5c81a25a5ec0889402f85157c23e9fe
---
M docs/topics/impala_iceberg.xml
1 file changed, 56 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/30/19530/1
--
To view, visit http://gerrit.cloudera.org:8080/19530
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Idd761a81f5c81a25a5ec0889402f85157c23e9fe
Gerrit-Change-Number: 19530
Gerrit-PatchSet: 1
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
[Impala-ASF-CR] IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Hello Andrew Sherman, Daniel Becker, Gabor Kaszab, Zoltan Borok-Nagy, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/19530
to look at the new patch set (#5).
Change subject: IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
......................................................................
IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
IMPALA-11658 implements Iceberg manifest caching for Impala. This patch
adds documentation for configuring the cache(s).
Testing:
- Built docs locally
Change-Id: Idd761a81f5c81a25a5ec0889402f85157c23e9fe
---
M docs/topics/impala_iceberg.xml
1 file changed, 60 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/30/19530/5
--
To view, visit http://gerrit.cloudera.org:8080/19530
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Idd761a81f5c81a25a5ec0889402f85157c23e9fe
Gerrit-Change-Number: 19530
Gerrit-PatchSet: 5
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19530 )
Change subject: IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
......................................................................
Patch Set 3: Verified+1
Build Successful
https://jenkins.impala.io/job/gerrit-docs-auto-test/703/ : Doc tests passed.
--
To view, visit http://gerrit.cloudera.org:8080/19530
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idd761a81f5c81a25a5ec0889402f85157c23e9fe
Gerrit-Change-Number: 19530
Gerrit-PatchSet: 3
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Fri, 24 Feb 2023 16:25:40 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19530 )
Change subject: IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
......................................................................
Patch Set 1:
Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/701/
Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstrong@cloudera.com or on this JIRA: IMPALA-7317
--
To view, visit http://gerrit.cloudera.org:8080/19530
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idd761a81f5c81a25a5ec0889402f85157c23e9fe
Gerrit-Change-Number: 19530
Gerrit-PatchSet: 1
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Fri, 24 Feb 2023 00:21:40 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
Posted by "Daniel Becker (Code Review)" <ge...@cloudera.org>.
Daniel Becker has posted comments on this change. ( http://gerrit.cloudera.org:8080/19530 )
Change subject: IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
......................................................................
Patch Set 5: Code-Review+1
Thanks Riza.
--
To view, visit http://gerrit.cloudera.org:8080/19530
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idd761a81f5c81a25a5ec0889402f85157c23e9fe
Gerrit-Change-Number: 19530
Gerrit-PatchSet: 5
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Fri, 24 Feb 2023 17:10:12 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
Posted by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>.
Zoltan Borok-Nagy has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/19530 )
Change subject: IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
......................................................................
IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
IMPALA-11658 implements Iceberg manifest caching for Impala. This patch
adds documentation for configuring the cache(s).
Testing:
- Built docs locally
Change-Id: Idd761a81f5c81a25a5ec0889402f85157c23e9fe
Reviewed-on: http://gerrit.cloudera.org:8080/19530
Reviewed-by: Daniel Becker <da...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
Reviewed-by: Zoltan Borok-Nagy <bo...@cloudera.com>
---
M docs/topics/impala_iceberg.xml
1 file changed, 60 insertions(+), 0 deletions(-)
Approvals:
Daniel Becker: Looks good to me, but someone else must approve
Impala Public Jenkins: Verified
Zoltan Borok-Nagy: Looks good to me, approved
--
To view, visit http://gerrit.cloudera.org:8080/19530
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Idd761a81f5c81a25a5ec0889402f85157c23e9fe
Gerrit-Change-Number: 19530
Gerrit-PatchSet: 6
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19530 )
Change subject: IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
......................................................................
Patch Set 2: Verified+1
Build Successful
https://jenkins.impala.io/job/gerrit-docs-auto-test/702/ : Doc tests passed.
--
To view, visit http://gerrit.cloudera.org:8080/19530
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idd761a81f5c81a25a5ec0889402f85157c23e9fe
Gerrit-Change-Number: 19530
Gerrit-PatchSet: 2
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Fri, 24 Feb 2023 00:47:43 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Hello Andrew Sherman, Daniel Becker, Gabor Kaszab, Zoltan Borok-Nagy, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/19530
to look at the new patch set (#4).
Change subject: IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
......................................................................
IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
IMPALA-11658 implements Iceberg manifest caching for Impala. This patch
adds documentation for configuring the cache(s).
Testing:
- Built docs locally
Change-Id: Idd761a81f5c81a25a5ec0889402f85157c23e9fe
---
M docs/topics/impala_iceberg.xml
1 file changed, 60 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/30/19530/4
--
To view, visit http://gerrit.cloudera.org:8080/19530
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Idd761a81f5c81a25a5ec0889402f85157c23e9fe
Gerrit-Change-Number: 19530
Gerrit-PatchSet: 4
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19530 )
Change subject: IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
......................................................................
Patch Set 5:
Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/704/
Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstrong@cloudera.com or on this JIRA: IMPALA-7317
--
To view, visit http://gerrit.cloudera.org:8080/19530
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idd761a81f5c81a25a5ec0889402f85157c23e9fe
Gerrit-Change-Number: 19530
Gerrit-PatchSet: 5
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Fri, 24 Feb 2023 17:04:14 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19530 )
Change subject: IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
......................................................................
Patch Set 1: Verified+1
Build Successful
https://jenkins.impala.io/job/gerrit-docs-auto-test/701/ : Doc tests passed.
--
To view, visit http://gerrit.cloudera.org:8080/19530
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idd761a81f5c81a25a5ec0889402f85157c23e9fe
Gerrit-Change-Number: 19530
Gerrit-PatchSet: 1
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Fri, 24 Feb 2023 00:31:00 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Hello Andrew Sherman, Gabor Kaszab, Zoltan Borok-Nagy, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/19530
to look at the new patch set (#2).
Change subject: IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
......................................................................
IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
IMPALA-11658 implement Iceberg manifest caching config for Impala. This
patch add documentation for that settings.
Testing:
- Built docs locally
Change-Id: Idd761a81f5c81a25a5ec0889402f85157c23e9fe
---
M docs/topics/impala_iceberg.xml
1 file changed, 58 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/30/19530/2
--
To view, visit http://gerrit.cloudera.org:8080/19530
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Idd761a81f5c81a25a5ec0889402f85157c23e9fe
Gerrit-Change-Number: 19530
Gerrit-PatchSet: 2
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
Posted by "Andrew Sherman (Code Review)" <ge...@cloudera.org>.
Andrew Sherman has posted comments on this change. ( http://gerrit.cloudera.org:8080/19530 )
Change subject: IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
......................................................................
Patch Set 2:
(7 comments)
I did a quick read through
http://gerrit.cloudera.org:8080/#/c/19530/2//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/19530/2//COMMIT_MSG@9
PS2, Line 9: IMPALA-11658 implement Iceberg manifest caching config for Impala. This
Nit: "implements Iceberg manifest caching for Impala."
http://gerrit.cloudera.org:8080/#/c/19530/2//COMMIT_MSG@10
PS2, Line 10: patch add documentation for that settings.
Nit: "adds documentation for configuring the cache(s).
http://gerrit.cloudera.org:8080/#/c/19530/2/docs/topics/impala_iceberg.xml
File docs/topics/impala_iceberg.xml:
http://gerrit.cloudera.org:8080/#/c/19530/2/docs/topics/impala_iceberg.xml@617
PS2, Line 617: by setting properties in Hadoop's core-site.xml as the following example:
Nit: "as in the following"
http://gerrit.cloudera.org:8080/#/c/19530/2/docs/topics/impala_iceberg.xml@631
PS2, Line 631: custom FileIO implementation to use in a catalog. Must be set to enable
Do we recommend that this not be changed? Maybe we should say that?
http://gerrit.cloudera.org:8080/#/c/19530/2/docs/topics/impala_iceberg.xml@632
PS2, Line 632: manifest caching. Impala default to HadoopFileIO.
Nit: "defaults to"
http://gerrit.cloudera.org:8080/#/c/19530/2/docs/topics/impala_iceberg.xml@640
PS2, Line 640: of a manifest file to be considered for caching. Manifest files with a length
Why would I want to set this to anything other than a high value? Oh. I see, is it because of max-total-bytes?
So we should set this to a value less than max-total-bytes?
http://gerrit.cloudera.org:8080/#/c/19530/2/docs/topics/impala_iceberg.xml@652
PS2, Line 652: <codeph>iceberg.io.manifest.cache.max-total-bytes</codeph>: maximum total
Maybe move this before iceberg.io.manifest.cache.expiration-interval-ms
That way the explanation in interval-ms that refers to max-total-bytes will be easier to read.
If we do this, maybe change the example code block too.
--
To view, visit http://gerrit.cloudera.org:8080/19530
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idd761a81f5c81a25a5ec0889402f85157c23e9fe
Gerrit-Change-Number: 19530
Gerrit-PatchSet: 2
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Fri, 24 Feb 2023 02:03:14 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19530 )
Change subject: IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
......................................................................
Patch Set 5: Verified+1
Build Successful
https://jenkins.impala.io/job/gerrit-docs-auto-test/704/ : Doc tests passed.
--
To view, visit http://gerrit.cloudera.org:8080/19530
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idd761a81f5c81a25a5ec0889402f85157c23e9fe
Gerrit-Change-Number: 19530
Gerrit-PatchSet: 5
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Fri, 24 Feb 2023 17:12:51 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Hello Andrew Sherman, Daniel Becker, Gabor Kaszab, Zoltan Borok-Nagy, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/19530
to look at the new patch set (#3).
Change subject: IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
......................................................................
IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
IMPALA-11658 implement Iceberg manifest caching config for Impala. This
patch add documentation for that settings.
Testing:
- Built docs locally
Change-Id: Idd761a81f5c81a25a5ec0889402f85157c23e9fe
---
M docs/topics/impala_iceberg.xml
1 file changed, 60 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/30/19530/3
--
To view, visit http://gerrit.cloudera.org:8080/19530
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Idd761a81f5c81a25a5ec0889402f85157c23e9fe
Gerrit-Change-Number: 19530
Gerrit-PatchSet: 3
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19530 )
Change subject: IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
......................................................................
Patch Set 3:
Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/703/
Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstrong@cloudera.com or on this JIRA: IMPALA-7317
--
To view, visit http://gerrit.cloudera.org:8080/19530
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idd761a81f5c81a25a5ec0889402f85157c23e9fe
Gerrit-Change-Number: 19530
Gerrit-PatchSet: 3
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Fri, 24 Feb 2023 16:17:14 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/19530 )
Change subject: IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
......................................................................
Patch Set 4:
(10 comments)
http://gerrit.cloudera.org:8080/#/c/19530/2//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/19530/2//COMMIT_MSG@9
PS2, Line 9: IMPALA-11658 implements Iceberg manifest caching for Impala. This patch
> Nit: "implements Iceberg manifest caching for Impala."
Done
http://gerrit.cloudera.org:8080/#/c/19530/2//COMMIT_MSG@10
PS2, Line 10: adds documentation for configuring the cache(s).
> Nit: "adds documentation for configuring the cache(s).
Done
http://gerrit.cloudera.org:8080/#/c/19530/2/docs/topics/impala_iceberg.xml
File docs/topics/impala_iceberg.xml:
http://gerrit.cloudera.org:8080/#/c/19530/2/docs/topics/impala_iceberg.xml@614
PS2, Line 614:
> Nit: contents?
Done
http://gerrit.cloudera.org:8080/#/c/19530/2/docs/topics/impala_iceberg.xml@617
PS2, Line 617: feature can be enabled for Impala by setting properties in Hadoop's core-site.xml
> Nit: "as in the following"
Done
http://gerrit.cloudera.org:8080/#/c/19530/2/docs/topics/impala_iceberg.xml@631
PS2, Line 631: <codeph>iceberg.io-impl</codeph>: custom FileIO implementation to use in a
> Do we recommend that this not be changed? Maybe we should say that?
Done
http://gerrit.cloudera.org:8080/#/c/19530/2/docs/topics/impala_iceberg.xml@632
PS2, Line 632: catalog. Must be set to enable manifest caching. Impala defaults to
> Nit: "defaults to"
Done
http://gerrit.cloudera.org:8080/#/c/19530/2/docs/topics/impala_iceberg.xml@639
PS2, Line 639:
> In the two cases above we started a new line after the title (in <codeph>),
Removed newline after </codeph> in the other two.
http://gerrit.cloudera.org:8080/#/c/19530/2/docs/topics/impala_iceberg.xml@640
PS2, Line 640: <codeph>iceberg.io.manifest.cache.max-total-bytes</codeph>: maximum total
> Why would I want to set this to anything other than a high value? Oh. I see
Yes. Added "and lower than <codeph>iceberg.io.manifest.cache.max-total-bytes</codeph>.".
http://gerrit.cloudera.org:8080/#/c/19530/2/docs/topics/impala_iceberg.xml@652
PS2, Line 652: of a manifest file to be considered for caching in bytes. Manifest files with
> Maybe move this before iceberg.io.manifest.cache.expiration-interval-ms
Yes, reordered the example and bullet points.
http://gerrit.cloudera.org:8080/#/c/19530/2/docs/topics/impala_iceberg.xml@659
PS2, Line 659:
> Shouldn't it be HadoopCatalogs and HiveCatalogs, i.e. plural?
Done
--
To view, visit http://gerrit.cloudera.org:8080/19530
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idd761a81f5c81a25a5ec0889402f85157c23e9fe
Gerrit-Change-Number: 19530
Gerrit-PatchSet: 4
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Fri, 24 Feb 2023 16:22:53 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
Posted by "Daniel Becker (Code Review)" <ge...@cloudera.org>.
Daniel Becker has posted comments on this change. ( http://gerrit.cloudera.org:8080/19530 )
Change subject: IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
......................................................................
Patch Set 2:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/19530/2/docs/topics/impala_iceberg.xml
File docs/topics/impala_iceberg.xml:
http://gerrit.cloudera.org:8080/#/c/19530/2/docs/topics/impala_iceberg.xml@614
PS2, Line 614: content
Nit: contents?
http://gerrit.cloudera.org:8080/#/c/19530/2/docs/topics/impala_iceberg.xml@639
PS2, Line 639:
In the two cases above we started a new line after the title (in <codeph>), from here on we don't. It would be better if it was consistent.
http://gerrit.cloudera.org:8080/#/c/19530/2/docs/topics/impala_iceberg.xml@659
PS2, Line 659: HadoopCatalog and HiveCatalog
Shouldn't it be HadoopCatalogs and HiveCatalogs, i.e. plural?
--
To view, visit http://gerrit.cloudera.org:8080/19530
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idd761a81f5c81a25a5ec0889402f85157c23e9fe
Gerrit-Change-Number: 19530
Gerrit-PatchSet: 2
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Fri, 24 Feb 2023 13:36:36 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
Posted by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>.
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/19530 )
Change subject: IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
......................................................................
Patch Set 4:
(2 comments)
Thanks for adding docs for it!
http://gerrit.cloudera.org:8080/#/c/19530/4/docs/topics/impala_iceberg.xml
File docs/topics/impala_iceberg.xml:
http://gerrit.cloudera.org:8080/#/c/19530/4/docs/topics/impala_iceberg.xml@617
PS4, Line 617: for Impala
Maybe we should mention that it is useful for the Coordinators and CatalogD as well.
http://gerrit.cloudera.org:8080/#/c/19530/4/docs/topics/impala_iceberg.xml@623
PS4, Line 623: 60000
nit: people might just blindly copying the values from here. So we probably want to provide a higher value as an example than one minute, e.g. one hour?
--
To view, visit http://gerrit.cloudera.org:8080/19530
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idd761a81f5c81a25a5ec0889402f85157c23e9fe
Gerrit-Change-Number: 19530
Gerrit-PatchSet: 4
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Fri, 24 Feb 2023 16:38:32 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
Posted by "Riza Suminto (Code Review)" <ge...@cloudera.org>.
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/19530 )
Change subject: IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
......................................................................
Patch Set 5:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/19530/4/docs/topics/impala_iceberg.xml
File docs/topics/impala_iceberg.xml:
http://gerrit.cloudera.org:8080/#/c/19530/4/docs/topics/impala_iceberg.xml@617
PS4, Line 617: gd. This f
> Maybe we should mention that it is useful for the Coordinators and CatalogD
Done
http://gerrit.cloudera.org:8080/#/c/19530/4/docs/topics/impala_iceberg.xml@623
PS4, Line 623: 36000
> nit: people might just blindly copying the values from here. So we probably
Make sense. Changed this to 3600000.
--
To view, visit http://gerrit.cloudera.org:8080/19530
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idd761a81f5c81a25a5ec0889402f85157c23e9fe
Gerrit-Change-Number: 19530
Gerrit-PatchSet: 5
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Fri, 24 Feb 2023 17:05:02 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
Posted by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>.
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/19530 )
Change subject: IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
......................................................................
Patch Set 5: Code-Review+2
Thanks Riza!
--
To view, visit http://gerrit.cloudera.org:8080/19530
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idd761a81f5c81a25a5ec0889402f85157c23e9fe
Gerrit-Change-Number: 19530
Gerrit-PatchSet: 5
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Fri, 24 Feb 2023 17:16:30 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19530 )
Change subject: IMPALA-11940: [DOCS] Document manifest caching settings for Iceberg
......................................................................
Patch Set 2:
Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/702/
Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstrong@cloudera.com or on this JIRA: IMPALA-7317
--
To view, visit http://gerrit.cloudera.org:8080/19530
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idd761a81f5c81a25a5ec0889402f85157c23e9fe
Gerrit-Change-Number: 19530
Gerrit-PatchSet: 2
Gerrit-Owner: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Andrew Sherman <as...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <ri...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Fri, 24 Feb 2023 00:37:55 +0000
Gerrit-HasComments: No