You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Sanjana Malhotra (Code Review)" <ge...@cloudera.org> on 2026/05/02 08:43:53 UTC
[Impala-ASF-CR] IMPALA-14899: [DOCS] Document max stmt metadata loader threads for parallel metadata loading
Sanjana Malhotra has uploaded this change for review. ( http://gerrit.cloudera.org:8080/24262
Change subject: IMPALA-14899: [DOCS] Document max_stmt_metadata_loader_threads for parallel metadata loading
......................................................................
IMPALA-14899: [DOCS] Document max_stmt_metadata_loader_threads for parallel metadata loading
This explains how to configure the thread pool size for loading statement metadata to improve
startup performance on systems with high schema complexity, as tracked in IMPALA-14131.
Change-Id: Ic347c3a4628ab4d0c22be689d3a8ff3dfdd22adf
---
M docs/topics/impala_metadata.xml
1 file changed, 30 insertions(+), 9 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/62/24262/2
--
To view, visit http://gerrit.cloudera.org:8080/24262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ic347c3a4628ab4d0c22be689d3a8ff3dfdd22adf
Gerrit-Change-Number: 24262
Gerrit-PatchSet: 2
Gerrit-Owner: Sanjana Malhotra <ma...@gmail.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@gmail.com>
Gerrit-Reviewer: Sanjana Malhotra <ma...@gmail.com>
[Impala-ASF-CR] IMPALA-14899: [DOCS] Document max stmt metadata loader threads for parallel metadata loading
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24262 )
Change subject: IMPALA-14899: [DOCS] Document max_stmt_metadata_loader_threads for parallel metadata loading
......................................................................
Patch Set 2:
Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/864/
Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstrong@cloudera.com or on this JIRA: IMPALA-7317
--
To view, visit http://gerrit.cloudera.org:8080/24262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic347c3a4628ab4d0c22be689d3a8ff3dfdd22adf
Gerrit-Change-Number: 24262
Gerrit-PatchSet: 2
Gerrit-Owner: Sanjana Malhotra <ma...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@gmail.com>
Gerrit-Reviewer: Sanjana Malhotra <ma...@gmail.com>
Gerrit-Comment-Date: Sat, 02 May 2026 08:44:04 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14899: [DOCS] Document max stmt metadata loader threads for parallel metadata loading
Posted by "Sanjana Malhotra (Code Review)" <ge...@cloudera.org>.
Sanjana Malhotra has uploaded a new patch set (#7). ( http://gerrit.cloudera.org:8080/24262 )
Change subject: IMPALA-14899: [DOCS] Document max_stmt_metadata_loader_threads for parallel metadata loading
......................................................................
IMPALA-14899: [DOCS] Document max_stmt_metadata_loader_threads for parallel metadata loading
This explains how to configure the thread pool size for loading
statement metadata to improve startup performance on queries
accessing multiple tables for the first time,
as tracked in IMPALA-14447.
Change-Id: Ic347c3a4628ab4d0c22be689d3a8ff3dfdd22adf
---
M docs/topics/impala_metadata.xml
1 file changed, 19 insertions(+), 9 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/62/24262/7
--
To view, visit http://gerrit.cloudera.org:8080/24262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ic347c3a4628ab4d0c22be689d3a8ff3dfdd22adf
Gerrit-Change-Number: 24262
Gerrit-PatchSet: 7
Gerrit-Owner: Sanjana Malhotra <ma...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@gmail.com>
Gerrit-Reviewer: Sanjana Malhotra <ma...@gmail.com>
[Impala-ASF-CR] IMPALA-14899: [DOCS] Document max stmt metadata loader threads for parallel metadata loading
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24262 )
Change subject: IMPALA-14899: [DOCS] Document max_stmt_metadata_loader_threads for parallel metadata loading
......................................................................
Patch Set 2: Verified+1
Build Successful
https://jenkins.impala.io/job/gerrit-docs-auto-test/864/ : Doc tests passed.
--
To view, visit http://gerrit.cloudera.org:8080/24262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic347c3a4628ab4d0c22be689d3a8ff3dfdd22adf
Gerrit-Change-Number: 24262
Gerrit-PatchSet: 2
Gerrit-Owner: Sanjana Malhotra <ma...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@gmail.com>
Gerrit-Reviewer: Sanjana Malhotra <ma...@gmail.com>
Gerrit-Comment-Date: Sat, 02 May 2026 08:49:05 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14899: [DOCS] Document max stmt metadata loader threads for parallel metadata loading
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/24262 )
Change subject: IMPALA-14899: [DOCS] Document max_stmt_metadata_loader_threads for parallel metadata loading
......................................................................
Patch Set 7: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/24262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic347c3a4628ab4d0c22be689d3a8ff3dfdd22adf
Gerrit-Change-Number: 24262
Gerrit-PatchSet: 7
Gerrit-Owner: Sanjana Malhotra <ma...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@gmail.com>
Gerrit-Reviewer: Sanjana Malhotra <ma...@gmail.com>
Gerrit-Comment-Date: Sat, 09 May 2026 12:05:12 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14899: [DOCS] Document max stmt metadata loader threads for parallel metadata loading
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/24262 )
Change subject: IMPALA-14899: [DOCS] Document max_stmt_metadata_loader_threads for parallel metadata loading
......................................................................
IMPALA-14899: [DOCS] Document max_stmt_metadata_loader_threads for parallel metadata loading
This explains how to configure the thread pool size for loading
statement metadata to improve startup performance on queries
accessing multiple tables for the first time,
as tracked in IMPALA-14447.
Change-Id: Ic347c3a4628ab4d0c22be689d3a8ff3dfdd22adf
Reviewed-on: http://gerrit.cloudera.org:8080/24262
Tested-by: Impala Public Jenkins <im...@cloudera.com>
Reviewed-by: Quanlong Huang <hu...@gmail.com>
---
M docs/topics/impala_metadata.xml
1 file changed, 19 insertions(+), 9 deletions(-)
Approvals:
Impala Public Jenkins: Verified
Quanlong Huang: Looks good to me, approved
--
To view, visit http://gerrit.cloudera.org:8080/24262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ic347c3a4628ab4d0c22be689d3a8ff3dfdd22adf
Gerrit-Change-Number: 24262
Gerrit-PatchSet: 9
Gerrit-Owner: Sanjana Malhotra <ma...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@gmail.com>
Gerrit-Reviewer: Sanjana Malhotra <ma...@gmail.com>
[Impala-ASF-CR] IMPALA-14899: [DOCS] Document max stmt metadata loader threads for parallel metadata loading
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24262 )
Change subject: IMPALA-14899: [DOCS] Document max_stmt_metadata_loader_threads for parallel metadata loading
......................................................................
Patch Set 7: Verified+1
Build Successful
https://jenkins.impala.io/job/gerrit-docs-auto-test/866/ : Doc tests passed.
--
To view, visit http://gerrit.cloudera.org:8080/24262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic347c3a4628ab4d0c22be689d3a8ff3dfdd22adf
Gerrit-Change-Number: 24262
Gerrit-PatchSet: 7
Gerrit-Owner: Sanjana Malhotra <ma...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@gmail.com>
Gerrit-Reviewer: Sanjana Malhotra <ma...@gmail.com>
Gerrit-Comment-Date: Sat, 09 May 2026 02:48:35 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14899: [DOCS] Document max stmt metadata loader threads for parallel metadata loading
Posted by "Sanjana Malhotra (Code Review)" <ge...@cloudera.org>.
Sanjana Malhotra has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/24262 )
Change subject: IMPALA-14899: [DOCS] Document max_stmt_metadata_loader_threads for parallel metadata loading
......................................................................
IMPALA-14899: [DOCS] Document max_stmt_metadata_loader_threads for parallel metadata loading
This explains how to configure the thread pool size for loading
statement metadata to improve startup performance on systems
with high schema complexity, as tracked in IMPALA-14131.
Change-Id: Ic347c3a4628ab4d0c22be689d3a8ff3dfdd22adf
---
M docs/topics/impala_metadata.xml
1 file changed, 19 insertions(+), 9 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/62/24262/4
--
To view, visit http://gerrit.cloudera.org:8080/24262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ic347c3a4628ab4d0c22be689d3a8ff3dfdd22adf
Gerrit-Change-Number: 24262
Gerrit-PatchSet: 4
Gerrit-Owner: Sanjana Malhotra <ma...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@gmail.com>
Gerrit-Reviewer: Sanjana Malhotra <ma...@gmail.com>
[Impala-ASF-CR] IMPALA-14899: [DOCS] Document max stmt metadata loader threads for parallel metadata loading
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24262 )
Change subject: IMPALA-14899: [DOCS] Document max_stmt_metadata_loader_threads for parallel metadata loading
......................................................................
Patch Set 8: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/24262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic347c3a4628ab4d0c22be689d3a8ff3dfdd22adf
Gerrit-Change-Number: 24262
Gerrit-PatchSet: 8
Gerrit-Owner: Sanjana Malhotra <ma...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@gmail.com>
Gerrit-Reviewer: Sanjana Malhotra <ma...@gmail.com>
Gerrit-Comment-Date: Sun, 10 May 2026 02:29:18 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14899: [DOCS] Document max stmt metadata loader threads for parallel metadata loading
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/24262 )
Change subject: IMPALA-14899: [DOCS] Document max_stmt_metadata_loader_threads for parallel metadata loading
......................................................................
Patch Set 2:
(5 comments)
http://gerrit.cloudera.org:8080/#/c/24262/2//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/24262/2//COMMIT_MSG@9
PS2, Line 9: This explains how to configure the thread pool size for loading statement metadata to improve
nit: line length should <= 72
http://gerrit.cloudera.org:8080/#/c/24262/2/docs/topics/impala_metadata.xml
File docs/topics/impala_metadata.xml:
http://gerrit.cloudera.org:8080/#/c/24262/2/docs/topics/impala_metadata.xml@131
PS2, Line 131: MAX_STMT_METADATA_LOADER_THREADS
nit: use lowercase to be consistent with other flags.
http://gerrit.cloudera.org:8080/#/c/24262/2/docs/topics/impala_metadata.xml@149
PS2, Line 149: MAX_STMT_METADATA_LOADER_THREADS
nit: use lowercase
http://gerrit.cloudera.org:8080/#/c/24262/2/docs/topics/impala_metadata.xml@151
PS2, Line 151: during query compilation. In local catalog mode, you can parallelize this process
To be clear, we should mention this is per-query, i.e., each query has up to this number of threads for loading metadata.
http://gerrit.cloudera.org:8080/#/c/24262/2/docs/topics/impala_metadata.xml@159
PS2, Line 159: <codeblock>SET MAX_STMT_METADATA_LOADER_THREADS=16;
This is not a query option. We can't use it in this way. It's set in the startup flags of impalad.
--
To view, visit http://gerrit.cloudera.org:8080/24262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic347c3a4628ab4d0c22be689d3a8ff3dfdd22adf
Gerrit-Change-Number: 24262
Gerrit-PatchSet: 2
Gerrit-Owner: Sanjana Malhotra <ma...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@gmail.com>
Gerrit-Reviewer: Sanjana Malhotra <ma...@gmail.com>
Gerrit-Comment-Date: Sun, 03 May 2026 09:40:29 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-14899: [DOCS] Document max stmt metadata loader threads for parallel metadata loading
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/24262 )
Change subject: IMPALA-14899: [DOCS] Document max_stmt_metadata_loader_threads for parallel metadata loading
......................................................................
Patch Set 4:
(5 comments)
http://gerrit.cloudera.org:8080/#/c/24262/4//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/24262/4//COMMIT_MSG@9
PS4, Line 9:
nit: remove trailing space
http://gerrit.cloudera.org:8080/#/c/24262/4//COMMIT_MSG@10
PS4, Line 10: systems
: with high schema complexity
Something like "queries accessing multiple tables at the first time" is more clear.
http://gerrit.cloudera.org:8080/#/c/24262/4//COMMIT_MSG@11
PS4, Line 11: IMPALA-14131
It's IMPALA-14447.
http://gerrit.cloudera.org:8080/#/c/24262/4/docs/topics/impala_metadata.xml
File docs/topics/impala_metadata.xml:
http://gerrit.cloudera.org:8080/#/c/24262/4/docs/topics/impala_metadata.xml@151
PS4, Line 151: ,
nit: use period "."
http://gerrit.cloudera.org:8080/#/c/24262/4/docs/topics/impala_metadata.xml@154
PS4, Line 154: Impala
Using "Coordinator" is more clear. Catalogd might already have the metadata loaded.
--
To view, visit http://gerrit.cloudera.org:8080/24262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic347c3a4628ab4d0c22be689d3a8ff3dfdd22adf
Gerrit-Change-Number: 24262
Gerrit-PatchSet: 4
Gerrit-Owner: Sanjana Malhotra <ma...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@gmail.com>
Gerrit-Reviewer: Sanjana Malhotra <ma...@gmail.com>
Gerrit-Comment-Date: Fri, 08 May 2026 13:29:37 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-14899: [DOCS] Document max stmt metadata loader threads for parallel metadata loading
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24262 )
Change subject: IMPALA-14899: [DOCS] Document max_stmt_metadata_loader_threads for parallel metadata loading
......................................................................
Patch Set 4: Verified+1
Build Successful
https://jenkins.impala.io/job/gerrit-docs-auto-test/865/ : Doc tests passed.
--
To view, visit http://gerrit.cloudera.org:8080/24262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic347c3a4628ab4d0c22be689d3a8ff3dfdd22adf
Gerrit-Change-Number: 24262
Gerrit-PatchSet: 4
Gerrit-Owner: Sanjana Malhotra <ma...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@gmail.com>
Gerrit-Reviewer: Sanjana Malhotra <ma...@gmail.com>
Gerrit-Comment-Date: Thu, 07 May 2026 04:31:12 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14899: [DOCS] Document max stmt metadata loader threads for parallel metadata loading
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24262 )
Change subject: IMPALA-14899: [DOCS] Document max_stmt_metadata_loader_threads for parallel metadata loading
......................................................................
Patch Set 8:
Build started: https://jenkins.impala.io/job/gerrit-docs-submit/396/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/24262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic347c3a4628ab4d0c22be689d3a8ff3dfdd22adf
Gerrit-Change-Number: 24262
Gerrit-PatchSet: 8
Gerrit-Owner: Sanjana Malhotra <ma...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@gmail.com>
Gerrit-Reviewer: Sanjana Malhotra <ma...@gmail.com>
Gerrit-Comment-Date: Sun, 10 May 2026 02:56:17 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14899: [DOCS] Document max stmt metadata loader threads for parallel metadata loading
Posted by "Sanjana Malhotra (Code Review)" <ge...@cloudera.org>.
Sanjana Malhotra has posted comments on this change. ( http://gerrit.cloudera.org:8080/24262 )
Change subject: IMPALA-14899: [DOCS] Document max_stmt_metadata_loader_threads for parallel metadata loading
......................................................................
Patch Set 2:
(4 comments)
> Patch Set 2:
>
> (5 comments)
http://gerrit.cloudera.org:8080/#/c/24262/2/docs/topics/impala_metadata.xml
File docs/topics/impala_metadata.xml:
http://gerrit.cloudera.org:8080/#/c/24262/2/docs/topics/impala_metadata.xml@131
PS2, Line 131: MAX_STMT_METADATA_LOADER_THREADS
> nit: use lowercase to be consistent with other flags.
Ack
http://gerrit.cloudera.org:8080/#/c/24262/2/docs/topics/impala_metadata.xml@149
PS2, Line 149: MAX_STMT_METADATA_LOADER_THREADS
> nit: use lowercase
Ack
http://gerrit.cloudera.org:8080/#/c/24262/2/docs/topics/impala_metadata.xml@151
PS2, Line 151: during query compilation. In local catalog mode, you can parallelize this process
> To be clear, we should mention this is per-query, i.e., each query has up t
The max_stmt_metadata_loader_threads configuration flag determines the maximum number of threads used to load table metadata during query compilation. This limit defines the capacity of the max_stmt_metadata_loader_threads flag in Apache Impala, allowing each query to use a specific thread count for loading metadata. In local catalog mode, ...
Does this deliver the info clearly?
http://gerrit.cloudera.org:8080/#/c/24262/2/docs/topics/impala_metadata.xml@159
PS2, Line 159: <codeblock>SET MAX_STMT_METADATA_LOADER_THREADS=16;
> This is not a query option. We can't use it in this way. It's set in the st
updating as
--max_stmt_metadata_loader_threads=16;
&
--max_stmt_metadata_loader_threads=1;
--
To view, visit http://gerrit.cloudera.org:8080/24262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic347c3a4628ab4d0c22be689d3a8ff3dfdd22adf
Gerrit-Change-Number: 24262
Gerrit-PatchSet: 2
Gerrit-Owner: Sanjana Malhotra <ma...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@gmail.com>
Gerrit-Reviewer: Sanjana Malhotra <ma...@gmail.com>
Gerrit-Comment-Date: Tue, 05 May 2026 09:38:11 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-14899: [DOCS] Document max stmt metadata loader threads for parallel metadata loading
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/24262 )
Change subject: IMPALA-14899: [DOCS] Document max_stmt_metadata_loader_threads for parallel metadata loading
......................................................................
Patch Set 2:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/24262/2/docs/topics/impala_metadata.xml
File docs/topics/impala_metadata.xml:
http://gerrit.cloudera.org:8080/#/c/24262/2/docs/topics/impala_metadata.xml@151
PS2, Line 151: during query compilation. In local catalog mode, you can parallelize this process
> This limit defines the capacity of the max_stmt_metadata_loader_threads flag in Apache Impala
I don't understand this. What do you mean by this flag defines the capacity of itself in Impala?
http://gerrit.cloudera.org:8080/#/c/24262/2/docs/topics/impala_metadata.xml@159
PS2, Line 159: <codeblock>SET MAX_STMT_METADATA_LOADER_THREADS=16;
> updating as
It might be OK to have no examples for a flag. I don't know how we do for other flags. Or you can just give examples and explain what they mean, e.g. --max_stmt_metadata_loader_threads=16 means coordinator can have 16 threads in loading table metadata for each query. --max_stmt_metadata_loader_threads=1 means coordinator will always load table metadata sequentially for a query.
--
To view, visit http://gerrit.cloudera.org:8080/24262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic347c3a4628ab4d0c22be689d3a8ff3dfdd22adf
Gerrit-Change-Number: 24262
Gerrit-PatchSet: 2
Gerrit-Owner: Sanjana Malhotra <ma...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@gmail.com>
Gerrit-Reviewer: Sanjana Malhotra <ma...@gmail.com>
Gerrit-Comment-Date: Wed, 06 May 2026 06:18:57 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-14899: [DOCS] Document max stmt metadata loader threads for parallel metadata loading
Posted by "Sanjana Malhotra (Code Review)" <ge...@cloudera.org>.
Hello Quanlong Huang, Riza Suminto, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/24262
to look at the new patch set (#5).
Change subject: IMPALA-14899: [DOCS] Document max_stmt_metadata_loader_threads for parallel metadata loading
......................................................................
IMPALA-14899: [DOCS] Document max_stmt_metadata_loader_threads for parallel metadata loading
This explains how to configure the thread pool size for loading
statement metadata to improve startup performance on queries
accessing multiple tables for the first time,
as tracked in IMPALA-14447.
Change-Id: Ic347c3a4628ab4d0c22be689d3a8ff3dfdd22adf
---
M docs/topics/impala_metadata.xml
1 file changed, 19 insertions(+), 9 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/62/24262/5
--
To view, visit http://gerrit.cloudera.org:8080/24262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ic347c3a4628ab4d0c22be689d3a8ff3dfdd22adf
Gerrit-Change-Number: 24262
Gerrit-PatchSet: 5
Gerrit-Owner: Sanjana Malhotra <ma...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@gmail.com>
Gerrit-Reviewer: Sanjana Malhotra <ma...@gmail.com>
[Impala-ASF-CR] IMPALA-14899: [DOCS] Document max stmt metadata loader threads for parallel metadata loading
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/24262 )
Change subject: IMPALA-14899: [DOCS] Document max_stmt_metadata_loader_threads for parallel metadata loading
......................................................................
Patch Set 8: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/24262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic347c3a4628ab4d0c22be689d3a8ff3dfdd22adf
Gerrit-Change-Number: 24262
Gerrit-PatchSet: 8
Gerrit-Owner: Sanjana Malhotra <ma...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@gmail.com>
Gerrit-Reviewer: Sanjana Malhotra <ma...@gmail.com>
Gerrit-Comment-Date: Sun, 10 May 2026 02:55:37 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14899: [DOCS] Document max stmt metadata loader threads for parallel metadata loading
Posted by "Sanjana Malhotra (Code Review)" <ge...@cloudera.org>.
Sanjana Malhotra has posted comments on this change. ( http://gerrit.cloudera.org:8080/24262 )
Change subject: IMPALA-14899: [DOCS] Document max_stmt_metadata_loader_threads for parallel metadata loading
......................................................................
Patch Set 5:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/24262/4/docs/topics/impala_metadata.xml
File docs/topics/impala_metadata.xml:
http://gerrit.cloudera.org:8080/#/c/24262/4/docs/topics/impala_metadata.xml@151
PS4, Line 151: ,
> nit: use period "."
Ack
http://gerrit.cloudera.org:8080/#/c/24262/4/docs/topics/impala_metadata.xml@154
PS4, Line 154: Impala
> Using "Coordinator" is more clear. Catalogd might already have the metadata
Ack
--
To view, visit http://gerrit.cloudera.org:8080/24262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic347c3a4628ab4d0c22be689d3a8ff3dfdd22adf
Gerrit-Change-Number: 24262
Gerrit-PatchSet: 5
Gerrit-Owner: Sanjana Malhotra <ma...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@gmail.com>
Gerrit-Reviewer: Sanjana Malhotra <ma...@gmail.com>
Gerrit-Comment-Date: Sat, 09 May 2026 02:41:30 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-14899: [DOCS] Document max stmt metadata loader threads for parallel metadata loading
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24262 )
Change subject: IMPALA-14899: [DOCS] Document max_stmt_metadata_loader_threads for parallel metadata loading
......................................................................
Patch Set 7:
Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/866/
Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstrong@cloudera.com or on this JIRA: IMPALA-7317
--
To view, visit http://gerrit.cloudera.org:8080/24262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic347c3a4628ab4d0c22be689d3a8ff3dfdd22adf
Gerrit-Change-Number: 24262
Gerrit-PatchSet: 7
Gerrit-Owner: Sanjana Malhotra <ma...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@gmail.com>
Gerrit-Reviewer: Sanjana Malhotra <ma...@gmail.com>
Gerrit-Comment-Date: Sat, 09 May 2026 02:43:38 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14899: [DOCS] Document max stmt metadata loader threads for parallel metadata loading
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24262 )
Change subject: IMPALA-14899: [DOCS] Document max_stmt_metadata_loader_threads for parallel metadata loading
......................................................................
Patch Set 8:
Build started: https://jenkins.impala.io/job/gerrit-docs-submit/395/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/24262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic347c3a4628ab4d0c22be689d3a8ff3dfdd22adf
Gerrit-Change-Number: 24262
Gerrit-PatchSet: 8
Gerrit-Owner: Sanjana Malhotra <ma...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@gmail.com>
Gerrit-Reviewer: Sanjana Malhotra <ma...@gmail.com>
Gerrit-Comment-Date: Sun, 10 May 2026 02:24:34 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-14899: [DOCS] Document max stmt metadata loader threads for parallel metadata loading
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24262 )
Change subject: IMPALA-14899: [DOCS] Document max_stmt_metadata_loader_threads for parallel metadata loading
......................................................................
Patch Set 4:
Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/865/
Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstrong@cloudera.com or on this JIRA: IMPALA-7317
--
To view, visit http://gerrit.cloudera.org:8080/24262
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic347c3a4628ab4d0c22be689d3a8ff3dfdd22adf
Gerrit-Change-Number: 24262
Gerrit-PatchSet: 4
Gerrit-Owner: Sanjana Malhotra <ma...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Riza Suminto <ri...@gmail.com>
Gerrit-Reviewer: Sanjana Malhotra <ma...@gmail.com>
Gerrit-Comment-Date: Thu, 07 May 2026 04:26:17 +0000
Gerrit-HasComments: No