You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Grant Henke (Code Review)" <ge...@cloudera.org> on 2020/06/29 14:05:21 UTC

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query 

Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16120


Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................

IMPALA-9903: Reduce Kudu openTable calls per query

This patch reduces the number of Kudu openTable calls for a query
by storing the KuduTable object in the FeKuduTable and using it
in places openTable was previously used.

Reducing the number of openTable calls is important because each
call results in a GetTableSchema RPC to the remote leader Kudu
master. With very high rates of queries against Kudu tables this
can overload the master leading to degraded query performance.

Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
---
M fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java
M fe/src/main/java/org/apache/impala/catalog/KuduTable.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java
M fe/src/main/java/org/apache/impala/planner/KuduScanNode.java
4 files changed, 27 insertions(+), 15 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/20/16120/1
-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 1
Gerrit-Owner: Grant Henke <gr...@apache.org>

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 1: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/6068/


-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 1
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Mon, 29 Jun 2020 15:19:15 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 5:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/16120/5/be/src/exec/kudu-scanner.cc
File be/src/exec/kudu-scanner.cc:

http://gerrit.cloudera.org:8080/#/c/16120/5/be/src/exec/kudu-scanner.cc@286
PS5, Line 286:             scanner_->AddConjunctPredicate(scanner_->GetKuduTable()->NewComparisonPredicate(
> line too long (92 > 90)
Done


http://gerrit.cloudera.org:8080/#/c/16120/5/be/src/exec/kudu-scanner.cc@293
PS5, Line 293:             scanner_->AddConjunctPredicate(scanner_->GetKuduTable()->NewComparisonPredicate(
> line too long (92 > 90)
Done



-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 5
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Thu, 30 Jul 2020 02:50:12 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Vihang Karajgaonkar (Code Review)" <ge...@cloudera.org>.
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 2:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/16120/2//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/16120/2//COMMIT_MSG@9
PS2, Line 9: This patch reduces the number of Kudu openTable calls for a query
           : by storing the KuduTable object in the FeKuduTable and using it
           : in places openTable was previously used.
I think it would be good to be more specific here. Looks like currently we get the KuduTable object every time in the KuduScanNode which can possibly mean multiple times per query.

Do we want to tie this with the invalidate/refresh table mechanism? Or do we want it at once per query granularity?


http://gerrit.cloudera.org:8080/#/c/16120/2/fe/src/main/java/org/apache/impala/catalog/KuduTable.java
File fe/src/main/java/org/apache/impala/catalog/KuduTable.java:

http://gerrit.cloudera.org:8080/#/c/16120/2/fe/src/main/java/org/apache/impala/catalog/KuduTable.java@185
PS2, Line 185:     if (kuduTable_ == null) {
             :       KuduClient client = KuduUtil.getKuduClient(getKuduMasterHosts());
             :       kuduTable_ = client.openTable(getKuduTableName());
             :     }
             :     return kuduTable_;
This would mean that once kuduTable_ is initialized, it never gets refreshed. Even after a explicit refresh command which would call the loadSchemaFromKudu().


http://gerrit.cloudera.org:8080/#/c/16120/2/fe/src/main/java/org/apache/impala/catalog/KuduTable.java@298
PS2, Line 298:       kuduTable = getKuduTable();
This probably should be kept as is otherwise we won't see a updated Kudu schema once the table is loaded.



-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 2
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Mon, 20 Jul 2020 19:40:54 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 5:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/16120/5/be/src/exec/kudu-scanner.cc
File be/src/exec/kudu-scanner.cc:

http://gerrit.cloudera.org:8080/#/c/16120/5/be/src/exec/kudu-scanner.cc@286
PS5, Line 286:             scanner_->AddConjunctPredicate(scanner_->GetKuduTable()->NewComparisonPredicate(
line too long (92 > 90)


http://gerrit.cloudera.org:8080/#/c/16120/5/be/src/exec/kudu-scanner.cc@293
PS5, Line 293:             scanner_->AddConjunctPredicate(scanner_->GetKuduTable()->NewComparisonPredicate(
line too long (92 > 90)



-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 5
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Thu, 30 Jul 2020 02:46:22 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 12:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/16120/12/bin/impala-config.sh
File bin/impala-config.sh:

http://gerrit.cloudera.org:8080/#/c/16120/12/bin/impala-config.sh@71
PS12, Line 71: export IMPALA_TOOLCHAIN_BUILD_ID=52-c3fa626d9b
> The first part is the Jenkins job id. The second part is the commit id of n
I'd found it was in granthenke's fork, but didn't know GitHub could find commits in other forks that way. Handy.



-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 12
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qf...@hotmail.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Tue, 20 Dec 2022 17:26:15 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Hello Qifan Chen, Vihang Karajgaonkar, Tim Armstrong, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/16120

to look at the new patch set (#8).

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................

IMPALA-9903: Reduce Kudu openTable calls per query

This patch reduces the number of Kudu openTable calls for the
lifetime of a query by storing the KuduTable object in the
Analyzer GlobalState and using it in the KuduScanNode.

It does not cache the KuduTable object longer than a single
query, does not impact DDL statements, and does not
introduce the need to invalidate metadata when interacting with
Kudu tables.

Additionally, this patch adjusts the backend scanner to use the
KuduTable instance from the KuduScanner instead of using
openTable to get a new instance.

Reducing the number of openTable calls is important because each
call results in a GetTableSchema RPC to the remote leader Kudu
master. With very high rates of queries against Kudu tables this
can overload the master leading to degraded query performance.

In manual testing this patched reduced the Kudu GetTableSchema
RPC calls to the master from 5 per query to 1 per query.

Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
---
M be/src/exec/kudu-scan-node-base.cc
M be/src/exec/kudu-scan-node-base.h
M be/src/exec/kudu-scanner.cc
M bin/impala-config.sh
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java
M fe/src/main/java/org/apache/impala/planner/KuduScanNode.java
8 files changed, 86 insertions(+), 32 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/20/16120/8
-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 8
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 2:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/16120/2/fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java
File fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java:

http://gerrit.cloudera.org:8080/#/c/16120/2/fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java@166
PS2, Line 166:     public static TResultSet getPartitions(FeKuduTable table)
These are methods that implement the show partitions DDL, so we don't need to worry about caching these - I think to maintain the current semantics we need to open the table once per DDL anyway.

I.e. I don't think we want to change the behaviour of these methods. Maybe we can be a comment to be explicit that we want to fetch the latest version of the Kudu table.


http://gerrit.cloudera.org:8080/#/c/16120/2/fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java
File fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java:

http://gerrit.cloudera.org:8080/#/c/16120/2/fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java@56
PS2, Line 56:   private org.apache.kudu.client.KuduTable kuduTable_;
Caching it in LocalTable makes sense since it's per-query anyway. So this part is fine - caching it here means we can open it once per query then re-use it.


http://gerrit.cloudera.org:8080/#/c/16120/2/fe/src/main/java/org/apache/impala/planner/KuduScanNode.java
File fe/src/main/java/org/apache/impala/planner/KuduScanNode.java:

http://gerrit.cloudera.org:8080/#/c/16120/2/fe/src/main/java/org/apache/impala/planner/KuduScanNode.java@135
PS2, Line 135:       org.apache.kudu.client.KuduTable rpcTable = kuduTable_.getKuduTable();
I think this invocation should go via 'analyzer' to retrieve the per-query cached version of the table. That would work for both catalog implementations.



-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 2
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Fri, 17 Jul 2020 17:33:51 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 12:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/16120/12/bin/impala-config.sh
File bin/impala-config.sh:

http://gerrit.cloudera.org:8080/#/c/16120/12/bin/impala-config.sh@71
PS12, Line 71: export IMPALA_TOOLCHAIN_BUILD_ID=52-c3fa626d9b
> Does anyone know where this toolchain version came from?
The first part is the Jenkins job id. The second part is the commit id of native-toolchain. So I think here it is:
https://github.com/cloudera/native-toolchain/commit/c3fa626d9b



-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 12
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qf...@hotmail.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Tue, 20 Dec 2022 01:16:42 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 12:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/16120/12/bin/impala-config.sh
File bin/impala-config.sh:

http://gerrit.cloudera.org:8080/#/c/16120/12/bin/impala-config.sh@71
PS12, Line 71: export IMPALA_TOOLCHAIN_BUILD_ID=52-c3fa626d9b
Does anyone know where this toolchain version came from?



-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 12
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qf...@hotmail.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Mon, 19 Dec 2022 21:29:49 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 7:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6200/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 7
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Fri, 31 Jul 2020 01:19:52 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 5:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/16120/4/fe/src/main/java/org/apache/impala/analysis/Analyzer.java
File fe/src/main/java/org/apache/impala/analysis/Analyzer.java:

http://gerrit.cloudera.org:8080/#/c/16120/4/fe/src/main/java/org/apache/impala/analysis/Analyzer.java@2817
PS4, Line 2817: 
> We talked about this directly, but if FeKuduTable is a per-query LocalKuduT
Done


http://gerrit.cloudera.org:8080/#/c/16120/4/fe/src/main/java/org/apache/impala/analysis/Analyzer.java@2829
PS4, Line 2829: uTable() != null) {
> Maybe reword as "Unable to open the Kudu table in a scan node". It is also 
Done



-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 5
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Thu, 30 Jul 2020 02:45:30 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 2:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/6448/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 2
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Mon, 29 Jun 2020 15:06:36 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 10: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/6205/


-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 10
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Sat, 01 Aug 2020 00:53:56 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 4:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/6691/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 4
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Wed, 22 Jul 2020 19:40:46 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Hello Qifan Chen, Vihang Karajgaonkar, Tim Armstrong, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/16120

to look at the new patch set (#9).

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................

IMPALA-9903: Reduce Kudu openTable calls per query

This patch reduces the number of Kudu openTable calls for the
lifetime of a query by storing the KuduTable object in the
Analyzer GlobalState and using it in the KuduScanNode.

It does not cache the KuduTable object longer than a single
query, does not impact DDL statements, and does not
introduce the need to invalidate metadata when interacting with
Kudu tables.

Additionally, this patch adjusts the backend scanner to use the
KuduTable instance from the KuduScanner instead of using
openTable to get a new instance.

Reducing the number of openTable calls is important because each
call results in a GetTableSchema RPC to the remote leader Kudu
master. With very high rates of queries against Kudu tables this
can overload the master leading to degraded query performance.

In manual testing this patched reduced the Kudu GetTableSchema
RPC calls to the master from 5 per query to 1 per query.

Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
---
M be/src/exec/kudu-scan-node-base.cc
M be/src/exec/kudu-scan-node-base.h
M be/src/exec/kudu-scanner.cc
M bin/impala-config.sh
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java
M fe/src/main/java/org/apache/impala/planner/KuduScanNode.java
8 files changed, 86 insertions(+), 32 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/20/16120/9
-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 9
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 4: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 4
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Thu, 23 Jul 2020 00:55:34 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 4:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/16120/4/fe/src/main/java/org/apache/impala/analysis/Analyzer.java
File fe/src/main/java/org/apache/impala/analysis/Analyzer.java:

http://gerrit.cloudera.org:8080/#/c/16120/4/fe/src/main/java/org/apache/impala/analysis/Analyzer.java@2817
PS4, Line 2817:   public org.apache.kudu.client.KuduTable getKuduTable(FeKuduTable feKuduTable)
We talked about this directly, but if FeKuduTable is a per-query LocalKuduTable we could cache the KuduTable in there and then retrieve it here.



-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 4
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Wed, 29 Jul 2020 18:43:26 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Hello Qifan Chen, Vihang Karajgaonkar, Tim Armstrong, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/16120

to look at the new patch set (#5).

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................

IMPALA-9903: Reduce Kudu openTable calls per query

This patch reduces the number of Kudu openTable calls for the
lifetime of a query by storing the KuduTable object in the
Analyzer GlobalState and using it in the KuduScanNode.

It does not cache the KuduTable object longer than a single
query, does not impact DDL statements, and does not
introduce the need to invalidate metadata when interacting with
Kudu tables.

Additionally, this patch adjusts the backend scanner to use the
KuduTable instance from the KuduScanner instead of using
openTable to get a new instance.

Reducing the number of openTable calls is important because each
call results in a GetTableSchema RPC to the remote leader Kudu
master. With very high rates of queries against Kudu tables this
can overload the master leading to degraded query performance.

In manual testing this patched reduced the Kudu GetTableSchema
RPC calls to the master from 5 per query to 1 per query.

Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
---
M be/src/exec/kudu-scan-node-base.cc
M be/src/exec/kudu-scan-node-base.h
M be/src/exec/kudu-scanner.cc
M bin/impala-config.sh
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java
M fe/src/main/java/org/apache/impala/planner/KuduScanNode.java
8 files changed, 83 insertions(+), 28 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/20/16120/5
-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 5
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 5:

Build Failed 

https://jenkins.impala.io/job/gerrit-code-review-checks/6735/ : Initial code review checks failed. See linked job for details on the failure.


-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 5
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Thu, 30 Jul 2020 02:57:29 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 2:

I think for the non-local catalog to get query lifetime you'd want to cache the tables in Analyzer.GlobalState that's reachable via any Analyzer object for the query. There's already StmtMetadataLoader and StmtTableCache that load objects from the catalog, that might be a natural place to tack on Kudu table loading since it's already collecting all tables referenced by the query.


-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 2
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Mon, 29 Jun 2020 18:03:21 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 7:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/6744/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 7
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Fri, 31 Jul 2020 01:17:05 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Hello Vihang Karajgaonkar, Tim Armstrong, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/16120

to look at the new patch set (#3).

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................

IMPALA-9903: Reduce Kudu openTable calls per query

This patch reduces the number of Kudu openTable calls for the
lifetime of a query by storing the KuduTable object in the
Analyzer GlobalState and using it in the KuduScanNode.

It does not cache the KuduTable object longer than a single
query, does not impact DDL statements, and does not
introduce the need to invalidate metadata when interacting with
Kudu tables.

Reducing the number of openTable calls is important because each
call results in a GetTableSchema RPC to the remote leader Kudu
master. With very high rates of queries against Kudu tables this
can overload the master leading to degraded query performance.

Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
---
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java
M fe/src/main/java/org/apache/impala/planner/KuduScanNode.java
3 files changed, 34 insertions(+), 5 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/20/16120/3
-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 3
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Hello Qifan Chen, Vihang Karajgaonkar, Tim Armstrong, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/16120

to look at the new patch set (#7).

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................

IMPALA-9903: Reduce Kudu openTable calls per query

This patch reduces the number of Kudu openTable calls for the
lifetime of a query by storing the KuduTable object in the
Analyzer GlobalState and using it in the KuduScanNode.

It does not cache the KuduTable object longer than a single
query, does not impact DDL statements, and does not
introduce the need to invalidate metadata when interacting with
Kudu tables.

Additionally, this patch adjusts the backend scanner to use the
KuduTable instance from the KuduScanner instead of using
openTable to get a new instance.

Reducing the number of openTable calls is important because each
call results in a GetTableSchema RPC to the remote leader Kudu
master. With very high rates of queries against Kudu tables this
can overload the master leading to degraded query performance.

In manual testing this patched reduced the Kudu GetTableSchema
RPC calls to the master from 5 per query to 1 per query.

Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
---
M be/src/exec/kudu-scan-node-base.cc
M be/src/exec/kudu-scan-node-base.h
M be/src/exec/kudu-scanner.cc
M bin/impala-config.sh
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java
M fe/src/main/java/org/apache/impala/planner/KuduScanNode.java
8 files changed, 86 insertions(+), 32 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/20/16120/7
-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 7
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 11:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6209/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 11
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Sat, 01 Aug 2020 18:55:27 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 1:

Build Failed 

https://jenkins.impala.io/job/gerrit-code-review-checks/6447/ : Initial code review checks failed. See linked job for details on the failure.


-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 1
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Mon, 29 Jun 2020 14:27:24 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 1:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6068/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 1
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Mon, 29 Jun 2020 14:13:19 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 6:

Build Failed 

https://jenkins.impala.io/job/gerrit-code-review-checks/6736/ : Initial code review checks failed. See linked job for details on the failure.


-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 6
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Thu, 30 Jul 2020 02:59:06 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 9:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6202/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 9
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Fri, 31 Jul 2020 15:47:54 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 10:

The test failure looks unrelated

query_test.test_mt_dop.TestMtDopScanNode.test_mt_dop_scan_node[protocol: beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format: text/none]


-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 10
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Sat, 01 Aug 2020 01:01:08 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Qifan Chen (Code Review)" <ge...@cloudera.org>.
Qifan Chen has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 4:

(1 comment)

Nice! I wonder similar techniques have been applied to other types of tables.

http://gerrit.cloudera.org:8080/#/c/16120/4/fe/src/main/java/org/apache/impala/analysis/Analyzer.java
File fe/src/main/java/org/apache/impala/analysis/Analyzer.java:

http://gerrit.cloudera.org:8080/#/c/16120/4/fe/src/main/java/org/apache/impala/analysis/Analyzer.java@2829
PS4, Line 2829: initialize the Kudu
Maybe reword as "Unable to open the Kudu table in a scan node". It is also good to name the table name in the error message.



-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 4
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Wed, 29 Jul 2020 17:31:21 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 7: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/6200/


-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 7
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Fri, 31 Jul 2020 06:21:51 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 11: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 11
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Sun, 02 Aug 2020 00:08:47 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Hello Qifan Chen, Vihang Karajgaonkar, Tim Armstrong, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/16120

to look at the new patch set (#6).

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................

IMPALA-9903: Reduce Kudu openTable calls per query

This patch reduces the number of Kudu openTable calls for the
lifetime of a query by storing the KuduTable object in the
Analyzer GlobalState and using it in the KuduScanNode.

It does not cache the KuduTable object longer than a single
query, does not impact DDL statements, and does not
introduce the need to invalidate metadata when interacting with
Kudu tables.

Additionally, this patch adjusts the backend scanner to use the
KuduTable instance from the KuduScanner instead of using
openTable to get a new instance.

Reducing the number of openTable calls is important because each
call results in a GetTableSchema RPC to the remote leader Kudu
master. With very high rates of queries against Kudu tables this
can overload the master leading to degraded query performance.

In manual testing this patched reduced the Kudu GetTableSchema
RPC calls to the master from 5 per query to 1 per query.

Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
---
M be/src/exec/kudu-scan-node-base.cc
M be/src/exec/kudu-scan-node-base.h
M be/src/exec/kudu-scanner.cc
M bin/impala-config.sh
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java
M fe/src/main/java/org/apache/impala/planner/KuduScanNode.java
8 files changed, 86 insertions(+), 31 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/20/16120/6
-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 6
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 2:

Some of these table objects will have longer than query lifetime. 

LocalKuduTable is per-query, but the other classes can live for longer in the catalogd cache or the legacy impalad catalog cache (i.e. when --use_local_catalog=false). Not sure if that's safe or not - I think it could affect behaviour if the Kudu schema changes in the meantime, no?


-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 2
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Mon, 29 Jun 2020 17:01:27 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 9:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/6752/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 9
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Fri, 31 Jul 2020 14:18:12 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 6:

This patch requires http://gerrit.cloudera.org/#/c/16257/


-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 6
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Thu, 30 Jul 2020 02:50:46 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 4:

I ran some manual tests and found this only reduces the GetTableSchema calls from 5 to 4. I will add some logging and find the additional source of requests.


-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 4
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Mon, 27 Jul 2020 15:59:02 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 9: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 9
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Fri, 31 Jul 2020 20:52:39 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 9: Code-Review+2

(1 comment)

Thanks for the patience with iterating on this, I think this is a really good outcome.

RE: Qifan's comment, I think we mostly avoid this from FS-based tables cause that state is all loaded by the catalogd and cached, but Kudu and HBase we fetch some info directly from the storage engine during planning. We did fix some things with HBase a while back like IMPALA-8912 but there could be other inefficiency.

http://gerrit.cloudera.org:8080/#/c/16120/9//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/16120/9//COMMIT_MSG@27
PS9, Line 27: In manual testing this patched reduced the Kudu GetTableSchema
Niiice



-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 9
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Fri, 31 Jul 2020 19:45:06 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 3:

Build Failed 

https://jenkins.impala.io/job/gerrit-code-review-checks/6689/ : Initial code review checks failed. See linked job for details on the failure.


-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 3
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Wed, 22 Jul 2020 18:56:52 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 11: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 11
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Sat, 01 Aug 2020 18:55:26 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 10: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 10
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Fri, 31 Jul 2020 19:45:19 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 10:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6205/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 10
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Fri, 31 Jul 2020 19:45:20 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Hello Vihang Karajgaonkar, Tim Armstrong, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/16120

to look at the new patch set (#4).

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................

IMPALA-9903: Reduce Kudu openTable calls per query

This patch reduces the number of Kudu openTable calls for the
lifetime of a query by storing the KuduTable object in the
Analyzer GlobalState and using it in the KuduScanNode.

It does not cache the KuduTable object longer than a single
query, does not impact DDL statements, and does not
introduce the need to invalidate metadata when interacting with
Kudu tables.

Reducing the number of openTable calls is important because each
call results in a GetTableSchema RPC to the remote leader Kudu
master. With very high rates of queries against Kudu tables this
can overload the master leading to degraded query performance.

Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
---
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java
M fe/src/main/java/org/apache/impala/planner/KuduScanNode.java
3 files changed, 34 insertions(+), 5 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/20/16120/4
-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 4
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 4:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6167/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 4
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Wed, 22 Jul 2020 19:41:49 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 2:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6069/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 2
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Mon, 29 Jun 2020 15:08:05 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 2: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/6069/


-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 2
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Mon, 29 Jun 2020 20:13:13 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Hello Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/16120

to look at the new patch set (#2).

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................

IMPALA-9903: Reduce Kudu openTable calls per query

This patch reduces the number of Kudu openTable calls for a query
by storing the KuduTable object in the FeKuduTable and using it
in places openTable was previously used.

Reducing the number of openTable calls is important because each
call results in a GetTableSchema RPC to the remote leader Kudu
master. With very high rates of queries against Kudu tables this
can overload the master leading to degraded query performance.

Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
---
M fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java
M fe/src/main/java/org/apache/impala/catalog/KuduTable.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java
M fe/src/main/java/org/apache/impala/planner/KuduScanNode.java
4 files changed, 40 insertions(+), 15 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/20/16120/2
-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 2
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................

IMPALA-9903: Reduce Kudu openTable calls per query

This patch reduces the number of Kudu openTable calls for the
lifetime of a query by storing the KuduTable object in the
Analyzer GlobalState and using it in the KuduScanNode.

It does not cache the KuduTable object longer than a single
query, does not impact DDL statements, and does not
introduce the need to invalidate metadata when interacting with
Kudu tables.

Additionally, this patch adjusts the backend scanner to use the
KuduTable instance from the KuduScanner instead of using
openTable to get a new instance.

Reducing the number of openTable calls is important because each
call results in a GetTableSchema RPC to the remote leader Kudu
master. With very high rates of queries against Kudu tables this
can overload the master leading to degraded query performance.

In manual testing this patched reduced the Kudu GetTableSchema
RPC calls to the master from 5 per query to 1 per query.

Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Reviewed-on: http://gerrit.cloudera.org:8080/16120
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M be/src/exec/kudu-scan-node-base.cc
M be/src/exec/kudu-scan-node-base.h
M be/src/exec/kudu-scanner.cc
M bin/impala-config.sh
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java
M fe/src/main/java/org/apache/impala/planner/KuduScanNode.java
8 files changed, 86 insertions(+), 32 deletions(-)

Approvals:
  Impala Public Jenkins: Looks good to me, approved; Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 12
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 11:

Agree it's unrelated - I filed IMPALA-10037


-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 11
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Sat, 01 Aug 2020 18:59:07 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 2:

I definitely would like to error on the side of safety and just improve the per-query behavior.

Would it be reasonable to use the stored KuduTable for LocalKuduTable.java and continue with a lookup every time for KuduTable.java?


-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 2
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Mon, 29 Jun 2020 17:29:39 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 8:

Build Failed 

https://jenkins.impala.io/job/gerrit-code-review-checks/6749/ : Initial code review checks failed. See linked job for details on the failure.


-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 8
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Qifan Chen <qc...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Fri, 31 Jul 2020 13:05:48 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-9903: Reduce Kudu openTable calls per query

Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )

Change subject: IMPALA-9903: Reduce Kudu openTable calls per query
......................................................................


Patch Set 3:

(6 comments)

http://gerrit.cloudera.org:8080/#/c/16120/2//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/16120/2//COMMIT_MSG@9
PS2, Line 9: This patch reduces the number of Kudu openTable calls for the
           : lifetime of a query by storing the KuduTable object in the
           : Analyzer GlobalState and using it in the
> I think it would be good to be more specific here. Looks like currently we 
Done


http://gerrit.cloudera.org:8080/#/c/16120/2/fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java
File fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java:

http://gerrit.cloudera.org:8080/#/c/16120/2/fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java@166
PS2, Line 166:       result.setSchema(resultSchema);
> These are methods that implement the show partitions DDL, so we don't need 
Done


http://gerrit.cloudera.org:8080/#/c/16120/2/fe/src/main/java/org/apache/impala/catalog/KuduTable.java
File fe/src/main/java/org/apache/impala/catalog/KuduTable.java:

http://gerrit.cloudera.org:8080/#/c/16120/2/fe/src/main/java/org/apache/impala/catalog/KuduTable.java@185
PS2, Line 185:   @Override
             :   public List<String> getPrimaryKeyColumnNames() {
             :     return ImmutableList.copyOf(primaryKeyColumnNames_);
             :   }
             : 
> This would mean that once kuduTable_ is initialized, it never gets refreshe
Done


http://gerrit.cloudera.org:8080/#/c/16120/2/fe/src/main/java/org/apache/impala/catalog/KuduTable.java@298
PS2, Line 298:     partitionBy_ = Utils.loadPartitionByParams(kuduTable);
> This probably should be kept as is otherwise we won't see a updated Kudu sc
Done


http://gerrit.cloudera.org:8080/#/c/16120/2/fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java
File fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java:

http://gerrit.cloudera.org:8080/#/c/16120/2/fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java@56
PS2, Line 56:   /**
> Caching it in LocalTable makes sense since it's per-query anyway. So this p
If we are going the analyzer route I don't think this is needed right?


http://gerrit.cloudera.org:8080/#/c/16120/2/fe/src/main/java/org/apache/impala/planner/KuduScanNode.java
File fe/src/main/java/org/apache/impala/planner/KuduScanNode.java:

http://gerrit.cloudera.org:8080/#/c/16120/2/fe/src/main/java/org/apache/impala/planner/KuduScanNode.java@135
PS2, Line 135:       // Get the KuduTable from the analyzer to retrieve the cached KuduTable
> I think this invocation should go via 'analyzer' to retrieve the per-query 
Done



-- 
To view, visit http://gerrit.cloudera.org:8080/16120
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63
Gerrit-Change-Number: 16120
Gerrit-PatchSet: 3
Gerrit-Owner: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Vihang Karajgaonkar <vi...@cloudera.com>
Gerrit-Comment-Date: Wed, 22 Jul 2020 18:39:14 +0000
Gerrit-HasComments: Yes