You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Anonymous Coward (Code Review)" <ge...@cloudera.org> on 2021/02/20 15:17:03 UTC
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
scarlin@cloudera.com has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17092
Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions.
......................................................................
IMPALA-10524: Changes to HdfsPartition for third party extensions.
Some changes are needed to HdfsPartition and other related classes
to allow for third party extensions. These changes include:
- A protected constructor which will allow a subclass to instantiate
HdfsPartition using its own Builder.
- Various changes of permissions to methods and variables to allow
third party extension visibility.
- Creation of the getHostIndex() method to allow the subclass to
override how the hostIndexes are retrieved.
- Added a new default method "getFileSystem()" to FeFsPartition which
will allow the third party extension to override how the filesystem
is obtained from the partition object.
Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46
---
M fe/src/main/java/org/apache/impala/catalog/FeFsPartition.java
M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java
M fe/src/main/java/org/apache/impala/catalog/HdfsPartitionLocationCompressor.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalFsPartition.java
M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
5 files changed, 50 insertions(+), 3 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/92/17092/3
--
To view, visit http://gerrit.cloudera.org:8080/17092
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46
Gerrit-Change-Number: 17092
Gerrit-PatchSet: 3
Gerrit-Owner: Anonymous Coward <sc...@cloudera.com>
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17092 )
Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions.
......................................................................
IMPALA-10524: Changes to HdfsPartition for third party extensions.
Some changes are needed to HdfsPartition and other related classes
to allow for third party extensions. These changes include:
- A protected constructor which will allow a subclass to instantiate
HdfsPartition using its own Builder.
- Various changes of permissions to methods and variables to allow
third party extension visibility.
- Creation of the getHostIndex() method to allow the subclass to
override how the hostIndexes are retrieved.
- Added a new default method "getFileSystem()" to FeFsPartition which
will allow the third party extension to override how the filesystem
is obtained from the partition object.
Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46
Reviewed-on: http://gerrit.cloudera.org:8080/17092
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M fe/src/main/java/org/apache/impala/analysis/AnalyticWindow.java
M fe/src/main/java/org/apache/impala/analysis/StatementBase.java
M fe/src/main/java/org/apache/impala/catalog/FeFsPartition.java
M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java
M fe/src/main/java/org/apache/impala/catalog/HdfsPartitionLocationCompressor.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalFsPartition.java
M fe/src/main/java/org/apache/impala/planner/AnalyticEvalNode.java
M fe/src/main/java/org/apache/impala/planner/CardinalityCheckNode.java
M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
M fe/src/main/java/org/apache/impala/planner/KuduScanNode.java
M fe/src/main/java/org/apache/impala/planner/PlanNode.java
M fe/src/main/java/org/apache/impala/planner/Planner.java
M fe/src/main/java/org/apache/impala/planner/SortNode.java
M fe/src/main/java/org/apache/impala/planner/SubplanNode.java
14 files changed, 82 insertions(+), 25 deletions(-)
Approvals:
Impala Public Jenkins: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/17092
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46
Gerrit-Change-Number: 17092
Gerrit-PatchSet: 13
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Steve Carlin <sc...@cloudera.com>
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/17092 )
Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions.
......................................................................
Patch Set 10:
(2 comments)
The change in HdfsTable may introduce an issue. Other changes look good to me.
http://gerrit.cloudera.org:8080/#/c/17092/10/be/src/rpc/hs2-http-test.cc
File be/src/rpc/hs2-http-test.cc:
http://gerrit.cloudera.org:8080/#/c/17092/10/be/src/rpc/hs2-http-test.cc@51
PS10, Line 51: virtual void GetBackendConfig(TGetBackendConfigResp& return_val,
Is this used somewhere?
http://gerrit.cloudera.org:8080/#/c/17092/10/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
File fe/src/main/java/org/apache/impala/catalog/HdfsTable.java:
http://gerrit.cloudera.org:8080/#/c/17092/10/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@1719
PS10, Line 1719: if (isSchemaLoaded_) return;
I don't think we can do this. This means the table schema will only be loaded once. However, when executiing REFRESH or some DDL commands, we need to reload these to update the schema.
Here's an example:
# Create a table in Impala and run any query on it to make its metadata loaded
impala> create table my_tbl (id int, name string);
impala> desc my_tbl;
# Change its schema in Hive
beeline> alter table my_tbl add columns (address string);
# Refresh in Impala. Impala should reload its metadata and get the latest schema.
impala> refresh my_tbl;
impala> desc my_tbl;
--
To view, visit http://gerrit.cloudera.org:8080/17092
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46
Gerrit-Change-Number: 17092
Gerrit-PatchSet: 10
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Steve Carlin <sc...@cloudera.com>
Gerrit-Comment-Date: Mon, 01 Mar 2021 13:29:34 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17092 )
Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions.
......................................................................
Patch Set 7:
Build Failed
https://jenkins.impala.io/job/gerrit-code-review-checks/8256/ : Initial code review checks failed. See linked job for details on the failure.
--
To view, visit http://gerrit.cloudera.org:8080/17092
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46
Gerrit-Change-Number: 17092
Gerrit-PatchSet: 7
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Steve Carlin <sc...@cloudera.com>
Gerrit-Comment-Date: Fri, 26 Feb 2021 19:49:38 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17092 )
Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions.
......................................................................
Patch Set 12: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/17092
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46
Gerrit-Change-Number: 17092
Gerrit-PatchSet: 12
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Steve Carlin <sc...@cloudera.com>
Gerrit-Comment-Date: Tue, 02 Mar 2021 05:57:25 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17092 )
Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions.
......................................................................
Patch Set 6:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/17092/6/fe/src/main/java/org/apache/impala/analysis/StatementBase.java
File fe/src/main/java/org/apache/impala/analysis/StatementBase.java:
http://gerrit.cloudera.org:8080/#/c/17092/6/fe/src/main/java/org/apache/impala/analysis/StatementBase.java@203
PS6, Line 203: public static Expr checkTypeCompatibility(String dstTableName, Column dstCol, Expr srcExpr,
line too long (93 > 90)
--
To view, visit http://gerrit.cloudera.org:8080/17092
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46
Gerrit-Change-Number: 17092
Gerrit-PatchSet: 6
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Steve Carlin <sc...@cloudera.com>
Gerrit-Comment-Date: Fri, 26 Feb 2021 18:18:30 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17092 )
Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions.
......................................................................
Patch Set 11:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/8271/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/17092
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46
Gerrit-Change-Number: 17092
Gerrit-PatchSet: 11
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Steve Carlin <sc...@cloudera.com>
Gerrit-Comment-Date: Mon, 01 Mar 2021 22:21:51 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17092 )
Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions.
......................................................................
Patch Set 3:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/8177/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/17092
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46
Gerrit-Change-Number: 17092
Gerrit-PatchSet: 3
Gerrit-Owner: Anonymous Coward <sc...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Sat, 20 Feb 2021 15:38:11 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17092 )
Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions.
......................................................................
Patch Set 7:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/17092/7/fe/src/main/java/org/apache/impala/analysis/StatementBase.java
File fe/src/main/java/org/apache/impala/analysis/StatementBase.java:
http://gerrit.cloudera.org:8080/#/c/17092/7/fe/src/main/java/org/apache/impala/analysis/StatementBase.java@203
PS7, Line 203: public static Expr checkTypeCompatibility(String dstTableName, Column dstCol, Expr srcExpr,
line too long (93 > 90)
--
To view, visit http://gerrit.cloudera.org:8080/17092
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46
Gerrit-Change-Number: 17092
Gerrit-PatchSet: 7
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Steve Carlin <sc...@cloudera.com>
Gerrit-Comment-Date: Fri, 26 Feb 2021 19:38:51 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17092 )
Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions.
......................................................................
Patch Set 6:
Build Failed
https://jenkins.impala.io/job/gerrit-code-review-checks/8254/ : Initial code review checks failed. See linked job for details on the failure.
--
To view, visit http://gerrit.cloudera.org:8080/17092
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46
Gerrit-Change-Number: 17092
Gerrit-PatchSet: 6
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Steve Carlin <sc...@cloudera.com>
Gerrit-Comment-Date: Fri, 26 Feb 2021 18:28:12 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Posted by "Steve Carlin (Code Review)" <ge...@cloudera.org>.
Steve Carlin has posted comments on this change. ( http://gerrit.cloudera.org:8080/17092 )
Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions.
......................................................................
Patch Set 3:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/17092/3/fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java
File fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java:
http://gerrit.cloudera.org:8080/#/c/17092/3/fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java@699
PS3, Line 699: this(table, partitionIdCounter_.getAndIncrement(), prevId, partName, partitionKeyValues,
> line too long (92 > 90)
Done
http://gerrit.cloudera.org:8080/#/c/17092/3/fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java@701
PS3, Line 701: encodedDeleteFileDescriptors, location, isMarkedCached, accessLevel, hmsParameters,
> line too long (91 > 90)
Done
http://gerrit.cloudera.org:8080/#/c/17092/3/fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java@702
PS3, Line 702: cachedMsPartitionDescriptor, partitionStats, hasIncrementalStats, numRows, writeId,
> line too long (91 > 90)
Done
--
To view, visit http://gerrit.cloudera.org:8080/17092
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46
Gerrit-Change-Number: 17092
Gerrit-PatchSet: 3
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Steve Carlin <sc...@cloudera.com>
Gerrit-Comment-Date: Sat, 20 Feb 2021 17:23:23 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17092 )
Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions.
......................................................................
Patch Set 12: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/17092
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46
Gerrit-Change-Number: 17092
Gerrit-PatchSet: 12
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Steve Carlin <sc...@cloudera.com>
Gerrit-Comment-Date: Tue, 02 Mar 2021 00:11:08 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Posted by "Steve Carlin (Code Review)" <ge...@cloudera.org>.
Steve Carlin has uploaded a new patch set (#11). ( http://gerrit.cloudera.org:8080/17092 )
Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions.
......................................................................
IMPALA-10524: Changes to HdfsPartition for third party extensions.
Some changes are needed to HdfsPartition and other related classes
to allow for third party extensions. These changes include:
- A protected constructor which will allow a subclass to instantiate
HdfsPartition using its own Builder.
- Various changes of permissions to methods and variables to allow
third party extension visibility.
- Creation of the getHostIndex() method to allow the subclass to
override how the hostIndexes are retrieved.
- Added a new default method "getFileSystem()" to FeFsPartition which
will allow the third party extension to override how the filesystem
is obtained from the partition object.
Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46
---
M fe/src/main/java/org/apache/impala/analysis/AnalyticWindow.java
M fe/src/main/java/org/apache/impala/analysis/StatementBase.java
M fe/src/main/java/org/apache/impala/catalog/FeFsPartition.java
M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java
M fe/src/main/java/org/apache/impala/catalog/HdfsPartitionLocationCompressor.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalFsPartition.java
M fe/src/main/java/org/apache/impala/planner/AnalyticEvalNode.java
M fe/src/main/java/org/apache/impala/planner/CardinalityCheckNode.java
M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
M fe/src/main/java/org/apache/impala/planner/KuduScanNode.java
M fe/src/main/java/org/apache/impala/planner/PlanNode.java
M fe/src/main/java/org/apache/impala/planner/Planner.java
M fe/src/main/java/org/apache/impala/planner/SortNode.java
M fe/src/main/java/org/apache/impala/planner/SubplanNode.java
14 files changed, 82 insertions(+), 25 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/92/17092/11
--
To view, visit http://gerrit.cloudera.org:8080/17092
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46
Gerrit-Change-Number: 17092
Gerrit-PatchSet: 11
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Steve Carlin <sc...@cloudera.com>
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Posted by "Steve Carlin (Code Review)" <ge...@cloudera.org>.
Steve Carlin has uploaded a new patch set (#10). ( http://gerrit.cloudera.org:8080/17092 )
Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions.
......................................................................
IMPALA-10524: Changes to HdfsPartition for third party extensions.
Some changes are needed to HdfsPartition and other related classes
to allow for third party extensions. These changes include:
- A protected constructor which will allow a subclass to instantiate
HdfsPartition using its own Builder.
- Various changes of permissions to methods and variables to allow
third party extension visibility.
- Creation of the getHostIndex() method to allow the subclass to
override how the hostIndexes are retrieved.
- Added a new default method "getFileSystem()" to FeFsPartition which
will allow the third party extension to override how the filesystem
is obtained from the partition object.
Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46
---
M be/src/rpc/hs2-http-test.cc
M fe/src/main/java/org/apache/impala/analysis/AnalyticWindow.java
M fe/src/main/java/org/apache/impala/analysis/StatementBase.java
M fe/src/main/java/org/apache/impala/catalog/FeFsPartition.java
M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java
M fe/src/main/java/org/apache/impala/catalog/HdfsPartitionLocationCompressor.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalFsPartition.java
M fe/src/main/java/org/apache/impala/planner/AnalyticEvalNode.java
M fe/src/main/java/org/apache/impala/planner/CardinalityCheckNode.java
M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
M fe/src/main/java/org/apache/impala/planner/KuduScanNode.java
M fe/src/main/java/org/apache/impala/planner/PlanNode.java
M fe/src/main/java/org/apache/impala/planner/Planner.java
M fe/src/main/java/org/apache/impala/planner/SortNode.java
M fe/src/main/java/org/apache/impala/planner/SubplanNode.java
16 files changed, 85 insertions(+), 25 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/92/17092/10
--
To view, visit http://gerrit.cloudera.org:8080/17092
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46
Gerrit-Change-Number: 17092
Gerrit-PatchSet: 10
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Steve Carlin <sc...@cloudera.com>
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17092 )
Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions.
......................................................................
Patch Set 5:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/8200/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/17092
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46
Gerrit-Change-Number: 17092
Gerrit-PatchSet: 5
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Steve Carlin <sc...@cloudera.com>
Gerrit-Comment-Date: Tue, 23 Feb 2021 03:01:32 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Posted by "Steve Carlin (Code Review)" <ge...@cloudera.org>.
Steve Carlin has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/17092 )
Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions.
......................................................................
IMPALA-10524: Changes to HdfsPartition for third party extensions.
Some changes are needed to HdfsPartition and other related classes
to allow for third party extensions. These changes include:
- A protected constructor which will allow a subclass to instantiate
HdfsPartition using its own Builder.
- Various changes of permissions to methods and variables to allow
third party extension visibility.
- Creation of the getHostIndex() method to allow the subclass to
override how the hostIndexes are retrieved.
- Added a new default method "getFileSystem()" to FeFsPartition which
will allow the third party extension to override how the filesystem
is obtained from the partition object.
Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46
---
M fe/src/main/java/org/apache/impala/catalog/FeFsPartition.java
M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java
M fe/src/main/java/org/apache/impala/catalog/HdfsPartitionLocationCompressor.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalFsPartition.java
M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
5 files changed, 50 insertions(+), 3 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/92/17092/4
--
To view, visit http://gerrit.cloudera.org:8080/17092
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46
Gerrit-Change-Number: 17092
Gerrit-PatchSet: 4
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Posted by "Steve Carlin (Code Review)" <ge...@cloudera.org>.
Steve Carlin has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/17092 )
Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions.
......................................................................
IMPALA-10524: Changes to HdfsPartition for third party extensions.
Some changes are needed to HdfsPartition and other related classes
to allow for third party extensions. These changes include:
- A protected constructor which will allow a subclass to instantiate
HdfsPartition using its own Builder.
- Various changes of permissions to methods and variables to allow
third party extension visibility.
- Creation of the getHostIndex() method to allow the subclass to
override how the hostIndexes are retrieved.
- Added a new default method "getFileSystem()" to FeFsPartition which
will allow the third party extension to override how the filesystem
is obtained from the partition object.
Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46
---
M fe/src/main/java/org/apache/impala/catalog/FeFsPartition.java
M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java
M fe/src/main/java/org/apache/impala/catalog/HdfsPartitionLocationCompressor.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalFsPartition.java
M fe/src/main/java/org/apache/impala/planner/CardinalityCheckNode.java
M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
6 files changed, 51 insertions(+), 4 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/92/17092/5
--
To view, visit http://gerrit.cloudera.org:8080/17092
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46
Gerrit-Change-Number: 17092
Gerrit-PatchSet: 5
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Steve Carlin <sc...@cloudera.com>
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Posted by "Steve Carlin (Code Review)" <ge...@cloudera.org>.
Steve Carlin has uploaded a new patch set (#7). ( http://gerrit.cloudera.org:8080/17092 )
Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions.
......................................................................
IMPALA-10524: Changes to HdfsPartition for third party extensions.
Some changes are needed to HdfsPartition and other related classes
to allow for third party extensions. These changes include:
- A protected constructor which will allow a subclass to instantiate
HdfsPartition using its own Builder.
- Various changes of permissions to methods and variables to allow
third party extension visibility.
- Creation of the getHostIndex() method to allow the subclass to
override how the hostIndexes are retrieved.
- Added a new default method "getFileSystem()" to FeFsPartition which
will allow the third party extension to override how the filesystem
is obtained from the partition object.
Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46
---
M be/src/rpc/hs2-http-test.cc
M fe/src/main/java/org/apache/impala/analysis/AnalyticWindow.java
M fe/src/main/java/org/apache/impala/analysis/StatementBase.java
M fe/src/main/java/org/apache/impala/catalog/FeFsPartition.java
M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java
M fe/src/main/java/org/apache/impala/catalog/HdfsPartitionLocationCompressor.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalFsPartition.java
M fe/src/main/java/org/apache/impala/planner/CardinalityCheckNode.java
M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
M fe/src/main/java/org/apache/impala/planner/Planner.java
11 files changed, 78 insertions(+), 19 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/92/17092/7
--
To view, visit http://gerrit.cloudera.org:8080/17092
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46
Gerrit-Change-Number: 17092
Gerrit-PatchSet: 7
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Steve Carlin <sc...@cloudera.com>
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17092 )
Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions.
......................................................................
Patch Set 3:
(3 comments)
http://gerrit.cloudera.org:8080/#/c/17092/3/fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java
File fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java:
http://gerrit.cloudera.org:8080/#/c/17092/3/fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java@699
PS3, Line 699: this(table, partitionIdCounter_.getAndIncrement(), prevId, partName, partitionKeyValues,
line too long (92 > 90)
http://gerrit.cloudera.org:8080/#/c/17092/3/fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java@701
PS3, Line 701: encodedDeleteFileDescriptors, location, isMarkedCached, accessLevel, hmsParameters,
line too long (91 > 90)
http://gerrit.cloudera.org:8080/#/c/17092/3/fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java@702
PS3, Line 702: cachedMsPartitionDescriptor, partitionStats, hasIncrementalStats, numRows, writeId,
line too long (91 > 90)
--
To view, visit http://gerrit.cloudera.org:8080/17092
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46
Gerrit-Change-Number: 17092
Gerrit-PatchSet: 3
Gerrit-Owner: Anonymous Coward <sc...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Sat, 20 Feb 2021 15:17:50 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17092 )
Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions.
......................................................................
Patch Set 4:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/8179/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/17092
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46
Gerrit-Change-Number: 17092
Gerrit-PatchSet: 4
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Steve Carlin <sc...@cloudera.com>
Gerrit-Comment-Date: Sat, 20 Feb 2021 17:49:05 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17092 )
Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions.
......................................................................
Patch Set 12:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6929/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/17092
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46
Gerrit-Change-Number: 17092
Gerrit-PatchSet: 12
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Steve Carlin <sc...@cloudera.com>
Gerrit-Comment-Date: Tue, 02 Mar 2021 00:11:09 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Posted by "Steve Carlin (Code Review)" <ge...@cloudera.org>.
Steve Carlin has uploaded a new patch set (#6). ( http://gerrit.cloudera.org:8080/17092 )
Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions.
......................................................................
IMPALA-10524: Changes to HdfsPartition for third party extensions.
Some changes are needed to HdfsPartition and other related classes
to allow for third party extensions. These changes include:
- A protected constructor which will allow a subclass to instantiate
HdfsPartition using its own Builder.
- Various changes of permissions to methods and variables to allow
third party extension visibility.
- Creation of the getHostIndex() method to allow the subclass to
override how the hostIndexes are retrieved.
- Added a new default method "getFileSystem()" to FeFsPartition which
will allow the third party extension to override how the filesystem
is obtained from the partition object.
Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46
---
M be/src/rpc/hs2-http-test.cc
M fe/src/main/java/org/apache/impala/analysis/AnalyticWindow.java
M fe/src/main/java/org/apache/impala/analysis/StatementBase.java
M fe/src/main/java/org/apache/impala/catalog/FeFsPartition.java
M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java
M fe/src/main/java/org/apache/impala/catalog/HdfsPartitionLocationCompressor.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalFsPartition.java
M fe/src/main/java/org/apache/impala/planner/CardinalityCheckNode.java
M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
M fe/src/main/java/org/apache/impala/planner/Planner.java
10 files changed, 77 insertions(+), 19 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/92/17092/6
--
To view, visit http://gerrit.cloudera.org:8080/17092
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46
Gerrit-Change-Number: 17092
Gerrit-PatchSet: 6
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Steve Carlin <sc...@cloudera.com>
[Impala-ASF-CR] IMPALA-10524: Changes to HdfsPartition for third party extensions.
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/17092 )
Change subject: IMPALA-10524: Changes to HdfsPartition for third party extensions.
......................................................................
Patch Set 11: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/17092
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5a792642f27228118ac8f2e8ef98e8ba7aee4a46
Gerrit-Change-Number: 17092
Gerrit-PatchSet: 11
Gerrit-Owner: Steve Carlin <sc...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Steve Carlin <sc...@cloudera.com>
Gerrit-Comment-Date: Tue, 02 Mar 2021 00:10:30 +0000
Gerrit-HasComments: No