You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Tim Armstrong (Code Review)" <ge...@cloudera.org> on 2018/10/31 15:54:50 UTC
[Impala-ASF-CR] IMPALA-7792: fix disabling of ORC scans
Tim Armstrong has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11835
Change subject: IMPALA-7792: fix disabling of ORC scans
......................................................................
IMPALA-7792: fix disabling of ORC scans
The previous approach could lead to hangs or cryptic error messages
because it removed the ORC data type from a lookup table.
Instead check explicitly in the planner for ORC scans and throw a
more helpful error message.
Testing:
Added custom cluster test to exercise code and check error message.
Change-Id: I209e79b18745c48d0182800a916d6566083f4609
---
M fe/src/main/java/org/apache/impala/catalog/HdfsFileFormat.java
M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
A testdata/workloads/functional-query/queries/QueryTest/disable-orc-scanner.test
A tests/custom_cluster/test_disable_features.py
5 files changed, 77 insertions(+), 20 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/35/11835/3
--
To view, visit http://gerrit.cloudera.org:8080/11835
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I209e79b18745c48d0182800a916d6566083f4609
Gerrit-Change-Number: 11835
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] IMPALA-7792: fix disabling of ORC scans
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11835 )
Change subject: IMPALA-7792: fix disabling of ORC scans
......................................................................
Patch Set 3:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/1227/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/11835
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I209e79b18745c48d0182800a916d6566083f4609
Gerrit-Change-Number: 11835
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Comment-Date: Wed, 31 Oct 2018 16:27:29 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7792: fix disabling of ORC scans
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11835 )
Change subject: IMPALA-7792: fix disabling of ORC scans
......................................................................
Patch Set 6:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3395/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/11835
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I209e79b18745c48d0182800a916d6566083f4609
Gerrit-Change-Number: 11835
Gerrit-PatchSet: 6
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Wed, 31 Oct 2018 23:31:42 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7792: fix disabling of ORC scans
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11835 )
Change subject: IMPALA-7792: fix disabling of ORC scans
......................................................................
Patch Set 7: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/11835
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I209e79b18745c48d0182800a916d6566083f4609
Gerrit-Change-Number: 11835
Gerrit-PatchSet: 7
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Thu, 01 Nov 2018 07:30:56 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7792: fix disabling of ORC scans
Posted by "Bharath Vissapragada (Code Review)" <ge...@cloudera.org>.
Bharath Vissapragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/11835 )
Change subject: IMPALA-7792: fix disabling of ORC scans
......................................................................
Patch Set 4:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/11835/4/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
File fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java:
http://gerrit.cloudera.org:8080/#/c/11835/4/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java@414
PS4, Line 414: if (!BackendConfig.INSTANCE.isOrcScannerEnabled()) {
: for (FeFsPartition part: partitions_) {
: if (part.getInputFormatDescriptor().getFileFormat() == HdfsFileFormat.ORC) {
: throw new NotImplementedException(
: "ORC scans are disabled by --enable_orc_scanner flag");
: }
: }
: }
Do we need this block? Shouldn't the check below inside the loop work?
http://gerrit.cloudera.org:8080/#/c/11835/4/tests/custom_cluster/test_disable_features.py
File tests/custom_cluster/test_disable_features.py:
http://gerrit.cloudera.org:8080/#/c/11835/4/tests/custom_cluster/test_disable_features.py@32
PS4, Line 32: vector
nit: Not sure if the intention here was to run with multiple dimensions but I think custom cluster by default limits to only specific set of dimensions (I could be wrong on this).
--
To view, visit http://gerrit.cloudera.org:8080/11835
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I209e79b18745c48d0182800a916d6566083f4609
Gerrit-Change-Number: 11835
Gerrit-PatchSet: 4
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Wed, 31 Oct 2018 23:11:04 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7792: fix disabling of ORC scans
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11835 )
Change subject: IMPALA-7792: fix disabling of ORC scans
......................................................................
Patch Set 7: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/11835
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I209e79b18745c48d0182800a916d6566083f4609
Gerrit-Change-Number: 11835
Gerrit-PatchSet: 7
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Thu, 01 Nov 2018 03:51:49 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7792: fix disabling of ORC scans
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11835 )
Change subject: IMPALA-7792: fix disabling of ORC scans
......................................................................
Patch Set 6: Verified-1
Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/3395/
--
To view, visit http://gerrit.cloudera.org:8080/11835
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I209e79b18745c48d0182800a916d6566083f4609
Gerrit-Change-Number: 11835
Gerrit-PatchSet: 6
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Thu, 01 Nov 2018 03:14:47 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7792: fix disabling of ORC scans
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11835 )
Change subject: IMPALA-7792: fix disabling of ORC scans
......................................................................
Patch Set 7:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3398/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/11835
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I209e79b18745c48d0182800a916d6566083f4609
Gerrit-Change-Number: 11835
Gerrit-PatchSet: 7
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Thu, 01 Nov 2018 03:51:50 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7792: fix disabling of ORC scans
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11835 )
Change subject: IMPALA-7792: fix disabling of ORC scans
......................................................................
Patch Set 4:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/1237/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/11835
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I209e79b18745c48d0182800a916d6566083f4609
Gerrit-Change-Number: 11835
Gerrit-PatchSet: 4
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Wed, 31 Oct 2018 23:09:18 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7792: fix disabling of ORC scans
Posted by "Bharath Vissapragada (Code Review)" <ge...@cloudera.org>.
Bharath Vissapragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/11835 )
Change subject: IMPALA-7792: fix disabling of ORC scans
......................................................................
Patch Set 4:
I can review this one. Lemme dig into the previous approach.
--
To view, visit http://gerrit.cloudera.org:8080/11835
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I209e79b18745c48d0182800a916d6566083f4609
Gerrit-Change-Number: 11835
Gerrit-PatchSet: 4
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Wed, 31 Oct 2018 22:32:35 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7792: fix disabling of ORC scans
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11835 )
Change subject: IMPALA-7792: fix disabling of ORC scans
......................................................................
IMPALA-7792: fix disabling of ORC scans
The previous approach could lead to hangs or cryptic error messages
because it removed the ORC data type from a lookup table.
Instead check explicitly in the planner for ORC scans and throw a
more helpful error message.
Testing:
Added custom cluster test to exercise code and check error message.
Change-Id: I209e79b18745c48d0182800a916d6566083f4609
Reviewed-on: http://gerrit.cloudera.org:8080/11835
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M fe/src/main/java/org/apache/impala/catalog/HdfsFileFormat.java
M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
A testdata/workloads/functional-query/queries/QueryTest/disable-orc-scanner.test
A tests/custom_cluster/test_disable_features.py
5 files changed, 71 insertions(+), 20 deletions(-)
Approvals:
Impala Public Jenkins: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/11835
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I209e79b18745c48d0182800a916d6566083f4609
Gerrit-Change-Number: 11835
Gerrit-PatchSet: 8
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
[Impala-ASF-CR] IMPALA-7792: fix disabling of ORC scans
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Hello Philip Zeyliger, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/11835
to look at the new patch set (#4).
Change subject: IMPALA-7792: fix disabling of ORC scans
......................................................................
IMPALA-7792: fix disabling of ORC scans
The previous approach could lead to hangs or cryptic error messages
because it removed the ORC data type from a lookup table.
Instead check explicitly in the planner for ORC scans and throw a
more helpful error message.
Testing:
Added custom cluster test to exercise code and check error message.
Change-Id: I209e79b18745c48d0182800a916d6566083f4609
---
M fe/src/main/java/org/apache/impala/catalog/HdfsFileFormat.java
M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
A testdata/workloads/functional-query/queries/QueryTest/disable-orc-scanner.test
A tests/custom_cluster/test_disable_features.py
5 files changed, 78 insertions(+), 20 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/35/11835/4
--
To view, visit http://gerrit.cloudera.org:8080/11835
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I209e79b18745c48d0182800a916d6566083f4609
Gerrit-Change-Number: 11835
Gerrit-PatchSet: 4
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
[Impala-ASF-CR] IMPALA-7792: fix disabling of ORC scans
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11835 )
Change subject: IMPALA-7792: fix disabling of ORC scans
......................................................................
Patch Set 4:
Any chance you could review this Bharath? I can provide any context if needed.
--
To view, visit http://gerrit.cloudera.org:8080/11835
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I209e79b18745c48d0182800a916d6566083f4609
Gerrit-Change-Number: 11835
Gerrit-PatchSet: 4
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Wed, 31 Oct 2018 22:19:57 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7792: fix disabling of ORC scans
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Hello Bharath Vissapragada, Philip Zeyliger, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/11835
to look at the new patch set (#5).
Change subject: IMPALA-7792: fix disabling of ORC scans
......................................................................
IMPALA-7792: fix disabling of ORC scans
The previous approach could lead to hangs or cryptic error messages
because it removed the ORC data type from a lookup table.
Instead check explicitly in the planner for ORC scans and throw a
more helpful error message.
Testing:
Added custom cluster test to exercise code and check error message.
Change-Id: I209e79b18745c48d0182800a916d6566083f4609
---
M fe/src/main/java/org/apache/impala/catalog/HdfsFileFormat.java
M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
A testdata/workloads/functional-query/queries/QueryTest/disable-orc-scanner.test
A tests/custom_cluster/test_disable_features.py
5 files changed, 71 insertions(+), 20 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/35/11835/5
--
To view, visit http://gerrit.cloudera.org:8080/11835
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I209e79b18745c48d0182800a916d6566083f4609
Gerrit-Change-Number: 11835
Gerrit-PatchSet: 5
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
[Impala-ASF-CR] IMPALA-7792: fix disabling of ORC scans
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11835 )
Change subject: IMPALA-7792: fix disabling of ORC scans
......................................................................
Patch Set 5:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/1241/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/11835
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I209e79b18745c48d0182800a916d6566083f4609
Gerrit-Change-Number: 11835
Gerrit-PatchSet: 5
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Wed, 31 Oct 2018 23:44:24 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7792: fix disabling of ORC scans
Posted by "Bharath Vissapragada (Code Review)" <ge...@cloudera.org>.
Bharath Vissapragada has posted comments on this change. ( http://gerrit.cloudera.org:8080/11835 )
Change subject: IMPALA-7792: fix disabling of ORC scans
......................................................................
Patch Set 5: Code-Review+2
(2 comments)
http://gerrit.cloudera.org:8080/#/c/11835/4/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
File fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java:
http://gerrit.cloudera.org:8080/#/c/11835/4/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java@414
PS4, Line 414: if (!BackendConfig.INSTANCE.isOrcScannerEnabled()) {
: for (FeFsPartition part: partitions_) {
: if (part.getInputFormatDescriptor().getFileFormat() == HdfsFileFormat.ORC) {
: throw new NotImplementedException(
: "ORC scans are disabled by --enable_orc_scanner flag");
: }
: }
: }
> I should have removed the block below. The below check doesn't work because
ah, ya!
http://gerrit.cloudera.org:8080/#/c/11835/4/tests/custom_cluster/test_disable_features.py
File tests/custom_cluster/test_disable_features.py:
http://gerrit.cloudera.org:8080/#/c/11835/4/tests/custom_cluster/test_disable_features.py@32
PS4, Line 32: vector
> I copied this from a different custom cluster test. Are you saying that it'
wfm, I think even with vector it only runs a specific dimension and not all combinations like in a normal e-e test.
--
To view, visit http://gerrit.cloudera.org:8080/11835
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I209e79b18745c48d0182800a916d6566083f4609
Gerrit-Change-Number: 11835
Gerrit-PatchSet: 5
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Wed, 31 Oct 2018 23:28:07 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7792: fix disabling of ORC scans
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11835 )
Change subject: IMPALA-7792: fix disabling of ORC scans
......................................................................
Patch Set 3:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/11835/3/tests/custom_cluster/test_disable_features.py
File tests/custom_cluster/test_disable_features.py:
http://gerrit.cloudera.org:8080/#/c/11835/3/tests/custom_cluster/test_disable_features.py@22
PS3, Line 22: class TestDisableFeatures(CustomClusterTestSuite):
flake8: E302 expected 2 blank lines, found 1
--
To view, visit http://gerrit.cloudera.org:8080/11835
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I209e79b18745c48d0182800a916d6566083f4609
Gerrit-Change-Number: 11835
Gerrit-PatchSet: 3
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Comment-Date: Wed, 31 Oct 2018 15:55:39 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7792: fix disabling of ORC scans
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11835 )
Change subject: IMPALA-7792: fix disabling of ORC scans
......................................................................
Patch Set 6: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/11835
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I209e79b18745c48d0182800a916d6566083f4609
Gerrit-Change-Number: 11835
Gerrit-PatchSet: 6
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Wed, 31 Oct 2018 23:31:41 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7792: fix disabling of ORC scans
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11835 )
Change subject: IMPALA-7792: fix disabling of ORC scans
......................................................................
Patch Set 4:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/11835/3/tests/custom_cluster/test_disable_features.py
File tests/custom_cluster/test_disable_features.py:
http://gerrit.cloudera.org:8080/#/c/11835/3/tests/custom_cluster/test_disable_features.py@22
PS3, Line 22:
> flake8: E302 expected 2 blank lines, found 1
Done
--
To view, visit http://gerrit.cloudera.org:8080/11835
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I209e79b18745c48d0182800a916d6566083f4609
Gerrit-Change-Number: 11835
Gerrit-PatchSet: 4
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Wed, 31 Oct 2018 22:18:14 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7792: fix disabling of ORC scans
Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11835 )
Change subject: IMPALA-7792: fix disabling of ORC scans
......................................................................
Patch Set 4:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/11835/4/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
File fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java:
http://gerrit.cloudera.org:8080/#/c/11835/4/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java@414
PS4, Line 414: if (!BackendConfig.INSTANCE.isOrcScannerEnabled()) {
: for (FeFsPartition part: partitions_) {
: if (part.getInputFormatDescriptor().getFileFormat() == HdfsFileFormat.ORC) {
: throw new NotImplementedException(
: "ORC scans are disabled by --enable_orc_scanner flag");
: }
: }
: }
> Do we need this block? Shouldn't the check below inside the loop work?
I should have removed the block below. The below check doesn't work because we bail out early if there's no complex typed column.
http://gerrit.cloudera.org:8080/#/c/11835/4/tests/custom_cluster/test_disable_features.py
File tests/custom_cluster/test_disable_features.py:
http://gerrit.cloudera.org:8080/#/c/11835/4/tests/custom_cluster/test_disable_features.py@32
PS4, Line 32: vector
> nit: Not sure if the intention here was to run with multiple dimensions but
I copied this from a different custom cluster test. Are you saying that it's unnecessary? I'd totally believe we just copied this blindly. It seems to work fine though :)
--
To view, visit http://gerrit.cloudera.org:8080/11835
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I209e79b18745c48d0182800a916d6566083f4609
Gerrit-Change-Number: 11835
Gerrit-PatchSet: 4
Gerrit-Owner: Tim Armstrong <ta...@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bh...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <ph...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Wed, 31 Oct 2018 23:16:05 +0000
Gerrit-HasComments: Yes