You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Anonymous Coward (Code Review)" <ge...@cloudera.org> on 2021/09/28 16:55:49 UTC
[Impala-ASF-CR] IMPALA-10934: Enable table definition over a single file
kirk@rxd.hu has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17878
Change subject: IMPALA-10934: Enable table definition over a single file
......................................................................
IMPALA-10934: Enable table definition over a single file
Implements an abstraction layer to show files in a single directory.
Suppose that the filesystem has a directory in which there are multiple files:
file://somedir/f1.txt
file://somedir/f2.txt
In case of a HMS backed table(s) - the contents of a directory could be considered as table.
This patch enables to use sfs+ wrappers to show a single file as a single file in a directory.
The directory which contains the file could be specified:
sfs+file://somedir/f1.txt/SINGLEFILE
This will be a directory containing only the f1.txt and nothing else.
(cherry picked from commit 998d5a8d72203c9ffead700b2e420049108417c3)
Change-Id: I32be936243aa4c8320f5d06d2b7fbf98822f82e7
---
M be/src/exec/hdfs-table-sink.cc
M be/src/runtime/io/disk-io-mgr-test.cc
M be/src/runtime/io/disk-io-mgr.cc
M be/src/runtime/io/disk-io-mgr.h
M be/src/util/hdfs-util.cc
M be/src/util/hdfs-util.h
M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java
M fe/src/main/java/org/apache/impala/common/FileSystemUtil.java
M java/shaded-deps/hive-exec/pom.xml
9 files changed, 37 insertions(+), 2 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/78/17878/1
--
To view, visit http://gerrit.cloudera.org:8080/17878
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I32be936243aa4c8320f5d06d2b7fbf98822f82e7
Gerrit-Change-Number: 17878
Gerrit-PatchSet: 1
Gerrit-Owner: Anonymous Coward <ki...@rxd.hu>
[Impala-ASF-CR] IMPALA-10934: Enable table definition over a single file
Posted by "Aman Sinha (Code Review)" <ge...@cloudera.org>.
Aman Sinha has posted comments on this change. ( http://gerrit.cloudera.org:8080/17878 )
Change subject: IMPALA-10934: Enable table definition over a single file
......................................................................
Patch Set 2:
(1 comment)
Could you pls add a commit
http://gerrit.cloudera.org:8080/#/c/17878/2//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/17878/2//COMMIT_MSG@22
PS2, Line 22:
Pls add a note about how this patch was tested even if manually.
--
To view, visit http://gerrit.cloudera.org:8080/17878
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I32be936243aa4c8320f5d06d2b7fbf98822f82e7
Gerrit-Change-Number: 17878
Gerrit-PatchSet: 2
Gerrit-Owner: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Wed, 27 Oct 2021 21:38:52 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10934: Enable table definition over a single file
Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/17878 )
Change subject: IMPALA-10934: Enable table definition over a single file
......................................................................
Patch Set 2:
(1 comment)
Is this expected to work on my dev setup? When I run:
create table sfstest like functional.alltypes location "sfs+hdfs://localhost:20500/test-warehouse/alltypes/year=2009/month=9/090901.txt/#SINGLEFILE#";
I get:
ERROR: AnalysisException: No FileSystem for scheme "sfs+hdfs"
CAUSED BY: UnsupportedFileSystemException: No FileSystem for scheme "sfs+hdfs"
Maybe it needs a newer version of something.
A few questions:
1. If a user creates a table based on a single file and then they drop the table, what happens? Does the external vs managed distinction continue to apply? (i.e. it could delete the file)
2. I'm assuming there are very limited operations that we can do for this table. No inserts, no loads, etc. Is that right? What errors do these throw?
3. If the create statement specifies a schema that has partitioning (which would be subdirectories), do we throw an error?
For Impala, I'm wondering if it is possible to keep the SFS code limited to a small piece of the frontend. Basically, detect that a table is SFS (and thus some statements are not allowed) and then convert to the underlying filename and go from there with the rest of the code not needing knowledge of SFS.
http://gerrit.cloudera.org:8080/#/c/17878/2/be/src/runtime/io/disk-io-mgr.cc
File be/src/runtime/io/disk-io-mgr.cc:
http://gerrit.cloudera.org:8080/#/c/17878/2/be/src/runtime/io/disk-io-mgr.cc@142
PS2, Line 142: // The maximum number of SFS I/O threads.
: DEFINE_int32(num_sfs_io_threads, 16, "Number of SFS I/O threads");
SFS maps down to some other storage type, and it should be using the logic for that underlying storage type. It's important for SFS+S3 to map down to S3 and SFS+Ozone to map down to Ozone, because we treat S3 differently from Ozone. For example, the file handle cache is only enabled for storage types that are known to work (and can lead to stability/performance issues if we don't do the check correctly).
If SFS is its own device name with its own set of threads, then it will lose those distinctions and there will be bugs.
If I'm understanding SFS correctly, then the Impala backend might not need to know about SFS at all. If the frontend knows that it is reading a single file table, it can convert the SFS filename to the actual real underlying file before sending it to the backend. The backend code then doesn't need any special changes for the read path.
--
To view, visit http://gerrit.cloudera.org:8080/17878
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I32be936243aa4c8320f5d06d2b7fbf98822f82e7
Gerrit-Change-Number: 17878
Gerrit-PatchSet: 2
Gerrit-Owner: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Wed, 27 Oct 2021 23:32:07 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10934: Enable table definition over a single file
Posted by "Aman Sinha (Code Review)" <ge...@cloudera.org>.
Aman Sinha has posted comments on this change. ( http://gerrit.cloudera.org:8080/17878 )
Change subject: IMPALA-10934: Enable table definition over a single file
......................................................................
Patch Set 2:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/17878/2//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/17878/2//COMMIT_MSG@12
PS2, Line 12: file://somedir/f1.txt
nit: could you use the more commonly used 'hdfs' path ?
http://gerrit.cloudera.org:8080/#/c/17878/2//COMMIT_MSG@17
PS2, Line 17: This patch enables to use sfs+ wrappers to show a single file as a single file in a directory.
nit: the wording here is a bit confusing. How about something like: 'This patch enables a new file system wrapper 'sfs+' (sfs = single file system) which provides a view of a single file in a directory.' The '+' indicates that this wrapper can be added on top of multiple underlying file systems/object storage such as HDFS, S3 etc.
Also, the JIRA IMPALA-10934 talks about the corresponding HIVE jira on which this patch depends. Could you mention that in the commit message ?
http://gerrit.cloudera.org:8080/#/c/17878/2//COMMIT_MSG@22
PS2, Line 22:
Could you add a comment about the Ranger authorization check that this patch also does.
http://gerrit.cloudera.org:8080/#/c/17878/2//COMMIT_MSG@23
PS2, Line 23: (cherry picked from commit 998d5a8d72203c9ffead700b2e420049108417c3)
Not sure which commit it is referring to, but this should be removed.
--
To view, visit http://gerrit.cloudera.org:8080/17878
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I32be936243aa4c8320f5d06d2b7fbf98822f82e7
Gerrit-Change-Number: 17878
Gerrit-PatchSet: 2
Gerrit-Owner: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Wed, 27 Oct 2021 20:19:35 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10934: Enable table definition over a single file
Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/17878
to look at the new patch set (#2).
Change subject: IMPALA-10934: Enable table definition over a single file
......................................................................
IMPALA-10934: Enable table definition over a single file
Implements an abstraction layer to show files in a single directory.
Suppose that the filesystem has a directory in which there are multiple files:
file://somedir/f1.txt
file://somedir/f2.txt
In case of a HMS backed table(s) - the contents of a directory could be considered as table.
This patch enables to use sfs+ wrappers to show a single file as a single file in a directory.
The directory which contains the file could be specified:
sfs+file://somedir/f1.txt/#SINGLEFILE#
This will be a directory containing only the f1.txt and nothing else.
(cherry picked from commit 998d5a8d72203c9ffead700b2e420049108417c3)
Change-Id: I32be936243aa4c8320f5d06d2b7fbf98822f82e7
---
M be/src/exec/hdfs-table-sink.cc
M be/src/runtime/io/disk-io-mgr-test.cc
M be/src/runtime/io/disk-io-mgr.cc
M be/src/runtime/io/disk-io-mgr.h
M be/src/util/hdfs-util.cc
M be/src/util/hdfs-util.h
M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java
M fe/src/main/java/org/apache/impala/common/FileSystemUtil.java
M java/shaded-deps/hive-exec/pom.xml
9 files changed, 38 insertions(+), 2 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/78/17878/2
--
To view, visit http://gerrit.cloudera.org:8080/17878
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I32be936243aa4c8320f5d06d2b7fbf98822f82e7
Gerrit-Change-Number: 17878
Gerrit-PatchSet: 2
Gerrit-Owner: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
[Impala-ASF-CR] IMPALA-10934: Enable table definition over a single file
Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
kirk@rxd.hu has posted comments on this change. ( http://gerrit.cloudera.org:8080/17878 )
Change subject: IMPALA-10934: Enable table definition over a single file
......................................................................
Patch Set 2:
(6 comments)
http://gerrit.cloudera.org:8080/#/c/17878/2//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/17878/2//COMMIT_MSG@12
PS2, Line 12: file://somedir/f1.txt
> nit: could you use the more commonly used 'hdfs' path ?
Done
http://gerrit.cloudera.org:8080/#/c/17878/2//COMMIT_MSG@17
PS2, Line 17: This patch enables to use sfs+ wrappers to show a single file as a single file in a directory.
> nit: the wording here is a bit confusing. How about something like: 'This p
Done
http://gerrit.cloudera.org:8080/#/c/17878/2//COMMIT_MSG@22
PS2, Line 22:
> Could you add a comment about the Ranger authorization check that this patc
removed that part - it was left there by mistake.
http://gerrit.cloudera.org:8080/#/c/17878/2//COMMIT_MSG@22
PS2, Line 22:
> Pls add a note about how this patch was tested even if manually.
Done
http://gerrit.cloudera.org:8080/#/c/17878/2//COMMIT_MSG@23
PS2, Line 23: (cherry picked from commit 998d5a8d72203c9ffead700b2e420049108417c3)
> Not sure which commit it is referring to, but this should be removed.
Done
http://gerrit.cloudera.org:8080/#/c/17878/2/be/src/runtime/io/disk-io-mgr.cc
File be/src/runtime/io/disk-io-mgr.cc:
http://gerrit.cloudera.org:8080/#/c/17878/2/be/src/runtime/io/disk-io-mgr.cc@142
PS2, Line 142: // The maximum number of SFS I/O threads.
: DEFINE_int32(num_sfs_io_threads, 16, "Number of SFS I/O threads");
> My understanding is that the file handle cache will be disabled for SFS unl
Although it might be possible to do that - I think it might not worth the effort to cover these cases: these are most likely small files so the performance impact might not be serious.
I also think that if a customer will run into perfomance issues by using sfs - if the issue is caused by this then he should consider migrating to a real table directory. The main usecase of this stuff is to help people testdrive and validate migration earlier.
--
To view, visit http://gerrit.cloudera.org:8080/17878
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I32be936243aa4c8320f5d06d2b7fbf98822f82e7
Gerrit-Change-Number: 17878
Gerrit-PatchSet: 2
Gerrit-Owner: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Wed, 03 Nov 2021 18:05:21 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10934: Enable table definition over a single file
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17878 )
Change subject: IMPALA-10934: Enable table definition over a single file
......................................................................
Patch Set 3:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/9717/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/17878
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I32be936243aa4c8320f5d06d2b7fbf98822f82e7
Gerrit-Change-Number: 17878
Gerrit-PatchSet: 3
Gerrit-Owner: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Wed, 03 Nov 2021 18:30:59 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10934 (Part 1): Enable table definition over a single file
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17878 )
Change subject: IMPALA-10934 (Part 1): Enable table definition over a single file
......................................................................
Patch Set 10:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/9966/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/17878
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I32be936243aa4c8320f5d06d2b7fbf98822f82e7
Gerrit-Change-Number: 17878
Gerrit-PatchSet: 10
Gerrit-Owner: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Wed, 05 Jan 2022 03:51:59 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10934 (Part 1): Enable table definition over a single file
Posted by "Fang-Yu Rao (Code Review)" <ge...@cloudera.org>.
Fang-Yu Rao has uploaded a new patch set (#5) to the change originally created by kirk@rxd.hu. ( http://gerrit.cloudera.org:8080/17878 )
Change subject: IMPALA-10934 (Part 1): Enable table definition over a single file
......................................................................
IMPALA-10934 (Part 1): Enable table definition over a single file
Implements an abstraction layer to show files in a single directory.
Impala side part - filesystem drivers are in HIVE-25569.
Suppose that the filesystem has a directory in which there are multiple
files:
hdfs://somedir/f1.txt
hdfs://somedir/f2.txt
In case of a HMS backed table(s) - the contents of a directory could be
considered as table.
This patch enables a new file system wrapper 'sfs+' (sfs = single file
system) which provides a view of a single file in a directory.' The '+'
indicates that this wrapper can be added on top of multiple underlying
file systems/object storage such as HDFS, S3 etc. The directory which
contains the file could be specified:
sfs+hdfs://somedir/f1.txt/#SINGLEFILE#
This will be a directory containing only the f1.txt and nothing else.
This patch was tested locally - with a custom build of Hive version
which also had HIVE-25569.
Change-Id: I32be936243aa4c8320f5d06d2b7fbf98822f82e7
---
M be/src/exec/hdfs-table-sink.cc
M be/src/runtime/io/disk-io-mgr-test.cc
M be/src/runtime/io/disk-io-mgr.cc
M be/src/runtime/io/disk-io-mgr.h
M be/src/util/hdfs-util.cc
M be/src/util/hdfs-util.h
M fe/src/main/java/org/apache/impala/common/FileSystemUtil.java
M java/shaded-deps/hive-exec/pom.xml
8 files changed, 32 insertions(+), 2 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/78/17878/5
--
To view, visit http://gerrit.cloudera.org:8080/17878
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I32be936243aa4c8320f5d06d2b7fbf98822f82e7
Gerrit-Change-Number: 17878
Gerrit-PatchSet: 5
Gerrit-Owner: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
[Impala-ASF-CR] IMPALA-10934: Enable table definition over a single file
Posted by "Aman Sinha (Code Review)" <ge...@cloudera.org>.
Aman Sinha has posted comments on this change. ( http://gerrit.cloudera.org:8080/17878 )
Change subject: IMPALA-10934: Enable table definition over a single file
......................................................................
Patch Set 2:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/17878/2/be/src/runtime/io/disk-io-mgr.cc
File be/src/runtime/io/disk-io-mgr.cc:
http://gerrit.cloudera.org:8080/#/c/17878/2/be/src/runtime/io/disk-io-mgr.cc@142
PS2, Line 142: // The maximum number of SFS I/O threads.
: DEFINE_int32(num_sfs_io_threads, 16, "Number of SFS I/O threads");
> As a second thought on this, if we are ok with turning off file handle cach
Agree that turning off file handle caching for the SFS case should not hurt much in terms of performance if indeed there is any problem with the file handle caching for SFS+S3 (will await testing).
And yes, ensuring that the data cache works as expected would be useful/important. Query profile metrics for the data cache can be used to manually verify.
On the previous point of potentially doing the single file processing through the frontend, based on offline discussion with Zoltan, this was considered but the backend alternative was more compelling because it allows - with relatively small changes - the same file system access mechanism to work for both Impala and Hive. This is kind of implied in the design doc attached to the corresponding Hive JIRA here: https://issues.apache.org/jira/browse/HIVE-25569.
Zoltan, it would be good if the drawback of the frontend-only approach is more explicitly described in the document.
Also, by representing the single file paths in a standard URI form, Ranger authorization gets transparently supported with minimal changes.
--
To view, visit http://gerrit.cloudera.org:8080/17878
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I32be936243aa4c8320f5d06d2b7fbf98822f82e7
Gerrit-Change-Number: 17878
Gerrit-PatchSet: 2
Gerrit-Owner: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Sun, 31 Oct 2021 23:44:45 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10934: Enable table definition over a single file
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17878 )
Change subject: IMPALA-10934: Enable table definition over a single file
......................................................................
Patch Set 4: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/17878
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I32be936243aa4c8320f5d06d2b7fbf98822f82e7
Gerrit-Change-Number: 17878
Gerrit-PatchSet: 4
Gerrit-Owner: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Mon, 08 Nov 2021 08:54:03 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10934: Enable table definition over a single file
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17878 )
Change subject: IMPALA-10934: Enable table definition over a single file
......................................................................
Patch Set 1:
Build Failed
https://jenkins.impala.io/job/gerrit-code-review-checks/9520/ : Initial code review checks failed. See linked job for details on the failure.
--
To view, visit http://gerrit.cloudera.org:8080/17878
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I32be936243aa4c8320f5d06d2b7fbf98822f82e7
Gerrit-Change-Number: 17878
Gerrit-PatchSet: 1
Gerrit-Owner: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Tue, 28 Sep 2021 17:07:16 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10934 (Part 1): Enable table definition over a single file
Posted by "Fang-Yu Rao (Code Review)" <ge...@cloudera.org>.
Fang-Yu Rao has posted comments on this change. ( http://gerrit.cloudera.org:8080/17878 )
Change subject: IMPALA-10934 (Part 1): Enable table definition over a single file
......................................................................
Patch Set 5:
I revised the commit message so that each line contains at most 72 characters.
--
To view, visit http://gerrit.cloudera.org:8080/17878
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I32be936243aa4c8320f5d06d2b7fbf98822f82e7
Gerrit-Change-Number: 17878
Gerrit-PatchSet: 5
Gerrit-Owner: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Wed, 17 Nov 2021 21:09:55 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10934 (Part 1): Enable table definition over a single file
Posted by "Aman Sinha (Code Review)" <ge...@cloudera.org>.
Aman Sinha has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17878 )
Change subject: IMPALA-10934 (Part 1): Enable table definition over a single file
......................................................................
IMPALA-10934 (Part 1): Enable table definition over a single file
Implements an abstraction layer to show files in a single directory.
Impala side part - filesystem drivers are in HIVE-25569.
Suppose that the filesystem has a directory in which there are multiple
files:
hdfs://somedir/f1.txt
hdfs://somedir/f2.txt
In case of a HMS backed table(s) - the contents of a directory could be
considered as table.
This patch enables a new file system wrapper 'sfs+' (sfs = single file
system) which provides a view of a single file in a directory.' The '+'
indicates that this wrapper can be added on top of multiple underlying
file systems/object storage such as HDFS, S3 etc. The directory which
contains the file could be specified:
sfs+hdfs://somedir/f1.txt/#SINGLEFILE#
This will be a directory containing only the f1.txt and nothing else.
This patch was tested locally - with a custom build of Hive version
which also had HIVE-25569.
Change-Id: I32be936243aa4c8320f5d06d2b7fbf98822f82e7
Reviewed-on: http://gerrit.cloudera.org:8080/17878
Reviewed-by: Aman Sinha <am...@cloudera.com>
Tested-by: Aman Sinha <am...@cloudera.com>
---
M be/src/exec/hdfs-table-sink.cc
M be/src/runtime/io/disk-io-mgr-test.cc
M be/src/runtime/io/disk-io-mgr.cc
M be/src/runtime/io/disk-io-mgr.h
M be/src/util/hdfs-util.cc
M be/src/util/hdfs-util.h
M fe/src/main/java/org/apache/impala/common/FileSystemUtil.java
M java/shaded-deps/hive-exec/pom.xml
8 files changed, 32 insertions(+), 2 deletions(-)
Approvals:
Aman Sinha: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/17878
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I32be936243aa4c8320f5d06d2b7fbf98822f82e7
Gerrit-Change-Number: 17878
Gerrit-PatchSet: 11
Gerrit-Owner: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
[Impala-ASF-CR] IMPALA-10934 (Part 1): Enable table definition over a single file
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17878 )
Change subject: IMPALA-10934 (Part 1): Enable table definition over a single file
......................................................................
Patch Set 8:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/9960/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/17878
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I32be936243aa4c8320f5d06d2b7fbf98822f82e7
Gerrit-Change-Number: 17878
Gerrit-PatchSet: 8
Gerrit-Owner: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Mon, 03 Jan 2022 19:34:48 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10934 (Part 1): Enable table definition over a single file
Posted by "Aman Sinha (Code Review)" <ge...@cloudera.org>.
Aman Sinha has uploaded a new patch set (#8) to the change originally created by kirk@rxd.hu. ( http://gerrit.cloudera.org:8080/17878 )
Change subject: IMPALA-10934 (Part 1): Enable table definition over a single file
......................................................................
IMPALA-10934 (Part 1): Enable table definition over a single file
Implements an abstraction layer to show files in a single directory.
Impala side part - filesystem drivers are in HIVE-25569.
Suppose that the filesystem has a directory in which there are multiple
files:
hdfs://somedir/f1.txt
hdfs://somedir/f2.txt
In case of a HMS backed table(s) - the contents of a directory could be
considered as table.
This patch enables a new file system wrapper 'sfs+' (sfs = single file
system) which provides a view of a single file in a directory.' The '+'
indicates that this wrapper can be added on top of multiple underlying
file systems/object storage such as HDFS, S3 etc. The directory which
contains the file could be specified:
sfs+hdfs://somedir/f1.txt/#SINGLEFILE#
This will be a directory containing only the f1.txt and nothing else.
This patch was tested locally - with a custom build of Hive version
which also had HIVE-25569.
Change-Id: I32be936243aa4c8320f5d06d2b7fbf98822f82e7
---
M be/src/exec/hdfs-table-sink.cc
M be/src/runtime/io/disk-io-mgr-test.cc
M be/src/runtime/io/disk-io-mgr.cc
M be/src/runtime/io/disk-io-mgr.h
M be/src/util/hdfs-util.cc
M be/src/util/hdfs-util.h
M fe/src/main/java/org/apache/impala/common/FileSystemUtil.java
M java/shaded-deps/hive-exec/pom.xml
8 files changed, 32 insertions(+), 2 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/78/17878/8
--
To view, visit http://gerrit.cloudera.org:8080/17878
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I32be936243aa4c8320f5d06d2b7fbf98822f82e7
Gerrit-Change-Number: 17878
Gerrit-PatchSet: 8
Gerrit-Owner: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
[Impala-ASF-CR] IMPALA-10934: Enable table definition over a single file
Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/17878 )
Change subject: IMPALA-10934: Enable table definition over a single file
......................................................................
Patch Set 2:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/17878/2/be/src/runtime/io/disk-io-mgr.cc
File be/src/runtime/io/disk-io-mgr.cc:
http://gerrit.cloudera.org:8080/#/c/17878/2/be/src/runtime/io/disk-io-mgr.cc@142
PS2, Line 142: // The maximum number of SFS I/O threads.
: DEFINE_int32(num_sfs_io_threads, 16, "Number of SFS I/O threads");
> SFS maps down to some other storage type, and it should be using the logic
As a second thought on this, if we are ok with turning off file handle caching, then this approach could work. We are losing some understanding of the underlying storage and some performance, but this is only one file. We should verify the data cache works, and that would generally handle most use cases.
--
To view, visit http://gerrit.cloudera.org:8080/17878
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I32be936243aa4c8320f5d06d2b7fbf98822f82e7
Gerrit-Change-Number: 17878
Gerrit-PatchSet: 2
Gerrit-Owner: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Fri, 29 Oct 2021 06:05:35 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10934: Enable table definition over a single file
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17878 )
Change subject: IMPALA-10934: Enable table definition over a single file
......................................................................
Patch Set 4:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/7599/ DRY_RUN=true
--
To view, visit http://gerrit.cloudera.org:8080/17878
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I32be936243aa4c8320f5d06d2b7fbf98822f82e7
Gerrit-Change-Number: 17878
Gerrit-PatchSet: 4
Gerrit-Owner: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Mon, 08 Nov 2021 02:39:47 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10934: Enable table definition over a single file
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17878 )
Change subject: IMPALA-10934: Enable table definition over a single file
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/17878/1/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java
File fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java:
http://gerrit.cloudera.org:8080/#/c/17878/1/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@157
PS1, Line 157: .uri(authorizable.getName().replaceAll("^sfs.", "").replaceAll("/SINGLEFILE.*", ""))
line too long (98 > 90)
--
To view, visit http://gerrit.cloudera.org:8080/17878
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I32be936243aa4c8320f5d06d2b7fbf98822f82e7
Gerrit-Change-Number: 17878
Gerrit-PatchSet: 1
Gerrit-Owner: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Tue, 28 Sep 2021 16:56:36 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10934: Enable table definition over a single file
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17878 )
Change subject: IMPALA-10934: Enable table definition over a single file
......................................................................
Patch Set 2:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/9524/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/17878
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I32be936243aa4c8320f5d06d2b7fbf98822f82e7
Gerrit-Change-Number: 17878
Gerrit-PatchSet: 2
Gerrit-Owner: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Wed, 29 Sep 2021 10:35:54 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10934 (Part 1): Enable table definition over a single file
Posted by "Aman Sinha (Code Review)" <ge...@cloudera.org>.
Aman Sinha has uploaded a new patch set (#10) to the change originally created by kirk@rxd.hu. ( http://gerrit.cloudera.org:8080/17878 )
Change subject: IMPALA-10934 (Part 1): Enable table definition over a single file
......................................................................
IMPALA-10934 (Part 1): Enable table definition over a single file
Implements an abstraction layer to show files in a single directory.
Impala side part - filesystem drivers are in HIVE-25569.
Suppose that the filesystem has a directory in which there are multiple
files:
hdfs://somedir/f1.txt
hdfs://somedir/f2.txt
In case of a HMS backed table(s) - the contents of a directory could be
considered as table.
This patch enables a new file system wrapper 'sfs+' (sfs = single file
system) which provides a view of a single file in a directory.' The '+'
indicates that this wrapper can be added on top of multiple underlying
file systems/object storage such as HDFS, S3 etc. The directory which
contains the file could be specified:
sfs+hdfs://somedir/f1.txt/#SINGLEFILE#
This will be a directory containing only the f1.txt and nothing else.
This patch was tested locally - with a custom build of Hive version
which also had HIVE-25569.
Change-Id: I32be936243aa4c8320f5d06d2b7fbf98822f82e7
---
M be/src/exec/hdfs-table-sink.cc
M be/src/runtime/io/disk-io-mgr-test.cc
M be/src/runtime/io/disk-io-mgr.cc
M be/src/runtime/io/disk-io-mgr.h
M be/src/util/hdfs-util.cc
M be/src/util/hdfs-util.h
M fe/src/main/java/org/apache/impala/common/FileSystemUtil.java
M java/shaded-deps/hive-exec/pom.xml
8 files changed, 32 insertions(+), 2 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/78/17878/10
--
To view, visit http://gerrit.cloudera.org:8080/17878
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I32be936243aa4c8320f5d06d2b7fbf98822f82e7
Gerrit-Change-Number: 17878
Gerrit-PatchSet: 10
Gerrit-Owner: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
[Impala-ASF-CR] IMPALA-10934 (Part 1): Enable table definition over a single file
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/17878 )
Change subject: IMPALA-10934 (Part 1): Enable table definition over a single file
......................................................................
Patch Set 5:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/9796/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/17878
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I32be936243aa4c8320f5d06d2b7fbf98822f82e7
Gerrit-Change-Number: 17878
Gerrit-PatchSet: 5
Gerrit-Owner: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Wed, 17 Nov 2021 21:30:36 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10934: Enable table definition over a single file
Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/17878 )
Change subject: IMPALA-10934: Enable table definition over a single file
......................................................................
Patch Set 2:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/17878/2/be/src/runtime/io/disk-io-mgr.cc
File be/src/runtime/io/disk-io-mgr.cc:
http://gerrit.cloudera.org:8080/#/c/17878/2/be/src/runtime/io/disk-io-mgr.cc@142
PS2, Line 142: // The maximum number of SFS I/O threads.
: DEFINE_int32(num_sfs_io_threads, 16, "Number of SFS I/O threads");
> Agree that turning off file handle caching for the SFS case should not hurt
My understanding is that the file handle cache will be disabled for SFS unless we explicitly try to enable it. That's probably ok. The path to enabling the file handle cache would be to understand the distinction between SFS+S3 vs SFS+HDFS vs whatnot and map them to the right thread pools. That probably isn't that hard if we want to go that way, and it could be done in the backend.
--
To view, visit http://gerrit.cloudera.org:8080/17878
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I32be936243aa4c8320f5d06d2b7fbf98822f82e7
Gerrit-Change-Number: 17878
Gerrit-PatchSet: 2
Gerrit-Owner: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Tue, 02 Nov 2021 03:31:20 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10934: Enable table definition over a single file
Posted by "Anonymous Coward (Code Review)" <ge...@cloudera.org>.
Hello Aman Sinha, Joe McDonnell, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/17878
to look at the new patch set (#3).
Change subject: IMPALA-10934: Enable table definition over a single file
......................................................................
IMPALA-10934: Enable table definition over a single file
Implements an abstraction layer to show files in a single directory.
Impala side part - filesystem drivers are in HIVE-25569.
Suppose that the filesystem has a directory in which there are multiple files:
hdfs://somedir/f1.txt
hdfs://somedir/f2.txt
In case of a HMS backed table(s) - the contents of a directory could be considered as table.
This patch enables a new file system wrapper 'sfs+' (sfs = single file system) which provides a view of a single file in a directory.' The '+' indicates that this wrapper can be added on top of multiple underlying file systems/object storage such as HDFS, S3 etc.
The directory which contains the file could be specified:
sfs+hdfs://somedir/f1.txt/#SINGLEFILE#
This will be a directory containing only the f1.txt and nothing else.
This patch was tested locally - with a custom build of Hive version which also had HIVE-25569.
Change-Id: I32be936243aa4c8320f5d06d2b7fbf98822f82e7
---
M be/src/exec/hdfs-table-sink.cc
M be/src/runtime/io/disk-io-mgr-test.cc
M be/src/runtime/io/disk-io-mgr.cc
M be/src/runtime/io/disk-io-mgr.h
M be/src/util/hdfs-util.cc
M be/src/util/hdfs-util.h
M fe/src/main/java/org/apache/impala/common/FileSystemUtil.java
M java/shaded-deps/hive-exec/pom.xml
8 files changed, 32 insertions(+), 2 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/78/17878/3
--
To view, visit http://gerrit.cloudera.org:8080/17878
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I32be936243aa4c8320f5d06d2b7fbf98822f82e7
Gerrit-Change-Number: 17878
Gerrit-PatchSet: 3
Gerrit-Owner: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
[Impala-ASF-CR] IMPALA-10934 (Part 1): Enable table definition over a single file
Posted by "Aman Sinha (Code Review)" <ge...@cloudera.org>.
Aman Sinha has posted comments on this change. ( http://gerrit.cloudera.org:8080/17878 )
Change subject: IMPALA-10934 (Part 1): Enable table definition over a single file
......................................................................
Patch Set 8: Code-Review+2
Published this as non-draft after rebasing to latest master. Carry forward Joe's implicit +1 and bumping to +2. Will run the full Jenkins verification on part 1 and part 2 together.
--
To view, visit http://gerrit.cloudera.org:8080/17878
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I32be936243aa4c8320f5d06d2b7fbf98822f82e7
Gerrit-Change-Number: 17878
Gerrit-PatchSet: 8
Gerrit-Owner: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <ki...@rxd.hu>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Comment-Date: Mon, 03 Jan 2022 19:17:48 +0000
Gerrit-HasComments: No