You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by Vlad-Storona <gi...@git.apache.org> on 2017/10/12 10:34:44 UTC
[GitHub] drill pull request #989: DRILL-5790: PCAP format explicitly opens local file
GitHub user Vlad-Storona opened a pull request:
https://github.com/apache/drill/pull/989
DRILL-5790: PCAP format explicitly opens local file
See DRILL-5790 for details.
In the current implementation in master used the local FS and it is working on MapR-FS but does not work on HDFS.
Now, for reading files used `org.apache.hadoop.fs.FileSystem`.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mapr-demos/drill DRILL-5790
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/drill/pull/989.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #989
----
commit f968708c0a428ab91fdff8f5e52303b500089b88
Author: Vlad Storona <vs...@cybervisiontech.com>
Date: 2017-10-12T10:16:19Z
Fixed problem with explicit opening a local file
----
---
[GitHub] drill pull request #989: DRILL-5790: PCAP format explicitly opens local file
Posted by arina-ielchiieva <gi...@git.apache.org>.
Github user arina-ielchiieva commented on a diff in the pull request:
https://github.com/apache/drill/pull/989#discussion_r144269017
--- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/pcap/PcapRecordReader.java ---
@@ -125,8 +129,8 @@ public int next() {
@Override
public void close() throws Exception {
-// buffer = null;
-// in.close();
+ in.close();
+ fs.close();
--- End diff --
Since you did not open fs, I guess it's not your responsibility to close it.
---
[GitHub] drill pull request #989: DRILL-5790: PCAP format explicitly opens local file
Posted by Vlad-Storona <gi...@git.apache.org>.
Github user Vlad-Storona commented on a diff in the pull request:
https://github.com/apache/drill/pull/989#discussion_r144286706
--- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/pcap/PcapRecordReader.java ---
@@ -100,14 +104,14 @@ public void setup(final OperatorContext context, final OutputMutator output) thr
this.output = output;
this.buffer = new byte[100000];
- this.in = new FileInputStream(inputPath);
+ this.in = fs.open(pathToFile);
this.decoder = new PacketDecoder(in);
this.validBytes = in.read(buffer);
this.projectedCols = getProjectedColsIfItNull();
setColumns(projectedColumns);
} catch (IOException io) {
throw UserException.dataReadError(io)
- .addContext("File name:", inputPath)
+ .addContext("File name:", pathToFile.toString())
--- End diff --
Thanks, I will replace it.
---
[GitHub] drill issue #989: DRILL-5790: PCAP format explicitly opens local file
Posted by arina-ielchiieva <gi...@git.apache.org>.
Github user arina-ielchiieva commented on the issue:
https://github.com/apache/drill/pull/989
+1, LGTM.
---
[GitHub] drill pull request #989: DRILL-5790: PCAP format explicitly opens local file
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/drill/pull/989
---
[GitHub] drill pull request #989: DRILL-5790: PCAP format explicitly opens local file
Posted by Vlad-Storona <gi...@git.apache.org>.
Github user Vlad-Storona commented on a diff in the pull request:
https://github.com/apache/drill/pull/989#discussion_r144286736
--- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/pcap/PcapRecordReader.java ---
@@ -125,8 +129,8 @@ public int next() {
@Override
public void close() throws Exception {
-// buffer = null;
-// in.close();
+ in.close();
+ fs.close();
--- End diff --
Sure, my mistake, I will remove it.
---
[GitHub] drill pull request #989: DRILL-5790: PCAP format explicitly opens local file
Posted by arina-ielchiieva <gi...@git.apache.org>.
Github user arina-ielchiieva commented on a diff in the pull request:
https://github.com/apache/drill/pull/989#discussion_r144269528
--- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/store/pcap/PcapRecordReader.java ---
@@ -100,14 +104,14 @@ public void setup(final OperatorContext context, final OutputMutator output) thr
this.output = output;
this.buffer = new byte[100000];
- this.in = new FileInputStream(inputPath);
+ this.in = fs.open(pathToFile);
this.decoder = new PacketDecoder(in);
this.validBytes = in.read(buffer);
this.projectedCols = getProjectedColsIfItNull();
setColumns(projectedColumns);
} catch (IOException io) {
throw UserException.dataReadError(io)
- .addContext("File name:", inputPath)
+ .addContext("File name:", pathToFile.toString())
--- End diff --
It's better you use `pathToFile.toUri().getPath()`.
---