[GitHub] [drill] vvysotskyi commented on a change in pull request #2184: DRILL-7874: Drill Fails to Read File Types on S3

[GitBox <gi...@apache.org>]

vvysotskyi commented on a change in pull request #2184:
URL: https://github.com/apache/drill/pull/2184#discussion_r585890334



##########
File path: contrib/format-esri/src/main/java/org/apache/drill/exec/store/esri/ShpBatchReader.java
##########
@@ -125,7 +125,7 @@ public boolean next() {
 
   private void openFile(FileSchemaNegotiator negotiator) {
     try {
-      fileReaderShp = negotiator.fileSystem().open(split.getPath());
+      fileReaderShp = negotiator.fileSystem().openDecompressedInputStream(split.getPath());

Review comment:
       I have some doubts regarding the safety of the `openDecompressedInputStream()` method. It contains the following line:
   ```
   byte[] bytes = IOUtils.toByteArray(compressedStream);
   ```
   that will read all the stream to the memory, but it may cause OOM.
   Why we can't use here another `openPossiblyCompressedStream()` method?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org