You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Ethan Xue (Code Review)" <ge...@cloudera.org> on 2019/07/12 23:46:14 UTC
[Impala-ASF-CR] IMPALA-8549: Add support for scanning DEFLATE text files
Ethan Xue has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13856
Change subject: IMPALA-8549: Add support for scanning DEFLATE text files
......................................................................
IMPALA-8549: Add support for scanning DEFLATE text files
Hadoop tools such as Hive and MapReduce support
reading and writing text files compressed using
the deflate algorithm. In Hadoop, the zlib library
is used to compress text files into .DEFLATE files,
which are not in the raw deflate format but rather
the zlib format.
This patch adds support to Impala for scanning
.DEFLATE files. To avoid confusion, it is important
to note that although these files have a compression
type of DEFLATE in Impala, they should be treated as
if their compression type is ZLIB.
Testing:
There is a pre-existing unit test that validates
compressing/decompressing data with compression type
DEFLATE. Also, modified existing end-to-end testing
that simulates querying files of various formats and
compression types.
Change-Id: I45e41ab5a12637d396fef0812a09d71fa839b27a
Changes
Change-Id: I59944d4359dde3a0a03e30a993b701cc7426397f
chagnes
Change-Id: I5d2cc8f889277ba62fa8455728e3809a6ee3c3f6
---
M be/src/exec/hdfs-text-scanner.cc
M be/src/exec/hdfs-text-scanner.h
M be/src/util/codec.cc
M testdata/workloads/functional-query/functional-query_exhaustive.csv
M tests/query_test/test_compressed_formats.py
5 files changed, 14 insertions(+), 20 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/56/13856/1
--
To view, visit http://gerrit.cloudera.org:8080/13856
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I5d2cc8f889277ba62fa8455728e3809a6ee3c3f6
Gerrit-Change-Number: 13856
Gerrit-PatchSet: 1
Gerrit-Owner: Ethan Xue <et...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Ethan Xue <et...@cloudera.com>
Gerrit-Reviewer: Sahil Takiar <st...@cloudera.com>
[Impala-ASF-CR] IMPALA-8549: Add support for scanning DEFLATE text files
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13856 )
Change subject: IMPALA-8549: Add support for scanning DEFLATE text files
......................................................................
Patch Set 1:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/3873/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/13856
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5d2cc8f889277ba62fa8455728e3809a6ee3c3f6
Gerrit-Change-Number: 13856
Gerrit-PatchSet: 1
Gerrit-Owner: Ethan Xue <et...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Ethan Xue <et...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Sahil Takiar <st...@cloudera.com>
Gerrit-Comment-Date: Sat, 13 Jul 2019 00:26:27 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8549: Add support for scanning DEFLATE text files
Posted by "Ethan Xue (Code Review)" <ge...@cloudera.org>.
Ethan Xue has abandoned this change. ( http://gerrit.cloudera.org:8080/13856 )
Change subject: IMPALA-8549: Add support for scanning DEFLATE text files
......................................................................
Abandoned
Forgot to delete extra change ids
--
To view, visit http://gerrit.cloudera.org:8080/13856
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: abandon
Gerrit-Change-Id: I5d2cc8f889277ba62fa8455728e3809a6ee3c3f6
Gerrit-Change-Number: 13856
Gerrit-PatchSet: 1
Gerrit-Owner: Ethan Xue <et...@cloudera.com>
Gerrit-Reviewer: Abhishek Rawat <ar...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Ethan Xue <et...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Sahil Takiar <st...@cloudera.com>