You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Szehon Ho <sz...@cloudera.com> on 2015/09/22 03:03:55 UTC
Review Request 38585: HIVE-11912 : Make snappy compression default for
parquet tables
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38585/
-----------------------------------------------------------
Review request for hive.
Bugs: HIVE-11912
https://issues.apache.org/jira/browse/HIVE-11912
Repository: hive-git
Description
-------
Snappy is a popular compression codec for Parquet, and is the default in many Parquet applications, increasing the performance.
This change would make it the default for new Hive Parquet tables.
Unfortunately, there's no Serde extension today to specify default table properties, like StorageHandler.
The most logical place seemed to be StorageFormat abstraction, which is the the rough equivalent of StorageHandler. By putting it there instead of in AbstractSerde, we don't have to waste time initializing the Serde.
Diffs
-----
ql/src/java/org/apache/hadoop/hive/ql/io/AbstractStorageFormatDescriptor.java 854a7a3
ql/src/java/org/apache/hadoop/hive/ql/io/ParquetFileStorageFormatDescriptor.java 166775f
ql/src/java/org/apache/hadoop/hive/ql/io/StorageFormatDescriptor.java 604abf8
ql/src/java/org/apache/hadoop/hive/ql/parse/StorageFormat.java 48aca4d
ql/src/test/results/clientpositive/parquet_array_null_element.q.out 387f01e
ql/src/test/results/clientpositive/parquet_create.q.out c6d33ff
ql/src/test/results/clientpositive/parquet_partitioned.q.out 3529d70
Diff: https://reviews.apache.org/r/38585/diff/
Testing
-------
Existing unit tests should cover this.
Thanks,
Szehon Ho
Re: Review Request 38585: HIVE-11912 : Make snappy compression
default for parquet tables
Posted by Sergio Pena <se...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38585/#review99988
-----------------------------------------------------------
Ship it!
Ship It!
- Sergio Pena
On Sept. 22, 2015, 1:03 a.m., Szehon Ho wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38585/
> -----------------------------------------------------------
>
> (Updated Sept. 22, 2015, 1:03 a.m.)
>
>
> Review request for hive.
>
>
> Bugs: HIVE-11912
> https://issues.apache.org/jira/browse/HIVE-11912
>
>
> Repository: hive-git
>
>
> Description
> -------
>
> Snappy is a popular compression codec for Parquet, and is the default in many Parquet applications, increasing the performance.
> This change would make it the default for new Hive Parquet tables.
>
> Unfortunately, there's no Serde extension today to specify default table properties, like StorageHandler.
> The most logical place seemed to be StorageFormat abstraction, which is the the rough equivalent of StorageHandler. By putting it there instead of in AbstractSerde, we don't have to waste time initializing the Serde.
>
>
> Diffs
> -----
>
> ql/src/java/org/apache/hadoop/hive/ql/io/AbstractStorageFormatDescriptor.java 854a7a3
> ql/src/java/org/apache/hadoop/hive/ql/io/ParquetFileStorageFormatDescriptor.java 166775f
> ql/src/java/org/apache/hadoop/hive/ql/io/StorageFormatDescriptor.java 604abf8
> ql/src/java/org/apache/hadoop/hive/ql/parse/StorageFormat.java 48aca4d
> ql/src/test/results/clientpositive/parquet_array_null_element.q.out 387f01e
> ql/src/test/results/clientpositive/parquet_create.q.out c6d33ff
> ql/src/test/results/clientpositive/parquet_partitioned.q.out 3529d70
>
> Diff: https://reviews.apache.org/r/38585/diff/
>
>
> Testing
> -------
>
> Existing unit tests should cover this.
>
>
> Thanks,
>
> Szehon Ho
>
>