You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Szehon Ho <sz...@cloudera.com> on 2015/09/22 03:03:55 UTC

Review Request 38585: HIVE-11912 : Make snappy compression default for parquet tables

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38585/
-----------------------------------------------------------

Review request for hive.


Bugs: HIVE-11912
    https://issues.apache.org/jira/browse/HIVE-11912


Repository: hive-git


Description
-------

Snappy is a popular compression codec for Parquet, and is the default in many Parquet applications, increasing the performance.
This change would make it the default for new Hive Parquet tables.

Unfortunately, there's no Serde extension today to specify default table properties, like StorageHandler.
The most logical place seemed to be StorageFormat abstraction, which is the the rough equivalent of StorageHandler. By putting it there instead of in AbstractSerde, we don't have to waste time initializing the Serde.


Diffs
-----

  ql/src/java/org/apache/hadoop/hive/ql/io/AbstractStorageFormatDescriptor.java 854a7a3 
  ql/src/java/org/apache/hadoop/hive/ql/io/ParquetFileStorageFormatDescriptor.java 166775f 
  ql/src/java/org/apache/hadoop/hive/ql/io/StorageFormatDescriptor.java 604abf8 
  ql/src/java/org/apache/hadoop/hive/ql/parse/StorageFormat.java 48aca4d 
  ql/src/test/results/clientpositive/parquet_array_null_element.q.out 387f01e 
  ql/src/test/results/clientpositive/parquet_create.q.out c6d33ff 
  ql/src/test/results/clientpositive/parquet_partitioned.q.out 3529d70 

Diff: https://reviews.apache.org/r/38585/diff/


Testing
-------

Existing unit tests should cover this.


Thanks,

Szehon Ho


Re: Review Request 38585: HIVE-11912 : Make snappy compression default for parquet tables

Posted by Sergio Pena <se...@cloudera.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38585/#review99988
-----------------------------------------------------------

Ship it!


Ship It!

- Sergio Pena


On Sept. 22, 2015, 1:03 a.m., Szehon Ho wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38585/
> -----------------------------------------------------------
> 
> (Updated Sept. 22, 2015, 1:03 a.m.)
> 
> 
> Review request for hive.
> 
> 
> Bugs: HIVE-11912
>     https://issues.apache.org/jira/browse/HIVE-11912
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> Snappy is a popular compression codec for Parquet, and is the default in many Parquet applications, increasing the performance.
> This change would make it the default for new Hive Parquet tables.
> 
> Unfortunately, there's no Serde extension today to specify default table properties, like StorageHandler.
> The most logical place seemed to be StorageFormat abstraction, which is the the rough equivalent of StorageHandler. By putting it there instead of in AbstractSerde, we don't have to waste time initializing the Serde.
> 
> 
> Diffs
> -----
> 
>   ql/src/java/org/apache/hadoop/hive/ql/io/AbstractStorageFormatDescriptor.java 854a7a3 
>   ql/src/java/org/apache/hadoop/hive/ql/io/ParquetFileStorageFormatDescriptor.java 166775f 
>   ql/src/java/org/apache/hadoop/hive/ql/io/StorageFormatDescriptor.java 604abf8 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/StorageFormat.java 48aca4d 
>   ql/src/test/results/clientpositive/parquet_array_null_element.q.out 387f01e 
>   ql/src/test/results/clientpositive/parquet_create.q.out c6d33ff 
>   ql/src/test/results/clientpositive/parquet_partitioned.q.out 3529d70 
> 
> Diff: https://reviews.apache.org/r/38585/diff/
> 
> 
> Testing
> -------
> 
> Existing unit tests should cover this.
> 
> 
> Thanks,
> 
> Szehon Ho
> 
>