You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hive.apache.org by vi...@apache.org on 2015/01/23 20:39:56 UTC
svn commit: r1654344 - in /hive/branches/branch-1.0/ql/src:
java/org/apache/hadoop/hive/ql/io/parquet/MapredParquetInputFormat.java
test/results/clientpositive/tez/vectorized_parquet.q.out
test/results/clientpositive/vectorized_parquet.q.out
Author: vikram
Date: Fri Jan 23 19:39:56 2015
New Revision: 1654344
URL: http://svn.apache.org/r1654344
Log:
HIVE-9235: Turn off Parquet Vectorization until all data types work: DECIMAL, DATE, TIMESTAMP, CHAR, and VARCHAR (Matt McCline via Vikram Dixit K)
Modified:
hive/branches/branch-1.0/ql/src/java/org/apache/hadoop/hive/ql/io/parquet/MapredParquetInputFormat.java
hive/branches/branch-1.0/ql/src/test/results/clientpositive/tez/vectorized_parquet.q.out
hive/branches/branch-1.0/ql/src/test/results/clientpositive/vectorized_parquet.q.out
Modified: hive/branches/branch-1.0/ql/src/java/org/apache/hadoop/hive/ql/io/parquet/MapredParquetInputFormat.java
URL: http://svn.apache.org/viewvc/hive/branches/branch-1.0/ql/src/java/org/apache/hadoop/hive/ql/io/parquet/MapredParquetInputFormat.java?rev=1654344&r1=1654343&r2=1654344&view=diff
==============================================================================
--- hive/branches/branch-1.0/ql/src/java/org/apache/hadoop/hive/ql/io/parquet/MapredParquetInputFormat.java (original)
+++ hive/branches/branch-1.0/ql/src/java/org/apache/hadoop/hive/ql/io/parquet/MapredParquetInputFormat.java Fri Jan 23 19:39:56 2015
@@ -31,9 +31,10 @@ import parquet.hadoop.ParquetInputFormat
*
* A Parquet InputFormat for Hive (with the deprecated package mapred)
*
+ * NOTE: With HIVE-9235 we removed "implements VectorizedParquetInputFormat" since all data types
+ * are not currently supported. Removing the interface turns off vectorization.
*/
-public class MapredParquetInputFormat extends FileInputFormat<Void, ArrayWritable>
- implements VectorizedInputFormatInterface {
+public class MapredParquetInputFormat extends FileInputFormat<Void, ArrayWritable> {
private static final Log LOG = LogFactory.getLog(MapredParquetInputFormat.class);
Modified: hive/branches/branch-1.0/ql/src/test/results/clientpositive/tez/vectorized_parquet.q.out
URL: http://svn.apache.org/viewvc/hive/branches/branch-1.0/ql/src/test/results/clientpositive/tez/vectorized_parquet.q.out?rev=1654344&r1=1654343&r2=1654344&view=diff
==============================================================================
--- hive/branches/branch-1.0/ql/src/test/results/clientpositive/tez/vectorized_parquet.q.out (original)
+++ hive/branches/branch-1.0/ql/src/test/results/clientpositive/tez/vectorized_parquet.q.out Fri Jan 23 19:39:56 2015
@@ -149,7 +149,6 @@ STAGE PLANS:
Map-reduce partition columns: _col0 (type: tinyint)
Statistics: Num rows: 12288 Data size: 73728 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: int), _col2 (type: smallint), _col3 (type: bigint), _col4 (type: struct<count:bigint,sum:double,input:float>), _col5 (type: struct<count:bigint,sum:double,variance:double>)
- Execution mode: vectorized
Reducer 2
Reduce Operator Tree:
Group By Operator
Modified: hive/branches/branch-1.0/ql/src/test/results/clientpositive/vectorized_parquet.q.out
URL: http://svn.apache.org/viewvc/hive/branches/branch-1.0/ql/src/test/results/clientpositive/vectorized_parquet.q.out?rev=1654344&r1=1654343&r2=1654344&view=diff
==============================================================================
--- hive/branches/branch-1.0/ql/src/test/results/clientpositive/vectorized_parquet.q.out (original)
+++ hive/branches/branch-1.0/ql/src/test/results/clientpositive/vectorized_parquet.q.out Fri Jan 23 19:39:56 2015
@@ -84,7 +84,6 @@ STAGE PLANS:
input format: org.apache.hadoop.mapred.TextInputFormat
output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
- Execution mode: vectorized
Stage: Stage-0
Fetch Operator
@@ -161,7 +160,6 @@ STAGE PLANS:
Map-reduce partition columns: _col0 (type: tinyint)
Statistics: Num rows: 12288 Data size: 73728 Basic stats: COMPLETE Column stats: NONE
value expressions: _col1 (type: int), _col2 (type: smallint), _col3 (type: bigint), _col4 (type: struct<count:bigint,sum:double,input:float>), _col5 (type: struct<count:bigint,sum:double,variance:double>)
- Execution mode: vectorized
Reduce Operator Tree:
Group By Operator
aggregations: max(VALUE._col0), min(VALUE._col1), count(VALUE._col2), avg(VALUE._col3), stddev_pop(VALUE._col4)