You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Naga Vijay <nv...@gmail.com> on 2013/06/13 06:17:02 UTC
Enhancing Query Join to speed up Query
Hi,
Which of the two query options is better?
SELECT a.item_id, a.create_dt
FROM a JOIN b
ON (a.item_id = b.item_id)
WHERE a.item_id = 'I001'
AND a.category_name = 'C001';
- or -
SELECT a.item_id, a.create_dt
FROM a JOIN b
ON (a.item_id = b.item_id AND a.item_id = 'I001')
WHERE a.category_name = 'C001';
Thanks
Naga
Re: Enhancing Query Join to speed up Query
Posted by Navis류승우 <na...@nexr.com>.
Yes, it's identical, as expected.
2013/6/16 Naga Vijay <nv...@gmail.com>:
> Hi,
>
> Thanks for all the responses!
>
> ------------------------------
>
> Here's output of "explain" for query option 1 ...
>
> ------------------------------
>
> ABSTRACT SYNTAX TREE:
> (TOK_QUERY (TOK_FROM (TOK_JOIN (TOK_TABREF (TOK_TABNAME a)) (TOK_TABREF
> (TOK_TABNAME b)) (= (. (TOK_TABLE_OR_COL a) item_id) (. (TOK_TABLE_OR_COL b)
> item_id)))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT
> (TOK_SELEXPR (. (TOK_TABLE_OR_COL a) item_id)) (TOK_SELEXPR (.
> (TOK_TABLE_OR_COL a) create_dt))) (TOK_WHERE (AND (= (. (TOK_TABLE_OR_COL a)
> item_id) 'I501') (= (. (TOK_TABLE_OR_COL a) category_name) 'C1')))))
>
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 is a root stage
>
> STAGE PLANS:
> Stage: Stage-1
> Map Reduce
> Alias -> Map Operator Tree:
> b
> TableScan
> alias: b
> GatherStats: false
> Filter Operator
> isSamplingPred: false
> predicate:
> expr: (item_id = 'I501')
> type: boolean
> Sorted Merge Bucket Map Join Operator
> condition map:
> Inner Join 0 to 1
> condition expressions:
> 0 {item_id} {create_dt}
> 1
> handleSkewJoin: false
> keys:
> 0 [Column[item_id]]
> 1 [Column[item_id]]
> outputColumnNames: _col0, _col3
> Position of Big Table: 1
> Select Operator
> expressions:
> expr: _col0
> type: string
> expr: _col3
> type: string
> outputColumnNames: _col0, _col1
> File Output Operator
> compressed: false
> GlobalTableId: 0
> directory:
> hdfs://sandbox:8020/tmp/hive-root/hive_2013-06-14_11-01-17_851_562334803109383952/-ext-10001
> NumFilesPerFileSink: 1
> Stats Publishing Key Prefix:
> hdfs://sandbox:8020/tmp/hive-root/hive_2013-06-14_11-01-17_851_562334803109383952/-ext-10001/
> table:
> input format:
> org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> columns _col0,_col1
> columns.types string:string
> escape.delim \
> serialization.format 1
> TotalFiles: 1
> GatherStats: false
> MultiFileSpray: false
> Needs Tagging: false
> Path -> Alias:
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-11 [b]
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-12 [b]
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-13 [b]
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-14 [b]
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-15 [b]
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-16 [b]
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-17 [b]
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-18 [b]
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-19 [b]
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-20 [b]
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-21 [b]
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-22 [b]
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-23 [b]
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-24 [b]
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-25 [b]
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-26 [b]
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-27 [b]
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-28 [b]
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-29 [b]
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-30 [b]
> Path -> Partition:
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-11
> Partition
> base file name: create_dt=2013-06-11
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> partition values:
> create_dt 2013-06-11
> properties:
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-11
> name default.b
> numFiles 16
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 16002000
> transient_lastDdlTime 1371159562
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
>
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> EXTERNAL TRUE
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location hdfs://sandbox:8020/apps/hive/warehouse/b
> name default.b
> numFiles 320
> numPartitions 20
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 320040000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> name: default.b
> name: default.b
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-12
> Partition
> base file name: create_dt=2013-06-12
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> partition values:
> create_dt 2013-06-12
> properties:
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-12
> name default.b
> numFiles 16
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 16002000
> transient_lastDdlTime 1371159562
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
>
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> EXTERNAL TRUE
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location hdfs://sandbox:8020/apps/hive/warehouse/b
> name default.b
> numFiles 320
> numPartitions 20
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 320040000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> name: default.b
> name: default.b
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-13
> Partition
> base file name: create_dt=2013-06-13
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> partition values:
> create_dt 2013-06-13
> properties:
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-13
> name default.b
> numFiles 16
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 16002000
> transient_lastDdlTime 1371159562
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
>
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> EXTERNAL TRUE
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location hdfs://sandbox:8020/apps/hive/warehouse/b
> name default.b
> numFiles 320
> numPartitions 20
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 320040000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> name: default.b
> name: default.b
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-14
> Partition
> base file name: create_dt=2013-06-14
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> partition values:
> create_dt 2013-06-14
> properties:
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-14
> name default.b
> numFiles 16
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 16002000
> transient_lastDdlTime 1371159563
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
>
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> EXTERNAL TRUE
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location hdfs://sandbox:8020/apps/hive/warehouse/b
> name default.b
> numFiles 320
> numPartitions 20
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 320040000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> name: default.b
> name: default.b
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-15
> Partition
> base file name: create_dt=2013-06-15
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> partition values:
> create_dt 2013-06-15
> properties:
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-15
> name default.b
> numFiles 16
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 16002000
> transient_lastDdlTime 1371159563
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
>
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> EXTERNAL TRUE
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location hdfs://sandbox:8020/apps/hive/warehouse/b
> name default.b
> numFiles 320
> numPartitions 20
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 320040000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> name: default.b
> name: default.b
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-16
> Partition
> base file name: create_dt=2013-06-16
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> partition values:
> create_dt 2013-06-16
> properties:
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-16
> name default.b
> numFiles 16
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 16002000
> transient_lastDdlTime 1371159563
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
>
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> EXTERNAL TRUE
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location hdfs://sandbox:8020/apps/hive/warehouse/b
> name default.b
> numFiles 320
> numPartitions 20
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 320040000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> name: default.b
> name: default.b
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-17
> Partition
> base file name: create_dt=2013-06-17
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> partition values:
> create_dt 2013-06-17
> properties:
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-17
> name default.b
> numFiles 16
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 16002000
> transient_lastDdlTime 1371159563
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
>
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> EXTERNAL TRUE
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location hdfs://sandbox:8020/apps/hive/warehouse/b
> name default.b
> numFiles 320
> numPartitions 20
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 320040000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> name: default.b
> name: default.b
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-18
> Partition
> base file name: create_dt=2013-06-18
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> partition values:
> create_dt 2013-06-18
> properties:
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-18
> name default.b
> numFiles 16
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 16002000
> transient_lastDdlTime 1371159563
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
>
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> EXTERNAL TRUE
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location hdfs://sandbox:8020/apps/hive/warehouse/b
> name default.b
> numFiles 320
> numPartitions 20
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 320040000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> name: default.b
> name: default.b
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-19
> Partition
> base file name: create_dt=2013-06-19
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> partition values:
> create_dt 2013-06-19
> properties:
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-19
> name default.b
> numFiles 16
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 16002000
> transient_lastDdlTime 1371159563
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
>
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> EXTERNAL TRUE
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location hdfs://sandbox:8020/apps/hive/warehouse/b
> name default.b
> numFiles 320
> numPartitions 20
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 320040000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> name: default.b
> name: default.b
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-20
> Partition
> base file name: create_dt=2013-06-20
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> partition values:
> create_dt 2013-06-20
> properties:
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-20
> name default.b
> numFiles 16
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 16002000
> transient_lastDdlTime 1371159563
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
>
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> EXTERNAL TRUE
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location hdfs://sandbox:8020/apps/hive/warehouse/b
> name default.b
> numFiles 320
> numPartitions 20
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 320040000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> name: default.b
> name: default.b
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-21
> Partition
> base file name: create_dt=2013-06-21
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> partition values:
> create_dt 2013-06-21
> properties:
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-21
> name default.b
> numFiles 16
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 16002000
> transient_lastDdlTime 1371159563
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
>
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> EXTERNAL TRUE
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location hdfs://sandbox:8020/apps/hive/warehouse/b
> name default.b
> numFiles 320
> numPartitions 20
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 320040000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> name: default.b
> name: default.b
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-22
> Partition
> base file name: create_dt=2013-06-22
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> partition values:
> create_dt 2013-06-22
> properties:
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-22
> name default.b
> numFiles 16
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 16002000
> transient_lastDdlTime 1371159563
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
>
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> EXTERNAL TRUE
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location hdfs://sandbox:8020/apps/hive/warehouse/b
> name default.b
> numFiles 320
> numPartitions 20
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 320040000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> name: default.b
> name: default.b
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-23
> Partition
> base file name: create_dt=2013-06-23
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> partition values:
> create_dt 2013-06-23
> properties:
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-23
> name default.b
> numFiles 16
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 16002000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
>
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> EXTERNAL TRUE
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location hdfs://sandbox:8020/apps/hive/warehouse/b
> name default.b
> numFiles 320
> numPartitions 20
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 320040000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> name: default.b
> name: default.b
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-24
> Partition
> base file name: create_dt=2013-06-24
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> partition values:
> create_dt 2013-06-24
> properties:
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-24
> name default.b
> numFiles 16
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 16002000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
>
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> EXTERNAL TRUE
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location hdfs://sandbox:8020/apps/hive/warehouse/b
> name default.b
> numFiles 320
> numPartitions 20
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 320040000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> name: default.b
> name: default.b
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-25
> Partition
> base file name: create_dt=2013-06-25
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> partition values:
> create_dt 2013-06-25
> properties:
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-25
> name default.b
> numFiles 16
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 16002000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
>
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> EXTERNAL TRUE
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location hdfs://sandbox:8020/apps/hive/warehouse/b
> name default.b
> numFiles 320
> numPartitions 20
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 320040000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> name: default.b
> name: default.b
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-26
> Partition
> base file name: create_dt=2013-06-26
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> partition values:
> create_dt 2013-06-26
> properties:
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-26
> name default.b
> numFiles 16
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 16002000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
>
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> EXTERNAL TRUE
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location hdfs://sandbox:8020/apps/hive/warehouse/b
> name default.b
> numFiles 320
> numPartitions 20
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 320040000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> name: default.b
> name: default.b
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-27
> Partition
> base file name: create_dt=2013-06-27
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> partition values:
> create_dt 2013-06-27
> properties:
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-27
> name default.b
> numFiles 16
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 16002000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
>
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> EXTERNAL TRUE
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location hdfs://sandbox:8020/apps/hive/warehouse/b
> name default.b
> numFiles 320
> numPartitions 20
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 320040000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> name: default.b
> name: default.b
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-28
> Partition
> base file name: create_dt=2013-06-28
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> partition values:
> create_dt 2013-06-28
> properties:
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-28
> name default.b
> numFiles 16
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 16002000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
>
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> EXTERNAL TRUE
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location hdfs://sandbox:8020/apps/hive/warehouse/b
> name default.b
> numFiles 320
> numPartitions 20
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 320040000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> name: default.b
> name: default.b
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-29
> Partition
> base file name: create_dt=2013-06-29
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> partition values:
> create_dt 2013-06-29
> properties:
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-29
> name default.b
> numFiles 16
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 16002000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
>
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> EXTERNAL TRUE
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location hdfs://sandbox:8020/apps/hive/warehouse/b
> name default.b
> numFiles 320
> numPartitions 20
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 320040000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> name: default.b
> name: default.b
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-30
> Partition
> base file name: create_dt=2013-06-30
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> partition values:
> create_dt 2013-06-30
> properties:
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-30
> name default.b
> numFiles 16
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 16002000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
>
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> EXTERNAL TRUE
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location hdfs://sandbox:8020/apps/hive/warehouse/b
> name default.b
> numFiles 320
> numPartitions 20
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 320040000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> name: default.b
> name: default.b
> Truncated Path -> Alias:
> /b/create_dt=2013-06-11 [b]
> /b/create_dt=2013-06-12 [b]
> /b/create_dt=2013-06-13 [b]
> /b/create_dt=2013-06-14 [b]
> /b/create_dt=2013-06-15 [b]
> /b/create_dt=2013-06-16 [b]
> /b/create_dt=2013-06-17 [b]
> /b/create_dt=2013-06-18 [b]
> /b/create_dt=2013-06-19 [b]
> /b/create_dt=2013-06-20 [b]
> /b/create_dt=2013-06-21 [b]
> /b/create_dt=2013-06-22 [b]
> /b/create_dt=2013-06-23 [b]
> /b/create_dt=2013-06-24 [b]
> /b/create_dt=2013-06-25 [b]
> /b/create_dt=2013-06-26 [b]
> /b/create_dt=2013-06-27 [b]
> /b/create_dt=2013-06-28 [b]
> /b/create_dt=2013-06-29 [b]
> /b/create_dt=2013-06-30 [b]
>
> Stage: Stage-0
> Fetch Operator
> limit: -1
>
> ------------------------------
>
> And, here's output of "explain" for query option 2 ...
>
> ------------------------------
>
> ABSTRACT SYNTAX TREE:
> (TOK_QUERY (TOK_FROM (TOK_JOIN (TOK_TABREF (TOK_TABNAME a)) (TOK_TABREF
> (TOK_TABNAME b)) (AND (= (. (TOK_TABLE_OR_COL a) item_id) (.
> (TOK_TABLE_OR_COL b) item_id)) (= (. (TOK_TABLE_OR_COL a) item_id)
> 'I501')))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT
> (TOK_SELEXPR (. (TOK_TABLE_OR_COL a) item_id)) (TOK_SELEXPR (.
> (TOK_TABLE_OR_COL a) create_dt))) (TOK_WHERE (= (. (TOK_TABLE_OR_COL a)
> category_name) 'C1'))))
>
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 is a root stage
>
> STAGE PLANS:
> Stage: Stage-1
> Map Reduce
> Alias -> Map Operator Tree:
> b
> TableScan
> alias: b
> GatherStats: false
> Filter Operator
> isSamplingPred: false
> predicate:
> expr: (item_id = 'I501')
> type: boolean
> Sorted Merge Bucket Map Join Operator
> condition map:
> Inner Join 0 to 1
> condition expressions:
> 0 {item_id} {create_dt}
> 1
> handleSkewJoin: false
> keys:
> 0 [Column[item_id]]
> 1 [Column[item_id]]
> outputColumnNames: _col0, _col3
> Position of Big Table: 1
> Select Operator
> expressions:
> expr: _col0
> type: string
> expr: _col3
> type: string
> outputColumnNames: _col0, _col1
> File Output Operator
> compressed: false
> GlobalTableId: 0
> directory:
> hdfs://sandbox:8020/tmp/hive-root/hive_2013-06-14_11-06-11_715_6885480848132014892/-ext-10001
> NumFilesPerFileSink: 1
> Stats Publishing Key Prefix:
> hdfs://sandbox:8020/tmp/hive-root/hive_2013-06-14_11-06-11_715_6885480848132014892/-ext-10001/
> table:
> input format:
> org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> columns _col0,_col1
> columns.types string:string
> escape.delim \
> serialization.format 1
> TotalFiles: 1
> GatherStats: false
> MultiFileSpray: false
> Needs Tagging: false
> Path -> Alias:
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-11 [b]
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-12 [b]
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-13 [b]
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-14 [b]
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-15 [b]
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-16 [b]
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-17 [b]
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-18 [b]
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-19 [b]
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-20 [b]
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-21 [b]
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-22 [b]
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-23 [b]
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-24 [b]
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-25 [b]
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-26 [b]
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-27 [b]
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-28 [b]
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-29 [b]
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-30 [b]
> Path -> Partition:
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-11
> Partition
> base file name: create_dt=2013-06-11
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> partition values:
> create_dt 2013-06-11
> properties:
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-11
> name default.b
> numFiles 16
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 16002000
> transient_lastDdlTime 1371159562
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
>
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> EXTERNAL TRUE
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location hdfs://sandbox:8020/apps/hive/warehouse/b
> name default.b
> numFiles 320
> numPartitions 20
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 320040000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> name: default.b
> name: default.b
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-12
> Partition
> base file name: create_dt=2013-06-12
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> partition values:
> create_dt 2013-06-12
> properties:
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-12
> name default.b
> numFiles 16
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 16002000
> transient_lastDdlTime 1371159562
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
>
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> EXTERNAL TRUE
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location hdfs://sandbox:8020/apps/hive/warehouse/b
> name default.b
> numFiles 320
> numPartitions 20
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 320040000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> name: default.b
> name: default.b
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-13
> Partition
> base file name: create_dt=2013-06-13
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> partition values:
> create_dt 2013-06-13
> properties:
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-13
> name default.b
> numFiles 16
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 16002000
> transient_lastDdlTime 1371159562
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
>
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> EXTERNAL TRUE
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location hdfs://sandbox:8020/apps/hive/warehouse/b
> name default.b
> numFiles 320
> numPartitions 20
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 320040000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> name: default.b
> name: default.b
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-14
> Partition
> base file name: create_dt=2013-06-14
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> partition values:
> create_dt 2013-06-14
> properties:
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-14
> name default.b
> numFiles 16
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 16002000
> transient_lastDdlTime 1371159563
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
>
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> EXTERNAL TRUE
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location hdfs://sandbox:8020/apps/hive/warehouse/b
> name default.b
> numFiles 320
> numPartitions 20
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 320040000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> name: default.b
> name: default.b
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-15
> Partition
> base file name: create_dt=2013-06-15
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> partition values:
> create_dt 2013-06-15
> properties:
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-15
> name default.b
> numFiles 16
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 16002000
> transient_lastDdlTime 1371159563
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
>
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> EXTERNAL TRUE
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location hdfs://sandbox:8020/apps/hive/warehouse/b
> name default.b
> numFiles 320
> numPartitions 20
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 320040000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> name: default.b
> name: default.b
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-16
> Partition
> base file name: create_dt=2013-06-16
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> partition values:
> create_dt 2013-06-16
> properties:
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-16
> name default.b
> numFiles 16
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 16002000
> transient_lastDdlTime 1371159563
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
>
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> EXTERNAL TRUE
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location hdfs://sandbox:8020/apps/hive/warehouse/b
> name default.b
> numFiles 320
> numPartitions 20
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 320040000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> name: default.b
> name: default.b
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-17
> Partition
> base file name: create_dt=2013-06-17
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> partition values:
> create_dt 2013-06-17
> properties:
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-17
> name default.b
> numFiles 16
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 16002000
> transient_lastDdlTime 1371159563
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
>
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> EXTERNAL TRUE
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location hdfs://sandbox:8020/apps/hive/warehouse/b
> name default.b
> numFiles 320
> numPartitions 20
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 320040000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> name: default.b
> name: default.b
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-18
> Partition
> base file name: create_dt=2013-06-18
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> partition values:
> create_dt 2013-06-18
> properties:
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-18
> name default.b
> numFiles 16
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 16002000
> transient_lastDdlTime 1371159563
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
>
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> EXTERNAL TRUE
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location hdfs://sandbox:8020/apps/hive/warehouse/b
> name default.b
> numFiles 320
> numPartitions 20
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 320040000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> name: default.b
> name: default.b
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-19
> Partition
> base file name: create_dt=2013-06-19
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> partition values:
> create_dt 2013-06-19
> properties:
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-19
> name default.b
> numFiles 16
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 16002000
> transient_lastDdlTime 1371159563
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
>
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> EXTERNAL TRUE
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location hdfs://sandbox:8020/apps/hive/warehouse/b
> name default.b
> numFiles 320
> numPartitions 20
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 320040000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> name: default.b
> name: default.b
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-20
> Partition
> base file name: create_dt=2013-06-20
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> partition values:
> create_dt 2013-06-20
> properties:
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-20
> name default.b
> numFiles 16
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 16002000
> transient_lastDdlTime 1371159563
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
>
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> EXTERNAL TRUE
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location hdfs://sandbox:8020/apps/hive/warehouse/b
> name default.b
> numFiles 320
> numPartitions 20
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 320040000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> name: default.b
> name: default.b
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-21
> Partition
> base file name: create_dt=2013-06-21
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> partition values:
> create_dt 2013-06-21
> properties:
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-21
> name default.b
> numFiles 16
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 16002000
> transient_lastDdlTime 1371159563
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
>
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> EXTERNAL TRUE
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location hdfs://sandbox:8020/apps/hive/warehouse/b
> name default.b
> numFiles 320
> numPartitions 20
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 320040000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> name: default.b
> name: default.b
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-22
> Partition
> base file name: create_dt=2013-06-22
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> partition values:
> create_dt 2013-06-22
> properties:
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-22
> name default.b
> numFiles 16
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 16002000
> transient_lastDdlTime 1371159563
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
>
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> EXTERNAL TRUE
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location hdfs://sandbox:8020/apps/hive/warehouse/b
> name default.b
> numFiles 320
> numPartitions 20
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 320040000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> name: default.b
> name: default.b
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-23
> Partition
> base file name: create_dt=2013-06-23
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> partition values:
> create_dt 2013-06-23
> properties:
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-23
> name default.b
> numFiles 16
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 16002000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
>
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> EXTERNAL TRUE
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location hdfs://sandbox:8020/apps/hive/warehouse/b
> name default.b
> numFiles 320
> numPartitions 20
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 320040000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> name: default.b
> name: default.b
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-24
> Partition
> base file name: create_dt=2013-06-24
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> partition values:
> create_dt 2013-06-24
> properties:
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-24
> name default.b
> numFiles 16
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 16002000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
>
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> EXTERNAL TRUE
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location hdfs://sandbox:8020/apps/hive/warehouse/b
> name default.b
> numFiles 320
> numPartitions 20
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 320040000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> name: default.b
> name: default.b
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-25
> Partition
> base file name: create_dt=2013-06-25
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> partition values:
> create_dt 2013-06-25
> properties:
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-25
> name default.b
> numFiles 16
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 16002000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
>
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> EXTERNAL TRUE
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location hdfs://sandbox:8020/apps/hive/warehouse/b
> name default.b
> numFiles 320
> numPartitions 20
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 320040000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> name: default.b
> name: default.b
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-26
> Partition
> base file name: create_dt=2013-06-26
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> partition values:
> create_dt 2013-06-26
> properties:
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-26
> name default.b
> numFiles 16
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 16002000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
>
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> EXTERNAL TRUE
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location hdfs://sandbox:8020/apps/hive/warehouse/b
> name default.b
> numFiles 320
> numPartitions 20
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 320040000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> name: default.b
> name: default.b
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-27
> Partition
> base file name: create_dt=2013-06-27
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> partition values:
> create_dt 2013-06-27
> properties:
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-27
> name default.b
> numFiles 16
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 16002000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
>
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> EXTERNAL TRUE
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location hdfs://sandbox:8020/apps/hive/warehouse/b
> name default.b
> numFiles 320
> numPartitions 20
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 320040000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> name: default.b
> name: default.b
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-28
> Partition
> base file name: create_dt=2013-06-28
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> partition values:
> create_dt 2013-06-28
> properties:
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-28
> name default.b
> numFiles 16
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 16002000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
>
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> EXTERNAL TRUE
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location hdfs://sandbox:8020/apps/hive/warehouse/b
> name default.b
> numFiles 320
> numPartitions 20
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 320040000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> name: default.b
> name: default.b
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-29
> Partition
> base file name: create_dt=2013-06-29
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> partition values:
> create_dt 2013-06-29
> properties:
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-29
> name default.b
> numFiles 16
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 16002000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
>
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> EXTERNAL TRUE
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location hdfs://sandbox:8020/apps/hive/warehouse/b
> name default.b
> numFiles 320
> numPartitions 20
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 320040000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> name: default.b
> name: default.b
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-30
> Partition
> base file name: create_dt=2013-06-30
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> partition values:
> create_dt 2013-06-30
> properties:
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location
> hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-30
> name default.b
> numFiles 16
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 16002000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
>
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> EXTERNAL TRUE
> bucket_count 16
> bucket_field_name item_id
> columns item_id,manufacturer_name,description_id
> columns.types string:string:bigint
> field.delim ,
> file.inputformat org.apache.hadoop.mapred.TextInputFormat
> file.outputformat
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> location hdfs://sandbox:8020/apps/hive/warehouse/b
> name default.b
> numFiles 320
> numPartitions 20
> numRows 0
> partition_columns create_dt
> rawDataSize 0
> serialization.ddl struct b { string item_id, string
> manufacturer_name, i64 description_id}
> serialization.format ,
> serialization.lib
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> totalSize 320040000
> transient_lastDdlTime 1371159564
> serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
> name: default.b
> name: default.b
> Truncated Path -> Alias:
> /b/create_dt=2013-06-11 [b]
> /b/create_dt=2013-06-12 [b]
> /b/create_dt=2013-06-13 [b]
> /b/create_dt=2013-06-14 [b]
> /b/create_dt=2013-06-15 [b]
> /b/create_dt=2013-06-16 [b]
> /b/create_dt=2013-06-17 [b]
> /b/create_dt=2013-06-18 [b]
> /b/create_dt=2013-06-19 [b]
> /b/create_dt=2013-06-20 [b]
> /b/create_dt=2013-06-21 [b]
> /b/create_dt=2013-06-22 [b]
> /b/create_dt=2013-06-23 [b]
> /b/create_dt=2013-06-24 [b]
> /b/create_dt=2013-06-25 [b]
> /b/create_dt=2013-06-26 [b]
> /b/create_dt=2013-06-27 [b]
> /b/create_dt=2013-06-28 [b]
> /b/create_dt=2013-06-29 [b]
> /b/create_dt=2013-06-30 [b]
>
> Stage: Stage-0
> Fetch Operator
> limit: -1
>
> ------------------------------
>
>
>
>
> On Thu, Jun 13, 2013 at 10:45 PM, Alexander Pivovarov <ap...@gmail.com>
> wrote:
>>
>> Basically
>> 1. if you join table try to filter out as much as possible in WHERE (to
>> reduce amount of data sent form map to reduce step)
>> 2. if you join big table with small table (< 500 MB) use SELECT /*+
>> MAPJOIN(small_table) */ hint to avoid reduce step.
>>
>> 3. if you join big table with big table make sure you choose right number
>> of reduce tasks. check "Reduce shuffle bytes" counter for several reducers
>> on Job Tracker UI.
>> from my experience it should not be more than 1 - 2 GB
>> play with
>> set mapred.reduce.tasks=xxx
>> to make Reduce shuffle bytes < 1-2 GB
>>
>>
>>
>>
>>
>> On Thu, Jun 13, 2013 at 10:08 PM, bharath vissapragada
>> <bh...@gmail.com> wrote:
>>>
>>> Hi,
>>>
>>> Can you please paste the output of "explain" for both the queries so that
>>> we can know if the predicate is being pushed down in both the cases.
>>>
>>> Thanks,
>>>
>>>
>>>
>>> On Fri, Jun 14, 2013 at 10:05 AM, Naga Vijay <nv...@gmail.com> wrote:
>>>>
>>>> Hi,
>>>>
>>>> Here are the test results for the two query options shown in my initial
>>>> email -
>>>>
>>>> Test Criteria - Both tables have 20 million rows each, 20 partitions, 16
>>>> buckets for item_id
>>>>
>>>> Query Option 1 - Took 25 minutes, 5 seconds
>>>> Query Option 2 - Took 24 minutes, 42 seconds
>>>>
>>>> Thanks for all the responses!
>>>>
>>>> Naga
>>>>
>>>>
>>>> On Thu, Jun 13, 2013 at 6:42 PM, Navis류승우 <na...@nexr.com> wrote:
>>>>>
>>>>> You can use "explain" for confirming differences. For inner joins, it
>>>>> would make the same plan.
>>>>>
>>>>> 2013/6/14 Igor Tatarinov <ig...@decide.com>:
>>>>> > I would expect no difference because of predicate pushdown.
>>>>> >
>>>>> > igor
>>>>> > decide.com
>>>>> >
>>>>> >
>>>>> > On Thu, Jun 13, 2013 at 11:31 AM, Naga Vijay <nv...@gmail.com>
>>>>> > wrote:
>>>>> >>
>>>>> >> Sure, Will do
>>>>> >>
>>>>> >>
>>>>> >> On Thu, Jun 13, 2013 at 10:42 AM, Stephen Sprague
>>>>> >> <sp...@gmail.com>
>>>>> >> wrote:
>>>>> >>>
>>>>> >>> Hi naja,
>>>>> >>> test those two versions (or three now) and report back to the
>>>>> >>> group. :)
>>>>> >>> even if some smarty-pants thinks he knows the answer its always
>>>>> >>> good to
>>>>> >>> confirm things are as they should be.
>>>>> >>>
>>>>> >>>
>>>>> >>> On Wed, Jun 12, 2013 at 11:54 PM, Sanjay Subramanian
>>>>> >>> <Sa...@wizecommerce.com> wrote:
>>>>> >>>>
>>>>> >>>> Hi
>>>>> >>>>
>>>>> >>>> I would actually do it like this…so that the set on the left of
>>>>> >>>> JOIN
>>>>> >>>> becomes smaller
>>>>> >>>>
>>>>> >>>> SELECT a.item_id, a.create_dt
>>>>> >>>> FROM
>>>>> >>>> ( SELECT
>>>>> >>>> item_id, create_dt
>>>>> >>>> FROM
>>>>> >>>> A
>>>>> >>>> WHERE
>>>>> >>>> item_id = 'I001'
>>>>> >>>> AND
>>>>> >>>> category_name = 'C001'
>>>>> >>>> ) a
>>>>> >>>> JOIN
>>>>> >>>> b
>>>>> >>>> ON
>>>>> >>>> a.item_id = b.item_id
>>>>> >>>> ;
>>>>> >>>>
>>>>> >>>>
>>>>> >>>> From: Naga Vijay <nv...@gmail.com>
>>>>> >>>> Reply-To: "user@hive.apache.org" <us...@hive.apache.org>
>>>>> >>>> Date: Wednesday, June 12, 2013 9:17 PM
>>>>> >>>> To: "user@hive.apache.org" <us...@hive.apache.org>
>>>>> >>>> Subject: Enhancing Query Join to speed up Query
>>>>> >>>>
>>>>> >>>> Hi,
>>>>> >>>>
>>>>> >>>> Which of the two query options is better?
>>>>> >>>>
>>>>> >>>> SELECT a.item_id, a.create_dt
>>>>> >>>> FROM a JOIN b
>>>>> >>>> ON (a.item_id = b.item_id)
>>>>> >>>> WHERE a.item_id = 'I001'
>>>>> >>>> AND a.category_name = 'C001';
>>>>> >>>>
>>>>> >>>> - or -
>>>>> >>>>
>>>>> >>>> SELECT a.item_id, a.create_dt
>>>>> >>>> FROM a JOIN b
>>>>> >>>> ON (a.item_id = b.item_id AND a.item_id = 'I001')
>>>>> >>>> WHERE a.category_name = 'C001';
>>>>> >>>>
>>>>> >>>> Thanks
>>>>> >>>> Naga
>>>>> >>>>
>>>>> >>>> CONFIDENTIALITY NOTICE
>>>>> >>>> ======================
>>>>> >>>> This email message and any attachments are for the exclusive use
>>>>> >>>> of the
>>>>> >>>> intended recipient(s) and may contain confidential and privileged
>>>>> >>>> information. Any unauthorized review, use, disclosure or
>>>>> >>>> distribution is
>>>>> >>>> prohibited. If you are not the intended recipient, please contact
>>>>> >>>> the sender
>>>>> >>>> by reply email and destroy all copies of the original message
>>>>> >>>> along with any
>>>>> >>>> attachments, from your computer system. If you are the intended
>>>>> >>>> recipient,
>>>>> >>>> please be advised that the content of this message is subject to
>>>>> >>>> access,
>>>>> >>>> review and disclosure by the sender's Email System Administrator.
>>>>> >>>
>>>>> >>>
>>>>> >>
>>>>> >
>>>>
>>>>
>>>
>>
>
Re: Enhancing Query Join to speed up Query
Posted by Naga Vijay <nv...@gmail.com>.
Hi,
Thanks for all the responses!
------------------------------
Here's output of "explain" for query option 1 ...
------------------------------
ABSTRACT SYNTAX TREE:
(TOK_QUERY (TOK_FROM (TOK_JOIN (TOK_TABREF (TOK_TABNAME a)) (TOK_TABREF
(TOK_TABNAME b)) (= (. (TOK_TABLE_OR_COL a) item_id) (. (TOK_TABLE_OR_COL
b) item_id)))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE))
(TOK_SELECT (TOK_SELEXPR (. (TOK_TABLE_OR_COL a) item_id)) (TOK_SELEXPR (.
(TOK_TABLE_OR_COL a) create_dt))) (TOK_WHERE (AND (= (. (TOK_TABLE_OR_COL
a) item_id) 'I501') (= (. (TOK_TABLE_OR_COL a) category_name) 'C1')))))
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 is a root stage
STAGE PLANS:
Stage: Stage-1
Map Reduce
Alias -> Map Operator Tree:
b
TableScan
alias: b
GatherStats: false
Filter Operator
isSamplingPred: false
predicate:
expr: (item_id = 'I501')
type: boolean
Sorted Merge Bucket Map Join Operator
condition map:
Inner Join 0 to 1
condition expressions:
0 {item_id} {create_dt}
1
handleSkewJoin: false
keys:
0 [Column[item_id]]
1 [Column[item_id]]
outputColumnNames: _col0, _col3
Position of Big Table: 1
Select Operator
expressions:
expr: _col0
type: string
expr: _col3
type: string
outputColumnNames: _col0, _col1
File Output Operator
compressed: false
GlobalTableId: 0
directory:
hdfs://sandbox:8020/tmp/hive-root/hive_2013-06-14_11-01-17_851_562334803109383952/-ext-10001
NumFilesPerFileSink: 1
Stats Publishing Key Prefix:
hdfs://sandbox:8020/tmp/hive-root/hive_2013-06-14_11-01-17_851_562334803109383952/-ext-10001/
table:
input format:
org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
columns _col0,_col1
columns.types string:string
escape.delim \
serialization.format 1
TotalFiles: 1
GatherStats: false
MultiFileSpray: false
Needs Tagging: false
Path -> Alias:
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-11 [b]
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-12 [b]
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-13 [b]
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-14 [b]
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-15 [b]
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-16 [b]
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-17 [b]
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-18 [b]
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-19 [b]
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-20 [b]
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-21 [b]
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-22 [b]
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-23 [b]
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-24 [b]
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-25 [b]
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-26 [b]
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-27 [b]
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-28 [b]
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-29 [b]
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-30 [b]
Path -> Partition:
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-11
Partition
base file name: create_dt=2013-06-11
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
create_dt 2013-06-11
properties:
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-11
name default.b
numFiles 16
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 16002000
transient_lastDdlTime 1371159562
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
EXTERNAL TRUE
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location hdfs://sandbox:8020/apps/hive/warehouse/b
name default.b
numFiles 320
numPartitions 20
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 320040000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.b
name: default.b
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-12
Partition
base file name: create_dt=2013-06-12
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
create_dt 2013-06-12
properties:
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-12
name default.b
numFiles 16
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 16002000
transient_lastDdlTime 1371159562
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
EXTERNAL TRUE
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location hdfs://sandbox:8020/apps/hive/warehouse/b
name default.b
numFiles 320
numPartitions 20
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 320040000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.b
name: default.b
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-13
Partition
base file name: create_dt=2013-06-13
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
create_dt 2013-06-13
properties:
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-13
name default.b
numFiles 16
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 16002000
transient_lastDdlTime 1371159562
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
EXTERNAL TRUE
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location hdfs://sandbox:8020/apps/hive/warehouse/b
name default.b
numFiles 320
numPartitions 20
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 320040000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.b
name: default.b
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-14
Partition
base file name: create_dt=2013-06-14
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
create_dt 2013-06-14
properties:
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-14
name default.b
numFiles 16
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 16002000
transient_lastDdlTime 1371159563
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
EXTERNAL TRUE
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location hdfs://sandbox:8020/apps/hive/warehouse/b
name default.b
numFiles 320
numPartitions 20
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 320040000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.b
name: default.b
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-15
Partition
base file name: create_dt=2013-06-15
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
create_dt 2013-06-15
properties:
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-15
name default.b
numFiles 16
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 16002000
transient_lastDdlTime 1371159563
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
EXTERNAL TRUE
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location hdfs://sandbox:8020/apps/hive/warehouse/b
name default.b
numFiles 320
numPartitions 20
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 320040000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.b
name: default.b
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-16
Partition
base file name: create_dt=2013-06-16
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
create_dt 2013-06-16
properties:
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-16
name default.b
numFiles 16
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 16002000
transient_lastDdlTime 1371159563
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
EXTERNAL TRUE
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location hdfs://sandbox:8020/apps/hive/warehouse/b
name default.b
numFiles 320
numPartitions 20
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 320040000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.b
name: default.b
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-17
Partition
base file name: create_dt=2013-06-17
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
create_dt 2013-06-17
properties:
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-17
name default.b
numFiles 16
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 16002000
transient_lastDdlTime 1371159563
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
EXTERNAL TRUE
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location hdfs://sandbox:8020/apps/hive/warehouse/b
name default.b
numFiles 320
numPartitions 20
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 320040000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.b
name: default.b
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-18
Partition
base file name: create_dt=2013-06-18
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
create_dt 2013-06-18
properties:
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-18
name default.b
numFiles 16
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 16002000
transient_lastDdlTime 1371159563
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
EXTERNAL TRUE
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location hdfs://sandbox:8020/apps/hive/warehouse/b
name default.b
numFiles 320
numPartitions 20
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 320040000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.b
name: default.b
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-19
Partition
base file name: create_dt=2013-06-19
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
create_dt 2013-06-19
properties:
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-19
name default.b
numFiles 16
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 16002000
transient_lastDdlTime 1371159563
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
EXTERNAL TRUE
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location hdfs://sandbox:8020/apps/hive/warehouse/b
name default.b
numFiles 320
numPartitions 20
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 320040000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.b
name: default.b
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-20
Partition
base file name: create_dt=2013-06-20
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
create_dt 2013-06-20
properties:
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-20
name default.b
numFiles 16
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 16002000
transient_lastDdlTime 1371159563
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
EXTERNAL TRUE
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location hdfs://sandbox:8020/apps/hive/warehouse/b
name default.b
numFiles 320
numPartitions 20
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 320040000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.b
name: default.b
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-21
Partition
base file name: create_dt=2013-06-21
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
create_dt 2013-06-21
properties:
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-21
name default.b
numFiles 16
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 16002000
transient_lastDdlTime 1371159563
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
EXTERNAL TRUE
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location hdfs://sandbox:8020/apps/hive/warehouse/b
name default.b
numFiles 320
numPartitions 20
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 320040000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.b
name: default.b
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-22
Partition
base file name: create_dt=2013-06-22
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
create_dt 2013-06-22
properties:
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-22
name default.b
numFiles 16
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 16002000
transient_lastDdlTime 1371159563
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
EXTERNAL TRUE
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location hdfs://sandbox:8020/apps/hive/warehouse/b
name default.b
numFiles 320
numPartitions 20
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 320040000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.b
name: default.b
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-23
Partition
base file name: create_dt=2013-06-23
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
create_dt 2013-06-23
properties:
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-23
name default.b
numFiles 16
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 16002000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
EXTERNAL TRUE
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location hdfs://sandbox:8020/apps/hive/warehouse/b
name default.b
numFiles 320
numPartitions 20
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 320040000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.b
name: default.b
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-24
Partition
base file name: create_dt=2013-06-24
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
create_dt 2013-06-24
properties:
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-24
name default.b
numFiles 16
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 16002000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
EXTERNAL TRUE
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location hdfs://sandbox:8020/apps/hive/warehouse/b
name default.b
numFiles 320
numPartitions 20
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 320040000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.b
name: default.b
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-25
Partition
base file name: create_dt=2013-06-25
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
create_dt 2013-06-25
properties:
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-25
name default.b
numFiles 16
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 16002000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
EXTERNAL TRUE
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location hdfs://sandbox:8020/apps/hive/warehouse/b
name default.b
numFiles 320
numPartitions 20
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 320040000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.b
name: default.b
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-26
Partition
base file name: create_dt=2013-06-26
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
create_dt 2013-06-26
properties:
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-26
name default.b
numFiles 16
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 16002000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
EXTERNAL TRUE
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location hdfs://sandbox:8020/apps/hive/warehouse/b
name default.b
numFiles 320
numPartitions 20
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 320040000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.b
name: default.b
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-27
Partition
base file name: create_dt=2013-06-27
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
create_dt 2013-06-27
properties:
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-27
name default.b
numFiles 16
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 16002000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
EXTERNAL TRUE
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location hdfs://sandbox:8020/apps/hive/warehouse/b
name default.b
numFiles 320
numPartitions 20
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 320040000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.b
name: default.b
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-28
Partition
base file name: create_dt=2013-06-28
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
create_dt 2013-06-28
properties:
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-28
name default.b
numFiles 16
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 16002000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
EXTERNAL TRUE
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location hdfs://sandbox:8020/apps/hive/warehouse/b
name default.b
numFiles 320
numPartitions 20
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 320040000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.b
name: default.b
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-29
Partition
base file name: create_dt=2013-06-29
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
create_dt 2013-06-29
properties:
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-29
name default.b
numFiles 16
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 16002000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
EXTERNAL TRUE
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location hdfs://sandbox:8020/apps/hive/warehouse/b
name default.b
numFiles 320
numPartitions 20
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 320040000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.b
name: default.b
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-30
Partition
base file name: create_dt=2013-06-30
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
create_dt 2013-06-30
properties:
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-30
name default.b
numFiles 16
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 16002000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
EXTERNAL TRUE
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location hdfs://sandbox:8020/apps/hive/warehouse/b
name default.b
numFiles 320
numPartitions 20
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 320040000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.b
name: default.b
Truncated Path -> Alias:
/b/create_dt=2013-06-11 [b]
/b/create_dt=2013-06-12 [b]
/b/create_dt=2013-06-13 [b]
/b/create_dt=2013-06-14 [b]
/b/create_dt=2013-06-15 [b]
/b/create_dt=2013-06-16 [b]
/b/create_dt=2013-06-17 [b]
/b/create_dt=2013-06-18 [b]
/b/create_dt=2013-06-19 [b]
/b/create_dt=2013-06-20 [b]
/b/create_dt=2013-06-21 [b]
/b/create_dt=2013-06-22 [b]
/b/create_dt=2013-06-23 [b]
/b/create_dt=2013-06-24 [b]
/b/create_dt=2013-06-25 [b]
/b/create_dt=2013-06-26 [b]
/b/create_dt=2013-06-27 [b]
/b/create_dt=2013-06-28 [b]
/b/create_dt=2013-06-29 [b]
/b/create_dt=2013-06-30 [b]
Stage: Stage-0
Fetch Operator
limit: -1
------------------------------
And, here's output of "explain" for query option 2 ...
------------------------------
ABSTRACT SYNTAX TREE:
(TOK_QUERY (TOK_FROM (TOK_JOIN (TOK_TABREF (TOK_TABNAME a)) (TOK_TABREF
(TOK_TABNAME b)) (AND (= (. (TOK_TABLE_OR_COL a) item_id) (.
(TOK_TABLE_OR_COL b) item_id)) (= (. (TOK_TABLE_OR_COL a) item_id)
'I501')))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT
(TOK_SELEXPR (. (TOK_TABLE_OR_COL a) item_id)) (TOK_SELEXPR (.
(TOK_TABLE_OR_COL a) create_dt))) (TOK_WHERE (= (. (TOK_TABLE_OR_COL a)
category_name) 'C1'))))
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 is a root stage
STAGE PLANS:
Stage: Stage-1
Map Reduce
Alias -> Map Operator Tree:
b
TableScan
alias: b
GatherStats: false
Filter Operator
isSamplingPred: false
predicate:
expr: (item_id = 'I501')
type: boolean
Sorted Merge Bucket Map Join Operator
condition map:
Inner Join 0 to 1
condition expressions:
0 {item_id} {create_dt}
1
handleSkewJoin: false
keys:
0 [Column[item_id]]
1 [Column[item_id]]
outputColumnNames: _col0, _col3
Position of Big Table: 1
Select Operator
expressions:
expr: _col0
type: string
expr: _col3
type: string
outputColumnNames: _col0, _col1
File Output Operator
compressed: false
GlobalTableId: 0
directory:
hdfs://sandbox:8020/tmp/hive-root/hive_2013-06-14_11-06-11_715_6885480848132014892/-ext-10001
NumFilesPerFileSink: 1
Stats Publishing Key Prefix:
hdfs://sandbox:8020/tmp/hive-root/hive_2013-06-14_11-06-11_715_6885480848132014892/-ext-10001/
table:
input format:
org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
columns _col0,_col1
columns.types string:string
escape.delim \
serialization.format 1
TotalFiles: 1
GatherStats: false
MultiFileSpray: false
Needs Tagging: false
Path -> Alias:
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-11 [b]
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-12 [b]
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-13 [b]
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-14 [b]
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-15 [b]
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-16 [b]
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-17 [b]
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-18 [b]
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-19 [b]
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-20 [b]
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-21 [b]
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-22 [b]
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-23 [b]
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-24 [b]
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-25 [b]
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-26 [b]
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-27 [b]
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-28 [b]
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-29 [b]
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-30 [b]
Path -> Partition:
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-11
Partition
base file name: create_dt=2013-06-11
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
create_dt 2013-06-11
properties:
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-11
name default.b
numFiles 16
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 16002000
transient_lastDdlTime 1371159562
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
EXTERNAL TRUE
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location hdfs://sandbox:8020/apps/hive/warehouse/b
name default.b
numFiles 320
numPartitions 20
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 320040000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.b
name: default.b
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-12
Partition
base file name: create_dt=2013-06-12
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
create_dt 2013-06-12
properties:
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-12
name default.b
numFiles 16
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 16002000
transient_lastDdlTime 1371159562
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
EXTERNAL TRUE
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location hdfs://sandbox:8020/apps/hive/warehouse/b
name default.b
numFiles 320
numPartitions 20
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 320040000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.b
name: default.b
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-13
Partition
base file name: create_dt=2013-06-13
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
create_dt 2013-06-13
properties:
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-13
name default.b
numFiles 16
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 16002000
transient_lastDdlTime 1371159562
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
EXTERNAL TRUE
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location hdfs://sandbox:8020/apps/hive/warehouse/b
name default.b
numFiles 320
numPartitions 20
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 320040000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.b
name: default.b
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-14
Partition
base file name: create_dt=2013-06-14
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
create_dt 2013-06-14
properties:
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-14
name default.b
numFiles 16
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 16002000
transient_lastDdlTime 1371159563
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
EXTERNAL TRUE
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location hdfs://sandbox:8020/apps/hive/warehouse/b
name default.b
numFiles 320
numPartitions 20
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 320040000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.b
name: default.b
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-15
Partition
base file name: create_dt=2013-06-15
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
create_dt 2013-06-15
properties:
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-15
name default.b
numFiles 16
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 16002000
transient_lastDdlTime 1371159563
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
EXTERNAL TRUE
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location hdfs://sandbox:8020/apps/hive/warehouse/b
name default.b
numFiles 320
numPartitions 20
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 320040000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.b
name: default.b
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-16
Partition
base file name: create_dt=2013-06-16
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
create_dt 2013-06-16
properties:
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-16
name default.b
numFiles 16
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 16002000
transient_lastDdlTime 1371159563
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
EXTERNAL TRUE
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location hdfs://sandbox:8020/apps/hive/warehouse/b
name default.b
numFiles 320
numPartitions 20
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 320040000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.b
name: default.b
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-17
Partition
base file name: create_dt=2013-06-17
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
create_dt 2013-06-17
properties:
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-17
name default.b
numFiles 16
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 16002000
transient_lastDdlTime 1371159563
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
EXTERNAL TRUE
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location hdfs://sandbox:8020/apps/hive/warehouse/b
name default.b
numFiles 320
numPartitions 20
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 320040000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.b
name: default.b
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-18
Partition
base file name: create_dt=2013-06-18
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
create_dt 2013-06-18
properties:
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-18
name default.b
numFiles 16
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 16002000
transient_lastDdlTime 1371159563
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
EXTERNAL TRUE
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location hdfs://sandbox:8020/apps/hive/warehouse/b
name default.b
numFiles 320
numPartitions 20
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 320040000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.b
name: default.b
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-19
Partition
base file name: create_dt=2013-06-19
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
create_dt 2013-06-19
properties:
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-19
name default.b
numFiles 16
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 16002000
transient_lastDdlTime 1371159563
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
EXTERNAL TRUE
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location hdfs://sandbox:8020/apps/hive/warehouse/b
name default.b
numFiles 320
numPartitions 20
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 320040000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.b
name: default.b
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-20
Partition
base file name: create_dt=2013-06-20
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
create_dt 2013-06-20
properties:
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-20
name default.b
numFiles 16
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 16002000
transient_lastDdlTime 1371159563
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
EXTERNAL TRUE
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location hdfs://sandbox:8020/apps/hive/warehouse/b
name default.b
numFiles 320
numPartitions 20
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 320040000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.b
name: default.b
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-21
Partition
base file name: create_dt=2013-06-21
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
create_dt 2013-06-21
properties:
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-21
name default.b
numFiles 16
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 16002000
transient_lastDdlTime 1371159563
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
EXTERNAL TRUE
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location hdfs://sandbox:8020/apps/hive/warehouse/b
name default.b
numFiles 320
numPartitions 20
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 320040000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.b
name: default.b
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-22
Partition
base file name: create_dt=2013-06-22
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
create_dt 2013-06-22
properties:
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-22
name default.b
numFiles 16
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 16002000
transient_lastDdlTime 1371159563
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
EXTERNAL TRUE
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location hdfs://sandbox:8020/apps/hive/warehouse/b
name default.b
numFiles 320
numPartitions 20
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 320040000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.b
name: default.b
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-23
Partition
base file name: create_dt=2013-06-23
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
create_dt 2013-06-23
properties:
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-23
name default.b
numFiles 16
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 16002000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
EXTERNAL TRUE
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location hdfs://sandbox:8020/apps/hive/warehouse/b
name default.b
numFiles 320
numPartitions 20
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 320040000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.b
name: default.b
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-24
Partition
base file name: create_dt=2013-06-24
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
create_dt 2013-06-24
properties:
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-24
name default.b
numFiles 16
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 16002000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
EXTERNAL TRUE
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location hdfs://sandbox:8020/apps/hive/warehouse/b
name default.b
numFiles 320
numPartitions 20
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 320040000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.b
name: default.b
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-25
Partition
base file name: create_dt=2013-06-25
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
create_dt 2013-06-25
properties:
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-25
name default.b
numFiles 16
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 16002000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
EXTERNAL TRUE
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location hdfs://sandbox:8020/apps/hive/warehouse/b
name default.b
numFiles 320
numPartitions 20
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 320040000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.b
name: default.b
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-26
Partition
base file name: create_dt=2013-06-26
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
create_dt 2013-06-26
properties:
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-26
name default.b
numFiles 16
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 16002000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
EXTERNAL TRUE
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location hdfs://sandbox:8020/apps/hive/warehouse/b
name default.b
numFiles 320
numPartitions 20
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 320040000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.b
name: default.b
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-27
Partition
base file name: create_dt=2013-06-27
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
create_dt 2013-06-27
properties:
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-27
name default.b
numFiles 16
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 16002000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
EXTERNAL TRUE
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location hdfs://sandbox:8020/apps/hive/warehouse/b
name default.b
numFiles 320
numPartitions 20
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 320040000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.b
name: default.b
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-28
Partition
base file name: create_dt=2013-06-28
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
create_dt 2013-06-28
properties:
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-28
name default.b
numFiles 16
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 16002000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
EXTERNAL TRUE
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location hdfs://sandbox:8020/apps/hive/warehouse/b
name default.b
numFiles 320
numPartitions 20
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 320040000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.b
name: default.b
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-29
Partition
base file name: create_dt=2013-06-29
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
create_dt 2013-06-29
properties:
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-29
name default.b
numFiles 16
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 16002000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
EXTERNAL TRUE
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location hdfs://sandbox:8020/apps/hive/warehouse/b
name default.b
numFiles 320
numPartitions 20
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 320040000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.b
name: default.b
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-30
Partition
base file name: create_dt=2013-06-30
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
partition values:
create_dt 2013-06-30
properties:
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location
hdfs://sandbox:8020/apps/hive/warehouse/b/create_dt=2013-06-30
name default.b
numFiles 16
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 16002000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
input format: org.apache.hadoop.mapred.TextInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
properties:
EXTERNAL TRUE
bucket_count 16
bucket_field_name item_id
columns item_id,manufacturer_name,description_id
columns.types string:string:bigint
field.delim ,
file.inputformat org.apache.hadoop.mapred.TextInputFormat
file.outputformat
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
location hdfs://sandbox:8020/apps/hive/warehouse/b
name default.b
numFiles 320
numPartitions 20
numRows 0
partition_columns create_dt
rawDataSize 0
serialization.ddl struct b { string item_id, string
manufacturer_name, i64 description_id}
serialization.format ,
serialization.lib
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
totalSize 320040000
transient_lastDdlTime 1371159564
serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
name: default.b
name: default.b
Truncated Path -> Alias:
/b/create_dt=2013-06-11 [b]
/b/create_dt=2013-06-12 [b]
/b/create_dt=2013-06-13 [b]
/b/create_dt=2013-06-14 [b]
/b/create_dt=2013-06-15 [b]
/b/create_dt=2013-06-16 [b]
/b/create_dt=2013-06-17 [b]
/b/create_dt=2013-06-18 [b]
/b/create_dt=2013-06-19 [b]
/b/create_dt=2013-06-20 [b]
/b/create_dt=2013-06-21 [b]
/b/create_dt=2013-06-22 [b]
/b/create_dt=2013-06-23 [b]
/b/create_dt=2013-06-24 [b]
/b/create_dt=2013-06-25 [b]
/b/create_dt=2013-06-26 [b]
/b/create_dt=2013-06-27 [b]
/b/create_dt=2013-06-28 [b]
/b/create_dt=2013-06-29 [b]
/b/create_dt=2013-06-30 [b]
Stage: Stage-0
Fetch Operator
limit: -1
------------------------------
On Thu, Jun 13, 2013 at 10:45 PM, Alexander Pivovarov
<ap...@gmail.com>wrote:
> Basically
> 1. if you join table try to filter out as much as possible in WHERE (to
> reduce amount of data sent form map to reduce step)
> 2. if you join big table with small table (< 500 MB) use SELECT /*+
> MAPJOIN(small_table) */ hint to avoid reduce step.
>
> 3. if you join big table with big table make sure you choose right number
> of reduce tasks. check "Reduce shuffle bytes" counter for several
> reducers on Job Tracker UI.
> from my experience it should not be more than 1 - 2 GB
> play with
> set mapred.reduce.tasks=xxx
> to make Reduce shuffle bytes < 1-2 GB
>
>
>
>
>
> On Thu, Jun 13, 2013 at 10:08 PM, bharath vissapragada <
> bharathvissapragada1990@gmail.com> wrote:
>
>> Hi,
>>
>> Can you please paste the output of "explain" for both the queries so that
>> we can know if the predicate is being pushed down in both the cases.
>>
>> Thanks,
>>
>>
>>
>> On Fri, Jun 14, 2013 at 10:05 AM, Naga Vijay <nv...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> Here are the test results for the two query options shown in my initial
>>> email -
>>>
>>> Test Criteria - Both tables have 20 million rows each, 20 partitions, 16
>>> buckets for item_id
>>>
>>> Query Option 1 - Took 25 minutes, 5 seconds
>>> Query Option 2 - Took 24 minutes, 42 seconds
>>>
>>> Thanks for all the responses!
>>>
>>> Naga
>>>
>>>
>>> On Thu, Jun 13, 2013 at 6:42 PM, Navis류승우 <na...@nexr.com> wrote:
>>>
>>>> You can use "explain" for confirming differences. For inner joins, it
>>>> would make the same plan.
>>>>
>>>> 2013/6/14 Igor Tatarinov <ig...@decide.com>:
>>>> > I would expect no difference because of predicate pushdown.
>>>> >
>>>> > igor
>>>> > decide.com
>>>> >
>>>> >
>>>> > On Thu, Jun 13, 2013 at 11:31 AM, Naga Vijay <nv...@gmail.com>
>>>> wrote:
>>>> >>
>>>> >> Sure, Will do
>>>> >>
>>>> >>
>>>> >> On Thu, Jun 13, 2013 at 10:42 AM, Stephen Sprague <
>>>> spragues@gmail.com>
>>>> >> wrote:
>>>> >>>
>>>> >>> Hi naja,
>>>> >>> test those two versions (or three now) and report back to the
>>>> group. :)
>>>> >>> even if some smarty-pants thinks he knows the answer its always
>>>> good to
>>>> >>> confirm things are as they should be.
>>>> >>>
>>>> >>>
>>>> >>> On Wed, Jun 12, 2013 at 11:54 PM, Sanjay Subramanian
>>>> >>> <Sa...@wizecommerce.com> wrote:
>>>> >>>>
>>>> >>>> Hi
>>>> >>>>
>>>> >>>> I would actually do it like this…so that the set on the left of
>>>> JOIN
>>>> >>>> becomes smaller
>>>> >>>>
>>>> >>>> SELECT a.item_id, a.create_dt
>>>> >>>> FROM
>>>> >>>> ( SELECT
>>>> >>>> item_id, create_dt
>>>> >>>> FROM
>>>> >>>> A
>>>> >>>> WHERE
>>>> >>>> item_id = 'I001'
>>>> >>>> AND
>>>> >>>> category_name = 'C001'
>>>> >>>> ) a
>>>> >>>> JOIN
>>>> >>>> b
>>>> >>>> ON
>>>> >>>> a.item_id = b.item_id
>>>> >>>> ;
>>>> >>>>
>>>> >>>>
>>>> >>>> From: Naga Vijay <nv...@gmail.com>
>>>> >>>> Reply-To: "user@hive.apache.org" <us...@hive.apache.org>
>>>> >>>> Date: Wednesday, June 12, 2013 9:17 PM
>>>> >>>> To: "user@hive.apache.org" <us...@hive.apache.org>
>>>> >>>> Subject: Enhancing Query Join to speed up Query
>>>> >>>>
>>>> >>>> Hi,
>>>> >>>>
>>>> >>>> Which of the two query options is better?
>>>> >>>>
>>>> >>>> SELECT a.item_id, a.create_dt
>>>> >>>> FROM a JOIN b
>>>> >>>> ON (a.item_id = b.item_id)
>>>> >>>> WHERE a.item_id = 'I001'
>>>> >>>> AND a.category_name = 'C001';
>>>> >>>>
>>>> >>>> - or -
>>>> >>>>
>>>> >>>> SELECT a.item_id, a.create_dt
>>>> >>>> FROM a JOIN b
>>>> >>>> ON (a.item_id = b.item_id AND a.item_id = 'I001')
>>>> >>>> WHERE a.category_name = 'C001';
>>>> >>>>
>>>> >>>> Thanks
>>>> >>>> Naga
>>>> >>>>
>>>> >>>> CONFIDENTIALITY NOTICE
>>>> >>>> ======================
>>>> >>>> This email message and any attachments are for the exclusive use
>>>> of the
>>>> >>>> intended recipient(s) and may contain confidential and privileged
>>>> >>>> information. Any unauthorized review, use, disclosure or
>>>> distribution is
>>>> >>>> prohibited. If you are not the intended recipient, please contact
>>>> the sender
>>>> >>>> by reply email and destroy all copies of the original message
>>>> along with any
>>>> >>>> attachments, from your computer system. If you are the intended
>>>> recipient,
>>>> >>>> please be advised that the content of this message is subject to
>>>> access,
>>>> >>>> review and disclosure by the sender's Email System Administrator.
>>>> >>>
>>>> >>>
>>>> >>
>>>> >
>>>>
>>>
>>>
>>
>
Re: Enhancing Query Join to speed up Query
Posted by Alexander Pivovarov <ap...@gmail.com>.
Basically
1. if you join table try to filter out as much as possible in WHERE (to
reduce amount of data sent form map to reduce step)
2. if you join big table with small table (< 500 MB) use SELECT /*+
MAPJOIN(small_table) */ hint to avoid reduce step.
3. if you join big table with big table make sure you choose right number
of reduce tasks. check "Reduce shuffle bytes" counter for several reducers
on Job Tracker UI.
from my experience it should not be more than 1 - 2 GB
play with
set mapred.reduce.tasks=xxx
to make Reduce shuffle bytes < 1-2 GB
On Thu, Jun 13, 2013 at 10:08 PM, bharath vissapragada <
bharathvissapragada1990@gmail.com> wrote:
> Hi,
>
> Can you please paste the output of "explain" for both the queries so that
> we can know if the predicate is being pushed down in both the cases.
>
> Thanks,
>
>
>
> On Fri, Jun 14, 2013 at 10:05 AM, Naga Vijay <nv...@gmail.com> wrote:
>
>> Hi,
>>
>> Here are the test results for the two query options shown in my initial
>> email -
>>
>> Test Criteria - Both tables have 20 million rows each, 20 partitions, 16
>> buckets for item_id
>>
>> Query Option 1 - Took 25 minutes, 5 seconds
>> Query Option 2 - Took 24 minutes, 42 seconds
>>
>> Thanks for all the responses!
>>
>> Naga
>>
>>
>> On Thu, Jun 13, 2013 at 6:42 PM, Navis류승우 <na...@nexr.com> wrote:
>>
>>> You can use "explain" for confirming differences. For inner joins, it
>>> would make the same plan.
>>>
>>> 2013/6/14 Igor Tatarinov <ig...@decide.com>:
>>> > I would expect no difference because of predicate pushdown.
>>> >
>>> > igor
>>> > decide.com
>>> >
>>> >
>>> > On Thu, Jun 13, 2013 at 11:31 AM, Naga Vijay <nv...@gmail.com>
>>> wrote:
>>> >>
>>> >> Sure, Will do
>>> >>
>>> >>
>>> >> On Thu, Jun 13, 2013 at 10:42 AM, Stephen Sprague <spragues@gmail.com
>>> >
>>> >> wrote:
>>> >>>
>>> >>> Hi naja,
>>> >>> test those two versions (or three now) and report back to the group.
>>> :)
>>> >>> even if some smarty-pants thinks he knows the answer its always good
>>> to
>>> >>> confirm things are as they should be.
>>> >>>
>>> >>>
>>> >>> On Wed, Jun 12, 2013 at 11:54 PM, Sanjay Subramanian
>>> >>> <Sa...@wizecommerce.com> wrote:
>>> >>>>
>>> >>>> Hi
>>> >>>>
>>> >>>> I would actually do it like this…so that the set on the left of JOIN
>>> >>>> becomes smaller
>>> >>>>
>>> >>>> SELECT a.item_id, a.create_dt
>>> >>>> FROM
>>> >>>> ( SELECT
>>> >>>> item_id, create_dt
>>> >>>> FROM
>>> >>>> A
>>> >>>> WHERE
>>> >>>> item_id = 'I001'
>>> >>>> AND
>>> >>>> category_name = 'C001'
>>> >>>> ) a
>>> >>>> JOIN
>>> >>>> b
>>> >>>> ON
>>> >>>> a.item_id = b.item_id
>>> >>>> ;
>>> >>>>
>>> >>>>
>>> >>>> From: Naga Vijay <nv...@gmail.com>
>>> >>>> Reply-To: "user@hive.apache.org" <us...@hive.apache.org>
>>> >>>> Date: Wednesday, June 12, 2013 9:17 PM
>>> >>>> To: "user@hive.apache.org" <us...@hive.apache.org>
>>> >>>> Subject: Enhancing Query Join to speed up Query
>>> >>>>
>>> >>>> Hi,
>>> >>>>
>>> >>>> Which of the two query options is better?
>>> >>>>
>>> >>>> SELECT a.item_id, a.create_dt
>>> >>>> FROM a JOIN b
>>> >>>> ON (a.item_id = b.item_id)
>>> >>>> WHERE a.item_id = 'I001'
>>> >>>> AND a.category_name = 'C001';
>>> >>>>
>>> >>>> - or -
>>> >>>>
>>> >>>> SELECT a.item_id, a.create_dt
>>> >>>> FROM a JOIN b
>>> >>>> ON (a.item_id = b.item_id AND a.item_id = 'I001')
>>> >>>> WHERE a.category_name = 'C001';
>>> >>>>
>>> >>>> Thanks
>>> >>>> Naga
>>> >>>>
>>> >>>> CONFIDENTIALITY NOTICE
>>> >>>> ======================
>>> >>>> This email message and any attachments are for the exclusive use of
>>> the
>>> >>>> intended recipient(s) and may contain confidential and privileged
>>> >>>> information. Any unauthorized review, use, disclosure or
>>> distribution is
>>> >>>> prohibited. If you are not the intended recipient, please contact
>>> the sender
>>> >>>> by reply email and destroy all copies of the original message along
>>> with any
>>> >>>> attachments, from your computer system. If you are the intended
>>> recipient,
>>> >>>> please be advised that the content of this message is subject to
>>> access,
>>> >>>> review and disclosure by the sender's Email System Administrator.
>>> >>>
>>> >>>
>>> >>
>>> >
>>>
>>
>>
>
Re: Enhancing Query Join to speed up Query
Posted by bharath vissapragada <bh...@gmail.com>.
Hi,
Can you please paste the output of "explain" for both the queries so that
we can know if the predicate is being pushed down in both the cases.
Thanks,
On Fri, Jun 14, 2013 at 10:05 AM, Naga Vijay <nv...@gmail.com> wrote:
> Hi,
>
> Here are the test results for the two query options shown in my initial
> email -
>
> Test Criteria - Both tables have 20 million rows each, 20 partitions, 16
> buckets for item_id
>
> Query Option 1 - Took 25 minutes, 5 seconds
> Query Option 2 - Took 24 minutes, 42 seconds
>
> Thanks for all the responses!
>
> Naga
>
>
> On Thu, Jun 13, 2013 at 6:42 PM, Navis류승우 <na...@nexr.com> wrote:
>
>> You can use "explain" for confirming differences. For inner joins, it
>> would make the same plan.
>>
>> 2013/6/14 Igor Tatarinov <ig...@decide.com>:
>> > I would expect no difference because of predicate pushdown.
>> >
>> > igor
>> > decide.com
>> >
>> >
>> > On Thu, Jun 13, 2013 at 11:31 AM, Naga Vijay <nv...@gmail.com> wrote:
>> >>
>> >> Sure, Will do
>> >>
>> >>
>> >> On Thu, Jun 13, 2013 at 10:42 AM, Stephen Sprague <sp...@gmail.com>
>> >> wrote:
>> >>>
>> >>> Hi naja,
>> >>> test those two versions (or three now) and report back to the group.
>> :)
>> >>> even if some smarty-pants thinks he knows the answer its always good
>> to
>> >>> confirm things are as they should be.
>> >>>
>> >>>
>> >>> On Wed, Jun 12, 2013 at 11:54 PM, Sanjay Subramanian
>> >>> <Sa...@wizecommerce.com> wrote:
>> >>>>
>> >>>> Hi
>> >>>>
>> >>>> I would actually do it like this…so that the set on the left of JOIN
>> >>>> becomes smaller
>> >>>>
>> >>>> SELECT a.item_id, a.create_dt
>> >>>> FROM
>> >>>> ( SELECT
>> >>>> item_id, create_dt
>> >>>> FROM
>> >>>> A
>> >>>> WHERE
>> >>>> item_id = 'I001'
>> >>>> AND
>> >>>> category_name = 'C001'
>> >>>> ) a
>> >>>> JOIN
>> >>>> b
>> >>>> ON
>> >>>> a.item_id = b.item_id
>> >>>> ;
>> >>>>
>> >>>>
>> >>>> From: Naga Vijay <nv...@gmail.com>
>> >>>> Reply-To: "user@hive.apache.org" <us...@hive.apache.org>
>> >>>> Date: Wednesday, June 12, 2013 9:17 PM
>> >>>> To: "user@hive.apache.org" <us...@hive.apache.org>
>> >>>> Subject: Enhancing Query Join to speed up Query
>> >>>>
>> >>>> Hi,
>> >>>>
>> >>>> Which of the two query options is better?
>> >>>>
>> >>>> SELECT a.item_id, a.create_dt
>> >>>> FROM a JOIN b
>> >>>> ON (a.item_id = b.item_id)
>> >>>> WHERE a.item_id = 'I001'
>> >>>> AND a.category_name = 'C001';
>> >>>>
>> >>>> - or -
>> >>>>
>> >>>> SELECT a.item_id, a.create_dt
>> >>>> FROM a JOIN b
>> >>>> ON (a.item_id = b.item_id AND a.item_id = 'I001')
>> >>>> WHERE a.category_name = 'C001';
>> >>>>
>> >>>> Thanks
>> >>>> Naga
>> >>>>
>> >>>> CONFIDENTIALITY NOTICE
>> >>>> ======================
>> >>>> This email message and any attachments are for the exclusive use of
>> the
>> >>>> intended recipient(s) and may contain confidential and privileged
>> >>>> information. Any unauthorized review, use, disclosure or
>> distribution is
>> >>>> prohibited. If you are not the intended recipient, please contact
>> the sender
>> >>>> by reply email and destroy all copies of the original message along
>> with any
>> >>>> attachments, from your computer system. If you are the intended
>> recipient,
>> >>>> please be advised that the content of this message is subject to
>> access,
>> >>>> review and disclosure by the sender's Email System Administrator.
>> >>>
>> >>>
>> >>
>> >
>>
>
>
Re: Enhancing Query Join to speed up Query
Posted by Naga Vijay <nv...@gmail.com>.
Hi,
Here are the test results for the two query options shown in my initial
email -
Test Criteria - Both tables have 20 million rows each, 20 partitions, 16
buckets for item_id
Query Option 1 - Took 25 minutes, 5 seconds
Query Option 2 - Took 24 minutes, 42 seconds
Thanks for all the responses!
Naga
On Thu, Jun 13, 2013 at 6:42 PM, Navis류승우 <na...@nexr.com> wrote:
> You can use "explain" for confirming differences. For inner joins, it
> would make the same plan.
>
> 2013/6/14 Igor Tatarinov <ig...@decide.com>:
> > I would expect no difference because of predicate pushdown.
> >
> > igor
> > decide.com
> >
> >
> > On Thu, Jun 13, 2013 at 11:31 AM, Naga Vijay <nv...@gmail.com> wrote:
> >>
> >> Sure, Will do
> >>
> >>
> >> On Thu, Jun 13, 2013 at 10:42 AM, Stephen Sprague <sp...@gmail.com>
> >> wrote:
> >>>
> >>> Hi naja,
> >>> test those two versions (or three now) and report back to the group.
> :)
> >>> even if some smarty-pants thinks he knows the answer its always good to
> >>> confirm things are as they should be.
> >>>
> >>>
> >>> On Wed, Jun 12, 2013 at 11:54 PM, Sanjay Subramanian
> >>> <Sa...@wizecommerce.com> wrote:
> >>>>
> >>>> Hi
> >>>>
> >>>> I would actually do it like this…so that the set on the left of JOIN
> >>>> becomes smaller
> >>>>
> >>>> SELECT a.item_id, a.create_dt
> >>>> FROM
> >>>> ( SELECT
> >>>> item_id, create_dt
> >>>> FROM
> >>>> A
> >>>> WHERE
> >>>> item_id = 'I001'
> >>>> AND
> >>>> category_name = 'C001'
> >>>> ) a
> >>>> JOIN
> >>>> b
> >>>> ON
> >>>> a.item_id = b.item_id
> >>>> ;
> >>>>
> >>>>
> >>>> From: Naga Vijay <nv...@gmail.com>
> >>>> Reply-To: "user@hive.apache.org" <us...@hive.apache.org>
> >>>> Date: Wednesday, June 12, 2013 9:17 PM
> >>>> To: "user@hive.apache.org" <us...@hive.apache.org>
> >>>> Subject: Enhancing Query Join to speed up Query
> >>>>
> >>>> Hi,
> >>>>
> >>>> Which of the two query options is better?
> >>>>
> >>>> SELECT a.item_id, a.create_dt
> >>>> FROM a JOIN b
> >>>> ON (a.item_id = b.item_id)
> >>>> WHERE a.item_id = 'I001'
> >>>> AND a.category_name = 'C001';
> >>>>
> >>>> - or -
> >>>>
> >>>> SELECT a.item_id, a.create_dt
> >>>> FROM a JOIN b
> >>>> ON (a.item_id = b.item_id AND a.item_id = 'I001')
> >>>> WHERE a.category_name = 'C001';
> >>>>
> >>>> Thanks
> >>>> Naga
> >>>>
> >>>> CONFIDENTIALITY NOTICE
> >>>> ======================
> >>>> This email message and any attachments are for the exclusive use of
> the
> >>>> intended recipient(s) and may contain confidential and privileged
> >>>> information. Any unauthorized review, use, disclosure or distribution
> is
> >>>> prohibited. If you are not the intended recipient, please contact the
> sender
> >>>> by reply email and destroy all copies of the original message along
> with any
> >>>> attachments, from your computer system. If you are the intended
> recipient,
> >>>> please be advised that the content of this message is subject to
> access,
> >>>> review and disclosure by the sender's Email System Administrator.
> >>>
> >>>
> >>
> >
>
Re: Enhancing Query Join to speed up Query
Posted by Navis류승우 <na...@nexr.com>.
You can use "explain" for confirming differences. For inner joins, it
would make the same plan.
2013/6/14 Igor Tatarinov <ig...@decide.com>:
> I would expect no difference because of predicate pushdown.
>
> igor
> decide.com
>
>
> On Thu, Jun 13, 2013 at 11:31 AM, Naga Vijay <nv...@gmail.com> wrote:
>>
>> Sure, Will do
>>
>>
>> On Thu, Jun 13, 2013 at 10:42 AM, Stephen Sprague <sp...@gmail.com>
>> wrote:
>>>
>>> Hi naja,
>>> test those two versions (or three now) and report back to the group. :)
>>> even if some smarty-pants thinks he knows the answer its always good to
>>> confirm things are as they should be.
>>>
>>>
>>> On Wed, Jun 12, 2013 at 11:54 PM, Sanjay Subramanian
>>> <Sa...@wizecommerce.com> wrote:
>>>>
>>>> Hi
>>>>
>>>> I would actually do it like this…so that the set on the left of JOIN
>>>> becomes smaller
>>>>
>>>> SELECT a.item_id, a.create_dt
>>>> FROM
>>>> ( SELECT
>>>> item_id, create_dt
>>>> FROM
>>>> A
>>>> WHERE
>>>> item_id = 'I001'
>>>> AND
>>>> category_name = 'C001'
>>>> ) a
>>>> JOIN
>>>> b
>>>> ON
>>>> a.item_id = b.item_id
>>>> ;
>>>>
>>>>
>>>> From: Naga Vijay <nv...@gmail.com>
>>>> Reply-To: "user@hive.apache.org" <us...@hive.apache.org>
>>>> Date: Wednesday, June 12, 2013 9:17 PM
>>>> To: "user@hive.apache.org" <us...@hive.apache.org>
>>>> Subject: Enhancing Query Join to speed up Query
>>>>
>>>> Hi,
>>>>
>>>> Which of the two query options is better?
>>>>
>>>> SELECT a.item_id, a.create_dt
>>>> FROM a JOIN b
>>>> ON (a.item_id = b.item_id)
>>>> WHERE a.item_id = 'I001'
>>>> AND a.category_name = 'C001';
>>>>
>>>> - or -
>>>>
>>>> SELECT a.item_id, a.create_dt
>>>> FROM a JOIN b
>>>> ON (a.item_id = b.item_id AND a.item_id = 'I001')
>>>> WHERE a.category_name = 'C001';
>>>>
>>>> Thanks
>>>> Naga
>>>>
>>>> CONFIDENTIALITY NOTICE
>>>> ======================
>>>> This email message and any attachments are for the exclusive use of the
>>>> intended recipient(s) and may contain confidential and privileged
>>>> information. Any unauthorized review, use, disclosure or distribution is
>>>> prohibited. If you are not the intended recipient, please contact the sender
>>>> by reply email and destroy all copies of the original message along with any
>>>> attachments, from your computer system. If you are the intended recipient,
>>>> please be advised that the content of this message is subject to access,
>>>> review and disclosure by the sender's Email System Administrator.
>>>
>>>
>>
>
Re: Enhancing Query Join to speed up Query
Posted by Igor Tatarinov <ig...@decide.com>.
I would expect no difference because of predicate pushdown.
igor
decide.com
On Thu, Jun 13, 2013 at 11:31 AM, Naga Vijay <nv...@gmail.com> wrote:
> Sure, Will do
>
>
> On Thu, Jun 13, 2013 at 10:42 AM, Stephen Sprague <sp...@gmail.com>wrote:
>
>> Hi naja,
>> test those two versions (or three now) and report back to the group. :)
>> even if some smarty-pants thinks he knows the answer its always good to
>> confirm things are as they should be.
>>
>>
>> On Wed, Jun 12, 2013 at 11:54 PM, Sanjay Subramanian <
>> Sanjay.Subramanian@wizecommerce.com> wrote:
>>
>>> Hi
>>>
>>> I would actually do it like this…so that the set on the left of JOIN
>>> becomes smaller
>>>
>>> SELECT a.item_id, a.create_dt
>>> FROM
>>> ( SELECT
>>> item_id, create_dt
>>> FROM
>>> A
>>> WHERE
>>> item_id = 'I001'
>>> AND
>>> category_name = 'C001'
>>> ) a
>>> JOIN
>>> b
>>> ON
>>> a.item_id = b.item_id
>>> ;
>>>
>>>
>>> From: Naga Vijay <nv...@gmail.com>
>>> Reply-To: "user@hive.apache.org" <us...@hive.apache.org>
>>> Date: Wednesday, June 12, 2013 9:17 PM
>>> To: "user@hive.apache.org" <us...@hive.apache.org>
>>> Subject: Enhancing Query Join to speed up Query
>>>
>>> Hi,
>>>
>>> Which of the two query options is better?
>>>
>>> SELECT a.item_id, a.create_dt
>>> FROM a JOIN b
>>> ON (a.item_id = b.item_id)
>>> WHERE a.item_id = 'I001'
>>> AND a.category_name = 'C001';
>>>
>>> - or -
>>>
>>> SELECT a.item_id, a.create_dt
>>> FROM a JOIN b
>>> ON (a.item_id = b.item_id AND a.item_id = 'I001')
>>> WHERE a.category_name = 'C001';
>>>
>>> Thanks
>>> Naga
>>>
>>> CONFIDENTIALITY NOTICE
>>> ======================
>>> This email message and any attachments are for the exclusive use of the
>>> intended recipient(s) and may contain confidential and privileged
>>> information. Any unauthorized review, use, disclosure or distribution is
>>> prohibited. If you are not the intended recipient, please contact the
>>> sender by reply email and destroy all copies of the original message along
>>> with any attachments, from your computer system. If you are the intended
>>> recipient, please be advised that the content of this message is subject to
>>> access, review and disclosure by the sender's Email System Administrator.
>>>
>>
>>
>
Re: Enhancing Query Join to speed up Query
Posted by Naga Vijay <nv...@gmail.com>.
Sure, Will do
On Thu, Jun 13, 2013 at 10:42 AM, Stephen Sprague <sp...@gmail.com>wrote:
> Hi naja,
> test those two versions (or three now) and report back to the group. :)
> even if some smarty-pants thinks he knows the answer its always good to
> confirm things are as they should be.
>
>
> On Wed, Jun 12, 2013 at 11:54 PM, Sanjay Subramanian <
> Sanjay.Subramanian@wizecommerce.com> wrote:
>
>> Hi
>>
>> I would actually do it like this…so that the set on the left of JOIN
>> becomes smaller
>>
>> SELECT a.item_id, a.create_dt
>> FROM
>> ( SELECT
>> item_id, create_dt
>> FROM
>> A
>> WHERE
>> item_id = 'I001'
>> AND
>> category_name = 'C001'
>> ) a
>> JOIN
>> b
>> ON
>> a.item_id = b.item_id
>> ;
>>
>>
>> From: Naga Vijay <nv...@gmail.com>
>> Reply-To: "user@hive.apache.org" <us...@hive.apache.org>
>> Date: Wednesday, June 12, 2013 9:17 PM
>> To: "user@hive.apache.org" <us...@hive.apache.org>
>> Subject: Enhancing Query Join to speed up Query
>>
>> Hi,
>>
>> Which of the two query options is better?
>>
>> SELECT a.item_id, a.create_dt
>> FROM a JOIN b
>> ON (a.item_id = b.item_id)
>> WHERE a.item_id = 'I001'
>> AND a.category_name = 'C001';
>>
>> - or -
>>
>> SELECT a.item_id, a.create_dt
>> FROM a JOIN b
>> ON (a.item_id = b.item_id AND a.item_id = 'I001')
>> WHERE a.category_name = 'C001';
>>
>> Thanks
>> Naga
>>
>> CONFIDENTIALITY NOTICE
>> ======================
>> This email message and any attachments are for the exclusive use of the
>> intended recipient(s) and may contain confidential and privileged
>> information. Any unauthorized review, use, disclosure or distribution is
>> prohibited. If you are not the intended recipient, please contact the
>> sender by reply email and destroy all copies of the original message along
>> with any attachments, from your computer system. If you are the intended
>> recipient, please be advised that the content of this message is subject to
>> access, review and disclosure by the sender's Email System Administrator.
>>
>
>
Re: Enhancing Query Join to speed up Query
Posted by Stephen Sprague <sp...@gmail.com>.
Hi naja,
test those two versions (or three now) and report back to the group. :)
even if some smarty-pants thinks he knows the answer its always good to
confirm things are as they should be.
On Wed, Jun 12, 2013 at 11:54 PM, Sanjay Subramanian <
Sanjay.Subramanian@wizecommerce.com> wrote:
> Hi
>
> I would actually do it like this…so that the set on the left of JOIN
> becomes smaller
>
> SELECT a.item_id, a.create_dt
> FROM
> ( SELECT
> item_id, create_dt
> FROM
> A
> WHERE
> item_id = 'I001'
> AND
> category_name = 'C001'
> ) a
> JOIN
> b
> ON
> a.item_id = b.item_id
> ;
>
>
> From: Naga Vijay <nv...@gmail.com>
> Reply-To: "user@hive.apache.org" <us...@hive.apache.org>
> Date: Wednesday, June 12, 2013 9:17 PM
> To: "user@hive.apache.org" <us...@hive.apache.org>
> Subject: Enhancing Query Join to speed up Query
>
> Hi,
>
> Which of the two query options is better?
>
> SELECT a.item_id, a.create_dt
> FROM a JOIN b
> ON (a.item_id = b.item_id)
> WHERE a.item_id = 'I001'
> AND a.category_name = 'C001';
>
> - or -
>
> SELECT a.item_id, a.create_dt
> FROM a JOIN b
> ON (a.item_id = b.item_id AND a.item_id = 'I001')
> WHERE a.category_name = 'C001';
>
> Thanks
> Naga
>
> CONFIDENTIALITY NOTICE
> ======================
> This email message and any attachments are for the exclusive use of the
> intended recipient(s) and may contain confidential and privileged
> information. Any unauthorized review, use, disclosure or distribution is
> prohibited. If you are not the intended recipient, please contact the
> sender by reply email and destroy all copies of the original message along
> with any attachments, from your computer system. If you are the intended
> recipient, please be advised that the content of this message is subject to
> access, review and disclosure by the sender's Email System Administrator.
>
Re: Enhancing Query Join to speed up Query
Posted by Sanjay Subramanian <Sa...@wizecommerce.com>.
Hi
I would actually do it like this…so that the set on the left of JOIN becomes smaller
SELECT a.item_id, a.create_dt
FROM
( SELECT
item_id, create_dt
FROM
A
WHERE
item_id = 'I001'
AND
category_name = 'C001'
) a
JOIN
b
ON
a.item_id = b.item_id
;
From: Naga Vijay <nv...@gmail.com>>
Reply-To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Date: Wednesday, June 12, 2013 9:17 PM
To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Subject: Enhancing Query Join to speed up Query
Hi,
Which of the two query options is better?
SELECT a.item_id, a.create_dt
FROM a JOIN b
ON (a.item_id = b.item_id)
WHERE a.item_id = 'I001'
AND a.category_name = 'C001';
- or -
SELECT a.item_id, a.create_dt
FROM a JOIN b
ON (a.item_id = b.item_id AND a.item_id = 'I001')
WHERE a.category_name = 'C001';
Thanks
Naga
CONFIDENTIALITY NOTICE
======================
This email message and any attachments are for the exclusive use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message along with any attachments, from your computer system. If you are the intended recipient, please be advised that the content of this message is subject to access, review and disclosure by the sender's Email System Administrator.