You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by KayVajj <va...@gmail.com> on 2013/09/27 00:28:23 UTC
What is the use of compressed field in the hive storage descriptor
A desc extended command results in the following
Table(
tableName:table_namet,
dbName:cybs_test,
owner:xyz@abc.com,
createTime:1380232668,
lastAccessTime:0,
retention:0,
sd:StorageDescriptor(
cols:[
FieldSchema(name:a, type:string, comment:null),
FieldSchema(name:b, type:string, comment:null),
FieldSchema(name:c, type:string, comment:null),
FieldSchema(name:d, type:string, comment:null),
FieldSchema(name:e, type:string, comment:null),
FieldSchema(name:f, type:timestamp, comment:null),
FieldSchema(name:date_added, type:timestamp, comment:null),
FieldSchema(name:g, type:string, comment:null),
FieldSchema(name:h, type:string, comment:null),
FieldSchema(name:i, type:string, comment:null),
FieldSchema(name:j, type:string, comment:null)
],
location:xyz
inputFormat:org.apache.hadoop.mapred.SequenceFileInputFormat,
outputFormat:org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat,
*compressed:false, *
numBuckets:-1,
serdeInfo:SerDeInfo(
name:null,
serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe,
parameters:{colelction.delim=, mapkey.delim=, serialization.format=,
field.delim=}
),
bucketCols:[],
sortCols:[],
parameters:{},
skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[],
skewedColValueLocationMaps:{}),
storedAsSubDirectories:false
),
partitionKeys:[FieldSchema(name:dt, type:int, comment:null)],
parameters:{numPartitions=1, numFiles=5, EXTERNAL=TRUE,
transient_lastDdlTime=1380233065, totalSize=340426, numRows=0,
comment=Request Table from ODS, rawDataSize=0},
viewOriginalText:null,
viewExpandedText:null,
tableType:EXTERNAL_TABLE
)
What is the meaning of the highlighted field?
Thanks
Re: What is the use of compressed field in the hive storage descriptor
Posted by Edward Capriolo <ed...@gmail.com>.
Some table types like rc file can have internal compression. It has
different meaning based on the serde used.
On Thursday, September 26, 2013, KayVajj <va...@gmail.com> wrote:
> A desc extended command results in the following
> Table(
> tableName:table_namet,
> dbName:cybs_test,
> owner:xyz@abc.com,
> createTime:1380232668,
> lastAccessTime:0,
> retention:0,
> sd:StorageDescriptor(
> cols:[
> FieldSchema(name:a, type:string, comment:null),
> FieldSchema(name:b, type:string, comment:null),
> FieldSchema(name:c, type:string, comment:null),
> FieldSchema(name:d, type:string, comment:null),
> FieldSchema(name:e, type:string, comment:null),
> FieldSchema(name:f, type:timestamp, comment:null),
> FieldSchema(name:date_added, type:timestamp, comment:null),
> FieldSchema(name:g, type:string, comment:null),
> FieldSchema(name:h, type:string, comment:null),
> FieldSchema(name:i, type:string, comment:null),
> FieldSchema(name:j, type:string, comment:null)
> ],
> location:xyz
> inputFormat:org.apache.hadoop.mapred.SequenceFileInputFormat,
> outputFormat:org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat,
> compressed:false,
> numBuckets:-1,
> serdeInfo:SerDeInfo(
> name:null,
> serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe,
> parameters:{colelction.delim=, mapkey.delim=, serialization.format=,
field.delim=}
> ),
> bucketCols:[],
> sortCols:[],
> parameters:{},
> skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[],
skewedColValueLocationMaps:{}),
> storedAsSubDirectories:false
> ),
> partitionKeys:[FieldSchema(name:dt, type:int, comment:null)],
> parameters:{numPartitions=1, numFiles=5, EXTERNAL=TRUE,
transient_lastDdlTime=1380233065, totalSize=340426, numRows=0,
comment=Request Table from ODS, rawDataSize=0},
> viewOriginalText:null,
> viewExpandedText:null,
> tableType:EXTERNAL_TABLE
> )
> What is the meaning of the highlighted field?
> Thanks
>
>
>