You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Miljan Markovic <mi...@yoterra.com> on 2014/12/01 14:26:02 UTC
How to make a table skewed on map value
Hi,
I am trying to make an avro backed table that will hold a map in one of
it's columns. One of map's keys has a highly skewed values so I want to
declare that in table definition. However when I do it like so:
CREATE TABLE my_table
PARTITIONED BY (rec_date date)
SKEWED BY (myDataMap['skewedMapKey']) ON ('value1','value2','value2',)
STORED AS DIRECTORIES
ROW FORMAT SERDE
'org.apache.hadoop.hive.serde2.avro.AvroSerDe'
STORED AS INPUTFORMAT
'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'
TBLPROPERTIES (
'avro.schema.literal'='{
"type" : "record",
"name" : "Record",
"namespace" : "namespace",
"fields" : [
{ "name" : "field1",
"type" : "string" },
,
{
"name" : "MyDataMap,
"type" : {
"type" : "map",
"values" : "string"
}
}
]
}')
I get the following error:
FAILED: ParseException line 3:22 mismatched input '[' expecting ) near
'myDataMap' in table skewed specification
Is this supported in hive? If yes, how should it be done? Otherwise, are
there any workarounds?
I am using hive 0.12.
Thank you!