You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Miljan Markovic <mi...@yoterra.com> on 2014/12/01 14:26:02 UTC

How to make a table skewed on map value

Hi,

I am trying to make an avro backed table that will hold a map in one of
it's columns. One of map's keys has a highly skewed values so I want to
declare that in table definition. However when I do it like so:

CREATE TABLE my_table
  PARTITIONED BY (rec_date date)
  SKEWED BY (myDataMap['skewedMapKey']) ON ('value1','value2','value2',)
  STORED AS DIRECTORIES
  ROW FORMAT SERDE
  'org.apache.hadoop.hive.serde2.avro.AvroSerDe'
  STORED AS INPUTFORMAT
  'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat'
  OUTPUTFORMAT
  'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'
TBLPROPERTIES (
    'avro.schema.literal'='{
  "type" : "record",
  "name" : "Record",
  "namespace" : "namespace",
  "fields" : [
{   "name" : "field1",
    "type" : "string" },
,
{
    "name" : "MyDataMap,
    "type" : {
      "type" : "map",
      "values" : "string"
    }
  }
]
}')

I get the following error:

FAILED: ParseException line 3:22 mismatched input '[' expecting ) near
'myDataMap' in table skewed specification

Is this supported in hive? If yes, how should it be done? Otherwise, are
there any workarounds?

I am using hive 0.12.

Thank you!