You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by vince plum <li...@gmail.com> on 2015/12/13 06:09:11 UTC

insert to table as map<> from hive mapreduce job

hi, Hive experts,

CREATE EXTERNAL TABLE IF NOT EXISTS myoutput (
       date STRING,
       id BIGINT,
       *count MAP<BIGINT, BIGINT>*
)
ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t'
COLLECTION ITEMS TERMINATED BY ','
MAP KEYS TERMINATED BY ':'
LINES TERMINATED BY '\n';

FROM (
     FROM sometable
     *MAP* date, complex
     USING 'py myscript.py mapper'
     AS date, sometype
     CLUSTER BY date) map_output
INSERT OVERWRITE TABLE myoutput
     *REDUCE* map_output.date, map_output.sometype
     USING 'py myscript.py reducer'
     AS date, id, str_to_map(*count_map*, ',', ':');


I got the error below.:


FAILED: ParseException line 11:27 cannot recognize input near '('
'count_map' ',' in serde specification


Since REDUCE can't output complex format like map<>, one alternative
solution is to use a tmp to save the result as string, then do another
select str_to_map().

any way to convert the string to map directly?