You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by vince plum <li...@gmail.com> on 2015/12/13 06:09:11 UTC
insert to table as map<> from hive mapreduce job
hi, Hive experts,
CREATE EXTERNAL TABLE IF NOT EXISTS myoutput (
date STRING,
id BIGINT,
*count MAP<BIGINT, BIGINT>*
)
ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t'
COLLECTION ITEMS TERMINATED BY ','
MAP KEYS TERMINATED BY ':'
LINES TERMINATED BY '\n';
FROM (
FROM sometable
*MAP* date, complex
USING 'py myscript.py mapper'
AS date, sometype
CLUSTER BY date) map_output
INSERT OVERWRITE TABLE myoutput
*REDUCE* map_output.date, map_output.sometype
USING 'py myscript.py reducer'
AS date, id, str_to_map(*count_map*, ',', ':');
I got the error below.:
FAILED: ParseException line 11:27 cannot recognize input near '('
'count_map' ',' in serde specification
Since REDUCE can't output complex format like map<>, one alternative
solution is to use a tmp to save the result as string, then do another
select str_to_map().
any way to convert the string to map directly?