You are viewing a plain text version of this content. The canonical link for it is here.

Posted to hdfs-user@hadoop.apache.org by Yuncong Chen <cy...@gmail.com> on 2013/01/25 03:08:10 UTC

Any streaming options that specify data type of key value pairs?

Hi, 

What would be the best way in hadoop streaming to send a binary object (i.e. a python dict, array) as the value in <key,value> pairs?

I know I can dump the object to string with pickle.dumps() and encode it to eliminate unintended '\t','\n' before sending it to stdout, but I wonder if there are native streaming options that specifies the data type of <key,value> pairs?

Thanks,
Yuncong