You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-user@hadoop.apache.org by Kasi Subrahmanyam <ka...@gmail.com> on 2015/10/27 22:31:37 UTC

Using NlineInputFormat in HIve

Hi,
I have a requirement where i have to send one line to one mapper in a file
but doing it using hive.
How can we implement the functionality of NLineInptFormat in Hive.
I couldnt find this and i tried the following configuration in hive

set hive.merge.mapfiles=false;
set hive.input.format=org.apache.hadoop.hive.ql.io.HiveInputFormat;

set mapreduce.input.fileinputformat.split.maxsize=100;

I have a file with onlt 10000 rows but i need each row to go different mapper

But if i used the above configuration in hive some of the rows are missing

based on the max split size i provide


Thanks,