You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by sam liu <sa...@gmail.com> on 2013/12/23 14:37:23 UTC

Any method to get input splits by column?

Hi,

By default, MR inputformat classes break input file into splits by rows.
However, we have a specilal requirement on MR app: get input splits by
column.

Is there any good method?

Thanks!

RE: Any method to get input splits by column?

Posted by java8964 <ja...@hotmail.com>.
You need to store your data into "column-based" format, checking out Hive RCFile, and its InputFormat option.
Yong

Date: Mon, 23 Dec 2013 21:37:23 +0800
Subject: Any method to get input splits by column?
From: samliuhadoop@gmail.com
To: user@hadoop.apache.org

Hi,

By default, MR inputformat classes break input file into splits by rows. However, we have a specilal requirement on MR app: get input splits by column.

Is there any good method?


Thanks!

RE: Any method to get input splits by column?

Posted by java8964 <ja...@hotmail.com>.
You need to store your data into "column-based" format, checking out Hive RCFile, and its InputFormat option.
Yong

Date: Mon, 23 Dec 2013 21:37:23 +0800
Subject: Any method to get input splits by column?
From: samliuhadoop@gmail.com
To: user@hadoop.apache.org

Hi,

By default, MR inputformat classes break input file into splits by rows. However, we have a specilal requirement on MR app: get input splits by column.

Is there any good method?


Thanks!

RE: Any method to get input splits by column?

Posted by java8964 <ja...@hotmail.com>.
You need to store your data into "column-based" format, checking out Hive RCFile, and its InputFormat option.
Yong

Date: Mon, 23 Dec 2013 21:37:23 +0800
Subject: Any method to get input splits by column?
From: samliuhadoop@gmail.com
To: user@hadoop.apache.org

Hi,

By default, MR inputformat classes break input file into splits by rows. However, we have a specilal requirement on MR app: get input splits by column.

Is there any good method?


Thanks!

RE: Any method to get input splits by column?

Posted by java8964 <ja...@hotmail.com>.
You need to store your data into "column-based" format, checking out Hive RCFile, and its InputFormat option.
Yong

Date: Mon, 23 Dec 2013 21:37:23 +0800
Subject: Any method to get input splits by column?
From: samliuhadoop@gmail.com
To: user@hadoop.apache.org

Hi,

By default, MR inputformat classes break input file into splits by rows. However, we have a specilal requirement on MR app: get input splits by column.

Is there any good method?


Thanks!