You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by novice user <pa...@gmail.com> on 2008/06/05 11:17:07 UTC

Possibility to specify some type of files in a directory as input

Hi,
  I need a help in setting my map-reduce job to  consider only certain type
of files as input in a specific directory.
For example, Suppose there is a directory dir1 and I have files like 
type1_1.txt 
type1_2.txt
type1_3.txt
type2_1.txt
type2_2.txt
and If I want to consider only those files whose name starting with type1 as
input to my mapper. Then, can some one please let me know how to specify
this while configuring job? 

Thanks


-- 
View this message in context: http://www.nabble.com/Possibility-to-specify-some-type-of-files-in-a-directory-as-input-tp17665598p17665598.html
Sent from the Hadoop core-user mailing list archive at Nabble.com.

Re: 答复: Possibility to specify some type of files in a directory as input

Posted by Andreas Kostyrka <an...@kostyrka.org>.
On Thursday 05 June 2008 11:38:41 志远 wrote:
> Put the input path like : dir1/type1*.txt
>
>
> Hi,
>   I need a help in setting my map-reduce job to  consider only certain type
> of files as input in a specific directory.
> For example, Suppose there is a directory dir1 and I have files like
> type1_1.txt
> type1_2.txt
> type1_3.txt
> type2_1.txt
> type2_2.txt
> and If I want to consider only those files whose name starting with type1
> as input to my mapper. Then, can some one please let me know how to specify
> this while configuring job?

Well, setInputPaths takes by default a comma seperated list of paths.

Andreas

>
> Thanks

答复: Possibility to specify some type of files in a directory as input

Posted by 志远 <zh...@taobao.com>.
Put the input path like : dir1/type1*.txt


Hi,
  I need a help in setting my map-reduce job to  consider only certain type
of files as input in a specific directory.
For example, Suppose there is a directory dir1 and I have files like 
type1_1.txt 
type1_2.txt
type1_3.txt
type2_1.txt
type2_2.txt
and If I want to consider only those files whose name starting with type1 as
input to my mapper. Then, can some one please let me know how to specify
this while configuring job? 

Thanks


-- 
View this message in context:
http://www.nabble.com/Possibility-to-specify-some-type-of-files-in-a-directo
ry-as-input-tp17665598p17665598.html
Sent from the Hadoop core-user mailing list archive at Nabble.com.