You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Parul Kudtarkar <Pa...@hms.harvard.edu> on 2009/04/24 21:29:16 UTC
HDFS files naming convention
The HDFS files generated after mapreduce run are strored in HDFS as
part-00000 and so on.....part-0000n
Is it possible to name these output files stored in HDFS as per my own
convention i.e. I would like to name these files my_file_1 and so
on....my_files_n
Please advice how this can be achieved?
Thanks,
Parul V. Kudtarkar
--
View this message in context: http://www.nabble.com/HDFS-files-naming-convention-tp23223348p23223348.html
Sent from the Hadoop core-user mailing list archive at Nabble.com.
Re: HDFS files naming convention
Posted by Pankil Doshi <fo...@gmail.com>.
hey
you can surely do that using MulipleFileOutputFormat. We have already
implemented that
Pankil
On Fri, Apr 24, 2009 at 8:58 PM, Aaron Kimball <aa...@cloudera.com> wrote:
> Alternatively, just use FileSystem.rename() on the normal output files
> after
> reducing is complete?
>
> On Sat, Apr 25, 2009 at 10:57 AM, Aaron Kimball <aa...@cloudera.com>
> wrote:
>
> > I think you can do this with MultiFileOutputFormat; specifically override
> > the getLeafFileName() method.
> > - Aaron
> >
> >
> > On Sat, Apr 25, 2009 at 4:29 AM, Parul Kudtarkar <
> > Parul_Kudtarkar@hms.harvard.edu> wrote:
> >
> >>
> >> The HDFS files generated after mapreduce run are strored in HDFS as
> >> part-00000 and so on.....part-0000n
> >>
> >> Is it possible to name these output files stored in HDFS as per my own
> >> convention i.e. I would like to name these files my_file_1 and so
> >> on....my_files_n
> >>
> >> Please advice how this can be achieved?
> >>
> >> Thanks,
> >> Parul V. Kudtarkar
> >> --
> >> View this message in context:
> >>
> http://www.nabble.com/HDFS-files-naming-convention-tp23223348p23223348.html
> >> Sent from the Hadoop core-user mailing list archive at Nabble.com.
> >>
> >>
> >
>
Re: HDFS files naming convention
Posted by Aaron Kimball <aa...@cloudera.com>.
Alternatively, just use FileSystem.rename() on the normal output files after
reducing is complete?
On Sat, Apr 25, 2009 at 10:57 AM, Aaron Kimball <aa...@cloudera.com> wrote:
> I think you can do this with MultiFileOutputFormat; specifically override
> the getLeafFileName() method.
> - Aaron
>
>
> On Sat, Apr 25, 2009 at 4:29 AM, Parul Kudtarkar <
> Parul_Kudtarkar@hms.harvard.edu> wrote:
>
>>
>> The HDFS files generated after mapreduce run are strored in HDFS as
>> part-00000 and so on.....part-0000n
>>
>> Is it possible to name these output files stored in HDFS as per my own
>> convention i.e. I would like to name these files my_file_1 and so
>> on....my_files_n
>>
>> Please advice how this can be achieved?
>>
>> Thanks,
>> Parul V. Kudtarkar
>> --
>> View this message in context:
>> http://www.nabble.com/HDFS-files-naming-convention-tp23223348p23223348.html
>> Sent from the Hadoop core-user mailing list archive at Nabble.com.
>>
>>
>
Re: HDFS files naming convention
Posted by Aaron Kimball <aa...@cloudera.com>.
I think you can do this with MultiFileOutputFormat; specifically override
the getLeafFileName() method.
- Aaron
On Sat, Apr 25, 2009 at 4:29 AM, Parul Kudtarkar <
Parul_Kudtarkar@hms.harvard.edu> wrote:
>
> The HDFS files generated after mapreduce run are strored in HDFS as
> part-00000 and so on.....part-0000n
>
> Is it possible to name these output files stored in HDFS as per my own
> convention i.e. I would like to name these files my_file_1 and so
> on....my_files_n
>
> Please advice how this can be achieved?
>
> Thanks,
> Parul V. Kudtarkar
> --
> View this message in context:
> http://www.nabble.com/HDFS-files-naming-convention-tp23223348p23223348.html
> Sent from the Hadoop core-user mailing list archive at Nabble.com.
>
>