You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Steve Lewis <lo...@gmail.com> on 2010/06/11 18:14:05 UTC
Setting a custom splitter and/or tracking files split?
I am running into an issue that the splitter is reading the wrong file and
causing my program to fail -
I cannot find which file is being read -
context.getConfiguration().get("map.input.file"); returns null and
while I have seem reference to a method getInputSplit I cannot see anything
implementing it
More generally it would be nice to be able tp track and filter files as they
are passed to the splitter -
Is there any way to set a custom splitter and if so how
--
Steven M. Lewis PhD
Institute for Systems Biology
Seattle WA
Re: Setting a custom splitter and/or tracking files split?
Posted by Ted Yu <yu...@gmail.com>.
Try this:
FileSplit fileSplit = (FileSplit) context.getInputSplit();
String sFileName = fileSplit.getPath().getName();
On Fri, Jun 11, 2010 at 9:14 AM, Steve Lewis <lo...@gmail.com> wrote:
> I am running into an issue that the splitter is reading the wrong file and
> causing my program to fail -
> I cannot find which file is being read -
> context.getConfiguration().get("map.input.file"); returns null and
> while I have seem reference to a method getInputSplit I cannot see anything
> implementing it
>
> More generally it would be nice to be able tp track and filter files as
> they are passed to the splitter -
> Is there any way to set a custom splitter and if so how
>
> --
> Steven M. Lewis PhD
> Institute for Systems Biology
> Seattle WA
>