You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Steve Lewis <lo...@gmail.com> on 2010/06/11 18:14:05 UTC

Setting a custom splitter and/or tracking files split?

I am running into an issue that the splitter is reading the wrong file and
causing my program to fail -
I cannot find which file is being read -
 context.getConfiguration().get("map.input.file"); returns null and
while I have seem reference to a method getInputSplit I cannot see anything
implementing it

More generally it would be nice to be able tp track and filter files as they
are passed to the splitter -
Is there any way to set a custom splitter and if so how

-- 
Steven M. Lewis PhD
Institute for Systems Biology
Seattle WA

Re: Setting a custom splitter and/or tracking files split?

Posted by Ted Yu <yu...@gmail.com>.
Try this:
FileSplit fileSplit = (FileSplit) context.getInputSplit();
String sFileName = fileSplit.getPath().getName();

On Fri, Jun 11, 2010 at 9:14 AM, Steve Lewis <lo...@gmail.com> wrote:

> I am running into an issue that the splitter is reading the wrong file and
> causing my program to fail -
> I cannot find which file is being read -
>  context.getConfiguration().get("map.input.file"); returns null and
> while I have seem reference to a method getInputSplit I cannot see anything
> implementing it
>
> More generally it would be nice to be able tp track and filter files as
> they are passed to the splitter -
> Is there any way to set a custom splitter and if so how
>
> --
> Steven M. Lewis PhD
> Institute for Systems Biology
> Seattle WA
>