You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@pig.apache.org by Scott Kester <sk...@gmail.com> on 2010/04/22 01:48:04 UTC

Help tracking down corrupted map input file given the task id

I have a pig job that is processing around 61,000 gzip files.  It is failing on 1 file, that appears to be corrupted.  I have looked at every log file I can find, but can't find the name of the bad file.  How can I find the filename given the task id that failed?

Also, is there a parameter setting that would allow the job to allow a small number of failures and run to completion? 

Thanks,
Scott