[jira] [Resolved] (MAPREDUCE-6016) hadoop yarn mapreduce skip failed records doesn't work

["Jakub Stransky (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/MAPREDUCE-6016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jakub Stransky resolved MAPREDUCE-6016.
---------------------------------------

    Resolution: Duplicate

Not sure what happen but issue report were duplicated MAPREDUCE-6017

> hadoop yarn mapreduce skip failed records doesn't work
> ------------------------------------------------------
>
>                 Key: MAPREDUCE-6016
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6016
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 2.2.0
>            Reporter: Jakub Stransky
>            Priority: Minor
>
> I am trying to use "skip failed records" map-reduce functionality during the map phase. I created special testing file with 8 corrupted records. I am using TextInputFormat and during the processing (of the record) map function fails with unhandled exception (parsing the record into expected structure). Job is using the old mapred api.
> My job settings for enabling "skip failed records feature":
>     <property>
>         <name>mapred.skip.mode.enabled</name>
>         <value>true</value>
>     </property>
>     <property>
>         <name>mapreduce.map.maxattempts</name>
>         <value>10</value>
>     </property>
>     <property>
>         <name>mapreduce.task.skip.start.attempts</name>
>         <value>1</value>
>     </property>
>     <property>
>         <name>mapreduce.map.skip.maxrecords</name>
>         <value>1</value>
>     </property>
> I verified that those properties are propagated via verification in job.xml. 
> I am using hadoop 2.2.0 (HDP 2.0). Job is still failing after 10 attempts.
> UPDATE:
> - obviously job is not entering skip record mode
> Q: Does this feature works on RecordReader level only? Hadoop definite guide (which is for v.1) descibes thais feature at the level of map/reduce funciton



--
This message was sent by Atlassian JIRA
(v6.2#6252)