[jira] [Created] (MAPREDUCE-6016) hadoop yarn mapreduce skip failed records doesn't work

["Jakub Stransky (JIRA)" <ji...@apache.org>]

Jakub Stransky created MAPREDUCE-6016:
-----------------------------------------

             Summary: hadoop yarn mapreduce skip failed records doesn't work
                 Key: MAPREDUCE-6016
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6016
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: mrv2
    Affects Versions: 2.2.0
            Reporter: Jakub Stransky
            Priority: Minor


I am trying to use "skip failed records" map-reduce functionality during the map phase. I created special testing file with 8 corrupted records. I am using TextInputFormat and during the processing (of the record) map function fails with unhandled exception (parsing the record into expected structure). Job is using the old mapred api.

My job settings for enabling "skip failed records feature":

    <property>
        <name>mapred.skip.mode.enabled</name>
        <value>true</value>
    </property>
    <property>
        <name>mapreduce.map.maxattempts</name>
        <value>10</value>
    </property>
    <property>
        <name>mapreduce.task.skip.start.attempts</name>
        <value>1</value>
    </property>
    <property>
        <name>mapreduce.map.skip.maxrecords</name>
        <value>1</value>
    </property>

I verified that those properties are propagated via verification in job.xml. 
I am using hadoop 2.2.0 (HDP 2.0). Job is still failing after 10 attempts.

UPDATE:
- obviously job is not entering skip record mode

Q: Does this feature works on RecordReader level only? Hadoop definite guide (which is for v.1) descibes thais feature at the level of map/reduce funciton



--
This message was sent by Atlassian JIRA
(v6.2#6252)