[jira] Updated: (HADOOP-3829) Narrown down skipped records based on user acceptable value

["Sharad Agarwal (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HADOOP-3829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sharad Agarwal updated HADOOP-3829:
-----------------------------------

    Attachment: 3829_v2.patch

fairly tested patch. few additional things:
- moved counters from Counters.java to SkipBadRecords.java as these are specific to skip feature.
- fixed hasNext in SortedRanges.SkipRangeIterator.
- skipped records are not written to HDFS, if it is a test attempt (test attempt is to figure out whether a range is good or bad. In this only records in test range are passed to the mapper/reducer, others are skipped).
- recordreader.next not called beyond the test range during test attempt.
- renamed failedRanges to skipRanges in Task.java
- added config params to hadoop-default

> Narrown down skipped records based on user acceptable value
> -----------------------------------------------------------
>
>                 Key: HADOOP-3829
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3829
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: mapred
>            Reporter: Sharad Agarwal
>            Assignee: Sharad Agarwal
>         Attachments: 3829_v1.153_7.patch, 3829_v1.patch, 3829_v2.patch
>
>
> This is an incremental step over HADOOP-153.
> If the number of skipped records in the neighborhood of a bad record are not acceptable to the user, then narrow down the skipped range to the user acceptable value.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.