[jira] Updated: (HADOOP-3829) Narrown down skipped records based on user acceptable value

["Sharad Agarwal (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HADOOP-3829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sharad Agarwal updated HADOOP-3829:
-----------------------------------

    Attachment: 3829_v1.153_7.patch

This patch depends on the patch from HADOOP-153. Please apply 153_7.patch before applying this one.

The approach has been discussed earlier in HADOOP-153 as well. Here is the brief:

Defines user configurable MAPPER_MAX_SKIP_RECORDS/MAPPER_REDUCE_SKIP_RECORDS -> acceptable skipped records in the neighborhood of a bad record. 
If skipped range is greater than this threshold, the task will try to narrow down the skipped range using a binary search kind of algorithm during task re-executions till this threshold is met or all task attempts are exhausted. The skipped range is divided into two halves and only one half get executed. Based on the subsequent failure, it figures out which half contains the bad record.


> Narrown down skipped records based on user acceptable value
> -----------------------------------------------------------
>
>                 Key: HADOOP-3829
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3829
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: mapred
>            Reporter: Sharad Agarwal
>            Assignee: Sharad Agarwal
>         Attachments: 3829_v1.153_7.patch
>
>
> This is an incremental step over HADOOP-153.
> If the number of skipped records in the neighborhood of a bad record are not acceptable to the user, then narrow down the skipped range to the user acceptable value.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.