[jira] Commented: (HADOOP-5572) The map progress value should have a separate phase for doing the final sort.

["Jothi Padmanabhan (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-5572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12705909#action_12705909 ] 

Jothi Padmanabhan commented on HADOOP-5572:
-------------------------------------------

Some initial comments:

# Ensure that the sum of weights for a phase does not cross 1
# Having a boolean variable to keep track of whether the weights are fixed or variable is a better option
# Merger -- Sort the segments only if numSegments > factor
# Relying on writesCounter to decide includeFinalMerge variable is not a good idea.
# computeBytesInMerges should disregard empty segments -- we probably need to add a isEmpty() API to Segment.

> The map progress value should have a separate phase for doing the final sort.
> -----------------------------------------------------------------------------
>
>                 Key: HADOOP-5572
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5572
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Owen O'Malley
>            Assignee: Ravi Gummadi
>         Attachments: HADOOP-5572.patch
>
>
> Currently, the final spill and sort doesn't record any progress while it runs, leading to the perception that the map is done, but "stuck".

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.