You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Jason Lowe (JIRA)" <ji...@apache.org> on 2013/12/05 18:42:37 UTC

[jira] [Commented] (MAPREDUCE-5670) CombineFileRecordReader should report progress when moving to the next file

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13840323#comment-13840323 ] 

Jason Lowe commented on MAPREDUCE-5670:
---------------------------------------

CombineFileRecordReader.initNextRecordReader should report progress as it moves to the next file to indicate that the task is making progress even though no records are being read.  Some input formats already report progress when their record reader is created, but adding a progress indicator here will catch the cases where the underlying record reader does not report progress upon construction.

> CombineFileRecordReader should report progress when moving to the next file
> ---------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-5670
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5670
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.9, 2.2.0
>            Reporter: Jason Lowe
>            Priority: Minor
>
> If a combine split consists of many "empty" files (i.e.: no record found by the underlying record reader) then theoretically a task can timeout due to lack of reported progress.



--
This message was sent by Atlassian JIRA
(v6.1#6144)