[jira] [Commented] (TEZ-3317) Speculative execution starts too early due to 0 progress

["Kuhu Shukla (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-3317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15382998#comment-15382998 ] 

Kuhu Shukla commented on TEZ-3317:
----------------------------------

There seems to be a disconnect between Processors and LogicalInput (and its derivatives) where the RecordReader progress is not used by {{RuntimeTask}} . Adding the progress calls to {{Input}}/{{LogicalInput}} class would break anyone who uses that interface, although {{AbstractLogicalInput}} can be augmented. Requesting for some comments from [~bikassaha], [~sseth] on how to propagate Input specific progress to each Processor.

> Speculative execution starts too early due to 0 progress
> --------------------------------------------------------
>
>                 Key: TEZ-3317
>                 URL: https://issues.apache.org/jira/browse/TEZ-3317
>             Project: Apache Tez
>          Issue Type: Improvement
>            Reporter: Jonathan Eagles
>
> Don't know at this point if this is a tez or a PigProcessor issue. There is some setProgress chain that is keeping task progress from being correctly reported. Task status is always zero, so as soon as the first task finishes, tasks up to the speculation limit are always launched.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)