[jira] [Updated] (TEZ-3317) Speculative execution starts too early due to 0 progress

["Kuhu Shukla (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/TEZ-3317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kuhu Shukla updated TEZ-3317:
-----------------------------
    Attachment: TEZ-3317.001.patch

Attaching an initial prototype patch that addresses MRInput and UnorderedKVInput. Added  {{getProgress}} at Input level and formulated the progress for the Unordered case to be : (totalBytesReadSofar/TotalfileBytes) * (numofCompletedInput/numofTotalInputs). Requesting comments/ideas on how to tackle different Readers for calculating and propagating progress to their inputs. Processors with multiple Inputs would need some more work to report correct/weighted progress. 

> Speculative execution starts too early due to 0 progress
> --------------------------------------------------------
>
>                 Key: TEZ-3317
>                 URL: https://issues.apache.org/jira/browse/TEZ-3317
>             Project: Apache Tez
>          Issue Type: Improvement
>            Reporter: Jonathan Eagles
>            Assignee: Kuhu Shukla
>         Attachments: TEZ-3317.001.patch
>
>
> Don't know at this point if this is a tez or a PigProcessor issue. There is some setProgress chain that is keeping task progress from being correctly reported. Task status is always zero, so as soon as the first task finishes, tasks up to the speculation limit are always launched.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)