[jira] [Commented] (TEZ-4106) Add Exponential Smooth RuntimeEstimator to the speculator

["Jonathan Turner Eagles (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17027021#comment-17027021 ] 

Jonathan Turner Eagles commented on TEZ-4106:
---------------------------------------------

Thanks, [~ahussein].

Let's clean up the TezConfiguration names if possible. Does it make sense to put them under a top level tez.am.speculation name? Right now there is speculator, speculative, speculation, so it may not be possible to be match perfectly with old configurations. 

Also, TEZ-4119 has been filed to address the flaky tests in TestSpeculation. Do we need to change the patch to account for this?

Otherwise the patch looks very close to put in. 

> Add Exponential Smooth RuntimeEstimator to the speculator
> ---------------------------------------------------------
>
>                 Key: TEZ-4106
>                 URL: https://issues.apache.org/jira/browse/TEZ-4106
>             Project: Apache Tez
>          Issue Type: Improvement
>            Reporter: Ahmed Hussein
>            Assignee: Ahmed Hussein
>            Priority: Major
>         Attachments: TEZ-4106.001.patch, TEZ-4106.002.patch, TEZ-4106.003.patch, TEZ-4106.004.patch
>
>
> Tez speculator implements start-end runtime estimator. Similar to [MAPREDUCE-7208|https://issues.apache.org/jira/browse/MAPREDUCE-7208], we need to implement an adaptive estimator based on smooth Exponential



--
This message was sent by Atlassian Jira
(v8.3.4#803005)