Fwd: Speculative Execution...

[Suresh S <su...@gmail.com>]

Hello Sir/Madam....

   I am doing PhD. I am interested to do research in Hadoop for publishing
paper.
I know little bit about speculative execution of slow tasks. I know it is
possible to enable or disable speculative execution.

But,
   Is there any idea published already for dynamically enable or disable
the speculative execution depending on application, cluster load and other
run time parameters?

   Is it worth to do research in this direction?

   Is this contribution is worth to publish a conference or journal
paper(s)?

*Regards*
*S.Suresh,*
*Research Scholar,*
*Department of Computer Applications,*
*National Institute of Technology,*
*Tiruchirappalli - 620015.*
*India*
*Mobile: +91-9941506562*

Re: Speculative Execution...

[Robert Evans <ev...@yahoo-inc.com>]

Under YARN (branch-2, branch-0.23, and trunk) the speculative execution
decision is pluggable, and can be replaced by a user.  If you could come
up with a better solution to speculative execution that would be great.
We have known for a while that it is not very good (most of the time we
run a speculative task it is just wasted).  In branch-2 we have a new
version that we think is better (more conservative), but I am not sure how
much of a study has been done on exactly how much better it is or what
else can be done to make it even better.  I would look at adding your
ideas into that plugin and not so much of using the config to turn
speculation on or off dynamically, because there are some map/reduce
applications that abuse map/reduce somewhat and will not run correctly if
speculative execution is enabled.

--Bobby Evans 

On 9/13/12 1:08 AM, "Suresh S" <su...@gmail.com> wrote:

>Hello Sir/Madam....
>
>   I am doing PhD. I am interested to do research in Hadoop for publishing
>paper.
>I know little bit about speculative execution of slow tasks. I know it is
>possible to enable or disable speculative execution.
>
>But,
>   Is there any idea published already for dynamically enable or disable
>the speculative execution depending on application, cluster load and other
>run time parameters?
>
>   Is it worth to do research in this direction?
>
>   Is this contribution is worth to publish a conference or journal
>paper(s)?
>
>*Regards*
>*S.Suresh,*
>*Research Scholar,*
>*Department of Computer Applications,*
>*National Institute of Technology,*
>*Tiruchirappalli - 620015.*
>*India*
>*Mobile: +91-9941506562*