You are viewing a plain text version of this content. The canonical link for it is here.

Posted to mapreduce-dev@hadoop.apache.org by "Clément MAHTIEU (JIRA)" <ji...@apache.org> on 2014/08/13 15:37:11 UTC

[jira] [Created] (MAPREDUCE-6034) JobControl: polling interval should not be hard coded

Clément MAHTIEU created MAPREDUCE-6034:
------------------------------------------

             Summary: JobControl: polling interval should not be hard coded
                 Key: MAPREDUCE-6034
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6034
             Project: Hadoop Map/Reduce
          Issue Type: Improvement
          Components: client
            Reporter: Clément MAHTIEU


The JobControl mechanism relies on a polling mechanism which is implemented in JobControl.run().

The polling period is currently hard coded to 5s in the run() method making it almost impossible to tune. However, I believe that it will be good to let the users set this value. Some applications could want it in production and it would speed up the local mode and integration tests a lot. In my current project, nearly half of the time of the test suite is spend in this single line. Pig had the same issue, and decided to "crazily" hack the JobControl class rather than fixing it in the upstream (see PIG-2702).

AFAIK it could be easily done by providing a new ctor or a new setter. This change would not break the compatibility. I can provide a patch if this change seems reasonable. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)