[jira] [Commented] (AIRFLOW-1036) Exponential backoff should use randomization

["ASF subversion and git services (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-1036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15989907#comment-15989907 ] 

ASF subversion and git services commented on AIRFLOW-1036:
----------------------------------------------------------

Commit 66168efa12de98a9c29b20e5cea28c7e34a2d90a in incubator-airflow's branch refs/heads/master from [~saguziel]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=66168ef ]

[AIRFLOW-1036] Randomize exponential backoff

This prevents the thundering herd problem. Using a
combination of
dag_run, task_id, and execution_date makes this
random with respect to
task instances, while still being deterministic
across machines. The
retry delay is within a range that doubles in
size.

Closes #2262 from saguziel/aguziel-random-
exponential-backoff


> Exponential backoff should use randomization
> --------------------------------------------
>
>                 Key: AIRFLOW-1036
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1036
>             Project: Apache Airflow
>          Issue Type: Improvement
>            Reporter: Alex Guziel
>            Assignee: Alex Guziel
>             Fix For: 1.9.0, 1.8.2
>
>
> This prevents the thundering herd problem. I think with the current way this is used, we would need to use some hashing function based on some subset of the dag_run, task_id, dag_id, and execution_date to emulate the RNG.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)