You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "jack (JIRA)" <ji...@apache.org> on 2019/03/07 07:58:00 UTC
[jira] [Updated] (AIRFLOW-3146) TimeDeltaSensor - add end time
stamp
[ https://issues.apache.org/jira/browse/AIRFLOW-3146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
jack updated AIRFLOW-3146:
--------------------------
Description:
Currently the TimeDeltaSensor takes these arguments:
{color:#4070a0} :param delta: time length to wait after execution_date before succeeding{color}
{color:#4070a0} :type delta: datetime.timedelta{color}{color:#4070a0} """{color}{color:#404040}
{color}{color:#555555}@apply_defaults{color}{color:#404040}
{color}{color:#007020}def{color} {color:#06287e}__init__{color}{color:#404040}({color}{color:#007020}self{color}{color:#404040},{color} {color:#404040}delta{color}{color:#404040},{color} {color:#666666}*{color}{color:#404040}args{color}{color:#404040},{color} {color:#666666}**{color}{color:#404040}kwargs{color}{color:#404040}):{color}{color:#404040}
{color}{color:#007020}super{color}{color:#404040}({color}{color:#404040}TimeDeltaSensor{color}{color:#404040},{color} {color:#007020}self{color}{color:#404040}){color}{color:#666666}.{color}{color:#06287e}__init__{color}{color:#404040}({color}{color:#666666}*{color}{color:#404040}args{color}{color:#404040},{color} {color:#666666}**{color}{color:#404040}kwargs{color}{color:#404040}){color}{color:#404040}
{color}{color:#007020}self{color}{color:#666666}.{color}{color:#404040}delta{color} {color:#666666}={color} {color:#404040}delta{color}
*The problem:*
The operator assumes that the user would like to wait execution_date + delta.
This isn't always the case, The user might want to wait delta after the task has finished running.
Lets take this example:
{color:#4070a0} 2016-01-01 can only start running on 2016-01-02. The timedelta here{color}{color:#4070a0} represents the time after the execution period has closed.{color}
Now, say the task ran for +4 hours.+ So if we set
{code:java}
delta = 5 minutes {code}
it actually doesn't wait at all. The sensor is useless in that senario. What the user may be intended is :
execution_date + duration to complete task + delta.
Or in simple words: Timestamp of execution end + delta.
*My suggestion:*
Add another Boolean parameter which will choose if the delta is from execution_date or execution_date + duration of task
was:
Currently the TimeDeltaSensor takes these arguments:
{color:#4070a0} :param delta: time length to wait after execution_date before succeeding{color}{color:#404040}
{color}{color:#4070a0} :type delta: datetime.timedelta{color}{color:#404040}
{color}{color:#4070a0} """{color}{color:#404040}
{color}{color:#555555}@apply_defaults{color}{color:#404040}
{color}{color:#007020}def{color} {color:#06287e}__init__{color}{color:#404040}({color}{color:#007020}self{color}{color:#404040},{color} {color:#404040}delta{color}{color:#404040},{color} {color:#666666}*{color}{color:#404040}args{color}{color:#404040},{color} {color:#666666}**{color}{color:#404040}kwargs{color}{color:#404040}):{color}{color:#404040}
{color}{color:#007020}super{color}{color:#404040}({color}{color:#404040}TimeDeltaSensor{color}{color:#404040},{color} {color:#007020}self{color}{color:#404040}){color}{color:#666666}.{color}{color:#06287e}__init__{color}{color:#404040}({color}{color:#666666}*{color}{color:#404040}args{color}{color:#404040},{color} {color:#666666}**{color}{color:#404040}kwargs{color}{color:#404040}){color}{color:#404040}
{color}{color:#007020}self{color}{color:#666666}.{color}{color:#404040}delta{color} {color:#666666}={color} {color:#404040}delta{color}
*The problem:*
The operator assumes that the user would like to wait execution_date + delta.
This isn't always the case, The user might want to wait delta after the task has finished running.
Lets take this example:
{color:#4070a0} 2016-01-01 can only start running on 2016-01-02. The timedelta here{color}{color:#404040}
{color}{color:#4070a0} represents the time after the execution period has closed.{color}
Now, say the task ran for +4 hours.+ So if we set
{code:java}
delta = 5 minutes {code}
it actually doesn't wait at all. The sensor is useless in that senario. What the user may be intended is :
execution_date + duration to complete task + delta.
Or in simple words: Timestamp of execution end + delta.
*My suggestion:*
Add another Boolean parameter which will choose if the delta is from execution_date or execution_date + duration of task
> TimeDeltaSensor - add end time stamp
> -------------------------------------
>
> Key: AIRFLOW-3146
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3146
> Project: Apache Airflow
> Issue Type: Wish
> Reporter: jack
> Priority: Minor
>
> Currently the TimeDeltaSensor takes these arguments:
> {color:#4070a0} :param delta: time length to wait after execution_date before succeeding{color}
> {color:#4070a0} :type delta: datetime.timedelta{color}{color:#4070a0} """{color}{color:#404040}
>
> {color}{color:#555555}@apply_defaults{color}{color:#404040}
> {color}{color:#007020}def{color} {color:#06287e}__init__{color}{color:#404040}({color}{color:#007020}self{color}{color:#404040},{color} {color:#404040}delta{color}{color:#404040},{color} {color:#666666}*{color}{color:#404040}args{color}{color:#404040},{color} {color:#666666}**{color}{color:#404040}kwargs{color}{color:#404040}):{color}{color:#404040}
> {color}{color:#007020}super{color}{color:#404040}({color}{color:#404040}TimeDeltaSensor{color}{color:#404040},{color} {color:#007020}self{color}{color:#404040}){color}{color:#666666}.{color}{color:#06287e}__init__{color}{color:#404040}({color}{color:#666666}*{color}{color:#404040}args{color}{color:#404040},{color} {color:#666666}**{color}{color:#404040}kwargs{color}{color:#404040}){color}{color:#404040}
> {color}{color:#007020}self{color}{color:#666666}.{color}{color:#404040}delta{color} {color:#666666}={color} {color:#404040}delta{color}
>
> *The problem:*
> The operator assumes that the user would like to wait execution_date + delta.
> This isn't always the case, The user might want to wait delta after the task has finished running.
>
> Lets take this example:
> {color:#4070a0} 2016-01-01 can only start running on 2016-01-02. The timedelta here{color}{color:#4070a0} represents the time after the execution period has closed.{color}
> Now, say the task ran for +4 hours.+ So if we set
> {code:java}
> delta = 5 minutes {code}
> it actually doesn't wait at all. The sensor is useless in that senario. What the user may be intended is :
>
> execution_date + duration to complete task + delta.
>
> Or in simple words: Timestamp of execution end + delta.
>
> *My suggestion:*
> Add another Boolean parameter which will choose if the delta is from execution_date or execution_date + duration of task
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)