[jira] [Created] (AIRFLOW-3244) Introduce offset on the execution date for data assessment

["Alberto Anceschi (JIRA)" <ji...@apache.org>]

Alberto Anceschi created AIRFLOW-3244:
-----------------------------------------

             Summary: Introduce offset on the execution date for data assessment
                 Key: AIRFLOW-3244
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-3244
             Project: Apache Airflow
          Issue Type: Improvement
          Components: DAG
    Affects Versions: 1.10.0
            Reporter: Alberto Anceschi


Hi everyone,

 

I'm trying to port my current cronjobs into Airflow. Let's consider a real case scenario: I've to send every week a report and through the pipeline data from Google Analytics needs to be collected, so I need 2 days before running the DAG (data assessment). Week starts on Monday and ends on Sunday, so I need the DAG to run on Wednesday at Midnight UTC.

In order to see on the Airflow dashboard start_date/exection_date that make sense to me, for now I've used a TimeDeltaSensor that adds that 2 day offset I need, but this is not its purpose. I also use Celery executor, so its workers keep polling during those 2 days, making them unavailable for other DAGs.

I think that the assumption that at the end of the period scheduled data are ready is not correct and at the same time it's much more intuitive seeing on the dashboard Monday execution dates instead of Tuesday ones.

 

What do you think about this request? Any suggestion? Thank you,

 

Alberto

 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)