You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by "Bumsoo Kim (Jira)" <ji...@apache.org> on 2019/10/25 02:00:00 UTC

[jira] [Created] (AIRFLOW-5753) Create DailyLatestOnlyOperator

Bumsoo Kim created AIRFLOW-5753:
-----------------------------------

             Summary: Create DailyLatestOnlyOperator
                 Key: AIRFLOW-5753
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-5753
             Project: Apache Airflow
          Issue Type: New Feature
          Components: operators
    Affects Versions: 1.10.5
            Reporter: Bumsoo Kim


In addition to {{LatestOnlyOperator}}, {{DailyLatestOnlyOperator}} is for cases that we need to run only the latest run in the series of task run, but to ensure that the task is run at least once in a day.

This is useful when we scrape external data from third-parties on an hourly basis/for multiple times in a day to keep data up-to-date. If the DAG is paused for several days, we want to avoid multiple runs in a day, but still want to ensure data is filled in every daily window.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)