You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/06/23 09:26:35 UTC

[GitHub] [airflow] CodingJonas opened a new issue #9485: Unwanted backfill when using empty database even with catchup set to false

CodingJonas opened a new issue #9485:
URL: https://github.com/apache/airflow/issues/9485

**Apache Airflow version**:v1.10

**Environment**: Deployed inside a Docker container

- **Cloud provider or hardware configuration**:
- **OS** (e.g. from /etc/os-release): Ubuntu 18.4
- **Install tools**: pipenv

**What happened**:
I have a DAG with start date of 1/1/2018 and schedule to run through a cron expression (e.g. every 2 hours). This is the first time I start Airflow with a new database. No DAG has ever run before. As soon as I unpause my DAG, the scheduler will start running one DAG. I set `catchup=False` to stop this behaviour from happening, yet it still schedules exactly one DAG for backfill.
If there was a previous DAG run this behaviour does not happen.

I found a [reference to this issue](https://stackoverflow.com/questions/52177418/how-to-stop-dag-from-backfilling-catchup-by-default-false-and-catchup-false-doe#comment92792366_52922501) already from 2018, which describes a very similar problem, but I couldn't find an open issue to this.

**What you expected to happen**:
If I set `catchup=False`, I expect no backfill to happen.

**How to reproduce it**:
These are the DAG settings I use:
```python
dag_params = dict(
dag_id='test',
schedule_interval='0 */2 * * *',
start_date=datetime(2018, 1, 1),
max_active_runs=1,
catchup=False,
)
```

**Comment**:
I read somewhere that setting a future `start_date` could help, I tried it with `datetime(3018, 1, 1)`, which got rid of the unwanted backfill, but no DAGs got executed. More interestingly it started a DAG and instantly set it to 'success´ without running a single task. But I think this is unrelated to this issue.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] github-actions[bot] commented on issue #9485: Unwanted backfill when using empty database even with catchup set to false

Posted by GitBox <gi...@apache.org>.

github-actions[bot] commented on issue #9485:
URL: https://github.com/apache/airflow/issues/9485#issuecomment-864478647


   This issue has been closed because it has not received response from the issue author.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] gflores1023 removed a comment on issue #9485: Unwanted backfill when using empty database even with catchup set to false

Posted by GitBox <gi...@apache.org>.

gflores1023 removed a comment on issue #9485:
URL: https://github.com/apache/airflow/issues/9485#issuecomment-662513165


   There was recently a bugfix involving catchup=False. Not sure if related, but you can try using the latest Airflow version 1.10.11 to see if the issue persists. 
   
   https://github.com/apache/airflow/pull/8776


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] eladkal commented on issue #9485: Unwanted backfill when using empty database even with catchup set to false

Posted by GitBox <gi...@apache.org>.

eladkal commented on issue #9485:
URL: https://github.com/apache/airflow/issues/9485#issuecomment-768557976


   You mention 2 issues:
   1. Why a single run is created when you use catchup=False.
   2. Why this single run is marked as success without running anything.
   
   For the **1st** question:
   This is by design.
   Kaxil explained it in the first paragraph of his answer to the stackoverflow question you mentioned in your question.
   
   For the **2nd** question:
   I'm unable to reproduce such behavior. Please provide a reproduce example.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] github-actions[bot] closed issue #9485: Unwanted backfill when using empty database even with catchup set to false

Posted by GitBox <gi...@apache.org>.

github-actions[bot] closed issue #9485:
URL: https://github.com/apache/airflow/issues/9485


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] github-actions[bot] commented on issue #9485: Unwanted backfill when using empty database even with catchup set to false

Posted by GitBox <gi...@apache.org>.

github-actions[bot] commented on issue #9485:
URL: https://github.com/apache/airflow/issues/9485#issuecomment-822092393


   This issue has been automatically marked as stale because it has been open for 30 days with no response from the author. It will be closed in next 7 days if no further activity occurs from the issue author.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] gflores1023 commented on issue #9485: Unwanted backfill when using empty database even with catchup set to false

Posted by GitBox <gi...@apache.org>.

gflores1023 commented on issue #9485:
URL: https://github.com/apache/airflow/issues/9485#issuecomment-662513165


   There was recently a bugfix involving catchup=False. Not sure if related, but you can try using the latest Airflow version 1.10.11 to see if the issue persists. 
   
   https://github.com/apache/airflow/pull/8776


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org