You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/11/22 12:41:47 UTC

[GitHub] [airflow] cstenkamp opened a new issue #19743: `catchup=False` doesn't stop airflow from backfilling if executed using the DebugExecutor

cstenkamp opened a new issue #19743:
URL: https://github.com/apache/airflow/issues/19743


   ### Apache Airflow version
   
   2.1.4
   
   ### Operating System
   
   Ubuntu 20.04.3 (airflow in docker-compose)
   
   ### Versions of Apache Airflow Providers
   
   _No response_
   
   ### Deployment
   
   Docker-Compose
   
   ### Deployment details
   
   Docker-Compose stack based on Docker-Image `apache/airflow:2.1.4`
   
   ### What happened
   
   When running a DAG using the DebugExecutor (see below), `catchup=False` is ignored and airflow starts backfilling the task.
   
   ### What you expected to happen
   
   With `catchup=False`, Airflow shouldn't schedule all previous tasks but only the newest one, regardless of the executor
   
   ### How to reproduce
   
   ```
   from datetime import datetime
   from airflow.models.dag import DAG
   from airflow.operators.dummy import DummyOperator
   
   default_args = {
       "start_date": datetime(2021, 11, 10),
       "catchup": False,
   }
   
   dag_args = dict(
       dag_id="accuracy_checker",
       schedule_interval=None,
       default_args=default_args,
       catchup=False,
       max_active_runs=1,
   )
   
   
   with DAG(**dag_args) as dag:
       start_pipeline = DummyOperator(
           task_id="start_pipeline",
           dag=dag,
       )
   
       finish_pipeline = DummyOperator(
           task_id="finish_pipeline",
           dag=dag,
       )
   
       start_pipeline >> finish_pipeline
   
   
   
   if __name__ == "__main__":
       from airflow.utils.state import State
   
       dag.clear(dag_run_state=State.NONE)
       dag.run()
   ```
   
   ### Anything else
   
   -
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [airflow] ashb commented on issue #19743: `catchup=False` doesn't stop airflow from backfilling if executed using the DebugExecutor

Posted by GitBox <gi...@apache.org>.
ashb commented on issue #19743:
URL: https://github.com/apache/airflow/issues/19743#issuecomment-980050943


   @CSammy if you aren't using that executor (it's unlikely unless you or someone else explicitly configured it) then your issue is separate to this one.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [airflow] github-actions[bot] commented on issue #19743: `catchup=False` doesn't stop airflow from backfilling if executed using the DebugExecutor

Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #19743:
URL: https://github.com/apache/airflow/issues/19743#issuecomment-1006176252


   This issue has been closed because it has not received response from the issue author.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [airflow] cstenkamp commented on issue #19743: `catchup=False` doesn't stop airflow from backfilling if executed using the DebugExecutor

Posted by GitBox <gi...@apache.org>.
cstenkamp commented on issue #19743:
URL: https://github.com/apache/airflow/issues/19743#issuecomment-980076768


   @ashb But for me the issue still persists.
   I am only using the DebugExecutor when I explicitly want to debug my code. In my `airflow.cfg` I specified the CeleryExecutor, and only when I want to be able to explicitly set Breakpoints into my code in the dag I overwrite the respective environment variable and execute the python-file directly - thus I only use the DebugExecutor when I invoke this dag using `dag.run()`.
   
   As a workaround for now I also change the start_date to something like `datetime(2021, 11, 10) if os.getenv("AIRFLOW__CORE__EXECUTOR") !== "DebugExecutor" else datetime.now()-timedelta(minutes=32)`, however I don't think that's the best way to solve this problem


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [airflow] CSammy commented on issue #19743: `catchup=False` doesn't stop airflow from backfilling if executed using the DebugExecutor

Posted by GitBox <gi...@apache.org>.
CSammy commented on issue #19743:
URL: https://github.com/apache/airflow/issues/19743#issuecomment-979984290


   I can confirm as well, using the official Helm chart with Docker tag `2.2.2-python3.9`.
   
   When setting a DAG to
   ```python
   schedule_interval="0 0 * * 6",
   start_date=datetime.datetime(2021, 11, 1),
   catchup=False,
   ```
   and running it manually for the first time, it starts runs for 2021-11-13 and 2021-11-20.
   
   I would've expected it to start a single run for today.
   
   One possible workaround for the moment is to use
   ```python
   start_date=airflow.utils.dates.days_ago(1),
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [airflow] potiuk edited a comment on issue #19743: `catchup=False` doesn't stop airflow from backfilling if executed using the DebugExecutor

Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on issue #19743:
URL: https://github.com/apache/airflow/issues/19743#issuecomment-980044945


   Or maybe add a warning in the UI as we do with "Sequential Executor". 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [airflow] CSammy commented on issue #19743: `catchup=False` doesn't stop airflow from backfilling if executed using the DebugExecutor

Posted by GitBox <gi...@apache.org>.
CSammy commented on issue #19743:
URL: https://github.com/apache/airflow/issues/19743#issuecomment-980078753


   Executing `airflow info` on the `scheduler` pod throws the following error:
   `airflow command error: argument subcommand: invalid choice: 'info'`
   
   I'm using the standard Helm chart in version 1.3.0, Docker tag pinned to `2.2.2-python3.9`. Configuration is adjusted only in minor details. Below is the complete DAG I'm using. It results in the following output in the UI (screenshot is including the button I'm using to run the DAG).
   
   ![image](https://user-images.githubusercontent.com/2164484/143602521-32c92709-90f2-4ace-bc1a-f9d792157a04.png)
   
   DAG file:
   ```python
   import datetime
   import os
   
   from airflow import DAG
   from airflow.contrib.operators.kubernetes_pod_operator import KubernetesPodOperator
   
   default_args = {
       "namespace": os.environ["K8S_NAMESPACE"],
       # K8s service account linked to the GCP service account
       "service_account_name": "airflow2-dag-default",
       "image_pull_policy": "Always",
       "get_logs": True,
       # -- DAG-SPECIFIC
       "image": "google/cloud-sdk:slim",
       "env_vars": {"GCP_PROJECT": os.environ["GCP_PROJECT"]},
   }
   
   with DAG(
       dag_id="debug_dag",
       # Saturday midnight
       schedule_interval="0 0 * * 6",
       start_date=datetime.datetime(2021, 11, 1),
       catchup=False,
       tags=["debug dag for catchup tests"],
       default_args=default_args,
   ) as dag:
       gcp_test_task = KubernetesPodOperator(
           # Task name in Airflow 2 UI
           task_id="gcp-test-task",
           # Pod name
           name="task-gcp-test-task",
           cmds=["sleep", "infinity"],
       )
   
   gcp_test_task
   ```
   
   My impression is that not the DebugExecutor is at fault. I believe the `catchup` parameter is not effective at the moment, for some reason. I'm happy to open a ticket for "`catchup` is always `True` no matter what you specify with the default Executor`" if that matches your guidelines for tickets better here, but I felt like it wouldn't make much sense to open a second ticket for what I perceive to be the same issue just with a different executor.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [airflow] github-actions[bot] closed issue #19743: `catchup=False` doesn't stop airflow from backfilling if executed using the DebugExecutor

Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed issue #19743:
URL: https://github.com/apache/airflow/issues/19743


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [airflow] ephraimbuddy commented on issue #19743: `catchup=False` doesn't stop airflow from backfilling if executed using the DebugExecutor

Posted by GitBox <gi...@apache.org>.
ephraimbuddy commented on issue #19743:
URL: https://github.com/apache/airflow/issues/19743#issuecomment-975542040


   Can you test with 2.2.2, there are changes in how catchup works since 2.1.4


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [airflow] ashb edited a comment on issue #19743: `catchup=False` doesn't stop airflow from backfilling if executed using the DebugExecutor

Posted by GitBox <gi...@apache.org>.
ashb edited a comment on issue #19743:
URL: https://github.com/apache/airflow/issues/19743#issuecomment-980037083


   Running with the scheduler with the DebugExecutor anywhere outside of the `airflow dags test` command is not recommended or supported.
   
   Oh I see, you have `dag.run()` which is also a valid way of using this executor, sorry.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [airflow] CSammy commented on issue #19743: `catchup=False` doesn't stop airflow from backfilling if executed using the DebugExecutor

Posted by GitBox <gi...@apache.org>.
CSammy commented on issue #19743:
URL: https://github.com/apache/airflow/issues/19743#issuecomment-980172293


   I have created a separate ticket (https://github.com/apache/airflow/issues/19844) due your observation, @cstenkamp .
   
   If they end up being caused by the same underlying issue, it's still possible to close it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [airflow] CSammy edited a comment on issue #19743: `catchup=False` doesn't stop airflow from backfilling if executed using the DebugExecutor

Posted by GitBox <gi...@apache.org>.
CSammy edited a comment on issue #19743:
URL: https://github.com/apache/airflow/issues/19743#issuecomment-980078753


   Executing `airflow info` on the `scheduler` pod throws the following error:
   `airflow command error: argument subcommand: invalid choice: 'info'`
   
   I'm using the standard Helm chart in version 1.3.0, Docker tag pinned to `2.2.2-python3.9`. Configuration is adjusted only in minor details. Below is the complete DAG I'm using. It results in the following output in the UI (screenshot is including the button I'm using to run the DAG).
   
   ![image](https://user-images.githubusercontent.com/2164484/143602521-32c92709-90f2-4ace-bc1a-f9d792157a04.png)
   
   DAG file:
   ```python
   import datetime
   import os
   
   from airflow import DAG
   from airflow.contrib.operators.kubernetes_pod_operator import KubernetesPodOperator
   
   default_args = {
       "namespace": os.environ["K8S_NAMESPACE"],
       # K8s service account linked to the GCP service account
       "service_account_name": "airflow2-dag-default",
       "image_pull_policy": "Always",
       "get_logs": True,
       # -- DAG-SPECIFIC
       "image": "google/cloud-sdk:slim",
   }
   
   with DAG(
       dag_id="debug_dag",
       # Saturday midnight
       schedule_interval="0 0 * * 6",
       start_date=datetime.datetime(2021, 11, 1),
       catchup=False,
       tags=["debug dag for catchup tests"],
       default_args=default_args,
   ) as dag:
       gcp_test_task = KubernetesPodOperator(
           # Task name in Airflow 2 UI
           task_id="gcp-test-task",
           # Pod name
           name="task-gcp-test-task",
           cmds=["sleep", "infinity"],
       )
   
   gcp_test_task
   ```
   
   My impression is that not the DebugExecutor is at fault. I believe the `catchup` parameter is not effective at the moment, for some reason. I'm happy to open a ticket for "`catchup` is always `True` no matter what you specify with the default Executor`" if that matches your guidelines for tickets better here, but I felt like it wouldn't make much sense to open a second ticket for what I perceive to be the same issue just with a different executor.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [airflow] cstenkamp commented on issue #19743: `catchup=False` doesn't stop airflow from backfilling if executed using the DebugExecutor

Posted by GitBox <gi...@apache.org>.
cstenkamp commented on issue #19743:
URL: https://github.com/apache/airflow/issues/19743#issuecomment-980106039


   @CSammy for me, only when I run it using the DebugExecutor it tries to backfill. If I schedule it normally (using the UI or using `airflow dags test`), `catchup=True` is not ignored and it only schedules the one latest task


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [airflow] potiuk commented on issue #19743: `catchup=False` doesn't stop airflow from backfilling if executed using the DebugExecutor

Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #19743:
URL: https://github.com/apache/airflow/issues/19743#issuecomment-980047516


   The `airflow info` command should tell you what you use.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [airflow] CSammy edited a comment on issue #19743: `catchup=False` doesn't stop airflow from backfilling if executed using the DebugExecutor

Posted by GitBox <gi...@apache.org>.
CSammy edited a comment on issue #19743:
URL: https://github.com/apache/airflow/issues/19743#issuecomment-979984290


   I can confirm as well, using the official Helm chart with Docker tag `2.2.2-python3.9`.
   
   When setting a DAG to
   ```python
   schedule_interval="0 0 * * 6",
   start_date=datetime.datetime(2021, 11, 1),
   catchup=False,
   ```
   and running it manually for the first time, it starts runs for 2021-11-13 and 2021-11-20.
   
   I would've expected it to start a single run for today.
   
   The DAG uses `KubernetesPodOperator` exclusively.
   
   One possible workaround for the moment is to use
   ```python
   start_date=airflow.utils.dates.days_ago(1),
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [airflow] abhi-quinyx commented on issue #19743: `catchup=False` doesn't stop airflow from backfilling if executed using the DebugExecutor

Posted by GitBox <gi...@apache.org>.
abhi-quinyx commented on issue #19743:
URL: https://github.com/apache/airflow/issues/19743#issuecomment-979905015


   I can confirm that similar issue exists in 2.2.2


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [airflow] ashb commented on issue #19743: `catchup=False` doesn't stop airflow from backfilling if executed using the DebugExecutor

Posted by GitBox <gi...@apache.org>.
ashb commented on issue #19743:
URL: https://github.com/apache/airflow/issues/19743#issuecomment-980044013


   @CSammy then my previous comment stands: using DebugExecutor with a scheduler is not really supported.
   
   @potiuk Should we disallow running `scheduler` with  the DebugExecutor do you think?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [airflow] boring-cyborg[bot] commented on issue #19743: `catchup=False` doesn't stop airflow from backfilling if executed using the DebugExecutor

Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #19743:
URL: https://github.com/apache/airflow/issues/19743#issuecomment-975482984


   Thanks for opening your first issue here! Be sure to follow the issue template!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [airflow] CSammy edited a comment on issue #19743: `catchup=False` doesn't stop airflow from backfilling if executed using the DebugExecutor

Posted by GitBox <gi...@apache.org>.
CSammy edited a comment on issue #19743:
URL: https://github.com/apache/airflow/issues/19743#issuecomment-980078753


   ~~Executing `airflow info` on the `scheduler` pod throws the following error:
   `airflow command error: argument subcommand: invalid choice: 'info'`~~
   
   My mistake. It says `CeleryExecutor`.
   
   I'm using the standard Helm chart in version 1.3.0, Docker tag pinned to `2.2.2-python3.9`. Configuration is adjusted only in minor details. Below is the complete DAG I'm using. It results in the following output in the UI (screenshot is including the button I'm using to run the DAG).
   
   ![image](https://user-images.githubusercontent.com/2164484/143602521-32c92709-90f2-4ace-bc1a-f9d792157a04.png)
   
   DAG file:
   ```python
   import datetime
   import os
   
   from airflow import DAG
   from airflow.contrib.operators.kubernetes_pod_operator import KubernetesPodOperator
   
   default_args = {
       "namespace": os.environ["K8S_NAMESPACE"],
       # K8s service account linked to the GCP service account
       "service_account_name": "airflow2-dag-default",
       "image_pull_policy": "Always",
       "get_logs": True,
       # -- DAG-SPECIFIC
       "image": "google/cloud-sdk:slim",
   }
   
   with DAG(
       dag_id="debug_dag",
       # Saturday midnight
       schedule_interval="0 0 * * 6",
       start_date=datetime.datetime(2021, 11, 1),
       catchup=False,
       tags=["debug dag for catchup tests"],
       default_args=default_args,
   ) as dag:
       gcp_test_task = KubernetesPodOperator(
           # Task name in Airflow 2 UI
           task_id="gcp-test-task",
           # Pod name
           name="task-gcp-test-task",
           cmds=["sleep", "infinity"],
       )
   
   gcp_test_task
   ```
   
   My impression is that not the DebugExecutor is at fault. I believe the `catchup` parameter is not effective at the moment, for some reason. I'm happy to open a ticket for "`catchup` is always `True` no matter what you specify with the default Executor`" if that matches your guidelines for tickets better here, but I felt like it wouldn't make much sense to open a second ticket for what I perceive to be the same issue just with a different executor.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [airflow] CSammy commented on issue #19743: `catchup=False` doesn't stop airflow from backfilling if executed using the DebugExecutor

Posted by GitBox <gi...@apache.org>.
CSammy commented on issue #19743:
URL: https://github.com/apache/airflow/issues/19743#issuecomment-980042843


   To be honest I'm not too familiar with the CLI or code side of Airflow itself. I'm pressing the "run" button of the DAG in the UI.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [airflow] github-actions[bot] commented on issue #19743: `catchup=False` doesn't stop airflow from backfilling if executed using the DebugExecutor

Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #19743:
URL: https://github.com/apache/airflow/issues/19743#issuecomment-1002819762


   This issue has been automatically marked as stale because it has been open for 30 days with no response from the author. It will be closed in next 7 days if no further activity occurs from the issue author.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [airflow] ashb commented on issue #19743: `catchup=False` doesn't stop airflow from backfilling if executed using the DebugExecutor

Posted by GitBox <gi...@apache.org>.
ashb commented on issue #19743:
URL: https://github.com/apache/airflow/issues/19743#issuecomment-980037083


   Running with the scheduler with the DebugExecutor anywhere outside of the `airflow dags test` command is not recommended or supported.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [airflow] ashb commented on issue #19743: `catchup=False` doesn't stop airflow from backfilling if executed using the DebugExecutor

Posted by GitBox <gi...@apache.org>.
ashb commented on issue #19743:
URL: https://github.com/apache/airflow/issues/19743#issuecomment-980038777


   @CSammy are you running a scheduler with the DebugExecutor, or using `dag.run()`/airflow dags test`?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [airflow] CSammy commented on issue #19743: `catchup=False` doesn't stop airflow from backfilling if executed using the DebugExecutor

Posted by GitBox <gi...@apache.org>.
CSammy commented on issue #19743:
URL: https://github.com/apache/airflow/issues/19743#issuecomment-980045703


   Please be aware that I'm using a completely normal DAG with KubernetesPodOperators. How do I find out if I'm using a DebugExecutor?
   
   I have the impression this issue may not be related to the DebugExecutor.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [airflow] potiuk commented on issue #19743: `catchup=False` doesn't stop airflow from backfilling if executed using the DebugExecutor

Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #19743:
URL: https://github.com/apache/airflow/issues/19743#issuecomment-980044945


   Or maybe add a warning as we do with "Sequential Executor". 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [airflow] CSammy edited a comment on issue #19743: `catchup=False` doesn't stop airflow from backfilling if executed using the DebugExecutor

Posted by GitBox <gi...@apache.org>.
CSammy edited a comment on issue #19743:
URL: https://github.com/apache/airflow/issues/19743#issuecomment-980172293


   I have created a separate ticket (https://github.com/apache/airflow/issues/19844) due to your observation, @cstenkamp .
   
   If they end up being caused by the same underlying issue, it's still possible to close it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org