[GitHub] [airflow] totalhack commented on issue #13625: Scheduler Dying/Hanging v2.0.0: WARNING - Killing DAGFileProcessorProcess

[GitBox <gi...@apache.org>]

totalhack commented on issue #13625:
URL: https://github.com/apache/airflow/issues/13625#issuecomment-758968595


   Below is my dag with some placeholders substituted. Note that this isn't even running (or attempting to run) when the problem occurs, as it's scheduled to run once a day. I was able to run this dag successfully.
   
   I also have several other disabled maintenance dags from [here](https://github.com/teamclairvoyant/airflow-maintenance-dags/tree/airflow2.0). They were not run or enabled around the time that the issue occurred but had run fine in previous testing.
   
   I have moved on to getting another scheduling solution in place as this effort is time-sensitive, but I will see if I can reproduce this quickly with debug logging and get back to you. I assume you just mean set logging->logging_level to `DEBUG` in my cfg, if you had other specific debug flags you wanted enabled let me know.
   
   ```
   import os
   
   import airflow
   from airflow import DAG
   from airflow.providers.docker.operators.docker import DockerOperator
   
   IMAGE = os.getenv("IMAGE")
   assert IMAGE, "No IMAGE defined"
   
   default_args = {
       "owner": "[redacted]",
       "start_date": airflow.utils.dates.days_ago(1),
       "email": ["redacted"],
       "email_on_failure": True,
       "email_on_retry": False,
       "depends_on_past": False,
       "retries": 0,
   }
   
   dag = DAG(
       "some_dag_id",
       default_args=default_args,
       schedule_interval="15 7 * * *",
       catchup=False,
   )
   
   t1 = DockerOperator(
       task_id="some_task_id",
       image=IMAGE,
       command="python /app/some_python_script.py",
       api_version="auto",
       auto_remove=True,
       dag=dag,
   )
   ```


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org