[GitHub] [airflow] kaxil commented on issue #13047: Occasional "KeyError" in dag_processing

[GitBox <gi...@apache.org>]

kaxil commented on issue #13047:
URL: https://github.com/apache/airflow/issues/13047#issuecomment-788138438


   > Hi Team ,
   > 
   > Our dags are failing to start with the below issue . We have not done any chages , had just restarted our scheduler and then onwards we are facing this issue.
   > 
   > ```
   > [2021-03-01 15:27:34,622] {scheduler_job.py:1229} ERROR - Executor reports task instance <TaskInstance: wmt.hw.clinic.create_cluster.create_cluster 2021-03-01 15:06:37.473524+00:00 [queued]> finished (failed) although the task says its queued. (Info: None) Was the task killed externally?
   > [2021-03-01 15:27:36,366] {dagrun.py:429} ERROR - Marking run <DagRun wmt.hw.clinic.create_cluster @ 2021-03-01 15:06:37.473524+00:00: manual__2021-03-01T15:06:37.473524+00:00, externally triggered: True> failed
   > Process ForkProcess-8:
   > Traceback (most recent call last):
   > File "/usr/local/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
   >   self.run()
   > File "/usr/local/lib/python3.8/multiprocessing/process.py", line 108, in run
   >   self._target(*self._args, **self._kwargs)
   > File "/home/airflow/.local/lib/python3.8/site-packages/airflow/utils/dag_processing.py", line 365, in _run_processor_manager
   >   processor_manager.start()
   > File "/home/airflow/.local/lib/python3.8/site-packages/airflow/utils/dag_processing.py", line 596, in start
   >   return self._run_parsing_loop()
   > File "/home/airflow/.local/lib/python3.8/site-packages/airflow/utils/dag_processing.py", line 659, in _run_parsing_loop
   >   self._processors.pop(processor.file_path)
   > KeyError: '/opt/airflow/dags/a0s03fc/rcm_clinic_create_cluster.py'
   > ```
   
   You need to update to 2.0.1


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org