[GitHub] [airflow] ashb commented on issue #6274: AIRFLOW-5608: Fix bug in SchedulerJob when calling executor.end

[GitBox <gi...@apache.org>]

ashb commented on issue #6274: AIRFLOW-5608: Fix bug in SchedulerJob when calling executor.end
URL: https://github.com/apache/airflow/pull/6274#issuecomment-543176050
 
 
   @kpathak13 This PR breaks the Scheduler stopping with LocalExecutor:
   
   
   ```
   Traceback (most recent call last):
     File "/Users/ash/code/python/incubator-airflow/airflow/jobs/scheduler_job.py", line 1358, in _execute
       self._execute_helper()
     File "/Users/ash/code/python/incubator-airflow/airflow/jobs/scheduler_job.py", line 1485, in _execute_helper
       sleep(sleep_length)
     File "/Users/ash/code/python/incubator-airflow/airflow/bin/cli.py", line 91, in sigint_handler
       sys.exit(0)
   SystemExit: 0
   
   During handling of the above exception, another exception occurred:
   
   Traceback (most recent call last):
     File "/Users/ash/.virtualenvs/airflow/bin/airflow", line 7, in <module>
       exec(compile(f.read(), __file__, 'exec'))
     File "/Users/ash/code/python/incubator-airflow/airflow/bin/airflow", line 37, in <module>
       args.func(args)
     File "/Users/ash/code/python/incubator-airflow/airflow/utils/cli.py", line 74, in wrapper
       return f(*args, **kwargs)
     File "/Users/ash/code/python/incubator-airflow/airflow/bin/cli.py", line 1042, in scheduler
       job.run()
     File "/Users/ash/code/python/incubator-airflow/airflow/jobs/base_job.py", line 222, in run
       self._execute()
     File "/Users/ash/code/python/incubator-airflow/airflow/jobs/scheduler_job.py", line 1363, in _execute
       self.executor.end()
     File "/Users/ash/code/python/incubator-airflow/airflow/executors/local_executor.py", line 233, in end
       self.impl.end()
     File "/Users/ash/code/python/incubator-airflow/airflow/executors/local_executor.py", line 209, in end
       self.queue.put((None, None))
     File "<string>", line 2, in put
     File "/Users/ash/.homebrew/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/managers.py", line 796, in _callmethod
       kind, result = conn.recv()
     File "/Users/ash/.homebrew/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/connection.py", line 250, in recv
       buf = self._recv_bytes()
     File "/Users/ash/.homebrew/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/connection.py", line 407, in _recv_bytes
       buf = self._recv(4)
     File "/Users/ash/.homebrew/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/connection.py", line 383, in _recv
       raise EOFError
   EOFError
   ```
   
   This makes what _was_ a clean shutdown for me an abnormal shutdown. I am reverting this change.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services