You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/05/10 05:39:41 UTC
[GitHub] [airflow] ajaymalikbol opened a new issue #15753: Airflow scheduler getting stop with sys.exit(os.EX_OK)
ajaymalikbol opened a new issue #15753:
URL: https://github.com/apache/airflow/issues/15753
We are using airflow Version: v2.0.0
Airflow scheduler is getting stopped, These are the logs that seem to be the only reason behind scheduler getting stop.
```[2021-04-26 11:47:07,335] {scheduler_job.py:641} WARNING - No viable dags retrieved from /opt/airflow/dags/icici_credit_card_scheduler.py
[2021-04-26 11:47:07,342] {scheduler_job.py:189} INFO - Processing /opt/airflow/dags/icici_credit_card_scheduler.py took 0.596 seconds
[2021-04-26 11:47:13,707] {scheduler_job.py:181} INFO - Started process (PID=27236) to work on /opt/airflow/dags/icici_credit_card_scheduler.py
[2021-04-26 11:47:13,708] {scheduler_job.py:629} INFO - Processing file /opt/airflow/dags/icici_credit_card_scheduler.py for tasks to queue
[2021-04-26 11:47:13,709] {logging_mixin.py:103} INFO - [2021-04-26 11:47:13,709] {dagbag.py:440} INFO - Filling up the DagBag from /opt/airflow/dags/icici_credit_card_scheduler.py
[2021-04-26 11:47:14,527] {logging_mixin.py:103} INFO - [2021-04-26 11:47:14,527] {process_utils.py:201} INFO - Waiting up to 5 seconds for processes to exit...
[2021-04-26 11:47:14,528] {logging_mixin.py:103} WARNING - Exception ignored in: <function WeakValueDictionary.__init__.<locals>.remove at 0x7f5435e5b9d0>
[2021-04-26 11:47:14,528] {logging_mixin.py:103} WARNING - Traceback (most recent call last):
[2021-04-26 11:47:14,528] {logging_mixin.py:103} WARNING - File "/usr/local/lib/python3.8/weakref.py", line 103, in remove
[2021-04-26 11:47:14,528] {logging_mixin.py:103} WARNING - def remove(wr, selfref=ref(self), _atomic_removal=_remove_dead_weakref):
[2021-04-26 11:47:14,529] {logging_mixin.py:103} WARNING - File "/home/airflow/.local/lib/python3.8/site-packages/airflow/utils/dag_processing.py", line 572, in _exit_gracefully
[2021-04-26 11:47:14,529] {logging_mixin.py:103} WARNING - self.log.debug("Current Stacktrace is: %s", '\n'.join(map(str, inspect.stack())))
[2021-04-26 11:47:14,529] {logging_mixin.py:103} WARNING - File "/usr/local/lib/python3.8/inspect.py", line 1514, in stack
[2021-04-26 11:47:14,529] {logging_mixin.py:103} WARNING - return getouterframes(sys._getframe(1), context)
[2021-04-26 11:47:14,529] {logging_mixin.py:103} WARNING - File "/usr/local/lib/python3.8/inspect.py", line 1491, in getouterframes
[2021-04-26 11:47:14,530] {logging_mixin.py:103} WARNING - frameinfo = (frame,) + getframeinfo(frame, context)
[2021-04-26 11:47:14,530] {logging_mixin.py:103} WARNING - File "/usr/local/lib/python3.8/inspect.py", line 1465, in getframeinfo
[2021-04-26 11:47:14,530] {logging_mixin.py:103} WARNING - lines, lnum = findsource(frame)
[2021-04-26 11:47:14,531] {logging_mixin.py:103} WARNING - File "/usr/local/lib/python3.8/inspect.py", line 792, in findsource
[2021-04-26 11:47:14,531] {logging_mixin.py:103} WARNING - module = getmodule(object, file)
[2021-04-26 11:47:14,531] {logging_mixin.py:103} WARNING - File "/usr/local/lib/python3.8/inspect.py", line 754, in getmodule
[2021-04-26 11:47:14,531] {logging_mixin.py:103} WARNING - os.path.realpath(f)] = module.__name__
[2021-04-26 11:47:14,531] {logging_mixin.py:103} WARNING - File "/usr/local/lib/python3.8/posixpath.py", line 391, in realpath
[2021-04-26 11:47:14,531] {logging_mixin.py:103} WARNING - path, ok = _joinrealpath(filename[:0], filename, {})
[2021-04-26 11:47:14,532] {logging_mixin.py:103} WARNING - File "/usr/local/lib/python3.8/posixpath.py", line 425, in _joinrealpath
[2021-04-26 11:47:14,532] {logging_mixin.py:103} WARNING - if not islink(newpath):
[2021-04-26 11:47:14,532] {logging_mixin.py:103} WARNING - File "/usr/local/lib/python3.8/posixpath.py", line 167, in islink
[2021-04-26 11:47:14,532] {logging_mixin.py:103} WARNING - st = os.lstat(path)
[2021-04-26 11:47:14,532] {logging_mixin.py:103} WARNING - File "/home/airflow/.local/lib/python3.8/site-packages/airflow/utils/dag_processing.py", line 572, in _exit_gracefully
[2021-04-26 11:47:14,532] {logging_mixin.py:103} WARNING - self.log.debug("Current Stacktrace is: %s", '\n'.join(map(str, inspect.stack())))
[2021-04-26 11:47:14,532] {logging_mixin.py:103} WARNING - File "/usr/local/lib/python3.8/inspect.py", line 1514, in stack
[2021-04-26 11:47:14,533] {logging_mixin.py:103} WARNING - return getouterframes(sys._getframe(1), context)
[2021-04-26 11:47:14,533] {logging_mixin.py:103} WARNING - File "/usr/local/lib/python3.8/inspect.py", line 1491, in getouterframes
[2021-04-26 11:47:14,533] {logging_mixin.py:103} WARNING - frameinfo = (frame,) + getframeinfo(frame, context)
[2021-04-26 11:47:14,533] {logging_mixin.py:103} WARNING - File "/usr/local/lib/python3.8/inspect.py", line 1465, in getframeinfo
[2021-04-26 11:47:14,534] {logging_mixin.py:103} WARNING - lines, lnum = findsource(frame)
[2021-04-26 11:47:14,534] {logging_mixin.py:103} WARNING - File "/usr/local/lib/python3.8/inspect.py", line 792, in findsource
[2021-04-26 11:47:14,534] {logging_mixin.py:103} WARNING - module = getmodule(object, file)
[2021-04-26 11:47:14,534] {logging_mixin.py:103} WARNING - File "/usr/local/lib/python3.8/inspect.py", line 744, in getmodule
[2021-04-26 11:47:14,535] {logging_mixin.py:103} WARNING - for modname, module in sys.modules.copy().items():
[2021-04-26 11:47:14,535] {logging_mixin.py:103} WARNING - File "/home/airflow/.local/lib/python3.8/site-packages/airflow/utils/dag_processing.py", line 576, in _exit_gracefully
[2021-04-26 11:47:14,535] {logging_mixin.py:103} WARNING - sys.exit(os.EX_OK)
[2021-04-26 11:47:14,535] {logging_mixin.py:103} WARNING - SystemExit: 0
```
This is the entry point which we are using for building the airflow docker image.
[entrypoint](https://github.com/apache/airflow/blob/v2-0-stable/scripts/in_container/prod/entrypoint_prod.sh)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #15753: Airflow scheduler getting stop with sys.exit(os.EX_OK)
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #15753:
URL: https://github.com/apache/airflow/issues/15753#issuecomment-836203019
Thanks for opening your first issue here! Be sure to follow the issue template!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #15753: Airflow scheduler getting stop with sys.exit(os.EX_OK)
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #15753:
URL: https://github.com/apache/airflow/issues/15753#issuecomment-836298456
I think you might have sys.exit() in your DAG file. The logs you have sent are just normal logs when you exit scheduler. And this will happen if your dag calls sys.exit() explicilty I believe.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #15753: Airflow scheduler getting stop with sys.exit(os.EX_OK)
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #15753:
URL: https://github.com/apache/airflow/issues/15753#issuecomment-836299715
In any case the infromation you posted is not enough to make any investigation - we need a reproducible case to be able to do anything with it. I am closing it for now, if you provide more information and reproducible case, we can re-open it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #15753: Airflow scheduler getting stop with sys.exit(os.EX_OK)
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #15753:
URL: https://github.com/apache/airflow/issues/15753
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org