[jira] [Commented] (AIRFLOW-2703) Scheduler crashes if Mysql Connectivity is lost

["ASF subversion and git services (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-2703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16747054#comment-16747054 ] 

ASF subversion and git services commented on AIRFLOW-2703:
----------------------------------------------------------

Commit b3918a658c358352348ecb3cb52532e605e7ecf3 in airflow's branch refs/heads/v1-10-test from mishikaSingh
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=b3918a6 ]

[AIRFLOW-2703] Catch transient DB exceptions from scheduler's heartbeat it does not crash (#3650)

If there is any issue in DB connection then rest of the functions take care of those exceptions but in heartbeat of scheduler, there is no handling for this kind of situation.

Airflow Scheduler should not crash if a "transient" DB exception occurs in the heartbeat of scheduler.

> Scheduler crashes if Mysql Connectivity is lost
> -----------------------------------------------
>
>                 Key: AIRFLOW-2703
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-2703
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: scheduler
>    Affects Versions: 1.9.0, 2.0.0
>            Reporter: raman
>            Assignee: Mishika Singh
>            Priority: Major
>             Fix For: 2.0.0
>
>
> Airflow scheduler crashes if connectivity to Mysql is lost.
> Below is the stack Trace
> Traceback (most recent call last): File "/usr/src/venv/local/lib/python2.7/site-packages/airflow/jobs.py", line 371, in helper pickle_dags) File "/usr/src/venv/local/lib/python2.7/site-packages/airflow/utils/db.py", line 50, in wrapper result = func(*args, **kwargs) File "/usr/src/venv/local/lib/python2.7/site-packages/airflow/jobs.py", line 1762, in process_file dag.sync_to_db() File "/usr/src/venv/local/lib/python2.7/site-packages/airflow/utils/db.py", line 50, in wrapper result = func(*args, **kwargs) File "/usr/src/venv/local/lib/python2.7/site-packages/airflow/models.py", line 3816, in sync_to_db session.commit() File "/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 943, in commit self.transaction.commit() File "/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 471, in commit t[1].commit() File "/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1643, in commit self._do_commit() File "/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1674, in _do_commit self.connection._commit_impl() File "/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 726, in _commit_impl self._handle_dbapi_exception(e, None, None, None, None) File "/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1413, in _handle_dbapi_exception exc_info File "/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause reraise(type(exception), exception, tb=exc_tb, cause=cause) File "/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 724, in _commit_impl self.engine.dialect.do_commit(self.connection) File "/usr/src/venv/local/lib/python2.7/site-packages/sqlalchemy/dialects/mysql/base.py", line 1784, in do_commit dbapi_connection.commit() OperationalError: (_mysql_exceptions.OperationalError) (2013, 'Lost connection to MySQL server during query') (Background on this error at: http://sqlalche.me/e/e3q8)
> Process DagFileProcessor141318-Process:



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)