[jira] [Updated] (AIRFLOW-5642) sqlalchemy.exc.OperationalError happens when sql is too long, which cause scheduler processor will be exited

["chen xianxin (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AIRFLOW-5642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

chen xianxin updated AIRFLOW-5642:
----------------------------------
    Summary: sqlalchemy.exc.OperationalError happens when sql is too long, which cause scheduler processor will be exited  (was: sqlalchemy.exc.OperationalError happens when sql is too long, which cause scheduler processor be exited)

> sqlalchemy.exc.OperationalError happens when sql is too long, which cause scheduler processor will be exited
> ------------------------------------------------------------------------------------------------------------
>
>                 Key: AIRFLOW-5642
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-5642
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: scheduler
>    Affects Versions: 1.10.5
>            Reporter: chen xianxin
>            Priority: Minor
>
> h2. code
> airflow\jobs\scheduler_job.py:_change_state_for_tasks_failed_to_execute(self, session)
> <
> ...
> filter_for_ti_state_change = (
>     [and_(
>     TI.dag_id == dag_id,
>     TI.task_id == task_id,
>     TI.execution_date == execution_date,
>     TI._try_number == try_number - 1,
>     TI.state == State.QUEUED)
>     for dag_id, task_id, execution_date, try_number
>     in self.executor.queued_tasks.keys()])
> ti_query = (session.query(TI)
>     .filter(or_(*filter_for_ti_state_change)))
> tis_to_set_to_scheduled = (ti_query
>     .with_for_update()
>     .all())
> ...
> >
> h2. Error description
> {{if there is too many queued_tasks, sqlalchemy.exc.OperationalError will occur. details are as follow:}}
> {{<}}
> {{sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (2006, "MySQL server has gone away (BrokenPipeError(32, 'Broken pipe'))")}}
> [SQL: SELECT ...]
> (Background on this error at: http://sqlalche.me/e/e3q8)
> {{> }}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)