[D] Task failed with exception【sqlalchemy.exc.OperationalError: (MySQLdb.OperationalError) (2006, 'MySQL server has gone away')】 [airflow]

["hrbust-liu (via GitHub)" <gi...@apache.org>]

GitHub user hrbust-liu created a discussion: Task failed with exception【sqlalchemy.exc.OperationalError: (MySQLdb.OperationalError) (2006, 'MySQL server has gone away')】

Should the session commit here be retried? Otherwise, network jitter may cause the commit to fail and the entire task to fail.

```
@contextlib.contextmanager
def create_session() -> Generator[settings.SASession, None, None]:
    """Contextmanager that will create and teardown a session."""
    Session = getattr(settings, "Session", None)
    if Session is None:
        raise RuntimeError("Session must be set before!")
    session = Session()
    try:
        yield session
        session.commit()
    except Exception:
        session.rollback()
        raise
    finally:
        session.close()

```

GitHub link: https://github.com/apache/airflow/discussions/52970

----
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@airflow.apache.org

Re: [D] Task failed with exception【sqlalchemy.exc.OperationalError: (MySQLdb.OperationalError) (2006, 'MySQL server has gone away')】 [airflow]

["potiuk (via GitHub)" <gi...@apache.org>]

GitHub user potiuk added a comment to the discussion: Task failed with exception【sqlalchemy.exc.OperationalError: (MySQLdb.OperationalError) (2006, 'MySQL server has gone away')】

We deliberately only do retries in places where we know it is safe to retry without leaving the side efects?. Is this the case? Can you consider all the scenarios and make sure that there are no side-effects of such failed connection ? 

Generally the approach is that if we are unsure about it - failing the whole process is the BEST option as default - then it means that either retries happen or simply "failed" task in the UI will be a signal to do the retry.

But if you can have certainty that retrying in a specific DB operation is safe and leaves no side effects (for example memory growth, connections open etc.) - feel free to propose a PR to handle it - we have a decorator for retrying db operations in certain specific cases where we discussed and agreed that it is safe to do so - but if you feel strong that this is the case and can convince the community to accept such PR - feel free to try it.

GitHub link: https://github.com/apache/airflow/discussions/52970#discussioncomment-13681724

----
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@airflow.apache.org