[jira] [Reopened] (AIRFLOW-2127) Airflow's Alembic migrations globally disable logging

["Aizhamal Nurmamat kyzy (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AIRFLOW-2127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Aizhamal Nurmamat kyzy reopened AIRFLOW-2127:
---------------------------------------------

Need to open for component refactor.

> Airflow's Alembic migrations globally disable logging
> -----------------------------------------------------
>
>                 Key: AIRFLOW-2127
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-2127
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: db
>            Reporter: Matt Davis
>            Priority: Major
>             Fix For: 1.10.0
>
>
> When running Airflow's {{[upgradedb|https://github.com/apache/incubator-airflow/blob/fc26cade87e181f162ecc8391ae16dccbe6f29c4/airflow/utils/db.py#L295]}}, {{[resetdb|https://github.com/apache/incubator-airflow/blob/fc26cade87e181f162ecc8391ae16dccbe6f29c4/airflow/utils/db.py#L311]}}, and {{[initdb|https://github.com/apache/incubator-airflow/blob/fc26cade87e181f162ecc8391ae16dccbe6f29c4/airflow/utils/db.py#L83]}} functions logging is disabled thereafter for all but the {{sqlalchemy.engine}} and {{alembic}} loggers. This is caused [this usage|https://github.com/apache/incubator-airflow/blob/fc26cade87e181f162ecc8391ae16dccbe6f29c4/airflow/migrations/env.py#L28] of Python's {{fileConfig}} function, which by default disables all loggers that aren't part of the supplied configuration. (See [Python 2 docs|https://docs.python.org/2/library/logging.config.html#logging.config.fileConfig] and [Python 3 docs|https://docs.python.org/3/library/logging.config.html#logging.config.fileConfig].) This can be fixed by adding {{disable_existing_loggers=False}} to the call of {{fileConfig}}.
> This has affected us at Clover Health because we use these database utility functions in some of our tooling, and particularly our _tests_ of the tooling. Having all logging disabled in the midst of our tests makes it more difficult to test our use of logging in completely unrelated parts of our codebase.
> As an example, we were trying to use [pytest's caplog feature|https://docs.pytest.org/en/latest/logging.html#caplog-fixture], but were unable to do so with logging globally disabled by {{fileConfig}}. Here's an example of a test that fails with {{disable_existing_loggers=True}} (the default), but passes with {{disable_existing_loggers=False}}.
> {code}
> import logging
> import pytest
> import airflow.utils.db as af_db
> LOGGER = logging.getLogger(__name__)
> @pytest.fixture(autouse=True)
> def resetdb():
>     af_db.resetdb()
> def test_caplog(caplog):
>     LOGGER.info('LINE 1')
>     assert caplog.record_tuples
>     assert 'LINE 1' in caplog.text
> {code}
> I'll submit a pull request shortly to add {{disable_existing_loggers=False}} to Airflow's {{[env.py|https://github.com/apache/incubator-airflow/blob/fc26cade87e181f162ecc8391ae16dccbe6f29c4/airflow/migrations/env.py#L28]}}.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)