[PR] feat(celery): honor json_logs config in Celery worker startup [airflow]

["hanxdatadog (via GitHub)" <gi...@apache.org>]

hanxdatadog opened a new pull request, #68915:
URL: https://github.com/apache/airflow/pull/68915

   Closes #68912
   
   ## Problem
   
   The Celery worker ignores `[logging] json_logs` / `AIRFLOW__LOGGING__JSON_LOGS`. It always calls `configure_logging(output=sys.stdout.buffer)` without passing `json_output`, so structured JSON logging is never enabled on the worker even when configured globally.
   
   ## Solution
   
   Add a two-level config lookup before calling `configure_logging`:
   
   1. `[celery] json_logs` — worker-specific override, takes precedence.
   2. `[logging] json_logs` — global setting, used when the celery key is absent.
   3. `False` — default when neither key is configured.
   
   This mirrors the existing `[celery] CELERY_LOGGING_LEVEL` / `[logging] LOGGING_LEVEL` fallback pattern already present in the worker startup code.
   
   ```ini
   # Global — affects the API server, scheduler, and workers:
   [logging]
   json_logs = True
   
   # Or override for the Celery worker only, leaving other components unchanged:
   [celery]
   json_logs = True
   ```
   
   ## Changes
   
   - **`providers/celery/src/airflow/providers/celery/cli/celery_command.py`**: read `[celery] json_logs` → `[logging] json_logs` → `False` and pass `json_output` to `configure_logging`.
   - **`providers/celery/provider.yaml`** + **`get_provider_info.py`**: new `json_logs` config option under `[celery]`.
   - **`providers/celery/tests/unit/celery/cli/test_celery_command.py`**: `TestWorkerJsonLogs` — 3 tests covering (a) default is False, (b) global `[logging]` setting is inherited, (c) `[celery]` override takes precedence.
   - **`providers/celery/docs/celery_executor.rst`**: new "Worker logging" section documenting the fallback order and Airflow 3.0/3.2 compatibility note.
   - **`providers/celery/newsfragments/68912.feature.rst`**: changelog entry.
   
   ## Notes
   
   - Only active on Airflow 3+ (`AIRFLOW_V_3_0_PLUS` guard is unchanged).
   - `conf.getboolean(..., fallback=None)` is safe on 3.0/3.1 even when the `[celery] json_logs` key is absent.
   - `version_added: ~` in `provider.yaml` — version not yet decided; to be set by the release manager.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] feat(celery): honor json_logs config in Celery worker startup [airflow]

["hanxdatadog (via GitHub)" <gi...@apache.org>]

hanxdatadog closed pull request #68915: feat(celery): honor json_logs config in Celery worker startup
URL: https://github.com/apache/airflow/pull/68915


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org