[PR] [v3-2-test] Install shared distros in CI image and tighten shared imports (#65783) [airflow]

["github-actions[bot] (via GitHub)" <gi...@apache.org>]

github-actions[bot] opened a new pull request, #65832:
URL: https://github.com/apache/airflow/pull/65832

   * Use airflow._shared for Stats import in airflow-core dag definition
   
   The new ``check_shared_distributions_structure`` prek hook (#65621)
   requires that ``airflow-core`` import shared code via the
   ``airflow._shared.*`` namespace rather than ``airflow.sdk._shared.*``
   (the task-sdk distribution's namespace). One stray ``Stats`` import in
   ``serialization/definitions/dag.py`` was using the task-sdk path; all
   other airflow-core modules already import the same ``Stats`` from
   ``airflow._shared.observability.metrics.stats``. Switch this one to
   match so it shares the singleton already initialized by the executors
   and job runners.
   
   * Install shared distributions in CI image so airflow_shared.* resolves
   
   The shared distributions under ``shared/<name>/`` are marked
   ``Private :: Do Not Upload`` and are not pulled in transitively when
   Airflow is installed from a published wheel — those wheels only embed
   the modules under ``airflow/_shared/<name>`` and
   ``airflow/sdk/_shared/<name>``. As a result, ``import airflow_shared.X``
   fails in the Lowest Deps and Compat test images even though the source
   tree is mounted.
   
   Install the shared distributions from local source inside
   ``install_development_dependencies.py`` (alongside ``devel-common``)
   so the canonical ``airflow_shared.*`` namespace is always available
   to test utilities. With that in place, switch ``devel-common`` back
   to importing from ``airflow_shared.*`` rather than reaching into
   ``airflow._shared`` / ``airflow.sdk._shared``:
   
   * ``sphinx_exts/metrics_tables_from_registry.py``
   * ``tests_common/pytest_plugin.py`` (``_get_grouped_entry_points``)
   * ``tests_common/test_utils/api_fastapi.py`` (``DEFAULT_SENSITIVE_FIELDS``)
   * ``tests_common/test_utils/logs.py`` (``DEFAULT_SENSITIVE_FIELDS``)
   
   The two ``DEFAULT_SENSITIVE_FIELDS`` try/except fallbacks collapse
   into a single direct import.
   
   * Re-install shared distributions after lowest-direct uv sync
   
   The lowest-direct `uv sync` in `check_force_lowest_dependencies` removes
   the `apache-airflow-shared-*` workspace members from the environment
   because they aren't transitively required by `airflow-core` or any
   provider. With devel-common test utils now importing from
   `airflow_shared.*`, test collection in the LowestDeps jobs fails with
   `ModuleNotFoundError: No module named 'airflow_shared'`.
   
   Re-install all `shared/<name>/` distributions with `--no-deps` after the
   sync so the namespace resolves without disturbing the lowest-direct
   resolution that was just applied.
   (cherry picked from commit 8eb433b842f2ef97001e636cbd3f2233a8aab36c)
   
   Co-authored-by: Jarek Potiuk <ja...@potiuk.com>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [v3-2-test] Install shared distros in CI image and tighten shared imports (#65783) [airflow]

["potiuk (via GitHub)" <gi...@apache.org>]

potiuk merged PR #65832:
URL: https://github.com/apache/airflow/pull/65832


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org