You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "potiuk (via GitHub)" <gi...@apache.org> on 2025/02/04 16:40:34 UTC
[I] The `test_refresh_dags_dir_deactivates_deleted_zipped_dags` is flaky [airflow]
potiuk opened a new issue, #46432:
URL: https://github.com/apache/airflow/issues/46432
Flaky test in DAG zipped dags
```python
=================================== FAILURES ===================================
_ TestDagFileProcessorManager.test_refresh_dags_dir_deactivates_deleted_zipped_dags _
tests/dag_processing/test_manager.py:636: in test_refresh_dags_dir_deactivates_deleted_zipped_dags
assert not dag.get_is_active()
E assert not True
E + where True = get_is_active()
E + where get_is_active = <DAG: test_zip_dag>.get_is_active
----------------------------- Captured stdout call -----------------------------
[2025-02-04T16:02:01.361+0000] {dagbag.py:574} INFO - Filling up the DagBag from /tmp/pytest-of-root/pytest-0/test_refresh_dags_dir_deactiva0
[2025-02-04T16:02:01.364+0000] {dagbag.py:442} INFO - File /opt/airflow/tests/dags/test_zip.zip:file_no_airflow_dag.py assumed to contain no DAGs. Skipping.
[2025-02-04T16:02:01.371+0000] {collection.py:83} INFO - Creating ORM DAG for test_zip_dag
[2025-02-04T16:02:01.391+0000] {manager.py:231} INFO - Processing files using up to 2 processes at a time
[2025-02-04T16:02:01.391+0000] {manager.py:232} INFO - Process each file at most once every 30 seconds
[2025-02-04T16:02:01.392+0000] {manager.py:98} INFO - DAG bundles loaded: testing
[2025-02-04T16:02:01.394+0000] {manager.py:98} INFO - DAG bundles loaded: testing
[2025-02-04T16:02:01.395+0000] {manager.py:443} INFO - Refreshing DAG bundles
[2025-02-04T16:02:01.398+0000] {manager.py:531} INFO - Searching for files in testing at /opt/airflow/tests/dags
[2025-02-04T16:02:01.416+0000] {manager.py:533} INFO - Found 41 files for bundle testing
[2025-02-04T16:02:01.466+0000] {manager.py:704} INFO -
================================================================================
```
Example failure: https://github.com/apache/airflow/actions/runs/13139235278/job/36662824378?pr=46431#step:6:3443
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The `test_refresh_dags_dir_deactivates_deleted_zipped_dags` is flaky [airflow]
Posted by "Andrushika (via GitHub)" <gi...@apache.org>.
Andrushika commented on issue #46432:
URL: https://github.com/apache/airflow/issues/46432#issuecomment-4990563861
I think this flaky test has been solved by #46443 and #46604, maybe we can close this issue now.
I traced the test history and I think the flakiness came from two places, the old version of the test looked like this:
```python
# Mock might_contain_dag to mimic deleting the python file from the zip
with mock.patch("airflow.dag_processing.manager.might_contain_dag", return_value=False):
with configure_testing_dag_bundle(TEST_DAGS_FOLDER):
manager = DagFileProcessorManager(max_runs=1)
manager.run()
...
assert not dag.get_is_active()
```
1. The test mocked `manager.might_contain_dag` to simulate deletion, but the zip was still present in the shared Dag folder and could still be parsed during `manager.run()`. When the manager collected that parse result, `DAG.bulk_write_to_db()` set `DagModel.is_active` back to `True`, making the final assertion timing-dependent.
2. The bundle pointed at the shared `TEST_DAGS_FOLDER`, so the manager had to parse all 41 files there. This made the parsing order and timing different on every run.
The current test avoids this conflict: it copies the zip into an isolated temporary bundle, parses it, actually deletes it with os.remove(), and runs the manager again. Both paths now observe that the zip is gone.
Can we close this issue now?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org