You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "choo121600 (via GitHub)" <gi...@apache.org> on 2026/03/05 06:30:29 UTC
[PR] Fix flaky backfill E2E tests with stable locators [airflow]
choo121600 opened a new pull request, #62913:
URL: https://github.com/apache/airflow/pull/62913
This PR stabilizes the backfill E2E tests, which were intermittently failing due to race conditions, non-deterministic date calculations, and brittle CSS selectors.
## Changes
### Flaky test root causes addressed
- **Relative dates → fixed dates**: `getPastDate(n)` produced different values depending on execution time, causing non-deterministic backfill ranges. Replaced with `FIXED_DATES` constants.
- **Race with scheduler**: Control tests (pause/resume/cancel) failed when the scheduler completed the backfill before the test had a chance to interact with it. The tests now pause the backfill immediately after creation via the API to guarantee the controls remain visible.
- **Brittle locators**: Replaced brittle CSS attribute selectors(`button[aria-label="..."]`, `table`, `[role="menuitem"]`) with Playwright recommended locators (`getByTestId`, `getByRole`, `getByText`).
- **`waitForTimeout` removal**: Eliminated hardcoded `waitForTimeout(500)` calls in favor of proper Playwright assertions and `waitForResponse`.
### Test setup/teardown improvements
- Added API helpers (`cancelAllActiveBackfillsViaApi`, `createBackfillViaApi`, `waitForBackfillComplete`, `waitForNoActiveBackfillViaApi`) for reliable setup and cleanup that don't depend on UI state.
- `createBackfill` (UI) now returns the backfill ID by intercepting the POST response, enabling precise completion polling.
- `beforeEach`/`afterEach` in the controls block ensures a clean state between tests.
### Code cleanup
- Consolidated 3 near-identical verification tests into a `for...of` loop.
- Unified reprocess behavior types via `REPROCESS_API_TO_UI` mapping.
- Separated pause/unpause into distinct locators with `.or()` composition.
- Removed unused methods and types.
## What is still tested through the UI
- Backfill creation dialog (4 times: 3 in `beforeAll` + 1 in cancel test)
- Date range validation error message
- Backfill table data verification (dates, reprocess behavior, timestamps)
- Table column filter toggle (hide/show)
- Pause/resume button toggle cycle
- Cancel button functionality
- Cancelled backfill state verification in table
API is used **only** for setup/teardown and race condition prevention, never as a substitute for UI interaction under test.
<!-- SPDX-License-Identifier: Apache-2.0
https://www.apache.org/licenses/LICENSE-2.0 -->
<!--
Thank you for contributing!
Please provide above a brief description of the changes made in this pull request.
Write a good git commit message following this guide: http://chris.beams.io/posts/git-commit/
Please make sure that your code changes are covered with tests.
And in case of new features or big changes remember to adjust the documentation.
Feel free to ping (in general) for the review if you do not see reaction for a few days
(72 Hours is the minimum reaction time you can expect from volunteers) - we sometimes miss notifications.
In case of an existing issue, reference it using one of the following:
* closes: #ISSUE
* related: #ISSUE
-->
---
##### Was generative AI tooling used to co-author this PR?
<!--
If generative AI tooling has been used in the process of authoring this PR, please
change below checkbox to `[X]` followed by the name of the tool, uncomment the "Generated-by".
-->
- [X] Yes (please specify the tool below)
claude
<!--
Generated-by: [Tool Name] following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
-->
---
* Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
* For fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
* When adding dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
* For significant user-facing changes create newsfragment: `{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in [airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix flaky backfill E2E tests with stable locators [airflow]
Posted by "choo121600 (via GitHub)" <gi...@apache.org>.
choo121600 commented on PR #62913:
URL: https://github.com/apache/airflow/pull/62913#issuecomment-4002931876
Hmm, not good :(
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix flaky backfill E2E tests with stable locators [airflow]
Posted by "vatsrahul1001 (via GitHub)" <gi...@apache.org>.
vatsrahul1001 merged PR #62913:
URL: https://github.com/apache/airflow/pull/62913
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix flaky backfill E2E tests with stable locators [airflow]
Posted by "vatsrahul1001 (via GitHub)" <gi...@apache.org>.
vatsrahul1001 commented on PR #62913:
URL: https://github.com/apache/airflow/pull/62913#issuecomment-4016009799
Nice
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org