You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "mikebridge (via GitHub)" <gi...@apache.org> on 2026/09/11 21:00:11 UTC
[PR] ci(python-unit): run the unit suite in parallel with pytest-xdist [superset]
mikebridge opened a new pull request, #44178:
URL: https://github.com/apache/superset/pull/44178
### SUMMARY
The `unit-tests (current)` job in `Python-Unit` has been getting killed at its 30-minute `timeout-minutes` wall even though every test passes. The serial suite has grown to ~27–28 minutes, so the *trailing* steps (the 100%-coverage gates and the codecov upload) no longer fit on an ordinary slow-runner day:
| run | `Python unit tests` step | then | total |
|---|---|---|---|
| passing | 26m45s ✅ | coverage 1m27s + upload 0m20s | 29m24s — 36 s to spare |
| killed | 28m04s ✅ | killed *during* the coverage gate | 30m21s |
| killed | 27m43s ✅ | coverage ✅, killed *during* codecov upload | 30m11s |
Same tree all three times; the only variable was ~1.3 min of runner speed. Master's own runs take 25–29 min, so the whole repo sits at 85–97% of the budget. On 2026-09-11 alone this killed five otherwise-green PR runs. The step ran `pytest` single-process with coverage instrumentation on, and `pytest-xdist` wasn't installed.
This PR runs the step with **`pytest -n auto --dist loadfile`**:
- `--dist loadfile` keeps every test in a file on one worker, so module-scoped fixture state is preserved.
- `pytest-cov` combines the per-worker data into the single `.coverage` that the later 100%-coverage gates and the codecov upload already read — verified: one combined, fully populated data file, no leftover fragments.
- The two short 100%-coverage gates stay serial (tiny subtrees; xdist startup would cost more than it saves). `timeout-minutes` is unchanged on purpose: with the main step at roughly 8–9 min on the 4-core runners it now has ~3× headroom instead of 36 seconds.
- Lockfile delta is exactly `pytest-xdist==3.8.0` plus its one dependency `execnet==2.1.2` (regenerated with `scripts/uv-pip-compile.sh`).
Parallelism exposed two latent problems in the suite itself. Both are fixed here, each with a fail-without / pass-with control:
1. **Non-deterministic collection order.** `tests/unit_tests/commands/report/base_test.py` parametrized over Python `set`s, so every worker collected the same ids in a different per-process hash order and xdist aborted with *"Different tests were collected between gw4 and gw2"*. Diffing full collections under two `PYTHONHASHSEED` values showed it was the **only** such file out of 15,132 tests. Sets → tuples, with a comment so it doesn't get tidied back.
2. **A hidden import-order dependency (a flake class, not a one-off).** `test_get_sql_results_oauth2` runs under `@freeze_time` and happened to be the first lazy importer of the engine-spec modules on its worker. `clickhouse_connect` reads the local timezone via `dateutil` **at import time**, which raises `AttributeError` under a frozen clock; Superset's `clickhouse.py` guard only catches `ImportError`. Serially, some earlier test had always paid that import outside `freeze_time`, hiding the dependency. **Fifteen** unit-test files use `freeze_time`, so on xdist whichever is first on its worker fails — a flake that moves with worker assignment. A session-scoped autouse fixture in `tests/unit_tests/conftest.py` now pre-loads the engine specs once per worker before any test can freeze the clock, giving every worker the import state a serial run reached by accident. Control: on a cold process the test fails with the exact `AttributeError` without the fixture and passes with it.
Local result (8 workers, coverage on, `TZ=UTC`): **15,126 passed, 5 skipped, 2 xfailed, 0 failed in 3m38s** — versus ~27 min serial.
### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
N/A (CI configuration).
### TESTING INSTRUCTIONS
1. The `Python-Unit` workflow on this PR is the primary test — `unit-tests (current)` should finish well inside the timeout with all lanes green and coverage uploaded.
2. Locally, with the dev requirements installed:
```bash
SUPERSET_TESTENV=true SUPERSET_SECRET_KEY=not-a-secret \
pytest -n auto --dist loadfile --cov=superset --cov-report= ./tests/common ./tests/unit_tests --cache-clear
```
then `coverage report` shows a populated combined file.
3. Collection determinism: `pytest --collect-only -q tests/unit_tests` under `PYTHONHASHSEED=1` and `=2` produces identical output.
4. The freeze_time control: stash the `conftest.py` change and run `pytest tests/unit_tests/sql_lab_test.py::test_get_sql_results_oauth2` in a fresh process — it fails with `AttributeError: 'NoneType' object has no attribute 'replace'` from `dateutil/tz`; restore the change and it passes.
### ADDITIONAL INFORMATION
- [ ] Has associated issue:
- [ ] Required feature flags:
- [ ] Changes UI
- [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351))
- [ ] Migration is atomic, supports rollback & is backwards-compatible
- [ ] Confirm DB migration upgrade and downgrade tested
- [ ] Runtime estimates and downtime expectations provided
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_01EA4MGEkUcDpyLReqjQBhPy
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] ci(python-unit): run the unit suite in parallel with pytest-xdist [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #44178:
URL: https://github.com/apache/superset/pull/44178#issuecomment-5642264011
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
<details>
<summary>
Bito Usage Guide
</summary>
**Commands**
Type the following command in the pull request comment and save the comment.
- `/review` - Manually triggers an incremental AI Review.
- `/review full` - Manually triggers a full AI Review.
- `/pause` - Pauses automatic reviews on this pull request.
- `/resume` - Resumes automatic reviews.
- `/resolve` - Marks all Bito-posted review comments as resolved.
- `/abort` - Cancels all in-progress reviews.
Refer to the <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/available-commands" target="_blank" rel="noopener noreferrer">documentation</a> for additional commands.
**Configuration**
This repository uses `Superset` You can customize the agent settings <a href="https://alpha.bito.ai/home/ai-agents/code-review-agent" target="_blank" rel="noopener noreferrer">here</a> or contact your Bito workspace admin at evan@preset.io.
**Documentation & Help**
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/getting-started/install-run-using-bito-cloud/create-or-customize-an-agent-instance" target="_blank" rel="noopener noreferrer">Customize agent settings</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/implementing-custom-code-review-rules" target="_blank" rel="noopener noreferrer">Review rules</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent" target="_blank" rel="noopener noreferrer">General documentation</a>
- <a href="https://docs.bito.ai/bito-dev-agents/ai-code-review-agent/faqs" target="_blank" rel="noopener noreferrer">FAQ</a>
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] ci(python-unit): run the unit suite in parallel with pytest-xdist [superset]
Posted by "hainenber (via GitHub)" <gi...@apache.org>.
hainenber merged PR #44178:
URL: https://github.com/apache/superset/pull/44178
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] ci(python-unit): run the unit suite in parallel with pytest-xdist [superset]
Posted by "hainenber (via GitHub)" <gi...@apache.org>.
hainenber commented on PR #44178:
URL: https://github.com/apache/superset/pull/44178#issuecomment-5643314657
Can confirm it landed at ~14m :D
<img width="1144" height="603" alt="image" src="https://github.com/user-attachments/assets/0accfe40-5e69-44c0-b744-a5163dcd951f" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] ci(python-unit): run the unit suite in parallel with pytest-xdist [superset]
Posted by "bito-code-review[bot] (via GitHub)" <gi...@apache.org>.
bito-code-review[bot] commented on PR #44178:
URL: https://github.com/apache/superset/pull/44178#issuecomment-5640575526
AI Code Review is in progress (usually takes 3 to 15 minutes unless it's a very large PR).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] ci(python-unit): run the unit suite in parallel with pytest-xdist [superset]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #44178:
URL: https://github.com/apache/superset/pull/44178#issuecomment-5640630401
## [Codecov](https://app.codecov.io/gh/apache/superset/pull/44178?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 57.45%. Comparing base ([`36361d9`](https://app.codecov.io/gh/apache/superset/commit/36361d9d0462b34114d4be0a87ca78d7a576f387?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6d741a4`](https://app.codecov.io/gh/apache/superset/commit/6d741a44dfa546cc8cb31651d8845aa2367eb8ae?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 3 commits behind head on master.
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #44178 +/- ##
==========================================
- Coverage 66.18% 57.45% -8.74%
==========================================
Files 2925 2925
Lines 172563 172563
Branches 40058 40058
==========================================
- Hits 114212 99142 -15070
- Misses 55970 72489 +16519
+ Partials 2381 932 -1449
```
| [Flag](https://app.codecov.io/gh/apache/superset/pull/44178/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hive](https://app.codecov.io/gh/apache/superset/pull/44178/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `37.34% <ø> (ø)` | |
| [presto](https://app.codecov.io/gh/apache/superset/pull/44178/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `39.22% <ø> (ø)` | |
| [python](https://app.codecov.io/gh/apache/superset/pull/44178/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `39.27% <ø> (-17.45%)` | :arrow_down: |
| [sqlite](https://app.codecov.io/gh/apache/superset/pull/44178/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/superset/pull/44178?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [PR] ci(python-unit): run the unit suite in parallel with pytest-xdist [superset]
Posted by "netlify[bot] (via GitHub)" <gi...@apache.org>.
netlify[bot] commented on PR #44178:
URL: https://github.com/apache/superset/pull/44178#issuecomment-5640967277
### <span aria-hidden="true">✅</span> Deploy Preview for *superset-docs-preview* ready!
| Name | Link |
|:-:|------------------------|
|<span aria-hidden="true">🔨</span> Latest commit | e365dc61250120e8074e54010b3d888d199d5d45 |
|<span aria-hidden="true">🔍</span> Latest deploy log | https://app.netlify.com/projects/superset-docs-preview/deploys/6aa47549692ceb000882553b |
|<span aria-hidden="true">😎</span> Deploy Preview | [https://deploy-preview-44178--superset-docs-preview.netlify.app](https://deploy-preview-44178--superset-docs-preview.netlify.app) |
|<span aria-hidden="true">📱</span> Preview on mobile | <details><summary> Toggle QR Code... </summary><br /><br /><br /><br />_Use your smartphone camera to open QR code link._</details> |
|<span aria-hidden="true">🤖</span> Make changes | [Run an agent on this branch](https://app.netlify.com/projects/superset-docs-preview/agent-runs#q1YqKMrPLShRslJS0lFKKkrMS84AspMzdUvzMkt0S1KLS4p1K1Iyi0uA0impBTn5lZ4pQAVmiYkm5qYmlmaWRsmpSQYGBhYWRqamxklKtQA) |
---
<!-- [superset-docs-preview Preview](https://deploy-preview-44178--superset-docs-preview.netlify.app) -->
_To edit notification comments on pull requests, go to your [Netlify project configuration](https://app.netlify.com/projects/superset-docs-preview/configuration/notifications#deploy-notifications)._
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org