You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@texera.apache.org by "mengw15 (via GitHub)" <gi...@apache.org> on 2026/08/25 19:59:44 UTC
[I] frontend: the macOS test leg dies on a V8 heap OOM — the configured 8 GB never reaches the Vitest workers [texera]
mengw15 opened a new issue, #7975:
URL: https://github.com/apache/texera/issues/7975
### Task Summary
`build / frontend (macos-latest)` fails on a **V8 heap exhaustion inside a Vitest worker**, not on any test. The
worker is killed mid-run, so the leg reports `200 passed (201)` — one spec file never reports at all — with zero
failed assertions:
```
[22268:0xcbe80c000] 408527 ms: Mark-Compact 2031.6 (2049.3) -> 2026.0 (2052.3) MB, pooled: 0 MB,
7444.75 / 0.00 ms (average mu = 0.211, current mu = 0.027) allocation failure
FATAL ERROR: Ineffective mark-compacts near heap limit - Allocation failed - JavaScript heap out of memory
Test Files 200 passed (201)
Error: [vitest-pool]: Worker forks emitted error.
Caused by: Error: Worker exited unexpectedly
```
It is currently red on **five open PRs** (#7917, #7961, #7966, #7967, #7968 — same signature in all five) and on
**5 of the last 10 pushes to `main`**. The spec that dies is different every time (`search-bar.component`,
`codearea-custom-template.component`, ...), because it depends only on which files a given worker was handed.
### Cause: the configured heap limit never reaches the worker
`frontend/package.json` asks for 8 GB:
```json
"test:ci": "node --max-old-space-size=8192 ./node_modules/nx/dist/bin/nx.js test ..."
```
That flag applies to the parent process only. When Vitest (4.1.10) spawns its fork-pool workers it rebuilds
`execArgv` from scratch and keeps **only profiling flags**:
```js
execArgv: [
...process.execArgv.filter(a => a.startsWith("--cpu-prof")
|| a.startsWith("--heap-prof")
|| a.startsWith("--diagnostic-dir")),
"--experimental-import-meta-resolve",
...
]
```
`--max-old-space-size=8192` is dropped, so every worker runs at V8's **default** heap limit. Reproduced locally:
| process | heap limit |
|---|---|
| parent, `node --max-old-space-size=8192` | 8288 MB |
| child forked with `execArgv: []` | **4192 MB** (the default — parent's flag has no effect) |
| child, with `NODE_OPTIONS=--max-old-space-size=6144` | **6240 MB** |
`NODE_OPTIONS` is an environment variable and *is* inherited by child processes; the command-line flag is not.
The macOS image (`macos-26-arm64`) defaults to ~2.05 GB per worker, which is exactly the ceiling in the crash
log; the ubuntu and windows images have more headroom and have never hit it. That is why this is a macOS-only
failure even though the suite is identical everywhere.
### Why now, and its relation to #7713
The suite is 201 unit spec files / 5069 tests and still growing. Running the 56 `src/app/dashboard` specs on a
single worker with `logHeapUsage: true` shows this is **not a monotonic leak** — the heap returns to a
130–275 MB baseline between files — but individual specs peak high:
| peak | spec |
|---|---|
| 712 MB | `search.component.spec.ts` |
| 586 MB | `workflow-execution-history.component.spec.ts` |
| 575 MB | `dataset-detail.component.spec.ts` (195 tests) |
| 379 MB | `filters.component.spec.ts` |
Those are the same specs #7713 listed as timing out on macOS a week ago. #7713 read the symptom as runner
variance against thin timeout margins and raised `testTimeout`/`hookTimeout`; what was actually happening is
that a worker near the 2 GB ceiling stalls in stop-the-world GC — single mark-compact pauses of **7.4 s** and
**29.6 s** appear in these logs — long enough to blow a `beforeEach`. #7966's one "real" failure this week is
that same effect: `filters.component.spec.ts` failed with `Hook timed out in 30000ms` inside
`TestBed.createComponent`, in the same run as the OOM.
Raising the timeouts bought about a week. The heap now exhausts outright, so the worker dies instead of stalling.
### An amplifier: the frontend matrix has no `fail-fast: false`
`build.yml` sets `fail-fast: false` on four other matrices but not on `frontend`, so when the macOS leg dies the
still-running legs are cancelled. On #7917 the windows leg was reported red purely for this reason — its own run
was `Test Files 201 passed (201)`, `Tests 5068 passed | 1 skipped`, then `The operation was canceled`.
### Proposed fix
1. Give the workers the memory the repo already intended, via an inherited variable, on the `frontend` job:
```yaml
env:
NODE_OPTIONS: --max-old-space-size=3072
```
The macOS leg runs 2 concurrent forks (two worker pids appear in the logs), so 3 GB each keeps the total
within the image's RAM. `poolOptions.forks.execArgv` in `vitest.config.ts` is an equivalent, more targeted
alternative that would also apply to local runs.
2. Add `fail-fast: false` to the `frontend` matrix, matching the other four matrices in the same workflow, so one
dead leg stops cancelling the others.
3. Follow-up, not a blocker: the four specs above account for a disproportionate share of the peak. Worth a look
once the leg is green.
### Task Type
- [ ] Refactor / Cleanup
- [x] DevOps / Deployment / CI
- [ ] Testing / QA
- [ ] Documentation
- [ ] Performance
- [ ] Other
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] frontend: the macOS test leg dies on a V8 heap OOM — the configured 8 GB never reaches the Vitest workers [texera]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #7975:
URL: https://github.com/apache/texera/issues/7975#issuecomment-5415995496
<!-- texera:template-compliance -->
👋 Thanks for opening this issue, @mengw15!
It looks like the issue description doesn't quite follow our template yet:
- This issue doesn't appear to use one of our templates (Bug, Feature, or Task). Please open it using a template so the required details are captured.
Filling out the template helps reviewers understand and triage your contribution faster. Please edit the description to complete it. This message will disappear automatically once the template is followed.
You can find the template prompts by editing the description, or see [CONTRIBUTING.md](https://github.com/apache/texera/blob/main/CONTRIBUTING.md) for the full contribution flow.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] frontend: the macOS test leg dies on a V8 heap OOM — the Vitest workers' heap is never configured [texera]
Posted by "mengw15 (via GitHub)" <gi...@apache.org>.
mengw15 closed issue #7975: frontend: the macOS test leg dies on a V8 heap OOM — the Vitest workers' heap is never configured
URL: https://github.com/apache/texera/issues/7975
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org