You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@texera.apache.org by GitBox <gi...@apache.org> on 2026/06/30 04:25:37 UTC
[GH] (texera/gh-readonly-queue/main/pr-6024-3f877734610ec4ae077bf87ce2c653d7ff78a250): Workflow run "Required Checks" is working again!
The GitHub Actions job "Required Checks" on texera.git/gh-readonly-queue/main/pr-6024-3f877734610ec4ae077bf87ce2c653d7ff78a250 has succeeded.
Run started by GitHub user Yicong-Huang (triggered by Yicong-Huang).
Head commit for run:
1a584332af50eb21e715df91a108a3d7c81736cc / Yicong Huang <17...@users.noreply.github.com>
test(amber): make PveResourceSpec hermetic via process-runner seam (#6024)
### What changes were proposed in this PR?
`PveResourceSpec` runs in the `amber` unit CI job
(`AMBER_TEST_FILTER=skip-integration`) but is not tagged
`@IntegrationTest`, so every run shelled out to **real `pip` over the
network**:
- each `createNewPve` ran `pip install -r requirements.txt` (the full
amber dependency set), and
- the lazy `resolveSystemPackages()` installed `requirements.txt` into a
throwaway venv, then `pip freeze`
— roughly 6 full installs per spec run. That made a PR-blocking unit
spec **network-dependent (flaky)** and **slow**.
This PR funnels every child process in `PveManager` (venv creation, pip
install / uninstall / freeze) through a single injectable seam:
```scala
private[pythonvirtualenvironment] var runProcess: ProcessRunner =
(command, env, logger) => Process(command, None, env: _*).!(logger)
```
Production wiring is unchanged (the default runner executes the command
for real). `PveResourceSpec` swaps in a **ScalaMock `mockFunction`**
whose handler fabricates the `<venv>/bin/{python,pip}` layout, emits the
resolved system set on `freeze`, and returns a configurable exit code —
so the spec is **fully hermetic: no venv, no pip, no network**.
`PveManager` still owns the metadata files (`user-packages.txt`) and
queue messages, so that logic stays under test.
Because failures are now cheap to trigger, this also adds negative
coverage that was previously impractical: venv-create failure,
requirements-install failure, user-package install failure, and
system-package rejection (using `pyarrow`, a hard amber dependency).
| Before | After |
| --- | --- |
| ~6 real `pip install` per run, hits PyPI | 0 network calls |
| flaky on network hiccups, ~minutes | deterministic, ~8s |
| only happy-path assertions | + 4 negative/failure cases |
### Any related issues, documentation, discussions?
Closes #6023
### How was this PR tested?
`PveManager` is an `object`, so the test points the shared `runProcess`
at the ScalaMock `mockFunction` in `beforeAll` and restores the real
runner in `afterAll`. ScalaMock expectations are per-test, so
`expectProcessCalls()` (an `anyNumberOfTimes` handler) is invoked at the
top of each test that exercises a process. Run with JDK 17:
```bash
STORAGE_JDBC_USERNAME=texera STORAGE_JDBC_PASSWORD=password \
sbt "WorkflowExecutionService/testOnly org.apache.texera.web.resource.pythonvirtualenvironment.PveResourceSpec"
# -> Tests: succeeded 25, failed 0, in ~8s
```
Also green:
```bash
sbt scalafmtCheckAll
sbt "WorkflowExecutionService/scalafixAll --check"
```
### Was this PR authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Claude Opus 4.8)
Report URL: https://github.com/apache/texera/actions/runs/28419874442
With regards,
GitHub Actions via GitBox