You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@texera.apache.org by "aglinxinyuan (via GitHub)" <gi...@apache.org> on 2026/08/14 09:35:57 UTC
[PR] test(amber): cover the runtime's serde bootstrap and worker startup [texera]
aglinxinyuan opened a new pull request, #7663:
URL: https://github.com/apache/texera/pull/7663
### What changes were proposed in this PR?
`AmberRuntime` had **no spec**. It owns the actor system the rest of amber leans on, the serialization extension every message goes through, and the scheduling helpers services use for recurring work.
Adds 15 tests, taking the file from **12.20% to 65.85% of lines** (5/41 → 27/41, branches 1/8 → 5/8).
Nothing needs a cluster: `startActorWorker(None)` builds its whole system with artery on port 0 in under 100 ms, makes no network call, and tears down again — so the worker startup path is drivable end to end.
Covered: the serde bootstrap and its memoization, extending an installed system rather than replacing it, both scheduling helpers including cancellation and the recurring delay, the seed-node URI and master address, the artery host binding, the config fallback chain, and the two child actors with their dead-letter subscription.
### Cross-suite safety
This matters more than the coverage number. amber sets neither `Test / fork` nor `Test / parallelExecution := false`, so every suite shares one JVM — and this file *is* the global state others depend on. The spec saves and restores `_actorSystem`, `_serde` and `AmberConfig.masterNodeAddr` by reflection, the way eight sibling specs already do, and terminates every system it starts.
Verified rather than asserted: run alongside `ClusterListenerSpec` and `SessionStateSpec`, **all 21 tests pass**. Review also drove twelve runtime-touching suites in one shared JVM with a sentinel system and address installed beforehand, and confirmed both were handed back untouched.
### Verification
21 mutations applied one at a time and reverted, production diff empty each time.
**Review then found three survivors, and two of the three are the same failure mode:**
| Survivor | Why it passed | Fix |
|---|---|---|
| the fallback chain dropping `pekkoConfig` | the assertion read `artery.transport == "tcp"`, which is pekko's **own** `reference.conf` default — it holds with `pekkoConfig` absent entirely | assert the kryo serializer binding, which only `cluster.conf` supplies |
| the serde bound to a different, same-named system | the test compared `system.name == "Amber"`, not identity | capture and compare with `eq` |
| `ClusterListener` swapped for `DeadLetterMonitorActor` at the same path | an `ActorRef` exposes no class, and nothing exchanges a message only one answers | **not fixed** — recorded in the spec instead; pinning it would mean adding a seam production does not need |
The first of those also exposed a measurement error worth stating: the build reported that mutation as *killed*, but the kill came from an earlier assertion in the same test failing first (`actor.provider == "cluster"`, which is genuine), so the transport line was never evaluated. **A red test is not evidence that the assertion you care about pins anything.**
One further mutation survived during the build and was resolved by deleting the assertion rather than keeping it: `cluster.conf` already sets `canonical.port = 0`, so the production override of it is unobservable.
### Deliberately not included
`getNodeIpAddress` and `startActorMaster` bind real network interfaces. One partial branch remains at the `mainNodeAddress.isDefined` guard, whose true arm needs that same address lookup.
No production file is touched.
### Any related issues, documentation, discussions?
Closes #7662
### How was this PR tested?
```
STORAGE_ICEBERG_CATALOG_TYPE=postgres sbt "WorkflowExecutionService/testOnly org.apache.texera.amber.engine.common.AmberRuntimeSpec"
```
```
[info] Total number of tests run: 15
[info] Tests: succeeded 15, failed 0, canceled 0, ignored 0, pending 0
```
`Test/scalafmtCheck` and `Test/scalafix --check` both pass.
### Was this PR authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Opus 5)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test(amber): cover the runtime's serde bootstrap and worker startup [texera]
Posted by "aglinxinyuan (via GitHub)" <gi...@apache.org>.
aglinxinyuan merged PR #7663:
URL: https://github.com/apache/texera/pull/7663
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test(amber): cover the runtime's serde bootstrap and worker startup [texera]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #7663:
URL: https://github.com/apache/texera/pull/7663#issuecomment-5291945909
<!-- texera-reviewer-suggestion -->
### Automated Reviewer Suggestions
Based on the `git blame` history of the changed files, we recommend the following reviewers:
- No candidates found from `git blame` history.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test(amber): cover the runtime's serde bootstrap and worker startup [texera]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #7663:
URL: https://github.com/apache/texera/pull/7663#issuecomment-5292143919
## [Codecov](https://app.codecov.io/gh/apache/texera/pull/7663?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 90.15%. Comparing base ([`8e80187`](https://app.codecov.io/gh/apache/texera/commit/8e80187a614fd8b82622c44594c6076eb69b80fe?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`f473f28`](https://app.codecov.io/gh/apache/texera/commit/f473f28218634f01e07994a2bc9910ad48ee0ebb?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:white_check_mark: All tests successful. No failed tests found.
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## main #7663 +/- ##
============================================
+ Coverage 90.13% 90.15% +0.02%
+ Complexity 4414 4410 -4
============================================
Files 1174 1174
Lines 46964 46964
Branches 5262 5262
============================================
+ Hits 42329 42342 +13
+ Misses 2877 2860 -17
- Partials 1758 1762 +4
```
| [Flag](https://app.codecov.io/gh/apache/texera/pull/7663/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | | *Carryforward flag |
|---|---|---|---|
| [access-control-service](https://app.codecov.io/gh/apache/texera/pull/7663/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `70.00% <ø> (ø)` | | Carriedforward from [8e80187](https://app.codecov.io/gh/apache/texera/commit/8e80187a614fd8b82622c44594c6076eb69b80fe?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [agent-service](https://app.codecov.io/gh/apache/texera/pull/7663/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `98.62% <ø> (ø)` | | Carriedforward from [8e80187](https://app.codecov.io/gh/apache/texera/commit/8e80187a614fd8b82622c44594c6076eb69b80fe?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [amber](https://app.codecov.io/gh/apache/texera/pull/7663/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `86.79% <ø> (+0.07%)` | :arrow_up: | |
| [computing-unit-managing-service](https://app.codecov.io/gh/apache/texera/pull/7663/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `72.46% <ø> (ø)` | | Carriedforward from [8e80187](https://app.codecov.io/gh/apache/texera/commit/8e80187a614fd8b82622c44594c6076eb69b80fe?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [config-service](https://app.codecov.io/gh/apache/texera/pull/7663/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `77.31% <ø> (ø)` | | Carriedforward from [8e80187](https://app.codecov.io/gh/apache/texera/commit/8e80187a614fd8b82622c44594c6076eb69b80fe?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [file-service](https://app.codecov.io/gh/apache/texera/pull/7663/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `68.90% <ø> (ø)` | | Carriedforward from [8e80187](https://app.codecov.io/gh/apache/texera/commit/8e80187a614fd8b82622c44594c6076eb69b80fe?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [frontend](https://app.codecov.io/gh/apache/texera/pull/7663/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `91.44% <ø> (ø)` | | Carriedforward from [8e80187](https://app.codecov.io/gh/apache/texera/commit/8e80187a614fd8b82622c44594c6076eb69b80fe?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [notebook-migration-service](https://app.codecov.io/gh/apache/texera/pull/7663/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `78.89% <ø> (ø)` | | Carriedforward from [8e80187](https://app.codecov.io/gh/apache/texera/commit/8e80187a614fd8b82622c44594c6076eb69b80fe?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [pyamber](https://app.codecov.io/gh/apache/texera/pull/7663/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `97.57% <ø> (ø)` | | Carriedforward from [8e80187](https://app.codecov.io/gh/apache/texera/commit/8e80187a614fd8b82622c44594c6076eb69b80fe?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [workflow-compiling-service](https://app.codecov.io/gh/apache/texera/pull/7663/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `57.89% <ø> (ø)` | | Carriedforward from [8e80187](https://app.codecov.io/gh/apache/texera/commit/8e80187a614fd8b82622c44594c6076eb69b80fe?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
*This pull request uses carry forward flags. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to find out more.
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/texera/pull/7663?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test(amber): cover the runtime's serde bootstrap and worker startup [texera]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #7663:
URL: https://github.com/apache/texera/pull/7663#issuecomment-5292141957
<!-- texera-benchmarks-comment -->
## ⚠️ Benchmark changes need a look
🟢 2 better · 🔴 6 worse · ⚪ 7 noise (<±5%) · 0 without baseline
> Compared against main `8e80187` benchmarked on **this same runner**, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.
[Dashboard](https://apache.github.io/texera/dev/bench/) · [Run](https://github.com/apache/texera/actions/runs/31788751687)
| | config | throughput | MB/s | latency | max Δ latest / 7d |
|---|---|---:|---:|---:|---:|
| 🔴 | bs=10 sw=10 sl=64 | 349 | 0.213 | 27,776/33,948/33,948 us | 🟢 -12.0% / 🔴 +118.1% |
| 🔴 | bs=100 sw=10 sl=64 | 749 | 0.457 | 131,353/162,507/162,507 us | 🔴 +8.6% / 🔴 +51.0% |
| ⚪ | bs=1000 sw=10 sl=64 | 867 | 0.529 | 1,147,592/1,270,796/1,270,796 us | ⚪ within ±5% / 🔴 +22.6% |
<details><summary>Baseline details</summary>
Latest main `8e80187` from same runner
| config | metric | PR | latest main | 7d avg | Δ latest | Δ 7d |
|---|---|---:|---:|---:|---:|---:|
| bs=10 sw=10 sl=64 | throughput | 349 tuples/sec | 375 tuples/sec | 774.89 tuples/sec | -6.9% | -55.0% |
| bs=10 sw=10 sl=64 | MB/s | 0.213 MB/s | 0.229 MB/s | 0.473 MB/s | -7.0% | -55.0% |
| bs=10 sw=10 sl=64 | p50 | 27,776 us | 26,148 us | 12,738 us | +6.2% | +118.1% |
| bs=10 sw=10 sl=64 | p95 | 33,948 us | 38,574 us | 16,348 us | -12.0% | +107.7% |
| bs=10 sw=10 sl=64 | p99 | 33,948 us | 38,574 us | 18,848 us | -12.0% | +80.1% |
| bs=100 sw=10 sl=64 | throughput | 749 tuples/sec | 791 tuples/sec | 1,005 tuples/sec | -5.3% | -25.4% |
| bs=100 sw=10 sl=64 | MB/s | 0.457 MB/s | 0.483 MB/s | 0.613 MB/s | -5.4% | -25.5% |
| bs=100 sw=10 sl=64 | p50 | 131,353 us | 120,958 us | 100,970 us | +8.6% | +30.1% |
| bs=100 sw=10 sl=64 | p95 | 162,507 us | 166,345 us | 107,605 us | -2.3% | +51.0% |
| bs=100 sw=10 sl=64 | p99 | 162,507 us | 166,345 us | 116,429 us | -2.3% | +39.6% |
| bs=1000 sw=10 sl=64 | throughput | 867 tuples/sec | 883 tuples/sec | 1,030 tuples/sec | -1.8% | -15.9% |
| bs=1000 sw=10 sl=64 | MB/s | 0.529 MB/s | 0.539 MB/s | 0.629 MB/s | -1.9% | -15.9% |
| bs=1000 sw=10 sl=64 | p50 | 1,147,592 us | 1,122,125 us | 991,433 us | +2.3% | +15.8% |
| bs=1000 sw=10 sl=64 | p95 | 1,270,796 us | 1,264,503 us | 1,036,668 us | +0.5% | +22.6% |
| bs=1000 sw=10 sl=64 | p99 | 1,270,796 us | 1,264,503 us | 1,070,470 us | +0.5% | +18.7% |
</details>
<details><summary>Raw CSV</summary>
```csv
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,573.34,200,128000,349,0.213,27776.44,33948.01,33948.01
1,100,10,64,20,2669.00,2000,1280000,749,0.457,131352.95,162507.42,162507.42
2,1000,10,64,20,23080.18,20000,12800000,867,0.529,1147592.08,1270795.80,1270795.80
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org