You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@texera.apache.org by "eugenegujing (via GitHub)" <gi...@apache.org> on 2026/08/13 00:11:58 UTC
[PR] test(amber): cover the worker StartChannelHandler [texera]
eugenegujing opened a new pull request, #7609:
URL: https://github.com/apache/texera/pull/7609
### What changes were proposed in this PR?
This PR adds `StartChannelHandlerSpec`, the first unit coverage for `amber/src/main/scala/org/apache/texera/amber/engine/architecture/worker/promisehandlers/StartChannelHandler.scala`. The handler was last changed by #6913, which adjusted the boundary-state emission but added only integration coverage (`LoopIntegrationSpec`) and Python-side unit tests, so the Scala handler's state-emission and exception paths were unasserted at the unit level.
No production code is changed; this is a test-only PR.
The spec drives a real `DataProcessor` and asserts on the worker's outgoing messages, so the marker and the emitted state are checked as the wire payloads a downstream worker would actually receive rather than as mocked calls. The 15 tests pin the handler's three steps and the order between them:
- the input port is resolved from the channel the ECM arrived on, and that port is what `produceStateOnStart` receives;
- the unaligned START_CHANNEL marker reaches every data channel and no control channel, after the pending output is flushed;
- the operator's boundary state is emitted to every data channel with the "no loop" envelope, an empty-but-present state is still emitted, and no state is emitted when the operator produces none;
- every marker precedes every state, and the marker still goes out first when the operator throws;
- a sink worker with no data channels emits nothing at all yet still replies successfully, and a second invocation repeats the whole sequence because the handler has no once-only guard;
- an operator exception, an operator `Error`, and a failure raised inside `emitState` are all swallowed, reported through `handleExecutorException`, and answered with a successful reply, while a `ControlThrowable` escapes and an unassigned port fails the RPC outright.
Four of these record current behavior that a reader may find surprising, and each says so in a comment rather than implying endorsement. `ErrorUtils.safely` swallows `java.lang.Error` because its `OutOfMemoryError` guard is a commented-out line, the port resolution sits before the `try` so an unassigned port escapes as an RPC failure instead of being reported like every other failure one line later, and the RPC replies successfully even after the operator fails because the failure is surfaced out of band. If any of these is later changed deliberately, the corresponding test turns red and forces that decision to be explicit, which is the point of pinning them.
### Any related issues, documentation, discussions?
Closes #7606
### How was this PR tested?
The 15 new tests were run locally, together with the three pre-existing specs in the same package and the wider worker package:
```
sbt "WorkflowExecutionService/testOnly org.apache.texera.amber.engine.architecture.worker.promisehandlers.StartChannelHandlerSpec"
-> Tests: succeeded 15, failed 0
sbt "WorkflowExecutionService/testOnly org.apache.texera.amber.engine.architecture.worker.promisehandlers.*"
-> Suites: completed 4, Tests: succeeded 31, failed 0
sbt "WorkflowExecutionService/testOnly org.apache.texera.amber.engine.architecture.worker.*"
-> Suites: completed 14, Tests: succeeded 112, failed 0
sbt "WorkflowExecutionService/Test/scalafmtCheck"
-> success
```
Both positive and negative directions are covered, along with the empty and boundary cases: a produced state and no produced state, an empty-but-present state, zero downstream data channels, a never-registered input channel, and a repeated invocation.
The assertions were mutation-checked rather than assumed to be meaningful. Fifteen mutations were applied to the production code one at a time and every one of them turned the spec red, including flipping `NO_ALIGNMENT` to `PORT_ALIGNMENT`, swapping `METHOD_START_CHANNEL` for `METHOD_END_CHANNEL`, inverting `isDefined`, deleting the `emitState` call, passing a constant port to `produceStateOnStart`, sending the marker after the `try` block instead of before it, dropping the `handleExecutorException` call, restricting the marker to the first data channel only, deleting the `outputManager.flush()` that precedes the marker, hoisting `emitState` out of the `try`, adding an idempotence guard, and pausing with a different `PauseType`. The production files were restored and verified byte-identical to `HEAD` afterwards.
### Was this PR authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Claude Opus 5)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test(amber): cover the worker StartChannelHandler [texera]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #7609:
URL: https://github.com/apache/texera/pull/7609#issuecomment-5274319591
<!-- texera-benchmarks-comment -->
## ⚠️ Benchmark changes need a look
🟢 2 better · 🔴 7 worse · ⚪ 6 noise (<±5%) · 0 without baseline
> Compared against main `c35bdb1` benchmarked on **this same runner**, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.
[Dashboard](https://apache.github.io/texera/dev/bench/) · [Run](https://github.com/apache/texera/actions/runs/31653608990)
| | config | throughput | MB/s | latency | max Δ latest / 7d |
|---|---|---:|---:|---:|---:|
| 🔴 | bs=10 sw=10 sl=64 | 415 | 0.253 | 24,144/29,247/29,247 us | 🔴 +14.4% / 🔴 +89.5% |
| 🔴 | bs=100 sw=10 sl=64 | 809 | 0.494 | 121,207/157,506/157,506 us | 🔴 +16.2% / 🔴 +45.6% |
| ⚪ | bs=1000 sw=10 sl=64 | 933 | 0.569 | 1,075,125/1,115,024/1,115,024 us | ⚪ within ±5% / 🔴 -9.2% |
<details><summary>Baseline details</summary>
Latest main `c35bdb1` from same runner
| config | metric | PR | latest main | 7d avg | Δ latest | Δ 7d |
|---|---|---:|---:|---:|---:|---:|
| bs=10 sw=10 sl=64 | throughput | 415 tuples/sec | 458 tuples/sec | 775.33 tuples/sec | -9.4% | -46.5% |
| bs=10 sw=10 sl=64 | MB/s | 0.253 MB/s | 0.28 MB/s | 0.473 MB/s | -9.6% | -46.5% |
| bs=10 sw=10 sl=64 | p50 | 24,144 us | 21,097 us | 12,743 us | +14.4% | +89.5% |
| bs=10 sw=10 sl=64 | p95 | 29,247 us | 32,651 us | 16,310 us | -10.4% | +79.3% |
| bs=10 sw=10 sl=64 | p99 | 29,247 us | 32,651 us | 18,926 us | -10.4% | +54.5% |
| bs=100 sw=10 sl=64 | throughput | 809 tuples/sec | 853 tuples/sec | 1,001 tuples/sec | -5.2% | -19.2% |
| bs=100 sw=10 sl=64 | MB/s | 0.494 MB/s | 0.521 MB/s | 0.611 MB/s | -5.2% | -19.1% |
| bs=100 sw=10 sl=64 | p50 | 121,207 us | 116,089 us | 101,399 us | +4.4% | +19.5% |
| bs=100 sw=10 sl=64 | p95 | 157,506 us | 135,585 us | 108,206 us | +16.2% | +45.6% |
| bs=100 sw=10 sl=64 | p99 | 157,506 us | 135,585 us | 118,195 us | +16.2% | +33.3% |
| bs=1000 sw=10 sl=64 | throughput | 933 tuples/sec | 941 tuples/sec | 1,026 tuples/sec | -0.9% | -9.1% |
| bs=1000 sw=10 sl=64 | MB/s | 0.569 MB/s | 0.575 MB/s | 0.626 MB/s | -1.0% | -9.2% |
| bs=1000 sw=10 sl=64 | p50 | 1,075,125 us | 1,061,678 us | 996,304 us | +1.3% | +7.9% |
| bs=1000 sw=10 sl=64 | p95 | 1,115,024 us | 1,113,279 us | 1,042,531 us | +0.2% | +7.0% |
| bs=1000 sw=10 sl=64 | p99 | 1,115,024 us | 1,113,279 us | 1,074,934 us | +0.2% | +3.7% |
</details>
<details><summary>Raw CSV</summary>
```csv
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,481.67,200,128000,415,0.253,24143.68,29247.33,29247.33
1,100,10,64,20,2473.04,2000,1280000,809,0.494,121207.05,157505.66,157505.66
2,1000,10,64,20,21443.88,20000,12800000,933,0.569,1075124.78,1115024.33,1115024.33
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test(amber): cover the worker StartChannelHandler [texera]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #7609:
URL: https://github.com/apache/texera/pull/7609#issuecomment-5274342210
## [Codecov](https://app.codecov.io/gh/apache/texera/pull/7609?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 88.54%. Comparing base ([`cb6e5c6`](https://app.codecov.io/gh/apache/texera/commit/cb6e5c649978215188ea92937ec583be9990c49e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`b73de57`](https://app.codecov.io/gh/apache/texera/commit/b73de57e84c66b9fb203299ba746a034a936cef8?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 12 commits behind head on main.
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## main #7609 +/- ##
============================================
+ Coverage 87.87% 88.54% +0.67%
- Complexity 4275 4340 +65
============================================
Files 1176 1178 +2
Lines 46890 46909 +19
Branches 5228 5229 +1
============================================
+ Hits 41205 41536 +331
+ Misses 3961 3612 -349
- Partials 1724 1761 +37
```
| [Flag](https://app.codecov.io/gh/apache/texera/pull/7609/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | | *Carryforward flag |
|---|---|---|---|
| [access-control-service](https://app.codecov.io/gh/apache/texera/pull/7609/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `70.00% <ø> (ø)` | | Carriedforward from [cb6e5c6](https://app.codecov.io/gh/apache/texera/commit/cb6e5c649978215188ea92937ec583be9990c49e?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [agent-service](https://app.codecov.io/gh/apache/texera/pull/7609/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `98.62% <ø> (ø)` | | Carriedforward from [cb6e5c6](https://app.codecov.io/gh/apache/texera/commit/cb6e5c649978215188ea92937ec583be9990c49e?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [amber](https://app.codecov.io/gh/apache/texera/pull/7609/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `84.69% <ø> (+1.81%)` | :arrow_up: | |
| [computing-unit-managing-service](https://app.codecov.io/gh/apache/texera/pull/7609/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `60.38% <ø> (ø)` | | Carriedforward from [cb6e5c6](https://app.codecov.io/gh/apache/texera/commit/cb6e5c649978215188ea92937ec583be9990c49e?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [config-service](https://app.codecov.io/gh/apache/texera/pull/7609/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `65.97% <ø> (ø)` | | Carriedforward from [cb6e5c6](https://app.codecov.io/gh/apache/texera/commit/cb6e5c649978215188ea92937ec583be9990c49e?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [file-service](https://app.codecov.io/gh/apache/texera/pull/7609/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `69.05% <ø> (ø)` | | Carriedforward from [cb6e5c6](https://app.codecov.io/gh/apache/texera/commit/cb6e5c649978215188ea92937ec583be9990c49e?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [frontend](https://app.codecov.io/gh/apache/texera/pull/7609/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `89.95% <ø> (ø)` | | Carriedforward from [cb6e5c6](https://app.codecov.io/gh/apache/texera/commit/cb6e5c649978215188ea92937ec583be9990c49e?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [notebook-migration-service](https://app.codecov.io/gh/apache/texera/pull/7609/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `78.89% <ø> (ø)` | | Carriedforward from [cb6e5c6](https://app.codecov.io/gh/apache/texera/commit/cb6e5c649978215188ea92937ec583be9990c49e?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [pyamber](https://app.codecov.io/gh/apache/texera/pull/7609/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `97.47% <ø> (ø)` | | Carriedforward from [cb6e5c6](https://app.codecov.io/gh/apache/texera/commit/cb6e5c649978215188ea92937ec583be9990c49e?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [workflow-compiling-service](https://app.codecov.io/gh/apache/texera/pull/7609/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `26.31% <ø> (ø)` | | Carriedforward from [cb6e5c6](https://app.codecov.io/gh/apache/texera/commit/cb6e5c649978215188ea92937ec583be9990c49e?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
*This pull request uses carry forward flags. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to find out more.
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/texera/pull/7609?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test(amber): cover the worker StartChannelHandler [texera]
Posted by "aglinxinyuan (via GitHub)" <gi...@apache.org>.
aglinxinyuan merged PR #7609:
URL: https://github.com/apache/texera/pull/7609
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test(amber): cover the worker StartChannelHandler [texera]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #7609:
URL: https://github.com/apache/texera/pull/7609#issuecomment-5274280584
<!-- texera-reviewer-suggestion -->
### Automated Reviewer Suggestions
Based on the `git blame` history of the changed files, we recommend the following reviewers:
- No candidates found from `git blame` history.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org