You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@texera.apache.org by "aglinxinyuan (via GitHub)" <gi...@apache.org> on 2026/08/12 23:39:31 UTC
[PR] test(amber): cover the sync endpoint's result truncation engine [texera]
aglinxinyuan opened a new pull request, #7604:
URL: https://github.com/apache/texera/pull/7604
### What changes were proposed in this PR?
`collectOperatorResult` decides what an external caller actually receives from a synchronous run — how many rows come back, **which rows are dropped** when a result exceeds the character budget, and how individual cells are shortened. Roughly 180 lines, none of it covered.
The spec's own scaladoc claimed the region was unreachable. It is not: `ExecutionResultServiceSpec` has been creating real Iceberg-backed documents in amber's test scope for some time via `DocumentFactory` against the ambient postgres catalog. The same pattern reaches this engine with **no build change and no new dependency**. That paragraph is rewritten in this PR to say what is true.
Adds 10 tests to the existing spec:
| | Before | After |
|---|---|---|
| Lines | 230/406 (56.7%) | **340/406 (83.7%)** |
| Branches | 109 | 143 |
The +110 lines are the whole of 523–708, including both catch arms.
Covered: the empty-result short circuit, the visualization single-tuple path and its `__is_visualization__` flag, a first row that alone fills the budget, the sliding window that drops the middle of an oversized result, the second window that walks the tail once the front half is exactly full, per-cell truncation at both call sites, a disabled-warehouse refusal reaching the caller, and a registered URI with no document behind it degrading rather than throwing.
### Verification
14 mutations from the build pass, then three more run independently afterwards — the all-rows-fit `truncated` flag, the empty-result short circuit, and the sliding window evicting the newest row instead of the oldest. All red, production diff empty.
Two things worth stating plainly:
- **One mutation initially looked like a survivor and was not.** Relaxing the first-row bound from `>=` to `>` appeared to survive, but only because it had been applied alongside two others that together reproduced the same tuple. Re-run in isolation it fails. Batched mutation runs can manufacture false survivors, and this one nearly went into the report as a hole.
- **One genuine survivor is an equivalent mutant.** The front-loop bound `frontSize < halfLimit` can be relaxed to `<=` with no observable difference: tuple sizes are strictly positive, so the relaxed loop immediately fails its next fit check and falls into the same window over the same iterator position. Verified by running it. Recorded in a comment rather than chased, and the test that would have claimed it still kills two other mutations.
Also, three of my own first-draft mutations were malformed — renaming a private method just breaks compilation, which proves nothing. Re-done against the real code.
### Deliberately not included
- **`processedCount`** is written in all three walk paths and never read. The tests execute those lines but assert nothing about it, so deleting the variable stays a safe cleanup.
- **`validateWorkflow`** remains dead with zero call sites, still reported rather than tested.
- The residual 66 missed lines are live-engine paths — the `Observable.amb` wait, the console-error and results-ready arms, `shutdownPreviousExecution` — plus the dead method above.
No production file is touched.
### Any related issues, documentation, discussions?
Closes #7603
### How was this PR tested?
```
STORAGE_ICEBERG_CATALOG_TYPE=postgres sbt "WorkflowExecutionService/testOnly org.apache.texera.web.resource.SyncExecutionResourceSpec"
```
```
[info] Total number of tests run: 33
[info] Tests: succeeded 33, failed 0, canceled 0, ignored 0, pending 0
```
10 new on top of the existing 23. CI already provides what this needs — `build.yml` creates `texera_iceberg_catalog` and sets `STORAGE_ICEBERG_CATALOG_TYPE=postgres` for the unit job — so no workflow change. `Test/scalafmtCheck` and `Test/scalafix --check` both pass.
### Was this PR authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Opus 5)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test(amber): cover the sync endpoint's result truncation engine [texera]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #7604:
URL: https://github.com/apache/texera/pull/7604#issuecomment-5274146744
## [Codecov](https://app.codecov.io/gh/apache/texera/pull/7604?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 88.85%. Comparing base ([`321fe45`](https://app.codecov.io/gh/apache/texera/commit/321fe45505ecadf3aba184c2caee2b948c0d86dc?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`6c345af`](https://app.codecov.io/gh/apache/texera/commit/6c345afcdef93771179b7db5ce62af88dc39a791?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 1 commits behind head on main.
:white_check_mark: All tests successful. No failed tests found.
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## main #7604 +/- ##
============================================
+ Coverage 88.63% 88.85% +0.21%
- Complexity 4349 4362 +13
============================================
Files 1179 1179
Lines 46919 46919
Branches 5230 5230
============================================
+ Hits 41586 41688 +102
+ Misses 3574 3465 -109
- Partials 1759 1766 +7
```
| [Flag](https://app.codecov.io/gh/apache/texera/pull/7604/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | | *Carryforward flag |
|---|---|---|---|
| [access-control-service](https://app.codecov.io/gh/apache/texera/pull/7604/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `70.00% <ø> (ø)` | | Carriedforward from [321fe45](https://app.codecov.io/gh/apache/texera/commit/321fe45505ecadf3aba184c2caee2b948c0d86dc?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [agent-service](https://app.codecov.io/gh/apache/texera/pull/7604/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `98.62% <ø> (ø)` | | Carriedforward from [321fe45](https://app.codecov.io/gh/apache/texera/commit/321fe45505ecadf3aba184c2caee2b948c0d86dc?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [amber](https://app.codecov.io/gh/apache/texera/pull/7604/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `85.30% <ø> (+0.58%)` | :arrow_up: | |
| [computing-unit-managing-service](https://app.codecov.io/gh/apache/texera/pull/7604/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `60.38% <ø> (ø)` | | Carriedforward from [321fe45](https://app.codecov.io/gh/apache/texera/commit/321fe45505ecadf3aba184c2caee2b948c0d86dc?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [config-service](https://app.codecov.io/gh/apache/texera/pull/7604/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `77.31% <ø> (ø)` | | Carriedforward from [321fe45](https://app.codecov.io/gh/apache/texera/commit/321fe45505ecadf3aba184c2caee2b948c0d86dc?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [file-service](https://app.codecov.io/gh/apache/texera/pull/7604/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `68.90% <ø> (ø)` | | Carriedforward from [321fe45](https://app.codecov.io/gh/apache/texera/commit/321fe45505ecadf3aba184c2caee2b948c0d86dc?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [frontend](https://app.codecov.io/gh/apache/texera/pull/7604/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `89.98% <ø> (ø)` | | Carriedforward from [321fe45](https://app.codecov.io/gh/apache/texera/commit/321fe45505ecadf3aba184c2caee2b948c0d86dc?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [notebook-migration-service](https://app.codecov.io/gh/apache/texera/pull/7604/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `78.89% <ø> (ø)` | | Carriedforward from [321fe45](https://app.codecov.io/gh/apache/texera/commit/321fe45505ecadf3aba184c2caee2b948c0d86dc?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [pyamber](https://app.codecov.io/gh/apache/texera/pull/7604/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `97.52% <ø> (ø)` | | Carriedforward from [321fe45](https://app.codecov.io/gh/apache/texera/commit/321fe45505ecadf3aba184c2caee2b948c0d86dc?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [workflow-compiling-service](https://app.codecov.io/gh/apache/texera/pull/7604/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `57.89% <ø> (ø)` | | Carriedforward from [321fe45](https://app.codecov.io/gh/apache/texera/commit/321fe45505ecadf3aba184c2caee2b948c0d86dc?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
*This pull request uses carry forward flags. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to find out more.
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/texera/pull/7604?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test(amber): cover the sync endpoint's result truncation engine [texera]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #7604:
URL: https://github.com/apache/texera/pull/7604#issuecomment-5274116302
<!-- texera-benchmarks-comment -->
## ⚠️ Benchmark changes need a look
🟢 0 better · 🔴 5 worse · ⚪ 10 noise (<±5%) · 0 without baseline
> Compared against main `321fe45` benchmarked on **this same runner**, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.
[Dashboard](https://apache.github.io/texera/dev/bench/) · [Run](https://github.com/apache/texera/actions/runs/31651687008)
| | config | throughput | MB/s | latency | max Δ latest / 7d |
|---|---|---:|---:|---:|---:|
| ⚪ | bs=10 sw=10 sl=64 | 374 | 0.228 | 26,116/38,326/38,326 us | ⚪ within ±5% / 🔴 +135.0% |
| 🔴 | bs=100 sw=10 sl=64 | 778 | 0.475 | 126,830/165,107/165,107 us | 🔴 +8.4% / 🔴 +52.6% |
| ⚪ | bs=1000 sw=10 sl=64 | 916 | 0.559 | 1,088,715/1,193,088/1,193,088 us | ⚪ within ±5% / 🔴 +14.4% |
<details><summary>Baseline details</summary>
Latest main `321fe45` from same runner
| config | metric | PR | latest main | 7d avg | Δ latest | Δ 7d |
|---|---|---:|---:|---:|---:|---:|
| bs=10 sw=10 sl=64 | throughput | 374 tuples/sec | 375 tuples/sec | 775.33 tuples/sec | -0.3% | -51.8% |
| bs=10 sw=10 sl=64 | MB/s | 0.228 MB/s | 0.229 MB/s | 0.473 MB/s | -0.4% | -51.8% |
| bs=10 sw=10 sl=64 | p50 | 26,116 us | 26,013 us | 12,743 us | +0.4% | +105.0% |
| bs=10 sw=10 sl=64 | p95 | 38,326 us | 36,591 us | 16,310 us | +4.7% | +135.0% |
| bs=10 sw=10 sl=64 | p99 | 38,326 us | 36,591 us | 18,926 us | +4.7% | +102.5% |
| bs=100 sw=10 sl=64 | throughput | 778 tuples/sec | 825 tuples/sec | 1,001 tuples/sec | -5.7% | -22.3% |
| bs=100 sw=10 sl=64 | MB/s | 0.475 MB/s | 0.504 MB/s | 0.611 MB/s | -5.8% | -22.2% |
| bs=100 sw=10 sl=64 | p50 | 126,830 us | 117,113 us | 101,399 us | +8.3% | +25.1% |
| bs=100 sw=10 sl=64 | p95 | 165,107 us | 152,349 us | 108,206 us | +8.4% | +52.6% |
| bs=100 sw=10 sl=64 | p99 | 165,107 us | 152,349 us | 118,195 us | +8.4% | +39.7% |
| bs=1000 sw=10 sl=64 | throughput | 916 tuples/sec | 921 tuples/sec | 1,026 tuples/sec | -0.5% | -10.7% |
| bs=1000 sw=10 sl=64 | MB/s | 0.559 MB/s | 0.562 MB/s | 0.626 MB/s | -0.5% | -10.8% |
| bs=1000 sw=10 sl=64 | p50 | 1,088,715 us | 1,078,678 us | 996,304 us | +0.9% | +9.3% |
| bs=1000 sw=10 sl=64 | p95 | 1,193,088 us | 1,169,734 us | 1,042,531 us | +2.0% | +14.4% |
| bs=1000 sw=10 sl=64 | p99 | 1,193,088 us | 1,169,734 us | 1,074,934 us | +2.0% | +11.0% |
</details>
<details><summary>Raw CSV</summary>
```csv
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,534.66,200,128000,374,0.228,26116.35,38325.99,38325.99
1,100,10,64,20,2570.04,2000,1280000,778,0.475,126829.55,165107.11,165107.11
2,1000,10,64,20,21845.29,20000,12800000,916,0.559,1088715.09,1193087.52,1193087.52
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test(amber): cover the sync endpoint's result truncation engine [texera]
Posted by "aglinxinyuan (via GitHub)" <gi...@apache.org>.
aglinxinyuan merged PR #7604:
URL: https://github.com/apache/texera/pull/7604
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test(amber): cover the sync endpoint's result truncation engine [texera]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #7604:
URL: https://github.com/apache/texera/pull/7604#issuecomment-5274075691
<!-- texera-reviewer-suggestion -->
### Automated Reviewer Suggestions
Based on the `git blame` history of the changed files, we recommend the following reviewers:
- No candidates found from `git blame` history.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org