You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@texera.apache.org by "eugenegujing (via GitHub)" <gi...@apache.org> on 2026/08/07 00:16:31 UTC
[PR] test(workflow-operator): cover the ScanSourceOpDesc base descriptor [texera]
eugenegujing opened a new pull request, #7373:
URL: https://github.com/apache/texera/pull/7373
### What changes were proposed in this PR?
One new test file, no source changes: `ScanSourceOpDescSpec` (18 tests), a characterization spec for the abstract base of the six file-scan descriptors, which had no direct test coverage. It pins the `operatorInfo` naming formula (the "Unknown" fallback, the empty-type "File Scan" form, the a/an article choice) and the default output port, the display names not asserted anywhere else (both CSV scans render as "CSV File Scan"), the property defaults, `setResolvedFileName`'s `toASCIIString` percent-encoding, `fileResolved()`, reflection-based equals/hashCode (fresh instances differ by the random-UUID operator id; the `@JsonIgnore`'d `INFER_READ_LIMIT` still participates in equality), and the serialized JSON shape (`@JsonIgnore` fields stay off the wire, unset `Option` fields are omitted).
The abstract class is instantiated through private stub subclasses, following the existing `SklearnModelOpDescSpec` / `SQLSourceOpDescSpec` pattern.
### Any related issues, documentation, discussions?
Closes #7369.
### How was this PR tested?
- `sbt "WorkflowOperator/testOnly org.apache.texera.amber.operator.source.scan.ScanSourceOpDescSpec"` — 18/18 pass.
- `sbt "WorkflowOperator/testOnly org.apache.texera.amber.operator.source.scan.*"` — 137/137 pass, no regressions in the scan package.
- `scalafmtCheck` passes.
- Mutation-checked: flipping the vowel set in `operatorInfo` or the output port's `blocking` flag each makes exactly the pinning test fail; both reverted.
### Was this PR authored or co-authored using generative AI tooling?
Co-authored by: Claude Code (Claude Fable 5)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test(workflow-operator): cover the ScanSourceOpDesc base descriptor [texera]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #7373:
URL: https://github.com/apache/texera/pull/7373#issuecomment-5210285150
<!-- texera-benchmarks-comment -->
## ⚠️ Benchmark changes need a look
🟢 6 better · 🔴 3 worse · ⚪ 6 noise (<±5%) · 0 without baseline
> Compared against main `18a277c` benchmarked on **this same runner**, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.
[Dashboard](https://apache.github.io/texera/dev/bench/) · [Run](https://github.com/apache/texera/actions/runs/31134071306)
| | config | throughput | MB/s | latency | max Δ latest / 7d |
|---|---|---:|---:|---:|---:|
| 🔴 | bs=10 sw=10 sl=64 | 614 | 0.375 | 15,741/21,114/21,114 us | 🟢 -17.5% / 🔴 +31.4% |
| 🟢 | bs=100 sw=10 sl=64 | 1,289 | 0.787 | 77,943/88,968/88,968 us | 🟢 -32.0% / 🟢 +34.6% |
| 🔴 | bs=1000 sw=10 sl=64 | 1,456 | 0.889 | 679,371/820,279/820,279 us | 🔴 +16.4% / 🟢 +47.3% |
<details><summary>Baseline details</summary>
Latest main `18a277c` from same runner
| config | metric | PR | latest main | 7d avg | Δ latest | Δ 7d |
|---|---|---:|---:|---:|---:|---:|
| bs=10 sw=10 sl=64 | throughput | 614 tuples/sec | 640 tuples/sec | 757.88 tuples/sec | -4.1% | -19.0% |
| bs=10 sw=10 sl=64 | MB/s | 0.375 MB/s | 0.391 MB/s | 0.463 MB/s | -4.1% | -18.9% |
| bs=10 sw=10 sl=64 | p50 | 15,741 us | 13,576 us | 12,904 us | +15.9% | +22.0% |
| bs=10 sw=10 sl=64 | p95 | 21,114 us | 25,592 us | 16,065 us | -17.5% | +31.4% |
| bs=10 sw=10 sl=64 | p99 | 21,114 us | 25,592 us | 19,384 us | -17.5% | +8.9% |
| bs=100 sw=10 sl=64 | throughput | 1,289 tuples/sec | 1,213 tuples/sec | 957.87 tuples/sec | +6.3% | +34.6% |
| bs=100 sw=10 sl=64 | MB/s | 0.787 MB/s | 0.74 MB/s | 0.585 MB/s | +6.4% | +34.6% |
| bs=100 sw=10 sl=64 | p50 | 77,943 us | 75,778 us | 104,502 us | +2.9% | -25.4% |
| bs=100 sw=10 sl=64 | p95 | 88,968 us | 130,850 us | 110,735 us | -32.0% | -19.7% |
| bs=100 sw=10 sl=64 | p99 | 88,968 us | 130,850 us | 120,386 us | -32.0% | -26.1% |
| bs=1000 sw=10 sl=64 | throughput | 1,456 tuples/sec | 1,523 tuples/sec | 988.88 tuples/sec | -4.4% | +47.2% |
| bs=1000 sw=10 sl=64 | MB/s | 0.889 MB/s | 0.93 MB/s | 0.604 MB/s | -4.4% | +47.3% |
| bs=1000 sw=10 sl=64 | p50 | 679,371 us | 652,911 us | 1,018,589 us | +4.1% | -33.3% |
| bs=1000 sw=10 sl=64 | p95 | 820,279 us | 704,624 us | 1,056,001 us | +16.4% | -22.3% |
| bs=1000 sw=10 sl=64 | p99 | 820,279 us | 704,624 us | 1,087,982 us | +16.4% | -24.6% |
</details>
<details><summary>Raw CSV</summary>
```csv
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,325.58,200,128000,614,0.375,15741.46,21114.09,21114.09
1,100,10,64,20,1551.24,2000,1280000,1289,0.787,77943.31,88967.67,88967.67
2,1000,10,64,20,13737.37,20000,12800000,1456,0.889,679371.36,820278.58,820278.58
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test(workflow-operator): cover the ScanSourceOpDesc base descriptor [texera]
Posted by "aglinxinyuan (via GitHub)" <gi...@apache.org>.
aglinxinyuan merged PR #7373:
URL: https://github.com/apache/texera/pull/7373
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test(workflow-operator): cover the ScanSourceOpDesc base descriptor [texera]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #7373:
URL: https://github.com/apache/texera/pull/7373#issuecomment-5210272015
## [Codecov](https://app.codecov.io/gh/apache/texera/pull/7373?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 83.72%. Comparing base ([`cc83087`](https://app.codecov.io/gh/apache/texera/commit/cc8308740361324f7f129657dad6dc395abeeb1c?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`485d482`](https://app.codecov.io/gh/apache/texera/commit/485d48247f794fd50aee53071fb002773dd7d5fd?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 4 commits behind head on main.
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## main #7373 +/- ##
=========================================
Coverage 83.72% 83.72%
Complexity 4137 4137
=========================================
Files 1169 1169
Lines 46520 46520
Branches 5179 5179
=========================================
Hits 38947 38947
Misses 5860 5860
Partials 1713 1713
```
| [Flag](https://app.codecov.io/gh/apache/texera/pull/7373/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | | *Carryforward flag |
|---|---|---|---|
| [access-control-service](https://app.codecov.io/gh/apache/texera/pull/7373/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `70.00% <ø> (ø)` | | |
| [agent-service](https://app.codecov.io/gh/apache/texera/pull/7373/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `83.65% <ø> (ø)` | | Carriedforward from [cc83087](https://app.codecov.io/gh/apache/texera/commit/cc8308740361324f7f129657dad6dc395abeeb1c?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [amber](https://app.codecov.io/gh/apache/texera/pull/7373/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `80.66% <ø> (ø)` | | Carriedforward from [cc83087](https://app.codecov.io/gh/apache/texera/commit/cc8308740361324f7f129657dad6dc395abeeb1c?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [computing-unit-managing-service](https://app.codecov.io/gh/apache/texera/pull/7373/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `50.72% <ø> (ø)` | | Carriedforward from [cc83087](https://app.codecov.io/gh/apache/texera/commit/cc8308740361324f7f129657dad6dc395abeeb1c?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [config-service](https://app.codecov.io/gh/apache/texera/pull/7373/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `65.97% <ø> (ø)` | | |
| [file-service](https://app.codecov.io/gh/apache/texera/pull/7373/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `69.05% <ø> (ø)` | | Carriedforward from [cc83087](https://app.codecov.io/gh/apache/texera/commit/cc8308740361324f7f129657dad6dc395abeeb1c?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [frontend](https://app.codecov.io/gh/apache/texera/pull/7373/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `84.71% <ø> (ø)` | | Carriedforward from [cc83087](https://app.codecov.io/gh/apache/texera/commit/cc8308740361324f7f129657dad6dc395abeeb1c?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [notebook-migration-service](https://app.codecov.io/gh/apache/texera/pull/7373/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `78.89% <ø> (ø)` | | |
| [pyamber](https://app.codecov.io/gh/apache/texera/pull/7373/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `97.36% <ø> (ø)` | | Carriedforward from [cc83087](https://app.codecov.io/gh/apache/texera/commit/cc8308740361324f7f129657dad6dc395abeeb1c?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [workflow-compiling-service](https://app.codecov.io/gh/apache/texera/pull/7373/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `26.31% <ø> (ø)` | | |
*This pull request uses carry forward flags. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to find out more.
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/texera/pull/7373?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test(workflow-operator): cover the ScanSourceOpDesc base descriptor [texera]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #7373:
URL: https://github.com/apache/texera/pull/7373#issuecomment-5210249185
<!-- texera-reviewer-suggestion -->
### Automated Reviewer Suggestions
Based on the `git blame` history of the changed files, we recommend the following reviewers:
- No candidates found from `git blame` history.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org