You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@texera.apache.org by "aglinxinyuan (via GitHub)" <gi...@apache.org> on 2026/06/18 01:45:45 UTC
[PR] test(workflow-operator): add unit test coverage for SklearnAdvanced trainer descriptors [texera]
aglinxinyuan opened a new pull request, #5768:
URL: https://github.com/apache/texera/pull/5768
### What changes were proposed in this PR?
Pin behavior of four previously-uncovered sklearn-trainer descriptors in `common/workflow-operator/operator/machineLearning/sklearnAdvanced/`. Each is a 30-line override of `SklearnMLOperatorDescriptor` that contributes just two values: the Python `import` statement and the operator-info label. Drift in either silently breaks generated Python code or the UI label. No production-code changes.
| Spec | Source class | Tests |
| --- | --- | --- |
| `SklearnAdvancedKNNClassifierTrainerOpDescSpec` | `SklearnAdvancedKNNClassifierTrainerOpDesc` | 5 |
| `SklearnAdvancedKNNRegressorTrainerOpDescSpec` | `SklearnAdvancedKNNRegressorTrainerOpDesc` | 6 |
| `SklearnAdvancedSVCTrainerOpDescSpec` | `SklearnAdvancedSVCTrainerOpDesc` | 5 |
| `SklearnAdvancedSVRTrainerOpDescSpec` | `SklearnAdvancedSVRTrainerOpDesc` | 6 |
All four spec files follow the `<srcClassName>Spec.scala` one-to-one convention.
**Behavior pinned (per descriptor)**
| Surface | Contract |
| --- | --- |
| `getImportStatements` | exact canonical Python import (`KNeighborsClassifier` / `KNeighborsRegressor` / `SVC` / `SVR` from the appropriate sklearn module) |
| `getOperatorInfo` | exact canonical label (`"KNN Classifier"` / `"KNN Regressor"` / `"SVM Classifier"` / `"SVM Regressor"`) |
| Stability across two instances | both methods return the same string regardless of which instance is queried |
| Type assignability | extends `SklearnMLOperatorDescriptor[ParamsT]` (compile-time enforced through a typed `val` binding) |
| Type-pattern matching | `case _: SklearnMLOperatorDescriptor[_]` matches a concrete instance |
The Regressor spec additionally cross-checks against the Classifier sibling (and SVR vs SVC) — catches copy-paste regressions where one subclass accidentally returned the other's strings.
### Any related issues, documentation, discussions?
Closes #5765.
### How was this PR tested?
Pure unit-test additions; verified locally with:
- `sbt "WorkflowOperator/testOnly org.apache.texera.amber.operator.machineLearning.sklearnAdvanced.KNNTrainer.SklearnAdvancedKNNClassifierTrainerOpDescSpec org.apache.texera.amber.operator.machineLearning.sklearnAdvanced.KNNTrainer.SklearnAdvancedKNNRegressorTrainerOpDescSpec org.apache.texera.amber.operator.machineLearning.sklearnAdvanced.SVCTrainer.SklearnAdvancedSVCTrainerOpDescSpec org.apache.texera.amber.operator.machineLearning.sklearnAdvanced.SVRTrainer.SklearnAdvancedSVRTrainerOpDescSpec"` — 22 tests, all green
- `sbt scalafmtCheckAll` — clean
- CI to confirm
### Was this PR authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Opus 4.7 [1M context])
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test(workflow-operator): add unit test coverage for SklearnAdvanced trainer descriptors [texera]
Posted by "mengw15 (via GitHub)" <gi...@apache.org>.
mengw15 merged PR #5768:
URL: https://github.com/apache/texera/pull/5768
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test(workflow-operator): add unit test coverage for SklearnAdvanced trainer descriptors [texera]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #5768:
URL: https://github.com/apache/texera/pull/5768#issuecomment-4737110382
## [Codecov](https://app.codecov.io/gh/apache/texera/pull/5768?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 53.27%. Comparing base ([`715bccc`](https://app.codecov.io/gh/apache/texera/commit/715bccc22278a7c8a91ba00014550e8d8d011947?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`1fe6b9b`](https://app.codecov.io/gh/apache/texera/commit/1fe6b9b18e7ce50b43d7b562c7a014aadef68e7b?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 1 commits behind head on main.
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## main #5768 +/- ##
=========================================
Coverage 53.27% 53.27%
Complexity 2660 2660
=========================================
Files 1095 1095
Lines 42398 42398
Branches 4560 4560
=========================================
Hits 22587 22587
Misses 18479 18479
Partials 1332 1332
```
| [Flag](https://app.codecov.io/gh/apache/texera/pull/5768/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | | *Carryforward flag |
|---|---|---|---|
| [access-control-service](https://app.codecov.io/gh/apache/texera/pull/5768/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `70.44% <ø> (ø)` | | Carriedforward from [715bccc](https://app.codecov.io/gh/apache/texera/commit/715bccc22278a7c8a91ba00014550e8d8d011947?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [agent-service](https://app.codecov.io/gh/apache/texera/pull/5768/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `34.36% <ø> (ø)` | | Carriedforward from [715bccc](https://app.codecov.io/gh/apache/texera/commit/715bccc22278a7c8a91ba00014550e8d8d011947?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [amber](https://app.codecov.io/gh/apache/texera/pull/5768/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `53.66% <ø> (ø)` | | Carriedforward from [715bccc](https://app.codecov.io/gh/apache/texera/commit/715bccc22278a7c8a91ba00014550e8d8d011947?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [computing-unit-managing-service](https://app.codecov.io/gh/apache/texera/pull/5768/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `1.65% <ø> (ø)` | | Carriedforward from [715bccc](https://app.codecov.io/gh/apache/texera/commit/715bccc22278a7c8a91ba00014550e8d8d011947?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [config-service](https://app.codecov.io/gh/apache/texera/pull/5768/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `56.71% <ø> (ø)` | | |
| [file-service](https://app.codecov.io/gh/apache/texera/pull/5768/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `57.06% <ø> (ø)` | | Carriedforward from [715bccc](https://app.codecov.io/gh/apache/texera/commit/715bccc22278a7c8a91ba00014550e8d8d011947?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [frontend](https://app.codecov.io/gh/apache/texera/pull/5768/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `48.02% <ø> (ø)` | | Carriedforward from [715bccc](https://app.codecov.io/gh/apache/texera/commit/715bccc22278a7c8a91ba00014550e8d8d011947?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [pyamber](https://app.codecov.io/gh/apache/texera/pull/5768/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `90.13% <ø> (ø)` | | Carriedforward from [715bccc](https://app.codecov.io/gh/apache/texera/commit/715bccc22278a7c8a91ba00014550e8d8d011947?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [python](https://app.codecov.io/gh/apache/texera/pull/5768/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `90.80% <ø> (ø)` | | Carriedforward from [715bccc](https://app.codecov.io/gh/apache/texera/commit/715bccc22278a7c8a91ba00014550e8d8d011947?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [workflow-compiling-service](https://app.codecov.io/gh/apache/texera/pull/5768/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `58.69% <ø> (ø)` | | Carriedforward from [715bccc](https://app.codecov.io/gh/apache/texera/commit/715bccc22278a7c8a91ba00014550e8d8d011947?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
*This pull request uses carry forward flags. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to find out more.
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/texera/pull/5768?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test(workflow-operator): add unit test coverage for SklearnAdvanced trainer descriptors [texera]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #5768:
URL: https://github.com/apache/texera/pull/5768#issuecomment-4737123233
<!-- texera-benchmarks-comment -->
## ⚠️ Benchmark changes need a look
🟢 2 better · 🔴 5 worse · ⚪ 8 noise (<±5%) · 0 without baseline
> Compared against main `715bccc` benchmarked on **this same runner**, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.
[Dashboard](https://apache.github.io/texera/dev/bench/) · [Run](https://github.com/apache/texera/actions/runs/27731304345)
| | config | throughput | MB/s | latency | max Δ latest / 7d |
|---|---|---:|---:|---:|---:|
| 🔴 | bs=10 sw=10 sl=64 | 380 | 0.232 | 24,527/36,911/36,911 us | 🔴 +12.1% / 🔴 -7.5% |
| 🟢 | bs=100 sw=10 sl=64 | 807 | 0.492 | 123,821/136,375/136,375 us | 🟢 -11.6% / 🔴 +10.3% |
| ⚪ | bs=1000 sw=10 sl=64 | 913 | 0.557 | 1,086,425/1,155,861/1,155,861 us | ⚪ within ±5% / 🔴 +13.0% |
<details><summary>Baseline details</summary>
Latest main `715bccc` from same runner
| config | metric | PR | latest main | 7d avg | Δ latest | Δ 7d |
|---|---|---:|---:|---:|---:|---:|
| bs=10 sw=10 sl=64 | throughput | 380 tuples/sec | 417 tuples/sec | 410.82 tuples/sec | -8.9% | -7.5% |
| bs=10 sw=10 sl=64 | MB/s | 0.232 MB/s | 0.255 MB/s | 0.251 MB/s | -9.0% | -7.5% |
| bs=10 sw=10 sl=64 | p50 | 24,527 us | 21,873 us | 23,785 us | +12.1% | +3.1% |
| bs=10 sw=10 sl=64 | p95 | 36,911 us | 33,879 us | 34,980 us | +9.0% | +5.5% |
| bs=10 sw=10 sl=64 | p99 | 36,911 us | 33,879 us | 34,980 us | +9.0% | +5.5% |
| bs=100 sw=10 sl=64 | throughput | 807 tuples/sec | 803 tuples/sec | 891.94 tuples/sec | +0.5% | -9.5% |
| bs=100 sw=10 sl=64 | MB/s | 0.492 MB/s | 0.49 MB/s | 0.544 MB/s | +0.4% | -9.6% |
| bs=100 sw=10 sl=64 | p50 | 123,821 us | 121,018 us | 112,277 us | +2.3% | +10.3% |
| bs=100 sw=10 sl=64 | p95 | 136,375 us | 154,309 us | 139,802 us | -11.6% | -2.5% |
| bs=100 sw=10 sl=64 | p99 | 136,375 us | 154,309 us | 139,802 us | -11.6% | -2.5% |
| bs=1000 sw=10 sl=64 | throughput | 913 tuples/sec | 913 tuples/sec | 1,041 tuples/sec | 0.0% | -12.3% |
| bs=1000 sw=10 sl=64 | MB/s | 0.557 MB/s | 0.557 MB/s | 0.635 MB/s | 0.0% | -12.3% |
| bs=1000 sw=10 sl=64 | p50 | 1,086,425 us | 1,095,160 us | 972,714 us | -0.8% | +11.7% |
| bs=1000 sw=10 sl=64 | p95 | 1,155,861 us | 1,172,833 us | 1,023,057 us | -1.4% | +13.0% |
| bs=1000 sw=10 sl=64 | p99 | 1,155,861 us | 1,172,833 us | 1,023,057 us | -1.4% | +13.0% |
</details>
<details><summary>Raw CSV</summary>
```csv
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,526.19,200,128000,380,0.232,24526.82,36911.09,36911.09
1,100,10,64,20,2479.23,2000,1280000,807,0.492,123821.20,136374.98,136374.98
2,1000,10,64,20,21899.95,20000,12800000,913,0.557,1086424.92,1155861.35,1155861.35
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org