You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@texera.apache.org by "eugenegujing (via GitHub)" <gi...@apache.org> on 2026/07/07 23:06:49 UTC
[PR] fix(amber): log betterproto stderr as info to avoid false IDE build failures [texera]
eugenegujing opened a new pull request, #6245:
URL: https://github.com/apache/texera/pull/6245
### What changes were proposed in this PR?
Log the `genPythonProto` subprocess's stderr at `info` instead of `error` (`amber/build.sbt`).
`genPythonProto` runs `bin/python-proto-gen.sh`, which invokes betterproto as a `protoc` plugin. A `protoc` plugin's stdout is reserved for the binary `CodeGeneratorResponse` sent back to `protoc`, so betterproto writes its normal progress (`Writing __init__.py`, `Writing org/…/__init__.py`, …) to stderr. The task routed that stderr to sbt's `error` level, so a fully successful generation printed a wall of `[error]` lines and still ended in `[success]`.
A command-line `sbt compile` is unaffected, but IDEs that delegate the build to sbt (e.g. IntelliJ IDEA with "Build and run using: sbt") parse those `[error]` lines as a build failure. This fails the "Build" before-launch step of the amber-based run configurations — `TexeraWebApplication`, `ComputingUnitMaster`, `ComputingUnitWorker` — so those services never launch, while non-amber services (which do not depend on `genPythonProto`) start normally.
**The change.** `ProcessLogger` takes two callbacks, one for the subprocess's stdout and one for its stderr; only the stderr callback is changed, from `log.error` to `log.info`, in the `genPythonProto` task in `amber/build.sbt`:
```diff
- val procLogger = scala.sys.process.ProcessLogger(line => log.info(line), line => log.error(line))
+ val procLogger = scala.sys.process.ProcessLogger(line => log.info(line), line => log.info(line))
```
Nothing else changes. Real failures are still detected via the script's exit code on the next line (`if (exit != 0) sys.error(...)`), which is untouched, so this only stops benign stderr from being labeled as an error.
### Any related issues, documentation, or discussions?
Follow-up to #5358
Closes: https://github.com/apache/texera/issues/6243
### How was this PR tested?
- Ran `sbt "WorkflowExecutionService/genPythonProto"` with `protoc` and `protoc-gen-python_betterproto` on `PATH`: the betterproto progress lines now appear as `[info] Writing …` (previously `[error] Writing …`), and the task still ends in `[success]`.
- Confirmed failure detection is unchanged: the exit-code check on the following line still fails the build when the script exits non-zero, so real failures are still surfaced.
- Confirmed the diff is limited to a single line in `amber/build.sbt`.
- `sbt scalafmtCheckAll` passed.
- `sbt "scalafixAll --check"` passed.
- `sbt test` (backend): all tests unrelated to external infrastructure passed.
### Was this PR authored or co-authored using generative AI tooling?
Co-authored by Claude Fable 5.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(amber): log betterproto stderr as info to avoid false IDE build failures [texera]
Posted by "Yicong-Huang (via GitHub)" <gi...@apache.org>.
Yicong-Huang commented on PR #6245:
URL: https://github.com/apache/texera/pull/6245#issuecomment-5126198989
Backport PR opened: #7108 (https://github.com/apache/texera/pull/7108) to `release/v1.2`, assigned to @eugenegujing — part of a backport-coverage audit (clean cherry-pick).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(amber): log betterproto stderr as info to avoid false IDE build failures [texera]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #6245:
URL: https://github.com/apache/texera/pull/6245#discussion_r3540309887
##########
amber/build.sbt:
##########
@@ -224,7 +224,7 @@ genPythonProto := {
"Install protoc and `pip install betterproto[compiler]` before launching a Python worker or running pytest."
)
} else {
- val procLogger = scala.sys.process.ProcessLogger(line => log.info(line), line => log.error(line))
+ val procLogger = scala.sys.process.ProcessLogger(line => log.info(line), line => log.info(line))
Review Comment:
After routing both stdout and stderr to `info`, log output no longer indicates whether a given line came from stderr. That makes it harder to diagnose failures or warnings from `python-proto-gen.sh` (especially when the task fails and only the final `sys.error(...)` is logged at error level). Consider prefixing stderr lines so provenance is preserved while still avoiding `[error]` output in IDEs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(amber): log betterproto stderr as info to avoid false IDE build failures [texera]
Posted by "eugenegujing (via GitHub)" <gi...@apache.org>.
eugenegujing commented on PR #6245:
URL: https://github.com/apache/texera/pull/6245#issuecomment-4910139199
@aglinxinyuan I used IntelliJ and encountered this issue, and my IntelliJ is set to delegate builds to sbt. Maybe you could try compile again to see if it went wrong?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(amber): log betterproto stderr as info to avoid false IDE build failures [texera]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #6245:
URL: https://github.com/apache/texera/pull/6245#issuecomment-4909886720
## [Codecov](https://app.codecov.io/gh/apache/texera/pull/6245?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 60.45%. Comparing base ([`7c135ee`](https://app.codecov.io/gh/apache/texera/commit/7c135ee0955d539de0f7905b88d8bc4e86cca5a4?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`5bc2f6c`](https://app.codecov.io/gh/apache/texera/commit/5bc2f6c5a2c525e25d92041ab3354dbc8bf53b96?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 1 commits behind head on main.
:white_check_mark: All tests successful. No failed tests found.
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## main #6245 +/- ##
============================================
- Coverage 60.46% 60.45% -0.01%
+ Complexity 3370 3368 -2
============================================
Files 1133 1133
Lines 43985 43985
Branches 4782 4782
============================================
- Hits 26596 26593 -3
- Misses 15937 15938 +1
- Partials 1452 1454 +2
```
| [Flag](https://app.codecov.io/gh/apache/texera/pull/6245/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | | *Carryforward flag |
|---|---|---|---|
| [access-control-service](https://app.codecov.io/gh/apache/texera/pull/6245/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `70.00% <ø> (ø)` | | Carriedforward from [7c135ee](https://app.codecov.io/gh/apache/texera/commit/7c135ee0955d539de0f7905b88d8bc4e86cca5a4?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [agent-service](https://app.codecov.io/gh/apache/texera/pull/6245/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.59% <ø> (ø)` | | Carriedforward from [7c135ee](https://app.codecov.io/gh/apache/texera/commit/7c135ee0955d539de0f7905b88d8bc4e86cca5a4?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [amber](https://app.codecov.io/gh/apache/texera/pull/6245/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `66.03% <ø> (-0.02%)` | :arrow_down: | |
| [computing-unit-managing-service](https://app.codecov.io/gh/apache/texera/pull/6245/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `0.00% <ø> (ø)` | | Carriedforward from [7c135ee](https://app.codecov.io/gh/apache/texera/commit/7c135ee0955d539de0f7905b88d8bc4e86cca5a4?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [config-service](https://app.codecov.io/gh/apache/texera/pull/6245/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `52.30% <ø> (ø)` | | Carriedforward from [7c135ee](https://app.codecov.io/gh/apache/texera/commit/7c135ee0955d539de0f7905b88d8bc4e86cca5a4?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [file-service](https://app.codecov.io/gh/apache/texera/pull/6245/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `63.97% <ø> (ø)` | | Carriedforward from [7c135ee](https://app.codecov.io/gh/apache/texera/commit/7c135ee0955d539de0f7905b88d8bc4e86cca5a4?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [frontend](https://app.codecov.io/gh/apache/texera/pull/6245/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `51.97% <ø> (ø)` | | Carriedforward from [7c135ee](https://app.codecov.io/gh/apache/texera/commit/7c135ee0955d539de0f7905b88d8bc4e86cca5a4?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [notebook-migration-service](https://app.codecov.io/gh/apache/texera/pull/6245/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `78.57% <ø> (ø)` | | Carriedforward from [7c135ee](https://app.codecov.io/gh/apache/texera/commit/7c135ee0955d539de0f7905b88d8bc4e86cca5a4?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [pyamber](https://app.codecov.io/gh/apache/texera/pull/6245/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `91.18% <ø> (ø)` | | Carriedforward from [7c135ee](https://app.codecov.io/gh/apache/texera/commit/7c135ee0955d539de0f7905b88d8bc4e86cca5a4?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [workflow-compiling-service](https://app.codecov.io/gh/apache/texera/pull/6245/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `55.14% <ø> (ø)` | | Carriedforward from [7c135ee](https://app.codecov.io/gh/apache/texera/commit/7c135ee0955d539de0f7905b88d8bc4e86cca5a4?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
*This pull request uses carry forward flags. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to find out more.
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/texera/pull/6245?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(amber): log betterproto stderr as info to avoid false IDE build failures [texera]
Posted by "aglinxinyuan (via GitHub)" <gi...@apache.org>.
aglinxinyuan commented on PR #6245:
URL: https://github.com/apache/texera/pull/6245#issuecomment-4910166992
> @aglinxinyuan I used IntelliJ and encountered this issue, and my IntelliJ is set to delegate builds to sbt. Maybe you could try compile again to see if it went wrong?
I see. It's a recent change, and my protoc files were already there, so the step was skipped.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(amber): log betterproto stderr as info to avoid false IDE build failures [texera]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #6245:
URL: https://github.com/apache/texera/pull/6245#issuecomment-4909861154
<!-- texera-benchmarks-comment -->
## ⚠️ Benchmark changes need a look
🟢 0 better · 🔴 7 worse · ⚪ 8 noise (<±5%) · 0 without baseline
> Compared against main `7c135ee` benchmarked on **this same runner**, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.
[Dashboard](https://apache.github.io/texera/dev/bench/) · [Run](https://github.com/apache/texera/actions/runs/28904914987)
| | config | throughput | MB/s | latency | max Δ latest / 7d |
|---|---|---:|---:|---:|---:|
| 🔴 | bs=10 sw=10 sl=64 | 345 | 0.21 | 28,875/40,096/40,096 us | 🔴 +15.5% / 🔴 +153.9% |
| ⚪ | bs=100 sw=10 sl=64 | 786 | 0.48 | 126,474/143,920/143,920 us | ⚪ within ±5% / 🔴 +30.3% |
| 🔴 | bs=1000 sw=10 sl=64 | 897 | 0.547 | 1,112,275/1,221,546/1,221,546 us | 🔴 +6.0% / 🔴 +15.4% |
<details><summary>Baseline details</summary>
Latest main `7c135ee` from same runner
| config | metric | PR | latest main | 7d avg | Δ latest | Δ 7d |
|---|---|---:|---:|---:|---:|---:|
| bs=10 sw=10 sl=64 | throughput | 345 tuples/sec | 390 tuples/sec | 749.87 tuples/sec | -11.5% | -54.0% |
| bs=10 sw=10 sl=64 | MB/s | 0.21 MB/s | 0.238 MB/s | 0.458 MB/s | -11.8% | -54.1% |
| bs=10 sw=10 sl=64 | p50 | 28,875 us | 25,002 us | 13,071 us | +15.5% | +120.9% |
| bs=10 sw=10 sl=64 | p95 | 40,096 us | 37,956 us | 15,795 us | +5.6% | +153.9% |
| bs=10 sw=10 sl=64 | p99 | 40,096 us | 37,956 us | 19,469 us | +5.6% | +105.9% |
| bs=100 sw=10 sl=64 | throughput | 786 tuples/sec | 819 tuples/sec | 960.12 tuples/sec | -4.0% | -18.1% |
| bs=100 sw=10 sl=64 | MB/s | 0.48 MB/s | 0.5 MB/s | 0.586 MB/s | -4.0% | -18.1% |
| bs=100 sw=10 sl=64 | p50 | 126,474 us | 121,460 us | 104,066 us | +4.1% | +21.5% |
| bs=100 sw=10 sl=64 | p95 | 143,920 us | 147,331 us | 110,469 us | -2.3% | +30.3% |
| bs=100 sw=10 sl=64 | p99 | 143,920 us | 147,331 us | 116,561 us | -2.3% | +23.5% |
| bs=1000 sw=10 sl=64 | throughput | 897 tuples/sec | 921 tuples/sec | 985.46 tuples/sec | -2.6% | -9.0% |
| bs=1000 sw=10 sl=64 | MB/s | 0.547 MB/s | 0.562 MB/s | 0.601 MB/s | -2.7% | -9.1% |
| bs=1000 sw=10 sl=64 | p50 | 1,112,275 us | 1,088,093 us | 1,019,691 us | +2.2% | +9.1% |
| bs=1000 sw=10 sl=64 | p95 | 1,221,546 us | 1,152,429 us | 1,058,233 us | +6.0% | +15.4% |
| bs=1000 sw=10 sl=64 | p99 | 1,221,546 us | 1,152,429 us | 1,088,106 us | +6.0% | +12.3% |
</details>
<details><summary>Raw CSV</summary>
```csv
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,580.50,200,128000,345,0.210,28874.75,40096.40,40096.40
1,100,10,64,20,2545.76,2000,1280000,786,0.480,126474.00,143920.11,143920.11
2,1000,10,64,20,22303.32,20000,12800000,897,0.547,1112275.05,1221546.03,1221546.03
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(amber): log betterproto stderr as info to avoid false IDE build failures [texera]
Posted by "aglinxinyuan (via GitHub)" <gi...@apache.org>.
aglinxinyuan merged PR #6245:
URL: https://github.com/apache/texera/pull/6245
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(amber): log betterproto stderr as info to avoid false IDE build failures [texera]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #6245:
URL: https://github.com/apache/texera/pull/6245#issuecomment-4909831927
<!-- texera-reviewer-suggestion -->
### Automated Reviewer Suggestions
Based on the `git blame` history of the changed files, we recommend the following reviewers:
- No candidates found from `git blame` history.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org