You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@texera.apache.org by "carloea2 (via GitHub)" <gi...@apache.org> on 2026/08/31 06:04:24 UTC

[PR] fix(pyamber): stop materialized state replay [texera]

carloea2 opened a new pull request, #8266:
URL: https://github.com/apache/texera/pull/8266

   ### What changes were proposed in this PR?
   
   Honor the materialized input reader's stop flag while replaying persisted state, matching the existing tuple replay behavior. The reader still sends its end marker so downstream port alignment can finish.
   
   ### Any related issues, documentation, discussions?
   
   Closes #8265
   
   ### How was this PR tested?
   
   The untouched live probe emitted one state frame after `stop`. After the fix it reported `state_frames_emitted=0` and `finished=True`.
   
   `C:\Users\carlo\texera\texera\.venv312\Scripts\python.exe -c "import sys,pytest; sys.path[:0]=[r'C:\Users\carlo\texera\texera-worktrees\investigate-bug69\amber\src\main\python',r'C:\Users\carlo\texera\texera\amber\src\main\python']; raise SystemExit(pytest.main([r'amber/src/test/python/core/storage/runnables/test_input_port_materialization_reader_runnable.py','-q','-p','no:cacheprovider']))"`
   
   `C:\Users\carlo\texera\texera\.venv312\Scripts\ruff.exe check amber/src/main/python amber/src/test/python`
   
   `C:\Users\carlo\texera\texera\.venv312\Scripts\ruff.exe format --check amber/src/main/python/core/storage/runnables/input_port_materialization_reader_runnable.py amber/src/test/python/core/storage/runnables/test_input_port_materialization_reader_runnable.py`
   
   ### Was this PR authored or co-authored using generative AI tooling?
   
   Generated-by: OpenAI Codex


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(pyamber): stop materialized state replay [texera]

Posted by "github-actions[bot] (via GitHub) via notifications" <no...@texera.apache.org>.
github-actions[bot] commented on PR #8266:
URL: https://github.com/apache/texera/pull/8266#issuecomment-6904611760

   This pull request has been automatically marked as **stale** because it has
   not had any activity for 30 days. It will be closed in **7 days** if no
   further activity occurs.
   
   If this PR is still relevant, please:
   - Push new commits or rebase onto the latest `main`, or
   - Comment with a status update, or
   - Address any open review feedback.
   
   To exempt this PR from auto-close, a maintainer can add the `pinned` label.
   Thank you for your contribution to Apache Texera (incubating)!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(pyamber): stop materialized state replay [texera]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8266:
URL: https://github.com/apache/texera/pull/8266#issuecomment-5474409728

   <!-- texera-reviewer-suggestion -->
   ### Automated Reviewer Suggestions
   
   Based on the `git blame` history of the changed files, we recommend the following reviewers:
   
   - **Contributors with relevant context:** `@aglinxinyuan`
     You can notify them by mentioning `@aglinxinyuan` in a comment.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(pyamber): stop materialized state replay [texera]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8266:
URL: https://github.com/apache/texera/pull/8266#issuecomment-5474410010

   <!-- backport-auto-label-report -->
   ### Backport auto-label report
   
   This `fix:` PR was checked against each actively-supported release branch. `release/*` labels drive the post-merge backport, so add or remove one to change where this fix lands.
   
   | Release branch | Analysis |
   | --- | --- |
   | ⚠️ `release/v1.3` | Not labeled automatically — none of the files this PR modifies exist on this branch (`amber/src/main/python/core/storage/runnables/input_port_materialization_reader_runnable.py`, `amber/src/test/python/core/storage/runnables/test_input_port_materialization_reader_runnable.py`). The fix may target code that isn't on this release, or the files were moved/renamed after the branch was cut. **Please check and add `release/v1.3` by hand if this fix should be backported here.** |
   | ✅ `release/v1.2` | Change detected on this branch — label added; this fix is queued to backport here. Requested review from @xuang7. |
   
   _[Auto-label run](https://github.com/apache/texera/actions/runs/33362742415)._


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(pyamber): stop materialized state replay [texera]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #8266:
URL: https://github.com/apache/texera/pull/8266#issuecomment-5474448086

   <!-- texera-benchmarks-comment -->
   ## ⚠️ Benchmark changes need a look
   
   🟢 2 better · 🔴 10 worse · ⚪ 3 noise (<±5%) · 0 without baseline
   
   > Compared against main `50321e4` benchmarked on **this same runner**, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.
   
   [Dashboard](https://apache.github.io/texera/dev/bench/) · [Run](https://github.com/apache/texera/actions/runs/33362758435)
   
   | | config | throughput | MB/s | latency | max Δ latest / 7d |
   |---|---|---:|---:|---:|---:|
   | 🔴 | bs=10 sw=10 sl=64 | 652 | 0.398 | 14,901/20,172/20,172 us | 🟢 -10.1% / 🔴 +36.4% |
   | 🔴 | bs=100 sw=10 sl=64 | 1,338 | 0.817 | 73,438/93,030/93,030 us | 🔴 +15.0% / 🟢 +23.5% |
   | 🔴 | bs=1000 sw=10 sl=64 | 1,532 | 0.935 | 651,036/702,243/702,243 us | 🔴 +5.7% / 🟢 +36.8% |
   
   <details><summary>Baseline details</summary>
   
   Latest main `50321e4` from same runner
   
   | config | metric | PR | latest main | 7d avg | Δ latest | Δ 7d |
   |---|---|---:|---:|---:|---:|---:|
   | bs=10 sw=10 sl=64 | throughput | 652 tuples/sec | 695 tuples/sec | 844.57 tuples/sec | -6.2% | -22.8% |
   | bs=10 sw=10 sl=64 | MB/s | 0.398 MB/s | 0.424 MB/s | 0.515 MB/s | -6.1% | -22.8% |
   | bs=10 sw=10 sl=64 | p50 | 14,901 us | 13,652 us | 11,837 us | +9.1% | +25.9% |
   | bs=10 sw=10 sl=64 | p95 | 20,172 us | 22,431 us | 14,788 us | -10.1% | +36.4% |
   | bs=10 sw=10 sl=64 | p99 | 20,172 us | 22,431 us | 17,651 us | -10.1% | +14.3% |
   | bs=100 sw=10 sl=64 | throughput | 1,338 tuples/sec | 1,458 tuples/sec | 1,084 tuples/sec | -8.2% | +23.4% |
   | bs=100 sw=10 sl=64 | MB/s | 0.817 MB/s | 0.89 MB/s | 0.662 MB/s | -8.2% | +23.5% |
   | bs=100 sw=10 sl=64 | p50 | 73,438 us | 67,441 us | 94,505 us | +8.9% | -22.3% |
   | bs=100 sw=10 sl=64 | p95 | 93,030 us | 80,896 us | 100,941 us | +15.0% | -7.8% |
   | bs=100 sw=10 sl=64 | p99 | 93,030 us | 80,896 us | 108,642 us | +15.0% | -14.4% |
   | bs=1000 sw=10 sl=64 | throughput | 1,532 tuples/sec | 1,595 tuples/sec | 1,119 tuples/sec | -3.9% | +36.8% |
   | bs=1000 sw=10 sl=64 | MB/s | 0.935 MB/s | 0.974 MB/s | 0.683 MB/s | -4.0% | +36.8% |
   | bs=1000 sw=10 sl=64 | p50 | 651,036 us | 631,390 us | 924,404 us | +3.1% | -29.6% |
   | bs=1000 sw=10 sl=64 | p95 | 702,243 us | 664,475 us | 967,785 us | +5.7% | -27.4% |
   | bs=1000 sw=10 sl=64 | p99 | 702,243 us | 664,475 us | 995,630 us | +5.7% | -29.5% |
   
   </details>
   
   <details><summary>Raw CSV</summary>
   
   ```csv
   config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
   0,10,10,64,20,306.67,200,128000,652,0.398,14900.64,20171.69,20171.69
   1,100,10,64,20,1494.86,2000,1280000,1338,0.817,73437.98,93030.25,93030.25
   2,1000,10,64,20,13053.64,20000,12800000,1532,0.935,651036.20,702243.31,702243.31
   ```
   
   </details>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(pyamber): stop materialized state replay [texera]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #8266:
URL: https://github.com/apache/texera/pull/8266#issuecomment-5474440948

   ## [Codecov](https://app.codecov.io/gh/apache/texera/pull/8266?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :white_check_mark: All modified and coverable lines are covered by tests.
   :white_check_mark: Project coverage is 93.93%. Comparing base ([`50321e4`](https://app.codecov.io/gh/apache/texera/commit/50321e403c82df299a13deb50a7f9849dd93bdba?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`b692a95`](https://app.codecov.io/gh/apache/texera/commit/b692a9592e87f61f9003cef0c21b3d6f704dd518?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@            Coverage Diff            @@
   ##               main    #8266   +/-   ##
   =========================================
     Coverage     93.93%   93.93%           
     Complexity     4761     4761           
   =========================================
     Files          1191     1191           
     Lines         48308    48310    +2     
     Branches       5377     5377           
   =========================================
   + Hits          45378    45380    +2     
     Misses         1495     1495           
     Partials       1435     1435           
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/texera/pull/8266/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | | *Carryforward flag |
   |---|---|---|---|
   | [access-control-service](https://app.codecov.io/gh/apache/texera/pull/8266/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `81.00% <ø> (ø)` | | Carriedforward from [50321e4](https://app.codecov.io/gh/apache/texera/commit/50321e403c82df299a13deb50a7f9849dd93bdba?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   | [agent-service](https://app.codecov.io/gh/apache/texera/pull/8266/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `99.32% <ø> (ø)` | | Carriedforward from [50321e4](https://app.codecov.io/gh/apache/texera/commit/50321e403c82df299a13deb50a7f9849dd93bdba?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   | [amber](https://app.codecov.io/gh/apache/texera/pull/8266/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `90.02% <ø> (ø)` | | Carriedforward from [50321e4](https://app.codecov.io/gh/apache/texera/commit/50321e403c82df299a13deb50a7f9849dd93bdba?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   | [computing-unit-managing-service](https://app.codecov.io/gh/apache/texera/pull/8266/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `73.67% <ø> (ø)` | | Carriedforward from [50321e4](https://app.codecov.io/gh/apache/texera/commit/50321e403c82df299a13deb50a7f9849dd93bdba?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   | [config-service](https://app.codecov.io/gh/apache/texera/pull/8266/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `86.86% <ø> (ø)` | | Carriedforward from [50321e4](https://app.codecov.io/gh/apache/texera/commit/50321e403c82df299a13deb50a7f9849dd93bdba?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   | [file-service](https://app.codecov.io/gh/apache/texera/pull/8266/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `87.91% <ø> (ø)` | | Carriedforward from [50321e4](https://app.codecov.io/gh/apache/texera/commit/50321e403c82df299a13deb50a7f9849dd93bdba?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   | [frontend](https://app.codecov.io/gh/apache/texera/pull/8266/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `96.41% <ø> (ø)` | | Carriedforward from [50321e4](https://app.codecov.io/gh/apache/texera/commit/50321e403c82df299a13deb50a7f9849dd93bdba?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   | [notebook-migration-service](https://app.codecov.io/gh/apache/texera/pull/8266/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `79.31% <ø> (ø)` | | Carriedforward from [50321e4](https://app.codecov.io/gh/apache/texera/commit/50321e403c82df299a13deb50a7f9849dd93bdba?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   | [pyamber](https://app.codecov.io/gh/apache/texera/pull/8266/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `98.87% <100.00%> (+<0.01%)` | :arrow_up: | |
   | [workflow-compiling-service](https://app.codecov.io/gh/apache/texera/pull/8266/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `77.19% <ø> (ø)` | | Carriedforward from [50321e4](https://app.codecov.io/gh/apache/texera/commit/50321e403c82df299a13deb50a7f9849dd93bdba?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   
   *This pull request uses carry forward flags. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to find out more.
   </details>
   
   [:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/texera/pull/8266?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).   
   :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org