You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hugegraph.apache.org by "imbajin (via GitHub)" <gi...@apache.org> on 2026/10/22 19:44:58 UTC
[PR] fix: harden worker and shuffle lifecycles [hugegraph]
imbajin opened a new pull request, #3393:
URL: https://github.com/apache/hugegraph/pull/3393
## Purpose of the PR
Fix worker failure reporting and sort/shuffle resource ownership after the Java 17 upgrade, while reusing existing implementations and preserving external compatibility.
## Main Changes
| Trigger | Before | After |
| --- | --- | --- |
| An algorithm worker or commit callback fails | The job can miss the failure, retry its callback, or leave a producer blocked | Shared consumers report the failure, run completion once, and release blocked producers |
| A small Store shuffle or repeated iterator probe | Results can be unsorted or records can be skipped | Both memory and disk paths respect the comparator and iterator contract |
| A shuffle read or cleanup fails | Truncated input can look like EOF; readers and spill files can remain open | Read failures propagate and owned resources close, preserving the first failure |
| PD connection waits are interrupted or snapshots are synced | Interrupts are lost and ZIP metadata can be written after sync | Interrupted work is cancelled with the flag retained; the complete archive is synced |
Also replace deprecated task reflection with constructor reflection, preserve errors without messages, use JDK stream/cleanup APIs, and remove duplicated deprecated compiler configuration. Commons keeps its Java 11 bytecode target. REST/Gremlin APIs, configuration, RPC definitions and persisted formats are unchanged.
## Verifying these changes
- [ ] Trivial rework / code cleanup without any test coverage. (No Need)
- [x] Existing consumer, traversal and task-scheduler regressions pass.
- [x] Added regression coverage for algorithm failure/interrupt handling, PD waits and archive checksums, shuffle ordering, malformed records, and reader/filter cleanup.
Validated with Java 17: repository formatting, full `clean compile`, focused Server unit tests, PD core test selection and Store core test selection. An independent review found no remaining blocking issue. Multi-node cluster validation is not included.
## Does this PR potentially affect the following parts?
- [ ] Dependencies
- [ ] Modify configurations
- [ ] The public API
- [x] Other affects: error propagation, cancellation and resource cleanup
- [ ] Nope
## Documentation Status
- [ ] `Doc - TODO`: required documentation is pending; complete it before merging.
- [x] `Doc - Done`: documentation is included here.
- [ ] `Doc - No Need`: no user-visible documentation is affected.
Documentation: [storage lifecycle contracts](../blob/master/docs/storage-lifecycle.md), including algorithm workers, Store shuffles, PD waits and snapshot I/O. No website configuration or API reference changes are required.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] fix: harden worker and shuffle lifecycles [hugegraph]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #3393:
URL: https://github.com/apache/hugegraph/pull/3393#issuecomment-6349881690
## [Codecov](https://app.codecov.io/gh/apache/hugegraph/pull/3393?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 56.00%. Comparing base ([`c2ec570`](https://app.codecov.io/gh/apache/hugegraph/commit/c2ec5704332fc1102ad277d6aae6658d65c73601?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`437893b`](https://app.codecov.io/gh/apache/hugegraph/commit/437893bd3017f3411ca1fccff6e4d1f43908d996?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 12 commits behind head on master.
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #3393 +/- ##
============================================
+ Coverage 55.93% 56.00% +0.07%
+ Complexity 2277 2275 -2
============================================
Files 774 774
Lines 68534 68304 -230
Branches 9333 9287 -46
============================================
- Hits 38334 38256 -78
+ Misses 26634 26491 -143
+ Partials 3566 3557 -9
```
| [Flag](https://app.codecov.io/gh/apache/hugegraph/pull/3393/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | | *Carryforward flag |
|---|---|---|---|
| [commons](https://app.codecov.io/gh/apache/hugegraph/pull/3393/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `96.37% <ø> (ø)` | | |
| [hstore](https://app.codecov.io/gh/apache/hugegraph/pull/3393/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `35.87% <ø> (+0.06%)` | :arrow_up: | Carriedforward from [c2ec570](https://app.codecov.io/gh/apache/hugegraph/commit/c2ec5704332fc1102ad277d6aae6658d65c73601?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [pd](https://app.codecov.io/gh/apache/hugegraph/pull/3393/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `49.21% <ø> (+0.03%)` | :arrow_up: | Carriedforward from [c2ec570](https://app.codecov.io/gh/apache/hugegraph/commit/c2ec5704332fc1102ad277d6aae6658d65c73601?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [server_hbase](https://app.codecov.io/gh/apache/hugegraph/pull/3393/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `41.93% <ø> (+0.03%)` | :arrow_up: | Carriedforward from [c2ec570](https://app.codecov.io/gh/apache/hugegraph/commit/c2ec5704332fc1102ad277d6aae6658d65c73601?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [server_memory](https://app.codecov.io/gh/apache/hugegraph/pull/3393/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `54.40% <ø> (+0.06%)` | :arrow_up: | Carriedforward from [c2ec570](https://app.codecov.io/gh/apache/hugegraph/commit/c2ec5704332fc1102ad277d6aae6658d65c73601?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [server_rocksdb](https://app.codecov.io/gh/apache/hugegraph/pull/3393/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `42.38% <ø> (+0.03%)` | :arrow_up: | Carriedforward from [c2ec570](https://app.codecov.io/gh/apache/hugegraph/commit/c2ec5704332fc1102ad277d6aae6658d65c73601?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [store](https://app.codecov.io/gh/apache/hugegraph/pull/3393/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.01% <ø> (+0.07%)` | :arrow_up: | Carriedforward from [c2ec570](https://app.codecov.io/gh/apache/hugegraph/commit/c2ec5704332fc1102ad277d6aae6658d65c73601?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
*This pull request uses carry forward flags. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to find out more.
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/hugegraph/pull/3393?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org
Re: [PR] fix: harden worker and shuffle lifecycles [hugegraph]
Posted by "imbajin (via GitHub)" <gi...@apache.org>.
imbajin merged PR #3393:
URL: https://github.com/apache/hugegraph/pull/3393
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@hugegraph.apache.org
For additional commands, e-mail: issues-help@hugegraph.apache.org