You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "PDGGK (via GitHub)" <gi...@apache.org> on 2026/08/28 08:47:49 UTC
[PR] [MINOR] Release every client resource on close even when an earlier one fails [hudi]
PDGGK opened a new pull request, #19778:
URL: https://github.com/apache/hudi/pull/19778
### Describe the issue this Pull Request addresses
`close()` on the write client releases several independent resources as a bare sequence, so the first failure strands everything after it.
```java
// BaseHoodieClient:153
stopEmbeddedServerView(true);
this.context.setJobStatus("", "");
this.heartbeatClient.close();
this.txnManager.close();
// BaseHoodieWriteClient:1663
super.close();
this.index.close();
this.tableServiceClient.close();
```
The one that matters most is last in the base chain: `txnManager` owns the write lock. If stopping the timeline server or closing the heartbeat client throws, the lock provider is never released, and the next writer waits for it to time out rather than getting a clean handover. The heartbeat client is in the same position — it holds a scheduled executor and leaves heartbeat files behind.
`BaseHoodieWriteClient` inherits the problem and adds to it: a failure in `super.close()` also skips the index and the table service client.
### Summary and Changelog
Each step is now closed independently. The first failure is what the caller sees, with any later ones attached via `addSuppressed` rather than dropped, so nothing that used to surface stops surfacing.
Tests: `TestBaseHoodieWriteClient#testCloseReleasesLaterResourcesWhenAnEarlierCloseFails` injects an index whose `close()` throws and asserts the table service client and the transaction manager are still closed, and that the original failure is what propagates.
Reverting the change turns it red with `Wanted but not invoked: tableServiceClient.close()`.
A note on how the test is shaped: an earlier version made the *last* resource throw, which passed with and without the fix — everything before it had already been released, so it proved nothing. The failure has to be injected ahead of the resources whose release is being asserted.
`mvn test -pl hudi-client/hudi-client-common -Dtest=TestBaseHoodieWriteClient#testCloseReleasesLaterResourcesWhenAnEarlierCloseFails` passes; `checkstyle:check` clean. The full `TestBaseHoodieWriteClient` class exhausts the heap on my machine both with and without this change, so I could not run it end to end locally.
### Impact
No public API change. `close()` still throws the same failure it threw before; it simply releases the remaining resources first.
### Risk Level
low
### Documentation Update
none
### Contributor's checklist
- [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [x] Enough context is provided in the sections above
- [x] Adequate tests were added if applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [MINOR] Release every client resource on close even when an earlier one fails [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19778:
URL: https://github.com/apache/hudi/pull/19778#issuecomment-5845740371
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5a314786a08105f0dcbc7b51ae6a13115fe95d7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16664",
"triggerID" : "5a314786a08105f0dcbc7b51ae6a13115fe95d7f",
"triggerType" : "PUSH"
}, {
"hash" : "fba93d23e1e373db9261737dbfdfb07fa2d48c4a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16739",
"triggerID" : "fba93d23e1e373db9261737dbfdfb07fa2d48c4a",
"triggerType" : "PUSH"
}, {
"hash" : "cfd155181248fbb3718a8582762fd11a75d22a2a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16741",
"triggerID" : "cfd155181248fbb3718a8582762fd11a75d22a2a",
"triggerType" : "PUSH"
}, {
"hash" : "dee61ecea7947ea273eb5c24dd4f7243ed1d155f",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17290",
"triggerID" : "dee61ecea7947ea273eb5c24dd4f7243ed1d155f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* dee61ecea7947ea273eb5c24dd4f7243ed1d155f Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17290)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [MINOR] Release every client resource on close even when an earlier one fails [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19778:
URL: https://github.com/apache/hudi/pull/19778#issuecomment-5495272167
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5a314786a08105f0dcbc7b51ae6a13115fe95d7f",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16664",
"triggerID" : "5a314786a08105f0dcbc7b51ae6a13115fe95d7f",
"triggerType" : "PUSH"
}, {
"hash" : "fba93d23e1e373db9261737dbfdfb07fa2d48c4a",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16739",
"triggerID" : "fba93d23e1e373db9261737dbfdfb07fa2d48c4a",
"triggerType" : "PUSH"
}, {
"hash" : "cfd155181248fbb3718a8582762fd11a75d22a2a",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "cfd155181248fbb3718a8582762fd11a75d22a2a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5a314786a08105f0dcbc7b51ae6a13115fe95d7f Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16664)
* fba93d23e1e373db9261737dbfdfb07fa2d48c4a Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16739)
* cfd155181248fbb3718a8582762fd11a75d22a2a UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [MINOR] Release every client resource on close even when an earlier one fails [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19778:
URL: https://github.com/apache/hudi/pull/19778#issuecomment-5451951823
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5a314786a08105f0dcbc7b51ae6a13115fe95d7f",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16664",
"triggerID" : "5a314786a08105f0dcbc7b51ae6a13115fe95d7f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5a314786a08105f0dcbc7b51ae6a13115fe95d7f Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16664)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [MINOR] Release every client resource on close even when an earlier one fails [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19778:
URL: https://github.com/apache/hudi/pull/19778#issuecomment-5844916509
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5a314786a08105f0dcbc7b51ae6a13115fe95d7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16664",
"triggerID" : "5a314786a08105f0dcbc7b51ae6a13115fe95d7f",
"triggerType" : "PUSH"
}, {
"hash" : "fba93d23e1e373db9261737dbfdfb07fa2d48c4a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16739",
"triggerID" : "fba93d23e1e373db9261737dbfdfb07fa2d48c4a",
"triggerType" : "PUSH"
}, {
"hash" : "cfd155181248fbb3718a8582762fd11a75d22a2a",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16741",
"triggerID" : "cfd155181248fbb3718a8582762fd11a75d22a2a",
"triggerType" : "PUSH"
}, {
"hash" : "dee61ecea7947ea273eb5c24dd4f7243ed1d155f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dee61ecea7947ea273eb5c24dd4f7243ed1d155f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* cfd155181248fbb3718a8582762fd11a75d22a2a Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16741)
* dee61ecea7947ea273eb5c24dd4f7243ed1d155f UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [MINOR] Release every client resource on close even when an earlier one fails [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #19778:
URL: https://github.com/apache/hudi/pull/19778#issuecomment-5505438977
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/19778?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `32.58427%` with `60 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 78.28%. Comparing base ([`c59987a`](https://app.codecov.io/gh/apache/hudi/commit/c59987a024cd021cbb9aa34bdc74d341ea834b4d?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`cfd1551`](https://app.codecov.io/gh/apache/hudi/commit/cfd155181248fbb3718a8582762fd11a75d22a2a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 2 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/19778?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [.../org/apache/hudi/client/BaseHoodieWriteClient.java](https://app.codecov.io/gh/apache/hudi/pull/19778?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2FBaseHoodieWriteClient.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC9CYXNlSG9vZGllV3JpdGVDbGllbnQuamF2YQ==) | 32.60% | [30 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/19778?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [.../java/org/apache/hudi/client/BaseHoodieClient.java](https://app.codecov.io/gh/apache/hudi/pull/19778?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2FBaseHoodieClient.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC9CYXNlSG9vZGllQ2xpZW50LmphdmE=) | 32.55% | [28 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/19778?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #19778 +/- ##
============================================
- Coverage 78.31% 78.28% -0.04%
Complexity 33888 33888
============================================
Files 2541 2541
Lines 141728 141798 +70
Branches 17182 17186 +4
============================================
+ Hits 111001 111009 +8
- Misses 23017 23076 +59
- Partials 7710 7713 +3
```
| [Components](https://app.codecov.io/gh/apache/hudi/pull/19778/components?src=pr&el=components&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hudi-common](https://app.codecov.io/gh/apache/hudi/pull/19778/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `83.65% <ø> (ø)` | |
| [hudi-client](https://app.codecov.io/gh/apache/hudi/pull/19778/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `83.06% <32.58%> (-0.16%)` | :arrow_down: |
| [hudi-flink](https://app.codecov.io/gh/apache/hudi/pull/19778/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `85.71% <ø> (ø)` | |
| [hudi-spark-datasource](https://app.codecov.io/gh/apache/hudi/pull/19778/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `73.21% <ø> (ø)` | |
| [hudi-utilities](https://app.codecov.io/gh/apache/hudi/pull/19778/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `74.55% <ø> (ø)` | |
| [hudi-cli](https://app.codecov.io/gh/apache/hudi/pull/19778/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `15.06% <ø> (ø)` | |
| [hudi-hadoop](https://app.codecov.io/gh/apache/hudi/pull/19778/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `70.72% <ø> (ø)` | |
| [hudi-sync](https://app.codecov.io/gh/apache/hudi/pull/19778/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `75.56% <ø> (ø)` | |
| [hudi-io](https://app.codecov.io/gh/apache/hudi/pull/19778/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `79.98% <ø> (ø)` | |
| [hudi-timeline-service](https://app.codecov.io/gh/apache/hudi/pull/19778/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `83.44% <ø> (ø)` | |
| [hudi-cloud](https://app.codecov.io/gh/apache/hudi/pull/19778/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `65.81% <ø> (ø)` | |
| [hudi-kafka-connect](https://app.codecov.io/gh/apache/hudi/pull/19778/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `53.20% <ø> (ø)` | |
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/19778/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/19778/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `44.04% <32.58%> (-0.05%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/19778?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...va/org/apache/hudi/client/SparkRDDWriteClient.java](https://app.codecov.io/gh/apache/hudi/pull/19778?src=pr&el=tree&filepath=hudi-client%2Fhudi-spark-client%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2FSparkRDDWriteClient.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1zcGFyay1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpZW50L1NwYXJrUkREV3JpdGVDbGllbnQuamF2YQ==) | `91.17% <ø> (-0.16%)` | :arrow_down: |
| [.../java/org/apache/hudi/client/BaseHoodieClient.java](https://app.codecov.io/gh/apache/hudi/pull/19778?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2FBaseHoodieClient.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC9CYXNlSG9vZGllQ2xpZW50LmphdmE=) | `78.96% <32.55%> (-10.83%)` | :arrow_down: |
| [.../org/apache/hudi/client/BaseHoodieWriteClient.java](https://app.codecov.io/gh/apache/hudi/pull/19778?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fclient%2FBaseHoodieWriteClient.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2NsaWVudC9CYXNlSG9vZGllV3JpdGVDbGllbnQuamF2YQ==) | `79.62% <32.60%> (-3.97%)` | :arrow_down: |
... and [1 file with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/19778/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [MINOR] Release every client resource on close even when an earlier one fails [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19778:
URL: https://github.com/apache/hudi/pull/19778#issuecomment-5450496800
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5a314786a08105f0dcbc7b51ae6a13115fe95d7f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5a314786a08105f0dcbc7b51ae6a13115fe95d7f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5a314786a08105f0dcbc7b51ae6a13115fe95d7f UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [MINOR] Release every client resource on close even when an earlier one fails [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19778:
URL: https://github.com/apache/hudi/pull/19778#issuecomment-5494135642
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5a314786a08105f0dcbc7b51ae6a13115fe95d7f",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16664",
"triggerID" : "5a314786a08105f0dcbc7b51ae6a13115fe95d7f",
"triggerType" : "PUSH"
}, {
"hash" : "fba93d23e1e373db9261737dbfdfb07fa2d48c4a",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "fba93d23e1e373db9261737dbfdfb07fa2d48c4a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5a314786a08105f0dcbc7b51ae6a13115fe95d7f Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16664)
* fba93d23e1e373db9261737dbfdfb07fa2d48c4a UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [MINOR] Release every client resource on close even when an earlier one fails [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19778:
URL: https://github.com/apache/hudi/pull/19778#issuecomment-5496655341
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5a314786a08105f0dcbc7b51ae6a13115fe95d7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16664",
"triggerID" : "5a314786a08105f0dcbc7b51ae6a13115fe95d7f",
"triggerType" : "PUSH"
}, {
"hash" : "fba93d23e1e373db9261737dbfdfb07fa2d48c4a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16739",
"triggerID" : "fba93d23e1e373db9261737dbfdfb07fa2d48c4a",
"triggerType" : "PUSH"
}, {
"hash" : "cfd155181248fbb3718a8582762fd11a75d22a2a",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16741",
"triggerID" : "cfd155181248fbb3718a8582762fd11a75d22a2a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* cfd155181248fbb3718a8582762fd11a75d22a2a Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16741)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [MINOR] Release every client resource on close even when an earlier one fails [hudi]
Posted by "hudi-agent (via GitHub)" <gi...@apache.org>.
hudi-agent commented on code in PR #19778:
URL: https://github.com/apache/hudi/pull/19778#discussion_r3904790420
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/BaseHoodieClient.java:
##########
@@ -142,19 +142,78 @@ private static TransactionManager buildTransactionManager(HoodieEngineContext co
this.metrics = new HoodieMetrics(config, storage);
this.txnManager = transactionManager;
this.timeGenerator = timeGenerator;
- startEmbeddedServerView();
- runClientInitCallbacks();
+ try {
+ startEmbeddedServerView();
+ runClientInitCallbacks();
+ } catch (RuntimeException | Error e) {
+ // The constructor is not returning, so no caller ever gets an instance to close(): release here or leak.
+ releaseAfterFailedInit(e);
Review Comment:
🤖 The guard here covers the base constructor's tail, so the bad-callback-class case is handled — nice. But the client is always a subclass, and the subclass work that runs after `super()` returns isn't covered: `BaseHoodieWriteClient`'s `createIndex(writeConfig)` / `emitIndexTypeMetrics(...)` and `SparkRDDWriteClient`'s `new SparkRDDTableServiceClient(...)` all execute once the base constructor has already started the embedded server, heartbeat, and txn-manager write lock. If any of those throw, the same resources leak exactly as before, since no instance escapes to call close(). Is closing that broader path in scope here, or intended as a follow-up?
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag quality.</i></sub>
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/BaseHoodieWriteClient.java:
##########
@@ -1661,12 +1661,35 @@ public void releaseResources(String instantTime) {
@Override
public void close() {
- // Stop timeline-server if running
- super.close();
- // Calling this here releases any resources used by your index, so make sure to finish any related operations
- // before this point
- this.index.close();
- this.tableServiceClient.close();
+ // The index and the table service client are this class's own resources; a failure while the
+ // base client releases its own must not leave them open.
+ Exception failure = null;
+ try {
+ // Stop timeline-server if running
+ super.close();
+ } catch (Exception e) {
+ failure = e;
+ }
+ try {
+ // Calling this here releases any resources used by your index, so make sure to finish any related operations
+ // before this point
+ this.index.close();
+ } catch (Exception e) {
+ failure = failure == null ? e : addSuppressed(failure, e);
+ }
+ try {
+ this.tableServiceClient.close();
+ } catch (Exception e) {
+ failure = failure == null ? e : addSuppressed(failure, e);
+ }
+ if (failure != null) {
+ throw failure instanceof RuntimeException ? (RuntimeException) failure : new HoodieException(failure);
+ }
+ }
Review Comment:
🤖 nit: the parent class calls this same helper `appendFailure` — could you use the same name here for consistency? `addSuppressed` also shadows `Throwable.addSuppressed`, which might cause a double-take. And since the parent's version handles `null` internally, the subclass's callers wouldn't need the inline `failure == null ? e :` guard if this were aligned.
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag quality.</i></sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [MINOR] Release every client resource on close even when an earlier one fails [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19778:
URL: https://github.com/apache/hudi/pull/19778#issuecomment-5495358331
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5a314786a08105f0dcbc7b51ae6a13115fe95d7f",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16664",
"triggerID" : "5a314786a08105f0dcbc7b51ae6a13115fe95d7f",
"triggerType" : "PUSH"
}, {
"hash" : "fba93d23e1e373db9261737dbfdfb07fa2d48c4a",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16739",
"triggerID" : "fba93d23e1e373db9261737dbfdfb07fa2d48c4a",
"triggerType" : "PUSH"
}, {
"hash" : "cfd155181248fbb3718a8582762fd11a75d22a2a",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16741",
"triggerID" : "cfd155181248fbb3718a8582762fd11a75d22a2a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5a314786a08105f0dcbc7b51ae6a13115fe95d7f Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16664)
* fba93d23e1e373db9261737dbfdfb07fa2d48c4a Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16739)
* cfd155181248fbb3718a8582762fd11a75d22a2a Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16741)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [MINOR] Release every client resource on close even when an earlier one fails [hudi]
Posted by "hudi-agent (via GitHub)" <gi...@apache.org>.
hudi-agent commented on code in PR #19778:
URL: https://github.com/apache/hudi/pull/19778#discussion_r3911253343
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/SparkRDDWriteClient.java:
##########
@@ -80,9 +80,22 @@ public SparkRDDWriteClient(HoodieEngineContext context, HoodieWriteConfig client
public SparkRDDWriteClient(HoodieEngineContext context, HoodieWriteConfig writeConfig,
Option<EmbeddedTimelineService> timelineService) {
super(context, writeConfig, timelineService, SparkUpgradeDowngradeHelper.getInstance());
- DistributedRegistryUtil.createWrapperFileSystemRegistries(context, writeConfig);
- this.tableServiceClient = new SparkRDDTableServiceClient<T>(context, writeConfig, getTimelineServer());
- checkSpeculativeExecution();
+ try {
+ DistributedRegistryUtil.createWrapperFileSystemRegistries(context, writeConfig);
+ this.tableServiceClient = new SparkRDDTableServiceClient<T>(context, writeConfig, getTimelineServer());
+ // The speculative-execution guardrail throws from here, after the base client is fully built.
+ checkSpeculativeExecution();
+ } catch (RuntimeException | Error e) {
+ if (this.tableServiceClient != null) {
+ try {
+ this.tableServiceClient.close();
+ } catch (Exception closeFailure) {
+ e.addSuppressed(closeFailure);
+ }
+ }
+ releaseAfterFailedInit(e);
Review Comment:
🤖 When this subclass body throws (e.g. checkSpeculativeExecution() with spark.speculation=true, or the SparkRDDTableServiceClient ctor), releaseAfterFailedInit(e) binds to the base one-arg method, which releases the timeline server/heartbeat/txnManager but not this.index that the base BaseHoodieWriteClient constructor already created. Both close() and the two-arg releaseAfterFailedInit close the index, so this is the one path that skips it. Inert today since every HoodieIndex.close() is a no-op, but would leak for any index that holds resources. Worth closing the index here too?
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag quality.</i></sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [MINOR] Release every client resource on close even when an earlier one fails [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19778:
URL: https://github.com/apache/hudi/pull/19778#issuecomment-5494211876
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5a314786a08105f0dcbc7b51ae6a13115fe95d7f",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16664",
"triggerID" : "5a314786a08105f0dcbc7b51ae6a13115fe95d7f",
"triggerType" : "PUSH"
}, {
"hash" : "fba93d23e1e373db9261737dbfdfb07fa2d48c4a",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16739",
"triggerID" : "fba93d23e1e373db9261737dbfdfb07fa2d48c4a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5a314786a08105f0dcbc7b51ae6a13115fe95d7f Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16664)
* fba93d23e1e373db9261737dbfdfb07fa2d48c4a Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16739)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [MINOR] Release every client resource on close even when an earlier one fails [hudi]
Posted by "hudi-agent (via GitHub)" <gi...@apache.org>.
hudi-agent commented on code in PR #19778:
URL: https://github.com/apache/hudi/pull/19778#discussion_r3901132025
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/BaseHoodieWriteClient.java:
##########
@@ -1661,12 +1661,35 @@ public void releaseResources(String instantTime) {
@Override
public void close() {
- // Stop timeline-server if running
- super.close();
- // Calling this here releases any resources used by your index, so make sure to finish any related operations
- // before this point
- this.index.close();
- this.tableServiceClient.close();
+ // The index and the table service client are this class's own resources; a failure while the
+ // base client releases its own must not leave them open.
+ Exception failure = null;
+ try {
+ // Stop timeline-server if running
+ super.close();
+ } catch (Exception e) {
+ failure = e;
+ }
+ try {
+ // Calling this here releases any resources used by your index, so make sure to finish any related operations
+ // before this point
+ this.index.close();
+ } catch (Exception e) {
+ failure = failure == null ? e : addSuppressed(failure, e);
+ }
+ try {
+ this.tableServiceClient.close();
+ } catch (Exception e) {
+ failure = failure == null ? e : addSuppressed(failure, e);
+ }
+ if (failure != null) {
+ throw failure instanceof RuntimeException ? (RuntimeException) failure : new HoodieException(failure);
+ }
+ }
Review Comment:
🤖 nit: `addSuppressed` does the same thing as `appendFailure` in the base class, just with a different name and without the null-guard (so the call sites need an inline ternary). Could you either promote `appendFailure` to `protected` and reuse it here, or at least align the name so the two close() paths are visually consistent?
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag quality.</i></sub>
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/BaseHoodieClient.java:
##########
@@ -151,10 +151,40 @@ private static TransactionManager buildTransactionManager(HoodieEngineContext co
*/
@Override
public void close() {
- stopEmbeddedServerView(true);
- this.context.setJobStatus("", "");
- this.heartbeatClient.close();
- this.txnManager.close();
+ // Each step owns a resource the others do not, and the transaction manager holds the write
+ // lock, so an earlier failure must not stop the rest from being released.
+ Exception failure = null;
+ try {
+ stopEmbeddedServerView(true);
+ } catch (Exception e) {
+ failure = e;
+ }
+ try {
+ this.context.setJobStatus("", "");
+ } catch (Exception e) {
+ failure = appendFailure(failure, e);
+ }
+ try {
+ this.heartbeatClient.close();
+ } catch (Exception e) {
+ failure = appendFailure(failure, e);
+ }
+ try {
+ this.txnManager.close();
Review Comment:
🤖 The new test covers the subclass path (index fails -> tableServiceClient still closes), but the base chain the description calls out as most important — a failure in stopEmbeddedServerView/heartbeatClient still releasing txnManager's write lock — isn't directly exercised (super.close() fully succeeds in that test). Might be worth a small test for that path too, e.g. inject a failure into stopEmbeddedServerView and assert txnManager.close() still runs.
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag quality.</i></sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [MINOR] Release every client resource on close even when an earlier one fails [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19778:
URL: https://github.com/apache/hudi/pull/19778#issuecomment-5844967872
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5a314786a08105f0dcbc7b51ae6a13115fe95d7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16664",
"triggerID" : "5a314786a08105f0dcbc7b51ae6a13115fe95d7f",
"triggerType" : "PUSH"
}, {
"hash" : "fba93d23e1e373db9261737dbfdfb07fa2d48c4a",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16739",
"triggerID" : "fba93d23e1e373db9261737dbfdfb07fa2d48c4a",
"triggerType" : "PUSH"
}, {
"hash" : "cfd155181248fbb3718a8582762fd11a75d22a2a",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16741",
"triggerID" : "cfd155181248fbb3718a8582762fd11a75d22a2a",
"triggerType" : "PUSH"
}, {
"hash" : "dee61ecea7947ea273eb5c24dd4f7243ed1d155f",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17290",
"triggerID" : "dee61ecea7947ea273eb5c24dd4f7243ed1d155f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* cfd155181248fbb3718a8582762fd11a75d22a2a Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16741)
* dee61ecea7947ea273eb5c24dd4f7243ed1d155f Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=17290)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [MINOR] Release every client resource on close even when an earlier one fails [hudi]
Posted by "hudi-agent (via GitHub)" <gi...@apache.org>.
hudi-agent commented on code in PR #19778:
URL: https://github.com/apache/hudi/pull/19778#discussion_r4127226350
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/BaseHoodieWriteClient.java:
##########
@@ -1661,12 +1691,38 @@ public void releaseResources(String instantTime) {
@Override
public void close() {
- // Stop timeline-server if running
- super.close();
- // Calling this here releases any resources used by your index, so make sure to finish any related operations
- // before this point
- this.index.close();
- this.tableServiceClient.close();
+ // The index and the table service client are this class's own resources; a failure while the
+ // base client releases its own must not leave them open.
+ Exception failure = null;
+ try {
+ // Stop timeline-server if running
+ super.close();
+ } catch (Exception e) {
+ failure = e;
+ }
+ try {
+ // Calling this here releases any resources used by your index, so make sure to finish any related operations
+ // before this point
+ this.index.close();
+ } catch (Exception e) {
+ failure = appendFailure(failure, e);
+ }
+ try {
+ this.tableServiceClient.close();
+ } catch (Exception e) {
+ failure = appendFailure(failure, e);
+ }
+ if (failure != null) {
+ throw failure instanceof RuntimeException ? (RuntimeException) failure : new HoodieException(failure);
+ }
+ }
+
+ private static Exception appendFailure(Exception previousFailure, Exception failure) {
Review Comment:
🤖 nit: `appendFailure` is an exact copy of the one in `BaseHoodieClient`. Could you make that one `protected static` and reuse it here? A small `closeQuietly(Runnable, failure)`-style helper might also cut down the repeated try/catch blocks in both `close()` methods and in `SparkRDDWriteClient`.
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag quality.</i></sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [MINOR] Release every client resource on close even when an earlier one fails [hudi]
Posted by "PDGGK (via GitHub)" <gi...@apache.org>.
PDGGK commented on code in PR #19778:
URL: https://github.com/apache/hudi/pull/19778#discussion_r4110743294
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/SparkRDDWriteClient.java:
##########
@@ -80,9 +80,22 @@ public SparkRDDWriteClient(HoodieEngineContext context, HoodieWriteConfig client
public SparkRDDWriteClient(HoodieEngineContext context, HoodieWriteConfig writeConfig,
Option<EmbeddedTimelineService> timelineService) {
super(context, writeConfig, timelineService, SparkUpgradeDowngradeHelper.getInstance());
- DistributedRegistryUtil.createWrapperFileSystemRegistries(context, writeConfig);
- this.tableServiceClient = new SparkRDDTableServiceClient<T>(context, writeConfig, getTimelineServer());
- checkSpeculativeExecution();
+ try {
+ DistributedRegistryUtil.createWrapperFileSystemRegistries(context, writeConfig);
+ this.tableServiceClient = new SparkRDDTableServiceClient<T>(context, writeConfig, getTimelineServer());
+ // The speculative-execution guardrail throws from here, after the base client is fully built.
+ checkSpeculativeExecution();
+ } catch (RuntimeException | Error e) {
+ if (this.tableServiceClient != null) {
+ try {
+ this.tableServiceClient.close();
+ } catch (Exception closeFailure) {
+ e.addSuppressed(closeFailure);
+ }
+ }
+ releaseAfterFailedInit(e);
Review Comment:
Good catch, fixed in dee61ec. The two-argument `releaseAfterFailedInit` is now `protected final`, and this path passes `getIndex()`, so the index is closed here too, the same as in `close()`. `testIndexIsClosedWhenSpeculativeExecutionGuardrailFails` enables `spark.speculation` with a mock index and checks that `close()` is called. It fails with the old one-argument call.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [MINOR] Release every client resource on close even when an earlier one fails [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19778:
URL: https://github.com/apache/hudi/pull/19778#issuecomment-5450549315
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5a314786a08105f0dcbc7b51ae6a13115fe95d7f",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16664",
"triggerID" : "5a314786a08105f0dcbc7b51ae6a13115fe95d7f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5a314786a08105f0dcbc7b51ae6a13115fe95d7f Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16664)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [MINOR] Release every client resource on close even when an earlier one fails [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19778:
URL: https://github.com/apache/hudi/pull/19778#issuecomment-5495537551
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5a314786a08105f0dcbc7b51ae6a13115fe95d7f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16664",
"triggerID" : "5a314786a08105f0dcbc7b51ae6a13115fe95d7f",
"triggerType" : "PUSH"
}, {
"hash" : "fba93d23e1e373db9261737dbfdfb07fa2d48c4a",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16739",
"triggerID" : "fba93d23e1e373db9261737dbfdfb07fa2d48c4a",
"triggerType" : "PUSH"
}, {
"hash" : "cfd155181248fbb3718a8582762fd11a75d22a2a",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16741",
"triggerID" : "cfd155181248fbb3718a8582762fd11a75d22a2a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* fba93d23e1e373db9261737dbfdfb07fa2d48c4a Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16739)
* cfd155181248fbb3718a8582762fd11a75d22a2a Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16741)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [MINOR] Release every client resource on close even when an earlier one fails [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #19778:
URL: https://github.com/apache/hudi/pull/19778#discussion_r4332548962
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/BaseHoodieClient.java:
##########
@@ -142,19 +142,79 @@ private static TransactionManager buildTransactionManager(HoodieEngineContext co
this.metrics = new HoodieMetrics(config, storage);
this.txnManager = transactionManager;
this.timeGenerator = timeGenerator;
- startEmbeddedServerView();
- runClientInitCallbacks();
+ try {
+ startEmbeddedServerView();
+ runClientInitCallbacks();
+ } catch (RuntimeException | Error e) {
+ // The constructor is not returning, so no caller ever gets an instance to close(): release here or leak.
+ releaseAfterFailedInit(e);
+ throw e;
+ }
+ }
+
+ /**
+ * Releases what the constructor already acquired, for the case where it cannot hand back an instance.
+ * This mirrors {@link #close()} but must not call it, because subclasses override close() and it would
+ * then run against a half-built object. Subclasses call this from their own constructor once they have
+ * released whatever they added on top.
+ */
+ protected final void releaseAfterFailedInit(Throwable initFailure) {
+ try {
+ stopEmbeddedServerView(true);
Review Comment:
see if we got a chance to use existing `CloseableUtils.closeSuppressing`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org