You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "linliu-code (via GitHub)" <gi...@apache.org> on 2024/01/17 19:50:19 UTC
[PR] [HUDI-6902] Shutdown metric hooks properly [hudi]
linliu-code opened a new pull request, #10520:
URL: https://github.com/apache/hudi/pull/10520
### Change Logs
Use try-finally statement to wrap the logic.
### Impact
Shutdown the hooks properly. Otherwise, we could have orphan processes in some corner cases.
### Risk level (write none, low medium or high below)
None.
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-6902] Shutdown metric hooks properly [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10520:
URL: https://github.com/apache/hudi/pull/10520#issuecomment-1896602580
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "699b453435f06e4f213051136932a16b5ac5a622",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "699b453435f06e4f213051136932a16b5ac5a622",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 699b453435f06e4f213051136932a16b5ac5a622 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-6902] Shutdown metric hooks properly [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10520:
URL: https://github.com/apache/hudi/pull/10520#issuecomment-1897585438
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "699b453435f06e4f213051136932a16b5ac5a622",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22011",
"triggerID" : "699b453435f06e4f213051136932a16b5ac5a622",
"triggerType" : "PUSH"
}, {
"hash" : "a8be064c7f9b7023b94fb543dc695c0ceeabc54c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a8be064c7f9b7023b94fb543dc695c0ceeabc54c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 699b453435f06e4f213051136932a16b5ac5a622 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22011)
* a8be064c7f9b7023b94fb543dc695c0ceeabc54c UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-6902] Shutdown metric hooks properly [hudi]
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan merged PR #10520:
URL: https://github.com/apache/hudi/pull/10520
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-6902] Shutdown metric hooks properly [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10520:
URL: https://github.com/apache/hudi/pull/10520#issuecomment-1897593614
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "699b453435f06e4f213051136932a16b5ac5a622",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22011",
"triggerID" : "699b453435f06e4f213051136932a16b5ac5a622",
"triggerType" : "PUSH"
}, {
"hash" : "a8be064c7f9b7023b94fb543dc695c0ceeabc54c",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22016",
"triggerID" : "a8be064c7f9b7023b94fb543dc695c0ceeabc54c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 699b453435f06e4f213051136932a16b5ac5a622 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22011)
* a8be064c7f9b7023b94fb543dc695c0ceeabc54c Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22016)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-6902] Shutdown metric hooks properly [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10520:
URL: https://github.com/apache/hudi/pull/10520#issuecomment-1897754588
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "699b453435f06e4f213051136932a16b5ac5a622",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22011",
"triggerID" : "699b453435f06e4f213051136932a16b5ac5a622",
"triggerType" : "PUSH"
}, {
"hash" : "a8be064c7f9b7023b94fb543dc695c0ceeabc54c",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22016",
"triggerID" : "a8be064c7f9b7023b94fb543dc695c0ceeabc54c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a8be064c7f9b7023b94fb543dc695c0ceeabc54c Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22016)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-6902] Shutdown metric hooks properly [hudi]
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan commented on code in PR #10520:
URL: https://github.com/apache/hudi/pull/10520#discussion_r1456675246
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DefaultSource.scala:
##########
@@ -138,17 +138,22 @@ class DefaultSource extends RelationProvider
mode: SaveMode,
optParams: Map[String, String],
df: DataFrame): BaseRelation = {
- if (optParams.get(OPERATION.key).contains(BOOTSTRAP_OPERATION_OPT_VAL)) {
- HoodieSparkSqlWriter.bootstrap(sqlContext, mode, optParams, df)
- HoodieSparkSqlWriter.cleanup()
- } else {
- val (success, _, _, _, _, _) = HoodieSparkSqlWriter.write(sqlContext, mode, optParams, df)
- HoodieSparkSqlWriter.cleanup()
- if (!success) {
- throw new HoodieException("Write to Hudi failed")
+ var shouldThrow = false
+ try {
+ if (optParams.get(OPERATION.key).contains(BOOTSTRAP_OPERATION_OPT_VAL)) {
Review Comment:
can we keep the code block as is. just add a try finally. and clean up within finally
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-6902] Shutdown metric hooks properly [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10520:
URL: https://github.com/apache/hudi/pull/10520#issuecomment-1897308508
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "699b453435f06e4f213051136932a16b5ac5a622",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22011",
"triggerID" : "699b453435f06e4f213051136932a16b5ac5a622",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 699b453435f06e4f213051136932a16b5ac5a622 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22011)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-6902] Shutdown metric hooks properly [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10520:
URL: https://github.com/apache/hudi/pull/10520#issuecomment-1896615890
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "699b453435f06e4f213051136932a16b5ac5a622",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22011",
"triggerID" : "699b453435f06e4f213051136932a16b5ac5a622",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 699b453435f06e4f213051136932a16b5ac5a622 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=22011)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org