You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "LiJie20190102 (via GitHub)" <gi...@apache.org> on 2023/04/11 03:23:01 UTC
[GitHub] [hudi] LiJie20190102 opened a new pull request, #8424: [HUDI-6057]Fixing shutting down deltastreamer properly when post writ…
LiJie20190102 opened a new pull request, #8424:
URL: https://github.com/apache/hudi/pull/8424
…e termination strategy is enabled
### Change Logs
Looks like table services are not propoerly shut down when post write termination kicks in. This patch fixes the flow, and in regular ingestion, any table service will shut down correctly when it is terminated after enabling write and meets the conditions (including compaction/clustering).
### Impact
Improves the usability of post write termination strategy for deltastreamer in continuous mode.
Also tested w/ a manual deltastremaer execution. w/o the fix, spark job just hangs, and w/ the fix, it shutsdown gracefully,including compaction/clustering.
### Risk level (write none, low medium or high below)
low.
### Documentation Update
_Describe any necessary documentation update if there is any new feature, config, or user-facing change_
- _The config description must be updated if new configs are added or the default value of the configs are changed_
- _Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the
ticket number here and follow the [instruction](https://hudi.apache.org/contribute/developer-setup#website) to make
changes to the website._
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8424: [HUDI-6057]Fixing shutting down deltastreamer properly when post writ…
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8424:
URL: https://github.com/apache/hudi/pull/8424#issuecomment-1502637932
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "bb35387f4dae37ede0283db54d314a6056f9a5a6",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "bb35387f4dae37ede0283db54d314a6056f9a5a6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* bb35387f4dae37ede0283db54d314a6056f9a5a6 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8424: [HUDI-6057]Fixing shutting down deltastreamer properly when post writ…
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8424:
URL: https://github.com/apache/hudi/pull/8424#issuecomment-1502672929
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "bb35387f4dae37ede0283db54d314a6056f9a5a6",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16244",
"triggerID" : "bb35387f4dae37ede0283db54d314a6056f9a5a6",
"triggerType" : "PUSH"
}, {
"hash" : "345130fd3abcb403bd64dbdc81951f7a39b6e062",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16245",
"triggerID" : "345130fd3abcb403bd64dbdc81951f7a39b6e062",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* bb35387f4dae37ede0283db54d314a6056f9a5a6 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16244)
* 345130fd3abcb403bd64dbdc81951f7a39b6e062 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16245)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] codope commented on pull request #8424: [HUDI-6057] Fix deltastreamer shutdown when post write termination strategy enabled
Posted by "codope (via GitHub)" <gi...@apache.org>.
codope commented on PR #8424:
URL: https://github.com/apache/hudi/pull/8424#issuecomment-1504419143
@LiJie20190102 Have you checked if this fix solves your problem? https://github.com/apache/hudi/pull/8173
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] LiJie20190102 commented on pull request #8424: [HUDI-6065] Fix deltastreamer shutdown when post write termination strategy enabled
Posted by "LiJie20190102 (via GitHub)" <gi...@apache.org>.
LiJie20190102 commented on PR #8424:
URL: https://github.com/apache/hudi/pull/8424#issuecomment-1504737203
> @LiJie20190102 Have you checked if this fix solves your problem? #8173
#8173 can indeed solve the shutdown problem, but if there are still pending compaction/clustering instants, forcibly closing them at this time is not reasonable in my opinion
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8424: [HUDI-6057]Fixing shutting down deltastreamer properly when post writ…
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8424:
URL: https://github.com/apache/hudi/pull/8424#issuecomment-1502665320
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "bb35387f4dae37ede0283db54d314a6056f9a5a6",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16244",
"triggerID" : "bb35387f4dae37ede0283db54d314a6056f9a5a6",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* bb35387f4dae37ede0283db54d314a6056f9a5a6 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16244)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] PrabhuJoseph commented on pull request #8424: [HUDI-6057]Fixing shutting down deltastreamer properly when post writ…
Posted by "PrabhuJoseph (via GitHub)" <gi...@apache.org>.
PrabhuJoseph commented on PR #8424:
URL: https://github.com/apache/hudi/pull/8424#issuecomment-1503046463
@LiJie20190102 Looks you have mentioned the wrong JIRA number. HUDI-6057 is related to Flink Upgrade.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] LiJie20190102 commented on pull request #8424: [HUDI-6057]Fixing shutting down deltastreamer properly when post writ…
Posted by "LiJie20190102 (via GitHub)" <gi...@apache.org>.
LiJie20190102 commented on PR #8424:
URL: https://github.com/apache/hudi/pull/8424#issuecomment-1502716352
@danny0405 @nsivabalan @codope I don't understand how to specify who to review the code for me. Can you help me specify it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] LiJie20190102 commented on pull request #8424: [HUDI-6065] Fix deltastreamer shutdown when post write termination strategy enabled
Posted by "LiJie20190102 (via GitHub)" <gi...@apache.org>.
LiJie20190102 commented on PR #8424:
URL: https://github.com/apache/hudi/pull/8424#issuecomment-1504737582
> @LiJie20190102 Looks you have mentioned the wrong JIRA number. [HUDI-6057](https://issues.apache.org/jira/browse/HUDI-6057) is related to Flink Upgrade.
Sorry, I have modified it to the correct one
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8424: [HUDI-6057]Fixing shutting down deltastreamer properly when post writ…
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8424:
URL: https://github.com/apache/hudi/pull/8424#issuecomment-1502669098
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "bb35387f4dae37ede0283db54d314a6056f9a5a6",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16244",
"triggerID" : "bb35387f4dae37ede0283db54d314a6056f9a5a6",
"triggerType" : "PUSH"
}, {
"hash" : "345130fd3abcb403bd64dbdc81951f7a39b6e062",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "345130fd3abcb403bd64dbdc81951f7a39b6e062",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* bb35387f4dae37ede0283db54d314a6056f9a5a6 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16244)
* 345130fd3abcb403bd64dbdc81951f7a39b6e062 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] codope commented on a diff in pull request #8424: [HUDI-6065] Fix deltastreamer shutdown when post write termination strategy enabled
Posted by "codope (via GitHub)" <gi...@apache.org>.
codope commented on code in PR #8424:
URL: https://github.com/apache/hudi/pull/8424#discussion_r1165047701
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/HoodieDeltaStreamer.java:
##########
@@ -742,7 +742,7 @@ protected Pair<CompletableFuture, ExecutorService> startService() {
scheduledCompactionInstantAndRDD.isPresent() ? HoodieJavaRDD.of(scheduledCompactionInstantAndRDD.get().getRight()) : null);
if (requestShutdownIfNeeded(lastWriteStatuses)) {
LOG.warn("Closing and shutting down ingestion service");
- error = true;
+ waitAsyncServicesFinishAndStop();
Review Comment:
Shouldn't the shutdown in line 747 below take of shutting down async services too?
cc @nsivabalan
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/HoodieDeltaStreamer.java:
##########
@@ -764,6 +764,33 @@ protected Pair<CompletableFuture, ExecutorService> startService() {
}, executor), executor);
}
+ /**
+ * Wait till outstanding pending compaction/clustering
+ */
+ private void waitAsyncServicesFinishAndStop() {
Review Comment:
Can you please add a test covering this scenario - deltastreamer shutdown with pending compaction or clustering?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #8424: [HUDI-6057]Fixing shutting down deltastreamer properly when post writ…
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #8424:
URL: https://github.com/apache/hudi/pull/8424#issuecomment-1502748402
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "bb35387f4dae37ede0283db54d314a6056f9a5a6",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16244",
"triggerID" : "bb35387f4dae37ede0283db54d314a6056f9a5a6",
"triggerType" : "PUSH"
}, {
"hash" : "345130fd3abcb403bd64dbdc81951f7a39b6e062",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16245",
"triggerID" : "345130fd3abcb403bd64dbdc81951f7a39b6e062",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 345130fd3abcb403bd64dbdc81951f7a39b6e062 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=16245)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org