You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/08/22 09:52:48 UTC
[GitHub] [hudi] junyuc25 opened a new pull request, #6468: Shutdown CloudWatch reporter when query completes
junyuc25 opened a new pull request, #6468:
URL: https://github.com/apache/hudi/pull/6468
Add delta streamer tests
Only shutdown in non-continuous mode
### Change Logs
_Describe context and summary for this change. Highlight if any code was copied._
### Impact
_Describe any public API or user-facing feature change or any performance impact._
**Risk level: none | low | medium | high**
_Choose one. If medium or high, explain what verification was done to mitigate the risks._
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6468: Shutdown CloudWatch reporter when query completes
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6468:
URL: https://github.com/apache/hudi/pull/6468#issuecomment-1222507984
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "314239443e55183226b790e4c353e0be74d42a40",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10873",
"triggerID" : "314239443e55183226b790e4c353e0be74d42a40",
"triggerType" : "PUSH"
}, {
"hash" : "314239443e55183226b790e4c353e0be74d42a40",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10873",
"triggerID" : "1222121369",
"triggerType" : "MANUAL"
}, {
"hash" : "6cfae1a094b5f5b60d664ecefb735072bfc1e39f",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10874",
"triggerID" : "6cfae1a094b5f5b60d664ecefb735072bfc1e39f",
"triggerType" : "PUSH"
}, {
"hash" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6cfae1a094b5f5b60d664ecefb735072bfc1e39f Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10874)
* 3fd2cb61fe8ae7f0b260a517383e6fe37233ec44 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6468: Shutdown CloudWatch reporter when query completes
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6468:
URL: https://github.com/apache/hudi/pull/6468#issuecomment-1222598975
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "314239443e55183226b790e4c353e0be74d42a40",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10873",
"triggerID" : "314239443e55183226b790e4c353e0be74d42a40",
"triggerType" : "PUSH"
}, {
"hash" : "314239443e55183226b790e4c353e0be74d42a40",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10873",
"triggerID" : "1222121369",
"triggerType" : "MANUAL"
}, {
"hash" : "6cfae1a094b5f5b60d664ecefb735072bfc1e39f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10874",
"triggerID" : "6cfae1a094b5f5b60d664ecefb735072bfc1e39f",
"triggerType" : "PUSH"
}, {
"hash" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10876",
"triggerID" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"triggerType" : "PUSH"
}, {
"hash" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1222541315",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 3fd2cb61fe8ae7f0b260a517383e6fe37233ec44 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10876)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6468: [HUDI-4731] Shutdown CloudWatch reporter when query completes
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6468:
URL: https://github.com/apache/hudi/pull/6468#issuecomment-1238206599
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "314239443e55183226b790e4c353e0be74d42a40",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10873",
"triggerID" : "314239443e55183226b790e4c353e0be74d42a40",
"triggerType" : "PUSH"
}, {
"hash" : "314239443e55183226b790e4c353e0be74d42a40",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10873",
"triggerID" : "1222121369",
"triggerType" : "MANUAL"
}, {
"hash" : "6cfae1a094b5f5b60d664ecefb735072bfc1e39f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10874",
"triggerID" : "6cfae1a094b5f5b60d664ecefb735072bfc1e39f",
"triggerType" : "PUSH"
}, {
"hash" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10876",
"triggerID" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"triggerType" : "PUSH"
}, {
"hash" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10876",
"triggerID" : "1222541315",
"triggerType" : "MANUAL"
}, {
"hash" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10876",
"triggerID" : "1227396610",
"triggerType" : "MANUAL"
}, {
"hash" : "a005404b94133b32508d6b8ef2bff77335ec7278",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10948",
"triggerID" : "a005404b94133b32508d6b8ef2bff77335ec7278",
"triggerType" : "PUSH"
}, {
"hash" : "1ba0c71be8f3ba9f0e3f663bc299b60b5360b409",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11174",
"triggerID" : "1ba0c71be8f3ba9f0e3f663bc299b60b5360b409",
"triggerType" : "PUSH"
}, {
"hash" : "2e4b3c86879105d680c419842078dbbff893434b",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11184",
"triggerID" : "2e4b3c86879105d680c419842078dbbff893434b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 2e4b3c86879105d680c419842078dbbff893434b Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11184)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xushiyan merged pull request #6468: [HUDI-4731] Shutdown CloudWatch reporter when query completes
Posted by GitBox <gi...@apache.org>.
xushiyan merged PR #6468:
URL: https://github.com/apache/hudi/pull/6468
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6468: Shutdown CloudWatch reporter when query completes
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6468:
URL: https://github.com/apache/hudi/pull/6468#issuecomment-1227408654
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "314239443e55183226b790e4c353e0be74d42a40",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10873",
"triggerID" : "314239443e55183226b790e4c353e0be74d42a40",
"triggerType" : "PUSH"
}, {
"hash" : "314239443e55183226b790e4c353e0be74d42a40",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10873",
"triggerID" : "1222121369",
"triggerType" : "MANUAL"
}, {
"hash" : "6cfae1a094b5f5b60d664ecefb735072bfc1e39f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10874",
"triggerID" : "6cfae1a094b5f5b60d664ecefb735072bfc1e39f",
"triggerType" : "PUSH"
}, {
"hash" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10876",
"triggerID" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"triggerType" : "PUSH"
}, {
"hash" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10876",
"triggerID" : "1222541315",
"triggerType" : "MANUAL"
}, {
"hash" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1227396610",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 3fd2cb61fe8ae7f0b260a517383e6fe37233ec44 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10876)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xushiyan commented on a diff in pull request #6468: [HUDI-4731] Shutdown CloudWatch reporter when query completes
Posted by GitBox <gi...@apache.org>.
xushiyan commented on code in PR #6468:
URL: https://github.com/apache/hudi/pull/6468#discussion_r964515489
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/HoodieDeltaStreamer.java:
##########
@@ -208,6 +209,7 @@ public void sync() throws Exception {
throw ex;
} finally {
deltaSyncService.ifPresent(DeltaSyncService::close);
+ Metrics.shutdown();
Review Comment:
functionality-wise, it should be ok. i think it's a question of where is the single place we can clean up all resources. `org.apache.hudi.utilities.deltastreamer.DeltaSync#close` looks like the right place. It looks more of a minor clean up to me now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6468: Shutdown CloudWatch reporter when query completes
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6468:
URL: https://github.com/apache/hudi/pull/6468#issuecomment-1222413216
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "314239443e55183226b790e4c353e0be74d42a40",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10873",
"triggerID" : "314239443e55183226b790e4c353e0be74d42a40",
"triggerType" : "PUSH"
}, {
"hash" : "314239443e55183226b790e4c353e0be74d42a40",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10873",
"triggerID" : "1222121369",
"triggerType" : "MANUAL"
}, {
"hash" : "6cfae1a094b5f5b60d664ecefb735072bfc1e39f",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10874",
"triggerID" : "6cfae1a094b5f5b60d664ecefb735072bfc1e39f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6cfae1a094b5f5b60d664ecefb735072bfc1e39f Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10874)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] junyuc25 commented on pull request #6468: Shutdown CloudWatch reporter when query completes
Posted by GitBox <gi...@apache.org>.
junyuc25 commented on PR #6468:
URL: https://github.com/apache/hudi/pull/6468#issuecomment-1222533236
Previous successful build: https://dev.azure.com/apache-hudi-ci-org/apache-hudi-ci/_build/results?buildId=10874&view=results
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] umehrot2 commented on a diff in pull request #6468: [HUDI-4731] Shutdown CloudWatch reporter when query completes
Posted by GitBox <gi...@apache.org>.
umehrot2 commented on code in PR #6468:
URL: https://github.com/apache/hudi/pull/6468#discussion_r961072009
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/HoodieDeltaStreamer.java:
##########
@@ -208,6 +209,7 @@ public void sync() throws Exception {
throw ex;
} finally {
deltaSyncService.ifPresent(DeltaSyncService::close);
+ Metrics.shutdown();
Review Comment:
The Metric's reporter already has a shutdown hook of its own, that should run when jvm exits. That should suffice the continuous mode case right ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6468: Shutdown CloudWatch reporter when query completes
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6468:
URL: https://github.com/apache/hudi/pull/6468#issuecomment-1227416879
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "314239443e55183226b790e4c353e0be74d42a40",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10873",
"triggerID" : "314239443e55183226b790e4c353e0be74d42a40",
"triggerType" : "PUSH"
}, {
"hash" : "314239443e55183226b790e4c353e0be74d42a40",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10873",
"triggerID" : "1222121369",
"triggerType" : "MANUAL"
}, {
"hash" : "6cfae1a094b5f5b60d664ecefb735072bfc1e39f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10874",
"triggerID" : "6cfae1a094b5f5b60d664ecefb735072bfc1e39f",
"triggerType" : "PUSH"
}, {
"hash" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10876",
"triggerID" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"triggerType" : "PUSH"
}, {
"hash" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10876",
"triggerID" : "1222541315",
"triggerType" : "MANUAL"
}, {
"hash" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10876",
"triggerID" : "1227396610",
"triggerType" : "MANUAL"
}, {
"hash" : "a005404b94133b32508d6b8ef2bff77335ec7278",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a005404b94133b32508d6b8ef2bff77335ec7278",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 3fd2cb61fe8ae7f0b260a517383e6fe37233ec44 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10876)
* a005404b94133b32508d6b8ef2bff77335ec7278 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6468: [HUDI-4731] Shutdown CloudWatch reporter when query completes
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6468:
URL: https://github.com/apache/hudi/pull/6468#issuecomment-1237903310
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "314239443e55183226b790e4c353e0be74d42a40",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10873",
"triggerID" : "314239443e55183226b790e4c353e0be74d42a40",
"triggerType" : "PUSH"
}, {
"hash" : "314239443e55183226b790e4c353e0be74d42a40",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10873",
"triggerID" : "1222121369",
"triggerType" : "MANUAL"
}, {
"hash" : "6cfae1a094b5f5b60d664ecefb735072bfc1e39f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10874",
"triggerID" : "6cfae1a094b5f5b60d664ecefb735072bfc1e39f",
"triggerType" : "PUSH"
}, {
"hash" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10876",
"triggerID" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"triggerType" : "PUSH"
}, {
"hash" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10876",
"triggerID" : "1222541315",
"triggerType" : "MANUAL"
}, {
"hash" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10876",
"triggerID" : "1227396610",
"triggerType" : "MANUAL"
}, {
"hash" : "a005404b94133b32508d6b8ef2bff77335ec7278",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10948",
"triggerID" : "a005404b94133b32508d6b8ef2bff77335ec7278",
"triggerType" : "PUSH"
}, {
"hash" : "1ba0c71be8f3ba9f0e3f663bc299b60b5360b409",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11174",
"triggerID" : "1ba0c71be8f3ba9f0e3f663bc299b60b5360b409",
"triggerType" : "PUSH"
}, {
"hash" : "2e4b3c86879105d680c419842078dbbff893434b",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11184",
"triggerID" : "2e4b3c86879105d680c419842078dbbff893434b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1ba0c71be8f3ba9f0e3f663bc299b60b5360b409 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11174)
* 2e4b3c86879105d680c419842078dbbff893434b Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11184)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on a diff in pull request #6468: [HUDI-4731] Shutdown CloudWatch reporter when query completes
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #6468:
URL: https://github.com/apache/hudi/pull/6468#discussion_r1250381027
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala:
##########
@@ -594,6 +595,10 @@ object HoodieSparkSqlWriter {
(syncHiveSuccess, common.util.Option.ofNullable(instantTime))
}
+ def cleanup() : Unit = {
Review Comment:
Nice catch ~, can you fire a fix then?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6468: Shutdown CloudWatch reporter when query completes
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6468:
URL: https://github.com/apache/hudi/pull/6468#issuecomment-1222607261
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "314239443e55183226b790e4c353e0be74d42a40",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10873",
"triggerID" : "314239443e55183226b790e4c353e0be74d42a40",
"triggerType" : "PUSH"
}, {
"hash" : "314239443e55183226b790e4c353e0be74d42a40",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10873",
"triggerID" : "1222121369",
"triggerType" : "MANUAL"
}, {
"hash" : "6cfae1a094b5f5b60d664ecefb735072bfc1e39f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10874",
"triggerID" : "6cfae1a094b5f5b60d664ecefb735072bfc1e39f",
"triggerType" : "PUSH"
}, {
"hash" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10876",
"triggerID" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"triggerType" : "PUSH"
}, {
"hash" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10876",
"triggerID" : "1222541315",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 3fd2cb61fe8ae7f0b260a517383e6fe37233ec44 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10876)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] junyuc25 commented on pull request #6468: Shutdown CloudWatch reporter when query completes
Posted by GitBox <gi...@apache.org>.
junyuc25 commented on PR #6468:
URL: https://github.com/apache/hudi/pull/6468#issuecomment-1222121369
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6468: [HUDI-4731] Shutdown CloudWatch reporter when query completes
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6468:
URL: https://github.com/apache/hudi/pull/6468#issuecomment-1237583835
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "314239443e55183226b790e4c353e0be74d42a40",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10873",
"triggerID" : "314239443e55183226b790e4c353e0be74d42a40",
"triggerType" : "PUSH"
}, {
"hash" : "314239443e55183226b790e4c353e0be74d42a40",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10873",
"triggerID" : "1222121369",
"triggerType" : "MANUAL"
}, {
"hash" : "6cfae1a094b5f5b60d664ecefb735072bfc1e39f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10874",
"triggerID" : "6cfae1a094b5f5b60d664ecefb735072bfc1e39f",
"triggerType" : "PUSH"
}, {
"hash" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10876",
"triggerID" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"triggerType" : "PUSH"
}, {
"hash" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10876",
"triggerID" : "1222541315",
"triggerType" : "MANUAL"
}, {
"hash" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10876",
"triggerID" : "1227396610",
"triggerType" : "MANUAL"
}, {
"hash" : "a005404b94133b32508d6b8ef2bff77335ec7278",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10948",
"triggerID" : "a005404b94133b32508d6b8ef2bff77335ec7278",
"triggerType" : "PUSH"
}, {
"hash" : "1ba0c71be8f3ba9f0e3f663bc299b60b5360b409",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1ba0c71be8f3ba9f0e3f663bc299b60b5360b409",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a005404b94133b32508d6b8ef2bff77335ec7278 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10948)
* 1ba0c71be8f3ba9f0e3f663bc299b60b5360b409 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] junyuc25 commented on a diff in pull request #6468: [HUDI-4731] Shutdown CloudWatch reporter when query completes
Posted by GitBox <gi...@apache.org>.
junyuc25 commented on code in PR #6468:
URL: https://github.com/apache/hudi/pull/6468#discussion_r963449470
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/HoodieDeltaStreamer.java:
##########
@@ -208,6 +209,7 @@ public void sync() throws Exception {
throw ex;
} finally {
deltaSyncService.ifPresent(DeltaSyncService::close);
+ Metrics.shutdown();
Review Comment:
Please correct me if I'm wrong. It seems like the DeltaStreamer continuous mode does not have this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6468: [HUDI-4731] Shutdown CloudWatch reporter when query completes
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6468:
URL: https://github.com/apache/hudi/pull/6468#issuecomment-1237586530
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "314239443e55183226b790e4c353e0be74d42a40",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10873",
"triggerID" : "314239443e55183226b790e4c353e0be74d42a40",
"triggerType" : "PUSH"
}, {
"hash" : "314239443e55183226b790e4c353e0be74d42a40",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10873",
"triggerID" : "1222121369",
"triggerType" : "MANUAL"
}, {
"hash" : "6cfae1a094b5f5b60d664ecefb735072bfc1e39f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10874",
"triggerID" : "6cfae1a094b5f5b60d664ecefb735072bfc1e39f",
"triggerType" : "PUSH"
}, {
"hash" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10876",
"triggerID" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"triggerType" : "PUSH"
}, {
"hash" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10876",
"triggerID" : "1222541315",
"triggerType" : "MANUAL"
}, {
"hash" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10876",
"triggerID" : "1227396610",
"triggerType" : "MANUAL"
}, {
"hash" : "a005404b94133b32508d6b8ef2bff77335ec7278",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10948",
"triggerID" : "a005404b94133b32508d6b8ef2bff77335ec7278",
"triggerType" : "PUSH"
}, {
"hash" : "1ba0c71be8f3ba9f0e3f663bc299b60b5360b409",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11174",
"triggerID" : "1ba0c71be8f3ba9f0e3f663bc299b60b5360b409",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a005404b94133b32508d6b8ef2bff77335ec7278 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10948)
* 1ba0c71be8f3ba9f0e3f663bc299b60b5360b409 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11174)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6468: [HUDI-4731] Shutdown CloudWatch reporter when query completes
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6468:
URL: https://github.com/apache/hudi/pull/6468#issuecomment-1237693390
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "314239443e55183226b790e4c353e0be74d42a40",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10873",
"triggerID" : "314239443e55183226b790e4c353e0be74d42a40",
"triggerType" : "PUSH"
}, {
"hash" : "314239443e55183226b790e4c353e0be74d42a40",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10873",
"triggerID" : "1222121369",
"triggerType" : "MANUAL"
}, {
"hash" : "6cfae1a094b5f5b60d664ecefb735072bfc1e39f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10874",
"triggerID" : "6cfae1a094b5f5b60d664ecefb735072bfc1e39f",
"triggerType" : "PUSH"
}, {
"hash" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10876",
"triggerID" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"triggerType" : "PUSH"
}, {
"hash" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10876",
"triggerID" : "1222541315",
"triggerType" : "MANUAL"
}, {
"hash" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10876",
"triggerID" : "1227396610",
"triggerType" : "MANUAL"
}, {
"hash" : "a005404b94133b32508d6b8ef2bff77335ec7278",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10948",
"triggerID" : "a005404b94133b32508d6b8ef2bff77335ec7278",
"triggerType" : "PUSH"
}, {
"hash" : "1ba0c71be8f3ba9f0e3f663bc299b60b5360b409",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11174",
"triggerID" : "1ba0c71be8f3ba9f0e3f663bc299b60b5360b409",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1ba0c71be8f3ba9f0e3f663bc299b60b5360b409 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11174)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6468: [HUDI-4731] Shutdown CloudWatch reporter when query completes
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6468:
URL: https://github.com/apache/hudi/pull/6468#issuecomment-1237896225
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "314239443e55183226b790e4c353e0be74d42a40",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10873",
"triggerID" : "314239443e55183226b790e4c353e0be74d42a40",
"triggerType" : "PUSH"
}, {
"hash" : "314239443e55183226b790e4c353e0be74d42a40",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10873",
"triggerID" : "1222121369",
"triggerType" : "MANUAL"
}, {
"hash" : "6cfae1a094b5f5b60d664ecefb735072bfc1e39f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10874",
"triggerID" : "6cfae1a094b5f5b60d664ecefb735072bfc1e39f",
"triggerType" : "PUSH"
}, {
"hash" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10876",
"triggerID" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"triggerType" : "PUSH"
}, {
"hash" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10876",
"triggerID" : "1222541315",
"triggerType" : "MANUAL"
}, {
"hash" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10876",
"triggerID" : "1227396610",
"triggerType" : "MANUAL"
}, {
"hash" : "a005404b94133b32508d6b8ef2bff77335ec7278",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10948",
"triggerID" : "a005404b94133b32508d6b8ef2bff77335ec7278",
"triggerType" : "PUSH"
}, {
"hash" : "1ba0c71be8f3ba9f0e3f663bc299b60b5360b409",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11174",
"triggerID" : "1ba0c71be8f3ba9f0e3f663bc299b60b5360b409",
"triggerType" : "PUSH"
}, {
"hash" : "2e4b3c86879105d680c419842078dbbff893434b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "2e4b3c86879105d680c419842078dbbff893434b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 1ba0c71be8f3ba9f0e3f663bc299b60b5360b409 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11174)
* 2e4b3c86879105d680c419842078dbbff893434b UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] boneanxs commented on a diff in pull request #6468: [HUDI-4731] Shutdown CloudWatch reporter when query completes
Posted by "boneanxs (via GitHub)" <gi...@apache.org>.
boneanxs commented on code in PR #6468:
URL: https://github.com/apache/hudi/pull/6468#discussion_r1250338414
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala:
##########
@@ -594,6 +595,10 @@ object HoodieSparkSqlWriter {
(syncHiveSuccess, common.util.Option.ofNullable(instantTime))
}
+ def cleanup() : Unit = {
Review Comment:
@junyuc25 Since we already have a shutdown hook to close the metric, is explicitly shutdown metric here still necessary?
I found an issue that if the metric server experiences problems such as high load, it fails to respond quickly, causing queries to be blocked until the timeout, the `Metrics.shutdown` here could block the query to finish until the timeout.
```java
23/07/03 14:16:31 WARN GraphiteReporter: Unable to report to Graphite
java.net.ConnectException: Connection timed out (Connection timed out)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:607)
at java.net.Socket.connect(Socket.java:556)
at java.net.Socket.<init>(Socket.java:452)
at java.net.Socket.<init>(Socket.java:262)
at javax.net.DefaultSocketFactory.createSocket(SocketFactory.java:277)
at org.apache.hudi.com.codahale.metrics.graphite.Graphite.connect(Graphite.java:130)
at org.apache.hudi.com.codahale.metrics.graphite.GraphiteReporter.report(GraphiteReporter.java:263)
at org.apache.hudi.com.codahale.metrics.ScheduledReporter.report(ScheduledReporter.java:237)
at org.apache.hudi.metrics.MetricsGraphiteReporter.report(MetricsGraphiteReporter.java:74)
at org.apache.hudi.metrics.Metrics.reportAndStopReporter(Metrics.java:61)
at org.apache.hudi.metrics.Metrics.shutdown(Metrics.java:105)
at java.lang.Thread.run(Thread.java:748)
```
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metrics/Metrics.java:
##########
@@ -53,14 +53,15 @@ private Metrics(HoodieWriteConfig metricConfig) {
}
reporter.start();
- Runtime.getRuntime().addShutdownHook(new Thread(this::reportAndCloseReporter));
+ Runtime.getRuntime().addShutdownHook(new Thread(Metrics::shutdown));
Review Comment:
we better to catch exceptions to fail it silently?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] boneanxs commented on a diff in pull request #6468: [HUDI-4731] Shutdown CloudWatch reporter when query completes
Posted by "boneanxs (via GitHub)" <gi...@apache.org>.
boneanxs commented on code in PR #6468:
URL: https://github.com/apache/hudi/pull/6468#discussion_r1250340527
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metrics/Metrics.java:
##########
@@ -53,14 +53,15 @@ private Metrics(HoodieWriteConfig metricConfig) {
}
reporter.start();
- Runtime.getRuntime().addShutdownHook(new Thread(this::reportAndCloseReporter));
+ Runtime.getRuntime().addShutdownHook(new Thread(Metrics::shutdown));
Review Comment:
we better to catch exceptions to fail it silently?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6468: Shutdown CloudWatch reporter when query completes
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6468:
URL: https://github.com/apache/hudi/pull/6468#issuecomment-1222320541
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "314239443e55183226b790e4c353e0be74d42a40",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10873",
"triggerID" : "314239443e55183226b790e4c353e0be74d42a40",
"triggerType" : "PUSH"
}, {
"hash" : "314239443e55183226b790e4c353e0be74d42a40",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10873",
"triggerID" : "1222121369",
"triggerType" : "MANUAL"
}, {
"hash" : "6cfae1a094b5f5b60d664ecefb735072bfc1e39f",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10874",
"triggerID" : "6cfae1a094b5f5b60d664ecefb735072bfc1e39f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 314239443e55183226b790e4c353e0be74d42a40 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10873)
* 6cfae1a094b5f5b60d664ecefb735072bfc1e39f Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10874)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6468: Shutdown CloudWatch reporter when query completes
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6468:
URL: https://github.com/apache/hudi/pull/6468#issuecomment-1227424654
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "314239443e55183226b790e4c353e0be74d42a40",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10873",
"triggerID" : "314239443e55183226b790e4c353e0be74d42a40",
"triggerType" : "PUSH"
}, {
"hash" : "314239443e55183226b790e4c353e0be74d42a40",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10873",
"triggerID" : "1222121369",
"triggerType" : "MANUAL"
}, {
"hash" : "6cfae1a094b5f5b60d664ecefb735072bfc1e39f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10874",
"triggerID" : "6cfae1a094b5f5b60d664ecefb735072bfc1e39f",
"triggerType" : "PUSH"
}, {
"hash" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10876",
"triggerID" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"triggerType" : "PUSH"
}, {
"hash" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10876",
"triggerID" : "1222541315",
"triggerType" : "MANUAL"
}, {
"hash" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10876",
"triggerID" : "1227396610",
"triggerType" : "MANUAL"
}, {
"hash" : "a005404b94133b32508d6b8ef2bff77335ec7278",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10948",
"triggerID" : "a005404b94133b32508d6b8ef2bff77335ec7278",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 3fd2cb61fe8ae7f0b260a517383e6fe37233ec44 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10876)
* a005404b94133b32508d6b8ef2bff77335ec7278 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10948)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] boneanxs commented on a diff in pull request #6468: [HUDI-4731] Shutdown CloudWatch reporter when query completes
Posted by "boneanxs (via GitHub)" <gi...@apache.org>.
boneanxs commented on code in PR #6468:
URL: https://github.com/apache/hudi/pull/6468#discussion_r1250435457
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala:
##########
@@ -594,6 +595,10 @@ object HoodieSparkSqlWriter {
(syncHiveSuccess, common.util.Option.ofNullable(instantTime))
}
+ def cleanup() : Unit = {
Review Comment:
Sure, let me fix it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xushiyan commented on a diff in pull request #6468: [HUDI-4731] Shutdown CloudWatch reporter when query completes
Posted by GitBox <gi...@apache.org>.
xushiyan commented on code in PR #6468:
URL: https://github.com/apache/hudi/pull/6468#discussion_r960218864
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/HoodieDeltaStreamer.java:
##########
@@ -208,6 +209,7 @@ public void sync() throws Exception {
throw ex;
} finally {
deltaSyncService.ifPresent(DeltaSyncService::close);
+ Metrics.shutdown();
Review Comment:
this only works with sync once mode. can we also hook it up with continuous-mode's shutdown callback as well?
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DefaultSource.scala:
##########
@@ -181,6 +181,7 @@ class DefaultSource extends RelationProvider
HoodieSparkSqlWriter.bootstrap(sqlContext, mode, optParams, dfWithoutMetaCols)
} else {
HoodieSparkSqlWriter.write(sqlContext, mode, optParams, dfWithoutMetaCols)
+ HoodieSparkSqlWriter.cleanup()
Review Comment:
i think a better place to shutdown metrics is `commitAndPerformPostOperations` in `HoodieSparkSqlWriter`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6468: Shutdown CloudWatch reporter when query completes
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6468:
URL: https://github.com/apache/hudi/pull/6468#issuecomment-1222180097
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "314239443e55183226b790e4c353e0be74d42a40",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10873",
"triggerID" : "314239443e55183226b790e4c353e0be74d42a40",
"triggerType" : "PUSH"
}, {
"hash" : "314239443e55183226b790e4c353e0be74d42a40",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10873",
"triggerID" : "1222121369",
"triggerType" : "MANUAL"
}, {
"hash" : "6cfae1a094b5f5b60d664ecefb735072bfc1e39f",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6cfae1a094b5f5b60d664ecefb735072bfc1e39f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 314239443e55183226b790e4c353e0be74d42a40 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10873)
* 6cfae1a094b5f5b60d664ecefb735072bfc1e39f UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6468: Shutdown CloudWatch reporter when query completes
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6468:
URL: https://github.com/apache/hudi/pull/6468#issuecomment-1222175300
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "314239443e55183226b790e4c353e0be74d42a40",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10873",
"triggerID" : "314239443e55183226b790e4c353e0be74d42a40",
"triggerType" : "PUSH"
}, {
"hash" : "314239443e55183226b790e4c353e0be74d42a40",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "1222121369",
"triggerType" : "MANUAL"
} ]
}-->
## CI report:
* 314239443e55183226b790e4c353e0be74d42a40 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10873)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6468: Shutdown CloudWatch reporter when query completes
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6468:
URL: https://github.com/apache/hudi/pull/6468#issuecomment-1222244707
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "314239443e55183226b790e4c353e0be74d42a40",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10873",
"triggerID" : "314239443e55183226b790e4c353e0be74d42a40",
"triggerType" : "PUSH"
}, {
"hash" : "314239443e55183226b790e4c353e0be74d42a40",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10873",
"triggerID" : "1222121369",
"triggerType" : "MANUAL"
}, {
"hash" : "6cfae1a094b5f5b60d664ecefb735072bfc1e39f",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10874",
"triggerID" : "6cfae1a094b5f5b60d664ecefb735072bfc1e39f",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 314239443e55183226b790e4c353e0be74d42a40 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10873)
* 6cfae1a094b5f5b60d664ecefb735072bfc1e39f Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10874)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] junyuc25 commented on pull request #6468: Shutdown CloudWatch reporter when query completes
Posted by GitBox <gi...@apache.org>.
junyuc25 commented on PR #6468:
URL: https://github.com/apache/hudi/pull/6468#issuecomment-1222541315
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6468: Shutdown CloudWatch reporter when query completes
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6468:
URL: https://github.com/apache/hudi/pull/6468#issuecomment-1222170017
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "314239443e55183226b790e4c353e0be74d42a40",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "314239443e55183226b790e4c353e0be74d42a40",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 314239443e55183226b790e4c353e0be74d42a40 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6468: Shutdown CloudWatch reporter when query completes
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6468:
URL: https://github.com/apache/hudi/pull/6468#issuecomment-1222515108
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "314239443e55183226b790e4c353e0be74d42a40",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10873",
"triggerID" : "314239443e55183226b790e4c353e0be74d42a40",
"triggerType" : "PUSH"
}, {
"hash" : "314239443e55183226b790e4c353e0be74d42a40",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10873",
"triggerID" : "1222121369",
"triggerType" : "MANUAL"
}, {
"hash" : "6cfae1a094b5f5b60d664ecefb735072bfc1e39f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10874",
"triggerID" : "6cfae1a094b5f5b60d664ecefb735072bfc1e39f",
"triggerType" : "PUSH"
}, {
"hash" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10876",
"triggerID" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 3fd2cb61fe8ae7f0b260a517383e6fe37233ec44 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10876)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on pull request #6468: Shutdown CloudWatch reporter when query completes
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on PR #6468:
URL: https://github.com/apache/hudi/pull/6468#issuecomment-1229359771
@zhedoubushishi : can you review this please.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #6468: Shutdown CloudWatch reporter when query completes
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6468:
URL: https://github.com/apache/hudi/pull/6468#issuecomment-1227580565
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "314239443e55183226b790e4c353e0be74d42a40",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10873",
"triggerID" : "314239443e55183226b790e4c353e0be74d42a40",
"triggerType" : "PUSH"
}, {
"hash" : "314239443e55183226b790e4c353e0be74d42a40",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10873",
"triggerID" : "1222121369",
"triggerType" : "MANUAL"
}, {
"hash" : "6cfae1a094b5f5b60d664ecefb735072bfc1e39f",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10874",
"triggerID" : "6cfae1a094b5f5b60d664ecefb735072bfc1e39f",
"triggerType" : "PUSH"
}, {
"hash" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10876",
"triggerID" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"triggerType" : "PUSH"
}, {
"hash" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10876",
"triggerID" : "1222541315",
"triggerType" : "MANUAL"
}, {
"hash" : "3fd2cb61fe8ae7f0b260a517383e6fe37233ec44",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10876",
"triggerID" : "1227396610",
"triggerType" : "MANUAL"
}, {
"hash" : "a005404b94133b32508d6b8ef2bff77335ec7278",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10948",
"triggerID" : "a005404b94133b32508d6b8ef2bff77335ec7278",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a005404b94133b32508d6b8ef2bff77335ec7278 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=10948)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] umehrot2 commented on a diff in pull request #6468: [HUDI-4731] Shutdown CloudWatch reporter when query completes
Posted by GitBox <gi...@apache.org>.
umehrot2 commented on code in PR #6468:
URL: https://github.com/apache/hudi/pull/6468#discussion_r961067147
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DefaultSource.scala:
##########
@@ -181,6 +181,7 @@ class DefaultSource extends RelationProvider
HoodieSparkSqlWriter.bootstrap(sqlContext, mode, optParams, dfWithoutMetaCols)
} else {
HoodieSparkSqlWriter.write(sqlContext, mode, optParams, dfWithoutMetaCols)
+ HoodieSparkSqlWriter.cleanup()
Review Comment:
The reason for doing it here is that, `HoodieSparkSqlWriter` is also used by `HoodieStreamingSink`. We don't want to be closing the reporter for the spark streaming job.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] junyuc25 commented on pull request #6468: Shutdown CloudWatch reporter when query completes
Posted by GitBox <gi...@apache.org>.
junyuc25 commented on PR #6468:
URL: https://github.com/apache/hudi/pull/6468#issuecomment-1227396610
@hudi-bot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org