You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "jonvex (via GitHub)" <gi...@apache.org> on 2023/08/17 17:48:08 UTC
[GitHub] [hudi] jonvex opened a new pull request, #9468: [HUDI-6718] Check Timeline Before Transitioning Inflight Clean in Multiwriter Scenario
jonvex opened a new pull request, #9468:
URL: https://github.com/apache/hudi/pull/9468
### Change Logs
If two cleans start at nearly the same time, they will both attempt to execute the same clean instances. This does not cause any data corruption, but will cause a writer to fail when they attempt to create the commit in the timeline. This is because the commit will have already been written by the first writer. Now, we check the timeline before transitioning state.
### Impact
No writers will fail in this scenario now
### Risk level (write none, low medium or high below)
low
### Documentation Update
N/A
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on a diff in pull request #9468: [HUDI-6718] Check Timeline Before Transitioning Inflight Clean in Multiwriter Scenario
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan commented on code in PR #9468:
URL: https://github.com/apache/hudi/pull/9468#discussion_r1303181318
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/clean/CleanActionExecutor.java:
##########
@@ -278,4 +280,14 @@ public HoodieCleanMetadata execute() {
// This requires the CleanActionExecutor to be refactored as BaseCommitActionExecutor
return cleanMetadataList.size() > 0 ? cleanMetadataList.get(cleanMetadataList.size() - 1) : null;
}
+
+ private void checkIfOtherWriterCommitted(HoodieInstant hoodieInstant, HoodieIOException e) {
+ table.getMetaClient().reloadActiveTimeline();
+ if (table.getCleanTimeline().filterCompletedInstants().containsInstant(hoodieInstant.getTimestamp())) {
+ LOG.warn("Clean operation was completed by another writer for instant: " + hoodieInstant);
+ } else {
+ LOG.warn("Failed to perform previous clean operation, instant: " + hoodieInstant, e);
Review Comment:
LOG.error()
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9468: [HUDI-6718] Check Timeline Before Transitioning Inflight Clean in Multiwriter Scenario
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #9468:
URL: https://github.com/apache/hudi/pull/9468#issuecomment-1683059173
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ac44e8c1ee6266c53a613ec96dbd89a7223da4c7",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19341",
"triggerID" : "ac44e8c1ee6266c53a613ec96dbd89a7223da4c7",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ac44e8c1ee6266c53a613ec96dbd89a7223da4c7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19341)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] jonvex commented on pull request #9468: [HUDI-6718] Check Timeline Before Transitioning Inflight Clean in Multiwriter Scenario
Posted by "jonvex (via GitHub)" <gi...@apache.org>.
jonvex commented on PR #9468:
URL: https://github.com/apache/hudi/pull/9468#issuecomment-1691715270
@nsivabalan Azure CI has passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9468: [HUDI-6718] Check Timeline Before Transitioning Inflight Clean in Multiwriter Scenario
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #9468:
URL: https://github.com/apache/hudi/pull/9468#issuecomment-1690448684
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ac44e8c1ee6266c53a613ec96dbd89a7223da4c7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19341",
"triggerID" : "ac44e8c1ee6266c53a613ec96dbd89a7223da4c7",
"triggerType" : "PUSH"
}, {
"hash" : "599baa2fe937dd35d24fcb2315e99d084cf31c10",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19360",
"triggerID" : "599baa2fe937dd35d24fcb2315e99d084cf31c10",
"triggerType" : "PUSH"
}, {
"hash" : "6451d5f7db3326eb691185ec1b4351dc6cbbdd1b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6451d5f7db3326eb691185ec1b4351dc6cbbdd1b",
"triggerType" : "PUSH"
}, {
"hash" : "9b2af18e26cea1d9ff7239d749a89ed35e9b4de8",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19427",
"triggerID" : "9b2af18e26cea1d9ff7239d749a89ed35e9b4de8",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6451d5f7db3326eb691185ec1b4351dc6cbbdd1b UNKNOWN
* 9b2af18e26cea1d9ff7239d749a89ed35e9b4de8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19427)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9468: [HUDI-6718] Check Timeline Before Transitioning Inflight Clean in Multiwriter Scenario
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #9468:
URL: https://github.com/apache/hudi/pull/9468#issuecomment-1684356223
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ac44e8c1ee6266c53a613ec96dbd89a7223da4c7",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19341",
"triggerID" : "ac44e8c1ee6266c53a613ec96dbd89a7223da4c7",
"triggerType" : "PUSH"
}, {
"hash" : "599baa2fe937dd35d24fcb2315e99d084cf31c10",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19360",
"triggerID" : "599baa2fe937dd35d24fcb2315e99d084cf31c10",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ac44e8c1ee6266c53a613ec96dbd89a7223da4c7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19341)
* 599baa2fe937dd35d24fcb2315e99d084cf31c10 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19360)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on a diff in pull request #9468: [HUDI-6718] Check Timeline Before Transitioning Inflight Clean in Multiwriter Scenario
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan commented on code in PR #9468:
URL: https://github.com/apache/hudi/pull/9468#discussion_r1298611005
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/clean/CleanActionExecutor.java:
##########
@@ -260,7 +264,12 @@ public HoodieCleanMetadata execute() {
} else {
LOG.info("Finishing previously unfinished cleaner instant=" + hoodieInstant);
try {
- cleanMetadataList.add(runPendingClean(table, hoodieInstant));
+ HoodieCleanMetadata finishedCleanMetadata = runPendingClean(table, hoodieInstant);
+ if (finishedCleanMetadata != null) {
+ cleanMetadataList.add(finishedCleanMetadata);
+ } else {
+ LOG.warn("Clean completed by ");
Review Comment:
can we fix the log statement properly
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9468: [HUDI-6718] Check Timeline Before Transitioning Inflight Clean in Multiwriter Scenario
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #9468:
URL: https://github.com/apache/hudi/pull/9468#issuecomment-1690218261
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ac44e8c1ee6266c53a613ec96dbd89a7223da4c7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19341",
"triggerID" : "ac44e8c1ee6266c53a613ec96dbd89a7223da4c7",
"triggerType" : "PUSH"
}, {
"hash" : "599baa2fe937dd35d24fcb2315e99d084cf31c10",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19360",
"triggerID" : "599baa2fe937dd35d24fcb2315e99d084cf31c10",
"triggerType" : "PUSH"
}, {
"hash" : "6451d5f7db3326eb691185ec1b4351dc6cbbdd1b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6451d5f7db3326eb691185ec1b4351dc6cbbdd1b",
"triggerType" : "PUSH"
}, {
"hash" : "9b2af18e26cea1d9ff7239d749a89ed35e9b4de8",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19427",
"triggerID" : "9b2af18e26cea1d9ff7239d749a89ed35e9b4de8",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 599baa2fe937dd35d24fcb2315e99d084cf31c10 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19360)
* 6451d5f7db3326eb691185ec1b4351dc6cbbdd1b UNKNOWN
* 9b2af18e26cea1d9ff7239d749a89ed35e9b4de8 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19427)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on a diff in pull request #9468: [HUDI-6718] Check Timeline Before Transitioning Inflight Clean in Multiwriter Scenario
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan commented on code in PR #9468:
URL: https://github.com/apache/hudi/pull/9468#discussion_r1298772243
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/clean/CleanActionExecutor.java:
##########
@@ -262,8 +262,13 @@ public HoodieCleanMetadata execute() {
try {
cleanMetadataList.add(runPendingClean(table, hoodieInstant));
} catch (Exception e) {
- LOG.warn("Failed to perform previous clean operation, instant: " + hoodieInstant, e);
- throw e;
+ table.getMetaClient().reloadActiveTimeline();
Review Comment:
can we catch a specific exception if its feasible ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on a diff in pull request #9468: [HUDI-6718] Check Timeline Before Transitioning Inflight Clean in Multiwriter Scenario
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan commented on code in PR #9468:
URL: https://github.com/apache/hudi/pull/9468#discussion_r1298610736
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/clean/CleanActionExecutor.java:
##########
@@ -223,6 +223,10 @@ private HoodieCleanMetadata runClean(HoodieTable<T, I, K, O> table, HoodieInstan
);
if (!skipLocking) {
this.txnManager.beginTransaction(Option.of(inflightInstant), Option.empty());
+ table.getMetaClient().reloadActiveTimeline();
Review Comment:
already we do reload the timeline at L218. So, do we still need this reload?
if we are concerned about TOCTOU, even there could be a gap here right. just after L 228, the clean could have been completed by another concurrent writer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9468: [HUDI-6718] Check Timeline Before Transitioning Inflight Clean in Multiwriter Scenario
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #9468:
URL: https://github.com/apache/hudi/pull/9468#issuecomment-1684501205
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ac44e8c1ee6266c53a613ec96dbd89a7223da4c7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19341",
"triggerID" : "ac44e8c1ee6266c53a613ec96dbd89a7223da4c7",
"triggerType" : "PUSH"
}, {
"hash" : "599baa2fe937dd35d24fcb2315e99d084cf31c10",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19360",
"triggerID" : "599baa2fe937dd35d24fcb2315e99d084cf31c10",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 599baa2fe937dd35d24fcb2315e99d084cf31c10 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19360)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9468: [HUDI-6718] Check Timeline Before Transitioning Inflight Clean in Multiwriter Scenario
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #9468:
URL: https://github.com/apache/hudi/pull/9468#issuecomment-1682773010
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ac44e8c1ee6266c53a613ec96dbd89a7223da4c7",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19341",
"triggerID" : "ac44e8c1ee6266c53a613ec96dbd89a7223da4c7",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ac44e8c1ee6266c53a613ec96dbd89a7223da4c7 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19341)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9468: [HUDI-6718] Check Timeline Before Transitioning Inflight Clean in Multiwriter Scenario
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #9468:
URL: https://github.com/apache/hudi/pull/9468#issuecomment-1682762105
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ac44e8c1ee6266c53a613ec96dbd89a7223da4c7",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "ac44e8c1ee6266c53a613ec96dbd89a7223da4c7",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ac44e8c1ee6266c53a613ec96dbd89a7223da4c7 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9468: [HUDI-6718] Check Timeline Before Transitioning Inflight Clean in Multiwriter Scenario
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #9468:
URL: https://github.com/apache/hudi/pull/9468#issuecomment-1690189258
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ac44e8c1ee6266c53a613ec96dbd89a7223da4c7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19341",
"triggerID" : "ac44e8c1ee6266c53a613ec96dbd89a7223da4c7",
"triggerType" : "PUSH"
}, {
"hash" : "599baa2fe937dd35d24fcb2315e99d084cf31c10",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19360",
"triggerID" : "599baa2fe937dd35d24fcb2315e99d084cf31c10",
"triggerType" : "PUSH"
}, {
"hash" : "6451d5f7db3326eb691185ec1b4351dc6cbbdd1b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6451d5f7db3326eb691185ec1b4351dc6cbbdd1b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 599baa2fe937dd35d24fcb2315e99d084cf31c10 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19360)
* 6451d5f7db3326eb691185ec1b4351dc6cbbdd1b UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan merged pull request #9468: [HUDI-6718] Check Timeline Before Transitioning Inflight Clean in Multiwriter Scenario
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan merged PR #9468:
URL: https://github.com/apache/hudi/pull/9468
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] jonvex commented on a diff in pull request #9468: [HUDI-6718] Check Timeline Before Transitioning Inflight Clean in Multiwriter Scenario
Posted by "jonvex (via GitHub)" <gi...@apache.org>.
jonvex commented on code in PR #9468:
URL: https://github.com/apache/hudi/pull/9468#discussion_r1298622560
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/clean/CleanActionExecutor.java:
##########
@@ -223,6 +223,10 @@ private HoodieCleanMetadata runClean(HoodieTable<T, I, K, O> table, HoodieInstan
);
if (!skipLocking) {
this.txnManager.beginTransaction(Option.of(inflightInstant), Option.empty());
+ table.getMetaClient().reloadActiveTimeline();
Review Comment:
When we are inside the lock it can't right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9468: [HUDI-6718] Check Timeline Before Transitioning Inflight Clean in Multiwriter Scenario
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #9468:
URL: https://github.com/apache/hudi/pull/9468#issuecomment-1690205920
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ac44e8c1ee6266c53a613ec96dbd89a7223da4c7",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19341",
"triggerID" : "ac44e8c1ee6266c53a613ec96dbd89a7223da4c7",
"triggerType" : "PUSH"
}, {
"hash" : "599baa2fe937dd35d24fcb2315e99d084cf31c10",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19360",
"triggerID" : "599baa2fe937dd35d24fcb2315e99d084cf31c10",
"triggerType" : "PUSH"
}, {
"hash" : "6451d5f7db3326eb691185ec1b4351dc6cbbdd1b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6451d5f7db3326eb691185ec1b4351dc6cbbdd1b",
"triggerType" : "PUSH"
}, {
"hash" : "9b2af18e26cea1d9ff7239d749a89ed35e9b4de8",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9b2af18e26cea1d9ff7239d749a89ed35e9b4de8",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 599baa2fe937dd35d24fcb2315e99d084cf31c10 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19360)
* 6451d5f7db3326eb691185ec1b4351dc6cbbdd1b UNKNOWN
* 9b2af18e26cea1d9ff7239d749a89ed35e9b4de8 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on pull request #9468: [HUDI-6718] Check Timeline Before Transitioning Inflight Clean in Multiwriter Scenario
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan commented on PR #9468:
URL: https://github.com/apache/hudi/pull/9468#issuecomment-1690896492
<img width="1231" alt="image" src="https://github.com/apache/hudi/assets/513218/645da90d-5ae4-4d4f-8c0a-7b6cc49e0c0a">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #9468: [HUDI-6718] Check Timeline Before Transitioning Inflight Clean in Multiwriter Scenario
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #9468:
URL: https://github.com/apache/hudi/pull/9468#issuecomment-1684347971
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "ac44e8c1ee6266c53a613ec96dbd89a7223da4c7",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19341",
"triggerID" : "ac44e8c1ee6266c53a613ec96dbd89a7223da4c7",
"triggerType" : "PUSH"
}, {
"hash" : "599baa2fe937dd35d24fcb2315e99d084cf31c10",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "599baa2fe937dd35d24fcb2315e99d084cf31c10",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* ac44e8c1ee6266c53a613ec96dbd89a7223da4c7 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=19341)
* 599baa2fe937dd35d24fcb2315e99d084cf31c10 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org