You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "nsivabalan (via GitHub)" <gi...@apache.org> on 2023/02/14 21:17:23 UTC
[GitHub] [hudi] nsivabalan opened a new pull request, #7948: [HUDI-5794] Failing new commits on any pending restore commits
nsivabalan opened a new pull request, #7948:
URL: https://github.com/apache/hudi/pull/7948
### Change Logs
Fail new writes if there are any pending restore commits in timeline.
### Impact
If restore is making progress, no new writes should be allowed. Also, if it failed, we should not let users start writing since the table will be in in-consistent state.
### Risk level (write none, low medium or high below)
low.
### Documentation Update
N/A
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7948: [HUDI-5794] Failing new commits on any pending restore commits
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #7948:
URL: https://github.com/apache/hudi/pull/7948#issuecomment-1440808843
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "44ccfacd787a566ff73343955389b080576ab225",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15176",
"triggerID" : "44ccfacd787a566ff73343955389b080576ab225",
"triggerType" : "PUSH"
}, {
"hash" : "0d3b0bbbf1aa55f5fe65f3a052d9d3839687898b",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15343",
"triggerID" : "0d3b0bbbf1aa55f5fe65f3a052d9d3839687898b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 44ccfacd787a566ff73343955389b080576ab225 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15176)
* 0d3b0bbbf1aa55f5fe65f3a052d9d3839687898b Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15343)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xushiyan commented on a diff in pull request #7948: [HUDI-5794] Failing new commits on any pending restore commits
Posted by "xushiyan (via GitHub)" <gi...@apache.org>.
xushiyan commented on code in PR #7948:
URL: https://github.com/apache/hudi/pull/7948#discussion_r1106547724
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/BaseHoodieWriteClient.java:
##########
@@ -839,6 +842,12 @@ private void startCommitWithTime(String instantTime, String actionType, HoodieTa
private void startCommit(String instantTime, String actionType, HoodieTableMetaClient metaClient) {
LOG.info("Generate a new instant time: " + instantTime + " action: " + actionType);
+ // check there are no inflight restore before starting a new commit.
+ ValidationUtils.checkArgument(metaClient.getActiveTimeline().getRestoreTimeline().filterInflightsAndRequested().countInstants() == 0,
+ "Found pending restore in active timeline. Please complete the restore fully before proceeding. As of now, table could be in an inconsistent state. "
+ + "Pending restores: " + Arrays.toString(metaClient.getActiveTimeline().getRestoreTimeline()
+ .filterInflightsAndRequested().getInstantsAsStream().map(instant -> instant.getTimestamp()).collect(Collectors.toList()).toArray()));
Review Comment:
`metaClient.getActiveTimeline().getRestoreTimeline().filterInflightsAndRequested()` avoid invoking this twice?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7948: [HUDI-5794] Failing new commits on any pending restore commits
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #7948:
URL: https://github.com/apache/hudi/pull/7948#issuecomment-1440986915
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "44ccfacd787a566ff73343955389b080576ab225",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15176",
"triggerID" : "44ccfacd787a566ff73343955389b080576ab225",
"triggerType" : "PUSH"
}, {
"hash" : "0d3b0bbbf1aa55f5fe65f3a052d9d3839687898b",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15343",
"triggerID" : "0d3b0bbbf1aa55f5fe65f3a052d9d3839687898b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0d3b0bbbf1aa55f5fe65f3a052d9d3839687898b Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15343)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7948: [HUDI-5794] Failing new commits on any pending restore commits
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #7948:
URL: https://github.com/apache/hudi/pull/7948#issuecomment-1430411544
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "44ccfacd787a566ff73343955389b080576ab225",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "44ccfacd787a566ff73343955389b080576ab225",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 44ccfacd787a566ff73343955389b080576ab225 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7948: [HUDI-5794] Failing new commits on any pending restore commits
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #7948:
URL: https://github.com/apache/hudi/pull/7948#issuecomment-1430665212
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "44ccfacd787a566ff73343955389b080576ab225",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15176",
"triggerID" : "44ccfacd787a566ff73343955389b080576ab225",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 44ccfacd787a566ff73343955389b080576ab225 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15176)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on pull request #7948: [HUDI-5794] Failing new commits on any pending restore commits
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan commented on PR #7948:
URL: https://github.com/apache/hudi/pull/7948#issuecomment-1442719726
CI is green
<img width="1246" alt="image" src="https://user-images.githubusercontent.com/513218/221079206-ec0c571f-2ce9-4338-ba36-7354902e3cb3.png">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on pull request #7948: [HUDI-5794] Failing new commits on any pending restore commits
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan commented on PR #7948:
URL: https://github.com/apache/hudi/pull/7948#issuecomment-1440783540
@xushiyan : ready for review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan merged pull request #7948: [HUDI-5794] Failing new commits on any pending restore commits
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan merged PR #7948:
URL: https://github.com/apache/hudi/pull/7948
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7948: [HUDI-5794] Failing new commits on any pending restore commits
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #7948:
URL: https://github.com/apache/hudi/pull/7948#issuecomment-1430418430
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "44ccfacd787a566ff73343955389b080576ab225",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15176",
"triggerID" : "44ccfacd787a566ff73343955389b080576ab225",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 44ccfacd787a566ff73343955389b080576ab225 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15176)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7948: [HUDI-5794] Failing new commits on any pending restore commits
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #7948:
URL: https://github.com/apache/hudi/pull/7948#issuecomment-1440800516
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "44ccfacd787a566ff73343955389b080576ab225",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15176",
"triggerID" : "44ccfacd787a566ff73343955389b080576ab225",
"triggerType" : "PUSH"
}, {
"hash" : "0d3b0bbbf1aa55f5fe65f3a052d9d3839687898b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0d3b0bbbf1aa55f5fe65f3a052d9d3839687898b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 44ccfacd787a566ff73343955389b080576ab225 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=15176)
* 0d3b0bbbf1aa55f5fe65f3a052d9d3839687898b UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org