You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "danny0405 (via GitHub)" <gi...@apache.org> on 2023/12/12 05:51:49 UTC
[PR] [HUDI-7132] Data may be lost in Flink checkpoint [hudi]
danny0405 opened a new pull request, #10312:
URL: https://github.com/apache/hudi/pull/10312
### Change Logs
See the context in https://issues.apache.org/jira/browse/HUDI-7132, there is no need to clean the event by failure task signal, the events would be overridden when new events got received.
### Impact
fix the potential data loss.
### Risk level (write none, low medium or high below)
none
### Documentation Update
_Describe any necessary documentation update if there is any new feature, config, or user-facing change_
- _The config description must be updated if new configs are added or the default value of the configs are changed_
- _Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the
ticket number here and follow the [instruction](https://hudi.apache.org/contribute/developer-setup#website) to make
changes to the website._
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7132] Data may be lost for flink task failure [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on PR #10312:
URL: https://github.com/apache/hudi/pull/10312#issuecomment-1864425480
> Which versions does this affect? @danny0405
This is a fix for 1.0, for older releases, it should be fine for mor tables, for cow and append only table, there is rare case to lost data but I didn't encounter any feedback in production. We still cherry pick this commit into 0.14.1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7132] Data may be lost for flink task failure [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 merged PR #10312:
URL: https://github.com/apache/hudi/pull/10312
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7132] Data may be lost for flink task failure [hudi]
Posted by "cuibo01 (via GitHub)" <gi...@apache.org>.
cuibo01 commented on PR #10312:
URL: https://github.com/apache/hudi/pull/10312#issuecomment-1851568190
LGTM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7132] Data may be lost for flink task failure [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10312:
URL: https://github.com/apache/hudi/pull/10312#issuecomment-1851388109
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5c971e1a0cafb635ad9cfed0f452751314bdb21c",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5c971e1a0cafb635ad9cfed0f452751314bdb21c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5c971e1a0cafb635ad9cfed0f452751314bdb21c UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7132] Data may be lost for flink task failure [hudi]
Posted by "waywtdcc (via GitHub)" <gi...@apache.org>.
waywtdcc commented on PR #10312:
URL: https://github.com/apache/hudi/pull/10312#issuecomment-1864067171
Which versions does this affect? @danny0405
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7132] Data may be lost for flink task failure [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10312:
URL: https://github.com/apache/hudi/pull/10312#issuecomment-1851635788
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5c971e1a0cafb635ad9cfed0f452751314bdb21c",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21465",
"triggerID" : "5c971e1a0cafb635ad9cfed0f452751314bdb21c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5c971e1a0cafb635ad9cfed0f452751314bdb21c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21465)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7132] Data may be lost for flink task failure [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous commented on PR #10312:
URL: https://github.com/apache/hudi/pull/10312#issuecomment-1851820623
@danny0405 @cuibo01 Read through the JIRA ticket. While I understand how the state of the TM and JM can cause the potential data loss, I am still not very sure how the TM and JM reaches that state.
Can you please describe the Flink job that i can use to try and replicate this?
Thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7132] Data may be lost for flink task failure [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on PR #10312:
URL: https://github.com/apache/hudi/pull/10312#issuecomment-1853186628
> @danny0405 @cuibo01 Read through the JIRA ticket. While I understand how the state of the TM and JM can cause the potential data loss, I am still not very sure how the TM and JM reaches that state.
>
> Can you please describe the Flink job that i can use to try and replicate this?
>
> Thank you!
The reporter is saying when a checkpoint finishes before the JM handles the success event, if a task fails, it would clean the buffer and when committing, some valid files may got deleted, that cause a data loss.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-7132] Data may be lost for flink task failure [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10312:
URL: https://github.com/apache/hudi/pull/10312#issuecomment-1851396107
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5c971e1a0cafb635ad9cfed0f452751314bdb21c",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21465",
"triggerID" : "5c971e1a0cafb635ad9cfed0f452751314bdb21c",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5c971e1a0cafb635ad9cfed0f452751314bdb21c Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21465)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org