You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2022/11/03 04:54:43 UTC
[GitHub] [flink-table-store] tsreaper opened a new pull request, #350: [FLINK-29805] Fix incorrect snapshot filter when snapshots are committing too slow
tsreaper opened a new pull request, #350:
URL: https://github.com/apache/flink-table-store/pull/350
Table Store sink continuously fails with "Trying to add file which is already added" when snapshot committing is slow.
This is due to a bug in `FileStoreCommitImpl#filterCommitted`. When this method finds an identifier, it removes the identifier from a map. However different snapshots may have the same identifier (for example an APPEND commit and the following COMPACT commit will have the same identifier), so we need to use another set to check for identifiers.
When snapshot committing is fast there is at most 1 identifier to check after the job restarts, so nothing happens. However when snapshot committing is slow, there will be multiple identifiers to check and some identifiers will be mistakenly kept.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-table-store] LadyForest commented on a diff in pull request #350: [FLINK-29805] Fix incorrect snapshot filter when snapshots are committing too slow
Posted by GitBox <gi...@apache.org>.
LadyForest commented on code in PR #350:
URL: https://github.com/apache/flink-table-store/pull/350#discussion_r1013585500
##########
flink-table-store-core/src/main/java/org/apache/flink/table/store/file/utils/SnapshotManager.java:
##########
@@ -109,7 +139,7 @@ public Long findLatest() throws IOException {
return findByListFiles(Math::max);
}
- public Long findEarliest() throws IOException {
+ private Long findEarliest() throws IOException {
Review Comment:
Nit: add `@Nullable` annotation?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-table-store] tsreaper merged pull request #350: [FLINK-29805] Fix incorrect snapshot filter when snapshots are committing too slow
Posted by GitBox <gi...@apache.org>.
tsreaper merged PR #350:
URL: https://github.com/apache/flink-table-store/pull/350
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org