You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "tsreaper (via GitHub)" <gi...@apache.org> on 2026/03/16 08:52:09 UTC
[PR] [core] Do not clean base and delta manifests in ChangelogDeletion [paimon]
tsreaper opened a new pull request, #7429:
URL: https://github.com/apache/paimon/pull/7429
### Purpose
Currently, `ChangelogDeletion` will clean base and delta manifests. However these manifests are also cleaned in `SnapshotDeletion`.
Consider the following scenario:
* `snapshot-1` is changed to `changelog-1`. However due to some issue, its base and delta manifests are not cleaned.
* `snapshot-1` is changed to `changelog-2`. Its base and delta manifests are cleaned.
* When expiring `changelog-1`, as `changelog-2` has no manifests, all manifests in `changelog-1` will be cleaned, even if they're still in use by later snapshots.
This PR fixes the bug by not cleaning base and delta manifests in `ChangelogDeletion`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [core] Do not clean base and delta manifests in ChangelogDeletion [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on code in PR #7429:
URL: https://github.com/apache/paimon/pull/7429#discussion_r2944146592
##########
paimon-core/src/main/java/org/apache/paimon/operation/ChangelogDeletion.java:
##########
@@ -63,25 +63,24 @@ public ChangelogDeletion(
public void cleanUnusedDataFiles(Changelog changelog, Predicate<ExpireFileEntry> skipper) {
if (changelog.changelogManifestList() != null) {
deleteAddedDataFiles(changelog.changelogManifestList());
- }
-
- if (manifestList.exists(changelog.deltaManifestList())) {
- cleanUnusedDataFiles(changelog.deltaManifestList(), skipper);
+ } else {
+ if (manifestList.exists(changelog.deltaManifestList())) {
+ cleanUnusedDataFiles(changelog.deltaManifestList(), skipper);
+ }
}
}
@Override
public void cleanUnusedManifests(Changelog changelog, Set<String> skippingSet) {
if (changelog.changelogManifestList() != null) {
cleanUnusedManifestList(changelog.changelogManifestList(), skippingSet);
- }
-
- if (manifestList.exists(changelog.deltaManifestList())) {
- cleanUnusedManifestList(changelog.deltaManifestList(), skippingSet);
- }
-
- if (manifestList.exists(changelog.baseManifestList())) {
- cleanUnusedManifestList(changelog.baseManifestList(), skippingSet);
+ } else {
+ if (manifestList.exists(changelog.deltaManifestList())) {
+ cleanUnusedManifestList(changelog.deltaManifestList(), skippingSet);
+ }
+ if (manifestList.exists(changelog.baseManifestList())) {
+ cleanUnusedManifestList(changelog.baseManifestList(), skippingSet);
Review Comment:
Why clean base here, it is very dangerous.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org