You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "yadavay-amzn (via GitHub)" <gi...@apache.org> on 2026/07/14 22:36:05 UTC
[PR] Core: Fail rewrite_data_files when the starting snapshot has been expired [iceberg]
yadavay-amzn opened a new pull request, #17204:
URL: https://github.com/apache/iceberg/pull/17204
Closes #17202.
## Problem
When `rewrite_data_files` runs with partial progress enabled and a concurrent `expire_snapshots` removes the rewrite's starting snapshot before the file-group commits complete, the action either throws a `NullPointerException` or silently completes with zero committed rewrites - losing all compaction progress with no error surfaced to the caller.
- `RewriteDataFilesCommitManager#commitFileGroups` calls `table.snapshot(startingSnapshotId).sequenceNumber()`, which throws `NullPointerException` because `Table#snapshot(long)` returns `null` for an expired snapshot (`use-starting-sequence-number` defaults to `true`).
- `BaseCommitService#commitReadyCommitGroups` catches and ignores the failure via a catch-all, so it is counted as a tolerable partial-progress failure and only a `LOG.warn` is emitted.
## Change
- Null-guard the starting snapshot in `commitFileGroups` with a clear message indicating the starting snapshot has been expired (likely by a concurrent `expire_snapshots`).
- Propagate the non-retriable `IllegalStateException` from `BaseCommitService` instead of swallowing it, while still tolerating genuine retriable commit conflicts (`ValidationException` / `CommitFailedException`) as before.
## Testing
Added `TestRewriteDataFilesCommitManager` with two tests that reproduce both failure paths (the NPE and the silent swallow). Both fail on the unpatched code and pass with the fix. Existing `TestCommitService` behavior is unchanged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Core: Fail rewrite_data_files when the starting snapshot has been expired [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #17204: Core: Fail rewrite_data_files when the starting snapshot has been expired
URL: https://github.com/apache/iceberg/pull/17204
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Core: Fail rewrite_data_files when the starting snapshot has been expired [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #17204:
URL: https://github.com/apache/iceberg/pull/17204#issuecomment-5287915097
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Core: Fail rewrite_data_files when the starting snapshot has been expired [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #17204:
URL: https://github.com/apache/iceberg/pull/17204#issuecomment-5376681347
This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org