You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@polaris.apache.org by "anxkhn (via GitHub)" <gi...@apache.org> on 2026/07/11 12:34:10 UTC
[PR] fix: return full commit log when a commit tail is shorter than the fetch page [polaris]
anxkhn opened a new pull request, #5040:
URL: https://github.com/apache/polaris/pull/5040
`Commits.commitLog` (the NoSQL persistence commit log, most-recent-first) paged
through history by allocating a fixed-size `ObjRef[REVERSE_COMMIT_FETCH_SIZE]` (20)
and filling only `tail.length` entries. When a commit's recent-ancestor `tail` is
shorter than 20, the trailing array slots stayed null. `Persistence.fetchMany`
returns a null at each null-id position, and the iterator treats the first null as
end-of-history:
```java
if (c == null) {
// stop at commits that do not exist, the history has been cut at that point.
return endOfData();
}
```
So a short tail silently truncated the natural-order commit log at that point.
Why it was not observed before: the default
`polaris.persistence.reference-previous-head-count` is 20, which equals the
internal fetch page size, so the trailing null always lined up with genuine
end-of-history. With a smaller `reference-previous-head-count` the log stops early.
Maintenance walks this commit log to decide which objects are still referenced
(`RetainedCollector`), so a truncated log can drop still-referenced objects.
The fix sizes the fetch array to `min(REVERSE_COMMIT_FETCH_SIZE, tail.length)` so
no null padding ever reaches `fetchMany` or the iterator. This mirrors the growing
`ArrayList` (capped at `REVERSE_COMMIT_FETCH_SIZE`, with an `isEmpty` guard) that
the sibling `commitLogReversed` already uses, so the two directions now allocate
consistently.
Added a parameterized regression test `commitLogShortTail` in the shared fixture
`BaseTestCommitLogImpl` (runs across all NoSQL backends). It commits 50 times with
`reference-previous-head-count` set below the page size (values 2, 3, 5), asserts
`commitLogReversed` returns the full history as a baseline, then asserts the
natural-order `commitLog` returns all 50 commits (it returned fewer before the fix).
No public interface or configuration changes.
#### Checklist
- [x] Don't disclose security issues! (contact security@apache.org)
- [x] Clearly explained why the changes are needed, or linked related issues
- [x] Added/updated tests with good coverage, or manually tested (and explained how)
- [ ] Added comments for complex logic (n/a: the fix is a 3-line array-sizing change; the test carries an explanatory Javadoc)
- [x] Updated `CHANGELOG.md` (Unreleased > Fixes)
- [ ] Updated documentation in `site/content/in-dev/unreleased` (n/a: bug fix, no user-facing config or behavior additions)
> No `Fixes #` line: there is no filed issue for this specific truncation. (Issue
> #4000 / closed PR #4007 concern bounding the offset-lookup traversal, a different
> code path, not this page-fetch null-padding.) If a maintainer prefers, I can file
> a short issue and link it.
---
## Files changed (4)
- `persistence/nosql/persistence/impl/src/main/java/org/apache/polaris/persistence/nosql/impl/commits/CommitsImpl.java`
(the fix: size the page array to `min(REVERSE_COMMIT_FETCH_SIZE, tail.length)`; 5 lines)
- `persistence/nosql/persistence/impl/src/testFixtures/java/org/apache/polaris/persistence/nosql/impl/commits/BaseTestCommitLogImpl.java`
(new parameterized `commitLogShortTail` regression test; runs on every backend via `TestCommitLogImpl`)
- `persistence/nosql/persistence/impl/build.gradle.kts`
(+1 `testFixturesImplementation(project(":polaris-idgen-api"))`: the test builds a
custom-config `Persistence` via `backend.newPersistence(...)`, which needs the
`IdGenerator`/`MonotonicClock` API on the test-fixture classpath, same shape as
`BaseCorrectness`)
- `CHANGELOG.md` (one bullet under `## [Unreleased]` > `### Fixes`)
Commit: `9363f8538` on `fix/commitlog-null-padding-truncation`. No `Signed-off-by`
(Polaris is ASF ICLA, not DCO). Conventional Commits.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix: return full commit log when a commit tail is shorter than the fetch page [polaris]
Posted by "dimas-b (via GitHub)" <gi...@apache.org>.
dimas-b merged PR #5040:
URL: https://github.com/apache/polaris/pull/5040
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix: return full commit log when a commit tail is shorter than the fetch page [polaris]
Posted by "anxkhn (via GitHub)" <gi...@apache.org>.
anxkhn commented on PR #5040:
URL: https://github.com/apache/polaris/pull/5040#issuecomment-4989734850
Pushed a revision addressing the latest review/CI feedback.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix: return full commit log when a commit tail is shorter than the fetch page [polaris]
Posted by "anxkhn (via GitHub)" <gi...@apache.org>.
anxkhn commented on PR #5040:
URL: https://github.com/apache/polaris/pull/5040#issuecomment-5094920539
rebased onto the latest main and resolved the changelog conflict while preserving the fix. formatting, compilation, and the relevant module check pass. please re-review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@polaris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org