You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@james.apache.org by "HesandaLiyanage (via GitHub)" <gi...@apache.org> on 2026/10/20 05:41:43 UTC
[PR] JAMES-4231 BlobIdUpdater SPI and Cassandra reference relocation [james-project]
HesandaLiyanage opened a new pull request, #3260:
URL: https://github.com/apache/james-project/pull/3260
## Scope
Third incremental PR for JAMES-4231, following merged #3244 and #3245. Adds the typed `BlobIdUpdater` SPI and a Cassandra implementation for relocating message header, body, and attachment references.
Targets `feature/s3-object-compaction`. The complete implementation remains available in draft #3193.
## Design
- Scan authoritative `messageV3` and `attachmentV2` references once, notify the liveness observer, and persist predicate-matching references in a paged Cassandra index scoped by run and blob ID.
- Apply conditional reference updates without overwriting changed references or recreating deleted rows. Retain the prepared index for retries after partial failure.
- Repair historical header references in both secondary tables when a header read reports a missing blob, using `messageV3` as the source of truth.
- Expose explicit asynchronous index cleanup. No index TTL is used, because expiry during relocation could hide live references.
- Keep mailbox code independent of chunk layout and compression.
## Validation and draft status
Production compilation passed with JDK 21:
```sh
mvn -o -q -pl mailbox/cassandra -am -Dmaven.test.skip=true compile
```
No new tests have been added or run. This PR stays **draft** pending Cassandra coverage for generated CQL, pagination, conditional writes, partial-failure retries, cleanup/cancellation, stale-header repair, and attachment TTL handling.
## Limits and follow-up
- The prepared updater assumes a completed generation with no new authoritative references into it; passes affecting that generation must be serialized.
- Cleanup currently scans index partition keys across retained runs and deletes only its own partitions. Highly shared blob IDs can still produce large Cassandra partitions.
- Very short remaining attachment TTL causes relocation to fail so the original physical object remains available for retry.
- Production schema registration, factory bindings, task lifecycle integration, and abandoned-run recovery are deferred to wiring work.
- PostgreSQL is next, followed by compaction core/property tests, wiring, WebAdmin/E2E, and final documentation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
Re: [PR] JAMES-4231 BlobIdUpdater SPI and Cassandra reference relocation [james-project]
Posted by "HesandaLiyanage (via GitHub)" <gi...@apache.org>.
HesandaLiyanage commented on PR #3260:
URL: https://github.com/apache/james-project/pull/3260#issuecomment-6285361289
> No tests at all here for this?
You're right, I missed the tests here. My bad. I'll add them and push an update :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
Re: [PR] JAMES-4231 BlobIdUpdater SPI and Cassandra reference relocation [james-project]
Posted by "Arsnael (via GitHub)" <gi...@apache.org>.
Arsnael commented on PR #3260:
URL: https://github.com/apache/james-project/pull/3260#issuecomment-6285461879
> You're right, I missed the tests here. My bad. I'll add them and push an update :)
No problem, it's alright. Thanks :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org