You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@james.apache.org by "HesandaLiyanage (via GitHub)" <gi...@apache.org> on 2026/09/29 06:38:21 UTC
[PR] JAMES-4231 PostgreSQL: Implement PostgresBlobIdUpdater for S3 compaction support [james-project]
HesandaLiyanage opened a new pull request, #3216:
URL: https://github.com/apache/james-project/pull/3216
### Proposed Changes
This PR provides PostgreSQL support for S3 Object Compaction by implementing `PostgresBlobIdUpdater` and binding it in `PostgresMailboxModule`.
#### Context & Motivation
In #3193, `BlobIdUpdater` was introduced to decouple blob storage from database reference updating during S3 object compaction. However, only `CassandraBlobIdUpdater` was implemented, leaving S3 compaction unavailable for PostgreSQL deployments.
This change brings PostgreSQL to feature parity:
- **`PostgresBlobIdUpdater`**:
- Scans both `message` (`body_blob_id`) and `attachment` (`blob_id`) tables across all domains using paginated queries.
- Observes all live references and tracks candidate IDs matching the target generation condition.
- Replaces references in safe batches (500 IDs per query) targeting primary keys (`message_id` and `id`) with strict checks against existing blob IDs.
- **`PostgresMailboxModule`**:
- Binds `BlobIdUpdater.Factory` to `PostgresBlobIdUpdater.Factory` in singleton scope.
- **Test Suite**:
- Adds `PostgresBlobIdUpdaterTest` covering message body replacement, attachment blob replacement, generation predicate filtering, and non-existent reference idempotency against PostgreSQL testcontainers.
> **Note**: Depends on #3193 for the `BlobIdUpdater` SPI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
Re: [PR] JAMES-4231 PostgreSQL: Implement PostgresBlobIdUpdater for S3 compaction support [james-project]
Posted by "Arsnael (via GitHub)" <gi...@apache.org>.
Arsnael commented on PR #3216:
URL: https://github.com/apache/james-project/pull/3216#issuecomment-5888500048
Hello, some ocmpilation issue reported by the CI:
08:24:41,160 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.14.0:compile (default-compile) on project apache-james-mailbox-postgres: Compilation failure: Compilation failure:
08:24:41,161 [ERROR] /home/jenkins/jenkins-agent/workspace/james_ApacheJames_PR-3216/mailbox/postgres/src/main/java/org/apache/james/mailbox/postgres/mail/PostgresBlobIdUpdater.java:[38,33] cannot find symbol
08:24:41,161 [ERROR] symbol: class BlobIdUpdater
08:24:41,161 [ERROR] location: package org.apache.james.blob.api
08:24:41,161 [ERROR] /home/jenkins/jenkins-agent/workspace/james_ApacheJames_PR-3216/mailbox/postgres/src/main/java/org/apache/james/mailbox/postgres/mail/PostgresBlobIdUpdater.java:[47,47] cannot find symbol
08:24:41,161 [ERROR] symbol: class BlobIdUpdater
08:24:41,161 [ERROR] /home/jenkins/jenkins-agent/workspace/james_ApacheJames_PR-3216/mailbox/postgres/src/main/java/org/apache/james/mailbox/postgres/mail/PostgresBlobIdUpdater.java:[48,57] package BlobIdUpdater does not exist
08:24:41,161 [ERROR] /home/jenkins/jenkins-agent/workspace/james_ApacheJames_PR-3216/mailbox/postgres/src/main/java/org/apache/james/mailbox/postgres/mail/PostgresBlobIdUpdater.java:[61,21] cannot find symbol
08:24:41,161 [ERROR] symbol: class BlobIdUpdater
08:24:41,161 [ERROR] location: class org.apache.james.mailbox.postgres.mail.PostgresBlobIdUpdater.Factory
08:24:41,161 [ERROR] /home/jenkins/jenkins-agent/workspace/james_ApacheJames_PR-3216/mailbox/postgres/src/main/java/org/apache/james/mailbox/postgres/mail/PostgresBlobIdUpdater.java:[60,9] method does not override or implement a method from a supertype
08:24:41,161 [ERROR] /home/jenkins/jenkins-agent/workspace/james_ApacheJames_PR-3216/mailbox/postgres/src/main/java/org/apache/james/mailbox/postgres/mail/PostgresBlobIdUpdater.java:[121,5] method does not override or implement a method from a supertype
08:24:41,161 [ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execut
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
Re: [PR] JAMES-4231 PostgreSQL: Implement PostgresBlobIdUpdater for S3 compaction support [james-project]
Posted by "HesandaLiyanage (via GitHub)" <gi...@apache.org>.
HesandaLiyanage commented on PR #3216:
URL: https://github.com/apache/james-project/pull/3216#issuecomment-5889563237
> Hello, some ocmpilation issue reported by the CI:
>
> 08:24:41,160 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.14.0:compile (default-compile) on project apache-james-mailbox-postgres: Compilation failure: Compilation failure: 08:24:41,161 [ERROR] /home/jenkins/jenkins-agent/workspace/james_ApacheJames_PR-3216/mailbox/postgres/src/main/java/org/apache/james/mailbox/postgres/mail/PostgresBlobIdUpdater.java:[38,33] cannot find symbol 08:24:41,161 [ERROR] symbol: class BlobIdUpdater 08:24:41,161 [ERROR] location: package org.apache.james.blob.api 08:24:41,161 [ERROR] /home/jenkins/jenkins-agent/workspace/james_ApacheJames_PR-3216/mailbox/postgres/src/main/java/org/apache/james/mailbox/postgres/mail/PostgresBlobIdUpdater.java:[47,47] cannot find symbol 08:24:41,161 [ERROR] symbol: class BlobIdUpdater 08:24:41,161 [ERROR] /home/jenkins/jenkins-agent/workspace/james_ApacheJames_PR-3216/mailbox/postgres/src/main/java/org/apache/james/mailbox/postgres/mail/PostgresBlobIdUpdater.java:[48,57] package BlobId
Updater does not exist 08:24:41,161 [ERROR] /home/jenkins/jenkins-agent/workspace/james_ApacheJames_PR-3216/mailbox/postgres/src/main/java/org/apache/james/mailbox/postgres/mail/PostgresBlobIdUpdater.java:[61,21] cannot find symbol 08:24:41,161 [ERROR] symbol: class BlobIdUpdater 08:24:41,161 [ERROR] location: class org.apache.james.mailbox.postgres.mail.PostgresBlobIdUpdater.Factory 08:24:41,161 [ERROR] /home/jenkins/jenkins-agent/workspace/james_ApacheJames_PR-3216/mailbox/postgres/src/main/java/org/apache/james/mailbox/postgres/mail/PostgresBlobIdUpdater.java:[60,9] method does not override or implement a method from a supertype 08:24:41,161 [ERROR] /home/jenkins/jenkins-agent/workspace/james_ApacheJames_PR-3216/mailbox/postgres/src/main/java/org/apache/james/mailbox/postgres/mail/PostgresBlobIdUpdater.java:[121,5] method does not override or implement a method from a supertype 08:24:41,161 [ERROR] -> [Help 1] org.apache.maven.lifecycle.LifecycleExecutionException: Failed to exec
ut
Yes there is a dependency issue
Dependency: This PR is branched off master and depends on https://github.com/apache/james-project/pull/3193 for the BlobIdUpdater SPI (server/blob/blob-api). CI compile checks will succeed once https://github.com/apache/james-project/pull/3193 is merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org