[jira] [Closed] (JAMES-3150) Implement Garbage Colletion for blobs

["Benoit Tellier (Jira)" <se...@james.apache.org>]

     [ https://issues.apache.org/jira/browse/JAMES-3150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Benoit Tellier closed JAMES-3150.
---------------------------------
    Fix Version/s: 3.7.0
       Resolution: Fixed

The BloomFilter algorithm prooved to be fast enough, and we reached 10.000.000 mails / hour.

Given run in the weekends, we could imagine scaling up to 600.000.000 emails.

A first optimisation could be to track recently deleted blobs and only feed them into the GC algorithm. This would allow to reduce the run time to "building the bloom filter from the references" - 40.000.000 mails per hour today, which could be further paralellized (process blob source parallely rather than sequuentially).

I'm closing this as we support deduplicated blob garbage collection for mid-sized deployments.

> Implement Garbage Colletion for blobs
> -------------------------------------
>
>                 Key: JAMES-3150
>                 URL: https://issues.apache.org/jira/browse/JAMES-3150
>             Project: James Server
>          Issue Type: Improvement
>          Components: Blob
>    Affects Versions: 3.3.0
>            Reporter: Gautier DI FOLCO
>            Priority: Major
>             Fix For: 3.7.0
>
>          Time Spent: 8h 40m
>  Remaining Estimate: 0h
>
> With the blob store deduplication, dropping a blob in a distributed environment is impossible if we want to keep an acceptable concurrency level.
> A Garbage Collector should be created in order to drop old blobs.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org