[collections]

["Balazs Attila-Mihaly (Cd-MaN)" <x_...@yahoo.com>]

Hello everybody.

I would like to know if there would be any interest of including a "disk-backed map" into the commons-collections? The problem statement for it would be:

- while we are moving to 64bit, it will still take a couple of years until all hardware is 64 bit and more importantly, all software (or at least the OS + JVM running on it) are 64 bit. In the meantime you can easily run out of memory space, even though you physically would have enough memory and you would accept some swapping (rather than the process dying with OOM). I had such an issue with running PMD CPD on a larger codebase (2+ million LOC).

The solution:

- a Map which keeps its keys in memory but writes the values to disk (to a temporary file, marked deleteOnExit). It also keeps WeakRefs to the values. When a request comes in for a value, it tries to satisfy it using the weakref, and if it fails, it reads it back from the temporary file. This would be efficient for a mostly read-intensive application (it does not try to efficiently handle the situations when there are many removes/updates to the file).

Would you be interested in including this solution in the framework?

Best regards (and happy holidays),
Attila



      

Re: [collections]

[Stephen Colebourne <sc...@joda.org>]

I would see this as more like caching functionality than collections
functionality. And there are plenty of other caching libraries.
Stephen


On 13 December 2010 03:27, Balazs Attila-Mihaly (Cd-MaN)
<x_...@yahoo.com> wrote:
> Hello everybody.
>
> I would like to know if there would be any interest of including a "disk-backed map" into the commons-collections? The problem statement for it would be:
>
> - while we are moving to 64bit, it will still take a couple of years until all hardware is 64 bit and more importantly, all software (or at least the OS + JVM running on it) are 64 bit. In the meantime you can easily run out of memory space, even though you physically would have enough memory and you would accept some swapping (rather than the process dying with OOM). I had such an issue with running PMD CPD on a larger codebase (2+ million LOC).
>
> The solution:
>
> - a Map which keeps its keys in memory but writes the values to disk (to a temporary file, marked deleteOnExit). It also keeps WeakRefs to the values. When a request comes in for a value, it tries to satisfy it using the weakref, and if it fails, it reads it back from the temporary file. This would be efficient for a mostly read-intensive application (it does not try to efficiently handle the situations when there are many removes/updates to the file).
>
> Would you be interested in including this solution in the framework?
>
> Best regards (and happy holidays),
> Attila
>
>
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org