JCS plugin that serializes the keys?..

[Vladimir Olenin <VO...@cihi.ca>]

Hi.
 
I wonder if there is a plugin available for JCS OR if existing Disk
Plugin can be tuned so, that keys are serialized as well, and the only
thing stored in memory would be a hashcode of the original key.
According to Disk Plugin doc:
 
   >>   It maintains the cached data on disk and the keys in memory for
the fastest possible lookup times.
 
Currently I need to quick-fix quite badly designed application which
uses insanely long _unique_ String values as keys in the cached lookup
tables. Probably in 50% of the cases the key takes up more memory than
the value itself. It's required to reduce memory consumption triplefold
(ie, from current 500M to at least 150M). Code refactoring is not an
option, since it's too late in the lifecycle.
 
I'm exploring the options of 'transparent' serialization/deserialization
of these cached lookup tables to/from hard drive. Performance can be
sacrificed within certain parameters, so want to keep my options open,
hence the question.
 
Providing the default String.hashCode() value should be quite unique and
very few collisions should happen (btw, does anyone has some stats on
this one?), it should be possible to have instead of:
 
key_in_memory -> value_on_disk
 
structure, the structure below:
 
key_in_memory.hashCode --> map_entries_on_disk{key, value}
 
which would load one or many entries from disk, after which possible
collisions can be resolved programmatically in code through 'equals'
call.
 
Does anyone have any insight into this kind of problem? Has this already
been done? Any approximations on performance degradation if key is
serialized together with value (vs 'just value' serialization)?
 
Any feedback is greatly appreciated!
 
Vlad

Re: JCS plugin that serializes the keys?..

[Aaron Smuts <as...@yahoo.com>]

You are proposing that we use the separate chaining
hash collision technique.  This is a bit hard to do on
disk, since you can't add another element to the list
(the file) without moving everything else on disk
forward.

Perhaps you should try to create a key encoder, if
there are some standard key parts you could encode. 
You'd use the encoded key when doing cache lookups.

Aaron

--- Vladimir Olenin <VO...@cihi.ca> wrote:

> Hi.
>  
> I wonder if there is a plugin available for JCS OR
> if existing Disk
> Plugin can be tuned so, that keys are serialized as
> well, and the only
> thing stored in memory would be a hashcode of the
> original key.
> According to Disk Plugin doc:
>  
>    >>   It maintains the cached data on disk and the
> keys in memory for
> the fastest possible lookup times.
>  
> Currently I need to quick-fix quite badly designed
> application which
> uses insanely long _unique_ String values as keys in
> the cached lookup
> tables. Probably in 50% of the cases the key takes
> up more memory than
> the value itself. It's required to reduce memory
> consumption triplefold
> (ie, from current 500M to at least 150M). Code
> refactoring is not an
> option, since it's too late in the lifecycle.
>  
> I'm exploring the options of 'transparent'
> serialization/deserialization
> of these cached lookup tables to/from hard drive.
> Performance can be
> sacrificed within certain parameters, so want to
> keep my options open,
> hence the question.
>  
> Providing the default String.hashCode() value should
> be quite unique and
> very few collisions should happen (btw, does anyone
> has some stats on
> this one?), it should be possible to have instead
> of:
>  
> key_in_memory -> value_on_disk
>  
> structure, the structure below:
>  
> key_in_memory.hashCode --> map_entries_on_disk{key,
> value}
>  
> which would load one or many entries from disk,
> after which possible
> collisions can be resolved programmatically in code
> through 'equals'
> call.
>  
> Does anyone have any insight into this kind of
> problem? Has this already
> been done? Any approximations on performance
> degradation if key is
> serialized together with value (vs 'just value'
> serialization)?
>  
> Any feedback is greatly appreciated!
>  
> Vlad
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: jcs-users-unsubscribe@jakarta.apache.org
For additional commands, e-mail: jcs-users-help@jakarta.apache.org