SimpleSession to JSON - and back?

[John Moore <jo...@jmsd.co.uk>]

I am trying to implement Cache and CacheManager to utilize Redis as a 
session store. Redis is a key/value store in which the keys and values 
are normally strings. I'm actually using Grails, and the inbuilt Grails 
JSON converters make it very straightforward to create a JSON string 
from a SimpleSession object (which is what I need to 'put' with the 
Cache implementation). My problem is implementing 'get'. How do I 
recreate a SimpleSession object from a JSON string? I imagine I could 
write some code to loop through the accessor methods and create one that 
way, but I was hoping there was some other much simpler way which I've 
overlooked.

Re: SimpleSession to JSON - and back?

[Les Hazlewood <lh...@apache.org>]

On Mon, Jan 30, 2012 at 10:40 AM, John Moore <jo...@jmsd.co.uk> wrote:
> On 30/01/12 18:22, Les Hazlewood wrote:
>>
>> There could be, depending on how long your TTL is.  However, this is
>> easily enough solved IMO using a ThreadLocal implementation that
>> simulates a local cache:
>>
>> Look up the object in the ThreadLocal - if its ID matches the DAO ID
>> argument, return it.  If it doesn't match, clear out the ThreadLocal,
>> look up the session object from the underlying store and then store it
>> in the ThreadLocal.  Also overwrite the ThreadLocal value during a
>> write operation.
>>
>> This logic probably will go into Shiro's implementation for 1.3, but
>> that kind of gives you an idea if you need this right away.  (Of
>> course if you do work on this and you are capable, any patches would
>> be appreciated!)
>>
> What I've done for the meantime, to check things out, is to use EHCache for
> my local cache, with a TTL of 1 second. In my 'get' method I first check to
> see if the session object exists in the cache and if so, use it, otherwise I
> get it from Redis and store it in the local cache. In the 'put' I put the
> object in Redis and the local cache as well. Does this sound right? It looks
> like it should reduce the network calls by 90%.

Sounds good - should work well.  I'd also ensure that the EHCache
config reflects local/in-memory storage only - overflow to disk would
probably slow things down unnecessarily.

I'm not sure what you mean
> by "if its ID matches the DAO ID argument, return it" above.

That comment was targeted specifically at a thread-local solution.
The reason is that if the ThreadLocal isn't cleared properly (for
whatever reason), a ThreadLocally-cached SimpleSession instance may
span across requests.  Therefore you'd only want to use the
ThreadLocal value IFF its ID matched the session ID used as a method
argument to SessionDAO.getSession.  If they don't match, the
ThreadLocal value would be considered stale and unusable.

HTH,

-- 
Les Hazlewood
CTO, Katasoft | http://www.katasoft.com | 888.391.5282
twitter: @lhazlewood | http://twitter.com/lhazlewood
katasoft blog: http://www.katasoft.com/blogs/lhazlewood
personal blog: http://leshazlewood.com

Re: SimpleSession to JSON - and back?

[John Moore <jo...@jmsd.co.uk>]

On 30/01/12 18:22, Les Hazlewood wrote:
> There could be, depending on how long your TTL is.  However, this is
> easily enough solved IMO using a ThreadLocal implementation that
> simulates a local cache:
>
> Look up the object in the ThreadLocal - if its ID matches the DAO ID
> argument, return it.  If it doesn't match, clear out the ThreadLocal,
> look up the session object from the underlying store and then store it
> in the ThreadLocal.  Also overwrite the ThreadLocal value during a
> write operation.
>
> This logic probably will go into Shiro's implementation for 1.3, but
> that kind of gives you an idea if you need this right away.  (Of
> course if you do work on this and you are capable, any patches would
> be appreciated!)
>
What I've done for the meantime, to check things out, is to use EHCache 
for my local cache, with a TTL of 1 second. In my 'get' method I first 
check to see if the session object exists in the cache and if so, use 
it, otherwise I get it from Redis and store it in the local cache. In 
the 'put' I put the object in Redis and the local cache as well. Does 
this sound right? It looks like it should reduce the network calls by 
90%. I'm not sure what you mean by "if its ID matches the DAO ID 
argument, return it" above.

Re: SimpleSession to JSON - and back?

[Les Hazlewood <lh...@apache.org>]

On Mon, Jan 30, 2012 at 9:40 AM, John Moore <jo...@jmsd.co.uk> wrote:
> Les, thanks for your very timely response. Since posting the earlier JSON
> message I figured I'd just try doing straight object serialization to a
> ByteArrayOutputStream and Base64 encoding the byte array, forgetting about
> JSON, and I'd just got that working nicely when I read your response
> (although I used standard Groovy stuff and not the Shiro utilities, which I
> was unaware of). I may well go down that path as in reality I have no real
> requirement for the human-readability of the stored session objects which
> JSON would provide.
>
> You're right about Shiro hitting the session cache frequently - I traced a
> request and was surprised how often the 'get' method of Cache is called.
> Redis is blindingly fast but you're right, the network round trips would be
> best avoided. Isn't there some staleness risk in using a local in-memory
> cache, though?

There could be, depending on how long your TTL is.  However, this is
easily enough solved IMO using a ThreadLocal implementation that
simulates a local cache:

Look up the object in the ThreadLocal - if its ID matches the DAO ID
argument, return it.  If it doesn't match, clear out the ThreadLocal,
look up the session object from the underlying store and then store it
in the ThreadLocal.  Also overwrite the ThreadLocal value during a
write operation.

This logic probably will go into Shiro's implementation for 1.3, but
that kind of gives you an idea if you need this right away.  (Of
course if you do work on this and you are capable, any patches would
be appreciated!)

Cheers,

Les

-- 
Les Hazlewood
CTO, Katasoft | http://www.katasoft.com | 888.391.5282
twitter: @lhazlewood | http://twitter.com/lhazlewood
katasoft blog: http://www.katasoft.com/blogs/lhazlewood
personal blog: http://leshazlewood.com

Re: SimpleSession to JSON - and back?

[John Moore <jo...@jmsd.co.uk>]

Les, thanks for your very timely response. Since posting the earlier 
JSON message I figured I'd just try doing straight object serialization 
to a ByteArrayOutputStream and Base64 encoding the byte array, 
forgetting about JSON, and I'd just got that working nicely when I read 
your response (although I used standard Groovy stuff and not the Shiro 
utilities, which I was unaware of). I may well go down that path as in 
reality I have no real requirement for the human-readability of the 
stored session objects which JSON would provide.

You're right about Shiro hitting the session cache frequently - I traced 
a request and was surprised how often the 'get' method of Cache is 
called. Redis is blindingly fast but you're right, the network round 
trips would be best avoided. Isn't there some staleness risk in using a 
local in-memory cache, though?

Re: SimpleSession to JSON - and back?

[Les Hazlewood <lh...@apache.org>]

Hi John,

With the exception of the Session's 'attributes' Map, all of the
internal properties of SimpleSession are all 'single value' properties
- Date, long, String, etc.  These can be serialized and deserialized
with no interaction on your part with most JSON serializer mechanisms
(Jackson, etc).

The attributes Map might be another story.  Of course JSON serializers
can handle maps easily, but there is no telling what is in that map
since its contents are at the mercy of the application developer's
preferences.  When SimpleSession is serialized via JVM serialization,
it works fine assuming that the Map contents are all Serializable
(this is a contract between Shiro and application developers - it is
expected that anything you put in the session can be serialized).

That means your JSON serialization mechanism should know how to
serialize the attributes Map as well, e.g.:

{ ...
  "attributes": {
      "someAttribute": {
          "someProp": "someValue",
          "anotherProp": "anotherValue"
      },
      "anotherAttribute": "hello world",
      ...
  }

If this is not possible (or not very feasible), the easiest thing to
do would be to serialize the attributes map via JVM serialization and
store value as a Base64-encoded string:

Serializer<Map> serializer = new org.apache.shiro.io.DefaultSerializer();
byte[] serialized = serializer.serialize(simpleSession.getAttributes());
String base64Attributes =
org.apache.shiro.codec.Base64.encodeToString(serialized);

{ ...
  "base64Attributes": base64Attributes
}

you'll of course reverse the process when converting from JSON to SimpleSession:

String base64Attributes = jsonObject['base64Attributes']
byte[] serialized = Base64.decode(base64Attributes);
Map attributes = serializer.deserialize(serialized);
simpleSession.setAttributes(attributes);

then return the simpleSession instance back to Shiro.

You'll probably want a local/in-memory cache to proxy Redis because
Shiro can hit the session cache dozens of times in a single request -
you wouldn't want dozens of network round-trips to Redis during a
request.  (Shiro 1.3 will probably reduce this need by storing a
thread-local copy of the Session, but for now that does not occur
because it expects the Cache to work efficiently).

HTH,

-- 
Les Hazlewood
CTO, Katasoft | http://www.katasoft.com | 888.391.5282
twitter: @lhazlewood | http://twitter.com/lhazlewood
katasoft blog: http://www.katasoft.com/blogs/lhazlewood
personal blog: http://leshazlewood.com

On Mon, Jan 30, 2012 at 6:48 AM, John Moore <jo...@jmsd.co.uk> wrote:
> I am trying to implement Cache and CacheManager to utilize Redis as a
> session store. Redis is a key/value store in which the keys and values are
> normally strings. I'm actually using Grails, and the inbuilt Grails JSON
> converters make it very straightforward to create a JSON string from a
> SimpleSession object (which is what I need to 'put' with the Cache
> implementation). My problem is implementing 'get'. How do I recreate a
> SimpleSession object from a JSON string? I imagine I could write some code
> to loop through the accessor methods and create one that way, but I was
> hoping there was some other much simpler way which I've overlooked.