You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@wicket.apache.org by DmitryM <ns...@aol.com> on 2010/06/03 20:06:27 UTC

SessionStore life cycle in cluster?

Hello, guys

I managed to have Tomcat cluster working with memcached-session-manager
(http://groups.google.com/group/memcached-session-manager) from Martin.

Everything works perfectly fine unless tomcats start getting shut down.
I have 2 tomcats running with the session replicated (see above). Then I
shut down one of them.
When on a page with ajax behavior after I click an ajax button the response
to the request gets delayed (about 2-3 seconds extra compared to the case
when all tomcats are live and running).

I tried this with Wicket's SecondLevelCacheSessionStore(+ DiskPageStore, I
assume) and I also tried to use a Hazelcast-based PageStore (ex. from here:
http://wicketbyexample.com/apache-wicket-clustering-with-multiple-options/).
That bizarre delay happens in both cases.

Can anyone please try to come up with some explanation what may be happening
in that case?
I guess, the scenario is as follows:
- tomcat cluster with proper session distribution in place,
- request hits tomcat1 and gets page with ajax button
- tomcat1 is shutdown
- ajax button is clicked
- request hits tomcat2 and gets processed successfully (but with delay)

Thanks in advance,
Dmitry
--
View this message in context: http://apache-wicket.1842946.n4.nabble.com/SessionStore-life-cycle-in-cluster-tp2242105p2242105.html
Sent from the Wicket - User mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org

Re: SessionStore life cycle in cluster?

Posted by Igor Vaynberg <ig...@gmail.com>.

On Fri, Jun 4, 2010 at 10:32 PM, DmitryM <ns...@aol.com> wrote:
>
> Igor,
>
> Can you please comment on the following couple of points I got?
>
> 1. "Pulling session from memcached node": I double checked the
> recommendation and it looks like fetching session from the memcached node
> doesn't take much time (around 200ms at most). But there was a
> recommendation about immediate session attributes deserialization after
> session replication
> (http://apache-wicket.1842946.n4.nabble.com/A-few-clustering-questions-td1863992.html#a1863993).
> Do you think it may be a potential issue causing the request processing
> delay I experienced?

i would use a profiler and see where the time goes, that way you dont
have to guess.

> 2. The memcached-session-manager's developer suggested I could try to use
> plain HttpSessionStore (instead of default SecondLevelCacheSessionStore).
> But when I tried his suggestion I got the following stacktrace:
>
> java.lang.ExceptionInInitializerError
...
> which is caused by the Injector usage in one of the classes used by a Page.
> Why do you think Application.get() method may blow up? Why isn't the
> ThreadLocal application instance get set?

unless memcached serializer works asynchronously in its own thread.

-igor

>
> Thanks,
> Dmitry
> --
> View this message in context: http://apache-wicket.1842946.n4.nabble.com/SessionStore-life-cycle-in-cluster-tp2242105p2244022.html
> Sent from the Wicket - User mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org

Re: SessionStore life cycle in cluster?

Posted by DmitryM <ns...@aol.com>.

Igor,

Can you please comment on the following couple of points I got?

1. "Pulling session from memcached node": I double checked the
recommendation and it looks like fetching session from the memcached node
doesn't take much time (around 200ms at most). But there was a
recommendation about immediate session attributes deserialization after
session replication
(http://apache-wicket.1842946.n4.nabble.com/A-few-clustering-questions-td1863992.html#a1863993).
Do you think it may be a potential issue causing the request processing
delay I experienced?

2. The memcached-session-manager's developer suggested I could try to use
plain HttpSessionStore (instead of default SecondLevelCacheSessionStore).
But when I tried his suggestion I got the following stacktrace:

java.lang.ExceptionInInitializerError
	at sun.misc.Unsafe.ensureClassInitialized(Native Method)
	at
sun.reflect.UnsafeFieldAccessorFactory.newFieldAccessor(UnsafeFieldAccessorFactory.java:25)
	at
sun.reflect.ReflectionFactory.newFieldAccessor(ReflectionFactory.java:122)
	at java.lang.reflect.Field.acquireFieldAccessor(Field.java:918)
	at java.lang.reflect.Field.getFieldAccessor(Field.java:899)
	at java.lang.reflect.Field.getLong(Field.java:528)
	at java.io.ObjectStreamClass.getDeclaredSUID(ObjectStreamClass.java:1614)
	at java.io.ObjectStreamClass.access$700(ObjectStreamClass.java:52)
	at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:425)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.io.ObjectStreamClass.<init>(ObjectStreamClass.java:413)
	at java.io.ObjectStreamClass.lookup(ObjectStreamClass.java:310)
	at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:547)
	at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1583)
	at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1496)
	at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1732)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1947)
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1871)
	at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1753)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1947)
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1871)
	at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1753)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1947)
	at java.io.ObjectInputStream.defaultReadObject(ObjectInputStream.java:480)
	at org.apache.wicket.Component.readObject(Component.java:4457)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:974)
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1849)
	at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1753)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
	at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1667)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1323)
	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1947)
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1871)
	at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1753)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
	at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1667)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1323)
	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1947)
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1871)
	at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1753)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
	at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1667)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1323)
	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1947)
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1871)
	at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1753)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
	at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1667)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1323)
	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1947)
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1871)
	at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1753)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1329)
	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:351)
	at
de.javakaffee.web.msm.JavaSerializationTranscoder.deserializeAttributes(JavaSerializationTranscoder.java:177)
	at
de.javakaffee.web.msm.TranscoderService.deserializeAttributes(TranscoderService.java:156)
	at
de.javakaffee.web.msm.TranscoderService.deserialize(TranscoderService.java:113)
	at
de.javakaffee.web.msm.MemcachedBackupSessionManager.loadFromMemcached(MemcachedBackupSessionManager.java:736)
	at
de.javakaffee.web.msm.MemcachedBackupSessionManager.findSession(MemcachedBackupSessionManager.java:481)
	at org.apache.catalina.connector.Request.doGetSession(Request.java:2363)
	at org.apache.catalina.connector.Request.getSession(Request.java:2098)
	at
org.apache.catalina.connector.RequestFacade.getSession(RequestFacade.java:833)
	at
org.springframework.security.web.context.HttpSessionSecurityContextRepository.loadContext(HttpSessionSecurityContextRepository.java:82)
	at
org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:75)
	at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:356)
	at
org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:150)
	at
org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237)
	at
org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)
	at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
	at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
	at
com.mobioid.core.utils.UserLocaleFilter.doFilter(UserLocaleFilter.java:34)
	at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
	at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
	at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
	at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
	at
de.javakaffee.web.msm.SessionTrackerValve.invoke(SessionTrackerValve.java:90)
	at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
	at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
	at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
	at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
	at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:852)
	at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
	at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
	at java.lang.Thread.run(Thread.java:637)
Caused by: org.apache.wicket.WicketRuntimeException: There is no application
attached to current thread http-8082-7
	at org.apache.wicket.Application.get(Application.java:179)
	at
org.apache.wicket.injection.web.InjectorHolder.getInjector(InjectorHolder.java:67)
	at
com.mobioid.modules.payment.ui.manage.organization.OrganizationController.<init>(OrganizationController.java:56)
	at
com.mobioid.modules.payment.ui.manage.organization.OrganizationController.<clinit>(OrganizationController.java:59)
	... 91 more

which is caused by the Injector usage in one of the classes used by a Page.
Why do you think Application.get() method may blow up? Why isn't the
ThreadLocal application instance get set?

Thanks,
Dmitry
-- 
View this message in context: http://apache-wicket.1842946.n4.nabble.com/SessionStore-life-cycle-in-cluster-tp2242105p2244022.html
Sent from the Wicket - User mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org

Re: SessionStore life cycle in cluster?

Posted by Igor Vaynberg <ig...@gmail.com>.

i would guess that is the time it takes to pull the session from
memcached to the tomcat node.

-igor

On Thu, Jun 3, 2010 at 12:12 PM, DmitryM <ns...@aol.com> wrote:
>
> Igor,
>
> I was wrong.
> When it's a first request hitting a page (after shutting down one of 2
> tomcats) then regardless of the type of request (ajax or bookmarkable page
> link) there is a delay of 2+ seconds.
>
> -Dmitry
> --
> View this message in context: http://apache-wicket.1842946.n4.nabble.com/SessionStore-life-cycle-in-cluster-tp2242105p2242198.html
> Sent from the Wicket - User mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org

Re: SessionStore life cycle in cluster?

Posted by DmitryM <ns...@aol.com>.

Igor,

I was wrong.
When it's a first request hitting a page (after shutting down one of 2
tomcats) then regardless of the type of request (ajax or bookmarkable page
link) there is a delay of 2+ seconds.

-Dmitry
-- 
View this message in context: http://apache-wicket.1842946.n4.nabble.com/SessionStore-life-cycle-in-cluster-tp2242105p2242198.html
Sent from the Wicket - User mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org

Re: SessionStore life cycle in cluster?

Posted by Igor Vaynberg <ig...@gmail.com>.

are you sure its ajax only? that seems rather strange to me. if its
true then i dont have a clue, you will have to use a profiler to see
where the time goes.

-igor

On Thu, Jun 3, 2010 at 11:26 AM, DmitryM <ns...@aol.com> wrote:
>
> Only the very first one.
>
> I'm not 100% sure but the session seems to be always retrieved from
> memcached...
>
> That's what I don't quite understand. When all nodes/tomcats in the cluster
> are up then request are fast.
> And the thing is, if that's not an Ajax action, then the request is
> responded at the same speed (like switching between pages in the app).
>
> -Dmitry
> --
> View this message in context: http://apache-wicket.1842946.n4.nabble.com/SessionStore-life-cycle-in-cluster-tp2242105p2242134.html
> Sent from the Wicket - User mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org

Re: SessionStore life cycle in cluster?

Posted by DmitryM <ns...@aol.com>.

Only the very first one.

I'm not 100% sure but the session seems to be always retrieved from
memcached...

That's what I don't quite understand. When all nodes/tomcats in the cluster
are up then request are fast.
And the thing is, if that's not an Ajax action, then the request is
responded at the same speed (like switching between pages in the app).

-Dmitry
-- 
View this message in context: http://apache-wicket.1842946.n4.nabble.com/SessionStore-life-cycle-in-cluster-tp2242105p2242134.html
Sent from the Wicket - User mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org

Re: SessionStore life cycle in cluster?

Posted by Igor Vaynberg <ig...@gmail.com>.

maybe because tomcat2 needs to retrieve the session from memcached,
are subsequent requests also slow? or just the first request after the
failover?

-igor

On Thu, Jun 3, 2010 at 11:06 AM, DmitryM <ns...@aol.com> wrote:
>
> Hello, guys
>
> I managed to have Tomcat cluster working with memcached-session-manager
> (http://groups.google.com/group/memcached-session-manager) from Martin.
>
> Everything works perfectly fine unless tomcats start getting shut down.
> I have 2 tomcats running with the session replicated (see above). Then I
> shut down one of them.
> When on a page with ajax behavior after I click an ajax button the response
> to the request gets delayed (about 2-3 seconds extra compared to the case
> when all tomcats are live and running).
>
> I tried this with Wicket's SecondLevelCacheSessionStore(+ DiskPageStore, I
> assume) and I also tried to use a Hazelcast-based PageStore (ex. from here:
> http://wicketbyexample.com/apache-wicket-clustering-with-multiple-options/).
> That bizarre delay happens in both cases.
>
> Can anyone please try to come up with some explanation what may be happening
> in that case?
> I guess, the scenario is as follows:
> - tomcat cluster with proper session distribution in place,
> - request hits tomcat1 and gets page with ajax button
> - tomcat1 is shutdown
> - ajax button is clicked
> - request hits tomcat2 and gets processed successfully (but with delay)
>
> Thanks in advance,
> Dmitry
> --
> View this message in context: http://apache-wicket.1842946.n4.nabble.com/SessionStore-life-cycle-in-cluster-tp2242105p2242105.html
> Sent from the Wicket - User mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org