You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Jürgen Link <ju...@googlemail.com> on 2011/12/07 17:04:03 UTC

Session replication fails with "Member send is failing"

Hi all,
I'm currently setting up a two-node cluster on a single machine (for
testing only).
I tried both on my Windows 7 box and on a Debian GNU/Linux 6.0.3 (squeeze)
server.
Tomcat versions are 7.0.23 and 6.0.35
Java versions are 1.6.0_18 (Linux) and 1.6.0_29 (Windows)

My setup is a simple cluster, serving two VAADIN based web applications.
The aim is to achieve fail-over security, i.e. a session started on one
node should be available on the other node as well.

This is how my server.xml looks like:

<?xml version='1.0' encoding='utf-8'?>
<Server port="7005" shutdown="SHUTDOWN">
  <Listener className="org.apache.catalina.core.AprLifecycleListener"
SSLEngine="on" />
  <Listener className="org.apache.catalina.core.JasperListener" />
  <Listener
className="org.apache.catalina.core.JreMemoryLeakPreventionListener" />
  <Listener className="org.apache.catalina.mbeans.ServerLifecycleListener"
/>
  <Listener
className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />

  <GlobalNamingResources>
    <Resource name="UserDatabase" auth="Container"
              type="org.apache.catalina.UserDatabase"
              description="User database that can be updated and saved"
              factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
              pathname="conf/tomcat-users.xml" />
  </GlobalNamingResources>

  <Service name="Catalina">
    <Connector port="7080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               redirectPort="8443" />
    <Connector port="7009" protocol="AJP/1.3" redirectPort="8443" />
    <Engine name="Catalina" defaultHost="localhost">
       <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
                 channelSendOptions="6">
          <Manager className="org.apache.catalina.ha.session.BackupManager"
                   expireSessionsOnShutdown="false"
                   notifyListenersOnReplication="true"
                   mapSendOptions="6"/>
          <Channel
className="org.apache.catalina.tribes.group.GroupChannel">
            <Membership
className="org.apache.catalina.tribes.membership.McastService"
                        address="228.0.0.4"
                        port="45564"
                        frequency="500"
                        dropTime="3000"/>
            <Receiver
className="org.apache.catalina.tribes.transport.nio.NioReceiver"
                      address="auto"
                      port="5000"
                      selectorTimeout="100"
                      maxThreads="6"/>

            <Sender
className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
              <Transport
className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
            </Sender>
            <Interceptor
className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
            <Interceptor
className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>
            <Interceptor
className="org.apache.catalina.tribes.group.interceptors.ThroughputInterceptor"/>
          </Channel>

          <Valve
className="org.apache.catalina.ha.tcp.ReplicationValve"

filter=".*\.gif|.*\.js|.*\.jpeg|.*\.jpg|.*\.png|.*\.htm|.*\.html|.*\.css|.*\.txt"/>

          <ClusterListener
className="org.apache.catalina.ha.session.ClusterSessionListener"/>
        </Cluster>

      <Realm className="org.apache.catalina.realm.UserDatabaseRealm"
             resourceName="UserDatabase"/>
      <Host name="localhost"  appBase="webapps"
            unpackWARs="true" autoDeploy="true"
            xmlValidation="false" xmlNamespaceAware="false">
      </Host>
    </Engine>
  </Service>
</Server>


I tried the following scenario:

1. start both cluster nodes, cluster membership is established successfully
2. submit request to web app on node1 (cookie JSESSIONID is established)
3. send request to web app on node2 (as both node run on the same machine,
the previous JSESSIONID is re-used)

I would expect that the second request would find the session data from the
original node1 (by session replication).
I can conform the desired behavior with a simple jsp which sets one session
property.
Anyhow, with the VAADIN web app, the following exception occurs:

07.12.2011 17:01:52
org.apache.catalina.tribes.transport.nio.ParallelNioSender doLoop
WARNUNG: Member send is failing for:tcp://{10, 1, 3, 181}:5000 ; Setting to
suspect and retrying.
07.12.2011 17:01:52 org.apache.catalina.tribes.tipis.LazyReplicatedMap
publishEntryInfo
SCHWERWIEGEND: Unable to replicate backup
key:C47C46A42F1D70A60AEC477F965F0C59 to
backup:org.apache.catalina.tribes.membership.MemberImpl[tcp://{10, 1, 3,
181}:
5000,{10, 1, 3, 181},5000, alive=5033,id={-42 -102 58 -89 94 -123 69 80 -83
85 90 -12 6 -108 -34 8 }, payload={}, command={}, domain={}, ]. Reason:Send
failed, attempt:2 max:1; Faulty members:tcp://{10, 1, 3,
181}:5000;org.apache.catalina.tribes.ChannelException: Send failed,
attempt:2 max:1; Faulty members:tcp://{10, 1, 3, 181}:5000;
        at
org.apache.catalina.tribes.transport.nio.ParallelNioSender.doLoop(ParallelNioSender.java:172)
        at
org.apache.catalina.tribes.transport.nio.ParallelNioSender.sendMessage(ParallelNioSender.java:78)
        at
org.apache.catalina.tribes.transport.nio.PooledParallelSender.sendMessage(PooledParallelSender.java:53)
        at
org.apache.catalina.tribes.transport.ReplicationTransmitter.sendMessage(ReplicationTransmitter.java:80)
        at
org.apache.catalina.tribes.group.ChannelCoordinator.sendMessage(ChannelCoordinator.java:78)
        at
org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessage(ChannelInterceptorBase.java:75)
        at
org.apache.catalina.tribes.group.interceptors.ThroughputInterceptor.sendMessage(ThroughputInterceptor.java:61)
        at
org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessage(ChannelInterceptorBase.java:75)
        at
org.apache.catalina.tribes.group.interceptors.MessageDispatchInterceptor.sendMessage(MessageDispatchInterceptor.java:73)
        at
org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessage(ChannelInterceptorBase.java:75)
        at
org.apache.catalina.tribes.group.interceptors.TcpFailureDetector.sendMessage(TcpFailureDetector.java:87)
        at
org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessage(ChannelInterceptorBase.java:75)
        at
org.apache.catalina.tribes.group.GroupChannel.send(GroupChannel.java:216)
        at
org.apache.catalina.tribes.group.GroupChannel.send(GroupChannel.java:175)
        at
org.apache.catalina.tribes.tipis.LazyReplicatedMap.publishEntryInfo(LazyReplicatedMap.java:158)
        at
org.apache.catalina.tribes.tipis.AbstractReplicatedMap.get(AbstractReplicatedMap.java:882)
        at
org.apache.catalina.session.ManagerBase.findSession(ManagerBase.java:921)
        at
org.apache.catalina.connector.Request.isRequestedSessionIdValid(Request.java:2221)
        at
org.apache.catalina.connector.CoyoteAdapter.parseSessionCookiesId(CoyoteAdapter.java:761)
        at
org.apache.catalina.connector.CoyoteAdapter.postParseRequest(CoyoteAdapter.java:574)
        at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:291)
        at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
        at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:602)
        at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
        at java.lang.Thread.run(Thread.java:662)
Caused by: org.apache.catalina.tribes.RemoteProcessException: Received a
failedack:org.apache.catalina.tribes.transport.Constants.FAIL_ACK_DATA
        at
org.apache.catalina.tribes.transport.nio.NioSender.read(NioSender.java:167)
        at
org.apache.catalina.tribes.transport.nio.NioSender.process(NioSender.java:117)
        at
org.apache.catalina.tribes.transport.nio.ParallelNioSender.doLoop(ParallelNioSender.java:130)
        ... 24 more

Anyone out there who has an idea?

Cheers
Jürgen

Re: Session replication fails with "Member send is failing"

Posted by Filip Hanik - Dev Lists <de...@hanik.com>.
On 12/7/2011 9:04 AM, Jürgen Link wrote:
> Caused by: org.apache.catalina.tribes.RemoteProcessException: Received a
> failedack:org.apache.catalina.tribes.transport.Constants.FAIL_ACK_DATA

sure thing, this is actually telling the "sender" that an exception happened on the "receiver" end.
Take a look at the logs for tcp://{10, 1, 3, 181}:5000 and it should contain the information in there

Filip

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org