You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Manak Bisht <ma...@iiitd.ac.in> on 2023/06/10 17:11:43 UTC

Error "Unable to send message through cluster sender" in a cluster with static members using delta manager

I am trying to run the carts example (*/examples/jsp/sessions/carts.jsp*)
with two nodes, on ports 8090 and 8190. The nodes are working fine
individually. However, I am trying to create a cluster with static members
(without multicast), but I keep encountering the following error -

09-Jun-2023 22:17:21.193 SEVERE
[ContainerBackgroundProcessor[StandardEngine[Catalina]]]
org.apache.catalina.ha.tcp.SimpleTcpCluster.send Unable to send
message through cluster sender.
    org.apache.catalina.tribes.ChannelException: Send failed,
attempt:[1] max:[1]; Faulty members:tcp://localhost:8190;
        at org.apache.catalina.tribes.transport.nio.ParallelNioSender.doLoop(ParallelNioSender.java:217)
        at org.apache.catalina.tribes.transport.nio.ParallelNioSender.sendMessage(ParallelNioSender.java:78)
        at org.apache.catalina.tribes.transport.nio.PooledParallelSender.sendMessage(PooledParallelSender.java:51)
        at org.apache.catalina.tribes.transport.ReplicationTransmitter.sendMessage(ReplicationTransmitter.java:65)
        at org.apache.catalina.tribes.group.ChannelCoordinator.sendMessage(ChannelCoordinator.java:83)
        at org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessage(ChannelInterceptorBase.java:89)
        at org.apache.catalina.tribes.group.interceptors.MessageDispatchInterceptor.sendMessage(MessageDispatchInterceptor.java:93)
        at org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessage(ChannelInterceptorBase.java:89)
        at org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessage(ChannelInterceptorBase.java:89)
        at org.apache.catalina.tribes.group.interceptors.TcpFailureDetector.sendMessage(TcpFailureDetector.java:89)
        at org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessage(ChannelInterceptorBase.java:89)
        at org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessage(ChannelInterceptorBase.java:89)
        at org.apache.catalina.tribes.group.GroupChannel.send(GroupChannel.java:268)
        at org.apache.catalina.tribes.group.GroupChannel.send(GroupChannel.java:220)
        at org.apache.catalina.ha.tcp.SimpleTcpCluster.send(SimpleTcpCluster.java:710)
        at org.apache.catalina.ha.tcp.SimpleTcpCluster.send(SimpleTcpCluster.java:681)
        at org.apache.catalina.ha.session.DeltaManager.send(DeltaManager.java:495)
        at org.apache.catalina.ha.session.DeltaManager.sessionExpired(DeltaManager.java:1160)
        at org.apache.catalina.ha.session.DeltaSession.expire(DeltaSession.java:540)
        at org.apache.catalina.ha.session.DeltaSession.expire(DeltaSession.java:495)
        at org.apache.catalina.ha.session.DeltaSession.isValid(DeltaSession.java:457)
        at org.apache.catalina.session.ManagerBase.processExpires(ManagerBase.java:617)
        at org.apache.catalina.session.ManagerBase.backgroundProcess(ManagerBase.java:600)
        at org.apache.catalina.core.StandardContext.backgroundProcess(StandardContext.java:5315)
        at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1331)
        at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1335)
        at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1335)
        at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1303)
        at java.lang.Thread.run(Thread.java:750)
    Caused by: java.io.IOException: Unable to receive an ack message.
EOF on socket channel has been reached.
        at org.apache.catalina.tribes.transport.nio.NioSender.read(NioSender.java:178)
        at org.apache.catalina.tribes.transport.nio.NioSender.process(NioSender.java:122)
        at org.apache.catalina.tribes.transport.nio.ParallelNioSender.doLoop(ParallelNioSender.java:163)
        ... 28 more

I am trying to do this to achieve non-sticky sessions. Please help me
identify the problem in my *server.xml*. I am using *tomcat 8.5* with the
default configuration, with the cluster settings added inside the <Engine>
tag. The *web.xml* for both tomcats already contains the <distributable/>
tag.

          <Cluster
className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
channelSendOptions="6" channelStartOptions="3">

              <Manager className="org.apache.catalina.ha.session.DeltaManager"
                expireSessionsOnShutdown="false"
                notifyListenersOnReplication="true"/>

                <Channel
className="org.apache.catalina.tribes.group.GroupChannel">

                    <Receiver
className="org.apache.catalina.tribes.transport.nio.NioReceiver"/>

                        <Sender
className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
                            <Transport
className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
                        </Sender>
                        <Interceptor
className="org.apache.catalina.tribes.group.interceptors.TcpPingInterceptor"/>
                        <Interceptor
className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
                        <Interceptor
className="org.apache.catalina.tribes.group.interceptors.StaticMembershipInterceptor">

                           <LocalMember
className="org.apache.catalina.tribes.membership.StaticMember"
                            domain="delta-static"
                            uniqueId="{2,3,4,5,6,7,8,9,10,11,12,13,14,15,0,1}"/>

                            <Member
className="org.apache.catalina.tribes.membership.StaticMember"
                                port="8090"
                                securePort="-1"
                                host="localhost"
                                domain="delta-static"

uniqueId="{1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,0}"
                                />

                            </Interceptor>
                            <Interceptor
className="org.apache.catalina.tribes.group.interceptors.MessageDispatchInterceptor"/>

                        </Channel>
                        <Valve
className="org.apache.catalina.ha.tcp.ReplicationValve" filter=""/>
                        <Valve
className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>

                        <ClusterListener
className="org.apache.catalina.ha.session.ClusterSessionListener"/>
                    </Cluster>

Re: Error "Unable to send message through cluster sender" in a cluster with static members using delta manager

Posted by Mark Thomas <ma...@apache.org>.
On 12/06/2023 21:00, Manak Bisht wrote:
> On Tue, Jun 13, 2023 at 1:26 AM Mark Thomas <ma...@apache.org> wrote:
> 
>> Try https://people.apache.org/~markt/dev/cluster-test.war instead.
>>
>> Mark
>>
> 
> Sure, I will give that a try. But is there some issue with the
> *carts.jsp *example?
> Or is it not meant for this purpose?

The carts.jsp example wasn't written with clustering in mind. Therefore 
it needs an additional line to ensure that the updated shopping cart is 
replicated.

What that change is is left as an exercise for the reader.

Hint: Look at carts.jsp and then look at how DeltaRequest

https://github.com/apache/tomcat/blob/main/java/org/apache/catalina/ha/session/DeltaRequest.java

tracks changes to the session for replication.

(The link is for the Tomcat 11 code but iyt is essentially the same in 
all current Tomcat versions.)

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Error "Unable to send message through cluster sender" in a cluster with static members using delta manager

Posted by Manak Bisht <ma...@iiitd.ac.in>.
On Tue, Jun 13, 2023 at 1:26 AM Mark Thomas <ma...@apache.org> wrote:

> Try https://people.apache.org/~markt/dev/cluster-test.war instead.
>
> Mark
>

Sure, I will give that a try. But is there some issue with the
*carts.jsp *example?
Or is it not meant for this purpose?

Re: Error "Unable to send message through cluster sender" in a cluster with static members using delta manager

Posted by Mark Thomas <ma...@apache.org>.
Try https://people.apache.org/~markt/dev/cluster-test.war instead.

Mark


On 12/06/2023 20:13, Manak Bisht wrote:
> Thank you for the clarification. Sorry, it took me a while to understand
> your point.
> I have successfully changed the cluster settings,
> For node with http port 8090
> ```
>            <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
> channelSendOptions="6" channelStartOptions="3">
>                <Manager
> className="org.apache.catalina.ha.session.DeltaManager"
>                  expireSessionsOnShutdown="false"
>                  notifyListenersOnReplication="true"/>
> 
>                  <Channel
> className="org.apache.catalina.tribes.group.GroupChannel">
> 
>                      <Receiver
> className="org.apache.catalina.tribes.transport.nio.NioReceiver"
>                          address="localhost"
>                          port="8091"
>                          autoBind="0"
>                          selectorTimeout="5000"
>                          maxThreads="6"/>
> 
>                          <Sender
> className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
>                              <Transport
> className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
>                          </Sender>
>                          <Interceptor
> className="org.apache.catalina.tribes.group.interceptors.TcpPingInterceptor"/>
>                          <Interceptor
> className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
>                          <Interceptor
> className="org.apache.catalina.tribes.group.interceptors.StaticMembershipInterceptor">
>                             <LocalMember
> className="org.apache.catalina.tribes.membership.StaticMember"
>                                port="8091"
>                                host="localhost"
>                                uniqueId="{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1}"
>                                domain="delta-static"
>                                />
>                                <Member
> className="org.apache.catalina.tribes.membership.StaticMember"
>                                    port="8191"
>                                    host="localhost"
> 
> uniqueId="{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2}"
>                                    domain="delta-static"
>                                    />
>                                </Interceptor>
>                                <Interceptor
> className="org.apache.catalina.tribes.group.interceptors.MessageDispatchInterceptor"/>
> 
> 
>                            </Channel>
>                            <Valve
> className="org.apache.catalina.ha.tcp.ReplicationValve" filter=""/>
>                            <Valve
> className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>
> 
>                            <ClusterListener
> className="org.apache.catalina.ha.session.ClusterSessionListener"/>
>                        </Cluster>
> ```
> 
> For node with http port
> ```
> <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
> channelSendOptions="6" channelStartOptions="3">
> 
>                <Manager
> className="org.apache.catalina.ha.session.DeltaManager"
>                  expireSessionsOnShutdown="false"
>                  notifyListenersOnReplication="true"/>
> 
>                  <Channel
> className="org.apache.catalina.tribes.group.GroupChannel">
> 
>                      <Receiver
> className="org.apache.catalina.tribes.transport.nio.NioReceiver"
>                          address="localhost"
>                          port="8191"
>                          autoBind="0"
>                          selectorTimeout="5000"
>                          maxThreads="6"/>
> 
>                          <Sender
> className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
>                              <Transport
> className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
>                          </Sender>
>                          <Interceptor
> className="org.apache.catalina.tribes.group.interceptors.TcpPingInterceptor"/>
>                          <Interceptor
> className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
>                          <Interceptor
> className="org.apache.catalina.tribes.group.interceptors.StaticMembershipInterceptor">
> 
>                              <LocalMember
> className="org.apache.catalina.tribes.membership.StaticMember"
>                                  port="8191"
>                                  host="localhost"
>                                  uniqueId="{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2}"
>                                  domain="delta-static"
>                                  />
>                                  <Member
> className="org.apache.catalina.tribes.membership.StaticMember"
>                                      port="8091"
>                                      host="localhost"
> 
> uniqueId="{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1}"
>                                      domain="delta-static"
>                                      />
> 
>                                  </Interceptor>
>                                  <Interceptor
> className="org.apache.catalina.tribes.group.interceptors.MessageDispatchInterceptor"/>
> 
>                              </Channel>
>                              <Valve
> className="org.apache.catalina.ha.tcp.ReplicationValve" filter=""/>
>                              <Valve
> className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>
> 
>                              <ClusterListener
> className="org.apache.catalina.ha.session.ClusterSessionListener"/>
>                          </Cluster>
> ```
> 
> I am trying to run *examples/jsp/sessions/carts.jsp*. Both the webapps are
> marked *<distributable /> *and I have implemented *Serializable *in
> the *DummyCart.java
> *class in both webapps. When viewed through the *manager/html* interface, I
> can see the session status alternating between *primary *and *backup *status
> as the requests alternate between the two nodes. The session id regardless
> of which node is serving the request. *However, the state of the DummyCart
> object is not shared between the two nodes even though the scope is defined
> as session and the cart attribute is visible in the manager web interface.*
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Error "Unable to send message through cluster sender" in a cluster with static members using delta manager

Posted by Manak Bisht <ma...@iiitd.ac.in>.
Thank you for the clarification. Sorry, it took me a while to understand
your point.
I have successfully changed the cluster settings,
For node with http port 8090
```
          <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
channelSendOptions="6" channelStartOptions="3">
              <Manager
className="org.apache.catalina.ha.session.DeltaManager"
                expireSessionsOnShutdown="false"
                notifyListenersOnReplication="true"/>

                <Channel
className="org.apache.catalina.tribes.group.GroupChannel">

                    <Receiver
className="org.apache.catalina.tribes.transport.nio.NioReceiver"
                        address="localhost"
                        port="8091"
                        autoBind="0"
                        selectorTimeout="5000"
                        maxThreads="6"/>

                        <Sender
className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
                            <Transport
className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
                        </Sender>
                        <Interceptor
className="org.apache.catalina.tribes.group.interceptors.TcpPingInterceptor"/>
                        <Interceptor
className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
                        <Interceptor
className="org.apache.catalina.tribes.group.interceptors.StaticMembershipInterceptor">
                           <LocalMember
className="org.apache.catalina.tribes.membership.StaticMember"
                              port="8091"
                              host="localhost"
                              uniqueId="{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1}"
                              domain="delta-static"
                              />
                              <Member
className="org.apache.catalina.tribes.membership.StaticMember"
                                  port="8191"
                                  host="localhost"

uniqueId="{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2}"
                                  domain="delta-static"
                                  />
                              </Interceptor>
                              <Interceptor
className="org.apache.catalina.tribes.group.interceptors.MessageDispatchInterceptor"/>


                          </Channel>
                          <Valve
className="org.apache.catalina.ha.tcp.ReplicationValve" filter=""/>
                          <Valve
className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>

                          <ClusterListener
className="org.apache.catalina.ha.session.ClusterSessionListener"/>
                      </Cluster>
```

For node with http port
```
<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
channelSendOptions="6" channelStartOptions="3">

              <Manager
className="org.apache.catalina.ha.session.DeltaManager"
                expireSessionsOnShutdown="false"
                notifyListenersOnReplication="true"/>

                <Channel
className="org.apache.catalina.tribes.group.GroupChannel">

                    <Receiver
className="org.apache.catalina.tribes.transport.nio.NioReceiver"
                        address="localhost"
                        port="8191"
                        autoBind="0"
                        selectorTimeout="5000"
                        maxThreads="6"/>

                        <Sender
className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
                            <Transport
className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
                        </Sender>
                        <Interceptor
className="org.apache.catalina.tribes.group.interceptors.TcpPingInterceptor"/>
                        <Interceptor
className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
                        <Interceptor
className="org.apache.catalina.tribes.group.interceptors.StaticMembershipInterceptor">

                            <LocalMember
className="org.apache.catalina.tribes.membership.StaticMember"
                                port="8191"
                                host="localhost"
                                uniqueId="{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2}"
                                domain="delta-static"
                                />
                                <Member
className="org.apache.catalina.tribes.membership.StaticMember"
                                    port="8091"
                                    host="localhost"

uniqueId="{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1}"
                                    domain="delta-static"
                                    />

                                </Interceptor>
                                <Interceptor
className="org.apache.catalina.tribes.group.interceptors.MessageDispatchInterceptor"/>

                            </Channel>
                            <Valve
className="org.apache.catalina.ha.tcp.ReplicationValve" filter=""/>
                            <Valve
className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>

                            <ClusterListener
className="org.apache.catalina.ha.session.ClusterSessionListener"/>
                        </Cluster>
```

I am trying to run *examples/jsp/sessions/carts.jsp*. Both the webapps are
marked *<distributable /> *and I have implemented *Serializable *in
the *DummyCart.java
*class in both webapps. When viewed through the *manager/html* interface, I
can see the session status alternating between *primary *and *backup *status
as the requests alternate between the two nodes. The session id regardless
of which node is serving the request. *However, the state of the DummyCart
object is not shared between the two nodes even though the scope is defined
as session and the cart attribute is visible in the manager web interface.*

Re: Error "Unable to send message through cluster sender" in a cluster with static members using delta manager

Posted by Mark Thomas <ma...@apache.org>.
Again, you are mixing HTTP ports and cluster ports. They must be different.


On 12/06/2023 17:22, Manak Bisht wrote:

<snip/>

Node 1: Cluster listening on 8090

>                      <Receiver
> className="org.apache.catalina.tribes.transport.nio.NioReceiver"
>                          address="localhost"
>                          port="8090"
>                          autoBind="0"
>                          selectorTimeout="5000"
>                          maxThreads="6"/>


<snip/>

Node 2: Cluster listening on 8190

>                      <Receiver
> className="org.apache.catalina.tribes.transport.nio.NioReceiver"
>                          address="localhost"
>                          port="8190"
>                          autoBind="0"
>                          selectorTimeout="5000"
>                          maxThreads="6"/>

<snip/>

> 12-Jun-2023 14:18:17.719 INFO [main]
> org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler
> ["http-nio-8090"]

HTTP connector also trying to listen on port 8090.

This WILL NOT work.

You MUST choose different ports for the cluster communications and the 
HTTP connector.

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Error "Unable to send message through cluster sender" in a cluster with static members using delta manager

Posted by Manak Bisht <ma...@iiitd.ac.in>.
Thanks for the response! I did follow this example
https://people.apache.org/~markt/dev/server-static-cluster-example.xml. The
only difference being that I am using tomcat 8.5 so I am using the
StaticMembershipInterceptor instead. In my second email, I have followed
this configuration, however I encounter the error -
*12-Jun-2023 14:18:26.507 SEVERE [Tribes-Task-Receiver[Catalina-Channel]-3]
org.apache.catalina.tribes.io.XByteBuffer.append Discarded the package,
invalid header*
I am copying that email here for your reference. *Cluster settings - *
``
          <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
channelSendOptions="6" channelStartOptions="3">
              <Manager
className="org.apache.catalina.ha.session.DeltaManager"
                expireSessionsOnShutdown="false"
                notifyListenersOnReplication="true"/>

                <Channel
className="org.apache.catalina.tribes.group.GroupChannel">

                    <Receiver
className="org.apache.catalina.tribes.transport.nio.NioReceiver"
                        address="localhost"
                        port="8090"
                        autoBind="0"
                        selectorTimeout="5000"
                        maxThreads="6"/>

                        <Sender
className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
                            <Transport
className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
                        </Sender>
                        <Interceptor
className="org.apache.catalina.tribes.group.interceptors.TcpPingInterceptor"/>
                        <Interceptor
className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
                        <Interceptor
className="org.apache.catalina.tribes.group.interceptors.MessageDispatchInterceptor"/>
                        <Interceptor
className="org.apache.catalina.tribes.group.interceptors.StaticMembershipInterceptor">
                         <LocalMember
className="org.apache.catalina.tribes.membership.StaticMember"
                          port="8090"
                          host="localhost"
                          uniqueId="{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1}"/>
                          <Member
className="org.apache.catalina.tribes.membership.StaticMember"
                              port="8190"
                              host="localhost"
                              uniqueId="{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2}"/>
                          </Interceptor>

                      </Channel>
                      <Valve
className="org.apache.catalina.ha.tcp.ReplicationValve" filter=""/>
                      <Valve
className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>

                      <ClusterListener
className="org.apache.catalina.ha.session.ClusterSessionListener"/>
                  </Cluster>
```

```
          <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
channelSendOptions="6" channelStartOptions="3">

              <Manager
className="org.apache.catalina.ha.session.DeltaManager"
                expireSessionsOnShutdown="false"
                notifyListenersOnReplication="true"/>

                <Channel
className="org.apache.catalina.tribes.group.GroupChannel">

                    <Receiver
className="org.apache.catalina.tribes.transport.nio.NioReceiver"
                        address="localhost"
                        port="8190"
                        autoBind="0"
                        selectorTimeout="5000"
                        maxThreads="6"/>

                        <Sender
className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
                            <Transport
className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
                        </Sender>
                        <Interceptor
className="org.apache.catalina.tribes.group.interceptors.TcpPingInterceptor"/>
                        <Interceptor
className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
                        <Interceptor
className="org.apache.catalina.tribes.group.interceptors.MessageDispatchInterceptor"/>
                        <Interceptor
className="org.apache.catalina.tribes.group.interceptors.StaticMembershipInterceptor">

                            <LocalMember
className="org.apache.catalina.tribes.membership.StaticMember"
                                port="8190"
                                host="localhost"

uniqueId="{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2}"/>
                                <Member
className="org.apache.catalina.tribes.membership.StaticMember"
                                    port="8090"
                                    host="localhost"

uniqueId="{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1}"/>

                                </Interceptor>

                            </Channel>
                            <Valve
className="org.apache.catalina.ha.tcp.ReplicationValve" filter=""/>
                            <Valve
className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>

                            <ClusterListener
className="org.apache.catalina.ha.session.ClusterSessionListener"/>
                        </Cluster>
```

*Logs,*
I get the following error log on the node with port 8090 when I try to
access it from the browser (through an nginx load balancer that I have
configured and even when accessed directly).
```
12-Jun-2023 14:18:17.675 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Server built:
 May 9 2023 16:21:47 UTC
12-Jun-2023 14:18:17.676 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Server version
number: 8.5.89.0
12-Jun-2023 14:18:17.676 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log OS Name:
  Mac OS X
12-Jun-2023 14:18:17.676 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log OS Version:
 10.16
12-Jun-2023 14:18:17.676 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Architecture:
 x86_64
12-Jun-2023 14:18:17.676 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Java Home:
  /Users/manak/.sdkman/candidates/java/8.0.372.fx-librca/jre
12-Jun-2023 14:18:17.676 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log JVM Version:
  1.8.0_372-b07
12-Jun-2023 14:18:17.676 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log JVM Vendor:
 BellSoft
12-Jun-2023 14:18:17.676 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log CATALINA_BASE:
  /Users/manak/apache-tomcat-8.5.89
12-Jun-2023 14:18:17.677 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log CATALINA_HOME:
  /Users/manak/apache-tomcat-8.5.89
12-Jun-2023 14:18:17.677 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Command line
argument:
-Djava.util.logging.config.file=/Users/manak/apache-tomcat-8.5.89/conf/logging.properties
12-Jun-2023 14:18:17.677 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Command line
argument: -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
12-Jun-2023 14:18:17.677 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Command line
argument: -Djdk.tls.ephemeralDHKeySize=2048
12-Jun-2023 14:18:17.677 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Command line
argument: -Djava.protocol.handler.pkgs=org.apache.catalina.webresources
12-Jun-2023 14:18:17.678 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Command line
argument: -Djava.net.preferIPv4Stack=true
12-Jun-2023 14:18:17.678 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Command line
argument: -Djava.net.preferIPv4Addresses=true
12-Jun-2023 14:18:17.679 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Command line
argument: -Dorg.apache.catalina.security.SecurityListener.UMASK=0027
12-Jun-2023 14:18:17.679 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Command line
argument: -Dignore.endorsed.dirs=
12-Jun-2023 14:18:17.679 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Command line
argument: -Dcatalina.base=/Users/manak/apache-tomcat-8.5.89
12-Jun-2023 14:18:17.679 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Command line
argument: -Dcatalina.home=/Users/manak/apache-tomcat-8.5.89
12-Jun-2023 14:18:17.680 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Command line
argument: -Djava.io.tmpdir=/Users/manak/apache-tomcat-8.5.89/temp
12-Jun-2023 14:18:17.680 INFO [main]
org.apache.catalina.core.AprLifecycleListener.lifecycleEvent The Apache
Tomcat Native library which allows using OpenSSL was not found on the
java.library.path:
[/Users/manak/Library/Java/Extensions:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java:.]
12-Jun-2023 14:18:17.719 INFO [main]
org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler
["http-nio-8090"]
12-Jun-2023 14:18:17.739 INFO [main]
org.apache.catalina.startup.Catalina.load Initialization processed in 536 ms
12-Jun-2023 14:18:17.844 INFO [main]
org.apache.catalina.core.StandardService.startInternal Starting service
[Catalina]
12-Jun-2023 14:18:17.845 INFO [main]
org.apache.catalina.core.StandardEngine.startInternal Starting Servlet
engine: [Apache Tomcat/8.5.89]
12-Jun-2023 14:18:17.850 INFO [main]
org.apache.catalina.ha.tcp.SimpleTcpCluster.startInternal Cluster is about
to start
12-Jun-2023 14:18:17.860 INFO [main]
org.apache.catalina.tribes.transport.ReceiverBase.bind Receiver Server
Socket bound to:[localhost/127.0.0.1:8090]
12-Jun-2023 14:18:17.873 INFO [Thread-2]
org.apache.catalina.ha.tcp.SimpleTcpCluster.memberAdded Replication member
added:[org.apache.catalina.tribes.membership.StaticMember[tcp://localhost:8190,localhost,8190,
alive=0, securePort=-1, UDP Port=-1, id={0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 },
payload={}, command={}, domain={}]]
12-Jun-2023 14:18:17.886 INFO [Tribes-Task-Receiver[Catalina-Channel]-1]
org.apache.catalina.tribes.io.BufferPool.getBufferPool Created a buffer
pool with max size:[104857600] bytes of type:
[org.apache.catalina.tribes.io.BufferPool15Impl]
12-Jun-2023 14:18:17.888 INFO [localhost-startStop-1]
org.apache.catalina.startup.HostConfig.deployDirectory Deploying web
application directory [/Users/manak/apache-tomcat-8.5.89/webapps/docs]
12-Jun-2023 14:18:18.081 INFO [localhost-startStop-1]
org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web
application directory [/Users/manak/apache-tomcat-8.5.89/webapps/docs] has
finished in [193] ms
12-Jun-2023 14:18:18.081 INFO [localhost-startStop-1]
org.apache.catalina.startup.HostConfig.deployDirectory Deploying web
application directory [/Users/manak/apache-tomcat-8.5.89/webapps/manager]
12-Jun-2023 14:18:18.112 INFO [localhost-startStop-1]
org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web
application directory [/Users/manak/apache-tomcat-8.5.89/webapps/manager]
has finished in [31] ms
12-Jun-2023 14:18:18.112 INFO [localhost-startStop-1]
org.apache.catalina.startup.HostConfig.deployDirectory Deploying web
application directory [/Users/manak/apache-tomcat-8.5.89/webapps/examples]
12-Jun-2023 14:18:18.370 INFO [localhost-startStop-1]
org.apache.catalina.ha.session.DeltaManager.startInternal Register manager
[localhost#/examples] to cluster element [Engine] with name [Catalina]
12-Jun-2023 14:18:18.370 INFO [localhost-startStop-1]
org.apache.catalina.ha.session.DeltaManager.startInternal Starting
clustering manager at [localhost#/examples]
12-Jun-2023 14:18:18.399 INFO [localhost-startStop-1]
org.apache.catalina.ha.session.DeltaManager.getAllClusterSessions Manager
[localhost#/examples], requesting session state from
[org.apache.catalina.tribes.membership.StaticMember[tcp://localhost:8190,localhost,8190,
alive=0, securePort=-1, UDP Port=-1, id={0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 },
payload={}, command={}, domain={}]]. This operation will timeout if no
session state has been received within [60] seconds.
12-Jun-2023 14:18:18.504 INFO [localhost-startStop-1]
org.apache.catalina.ha.session.DeltaManager.waitForSendAllSessions Manager
[localhost#/examples]; session state sent at [6/12/23 2:18 PM] received in
[132] ms.
12-Jun-2023 14:18:18.509 INFO [localhost-startStop-1]
org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web
application directory [/Users/manak/apache-tomcat-8.5.89/webapps/examples]
has finished in [397] ms
12-Jun-2023 14:18:18.509 INFO [localhost-startStop-1]
org.apache.catalina.startup.HostConfig.deployDirectory Deploying web
application directory [/Users/manak/apache-tomcat-8.5.89/webapps/ROOT]
12-Jun-2023 14:18:18.528 INFO [localhost-startStop-1]
org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web
application directory [/Users/manak/apache-tomcat-8.5.89/webapps/ROOT] has
finished in [19] ms
12-Jun-2023 14:18:18.528 INFO [localhost-startStop-1]
org.apache.catalina.startup.HostConfig.deployDirectory Deploying web
application directory
[/Users/manak/apache-tomcat-8.5.89/webapps/host-manager]
12-Jun-2023 14:18:18.548 INFO [localhost-startStop-1]
org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web
application directory
[/Users/manak/apache-tomcat-8.5.89/webapps/host-manager] has finished in
[20] ms
12-Jun-2023 14:18:18.549 INFO [main]
org.apache.catalina.ha.session.JvmRouteBinderValve.startInternal
JvmRouteBinderValve started
12-Jun-2023 14:18:18.551 INFO [main]
org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler
["http-nio-8090"]
12-Jun-2023 14:18:18.561 INFO [main]
org.apache.catalina.startup.Catalina.start Server startup in 822 ms
*12-Jun-2023 14:18:26.507 SEVERE [Tribes-Task-Receiver[Catalina-Channel]-3]
org.apache.catalina.tribes.io.XByteBuffer.append Discarded the package,
invalid header*
```

Re: Error "Unable to send message through cluster sender" in a cluster with static members using delta manager

Posted by Mark Thomas <ma...@apache.org>.
On 12/06/2023 16:44, Manak Bisht wrote:
> I am not sure if the receiver port should be the same or different from the
> localmember port. However, changing that to 4001 for node with localmember
> at 8090 and to 4002 for node with localmember at 8190, I am encountering
> the following error -

You are mixing the HTTP ports and the cluster ports. They should be 
completely separate.

Look at this working example:

https://people.apache.org/~markt/dev/server-static-cluster-example.xml

It uses StaticMembershipService and the nodes are each on their own VM 
so you'll need to tweak it but it should give you the idea for how to 
configure the ports.

The port for the Member needs to match the port in the Receiver element 
for that Member.

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Error "Unable to send message through cluster sender" in a cluster with static members using delta manager

Posted by Manak Bisht <ma...@iiitd.ac.in>.
I am not sure if the receiver port should be the same or different from the
localmember port. However, changing that to 4001 for node with localmember
at 8090 and to 4002 for node with localmember at 8190, I am encountering
the following error -
```
12-Jun-2023 15:00:29.885 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Server version name:
  Apache Tomcat/8.5.89
12-Jun-2023 15:00:29.887 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Server built:
 May 9 2023 16:21:47 UTC
12-Jun-2023 15:00:29.887 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Server version
number: 8.5.89.0
12-Jun-2023 15:00:29.887 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log OS Name:
  Mac OS X
12-Jun-2023 15:00:29.887 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log OS Version:
 10.16
12-Jun-2023 15:00:29.887 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Architecture:
 x86_64
12-Jun-2023 15:00:29.887 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Java Home:
  /Users/manak/.sdkman/candidates/java/8.0.372.fx-librca/jre
12-Jun-2023 15:00:29.887 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log JVM Version:
  1.8.0_372-b07
12-Jun-2023 15:00:29.887 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log JVM Vendor:
 BellSoft
12-Jun-2023 15:00:29.888 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log CATALINA_BASE:
  /Users/manak/apache-tomcat-8.5.89
12-Jun-2023 15:00:29.888 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log CATALINA_HOME:
  /Users/manak/apache-tomcat-8.5.89
12-Jun-2023 15:00:29.888 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Command line
argument:
-Djava.util.logging.config.file=/Users/manak/apache-tomcat-8.5.89/conf/logging.properties
12-Jun-2023 15:00:29.889 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Command line
argument: -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
12-Jun-2023 15:00:29.889 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Command line
argument: -Djdk.tls.ephemeralDHKeySize=2048
12-Jun-2023 15:00:29.889 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Command line
argument: -Djava.protocol.handler.pkgs=org.apache.catalina.webresources
12-Jun-2023 15:00:29.889 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Command line
argument: -Djava.net.preferIPv4Stack=true
12-Jun-2023 15:00:29.890 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Command line
argument: -Djava.net.preferIPv4Addresses=true
12-Jun-2023 15:00:29.890 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Command line
argument: -Dorg.apache.catalina.security.SecurityListener.UMASK=0027
12-Jun-2023 15:00:29.890 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Command line
argument: -Dignore.endorsed.dirs=
12-Jun-2023 15:00:29.890 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Command line
argument: -Dcatalina.base=/Users/manak/apache-tomcat-8.5.89
12-Jun-2023 15:00:29.891 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Command line
argument: -Dcatalina.home=/Users/manak/apache-tomcat-8.5.89
12-Jun-2023 15:00:29.891 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Command line
argument: -Djava.io.tmpdir=/Users/manak/apache-tomcat-8.5.89/temp
12-Jun-2023 15:00:29.891 INFO [main]
org.apache.catalina.core.AprLifecycleListener.lifecycleEvent The Apache
Tomcat Native library which allows using OpenSSL was not found on the
java.library.path:
[/Users/manak/Library/Java/Extensions:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java:.]
12-Jun-2023 15:00:29.931 INFO [main]
org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler
["http-nio-8090"]
12-Jun-2023 15:00:29.954 INFO [main]
org.apache.catalina.startup.Catalina.load Initialization processed in 548 ms
12-Jun-2023 15:00:30.046 INFO [main]
org.apache.catalina.core.StandardService.startInternal Starting service
[Catalina]
12-Jun-2023 15:00:30.046 INFO [main]
org.apache.catalina.core.StandardEngine.startInternal Starting Servlet
engine: [Apache Tomcat/8.5.89]
12-Jun-2023 15:00:30.050 INFO [main]
org.apache.catalina.ha.tcp.SimpleTcpCluster.startInternal Cluster is about
to start
12-Jun-2023 15:00:30.059 INFO [main]
org.apache.catalina.tribes.transport.ReceiverBase.bind Receiver Server
Socket bound to:[localhost/127.0.0.1:4001]
12-Jun-2023 15:00:30.073 INFO [localhost-startStop-1]
org.apache.catalina.startup.HostConfig.deployDirectory Deploying web
application directory [/Users/manak/apache-tomcat-8.5.89/webapps/docs]
12-Jun-2023 15:00:30.301 INFO [localhost-startStop-1]
org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web
application directory [/Users/manak/apache-tomcat-8.5.89/webapps/docs] has
finished in [228] ms
12-Jun-2023 15:00:30.302 INFO [localhost-startStop-1]
org.apache.catalina.startup.HostConfig.deployDirectory Deploying web
application directory [/Users/manak/apache-tomcat-8.5.89/webapps/manager]
12-Jun-2023 15:00:30.332 INFO [localhost-startStop-1]
org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web
application directory [/Users/manak/apache-tomcat-8.5.89/webapps/manager]
has finished in [30] ms
12-Jun-2023 15:00:30.332 INFO [localhost-startStop-1]
org.apache.catalina.startup.HostConfig.deployDirectory Deploying web
application directory [/Users/manak/apache-tomcat-8.5.89/webapps/examples]
12-Jun-2023 15:00:30.596 INFO [localhost-startStop-1]
org.apache.catalina.ha.session.DeltaManager.startInternal Register manager
[localhost#/examples] to cluster element [Engine] with name [Catalina]
12-Jun-2023 15:00:30.597 INFO [localhost-startStop-1]
org.apache.catalina.ha.session.DeltaManager.startInternal Starting
clustering manager at [localhost#/examples]
12-Jun-2023 15:00:30.597 INFO [localhost-startStop-1]
org.apache.catalina.ha.session.DeltaManager.getAllClusterSessions Manager
[localhost#/examples]: skipping state transfer. No members active in
cluster group.
12-Jun-2023 15:00:30.602 INFO [localhost-startStop-1]
org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web
application directory [/Users/manak/apache-tomcat-8.5.89/webapps/examples]
has finished in [270] ms
12-Jun-2023 15:00:30.602 INFO [localhost-startStop-1]
org.apache.catalina.startup.HostConfig.deployDirectory Deploying web
application directory [/Users/manak/apache-tomcat-8.5.89/webapps/ROOT]
12-Jun-2023 15:00:30.618 INFO [localhost-startStop-1]
org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web
application directory [/Users/manak/apache-tomcat-8.5.89/webapps/ROOT] has
finished in [15] ms
12-Jun-2023 15:00:30.618 INFO [localhost-startStop-1]
org.apache.catalina.startup.HostConfig.deployDirectory Deploying web
application directory
[/Users/manak/apache-tomcat-8.5.89/webapps/host-manager]
12-Jun-2023 15:00:30.635 INFO [localhost-startStop-1]
org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web
application directory
[/Users/manak/apache-tomcat-8.5.89/webapps/host-manager] has finished in
[17] ms
12-Jun-2023 15:00:30.636 INFO [main]
org.apache.catalina.ha.session.JvmRouteBinderValve.startInternal
JvmRouteBinderValve started
12-Jun-2023 15:00:30.638 INFO [main]
org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler
["http-nio-8090"]
12-Jun-2023 15:00:30.647 INFO [main]
org.apache.catalina.startup.Catalina.start Server startup in 692 ms


*12-Jun-2023 15:00:34.987 INFO [http-nio-8090-exec-1]
org.apache.coyote.http11.Http11Processor.service Error parsing HTTP request
header Note: further occurrences of HTTP request parsing errors will be
logged at DEBUG level. java.lang.IllegalArgumentException: Invalid
character found in method name
[FLT20020x000x000x000xb50x000x000x000x010x000x000x010x880xae0xf1bA0x000x000x000x100xa30x1ckY0xd70xa0L0xfc0x8f0xe8i0x9e0xce0x830xb2P0x000x000x00MTRIBES-B0x010x000x000x000x0050x000x000x010x880xae0xf1bA0x000x000x0f0xa20xff0xff0xff0xff0xff0xff0xff0xff0x040x7f0x000x000x010x000x000x000x000x000x000x000x000x000x000x000x000x000x000x000x000x000x000x000x000x000x000x000x020x000x000x000x00TRIBES-E0x010x000x000x000x00@O0xa7sHy0x82C0xc90x9fo0x890x800xa1[0x070x14}0xd9R[0xeb0xf1C0x9a0xb7~0xbe0x8f0x81g0x1e0xb670x150xbe0x87E~L0xa80xbf0x0aM0x13S80x152U0xf60x940xb7:0xfa@x0x910x04}0xd7r0x840xc00xd5TLF2003...].
HTTP method names must be tokens*
at
org.apache.coyote.http11.Http11InputBuffer.parseRequestLine(Http11InputBuffer.java:422)
at
org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:492)
at
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63)
at
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:932)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1695)
at
org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
at
org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
at
org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:750)


*12-Jun-2023 15:00:34.987 INFO [http-nio-8090-exec-2]
org.apache.coyote.http11.Http11Processor.service Error parsing HTTP request
header Note: further occurrences of HTTP request parsing errors will be
logged at DEBUG level. java.lang.IllegalArgumentException: Invalid
character found in method name
[FLT20020x000x000x000x990x000x000x000x000x000x000x010x880xae0xf1bC0x000x000x000x10P0xee)-0xc40xd6A0x120x8a0x9c0xab0x09C0x940xf9p0x000x000x00MTRIBES-B0x010x000x000x000x0050x000x000x010x880xae0xf1bA0x000x000x0f0xa20xff0xff0xff0xff0xff0xff0xff0xff0x040x7f0x000x000x010x000x000x000x000x000x000x000x000x000x000x000x000x000x000x000x000x000x000x000x000x000x000x000x020x000x000x000x00TRIBES-E0x010x000x000x000x00$Local
]. HTTP method names must be tokens*
at
org.apache.coyote.http11.Http11InputBuffer.parseRequestLine(Http11InputBuffer.java:422)
at
org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:492)
at
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63)
at
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:932)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1695)
at
org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
at
org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
at
org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:750)
12-Jun-2023 15:00:35.066 INFO [GroupChannel-Heartbeat[Catalina-Channel]-1]
org.apache.catalina.ha.tcp.SimpleTcpCluster.memberAdded Replication member
added:[org.apache.catalina.tribes.membership.StaticMember[tcp://localhost:8190,localhost,8190,
alive=0, securePort=-1, UDP Port=-1, id={0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 },
payload={}, command={}, domain={}]]
12-Jun-2023 15:00:35.067 INFO [GroupChannel-Heartbeat[Catalina-Channel]-1]
org.apache.catalina.tribes.group.interceptors.TcpFailureDetector.performBasicCheck
Suspect member, confirmed
alive.[org.apache.catalina.tribes.membership.StaticMember[tcp://localhost:8190,localhost,8190,
alive=0, securePort=-1, UDP Port=-1, id={0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 },
payload={}, command={}, domain={}]]
12-Jun-2023 15:01:58.330 INFO [http-nio-8090-exec-7]
org.apache.catalina.tribes.io.BufferPool.getBufferPool Created a buffer
pool with max size:[104857600] bytes of type:
[org.apache.catalina.tribes.io.BufferPool15Impl]
12-Jun-2023 15:01:58.343 WARNING [http-nio-8090-exec-7]
org.apache.catalina.tribes.transport.nio.ParallelNioSender.doLoop Member
send is failing for:[tcp://localhost:8190] ; Setting to suspect and
retrying.
12-Jun-2023 15:01:58.346 INFO [http-nio-8090-exec-7]
org.apache.catalina.tribes.group.interceptors.TcpFailureDetector.memberDisappeared
Received
memberDisappeared[org.apache.catalina.tribes.membership.StaticMember[tcp://localhost:8190,localhost,8190,
alive=0, securePort=-1, UDP Port=-1, id={0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 },
payload={}, command={}, domain={}]] message. Will verify.
12-Jun-2023 15:01:58.346 INFO [http-nio-8090-exec-7]
org.apache.catalina.tribes.group.interceptors.TcpFailureDetector.memberDisappeared
Verification complete. Member still
alive[org.apache.catalina.tribes.membership.StaticMember[tcp://localhost:8190,localhost,8190,
alive=0, securePort=-1, UDP Port=-1, id={0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 },
payload={}, command={}, domain={}]]
```

Re: Error "Unable to send message through cluster sender" in a cluster with static members using delta manager

Posted by Manak Bisht <ma...@iiitd.ac.in>.
On Mon, Jun 12, 2023 at 1:50 PM Mark Thomas <ma...@apache.org> wrote:

> On 10/06/2023 18:11, Manak Bisht wrote:
> > I am trying to run the carts example (*/examples/jsp/sessions/carts.jsp*)
> > with two nodes, on ports 8090 and 8190. The nodes are working fine
> > individually. However, I am trying to create a cluster with static
> members
> > (without multicast), but I keep encountering the following error -
>
> <snip/>
>
> >                      <Receiver
> > className="org.apache.catalina.tribes.transport.nio.NioReceiver"/>
>
> You are listening on the default port of 4000 ...
>
> >                              <Member
> > className="org.apache.catalina.tribes.membership.StaticMember"
> >                                  port="8090"
> >                                  securePort="-1"
> >                                  host="localhost"
> >                                  domain="delta-static"
> >
> > uniqueId="{1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,0}"
> >                                  />
>
> ... but trying to send to port 8009.
>
> The send/receive ports for the cluster need to match.
>
> Since you are running on a single machine I would explicitly configure
> the ports as:
> node1 - 4001
> node2 - 4002
>
> and so on.
>
> Keep in mind you will also need unique HTTP, HTTPS, AJP and shutdown
> ports for each node (as required depending on what is enabled).
>
> Mark
>

Thanks for the reply. I did figure this out eventually, I have explicitly
changed the receiver ports for both the clusters,
This is what the current cluster settings look like -
```
          <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
channelSendOptions="6" channelStartOptions="3">
              <Manager
className="org.apache.catalina.ha.session.DeltaManager"
                expireSessionsOnShutdown="false"
                notifyListenersOnReplication="true"/>

                <Channel
className="org.apache.catalina.tribes.group.GroupChannel">

                    <Receiver
className="org.apache.catalina.tribes.transport.nio.NioReceiver"
                        address="localhost"
                        port="8090"
                        autoBind="0"
                        selectorTimeout="5000"
                        maxThreads="6"/>

                        <Sender
className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
                            <Transport
className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
                        </Sender>
                        <Interceptor
className="org.apache.catalina.tribes.group.interceptors.TcpPingInterceptor"/>
                        <Interceptor
className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
                        <Interceptor
className="org.apache.catalina.tribes.group.interceptors.MessageDispatchInterceptor"/>
                        <Interceptor
className="org.apache.catalina.tribes.group.interceptors.StaticMembershipInterceptor">
                         <LocalMember
className="org.apache.catalina.tribes.membership.StaticMember"
                          port="8090"
                          host="localhost"
                          uniqueId="{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1}"/>
                          <Member
className="org.apache.catalina.tribes.membership.StaticMember"
                              port="8190"
                              host="localhost"
                              uniqueId="{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2}"/>
                          </Interceptor>

                      </Channel>
                      <Valve
className="org.apache.catalina.ha.tcp.ReplicationValve" filter=""/>
                      <Valve
className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>

                      <ClusterListener
className="org.apache.catalina.ha.session.ClusterSessionListener"/>
                  </Cluster>
```

```
          <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
channelSendOptions="6" channelStartOptions="3">

              <Manager
className="org.apache.catalina.ha.session.DeltaManager"
                expireSessionsOnShutdown="false"
                notifyListenersOnReplication="true"/>

                <Channel
className="org.apache.catalina.tribes.group.GroupChannel">

                    <Receiver
className="org.apache.catalina.tribes.transport.nio.NioReceiver"
                        address="localhost"
                        port="8190"
                        autoBind="0"
                        selectorTimeout="5000"
                        maxThreads="6"/>

                        <Sender
className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
                            <Transport
className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
                        </Sender>
                        <Interceptor
className="org.apache.catalina.tribes.group.interceptors.TcpPingInterceptor"/>
                        <Interceptor
className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
                        <Interceptor
className="org.apache.catalina.tribes.group.interceptors.MessageDispatchInterceptor"/>
                        <Interceptor
className="org.apache.catalina.tribes.group.interceptors.StaticMembershipInterceptor">

                            <LocalMember
className="org.apache.catalina.tribes.membership.StaticMember"
                                port="8190"
                                host="localhost"

uniqueId="{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2}"/>
                                <Member
className="org.apache.catalina.tribes.membership.StaticMember"
                                    port="8090"
                                    host="localhost"

uniqueId="{0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1}"/>

                                </Interceptor>

                            </Channel>
                            <Valve
className="org.apache.catalina.ha.tcp.ReplicationValve" filter=""/>
                            <Valve
className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>

                            <ClusterListener
className="org.apache.catalina.ha.session.ClusterSessionListener"/>
                        </Cluster>
```

However, I get the following error log on the node with port 8090 when I
try to access it from the browser (through an nginx load balancer that I
have configured).
```
12-Jun-2023 14:18:17.675 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Server built:
 May 9 2023 16:21:47 UTC
12-Jun-2023 14:18:17.676 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Server version
number: 8.5.89.0
12-Jun-2023 14:18:17.676 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log OS Name:
  Mac OS X
12-Jun-2023 14:18:17.676 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log OS Version:
 10.16
12-Jun-2023 14:18:17.676 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Architecture:
 x86_64
12-Jun-2023 14:18:17.676 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Java Home:
  /Users/manak/.sdkman/candidates/java/8.0.372.fx-librca/jre
12-Jun-2023 14:18:17.676 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log JVM Version:
  1.8.0_372-b07
12-Jun-2023 14:18:17.676 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log JVM Vendor:
 BellSoft
12-Jun-2023 14:18:17.676 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log CATALINA_BASE:
  /Users/manak/apache-tomcat-8.5.89
12-Jun-2023 14:18:17.677 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log CATALINA_HOME:
  /Users/manak/apache-tomcat-8.5.89
12-Jun-2023 14:18:17.677 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Command line
argument:
-Djava.util.logging.config.file=/Users/manak/apache-tomcat-8.5.89/conf/logging.properties
12-Jun-2023 14:18:17.677 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Command line
argument: -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
12-Jun-2023 14:18:17.677 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Command line
argument: -Djdk.tls.ephemeralDHKeySize=2048
12-Jun-2023 14:18:17.677 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Command line
argument: -Djava.protocol.handler.pkgs=org.apache.catalina.webresources
12-Jun-2023 14:18:17.678 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Command line
argument: -Djava.net.preferIPv4Stack=true
12-Jun-2023 14:18:17.678 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Command line
argument: -Djava.net.preferIPv4Addresses=true
12-Jun-2023 14:18:17.679 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Command line
argument: -Dorg.apache.catalina.security.SecurityListener.UMASK=0027
12-Jun-2023 14:18:17.679 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Command line
argument: -Dignore.endorsed.dirs=
12-Jun-2023 14:18:17.679 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Command line
argument: -Dcatalina.base=/Users/manak/apache-tomcat-8.5.89
12-Jun-2023 14:18:17.679 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Command line
argument: -Dcatalina.home=/Users/manak/apache-tomcat-8.5.89
12-Jun-2023 14:18:17.680 INFO [main]
org.apache.catalina.startup.VersionLoggerListener.log Command line
argument: -Djava.io.tmpdir=/Users/manak/apache-tomcat-8.5.89/temp
12-Jun-2023 14:18:17.680 INFO [main]
org.apache.catalina.core.AprLifecycleListener.lifecycleEvent The Apache
Tomcat Native library which allows using OpenSSL was not found on the
java.library.path:
[/Users/manak/Library/Java/Extensions:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java:.]
12-Jun-2023 14:18:17.719 INFO [main]
org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler
["http-nio-8090"]
12-Jun-2023 14:18:17.739 INFO [main]
org.apache.catalina.startup.Catalina.load Initialization processed in 536 ms
12-Jun-2023 14:18:17.844 INFO [main]
org.apache.catalina.core.StandardService.startInternal Starting service
[Catalina]
12-Jun-2023 14:18:17.845 INFO [main]
org.apache.catalina.core.StandardEngine.startInternal Starting Servlet
engine: [Apache Tomcat/8.5.89]
12-Jun-2023 14:18:17.850 INFO [main]
org.apache.catalina.ha.tcp.SimpleTcpCluster.startInternal Cluster is about
to start
12-Jun-2023 14:18:17.860 INFO [main]
org.apache.catalina.tribes.transport.ReceiverBase.bind Receiver Server
Socket bound to:[localhost/127.0.0.1:8090]
12-Jun-2023 14:18:17.873 INFO [Thread-2]
org.apache.catalina.ha.tcp.SimpleTcpCluster.memberAdded Replication member
added:[org.apache.catalina.tribes.membership.StaticMember[tcp://localhost:8190,localhost,8190,
alive=0, securePort=-1, UDP Port=-1, id={0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 },
payload={}, command={}, domain={}]]
12-Jun-2023 14:18:17.886 INFO [Tribes-Task-Receiver[Catalina-Channel]-1]
org.apache.catalina.tribes.io.BufferPool.getBufferPool Created a buffer
pool with max size:[104857600] bytes of type:
[org.apache.catalina.tribes.io.BufferPool15Impl]
12-Jun-2023 14:18:17.888 INFO [localhost-startStop-1]
org.apache.catalina.startup.HostConfig.deployDirectory Deploying web
application directory [/Users/manak/apache-tomcat-8.5.89/webapps/docs]
12-Jun-2023 14:18:18.081 INFO [localhost-startStop-1]
org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web
application directory [/Users/manak/apache-tomcat-8.5.89/webapps/docs] has
finished in [193] ms
12-Jun-2023 14:18:18.081 INFO [localhost-startStop-1]
org.apache.catalina.startup.HostConfig.deployDirectory Deploying web
application directory [/Users/manak/apache-tomcat-8.5.89/webapps/manager]
12-Jun-2023 14:18:18.112 INFO [localhost-startStop-1]
org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web
application directory [/Users/manak/apache-tomcat-8.5.89/webapps/manager]
has finished in [31] ms
12-Jun-2023 14:18:18.112 INFO [localhost-startStop-1]
org.apache.catalina.startup.HostConfig.deployDirectory Deploying web
application directory [/Users/manak/apache-tomcat-8.5.89/webapps/examples]
12-Jun-2023 14:18:18.370 INFO [localhost-startStop-1]
org.apache.catalina.ha.session.DeltaManager.startInternal Register manager
[localhost#/examples] to cluster element [Engine] with name [Catalina]
12-Jun-2023 14:18:18.370 INFO [localhost-startStop-1]
org.apache.catalina.ha.session.DeltaManager.startInternal Starting
clustering manager at [localhost#/examples]
12-Jun-2023 14:18:18.399 INFO [localhost-startStop-1]
org.apache.catalina.ha.session.DeltaManager.getAllClusterSessions Manager
[localhost#/examples], requesting session state from
[org.apache.catalina.tribes.membership.StaticMember[tcp://localhost:8190,localhost,8190,
alive=0, securePort=-1, UDP Port=-1, id={0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 },
payload={}, command={}, domain={}]]. This operation will timeout if no
session state has been received within [60] seconds.
12-Jun-2023 14:18:18.504 INFO [localhost-startStop-1]
org.apache.catalina.ha.session.DeltaManager.waitForSendAllSessions Manager
[localhost#/examples]; session state sent at [6/12/23 2:18 PM] received in
[132] ms.
12-Jun-2023 14:18:18.509 INFO [localhost-startStop-1]
org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web
application directory [/Users/manak/apache-tomcat-8.5.89/webapps/examples]
has finished in [397] ms
12-Jun-2023 14:18:18.509 INFO [localhost-startStop-1]
org.apache.catalina.startup.HostConfig.deployDirectory Deploying web
application directory [/Users/manak/apache-tomcat-8.5.89/webapps/ROOT]
12-Jun-2023 14:18:18.528 INFO [localhost-startStop-1]
org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web
application directory [/Users/manak/apache-tomcat-8.5.89/webapps/ROOT] has
finished in [19] ms
12-Jun-2023 14:18:18.528 INFO [localhost-startStop-1]
org.apache.catalina.startup.HostConfig.deployDirectory Deploying web
application directory
[/Users/manak/apache-tomcat-8.5.89/webapps/host-manager]
12-Jun-2023 14:18:18.548 INFO [localhost-startStop-1]
org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web
application directory
[/Users/manak/apache-tomcat-8.5.89/webapps/host-manager] has finished in
[20] ms
12-Jun-2023 14:18:18.549 INFO [main]
org.apache.catalina.ha.session.JvmRouteBinderValve.startInternal
JvmRouteBinderValve started
12-Jun-2023 14:18:18.551 INFO [main]
org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler
["http-nio-8090"]
12-Jun-2023 14:18:18.561 INFO [main]
org.apache.catalina.startup.Catalina.start Server startup in 822 ms
*12-Jun-2023 14:18:26.507 SEVERE [Tribes-Task-Receiver[Catalina-Channel]-3]
org.apache.catalina.tribes.io.XByteBuffer.append Discarded the package,
invalid header*
```

I tried searching through the mail archive, stackoverflow, and other
sources. However, I haven't found the reason for this error. I would really
appreciate your further help.

Sincerely
Manak

Re: Error "Unable to send message through cluster sender" in a cluster with static members using delta manager

Posted by Mark Thomas <ma...@apache.org>.
On 10/06/2023 18:11, Manak Bisht wrote:
> I am trying to run the carts example (*/examples/jsp/sessions/carts.jsp*)
> with two nodes, on ports 8090 and 8190. The nodes are working fine
> individually. However, I am trying to create a cluster with static members
> (without multicast), but I keep encountering the following error -

<snip/>

>                      <Receiver
> className="org.apache.catalina.tribes.transport.nio.NioReceiver"/>

You are listening on the default port of 4000 ...

>                              <Member
> className="org.apache.catalina.tribes.membership.StaticMember"
>                                  port="8090"
>                                  securePort="-1"
>                                  host="localhost"
>                                  domain="delta-static"
> 
> uniqueId="{1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,0}"
>                                  />

... but trying to send to port 8009.

The send/receive ports for the cluster need to match.

Since you are running on a single machine I would explicitly configure 
the ports as:
node1 - 4001
node2 - 4002

and so on.

Keep in mind you will also need unique HTTP, HTTPS, AJP and shutdown 
ports for each node (as required depending on what is enabled).

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org