You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by cnnfntop <wo...@yahoo.com> on 2010/08/06 18:25:49 UTC
2 separate clusters, crosstalk?
Hi,
We have a stage cluster and a prod cluster. Yet if the stage cluster is UP, and
we start the prod cluster, the logs report prod members and stage members seeing
one another as part of the same tribes group! How do we separate this?
We already have them listening on separate mcastAddresses... prod id 228.0.0.9
and stage is 228.0.0.10... really at a loss here. Help!
<Engine....
....
<Cluster channelSendOptions="8"
className="org.apache.catalina.ha.tcp.SimpleTcpCluster">
<Membership
className="org.apache.catalina.cluster.mcast.McastService"
mcastAddr="228.0.0.9"
mcastPort="45564"
mcastFrequency="500"
mcastDropTime="3000"/>
<Manager
className="org.apache.catalina.ha.session.DeltaManager"
expireSessionsOnShutdown="false" domainReplication="true"
notifyListenersOnReplication="true" />
<Channel className="org.apache.catalina.tribes.group.GroupChannel">
<!-- this is what THIS node listens on in the cluster
-->
<Receiver address="10.xx.xx.me"
className="org.apache.catalina.tribes.transport.nio.NioReceiver"
maxThreads="6" port="4000" selectorTimeout="5000" />
<Sender
className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
<Transport
className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"
/> </Sender>
<Interceptor
className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"
/>
<Interceptor
className="org.apache.catalina.tribes.group.interceptors.StaticMembershipInterceptor">
<Member
className="org.apache.catalina.tribes.membership.StaticMember"
port="4000" securePort="-1"
host="10.xx.xx.othernode" domain="PROD-domain"
uniqueId="{0,0,0,0,0,0,0,0,0,191}" />
</Interceptor>
<Interceptor
className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"
/>
</Channel>
<Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
filter="" />
<Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"
/>
<ClusterListener
className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener" />
</Cluster>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: 2 separate clusters, crosstalk?
Posted by Felix Schumacher <fe...@internetallee.de>.
Hi Chris,
On Tue, 10 Aug 2010 17:05:11 -0400, Christopher Schultz
<ch...@christopherschultz.net> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>
> On 8/6/2010 12:35 PM, cnnfntop wrote:
>> Below are the configs, one from one of the prod servers, the other from
>> one of
>> the stage servers (note that the member/receiver settings are correct
>> for each
>> pair in each cluster, they reference each other correctly)
>
>> PROD SETUP (stage below)
>> -------------------
>>
>> <Membership className="org.apache.catalina.cluster.mcast.McastService"
>> mcastAddr="228.0.0.9"
>> mcastPort="45564"
>>
>> ...
>>
>> <Receiver address="10.xx.xx.me"
>>
>> ...
>> <Member
>> className="org.apache.catalina.tribes.membership.StaticMember"
>> port="4000" securePort="-1"
>> host="10.xx.xx.othernode" domain="PROD-domain"
>> uniqueId="{0,0,0,0,0,0,0,0,0,191}" />
>
> ...
>
>> [STAGING] SETUP
>> ------------------------
>> <Membership className="org.apache.catalina.cluster.mcast.McastService"
>> mcastAddr="228.0.0.10"
>> mcastPort="45564"
>>
>> ...
>>
>> <Receiver address="10.xx.xx.211"
...
>>
>> Aug 6, 2010 4:02:26 PM org.apache.catalina.ha.tcp.SimpleTcpCluster
>> memberAdded
>> INFO: Replication member
>> added:org.apache.catalina.tribes.membership.MemberImpl[tcp://{10, 6,
>> -94,
>> -45}:4000,{10, 6, -94, -45},4000, alive=1018,id={-98 14 -16 65 -45 -70
>> 77 -112
>> -91 -67 124 -106 -44 99 -46 122 }, payload={}, command={}, domain={}, ]
>
> I'm not sure what the above is supposed to prove... neither the IP
> addresses nor the "id" from that log message matches what you have in
> your staging configuration. How do you know there's crosstalk?
I think the ip addresses are printed as bytes and that means in java
(int) 211 == (byte)-45 :)
So at least one of the members is found and the hidden parts revealed.
Bye
Felix
>
> - -chris
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (MingW32)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAkxhvwcACgkQ9CaO5/Lv0PASvwCgpMvOWcNs8P0xskho//QJQYY/
> i8kAoJwRVq2NPAlwzjEQwB4LWYuWejyH
> =R8wq
> -----END PGP SIGNATURE-----
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: 2 separate clusters, crosstalk?
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 8/6/2010 12:35 PM, cnnfntop wrote:
> Below are the configs, one from one of the prod servers, the other from one of
> the stage servers (note that the member/receiver settings are correct for each
> pair in each cluster, they reference each other correctly)
> PROD SETUP (stage below)
> -------------------
>
> <Membership className="org.apache.catalina.cluster.mcast.McastService"
> mcastAddr="228.0.0.9"
> mcastPort="45564"
>
> ...
>
> <Receiver address="10.xx.xx.me"
>
> ...
> <Member
> className="org.apache.catalina.tribes.membership.StaticMember"
> port="4000" securePort="-1"
> host="10.xx.xx.othernode" domain="PROD-domain"
> uniqueId="{0,0,0,0,0,0,0,0,0,191}" />
...
> [STAGING] SETUP
> ------------------------
> <Membership className="org.apache.catalina.cluster.mcast.McastService"
> mcastAddr="228.0.0.10"
> mcastPort="45564"
>
> ...
>
> <Receiver address="10.xx.xx.211"
>
> ...
>
> <Member
> className="org.apache.catalina.tribes.membership.StaticMember"
> port="4000" securePort="-1"
> host="10.xx.xx.212" domain="lfstg-domain"
> uniqueId="{0,0,0,0,0,0,0,0,0,99}" />
- From a quick glance at these settings, they all seem reasonable. I'm not
expert on Tomcat clustering, though. If you don't hear anything for a
day or two, try bumping the topic to see if someone else can comment.
You don't have any weird multicast configuration on the NIC do you? Are
you sure that all your network infrastructure is capable of doing
multicast in the way you expect? It's possible that you have a stupid
router or switch that just broadcasts multicast messages instead of
properly sending them.
> Prod/stage report this in the their logs as members come up (across both
> prod/stage, hence evidence of crosstalk)
>
> Aug 6, 2010 4:02:26 PM org.apache.catalina.ha.tcp.SimpleTcpCluster memberAdded
> INFO: Replication member
> added:org.apache.catalina.tribes.membership.MemberImpl[tcp://{10, 6, -94,
> -45}:4000,{10, 6, -94, -45},4000, alive=1018,id={-98 14 -16 65 -45 -70 77 -112
> -91 -67 124 -106 -44 99 -46 122 }, payload={}, command={}, domain={}, ]
I'm not sure what the above is supposed to prove... neither the IP
addresses nor the "id" from that log message matches what you have in
your staging configuration. How do you know there's crosstalk?
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAkxhvwcACgkQ9CaO5/Lv0PASvwCgpMvOWcNs8P0xskho//QJQYY/
i8kAoJwRVq2NPAlwzjEQwB4LWYuWejyH
=R8wq
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: 2 separate clusters, crosstalk?
Posted by cnnfntop <wo...@yahoo.com>.
Thanks Chris,
Below are the configs, one from one of the prod servers, the other from one of
the stage servers (note that the member/receiver settings are correct for each
pair in each cluster, they reference eachother correctly)
Prod/stage report this in the their logs as members come up (across both
prod/stage, hence evidence of crosstalk)
Aug 6, 2010 4:02:26 PM org.apache.catalina.ha.tcp.SimpleTcpCluster memberAdded
INFO: Replication member
added:org.apache.catalina.tribes.membership.MemberImpl[tcp://{10, 6, -94,
-45}:4000,{10, 6, -94, -45},4000, alive=1018,id={-98 14 -16 65 -45 -70 77 -112
-91 -67 124 -106 -44 99 -46 122 }, payload={}, command={}, domain={}, ]
PROD SETUP (stage below)
-------------------
<Engine....
....
<Cluster channelSendOptions="8"
className="org.apache.catalina.ha.tcp.SimpleTcpCluster">
<Membership
className="org.apache.catalina.cluster.mcast.McastService"
mcastAddr="228.0.0.9"
mcastPort="45564"
mcastFrequency="500"
mcastDropTime="3000"/>
<Manager
className="org.apache.catalina.ha.session.DeltaManager"
expireSessionsOnShutdown="false" domainReplication="true"
notifyListenersOnReplication="true" />
<Channel className="org.apache.catalina.tribes.group.GroupChannel">
<!-- this is what THIS node listens on in the cluster
-->
<Receiver address="10.xx.xx.me"
className="org.apache.catalina.tribes.transport.nio.NioReceiver"
maxThreads="6" port="4000" selectorTimeout="5000" />
<Sender
className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
<Transport
className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"
/> </Sender>
<Interceptor
className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"
/>
<Interceptor
className="org.apache.catalina.tribes.group.interceptors.StaticMembershipInterceptor">
<Member
className="org.apache.catalina.tribes.membership.StaticMember"
port="4000" securePort="-1"
host="10.xx.xx.othernode" domain="PROD-domain"
uniqueId="{0,0,0,0,0,0,0,0,0,191}" />
</Interceptor>
<Interceptor
className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"
/>
</Channel>
<Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
filter="" />
<Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"
/>
<ClusterListener
className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener" />
</Cluster>
STAGE SETUP
------------------------
<Cluster channelSendOptions="8"
className="org.apache.catalina.ha.tcp.SimpleTcpCluster">
<Membership
className="org.apache.catalina.cluster.mcast.McastService"
mcastAddr="228.0.0.10"
mcastPort="45564"
mcastFrequency="500"
mcastDropTime="3000"/>
<Manager
className="org.apache.catalina.ha.session.DeltaManager"
expireSessionsOnShutdown="false" domainReplication="true"
notifyListenersOnReplication="true" />
<Channel className="org.apache.catalina.tribes.group.GroupChannel">
<Receiver address="10.xx.xx.211"
className="org.apache.catalina.tribes.transport.nio.NioReceiver"
maxThreads="6" port="4000" selectorTimeout="5000" />
<Sender
className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
<Transport
className="org.apache.catalina.tribes.transport.nio.PooledParallelSender" />
</Sender>
<Interceptor
className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector" />
<Interceptor
className="org.apache.catalina.tribes.group.interceptors.StaticMembershipInterceptor">
<!-- FOR Each member peer host in the cluster, create
an entry like this -->
<Member
className="org.apache.catalina.tribes.membership.StaticMember"
port="4000" securePort="-1" host="10.xx.xx.212"
domain="lfstg-domain"
uniqueId="{0,0,0,0,0,0,0,0,0,99}" />
</Interceptor>
<Interceptor
className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"
/>
</Channel>
<Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
filter="" />
<Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"
/>
<ClusterListener
className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener" />
<ClusterListener
className="org.apache.catalina.ha.session.ClusterSessionListener" />
</Cluster>
----- Original Message ----
From: Christopher Schultz <ch...@christopherschultz.net>
To: Tomcat Users List <us...@tomcat.apache.org>
Sent: Fri, August 6, 2010 10:30:10 AM
Subject: Re: 2 separate clusters, crosstalk?
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 8/6/2010 12:25 PM, cnnfntop wrote:
> We have a stage cluster and a prod cluster. Yet if the stage cluster is UP, and
>
> we start the prod cluster, the logs report prod members and stage members
>seeing
>
> one another as part of the same tribes group! How do we separate this?
>
> We already have them listening on separate mcastAddresses... prod id 228.0.0.9
> and stage is 228.0.0.10... really at a loss here. Help!
Could you post /both/ configurations? Or, at least, one full (looks like
you did that) plus the differences between the first and the second?
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAkxcOJEACgkQ9CaO5/Lv0PD7GgCeKJYrfqyngL0wZeuUIO7e/TYp
ZewAni9KwiTGvSknmXI4MPAExuh1YVtN
=oy9o
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: 2 separate clusters, crosstalk?
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 8/6/2010 12:25 PM, cnnfntop wrote:
> We have a stage cluster and a prod cluster. Yet if the stage cluster is UP, and
> we start the prod cluster, the logs report prod members and stage members seeing
> one another as part of the same tribes group! How do we separate this?
>
> We already have them listening on separate mcastAddresses... prod id 228.0.0.9
> and stage is 228.0.0.10... really at a loss here. Help!
Could you post /both/ configurations? Or, at least, one full (looks like
you did that) plus the differences between the first and the second?
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAkxcOJEACgkQ9CaO5/Lv0PD7GgCeKJYrfqyngL0wZeuUIO7e/TYp
ZewAni9KwiTGvSknmXI4MPAExuh1YVtN
=oy9o
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org