You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Mark Osborne <Ma...@ni.com> on 2008/02/08 16:28:53 UTC
Cluster Membership not picking up ttl setting
Hello All,
I actually sent this yesterday, but never saw it show up in the archive or
in my inbox so trying again. Sorry if y'all are getting this twice.
I'm having a problem setting up clustering in Tomcat 6.0 on RedHat ES 4.
I'm hoping someone can help me with this. I'm using a very vanilla
multicast clustering implementation. The problem is that the 2 machines I
want to cluster are on different subnets with 4 hops between them. Because
of this I need to up the Membership multicast ttl. Unfortunately for some
reason it doesn't appear that setting the ttl is having any effect for me.
I have verified that I can change the multicast address, port, and
frequency and those settings seem to work. It is only the ttl that
doesn't seem to change. (I've reverted all of those changes back to
default now).
Here is my clustering set up from server.xml
<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
channelSendOptions="8">
<Manager className="org.apache.catalina.ha.session.DeltaManager"
expireSessionsOnShutdown="false"
notifyListenersOnReplication="true"/>
<Channel
className="org.apache.catalina.tribes.group.GroupChannel">
<Membership
className="org.apache.catalina.tribes.membership.McastService"
address="228.0.0.4"
port="45564"
ttl="15"
frequency="500"
dropTime="3000"/>
<Sender
className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
<Transport
className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
</Sender>
<Receiver
className="org.apache.catalina.tribes.transport.nio.NioReceiver"
address="auto"
port="4000"
autoBind="100"
selectorTimeout="5000"
maxThreads="6"/>
<Interceptor
className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
<Interceptor
className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>
</Channel>
<Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
filter=""/>
<Valve
className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>
<ClusterListener
className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener"/>
<ClusterListener
className="org.apache.catalina.ha.session.ClusterSessionListener"/>
</Cluster>
As you can see I've changed the Membership ttl to 15. However, when I
start up apache and run a tcpdump it appears that the TTL is still set to
1. Our network admins have also verified from their end that it appears
the ttl on the multicast traffic is 1.
[ root on godofwar ] tcpdump host 228.0.0.4 -v
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 96
bytes
17:32:45.852847 IP (tos 0xc0, ttl 1, id 63038, offset 0, flags [none],
proto 2, length: 28) mp3-1n14-c1-e-gw-vlan-1329.natinst.com >
reserved-multicast-range-not-delegated.example.com: igmp query v2 [max
resp time 10] [gaddr reserved-multicast-range-not-delegated.example.com]
17:32:50.560751 IP (tos 0xc0, ttl 1, id 0, offset 0, flags [DF], proto
2, length: 32, optlength: 4 ( RA )) godofwar.natinst.com >
reserved-multicast-range-not-delegated.example.com: igmp v2 report
reserved-multicast-range-not-delegated.example.com
17:32:51.583260 IP (tos 0x0, ttl 1, id 0, offset 0, flags [DF], proto
17, length: 97) godofwar.natinst.com.45564 >
reserved-multicast-range-not-delegated.example.com.45564: UDP, length 69
17:32:51.585935 IP (tos 0x0, ttl 1, id 0, offset 0, flags [DF], proto
17, length: 97) godofwar.natinst.com.45564 >
reserved-multicast-range-not-delegated.example.com.45564: UDP, length 69
17:32:52.087227 IP (tos 0x0, ttl 1, id 0, offset 0, flags [DF], proto
17, length: 97) godofwar.natinst.com.45564 >
reserved-multicast-range-not-delegated.example.com.45564: UDP, length 69
17:32:52.589311 IP (tos 0x0, ttl 1, id 0, offset 0, flags [DF], proto
17, length: 97) godofwar.natinst.com.45564 >
reserved-multicast-range-not-delegated.example.com.45564: UDP, length 69
17:32:53.091522 IP (tos 0x0, ttl 1, id 0, offset 0, flags [DF], proto
17, length: 97) godofwar.natinst.com.45564 >
reserved-multicast-range-not-delegated.example.com.45564: UDP, length 69
Does anybody have any ideas on why this is?
Thanks in advance
~^~^~^~^~^~^~^~^~^~^~^~^~^~^~^~^~^~^~^~
Mark Osborne
Web Systems Engineer
mark.osborne@ni.com
(512) 683-5019
~^~^~^~^~^~^~^~^~^~^~^~^~^~^~^~^~^~^~^~
Re: Cluster Membership not picking up ttl setting
Posted by Mark Osborne <Ma...@ni.com>.
Filip . . . you are most definitely the man. That solved my problem. I
Added -Djava.net.preferIPv4Stack=true to the start options in catalina.sh
and now my servers are communicating beautifully. Thanks a million.
~^~^~^~^~^~^~^~^~^~^~^~^~^~^~^~^~^~^~^~
Mark Osborne
Web Systems Engineer
mark.osborne@ni.com
(512) 683-5019
~^~^~^~^~^~^~^~^~^~^~^~^~^~^~^~^~^~^~^~
Filip Hanik - Dev Lists <de...@hanik.com>
02/08/2008 11:47 AM
Please respond to
"Tomcat Users List" <us...@tomcat.apache.org>
To
Tomcat Users List <us...@tomcat.apache.org>
cc
Subject
Re: Cluster Membership not picking up ttl setting
running your config, the print out is
Feb 8, 2008 10:13:51 AM
org.apache.catalina.tribes.membership.McastServiceImpl setupSocket
INFO: Setting cluster mcast TTL to 15
and that executes the code
if ( mcastTTL >= 0 ) {
if(log.isInfoEnabled())
log.info("Setting cluster mcast TTL to " + mcastTTL);
socket.setTimeToLive(mcastTTL);
}
which is this method
http://java.sun.com/j2se/1.5.0/docs/api/java/net/MulticastSocket.html#setTimeToLive(int)
then I capture this in wireshark (two systems running
WINDOWS
Frame 1 (111 bytes on wire, 111 bytes captured)
Ethernet II, Src: Dell_08:6c:61 (00:15:c5:08:6c:61), Dst:
01:00:5e:00:00:04 (01:00:5e:00:00:04)
Internet Protocol, Src: 192.168.3.102 (192.168.3.102), Dst: 228.0.0.4
(228.0.0.4)
Version: 4
Header length: 20 bytes
Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00)
Total Length: 97
Identification: 0x6662 (26210)
Flags: 0x00
Fragment offset: 0
Time to live: 15
Protocol: UDP (0x11)
Header checksum: 0x9d17 [correct]
Source: 192.168.3.102 (192.168.3.102)
Destination: 228.0.0.4 (228.0.0.4)
all good, TTL is 15
LINUX-FEDORA 6
No. Time Source Destination Protocol
Info
2 0.229936 192.168.3.111 228.0.0.4 UDP
Source port: 45564 Destination port: 45564
Frame 2 (111 bytes on wire, 111 bytes captured)
Ethernet II, Src: IntelCor_73:41:5a (00:13:20:73:41:5a), Dst:
01:00:5e:00:00:04 (01:00:5e:00:00:04)
Internet Protocol, Src: 192.168.3.111 (192.168.3.111), Dst: 228.0.0.4
(228.0.0.4)
Version: 4
Header length: 20 bytes
Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00)
Total Length: 97
Identification: 0x0000 (0)
Flags: 0x04 (Don't Fragment)
Fragment offset: 0
Time to live: 1
Protocol: UDP (0x11)
Header checksum: 0xd170 [correct]
Source: 192.168.3.111 (192.168.3.111)
Destination: 228.0.0.4 (228.0.0.4)
not good at all, TTL is 1 here, even though the Java code set it to 15
so I googled a little bit, the bug is either in the JVM or on linux,
most likely linux.
to solve this problem add
-Djava.net.preferIPv4Stack=true
to your startup script. and when I did that I got
Frame 1 (111 bytes on wire, 111 bytes captured)
Ethernet II, Src: IntelCor_73:41:5a (00:13:20:73:41:5a), Dst:
01:00:5e:00:00:04 (01:00:5e:00:00:04)
Internet Protocol, Src: 192.168.3.111 (192.168.3.111), Dst: 228.0.0.4
(228.0.0.4)
Version: 4
Header length: 20 bytes
Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00)
Total Length: 97
Identification: 0x0000 (0)
Flags: 0x04 (Don't Fragment)
Fragment offset: 0
Time to live: 15
Protocol: UDP (0x11)
Header checksum: 0xc370 [correct]
Source: 192.168.3.111 (192.168.3.111)
Destination: 228.0.0.4 (228.0.0.4)
and all is good
Filip
Mark Osborne wrote:
> Hello All,
>
> I actually sent this yesterday, but never saw it show up in the archive
or
> in my inbox so trying again. Sorry if y'all are getting this twice.
>
> I'm having a problem setting up clustering in Tomcat 6.0 on RedHat ES 4.
> I'm hoping someone can help me with this. I'm using a very vanilla
> multicast clustering implementation. The problem is that the 2 machines
I
> want to cluster are on different subnets with 4 hops between them.
Because
> of this I need to up the Membership multicast ttl. Unfortunately for
some
> reason it doesn't appear that setting the ttl is having any effect for
me.
> I have verified that I can change the multicast address, port, and
> frequency and those settings seem to work. It is only the ttl that
> doesn't seem to change. (I've reverted all of those changes back to
> default now).
>
> Here is my clustering set up from server.xml
>
> <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
> channelSendOptions="8">
>
> <Manager className="org.apache.catalina.ha.session.DeltaManager"
> expireSessionsOnShutdown="false"
> notifyListenersOnReplication="true"/>
>
> <Channel
> className="org.apache.catalina.tribes.group.GroupChannel">
> <Membership
> className="org.apache.catalina.tribes.membership.McastService"
> address="228.0.0.4"
> port="45564"
> ttl="15"
> frequency="500"
> dropTime="3000"/>
> <Sender
> className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
> <Transport
>
className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
> </Sender>
> <Receiver
> className="org.apache.catalina.tribes.transport.nio.NioReceiver"
> address="auto"
> port="4000"
> autoBind="100"
> selectorTimeout="5000"
> maxThreads="6"/>
>
> <Interceptor
>
className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
> <Interceptor
>
className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>
> </Channel>
>
> <Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
> filter=""/>
> <Valve
> className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>
>
> <ClusterListener
>
className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener"/>
> <ClusterListener
> className="org.apache.catalina.ha.session.ClusterSessionListener"/>
> </Cluster>
>
> As you can see I've changed the Membership ttl to 15. However, when I
> start up apache and run a tcpdump it appears that the TTL is still set
to
> 1. Our network admins have also verified from their end that it appears
> the ttl on the multicast traffic is 1.
>
> [ root on godofwar ] tcpdump host 228.0.0.4 -v
> tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 96
> bytes
> 17:32:45.852847 IP (tos 0xc0, ttl 1, id 63038, offset 0, flags [none],
> proto 2, length: 28) mp3-1n14-c1-e-gw-vlan-1329.natinst.com >
> reserved-multicast-range-not-delegated.example.com: igmp query v2 [max
> resp time 10] [gaddr reserved-multicast-range-not-delegated.example.com]
> 17:32:50.560751 IP (tos 0xc0, ttl 1, id 0, offset 0, flags [DF], proto
> 2, length: 32, optlength: 4 ( RA )) godofwar.natinst.com >
> reserved-multicast-range-not-delegated.example.com: igmp v2 report
> reserved-multicast-range-not-delegated.example.com
> 17:32:51.583260 IP (tos 0x0, ttl 1, id 0, offset 0, flags [DF], proto
> 17, length: 97) godofwar.natinst.com.45564 >
> reserved-multicast-range-not-delegated.example.com.45564: UDP, length 69
> 17:32:51.585935 IP (tos 0x0, ttl 1, id 0, offset 0, flags [DF], proto
> 17, length: 97) godofwar.natinst.com.45564 >
> reserved-multicast-range-not-delegated.example.com.45564: UDP, length 69
> 17:32:52.087227 IP (tos 0x0, ttl 1, id 0, offset 0, flags [DF], proto
> 17, length: 97) godofwar.natinst.com.45564 >
> reserved-multicast-range-not-delegated.example.com.45564: UDP, length 69
> 17:32:52.589311 IP (tos 0x0, ttl 1, id 0, offset 0, flags [DF], proto
> 17, length: 97) godofwar.natinst.com.45564 >
> reserved-multicast-range-not-delegated.example.com.45564: UDP, length 69
> 17:32:53.091522 IP (tos 0x0, ttl 1, id 0, offset 0, flags [DF], proto
> 17, length: 97) godofwar.natinst.com.45564 >
> reserved-multicast-range-not-delegated.example.com.45564: UDP, length 69
>
> Does anybody have any ideas on why this is?
>
> Thanks in advance
>
> ~^~^~^~^~^~^~^~^~^~^~^~^~^~^~^~^~^~^~^~
> Mark Osborne
> Web Systems Engineer
> mark.osborne@ni.com
> (512) 683-5019
> ~^~^~^~^~^~^~^~^~^~^~^~^~^~^~^~^~^~^~^~
>
> ------------------------------------------------------------------------
>
> No virus found in this incoming message.
> Checked by AVG Free Edition.
> Version: 7.5.516 / Virus Database: 269.19.21/1263 - Release Date:
2/6/2008 8:14 PM
>
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Cluster Membership not picking up ttl setting
Posted by Filip Hanik - Dev Lists <de...@hanik.com>.
running your config, the print out is
Feb 8, 2008 10:13:51 AM
org.apache.catalina.tribes.membership.McastServiceImpl setupSocket
INFO: Setting cluster mcast TTL to 15
and that executes the code
if ( mcastTTL >= 0 ) {
if(log.isInfoEnabled())
log.info("Setting cluster mcast TTL to " + mcastTTL);
socket.setTimeToLive(mcastTTL);
}
which is this method
http://java.sun.com/j2se/1.5.0/docs/api/java/net/MulticastSocket.html#setTimeToLive(int)
then I capture this in wireshark (two systems running
WINDOWS
Frame 1 (111 bytes on wire, 111 bytes captured)
Ethernet II, Src: Dell_08:6c:61 (00:15:c5:08:6c:61), Dst:
01:00:5e:00:00:04 (01:00:5e:00:00:04)
Internet Protocol, Src: 192.168.3.102 (192.168.3.102), Dst: 228.0.0.4
(228.0.0.4)
Version: 4
Header length: 20 bytes
Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00)
Total Length: 97
Identification: 0x6662 (26210)
Flags: 0x00
Fragment offset: 0
Time to live: 15
Protocol: UDP (0x11)
Header checksum: 0x9d17 [correct]
Source: 192.168.3.102 (192.168.3.102)
Destination: 228.0.0.4 (228.0.0.4)
all good, TTL is 15
LINUX-FEDORA 6
No. Time Source Destination Protocol
Info
2 0.229936 192.168.3.111 228.0.0.4 UDP
Source port: 45564 Destination port: 45564
Frame 2 (111 bytes on wire, 111 bytes captured)
Ethernet II, Src: IntelCor_73:41:5a (00:13:20:73:41:5a), Dst:
01:00:5e:00:00:04 (01:00:5e:00:00:04)
Internet Protocol, Src: 192.168.3.111 (192.168.3.111), Dst: 228.0.0.4
(228.0.0.4)
Version: 4
Header length: 20 bytes
Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00)
Total Length: 97
Identification: 0x0000 (0)
Flags: 0x04 (Don't Fragment)
Fragment offset: 0
Time to live: 1
Protocol: UDP (0x11)
Header checksum: 0xd170 [correct]
Source: 192.168.3.111 (192.168.3.111)
Destination: 228.0.0.4 (228.0.0.4)
not good at all, TTL is 1 here, even though the Java code set it to 15
so I googled a little bit, the bug is either in the JVM or on linux,
most likely linux.
to solve this problem add
-Djava.net.preferIPv4Stack=true
to your startup script. and when I did that I got
Frame 1 (111 bytes on wire, 111 bytes captured)
Ethernet II, Src: IntelCor_73:41:5a (00:13:20:73:41:5a), Dst:
01:00:5e:00:00:04 (01:00:5e:00:00:04)
Internet Protocol, Src: 192.168.3.111 (192.168.3.111), Dst: 228.0.0.4
(228.0.0.4)
Version: 4
Header length: 20 bytes
Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00)
Total Length: 97
Identification: 0x0000 (0)
Flags: 0x04 (Don't Fragment)
Fragment offset: 0
Time to live: 15
Protocol: UDP (0x11)
Header checksum: 0xc370 [correct]
Source: 192.168.3.111 (192.168.3.111)
Destination: 228.0.0.4 (228.0.0.4)
and all is good
Filip
Mark Osborne wrote:
> Hello All,
>
> I actually sent this yesterday, but never saw it show up in the archive or
> in my inbox so trying again. Sorry if y'all are getting this twice.
>
> I'm having a problem setting up clustering in Tomcat 6.0 on RedHat ES 4.
> I'm hoping someone can help me with this. I'm using a very vanilla
> multicast clustering implementation. The problem is that the 2 machines I
> want to cluster are on different subnets with 4 hops between them. Because
> of this I need to up the Membership multicast ttl. Unfortunately for some
> reason it doesn't appear that setting the ttl is having any effect for me.
> I have verified that I can change the multicast address, port, and
> frequency and those settings seem to work. It is only the ttl that
> doesn't seem to change. (I've reverted all of those changes back to
> default now).
>
> Here is my clustering set up from server.xml
>
> <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
> channelSendOptions="8">
>
> <Manager className="org.apache.catalina.ha.session.DeltaManager"
> expireSessionsOnShutdown="false"
> notifyListenersOnReplication="true"/>
>
> <Channel
> className="org.apache.catalina.tribes.group.GroupChannel">
> <Membership
> className="org.apache.catalina.tribes.membership.McastService"
> address="228.0.0.4"
> port="45564"
> ttl="15"
> frequency="500"
> dropTime="3000"/>
> <Sender
> className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
> <Transport
> className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
> </Sender>
> <Receiver
> className="org.apache.catalina.tribes.transport.nio.NioReceiver"
> address="auto"
> port="4000"
> autoBind="100"
> selectorTimeout="5000"
> maxThreads="6"/>
>
> <Interceptor
> className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
> <Interceptor
> className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>
> </Channel>
>
> <Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
> filter=""/>
> <Valve
> className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>
>
> <ClusterListener
> className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener"/>
> <ClusterListener
> className="org.apache.catalina.ha.session.ClusterSessionListener"/>
> </Cluster>
>
> As you can see I've changed the Membership ttl to 15. However, when I
> start up apache and run a tcpdump it appears that the TTL is still set to
> 1. Our network admins have also verified from their end that it appears
> the ttl on the multicast traffic is 1.
>
> [ root on godofwar ] tcpdump host 228.0.0.4 -v
> tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 96
> bytes
> 17:32:45.852847 IP (tos 0xc0, ttl 1, id 63038, offset 0, flags [none],
> proto 2, length: 28) mp3-1n14-c1-e-gw-vlan-1329.natinst.com >
> reserved-multicast-range-not-delegated.example.com: igmp query v2 [max
> resp time 10] [gaddr reserved-multicast-range-not-delegated.example.com]
> 17:32:50.560751 IP (tos 0xc0, ttl 1, id 0, offset 0, flags [DF], proto
> 2, length: 32, optlength: 4 ( RA )) godofwar.natinst.com >
> reserved-multicast-range-not-delegated.example.com: igmp v2 report
> reserved-multicast-range-not-delegated.example.com
> 17:32:51.583260 IP (tos 0x0, ttl 1, id 0, offset 0, flags [DF], proto
> 17, length: 97) godofwar.natinst.com.45564 >
> reserved-multicast-range-not-delegated.example.com.45564: UDP, length 69
> 17:32:51.585935 IP (tos 0x0, ttl 1, id 0, offset 0, flags [DF], proto
> 17, length: 97) godofwar.natinst.com.45564 >
> reserved-multicast-range-not-delegated.example.com.45564: UDP, length 69
> 17:32:52.087227 IP (tos 0x0, ttl 1, id 0, offset 0, flags [DF], proto
> 17, length: 97) godofwar.natinst.com.45564 >
> reserved-multicast-range-not-delegated.example.com.45564: UDP, length 69
> 17:32:52.589311 IP (tos 0x0, ttl 1, id 0, offset 0, flags [DF], proto
> 17, length: 97) godofwar.natinst.com.45564 >
> reserved-multicast-range-not-delegated.example.com.45564: UDP, length 69
> 17:32:53.091522 IP (tos 0x0, ttl 1, id 0, offset 0, flags [DF], proto
> 17, length: 97) godofwar.natinst.com.45564 >
> reserved-multicast-range-not-delegated.example.com.45564: UDP, length 69
>
> Does anybody have any ideas on why this is?
>
> Thanks in advance
>
> ~^~^~^~^~^~^~^~^~^~^~^~^~^~^~^~^~^~^~^~
> Mark Osborne
> Web Systems Engineer
> mark.osborne@ni.com
> (512) 683-5019
> ~^~^~^~^~^~^~^~^~^~^~^~^~^~^~^~^~^~^~^~
>
> ------------------------------------------------------------------------
>
> No virus found in this incoming message.
> Checked by AVG Free Edition.
> Version: 7.5.516 / Virus Database: 269.19.21/1263 - Release Date: 2/6/2008 8:14 PM
>
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org