You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Dave Ford <D....@bath.ac.uk> on 2017/10/30 10:19:22 UTC
Beginner help setting up test vertical cluster
Hello,
I should apologise in advance as I'm very new to Tomcat and, I'm sure,
will be making some daft mistakes and silly errors. I hope this
question and any that follow it aren't too dumb.
I've recently started a new job and have inherited a half-finished
tomcat cluster on solaris.
While I'm not going to immediately try and finish that cluster, I would
like to try setting up some test instances to ensure I know what to
expect - but I'm having trouble doing even this.
I've gotten my head round how to set up individual instances of Tomcat
8.5.23 running from different sub folders of a common CATALINA_HOME
folder, and now I'm trying to get the two instances to talk to each
other using StaticMember on specific ports rather than the automatic
multicasting.
But instances start up, and the logs at least suggest they're aware of
each other. But one server takes forever to start, and I'm not
convinced they're talking to each other properly. For example, in the
catalina.out logs I'm seeing messages such as:
30-Oct-2017 09:28:29.632 INFO [MyHost-startStop-1]
org.apache.catalina.ha.session.DeltaManager.getAllClusterSessions
Manager [/sample], requesting session state from
[org.apache.catalina.tribes.membership.StaticMember[tcp://my.ip.addr:40
01,my.ip.addr,4001, alive=0, securePort=-1, UDP Port=-1, id={1 2 3 4 5
6 7 8 9 10 11 12 13 14 15 0 }, payload={}, command={}, domain={99 108
117 115 116 101 114 116 101 ...(11)}, ]]. This operation will timeout
if no session state has been received within [60] seconds.
30-Oct-2017 09:28:34.111 WARNING [Tribes-Task-Receiver[MyHost-Channel]-
2]
org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor.m
essageReceived Received message from
cluster[org.apache.catalina.tribes.membership.MemberImpl[tcp://{my, ip,
nn, nn}:4001,{my, ip, nn, nn},4001, alive=1509355714104, securePort=-
1,
UDP Port=-1, id={1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 0 }, payload={},
command={}, domain={99 108 117 115 116 101 114 116 101 ...(11)}, ]] was
refused.
Which does not seem encouraging. Yet on the other instance, I'm
seeing:
30-Oct-2017 09:28:34.105 INFO [GroupChannel-Heartbeat[MyHost-Channel]-
1] org.apache.catalina.ha.tcp.SimpleTcpCluster.memberAdded Replication
member
added:[org.apache.catalina.tribes.membership.StaticMember[tcp://my.ip.a
ddr:4000,my.ip.addr,4000, alive=0, securePort=-1, UDP Port=-1, id={0 1
2 3 4 5 6 7 8 9 10 11 12 13 14 15 }, payload={}, command={}, domain={99
108 117 115 116 101 114 116 101 ...(11)}, ]]
30-Oct-2017 09:28:34.105 INFO [GroupChannel-Heartbeat[MyHost-Channel]-
1]
org.apache.catalina.tribes.group.interceptors.TcpFailureDetector.perfor
mBasicCheck Suspect member, confirmed
alive.[org.apache.catalina.tribes.membership.StaticMember[tcp://my.ip.a
ddr:4000,my.ip.addr,4000, alive=0, securePort=-1, UDP Port=-1, id={0 1
2 3 4 5 6 7 8 9 10 11 12 13 14 15 }, payload={}, command={}, domain={99
108 117 115 116 101 114 116 101 ...(11)}, ]]
Which seems better... But then I see back on instance one,
30-Oct-2017 09:32:30.178 INFO [main]
org.apache.catalina.startup.Catalina.start Server startup in 241109 ms
Which seems really bad.
I've installed the 'examples' app on both instances, have added
'<distributable/>' to both's web.xml file.
I can see on one instance a message such as
30-Oct-2017 09:35:22.600 INFO [http-nio-8081-exec-7]
org.apache.catalina.core.ApplicationContext.log SessionListener:
sessionCreated('C2C3720879E72D558EEF4B0655188EBD.beta')
30-Oct-2017 09:35:26.912 INFO [http-nio-8081-exec-8]
org.apache.catalina.core.ApplicationContext.log SessionListener:
attributeAdded('C2C3720879E72D558EEF4B0655188EBD.beta', 'foo', 'bar')
When I play with the sessions example code. But nothing on the other
instance that would suggest this is being replicated.
Before I make this post too long and post my server.xml code, could
someone advise me as to what I should be looking for, or what these
messages suggest? Any pointers at this stage would be grateful as I'm
unsure even what a working cluster should look like.
Thanks
Dave
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Beginner help setting up test vertical cluster
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Keiichi,
On 11/1/17 3:28 AM, Keiichi Fujino wrote:
> Hi Dave.
>
> Your Interceptor settings are as follows.
>
>
>> <Interceptor className="org.apache.catalina.tribes.group.
>> interceptors.TcpPingInterceptor"></Interceptor> <Interceptor
>> className="org.apache.catalina.tribes.group.
>> interceptors.TcpFailureDetector"></Interceptor> <Interceptor
>> className="org.apache.catalina.tribes.group.
>> interceptors.ThroughputInterceptor" /> <Interceptor
>> className="org.apache.catalina.tribes.group.
>> interceptors.StaticMembershipInterceptor"> <LocalMember
>> className="org.apache. catalina.tribes.membership.StaticMember"
>> domain="clustertest"
>> uniqueId="{0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}"></LocalMember>
>>
>>
<Member className="org.apache.catalina.tribes.membership.StaticMember"
>> domain="clustertest" host="xxx.xxx.xxx.xxx" port="4001"
>> securePort="-1"
>> uniqueId="{1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,0}"></Member>
>> </Interceptor> <Interceptor
>> className="org.apache.catalina.tribes.group.
>> interceptors.DomainFilterInterceptor"></Interceptor> <Interceptor
>> className="org.apache.catalina.tribes.group.
>> interceptors.MessageDispatchInterceptor"></Interceptor>
>>
>>
>>
> You specified domain="clustertest" in <Member>, but
> DomainFilterInterceptor does not have a domain setting. If you want
> to filter by domain, you have to set domain="clustertest" to
> DomainFilterInterceptor. if you do not want to filter by domain,
> you have to remove domain="clustertest" from <Member> or remove
> DomainFilterInterceptor.
>
> Also, if you use DomainFilterInterceptor with static membership,
> you must list it above StaticMembershipInterceptor.
>
> e.g.
> TcpPingInterceptor->TcpFailureDetector->DomainFilterInterceptor->Stati
cMembershipInterceptor
>
>
or
> DomainFilterInterceptor->TcpPingInterceptor->TcpFailureDetector->Stati
cMembershipInterceptor
Would
>
it be appropriate for Tomcat to sanity-check some of the
settings above to catch this kind of oversight? Or are there too many
possibilities of valid configuration that it's not possible to
validate in this way?
Thanks,
- -chris
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAln6Ht8ACgkQHPApP6U8
pFhr6hAAtLVR0j+C5UorKkIJv1EIBZJcTS+ZDr/69U89ELLf91OKz0K+6mqmgXl/
TGpkKjKPAhDsmR+1uGh7SR89i0jZVMc29plo1qjd0XIYMJtgViGzA841XFN53Ndh
XI2vIVh+XdunxKO4kWzWaAXdUPVmYkKiZ9A8Sc+c5hYkJA9Pa3PzMZoXWBfhcKHM
hbmGyd1wsFCN+1iWrj2Hd9QAIG9v+gNfJvRQ9e773EOorQz3YMrgjIBUSPy5amC9
E/tng/el9x6jv7FaWIHTH2a12OGZAGSldzDV25w3a3tLq5dIzBGzQytYCPUkPpVr
JOaQewjLLEGgfw39v5EwI4Z/WJAzOP+vozmdYRFyxPgvOwHOkWP7CuXEvHszZD9T
Tggir4TLninyU9Hotq6mzsYeWnaF63OHAPECowOhxro7plZjHdBEqEy+kcn2/r+F
CiGhZI4B7B+RqjK62sx2VNIxNPjiPZycSlt7zXkljkBCRQRWSip1JKcXmQjJgeCB
16mB4rYHSDsAezfvD+/3G20V0j1evl5MPSrn/f9bOj4blOIhMi48m1zEGSyihFr3
SiocXLgje+vXWRXX82IoqH6lR2rcGOwvrGfvO5qzQzT7G7OCRShMZNEBqgWclMOh
tqy/ernLtrp7BuGpJpRn0twqK8qlhGlMyr4kYZ7GIcZEdq7hG/0=
=fSTg
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Beginner help setting up test vertical cluster
Posted by Keiichi Fujino <kf...@apache.org>.
Hi Dave.
Your Interceptor settings are as follows.
> <Interceptor className="org.apache.catalina.tribes.group.
> interceptors.TcpPingInterceptor"></Interceptor>
> <Interceptor className="org.apache.catalina.tribes.group.
> interceptors.TcpFailureDetector"></Interceptor>
> <Interceptor className="org.apache.catalina.tribes.group.
> interceptors.ThroughputInterceptor" />
> <Interceptor className="org.apache.catalina.tribes.group.
> interceptors.StaticMembershipInterceptor">
> <LocalMember className="org.apache.
> catalina.tribes.membership.StaticMember" domain="clustertest"
> uniqueId="{0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}"></LocalMember>
> <Member className="org.apache.catalina.tribes.membership.StaticMember"
> domain="clustertest" host="xxx.xxx.xxx.xxx" port="4001" securePort="-1"
> uniqueId="{1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,0}"></Member>
> </Interceptor>
> <Interceptor className="org.apache.catalina.tribes.group.
> interceptors.DomainFilterInterceptor"></Interceptor>
> <Interceptor className="org.apache.catalina.tribes.group.
> interceptors.MessageDispatchInterceptor"></Interceptor>
>
>
>
You specified domain="clustertest" in <Member>, but DomainFilterInterceptor
does not have a domain setting.
If you want to filter by domain, you have to set domain="clustertest" to
DomainFilterInterceptor.
if you do not want to filter by domain, you have to remove
domain="clustertest" from <Member> or remove DomainFilterInterceptor.
Also, if you use DomainFilterInterceptor with static membership, you must
list it above StaticMembershipInterceptor.
e.g.
TcpPingInterceptor->TcpFailureDetector->DomainFilterInterceptor->StaticMembershipInterceptor
or
DomainFilterInterceptor->TcpPingInterceptor->TcpFailureDetector->StaticMembershipInterceptor
--
Keiichi.Fujino
Re: Beginner help setting up test vertical cluster
Posted by Dave Ford <D....@bath.ac.uk>.
On Mon, 2017-10-30 at 09:15 -0400, Christopher Schultz wrote:
> Dave,
>
> Can you please post your <Cluster> and associated elements from
> conf/server.xml -- minus any secrets that may have crept in there?
> Also, what does your network look like? Any intermediates such as
> load
> balancers/firewalls? What about software firewalls?
Hi Chris - thanks for responding.
This is all running withing a single machine with no firewall, so
nothing should be getting in the way network wise.
Here's the my server.xml from one of the instances, with the comments
removed and IPs and Hostnames removed also. The IPs/Hostnames are the
same on the other server.xml, with just the Member and StaticMember
UniqueIDs and port number entries appropriately reversed.
Dave
--
<?xml version="1.0" encoding="utf-8"?>
<Server port="8005" shutdown="SHUTDOWN">
<Listener className="org.apache.catalina.startup.VersionLoggerListener" />
<Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" />
<Listener className="org.apache.catalina.core.JreMemoryLeakPreventionListener" />
<Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />
<Listener className="org.apache.catalina.core.ThreadLocalLeakPreventionListener" />
<GlobalNamingResources>
<Resource name="UserDatabase" auth="Container" type="org.apache.catalina.UserDatabase" description="User database that can be updated and saved" factory="org.apache.catalina.users.MemoryUserDatabaseFactory" pathname="conf/tomcat-users.xml" />
</GlobalNamingResources>
<Service name="Catalina">
<Connector port="8080" protocol="HTTP/1.1" connectionTimeout="20000" redirectPort="8444" />
<Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />
<Engine name="Catalina" defaultHost="XXXXX" jvmRoute="alpha">
<Realm className="org.apache.catalina.realm.LockOutRealm">
<Realm className="org.apache.catalina.realm.UserDatabaseRealm" resourceName="UserDatabase" />
</Realm>
<Host name="XXXXX" appBase="webapps" unpackWARs="true" autoDeploy="true">
<Cluster channelSendOptions="8" channelStartOptions="3" className="org.apache.catalina.ha.tcp.SimpleTcpCluster">
<Manager className="org.apache.catalina.ha.session.DeltaManager" expireSessionsOnShutdown="false" notifyListenersOnReplication="true"></Manager>
<Channel className="org.apache.catalina.tribes.group.GroupChannel">
<Receiver address="xxx.xxx.xxx.xxx" autoBind="0" className="org.apache.catalina.tribes.transport.nio.NioReceiver" maxThreads="6" port="4000" selectorTimeout="5000"></Receiver>
<Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
<Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"></Transport>
</Sender>
<Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpPingInterceptor"></Interceptor>
<Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"></Interceptor>
<Interceptor className="org.apache.catalina.tribes.group.interceptors.ThroughputInterceptor" />
<Interceptor className="org.apache.catalina.tribes.group.interceptors.StaticMembershipInterceptor">
<LocalMember className="org.apache.catalina.tribes.membership.StaticMember" domain="clustertest" uniqueId="{0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}"></LocalMember>
<Member className="org.apache.catalina.tribes.membership.StaticMember" domain="clustertest" host="xxx.xxx.xxx.xxx" port="4001" securePort="-1" uniqueId="{1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,0}"></Member>
</Interceptor>
<Interceptor className="org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor"></Interceptor>
<Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatchInterceptor"></Interceptor>
</Channel>
<Valve className="org.apache.catalina.ha.tcp.ReplicationValve" filter=""></Valve>
<Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"></Valve>
<ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener" />
<Deployer className="org.apache.catalina.ha.deploy.FarmWarDeployer" deployDir="/opt/tomcat85/instances/alpha/war-deploy/" tempDir="/opt/tomcat85/instances/alpha/war-temp/" watchDir="/opt/tomcat85/instances/alpha/war-listen/" watchEnabled="false"></Deployer>
</Cluster>
<Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs" prefix="localhost_access_log" suffix=".txt" pattern="%h %l %u %t "%r" %s %b" />
</Host>
</Engine>
</Service>
</Server>
Re: Beginner help setting up test vertical cluster
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Dave,
On 10/30/17 6:19 AM, Dave Ford wrote:
> Hello,
>
> I should apologise in advance as I'm very new to Tomcat and, I'm
> sure, will be making some daft mistakes and silly errors. I hope
> this question and any that follow it aren't too dumb.
>
> I've recently started a new job and have inherited a half-finished
> tomcat cluster on solaris.
>
> While I'm not going to immediately try and finish that cluster, I
> would like to try setting up some test instances to ensure I know
> what to expect - but I'm having trouble doing even this.
>
> I've gotten my head round how to set up individual instances of
> Tomcat 8.5.23 running from different sub folders of a common
> CATALINA_HOME folder, and now I'm trying to get the two instances
> to talk to each other using StaticMember on specific ports rather
> than the automatic multicasting.
>
> But instances start up, and the logs at least suggest they're aware
> of each other. But one server takes forever to start, and I'm not
> convinced they're talking to each other properly. For example, in
> the catalina.out logs I'm seeing messages such as:
>
>
> 30-Oct-2017 09:28:29.632 INFO [MyHost-startStop-1]
> org.apache.catalina.ha.session.DeltaManager.getAllClusterSessions
> Manager [/sample], requesting session state from
> [org.apache.catalina.tribes.membership.StaticMember[tcp://my.ip.addr:4
0
>
>
01,my.ip.addr,4001, alive=0, securePort=-1, UDP Port=-1, id={1 2 3 4 5
> 6 7 8 9 10 11 12 13 14 15 0 }, payload={}, command={}, domain={99
> 108 117 115 116 101 114 116 101 ...(11)}, ]]. This operation will
> timeout if no session state has been received within [60] seconds.
> 30-Oct-2017 09:28:34.111 WARNING
> [Tribes-Task-Receiver[MyHost-Channel]- 2]
> org.apache.catalina.tribes.group.interceptors.DomainFilterInterceptor.
m
>
>
essageReceived Received message from
> cluster[org.apache.catalina.tribes.membership.MemberImpl[tcp://{my,
> ip, nn, nn}:4001,{my, ip, nn, nn},4001, alive=1509355714104,
> securePort=- 1, UDP Port=-1, id={1 2 3 4 5 6 7 8 9 10 11 12 13 14
> 15 0 }, payload={}, command={}, domain={99 108 117 115 116 101 114
> 116 101 ...(11)}, ]] was refused.
>
> Which does not seem encouraging. Yet on the other instance, I'm
> seeing:
>
> 30-Oct-2017 09:28:34.105 INFO
> [GroupChannel-Heartbeat[MyHost-Channel]- 1]
> org.apache.catalina.ha.tcp.SimpleTcpCluster.memberAdded
> Replication member
> added:[org.apache.catalina.tribes.membership.StaticMember[tcp://my.ip.
a
>
>
ddr:4000,my.ip.addr,4000, alive=0, securePort=-1, UDP Port=-1, id={0 1
> 2 3 4 5 6 7 8 9 10 11 12 13 14 15 }, payload={}, command={},
> domain={99 108 117 115 116 101 114 116 101 ...(11)}, ]] 30-Oct-2017
> 09:28:34.105 INFO [GroupChannel-Heartbeat[MyHost-Channel]- 1]
> org.apache.catalina.tribes.group.interceptors.TcpFailureDetector.perfo
r
>
>
mBasicCheck Suspect member, confirmed
> alive.[org.apache.catalina.tribes.membership.StaticMember[tcp://my.ip.
a
>
>
ddr:4000,my.ip.addr,4000, alive=0, securePort=-1, UDP Port=-1, id={0 1
> 2 3 4 5 6 7 8 9 10 11 12 13 14 15 }, payload={}, command={},
> domain={99 108 117 115 116 101 114 116 101 ...(11)}, ]]
>
> Which seems better... But then I see back on instance one,
>
> 30-Oct-2017 09:32:30.178 INFO [main]
> org.apache.catalina.startup.Catalina.start Server startup in 241109
> ms
>
> Which seems really bad.
>
> I've installed the 'examples' app on both instances, have added
> '<distributable/>' to both's web.xml file.
>
> I can see on one instance a message such as
>
> 30-Oct-2017 09:35:22.600 INFO [http-nio-8081-exec-7]
> org.apache.catalina.core.ApplicationContext.log SessionListener:
> sessionCreated('C2C3720879E72D558EEF4B0655188EBD.beta') 30-Oct-2017
> 09:35:26.912 INFO [http-nio-8081-exec-8]
> org.apache.catalina.core.ApplicationContext.log SessionListener:
> attributeAdded('C2C3720879E72D558EEF4B0655188EBD.beta', 'foo',
> 'bar')
>
> When I play with the sessions example code. But nothing on the
> other instance that would suggest this is being replicated.
>
> Before I make this post too long and post my server.xml code,
> could someone advise me as to what I should be looking for, or what
> these messages suggest? Any pointers at this stage would be
> grateful as I'm unsure even what a working cluster should look
> like.
Can you please post your <Cluster> and associated elements from
conf/server.xml -- minus any secrets that may have crept in there?
Also, what does your network look like? Any intermediates such as load
balancers/firewalls? What about software firewalls?
- -chris
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAln3JdQACgkQHPApP6U8
pFjqOA/+IV1PkQRjz8M2BNAqjVgB5F8ilsjm9pRt9O+CgGhvfaC056iPsfifvwdl
s5sWyAJ7Jr+X/uv1idJSdWVc8aDzKDsg2bnqdE6T9r7qlio2CnqgirOVrYKF3ykC
WKpx8P7Nkl1fY0vDFF/klzh8yEu5SZm16fUCRuuyV+0ydlPMF82jHaMa5pVXby8b
jLkdz7mxYC/dPWCt6niUY5M918ioJ3g3qV3CXDgMRiHXB99wRSnQxGARFAGyGKV5
GQ/Ioo4lPCX01C+KXtLDMQ7/Z3dS65svcXC0hyeZodp90uZdSrsv+phX1KBa1Idv
dL4HhZ0oBwRxJVEWogClRN2DmoNvo6brEuBssmaXT9a7JHINAxRfkE0SRU9cfGav
Xxq1hDz3xpmim43xfNp0RdPXRF0diYcs0+Q+EhbMgu4ZOfh/G+kAZOXyCzPda1Xz
uzmw+AuBK9gebdZUs9BnurNEleyvlK06ogAOVYlvvm3Hs8Xg2a8eRKJ0PHlhSC+t
WwEAy7XfPmgVWpaq1UY/cFcnOZLnhX8YfS/njIv6931AjSyGr/Opo8SFnLCvXrvX
bJe9DKtPbjcet6GtavOTH4+KV9fhQU7CNsEv2ntHoQk1W5POPRl4xOCBoPdzSnCX
lWEQo4JTZnPK01bRASmXzWL22HjQS2xSX+f64oj7qbaWOgca690=
=UeG8
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org