You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomcat.apache.org by bu...@apache.org on 2020/04/06 10:10:24 UTC
[Bug 64312] New: Many memberDisappeared messages are logged on all
nodes of the cluster with JDK versions 12 or above, even when system is idle
https://bz.apache.org/bugzilla/show_bug.cgi?id=64312
Bug ID: 64312
Summary: Many memberDisappeared messages are logged on all
nodes of the cluster with JDK versions 12 or above,
even when system is idle
Product: Tomcat 8
Version: 8.5.50
Hardware: PC
Status: NEW
Severity: normal
Priority: P2
Component: Cluster
Assignee: dev@tomcat.apache.org
Reporter: mahathivavilala97@gmail.com
Target Milestone: ----
When a cluster is installed with TOMEE 7.0.7 (uses Tomcat 8.5.50.0) with JDK
12 or above only, for default values of frequency (=500) and dropTime (=3000),
many memberDisappeared messages are logged on all nodes of the cluster. This is
occuring even when the system is idle.
The following messages are seen in the logs:
Mar 19, 2020 9:43:56 AM jdk.internal.reflect.DelegatingMethodAccessorImpl
invoke
INFO: Received
memberDisappeared[org.apache.catalina.tribes.membership.MemberImpl[tcp://{10,
18, 88, 143}:4001,{10, 18, 88, 143},4001, alive=1683768, securePort=-1, UDP
Port=-1, id={-16 81 -2 100 -1 -27 65 77 -100 123 96 -67 -54 -14 107 84 },
payload={}, command={}, domain={}]] message. Will verify.
Mar 19, 2020 9:43:56 AM jdk.internal.reflect.DelegatingMethodAccessorImpl
invoke
INFO: Verification complete. Member still
alive[org.apache.catalina.tribes.membership.MemberImpl[tcp://{10, 18, 88,
143}:4001,{10, 18, 88, 143},4001, alive=1683768, securePort=-1, UDP Port=-1,
id={-16 81 -2 100 -1 -27 65 77 -100 123 96 -67 -54 -14 107 84 }, payload={},
command={}, domain={}]]
We have tried to increase the dropTime value to 8500 and observed very few
messages logged in some of the nodes.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
[Bug 64312] Many memberDisappeared messages are logged on all nodes
of the cluster with JDK versions 12 or above, even when system is idle
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=64312
Pavana Sai Mahathi Vavilala <ma...@gmail.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Resolution|WORKSFORME |---
Status|RESOLVED |REOPENED
--- Comment #8 from Pavana Sai Mahathi Vavilala <ma...@gmail.com> ---
Thanks for re-testing the cluster setup.
As we are able to consistently reproduce the issue with 1-cpu configured nodes,
can you please confirm if you have tested with the same configuration?
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
[Bug 64312] Many memberDisappeared messages are logged on all nodes
of the cluster with JDK versions 12 or above, even when system is idle
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=64312
Remy Maucherat <re...@apache.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |NEEDINFO
--- Comment #2 from Remy Maucherat <re...@apache.org> ---
Ok, so basically it still works 100% fine, but with an extra step + noise ? Can
you confirm the exact same setup works without that problem on Java 11 and has
the issue on Java 14 ?
[reminder: Java 11 is the supported release, and Java 14 will be the next
supported release]
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
[Bug 64312] Many memberDisappeared messages are logged on all nodes
of the cluster with JDK versions 12 or above, even when system is idle
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=64312
Mark Thomas <ma...@apache.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |RESOLVED
Resolution|--- |WORKSFORME
--- Comment #4 from Mark Thomas <ma...@apache.org> ---
I've tested this with Adopt OpenJDK 8.242_b08 and 14_36 and I don't see the
memberDisappeared messages.
This looks like an environmental issue that becomes (more) obvious with newer
JDK versions rather than a Tomcat issue.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
[Bug 64312] Many memberDisappeared messages are logged on all nodes
of the cluster with JDK versions 12 or above, even when system is idle
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=64312
Remy Maucherat <re...@apache.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Resolution|--- |WORKSFORME
Status|REOPENED |RESOLVED
--- Comment #9 from Remy Maucherat <re...@apache.org> ---
Please do not reopen this bug report and use the user list if you would like to
investigate further.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
[Bug 64312] Many memberDisappeared messages are logged on all nodes
of the cluster with JDK versions 12 or above, even when system is idle
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=64312
--- Comment #6 from Pavana Sai Mahathi Vavilala <ma...@gmail.com> ---
Please ignore the following from the configuration posted above:
1) Filter attribute in: <Valve
className="org.apache.catalina.ha.tcp.ReplicationValve"
filter=".*\.gif;.*\.js;.*\.jpg;.*\.png;.*\.htm;.*\.html;.*\.css;.*\.txt;" />
2) The custom ClusterListener.
These were inadvertently added in the comment and were not used during the
Tomcat Cluster testing.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
[Bug 64312] Many memberDisappeared messages are logged on all nodes
of the cluster with JDK versions 12 or above, even when system is idle
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=64312
Pavana Sai Mahathi Vavilala <ma...@gmail.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
OS| |All
--- Comment #1 from Pavana Sai Mahathi Vavilala <ma...@gmail.com> ---
The server.xml entry for Membership tag:
<Membership className="org.apache.catalina.tribes.membership.McastService"
address="228.0.0.4" port="33344" frequency="500" dropTime="3000" />
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
[Bug 64312] Many memberDisappeared messages are logged on all nodes
of the cluster with JDK versions 12 or above, even when system is idle
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=64312
Pavana Sai Mahathi Vavilala <ma...@gmail.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Resolution|WORKSFORME |---
Status|RESOLVED |REOPENED
--- Comment #5 from Pavana Sai Mahathi Vavilala <ma...@gmail.com> ---
We reproduced this issue on Tomcat-only (v8.5.54 and v8.5.50) cluster setup
using Oracle JDK 12.0.2 (64-bit, java version "12.0.2" 2019-07-16) and JDK14
(64-bit, java version "14" 2020-03-17).
Cluster setup Details:
1) Nodes Configurations:
Count : 2
Resources : 1-cpu 2.29 GHZ, 1-core and 4 GB RAM.
2) Server.xml Configuration:
a) We used the default cluster settings as per Tomcat 8.5 docs
(https://tomcat.apache.org/tomcat-8.5-doc/cluster-howto.html) using
asynchronous replication mode:
<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
channelSendOptions="8">
<Manager className="org.apache.catalina.ha.session.DeltaManager"
expireSessionsOnShutdown="false" notifyListenersOnReplication="true" />
<Channel className="org.apache.catalina.tribes.group.GroupChannel">
<Membership
className="org.apache.catalina.tribes.membership.McastService"
address="228.0.0.4" port="45564" frequency="500" dropTime="3000" />
<Receiver
className="org.apache.catalina.tribes.transport.nio.NioReceiver" address="auto"
port="4000" autoBind="100" selectorTimeout="5000" maxThreads="6" />
<Sender
className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
<Transport
className="org.apache.catalina.tribes.transport.nio.PooledParallelSender" />
</Sender>
<Interceptor
className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector" />
<Interceptor
className="org.apache.catalina.tribes.group.interceptors.MessageDispatchInterceptor"
/>
</Channel>
<Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
filter=".*\.gif;.*\.js;.*\.jpg;.*\.png;.*\.htm;.*\.html;.*\.css;.*\.txt;" />
<Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"
/>
<Deployer className="org.apache.catalina.ha.deploy.FarmWarDeployer"
tempDir="/tmp/war-temp/" deployDir="/tmp/war-deploy/"
watchDir="/tmp/war-listen/" watchEnabled="false" />
<ClusterListener
className="com.cardiff.cluster.LOClusterImplementation" />
<ClusterListener
className="org.apache.catalina.ha.session.ClusterSessionListener" />
</Cluster>
b) Also for non-default channelSendOptions:
<Cluster
className="org.apache.catalina.ha.tcp.SimpleTcpCluster" channelSendOptions="6">
In both instances, we reproduced the issue.
Note: We were unable to reproduce it with increased cpu resources (cpu count >=
2) or higher dropTime (>= 8500 ms).
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
[Bug 64312] Many memberDisappeared messages are logged on all nodes
of the cluster with JDK versions 12 or above, even when system is idle
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=64312
Pavana Sai Mahathi Vavilala <ma...@gmail.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEEDINFO |NEW
--- Comment #3 from Pavana Sai Mahathi Vavilala <ma...@gmail.com> ---
The exact same setup works without any problem when configured with JDK 11.
However, with JDK 12 and 14, there are too many memberDisappeared messages
logged (almost for every 2 minutes).
Note:
In the JDK 12 environment, on increasing the dropTime to 10000, we identified
that they are scarcely generated.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
[Bug 64312] Many memberDisappeared messages are logged on all nodes
of the cluster with JDK versions 12 or above, even when system is idle
Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=64312
Mark Thomas <ma...@apache.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Resolution|--- |WORKSFORME
Status|REOPENED |RESOLVED
--- Comment #7 from Mark Thomas <ma...@apache.org> ---
I've retested this with a 4 node cluster of 8.5.x using the defaults from the
documentation with dropTime as low as 1000ms with Adopt OpenJDK 14_36 and I
don't see this issue.
The evidence so far isn't pointing towards a Tomcat issue. If you want help
debugging this further, the users list is the place to seek such help.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org