You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomcat.apache.org by bu...@apache.org on 2009/05/31 17:58:37 UTC
DO NOT REPLY [Bug 47292] New: Attributes are lost in newly created
DeltaSession
https://issues.apache.org/bugzilla/show_bug.cgi?id=47292
Summary: Attributes are lost in newly created DeltaSession
Product: Tomcat 6
Version: unspecified
Platform: PC
OS/Version: Linux
Status: NEW
Keywords: PatchAvailable
Severity: critical
Priority: P2
Component: Cluster
AssignedTo: dev@tomcat.apache.org
ReportedBy: martin.sulak@nam.cz
It seems SessionMessages spread in multicast are processed by all nodes
including sender.
Problem timing:
1) http request received
2) session is created
3) session_created message sent
4) attribute added into session
5) session_created received back
6) new session createde with same session_id and stored into sessions map
replacing original session with attributes
7) http request is done
8) deltarequest sent but for empty session from step 6
My quick fix:
/**
* handle receive new session is created at other node (create backup -
primary false)
* @param msg
* @param sender
*/
protected void handleSESSION_CREATED(SessionMessage msg,Member sender) {
// Bug fix begin
if (sessions.containsKey(msg.getSessionID())) {
log.debug("Rejected session_created with duplicit id");
return;
}
// Bug fix end
counterReceive_EVT_SESSION_CREATED++;
if (log.isDebugEnabled())
log.debug(sm.getString("deltaManager.receiveMessage.createNewSession",getName(),
msg.getSessionID()));
DeltaSession session = (DeltaSession) createEmptySession();
session.setManager(this);
session.setValid(true);
session.setPrimarySession(false);
session.setCreationTime(msg.getTimestamp());
// use container maxInactiveInterval so that session will expire
correctly in case of primary transfer
session.setMaxInactiveInterval(getMaxInactiveInterval());
session.access();
if(notifySessionListenersOnReplication)
session.setId(msg.getSessionID());
else
session.setIdInternal(msg.getSessionID());
session.resetDeltaRequest();
session.endAccess();
}
My suggested fixes:
1) in sender node drop SessionMessage sent back by network layer
Check worker id in SessionMessage, which is part of session_id.
2) replace SESSION_CREATED functionality by DeltaRequest.
If SessionMessage with DeltaRequest is received with unknown session_id, create
new session.
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
DO NOT REPLY [Bug 47292] Attributes are lost in newly created
DeltaSession
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=47292
Martin Sulak <ma...@nam.cz> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEEDINFO |NEW
--- Comment #6 from Martin Sulak <ma...@nam.cz> 2009-07-08 14:02:13 PST ---
I've also another problem: First node have tcpListenPort = 4000 and second node
4001. Second node starts ok, but first node always write SEVERE: No session
state send ... into catalina.out.
According to catalina.out, both nodes are sending request to port 4000. So
first node sends request to itself.
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
DO NOT REPLY [Bug 47292] Attributes are lost in newly created
DeltaSession
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=47292
Filip Hanik <fh...@apache.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |RESOLVED
Resolution| |WORKSFORME
--- Comment #10 from Filip Hanik <fh...@apache.org> 2009-07-09 14:39:33 PST ---
Seems like you've worked out your kinks, and also used some funky configs.
The configuration for tomcat 6 is documented here
http://tomcat.apache.org/tomcat-6.0-doc/config/cluster-receiver.html
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
DO NOT REPLY [Bug 47292] Attributes are lost in newly created
DeltaSession
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=47292
--- Comment #2 from Filip Hanik <fh...@apache.org> 2009-06-01 10:09:35 PST ---
Martin, what does this mean
>It seems SessionMessages spread in multicast are processed by all nodes
>including sender.
We're not using multicast for sending messages.
Are you saying that the sending node is receiving the SESSION_CREATED message?
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
DO NOT REPLY [Bug 47292] Attributes are lost in newly created
DeltaSession
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=47292
Martin Sulak <ma...@nam.cz> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEEDINFO |NEW
--- Comment #8 from Martin Sulak <ma...@nam.cz> 2009-07-08 22:12:15 PST ---
Hello,
In node 2 session request was sent to node 1 and reply arrived in 105 ms.
8.7.2009 22:13:02 org.apache.catalina.ha.session.DeltaManager
getAllClusterSessions
WARNING: Manager [localhost#/inetgweb2], requesting session state from
org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 1,
1}:4000,{127, 0, 1, 1},4000, alive=482259865,id={77 104 -45 52 13 -18 77 -88
-123 -28 -35 -2 111 103 73 -108 }, payload={}, command={}, domain={}, ]. This
operation will timeout if no session state has been received within 60 seconds.
8.7.2009 22:13:02 org.apache.catalina.ha.session.DeltaManager
waitForSendAllSessions
INFO: Manager [localhost#/inetgweb2]; session state send at 8.7.09 22:13
received in 105 ms.
----------------------------------------------------------------------
In node 1 session request was sent to node 1 again and reply did not arrive.
8.7.2009 22:36:59 org.apache.catalina.ha.session.DeltaManager
getAllClusterSessions
WARNING: Manager [localhost#/inetgweb2], requesting session state from
org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 1,
1}:4000,{127, 0, 1, 1},4000, alive=483696349,id={77 104 -45 52 13 -18 77 -88
-123 -28 -35 -2 111 103 7
3 -108 }, payload={}, command={}, domain={}, ]. This operation will timeout if
no session state has been received within 60 seconds.
...
8.7.2009 22:37:59 org.apache.catalina.ha.session.DeltaManager
waitForSendAllSessions
SEVERE: Manager [localhost#/inetgweb2]: No session state send at 8.7.09 22:36
received, timing out after 60 009 ms.
8.7.2009 22:37:59 org.apache.catalina.ha.session.DeltaManager
getAllClusterSessions
WARNING: Manager [localhost#/inetgweb2]: Drop message SESSION-GET-ALL inside
GET_ALL_SESSIONS sync phase start date 8.7.09 22:36 message date 1.1.70 1:00
----------------------------------------------------------------------
More interesting are logs about memberAdded events (on node1,
tcplistenport=4000):
INFO: Starting Servlet Engine: Apache Tomcat/6.0.18
8.7.2009 22:34:52 org.apache.catalina.ha.tcp.SimpleTcpCluster start
INFO: Cluster is about to start
8.7.2009 22:34:52 org.apache.catalina.tribes.transport.ReceiverBase bind
INFO: Receiver Server Socket bound to:/127.0.1.1:4000
8.7.2009 22:34:52 org.apache.catalina.tribes.membership.McastServiceImpl
setupSocket
INFO: Setting cluster mcast soTimeout to 500
8.7.2009 22:34:52 org.apache.catalina.tribes.membership.McastServiceImpl
waitForMembers
INFO: Sleeping for 1000 milliseconds to establish cluster membership, start
level:4
8.7.2009 22:34:52 org.apache.catalina.ha.tcp.SimpleTcpCluster memberAdded
INFO: Replication member
added:org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 1,
1}:4000,{127, 0, 1, 1},4000, alive=483569817,id={77 104 -45 52 13 -18 77 -88
-123 -28 -35 -2 111 103 73 -108 }, payload={}, command={}, domain={}, ]
8.7.2009 22:34:52 org.apache.catalina.tribes.io.BufferPool getBufferPool
INFO: Created a buffer pool with max size:104857600 bytes of
type:org.apache.catalina.tribes.io.BufferPool15Impl
8.7.2009 22:34:52 org.apache.catalina.ha.tcp.SimpleTcpCluster memberAdded
INFO: Replication member
added:org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 1,
1}:4001,{127, 0, 1, 1},4001, alive=483264299,id={62 80 120 -81 31 -10 76 115
-123 78 5 44 -86 -85 -85 -59 }, payload={}, command={}, domain={}, ]
8.7.2009 22:34:53 org.apache.catalina.ha.tcp.SimpleTcpCluster memberAdded
INFO: Replication member
added:org.apache.catalina.tribes.membership.MemberImpl[tcp://{127, 0, 1,
1}:4001,{127, 0, 1, 1},4001, alive=81647,id={-113 -69 -19 41 -24 -23 76 53 -89
-13 125 -49 95 -62 63 2 }, payload={}, command={}, domain={}, ]
8.7.2009 22:34:53 org.apache.catalina.tribes.membership.McastServiceImpl
waitForMembers
INFO: Done sleeping, membership established, start level:4
8.7.2009 22:34:53 org.apache.catalina.tribes.membership.McastServiceImpl
waitForMembers
INFO: Sleeping for 1000 milliseconds to establish cluster membership, start
level:8
8.7.2009 22:34:54 org.apache.catalina.tribes.membership.McastServiceImpl
waitForMembers
INFO: Done sleeping, membership established, start level:8
----------------------------------------------------------------------
I'm not sure why there are two memberadded events for port 4001 - node2.
However there should definitely not be memberadded for port 4000 - node1
itself, which is first event in log.
This is cause of both my bugs:
As I've seen in SimpleTcpCluster.java, session messages are sent to all
members. This can explain my first bug.
In DeltaManager.findSessionMasterMember first member is in
getAllClusterSessions.
This explains my second bug.
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
DO NOT REPLY [Bug 47292] Attributes are lost in newly created
DeltaSession
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=47292
--- Comment #5 from Martin Sulak <ma...@nam.cz> 2009-07-08 13:56:55 PST ---
Created an attachment (id=23945)
--> (https://issues.apache.org/bugzilla/attachment.cgi?id=23945)
Zipped configuration files
Here are requested configuration files from tomcat/conf, conf1 for node1 and
conf2 for node2. Both nodes are on the same machine.
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
DO NOT REPLY [Bug 47292] Attributes are lost in newly created
DeltaSession
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=47292
--- Comment #1 from Martin Sulak <ma...@nam.cz> 2009-05-31 23:01:31 PST ---
Sorry, I forgot to mention : the handleSESSION_CREATED method I modified is
located in class org.apache.catalina.ha.session.DeltaManager.
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
DO NOT REPLY [Bug 47292] Attributes are lost in newly created
DeltaSession
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=47292
--- Comment #3 from Martin Sulak <ma...@nam.cz> 2009-06-01 22:16:48 PST ---
> We're not using multicast for sending messages.
Sorry, I just guessed.
> Are you saying that the sending node is receiving the SESSION_CREATED message?
Yes, handleSESSION_CREATED was actually called while processing the request. At
least log file said so. To be sure with this I added some logging into this
method. Session_id was identical to session created in this request.
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
DO NOT REPLY [Bug 47292] Attributes are lost in newly created
DeltaSession
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=47292
Casey Lucas <cl...@e-miles.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |clucas@e-miles.com
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
DO NOT REPLY [Bug 47292] Attributes are lost in newly created
DeltaSession
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=47292
Martin Sulak <ma...@nam.cz> changed:
What |Removed |Added
----------------------------------------------------------------------------
Severity|critical |normal
--- Comment #9 from Martin Sulak <ma...@nam.cz> 2009-07-09 03:24:00 PST ---
Now I'm closer to the "route cause".
It seems during McastServiceImpl.waitForMembers ping is received with same ip
and port but with different uniqueid. The ip is 127.0.1.1 so it actually may be
tomcat on another machine. Member is added for 127.0.1.1 because of different
uniqueid.
I resolved it in /etc/hosts by adding "self" ip address of the machine.
It seems cluster.tcpListenAddressip adress in server.xml is ignored and system
API is used instead to determine host's ip.
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
DO NOT REPLY [Bug 47292] Attributes are lost in newly created
DeltaSession
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=47292
Filip Hanik <fh...@apache.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |NEEDINFO
--- Comment #7 from Filip Hanik <fh...@apache.org> 2009-07-08 14:15:35 PST ---
Sounds like you have some useful info in the logs, are you able to give us
those too?
thanks in advance
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
DO NOT REPLY [Bug 47292] Attributes are lost in newly created
DeltaSession
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=47292
Filip Hanik <fh...@apache.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |NEEDINFO
--- Comment #4 from Filip Hanik <fh...@apache.org> 2009-07-08 12:40:11 PST ---
Could you attach your configurations from your different nodes, we shouldn't be
receiving our own sessions locally.
so while the quick fix takes care of the problem, its not addressing the actual
root cause.
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
DO NOT REPLY [Bug 47292] Attributes are lost in newly created
DeltaSession
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=47292
Martin Sulak <ma...@nam.cz> changed:
What |Removed |Added
----------------------------------------------------------------------------
Version|unspecified |6.0.18
--
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org