You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@activemq.apache.org by "Joel Schaubert (JIRA)" <ji...@apache.org> on 2007/03/14 04:41:34 UTC
[jira] Created: (AMQ-1203) Killing STOMP client stalls the Q
forever until a new message is written to that Q
Killing STOMP client stalls the Q forever until a new message is written to that Q
----------------------------------------------------------------------------------
Key: AMQ-1203
URL: https://issues.apache.org/activemq/browse/AMQ-1203
Project: ActiveMQ
Issue Type: Bug
Affects Versions: 4.1.1
Environment: AS3 linux
Reporter: Joel Schaubert
Priority: Minor
Using STOMP C client subscribed in client ack mode
A) send 20 messages into a Q
B) startup slow reader, read 5 messages and control-C without allowing client to send DISCONNECT
C) restart a reader (or more readers if desired)
D) the remaining 15 messages are still present but none will be delivered to the STOMP clients until at least 1 new message is written into the Q.
So no loss of messages under these conditions but Q is effectively stalled until a new message is written in.
------ copy of Hiram's comments from AMQ-1192 ------------
I think this is due to the socket being killed but the server side not noticing. But it will notice once it tries to send it a message, so the it shutdown the dead socket and redelivers all messages queued for the client. On some OSes the TCP timeout interval can be tweaked. Otherwise, keep alive packets would need to be periodically transmitted in Stomp so that the server could detect a dead client sooner. Keep alives have not been added to Stomp yet.. but could be in a future version.
------------------------------------------------------------------------------
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (AMQ-1203) Killing STOMP client stalls
the Q forever until a new message is written to that Q
Posted by "Bryan Taylor (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-1203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=43463#action_43463 ]
bwtaylor edited comment on AMQ-1203 at 6/14/08 3:16 PM:
------------------------------------------------------------
I'm seeing a socket leak that is hanging my server that I think is related to this bug, given Hiram's comment. My producers connect by stomp to my broker. I believe a firewall in between the producers and the broker drops the connection when it's quiet for longer than a timeout. When the producers try to send another message, they see the dead socket and reopen a new one. However, my broker still sees the original connections as ESTABLISHED. This game plays out on any stomp connection that is quiet for longer than the firewall's timeout, which gives me about 25-35 leaked sockets a day until I hit the ulimit for file descriptors. At this time, my broker is hung: it cannot accept new connections, but has not crashed.
I have several suggestions:
1) bump the priority of this ticket
2) implement keep alives and the maxInactivityDuration parameter for stomp
3) when sockets are maxed out, kill old sockets LRU style so as to always accept new connections
was (Author: bwtaylor):
I'm seeing a socket leak that is hanging my server that I think is related to this bug, given Hiram's comment. My producers connect by stomp to my broker. I believe a firewall in between the producers and the broker drops the connection. When the producers try to send another message, they see the dead socket and reopen a new one. However, my broker still sees the original connections as ESTABLISHED. This game plays out on any stomp connection that is quiet for longer than the firewall's timeout, which gives me about 25-35 leaked sockets a day until I hit the ulimit for file descriptors. At this time, my broker is hung: it cannot accept new connections, but has not crashed.
I have several suggestions:
1) bump the priority of this ticket
2) implement keep alives and the maxInactivityDuration parameter for stomp
3) when sockets are maxed out, kill old sockets LRU style so as to always accept new connections
> Killing STOMP client stalls the Q forever until a new message is written to that Q
> ----------------------------------------------------------------------------------
>
> Key: AMQ-1203
> URL: https://issues.apache.org/activemq/browse/AMQ-1203
> Project: ActiveMQ
> Issue Type: Bug
> Affects Versions: 4.1.1
> Environment: AS3 linux
> Reporter: Joel Schaubert
> Priority: Minor
> Fix For: 5.2.0
>
>
> Using STOMP C client subscribed in client ack mode
> A) send 20 messages into a Q
> B) startup slow reader, read 5 messages and control-C without allowing client to send DISCONNECT
> C) restart a reader (or more readers if desired)
> D) the remaining 15 messages are still present but none will be delivered to the STOMP clients until at least 1 new message is written into the Q.
> So no loss of messages under these conditions but Q is effectively stalled until a new message is written in.
> ------ copy of Hiram's comments from AMQ-1192 ------------
> I think this is due to the socket being killed but the server side not noticing. But it will notice once it tries to send it a message, so the it shutdown the dead socket and redelivers all messages queued for the client. On some OSes the TCP timeout interval can be tweaked. Otherwise, keep alive packets would need to be periodically transmitted in Stomp so that the server could detect a dead client sooner. Keep alives have not been added to Stomp yet.. but could be in a future version.
> ------------------------------------------------------------------------------
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AMQ-1203) Killing STOMP client stalls the Q
forever until a new message is written to that Q
Posted by "Gary Tully (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-1203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gary Tully updated AMQ-1203:
----------------------------
Fix Version/s: 5.3.0
(was: 5.2.0)
> Killing STOMP client stalls the Q forever until a new message is written to that Q
> ----------------------------------------------------------------------------------
>
> Key: AMQ-1203
> URL: https://issues.apache.org/activemq/browse/AMQ-1203
> Project: ActiveMQ
> Issue Type: Bug
> Affects Versions: 4.1.1
> Environment: AS3 linux
> Reporter: Joel Schaubert
> Priority: Minor
> Fix For: 5.3.0
>
>
> Using STOMP C client subscribed in client ack mode
> A) send 20 messages into a Q
> B) startup slow reader, read 5 messages and control-C without allowing client to send DISCONNECT
> C) restart a reader (or more readers if desired)
> D) the remaining 15 messages are still present but none will be delivered to the STOMP clients until at least 1 new message is written into the Q.
> So no loss of messages under these conditions but Q is effectively stalled until a new message is written in.
> ------ copy of Hiram's comments from AMQ-1192 ------------
> I think this is due to the socket being killed but the server side not noticing. But it will notice once it tries to send it a message, so the it shutdown the dead socket and redelivers all messages queued for the client. On some OSes the TCP timeout interval can be tweaked. Otherwise, keep alive packets would need to be periodically transmitted in Stomp so that the server could detect a dead client sooner. Keep alives have not been added to Stomp yet.. but could be in a future version.
> ------------------------------------------------------------------------------
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AMQ-1203) Killing STOMP client stalls the Q
forever until a new message is written to that Q
Posted by "Bruce Snyder (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-1203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bruce Snyder updated AMQ-1203:
------------------------------
Fix Version/s: (was: AGING_TO_DIE)
NEED_REVIEWED
> Killing STOMP client stalls the Q forever until a new message is written to that Q
> ----------------------------------------------------------------------------------
>
> Key: AMQ-1203
> URL: https://issues.apache.org/activemq/browse/AMQ-1203
> Project: ActiveMQ
> Issue Type: Bug
> Affects Versions: 4.1.1
> Environment: AS3 linux
> Reporter: Joel Schaubert
> Priority: Minor
> Fix For: NEED_REVIEWED
>
>
> Using STOMP C client subscribed in client ack mode
> A) send 20 messages into a Q
> B) startup slow reader, read 5 messages and control-C without allowing client to send DISCONNECT
> C) restart a reader (or more readers if desired)
> D) the remaining 15 messages are still present but none will be delivered to the STOMP clients until at least 1 new message is written into the Q.
> So no loss of messages under these conditions but Q is effectively stalled until a new message is written in.
> ------ copy of Hiram's comments from AMQ-1192 ------------
> I think this is due to the socket being killed but the server side not noticing. But it will notice once it tries to send it a message, so the it shutdown the dead socket and redelivers all messages queued for the client. On some OSes the TCP timeout interval can be tweaked. Otherwise, keep alive packets would need to be periodically transmitted in Stomp so that the server could detect a dead client sooner. Keep alives have not been added to Stomp yet.. but could be in a future version.
> ------------------------------------------------------------------------------
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AMQ-1203) Killing STOMP client stalls the Q
forever until a new message is written to that Q
Posted by "Bryan Taylor (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-1203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=43463#action_43463 ]
Bryan Taylor commented on AMQ-1203:
-----------------------------------
I'm seeing a socket leak that is hanging my server that I think is related to this bug, given Hiram's comment. My producers connect by stomp to my broker. I believe a firewall in between the producers and the broker drops the connection. When the producers try to send another message, they see the dead socket and reopen a new one. However, my broker still sees the original connections as ESTABLISHED. This game plays out on any stomp connection that is quiet for longer than the firewall's timeout, which gives me about 25-35 leaked sockets a day until I hit the ulimit for file descriptors. At this time, my broker is hung: it cannot accept new connections, but has not crashed.
I have several suggestions:
1) bump the priority of this ticket
2) implement keep alives and the maxInactivityDuration parameter for stomp
3) when sockets are maxed out, kill old sockets LRU style so as to always accept new connections
> Killing STOMP client stalls the Q forever until a new message is written to that Q
> ----------------------------------------------------------------------------------
>
> Key: AMQ-1203
> URL: https://issues.apache.org/activemq/browse/AMQ-1203
> Project: ActiveMQ
> Issue Type: Bug
> Affects Versions: 4.1.1
> Environment: AS3 linux
> Reporter: Joel Schaubert
> Priority: Minor
> Fix For: 5.2.0
>
>
> Using STOMP C client subscribed in client ack mode
> A) send 20 messages into a Q
> B) startup slow reader, read 5 messages and control-C without allowing client to send DISCONNECT
> C) restart a reader (or more readers if desired)
> D) the remaining 15 messages are still present but none will be delivered to the STOMP clients until at least 1 new message is written into the Q.
> So no loss of messages under these conditions but Q is effectively stalled until a new message is written in.
> ------ copy of Hiram's comments from AMQ-1192 ------------
> I think this is due to the socket being killed but the server side not noticing. But it will notice once it tries to send it a message, so the it shutdown the dead socket and redelivers all messages queued for the client. On some OSes the TCP timeout interval can be tweaked. Otherwise, keep alive packets would need to be periodically transmitted in Stomp so that the server could detect a dead client sooner. Keep alives have not been added to Stomp yet.. but could be in a future version.
> ------------------------------------------------------------------------------
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.