You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@activemq.apache.org by "Jim Gomes (JIRA)" <ji...@apache.org> on 2012/08/18 01:06:37 UTC
[jira] [Created] (AMQNET-394) Zombie Consumer is created after
failover
Jim Gomes created AMQNET-394:
--------------------------------
Summary: Zombie Consumer is created after failover
Key: AMQNET-394
URL: https://issues.apache.org/jira/browse/AMQNET-394
Project: ActiveMQ .Net
Issue Type: Bug
Components: ActiveMQ, Stomp
Affects Versions: 1.5.6
Reporter: Jim Gomes
Assignee: Jim Gomes
Fix For: 1.5.7, 1.6.0
Steps to reproduce:
1. Create a consumer on a topic.
2. Disconnect the broker (easiest is to shut it down).
3. Destroy the consumer while the low-level failover is retrying to connect.
4. Bring the broker online and let the failover reconnect.
Result:
A zombie consumer is re-created on the topic. This will eventually lead to producer flow-control kicking in because it will never acknowledge messages sent to it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AMQNET-394) Zombie Consumer is created after
failover
Posted by "Jim Gomes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AMQNET-394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13438271#comment-13438271 ]
Jim Gomes commented on AMQNET-394:
----------------------------------
That does look like a better solution. I was afraid that there would be a potential race condition where the failover would reconnect while the consumer was being removed, but I see now that the reconnectMutex should avoid that condition. I'll update with this change.
> Zombie Consumer is created after failover
> -----------------------------------------
>
> Key: AMQNET-394
> URL: https://issues.apache.org/jira/browse/AMQNET-394
> Project: ActiveMQ .Net
> Issue Type: Bug
> Components: ActiveMQ, Stomp
> Affects Versions: 1.5.6
> Reporter: Jim Gomes
> Assignee: Jim Gomes
> Priority: Critical
> Labels: consumer, failover
> Fix For: 1.5.3, 1.5.7, 1.6.0
>
> Original Estimate: 24h
> Time Spent: 24h
> Remaining Estimate: 0h
>
> Steps to reproduce:
> 1. Create a consumer on a topic.
> 2. Disconnect the broker (easiest is to shut it down).
> 3. Destroy the consumer while the low-level failover is retrying to connect.
> 4. Bring the broker online and let the failover reconnect.
> Result:
> A zombie consumer is re-created on the topic. This will eventually lead to producer flow-control kicking in because it will never acknowledge messages sent to it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (AMQNET-394) Zombie Consumer is created after
failover
Posted by "Jim Gomes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AMQNET-394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jim Gomes resolved AMQNET-394.
------------------------------
Resolution: Fixed
> Zombie Consumer is created after failover
> -----------------------------------------
>
> Key: AMQNET-394
> URL: https://issues.apache.org/jira/browse/AMQNET-394
> Project: ActiveMQ .Net
> Issue Type: Bug
> Components: ActiveMQ, Stomp
> Affects Versions: 1.5.6
> Reporter: Jim Gomes
> Assignee: Jim Gomes
> Priority: Critical
> Labels: consumer, failover
> Fix For: 1.5.7, 1.6.0, 1.5.3
>
> Original Estimate: 24h
> Time Spent: 24h
> Remaining Estimate: 0h
>
> Steps to reproduce:
> 1. Create a consumer on a topic.
> 2. Disconnect the broker (easiest is to shut it down).
> 3. Destroy the consumer while the low-level failover is retrying to connect.
> 4. Bring the broker online and let the failover reconnect.
> Result:
> A zombie consumer is re-created on the topic. This will eventually lead to producer flow-control kicking in because it will never acknowledge messages sent to it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (AMQNET-394) Zombie Consumer is created after
failover
Posted by "Jim Gomes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AMQNET-394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jim Gomes resolved AMQNET-394.
------------------------------
Resolution: Fixed
Fix Version/s: 1.5.3
Fixed in 1.5.x branches for Apache.NMS.ActiveMQ and Apache.NMS.Stomp. Merged changes into trunk for both projects.
> Zombie Consumer is created after failover
> -----------------------------------------
>
> Key: AMQNET-394
> URL: https://issues.apache.org/jira/browse/AMQNET-394
> Project: ActiveMQ .Net
> Issue Type: Bug
> Components: ActiveMQ, Stomp
> Affects Versions: 1.5.6
> Reporter: Jim Gomes
> Assignee: Jim Gomes
> Priority: Critical
> Labels: consumer, failover
> Fix For: 1.5.7, 1.6.0, 1.5.3
>
> Original Estimate: 24h
> Time Spent: 24h
> Remaining Estimate: 0h
>
> Steps to reproduce:
> 1. Create a consumer on a topic.
> 2. Disconnect the broker (easiest is to shut it down).
> 3. Destroy the consumer while the low-level failover is retrying to connect.
> 4. Bring the broker online and let the failover reconnect.
> Result:
> A zombie consumer is re-created on the topic. This will eventually lead to producer flow-control kicking in because it will never acknowledge messages sent to it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Work logged] (AMQNET-394) Zombie Consumer is created after
failover
Posted by "Jim Gomes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AMQNET-394?page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel\#worklog-{worklog.getId()} ]
Jim Gomes logged work on AMQNET-394:
------------------------------------
Author: Jim Gomes
Created on: 18/Aug/12 10:33
Start Date: 18/Aug/12 10:33
Worklog Time Spent: 24h
Issue Time Tracking
-------------------
Worklog Id: (was: 13934)
Time Spent: 24h
Remaining Estimate: 0h (was: 24h)
> Zombie Consumer is created after failover
> -----------------------------------------
>
> Key: AMQNET-394
> URL: https://issues.apache.org/jira/browse/AMQNET-394
> Project: ActiveMQ .Net
> Issue Type: Bug
> Components: ActiveMQ, Stomp
> Affects Versions: 1.5.6
> Reporter: Jim Gomes
> Assignee: Jim Gomes
> Priority: Critical
> Labels: consumer, failover
> Fix For: 1.5.7, 1.6.0
>
> Original Estimate: 24h
> Time Spent: 24h
> Remaining Estimate: 0h
>
> Steps to reproduce:
> 1. Create a consumer on a topic.
> 2. Disconnect the broker (easiest is to shut it down).
> 3. Destroy the consumer while the low-level failover is retrying to connect.
> 4. Bring the broker online and let the failover reconnect.
> Result:
> A zombie consumer is re-created on the topic. This will eventually lead to producer flow-control kicking in because it will never acknowledge messages sent to it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AMQNET-394) Zombie Consumer is created after
failover
Posted by "Timothy Bish (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AMQNET-394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13438272#comment-13438272 ]
Timothy Bish commented on AMQNET-394:
-------------------------------------
The other thing that we want to avoid is sending multiple removeInfo's in the case where the broker has async dispatch enabled and the consumer has a prefetch, you'd likely get a batch of messages for that consumer out before your remove gets fired off and then end up sending several removes and creating a lot of log spam on the broker.
> Zombie Consumer is created after failover
> -----------------------------------------
>
> Key: AMQNET-394
> URL: https://issues.apache.org/jira/browse/AMQNET-394
> Project: ActiveMQ .Net
> Issue Type: Bug
> Components: ActiveMQ, Stomp
> Affects Versions: 1.5.6
> Reporter: Jim Gomes
> Assignee: Jim Gomes
> Priority: Critical
> Labels: consumer, failover
> Fix For: 1.5.3, 1.5.7, 1.6.0
>
> Original Estimate: 24h
> Time Spent: 24h
> Remaining Estimate: 0h
>
> Steps to reproduce:
> 1. Create a consumer on a topic.
> 2. Disconnect the broker (easiest is to shut it down).
> 3. Destroy the consumer while the low-level failover is retrying to connect.
> 4. Bring the broker online and let the failover reconnect.
> Result:
> A zombie consumer is re-created on the topic. This will eventually lead to producer flow-control kicking in because it will never acknowledge messages sent to it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (AMQNET-394) Zombie Consumer is created after
failover
Posted by "Timothy Bish (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AMQNET-394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Timothy Bish reopened AMQNET-394:
---------------------------------
Actually I think this solution is not so great, the simplest solution would be for the FailoverTransport to not create the zombie consumers in the first place. This can be accomplished by adding the missing call to:
{code}
stateTracker.Track(command);
{code}
In FailoverTransport oneway:
{code}
else if(command.IsRemoveInfo || command.IsMessageAck)
{
stateTracker.Track(command);
// Simulate response to RemoveInfo command or a MessageAck
// since it would be stale at this point.
if(command.ResponseRequired)
{
OnCommand(this, new Response() { CorrelationId = command.CommandId });
}
return;
}
{code}
This then also accounts for Producers and Sessions which the applied change does not.
> Zombie Consumer is created after failover
> -----------------------------------------
>
> Key: AMQNET-394
> URL: https://issues.apache.org/jira/browse/AMQNET-394
> Project: ActiveMQ .Net
> Issue Type: Bug
> Components: ActiveMQ, Stomp
> Affects Versions: 1.5.6
> Reporter: Jim Gomes
> Assignee: Jim Gomes
> Priority: Critical
> Labels: consumer, failover
> Fix For: 1.5.3, 1.5.7, 1.6.0
>
> Original Estimate: 24h
> Time Spent: 24h
> Remaining Estimate: 0h
>
> Steps to reproduce:
> 1. Create a consumer on a topic.
> 2. Disconnect the broker (easiest is to shut it down).
> 3. Destroy the consumer while the low-level failover is retrying to connect.
> 4. Bring the broker online and let the failover reconnect.
> Result:
> A zombie consumer is re-created on the topic. This will eventually lead to producer flow-control kicking in because it will never acknowledge messages sent to it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AMQNET-394) Zombie Consumer is created after
failover
Posted by "Jim Gomes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AMQNET-394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jim Gomes updated AMQNET-394:
-----------------------------
Priority: Critical (was: Major)
> Zombie Consumer is created after failover
> -----------------------------------------
>
> Key: AMQNET-394
> URL: https://issues.apache.org/jira/browse/AMQNET-394
> Project: ActiveMQ .Net
> Issue Type: Bug
> Components: ActiveMQ, Stomp
> Affects Versions: 1.5.6
> Reporter: Jim Gomes
> Assignee: Jim Gomes
> Priority: Critical
> Labels: failover
> Fix For: 1.5.7, 1.6.0
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> Steps to reproduce:
> 1. Create a consumer on a topic.
> 2. Disconnect the broker (easiest is to shut it down).
> 3. Destroy the consumer while the low-level failover is retrying to connect.
> 4. Bring the broker online and let the failover reconnect.
> Result:
> A zombie consumer is re-created on the topic. This will eventually lead to producer flow-control kicking in because it will never acknowledge messages sent to it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AMQNET-394) Zombie Consumer is created after
failover
Posted by "Jim Gomes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AMQNET-394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13437127#comment-13437127 ]
Jim Gomes commented on AMQNET-394:
----------------------------------
The simplest and most robust solution is to simply send a RemoveInfo command to the broker when a message is dispatched for a non-existent consumer. This will clean up any zombie consumers.
> Zombie Consumer is created after failover
> -----------------------------------------
>
> Key: AMQNET-394
> URL: https://issues.apache.org/jira/browse/AMQNET-394
> Project: ActiveMQ .Net
> Issue Type: Bug
> Components: ActiveMQ, Stomp
> Affects Versions: 1.5.6
> Reporter: Jim Gomes
> Assignee: Jim Gomes
> Priority: Critical
> Labels: failover
> Fix For: 1.5.7, 1.6.0
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> Steps to reproduce:
> 1. Create a consumer on a topic.
> 2. Disconnect the broker (easiest is to shut it down).
> 3. Destroy the consumer while the low-level failover is retrying to connect.
> 4. Bring the broker online and let the failover reconnect.
> Result:
> A zombie consumer is re-created on the topic. This will eventually lead to producer flow-control kicking in because it will never acknowledge messages sent to it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira