You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by "Keith Wall (JIRA)" <ji...@apache.org> on 2016/08/11 16:33:20 UTC
[jira] [Comment Edited] (QPID-7387) [0-8..0-91] Mishandling of
message credit allows consumers to starve
[ https://issues.apache.org/jira/browse/QPID-7387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15417530#comment-15417530 ]
Keith Wall edited comment on QPID-7387 at 8/11/16 4:33 PM:
-----------------------------------------------------------
Application using a synchronous receive calls with a transactional session (prefetch 1) at first rolls back every incoming message (Broker configured to DLQ). After a number of messages rolled back, it application flips state and begins to commit the transactions instead. Internally the Qpid Client tries to revert the session's prefetch by sending a BasicQOS. There is a defect on the Broker side that means the _messageCredit is updated and becomes a negative value. Once this has happened, the session is broken. It can no longer send further messages to consumers (either existing or new).
(Following log from 0.32 derivative. Equivalent trunk line, 80)
{noformat}
2016-08-11 16:28:26,630 WARN [IoRcvr-/127.0.0.1:5672-/127.0.0.1:49680] (flow.Pre0_10CreditManager) - KWDEBUG message credit gone negative: messageCreditLimit 1 _messageCredit -342 _messageCreditLimit 11284
java.lang.Exception
at org.apache.qpid.server.flow.Pre0_10CreditManager.setCreditLimits(Pre0_10CreditManager.java:77)
at org.apache.qpid.server.protocol.v0_8.AMQChannel.setCredit(AMQChannel.java:1369)
at org.apache.qpid.server.protocol.v0_8.AMQChannel.receiveBasicQos(AMQChannel.java:2346)
at org.apache.qpid.framing.BasicQosBody.process(BasicQosBody.java:136)
at org.apache.qpid.codec.ServerDecoder.processMethod(ServerDecoder.java:170)
at org.apache.qpid.codec.AMQDecoder.processFrame(AMQDecoder.java:388)
at org.apache.qpid.server.protocol.v0_8.BrokerDecoder.doProcessFrame(BrokerDecoder.java:113)
at org.apache.qpid.server.protocol.v0_8.BrokerDecoder.access$000(BrokerDecoder.java:36)
at org.apache.qpid.server.protocol.v0_8.BrokerDecoder$1.run(BrokerDecoder.java:79)
at org.apache.qpid.server.protocol.v0_8.BrokerDecoder$1.run(BrokerDecoder.java:75)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.qpid.server.protocol.v0_8.BrokerDecoder.processFrame(BrokerDecoder.java:74)
at org.apache.qpid.codec.AMQDecoder.processInput(AMQDecoder.java:370)
at org.apache.qpid.codec.AMQDecoder.decodeBuffer(AMQDecoder.java:259)
at org.apache.qpid.server.protocol.v0_8.AMQProtocolEngine$2.run(AMQProtocolEngine.java:334)
at org.apache.qpid.server.protocol.v0_8.AMQProtocolEngine$2.run(AMQProtocolEngine.java:311)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:360)
at org.apache.qpid.server.protocol.v0_8.AMQProtocolEngine.received(AMQProtocolEngine.java:310)
at org.apache.qpid.server.protocol.v0_8.AMQProtocolEngine.received(AMQProtocolEngine.java:95)
at org.apache.qpid.server.protocol.MultiVersionProtocolEngine.received(MultiVersionProtocolEngine.java:153)
at org.apache.qpid.server.protocol.MultiVersionProtocolEngine.received(MultiVersionProtocolEngine.java:51)
at org.apache.qpid.transport.network.io.IoReceiver.run(IoReceiver.java:163)
at java.lang.Thread.run(Thread.java:745)
{noformat}
was (Author: k-wall):
Application using a synchronous receive calls with a transactional session (prefetch 1) at first rolls back every incoming message (Broker configured to DLQ). After a number of messages rolled back, it application flips state and begins to commit the transactions instead. Internally the Qpid Client tries to revert the session's prefetch by sending a BasicQOS. There is a defect on the Broker side that means the _messageCredit is updated and becomes a negative value. Once this has happened, the session is broken. It can no longer send further messages to consumers (either existing or new).
(Following log from 0.32 derivative. Equivalent trunk line, 80)
{{noformat}}
2016-08-11 16:28:26,630 WARN [IoRcvr-/127.0.0.1:5672-/127.0.0.1:49680] (flow.Pre0_10CreditManager) - KWDEBUG message credit gone negative: messageCreditLimit 1 _messageCredit -342 _messageCreditLimit 11284
java.lang.Exception
at org.apache.qpid.server.flow.Pre0_10CreditManager.setCreditLimits(Pre0_10CreditManager.java:77)
at org.apache.qpid.server.protocol.v0_8.AMQChannel.setCredit(AMQChannel.java:1369)
at org.apache.qpid.server.protocol.v0_8.AMQChannel.receiveBasicQos(AMQChannel.java:2346)
at org.apache.qpid.framing.BasicQosBody.process(BasicQosBody.java:136)
at org.apache.qpid.codec.ServerDecoder.processMethod(ServerDecoder.java:170)
at org.apache.qpid.codec.AMQDecoder.processFrame(AMQDecoder.java:388)
at org.apache.qpid.server.protocol.v0_8.BrokerDecoder.doProcessFrame(BrokerDecoder.java:113)
at org.apache.qpid.server.protocol.v0_8.BrokerDecoder.access$000(BrokerDecoder.java:36)
at org.apache.qpid.server.protocol.v0_8.BrokerDecoder$1.run(BrokerDecoder.java:79)
at org.apache.qpid.server.protocol.v0_8.BrokerDecoder$1.run(BrokerDecoder.java:75)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.qpid.server.protocol.v0_8.BrokerDecoder.processFrame(BrokerDecoder.java:74)
at org.apache.qpid.codec.AMQDecoder.processInput(AMQDecoder.java:370)
at org.apache.qpid.codec.AMQDecoder.decodeBuffer(AMQDecoder.java:259)
at org.apache.qpid.server.protocol.v0_8.AMQProtocolEngine$2.run(AMQProtocolEngine.java:334)
at org.apache.qpid.server.protocol.v0_8.AMQProtocolEngine$2.run(AMQProtocolEngine.java:311)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:360)
at org.apache.qpid.server.protocol.v0_8.AMQProtocolEngine.received(AMQProtocolEngine.java:310)
at org.apache.qpid.server.protocol.v0_8.AMQProtocolEngine.received(AMQProtocolEngine.java:95)
at org.apache.qpid.server.protocol.MultiVersionProtocolEngine.received(MultiVersionProtocolEngine.java:153)
at org.apache.qpid.server.protocol.MultiVersionProtocolEngine.received(MultiVersionProtocolEngine.java:51)
at org.apache.qpid.transport.network.io.IoReceiver.run(IoReceiver.java:163)
at java.lang.Thread.run(Thread.java:745)
{{noformat}}
> [0-8..0-91] Mishandling of message credit allows consumers to starve
> --------------------------------------------------------------------
>
> Key: QPID-7387
> URL: https://issues.apache.org/jira/browse/QPID-7387
> Project: Qpid
> Issue Type: Bug
> Components: Java Broker
> Affects Versions: 0.32, qpid-java-6.0, qpid-java-6.1
> Reporter: Keith Wall
>
> The {{Pre0_10CreditManager}} mishandles message credit. In some circumstances allows message credit to fall beneath 0. Once this has occurred, messages cease to flow to all consumers associated with the session (messages appear stuck on the queue). Recreating the session (or connection) will allow messages to flow again.
> This problem was reproduced on a 0.32 derivative but it appears the same issue will affect newer releases too.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org