You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by ndmigration <ra...@americantours.com> on 2012/10/23 03:44:39 UTC
Request/Response Model Not Working in Network of Brokers. Please
Help
Hello,
I have set up 3 brokers in network of brokers. BrokerA connects to BrokerB,
BrokerC, and BrokerB connects to BrokerA, BrokerC, and BrokerC connects to
BrokerB, BrokerA. I tried duplex on all the brokers to reduce the
configuration but that didn't help. My set up involves consumers and
producers in request/response model. Producers on client side sends messages
to REQUEST queue and Consumers on server side receive messages, process it
and sends the response to RESPONSE queue where the consumers on client side
are listening. Please note that I'm not using temporary queues for
Request/Response model but physical queues.
The request and response is based on correlationID so that the server
response reaches proper consumer on the client side. The NOB is forwarding
client request messages across the brokers, severs process them and put back
the message on the response queue. However, the consumers are timing out
waiting for the responses.
*Example:*
Consumer1 randomly connects to a broker using failover protocol and sends a
message to BrokerB. BrokerB forwards the message to BrokerC and the message
is processed by the consumer on the server that is listening on BrokerC. The
response message is put back on the RESPONSE queue on BrokerC but not sure
what's happening after that. The consumer on the client side waiting on
BrokerB is timing out. I have set 3 minutes as timeout on receive() method,
and 2 minutes for message TTL. Again, the server immediately process the
messages and so there is no delay in there. I'm suspecting that the messages
in RESPONSE queue within the brokers are not forwarding properly to reach
the consumer which is using correlation selector key. I have also verified
the uniqueness of correlationid so that no other consumer can consume the
message other than the intended consumer.
I have tried all suggestions: making duplex to true, ProducerControl to
false, allocated more memory, changed networkTTL to 6, added
receiveWhenNoConsumers=true flag with useCache=false and auditEnable=false.
I've been fighting with this for the past few days before seeking help.
*One of my broker config:*
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry topic=">" producerFlowControl="true"
memoryLimit="10mb">
<pendingSubscriberPolicy>
<vmCursor />
</pendingSubscriberPolicy>
</policyEntry>
<policyEntry queue=">" producerFlowControl="false" memoryLimit="10mb"
enableAudit="false" useCache="false">
<networkBridgeFilterFactory>
<conditionalNetworkBridgeFilterFactory replayWhenNoConsumers="true" />
</networkBridgeFilterFactory>
<deadLetterStrategy>
<sharedDeadLetterStrategy processExpired="false" />
</deadLetterStrategy>
</policyEntry>
</policyEntries>
</policyMap>
</destinationPolicy>
<networkConnectors>
<networkConnector name="queuesOnly" networkTTL="6"
uri="static:(tcp://10.0.0.178:61616,tcp://10.0.0.179:61616)"
conduitSubscriptions="false">
<dynamicallyIncludedDestinations>
<queue physicalName=">"/>
</dynamicallyIncludedDestinations>
</networkConnector>
<networkConnector name="topicsOnly" networkTTL="6"
uri="static:(tcp://10.0.0.178:61616,tcp://10.0.0.179:61616)"
conduitSubscriptions="true">
</networkConnector>
</networkConnectors>
<transportConnectors>
<transportConnector name="openwire" uri="tcp://10.0.0.177:61616"
/>
</transportConnectors>
Thanks,
Ragu
--
View this message in context: http://activemq.2283324.n4.nabble.com/Request-Response-Model-Not-Working-in-Network-of-Brokers-Please-Help-tp4658071.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Request/Response Model Not Working in Network of Brokers. Please Help
Posted by Gaurav Sharma <ga...@gmail.com>.
Some points/observations:
1. Do you have NOB config for HA reasons only? Because if you are
immediately consuming published messages, what's the benefit of paying
the price of broker-to-broker message transfer.. there are other means
to achieve availability and message durability without having to deal
with "replication lag" type issues.
2. That said, your config looks okay: replayWhenNoConsumers set,
enableAudit unset. So, it should work okay on paper. Please give it
some more memory (your current memoryLimit is 10mb)
3. From your debug logs, why is the JMSReplyTo header set to null? And
some of the messages are persistent as well while some are
non-persistent - is that by design?
4. You might also want to consider testing the Master/Slave NOB
option. See https://issues.apache.org/jira/browse/AMQ-3564
On Tue, Oct 23, 2012 at 11:18 PM, ndmigration
<ra...@americantours.com> wrote:
> Can somebody help me on this? We are planning to go to production in another
> few days and this issue is a blocker. Please let me know if you need any
> more information.
>
> Thanks,
> Raghu
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/Request-Response-Model-Not-Working-in-Network-of-Brokers-Please-Help-tp4658071p4658152.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Request/Response Model Not Working in Network of Brokers.
Please Help
Posted by ndmigration <ra...@americantours.com>.
Can somebody help me on this? We are planning to go to production in another
few days and this issue is a blocker. Please let me know if you need any
more information.
Thanks,
Raghu
--
View this message in context: http://activemq.2283324.n4.nabble.com/Request-Response-Model-Not-Working-in-Network-of-Brokers-Please-Help-tp4658071p4658152.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Request/Response Model Not Working in Network of Brokers.
Please Help
Posted by ndmigration <ra...@americantours.com>.
Dear Gary,
Thanks for your response. I have enabled the logging and sent a message. The
consumer sent the message to BrokerB which was forwarded to BrokerC and
processed. The response was put back on BrokerC which forwarded the message
to BrokerB and later BrokerA. However, I can see lot of "already routed
through this broker once" messages in the log in all the brokers till the
message expires after 2 minutes. When the response was forwarded to BrokerB
from BrokerC where the client was listening (BrokerB), the BrokerB tries to
forward the message to BrokerA or BrokerC instead of letting it consume by
the consumer. After much hoping around, the message got finally timed out
and removed from the queue leading the consumer to wait for 3 minutes and
time out.
I'm adding some important logs from three brokers in the order the response
was put back and forwarded:
Note: BrokerA = ValhallaMQ, BrokerB = MidgardMQ , and BrokerC = JotunheimMQ,
and Selector Key/Correlation Id = 7430d0bb-cae4-47b5-b0b9-710b5b79e6d0
*BrokerC:*
2012-10-23 10:03:22,610 | DEBUG | JotunheimMQ Ignoring sub from MidgardMQ,
destination queue://PAYMENT_RESPONSE is not permiited :ConsumerInfo
{commandId = 5, responseRequired = true, consumerId =
ID:thor.amt.com-40176-1351011796558-1:7:1:1, destination =
queue://PAYMENT_RESPONSE, prefetchSize = 1000, maximumPendingMessageLimit =
0, browser = false, dispatchAsync = true, selector = PAYMENT_REQUEST_KEY =
'7430d0bb-cae4-47b5-b0b9-710b5b79e6d0', subscriptionName = null, noLocal =
false, exclusive = false, retroactive = false, priority = 0, brokerPath =
null, optimizedAcknowledge = false, noRangeAcks = false, additionalPredicate
= null} | org.apache.activemq.network.DemandForwardingBridgeSupport |
ActiveMQ Transport: tcp:///10.0.0.178:61616@46883
2012-10-23 10:03:22,611 | DEBUG | queue://PAYMENT_RESPONSE add sub:
QueueSubscription:
consumer=ID:jotunheim.amt.com-50445-1351010815753-2:2:1:31, destinations=0,
dispatched=0, delivered=0, pending=0, dequeues: 41, dispatched: 41,
inflight: 0 | org.apache.activemq.broker.region.Queue | ActiveMQ Transport:
tcp:///10.0.0.178:61616@46881
2012-10-23 10:03:22,613 | DEBUG | JotunheimMQ Ignoring sub from ValhallaMQ,
destination queue://PAYMENT_RESPONSE is not permiited :ConsumerInfo
{commandId = 96, responseRequired = false, consumerId =
ID:valhalla.amt.com-47978-1351010795198-23:2:1:42, destination =
queue://PAYMENT_RESPONSE, prefetchSize = 1000, maximumPendingMessageLimit =
0, browser = false, dispatchAsync = true, selector = PAYMENT_REQUEST_KEY =
'7430d0bb-cae4-47b5-b0b9-710b5b79e6d0', subscriptionName = null, noLocal =
false, exclusive = false, retroactive = false, priority = 0, brokerPath =
[ID:midgard.amt.com-34058-1351010814364-0:1], optimizedAcknowledge = false,
noRangeAcks = false, additionalPredicate =
org.apache.activemq.command.NetworkBridgeFilter@1261824e} |
org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: tcp:///10.0.0.177:61616@41995
2012-10-23 10:03:22,613 | DEBUG | JotunheimMQ bridged sub on
vm://JotunheimMQ#0 from MidgardMQ : ConsumerInfo {commandId = 5,
responseRequired = true, consumerId =
ID:thor.amt.com-40176-1351011796558-1:7:1:1, destination =
queue://PAYMENT_RESPONSE, prefetchSize = 1000, maximumPendingMessageLimit =
0, browser = false, dispatchAsync = true, selector = PAYMENT_REQUEST_KEY =
'7430d0bb-cae4-47b5-b0b9-710b5b79e6d0', subscriptionName = null, noLocal =
false, exclusive = false, retroactive = false, priority = 0, brokerPath =
null, optimizedAcknowledge = false, noRangeAcks = false, additionalPredicate
= null} | org.apache.activemq.network.DemandForwardingBridgeSupport |
ActiveMQ Transport: tcp:///10.0.0.178:61616@46881
2012-10-23 10:03:22,613 | DEBUG | JotunheimMQ adding consumer:
ID:jotunheim.amt.com-50445-1351010815753-3:2:1:37 for destination:
queue://PAYMENT_RESPONSE | org.apache.activemq.broker.region.AbstractRegion
| ActiveMQ Transport: tcp:///10.0.0.177:61616@41993
2012-10-23 10:03:22,623 | DEBUG | JotunheimMQ bridged sub on
vm://JotunheimMQ#2 from ValhallaMQ : ConsumerInfo {commandId = 96,
responseRequired = false, consumerId =
ID:valhalla.amt.com-47978-1351010795198-23:2:1:42, destination =
queue://PAYMENT_RESPONSE, prefetchSize = 1000, maximumPendingMessageLimit =
0, browser = false, dispatchAsync = true, selector = PAYMENT_REQUEST_KEY =
'7430d0bb-cae4-47b5-b0b9-710b5b79e6d0', subscriptionName = null, noLocal =
false, exclusive = false, retroactive = false, priority = 0, brokerPath =
[ID:midgard.amt.com-34058-1351010814364-0:1], optimizedAcknowledge = false,
noRangeAcks = false, additionalPredicate =
org.apache.activemq.command.NetworkBridgeFilter@3c62de83} |
org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: tcp:///10.0.0.177:61616@41993
2012-10-23 10:03:23,677 | DEBUG | bridging (JotunheimMQ -> MidgardMQ)
ID:frigga.amt.com-46716-1351011227838-1:1:1:1:14, consumer:
ID:jotunheim.amt.com-50445-1351010815753-2:2:1:31, destination
queue://PAYMENT_RESPONSE, brokerPath:
[ID:jotunheim.amt.com-50445-1351010815753-0:1], message:
ActiveMQObjectMessage {commandId = 34, responseRequired = true, messageId =
ID:frigga.amt.com-46716-1351011227838-1:1:1:1:14, originalDestination =
null, originalTransactionId = null, producerId =
ID:jotunheim.amt.com-50445-1351010815753-2:1:1:1, destination =
queue://PAYMENT_RESPONSE, transactionId = null, expiration = 1351011923666,
timestamp = 1351011803666, arrival = 0, brokerInTime = 1351011803674,
brokerOutTime = 1351011803677, correlationId =
7430d0bb-cae4-47b5-b0b9-710b5b79e6d0, replyTo = null, persistent = true,
type = null, priority = 4, groupID = null, groupSequence = 0,
targetConsumerId = null, compressed = false, userID = null, content =
org.apache.activemq.util.ByteSequence@37bc2e0, marshalledProperties =
org.apache.activemq.util.ByteSequence@323701e9, dataStructure = null,
redeliveryCounter = 0, size = 1779, properties =
{PAYMENT_REQUEST_KEY=7430d0bb-cae4-47b5-b0b9-710b5b79e6d0},
readOnlyProperties = false, readOnlyBody = false, droppable = false} |
org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
BrokerService[JotunheimMQ] Task-28
2012-10-23 10:05:22,627 | DEBUG | JotunheimMQ remove local subscription for
remote ID:thor.amt.com-40176-1351011796558-1:7:1:1 |
org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: tcp:///10.0.0.178:61616@46881
2012-10-23 10:05:22,627 | DEBUG | JotunheimMQ removed sub on
vm://JotunheimMQ#0 from MidgardMQ : ConsumerInfo {commandId = 5,
responseRequired = true, consumerId =
ID:thor.amt.com-40176-1351011796558-1:7:1:1, destination =
queue://PAYMENT_RESPONSE, prefetchSize = 1000, maximumPendingMessageLimit =
0, browser = false, dispatchAsync = true, selector = PAYMENT_REQUEST_KEY =
'7430d0bb-cae4-47b5-b0b9-710b5b79e6d0', subscriptionName = null, noLocal =
false, exclusive = false, retroactive = false, priority = 0, brokerPath =
[ID:midgard.amt.com-34058-1351010814364-0:1], optimizedAcknowledge = false,
noRangeAcks = false, additionalPredicate = null} |
org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: tcp:///10.0.0.178:61616@46881
2012-10-23 10:05:22,629 | DEBUG | JotunheimMQ removed sub on
vm://JotunheimMQ#2 from ValhallaMQ : ConsumerInfo {commandId = 96,
responseRequired = false, consumerId =
ID:valhalla.amt.com-47978-1351010795198-23:2:1:42, destination =
queue://PAYMENT_RESPONSE, prefetchSize = 1000, maximumPendingMessageLimit =
0, browser = false, dispatchAsync = true, selector = PAYMENT_REQUEST_KEY =
'7430d0bb-cae4-47b5-b0b9-710b5b79e6d0', subscriptionName = null, noLocal =
false, exclusive = false, retroactive = false, priority = 0, brokerPath =
[ID:midgard.amt.com-34058-1351010814364-0:1,
ID:valhalla.amt.com-47978-1351010795198-0:1], optimizedAcknowledge = false,
noRangeAcks = false, additionalPredicate = null} |
org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: tcp:///10.0.0.177:61616@41993
*BrokerB:*
2012-10-23 10:03:22,670 | DEBUG | MidgardMQ Ignoring sub from JotunheimMQ,
already routed through this broker once : ConsumerInfo {commandId = 81,
responseRequired = false, consumerId =
ID:jotunheim.amt.com-50445-1351010815753-2:2:1:31, destination =
queue://PAYMENT_RESPONSE, prefetchSize = 1000, maximumPendingMessageLimit =
0, browser = false, dispatchAsync = true, selector = PAYMENT_REQUEST_KEY =
'7430d0bb-cae4-47b5-b0b9-710b5b79e6d0', subscriptionName = null, noLocal =
false, exclusive = false, retroactive = false, priority = 0, brokerPath =
[ID:midgard.amt.com-34058-1351010814364-0:1], optimizedAcknowledge = false,
noRangeAcks = false, additionalPredicate =
org.apache.activemq.command.NetworkBridgeFilter@6309b022} |
org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: tcp:///10.0.0.179:61616@35360
2012-10-23 10:03:22,670 | DEBUG | MidgardMQ Ignoring sub from ValhallaMQ,
already routed through this broker once : ConsumerInfo {commandId = 96,
responseRequired = false, consumerId =
ID:valhalla.amt.com-47978-1351010795198-23:2:1:42, destination =
queue://PAYMENT_RESPONSE, prefetchSize = 1000, maximumPendingMessageLimit =
0, browser = false, dispatchAsync = true, selector = PAYMENT_REQUEST_KEY =
'7430d0bb-cae4-47b5-b0b9-710b5b79e6d0', subscriptionName = null, noLocal =
false, exclusive = false, retroactive = false, priority = 0, brokerPath =
[ID:midgard.amt.com-34058-1351010814364-0:1], optimizedAcknowledge = false,
noRangeAcks = false, additionalPredicate =
org.apache.activemq.command.NetworkBridgeFilter@110abafa} |
org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: tcp:///10.0.0.177:61616@60343
2012-10-23 10:03:22,676 | DEBUG | bridging (MidgardMQ -> ValhallaMQ)
ID:thor.amt.com-40176-1351011796558-1:7:1:1:1, consumer:
ID:midgard.amt.com-34058-1351010814364-3:2:1:2, destination
queue://PAYMENT_REQUEST, brokerPath:
[ID:midgard.amt.com-34058-1351010814364-0:1], message: ActiveMQObjectMessage
{commandId = 6, responseRequired = false, messageId =
ID:thor.amt.com-40176-1351011796558-1:7:1:1:1, originalDestination = null,
originalTransactionId = null, producerId =
ID:midgard.amt.com-34058-1351010814364-3:1:1:1, destination =
queue://PAYMENT_REQUEST, transactionId = null, expiration = 1351011892821,
timestamp = 1351011802821, arrival = 0, brokerInTime = 1351011802675,
brokerOutTime = 1351011802675, correlationId =
7430d0bb-cae4-47b5-b0b9-710b5b79e6d0, replyTo = null, persistent = false,
type = null, priority = 4, groupID = null, groupSequence = 0,
targetConsumerId = null, compressed = false, userID = null, content =
org.apache.activemq.util.ByteSequence@313239d8, marshalledProperties = null,
dataStructure = null, redeliveryCounter = 0, size = 2147, properties = null,
readOnlyProperties = false, readOnlyBody = false, droppable = false} |
org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
BrokerService[MidgardMQ] Task-35
2012-10-23 10:03:22,680 | DEBUG | MidgardMQ Ignoring sub from JotunheimMQ,
already routed through this broker once : ConsumerInfo {commandId = 103,
responseRequired = false, consumerId =
ID:jotunheim.amt.com-50445-1351010815753-3:2:1:37, destination =
queue://PAYMENT_RESPONSE, prefetchSize = 1000, maximumPendingMessageLimit =
0, browser = false, dispatchAsync = true, selector = PAYMENT_REQUEST_KEY =
'7430d0bb-cae4-47b5-b0b9-710b5b79e6d0', subscriptionName = null, noLocal =
false, exclusive = false, retroactive = false, priority = 0, brokerPath =
[ID:midgard.amt.com-34058-1351010814364-0:1,
ID:valhalla.amt.com-47978-1351010795198-0:1], optimizedAcknowledge = false,
noRangeAcks = false, additionalPredicate =
org.apache.activemq.command.NetworkBridgeFilter@126c0233} |
org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: tcp:///10.0.0.179:61616@35360
2012-10-23 10:03:23,740 | DEBUG | bridging (MidgardMQ -> ValhallaMQ)
ID:frigga.amt.com-46716-1351011227838-1:1:1:1:14, consumer:
ID:midgard.amt.com-34058-1351010814364-3:2:1:43, destination
queue://PAYMENT_RESPONSE, brokerPath:
[ID:jotunheim.amt.com-50445-1351010815753-0:1,
ID:midgard.amt.com-34058-1351010814364-0:1], message: ActiveMQObjectMessage
{commandId = 29, responseRequired = true, messageId =
ID:frigga.amt.com-46716-1351011227838-1:1:1:1:14, originalDestination =
null, originalTransactionId = null, producerId =
ID:midgard.amt.com-34058-1351010814364-3:1:1:1, destination =
queue://PAYMENT_RESPONSE, transactionId = null, expiration = 1351011923666,
timestamp = 1351011803666, arrival = 0, brokerInTime = 1351011803736,
brokerOutTime = 1351011803739, correlationId =
7430d0bb-cae4-47b5-b0b9-710b5b79e6d0, replyTo = null, persistent = true,
type = null, priority = 4, groupID = null, groupSequence = 0,
targetConsumerId = null, compressed = false, userID = null, content =
org.apache.activemq.util.ByteSequence@522f3bfa, marshalledProperties =
org.apache.activemq.util.ByteSequence@1b0ac43d, dataStructure = null,
redeliveryCounter = 0, size = 1779, properties =
{PAYMENT_REQUEST_KEY=7430d0bb-cae4-47b5-b0b9-710b5b79e6d0},
readOnlyProperties = false, readOnlyBody = false, droppable = false} |
org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
BrokerService[MidgardMQ] Task-32
*BrokerA:*
2012-10-23 10:03:22,712 | DEBUG | ValhallaMQ bridged sub on
vm://ValhallaMQ#42 from MidgardMQ : ConsumerInfo {commandId = 5,
responseRequired = true, consumerId =
ID:thor.amt.com-40176-1351011796558-1:7:1:1, destination =
queue://PAYMENT_RESPONSE, prefetchSize = 1000, maximumPendingMessageLimit =
0, browser = false, dispatchAsync = true, selector = PAYMENT_REQUEST_KEY =
'7430d0bb-cae4-47b5-b0b9-710b5b79e6d0', subscriptionName = null, noLocal =
false, exclusive = false, retroactive = false, priority = 0, brokerPath =
null, optimizedAcknowledge = false, noRangeAcks = false, additionalPredicate
= null} | org.apache.activemq.network.DemandForwardingBridgeSupport |
ActiveMQ Transport: tcp:///10.0.0.178:61616@55531
2012-10-23 10:03:22,713 | DEBUG | ValhallaMQ Ignoring sub from JotunheimMQ
as already subscribed to matching destination : ConsumerInfo {commandId =
81, responseRequired = false, consumerId =
ID:jotunheim.amt.com-50445-1351010815753-2:2:1:31, destination =
queue://PAYMENT_RESPONSE, prefetchSize = 1000, maximumPendingMessageLimit =
0, browser = false, dispatchAsync = true, selector = PAYMENT_REQUEST_KEY =
'7430d0bb-cae4-47b5-b0b9-710b5b79e6d0', subscriptionName = null, noLocal =
false, exclusive = false, retroactive = false, priority = 0, brokerPath =
[ID:midgard.amt.com-34058-1351010814364-0:1], optimizedAcknowledge = false,
noRangeAcks = false, additionalPredicate =
org.apache.activemq.command.NetworkBridgeFilter@352096f2} |
org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: tcp:///10.0.0.179:61616@40611
2012-10-23 10:03:22,717 | DEBUG | ValhallaMQ bridged sub on
vm://ValhallaMQ#46 from JotunheimMQ : ConsumerInfo {commandId = 81,
responseRequired = false, consumerId =
ID:jotunheim.amt.com-50445-1351010815753-2:2:1:31, destination =
queue://PAYMENT_RESPONSE, prefetchSize = 1000, maximumPendingMessageLimit =
0, browser = false, dispatchAsync = true, selector = PAYMENT_REQUEST_KEY =
'7430d0bb-cae4-47b5-b0b9-710b5b79e6d0', subscriptionName = null, noLocal =
false, exclusive = false, retroactive = false, priority = 0, brokerPath =
[ID:midgard.amt.com-34058-1351010814364-0:1], optimizedAcknowledge = false,
noRangeAcks = false, additionalPredicate =
org.apache.activemq.command.NetworkBridgeFilter@3b29eba3} |
org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: tcp:///10.0.0.179:61616@40612
2012-10-23 10:03:22,722 | DEBUG | ValhallaMQ Ignoring sub from JotunheimMQ,
already routed through this broker once : ConsumerInfo {commandId = 103,
responseRequired = false, consumerId =
ID:jotunheim.amt.com-50445-1351010815753-3:2:1:37, destination =
queue://PAYMENT_RESPONSE, prefetchSize = 1000, maximumPendingMessageLimit =
0, browser = false, dispatchAsync = true, selector = PAYMENT_REQUEST_KEY =
'7430d0bb-cae4-47b5-b0b9-710b5b79e6d0', subscriptionName = null, noLocal =
false, exclusive = false, retroactive = false, priority = 0, brokerPath =
[ID:midgard.amt.com-34058-1351010814364-0:1,
ID:valhalla.amt.com-47978-1351010795198-0:1], optimizedAcknowledge = false,
noRangeAcks = false, additionalPredicate =
org.apache.activemq.command.NetworkBridgeFilter@4f4dfa6a} |
org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: tcp:///10.0.0.179:61616@40611
2012-10-23 10:05:22,727 | DEBUG | ValhallaMQ removed sub on
vm://ValhallaMQ#42 from MidgardMQ : ConsumerInfo {commandId = 5,
responseRequired = true, consumerId =
ID:thor.amt.com-40176-1351011796558-1:7:1:1, destination =
queue://PAYMENT_RESPONSE, prefetchSize = 1000, maximumPendingMessageLimit =
0, browser = false, dispatchAsync = true, selector = PAYMENT_REQUEST_KEY =
'7430d0bb-cae4-47b5-b0b9-710b5b79e6d0', subscriptionName = null, noLocal =
false, exclusive = false, retroactive = false, priority = 0, brokerPath =
[ID:midgard.amt.com-34058-1351010814364-0:1], optimizedAcknowledge = false,
noRangeAcks = false, additionalPredicate = null} |
org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: tcp:///10.0.0.178:61616@55531
2012-10-23 10:05:22,732 | DEBUG | ValhallaMQ removed sub on
vm://ValhallaMQ#46 from JotunheimMQ : ConsumerInfo {commandId = 81,
responseRequired = false, consumerId =
ID:jotunheim.amt.com-50445-1351010815753-2:2:1:31, destination =
queue://PAYMENT_RESPONSE, prefetchSize = 1000, maximumPendingMessageLimit =
0, browser = false, dispatchAsync = true, selector = PAYMENT_REQUEST_KEY =
'7430d0bb-cae4-47b5-b0b9-710b5b79e6d0', subscriptionName = null, noLocal =
false, exclusive = false, retroactive = false, priority = 0, brokerPath =
[ID:midgard.amt.com-34058-1351010814364-0:1,
ID:jotunheim.amt.com-50445-1351010815753-0:1], optimizedAcknowledge = false,
noRangeAcks = false, additionalPredicate = null} |
org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ
Transport: tcp:///10.0.0.179:61616@40612
Thanks,
Raghu
--
View this message in context: http://activemq.2283324.n4.nabble.com/Request-Response-Model-Not-Working-in-Network-of-Brokers-Please-Help-tp4658071p4658121.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Request/Response Model Not Working in Network of Brokers. Please Help
Posted by Gary Tully <ga...@gmail.com>.
enable debug logging on all brokers and produce a single message. you
should be able to trace the progress of that message till it expires,
gets consumed or gets orphaned.
Once you know what happens we will be able to figure out the next step.
http://activemq.apache.org/how-do-i-enable-debug-logging.html
On 23 October 2012 02:44, ndmigration <ra...@americantours.com> wrote:
> Hello,
>
> I have set up 3 brokers in network of brokers. BrokerA connects to BrokerB,
> BrokerC, and BrokerB connects to BrokerA, BrokerC, and BrokerC connects to
> BrokerB, BrokerA. I tried duplex on all the brokers to reduce the
> configuration but that didn't help. My set up involves consumers and
> producers in request/response model. Producers on client side sends messages
> to REQUEST queue and Consumers on server side receive messages, process it
> and sends the response to RESPONSE queue where the consumers on client side
> are listening. Please note that I'm not using temporary queues for
> Request/Response model but physical queues.
>
> The request and response is based on correlationID so that the server
> response reaches proper consumer on the client side. The NOB is forwarding
> client request messages across the brokers, severs process them and put back
> the message on the response queue. However, the consumers are timing out
> waiting for the responses.
>
> *Example:*
>
> Consumer1 randomly connects to a broker using failover protocol and sends a
> message to BrokerB. BrokerB forwards the message to BrokerC and the message
> is processed by the consumer on the server that is listening on BrokerC. The
> response message is put back on the RESPONSE queue on BrokerC but not sure
> what's happening after that. The consumer on the client side waiting on
> BrokerB is timing out. I have set 3 minutes as timeout on receive() method,
> and 2 minutes for message TTL. Again, the server immediately process the
> messages and so there is no delay in there. I'm suspecting that the messages
> in RESPONSE queue within the brokers are not forwarding properly to reach
> the consumer which is using correlation selector key. I have also verified
> the uniqueness of correlationid so that no other consumer can consume the
> message other than the intended consumer.
>
> I have tried all suggestions: making duplex to true, ProducerControl to
> false, allocated more memory, changed networkTTL to 6, added
> receiveWhenNoConsumers=true flag with useCache=false and auditEnable=false.
> I've been fighting with this for the past few days before seeking help.
>
> *One of my broker config:*
>
> <destinationPolicy>
> <policyMap>
> <policyEntries>
> <policyEntry topic=">" producerFlowControl="true"
> memoryLimit="10mb">
> <pendingSubscriberPolicy>
> <vmCursor />
> </pendingSubscriberPolicy>
> </policyEntry>
> <policyEntry queue=">" producerFlowControl="false" memoryLimit="10mb"
> enableAudit="false" useCache="false">
>
> <networkBridgeFilterFactory>
> <conditionalNetworkBridgeFilterFactory replayWhenNoConsumers="true" />
> </networkBridgeFilterFactory>
> <deadLetterStrategy>
> <sharedDeadLetterStrategy processExpired="false" />
> </deadLetterStrategy>
> </policyEntry>
> </policyEntries>
> </policyMap>
> </destinationPolicy>
>
>
> <networkConnectors>
> <networkConnector name="queuesOnly" networkTTL="6"
> uri="static:(tcp://10.0.0.178:61616,tcp://10.0.0.179:61616)"
> conduitSubscriptions="false">
> <dynamicallyIncludedDestinations>
> <queue physicalName=">"/>
> </dynamicallyIncludedDestinations>
> </networkConnector>
> <networkConnector name="topicsOnly" networkTTL="6"
> uri="static:(tcp://10.0.0.178:61616,tcp://10.0.0.179:61616)"
> conduitSubscriptions="true">
> </networkConnector>
> </networkConnectors>
>
>
> <transportConnectors>
> <transportConnector name="openwire" uri="tcp://10.0.0.177:61616"
> />
> </transportConnectors>
>
> Thanks,
> Ragu
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/Request-Response-Model-Not-Working-in-Network-of-Brokers-Please-Help-tp4658071.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
--
http://redhat.com
http://blog.garytully.com
Re: Request/Response Model Not Working in Network of Brokers. Please Help
Posted by Gary Tully <ga...@gmail.com>.
from your network connectors config, remove:
<networkConnector name="topicsOnly" networkTTL="6"
uri="static:(tcp://10.0.0.178:61616,tcp://10.0.0.179:61616)"
conduitSubscriptions="true">
b/c this is bridging all destinations as there are no excludes, so you
have two paths to each broker which will complicate it a bit.
On the ttl, this should be the minimum number of hops you require, I
don't see why you want 6 in there. 3 or 2 should be sufficient.
What is your intention with three brokers? If the intention is to
build a mesh topology, then all brokers can talk directly to each
other and a ttl of 1 is sufficient.
If you are building a ring, then a message can have multiple routes,
but ttl=2 is sufficient.
If your local consumer and network consumers are competing, then use
decreaseNetworkConsumerPriority=true in the network connector so that
local consumers get priority.
It looks like you need to go back to first principals and understand
what is being configured and the intent of every option that you
choose.
Maybe have a read of
http://blog.garytully.com/2012/07/activemq-broker-networks-think-demand.html
to get a better understanding of the principals.
hope this helps.
On 23 October 2012 02:44, ndmigration <ra...@americantours.com> wrote:
> Hello,
>
> I have set up 3 brokers in network of brokers. BrokerA connects to BrokerB,
> BrokerC, and BrokerB connects to BrokerA, BrokerC, and BrokerC connects to
> BrokerB, BrokerA. I tried duplex on all the brokers to reduce the
> configuration but that didn't help. My set up involves consumers and
> producers in request/response model. Producers on client side sends messages
> to REQUEST queue and Consumers on server side receive messages, process it
> and sends the response to RESPONSE queue where the consumers on client side
> are listening. Please note that I'm not using temporary queues for
> Request/Response model but physical queues.
>
> The request and response is based on correlationID so that the server
> response reaches proper consumer on the client side. The NOB is forwarding
> client request messages across the brokers, severs process them and put back
> the message on the response queue. However, the consumers are timing out
> waiting for the responses.
>
> *Example:*
>
> Consumer1 randomly connects to a broker using failover protocol and sends a
> message to BrokerB. BrokerB forwards the message to BrokerC and the message
> is processed by the consumer on the server that is listening on BrokerC. The
> response message is put back on the RESPONSE queue on BrokerC but not sure
> what's happening after that. The consumer on the client side waiting on
> BrokerB is timing out. I have set 3 minutes as timeout on receive() method,
> and 2 minutes for message TTL. Again, the server immediately process the
> messages and so there is no delay in there. I'm suspecting that the messages
> in RESPONSE queue within the brokers are not forwarding properly to reach
> the consumer which is using correlation selector key. I have also verified
> the uniqueness of correlationid so that no other consumer can consume the
> message other than the intended consumer.
>
> I have tried all suggestions: making duplex to true, ProducerControl to
> false, allocated more memory, changed networkTTL to 6, added
> receiveWhenNoConsumers=true flag with useCache=false and auditEnable=false.
> I've been fighting with this for the past few days before seeking help.
>
> *One of my broker config:*
>
> <destinationPolicy>
> <policyMap>
> <policyEntries>
> <policyEntry topic=">" producerFlowControl="true"
> memoryLimit="10mb">
> <pendingSubscriberPolicy>
> <vmCursor />
> </pendingSubscriberPolicy>
> </policyEntry>
> <policyEntry queue=">" producerFlowControl="false" memoryLimit="10mb"
> enableAudit="false" useCache="false">
>
> <networkBridgeFilterFactory>
> <conditionalNetworkBridgeFilterFactory replayWhenNoConsumers="true" />
> </networkBridgeFilterFactory>
> <deadLetterStrategy>
> <sharedDeadLetterStrategy processExpired="false" />
> </deadLetterStrategy>
> </policyEntry>
> </policyEntries>
> </policyMap>
> </destinationPolicy>
>
>
> <networkConnectors>
> <networkConnector name="queuesOnly" networkTTL="6"
> uri="static:(tcp://10.0.0.178:61616,tcp://10.0.0.179:61616)"
> conduitSubscriptions="false">
> <dynamicallyIncludedDestinations>
> <queue physicalName=">"/>
> </dynamicallyIncludedDestinations>
> </networkConnector>
> <networkConnector name="topicsOnly" networkTTL="6"
> uri="static:(tcp://10.0.0.178:61616,tcp://10.0.0.179:61616)"
> conduitSubscriptions="true">
> </networkConnector>
> </networkConnectors>
>
>
> <transportConnectors>
> <transportConnector name="openwire" uri="tcp://10.0.0.177:61616"
> />
> </transportConnectors>
>
> Thanks,
> Ragu
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/Request-Response-Model-Not-Working-in-Network-of-Brokers-Please-Help-tp4658071.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
--
http://redhat.com
http://blog.garytully.com