You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by ievanliu <ie...@qq.com> on 2018/08/27 07:08:11 UTC
cannot consume msg by a duplex network connect broke.
version:5.14.0
architecture： Broke A(a 3 instances jdbc cluster) and Broke B(a 3 instances
jdbc cluster), use duplex network connect, from brokeB to brokeA
productor produce messages into brokeA, consumers consume messages from
broke B, bridge queue like queueA,queueB
networkConnector config:
<networkConnectors>
<networkConnector
uri="masterslave:(tcp://172.16.202.62:61616,tcp://172.16.202.65:61616,tcp://172.16.202.68:61616)"
duplex="true" name="bridge_111"></networkConnector>
</networkConnectors>
problem:
early this morning, brokeB happened a master change, by the pre-master lost
the jdbc lock.
broke B2 in the broke B cluster, successfuly became the master, and build
the network connection to broke A.
But, there is a problem that , producer can produce messages to BrokeA, but
consumers cannot consumer message from brokeB.
process:
1、as the log shows, the network bridge has successfully been built。and there
are no other error logs info.
2、I found that, when consumer connect to brokeB to consume queueA, in broke
A, there is no consumers.Normally, here should be a network bridge consumer
to brokeA.maybe this is the reason.
3、restart the broke A and broke B，the problem happened as same
result:
about 1 hours later, the problem disappear automaticly, and I can see there
is a network bridge consumer to brokeA.
who can help me how to troubleshooting this problem?
--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Re: cannot consume msg by a duplex network connect broke.
Posted by ievanliu <ie...@qq.com>.
sorry Tim, I didn't describe clearly,and I have start a new topic to explain
this problem more clearly,please see there:
http://activemq.2283324.n4.nabble.com/no-consumers-for-the-queue-by-a-duplex-network-connector-td4743534.html
thanks a lot.
--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Re: cannot consume msg by a duplex network connect broke.
Posted by Tim Bain <tb...@alumni.duke.edu>.
You previously said that you were statically configuring the network
connectors, but in your last email you talked about advisory topics, which
are only used for dynamically included destinations. Which approach are you
using?
More generally, please post your broker configuration, since that may
explain the behavior you're observing.
Also, are you saying that the network behavior you see is different before
a broker failover than after one? If so, which broker was the one that
failed over?
And to which broker was the consumer connected in each case you described?
Your description of the missing logs in A sound like the consumer is
connected to B, but your paragraph at the end sounded like the opposite. So
a detailed description of how you're interacting with the network of
brokers would help clarify what you're describing.
Tim
On Fri, Sep 14, 2018, 11:23 AM ievanliu <ie...@qq.com> wrote:
> Hi,Tim
>
> this problem happened very often recently.
>
> as run in this architecture： Broke A(a 3 instances jdbc cluster) and Broke
> B(a 3 instances
> jdbc cluster), use duplex network connect, from brokeB to brokeA
>
> once Broker B cluster happened fail-over event(master changed from B1 to
> B2),then the bridge problem happened.
>
> I test that subscribe a queue test_queue to broke B,but in broke A ,there
> even has no queue named test_queue,and there has no advisory topic in broke
> A either
>
> normally,when subscribe a queue,the log is as below:
> [log in brokerA]
> [DEBUG][2018-09-14 16:33:46] localhost adding consumer:
> localhost->bj_172.20.32.111-44717-1536908810883-4:1:1:13 for destination:
>
> queue://amq_admin_test45<--[org.apache.activemq.broker.region.AbstractRegion:338
> addConsumer]
> [DEBUG][2018-09-14 16:33:46] localhost adding destination:
>
> queue://amq_admin_test45<--[org.apache.activemq.broker.region.AbstractRegion:154
> addDestination]
> [DEBUG][2018-09-14 16:33:46] queue://amq_admin_test45, subscriptions=0,
> memory=0%, size=0, pending=0 toPageIn: 0, force:false, Inflight: 0,
> pagedInMessages.size 0, pagedInPendingDispatch.size 0, enqueueCount: 0,
> dequeueCount: 0, memUsage:0,
> maxPageSize:200<--[org.apache.activemq.broker.region.Queue:1913
> doPageInForDispatch]
> [DEBUG][2018-09-14 16:33:46] queue://amq_admin_test45 add sub:
> QueueSubscription:
> consumer=localhost->bj_172.20.32.111-44717-1536908810883-4:1:1:13,
> destinations=0, dispatched=0, delivered=0, pending=0, dequeues: 0,
> dispatched: 0, inflight: 0<--[org.apache.activemq.broker.region.Queue:422
> addSubscription]
> [DEBUG][2018-09-14 16:33:46] queue://amq_admin_test45, subscriptions=1,
> memory=0%, size=0, pending=0 toPageIn: 0, force:false, Inflight: 0,
> pagedInMessages.size 0, pagedInPendingDispatch.size 0, enqueueCount: 0,
> dequeueCount: 0, memUsage:0,
> maxPageSize:200<--[org.apache.activemq.broker.region.Queue:1913
> doPageInForDispatch]
> [DEBUG][2018-09-14 16:33:46] localhost adding destination:
>
> topic://ActiveMQ.Advisory.Consumer.Queue.amq_admin_test45<--[org.apache.activemq.broker.region.AbstractRegion:154
> addDestination]
>
>
> [log in BrokerB]
> [DEBUG][2018-09-14 16:33:46] bj_172.20.32.111 adding consumer:
> ID:cloudlab042-46597-1536908807725-7:22:-1:1 for destination:
>
> queue://amq_admin_test45<--[org.apache.activemq.broker.region.AbstractRegion:338
> addConsumer]
> [DEBUG][2018-09-14 16:33:46] bj_172.20.32.111 adding destination:
>
> queue://amq_admin_test45<--[org.apache.activemq.broker.region.AbstractRegion:154
> addDestination]
> [DEBUG][2018-09-14 16:33:46] queue://amq_admin_test45, subscriptions=0,
> memory=0%, size=0, pending=0 toPageIn: 0, force:false, Inflight: 0,
> pagedInMessages.size 0, pagedInPendingDispatch.size 0, enqueueCount: 0,
> dequeueCount: 0, memUsage:0,
> maxPageSize:200<--[org.apache.activemq.broker.region.Queue:1913
> doPageInForDispatch]
> [DEBUG][2018-09-14 16:33:46] queue://amq_admin_test45 add sub:
> QueueSubscription: consumer=ID:cloudlab042-46597-1536908807725-7:22:-1:1,
> destinations=0, dispatched=0, delivered=0, pending=0, dequeues: 0,
> dispatched: 0, inflight: 0<--[org.apache.activemq.broker.region.Queue:422
> addSubscription]
> [DEBUG][2018-09-14 16:33:46] queue://amq_admin_test45, subscriptions=1,
> memory=0%, size=0, pending=0 toPageIn: 0, force:false, Inflight: 0,
> pagedInMessages.size 0, pagedInPendingDispatch.size 0, enqueueCount: 0,
> dequeueCount: 0, memUsage:0,
> maxPageSize:200<--[org.apache.activemq.broker.region.Queue:1913
> doPageInForDispatch]
> [DEBUG][2018-09-14 16:33:46] bj_172.20.32.111 adding destination:
>
> topic://ActiveMQ.Advisory.Consumer.Queue.amq_admin_test45<--[org.apache.activemq.broker.region.AbstractRegion:154
> addDestination]
> [DEBUG][2018-09-14 16:33:46] bridging (bj_172.20.32.111 -> localhost),
> consumer: bj_172.20.32.111->localhost-46597-1536908807725-6:1:1:1,
> destination: topic://ActiveMQ.Advisory.Consumer.Queue.amq_admin_test45,
> brokerPath: [ID:cloudlab042-46597-1536908807725-0:1], message:
>
> ID:cloudlab042-46597-1536908807725-1:1:0:0:87<--[org.apache.activemq.network.DemandForwardingBridgeSupport:1070
> serviceLocalCommand]
>
>
> but when problem happened,and then I subscribe anther queue in brokeB, I
> found,there has no log of this queue bin BrokeA at all.
> in BrokerB,log is as below:[DEBUG][2018-09-14 22:31:53] bj_172.20.32.109
> adding consumer: ID:bfjd-psc-bcop-vm16-60932-1536934894156-5:2:-1:1 for
> destination:
>
> queue://amq_admin_test_nj<--[org.apache.activemq.broker.region.AbstractRegion:338
> addConsumer]
> [DEBUG][2018-09-14 22:31:53] bj_172.20.32.109 adding destination:
>
> queue://amq_admin_test_nj<--[org.apache.activemq.broker.region.AbstractRegion:154
> addDestination]
> [DEBUG][2018-09-14 22:31:53] queue://amq_admin_test_nj, subscriptions=0,
> memory=0%, size=0, pending=0 toPageIn: 0, force:false, Inflight: 0,
> pagedInMessages.size 0, pagedInPendingDispatch.size 0, enqueueCount: 0,
> dequeueCount: 0, memUsage:0,
> maxPageSize:200<--[org.apache.activemq.broker.region.Queue:1913
> doPageInForDispatch]
> [DEBUG][2018-09-14 22:31:53] queue://amq_admin_test_nj add sub:
> QueueSubscription:
> consumer=ID:bfjd-psc-bcop-vm16-60932-1536934894156-5:2:-1:1,
> destinations=0,
> dispatched=0, delivered=0, pending=0, dequeues: 0, dispatched: 0, inflight:
> 0<--[org.apache.activemq.broker.region.Queue:422 addSubscription]
> [DEBUG][2018-09-14 22:31:53] queue://amq_admin_test_nj, subscriptions=1,
> memory=0%, size=0, pending=0 toPageIn: 0, force:false, Inflight: 0,
> pagedInMessages.size 0, pagedInPendingDispatch.size 0, enqueueCount: 0,
> dequeueCount: 0, memUsage:0,
> maxPageSize:200<--[org.apache.activemq.broker.region.Queue:1913
> doPageInForDispatch]
> [DEBUG][2018-09-14 22:31:53] bj_172.20.32.109 adding destination:
>
> topic://ActiveMQ.Advisory.Consumer.Queue.amq_admin_test_nj<--[org.apache.activemq.broker.region.AbstractRegion:154
> addDestination]
> [DEBUG][2018-09-14 22:32:23] queue://amq_admin_test_nj expiring messages
> ..<--[org.apache.activemq.broker.region.Queue:904 expireMessages]
> [DEBUG][2018-09-14 22:32:23] queue://amq_admin_test_nj expiring messages
> done.<--[org.apache.activemq.broker.region.Queue:910 expireMessages]
>
>
> I can see that this log did not happed:
> "bridging (bj_172.20.32.111 -> localhost), consumer:
> bj_172.20.32.111->localhost-46597-1536908807725-6:1:1:1, destination:
> topic://ActiveMQ.Advisory.Consumer.Queue.amq_admin_test45, brokerPath:
> [ID:cloudlab042-46597-1536908807725-0:1], message"
>
>
> but there are some queues successfully built network connector from BrokeA
> to BrokeB(the queue in BrokeB has a bridge connect as a consumer)
>
> this really puzzle me.
>
>
>
> --
> Sent from:
> http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
>
Re: cannot consume msg by a duplex network connect broke.
Posted by ievanliu <ie...@qq.com>.
Hi,Tim
this problem happened very often recently.
as run in this architecture： Broke A(a 3 instances jdbc cluster) and Broke
B(a 3 instances
jdbc cluster), use duplex network connect, from brokeB to brokeA
once Broker B cluster happened fail-over event(master changed from B1 to
B2),then the bridge problem happened.
I test that subscribe a queue test_queue to broke B,but in broke A ,there
even has no queue named test_queue,and there has no advisory topic in broke
A either
normally,when subscribe a queue,the log is as below:
[log in brokerA]
[DEBUG][2018-09-14 16:33:46] localhost adding consumer:
localhost->bj_172.20.32.111-44717-1536908810883-4:1:1:13 for destination:
queue://amq_admin_test45<--[org.apache.activemq.broker.region.AbstractRegion:338
addConsumer]
[DEBUG][2018-09-14 16:33:46] localhost adding destination:
queue://amq_admin_test45<--[org.apache.activemq.broker.region.AbstractRegion:154
addDestination]
[DEBUG][2018-09-14 16:33:46] queue://amq_admin_test45, subscriptions=0,
memory=0%, size=0, pending=0 toPageIn: 0, force:false, Inflight: 0,
pagedInMessages.size 0, pagedInPendingDispatch.size 0, enqueueCount: 0,
dequeueCount: 0, memUsage:0,
maxPageSize:200<--[org.apache.activemq.broker.region.Queue:1913
doPageInForDispatch]
[DEBUG][2018-09-14 16:33:46] queue://amq_admin_test45 add sub:
QueueSubscription:
consumer=localhost->bj_172.20.32.111-44717-1536908810883-4:1:1:13,
destinations=0, dispatched=0, delivered=0, pending=0, dequeues: 0,
dispatched: 0, inflight: 0<--[org.apache.activemq.broker.region.Queue:422
addSubscription]
[DEBUG][2018-09-14 16:33:46] queue://amq_admin_test45, subscriptions=1,
memory=0%, size=0, pending=0 toPageIn: 0, force:false, Inflight: 0,
pagedInMessages.size 0, pagedInPendingDispatch.size 0, enqueueCount: 0,
dequeueCount: 0, memUsage:0,
maxPageSize:200<--[org.apache.activemq.broker.region.Queue:1913
doPageInForDispatch]
[DEBUG][2018-09-14 16:33:46] localhost adding destination:
topic://ActiveMQ.Advisory.Consumer.Queue.amq_admin_test45<--[org.apache.activemq.broker.region.AbstractRegion:154
addDestination]
[log in BrokerB]
[DEBUG][2018-09-14 16:33:46] bj_172.20.32.111 adding consumer:
ID:cloudlab042-46597-1536908807725-7:22:-1:1 for destination:
queue://amq_admin_test45<--[org.apache.activemq.broker.region.AbstractRegion:338
addConsumer]
[DEBUG][2018-09-14 16:33:46] bj_172.20.32.111 adding destination:
queue://amq_admin_test45<--[org.apache.activemq.broker.region.AbstractRegion:154
addDestination]
[DEBUG][2018-09-14 16:33:46] queue://amq_admin_test45, subscriptions=0,
memory=0%, size=0, pending=0 toPageIn: 0, force:false, Inflight: 0,
pagedInMessages.size 0, pagedInPendingDispatch.size 0, enqueueCount: 0,
dequeueCount: 0, memUsage:0,
maxPageSize:200<--[org.apache.activemq.broker.region.Queue:1913
doPageInForDispatch]
[DEBUG][2018-09-14 16:33:46] queue://amq_admin_test45 add sub:
QueueSubscription: consumer=ID:cloudlab042-46597-1536908807725-7:22:-1:1,
destinations=0, dispatched=0, delivered=0, pending=0, dequeues: 0,
dispatched: 0, inflight: 0<--[org.apache.activemq.broker.region.Queue:422
addSubscription]
[DEBUG][2018-09-14 16:33:46] queue://amq_admin_test45, subscriptions=1,
memory=0%, size=0, pending=0 toPageIn: 0, force:false, Inflight: 0,
pagedInMessages.size 0, pagedInPendingDispatch.size 0, enqueueCount: 0,
dequeueCount: 0, memUsage:0,
maxPageSize:200<--[org.apache.activemq.broker.region.Queue:1913
doPageInForDispatch]
[DEBUG][2018-09-14 16:33:46] bj_172.20.32.111 adding destination:
topic://ActiveMQ.Advisory.Consumer.Queue.amq_admin_test45<--[org.apache.activemq.broker.region.AbstractRegion:154
addDestination]
[DEBUG][2018-09-14 16:33:46] bridging (bj_172.20.32.111 -> localhost),
consumer: bj_172.20.32.111->localhost-46597-1536908807725-6:1:1:1,
destination: topic://ActiveMQ.Advisory.Consumer.Queue.amq_admin_test45,
brokerPath: [ID:cloudlab042-46597-1536908807725-0:1], message:
ID:cloudlab042-46597-1536908807725-1:1:0:0:87<--[org.apache.activemq.network.DemandForwardingBridgeSupport:1070
serviceLocalCommand]
but when problem happened,and then I subscribe anther queue in brokeB, I
found,there has no log of this queue bin BrokeA at all.
in BrokerB,log is as below:[DEBUG][2018-09-14 22:31:53] bj_172.20.32.109
adding consumer: ID:bfjd-psc-bcop-vm16-60932-1536934894156-5:2:-1:1 for
destination:
queue://amq_admin_test_nj<--[org.apache.activemq.broker.region.AbstractRegion:338
addConsumer]
[DEBUG][2018-09-14 22:31:53] bj_172.20.32.109 adding destination:
queue://amq_admin_test_nj<--[org.apache.activemq.broker.region.AbstractRegion:154
addDestination]
[DEBUG][2018-09-14 22:31:53] queue://amq_admin_test_nj, subscriptions=0,
memory=0%, size=0, pending=0 toPageIn: 0, force:false, Inflight: 0,
pagedInMessages.size 0, pagedInPendingDispatch.size 0, enqueueCount: 0,
dequeueCount: 0, memUsage:0,
maxPageSize:200<--[org.apache.activemq.broker.region.Queue:1913
doPageInForDispatch]
[DEBUG][2018-09-14 22:31:53] queue://amq_admin_test_nj add sub:
QueueSubscription:
consumer=ID:bfjd-psc-bcop-vm16-60932-1536934894156-5:2:-1:1, destinations=0,
dispatched=0, delivered=0, pending=0, dequeues: 0, dispatched: 0, inflight:
0<--[org.apache.activemq.broker.region.Queue:422 addSubscription]
[DEBUG][2018-09-14 22:31:53] queue://amq_admin_test_nj, subscriptions=1,
memory=0%, size=0, pending=0 toPageIn: 0, force:false, Inflight: 0,
pagedInMessages.size 0, pagedInPendingDispatch.size 0, enqueueCount: 0,
dequeueCount: 0, memUsage:0,
maxPageSize:200<--[org.apache.activemq.broker.region.Queue:1913
doPageInForDispatch]
[DEBUG][2018-09-14 22:31:53] bj_172.20.32.109 adding destination:
topic://ActiveMQ.Advisory.Consumer.Queue.amq_admin_test_nj<--[org.apache.activemq.broker.region.AbstractRegion:154
addDestination]
[DEBUG][2018-09-14 22:32:23] queue://amq_admin_test_nj expiring messages
..<--[org.apache.activemq.broker.region.Queue:904 expireMessages]
[DEBUG][2018-09-14 22:32:23] queue://amq_admin_test_nj expiring messages
done.<--[org.apache.activemq.broker.region.Queue:910 expireMessages]
I can see that this log did not happed:
"bridging (bj_172.20.32.111 -> localhost), consumer:
bj_172.20.32.111->localhost-46597-1536908807725-6:1:1:1, destination:
topic://ActiveMQ.Advisory.Consumer.Queue.amq_admin_test45, brokerPath:
[ID:cloudlab042-46597-1536908807725-0:1], message"
but there are some queues successfully built network connector from BrokeA
to BrokeB(the queue in BrokeB has a bridge connect as a consumer)
this really puzzle me.
--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Re: cannot consume msg by a duplex network connect broke.
Posted by Tim Bain <tb...@alumni.duke.edu>.
I typically find that dynamically including the destinations is a more
flexible approach rather than statically including them (among other
things, you can use wildcards so you don't have to know the names at
configuration time); would that solve the problem you're hitting with the
temp queues? If not, can you describe the configuration you're using and
the behavior you're seeing?
Make sure you set bridgeTempDestinations=true, as described in
http://activemq.apache.org/networks-of-brokers.html.
Tim
On Sep 13, 2018 3:28 AM, "ievanliu" <ie...@qq.com> wrote:
Hi,Tim
This problem happened twice recently...
I try to use static bridge, it really worded,and solve the problem.but we
steel use temp-queue to use request-reply mode by camel,and this need
translate message between different broke,so I can not use static bridge.
I found another this that, when the problem happened , I restart the broker
,but the problem is steel as same.In the end, I shutdown all the broker,and
then start Broke A,later,start Broker B.By this, the problem disappeared.
please help .
--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Re: cannot consume msg by a duplex network connect broke.
Posted by ievanliu <ie...@qq.com>.
Hi,Tim
This problem happened twice recently...
I try to use static bridge, it really worded,and solve the problem.but we
steel use temp-queue to use request-reply mode by camel,and this need
translate message between different broke,so I can not use static bridge.
I found another this that, when the problem happened , I restart the broker
,but the problem is steel as same.In the end, I shutdown all the broker,and
then start Broke A,later,start Broker B.By this, the problem disappeared.
please help .
--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Re: cannot consume msg by a duplex network connect broke.
Posted by Tim Bain <tb...@alumni.duke.edu>.
If you're not explicitly including destinations to be propagated across the
bridge, this is expected behavior. Configure destinations to include (I
recommend dynamic inclusion) and give it another try.
Tim
On Wed, Aug 29, 2018, 1:33 AM ievanliu <ie...@qq.com> wrote:
> Hi,Tim.
> thank you very much for your help.
> here are some more information:
>
> 1.the network connect configuration is the same as what we use in this
> problem,so most values should be defaults value.
> 2.these days, I compared the log very carefully,and I found that, when the
> problem happens, broke did not create a bridge-consumer.
>
> but why that happened?
> please help me?
>
> ------------------------------------------------------
> test case:
> produce messages into broke A,and consume messages from broke B.broke B
> config duplex network connect to Broke A
> ------------------------------------------------------
> #####when the problem happen,the log is as below:
> [broke A]
> [DEBUG][2018-08-21 10:05:24] brokeA adding destination:
>
> queue://amq_admin_test<--[org.apache.activemq.broker.region.AbstractRegion:154
> addDestination]
> [DEBUG][2018-08-21 10:05:24] queue://amq_admin_test, subscriptions=0,
> memory=0%, size=0, pending=0 toPageIn: 0, force:false, Inflight: 0,
> pagedInMessages.size 0, pagedInPendingDispatch.size 0, enqueueCount: 0,
> dequeueCount: 0, memUsage:0,
> maxPageSize:200<--[org.apache.activemq.broker.region.Queue:1913
> doPageInForDispatch]
> [DEBUG][2018-08-21 10:05:24] brokeA Message
> ID:NFJD-PSC-BCOP-COREVM25-59961-1534816937032-9:1:-1:1:1 sent to
> queue://amq_admin_test<--[org.apache.activemq.broker.region.Queue:1842
> messageSent]
> [DEBUG][2018-08-21 10:05:24] queue://amq_admin_test, subscriptions=0,
> memory=0%, size=1, pending=0 toPageIn: 1, force:false, Inflight: 0,
> pagedInMessages.size 0, pagedInPendingDispatch.size 0, enqueueCount: 1,
> dequeueCount: 0, memUsage:1065,
> maxPageSize:200<--[org.apache.activemq.broker.region.Queue:1913
> doPageInForDispatch]
> [DEBUG][2018-08-21 10:05:24] localhost Message
> ID:NFJD-PSC-BCOP-COREVM25-59961-1534816937032-9:1:-1:1:2 sent to
> queue://amq_admin_test<--[org.apache.activemq.broker.region.Queue:1842
> messageSent]
> [DEBUG][2018-08-21 10:05:24] queue://amq_admin_test, subscriptions=0,
> memory=0%, size=2, pending=0 toPageIn: 2, force:false, Inflight: 0,
> pagedInMessages.size 0, pagedInPendingDispatch.size 0, enqueueCount: 2,
> dequeueCount: 0, memUsage:2130,
> maxPageSize:200<--[org.apache.activemq.broker.region.Queue:1913
> doPageInForDispatch]
> [DEBUG][2018-08-21 10:05:24] localhost Message
> ID:NFJD-PSC-BCOP-COREVM25-59961-1534816937032-9:1:-1:1:3 sent to
> queue://amq_admin_test<--[org.apache.activemq.broker.region.Queue:1842
> messageSent]
> [DEBUG][2018-08-21 10:05:24] queue://amq_admin_test, subscriptions=0,
> memory=0%, size=3, pending=0 toPageIn: 3, force:false, Inflight: 0,
> pagedInMessages.size 0, pagedInPendingDispatch.size 0, enqueueCount: 3,
> dequeueCount: 0, memUsage:3195,
> maxPageSize:200<--[org.apache.activemq.broker.region.Queue:1913
> doPageInForDispatch]
>
>
>
> [broke B]
> [DEBUG][2018-08-21 10:07:22] brokeB adding consumer:
> ID:XXXXXX-vm18-47733-1534816696571-13:1:-1:1 for destination:
>
> queue://amq_admin_test<--[org.apache.activemq.broker.region.AbstractRegion:338
> addConsumer]
> [DEBUG][2018-08-21 10:07:22] brokeB adding destination:
>
> queue://amq_admin_test<--[org.apache.activemq.broker.region.AbstractRegion:154
> addDestination]
> [DEBUG][2018-08-21 10:07:22] queue://amq_admin_test, subscriptions=0,
> memory=0%, size=0, pending=0 toPageIn: 0, force:false, Inflight: 0,
> pagedInMessages.size 0, pagedInPendingDispatch.size 0, enqueueCount: 0,
> dequeueCount: 0, memUsage:0,
> maxPageSize:200<--[org.apache.activemq.broker.region.Queue:1913
> doPageInForDispatch]
> [DEBUG][2018-08-21 10:07:22] queue://amq_admin_test add sub:
> QueueSubscription: consumer=ID:XXXXXX-vm18-47733-1534816696571-13:1:-1:1,
> destinations=0, dispatched=0, delivered=0, pending=0, dequeues: 0,
> dispatched: 0, inflight: 0<--[org.apache.activemq.broker.region.Queue:422
> addSubscription]
> [DEBUG][2018-08-21 10:07:22] queue://amq_admin_test, subscriptions=1,
> memory=0%, size=0, pending=0 toPageIn: 0, force:false, Inflight: 0,
> pagedInMessages.size 0, pagedInPendingDispatch.size 0, enqueueCount: 0,
> dequeueCount: 0, memUsage:0,
> maxPageSize:200<--[org.apache.activemq.broker.region.Queue:1913
> doPageInForDispatch]
> [DEBUG][2018-08-21 10:07:22] brokeB adding destination:
>
> topic://ActiveMQ.Advisory.Consumer.Queue.amq_admin_test<--[org.apache.activemq.broker.region.AbstractRegion:154
> addDestination]
>
>
>
> ######normally, the log is as below
> [broke A]
> [DEBUG][2018-08-29 14:48:24] brokeA adding consumer:
> brokeA->brokeB-45659-1535359315047-21:1:1:8 for destination:
>
> queue://amq_admin_test<--[org.apache.activemq.broker.region.AbstractRegion:338
> addConsumer]
> [DEBUG][2018-08-29 14:48:24] queue://amq_admin_test add sub:
> QueueSubscription:
> consumer=localhost->bj_172.20.32.111-45659-1535359315047-21:1:1:8,
> destinations=0, dispatched=0, delivered=0, pending=0, dequeues: 0,
> dispatched: 0, inflight: 0<--[org.apache.activemq.broker.region.Queue:422
> addSubscription]
> [DEBUG][2018-08-29 14:48:24] queue://amq_admin_test, subscriptions=1,
> memory=0%, size=3, pending=0 toPageIn: 0, force:false, Inflight: 0,
> pagedInMessages.size 3, pagedInPendingDispatch.size 3, enqueueCount: 3,
> dequeueCount: 0, memUsage:3195,
> maxPageSize:200<--[org.apache.activemq.broker.region.Queue:1913
> doPageInForDispatch]
> [DEBUG][2018-08-29 14:48:24] Publishing: vm://localhost for broker
> transport
> URI: vm://localhost<--[org.apache.activemq.broker.TransportConnector:268
> getPublishableConnectString]
> [DEBUG][2018-08-29 14:48:24] bridging (brokeA -> brokeB), consumer:
> localhost->bj_172.20.32.111-45659-1535359315047-21:1:1:8, destination:
> queue://amq_admin_test, brokerPath:
> [ID:cloudlab043-45659-1535359315047-0:1], message:
>
> ID:cloudlab043-45659-1535359315047-13:3:-1:1:1<--[org.apache.activemq.network.DemandForwardingBridgeSupport:1070
> serviceLocalCommand]
> [DEBUG][2018-08-29 14:48:24] localhost new demand subscription:
>
> DemandSub{brokeA->brokeB-45659-1535359315047-21:1:1:8,remotes:[ID:XXXXXXXXX-54561-1535506233797-5:12:-1:1]}<--[org.apache.activemq.network.DemandForwardingBridgeSupport:1305
> addConsumerInfo]
>
> [broke B]
> [DEBUG][2018-08-29 14:48:25] brokeB adding consumer:
> ID:cloudlab042-54561-1535506233797-5:12:-1:1 for destination:
>
> queue://amq_admin_test<--[org.apache.activemq.broker.region.AbstractRegion:338
> addConsumer]
> [DEBUG][2018-08-29 14:48:25] brokeB adding destination:
>
> queue://amq_admin_test<--[org.apache.activemq.broker.region.AbstractRegion:154
> addDestination]
> [DEBUG][2018-08-29 14:48:25] queue://amq_admin_test, subscriptions=0,
> memory=0%, size=0, pending=0 toPageIn: 0, force:false, Inflight: 0,
> pagedInMessages.size 0, pagedInPendingDispatch.size 0, enqueueCount: 0,
> dequeueCount: 0, memUsage:0,
> maxPageSize:200<--[org.apache.activemq.broker.region.Queue:1913
> doPageInForDispatch]
> [DEBUG][2018-08-29 14:48:25] queue://amq_admin_test add sub:
> QueueSubscription: consumer=ID:XXXXXXXX-54561-1535506233797-5:12:-1:1,
> destinations=0, dispatched=0, delivered=0, pending=0, dequeues: 0,
> dispatched: 0, inflight: 0<--[org.apache.activemq.broker.region.Queue:422
> addSubscription]
> [DEBUG][2018-08-29 14:48:25] brokeB adding destination:
>
> topic://ActiveMQ.Advisory.Consumer.Queue.amq_admin_test<--[org.apache.activemq.broker.region.AbstractRegion:154
> addDestination]
> [DEBUG][2018-08-29 14:48:25] queue://amq_admin_test, subscriptions=1,
> memory=0%, size=0, pending=0 toPageIn: 0, force:false, Inflight: 0,
> pagedInMessages.size 0, pagedInPendingDispatch.size 0, enqueueCount: 0,
> dequeueCount: 0, memUsage:0,
> maxPageSize:200<--[org.apache.activemq.broker.region.Queue:1913
> doPageInForDispatch]
> [DEBUG][2018-08-29 14:48:25] bridging (brokeB -> localhost), consumer:
> bj_172.20.32.111->brokeA-54561-1535506233797-4:1:1:1, destination:
> topic://ActiveMQ.Advisory.Consumer.Queue.amq_admin_test, brokerPath:
> [ID:cloudlab042-54561-1535506233797-0:1], message:
>
> ID:cloudlab042-54561-1535506233797-1:1:0:0:275<--[org.apache.activemq.network.DemandForwardingBridgeSupport:1070
> serviceLocalCommand]
> [DEBUG][2018-08-29 14:48:25] brokeB Message
> ID:cloudlab043-45659-1535359315047-13:3:-1:1:1 sent to
> queue://amq_admin_test<--[org.apache.activemq.broker.region.Queue:1842
> messageSent]
> [DEBUG][2018-08-29 14:48:25] queue://amq_admin_test, subscriptions=1,
> memory=0%, size=1, pending=0 toPageIn: 1, force:false, Inflight: 0,
> pagedInMessages.size 0, pagedInPendingDispatch.size 0, enqueueCount: 1,
> dequeueCount: 0, memUsage:1065,
> maxPageSize:200<--[org.apache.activemq.broker.region.Queue:1913
> doPageInForDispatch]
>
>
>
>
> --
> Sent from:
> http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
>
Re: cannot consume msg by a duplex network connect broke.
Posted by ievanliu <ie...@qq.com>.
Hi,Tim.
thank you very much for your help.
here are some more information:
1.the network connect configuration is the same as what we use in this
problem,so most values should be defaults value.
2.these days, I compared the log very carefully,and I found that, when the
problem happens, broke did not create a bridge-consumer.
but why that happened?
please help me?
------------------------------------------------------
test case:
produce messages into broke A,and consume messages from broke B.broke B
config duplex network connect to Broke A
------------------------------------------------------
#####when the problem happen,the log is as below:
[broke A]
[DEBUG][2018-08-21 10:05:24] brokeA adding destination:
queue://amq_admin_test<--[org.apache.activemq.broker.region.AbstractRegion:154
addDestination]
[DEBUG][2018-08-21 10:05:24] queue://amq_admin_test, subscriptions=0,
memory=0%, size=0, pending=0 toPageIn: 0, force:false, Inflight: 0,
pagedInMessages.size 0, pagedInPendingDispatch.size 0, enqueueCount: 0,
dequeueCount: 0, memUsage:0,
maxPageSize:200<--[org.apache.activemq.broker.region.Queue:1913
doPageInForDispatch]
[DEBUG][2018-08-21 10:05:24] brokeA Message
ID:NFJD-PSC-BCOP-COREVM25-59961-1534816937032-9:1:-1:1:1 sent to
queue://amq_admin_test<--[org.apache.activemq.broker.region.Queue:1842
messageSent]
[DEBUG][2018-08-21 10:05:24] queue://amq_admin_test, subscriptions=0,
memory=0%, size=1, pending=0 toPageIn: 1, force:false, Inflight: 0,
pagedInMessages.size 0, pagedInPendingDispatch.size 0, enqueueCount: 1,
dequeueCount: 0, memUsage:1065,
maxPageSize:200<--[org.apache.activemq.broker.region.Queue:1913
doPageInForDispatch]
[DEBUG][2018-08-21 10:05:24] localhost Message
ID:NFJD-PSC-BCOP-COREVM25-59961-1534816937032-9:1:-1:1:2 sent to
queue://amq_admin_test<--[org.apache.activemq.broker.region.Queue:1842
messageSent]
[DEBUG][2018-08-21 10:05:24] queue://amq_admin_test, subscriptions=0,
memory=0%, size=2, pending=0 toPageIn: 2, force:false, Inflight: 0,
pagedInMessages.size 0, pagedInPendingDispatch.size 0, enqueueCount: 2,
dequeueCount: 0, memUsage:2130,
maxPageSize:200<--[org.apache.activemq.broker.region.Queue:1913
doPageInForDispatch]
[DEBUG][2018-08-21 10:05:24] localhost Message
ID:NFJD-PSC-BCOP-COREVM25-59961-1534816937032-9:1:-1:1:3 sent to
queue://amq_admin_test<--[org.apache.activemq.broker.region.Queue:1842
messageSent]
[DEBUG][2018-08-21 10:05:24] queue://amq_admin_test, subscriptions=0,
memory=0%, size=3, pending=0 toPageIn: 3, force:false, Inflight: 0,
pagedInMessages.size 0, pagedInPendingDispatch.size 0, enqueueCount: 3,
dequeueCount: 0, memUsage:3195,
maxPageSize:200<--[org.apache.activemq.broker.region.Queue:1913
doPageInForDispatch]
[broke B]
[DEBUG][2018-08-21 10:07:22] brokeB adding consumer:
ID:XXXXXX-vm18-47733-1534816696571-13:1:-1:1 for destination:
queue://amq_admin_test<--[org.apache.activemq.broker.region.AbstractRegion:338
addConsumer]
[DEBUG][2018-08-21 10:07:22] brokeB adding destination:
queue://amq_admin_test<--[org.apache.activemq.broker.region.AbstractRegion:154
addDestination]
[DEBUG][2018-08-21 10:07:22] queue://amq_admin_test, subscriptions=0,
memory=0%, size=0, pending=0 toPageIn: 0, force:false, Inflight: 0,
pagedInMessages.size 0, pagedInPendingDispatch.size 0, enqueueCount: 0,
dequeueCount: 0, memUsage:0,
maxPageSize:200<--[org.apache.activemq.broker.region.Queue:1913
doPageInForDispatch]
[DEBUG][2018-08-21 10:07:22] queue://amq_admin_test add sub:
QueueSubscription: consumer=ID:XXXXXX-vm18-47733-1534816696571-13:1:-1:1,
destinations=0, dispatched=0, delivered=0, pending=0, dequeues: 0,
dispatched: 0, inflight: 0<--[org.apache.activemq.broker.region.Queue:422
addSubscription]
[DEBUG][2018-08-21 10:07:22] queue://amq_admin_test, subscriptions=1,
memory=0%, size=0, pending=0 toPageIn: 0, force:false, Inflight: 0,
pagedInMessages.size 0, pagedInPendingDispatch.size 0, enqueueCount: 0,
dequeueCount: 0, memUsage:0,
maxPageSize:200<--[org.apache.activemq.broker.region.Queue:1913
doPageInForDispatch]
[DEBUG][2018-08-21 10:07:22] brokeB adding destination:
topic://ActiveMQ.Advisory.Consumer.Queue.amq_admin_test<--[org.apache.activemq.broker.region.AbstractRegion:154
addDestination]
######normally, the log is as below
[broke A]
[DEBUG][2018-08-29 14:48:24] brokeA adding consumer:
brokeA->brokeB-45659-1535359315047-21:1:1:8 for destination:
queue://amq_admin_test<--[org.apache.activemq.broker.region.AbstractRegion:338
addConsumer]
[DEBUG][2018-08-29 14:48:24] queue://amq_admin_test add sub:
QueueSubscription:
consumer=localhost->bj_172.20.32.111-45659-1535359315047-21:1:1:8,
destinations=0, dispatched=0, delivered=0, pending=0, dequeues: 0,
dispatched: 0, inflight: 0<--[org.apache.activemq.broker.region.Queue:422
addSubscription]
[DEBUG][2018-08-29 14:48:24] queue://amq_admin_test, subscriptions=1,
memory=0%, size=3, pending=0 toPageIn: 0, force:false, Inflight: 0,
pagedInMessages.size 3, pagedInPendingDispatch.size 3, enqueueCount: 3,
dequeueCount: 0, memUsage:3195,
maxPageSize:200<--[org.apache.activemq.broker.region.Queue:1913
doPageInForDispatch]
[DEBUG][2018-08-29 14:48:24] Publishing: vm://localhost for broker transport
URI: vm://localhost<--[org.apache.activemq.broker.TransportConnector:268
getPublishableConnectString]
[DEBUG][2018-08-29 14:48:24] bridging (brokeA -> brokeB), consumer:
localhost->bj_172.20.32.111-45659-1535359315047-21:1:1:8, destination:
queue://amq_admin_test, brokerPath:
[ID:cloudlab043-45659-1535359315047-0:1], message:
ID:cloudlab043-45659-1535359315047-13:3:-1:1:1<--[org.apache.activemq.network.DemandForwardingBridgeSupport:1070
serviceLocalCommand]
[DEBUG][2018-08-29 14:48:24] localhost new demand subscription:
DemandSub{brokeA->brokeB-45659-1535359315047-21:1:1:8,remotes:[ID:XXXXXXXXX-54561-1535506233797-5:12:-1:1]}<--[org.apache.activemq.network.DemandForwardingBridgeSupport:1305
addConsumerInfo]
[broke B]
[DEBUG][2018-08-29 14:48:25] brokeB adding consumer:
ID:cloudlab042-54561-1535506233797-5:12:-1:1 for destination:
queue://amq_admin_test<--[org.apache.activemq.broker.region.AbstractRegion:338
addConsumer]
[DEBUG][2018-08-29 14:48:25] brokeB adding destination:
queue://amq_admin_test<--[org.apache.activemq.broker.region.AbstractRegion:154
addDestination]
[DEBUG][2018-08-29 14:48:25] queue://amq_admin_test, subscriptions=0,
memory=0%, size=0, pending=0 toPageIn: 0, force:false, Inflight: 0,
pagedInMessages.size 0, pagedInPendingDispatch.size 0, enqueueCount: 0,
dequeueCount: 0, memUsage:0,
maxPageSize:200<--[org.apache.activemq.broker.region.Queue:1913
doPageInForDispatch]
[DEBUG][2018-08-29 14:48:25] queue://amq_admin_test add sub:
QueueSubscription: consumer=ID:XXXXXXXX-54561-1535506233797-5:12:-1:1,
destinations=0, dispatched=0, delivered=0, pending=0, dequeues: 0,
dispatched: 0, inflight: 0<--[org.apache.activemq.broker.region.Queue:422
addSubscription]
[DEBUG][2018-08-29 14:48:25] brokeB adding destination:
topic://ActiveMQ.Advisory.Consumer.Queue.amq_admin_test<--[org.apache.activemq.broker.region.AbstractRegion:154
addDestination]
[DEBUG][2018-08-29 14:48:25] queue://amq_admin_test, subscriptions=1,
memory=0%, size=0, pending=0 toPageIn: 0, force:false, Inflight: 0,
pagedInMessages.size 0, pagedInPendingDispatch.size 0, enqueueCount: 0,
dequeueCount: 0, memUsage:0,
maxPageSize:200<--[org.apache.activemq.broker.region.Queue:1913
doPageInForDispatch]
[DEBUG][2018-08-29 14:48:25] bridging (brokeB -> localhost), consumer:
bj_172.20.32.111->brokeA-54561-1535506233797-4:1:1:1, destination:
topic://ActiveMQ.Advisory.Consumer.Queue.amq_admin_test, brokerPath:
[ID:cloudlab042-54561-1535506233797-0:1], message:
ID:cloudlab042-54561-1535506233797-1:1:0:0:275<--[org.apache.activemq.network.DemandForwardingBridgeSupport:1070
serviceLocalCommand]
[DEBUG][2018-08-29 14:48:25] brokeB Message
ID:cloudlab043-45659-1535359315047-13:3:-1:1:1 sent to
queue://amq_admin_test<--[org.apache.activemq.broker.region.Queue:1842
messageSent]
[DEBUG][2018-08-29 14:48:25] queue://amq_admin_test, subscriptions=1,
memory=0%, size=1, pending=0 toPageIn: 1, force:false, Inflight: 0,
pagedInMessages.size 0, pagedInPendingDispatch.size 0, enqueueCount: 1,
dequeueCount: 0, memUsage:1065,
maxPageSize:200<--[org.apache.activemq.broker.region.Queue:1913
doPageInForDispatch]
--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
Re: cannot consume msg by a duplex network connect broke.
Posted by Tim Bain <tb...@alumni.duke.edu>.
As you said in #2, without a network bridge consumer, there will be no
message flow through the network of brokers. So the question then is why
network consumers aren't being created, given that in #1 you said that the
logs indicate that the network bridge was successfully built.
I notice that the snippet of activemq.xml that you quoted doesn't include
any indication that you're including destinations either dynamically or
statically (see Dynamic Networks and Pure Static Networks in
http://activemq.apache.org/networks-of-brokers.html). Is that an accurate
and complete quote from the file, or did you omit some of the content to
focus our attention on the pieces you did include? What you quoted would be
expected to result in no message flow between A and B, which matches your
description of the problematic behavior wouldn't explain why this would
have ever worked for you.
If that doesn't lead us to an answer and you're able to reproduce the
problem, I'd start by attempting to see if a socket connection between the
brokers exists, and then using JMX to see whether a network consumer is
built for the topic in question. If there's a network connection but no
network consumer, then I'd want to attach a debugger and step through the
code where the subscriptions are propagated.
https://activemq.apache.org/maven/apidocs/org/apache/activemq/network/DemandForwardingBridgeSupport.html#isPermissableDestination-org.apache.activemq.command.ActiveMQDestination-boolean-
should be the right place to set your breakpoint, should you need to go
down this path.
Tim
On Mon, Aug 27, 2018 at 7:26 AM ievanliu <ie...@qq.com> wrote:
> version:5.14.0
> architecture： Broke A(a 3 instances jdbc cluster) and Broke B(a 3 instances
> jdbc cluster), use duplex network connect, from brokeB to brokeA
> productor produce messages into brokeA, consumers consume messages from
> broke B, bridge queue like queueA,queueB
>
>
> networkConnector config:
>
> <networkConnectors>
> <networkConnector
> uri="masterslave:(tcp://172.16.202.62:61616,tcp://172.16.202.65:61616
> ,tcp://172.16.202.68:61616)"
> duplex="true" name="bridge_111"></networkConnector>
> </networkConnectors>
>
> problem:
> early this morning, brokeB happened a master change, by the pre-master lost
> the jdbc lock.
> broke B2 in the broke B cluster, successfuly became the master, and build
> the network connection to broke A.
> But, there is a problem that , producer can produce messages to BrokeA, but
> consumers cannot consumer message from brokeB.
>
> process:
> 1、as the log shows, the network bridge has successfully been built。and
> there
> are no other error logs info.
> 2、I found that, when consumer connect to brokeB to consume queueA, in broke
> A, there is no consumers.Normally, here should be a network bridge consumer
> to brokeA.maybe this is the reason.
> 3、restart the broke A and broke B，the problem happened as same
>
>
> result:
> about 1 hours later, the problem disappear automaticly, and I can see there
> is a network bridge consumer to brokeA.
>
>
> who can help me how to troubleshooting this problem?
>
>
>
> --
> Sent from:
> http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html
>