Re: Network of brokers: consumers not synchronized

[Tim Bain <tb...@alumni.duke.edu>]

So I was thinking about something more like netstat, to answer the question
at the TCP layer. I've never used Karaf, but the documentation doesn't seem
to provide a way to do that from within the container, but at a minimum you
can run netstat on the host on which Karaf is running.

I'm wondering whether your mobile broker is connecting to a different
broker (not your backoffice one), which could explain why the mobile broker
says it's connected but the backoffice one says it's not.  Knowing where
the TCP connection is going would help with that.

Tim

On Nov 25, 2016 6:09 AM, "jochenw" <jo...@googlemail.com> wrote:

> Hello,
>
> found the right command: activemq:query | grep -A 10 -B 10
> networkConnectors.
>
> And the problem occurred again, so I could check this. The network
> connector
> is present on the mobile broker, but the web console of the backoffice
> broker doesn't show a connection.
>
> Regards,
> Jochen
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.
> nabble.com/Network-of-brokers-consumers-not-synchronized-
> tp4718852p4719488.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>

Re: Network of brokers: consumers not synchronized

[jochenw <jo...@googlemail.com>]

Some news on that: in the meantime, we have switched to only using static
bridges in our project for connection of the mobile brokers to the back
office. Is much more stable now.

However, there is still some problem which shows up now and then: sometimes
the back office broker "forgets" to start the responder end of the
duplex="true" bridge. In this case there is no consumer on the respective
queues in the backoffice broker, an the network connector on the mobile
broker doesn't get an active bridge, although the connection is there (and
the duplex="false" bridge for the direction mobile->back office works
absolutely fine). Happens in one out of ~ 500 reconnections. I have tested
this by turning up and down the network interface in an endless loop on one
of the mobile target systems.

So there still seems to be some instability when building the duplex bridge,
even for static bridges. I have a "workaround" solution by monitoring the
mobile broker via JMX beans, checking whether the duplex network connector
has an active bridge when the connection is there. If not, the activemq-osgi
bundle is restarted, which heals the problem.

I'm not sure whether is makes sense to create a Jira issue for that.

Regards,
Jochen



--
View this message in context: http://activemq.2283324.n4.nabble.com/Network-of-brokers-consumers-not-synchronized-tp4718852p4728365.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Network of brokers: consumers not synchronized

[jochenw <jo...@googlemail.com>]

Saw it again.

- The backoffice broker had shut down the connection (Channel was inactive
for too (>30000) long).
- The mobile broker logged "Transport failed, not attempting to
automatically reconnect; java.io.EOFException" and "bridge to .... stopped"
- One second later, the mobile broker tries reconnection, "Establishing
network connection from ... to ..."
- 16 seconds after this, the mobile broker logs "Successfully connected to
..."
- The backoffice broker has no log entry, and also doesn't show the
connection in the Web console
- On tcp level, the connection is there (netstat -an | grep 616):
     on mobile side: tcp <ip mobile>:45997 <ip backoffice>:61617 ESTABLISHED
     on backoffice side: tcp6 <ip backoffice>:61617 <ip mobile>:45997
ESTABLISHED

Regards,
Jochen



--
View this message in context: http://activemq.2283324.n4.nabble.com/Network-of-brokers-consumers-not-synchronized-tp4718852p4720243.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Network of brokers: consumers not synchronized

[jochenw <jo...@googlemail.com>]

I'll check this next time the issue shows up. But I'm pretty sure that if the
connection is there on TCP level, it can be to nothing else than one of the
two backoffice brokers, since only these are defined in the connection URI
for the network connection. But anyways a good idea to check whether the TCP
connection is there at all.

Regards,
Jochen



--
View this message in context: http://activemq.2283324.n4.nabble.com/Network-of-brokers-consumers-not-synchronized-tp4718852p4719762.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.