[C++ Broker] dynamically federated links

[Matt Broadstone <mb...@gmail.com>]

I'm having some trouble linking two brokers together using the
documentation in section 1.4 (Broker Federation), and was looking for some
advice.

* I currently have two brokers (192.168.1.1, 192.168.1.2) which use PLAIN
SASL, and have appropriate ACL files allowing a user "test" access to
everything once authenticated.

* I then have two previously existing (not sure if this is where the
problem starts, very little guiding debug information is printed) durable
fanout topics on each broker called `test.fanout` -- this is the "initial"
unfederated state.

* At this point I run the following command on 192.168.1.2, in order to
dynamically federate the `test.fanout` exchange to the "leader":

> qpid-route -v dynamic add test/password@192.168.1.1:5672
test/password@localhost:5672 test.fanout

Link state is Operational
Creating inter-broker binding...
Bridge method returned: 0 OK

* Cool, seems like everything worked. However, on the other side my qpidd
log file has the following:

2016-01-06 09:50:58 [Management] error Detected two management objects with
the same identifier:
0-3-1--653(org.apache.qpid.broker:session:test@TEST.qpid.bridge_session_qpid.tcp
:localhost:5672!test.fanout!test.fanout!_25c065aa-5b1b-465a-a6d2-510666f89248)

which seems problematic, however the link is operational and there are no
other errors.

* Now I try to use the fanout by connecting to the local broker on
192.168.1.2 and sending some data. The data is indeed sent to the local
fanout and I'm able to pick up those messages, but nothing is sent to the
federated broker.

Any hints as to where I'm going wrong here would be much appreciated!

Matt

Re: [C++ Broker] dynamically federated links

[Gordon Sim <gs...@redhat.com>]

On 01/06/2016 04:34 PM, Matt Broadstone wrote:
> On Wed, Jan 6, 2016 at 11:14 AM, Matt Broadstone <mb...@gmail.com> wrote:
>> Just one more question that doesn't seem to be covered in the
>> documentation: is it possible with just `qpid-route` to configure
>> replication of queues across federated brokers?  It seems that the
>> `qpid-route queue` related commands all bind a given queue to a remote
>> exchange, where ideally it would be possible to federate two brokers and
>> maintain a shared queue between them.
>>
>> Matt
>>
>
> It might be better to clarify my particular use case here. I know that it
> seems that using qpid-ha is probably the way to go in my case, however, it
> seems that qpid-ha is designed in such a way that clients are all meant to
> connect to the primary broker.  In our case, the desired behavior would be
> for all clients on a local machine connect to that machine, which is in
> turn federated to a cluster (thus alleviating the need in our case to track
> and dynamically connect to the IP of the primary broker).  So this leads me
> to a number of questions:
>
> a) it seems I can accomplish this mostly (with exchange types) through
> plain old federation, and as such have great flexibility over my design
>
> b) (copied from above) is it possible to easily replicate queues using
> plain federation?

No, the 'federation' feature only transfers messages. It cannot 
propagate any dequeue operations. As such it can't be used on its own to 
create a replica of a queue (which tracks the original queues state).

(The ha feature uses federation, but adds extra capabilities on top of it).

> c) If, instead, qpid-ha is used to satisfy these needs, is it possible to
> configure qpid-ha such that clients can directly connect to backup cluster
> members and interact with the cluster in some sort of "proxy" mode
> (proxying everything up to the primary broker, while maintaining the back
> channel that backs up the primary broker)

The replication uses queue sequence for identifying messages so if you 
alter a replica queue it will almost certainly break replication.


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org

Re: [C++ Broker] dynamically federated links

[Matt Broadstone <mb...@gmail.com>]

On Wed, Jan 6, 2016 at 11:14 AM, Matt Broadstone <mb...@gmail.com> wrote:

> On Wed, Jan 6, 2016 at 10:23 AM, Matt Broadstone <mb...@gmail.com>
> wrote:
>
>> On Wed, Jan 6, 2016 at 10:12 AM, Ted Ross <tr...@redhat.com> wrote:
>>
>>> On 01/06/2016 10:03 AM, Matt Broadstone wrote:
>>>
>>>> I'm having some trouble linking two brokers together using the
>>>> documentation in section 1.4 (Broker Federation), and was looking for
>>>> some
>>>> advice.
>>>>
>>>> * I currently have two brokers (192.168.1.1, 192.168.1.2) which use
>>>> PLAIN
>>>> SASL, and have appropriate ACL files allowing a user "test" access to
>>>> everything once authenticated.
>>>>
>>>> * I then have two previously existing (not sure if this is where the
>>>> problem starts, very little guiding debug information is printed)
>>>> durable
>>>> fanout topics on each broker called `test.fanout` -- this is the
>>>> "initial"
>>>> unfederated state.
>>>>
>>>> * At this point I run the following command on 192.168.1.2, in order to
>>>> dynamically federate the `test.fanout` exchange to the "leader":
>>>>
>>>> qpid-route -v dynamic add test/password@192.168.1.1:5672
>>>>>
>>>> test/password@localhost:5672 test.fanout
>>>>
>>>
>>> I wonder if you are running into problems because you used "localhost"
>>> instead of the real address (192.168.1.2).  Both systems think of
>>> themselves as localhost so there may be a collision.
>>
>>
>> Ted,
>> That looks like it solved the problem! Many thanks.  Might I suggest an
>> update to the documentation to remove all references to localhost.  Also
>> there is a typo in the federation docs about `qpid-route list connections`
>> which is a command that does not exist, rather it shouldbe `qpid-route link
>> list`
>>
>> Cheers,
>> Matt
>>
>>
> Just one more question that doesn't seem to be covered in the
> documentation: is it possible with just `qpid-route` to configure
> replication of queues across federated brokers?  It seems that the
> `qpid-route queue` related commands all bind a given queue to a remote
> exchange, where ideally it would be possible to federate two brokers and
> maintain a shared queue between them.
>
> Matt
>

It might be better to clarify my particular use case here. I know that it
seems that using qpid-ha is probably the way to go in my case, however, it
seems that qpid-ha is designed in such a way that clients are all meant to
connect to the primary broker.  In our case, the desired behavior would be
for all clients on a local machine connect to that machine, which is in
turn federated to a cluster (thus alleviating the need in our case to track
and dynamically connect to the IP of the primary broker).  So this leads me
to a number of questions:

a) it seems I can accomplish this mostly (with exchange types) through
plain old federation, and as such have great flexibility over my design

b) (copied from above) is it possible to easily replicate queues using
plain federation?

c) If, instead, qpid-ha is used to satisfy these needs, is it possible to
configure qpid-ha such that clients can directly connect to backup cluster
members and interact with the cluster in some sort of "proxy" mode
(proxying everything up to the primary broker, while maintaining the back
channel that backs up the primary broker)

Thanks,
Matt



>
>
>
>>
>>>
>>>
>>>> Link state is Operational
>>>> Creating inter-broker binding...
>>>> Bridge method returned: 0 OK
>>>>
>>>> * Cool, seems like everything worked. However, on the other side my
>>>> qpidd
>>>> log file has the following:
>>>>
>>>> 2016-01-06 09:50:58 [Management] error Detected two management objects
>>>> with
>>>> the same identifier:
>>>>
>>>> 0-3-1--653(org.apache.qpid.broker:session:test@TEST.qpid.bridge_session_qpid.tcp
>>>>
>>>> :localhost:5672!test.fanout!test.fanout!_25c065aa-5b1b-465a-a6d2-510666f89248)
>>>>
>>>> which seems problematic, however the link is operational and there are
>>>> no
>>>> other errors.
>>>>
>>>> * Now I try to use the fanout by connecting to the local broker on
>>>> 192.168.1.2 and sending some data. The data is indeed sent to the local
>>>> fanout and I'm able to pick up those messages, but nothing is sent to
>>>> the
>>>> federated broker.
>>>>
>>>> Any hints as to where I'm going wrong here would be much appreciated!
>>>>
>>>> Matt
>>>>
>>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
>>> For additional commands, e-mail: users-help@qpid.apache.org
>>>
>>>
>>
>

Re: [C++ Broker] dynamically federated links

[Matt Broadstone <mb...@gmail.com>]

On Wed, Jan 6, 2016 at 10:23 AM, Matt Broadstone <mb...@gmail.com> wrote:

> On Wed, Jan 6, 2016 at 10:12 AM, Ted Ross <tr...@redhat.com> wrote:
>
>> On 01/06/2016 10:03 AM, Matt Broadstone wrote:
>>
>>> I'm having some trouble linking two brokers together using the
>>> documentation in section 1.4 (Broker Federation), and was looking for
>>> some
>>> advice.
>>>
>>> * I currently have two brokers (192.168.1.1, 192.168.1.2) which use PLAIN
>>> SASL, and have appropriate ACL files allowing a user "test" access to
>>> everything once authenticated.
>>>
>>> * I then have two previously existing (not sure if this is where the
>>> problem starts, very little guiding debug information is printed) durable
>>> fanout topics on each broker called `test.fanout` -- this is the
>>> "initial"
>>> unfederated state.
>>>
>>> * At this point I run the following command on 192.168.1.2, in order to
>>> dynamically federate the `test.fanout` exchange to the "leader":
>>>
>>> qpid-route -v dynamic add test/password@192.168.1.1:5672
>>>>
>>> test/password@localhost:5672 test.fanout
>>>
>>
>> I wonder if you are running into problems because you used "localhost"
>> instead of the real address (192.168.1.2).  Both systems think of
>> themselves as localhost so there may be a collision.
>
>
> Ted,
> That looks like it solved the problem! Many thanks.  Might I suggest an
> update to the documentation to remove all references to localhost.  Also
> there is a typo in the federation docs about `qpid-route list connections`
> which is a command that does not exist, rather it shouldbe `qpid-route link
> list`
>
> Cheers,
> Matt
>
>
Just one more question that doesn't seem to be covered in the
documentation: is it possible with just `qpid-route` to configure
replication of queues across federated brokers?  It seems that the
`qpid-route queue` related commands all bind a given queue to a remote
exchange, where ideally it would be possible to federate two brokers and
maintain a shared queue between them.

Matt



>
>>
>>
>>> Link state is Operational
>>> Creating inter-broker binding...
>>> Bridge method returned: 0 OK
>>>
>>> * Cool, seems like everything worked. However, on the other side my qpidd
>>> log file has the following:
>>>
>>> 2016-01-06 09:50:58 [Management] error Detected two management objects
>>> with
>>> the same identifier:
>>>
>>> 0-3-1--653(org.apache.qpid.broker:session:test@TEST.qpid.bridge_session_qpid.tcp
>>>
>>> :localhost:5672!test.fanout!test.fanout!_25c065aa-5b1b-465a-a6d2-510666f89248)
>>>
>>> which seems problematic, however the link is operational and there are no
>>> other errors.
>>>
>>> * Now I try to use the fanout by connecting to the local broker on
>>> 192.168.1.2 and sending some data. The data is indeed sent to the local
>>> fanout and I'm able to pick up those messages, but nothing is sent to the
>>> federated broker.
>>>
>>> Any hints as to where I'm going wrong here would be much appreciated!
>>>
>>> Matt
>>>
>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
>> For additional commands, e-mail: users-help@qpid.apache.org
>>
>>
>

Re: [C++ Broker] dynamically federated links

[Matt Broadstone <mb...@gmail.com>]

On Wed, Jan 6, 2016 at 10:12 AM, Ted Ross <tr...@redhat.com> wrote:

> On 01/06/2016 10:03 AM, Matt Broadstone wrote:
>
>> I'm having some trouble linking two brokers together using the
>> documentation in section 1.4 (Broker Federation), and was looking for some
>> advice.
>>
>> * I currently have two brokers (192.168.1.1, 192.168.1.2) which use PLAIN
>> SASL, and have appropriate ACL files allowing a user "test" access to
>> everything once authenticated.
>>
>> * I then have two previously existing (not sure if this is where the
>> problem starts, very little guiding debug information is printed) durable
>> fanout topics on each broker called `test.fanout` -- this is the "initial"
>> unfederated state.
>>
>> * At this point I run the following command on 192.168.1.2, in order to
>> dynamically federate the `test.fanout` exchange to the "leader":
>>
>> qpid-route -v dynamic add test/password@192.168.1.1:5672
>>>
>> test/password@localhost:5672 test.fanout
>>
>
> I wonder if you are running into problems because you used "localhost"
> instead of the real address (192.168.1.2).  Both systems think of
> themselves as localhost so there may be a collision.


Ted,
That looks like it solved the problem! Many thanks.  Might I suggest an
update to the documentation to remove all references to localhost.  Also
there is a typo in the federation docs about `qpid-route list connections`
which is a command that does not exist, rather it shouldbe `qpid-route link
list`

Cheers,
Matt


>
>
>> Link state is Operational
>> Creating inter-broker binding...
>> Bridge method returned: 0 OK
>>
>> * Cool, seems like everything worked. However, on the other side my qpidd
>> log file has the following:
>>
>> 2016-01-06 09:50:58 [Management] error Detected two management objects
>> with
>> the same identifier:
>>
>> 0-3-1--653(org.apache.qpid.broker:session:test@TEST.qpid.bridge_session_qpid.tcp
>>
>> :localhost:5672!test.fanout!test.fanout!_25c065aa-5b1b-465a-a6d2-510666f89248)
>>
>> which seems problematic, however the link is operational and there are no
>> other errors.
>>
>> * Now I try to use the fanout by connecting to the local broker on
>> 192.168.1.2 and sending some data. The data is indeed sent to the local
>> fanout and I'm able to pick up those messages, but nothing is sent to the
>> federated broker.
>>
>> Any hints as to where I'm going wrong here would be much appreciated!
>>
>> Matt
>>
>>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>

Re: [C++ Broker] dynamically federated links

[Ted Ross <tr...@redhat.com>]

On 01/06/2016 10:03 AM, Matt Broadstone wrote:
> I'm having some trouble linking two brokers together using the
> documentation in section 1.4 (Broker Federation), and was looking for some
> advice.
>
> * I currently have two brokers (192.168.1.1, 192.168.1.2) which use PLAIN
> SASL, and have appropriate ACL files allowing a user "test" access to
> everything once authenticated.
>
> * I then have two previously existing (not sure if this is where the
> problem starts, very little guiding debug information is printed) durable
> fanout topics on each broker called `test.fanout` -- this is the "initial"
> unfederated state.
>
> * At this point I run the following command on 192.168.1.2, in order to
> dynamically federate the `test.fanout` exchange to the "leader":
>
>> qpid-route -v dynamic add test/password@192.168.1.1:5672
> test/password@localhost:5672 test.fanout

I wonder if you are running into problems because you used "localhost" 
instead of the real address (192.168.1.2).  Both systems think of 
themselves as localhost so there may be a collision.

>
> Link state is Operational
> Creating inter-broker binding...
> Bridge method returned: 0 OK
>
> * Cool, seems like everything worked. However, on the other side my qpidd
> log file has the following:
>
> 2016-01-06 09:50:58 [Management] error Detected two management objects with
> the same identifier:
> 0-3-1--653(org.apache.qpid.broker:session:test@TEST.qpid.bridge_session_qpid.tcp
> :localhost:5672!test.fanout!test.fanout!_25c065aa-5b1b-465a-a6d2-510666f89248)
>
> which seems problematic, however the link is operational and there are no
> other errors.
>
> * Now I try to use the fanout by connecting to the local broker on
> 192.168.1.2 and sending some data. The data is indeed sent to the local
> fanout and I'm able to pick up those messages, but nothing is sent to the
> federated broker.
>
> Any hints as to where I'm going wrong here would be much appreciated!
>
> Matt
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org