You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by Gordon Sim <gs...@redhat.com> on 2013/06/21 12:25:11 UTC
Federation on Windows (was Re: Qpid post-mortem and request for suggestions
for (my) next release challenge (10M msgs/sec on Windows))
On 06/17/2013 07:31 PM, Steve Huston wrote:
>>> And anything you can think of for dynamically load balancing across
>> brokers?
>>
>> Honestly, I think the simplest solution overall is for us to get federation
>> working on windows. I assume its some issue in the IO layer.
>> Does anyone have a concrete understanding of what the problem is and
>> what is required to fix it?
>>
>> Any volunteers from our windows experts to take a look (Cliff, Chuck,
>> Andrew, Steve)?
>
> https://issues.apache.org/jira/browse/QPID-2199
>
> It is likely not a very complicated issue once it's understood more. Getting the time/funding to do it has been the only impediment.
A view was expressed that this was because the AsynchConnector for
windows was not in fact asynchronous. I'm a little sceptical of that,
though I admit I haven't looked into the issue at all.
The underlying socket is reported to be established. As Ted suggests in
his comment, perhaps that the link registry has not been notified or was
unable to match the notification to the pending link.
The connection id and federation link matching was very fragile in
earlier versions (and seems now to be a little more robust). For example
https://issues.apache.org/jira/browse/QPID-4315, where SSL federation
broke if hostnames were used qith qpid-route.
Could a similar issue be (or have been) behind the windows problem?
Has anyone tried with a recent broker? E.g. 0.22 or trunk? There have
been quite a few changes in the codepaths involved.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
Re: Federation on Windows (was Re: Qpid post-mortem and request for
suggestions for (my) next release challenge (10M msgs/sec on Windows))
Posted by Gordon Sim <gs...@redhat.com>.
On 06/21/2013 04:31 PM, Gordon Sim wrote:
> Federation as is provides a way of spreading connections across brokers
> for pub-sub style traffic. It doesn't yet support a federated queue
> (i.e. a queue which is split between different brokers abut for which
> each message is consumed by one consumer in the network).
Or putting it another way, it works reasonably well for balancing load
between non-competing consumers, but doesn't do the coordination
required for competing consumers.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
Re: Federation on Windows (was Re: Qpid post-mortem and request for
suggestions for (my) next release challenge (10M msgs/sec on Windows))
Posted by Gordon Sim <gs...@redhat.com>.
On 06/21/2013 04:23 PM, Kerry Bonin wrote:
> Is there a clear explanation anywhere of exactly what Broker federation
> offers without corosync?
Broker federation (which has never used corosync) is described here:
http://qpid.apache.org/books/0.20/AMQP-Messaging-Broker-CPP-Book/html/chap-Messaging_User_Guide-Broker_Federation.html
The new HA functionality (which replaces the previous cluster plugin
which required direct corosync support) is described here:
http://qpid.apache.org/books/0.20/AMQP-Messaging-Broker-CPP-Book/html/chap-Messaging_User_Guide-Active_Passive_Cluster.html
> If I can meet my requirements by fixing this, and
> the scope of work is within an order of magnitude of rolling my own load
> balancing, I'd rather fix Qpid.
Federation as is provides a way of spreading connections across brokers
for pub-sub style traffic. It doesn't yet support a federated queue
(i.e. a queue which is split between different brokers abut for which
each message is consumed by one consumer in the network).
> I looked at it once, and it was in some
> overly clever and difficult to follow code, so I stopped, but I've spent a
> lot of time in the broker and low level Windows transport layers fixing
> other bugs, as has another senior member of my team...
If you haven't already done so I would strongly recommend a quick retest
on 0.22 (or trunk) as there have been a lot of changes in the codepaths
involved.
If not I'd be happy to try and help debugging in answering questions and
suggesting things to check...
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
Re: Federation on Windows (was Re: Qpid post-mortem and request for
suggestions for (my) next release challenge (10M msgs/sec on Windows))
Posted by Kerry Bonin <ke...@gmail.com>.
Is there a clear explanation anywhere of exactly what Broker federation
offers without corosync? If I can meet my requirements by fixing this, and
the scope of work is within an order of magnitude of rolling my own load
balancing, I'd rather fix Qpid. I looked at it once, and it was in some
overly clever and difficult to follow code, so I stopped, but I've spent a
lot of time in the broker and low level Windows transport layers fixing
other bugs, as has another senior member of my team...
On Fri, Jun 21, 2013 at 5:25 AM, Gordon Sim <gs...@redhat.com> wrote:
> On 06/17/2013 07:31 PM, Steve Huston wrote:
>
>> And anything you can think of for dynamically load balancing across
>>>>
>>> brokers?
>>>
>>> Honestly, I think the simplest solution overall is for us to get
>>> federation
>>> working on windows. I assume its some issue in the IO layer.
>>> Does anyone have a concrete understanding of what the problem is and
>>> what is required to fix it?
>>>
>>> Any volunteers from our windows experts to take a look (Cliff, Chuck,
>>> Andrew, Steve)?
>>>
>>
>> https://issues.apache.org/**jira/browse/QPID-2199<https://issues.apache.org/jira/browse/QPID-2199>
>>
>> It is likely not a very complicated issue once it's understood more.
>> Getting the time/funding to do it has been the only impediment.
>>
>
> A view was expressed that this was because the AsynchConnector for windows
> was not in fact asynchronous. I'm a little sceptical of that, though I
> admit I haven't looked into the issue at all.
>
> The underlying socket is reported to be established. As Ted suggests in
> his comment, perhaps that the link registry has not been notified or was
> unable to match the notification to the pending link.
>
> The connection id and federation link matching was very fragile in earlier
> versions (and seems now to be a little more robust). For example
> https://issues.apache.org/**jira/browse/QPID-4315<https://issues.apache.org/jira/browse/QPID-4315>,
> where SSL federation broke if hostnames were used qith qpid-route.
>
> Could a similar issue be (or have been) behind the windows problem?
>
> Has anyone tried with a recent broker? E.g. 0.22 or trunk? There have been
> quite a few changes in the codepaths involved.
>
>
> ------------------------------**------------------------------**---------
> To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.**org<de...@qpid.apache.org>
> For additional commands, e-mail: dev-help@qpid.apache.org
>
>