Bug AMQ-810 not fixed properly yet?

[xa...@nexgo.de]

 Hello,

with a network of brokers and conduit subscriptions there is a known bug in version 4.0.1, that under certain conditions prevents subscribers from getting events. 
(https://issues.apache.org/activemq/browse/AMQ-810)
According to the issue tracker the bug is fixed with version 4.0.2.
I know, that 4.0.2 is not released yet, so maybe I'm a little to early, but I retested the problem yesterday with 4.0.2-RC4 (http://people.apache.org/~chirino/incubator-activemq-4.0.2-RC4/maven1/incubator-activemq/distributions/) and found the behaviour has changed, but it's still not correct. Now there are duplicated events.
Should I submit a bug report, or is the bug-fixing for AMQ-810 still ongoing? 

Regards
Linda Floren
----

My test does the following:
I have two brokers A and B linked to each other with conduitSubscriptions=true.
I have a publisher on A and subscribers with messageSelectors on B (and an identical set of subscribers on A for sanity check).

In 4.0.1 the behavior was as follows:
On broker B only the subscribers with a messageSelector that was a subset of the messageSelector of the first subscription made on B received events. Subscribers with different messageSelectors received nothing.

With 4.0.2-RC4 the behaviour is as follows:
If on broker B n subscribers have identical messageSelectors, they receive each matching event n times!

Without messageSelectors everything works fine.


Viel oder wenig? Schnell oder langsam? Unbegrenzt surfen + telefonieren
ohne Zeit- und Volumenbegrenzung? DAS TOP ANGEBOT JETZT bei Arcor: günstig
und schnell mit DSL - das All-Inclusive-Paket für clevere Doppel-Sparer,
nur  44,85 €  inkl. DSL- und ISDN-Grundgebühr!
http://www.arcor.de/rd/emf-dsl-2

Re: Bug AMQ-810 not fixed properly yet?

[Xalibur <xa...@nexgo.de>]

Hello Hiram!

> 
> We need to think of a better approach. Off the top of my head, we 
> could... :
> 
> 1) only forward the first subscription, but always strip off any 
> selector. That way message to the topic are always properly 
> replicated. The down side: if that in a big network this kills your 
> scaleability since every broker has to process every message. Might 
> not be a problem if you have consumers on every node that need to see
> every message (but this is typically not the case).
> 
> 2) if a subscription has a selector, then send a copy per 
> subscription but somehow target a specific consumer on the remote 
> broker so that the remote broker does not broadcast the message again
>  to all it subscribers. Implementing this might be a bit trickier but
>  should preseve scalability of a broker network.
> 
> Any other ideas?

Well, what about delivering an event only once per connection?
If the dispatching process for an event kept a list of connections the
event was already delivered to, it would be possible to prevent
delivering it over the same connection a second time. The event would
reach the remote broker only once, even if there were more than one
matching conduit subscription. And the remote broker would dispatch it
to any matching local subscription. That approach would maintain the
scalability of the network, but I don't know how much it would spoil the
efficiency of the dispatching process.


Best regards
Linda

Re: Bug AMQ-810 not fixed properly yet?

[Hiram Chirino <hi...@hiramchirino.com>]

Hi xalibur..

you have hit a situation where we need some further redesign to fix this
properly.  The original bug was reported because of the exact opposite
behaviour of what your are seeing.

Basically, we have 2 subscribers on the same topic where the selectors are
partitioning the trafic.  Think selectorA = 'price > 10' and selectorB =
'price <= 10'.  If selectorB was registered after selectorA, then it would
not receive any messages since the 2nd subscription was not being forwarded.

The fix was simplistic in that it said, hey if the consumer is using
selectors, then lets forward the subscription.  But in your case your
selectors do not partition the traffic.. they actually share some of the
traffic.  Those shared messages that mach both selectors get duplicated :(

We need to think of a better approach.  Off the top of my head, we could...
:

1) only forward the first subscription, but always strip off any selector.
That way message to the topic are always properly replicated.  The down side
if that in a big network this kills your scaleability since every broker has
to process every message.  Might not be a problem if you have consumers on
every node that need to see every message (but this is typically not the
case).

2) if a subscription has a selector, then send a copy per subscription but
somehow target a specific consumer on the remote broker so that the remote
broker does not broadcast the message again to all it subscribers.
Implementing this might be a bit trickier but should preseve scalability of
a broker network.

Any other ideas?


On 10/18/06, xalibur@nexgo.de <xa...@nexgo.de> wrote:
>
> Hello,
>
> with a network of brokers and conduit subscriptions there is a known bug
> in version 4.0.1, that under certain conditions prevents subscribers from
> getting events.
> (https://issues.apache.org/activemq/browse/AMQ-810)
> According to the issue tracker the bug is fixed with version 4.0.2.
> I know, that 4.0.2 is not released yet, so maybe I'm a little to early,
> but I retested the problem yesterday with 4.0.2-RC4 (
> http://people.apache.org/~chirino/incubator-activemq-4.0.2-RC4/maven1/incubator-activemq/distributions/)
> and found the behaviour has changed, but it's still not correct. Now there
> are duplicated events.
> Should I submit a bug report, or is the bug-fixing for AMQ-810 still
> ongoing?
>
> Regards
> Linda Floren
> ----
>
> My test does the following:
> I have two brokers A and B linked to each other with
> conduitSubscriptions=true.
> I have a publisher on A and subscribers with messageSelectors on B (and an
> identical set of subscribers on A for sanity check).
>
> In 4.0.1 the behavior was as follows:
> On broker B only the subscribers with a messageSelector that was a subset
> of the messageSelector of the first subscription made on B received events.
> Subscribers with different messageSelectors received nothing.
>
> With 4.0.2-RC4 the behaviour is as follows:
> If on broker B n subscribers have identical messageSelectors, they receive
> each matching event n times!
>
> Without messageSelectors everything works fine.
>
>
> Viel oder wenig? Schnell oder langsam? Unbegrenzt surfen + telefonieren
> ohne Zeit- und Volumenbegrenzung? DAS TOP ANGEBOT JETZT bei Arcor: günstig
> und schnell mit DSL - das All-Inclusive-Paket für clevere Doppel-Sparer,
> nur  44,85 €  inkl. DSL- und ISDN-Grundgebühr!
> http://www.arcor.de/rd/emf-dsl-2
>



-- 
Regards,
Hiram

Blog: http://hiramchirino.com