You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by Adel Boutros <ad...@live.com> on 2017/05/13 09:29:53 UTC
Dispatch router 2-phase start
Hello,
We have noticed a race conditon with the dispatch router's behavior.
If you have a producer and a consumer exchanging messages on a queue configured on a broker and accessible via the router. The consumer and producers are JMS clients configured with Failover options for retry.
If the router goes down, the retry mechanism will kick in until it is up again.
As we are configuring th addresses and connectors on the router dynamically, the producer and consumer might connect to the router before the waypointed address is created on it. In this case, a local address will be created on the router. The clients would exchange the messages directly from the router and the messages on the broker would never be consumed.
I discussed this issue with Justin and Ulf during the RivieraDev conference and we were wondering if it was possible to implement a 2-phase startup on the router to avoid this issue.
In that case, the router would start but not accept any connetion except for management. Once all dynamic configuration is done, we send a management message to allow the router to start accepting connections.
Any thoughts on this?
Regards,
Adel
Get Outlook for Android<https://aka.ms/ghei36>
Re: Dispatch router 2-phase start
Posted by Adel Boutros <Ad...@live.com>.
Hello Ganesh,
Done: https://issues.apache.org/jira/browse/DISPATCH-773
Indeed, I didn't see it could happen even to static config files.
Regards,
Adel
________________________________
From: Ganesh Murthy <gm...@redhat.com>
Sent: Monday, May 15, 2017 4:09:17 PM
To: users@qpid.apache.org
Subject: Re: Dispatch router 2-phase start
Hi Adel,
Can you please enter an enhancement JIRA for this so we can track this. This problem definitely needs to be addressed. I see this problem happening even in the case of static config files.
If you see here - https://github.com/apache/qpid-dispatch/blob/master/python/qpid_dispatch_internal/management/config.py#L167 - the listener is configured before the waypoint and address entities and there is a possibility that a producer and consumer might start directly exchanging messages if the listeners are setup are setup and the clients attach immediately thereafter and before the router could run thru the rest of the static configuration.
Thanks.
----- Original Message -----
> From: "Jiri Danek" <jd...@redhat.com>
> To: "users" <us...@qpid.apache.org>
> Sent: Monday, May 15, 2017 6:21:01 AM
> Subject: Re: Dispatch router 2-phase start
>
> On Mon, May 15, 2017 at 12:14 PM, Adel Boutros <Ad...@live.com> wrote:
>
> > Indeed, but I would not need an extra port in this case.
> >
> > Also, the extra management port could be used by mistake by any
> > misconfigured consumer/producer.
> >
>
> You could setup a policy to allow only management on this extra port...
>
>
> >
> > ________________________________
> > From: Jiri Danek <jd...@redhat.com>
> > Sent: Monday, May 15, 2017 12:04:50 PM
> > To: users
> > Subject: Re: Dispatch router 2-phase start
> >
> > On Mon, May 15, 2017 at 12:00 PM, Adel Boutros <Ad...@live.com>
> > wrote:
> >
> > > Hello Gordon,
> > >
> > >
> > > With what you are proposing, the order of the configuration becomes
> > > critical because if the public listener is configured before the
> > connectors
> > > and autolinks, I would have the same issue with the producer/consumer. So
> > > my management process would have to send the management commands in a
> > > predefined order.
> > >
> > >
> > > In the case of the 2-phase start, the order of configuration is
> > irrelevant
> > > and the management is thus easier.
> > >
> > >
> > > Regards,
> > >
> > > Adel
> > >
> >
> > On Sat, May 13, 2017 at 11:29 AM, Adel Boutros <ad...@live.com>
> > wrote:
> >
> > > Once all dynamic configuration is done, we send a management message to
> > > allow the router to start accepting connections.
> >
> >
> > So even with the 2-phase startup you'd have to keep the order of commands
> > in mind. You'd have to send this special startup command last.
> > --
> > Jiří Daněk
> > Messaging QA
> >
>
>
>
> --
> Jiří Daněk
> Messaging QA
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Dispatch router 2-phase start
Posted by Ganesh Murthy <gm...@redhat.com>.
Hi Adel,
Can you please enter an enhancement JIRA for this so we can track this. This problem definitely needs to be addressed. I see this problem happening even in the case of static config files.
If you see here - https://github.com/apache/qpid-dispatch/blob/master/python/qpid_dispatch_internal/management/config.py#L167 - the listener is configured before the waypoint and address entities and there is a possibility that a producer and consumer might start directly exchanging messages if the listeners are setup are setup and the clients attach immediately thereafter and before the router could run thru the rest of the static configuration.
Thanks.
----- Original Message -----
> From: "Jiri Danek" <jd...@redhat.com>
> To: "users" <us...@qpid.apache.org>
> Sent: Monday, May 15, 2017 6:21:01 AM
> Subject: Re: Dispatch router 2-phase start
>
> On Mon, May 15, 2017 at 12:14 PM, Adel Boutros <Ad...@live.com> wrote:
>
> > Indeed, but I would not need an extra port in this case.
> >
> > Also, the extra management port could be used by mistake by any
> > misconfigured consumer/producer.
> >
>
> You could setup a policy to allow only management on this extra port...
>
>
> >
> > ________________________________
> > From: Jiri Danek <jd...@redhat.com>
> > Sent: Monday, May 15, 2017 12:04:50 PM
> > To: users
> > Subject: Re: Dispatch router 2-phase start
> >
> > On Mon, May 15, 2017 at 12:00 PM, Adel Boutros <Ad...@live.com>
> > wrote:
> >
> > > Hello Gordon,
> > >
> > >
> > > With what you are proposing, the order of the configuration becomes
> > > critical because if the public listener is configured before the
> > connectors
> > > and autolinks, I would have the same issue with the producer/consumer. So
> > > my management process would have to send the management commands in a
> > > predefined order.
> > >
> > >
> > > In the case of the 2-phase start, the order of configuration is
> > irrelevant
> > > and the management is thus easier.
> > >
> > >
> > > Regards,
> > >
> > > Adel
> > >
> >
> > On Sat, May 13, 2017 at 11:29 AM, Adel Boutros <ad...@live.com>
> > wrote:
> >
> > > Once all dynamic configuration is done, we send a management message to
> > > allow the router to start accepting connections.
> >
> >
> > So even with the 2-phase startup you'd have to keep the order of commands
> > in mind. You'd have to send this special startup command last.
> > --
> > Jiří Daněk
> > Messaging QA
> >
>
>
>
> --
> Jiří Daněk
> Messaging QA
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Dispatch router 2-phase start
Posted by Jiri Danek <jd...@redhat.com>.
On Mon, May 15, 2017 at 12:14 PM, Adel Boutros <Ad...@live.com> wrote:
> Indeed, but I would not need an extra port in this case.
>
> Also, the extra management port could be used by mistake by any
> misconfigured consumer/producer.
>
You could setup a policy to allow only management on this extra port...
>
> ________________________________
> From: Jiri Danek <jd...@redhat.com>
> Sent: Monday, May 15, 2017 12:04:50 PM
> To: users
> Subject: Re: Dispatch router 2-phase start
>
> On Mon, May 15, 2017 at 12:00 PM, Adel Boutros <Ad...@live.com>
> wrote:
>
> > Hello Gordon,
> >
> >
> > With what you are proposing, the order of the configuration becomes
> > critical because if the public listener is configured before the
> connectors
> > and autolinks, I would have the same issue with the producer/consumer. So
> > my management process would have to send the management commands in a
> > predefined order.
> >
> >
> > In the case of the 2-phase start, the order of configuration is
> irrelevant
> > and the management is thus easier.
> >
> >
> > Regards,
> >
> > Adel
> >
>
> On Sat, May 13, 2017 at 11:29 AM, Adel Boutros <ad...@live.com>
> wrote:
>
> > Once all dynamic configuration is done, we send a management message to
> > allow the router to start accepting connections.
>
>
> So even with the 2-phase startup you'd have to keep the order of commands
> in mind. You'd have to send this special startup command last.
> --
> Jiří Daněk
> Messaging QA
>
--
Jiří Daněk
Messaging QA
Re: Dispatch router 2-phase start
Posted by Adel Boutros <Ad...@live.com>.
Indeed, but I would not need an extra port in this case.
Also, the extra management port could be used by mistake by any misconfigured consumer/producer.
________________________________
From: Jiri Danek <jd...@redhat.com>
Sent: Monday, May 15, 2017 12:04:50 PM
To: users
Subject: Re: Dispatch router 2-phase start
On Mon, May 15, 2017 at 12:00 PM, Adel Boutros <Ad...@live.com> wrote:
> Hello Gordon,
>
>
> With what you are proposing, the order of the configuration becomes
> critical because if the public listener is configured before the connectors
> and autolinks, I would have the same issue with the producer/consumer. So
> my management process would have to send the management commands in a
> predefined order.
>
>
> In the case of the 2-phase start, the order of configuration is irrelevant
> and the management is thus easier.
>
>
> Regards,
>
> Adel
>
On Sat, May 13, 2017 at 11:29 AM, Adel Boutros <ad...@live.com> wrote:
> Once all dynamic configuration is done, we send a management message to
> allow the router to start accepting connections.
So even with the 2-phase startup you'd have to keep the order of commands
in mind. You'd have to send this special startup command last.
--
Jiří Daněk
Messaging QA
Re: Dispatch router 2-phase start
Posted by Jiri Danek <jd...@redhat.com>.
On Mon, May 15, 2017 at 12:00 PM, Adel Boutros <Ad...@live.com> wrote:
> Hello Gordon,
>
>
> With what you are proposing, the order of the configuration becomes
> critical because if the public listener is configured before the connectors
> and autolinks, I would have the same issue with the producer/consumer. So
> my management process would have to send the management commands in a
> predefined order.
>
>
> In the case of the 2-phase start, the order of configuration is irrelevant
> and the management is thus easier.
>
>
> Regards,
>
> Adel
>
On Sat, May 13, 2017 at 11:29 AM, Adel Boutros <ad...@live.com> wrote:
> Once all dynamic configuration is done, we send a management message to
> allow the router to start accepting connections.
So even with the 2-phase startup you'd have to keep the order of commands
in mind. You'd have to send this special startup command last.
--
Jiří Daněk
Messaging QA
Re: Dispatch router 2-phase start
Posted by Adel Boutros <Ad...@live.com>.
Hello Gordon,
With what you are proposing, the order of the configuration becomes critical because if the public listener is configured before the connectors and autolinks, I would have the same issue with the producer/consumer. So my management process would have to send the management commands in a predefined order.
In the case of the 2-phase start, the order of configuration is irrelevant and the management is thus easier.
Regards,
Adel
________________________________
From: Gordon Sim <gs...@redhat.com>
Sent: Monday, May 15, 2017 9:58:50 AM
To: users@qpid.apache.org
Subject: Re: Dispatch router 2-phase start
On 13/05/17 10:29, Adel Boutros wrote:
> Hello,
>
> We have noticed a race conditon with the dispatch router's behavior.
> If you have a producer and a consumer exchanging messages on a queue configured on a broker and accessible via the router. The consumer and producers are JMS clients configured with Failover options for retry.
> If the router goes down, the retry mechanism will kick in until it is up again.
>
> As we are configuring th addresses and connectors on the router dynamically, the producer and consumer might connect to the router before the waypointed address is created on it. In this case, a local address will be created on the router. The clients would exchange the messages directly from the router and the messages on the broker would never be consumed.
>
> I discussed this issue with Justin and Ulf during the RivieraDev conference and we were wondering if it was possible to implement a 2-phase startup on the router to avoid this issue.
> In that case, the router would start but not accept any connetion except for management. Once all dynamic configuration is done, we send a management message to allow the router to start accepting connections.
>
> Any thoughts on this?
You could do something similar already by defining a specific listener
for management only in the static config, and then only define the
'public' listener for real clients once any other configuration has been
done.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Dispatch router 2-phase start
Posted by Gordon Sim <gs...@redhat.com>.
On 13/05/17 10:29, Adel Boutros wrote:
> Hello,
>
> We have noticed a race conditon with the dispatch router's behavior.
> If you have a producer and a consumer exchanging messages on a queue configured on a broker and accessible via the router. The consumer and producers are JMS clients configured with Failover options for retry.
> If the router goes down, the retry mechanism will kick in until it is up again.
>
> As we are configuring th addresses and connectors on the router dynamically, the producer and consumer might connect to the router before the waypointed address is created on it. In this case, a local address will be created on the router. The clients would exchange the messages directly from the router and the messages on the broker would never be consumed.
>
> I discussed this issue with Justin and Ulf during the RivieraDev conference and we were wondering if it was possible to implement a 2-phase startup on the router to avoid this issue.
> In that case, the router would start but not accept any connetion except for management. Once all dynamic configuration is done, we send a management message to allow the router to start accepting connections.
>
> Any thoughts on this?
You could do something similar already by defining a specific listener
for management only in the static config, and then only define the
'public' listener for real clients once any other configuration has been
done.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org