You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by "Petrenko, Vadim" <va...@ns.nl> on 2020/10/14 12:16:54 UTC
Non-blocking Sender with no Receivers
Good day,
We’re trying to implement our Qpid dispatch router network in such a way that a sender can connect to it at any moment and start sending messages to a topic.
There might be no receivers at this moment, so the messages would be just lost. This is ok.
Then an any moment a receiver(s) can connect and start receiving messages currently being sent, then drop off. The sender will just continue sending messages, probably into the void.
It’s about sending data from sensors. Sensor data is short living and there is no need to buffer it or persist. If no one consumes this data in realtime, it can be lost.
Senders and receivers in our situation are quite volatile, they appear and drop off.
As per docs of Qpid:
===
Dispatch Router uses a credit-based flow control mechanism to ensure that producers can only send messages to a router if at least one consumer is available to receive them. Because Dispatch Router does not store messages, this credit-based flow control prevents producers from sending messages when there are no consumers present.
===
I was wondering, maybe there is still a workaround or a trick to implement the router network the way I described in the beginning?
If possible without extra brokers and only with Qpid dispatch routers?
Thanks.
________________________________
Deze e-mail, inclusief eventuele bijlagen, is uitsluitend bestemd voor (gebruik door) de geadresseerde. De e-mail kan persoonlijke of vertrouwelijke informatie bevatten. Openbaarmaking, vermenigvuldiging, verspreiding en/of verstrekking van (de inhoud van) deze e-mail (en eventuele bijlagen) aan derden is uitdrukkelijk niet toegestaan. Indien u niet de bedoelde geadresseerde bent, wordt u vriendelijk verzocht degene die de e-mail verzond hiervan direct op de hoogte te brengen en de e-mail (en eventuele bijlagen) te vernietigen.
Informatie vennootschap<http://www.ns.nl/emaildisclaimer>
Re: Non-blocking Sender with no Receivers
Posted by Robbie Gemmell <ro...@gmail.com>.
I expect Gerard means that if you also created a receiver that
received your multicasted messages, then you would definitely be
granted credit to send to the address, since you have ensured there is
definitely a receiver (you, plus maybe others)
On Wed, 14 Oct 2020 at 14:27, Petrenko, Vadim <va...@ns.nl> wrote:
>
> Hi Gerard, could you possibly elaborate on this? We want senders and receivers be completely separated, they are different applications.
>
> -----Original Message-----
> From: Weatherby,Gerard <gw...@uchc.edu>
> Sent: woensdag 14 oktober 2020 14:20
> To: users@qpid.apache.org
> Subject: Re: Non-blocking Sender with no Receivers
>
> I don’t know if this would work, but have you tried making your senders receivers of their own messages?
> --
> Gerard Weatherby | Application Architect NMRbox | Department of Molecular Biology and Biophysics | UConn Health
> 263 Farmington Avenue, Farmington, CT 06030-6406 uchc.edu
>
> > On Oct 14, 2020, at 8:16 AM, Petrenko, Vadim <va...@ns.nl> wrote:
> >
> > *** Attention: This is an external email. Use caution responding,
> > opening attachments or clicking on links. ***
> >
> > Good day,
> >
> > We’re trying to implement our Qpid dispatch router network in such a way that a sender can connect to it at any moment and start sending messages to a topic.
> > There might be no receivers at this moment, so the messages would be just lost. This is ok.
> > Then an any moment a receiver(s) can connect and start receiving messages currently being sent, then drop off. The sender will just continue sending messages, probably into the void.
> >
> > It’s about sending data from sensors. Sensor data is short living and there is no need to buffer it or persist. If no one consumes this data in realtime, it can be lost.
> > Senders and receivers in our situation are quite volatile, they appear and drop off.
> >
> > As per docs of Qpid:
> > ===
> > Dispatch Router uses a credit-based flow control mechanism to ensure that producers can only send messages to a router if at least one consumer is available to receive them. Because Dispatch Router does not store messages, this credit-based flow control prevents producers from sending messages when there are no consumers present.
> > ===
> >
> > I was wondering, maybe there is still a workaround or a trick to implement the router network the way I described in the beginning?
> > If possible without extra brokers and only with Qpid dispatch routers?
> >
> > Thanks.
> >
> >
> > ________________________________
> >
> > Deze e-mail, inclusief eventuele bijlagen, is uitsluitend bestemd voor (gebruik door) de geadresseerde. De e-mail kan persoonlijke of vertrouwelijke informatie bevatten. Openbaarmaking, vermenigvuldiging, verspreiding en/of verstrekking van (de inhoud van) deze e-mail (en eventuele bijlagen) aan derden is uitdrukkelijk niet toegestaan. Indien u niet de bedoelde geadresseerde bent, wordt u vriendelijk verzocht degene die de e-mail verzond hiervan direct op de hoogte te brengen en de e-mail (en eventuele bijlagen) te vernietigen.
> >
> > Informatie
> > vennootschap<https://urldefense.com/v3/__http://www.ns.nl/emaildisclai
> > mer__;!!N0rdg9Wr!6nw7f_PTi-MsxBR4wWT6OhEcRjT3xrahFXs8pbYzYdtf6O6DZb-nj
> > fmu49tteveosDU$ >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org For additional commands, e-mail: users-help@qpid.apache.org
>
>
> ________________________________
>
> Deze e-mail, inclusief eventuele bijlagen, is uitsluitend bestemd voor (gebruik door) de geadresseerde. De e-mail kan persoonlijke of vertrouwelijke informatie bevatten. Openbaarmaking, vermenigvuldiging, verspreiding en/of verstrekking van (de inhoud van) deze e-mail (en eventuele bijlagen) aan derden is uitdrukkelijk niet toegestaan. Indien u niet de bedoelde geadresseerde bent, wordt u vriendelijk verzocht degene die de e-mail verzond hiervan direct op de hoogte te brengen en de e-mail (en eventuele bijlagen) te vernietigen.
>
> Informatie vennootschap<http://www.ns.nl/emaildisclaimer>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
RE: Non-blocking Sender with no Receivers
Posted by "Petrenko, Vadim" <va...@ns.nl>.
Hi Gerard, could you possibly elaborate on this? We want senders and receivers be completely separated, they are different applications.
-----Original Message-----
From: Weatherby,Gerard <gw...@uchc.edu>
Sent: woensdag 14 oktober 2020 14:20
To: users@qpid.apache.org
Subject: Re: Non-blocking Sender with no Receivers
I don’t know if this would work, but have you tried making your senders receivers of their own messages?
--
Gerard Weatherby | Application Architect NMRbox | Department of Molecular Biology and Biophysics | UConn Health
263 Farmington Avenue, Farmington, CT 06030-6406 uchc.edu
> On Oct 14, 2020, at 8:16 AM, Petrenko, Vadim <va...@ns.nl> wrote:
>
> *** Attention: This is an external email. Use caution responding,
> opening attachments or clicking on links. ***
>
> Good day,
>
> We’re trying to implement our Qpid dispatch router network in such a way that a sender can connect to it at any moment and start sending messages to a topic.
> There might be no receivers at this moment, so the messages would be just lost. This is ok.
> Then an any moment a receiver(s) can connect and start receiving messages currently being sent, then drop off. The sender will just continue sending messages, probably into the void.
>
> It’s about sending data from sensors. Sensor data is short living and there is no need to buffer it or persist. If no one consumes this data in realtime, it can be lost.
> Senders and receivers in our situation are quite volatile, they appear and drop off.
>
> As per docs of Qpid:
> ===
> Dispatch Router uses a credit-based flow control mechanism to ensure that producers can only send messages to a router if at least one consumer is available to receive them. Because Dispatch Router does not store messages, this credit-based flow control prevents producers from sending messages when there are no consumers present.
> ===
>
> I was wondering, maybe there is still a workaround or a trick to implement the router network the way I described in the beginning?
> If possible without extra brokers and only with Qpid dispatch routers?
>
> Thanks.
>
>
> ________________________________
>
> Deze e-mail, inclusief eventuele bijlagen, is uitsluitend bestemd voor (gebruik door) de geadresseerde. De e-mail kan persoonlijke of vertrouwelijke informatie bevatten. Openbaarmaking, vermenigvuldiging, verspreiding en/of verstrekking van (de inhoud van) deze e-mail (en eventuele bijlagen) aan derden is uitdrukkelijk niet toegestaan. Indien u niet de bedoelde geadresseerde bent, wordt u vriendelijk verzocht degene die de e-mail verzond hiervan direct op de hoogte te brengen en de e-mail (en eventuele bijlagen) te vernietigen.
>
> Informatie
> vennootschap<https://urldefense.com/v3/__http://www.ns.nl/emaildisclai
> mer__;!!N0rdg9Wr!6nw7f_PTi-MsxBR4wWT6OhEcRjT3xrahFXs8pbYzYdtf6O6DZb-nj
> fmu49tteveosDU$ >
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org For additional commands, e-mail: users-help@qpid.apache.org
________________________________
Deze e-mail, inclusief eventuele bijlagen, is uitsluitend bestemd voor (gebruik door) de geadresseerde. De e-mail kan persoonlijke of vertrouwelijke informatie bevatten. Openbaarmaking, vermenigvuldiging, verspreiding en/of verstrekking van (de inhoud van) deze e-mail (en eventuele bijlagen) aan derden is uitdrukkelijk niet toegestaan. Indien u niet de bedoelde geadresseerde bent, wordt u vriendelijk verzocht degene die de e-mail verzond hiervan direct op de hoogte te brengen en de e-mail (en eventuele bijlagen) te vernietigen.
Informatie vennootschap<http://www.ns.nl/emaildisclaimer>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Non-blocking Sender with no Receivers
Posted by "Weatherby,Gerard" <gw...@uchc.edu>.
I don’t know if this would work, but have you tried making your senders receivers of their own messages?
--
Gerard Weatherby | Application Architect
NMRbox | Department of Molecular Biology and Biophysics | UConn Health
263 Farmington Avenue, Farmington, CT 06030-6406
uchc.edu
> On Oct 14, 2020, at 8:16 AM, Petrenko, Vadim <va...@ns.nl> wrote:
>
> *** Attention: This is an external email. Use caution responding, opening attachments or clicking on links. ***
>
> Good day,
>
> We’re trying to implement our Qpid dispatch router network in such a way that a sender can connect to it at any moment and start sending messages to a topic.
> There might be no receivers at this moment, so the messages would be just lost. This is ok.
> Then an any moment a receiver(s) can connect and start receiving messages currently being sent, then drop off. The sender will just continue sending messages, probably into the void.
>
> It’s about sending data from sensors. Sensor data is short living and there is no need to buffer it or persist. If no one consumes this data in realtime, it can be lost.
> Senders and receivers in our situation are quite volatile, they appear and drop off.
>
> As per docs of Qpid:
> ===
> Dispatch Router uses a credit-based flow control mechanism to ensure that producers can only send messages to a router if at least one consumer is available to receive them. Because Dispatch Router does not store messages, this credit-based flow control prevents producers from sending messages when there are no consumers present.
> ===
>
> I was wondering, maybe there is still a workaround or a trick to implement the router network the way I described in the beginning?
> If possible without extra brokers and only with Qpid dispatch routers?
>
> Thanks.
>
>
> ________________________________
>
> Deze e-mail, inclusief eventuele bijlagen, is uitsluitend bestemd voor (gebruik door) de geadresseerde. De e-mail kan persoonlijke of vertrouwelijke informatie bevatten. Openbaarmaking, vermenigvuldiging, verspreiding en/of verstrekking van (de inhoud van) deze e-mail (en eventuele bijlagen) aan derden is uitdrukkelijk niet toegestaan. Indien u niet de bedoelde geadresseerde bent, wordt u vriendelijk verzocht degene die de e-mail verzond hiervan direct op de hoogte te brengen en de e-mail (en eventuele bijlagen) te vernietigen.
>
> Informatie vennootschap<https://urldefense.com/v3/__http://www.ns.nl/emaildisclaimer__;!!N0rdg9Wr!6nw7f_PTi-MsxBR4wWT6OhEcRjT3xrahFXs8pbYzYdtf6O6DZb-njfmu49tteveosDU$ >
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
RE: Non-blocking Sender with no Receivers
Posted by "Petrenko, Vadim" <va...@ns.nl>.
Many thanks to all of you guys for the valuable feedback. I'm going to evaluate suggested approaches and will communicate results back here.
Have a great weekend.
-----Original Message-----
From: Ken Giusti <kg...@redhat.com>
Sent: woensdag 14 oktober 2020 20:05
To: users <us...@qpid.apache.org>
Subject: Re: Non-blocking Sender with no Receivers
On Wed, Oct 14, 2020 at 11:05 AM Petrenko, Vadim <va...@ns.nl>
wrote:
> Hi Ken,
>
> Yes, we're pre-settling messages already:
> connectionfactory.myFactoryLookupSender =
> amqp://localhost:5674?jms.presettlePolicy.presettleProducers=true
> (dockerized Qpid with an exposed port).
>
> Making the sending client first check for available credits is a
> possibility, but not the ideal option we'd like to use. These clients
> are not in our control, it's other teams that develop their
> applications using our Qpid network. So we can't really enforce them. Only advise.
>
> I also noticed some strange behavior which I don't understand. Maybe
> you guys can explain.
> I've taken the Sender and Receiver from qpid-jms-examples, connected
> them to our router network and observed how this works.
> 1. When I start the Sender, it gets blocked because there is no
> receiver (in the logs I see "Holding Message send until credit is
> available"). This is as expected.
> 2. I start the Receiver, messages begin to flow, all good.
> 3. I kill the Receiver process, but the Sender is still sending messages.
> To nowhere. It doesn't timeout either.
>
> I then proceeded debugging AmqpFixedProducer, which works with
> credits, and noticed that on step 1 number of credits it gets via
> getEndpoint().getCredit() is 0. So it holds the message.
> On the next step Receiver gives it 250 credits. After that, even
> though the Receiver process is killed, the number of credits is still
> 250 and doesn't get decremented. The Sender continues to send. I
> expect that credits would be exhausted at some point.
> Is this as it should be?
>
If the sender's link is anonymous as Robbie described then yes credit will be replenished regardless of the presence of your receiver.
If the sender's link is not anonymous I would expect that 250 outstanding credits will be exhausted eventually. Once the router learns of the loss of all receivers it should no longer replenish credit to the sender.
The router doesn't have a way to revoke credit btw - any extra credit outstanding at the sender when the receivers go away will remain until the sender either uses them or disconnects.
The 250 value is configurable - use the "linkCapacity" attribute on the "listener" configuration entity. For example this changes the maximum outstanding credit limit from 250 (default) down to 10 :
listener {
host: 0.0.0.0
port: amqp
authenticatePeer: no
saslMechanisms: ANONYMOUS
linkCapacity: 10
}
>
> Thanks.
>
>
> -----Original Message-----
> From: Ken Giusti <kg...@redhat.com>
> Sent: woensdag 14 oktober 2020 15:18
> To: users <us...@qpid.apache.org>
> Subject: Re: Non-blocking Sender with no Receivers
>
> On Wed, Oct 14, 2020 at 8:25 AM Petrenko, Vadim <va...@ns.nl>
> wrote:
>
> > Good day,
> >
> > We’re trying to implement our Qpid dispatch router network in such a
> > way that a sender can connect to it at any moment and start sending
> > messages to a topic.
> > There might be no receivers at this moment, so the messages would be
> > just lost. This is ok.
> > Then an any moment a receiver(s) can connect and start receiving
> > messages currently being sent, then drop off. The sender will just
> > continue sending messages, probably into the void.
> >
> > It’s about sending data from sensors. Sensor data is short living
> > and there is no need to buffer it or persist. If no one consumes
> > this data in realtime, it can be lost.
> > Senders and receivers in our situation are quite volatile, they
> > appear and drop off.
> >
> > As per docs of Qpid:
> > ===
> > Dispatch Router uses a credit-based flow control mechanism to ensure
> > that producers can only send messages to a router if at least one
> > consumer is available to receive them. Because Dispatch Router does
> > not store messages, this credit-based flow control prevents
> > producers from sending messages when there are no consumers present.
> > ===
> >
> > I was wondering, maybe there is still a workaround or a trick to
> > implement the router network the way I described in the beginning?
> > If possible without extra brokers and only with Qpid dispatch routers?
> >
> > Thanks.
> >
> >
> Can your sending client first check whether or not there is credit on
> the link and if there is no credit simply drop the message rather than
> sending it?
> Getting the value of available credit depends on the client API you
> are using. For example in Proton-C your sender would call
> pn_link_credit(sending_link_ptr) to get the credit for the sending link.
>
> Also keep in mind that since the application is OK with messages being
> dropped you'll want the sender to settle the message after it is sent
> (a.k.a. "pre-settled).
>
>
>
> >
> > ________________________________
> >
> > Deze e-mail, inclusief eventuele bijlagen, is uitsluitend bestemd
> > voor (gebruik door) de geadresseerde. De e-mail kan persoonlijke of
> > vertrouwelijke informatie bevatten. Openbaarmaking,
> > vermenigvuldiging, verspreiding en/of verstrekking van (de inhoud
> > van) deze e-mail (en eventuele bijlagen) aan derden is uitdrukkelijk niet toegestaan.
> > Indien u niet de bedoelde geadresseerde bent, wordt u vriendelijk
> > verzocht degene die de e-mail verzond hiervan direct op de hoogte te
> > brengen en de e-mail (en eventuele bijlagen) te vernietigen.
> >
> > Informatie vennootschap<http://www.ns.nl/emaildisclaimer>
> >
>
>
> --
> -K
>
> ________________________________
>
> Deze e-mail, inclusief eventuele bijlagen, is uitsluitend bestemd voor
> (gebruik door) de geadresseerde. De e-mail kan persoonlijke of
> vertrouwelijke informatie bevatten. Openbaarmaking, vermenigvuldiging,
> verspreiding en/of verstrekking van (de inhoud van) deze e-mail (en
> eventuele bijlagen) aan derden is uitdrukkelijk niet toegestaan.
> Indien u niet de bedoelde geadresseerde bent, wordt u vriendelijk
> verzocht degene die de e-mail verzond hiervan direct op de hoogte te
> brengen en de e-mail (en eventuele bijlagen) te vernietigen.
>
> Informatie vennootschap<http://www.ns.nl/emaildisclaimer>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org For
> additional commands, e-mail: users-help@qpid.apache.org
>
>
--
-K
________________________________
Deze e-mail, inclusief eventuele bijlagen, is uitsluitend bestemd voor (gebruik door) de geadresseerde. De e-mail kan persoonlijke of vertrouwelijke informatie bevatten. Openbaarmaking, vermenigvuldiging, verspreiding en/of verstrekking van (de inhoud van) deze e-mail (en eventuele bijlagen) aan derden is uitdrukkelijk niet toegestaan. Indien u niet de bedoelde geadresseerde bent, wordt u vriendelijk verzocht degene die de e-mail verzond hiervan direct op de hoogte te brengen en de e-mail (en eventuele bijlagen) te vernietigen.
Informatie vennootschap<http://www.ns.nl/emaildisclaimer>
Re: Non-blocking Sender with no Receivers
Posted by Ken Giusti <kg...@redhat.com>.
On Wed, Oct 14, 2020 at 11:05 AM Petrenko, Vadim <va...@ns.nl>
wrote:
> Hi Ken,
>
> Yes, we're pre-settling messages already:
> connectionfactory.myFactoryLookupSender =
> amqp://localhost:5674?jms.presettlePolicy.presettleProducers=true
> (dockerized Qpid with an exposed port).
>
> Making the sending client first check for available credits is a
> possibility, but not the ideal option we'd like to use. These clients are
> not in our control, it's other teams that develop their applications using
> our Qpid network. So we can't really enforce them. Only advise.
>
> I also noticed some strange behavior which I don't understand. Maybe you
> guys can explain.
> I've taken the Sender and Receiver from qpid-jms-examples, connected them
> to our router network and observed how this works.
> 1. When I start the Sender, it gets blocked because there is no receiver
> (in the logs I see "Holding Message send until credit is available"). This
> is as expected.
> 2. I start the Receiver, messages begin to flow, all good.
> 3. I kill the Receiver process, but the Sender is still sending messages.
> To nowhere. It doesn't timeout either.
>
> I then proceeded debugging AmqpFixedProducer, which works with credits,
> and noticed that on step 1 number of credits it gets via
> getEndpoint().getCredit() is 0. So it holds the message.
> On the next step Receiver gives it 250 credits. After that, even though
> the Receiver process is killed, the number of credits is still 250 and
> doesn't get decremented. The Sender continues to send. I expect that
> credits would be exhausted at some point.
> Is this as it should be?
>
If the sender's link is anonymous as Robbie described then yes credit will
be replenished regardless of the presence of your receiver.
If the sender's link is not anonymous I would expect that 250 outstanding
credits will be exhausted eventually. Once the router learns of the loss of
all receivers it should no longer replenish credit to the sender.
The router doesn't have a way to revoke credit btw - any extra credit
outstanding at the sender when the receivers go away will remain until the
sender either uses them or disconnects.
The 250 value is configurable - use the "linkCapacity" attribute on the
"listener" configuration entity. For example this changes the maximum
outstanding credit limit from 250 (default) down to 10 :
listener {
host: 0.0.0.0
port: amqp
authenticatePeer: no
saslMechanisms: ANONYMOUS
linkCapacity: 10
}
>
> Thanks.
>
>
> -----Original Message-----
> From: Ken Giusti <kg...@redhat.com>
> Sent: woensdag 14 oktober 2020 15:18
> To: users <us...@qpid.apache.org>
> Subject: Re: Non-blocking Sender with no Receivers
>
> On Wed, Oct 14, 2020 at 8:25 AM Petrenko, Vadim <va...@ns.nl>
> wrote:
>
> > Good day,
> >
> > We’re trying to implement our Qpid dispatch router network in such a
> > way that a sender can connect to it at any moment and start sending
> > messages to a topic.
> > There might be no receivers at this moment, so the messages would be
> > just lost. This is ok.
> > Then an any moment a receiver(s) can connect and start receiving
> > messages currently being sent, then drop off. The sender will just
> > continue sending messages, probably into the void.
> >
> > It’s about sending data from sensors. Sensor data is short living and
> > there is no need to buffer it or persist. If no one consumes this data
> > in realtime, it can be lost.
> > Senders and receivers in our situation are quite volatile, they appear
> > and drop off.
> >
> > As per docs of Qpid:
> > ===
> > Dispatch Router uses a credit-based flow control mechanism to ensure
> > that producers can only send messages to a router if at least one
> > consumer is available to receive them. Because Dispatch Router does
> > not store messages, this credit-based flow control prevents producers
> > from sending messages when there are no consumers present.
> > ===
> >
> > I was wondering, maybe there is still a workaround or a trick to
> > implement the router network the way I described in the beginning?
> > If possible without extra brokers and only with Qpid dispatch routers?
> >
> > Thanks.
> >
> >
> Can your sending client first check whether or not there is credit on the
> link and if there is no credit simply drop the message rather than sending
> it?
> Getting the value of available credit depends on the client API you are
> using. For example in Proton-C your sender would call
> pn_link_credit(sending_link_ptr) to get the credit for the sending link.
>
> Also keep in mind that since the application is OK with messages being
> dropped you'll want the sender to settle the message after it is sent
> (a.k.a. "pre-settled).
>
>
>
> >
> > ________________________________
> >
> > Deze e-mail, inclusief eventuele bijlagen, is uitsluitend bestemd voor
> > (gebruik door) de geadresseerde. De e-mail kan persoonlijke of
> > vertrouwelijke informatie bevatten. Openbaarmaking, vermenigvuldiging,
> > verspreiding en/of verstrekking van (de inhoud van) deze e-mail (en
> > eventuele bijlagen) aan derden is uitdrukkelijk niet toegestaan.
> > Indien u niet de bedoelde geadresseerde bent, wordt u vriendelijk
> > verzocht degene die de e-mail verzond hiervan direct op de hoogte te
> > brengen en de e-mail (en eventuele bijlagen) te vernietigen.
> >
> > Informatie vennootschap<http://www.ns.nl/emaildisclaimer>
> >
>
>
> --
> -K
>
> ________________________________
>
> Deze e-mail, inclusief eventuele bijlagen, is uitsluitend bestemd voor
> (gebruik door) de geadresseerde. De e-mail kan persoonlijke of
> vertrouwelijke informatie bevatten. Openbaarmaking, vermenigvuldiging,
> verspreiding en/of verstrekking van (de inhoud van) deze e-mail (en
> eventuele bijlagen) aan derden is uitdrukkelijk niet toegestaan. Indien u
> niet de bedoelde geadresseerde bent, wordt u vriendelijk verzocht degene
> die de e-mail verzond hiervan direct op de hoogte te brengen en de e-mail
> (en eventuele bijlagen) te vernietigen.
>
> Informatie vennootschap<http://www.ns.nl/emaildisclaimer>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
--
-K
Re: Non-blocking Sender with no Receivers
Posted by Robbie Gemmell <ro...@gmail.com>.
On Wed, 14 Oct 2020 at 17:18, Robbie Gemmell <ro...@gmail.com> wrote:
>
> On Wed, 14 Oct 2020 at 15:55, Petrenko, Vadim <va...@ns.nl> wrote:
> >
> > Hi Ken,
> >
> > Yes, we're pre-settling messages already:
> > connectionfactory.myFactoryLookupSender = amqp://localhost:5674?jms.presettlePolicy.presettleProducers=true
> > (dockerized Qpid with an exposed port).
> >
> > Making the sending client first check for available credits is a possibility, but not the ideal option we'd like to use. These clients are not in our control, it's other teams that develop their applications using our Qpid network. So we can't really enforce them. Only advise.
> >
>
> The JMS client can't do that anyway.
>
> > I also noticed some strange behavior which I don't understand. Maybe you guys can explain.
> > I've taken the Sender and Receiver from qpid-jms-examples, connected them to our router network and observed how this works.
> > 1. When I start the Sender, it gets blocked because there is no receiver (in the logs I see "Holding Message send until credit is available"). This is as expected.
> > 2. I start the Receiver, messages begin to flow, all good.
> > 3. I kill the Receiver process, but the Sender is still sending messages. To nowhere. It doesn't timeout either.
>
> It's sending them to the router, which it still has outstanding credit
> to do. It knows nothing of the receiver.
>
> It won't time out a send unless you configure it to, and note it
> certainly won't time out in this case if it has credit to send and is
> sending pre-settled messages, since there is also no response to wait
> for.
>
> >
> > I then proceeded debugging AmqpFixedProducer, which works with credits, and noticed that on step 1 number of credits it gets via getEndpoint().getCredit() is 0. So it holds the message.
> > On the next step Receiver gives it 250 credits. After that, even though the Receiver process is killed, the number of credits is still 250 and doesn't get decremented. The Sender continues to send. I expect that credits would be exhausted at some point.
> > Is this as it should be?
>
> Are you sure it 'doesn't get decremented' (which happens locally as
> sends occur), or are you perhaps just not observing the credit is
> being replenished by the router? You can see the protocol trace on
> router and/or the client by running them with env variable
> PN_TRACE_FRM=1 set (or also for client, see:
> http://qpid.apache.org/releases/qpid-jms-0.54.0/docs/index.html#logging)
>
> As you are using message-routing, the router itself is what grants the
> sender credit to send. The receiver also grants credit but that is to
> the router, and is quite independent of the credit the sender will see
> from the router (credit would be aligned end to end for a link-routing
> case, e.g through router to/from a broker).
>
> https://issues.apache.org/jira/browse/DISPATCH-1090 is the last I can
> think of the behaviour in this area being tweaked, which was to try
> and drain the outstanding credit in similar cases, however in checking
> the vicinity of those changes now, I see edge routers being called out
> and handled differently, indicating it will replenish the credit for
> such messages. Your other thread mentioned edge routers so I think
> perhaps this is in play here:
> https://github.com/apache/qpid-dispatch/blob/1.14.0/src/router_core/transfer.c#L467-L476
>
Actually, though it may still be in play in the larger picture, it
won't be in the way I was originally thinking since I wasn't
considering the correct link. Needs folks with more(/any) router
knowledge than me to reason about :)
> >
> > Thanks.
> >
> >
> > -----Original Message-----
> > From: Ken Giusti <kg...@redhat.com>
> > Sent: woensdag 14 oktober 2020 15:18
> > To: users <us...@qpid.apache.org>
> > Subject: Re: Non-blocking Sender with no Receivers
> >
> > On Wed, Oct 14, 2020 at 8:25 AM Petrenko, Vadim <va...@ns.nl>
> > wrote:
> >
> > > Good day,
> > >
> > > We’re trying to implement our Qpid dispatch router network in such a
> > > way that a sender can connect to it at any moment and start sending
> > > messages to a topic.
> > > There might be no receivers at this moment, so the messages would be
> > > just lost. This is ok.
> > > Then an any moment a receiver(s) can connect and start receiving
> > > messages currently being sent, then drop off. The sender will just
> > > continue sending messages, probably into the void.
> > >
> > > It’s about sending data from sensors. Sensor data is short living and
> > > there is no need to buffer it or persist. If no one consumes this data
> > > in realtime, it can be lost.
> > > Senders and receivers in our situation are quite volatile, they appear
> > > and drop off.
> > >
> > > As per docs of Qpid:
> > > ===
> > > Dispatch Router uses a credit-based flow control mechanism to ensure
> > > that producers can only send messages to a router if at least one
> > > consumer is available to receive them. Because Dispatch Router does
> > > not store messages, this credit-based flow control prevents producers
> > > from sending messages when there are no consumers present.
> > > ===
> > >
> > > I was wondering, maybe there is still a workaround or a trick to
> > > implement the router network the way I described in the beginning?
> > > If possible without extra brokers and only with Qpid dispatch routers?
> > >
> > > Thanks.
> > >
> > >
> > Can your sending client first check whether or not there is credit on the link and if there is no credit simply drop the message rather than sending it?
> > Getting the value of available credit depends on the client API you are using. For example in Proton-C your sender would call
> > pn_link_credit(sending_link_ptr) to get the credit for the sending link.
> >
> > Also keep in mind that since the application is OK with messages being dropped you'll want the sender to settle the message after it is sent (a.k.a. "pre-settled).
> >
> >
> >
> > >
> > > ________________________________
> > >
> > > Deze e-mail, inclusief eventuele bijlagen, is uitsluitend bestemd voor
> > > (gebruik door) de geadresseerde. De e-mail kan persoonlijke of
> > > vertrouwelijke informatie bevatten. Openbaarmaking, vermenigvuldiging,
> > > verspreiding en/of verstrekking van (de inhoud van) deze e-mail (en
> > > eventuele bijlagen) aan derden is uitdrukkelijk niet toegestaan.
> > > Indien u niet de bedoelde geadresseerde bent, wordt u vriendelijk
> > > verzocht degene die de e-mail verzond hiervan direct op de hoogte te
> > > brengen en de e-mail (en eventuele bijlagen) te vernietigen.
> > >
> > > Informatie vennootschap<http://www.ns.nl/emaildisclaimer>
> > >
> >
> >
> > --
> > -K
> >
> > ________________________________
> >
> > Deze e-mail, inclusief eventuele bijlagen, is uitsluitend bestemd voor (gebruik door) de geadresseerde. De e-mail kan persoonlijke of vertrouwelijke informatie bevatten. Openbaarmaking, vermenigvuldiging, verspreiding en/of verstrekking van (de inhoud van) deze e-mail (en eventuele bijlagen) aan derden is uitdrukkelijk niet toegestaan. Indien u niet de bedoelde geadresseerde bent, wordt u vriendelijk verzocht degene die de e-mail verzond hiervan direct op de hoogte te brengen en de e-mail (en eventuele bijlagen) te vernietigen.
> >
> > Informatie vennootschap<http://www.ns.nl/emaildisclaimer>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Non-blocking Sender with no Receivers
Posted by Robbie Gemmell <ro...@gmail.com>.
On Wed, 14 Oct 2020 at 15:55, Petrenko, Vadim <va...@ns.nl> wrote:
>
> Hi Ken,
>
> Yes, we're pre-settling messages already:
> connectionfactory.myFactoryLookupSender = amqp://localhost:5674?jms.presettlePolicy.presettleProducers=true
> (dockerized Qpid with an exposed port).
>
> Making the sending client first check for available credits is a possibility, but not the ideal option we'd like to use. These clients are not in our control, it's other teams that develop their applications using our Qpid network. So we can't really enforce them. Only advise.
>
The JMS client can't do that anyway.
> I also noticed some strange behavior which I don't understand. Maybe you guys can explain.
> I've taken the Sender and Receiver from qpid-jms-examples, connected them to our router network and observed how this works.
> 1. When I start the Sender, it gets blocked because there is no receiver (in the logs I see "Holding Message send until credit is available"). This is as expected.
> 2. I start the Receiver, messages begin to flow, all good.
> 3. I kill the Receiver process, but the Sender is still sending messages. To nowhere. It doesn't timeout either.
It's sending them to the router, which it still has outstanding credit
to do. It knows nothing of the receiver.
It won't time out a send unless you configure it to, and note it
certainly won't time out in this case if it has credit to send and is
sending pre-settled messages, since there is also no response to wait
for.
>
> I then proceeded debugging AmqpFixedProducer, which works with credits, and noticed that on step 1 number of credits it gets via getEndpoint().getCredit() is 0. So it holds the message.
> On the next step Receiver gives it 250 credits. After that, even though the Receiver process is killed, the number of credits is still 250 and doesn't get decremented. The Sender continues to send. I expect that credits would be exhausted at some point.
> Is this as it should be?
Are you sure it 'doesn't get decremented' (which happens locally as
sends occur), or are you perhaps just not observing the credit is
being replenished by the router? You can see the protocol trace on
router and/or the client by running them with env variable
PN_TRACE_FRM=1 set (or also for client, see:
http://qpid.apache.org/releases/qpid-jms-0.54.0/docs/index.html#logging)
As you are using message-routing, the router itself is what grants the
sender credit to send. The receiver also grants credit but that is to
the router, and is quite independent of the credit the sender will see
from the router (credit would be aligned end to end for a link-routing
case, e.g through router to/from a broker).
https://issues.apache.org/jira/browse/DISPATCH-1090 is the last I can
think of the behaviour in this area being tweaked, which was to try
and drain the outstanding credit in similar cases, however in checking
the vicinity of those changes now, I see edge routers being called out
and handled differently, indicating it will replenish the credit for
such messages. Your other thread mentioned edge routers so I think
perhaps this is in play here:
https://github.com/apache/qpid-dispatch/blob/1.14.0/src/router_core/transfer.c#L467-L476
>
> Thanks.
>
>
> -----Original Message-----
> From: Ken Giusti <kg...@redhat.com>
> Sent: woensdag 14 oktober 2020 15:18
> To: users <us...@qpid.apache.org>
> Subject: Re: Non-blocking Sender with no Receivers
>
> On Wed, Oct 14, 2020 at 8:25 AM Petrenko, Vadim <va...@ns.nl>
> wrote:
>
> > Good day,
> >
> > We’re trying to implement our Qpid dispatch router network in such a
> > way that a sender can connect to it at any moment and start sending
> > messages to a topic.
> > There might be no receivers at this moment, so the messages would be
> > just lost. This is ok.
> > Then an any moment a receiver(s) can connect and start receiving
> > messages currently being sent, then drop off. The sender will just
> > continue sending messages, probably into the void.
> >
> > It’s about sending data from sensors. Sensor data is short living and
> > there is no need to buffer it or persist. If no one consumes this data
> > in realtime, it can be lost.
> > Senders and receivers in our situation are quite volatile, they appear
> > and drop off.
> >
> > As per docs of Qpid:
> > ===
> > Dispatch Router uses a credit-based flow control mechanism to ensure
> > that producers can only send messages to a router if at least one
> > consumer is available to receive them. Because Dispatch Router does
> > not store messages, this credit-based flow control prevents producers
> > from sending messages when there are no consumers present.
> > ===
> >
> > I was wondering, maybe there is still a workaround or a trick to
> > implement the router network the way I described in the beginning?
> > If possible without extra brokers and only with Qpid dispatch routers?
> >
> > Thanks.
> >
> >
> Can your sending client first check whether or not there is credit on the link and if there is no credit simply drop the message rather than sending it?
> Getting the value of available credit depends on the client API you are using. For example in Proton-C your sender would call
> pn_link_credit(sending_link_ptr) to get the credit for the sending link.
>
> Also keep in mind that since the application is OK with messages being dropped you'll want the sender to settle the message after it is sent (a.k.a. "pre-settled).
>
>
>
> >
> > ________________________________
> >
> > Deze e-mail, inclusief eventuele bijlagen, is uitsluitend bestemd voor
> > (gebruik door) de geadresseerde. De e-mail kan persoonlijke of
> > vertrouwelijke informatie bevatten. Openbaarmaking, vermenigvuldiging,
> > verspreiding en/of verstrekking van (de inhoud van) deze e-mail (en
> > eventuele bijlagen) aan derden is uitdrukkelijk niet toegestaan.
> > Indien u niet de bedoelde geadresseerde bent, wordt u vriendelijk
> > verzocht degene die de e-mail verzond hiervan direct op de hoogte te
> > brengen en de e-mail (en eventuele bijlagen) te vernietigen.
> >
> > Informatie vennootschap<http://www.ns.nl/emaildisclaimer>
> >
>
>
> --
> -K
>
> ________________________________
>
> Deze e-mail, inclusief eventuele bijlagen, is uitsluitend bestemd voor (gebruik door) de geadresseerde. De e-mail kan persoonlijke of vertrouwelijke informatie bevatten. Openbaarmaking, vermenigvuldiging, verspreiding en/of verstrekking van (de inhoud van) deze e-mail (en eventuele bijlagen) aan derden is uitdrukkelijk niet toegestaan. Indien u niet de bedoelde geadresseerde bent, wordt u vriendelijk verzocht degene die de e-mail verzond hiervan direct op de hoogte te brengen en de e-mail (en eventuele bijlagen) te vernietigen.
>
> Informatie vennootschap<http://www.ns.nl/emaildisclaimer>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Non-blocking Sender with no Receivers
Posted by Chuck Rolke <cr...@redhat.com>.
When the receiver starts then the sender gets (by default) 250 credits.
As long as the receiver is there the credits keep getting replenished.
When the receiver exits then the router stops giving the sender new
credits. The sender still can send the 250 messages to use up the
current credits. If you wait long enough then the sender will eventually
hang.
The messages that are sent to the router when there is no receiver
receive 'released' disposition indicating that no receiver on the
router got the message.
-Chuck
----- Original Message -----
> From: "Vadim Petrenko" <va...@ns.nl>
> To: users@qpid.apache.org
> Sent: Wednesday, October 14, 2020 10:54:53 AM
> Subject: RE: Non-blocking Sender with no Receivers
>
> Hi Ken,
>
> Yes, we're pre-settling messages already:
> connectionfactory.myFactoryLookupSender =
> amqp://localhost:5674?jms.presettlePolicy.presettleProducers=true
> (dockerized Qpid with an exposed port).
>
> Making the sending client first check for available credits is a possibility,
> but not the ideal option we'd like to use. These clients are not in our
> control, it's other teams that develop their applications using our Qpid
> network. So we can't really enforce them. Only advise.
>
> I also noticed some strange behavior which I don't understand. Maybe you guys
> can explain.
> I've taken the Sender and Receiver from qpid-jms-examples, connected them to
> our router network and observed how this works.
> 1. When I start the Sender, it gets blocked because there is no receiver (in
> the logs I see "Holding Message send until credit is available"). This is as
> expected.
> 2. I start the Receiver, messages begin to flow, all good.
> 3. I kill the Receiver process, but the Sender is still sending messages. To
> nowhere. It doesn't timeout either.
>
> I then proceeded debugging AmqpFixedProducer, which works with credits, and
> noticed that on step 1 number of credits it gets via
> getEndpoint().getCredit() is 0. So it holds the message.
> On the next step Receiver gives it 250 credits. After that, even though the
> Receiver process is killed, the number of credits is still 250 and doesn't
> get decremented. The Sender continues to send. I expect that credits would
> be exhausted at some point.
> Is this as it should be?
>
> Thanks.
>
>
> -----Original Message-----
> From: Ken Giusti <kg...@redhat.com>
> Sent: woensdag 14 oktober 2020 15:18
> To: users <us...@qpid.apache.org>
> Subject: Re: Non-blocking Sender with no Receivers
>
> On Wed, Oct 14, 2020 at 8:25 AM Petrenko, Vadim <va...@ns.nl>
> wrote:
>
> > Good day,
> >
> > We’re trying to implement our Qpid dispatch router network in such a
> > way that a sender can connect to it at any moment and start sending
> > messages to a topic.
> > There might be no receivers at this moment, so the messages would be
> > just lost. This is ok.
> > Then an any moment a receiver(s) can connect and start receiving
> > messages currently being sent, then drop off. The sender will just
> > continue sending messages, probably into the void.
> >
> > It’s about sending data from sensors. Sensor data is short living and
> > there is no need to buffer it or persist. If no one consumes this data
> > in realtime, it can be lost.
> > Senders and receivers in our situation are quite volatile, they appear
> > and drop off.
> >
> > As per docs of Qpid:
> > ===
> > Dispatch Router uses a credit-based flow control mechanism to ensure
> > that producers can only send messages to a router if at least one
> > consumer is available to receive them. Because Dispatch Router does
> > not store messages, this credit-based flow control prevents producers
> > from sending messages when there are no consumers present.
> > ===
> >
> > I was wondering, maybe there is still a workaround or a trick to
> > implement the router network the way I described in the beginning?
> > If possible without extra brokers and only with Qpid dispatch routers?
> >
> > Thanks.
> >
> >
> Can your sending client first check whether or not there is credit on the
> link and if there is no credit simply drop the message rather than sending
> it?
> Getting the value of available credit depends on the client API you are
> using. For example in Proton-C your sender would call
> pn_link_credit(sending_link_ptr) to get the credit for the sending link.
>
> Also keep in mind that since the application is OK with messages being
> dropped you'll want the sender to settle the message after it is sent
> (a.k.a. "pre-settled).
>
>
>
> >
> > ________________________________
> >
> > Deze e-mail, inclusief eventuele bijlagen, is uitsluitend bestemd voor
> > (gebruik door) de geadresseerde. De e-mail kan persoonlijke of
> > vertrouwelijke informatie bevatten. Openbaarmaking, vermenigvuldiging,
> > verspreiding en/of verstrekking van (de inhoud van) deze e-mail (en
> > eventuele bijlagen) aan derden is uitdrukkelijk niet toegestaan.
> > Indien u niet de bedoelde geadresseerde bent, wordt u vriendelijk
> > verzocht degene die de e-mail verzond hiervan direct op de hoogte te
> > brengen en de e-mail (en eventuele bijlagen) te vernietigen.
> >
> > Informatie vennootschap<http://www.ns.nl/emaildisclaimer>
> >
>
>
> --
> -K
>
> ________________________________
>
> Deze e-mail, inclusief eventuele bijlagen, is uitsluitend bestemd voor
> (gebruik door) de geadresseerde. De e-mail kan persoonlijke of
> vertrouwelijke informatie bevatten. Openbaarmaking, vermenigvuldiging,
> verspreiding en/of verstrekking van (de inhoud van) deze e-mail (en
> eventuele bijlagen) aan derden is uitdrukkelijk niet toegestaan. Indien u
> niet de bedoelde geadresseerde bent, wordt u vriendelijk verzocht degene die
> de e-mail verzond hiervan direct op de hoogte te brengen en de e-mail (en
> eventuele bijlagen) te vernietigen.
>
> Informatie vennootschap<http://www.ns.nl/emaildisclaimer>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
RE: Non-blocking Sender with no Receivers
Posted by "Petrenko, Vadim" <va...@ns.nl>.
Hi Ken,
Yes, we're pre-settling messages already:
connectionfactory.myFactoryLookupSender = amqp://localhost:5674?jms.presettlePolicy.presettleProducers=true
(dockerized Qpid with an exposed port).
Making the sending client first check for available credits is a possibility, but not the ideal option we'd like to use. These clients are not in our control, it's other teams that develop their applications using our Qpid network. So we can't really enforce them. Only advise.
I also noticed some strange behavior which I don't understand. Maybe you guys can explain.
I've taken the Sender and Receiver from qpid-jms-examples, connected them to our router network and observed how this works.
1. When I start the Sender, it gets blocked because there is no receiver (in the logs I see "Holding Message send until credit is available"). This is as expected.
2. I start the Receiver, messages begin to flow, all good.
3. I kill the Receiver process, but the Sender is still sending messages. To nowhere. It doesn't timeout either.
I then proceeded debugging AmqpFixedProducer, which works with credits, and noticed that on step 1 number of credits it gets via getEndpoint().getCredit() is 0. So it holds the message.
On the next step Receiver gives it 250 credits. After that, even though the Receiver process is killed, the number of credits is still 250 and doesn't get decremented. The Sender continues to send. I expect that credits would be exhausted at some point.
Is this as it should be?
Thanks.
-----Original Message-----
From: Ken Giusti <kg...@redhat.com>
Sent: woensdag 14 oktober 2020 15:18
To: users <us...@qpid.apache.org>
Subject: Re: Non-blocking Sender with no Receivers
On Wed, Oct 14, 2020 at 8:25 AM Petrenko, Vadim <va...@ns.nl>
wrote:
> Good day,
>
> We’re trying to implement our Qpid dispatch router network in such a
> way that a sender can connect to it at any moment and start sending
> messages to a topic.
> There might be no receivers at this moment, so the messages would be
> just lost. This is ok.
> Then an any moment a receiver(s) can connect and start receiving
> messages currently being sent, then drop off. The sender will just
> continue sending messages, probably into the void.
>
> It’s about sending data from sensors. Sensor data is short living and
> there is no need to buffer it or persist. If no one consumes this data
> in realtime, it can be lost.
> Senders and receivers in our situation are quite volatile, they appear
> and drop off.
>
> As per docs of Qpid:
> ===
> Dispatch Router uses a credit-based flow control mechanism to ensure
> that producers can only send messages to a router if at least one
> consumer is available to receive them. Because Dispatch Router does
> not store messages, this credit-based flow control prevents producers
> from sending messages when there are no consumers present.
> ===
>
> I was wondering, maybe there is still a workaround or a trick to
> implement the router network the way I described in the beginning?
> If possible without extra brokers and only with Qpid dispatch routers?
>
> Thanks.
>
>
Can your sending client first check whether or not there is credit on the link and if there is no credit simply drop the message rather than sending it?
Getting the value of available credit depends on the client API you are using. For example in Proton-C your sender would call
pn_link_credit(sending_link_ptr) to get the credit for the sending link.
Also keep in mind that since the application is OK with messages being dropped you'll want the sender to settle the message after it is sent (a.k.a. "pre-settled).
>
> ________________________________
>
> Deze e-mail, inclusief eventuele bijlagen, is uitsluitend bestemd voor
> (gebruik door) de geadresseerde. De e-mail kan persoonlijke of
> vertrouwelijke informatie bevatten. Openbaarmaking, vermenigvuldiging,
> verspreiding en/of verstrekking van (de inhoud van) deze e-mail (en
> eventuele bijlagen) aan derden is uitdrukkelijk niet toegestaan.
> Indien u niet de bedoelde geadresseerde bent, wordt u vriendelijk
> verzocht degene die de e-mail verzond hiervan direct op de hoogte te
> brengen en de e-mail (en eventuele bijlagen) te vernietigen.
>
> Informatie vennootschap<http://www.ns.nl/emaildisclaimer>
>
--
-K
________________________________
Deze e-mail, inclusief eventuele bijlagen, is uitsluitend bestemd voor (gebruik door) de geadresseerde. De e-mail kan persoonlijke of vertrouwelijke informatie bevatten. Openbaarmaking, vermenigvuldiging, verspreiding en/of verstrekking van (de inhoud van) deze e-mail (en eventuele bijlagen) aan derden is uitdrukkelijk niet toegestaan. Indien u niet de bedoelde geadresseerde bent, wordt u vriendelijk verzocht degene die de e-mail verzond hiervan direct op de hoogte te brengen en de e-mail (en eventuele bijlagen) te vernietigen.
Informatie vennootschap<http://www.ns.nl/emaildisclaimer>
Re: Non-blocking Sender with no Receivers
Posted by Robbie Gemmell <ro...@gmail.com>.
On Wed, 14 Oct 2020 at 14:18, Ken Giusti <kg...@redhat.com> wrote:
>
> On Wed, Oct 14, 2020 at 8:25 AM Petrenko, Vadim <va...@ns.nl>
> wrote:
>
> > Good day,
> >
> > We’re trying to implement our Qpid dispatch router network in such a way
> > that a sender can connect to it at any moment and start sending messages to
> > a topic.
> > There might be no receivers at this moment, so the messages would be just
> > lost. This is ok.
> > Then an any moment a receiver(s) can connect and start receiving messages
> > currently being sent, then drop off. The sender will just continue sending
> > messages, probably into the void.
> >
> > It’s about sending data from sensors. Sensor data is short living and
> > there is no need to buffer it or persist. If no one consumes this data in
> > realtime, it can be lost.
> > Senders and receivers in our situation are quite volatile, they appear and
> > drop off.
> >
> > As per docs of Qpid:
> > ===
> > Dispatch Router uses a credit-based flow control mechanism to ensure that
> > producers can only send messages to a router if at least one consumer is
> > available to receive them. Because Dispatch Router does not store messages,
> > this credit-based flow control prevents producers from sending messages
> > when there are no consumers present.
> > ===
> >
> > I was wondering, maybe there is still a workaround or a trick to implement
> > the router network the way I described in the beginning?
> > If possible without extra brokers and only with Qpid dispatch routers?
> >
> > Thanks.
> >
> >
> Can your sending client first check whether or not there is credit on the
> link and if there is no credit simply drop the message rather than sending
> it?
> Getting the value of available credit depends on the client API you are
> using. For example in Proton-C your sender would call
> pn_link_credit(sending_link_ptr) to get the credit for the sending link.
>
> Also keep in mind that since the application is OK with messages being
> dropped you'll want the sender to settle the message after it is sent
> (a.k.a. "pre-settled).
>
>
Though it may not be the only type of client in use, Qpid JMS was
mentioned in another thread. It isn't possible to check credit there,
the client blocks for credit (you can set a sendTimeout and the client
will fail a send if it doesn't get credit).
Alternatively, might I suggest use of anonymous producer links
(producers to the null address)? I believe they are always granted
credit to send, since it's not possible for the router to tell in
advance where their messages are going to be sent, it can only
identify that once they arrive. If so, you can create JMS senders that
use the anonymous sender, either by omitting the destination when
creating a MessageProducer instance, or using JMSProducer instances
which always use anonymous senders.
You can also configure the client via the URI to send messages
pre-settled if desired. With talk of pre-settling however, also note
the router aggressively manages incoming pre-settled message backlog
while messag-routing (as opposed to link routing) in order to avoid
congestion.
Robbie
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Non-blocking Sender with no Receivers
Posted by Ken Giusti <kg...@redhat.com>.
On Wed, Oct 14, 2020 at 8:25 AM Petrenko, Vadim <va...@ns.nl>
wrote:
> Good day,
>
> We’re trying to implement our Qpid dispatch router network in such a way
> that a sender can connect to it at any moment and start sending messages to
> a topic.
> There might be no receivers at this moment, so the messages would be just
> lost. This is ok.
> Then an any moment a receiver(s) can connect and start receiving messages
> currently being sent, then drop off. The sender will just continue sending
> messages, probably into the void.
>
> It’s about sending data from sensors. Sensor data is short living and
> there is no need to buffer it or persist. If no one consumes this data in
> realtime, it can be lost.
> Senders and receivers in our situation are quite volatile, they appear and
> drop off.
>
> As per docs of Qpid:
> ===
> Dispatch Router uses a credit-based flow control mechanism to ensure that
> producers can only send messages to a router if at least one consumer is
> available to receive them. Because Dispatch Router does not store messages,
> this credit-based flow control prevents producers from sending messages
> when there are no consumers present.
> ===
>
> I was wondering, maybe there is still a workaround or a trick to implement
> the router network the way I described in the beginning?
> If possible without extra brokers and only with Qpid dispatch routers?
>
> Thanks.
>
>
Can your sending client first check whether or not there is credit on the
link and if there is no credit simply drop the message rather than sending
it?
Getting the value of available credit depends on the client API you are
using. For example in Proton-C your sender would call
pn_link_credit(sending_link_ptr) to get the credit for the sending link.
Also keep in mind that since the application is OK with messages being
dropped you'll want the sender to settle the message after it is sent
(a.k.a. "pre-settled).
>
> ________________________________
>
> Deze e-mail, inclusief eventuele bijlagen, is uitsluitend bestemd voor
> (gebruik door) de geadresseerde. De e-mail kan persoonlijke of
> vertrouwelijke informatie bevatten. Openbaarmaking, vermenigvuldiging,
> verspreiding en/of verstrekking van (de inhoud van) deze e-mail (en
> eventuele bijlagen) aan derden is uitdrukkelijk niet toegestaan. Indien u
> niet de bedoelde geadresseerde bent, wordt u vriendelijk verzocht degene
> die de e-mail verzond hiervan direct op de hoogte te brengen en de e-mail
> (en eventuele bijlagen) te vernietigen.
>
> Informatie vennootschap<http://www.ns.nl/emaildisclaimer>
>
--
-K