You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by Nitin Shah <ns...@btisystems.com> on 2012/08/15 23:14:46 UTC
Can someone help to understand
Hi,
I wanted to run this by the developers to understand why this happens. We are running the release 0.16 C++ broker and have written some applications to use the infrastructure. I have developed a simple applications that sends messages to a receiving application on from the amq.topic exchange using the auto created queues. The connection is setup to reconnect in the broker goes down by setting connection option as reconnect:true.
I am using the receive loop for the messages s getNextReceiver calls. In order for that to work the setCapacity for receivers is set at 10 for that matter could be higher. The applications are linked with the qpidclient library -background info. The senderstarts up and sends 20 messages which the receiver receives. After that the sender does not send any more messages. The test that I was trying was , what happens if the broker is killed and then restarted. What I see if I do that is that all the messages are resent by the broker to the receiving loop.
I haven't done anything to the link options so they must be the default ones. Each message received is acked with the session acknowledge calls, so I assume the buffers must be released and the messages not kept in the qpid subsystem. However that does not seem to be the case and each time I restart the qpidd deaemon, the messages are resent to the receiving app. I have played around with setting capacity in the sender and the receiver. If I set the capacity of the sender to 1 then only one message is retransmitted when the broker comes up. If I set the capacity on the sender and the receiver to 100 i.e. higher than 20 then all messages get resent on every restart of the broker.
Can someone tell me what I am doing wrong. It seems that the messages are persistant.
Thanks
Nitin
[cid:image001.png@01CD7B06.4AAD3D20]
Re: Can someone help to understand
Posted by Gordon Sim <gs...@redhat.com>.
On 08/21/2012 06:32 PM, Nitin Shah wrote:
> the producer and the consumer are different thread on the same process, could that cause a problem?
No, that should not cause any problem. If they share the same session
then a sync() will affect both the sent messages and the
acknowledgements. If they each have their own session there should be no
real difference as far as in doubt messages go between having them in
the same process or in a separate process.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
Re: Can someone help to understand
Posted by Gordon Sim <gs...@redhat.com>.
On 08/21/2012 06:32 PM, Nitin Shah wrote:
> After a Session::sync() that should also be 0 (assuming no other thread
> is using the session).
> Nitin:: I should check that count, i suspect it may not be zero.
If not, that would be indicative of a bug (assuming nothing else was
sending on that session in the meantime). If you confirm it is not zero,
then please open a Jira and attach your test program. You can assign it
to me and I'll take a look.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
RE: Can someone help to understand
Posted by Nitin Shah <ns...@btisystems.com>.
My answers enclosed
________________________________________
From: Gordon Sim [gsim@redhat.com]
Sent: Tuesday, August 21, 2012 11:52 AM
To: dev@qpid.apache.org
Subject: Re: Can someone help to understand
On 08/21/2012 04:21 PM, Nitin Shah wrote:
> I did not specify a parameter to the sync call, which means it is
> default i.e. true. That means it blocks when called. If that is the
> case, why does it repeat the messages since I assume they have been
> acked since we forced a sync and waited a while before killing the
> broker.
Did you kill the broker during a sync() or in between syncs? How manay
messages did it replay? Nitin:: the broker was killed a long time after the sync call was done, minutes
> Also in our scenario, the broker ends up starting on another
> VM in our test setup.
>
> On the receiver side, we acknowledge every message received. It is
> the unsettled count that seems to be the issue.
Right, unsettled == in doubt.
> If that count on the
> sender is 0, no messages are repeated. I can only get that if the
> link options on the sender are set to "unreliable".
After a Session::sync() that should also be 0 (assuming no other thread
is using the session).
Nitin:: I should check that count, i suspect it may not be zero.
> No other setup
> produces the result we want. Explicitly sinking the session does not
> seem to stop the repeated messages on a failover of the broker unless
> we have the link option as stated above.
As mentioned you can choose between message loss or message duplication
at present. If you choose duplication you can't eliminate it. There is
no exactly once guarantee supported yet. Nitin:: I understand that from the documentation, so i have not assumed anything else.
Nitin:: one more thing I should add, the producer and the consumer are different thread on the same process, could that cause a problem?
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
Re: Can someone help to understand
Posted by Gordon Sim <gs...@redhat.com>.
On 08/21/2012 04:21 PM, Nitin Shah wrote:
> I did not specify a parameter to the sync call, which means it is
> default i.e. true. That means it blocks when called. If that is the
> case, why does it repeat the messages since I assume they have been
> acked since we forced a sync and waited a while before killing the
> broker.
Did you kill the broker during a sync() or in between syncs? How manay
messages did it replay?
> Also in our scenario, the broker ends up starting on another
> VM in our test setup.
>
> On the receiver side, we acknowledge every message received. It is
> the unsettled count that seems to be the issue.
Right, unsettled == in doubt.
> If that count on the
> sender is 0, no messages are repeated. I can only get that if the
> link options on the sender are set to "unreliable".
After a Session::sync() that should also be 0 (assuming no other thread
is using the session).
> No other setup
> produces the result we want. Explicitly sinking the session does not
> seem to stop the repeated messages on a failover of the broker unless
> we have the link option as stated above.
As mentioned you can choose between message loss or message duplication
at present. If you choose duplication you can't eliminate it. There is
no exactly once guarantee supported yet.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
RE: Can someone help to understand
Posted by Nitin Shah <ns...@btisystems.com>.
Hi Gordon,
I did not specify a parameter to the sync call, which means it is default i.e. true. That means it blocks when called. If that is the case, why does it repeat the messages since I assume they have been acked since we forced a sync and waited a while before killing the broker. Also in our scenario, the broker ends up starting on another VM in our test setup.
On the receiver side, we acknowledge every message received. It is the unsettled count that seems to be the issue. If that count on the sender is 0, no messages are repeated. I can only get that if the link options on the sender are set to "unreliable". No other setup produces the result we want. Explicitly sinking the session does not seem to stop the repeated messages on a failover of the broker unless we have the link option as stated above.
Nitin
-----Original Message-----
From: Gordon Sim [mailto:gsim@redhat.com]
Sent: Tuesday, August 21, 2012 11:00 AM
To: dev@qpid.apache.org
Subject: Re: Can someone help to understand
On 08/21/2012 03:27 PM, Nitin Shah wrote:
> I tried the suggestion you had. I found that the scenario still exists
> and messages are repeated when the broker killed and restarted. I
> tried 2 things. I have a loop that sends 20 messages and then after I
> have the session.sync() call.
Are you specifying a parameter to that call? (If so is it true or false)? A blocking sync() should prevent you sending any more messages until the first 20 are confirmed which should reduce the number of resends by the sender to at most 20.
> In this case the messages
> that are repeated are the same as the setCapacity() call I had made
> when the sender is created. If I perform the sync call every after
> every send(), then the number of messages are reduced to 2 on a
> restart of the broker.
That's probably one resent by the sender and one resent by the broker.
> Of course if I set the link option to be unreliable when I create a
> sender, the messages do not get repeated.
No, but then they may get lost.
> I would like figure out why
> this happens and maybe it is an education on my part.
At present the supported guarantees are at-least-once or unreliable. The former may result in duplicates, the latter in message loss. In both cases the scope of the problem is the set of 'in doubt' messages for both sender and receiver.
The number in doubt messages for the sender is limited by the capacity.
You can also explicitly sync() the session.
For the receiver, the number of in doubt messages depends also on the acknowledgement frequency.
Does this help?
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org For additional commands, e-mail: dev-help@qpid.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
Re: Can someone help to understand
Posted by Gordon Sim <gs...@redhat.com>.
On 08/21/2012 03:27 PM, Nitin Shah wrote:
> I tried the suggestion you had. I found that the scenario still
> exists and messages are repeated when the broker killed and
> restarted. I tried 2 things. I have a loop that sends 20 messages and
> then after I have the session.sync() call.
Are you specifying a parameter to that call? (If so is it true or
false)? A blocking sync() should prevent you sending any more messages
until the first 20 are confirmed which should reduce the number of
resends by the sender to at most 20.
> In this case the messages
> that are repeated are the same as the setCapacity() call I had made
> when the sender is created. If I perform the sync call every after
> every send(), then the number of messages are reduced to 2 on a
> restart of the broker.
That's probably one resent by the sender and one resent by the broker.
> Of course if I set the link option to be unreliable when I create a
> sender, the messages do not get repeated.
No, but then they may get lost.
> I would like figure out why
> this happens and maybe it is an education on my part.
At present the supported guarantees are at-least-once or unreliable. The
former may result in duplicates, the latter in message loss. In both
cases the scope of the problem is the set of 'in doubt' messages for
both sender and receiver.
The number in doubt messages for the sender is limited by the capacity.
You can also explicitly sync() the session.
For the receiver, the number of in doubt messages depends also on the
acknowledgement frequency.
Does this help?
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
RE: Can someone help to understand
Posted by Nitin Shah <ns...@btisystems.com>.
Hi Gordon,
I tried the suggestion you had. I found that the scenario still exists and messages are repeated when the broker killed and restarted. I tried 2 things. I have a loop that sends 20 messages and then after I have the session.sync() call. In this case the messages that are repeated are the same as the setCapacity() call I had made when the sender is created. If I perform the sync call every after every send(), then the number of messages are reduced to 2 on a restart of the broker.
Of course if I set the link option to be unreliable when I create a sender, the messages do not get repeated. I would like figure out why this happens and maybe it is an education on my part.
Thanks
Nitin
-----Original Message-----
From: Gordon Sim [mailto:gsim@redhat.com]
Sent: Thursday, August 16, 2012 10:05 AM
To: dev@qpid.apache.org
Subject: Re: Can someone help to understand
On 08/15/2012 10:14 PM, Nitin Shah wrote:
> Hi,
>
> I wanted to run this by the developers to understand why this happens.
> We are running the release 0.16 C++ broker and have written some
> applications to use the infrastructure. I have developed a simple
> applications that sends messages to a receiving application on from
> the amq.topic exchange using the auto created queues. The connection
> is setup to reconnect in the broker goes down by setting connection
> option as reconnect:true.
>
> I am using the receive loop for the messages s getNextReceiver calls.
> In order for that to work the setCapacity for receivers is set at 10
> for that matter could be higher. The applications are linked with the
> qpidclient library -background info. The senderstarts up and sends 20
> messages which the receiver receives. After that the sender does not
> send any more messages. The test that I was trying was , what happens
> if the broker is killed and then restarted. What I see if I do that is
> that all the messages are resent by the broker to the receiving loop.
>
> I haven't done anything to the link options so they must be the
> default ones. Each message received is acked with the session
> acknowledge calls, so I assume the buffers must be released and the
> messages not kept in the qpid subsystem. However that does not seem to
> be the case and each time I restart the qpidd deaemon, the messages
> are resent to the receiving app. I have played around with setting
> capacity in the sender and the receiver. If I set the capacity of the
> sender to 1 then only one message is retransmitted when the broker
> comes up. If I set the capacity on the sender and the receiver to 100
> i.e. higher than 20 then all messages get resent on every restart of the broker.
It is in fact the sender that is resending the messages. The sender maintains a buffer of messages it has sent until the broker confirms them. On failover it will resend these 'in-doubt' messages. The capacity dictates the frequency at which the sender requests confirmation from the broker. You can also call Session::sync() on the senders session at any time to request the broker to indicate when it has received all the outstanding messages.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org For additional commands, e-mail: dev-help@qpid.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org
Re: Can someone help to understand
Posted by Gordon Sim <gs...@redhat.com>.
On 08/15/2012 10:14 PM, Nitin Shah wrote:
> Hi,
>
> I wanted to run this by the developers to understand why this happens.
> We are running the release 0.16 C++ broker and have written some
> applications to use the infrastructure. I have developed a simple
> applications that sends messages to a receiving application on from the
> amq.topic exchange using the auto created queues. The connection is
> setup to reconnect in the broker goes down by setting connection option
> as reconnect:true.
>
> I am using the receive loop for the messages s getNextReceiver calls. In
> order for that to work the setCapacity for receivers is set at 10 for
> that matter could be higher. The applications are linked with the
> qpidclient library -background info. The senderstarts up and sends 20
> messages which the receiver receives. After that the sender does not
> send any more messages. The test that I was trying was , what happens if
> the broker is killed and then restarted. What I see if I do that is that
> all the messages are resent by the broker to the receiving loop.
>
> I haven’t done anything to the link options so they must be the default
> ones. Each message received is acked with the session acknowledge calls,
> so I assume the buffers must be released and the messages not kept in
> the qpid subsystem. However that does not seem to be the case and each
> time I restart the qpidd deaemon, the messages are resent to the
> receiving app. I have played around with setting capacity in the sender
> and the receiver. If I set the capacity of the sender to 1 then only one
> message is retransmitted when the broker comes up. If I set the
> capacity on the sender and the receiver to 100 i.e. higher than 20 then
> all messages get resent on every restart of the broker.
It is in fact the sender that is resending the messages. The sender
maintains a buffer of messages it has sent until the broker confirms
them. On failover it will resend these 'in-doubt' messages. The capacity
dictates the frequency at which the sender requests confirmation from
the broker. You can also call Session::sync() on the senders session at
any time to request the broker to indicate when it has received all the
outstanding messages.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org