You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by Tim Fox <ti...@jboss.com> on 2006/09/21 12:43:29 UTC
Using AMQP pre-fetch windowing with JMS transactional sessions and
client acknowledge
Hi all-
I'm trying to understand how AMQP pre-fetch windowing will work with JMS
consumers that are using JMS client acknowledge or JMS transacted session.
It's my understanding that pre-fetch windowing works by sending another
batch of messages to the consumer only when the server has received a
certain number of acknowledgements, in such a way consumers can be
prevented from being overwhelmed with messages.
However, when JMS consumers have a session with an acknowledgement mode
of client acknowledge, then acks are only sent for the session when the
client explicitly calls acknowledge() on a message.
Similarly for JMS transacted sessions, acks are sent when commit() is
called on the session.
So, if the pre-fetch window size is set to x messages, but the client
doesn't want to call acknowledge() or commit until it has processed y
messages, where y > x, then the consumer will starve at x messages.
How do we reconcile the JMS semantic with the way AMQP is supposed to work?
Thanks in advance
Re: Using AMQP pre-fetch windowing with JMS transactional sessions and client acknowledge
Posted by Rob Davies <ra...@gmail.com>.
On 21 Sep 2006, at 13:49, Gordon Sim wrote:
> Gordon Sim wrote:
>> Tim Fox wrote:
>>> Perhaps the protocol needs to be extended?
>> It does seem that it could be useful to be able to move the
>> prefetch window without having to acknowledge the message.
>
> A further thought here: it would be possible to resize the prefetch
> winddow instead of moving it. i.e. if the prefetch is 100 and the
> application is in control of when acknowledgements will be sent,
> then after passing say the 80th message to the application, we
> could resize the window to 180. When the acknowledgement is sent,
> we can resize it back to 100.
>
> I'm not saying that is ideal, but it seems like a workaround.
>
> Alternatively if we were able to modify the channel.flow message
> then instead of just a simple stop/go flag it could allow the
> 'position' prefetch window to be stated (e.g. an unacknowledged
> messages received count). (Theres no real reason this has to be put
> into channel.flow apart from the fact that it does seem like flow
> control). That would obviously require a protocol modification though.
seems like you're suggesting a sliding window - like in TCP/IP -
where you could increase/decrease the prefetch up to the max prefetch
size, but ensure that you inform the server to send more messages
before the client size is empty.
>
> Thoughts?
Re: Using AMQP pre-fetch windowing with JMS transactional sessions
and client acknowledge
Posted by Gordon Sim <gs...@redhat.com>.
Tim Fox wrote:
>
>
> Gordon Sim wrote:
>> Gordon Sim wrote:
>>> Tim Fox wrote:
>>>> Perhaps the protocol needs to be extended?
>>>
>>> It does seem that it could be useful to be able to move the prefetch
>>> window without having to acknowledge the message.
>>
>> A further thought here: it would be possible to resize the prefetch
>> winddow instead of moving it. i.e. if the prefetch is 100 and the
>> application is in control of when acknowledgements will be sent, then
>> after passing say the 80th message to the application, we could resize
>> the window to 180. When the acknowledgement is sent, we can resize it
>> back to 100.
>>
>>
>
> I'm not sure how this would prevent the consumer from being overwhelmed.
>
> If the prefetch window is resized to larger values as long as messages
> aren't acknowledged, then, assuming messages can be sent to the consumer
> faster then they can be processed by the consumer, they could
> catastrophically build up on the client.
You would only increase the size to compensate for messages processed by
the application, but not yet acknowledged. So if the logical prefetch is
x (i.e. the maximum unprocessed messages allowed), and the application
has processed y messages but not acknowledged them then the actual
prefetch as far as the broker is concerned would be x + y, though the
number of unprocessed messages would still never go above x.
Re: Using AMQP pre-fetch windowing with JMS transactional sessions
and client acknowledge
Posted by Tim Fox <ti...@jboss.com>.
Gordon Sim wrote:
> Gordon Sim wrote:
>> Tim Fox wrote:
>>> Perhaps the protocol needs to be extended?
>>
>> It does seem that it could be useful to be able to move the prefetch
>> window without having to acknowledge the message.
>
> A further thought here: it would be possible to resize the prefetch
> winddow instead of moving it. i.e. if the prefetch is 100 and the
> application is in control of when acknowledgements will be sent, then
> after passing say the 80th message to the application, we could resize
> the window to 180. When the acknowledgement is sent, we can resize it
> back to 100.
>
>
I'm not sure how this would prevent the consumer from being overwhelmed.
If the prefetch window is resized to larger values as long as messages
aren't acknowledged, then, assuming messages can be sent to the consumer
faster then they can be processed by the consumer, they could
catastrophically build up on the client.
Or perhaps I have misunderstood :)
Re: Using AMQP pre-fetch windowing with JMS transactional sessions
and client acknowledge
Posted by Gordon Sim <gs...@redhat.com>.
Gordon Sim wrote:
> A further thought here: it would be possible to resize the prefetch
> winddow instead of moving it. i.e. if the prefetch is 100 and the
> application is in control of when acknowledgements will be sent, then
> after passing say the 80th message to the application, we could resize
> the window to 180. When the acknowledgement is sent, we can resize it
> back to 100.
In the transactional case, I don't think we need to do this as we can
send acknowledgements periodically and they won't be processed until commit.
Re: Using AMQP pre-fetch windowing with JMS transactional sessions and
client acknowledge
Posted by ro...@jpmorgan.com.
Yes, the "delay" indication is what I have been thinking of.
It would allow us to regulate the rate without introducing extra
"chattiness" to the protocol - or at least that is what I think it should
allow.
RG
|---------+---------------------------->
| | "John O'Hara" |
| | <john.r.ohara@gma|
| | il.com> |
| | |
| | 21/09/2006 23:10 |
| | Please respond to|
| | qpid-dev |
|---------+---------------------------->
>------------------------------------------------------------------------------------------------------------------------------|
| |
| To: qpid-dev@incubator.apache.org |
| cc: |
| Subject: Re: Using AMQP pre-fetch windowing with JMS transactional sessions and client acknowledge |
>------------------------------------------------------------------------------------------------------------------------------|
I kind of like this - it's minimalist.
So what Gordon is saying is that a prefetch resize is like a block ack?
I also agree that our Flow Control is perhaps too simplistic and should
have
some kind of rate specifier or hold off timer sent by the server (i.e
server
says to client "no more messages for X milliseconds"). Or zero for stop
until told otherwise.
This would be cool because then the server can measure the rate of flow for
each client and then send a flow control message to tell the client to hold
off a bit - effectively altering the rate the client is producing to a more
acceptable rate. Perhaps the client should also have a way of telling the
server how much back pressure it is experiencing as a publisher -- ie. the
client can say "give me more bandwidth I'm about to die". Then the server
could make choices about who to penalise... or is this all too complex?
At some stage, soon, we should write a precise description of how the JMS
ack/tran modes map to the protocol. Its important that the clients and
servers interact consistently even across implementations, imho. Sounds
like something we could proposed to those at the AMQP WG offsite...
John
On 21/09/06, Gordon Sim < gsim@redhat.com> wrote:
>
> robert.j.greig@jpmorgan.com wrote:
> > My initial reaction to your suggestion is: does this actually change
> > anything?
> >
> > i.e. if the prefetch is 100 and you resize it to 180, what do you do
> when
> > you reach that limit? Resize again? If the prefetch is in place to
> prevent
> > the memory footprint getting too large, resizing it may not be
> desirable.
>
> You only resize it once messages have been processed. It seems to me
> that the memory footprint is controlled by preventing the number of
> unprocessed messages in the client going above a specific value . But as
> messages are processed, they in effect 'make room' for more. However as
> the server has not received acks it is unaware of this extra space and
> wont send more messages unless the prefetch is increased.
>
> Once the application acknowledges a message the prefetch can be set back
> to the original value.
>
>
This communication is for informational purposes only. It is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice. Any comments or statements made herein do not necessarily reflect those of JPMorgan Chase & Co., its subsidiaries and affiliates.
This transmission may contain information that is privileged, confidential, legally privileged, and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. Although this transmission and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by JPMorgan Chase & Co., its subsidiaries and affiliates, as applicable, for any loss or damage arising in any way from its use. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Thank you.
Re: Using AMQP pre-fetch windowing with JMS transactional sessions and client acknowledge
Posted by John O'Hara <jo...@gmail.com>.
I kind of like this - it's minimalist.
So what Gordon is saying is that a prefetch resize is like a block ack?
I also agree that our Flow Control is perhaps too simplistic and should have
some kind of rate specifier or hold off timer sent by the server (i.e server
says to client "no more messages for X milliseconds"). Or zero for stop
until told otherwise.
This would be cool because then the server can measure the rate of flow for
each client and then send a flow control message to tell the client to hold
off a bit - effectively altering the rate the client is producing to a more
acceptable rate. Perhaps the client should also have a way of telling the
server how much back pressure it is experiencing as a publisher -- ie. the
client can say "give me more bandwidth I'm about to die". Then the server
could make choices about who to penalise... or is this all too complex?
At some stage, soon, we should write a precise description of how the JMS
ack/tran modes map to the protocol. Its important that the clients and
servers interact consistently even across implementations, imho. Sounds
like something we could proposed to those at the AMQP WG offsite...
John
On 21/09/06, Gordon Sim < gsim@redhat.com> wrote:
>
> robert.j.greig@jpmorgan.com wrote:
> > My initial reaction to your suggestion is: does this actually change
> > anything?
> >
> > i.e. if the prefetch is 100 and you resize it to 180, what do you do
> when
> > you reach that limit? Resize again? If the prefetch is in place to
> prevent
> > the memory footprint getting too large, resizing it may not be
> desirable.
>
> You only resize it once messages have been processed. It seems to me
> that the memory footprint is controlled by preventing the number of
> unprocessed messages in the client going above a specific value . But as
> messages are processed, they in effect 'make room' for more. However as
> the server has not received acks it is unaware of this extra space and
> wont send more messages unless the prefetch is increased.
>
> Once the application acknowledges a message the prefetch can be set back
> to the original value.
>
>
Re: Using AMQP pre-fetch windowing with JMS transactional sessions
and client acknowledge
Posted by Gordon Sim <gs...@redhat.com>.
robert.j.greig@jpmorgan.com wrote:
> My initial reaction to your suggestion is: does this actually change
> anything?
>
> i.e. if the prefetch is 100 and you resize it to 180, what do you do when
> you reach that limit? Resize again? If the prefetch is in place to prevent
> the memory footprint getting too large, resizing it may not be desirable.
You only resize it once messages have been processed. It seems to me
that the memory footprint is controlled by preventing the number of
unprocessed messages in the client going above a specific value . But as
messages are processed, they in effect 'make room' for more. However as
the server has not received acks it is unaware of this extra space and
wont send more messages unless the prefetch is increased.
Once the application acknowledges a message the prefetch can be set back
to the original value.
Re: Using AMQP pre-fetch windowing with JMS transactional sessions and
client acknowledge
Posted by ro...@jpmorgan.com.
My initial reaction to your suggestion is: does this actually change
anything?
i.e. if the prefetch is 100 and you resize it to 180, what do you do when
you reach that limit? Resize again? If the prefetch is in place to prevent
the memory footprint getting too large, resizing it may not be desirable.
Or do you envisage only being able to resize it once? If so is it really
any different from just having the limit set at the larger value in the
first place?
I was thinking along the lines of allowing the client to indicate some kind
of "rate" information to the broker so that it can slow down (or if
possible speed up) the rate of delivery, thereby providing a more graceful
flow control. I believe that SwiftMQ does something along those lines.
RG
|---------+---------------------------->
| | Gordon Sim |
| | <gs...@redhat.com>|
| | |
| | 21/09/2006 13:49 |
| | Please respond to|
| | qpid-dev |
|---------+---------------------------->
>------------------------------------------------------------------------------------------------------------------------------|
| |
| To: qpid-dev@incubator.apache.org |
| cc: |
| Subject: Re: Using AMQP pre-fetch windowing with JMS transactional sessions and client acknowledge |
>------------------------------------------------------------------------------------------------------------------------------|
Gordon Sim wrote:
> Tim Fox wrote:
>> Perhaps the protocol needs to be extended?
>
> It does seem that it could be useful to be able to move the prefetch
> window without having to acknowledge the message.
A further thought here: it would be possible to resize the prefetch
winddow instead of moving it. i.e. if the prefetch is 100 and the
application is in control of when acknowledgements will be sent, then
after passing say the 80th message to the application, we could resize
the window to 180. When the acknowledgement is sent, we can resize it
back to 100.
I'm not saying that is ideal, but it seems like a workaround.
Alternatively if we were able to modify the channel.flow message then
instead of just a simple stop/go flag it could allow the 'position'
prefetch window to be stated (e.g. an unacknowledged messages received
count). (Theres no real reason this has to be put into channel.flow
apart from the fact that it does seem like flow control). That would
obviously require a protocol modification though.
Thoughts?
This communication is for informational purposes only. It is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice. Any comments or statements made herein do not necessarily reflect those of JPMorgan Chase & Co., its subsidiaries and affiliates.
This transmission may contain information that is privileged, confidential, legally privileged, and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. Although this transmission and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by JPMorgan Chase & Co., its subsidiaries and affiliates, as applicable, for any loss or damage arising in any way from its use. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Thank you.
Re: Using AMQP pre-fetch windowing with JMS transactional sessions
and client acknowledge
Posted by Gordon Sim <gs...@redhat.com>.
Gordon Sim wrote:
> Tim Fox wrote:
>> Perhaps the protocol needs to be extended?
>
> It does seem that it could be useful to be able to move the prefetch
> window without having to acknowledge the message.
A further thought here: it would be possible to resize the prefetch
winddow instead of moving it. i.e. if the prefetch is 100 and the
application is in control of when acknowledgements will be sent, then
after passing say the 80th message to the application, we could resize
the window to 180. When the acknowledgement is sent, we can resize it
back to 100.
I'm not saying that is ideal, but it seems like a workaround.
Alternatively if we were able to modify the channel.flow message then
instead of just a simple stop/go flag it could allow the 'position'
prefetch window to be stated (e.g. an unacknowledged messages received
count). (Theres no real reason this has to be put into channel.flow
apart from the fact that it does seem like flow control). That would
obviously require a protocol modification though.
Thoughts?
Re: Using AMQP pre-fetch windowing with JMS transactional sessions
and client acknowledge
Posted by Gordon Sim <gs...@redhat.com>.
Tim Fox wrote:
>
>
> Gordon Sim wrote:
>> Tim Fox wrote:
>>> So, if the pre-fetch window size is set to x messages, but the client
>>> doesn't want to call acknowledge() or commit until it has processed y
>>> messages, where y > x, then the consumer will starve at x messages.
>>>
>>> How do we reconcile the JMS semantic with the way AMQP is supposed to
>>> work?
>>
>> Good questions. I'd say that client acknowledgement or transacted
>> sessions should not use prefetch. I'd need to look at the AMQP spec to
>> see if it says anything on this but I would imagine a prefetch of 0
>> implies prefetch limits aren't required.
>
> I would have thought that constraining AMQP compatible applications to
> not be able to use prefetch in this situations may be a serious limitation.
Yes, I think you (and Robert) are right. Sorry for the rash response.
> Perhaps the protocol needs to be extended?
It does seem that it could be useful to be able to move the prefetch
window without having to acknowledge the message.
Re: Using AMQP pre-fetch windowing with JMS transactional sessions
and client acknowledge
Posted by Tim Fox <ti...@jboss.com>.
Gordon Sim wrote:
> Tim Fox wrote:
>> So, if the pre-fetch window size is set to x messages, but the client
>> doesn't want to call acknowledge() or commit until it has processed y
>> messages, where y > x, then the consumer will starve at x messages.
>>
>> How do we reconcile the JMS semantic with the way AMQP is supposed to
>> work?
>
> Good questions. I'd say that client acknowledgement or transacted
> sessions should not use prefetch. I'd need to look at the AMQP spec to
> see if it says anything on this but I would imagine a prefetch of 0
> implies prefetch limits aren't required.
I would have thought that constraining AMQP compatible applications to
not be able to use prefetch in this situations may be a serious limitation.
Perhaps the protocol needs to be extended?
>
Re: Using AMQP pre-fetch windowing with JMS transactional sessions and
client acknowledge
Posted by ro...@jpmorgan.com.
Surely you would still want to use prefetch even in the case of client ack
or txn?
e.g. consider a slow consumer, you wouldn't necessarily want to received a
huge number of messages if you could not keep up processing them.
Incidentally I think that the prefetch for flow control could be improved
in the protocol - it is perhaps not quite flexible enough.
RG
|---------+---------------------------->
| | Gordon Sim |
| | <gs...@redhat.com>|
| | |
| | 21/09/2006 12:00 |
| | Please respond to|
| | qpid-dev |
|---------+---------------------------->
>--------------------------------------------------------------------------------------------------------------------------------|
| |
| To: qpid-dev@incubator.apache.org |
| cc: |
| Subject: Re: Using AMQP pre-fetch windowing with JMS transactional sessions and client acknowledge |
>--------------------------------------------------------------------------------------------------------------------------------|
Tim Fox wrote:
> So, if the pre-fetch window size is set to x messages, but the client
> doesn't want to call acknowledge() or commit until it has processed y
> messages, where y > x, then the consumer will starve at x messages.
>
> How do we reconcile the JMS semantic with the way AMQP is supposed to
work?
Good questions. I'd say that client acknowledgement or transacted
sessions should not use prefetch. I'd need to look at the AMQP spec to
see if it says anything on this but I would imagine a prefetch of 0
implies prefetch limits aren't required.
This communication is for informational purposes only. It is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice. Any comments or statements made herein do not necessarily reflect those of JPMorgan Chase & Co., its subsidiaries and affiliates.
This transmission may contain information that is privileged, confidential, legally privileged, and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. Although this transmission and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by JPMorgan Chase & Co., its subsidiaries and affiliates, as applicable, for any loss or damage arising in any way from its use. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Thank you.
Re: Using AMQP pre-fetch windowing with JMS transactional sessions
and client acknowledge
Posted by Gordon Sim <gs...@redhat.com>.
Tim Fox wrote:
> So, if the pre-fetch window size is set to x messages, but the client
> doesn't want to call acknowledge() or commit until it has processed y
> messages, where y > x, then the consumer will starve at x messages.
>
> How do we reconcile the JMS semantic with the way AMQP is supposed to work?
Good questions. I'd say that client acknowledgement or transacted
sessions should not use prefetch. I'd need to look at the AMQP spec to
see if it says anything on this but I would imagine a prefetch of 0
implies prefetch limits aren't required.
Re: Using AMQP pre-fetch windowing with JMS transactional sessions
and client acknowledge
Posted by Tim Fox <ti...@jboss.com>.
robert.j.greig@jpmorgan.com wrote:
> Hi,
>
> Ah ok, I understand where you are coming from now.
>
> Yes you could set it to zero to turn it off.
>
> "although setting to a very large value could also break the app by
> overwhelming the client and leading to out of memory errors and the like."
>
> How do other implementation reconcile this kind of issue?
Well, the way we do it in JBoss Messaging is to send a special "more"
message from the consumer to the server when the consumer has depleted
it's local cache, then the server will send another batch if it has any.
>
> I definitely do not like the idea of sending an ack every time for the
> efficiency reason you mention.
>
> RG
>
>
> |---------+---------------------------->
> | | Tim Fox |
> | | <tim.fox@jboss.co|
> | | m> |
> | | |
> | | 21/09/2006 12:04 |
> | | Please respond to|
> | | qpid-dev |
> |---------+---------------------------->
> >------------------------------------------------------------------------------------------------------------------------------|
> | |
> | To: qpid-dev@incubator.apache.org |
> | cc: |
> | Subject: Re: Using AMQP pre-fetch windowing with JMS transactional sessions and client acknowledge |
> >------------------------------------------------------------------------------------------------------------------------------|
>
>
>
>
>
>
> robert.j.greig@jpmorgan.com wrote:
>> Hi,
>>
>> Is this not just "user error"? i.e. if the user sets the prefetch to x
> then
>> they must accept that they have to ack at least every x messages.
>
> For JMS applications that are already in production, forcing them to
> rewrite their code to ack or commit after every x messages may not be
> feasible. ;)
>
> Perhaps the only safe option, so as not to break JMS apps would be to
> turn it off altogether, or set the pre-fetch size to a very large value,
> although setting to a very large value could also break the app by
> overwhelming the client and leading to out of memory errors and the like.
>
> Another option would be *always* to send acks from the client to the
> server irrespective of whether JMS client acknowlege is being used or
> the jms session is transacted, and caching them on the server, and only
> *processing* them when the client explicitly acknowledges.
>
> However I guess the latter is bound to be less efficient than sending
> the acks in one go.
>
>
>
>> RG
>>
>>
>> |---------+---------------------------->
>> | | Tim Fox |
>> | | <tim.fox@jboss.co|
>> | | m> |
>> | | |
>> | | 21/09/2006 11:43 |
>> | | Please respond to|
>> | | qpid-dev |
>> |---------+---------------------------->
>>
>> ------------------------------------------------------------------------------------------------------------------------------|
>
>> |
> |
>> | To: qpid-dev@incubator.apache.org
> |
>> | cc:
> |
>> | Subject: Using AMQP pre-fetch windowing with JMS transactional
> sessions and client acknowledge |
>> ------------------------------------------------------------------------------------------------------------------------------|
>
>>
>>
>>
>> Hi all-
>>
>> I'm trying to understand how AMQP pre-fetch windowing will work with JMS
>> consumers that are using JMS client acknowledge or JMS transacted
> session.
>> It's my understanding that pre-fetch windowing works by sending another
>> batch of messages to the consumer only when the server has received a
>> certain number of acknowledgements, in such a way consumers can be
>> prevented from being overwhelmed with messages.
>>
>> However, when JMS consumers have a session with an acknowledgement mode
>> of client acknowledge, then acks are only sent for the session when the
>> client explicitly calls acknowledge() on a message.
>>
>> Similarly for JMS transacted sessions, acks are sent when commit() is
>> called on the session.
>>
>> So, if the pre-fetch window size is set to x messages, but the client
>> doesn't want to call acknowledge() or commit until it has processed y
>> messages, where y > x, then the consumer will starve at x messages.
>>
>> How do we reconcile the JMS semantic with the way AMQP is supposed to
> work?
>> Thanks in advance
>>
>>
>>
>>
>> This communication is for informational purposes only. It is not intended
> as an offer or solicitation for the purchase or sale of any financial
> instrument or as an official confirmation of any transaction. All market
> prices, data and other information are not warranted as to completeness or
> accuracy and are subject to change without notice. Any comments or
> statements made herein do not necessarily reflect those of JPMorgan Chase &
> Co., its subsidiaries and affiliates.
>> This transmission may contain information that is privileged,
> confidential, legally privileged, and/or exempt from disclosure under
> applicable law. If you are not the intended recipient, you are hereby
> notified that any disclosure, copying, distribution, or use of the
> information contained herein (including any reliance thereon) is STRICTLY
> PROHIBITED. Although this transmission and any attachments are believed to
> be free of any virus or other defect that might affect any computer system
> into which it is received and opened, it is the responsibility of the
> recipient to ensure that it is virus free and no responsibility is accepted
> by JPMorgan Chase & Co., its subsidiaries and affiliates, as applicable,
> for any loss or damage arising in any way from its use. If you received
> this transmission in error, please immediately contact the sender and
> destroy the material in its entirety, whether in electronic or hard copy
> format. Thank you.
>>
>
>
>
>
> This communication is for informational purposes only. It is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice. Any comments or statements made herein do not necessarily reflect those of JPMorgan Chase & Co., its subsidiaries and affiliates.
>
> This transmission may contain information that is privileged, confidential, legally privileged, and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. Although this transmission and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by JPMorgan Chase & Co., its subsidiaries and affiliates, as applicable, for any loss or damage arising in any way from its use. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Thank you.
>
>
Re: Using AMQP pre-fetch windowing with JMS transactional sessions and
client acknowledge
Posted by ro...@jpmorgan.com.
Hi,
Ah ok, I understand where you are coming from now.
Yes you could set it to zero to turn it off.
"although setting to a very large value could also break the app by
overwhelming the client and leading to out of memory errors and the like."
How do other implementation reconcile this kind of issue?
I definitely do not like the idea of sending an ack every time for the
efficiency reason you mention.
RG
|---------+---------------------------->
| | Tim Fox |
| | <tim.fox@jboss.co|
| | m> |
| | |
| | 21/09/2006 12:04 |
| | Please respond to|
| | qpid-dev |
|---------+---------------------------->
>------------------------------------------------------------------------------------------------------------------------------|
| |
| To: qpid-dev@incubator.apache.org |
| cc: |
| Subject: Re: Using AMQP pre-fetch windowing with JMS transactional sessions and client acknowledge |
>------------------------------------------------------------------------------------------------------------------------------|
robert.j.greig@jpmorgan.com wrote:
> Hi,
>
> Is this not just "user error"? i.e. if the user sets the prefetch to x
then
> they must accept that they have to ack at least every x messages.
For JMS applications that are already in production, forcing them to
rewrite their code to ack or commit after every x messages may not be
feasible. ;)
Perhaps the only safe option, so as not to break JMS apps would be to
turn it off altogether, or set the pre-fetch size to a very large value,
although setting to a very large value could also break the app by
overwhelming the client and leading to out of memory errors and the like.
Another option would be *always* to send acks from the client to the
server irrespective of whether JMS client acknowlege is being used or
the jms session is transacted, and caching them on the server, and only
*processing* them when the client explicitly acknowledges.
However I guess the latter is bound to be less efficient than sending
the acks in one go.
>
> RG
>
>
> |---------+---------------------------->
> | | Tim Fox |
> | | <tim.fox@jboss.co|
> | | m> |
> | | |
> | | 21/09/2006 11:43 |
> | | Please respond to|
> | | qpid-dev |
> |---------+---------------------------->
>
>------------------------------------------------------------------------------------------------------------------------------|
> |
|
> | To: qpid-dev@incubator.apache.org
|
> | cc:
|
> | Subject: Using AMQP pre-fetch windowing with JMS transactional
sessions and client acknowledge |
>
>------------------------------------------------------------------------------------------------------------------------------|
>
>
>
>
> Hi all-
>
> I'm trying to understand how AMQP pre-fetch windowing will work with JMS
> consumers that are using JMS client acknowledge or JMS transacted
session.
>
> It's my understanding that pre-fetch windowing works by sending another
> batch of messages to the consumer only when the server has received a
> certain number of acknowledgements, in such a way consumers can be
> prevented from being overwhelmed with messages.
>
> However, when JMS consumers have a session with an acknowledgement mode
> of client acknowledge, then acks are only sent for the session when the
> client explicitly calls acknowledge() on a message.
>
> Similarly for JMS transacted sessions, acks are sent when commit() is
> called on the session.
>
> So, if the pre-fetch window size is set to x messages, but the client
> doesn't want to call acknowledge() or commit until it has processed y
> messages, where y > x, then the consumer will starve at x messages.
>
> How do we reconcile the JMS semantic with the way AMQP is supposed to
work?
>
> Thanks in advance
>
>
>
>
> This communication is for informational purposes only. It is not intended
as an offer or solicitation for the purchase or sale of any financial
instrument or as an official confirmation of any transaction. All market
prices, data and other information are not warranted as to completeness or
accuracy and are subject to change without notice. Any comments or
statements made herein do not necessarily reflect those of JPMorgan Chase &
Co., its subsidiaries and affiliates.
>
> This transmission may contain information that is privileged,
confidential, legally privileged, and/or exempt from disclosure under
applicable law. If you are not the intended recipient, you are hereby
notified that any disclosure, copying, distribution, or use of the
information contained herein (including any reliance thereon) is STRICTLY
PROHIBITED. Although this transmission and any attachments are believed to
be free of any virus or other defect that might affect any computer system
into which it is received and opened, it is the responsibility of the
recipient to ensure that it is virus free and no responsibility is accepted
by JPMorgan Chase & Co., its subsidiaries and affiliates, as applicable,
for any loss or damage arising in any way from its use. If you received
this transmission in error, please immediately contact the sender and
destroy the material in its entirety, whether in electronic or hard copy
format. Thank you.
>
>
This communication is for informational purposes only. It is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice. Any comments or statements made herein do not necessarily reflect those of JPMorgan Chase & Co., its subsidiaries and affiliates.
This transmission may contain information that is privileged, confidential, legally privileged, and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. Although this transmission and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by JPMorgan Chase & Co., its subsidiaries and affiliates, as applicable, for any loss or damage arising in any way from its use. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Thank you.
Re: Using AMQP pre-fetch windowing with JMS transactional sessions
and client acknowledge
Posted by Tim Fox <ti...@jboss.com>.
robert.j.greig@jpmorgan.com wrote:
> Hi,
>
> Is this not just "user error"? i.e. if the user sets the prefetch to x then
> they must accept that they have to ack at least every x messages.
For JMS applications that are already in production, forcing them to
rewrite their code to ack or commit after every x messages may not be
feasible. ;)
Perhaps the only safe option, so as not to break JMS apps would be to
turn it off altogether, or set the pre-fetch size to a very large value,
although setting to a very large value could also break the app by
overwhelming the client and leading to out of memory errors and the like.
Another option would be *always* to send acks from the client to the
server irrespective of whether JMS client acknowlege is being used or
the jms session is transacted, and caching them on the server, and only
*processing* them when the client explicitly acknowledges.
However I guess the latter is bound to be less efficient than sending
the acks in one go.
>
> RG
>
>
> |---------+---------------------------->
> | | Tim Fox |
> | | <tim.fox@jboss.co|
> | | m> |
> | | |
> | | 21/09/2006 11:43 |
> | | Please respond to|
> | | qpid-dev |
> |---------+---------------------------->
> >------------------------------------------------------------------------------------------------------------------------------|
> | |
> | To: qpid-dev@incubator.apache.org |
> | cc: |
> | Subject: Using AMQP pre-fetch windowing with JMS transactional sessions and client acknowledge |
> >------------------------------------------------------------------------------------------------------------------------------|
>
>
>
>
> Hi all-
>
> I'm trying to understand how AMQP pre-fetch windowing will work with JMS
> consumers that are using JMS client acknowledge or JMS transacted session.
>
> It's my understanding that pre-fetch windowing works by sending another
> batch of messages to the consumer only when the server has received a
> certain number of acknowledgements, in such a way consumers can be
> prevented from being overwhelmed with messages.
>
> However, when JMS consumers have a session with an acknowledgement mode
> of client acknowledge, then acks are only sent for the session when the
> client explicitly calls acknowledge() on a message.
>
> Similarly for JMS transacted sessions, acks are sent when commit() is
> called on the session.
>
> So, if the pre-fetch window size is set to x messages, but the client
> doesn't want to call acknowledge() or commit until it has processed y
> messages, where y > x, then the consumer will starve at x messages.
>
> How do we reconcile the JMS semantic with the way AMQP is supposed to work?
>
> Thanks in advance
>
>
>
>
> This communication is for informational purposes only. It is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice. Any comments or statements made herein do not necessarily reflect those of JPMorgan Chase & Co., its subsidiaries and affiliates.
>
> This transmission may contain information that is privileged, confidential, legally privileged, and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. Although this transmission and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by JPMorgan Chase & Co., its subsidiaries and affiliates, as applicable, for any loss or damage arising in any way from its use. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Thank you.
>
>
Re: Using AMQP pre-fetch windowing with JMS transactional sessions and
client acknowledge
Posted by ro...@jpmorgan.com.
Hi,
Is this not just "user error"? i.e. if the user sets the prefetch to x then
they must accept that they have to ack at least every x messages.
RG
|---------+---------------------------->
| | Tim Fox |
| | <tim.fox@jboss.co|
| | m> |
| | |
| | 21/09/2006 11:43 |
| | Please respond to|
| | qpid-dev |
|---------+---------------------------->
>------------------------------------------------------------------------------------------------------------------------------|
| |
| To: qpid-dev@incubator.apache.org |
| cc: |
| Subject: Using AMQP pre-fetch windowing with JMS transactional sessions and client acknowledge |
>------------------------------------------------------------------------------------------------------------------------------|
Hi all-
I'm trying to understand how AMQP pre-fetch windowing will work with JMS
consumers that are using JMS client acknowledge or JMS transacted session.
It's my understanding that pre-fetch windowing works by sending another
batch of messages to the consumer only when the server has received a
certain number of acknowledgements, in such a way consumers can be
prevented from being overwhelmed with messages.
However, when JMS consumers have a session with an acknowledgement mode
of client acknowledge, then acks are only sent for the session when the
client explicitly calls acknowledge() on a message.
Similarly for JMS transacted sessions, acks are sent when commit() is
called on the session.
So, if the pre-fetch window size is set to x messages, but the client
doesn't want to call acknowledge() or commit until it has processed y
messages, where y > x, then the consumer will starve at x messages.
How do we reconcile the JMS semantic with the way AMQP is supposed to work?
Thanks in advance
This communication is for informational purposes only. It is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice. Any comments or statements made herein do not necessarily reflect those of JPMorgan Chase & Co., its subsidiaries and affiliates.
This transmission may contain information that is privileged, confidential, legally privileged, and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. Although this transmission and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by JPMorgan Chase & Co., its subsidiaries and affiliates, as applicable, for any loss or damage arising in any way from its use. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Thank you.