AMQP Commit problems with 0-8/0-9 Spec

[Martin Ritchie <ri...@apache.org>]

Hi,

Just raised a bug as a result of a CI failure for the SyncWaitTimeoutDelayTest.

It appears to me to be a protocol bug anyone fluent in 0-10 able to
say if the bug is also in 0-10?

Is there going to be a 0-9 update that might address this?

https://issues.apache.org/jira/browse/QPID-1262

The problem in a nutshell:

TxCommitOk is not correlated with the TxCommit that initiated the work
on the broker.
So if our broker takes a long time (using SlowMessageStore) to perform
commit and client times out the wait for the TxCommitOK (as in the
SyncWaitTimeoutDelayTest) then it is possible that if a subsequent
TxCommit is sent that the TxCommitOk that is returned signals the wait
by mistake.

AMQP Method Sequence:
[C]lient
[B]roker
[S]end
[R]eceive

CS: TxCommit  (a)
BR: TxCommit  (a)
// Broker takes a lot of time
// Client times out waiting for TxCommit (a)
CS: TxCommit  (b)
BS: TxCommitOk (a)
CR: TxCommitOk  (a)
// At this point the the client thinks that its commit (a) has
succeeded, it hasn't.

My only thoughts were
a) add correlation ids to the TxCommit TxCommitOk pairs, as was done
above for clarity in the explanation.
b) close the session in the event of a timeout and re-establish session.

thoughts?
-- 
Martin Ritchie

Re: AMQP Commit problems with 0-8/0-9 Spec

[Robert Godfrey <ro...@gmail.com>]

2008/8/26 Martin Ritchie <ri...@apache.org>:
> Hi,
>
> Just raised a bug as a result of a CI failure for the SyncWaitTimeoutDelayTest.
>
> It appears to me to be a protocol bug anyone fluent in 0-10 able to
> say if the bug is also in 0-10?
>
> Is there going to be a 0-9 update that might address this?
>
> https://issues.apache.org/jira/browse/QPID-1262
>
> The problem in a nutshell:
>
> TxCommitOk is not correlated with the TxCommit that initiated the work
> on the broker.
> So if our broker takes a long time (using SlowMessageStore) to perform
> commit and client times out the wait for the TxCommitOK (as in the
> SyncWaitTimeoutDelayTest) then it is possible that if a subsequent
> TxCommit is sent that the TxCommitOk that is returned signals the wait
> by mistake.
>
> AMQP Method Sequence:
> [C]lient
> [B]roker
> [S]end
> [R]eceive
>
> CS: TxCommit  (a)
> BR: TxCommit  (a)
> // Broker takes a lot of time
> // Client times out waiting for TxCommit (a)
> CS: TxCommit  (b)
> BS: TxCommitOk (a)
> CR: TxCommitOk  (a)
> // At this point the the client thinks that its commit (a) has
> succeeded, it hasn't.
>
> My only thoughts were
> a) add correlation ids to the TxCommit TxCommitOk pairs, as was done
> above for clarity in the explanation.
> b) close the session in the event of a timeout and re-establish session.
>

Option b) is the only safe alternative for 0-8/0-9.

Completion of commands is correlated in 0-10 so this is no longer an issue...

-- Rob

> thoughts?
> --
> Martin Ritchie
>

Re: Now that M3 is complete...

[Marnie McCormack <ma...@googlemail.com>]

Hi Carl,

Thanks for the info, think the date sounds fine !

I'm on vacation this week but will take a closer look and consider in more
detail asap.

Rgds,
Marnie


On 8/26/08, Joshua Kramer <jo...@globalherald.net> wrote:
>
>
> I should have something better by the end of the week.  I need to make sure
> my build environment is up to snuff because I haven't touched it in a couple
> of months.
>
> On Tue, 26 Aug 2008, Carl Trieloff wrote:
>
> Date: Tue, 26 Aug 2008 09:44:46 -0400
>> From: Carl Trieloff <cc...@redhat.com>
>> Reply-To: qpid-dev@incubator.apache.org, cctrieloff@redhat.com
>> To: Joshua Kramer <jo...@globalherald.net>
>> Cc: qpid-dev@incubator.apache.org
>> Subject: Re: Now that M3 is complete...
>>
>>
>> Joshua,
>>
>> That is fantastic, do you have a patch yet that can be looked at?
>>
>> Carl
>>
>>
>> Joshua Kramer wrote:
>>
>>>
>>> Hello Carl,
>>>
>>> On Tue, 26 Aug 2008, Carl Trieloff wrote:
>>>
>>> - Encryption (SASL/TLS) (all clients)
>>>>
>>>
>>> I started work on adding SSL to the C++ server/client a while back but
>>> got very busy at work.  Now that my schedule has freed up a bit, I'll start
>>> working on this again.
>>>
>>> Thanks!
>>> -Joshua Kramer
>>>
>>> -----
>>> http://www.globalherald.net/jb01
>>> GlobalHerald.NET, the Smarter Social Network! (tm)
>>>
>>
>>
>>
> --
>
> -----
> http://www.globalherald.net/jb01
> GlobalHerald.NET, the Smarter Social Network! (tm)
>

Re: Now that M3 is complete...

[Joshua Kramer <jo...@globalherald.net>]

I should have something better by the end of the week.  I need to make 
sure my build environment is up to snuff because I haven't touched it in a 
couple of months.

On Tue, 26 Aug 2008, Carl Trieloff wrote:

> Date: Tue, 26 Aug 2008 09:44:46 -0400
> From: Carl Trieloff <cc...@redhat.com>
> Reply-To: qpid-dev@incubator.apache.org, cctrieloff@redhat.com
> To: Joshua Kramer <jo...@globalherald.net>
> Cc: qpid-dev@incubator.apache.org
> Subject: Re: Now that M3 is complete...
> 
>
> Joshua,
>
> That is fantastic, do you have a patch yet that can be looked at?
>
> Carl
>
>
> Joshua Kramer wrote:
>> 
>> Hello Carl,
>> 
>> On Tue, 26 Aug 2008, Carl Trieloff wrote:
>> 
>>> - Encryption (SASL/TLS) (all clients)
>> 
>> I started work on adding SSL to the C++ server/client a while back but got 
>> very busy at work.  Now that my schedule has freed up a bit, I'll start 
>> working on this again.
>> 
>> Thanks!
>> -Joshua Kramer
>> 
>> -----
>> http://www.globalherald.net/jb01
>> GlobalHerald.NET, the Smarter Social Network! (tm)
>
>

-- 

-----
http://www.globalherald.net/jb01
GlobalHerald.NET, the Smarter Social Network! (tm)

Re: Now that M3 is complete...

[Carl Trieloff <cc...@redhat.com>]

Joshua,

That is fantastic, do you have a patch yet that can be looked at?

Carl


Joshua Kramer wrote:
>
> Hello Carl,
>
> On Tue, 26 Aug 2008, Carl Trieloff wrote:
>
>> - Encryption (SASL/TLS) (all clients)
>
> I started work on adding SSL to the C++ server/client a while back but 
> got very busy at work.  Now that my schedule has freed up a bit, I'll 
> start working on this again.
>
> Thanks!
> -Joshua Kramer
>
> -----
> http://www.globalherald.net/jb01
> GlobalHerald.NET, the Smarter Social Network! (tm)

Re: Now that M3 is complete...

[Joshua Kramer <jo...@globalherald.net>]

Hello Carl,

On Tue, 26 Aug 2008, Carl Trieloff wrote:

> - Encryption (SASL/TLS) (all clients)

I started work on adding SSL to the C++ server/client a while back but got 
very busy at work.  Now that my schedule has freed up a bit, I'll start 
working on this again.

Thanks!
-Joshua Kramer

-----
http://www.globalherald.net/jb01
GlobalHerald.NET, the Smarter Social Network! (tm)

Last image caching (was Re: Now that M3 is complete...)

[Gordon Sim <gs...@redhat.com>]

Robert Greig wrote:
> I think it would be interesting to play this out in one of the most
> common use cases - equity prices. So assume that we have a topic
> hierarchy that people subscribe to e.g. stocks.nasdaq, stocks.ftse
> etc. Messages are published with routing keys like stocks.nasdaq.MSFT
> etc. Consumers subscribe to e.g. stocks.nasdaq.*
> 
> With a single replay queue, it would seem to me that it would be
> necessary to store the routing key used to publish the message (i.e.
> this is some special queue). Then when a new consumer subscribes, the
> replay queue is given the routing key and filters using some variant
> of a message selector to get the relevant messages.

Yes, my thought was that we could support this by using another (as yet 
unimplemented) feature, whereby messages with a particular value for the 
routing_key (or indeed any other property) would 'overwrite' any queued 
message with the same value.

I.e. the queue would always hold the last message sent matching a 
particular criteria.

However, perhaps thats overkill...

> I can see you could do it with multiple replay queues (e.g. one per
> routing key) which would mean you could use standard queues and avoid
> any kind of selector, pushing that bit into the exchange.
> 
> I don't think what I was thinking about was too different, I just
> thought that since the exchange had access to the routing key it would
> be quite easy to cache a message keyed on the routing key.

The simplicity of that is certainly attractive. I need to think some 
more and am keen to hear any/all thoughts and ideas on this.

Re: Now that M3 is complete...

[Robert Greig <ro...@gmail.com>]

2008/9/9 Gordon Sim <gs...@redhat.com>:

> I haven't yet thought about it in sufficient detail, but I was imagining
> allowing a 'replay queue' to be attached to an exchange instance (of any of
> the existing types), and in bind requests for that exchange allowing clients
> to ask that the queue they are binding first be populated with messages from
> that replay queue (synchronised w.r.t. the incoming message stream so
> nothing is lost).
>
> The replay queue could then have policies on it restricting the number of
> messages it retains etc. In the simple case it might just hold a single
> message, but it could hold more than that if required.

I think it would be interesting to play this out in one of the most
common use cases - equity prices. So assume that we have a topic
hierarchy that people subscribe to e.g. stocks.nasdaq, stocks.ftse
etc. Messages are published with routing keys like stocks.nasdaq.MSFT
etc. Consumers subscribe to e.g. stocks.nasdaq.*

With a single replay queue, it would seem to me that it would be
necessary to store the routing key used to publish the message (i.e.
this is some special queue). Then when a new consumer subscribes, the
replay queue is given the routing key and filters using some variant
of a message selector to get the relevant messages.

I can see you could do it with multiple replay queues (e.g. one per
routing key) which would mean you could use standard queues and avoid
any kind of selector, pushing that bit into the exchange.

I don't think what I was thinking about was too different, I just
thought that since the exchange had access to the routing key it would
be quite easy to cache a message keyed on the routing key. That does
assume a single message per routing key (which fits the stock price
use case but I can see that multiple values per routing key may be
useful).

RG

Re: Now that M3 is complete...

[Marnie McCormack <ma...@googlemail.com>]

Hi All,

In case it's useful, there's a page dedicated to replay (which I *think*
Colin Christ wrote) here:

http://cwiki.apache.org/qpid/queue-replay.html

Marnie

On Tue, Sep 9, 2008 at 10:44 AM, Gordon Sim <gs...@redhat.com> wrote:

>  Robert Greig wrote:
>
>> 2008/9/8 Gordon Sim <gs...@redhat.com>:
>>
>>> Andrew M wrote:
>>>
>>>> Do you guys think someone could implement this for M4?
>>>> https://issues.apache.org/jira/browse/QPID-1248
>>>>
>>> Yes, I'm going to try and get that done, its a useful feature to have.
>>>
>>
>> Out of curiosity, is the easiest way to do this with a custom
>> exchange? Does the C++ broker support chaining of exchanges?
>>
>
> I haven't yet thought about it in sufficient detail, but I was imagining
> allowing a 'replay queue' to be attached to an exchange instance (of any of
> the existing types), and in bind requests for that exchange allowing clients
> to ask that the queue they are binding first be populated with messages from
> that replay queue (synchronised w.r.t. the incoming message stream so
> nothing is lost).
>
> The replay queue could then have policies on it restricting the number of
> messages it retains etc. In the simple case it might just hold a single
> message, but it could hold more than that if required.
>
> Thoughts?
>

Re: Now that M3 is complete...

[Gordon Sim <gs...@redhat.com>]

Robert Greig wrote:
> 2008/9/8 Gordon Sim <gs...@redhat.com>:
>> Andrew M wrote:
>>> Do you guys think someone could implement this for M4?
>>> https://issues.apache.org/jira/browse/QPID-1248
>> Yes, I'm going to try and get that done, its a useful feature to have.
> 
> Out of curiosity, is the easiest way to do this with a custom
> exchange? Does the C++ broker support chaining of exchanges?

I haven't yet thought about it in sufficient detail, but I was imagining 
allowing a 'replay queue' to be attached to an exchange instance (of any 
of the existing types), and in bind requests for that exchange allowing 
clients to ask that the queue they are binding first be populated with 
messages from that replay queue (synchronised w.r.t. the incoming 
message stream so nothing is lost).

The replay queue could then have policies on it restricting the number 
of messages it retains etc. In the simple case it might just hold a 
single message, but it could hold more than that if required.

Thoughts?

Re: Now that M3 is complete...

[Robert Greig <ro...@gmail.com>]

2008/9/8 Gordon Sim <gs...@redhat.com>:
> Andrew M wrote:
>>
>> Do you guys think someone could implement this for M4?
>> https://issues.apache.org/jira/browse/QPID-1248
>
> Yes, I'm going to try and get that done, its a useful feature to have.

Out of curiosity, is the easiest way to do this with a custom
exchange? Does the C++ broker support chaining of exchanges?

RG

Re: Now that M3 is complete...

[Gordon Sim <gs...@redhat.com>]

Andrew M wrote:
> Do you guys think someone could implement this for M4?
> https://issues.apache.org/jira/browse/QPID-1248

Yes, I'm going to try and get that done, its a useful feature to have.

Re: Now that M3 is complete...

[Carl Trieloff <cc...@redhat.com>]

Andrew M,

This has been committed as of svn revision 705443.

See
http://cwiki.apache.org/confluence/display/qpid/Cheat+Sheet+for+configuring+Exchange+Options

on how to use, let me know if it works for you

enjoy,
Carl.


Andrew M wrote:
> Do you guys think someone could implement this for M4?
> https://issues.apache.org/jira/browse/QPID-1248
>
> thanks!
>
> -----Original Message-----
> From: Robert Greig [mailto:robert.j.greig@gmail.com] 
> Sent: Monday, September 08, 2008 1:00 PM
> To: qpid-dev@incubator.apache.org
> Subject: Re: Now that M3 is complete...
>
> 2008/8/26 Carl Trieloff <cc...@redhat.com>:
>   
>> I know that we had planned to code freeze M4 on October 10th. Doing some
>> planning
>> would anyone object if we pushed the code freeze date for M4 back to Oct
>> 22nd.
>>     
>
> Just back from holiday so apologies for the late response.
>
> Do we have any effort estimates for the listed items you would like to
> see? Are we looking to do get a list of items then figure out what can
> be done in the time available? Or have you done this already hence the
> date change?
>
> RG
>
>   

RE: Now that M3 is complete...

[Andrew M <an...@oc384.net>]

Do you guys think someone could implement this for M4?
https://issues.apache.org/jira/browse/QPID-1248

thanks!

-----Original Message-----
From: Robert Greig [mailto:robert.j.greig@gmail.com] 
Sent: Monday, September 08, 2008 1:00 PM
To: qpid-dev@incubator.apache.org
Subject: Re: Now that M3 is complete...

2008/8/26 Carl Trieloff <cc...@redhat.com>:
>
> I know that we had planned to code freeze M4 on October 10th. Doing some
> planning
> would anyone object if we pushed the code freeze date for M4 back to Oct
> 22nd.

Just back from holiday so apologies for the late response.

Do we have any effort estimates for the listed items you would like to
see? Are we looking to do get a list of items then figure out what can
be done in the time available? Or have you done this already hence the
date change?

RG

Re: Now that M3 is complete...

[Robert Greig <ro...@gmail.com>]

2008/8/26 Carl Trieloff <cc...@redhat.com>:
>
> I know that we had planned to code freeze M4 on October 10th. Doing some
> planning
> would anyone object if we pushed the code freeze date for M4 back to Oct
> 22nd.

Just back from holiday so apologies for the late response.

Do we have any effort estimates for the listed items you would like to
see? Are we looking to do get a list of items then figure out what can
be done in the time available? Or have you done this already hence the
date change?

RG

Now that M3 is complete...

[Carl Trieloff <cc...@redhat.com>]

I know that we had planned to code freeze M4 on October 10th. Doing some 
planning
would anyone object if we pushed the code freeze date for M4 back to Oct 
22nd.

Please add items you want to or would like to address, The list of 
larger items I would like
to see us target in M4 are,
- Cluster complete
- IB complete
- Ports complete
- ttl and expiry of stale messages
- Java client kerberos support (ok, all clients)
- Encryption (SASL/TLS) (all clients)
- .NET client 0-10 support
- Java broker mostly if not fully supporting 0-10
- ACL complete
- mgnt agents for C++ & python
- Cluster complete
- More performance work
- Broker audit logs/ events
- (JMX bridge started by Rahaul)?

regards
Carl.