"Initial value exchange" behaviour; what does it keep? How does it compare to last-value queues?

[Toralf Lund <to...@pgs.com>]

I'd now like to turn to the "initial value" behaviour for exchanges. Can 
anyone explain what exactly this means? Is the mechanism documented 
anywhere? Obviously, I understand that an exchange with this behaviour 
will store some values in addition to forwarding them to queues but what 
exactly does it keep? Only one message for the entire exchange? Or one 
per routing key? Or something else entirely?

Maybe should explain exactly what I'm trying to do, so that it will be 
easier to say if this behaviour is appropriate for my needs:
I won't go into details, but essentially I'm setting up a server, or 
publisher if you like, that will distribute different status readings 
from external hardware to one or more client application. Or actually, 
the setup already exists, but using a different communication 
mechanism... There are several different, but identical (and 
identifiable), hardware units, several different status values for each 
of them, and several samples over time for each value. In normal 
operation, all the values should simply be forwarded to every client, 
which is the easy bit. But, I also want to support clients that start up 
after the data has started appearing. When they do, they should get 
exactly one sample of each status value for each of the hardware units 
from the data distributed earlier - specifically, the latest instance of 
everything. This is important to get a complete state within a 
reasonable amount of time, as some of the data is sampled at a low rate.

What setup(s) would be appropriate in this case? My current prototype 
implementation has:

    * One topic exchange.
    * One routing key (topic) per status value.
    * A hardware unit identifier stored as a custom message property.
    * A last-value queue per topic, using the above mentioned property
      as "last value queue key". This is shared between clients, which
      will only ever browse messages.

It seems like this might be a way forward, but it does have some issues, 
notably that it only works correctly if the queues are created and 
correctly bound to the exchange before the publisher is started, and are 
kept that way throughout its lifetime.

Would an initial value exchange be appropriate in such a scenario? And, 
do you think it would make it easier to get a robust setup? I mean, 
ideally, the publisher should re-issue the last samples if a problem has 
occurred. This is no problem as such as the data is still available on 
the server-side, but how do I know that I have to re-send it?

TIA,


- Toralf



This e-mail, including any attachments and response string, may contain proprietary information which is confidential and may be legally privileged. It is for the intended recipient only. If you are not the intended recipient or transmission error has misdirected this e-mail, please notify the author by return e-mail and delete this message and any attachment immediately. If you are not the intended recipient you must not use, disclose, distribute, forward, copy, print or rely on this e-mail in any way except as permitted by the author.

---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org

Re: "Initial value exchange" behaviour; what does it keep? How does it compare to last-value queues?

[Gordon Sim <gs...@redhat.com>]

On 08/25/2011 01:05 PM, Toralf Lund wrote:
> I'd now like to turn to the "initial value" behaviour for exchanges. Can
> anyone explain what exactly this means? Is the mechanism documented
> anywhere? Obviously, I understand that an exchange with this behaviour
> will store some values in addition to forwarding them to queues but what
> exactly does it keep? Only one message for the entire exchange? Or one
> per routing key? Or something else entirely?

Only one message for the entire exchange.

> Maybe should explain exactly what I'm trying to do, so that it will be
> easier to say if this behaviour is appropriate for my needs:
> I won't go into details, but essentially I'm setting up a server, or
> publisher if you like, that will distribute different status readings
> from external hardware to one or more client application. Or actually,
> the setup already exists, but using a different communication
> mechanism... There are several different, but identical (and
> identifiable), hardware units, several different status values for each
> of them, and several samples over time for each value. In normal
> operation, all the values should simply be forwarded to every client,
> which is the easy bit. But, I also want to support clients that start up
> after the data has started appearing. When they do, they should get
> exactly one sample of each status value for each of the hardware units
> from the data distributed earlier - specifically, the latest instance of
> everything. This is important to get a complete state within a
> reasonable amount of time, as some of the data is sampled at a low rate.

So at any point in time each hardware unit has a value for several 
different categories of status?

> What setup(s) would be appropriate in this case? My current prototype
> implementation has:
>
> * One topic exchange.
> * One routing key (topic) per status value.
> * A hardware unit identifier stored as a custom message property.
> * A last-value queue per topic, using the above mentioned property
> as "last value queue key". This is shared between clients, which
> will only ever browse messages.
>
> It seems like this might be a way forward, but it does have some issues,
> notably that it only works correctly if the queues are created and
> correctly bound to the exchange before the publisher is started, and are
> kept that way throughout its lifetime.

What is your concern here? That the broker would crash and lose the 
queues? That there would be configuration error? That someone would 
delete or unbind the queues?

It may not address your concerns, but another suggestion would be to use 
the combination of hardware unit identifier and status as a property on 
each message and use that combined property as the key for a single 
last-value queue. All this really offers is simplicity of configuration; 
i.e. one queue that the publisher sends to and that the subscribers 
browse. It would however mean you couldn't simply request the values for 
a particular category of status.

> Would an initial value exchange be appropriate in such a scenario? And,
> do you think it would make it easier to get a robust setup? I mean,
> ideally, the publisher should re-issue the last samples if a problem has
> occurred. This is no problem as such as the data is still available on
> the server-side, but how do I know that I have to re-send it?


---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org