Losing messages

[Caio Brentano <ca...@gmail.com>]

Hello folks

I've noticed that I'm losing some messages and I need to elimanate these
loses.
I'm using python 2.4 and qpid 0.5.

This is how the client send messages to the queue.

        # Create a connection.
        try:
                lvSocket = connect(lvHost, lvPort)
        except Exception, e:
                log.critical('Error: host [%s] port [%d]'%(lvHost, lvPort))
                sys.exit(-1)

        lvConnection = Connection(sock=lvSocket, username=lvUser,
password=lvPassword)
        lvConnection.start()

        lvSession = lvConnection.session(str(uuid4()))

        lvProps = lvSession.delivery_properties(routing_key=pvRoutingKey)

        try:
                lvSession.message_transfer(destination=pvQpidExch,
message=Message(lvProps, pvMsg))
        except qpid.exceptions.Timeout:
                log.warning('message_transfer exception')


My guess is that I need to configure the session to send sync messages,
right?

What should I do to really guarantee the delivery?

Btw, Python documentation is very unfriendly! =(


Thanks in advance
-- 
Caio Brentano

Re: Losing messages

[Rafael Schloming <ra...@redhat.com>]

Caio Brentano wrote:
> Hello folks
> 
> I've noticed that I'm losing some messages and I need to elimanate these
> loses.
> I'm using python 2.4 and qpid 0.5.
> 
> This is how the client send messages to the queue.
> 
>         # Create a connection.
>         try:
>                 lvSocket = connect(lvHost, lvPort)
>         except Exception, e:
>                 log.critical('Error: host [%s] port [%d]'%(lvHost, lvPort))
>                 sys.exit(-1)
> 
>         lvConnection = Connection(sock=lvSocket, username=lvUser,
> password=lvPassword)
>         lvConnection.start()
> 
>         lvSession = lvConnection.session(str(uuid4()))
> 
>         lvProps = lvSession.delivery_properties(routing_key=pvRoutingKey)
> 
>         try:
>                 lvSession.message_transfer(destination=pvQpidExch,
> message=Message(lvProps, pvMsg))
>         except qpid.exceptions.Timeout:
>                 log.warning('message_transfer exception')
> 
> 
> My guess is that I need to configure the session to send sync messages,
> right?

Actually it should do that by default.

> What should I do to really guarantee the delivery?

You would need to retry rather than ignore failed transfer attempts.

Note that depending on what you mean, even retrying on publish may not 
actually guarantee *delivery*, it would just guarantee that the message 
made it from the publisher to the broker. Messages could still be 
dropped later on for a variety of reasons, e.g. if no queue is bound to 
the exchange, if the ttl expires before the messages is consumed, if a 
consumer uses accept-mode=None.

> Btw, Python documentation is very unfriendly! =(

FWIW, I would strongly recommend giving the new python client a try. 
It's much simpler, easier to use, and it is better documented. You can 
configure it to automatically reconnect and replay messages for you 
rather than having to code that yourself.

You can get the preview version from the 0.6 release, but I'd actually 
recommend pulling it from trunk as there have been numerous fixes since 
0.6.

It should work fine with older versions of the broker, so you can just 
try out the new client if you like. If you're familiar with distutils 
you can use trunk/qpid/python/setup.py to install the client.

--Rafael


---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org