[jira] [Commented] (QPID-5799) Allow qpid.messaging.endpoints.session to have all associated receivers mointored via epoll mechanism

["Ken Giusti (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/QPID-5799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14306127#comment-14306127 ] 

Ken Giusti commented on QPID-5799:
----------------------------------

Proposed patch:

https://reviews.apache.org/r/30640/

This callback can be used to cause a write on a socket that is being monitored from the application's epoll loop.  This will cause the application to 'wake up' and it can poll for the next ready receiver.

Since this callback is being invoked by the driver thread, the callback code cannot call any of the qpid::messaging APIs.  The qpid::messaging APIs are expected to be called from the application thread - calling them from the driver thread (as via the callback) will lead to undefined behavior.

> Allow qpid.messaging.endpoints.session to have all associated receivers mointored via epoll mechanism
> -----------------------------------------------------------------------------------------------------
>
>                 Key: QPID-5799
>                 URL: https://issues.apache.org/jira/browse/QPID-5799
>             Project: Qpid
>          Issue Type: Improvement
>          Components: Python Client
>            Reporter: Brian Bouterse
>            Assignee: Ken Giusti
>
> Kombu [0] is a python producer/consumer software that I've written a qpid transport[1] for. This qpid transport for Kombu, uses qpid.messaging, and has all consuming receivers associated with a single session endpoint.
> Celery[2] uses Kombu, and has an asynchronous I/O event loop, and would like to monitor a single file descriptor for reading to know when there is any message available on any receiver associated with the session endpoint.
> Currently as a workaround, I create a pipe with one read and one write file descriptors. I register the read file descriptor with Celery to monitor. I then start a separate thread (Thread1) that calls next_receiver() on the session in an infinite loop. Each time next_receiver() returns instead of fetching the message, Thread1 writes a symbol to the file descriptor Celery is monitoring. Celery then calls the on_readable() callback from the MainThread, which removes a symbol from the pipe Celery is monitoring (clearing the epoll readable attribute), and then calls next_receiver() again only this time from MainThread, to acquire the receiver, and fetch() the message and process it.
> This architecture is not ideal in two ways.
> 1. There is a redundant call to next_receiver() that happens with every read.
> 2. A separate thread to "monitor" qpid messaging should not be required, when Celery is already capable of monitoring a single file descriptor, and can call the read method which would use next_receiver() to drain from all receivers associated with the session.
> [0]:  https://github.com/celery/kombu
> [1]:  https://github.com/pulp/kombu/blob/pulp-dep-3.0.15-with-qpid/kombu/transport/qpid.py
> [2]:  https://github.com/celery/celery



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org