How to use connection_driver

["Schaik, Jacco van" <Ja...@nlr.nl>]

Hello,


Absolute beginner in the world of AMQP and Qpid here.


I'm trying to set up a C application where the AMQP traffic takes place over a socket that is under control of the application, effectively using select() to wait for data and then having Qpid handle it. As I understand it, I need to use a "connection driver" for this, but I haven't been able to find any examples that show me how.


I have looked at the C examples in the Qpid software distribution (particularly send.c and receive.c), and I'm beginning to see how things are started rolling after the PN_CONNECTION_INIT event comes in. Can I simply call pn_connection_driver_init at that point to create a connection driver? I assume the code as-is creates its own network connection, and I want to supply my own, so I assume I need to take something out too. But what?


Can anyone tell me how to proceed? Are there examples that I simply haven't found yet? Or am I barking up entirely the wrong tree here? Any guidance would be appreciated.


Groeten, Jacco

Re: How to use connection_driver

["Schaik, Jacco van" <Ja...@nlr.nl>]

Thanks for the info everyone.

Regards, Jacco

________________________________________
Van: Andrew Stitcher <as...@apache.org>
Verzonden: woensdag 20 maart 2019 04:46
Aan: users@qpid.apache.org
Onderwerp: Re: How to use connection_driver

On Tue, 2019-03-19 at 23:15 -0400, Andrew Stitcher wrote:
> ...
> > > Can anyone tell me how to proceed? Are there examples that I
> > > simply
> > > haven't found yet? Or am I barking up entirely the wrong tree
> > > here?
> > > Any guidance would be appreciated.
>
> I don't think there is a current example using the connection_driver.
> But there is test code that you could look at - although it is not
> the
> easiet thing to understand.
>
> pn_test.cpp has a memory transport implementation that is implemented
> with 2 back to back connections drivers - this should give you some
> idea.

Actually there is a much better example in the test code:
fuzz_connection_driver.c.

This has almost exactly what you need - substitute your socket reading
code for fcd_read(); your socket writing code for fdc_write() and your
event handling code for handle().

Andrew


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org

Re: How to use connection_driver

[Andrew Stitcher <as...@apache.org>]

On Tue, 2019-03-19 at 23:15 -0400, Andrew Stitcher wrote:
> ...
> > > Can anyone tell me how to proceed? Are there examples that I
> > > simply
> > > haven't found yet? Or am I barking up entirely the wrong tree
> > > here?
> > > Any guidance would be appreciated.
> 
> I don't think there is a current example using the connection_driver.
> But there is test code that you could look at - although it is not
> the
> easiet thing to understand.
> 
> pn_test.cpp has a memory transport implementation that is implemented
> with 2 back to back connections drivers - this should give you some
> idea.

Actually there is a much better example in the test code:
fuzz_connection_driver.c.

This has almost exactly what you need - substitute your socket reading
code for fcd_read(); your socket writing code for fdc_write() and your
event handling code for handle().

Andrew


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org

Re: How to use connection_driver

[Andrew Stitcher <as...@apache.org>]

On Tue, 2019-03-19 at 21:24 +0000, Gordon Sim wrote:
> On 19/03/2019 8:33 am, Schaik, Jacco van wrote:
> > Hello,
> > 
> > 
> > Absolute beginner in the world of AMQP and Qpid here.
> > 
> > 
> > I'm trying to set up a C application where the AMQP traffic takes
> > place over a socket that is under control of the application,
> > effectively using select() to wait for data and then having Qpid
> > handle it. As I understand it, I need to use a "connection driver"
> > for this, but I haven't been able to find any examples that show me
> > how.
> 
> I'm not an expert on the c api, but you don't *need* to use 
> connection_driver. It is there as a convenience.

Whilst this is technically true, you will just end up implementing the
same code (with bugs!) that is in the connection_driver code.

> 
> You can just manage the transport, connection and event collector 
> yourself, pushing data from your socket into the transport, pulling
> data 
> out the transport and writing it to your socket etc. In fact this is 
> 
> what qpidd does (i.e. the c++ broker).

I seriously don't recommend this if you want to use the proton in an
event driven way - simply because the code is subtle and bugs are too
easy to write.


> 
> 
> I have looked at the C examples in the Qpid software distribution
> > (particularly send.c and receive.c), and I'm beginning to see how
> > things are started rolling after the PN_CONNECTION_INIT event comes
> > in.

These examples are exactly how you would write the "application code"
part of your software.

The connection driver is used to handle translating bytes from the tcp
socket to events and then back to bytes going out to the socket.

> >  Can I simply call pn_connection_driver_init at that point to
> > create a connection driver? I assume the code as-is creates its own
> > network connection, and I want to supply my own, so I assume I need
> > to take something out too. But what?

The basic idea is create the connection_driver with

pn_connection_driver_init()

And then to loop "forever":

* read bytes from the socket and feed them to the connection_driver:
** get a buffer from pn_connection_driver_read_buffer()
** get bytes from the socket using read()/recv() either directly into
the buffer or copy in the bytes (with memcpy) up to the size of the
buffer.
** Tell the connection_driver it has new bytes to process with
pn_connection_driver_read_done() with the length of the new bytes.

* Process any events that have been generated:
** get the next event with pn_connection_driver_next_event()
** process this event as in the examples (Except you will never see
proactor related events as the connection driver replaces the role of
the proactor.

* write any bytes that the connection_driver needs to put to the TCP
socket.
** get a buffer to write using pn_connection_driver_write_buffer()
** use write()/send() to write the buffer to the TCP socket.
** use pn_connection_driver_write_done() to tell the connection_driver
how many bytes have been written.

This is the essential loop.
> 
> I don't think connection_driver is actually tied to any io 
> implementation, so you can still use it with your own sockets and
> your 
> own io loop.
> 
> Unfortunately, (and ironically since that was originally its main 
> purpose) there is not a great deal of documentation about
> integrating 
> the proton engine into an existing c event/io loop.

Indeed this is just too ironic.

However in our defense, this is a somewhat rare use case. Most
applications are happy for proton to manage its own TCP connections and
to poke the proton proactor event loop from its outside (perhaps
dedicating a thread to it).

> 
> The basic idea is that there is a'top half', the pn_connection_t, 
> pn_link_t etc that lets you invoke the AMQP behaviour you want and
> react 
> to AMQP events. Then there is a bootom half (the pn_transport_t, or 
> pn_connection_driver_t if you prefer, which hides the transport),
> which 
> operates on a bytes in and bytes out abstraction.
> 
> > Can anyone tell me how to proceed? Are there examples that I simply
> > haven't found yet? Or am I barking up entirely the wrong tree here?
> > Any guidance would be appreciated.

I don't think there is a current example using the connection_driver.
But there is test code that you could look at - although it is not the
easiet thing to understand.

pn_test.cpp has a memory transport implementation that is implemented
with 2 back to back connections drivers - this should give you some
idea.

This is actually written in C++ but uses C++ features beyond C lightly,
and only uses the C interface to the connection_driver.

HTH

Andrew



---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org

Re: How to use connection_driver

[Gordon Sim <gs...@redhat.com>]

On 19/03/2019 8:33 am, Schaik, Jacco van wrote:
> Hello,
> 
> 
> Absolute beginner in the world of AMQP and Qpid here.
> 
> 
> I'm trying to set up a C application where the AMQP traffic takes place over a socket that is under control of the application, effectively using select() to wait for data and then having Qpid handle it. As I understand it, I need to use a "connection driver" for this, but I haven't been able to find any examples that show me how.

I'm not an expert on the c api, but you don't *need* to use 
connection_driver. It is there as a convenience.

You can just manage the transport, connection and event collector 
yourself, pushing data from your socket into the transport, pulling data 
out the transport and writing it to your socket etc. In fact this is 
what qpidd does (i.e. the c++ broker).

The connection_driver is a utility that simplifies some of the 
interactions for that (and gives them more intuitive names).

> I have looked at the C examples in the Qpid software distribution (particularly send.c and receive.c), and I'm beginning to see how things are started rolling after the PN_CONNECTION_INIT event comes in. Can I simply call pn_connection_driver_init at that point to create a connection driver? I assume the code as-is creates its own network connection, and I want to supply my own, so I assume I need to take something out too. But what?

I don't think connection_driver is actually tied to any io 
implementation, so you can still use it with your own sockets and your 
own io loop.

Unfortunately, (and ironically since that was originally its main 
purpose) there is not a great deal of documentation about integrating 
the proton engine into an existing c event/io loop.

The basic idea is that there is a'top half', the pn_connection_t, 
pn_link_t etc that lets you invoke the AMQP behaviour you want and react 
to AMQP events. Then there is a bootom half (the pn_transport_t, or 
pn_connection_driver_t if you prefer, which hides the transport), which 
operates on a bytes in and bytes out abstraction.

> Can anyone tell me how to proceed? Are there examples that I simply haven't found yet? Or am I barking up entirely the wrong tree here? Any guidance would be appreciated.
> 
> 
> Groeten, Jacco
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org