You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by Justin Ross <ju...@gmail.com> on 2018/02/01 13:37:02 UTC
Re: C++ imperative client API
Hi, Olivier. It's still in prototype and research mode. We don't have
anything like a design proposal yet. I hope we'll have more time to work
on it soon.
When we have some concrete design points and sample code for evaluation,
we'll share it here so we can have an open design discussion.
On Tue, Jan 30, 2018 at 12:29 AM, VERMEULEN Olivier <
Olivier.VERMEULEN@murex.com> wrote:
> Hello,
>
> We already discussed with some of you our need @Murex for a C++ imperative
> client API.
> I remember Justin saying that you were making progress on this subject but
> I can't seem to find any information about it.
> Could you give us a quick status on this? Note that we would also be
> interested in contributing to the development of this new client.
>
> Thanks,
> Olivier
> *******************************
>
> This e-mail contains information for the intended recipient only. It may
> contain proprietary material or confidential information. If you are not
> the intended recipient you are not authorised to distribute, copy or use
> this e-mail or any attachment to it. Murex cannot guarantee that it is
> virus free and accepts no responsibility for any loss or damage arising
> from its use. If you have received this e-mail in error please notify
> immediately the sender and delete the original email received, any
> attachments and all copies from your system.
>
Re: C++ imperative client API
Posted by Rabih M <ra...@gmail.com>.
Hello,
In Murex, we advanced a little bit on the subject of having an imperative
API in proton.
We thought about 2 ways of doing this:
The first way is to have a wrapper over Proton C++ with an imperative style
(like JMS API over proton-J in Java or like multithreaded_client.cpp
example):
The communication between the 2 threads will be done using futures and
queues.
To establish the connection, we can have Jms or Qpid messaging like Api.
Then have synchronous and asynchronous capabilities in the sender/receiver.
For example:
class Receiver {
/* Synchronous receive with a timeout.
* Receiver will add 1 credit on the link, and when the timeout is
reached, it asks for a drain.
* If the parameter timeout is 0, the drain will be sent right after
the add credit (equivalent for a tryReceive)
* If the parameter timeout is -1, we will wait indefinitely for the
answer (like JMS)
* If the timeout is expired and a message is received in the container
thread after the exit of this method, we will release the message, so it
comes back to the peer.
* Exceptions are thrown when the timeout is expired or when the
container thread signals an error.
*/
Message syncReceive(long timeout);
/* A batch synchronous receive with a timeout.
* Receiver will add n credit on the link, and when the timeout is
reached, it asks for a drain.
* If the timeout is 0, the drain will be sent right after the add
credit (equivalent for a tryReceive)
* If the timeout is -1 we will wait indefinitely for the answer
* If the timeout is expired and some messages are received in the
container thread after the exit of this method, we will release those
messages.
* This method has batch performance advantages over the last one.
* No exception is thrown, the vector will return the received
messages.
*/
std::vector<Message> batchReceive(int numberOfRequestedMessages, long
timeout);
/* Setting a listener.
* Similar to Jms setListner
*/
void setMessageListener(MessageListener /* can be a std::function */
listener);
/* Asynchronous receive.
* Receiver will add 1 credit on the link and we will set the value of
the future when a message or an error is received.
* If there are no messages on the other side the future.get() will
hang.
*/
std::future<Message> receive();
/* A batch asynchronous receive.
*/
std::vector<std::future<Message>> asyncBatchReceive(int
numberOfRequestedMessages);
}
The second way is just an idea I had, I did not go much into details if it
feasible, the idea is to enhance the proton container API to become thread
safe:
Let's say the container object can return thread safe objects, for example:
it can return thread safe sender object on the open_sender() method that
internally can use the work_queue to send message. For the receiver object,
we can return a future of a message when receive() is called.
What do you think of those propositions?
If you wish, we can have some voice calls to discuss more...
Best regards,
Rabih
On Tue, Feb 13, 2018 at 8:44 PM Alan Conway <ac...@redhat.com> wrote:
> On Tue, Feb 13, 2018 at 8:30 AM, Rabih M <ra...@gmail.com> wrote:
>
> > Hello,
> >
> > As Olivier said at Murex we wrote a small C++ client with an imperative
> API
> > that wraps proton. At that time we did not know that the community had
> > plans to develop one.
> >
> > The API we wrote was inspired from JMS (connection, session, messaging
> > producer, messaging consumer...). So the whole thing is similar to
> Qpid-Jms
> > wrapping Proton-J.
> >
> > What design are you thinking about?
> >
> >
> We are thinking along similar lines, see also the old qpid::messaging API
> which is quite JMS like.
>
> The main weakness of those APIs is the difficulty of handling asynchronous
> events and flow control, and an inability to write server-like
> applications.
>
> The goal is to come up with something that can be used like JMS or
> qpid::messaging for simple cases, but also allows a more asynchronous
> style of programming - without inversion of control. The thinking so far is
> to use queues and futures (or similar features depending on language) to
> handle async events so that you can write multi-threaded clients/servers or
> single-threaded apps that fire off a bunch of work in parallel, then react
> as things complete (rather than a synchronous send/wait; send/wait;
> send/wait pattern.)
>
> We have one imperative API so far in Go, with a good example of how the
> imperative style reduces coding for a simple broker example:
>
>
>
> https://github.com/apache/qpid-proton/tree/master/examples/go#a-tale-of-two-brokers
>
> In Go we use the native channel mechanism, which combines the features of
> futures and queues. In C++ we might be able to use std::future, but it is
> likely that we will need to add some queue-like primitive of our own.
>
> The big difference between Go coroutines and posix-like threads is
> scheduling efficiency. In Go it is reasonable to make every concurrent task
> into its own goroutine. In C/C++ it is not efficient to create a thread for
> every activity in a server that handles 1000s of concurrent connections -
> hence the event-driven model to multiplex onto a small thread pool. However
> for small-scale servers or clients that just want to create a handful of
> threads (e.g. to talk to a handful of back-end services) it *is* efficient
> to use a thread-per-activity - that's where a concurrent-friendly
> imperative API comes in. Also many languages (including C++) are moving
> towards some form of coroutine support so eventually the
> concurrent+imperative model will be efficient even for large scale servers.
>
> We would like to unify our effort with the community to have a working
> > C++ imperative
> > client.
> >
>
> +1 - we are still in discussion & early prototyping so your input much
> appreciated.
>
>
>
> > Best regards,
> > Rabih
> >
> > On Tue, Feb 6, 2018 at 5:54 PM, VERMEULEN Olivier <
> > Olivier.VERMEULEN@murex.com> wrote:
> >
> > > Hello Justin.
> > >
> > > Good to hear.
> > > Maybe we could also send you what we did on our side.
> > > It's still pretty basic but it can be a starting point.
> > >
> > > Olivier
> > >
> > > -----Original Message-----
> > > From: Justin Ross [mailto:justin.ross@gmail.com]
> > > Sent: jeudi 1 février 2018 14:37
> > > To: users@qpid.apache.org
> > > Subject: Re: C++ imperative client API
> > >
> > > Hi, Olivier. It's still in prototype and research mode. We don't have
> > > anything like a design proposal yet. I hope we'll have more time to
> work
> > > on it soon.
> > >
> > > When we have some concrete design points and sample code for
> evaluation,
> > > we'll share it here so we can have an open design discussion.
> > >
> > > On Tue, Jan 30, 2018 at 12:29 AM, VERMEULEN Olivier <
> > > Olivier.VERMEULEN@murex.com> wrote:
> > >
> > > > Hello,
> > > >
> > > > We already discussed with some of you our need @Murex for a C++
> > > > imperative client API.
> > > > I remember Justin saying that you were making progress on this
> subject
> > > > but I can't seem to find any information about it.
> > > > Could you give us a quick status on this? Note that we would also be
> > > > interested in contributing to the development of this new client.
> > > >
> > > > Thanks,
> > > > Olivier
> > > > *******************************
> > > >
> > > > This e-mail contains information for the intended recipient only. It
> > > > may contain proprietary material or confidential information. If you
> > > > are not the intended recipient you are not authorised to distribute,
> > > > copy or use this e-mail or any attachment to it. Murex cannot
> > > > guarantee that it is virus free and accepts no responsibility for any
> > > > loss or damage arising from its use. If you have received this e-mail
> > > > in error please notify immediately the sender and delete the original
> > > > email received, any attachments and all copies from your system.
> > > >
> > > *******************************
> > >
> > > This e-mail contains information for the intended recipient only. It
> may
> > > contain proprietary material or confidential information. If you are
> not
> > > the intended recipient you are not authorised to distribute, copy or
> use
> > > this e-mail or any attachment to it. Murex cannot guarantee that it is
> > > virus free and accepts no responsibility for any loss or damage arising
> > > from its use. If you have received this e-mail in error please notify
> > > immediately the sender and delete the original email received, any
> > > attachments and all copies from your system.
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> > > For additional commands, e-mail: users-help@qpid.apache.org
> > >
> > >
> >
>
Re: C++ imperative client API
Posted by Alan Conway <ac...@redhat.com>.
On Tue, Feb 13, 2018 at 8:30 AM, Rabih M <ra...@gmail.com> wrote:
> Hello,
>
> As Olivier said at Murex we wrote a small C++ client with an imperative API
> that wraps proton. At that time we did not know that the community had
> plans to develop one.
>
> The API we wrote was inspired from JMS (connection, session, messaging
> producer, messaging consumer...). So the whole thing is similar to Qpid-Jms
> wrapping Proton-J.
>
> What design are you thinking about?
>
>
We are thinking along similar lines, see also the old qpid::messaging API
which is quite JMS like.
The main weakness of those APIs is the difficulty of handling asynchronous
events and flow control, and an inability to write server-like applications.
The goal is to come up with something that can be used like JMS or
qpid::messaging for simple cases, but also allows a more asynchronous
style of programming - without inversion of control. The thinking so far is
to use queues and futures (or similar features depending on language) to
handle async events so that you can write multi-threaded clients/servers or
single-threaded apps that fire off a bunch of work in parallel, then react
as things complete (rather than a synchronous send/wait; send/wait;
send/wait pattern.)
We have one imperative API so far in Go, with a good example of how the
imperative style reduces coding for a simple broker example:
https://github.com/apache/qpid-proton/tree/master/examples/go#a-tale-of-two-brokers
In Go we use the native channel mechanism, which combines the features of
futures and queues. In C++ we might be able to use std::future, but it is
likely that we will need to add some queue-like primitive of our own.
The big difference between Go coroutines and posix-like threads is
scheduling efficiency. In Go it is reasonable to make every concurrent task
into its own goroutine. In C/C++ it is not efficient to create a thread for
every activity in a server that handles 1000s of concurrent connections -
hence the event-driven model to multiplex onto a small thread pool. However
for small-scale servers or clients that just want to create a handful of
threads (e.g. to talk to a handful of back-end services) it *is* efficient
to use a thread-per-activity - that's where a concurrent-friendly
imperative API comes in. Also many languages (including C++) are moving
towards some form of coroutine support so eventually the
concurrent+imperative model will be efficient even for large scale servers.
We would like to unify our effort with the community to have a working
> C++ imperative
> client.
>
+1 - we are still in discussion & early prototyping so your input much
appreciated.
> Best regards,
> Rabih
>
> On Tue, Feb 6, 2018 at 5:54 PM, VERMEULEN Olivier <
> Olivier.VERMEULEN@murex.com> wrote:
>
> > Hello Justin.
> >
> > Good to hear.
> > Maybe we could also send you what we did on our side.
> > It's still pretty basic but it can be a starting point.
> >
> > Olivier
> >
> > -----Original Message-----
> > From: Justin Ross [mailto:justin.ross@gmail.com]
> > Sent: jeudi 1 février 2018 14:37
> > To: users@qpid.apache.org
> > Subject: Re: C++ imperative client API
> >
> > Hi, Olivier. It's still in prototype and research mode. We don't have
> > anything like a design proposal yet. I hope we'll have more time to work
> > on it soon.
> >
> > When we have some concrete design points and sample code for evaluation,
> > we'll share it here so we can have an open design discussion.
> >
> > On Tue, Jan 30, 2018 at 12:29 AM, VERMEULEN Olivier <
> > Olivier.VERMEULEN@murex.com> wrote:
> >
> > > Hello,
> > >
> > > We already discussed with some of you our need @Murex for a C++
> > > imperative client API.
> > > I remember Justin saying that you were making progress on this subject
> > > but I can't seem to find any information about it.
> > > Could you give us a quick status on this? Note that we would also be
> > > interested in contributing to the development of this new client.
> > >
> > > Thanks,
> > > Olivier
> > > *******************************
> > >
> > > This e-mail contains information for the intended recipient only. It
> > > may contain proprietary material or confidential information. If you
> > > are not the intended recipient you are not authorised to distribute,
> > > copy or use this e-mail or any attachment to it. Murex cannot
> > > guarantee that it is virus free and accepts no responsibility for any
> > > loss or damage arising from its use. If you have received this e-mail
> > > in error please notify immediately the sender and delete the original
> > > email received, any attachments and all copies from your system.
> > >
> > *******************************
> >
> > This e-mail contains information for the intended recipient only. It may
> > contain proprietary material or confidential information. If you are not
> > the intended recipient you are not authorised to distribute, copy or use
> > this e-mail or any attachment to it. Murex cannot guarantee that it is
> > virus free and accepts no responsibility for any loss or damage arising
> > from its use. If you have received this e-mail in error please notify
> > immediately the sender and delete the original email received, any
> > attachments and all copies from your system.
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> > For additional commands, e-mail: users-help@qpid.apache.org
> >
> >
>
Re: C++ imperative client API
Posted by Rabih M <ra...@gmail.com>.
Hello,
As Olivier said at Murex we wrote a small C++ client with an imperative API
that wraps proton. At that time we did not know that the community had
plans to develop one.
The API we wrote was inspired from JMS (connection, session, messaging
producer, messaging consumer...). So the whole thing is similar to Qpid-Jms
wrapping Proton-J.
What design are you thinking about?
We would like to unify our effort with the community to have a working
C++ imperative
client.
Best regards,
Rabih
On Tue, Feb 6, 2018 at 5:54 PM, VERMEULEN Olivier <
Olivier.VERMEULEN@murex.com> wrote:
> Hello Justin.
>
> Good to hear.
> Maybe we could also send you what we did on our side.
> It's still pretty basic but it can be a starting point.
>
> Olivier
>
> -----Original Message-----
> From: Justin Ross [mailto:justin.ross@gmail.com]
> Sent: jeudi 1 février 2018 14:37
> To: users@qpid.apache.org
> Subject: Re: C++ imperative client API
>
> Hi, Olivier. It's still in prototype and research mode. We don't have
> anything like a design proposal yet. I hope we'll have more time to work
> on it soon.
>
> When we have some concrete design points and sample code for evaluation,
> we'll share it here so we can have an open design discussion.
>
> On Tue, Jan 30, 2018 at 12:29 AM, VERMEULEN Olivier <
> Olivier.VERMEULEN@murex.com> wrote:
>
> > Hello,
> >
> > We already discussed with some of you our need @Murex for a C++
> > imperative client API.
> > I remember Justin saying that you were making progress on this subject
> > but I can't seem to find any information about it.
> > Could you give us a quick status on this? Note that we would also be
> > interested in contributing to the development of this new client.
> >
> > Thanks,
> > Olivier
> > *******************************
> >
> > This e-mail contains information for the intended recipient only. It
> > may contain proprietary material or confidential information. If you
> > are not the intended recipient you are not authorised to distribute,
> > copy or use this e-mail or any attachment to it. Murex cannot
> > guarantee that it is virus free and accepts no responsibility for any
> > loss or damage arising from its use. If you have received this e-mail
> > in error please notify immediately the sender and delete the original
> > email received, any attachments and all copies from your system.
> >
> *******************************
>
> This e-mail contains information for the intended recipient only. It may
> contain proprietary material or confidential information. If you are not
> the intended recipient you are not authorised to distribute, copy or use
> this e-mail or any attachment to it. Murex cannot guarantee that it is
> virus free and accepts no responsibility for any loss or damage arising
> from its use. If you have received this e-mail in error please notify
> immediately the sender and delete the original email received, any
> attachments and all copies from your system.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
RE: C++ imperative client API
Posted by VERMEULEN Olivier <Ol...@murex.com>.
Hello Justin.
Good to hear.
Maybe we could also send you what we did on our side.
It's still pretty basic but it can be a starting point.
Olivier
-----Original Message-----
From: Justin Ross [mailto:justin.ross@gmail.com]
Sent: jeudi 1 février 2018 14:37
To: users@qpid.apache.org
Subject: Re: C++ imperative client API
Hi, Olivier. It's still in prototype and research mode. We don't have anything like a design proposal yet. I hope we'll have more time to work on it soon.
When we have some concrete design points and sample code for evaluation, we'll share it here so we can have an open design discussion.
On Tue, Jan 30, 2018 at 12:29 AM, VERMEULEN Olivier < Olivier.VERMEULEN@murex.com> wrote:
> Hello,
>
> We already discussed with some of you our need @Murex for a C++
> imperative client API.
> I remember Justin saying that you were making progress on this subject
> but I can't seem to find any information about it.
> Could you give us a quick status on this? Note that we would also be
> interested in contributing to the development of this new client.
>
> Thanks,
> Olivier
> *******************************
>
> This e-mail contains information for the intended recipient only. It
> may contain proprietary material or confidential information. If you
> are not the intended recipient you are not authorised to distribute,
> copy or use this e-mail or any attachment to it. Murex cannot
> guarantee that it is virus free and accepts no responsibility for any
> loss or damage arising from its use. If you have received this e-mail
> in error please notify immediately the sender and delete the original
> email received, any attachments and all copies from your system.
>
*******************************
This e-mail contains information for the intended recipient only. It may contain proprietary material or confidential information. If you are not the intended recipient you are not authorised to distribute, copy or use this e-mail or any attachment to it. Murex cannot guarantee that it is virus free and accepts no responsibility for any loss or damage arising from its use. If you have received this e-mail in error please notify immediately the sender and delete the original email received, any attachments and all copies from your system.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org