You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by "Cliff Jansen (Interop Systems Inc)" <v-...@microsoft.com> on 2009/08/04 22:39:51 UTC
RE: c++ messaging api: a map message example
Hi Gordon,
> Jonathan previously requested support for the stream operator and I've
> had a tentative stab at exploring what that might look like as well.
In addition, I would like to see the some stream-ish functions that
are friendly to char* blobs that are not null terminated. Something
along the lines of:
MessageContent& MessageContent::write (const char* s, int n);
MessageContent& MessageContent::read (const char* s, int n); // + rewind() or seek
As well as something like
int MessageContent::size(); // (or getSize(), getLength()...)
Message::Message(const char* s, int n);
Note that std::string is not copy-on-write in all C++ implementations.
On Windows, forcing an application to use strings for its raw data (from
a char* perspective) results in needless copies of the content:
string tmpdata(myBlobPtr, myBlobSize); // memory copy #1
Message m(tmpdata); // m.impl->bytes gets copy #2 when constructed
// tmpdata never used again, freed when out of scope
The programmer can take evasive action:
Message m; // m.impl->bytes starts out as a sacrificial empty string
m.getBytes().assign(myBlobPtr, myBlobSize); // just one copy
but
Message m(myBlobPtr, myBlobSize);
might create the FrameSet directly, and cut out another middleman.
Since MessageContent already has abstract content (string, MAP, and
LIST), extentending the set to include char* could allow performance
optimizations:
For incoming messages, the actual content could be copied from the
FrameSet directly to an application buffer. (This can be a win even
on Linux when messages are larger than the connection's
maxFrameSize.) The string form can be lazilly created when needed.
For outgoing messages, the content may be placed directly in special
OS memory in preparation for network transfer.
Cliff
-----Original Message-----
From: Gordon Sim [mailto:gsim@redhat.com]
Sent: Friday, July 31, 2009 10:11 AM
To: Qpid Dev; users@qpid.apache.org
Subject: c++ messaging api: a map message example
Attached is a patch for a higher level, protocol independent messaging api from c++ as last discussed some months ago. I have not had much time to spend on this since then and so there is not a huge amount of change (mainly moving to a better namespace and adjusting for the revised include directory layout).
There is however now an example of a sending a map message using the amqp0-10 encoding (list messages are also supported) and the message headers are now correctly sent. Jonathan previously requested support for the stream operator and I've had a tentative stab at exploring what that might look like as well.
As always keen to get any feedback. I'm hoping to have more time to spend on this for the next few weeks. I'm aiming to align the c++ api with the similar work on python (at least conceptually) and to flesh out the functionality (determining how best to expose asynchronous completion of sends and acknowledgements will be a likely next step).
--Gordon.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
Re: c++ messaging api: a map message example
Posted by Alan Conway <ac...@redhat.com>.
On 08/04/2009 04:39 PM, Cliff Jansen (Interop Systems Inc) wrote:
> Hi Gordon,
>
>> Jonathan previously requested support for the stream operator and I've
>> had a tentative stab at exploring what that might look like as well.
>
> In addition, I would like to see the some stream-ish functions that
> are friendly to char* blobs that are not null terminated. Something
> along the lines of:
>
> MessageContent& MessageContent::write (const char* s, int n);
I think append() would be more self-explanatory (if that's what you meant)
> MessageContent& MessageContent::read (const char* s, int n); // + rewind() or seek
That raises an interesting question: do we force message to keep its content in
a single contiguous buffer? If not we need a way for the user to iterate over
the data in non-contiguous buffers. An iterator of sorts that returns successive
pair<char*, size_t> would do it.
>
> As well as something like
>
> int MessageContent::size(); // (or getSize(), getLength()...)
>
> Message::Message(const char* s, int n);
>
> Note that std::string is not copy-on-write in all C++ implementations.
> On Windows, forcing an application to use strings for its raw data (from
> a char* perspective) results in needless copies of the content:
>
> string tmpdata(myBlobPtr, myBlobSize); // memory copy #1
> Message m(tmpdata); // m.impl->bytes gets copy #2 when constructed
> // tmpdata never used again, freed when out of scope
>
> The programmer can take evasive action:
>
> Message m; // m.impl->bytes starts out as a sacrificial empty string
> m.getBytes().assign(myBlobPtr, myBlobSize); // just one copy
>
> but
>
> Message m(myBlobPtr, myBlobSize);
>
> might create the FrameSet directly, and cut out another middleman.
>
> Since MessageContent already has abstract content (string, MAP, and
> LIST), extentending the set to include char* could allow performance
> optimizations:
>
> For incoming messages, the actual content could be copied from the
> FrameSet directly to an application buffer. (This can be a win even
> on Linux when messages are larger than the connection's
> maxFrameSize.) The string form can be lazilly created when needed.
>
> For outgoing messages, the content may be placed directly in special
> OS memory in preparation for network transfer.
>
>
> Cliff
>
>
> -----Original Message-----
> From: Gordon Sim [mailto:gsim@redhat.com]
> Sent: Friday, July 31, 2009 10:11 AM
> To: Qpid Dev; users@qpid.apache.org
> Subject: c++ messaging api: a map message example
>
> Attached is a patch for a higher level, protocol independent messaging api from c++ as last discussed some months ago. I have not had much time to spend on this since then and so there is not a huge amount of change (mainly moving to a better namespace and adjusting for the revised include directory layout).
>
> There is however now an example of a sending a map message using the amqp0-10 encoding (list messages are also supported) and the message headers are now correctly sent. Jonathan previously requested support for the stream operator and I've had a tentative stab at exploring what that might look like as well.
>
> As always keen to get any feedback. I'm hoping to have more time to spend on this for the next few weeks. I'm aiming to align the c++ api with the similar work on python (at least conceptually) and to flesh out the functionality (determining how best to expose asynchronous completion of sends and acknowledgements will be a likely next step).
>
> --Gordon.
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:dev-subscribe@qpid.apache.org
>
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
Re: c++ messaging api: a map message example
Posted by Gordon Sim <gs...@redhat.com>.
Cliff Jansen (Interop Systems Inc) wrote:
> Hi Gordon,
Hi Cliff,
Thanks for the feedback, much appreciated!
>> Jonathan previously requested support for the stream operator and I've
>> had a tentative stab at exploring what that might look like as well.
>
> In addition, I would like to see the some stream-ish functions that
> are friendly to char* blobs that are not null terminated. Something
> along the lines of:
>
> MessageContent& MessageContent::write (const char* s, int n);
> MessageContent& MessageContent::read (const char* s, int n); // + rewind() or seek
>
> As well as something like
>
> int MessageContent::size(); // (or getSize(), getLength()...)
>
> Message::Message(const char* s, int n);
>
> Note that std::string is not copy-on-write in all C++ implementations.
> On Windows, forcing an application to use strings for its raw data (from
> a char* perspective) results in needless copies of the content:
Good point, I'll try and incorporate this into the next patch.
> string tmpdata(myBlobPtr, myBlobSize); // memory copy #1
> Message m(tmpdata); // m.impl->bytes gets copy #2 when constructed
> // tmpdata never used again, freed when out of scope
>
> The programmer can take evasive action:
>
> Message m; // m.impl->bytes starts out as a sacrificial empty string
> m.getBytes().assign(myBlobPtr, myBlobSize); // just one copy
>
> but
>
> Message m(myBlobPtr, myBlobSize);
>
> might create the FrameSet directly, and cut out another middleman.
>
> Since MessageContent already has abstract content (string, MAP, and
> LIST), extentending the set to include char* could allow performance
> optimizations:
Agreed.
> For incoming messages, the actual content could be copied from the
> FrameSet directly to an application buffer. (This can be a win even
> on Linux when messages are larger than the connection's
> maxFrameSize.) The string form can be lazilly created when needed.
>
> For outgoing messages, the content may be placed directly in special
> OS memory in preparation for network transfer.
>
>
> Cliff
>
>
> -----Original Message-----
> From: Gordon Sim [mailto:gsim@redhat.com]
> Sent: Friday, July 31, 2009 10:11 AM
> To: Qpid Dev; users@qpid.apache.org
> Subject: c++ messaging api: a map message example
>
> Attached is a patch for a higher level, protocol independent messaging api from c++ as last discussed some months ago. I have not had much time to spend on this since then and so there is not a huge amount of change (mainly moving to a better namespace and adjusting for the revised include directory layout).
>
> There is however now an example of a sending a map message using the amqp0-10 encoding (list messages are also supported) and the message headers are now correctly sent. Jonathan previously requested support for the stream operator and I've had a tentative stab at exploring what that might look like as well.
>
> As always keen to get any feedback. I'm hoping to have more time to spend on this for the next few weeks. I'm aiming to align the c++ api with the similar work on python (at least conceptually) and to flesh out the functionality (determining how best to expose asynchronous completion of sends and acknowledgements will be a likely next step).
>
> --Gordon.
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:dev-subscribe@qpid.apache.org
>
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org