You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Scott Banachowski (JIRA)" <ji...@apache.org> on 2010/04/05 19:29:27 UTC
[jira] Created: (AVRO-508) Use page-backed buffers for C++
serialization input or output
Use page-backed buffers for C++ serialization input or output
-------------------------------------------------------------
Key: AVRO-508
URL: https://issues.apache.org/jira/browse/AVRO-508
Project: Avro
Issue Type: New Feature
Components: c++
Reporter: Scott Banachowski
This change came about when I looked at how to tie the object container and http changes to the serializer.
The current implementation of C++ allows serialization to and from C++ iostreams. However, typically I want to serialize to a buffer, which I can write to disk or network (or both). While iostreams do buffering, they're not well-suited if you just want to store to memory (the unittest examples serialize to ostringstream then extract the data as a string--not so efficient).
So I've added a buffering library, that does page-backed allocations (adds a page as buffer grows). It also supports the concept classes defined by boost::asio for Constant and Mutable buffer sequences (this allows the buffers to be used directly in asio networking calls), conversion to iovec, appending of foreign buffers, and iostream interface wrapper, so it's pretty flexible in the way it can be used.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-508) Use page-backed buffers for C++
serialization input or output
Posted by "Scott Banachowski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Scott Banachowski updated AVRO-508:
-----------------------------------
Attachment: applypatch508.sh
AVRO-508.patch
Patch, and script to apply patch
> Use page-backed buffers for C++ serialization input or output
> -------------------------------------------------------------
>
> Key: AVRO-508
> URL: https://issues.apache.org/jira/browse/AVRO-508
> Project: Avro
> Issue Type: New Feature
> Components: c++
> Reporter: Scott Banachowski
> Attachments: applypatch508.sh, AVRO-508.patch
>
>
> This change came about when I looked at how to tie the object container and http changes to the serializer.
> The current implementation of C++ allows serialization to and from C++ iostreams. However, typically I want to serialize to a buffer, which I can write to disk or network (or both). While iostreams do buffering, they're not well-suited if you just want to store to memory (the unittest examples serialize to ostringstream then extract the data as a string--not so efficient).
> So I've added a buffering library, that does page-backed allocations (adds a page as buffer grows). It also supports the concept classes defined by boost::asio for Constant and Mutable buffer sequences (this allows the buffers to be used directly in asio networking calls), conversion to iovec, appending of foreign buffers, and iostream interface wrapper, so it's pretty flexible in the way it can be used.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-508) Use page-backed buffers for C++
serialization input or output
Posted by "Scott Banachowski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Scott Banachowski updated AVRO-508:
-----------------------------------
Status: Patch Available (was: Open)
> Use page-backed buffers for C++ serialization input or output
> -------------------------------------------------------------
>
> Key: AVRO-508
> URL: https://issues.apache.org/jira/browse/AVRO-508
> Project: Avro
> Issue Type: New Feature
> Components: c++
> Reporter: Scott Banachowski
> Attachments: applypatch508.sh, AVRO-508.patch
>
>
> This change came about when I looked at how to tie the object container and http changes to the serializer.
> The current implementation of C++ allows serialization to and from C++ iostreams. However, typically I want to serialize to a buffer, which I can write to disk or network (or both). While iostreams do buffering, they're not well-suited if you just want to store to memory (the unittest examples serialize to ostringstream then extract the data as a string--not so efficient).
> So I've added a buffering library, that does page-backed allocations (adds a page as buffer grows). It also supports the concept classes defined by boost::asio for Constant and Mutable buffer sequences (this allows the buffers to be used directly in asio networking calls), conversion to iovec, appending of foreign buffers, and iostream interface wrapper, so it's pretty flexible in the way it can be used.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-508) Use page-backed buffers for C++
serialization input or output
Posted by "Scott Banachowski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Scott Banachowski updated AVRO-508:
-----------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
Committed revision 931999.
> Use page-backed buffers for C++ serialization input or output
> -------------------------------------------------------------
>
> Key: AVRO-508
> URL: https://issues.apache.org/jira/browse/AVRO-508
> Project: Avro
> Issue Type: Improvement
> Components: c++
> Reporter: Scott Banachowski
> Attachments: applypatch508.sh, AVRO-508.patch
>
>
> This change came about when I looked at how to tie the object container and http changes to the serializer.
> The current implementation of C++ allows serialization to and from C++ iostreams. However, typically I want to serialize to a buffer, which I can write to disk or network (or both). While iostreams do buffering, they're not well-suited if you just want to store to memory (the unittest examples serialize to ostringstream then extract the data as a string--not so efficient).
> So I've added a buffering library, that does page-backed allocations (adds a page as buffer grows). It also supports the concept classes defined by boost::asio for Constant and Mutable buffer sequences (this allows the buffers to be used directly in asio networking calls), conversion to iovec, appending of foreign buffers, and iostream interface wrapper, so it's pretty flexible in the way it can be used.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-508) Use page-backed buffers for C++
serialization input or output
Posted by "Scott Banachowski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Scott Banachowski updated AVRO-508:
-----------------------------------
Issue Type: Improvement (was: New Feature)
> Use page-backed buffers for C++ serialization input or output
> -------------------------------------------------------------
>
> Key: AVRO-508
> URL: https://issues.apache.org/jira/browse/AVRO-508
> Project: Avro
> Issue Type: Improvement
> Components: c++
> Reporter: Scott Banachowski
> Attachments: applypatch508.sh, AVRO-508.patch
>
>
> This change came about when I looked at how to tie the object container and http changes to the serializer.
> The current implementation of C++ allows serialization to and from C++ iostreams. However, typically I want to serialize to a buffer, which I can write to disk or network (or both). While iostreams do buffering, they're not well-suited if you just want to store to memory (the unittest examples serialize to ostringstream then extract the data as a string--not so efficient).
> So I've added a buffering library, that does page-backed allocations (adds a page as buffer grows). It also supports the concept classes defined by boost::asio for Constant and Mutable buffer sequences (this allows the buffers to be used directly in asio networking calls), conversion to iovec, appending of foreign buffers, and iostream interface wrapper, so it's pretty flexible in the way it can be used.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.