You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@avro.apache.org by "Douglas Creager (Created) (JIRA)" <ji...@apache.org> on 2011/10/10 15:51:29 UTC

[jira] [Created] (AVRO-921) Zero-copy implementation of wrapped buffer interface

Zero-copy implementation of wrapped buffer interface
----------------------------------------------------

                 Key: AVRO-921
                 URL: https://issues.apache.org/jira/browse/AVRO-921
             Project: Avro
          Issue Type: New Feature
          Components: c
    Affects Versions: 1.6.0
            Reporter: Douglas Creager
            Assignee: Douglas Creager


The {{avro_wrapped_buffer_t}} is used by the bytes and string types to store the content of the Avro value.  The wrapped buffer interface is capable of being zero-copy, where the underlying content buffer is shared across multiple Avro values.  But our default implementation would create a new physical copy of the data every time the {{copy}} method is called.  It would be good to have a default zero-copy implementation.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (AVRO-921) Zero-copy implementation of wrapped buffer interface

Posted by "Douglas Creager (Resolved) (JIRA)" <ji...@apache.org>.

     [ https://issues.apache.org/jira/browse/AVRO-921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Douglas Creager resolved AVRO-921.
----------------------------------

       Resolution: Fixed
    Fix Version/s: 1.6.0

Checked into SVN trunk
                
> Zero-copy implementation of wrapped buffer interface
> ----------------------------------------------------
>
>                 Key: AVRO-921
>                 URL: https://issues.apache.org/jira/browse/AVRO-921
>             Project: Avro
>          Issue Type: New Feature
>          Components: c
>    Affects Versions: 1.6.0
>            Reporter: Douglas Creager
>            Assignee: Douglas Creager
>             Fix For: 1.6.0
>
>         Attachments: 0001-Default-wrapped-buffer-implementation-is-zero-copy.patch
>
>
> The {{avro_wrapped_buffer_t}} is used by the bytes and string types to store the content of the Avro value.  The wrapped buffer interface is capable of being zero-copy, where the underlying content buffer is shared across multiple Avro values.  But our default implementation would create a new physical copy of the data every time the {{copy}} method is called.  It would be good to have a default zero-copy implementation.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (AVRO-921) Zero-copy implementation of wrapped buffer interface

Posted by "Douglas Creager (Updated) (JIRA)" <ji...@apache.org>.

     [ https://issues.apache.org/jira/browse/AVRO-921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Douglas Creager updated AVRO-921:
---------------------------------

    Attachment: 0001-Default-wrapped-buffer-implementation-is-zero-copy.patch

This patch adds a zero-copy wrapped buffer.

The {{avro_wrapped_buffer_new_copy}} function creates a copy of an external buffer, but that new copy is reference-counted.  Any additional copies will just increment the reference count, and will share the underlying content buffer.

The {{avro_wrapped_buffer_new}} function wraps an existing buffer, but since we don't have any real control over that buffer, we have to make a real copy of it if its {{copy}} method is called.  This first copy, however, becomes properly reference-counted, just like a wrapped buffer created via {{avro_wrapped_buffer_new_copy}}.  Therefore, any additional copies (after the first) will be reference-counted and shared.
                
> Zero-copy implementation of wrapped buffer interface
> ----------------------------------------------------
>
>                 Key: AVRO-921
>                 URL: https://issues.apache.org/jira/browse/AVRO-921
>             Project: Avro
>          Issue Type: New Feature
>          Components: c
>    Affects Versions: 1.6.0
>            Reporter: Douglas Creager
>            Assignee: Douglas Creager
>         Attachments: 0001-Default-wrapped-buffer-implementation-is-zero-copy.patch
>
>
> The {{avro_wrapped_buffer_t}} is used by the bytes and string types to store the content of the Avro value.  The wrapped buffer interface is capable of being zero-copy, where the underlying content buffer is shared across multiple Avro values.  But our default implementation would create a new physical copy of the data every time the {{copy}} method is called.  It would be good to have a default zero-copy implementation.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira