[jira] Issue Comment Edited: (AVRO-25) Blocking for value output (with API change)

["Thiruvalluvan M. G. (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AVRO-25?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12716211#action_12716211 ] 

Thiruvalluvan M. G. edited comment on AVRO-25 at 6/4/09 2:38 AM:
-----------------------------------------------------------------

Taken care of all four issues:

- No more CRLF line endings
- BasicValueReader(Writer) and now ValueReader(Writer). No more abstract base classes
- Moved ByteBufferInput(Output)Streams to avro.ipc package. They are now public (from being package-protected) so that they are accessible from avro.io package.
- Reverted the changes w.r.t Strings and ByteBuffers in GenericDatumReader/Writer. 

Again there is no performance impact due to these changes.

      was (Author: thiru_mg):
    Taken care of all four issues:

- No more CRLF line endings
- BasicValueReader(Writer) and now ValueReader(Writer). No more abstract base classes
- Moved ByteBufferInput(Output)Streams to avro.ipc package. They are now public (from being package-protected) so that they are accessible from avro.io package.
- Reverted the changes w.r.t Strings and ByteBuffers in GenericDatumReader/Writer. 
  
> Blocking for value output (with API change)
> -------------------------------------------
>
>                 Key: AVRO-25
>                 URL: https://issues.apache.org/jira/browse/AVRO-25
>             Project: Avro
>          Issue Type: Improvement
>          Components: java
>            Reporter: Raymie Stata
>            Assignee: Thiruvalluvan M. G.
>         Attachments: AVRO-25.patch, AVRO-25.patch, AVRO-25.patch, AVRO-25.patch, AVRO-25.patch
>
>
> The Avro specification has provisions for decomposing very large arrays and maps into "blocks."  These provisions allow for streaming implementations that would allow one to, for example, write the contents of a file out as an Avro array w/out knowing in advance how many records are in the file.
> The current Java implementation of Avro does support this provision.  My colleague Thiru will be attaching a patch which implements blocking.  It turns out that the buffering required to do blocking is non-trivial, so it seem beneficial to include a standard implementation of blocking as part of the reference Avro implementation.
> This is an early version of the code.  We are still working on testing and performance tuning.  But we wanted early feedback.
> This patch also includes a new set of classes called ValueInput and ValueOutput, which are meant to replace ValueReader and ValueWriter.  These classes have largely the same API as ValueReader/Writer, but they include a few more methods to "bracket" items that appear inside of arrays and maps.  Shortly, we'll be posting a separate patch which implements further subclasses of ValueInput/Output that do "validation" of input and output against a schema (and also do automatic schema resolution for readers).
> We're implementing these classes separate from ValueInput/Output to allow you to kick our tires w/out causing too much disruption to your source trees.  Let's validate the basic idea behind these patches first, and then determine the details of integrating them into the rest of Avro.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.