You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Raymie Stata (JIRA)" <ji...@apache.org> on 2009/07/02 17:33:47 UTC

[jira] Commented: (AVRO-50) bidrectional text representation of AVRO data

    [ https://issues.apache.org/jira/browse/AVRO-50?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12726519#action_12726519 ] 

Raymie Stata commented on AVRO-50:
----------------------------------

Here's the conversation Doug was alluding to:

Doug mentioned that he added start-record/end-record and related methods to value-reader/writer to support reading and writing from the JSON syntax.  Neither of us really like this, because it adds a lot of noise to clients of the API, and also because it's the tail (JSON format) wagging the dog (Avro binary).

To understand the motivation for methods like start-record, consider writing the schema:
{noformat}
{type:record, name:r1, fields:[
    {name:f11, value:int}
    {name:f12, value:{type:record,name:r2,fields:[{name:f21, value:int}]}}
  ]
}
{noformat}

In the current design of value-writer, all the client needs to do is call writeInt and writeInt, and all the record stuff is handled implicitly.  This implicit handling works well for the Avro binary representation, which does not surround records or their fields with any delimiters.  But the JSON format _does_ surround records and their fields with brackets and field-labels, which raises the question: How does the JSON-implementation of value-reader/writer know how to handle this bracketing and labeling?

The solution of adding start/end-record and related methods to the value-reader/writer API forces this task of bracketing and labeling onto the client of value-writer,  which has the problems listed above.

However, Doug and I realized that the parser has all the state needed to output the bracketing and labeling.  So instead of adding a bunch of noise-methods to value-reader/writer, we can use the existing APIs, and use the parsing code to implement value reader/writers that handle the JSON format.  (The parser would be something closer to the Resolving parser than the validating one.)

> bidrectional text representation of AVRO data
> ---------------------------------------------
>
>                 Key: AVRO-50
>                 URL: https://issues.apache.org/jira/browse/AVRO-50
>             Project: Avro
>          Issue Type: New Feature
>          Components: spec
>            Reporter: Philip Zeyliger
>            Assignee: Doug Cutting
>         Attachments: AVRO-50.patch
>
>
> It would be very useful to add a text representation of AVRO data to the spec, and implement toString() and fromString() in all implementations.  Faced with binary data, it'll be a useful operation to decode it for debugging, ad-hoc manipulation, etc.
> I suspect the text format will:
>  * be JSON
>  * require the schema for full interpretation
>  * map easily onto the binary format (if the binary format has a signifier to take a specific branch of a union, the text format will have such a signifier as well)
>  * not be unique (there's more than one way to encode a given number (e.g., {{0x0 == 0}}) or string (e.g., {{"\u0061" == "a"}}, not to mention flexible whitespace)
>  * be compatible, for the binary type, with whatever is decided in AVRO-36

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.