Questions about spec

[Scott Banachowski <sb...@yahoo-inc.com>]

Hi,

Recently I was rereading the spec, while making the parser used by C++ more
compliant.

But there's some points that seem vague (at least to me), and I have some
idea about what they mean, but I want to verify my assumptions are true
before fixing the parser to conform:

1)  It says that an object is {"type":"typename" ...attributes...}

Does this means that type must always appear first, before other attributes?
(I assume yes)  And that after type, the remaining attributes can appear in
any order? (again, I assume yes).

2)  "Attributes not defined in this document are permitted as metadata"

It talks about metadata attributes, but doesn't say explicitly what format
is allowed.  I assume additional attributes are all of the form:

"attribute_name" : attribute_value

Can we assume attribute_name conforms to the same schema as
record/field/enum names, or is any unicode string allowed as per json?

And can attribute_value can be ANY json value?  Or only string?

3) Are field declarations order-dependent (i.e. must "name" always come
before "type", and before "default")?  Can objects used as field
declarations also include other metadata attributes?

4) The table for field defaults does not include an entry for "fixed" type.
Is that intentional?

5) And unrelated to spec parsing, "Blocking has not yet been ..."  Any news
on that front?

thanks,
Scott

Re: Questions about spec

[Doug Cutting <cu...@apache.org>]

Scott Banachowski wrote:
> The spec has an italic comment that says blocking is still a draft and
> subject to change.  I was basically asking if this clause of the spec can be
> removed now? 

Ah, yes, it probably can.  I think that was written before we had an 
implementation.  (I don't like to add things to the spec until they're 
proven to work in an implementation.) Now that we have multiple 
implementations we can remove that comment.  Do you want to file a Jira 
issue for this?  Thanks!

Doug

Re: Questions about spec

[Scott Banachowski <sb...@yahoo-inc.com>]

On 8/18/09 8:49 AM, "Doug Cutting" said:
> 
>> 5) And unrelated to spec parsing, "Blocking has not yet been ..."  Any news
>> on that front?
> 
> Not sure what you're asking here.  Blocking is implemented in Java.
> There is a patch to add blocking to Python that I need to commit.

Thanks for the reply.
The spec has an italic comment that says blocking is still a draft and
subject to change.  I was basically asking if this clause of the spec can be
removed now? 

thanks,
Scott

 

Re: Questions about spec

[Doug Cutting <cu...@apache.org>]

Scott Banachowski wrote:
> 1)  It says that an object is {"type":"typename" ...attributes...}
> 
> Does this means that type must always appear first, before other attributes?
> (I assume yes)  And that after type, the remaining attributes can appear in
> any order? (again, I assume yes).

The type need not occur first.  JSON does not guarantee ordering of 
fields within objects, so we cannot rely on it.

> 2)  "Attributes not defined in this document are permitted as metadata"
> 
> It talks about metadata attributes, but doesn't say explicitly what format
> is allowed.  I assume additional attributes are all of the form:
> 
> "attribute_name" : attribute_value
> 
> Can we assume attribute_name conforms to the same schema as
> record/field/enum names, or is any unicode string allowed as per json?
> 
> And can attribute_value can be ANY json value?  Or only string?

Good question.  My instinct is to permit any values.  Can you think of a 
reason not to?  Implementations should simply ignore any attributes not 
named in the spec.

> 3) Are field declarations order-dependent (i.e. must "name" always come
> before "type", and before "default")?  Can objects used as field
> declarations also include other metadata attributes?

Again, JSON does not permit us to depend on order within objects, only 
within arrays.  So attributes within a field declaration are unordered. 
  Fields within a record are ordered.  And, yes, it was again my intent 
to permit arbitrary other attributes here too, to permit extensions.

> 4) The table for field defaults does not include an entry for "fixed" type.
> Is that intentional?

This is fixed in trunk.

https://issues.apache.org/jira/browse/AVRO-79

> 5) And unrelated to spec parsing, "Blocking has not yet been ..."  Any news
> on that front?

Not sure what you're asking here.  Blocking is implemented in Java. 
There is a patch to add blocking to Python that I need to commit.

https://issues.apache.org/jira/browse/AVRO-61

Doug