You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by Jeff Hammerbacher <ha...@cloudera.com> on 2009/11/16 11:05:17 UTC
Why do named schemas have namespaces?
Hey,
In the Avro spec, namespaces are defined for protocols, but not for named
schemas. Yet in both the Java and Python codebases, it appears that schemas
can have a namespace. Should the code be removed, or the spec be updated?
Thanks,
Jeff
Re: Why do named schemas have namespaces?
Posted by Doug Cutting <cu...@apache.org>.
I think AVRO-153 is fine for this. Thanks!
Doug
Jeff Hammerbacher wrote:
> Hey Doug,
>
> Great, would you like to do the work on the spec under the aegis of 153 or
> shall I open another ticket for adding namespaces to the schema portion of
> the spec?
>
> Thanks,
> Jeff
>
> On Mon, Nov 16, 2009 at 11:36 AM, Doug Cutting <cu...@apache.org> wrote:
>
>> Jeff Hammerbacher wrote:
>>
>>> In the Avro spec, namespaces are defined for protocols, but not for named
>>> schemas. Yet in both the Java and Python codebases, it appears that
>>> schemas
>>> can have a namespace. Should the code be removed, or the spec be updated?
>>>
>> The spec should be updated. Schemas can specify a namespace. If they're
>> nested in another schema or protocol then the namespace defaults to the
>> namespace of the containing schema or protocol.
>>
>> Another thing that should be updated in the spec is that a name can be
>> namespace-qualified. This is useful to refer to types in a different
>> namespace, e.g., a field like:
>> {"name": "foo", "type": "org.apache.hadoop.util.MD5"}
>>
>> This is related to https://issues.apache.org/jira/browse/AVRO-153.
>>
>> The spec currently prohibits dots in identifiers. We should clarify that
>> dots are permitted in namespaces, and that, if present in a name, the last
>> dot separates the name from the namespace.
>>
>> Doug
>>
>>
>>
>
Re: Why do named schemas have namespaces?
Posted by Jeff Hammerbacher <ha...@cloudera.com>.
Hey Doug,
Great, would you like to do the work on the spec under the aegis of 153 or
shall I open another ticket for adding namespaces to the schema portion of
the spec?
Thanks,
Jeff
On Mon, Nov 16, 2009 at 11:36 AM, Doug Cutting <cu...@apache.org> wrote:
> Jeff Hammerbacher wrote:
>
>> In the Avro spec, namespaces are defined for protocols, but not for named
>> schemas. Yet in both the Java and Python codebases, it appears that
>> schemas
>> can have a namespace. Should the code be removed, or the spec be updated?
>>
>
> The spec should be updated. Schemas can specify a namespace. If they're
> nested in another schema or protocol then the namespace defaults to the
> namespace of the containing schema or protocol.
>
> Another thing that should be updated in the spec is that a name can be
> namespace-qualified. This is useful to refer to types in a different
> namespace, e.g., a field like:
> {"name": "foo", "type": "org.apache.hadoop.util.MD5"}
>
> This is related to https://issues.apache.org/jira/browse/AVRO-153.
>
> The spec currently prohibits dots in identifiers. We should clarify that
> dots are permitted in namespaces, and that, if present in a name, the last
> dot separates the name from the namespace.
>
> Doug
>
>
>
Re: Why do named schemas have namespaces?
Posted by Philip Zeyliger <ph...@cloudera.com>.
On Thu, Nov 19, 2009 at 4:40 PM, Doug Cutting <cu...@apache.org> wrote:
> Philip Zeyliger wrote:
>>
>> I think that for keywords, that language's implementation should
>> generate something sensible if there's code generation installed. So,
>> the Java specific compiler could have a list of Java keywords
>> embedded. If it sees "long", it just replaces it with "long_" in the
>> generated code. The only reserved words in Avro are the names of
>> built-in avro types...
>>
>> Would that be sensible?
>
> +1 I agree that we should only mangle names when they conflict.
AVRO-221.
Re: Why do named schemas have namespaces?
Posted by Doug Cutting <cu...@apache.org>.
Philip Zeyliger wrote:
> I think that for keywords, that language's implementation should
> generate something sensible if there's code generation installed. So,
> the Java specific compiler could have a list of Java keywords
> embedded. If it sees "long", it just replaces it with "long_" in the
> generated code. The only reserved words in Avro are the names of
> built-in avro types...
>
> Would that be sensible?
+1 I agree that we should only mangle names when they conflict.
Doug
Re: Why do named schemas have namespaces?
Posted by Philip Zeyliger <ph...@cloudera.com>.
I think that for keywords, that language's implementation should
generate something sensible if there's code generation installed. So,
the Java specific compiler could have a list of Java keywords
embedded. If it sees "long", it just replaces it with "long_" in the
generated code. The only reserved words in Avro are the names of
built-in avro types...
Would that be sensible?
-- Philip
On Mon, Nov 16, 2009 at 12:13 PM, Patrick Hunt <ph...@apache.org> wrote:
> Same problem for enums btw. If you use symbols:["long", "null"] etc... the
> java will fail to compile (fine for python though on both these issues).
>
> Patrick
>
> Doug Cutting wrote:
>>
>> Patrick Hunt wrote:
>>>
>>> Re 153 - last night I noticed that if you spec a record/enum/etc... with
>>> a name that starts with lower case it fails to compile in Java:
>>>
>>> * start with [A-Za-z_]
>>>
>>> Add this to 153 or a new JIRA?
>>
>> Do we want to enforce naming conventions or just leave them conventions?
>> If they're just conventions, this is a bug in java, if they're to be
>> enforced, this could be part of 153's updates to the spec.
>>
>> Also, the Java compiler currently fails when names are Java reserved
>> words. I guess we could prefix every name somehow to prevent this, but
>> that's soooo ugly. Should we perhaps prefix just reserved words? This is
>> certainly a separate issue.
>>
>> Doug
>
Re: Why do named schemas have namespaces?
Posted by Patrick Hunt <ph...@apache.org>.
Same problem for enums btw. If you use symbols:["long", "null"] etc...
the java will fail to compile (fine for python though on both these issues).
Patrick
Doug Cutting wrote:
> Patrick Hunt wrote:
>> Re 153 - last night I noticed that if you spec a record/enum/etc...
>> with a name that starts with lower case it fails to compile in Java:
>>
>> * start with [A-Za-z_]
>>
>> Add this to 153 or a new JIRA?
>
> Do we want to enforce naming conventions or just leave them conventions?
> If they're just conventions, this is a bug in java, if they're to be
> enforced, this could be part of 153's updates to the spec.
>
> Also, the Java compiler currently fails when names are Java reserved
> words. I guess we could prefix every name somehow to prevent this, but
> that's soooo ugly. Should we perhaps prefix just reserved words? This
> is certainly a separate issue.
>
> Doug
Re: Why do named schemas have namespaces?
Posted by Doug Cutting <cu...@apache.org>.
Patrick Hunt wrote:
> Re 153 - last night I noticed that if you spec a record/enum/etc... with
> a name that starts with lower case it fails to compile in Java:
>
> * start with [A-Za-z_]
>
> Add this to 153 or a new JIRA?
Do we want to enforce naming conventions or just leave them conventions?
If they're just conventions, this is a bug in java, if they're to be
enforced, this could be part of 153's updates to the spec.
Also, the Java compiler currently fails when names are Java reserved
words. I guess we could prefix every name somehow to prevent this, but
that's soooo ugly. Should we perhaps prefix just reserved words? This
is certainly a separate issue.
Doug
Re: Why do named schemas have namespaces?
Posted by Patrick Hunt <ph...@apache.org>.
Re 153 - last night I noticed that if you spec a record/enum/etc... with
a name that starts with lower case it fails to compile in Java:
* start with [A-Za-z_]
Add this to 153 or a new JIRA?
Patrick
Doug Cutting wrote:
> Jeff Hammerbacher wrote:
>> In the Avro spec, namespaces are defined for protocols, but not for named
>> schemas. Yet in both the Java and Python codebases, it appears that
>> schemas
>> can have a namespace. Should the code be removed, or the spec be updated?
>
> The spec should be updated. Schemas can specify a namespace. If
> they're nested in another schema or protocol then the namespace defaults
> to the namespace of the containing schema or protocol.
>
> Another thing that should be updated in the spec is that a name can be
> namespace-qualified. This is useful to refer to types in a different
> namespace, e.g., a field like:
> {"name": "foo", "type": "org.apache.hadoop.util.MD5"}
>
> This is related to https://issues.apache.org/jira/browse/AVRO-153.
>
> The spec currently prohibits dots in identifiers. We should clarify
> that dots are permitted in namespaces, and that, if present in a name,
> the last dot separates the name from the namespace.
>
> Doug
>
>
Re: Why do named schemas have namespaces?
Posted by Doug Cutting <cu...@apache.org>.
Jeff Hammerbacher wrote:
> In the Avro spec, namespaces are defined for protocols, but not for named
> schemas. Yet in both the Java and Python codebases, it appears that schemas
> can have a namespace. Should the code be removed, or the spec be updated?
The spec should be updated. Schemas can specify a namespace. If
they're nested in another schema or protocol then the namespace defaults
to the namespace of the containing schema or protocol.
Another thing that should be updated in the spec is that a name can be
namespace-qualified. This is useful to refer to types in a different
namespace, e.g., a field like:
{"name": "foo", "type": "org.apache.hadoop.util.MD5"}
This is related to https://issues.apache.org/jira/browse/AVRO-153.
The spec currently prohibits dots in identifiers. We should clarify
that dots are permitted in namespaces, and that, if present in a name,
the last dot separates the name from the namespace.
Doug