You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Rafael Merlin (Jira)" <ji...@apache.org> on 2020/04/30 00:55:00 UTC
[jira] [Updated] (AVRO-2823) Can't find types if namespace has
reserved word.
[ https://issues.apache.org/jira/browse/AVRO-2823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rafael Merlin updated AVRO-2823:
--------------------------------
Description:
Let's say I have this schema for my Values:
{code:java}
{
"type": [{
"type": "record",
"name": "TestRecord",
"namespace": "test.public.example",
"fields": [{
"name": "name",
"type": "string"
}, {
"name": "uuid",
"type": ["string", "null"]
}
]
}, {
"type": "record",
"name": "TestValue",
"namespace": "test.public.example",
"fields": [{
"name": "message_uuid",
"type": "string"
}, {
"name": "trace_id",
"type": "string"
}, {
"name": "testone",
"type": {
"type": "record",
"name": "TestOne",
"namespace": "test.public.example",
"fields": [{
"name": "uuid",
"type": "string"
}, {
"name": "message_number",
"type": "long"
}, {
"name": "name",
"type": "string"
}, {
"name": "internalarray",
"type": {
"type": "array",
"items": {
"type": "record",
"name": "InternalRecord",
"namespace": "test.public.example",
"fields": [{
"name": "uuid",
"type": "string"
}, {
"name": "generic_field",
"type": "string"
}
]
}
}
}
]
}
}
]
}
]
}
{code}
When I run codegen the InternalRecord class will be generated as:
{code:java}
namespace test.@public.example
{
public partial class InternalRecord : ISpecificRecord
{code}
However, when deserialising to this object (using the AvroDeserializer on ConsumerBuilder from Confluent.SchemaRegistry.Serdes.Avro.
When it gets to the Array the CodeGen step will eventually get to this method:
{code:java}
internal static string getType(Schema schema, bool nullible, ref bool nullibleEnum{code}
And due to the schema's tag there being a Record it will hit this:
{code:java}
case Schema.Type.Fixed:
case Schema.Type.Record:
case Schema.Type.Error:
namedSchema = schema as NamedSchema;
if (null == namedSchema)
throw new CodeGenException("Unable to cast schema into a named schema");
return CodeGenUtil.Instance.Mangle(namedSchema.Fullname);
{code}
This will Mangle the types again in here so now the type will be "test.@public.example.InternalRecord" due to the Mangling. I reckon this may have been added due to protected names in properties or something like that. So a possible solution that I hope shouldn't affect anything else is to in the Object creator's FindType method to add this:
{code:java}
// Original Code:
if (diffAssembly)
{
type = entryAssembly.GetType(name);
}// try current assembly and mscorlib
if (type == null)
{
type = Type.GetType(name);
}
//Code to add to Original code.
if (name.Contains("@") && type == null)
{
var unmangledName = CodeGenUtil.Instance.UnMangle(name);
if (diffAssembly)
type = entryAssembly.GetType(unmangledName);
if (type == null)
type = Type.GetType(unmangledName);
}
{code}
was:
Let's say I have this schema for my Values:
{code:java}
{
"type": [{
"type": "record",
"name": "TestRecord",
"namespace": "test.public.example",
"fields": [{
"name": "name",
"type": "string"
}, {
"name": "uuid",
"type": ["string", "null"]
}
]
}, {
"type": "record",
"name": "TestValue",
"namespace": "test.public.example",
"fields": [{
"name": "message_uuid",
"type": "string"
}, {
"name": "trace_id",
"type": "string"
}, {
"name": "testone",
"type": {
"type": "record",
"name": "TestOne",
"namespace": "test.public.example",
"fields": [{
"name": "uuid",
"type": "string"
}, {
"name": "message_number",
"type": "long"
}, {
"name": "name",
"type": "string"
}, {
"name": "internalarray",
"type": {
"type": "array",
"items": {
"type": "record",
"name": "InternalRecord",
"namespace": "test.public.example",
"fields": [{
"name": "uuid",
"type": "string"
}, {
"name": "generic_field",
"type": "string"
}
]
}
}
}
]
}
}
]
}
]
}
{code}
When I run codegen the InternalRecord class will be generated as:
{code:java}
namespace test.@public.example
{
public partial class InternalRecord : ISpecificRecord
{code}
However, when deserialising to this object (using the AvroDeserializer on ConsumerBuilder from Confluent.SchemaRegistry.Serdes.Avro.
When it gets to the Array the CodeGen step will eventually get to this method:
`internal static string getType(Schema schema, bool nullible, ref bool nullibleEnum)`
This will Mangle the types again in here so now the type will be "test.@public.example.InternalRecord" due to the Mangling. I reckon this may have been added due to protected names in properties or something like that. So a possible solution that I hope shouldn't affect anything else is to in the Object creator's FindType method to add this:
{code:java}
// Original Code:
if (diffAssembly)
{
type = entryAssembly.GetType(name);
}// try current assembly and mscorlib
if (type == null)
{
type = Type.GetType(name);
}
//Code to add to Original code.
if (name.Contains("@") && type == null)
{
var unmangledName = CodeGenUtil.Instance.UnMangle(name);
if (diffAssembly)
type = entryAssembly.GetType(unmangledName);
if (type == null)
type = Type.GetType(unmangledName);
}
{code}
> Can't find types if namespace has reserved word.
> ------------------------------------------------
>
> Key: AVRO-2823
> URL: https://issues.apache.org/jira/browse/AVRO-2823
> Project: Apache Avro
> Issue Type: Bug
> Components: csharp
> Reporter: Rafael Merlin
> Priority: Major
>
> Let's say I have this schema for my Values:
> {code:java}
> {
> "type": [{
> "type": "record",
> "name": "TestRecord",
> "namespace": "test.public.example",
> "fields": [{
> "name": "name",
> "type": "string"
> }, {
> "name": "uuid",
> "type": ["string", "null"]
> }
> ]
> }, {
> "type": "record",
> "name": "TestValue",
> "namespace": "test.public.example",
> "fields": [{
> "name": "message_uuid",
> "type": "string"
> }, {
> "name": "trace_id",
> "type": "string"
> }, {
> "name": "testone",
> "type": {
> "type": "record",
> "name": "TestOne",
> "namespace": "test.public.example",
> "fields": [{
> "name": "uuid",
> "type": "string"
> }, {
> "name": "message_number",
> "type": "long"
> }, {
> "name": "name",
> "type": "string"
> }, {
> "name": "internalarray",
> "type": {
> "type": "array",
> "items": {
> "type": "record",
> "name": "InternalRecord",
> "namespace": "test.public.example",
> "fields": [{
> "name": "uuid",
> "type": "string"
> }, {
> "name": "generic_field",
> "type": "string"
> }
> ]
> }
> }
> }
> ]
> }
> }
> ]
> }
> ]
> }
> {code}
>
>
> When I run codegen the InternalRecord class will be generated as:
>
> {code:java}
> namespace test.@public.example
> {
> public partial class InternalRecord : ISpecificRecord
> {code}
>
>
> However, when deserialising to this object (using the AvroDeserializer on ConsumerBuilder from Confluent.SchemaRegistry.Serdes.Avro.
>
> When it gets to the Array the CodeGen step will eventually get to this method:
> {code:java}
> internal static string getType(Schema schema, bool nullible, ref bool nullibleEnum{code}
>
> And due to the schema's tag there being a Record it will hit this:
> {code:java}
> case Schema.Type.Fixed:
> case Schema.Type.Record:
> case Schema.Type.Error:
> namedSchema = schema as NamedSchema;
> if (null == namedSchema)
> throw new CodeGenException("Unable to cast schema into a named schema");
> return CodeGenUtil.Instance.Mangle(namedSchema.Fullname);
> {code}
> This will Mangle the types again in here so now the type will be "test.@public.example.InternalRecord" due to the Mangling. I reckon this may have been added due to protected names in properties or something like that. So a possible solution that I hope shouldn't affect anything else is to in the Object creator's FindType method to add this:
>
> {code:java}
> // Original Code:
> if (diffAssembly)
> {
> type = entryAssembly.GetType(name);
> }// try current assembly and mscorlib
> if (type == null)
> {
> type = Type.GetType(name);
> }
> //Code to add to Original code.
> if (name.Contains("@") && type == null)
> {
> var unmangledName = CodeGenUtil.Instance.UnMangle(name);
> if (diffAssembly)
> type = entryAssembly.GetType(unmangledName);
> if (type == null)
> type = Type.GetType(unmangledName);
> }
> {code}
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)