You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@avro.apache.org by Charles Cazabon <ch...@pyropus.ca> on 2020/05/11 18:06:08 UTC
enum as record member field? (Python 3)
Hi,
I'm using avro_python3-1.9.2.1 with Python 3. I have a custom record schema
that I would like to include an enum field in, but I get a
SchemaParseException when I do so.
For a minimal test case, using the example from the Python getting started
guide as a complete schema is successful:
schema_txt = """
{ "type": "enum",
"name": "Suit",
"symbols" : ["SPADES", "HEARTS", "DIAMONDS", "CLUBS"]
}
"""
test_schema = avro.schema.parse(schema_txt) # success
However, if I create a record schema with such an enum as a member field (here
the only field, but behaviour is the same with other fields present):
schema_txt = """
{
"name": "Test",
"type": "record",
"fields": [
{ "type": "enum",
"name": "Suit",
"symbols" : ["SPADES", "HEARTS", "DIAMONDS", "CLUBS"]
}
]
}
"""
... then avro.schema.parse() fails with:
Traceback (most recent call last):
File "./simple-schema.py", line 43, in <module>
test_schema = avro.schema.parse(schema_txt)
File ".../.venv/lib/python3.7/site-packages/avro/schema.py", line 1244, in parse
return SchemaFromJSONData(json_data, names)
File ".../.venv/lib/python3.7/site-packages/avro/schema.py", line 1215, in SchemaFromJSONData
return parser(json_data, names=names)
File ".../.venv/lib/python3.7/site-packages/avro/schema.py", line 1143, in _SchemaFromJSONObject
other_props=other_props,
File ".../.venv/lib/python3.7/site-packages/avro/schema.py", line 1022, in __init__
fields = make_fields(names=nested_names)
File ".../.venv/lib/python3.7/site-packages/avro/schema.py", line 1134, in MakeFields
return tuple(RecordSchema._MakeFieldList(field_desc_list, names))
File ".../.venv/lib/python3.7/site-packages/avro/schema.py", line 947, in _MakeFieldList
yield RecordSchema._MakeField(index, field_desc, names)
File ".../.venv/lib/python3.7/site-packages/avro/schema.py", line 918, in _MakeField
names=names,
File ".../.venv/lib/python3.7/site-packages/avro/schema.py", line 1215, in SchemaFromJSONData
return parser(json_data, names=names)
File ".../.venv/lib/python3.7/site-packages/avro/schema.py", line 1096, in _SchemaFromJSONString
% (json_string, sorted(names.names)))
avro.schema.SchemaParseException: Unknown named schema 'enum', known names: ['Test'].
What am I doing wrong?
I've looked through the last six months of the avro users' list archive. The
search function for the list appears to be offline so I haven't dug further
than that. Haven't found anything relevant with web searches either.
Any help appreciated.
Thanks,
Charles
--
------------------------------------------------------------------
Charles Cazabon <ch...@pyropus.ca>
Software, consulting, and services available at http://pyropus.ca/
------------------------------------------------------------------
Re: enum as record member field? (Python 3)
Posted by Charles Cazabon <ch...@pyropus.ca>.
Thank you to both Hisham and Michael for the pointers - this does indeed
explain what I was doing wrong.
For the record, the correct way to specify the enum in the record's fields
member is:
{
"name": "Test",
"type": "record",
"fields": [
{
"name": "Suit",
"type": {
"type": "enum",
"name": "Suit",
"symbols" : ["SPADES", "HEARTS", "DIAMONDS", "CLUBS"]
}
}
]
}
Charles
--
------------------------------------------------------------------
Charles Cazabon <ch...@pyropus.ca>
Software, consulting, and services available at http://pyropus.ca/
------------------------------------------------------------------
Re: enum as record member field? (Python 3)
Posted by "Michael A. Smith" <mi...@smith-li.com>.
Right. To explain a little differently, an enum has a name, and a field has
a name. You have to define them distinctly.
On Mon, May 11, 2020 at 17:07 Hisham El-Sheshtawy <
hisham.elsheshtawy@sparkmeter.io> wrote:
> I think you may need to adjust it as follows:
>
> schema_txt = """
> {
> "name": "Test",
> "type": "record",
> "fields": [
> { "name": "Suit", "type": {"enum",
> "name": "Suit",
> "symbols" : ["SPADES", "HEARTS", "DIAMONDS", "CLUBS"]
> }
> }
> ]
> }
> """
>
> On Mon, May 11, 2020 at 2:06 PM Charles Cazabon <
> charlesc-avro.apache.org@pyropus.ca> wrote:
>
>> Hi,
>>
>> I'm using avro_python3-1.9.2.1 with Python 3. I have a custom record
>> schema
>> that I would like to include an enum field in, but I get a
>> SchemaParseException when I do so.
>>
>> For a minimal test case, using the example from the Python getting started
>> guide as a complete schema is successful:
>>
>> schema_txt = """
>> { "type": "enum",
>> "name": "Suit",
>> "symbols" : ["SPADES", "HEARTS", "DIAMONDS", "CLUBS"]
>> }
>> """
>>
>> test_schema = avro.schema.parse(schema_txt) # success
>>
>> However, if I create a record schema with such an enum as a member field
>> (here
>> the only field, but behaviour is the same with other fields present):
>>
>> schema_txt = """
>> {
>> "name": "Test",
>> "type": "record",
>> "fields": [
>> { "type": "enum",
>> "name": "Suit",
>> "symbols" : ["SPADES", "HEARTS", "DIAMONDS", "CLUBS"]
>> }
>> ]
>> }
>> """
>>
>> ... then avro.schema.parse() fails with:
>>
>> Traceback (most recent call last):
>> File "./simple-schema.py", line 43, in <module>
>> test_schema = avro.schema.parse(schema_txt)
>> File ".../.venv/lib/python3.7/site-packages/avro/schema.py", line 1244,
>> in parse
>> return SchemaFromJSONData(json_data, names)
>> File ".../.venv/lib/python3.7/site-packages/avro/schema.py", line 1215,
>> in SchemaFromJSONData
>> return parser(json_data, names=names)
>> File ".../.venv/lib/python3.7/site-packages/avro/schema.py", line 1143,
>> in _SchemaFromJSONObject
>> other_props=other_props,
>> File ".../.venv/lib/python3.7/site-packages/avro/schema.py", line 1022,
>> in __init__
>> fields = make_fields(names=nested_names)
>> File ".../.venv/lib/python3.7/site-packages/avro/schema.py", line 1134,
>> in MakeFields
>> return tuple(RecordSchema._MakeFieldList(field_desc_list, names))
>> File ".../.venv/lib/python3.7/site-packages/avro/schema.py", line 947,
>> in _MakeFieldList
>> yield RecordSchema._MakeField(index, field_desc, names)
>> File ".../.venv/lib/python3.7/site-packages/avro/schema.py", line 918,
>> in _MakeField
>> names=names,
>> File ".../.venv/lib/python3.7/site-packages/avro/schema.py", line 1215,
>> in SchemaFromJSONData
>> return parser(json_data, names=names)
>> File ".../.venv/lib/python3.7/site-packages/avro/schema.py", line 1096,
>> in _SchemaFromJSONString
>> % (json_string, sorted(names.names)))
>> avro.schema.SchemaParseException: Unknown named schema 'enum', known
>> names: ['Test'].
>>
>>
>> What am I doing wrong?
>>
>> I've looked through the last six months of the avro users' list archive.
>> The
>> search function for the list appears to be offline so I haven't dug
>> further
>> than that. Haven't found anything relevant with web searches either.
>>
>> Any help appreciated.
>>
>> Thanks,
>>
>> Charles
>> --
>> ------------------------------------------------------------------
>> Charles Cazabon <ch...@pyropus.ca>
>> Software, consulting, and services available at http://pyropus.ca/
>> ------------------------------------------------------------------
>>
>
>
> --
>
> *Hisham ElSheshtawy*
> Software Engineer, SparkMeter
> Washington D.C., USA
> *www.sparkmeter.io
> <http://www.sparkmeter.io/?utm_source=email_signature&utm_medium=referral> *
>
> *Have you heard about our New & Upcoming Products
> <https://go.sparkmeter.io/new-product-announcements?utm_source=email_signature&utm_medium=referral>?*
>
>
Re: enum as record member field? (Python 3)
Posted by Hisham El-Sheshtawy <hi...@sparkmeter.io>.
I think you may need to adjust it as follows:
schema_txt = """
{
"name": "Test",
"type": "record",
"fields": [
{ "name": "Suit", "type": {"enum",
"name": "Suit",
"symbols" : ["SPADES", "HEARTS", "DIAMONDS", "CLUBS"]
}
}
]
}
"""
On Mon, May 11, 2020 at 2:06 PM Charles Cazabon <
charlesc-avro.apache.org@pyropus.ca> wrote:
> Hi,
>
> I'm using avro_python3-1.9.2.1 with Python 3. I have a custom record
> schema
> that I would like to include an enum field in, but I get a
> SchemaParseException when I do so.
>
> For a minimal test case, using the example from the Python getting started
> guide as a complete schema is successful:
>
> schema_txt = """
> { "type": "enum",
> "name": "Suit",
> "symbols" : ["SPADES", "HEARTS", "DIAMONDS", "CLUBS"]
> }
> """
>
> test_schema = avro.schema.parse(schema_txt) # success
>
> However, if I create a record schema with such an enum as a member field
> (here
> the only field, but behaviour is the same with other fields present):
>
> schema_txt = """
> {
> "name": "Test",
> "type": "record",
> "fields": [
> { "type": "enum",
> "name": "Suit",
> "symbols" : ["SPADES", "HEARTS", "DIAMONDS", "CLUBS"]
> }
> ]
> }
> """
>
> ... then avro.schema.parse() fails with:
>
> Traceback (most recent call last):
> File "./simple-schema.py", line 43, in <module>
> test_schema = avro.schema.parse(schema_txt)
> File ".../.venv/lib/python3.7/site-packages/avro/schema.py", line 1244,
> in parse
> return SchemaFromJSONData(json_data, names)
> File ".../.venv/lib/python3.7/site-packages/avro/schema.py", line 1215,
> in SchemaFromJSONData
> return parser(json_data, names=names)
> File ".../.venv/lib/python3.7/site-packages/avro/schema.py", line 1143,
> in _SchemaFromJSONObject
> other_props=other_props,
> File ".../.venv/lib/python3.7/site-packages/avro/schema.py", line 1022,
> in __init__
> fields = make_fields(names=nested_names)
> File ".../.venv/lib/python3.7/site-packages/avro/schema.py", line 1134,
> in MakeFields
> return tuple(RecordSchema._MakeFieldList(field_desc_list, names))
> File ".../.venv/lib/python3.7/site-packages/avro/schema.py", line 947,
> in _MakeFieldList
> yield RecordSchema._MakeField(index, field_desc, names)
> File ".../.venv/lib/python3.7/site-packages/avro/schema.py", line 918,
> in _MakeField
> names=names,
> File ".../.venv/lib/python3.7/site-packages/avro/schema.py", line 1215,
> in SchemaFromJSONData
> return parser(json_data, names=names)
> File ".../.venv/lib/python3.7/site-packages/avro/schema.py", line 1096,
> in _SchemaFromJSONString
> % (json_string, sorted(names.names)))
> avro.schema.SchemaParseException: Unknown named schema 'enum', known
> names: ['Test'].
>
>
> What am I doing wrong?
>
> I've looked through the last six months of the avro users' list archive.
> The
> search function for the list appears to be offline so I haven't dug further
> than that. Haven't found anything relevant with web searches either.
>
> Any help appreciated.
>
> Thanks,
>
> Charles
> --
> ------------------------------------------------------------------
> Charles Cazabon <ch...@pyropus.ca>
> Software, consulting, and services available at http://pyropus.ca/
> ------------------------------------------------------------------
>
--
*Hisham ElSheshtawy*
Software Engineer, SparkMeter
Washington D.C., USA
*www.sparkmeter.io
<http://www.sparkmeter.io/?utm_source=email_signature&utm_medium=referral> *
*Have you heard about our New & Upcoming Products
<https://go.sparkmeter.io/new-product-announcements?utm_source=email_signature&utm_medium=referral>?*