You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@avro.apache.org by Patrick Farry <pa...@gmail.com> on 2019/08/05 23:10:13 UTC
Schema parses in C# Avro lib but not in Kafka Schema registry (assume
that it is the java lib)
The schema was generated from a protocol using the C# 1.9 code. When we try to load it into the Kafka schema registry we get an error that the schema is invalid.
The only feedback we get is “Length is not valid”. If we remove Length then the error says that “Address is not valid” - so presumably there is something wrong referencing previously defined named types.
Is there anything obviously wrong?
{
"type": "record",
"name": "PackageCreateInformation",
"namespace": "com.pb.fdr.delivery",
"fields": [
{
"name": "sendSms",
"type": [
"null",
"boolean"
]
},
{
"name": "trackingId",
"type": "string"
},
{
"name": "packageId",
"type": "string"
},
{
"name": "clientFacilityId",
"type": "string"
},
{
"name": "sortCode",
"type": "string"
},
{
"name": "ngsFacilityId",
"type": "string"
},
{
"name": "merchantId",
"type": "string"
},
{
"name": "ClassOfService",
"type": {
"type": "enum",
"name": "ClassesOfService",
"namespace": "com.pb.fdr.delivery",
"symbols": [
"Ground",
"Express",
"FirstClass",
"Priority",
"BPM",
"PBD3D"
]
}
},
{
"name": "carrier",
"type": {
"type": "enum",
"name": "Carriers",
"namespace": "com.pb.fdr.delivery",
"symbols": [
"USPS",
"DHL",
"UPS",
"FedEx"
]
}
},
{
"name": "carrierClassOfService",
"type": {
"type": "enum",
"name": "CarrierClassesOfService",
"namespace": "com.pb.fdr.delivery",
"symbols": [
"ParcelSelect",
"ParcelSelectLightweight",
"FirstClass",
"Priority",
"BPM"
]
}
},
{
"name": "weight",
"type": {
"type": "record",
"name": "Weight",
"namespace": "com.pb.fdr.delivery",
"fields": [
{
"name": "unitOfMeasure",
"type": {
"type": "enum",
"name": "WeightUnitOfMeasure",
"namespace": "com.pb.fdr.delivery",
"symbols": [
"Pounds",
"Ounces",
"Kilograms",
"Grams"
]
}
},
{
"name": "measurmentValue",
"type": "double"
}
]
}
},
{
"name": "dimensions",
"type": {
"type": "record",
"name": "Dimensions",
"namespace": "com.pb.fdr.delivery",
"fields": [
{
"name": "length",
"type": {
"type": "record",
"name": "Length",
"namespace": "com.pb.fdr.delivery",
"fields": [
{
"name": "unitOfMeasure",
"type": {
"type": "enum",
"name": "LengthUnitOfMeasure",
"namespace": "com.pb.fdr.delivery",
"symbols": [
"Inches",
"Meters",
"Centimeters"
]
}
},
{
"name": "measurmentValue",
"type": "double"
}
]
}
},
{
"name": "width",
"type": "Length"
},
{
"name": "height",
"type": "Length"
},
{
"name": "girth",
"type": "Length"
},
{
"name": "isRectangular",
"type": "boolean"
}
]
}
},
{
"name": "shipToAddress",
"type": {
"type": "record",
"name": "Address",
"namespace": "com.pb.fdr.delivery",
"fields": [
{
"name": "name",
"type": "string"
},
{
"name": "attention",
"type": [
"null",
"string"
]
},
{
"name": "address1",
"type": "string"
},
{
"name": "address2",
"type": [
"null",
"string"
]
},
{
"name": "city",
"type": [
"null",
"string"
]
},
{
"name": "stateOrProvince",
"type": [
"null",
"string"
]
},
{
"name": "country",
"type": [
"null",
"string"
]
},
{
"name": "isResidential",
"type": [
"null",
"boolean"
]
}
]
}
},
{
"name": "returnAddress",
"type": "Address"
},
{
"name": "ReferenceNumbers",
"type": [
"null",
{
"type": "array",
"items": {
"type": "record",
"name": "ReferenceNumbers",
"namespace": "com.pb.fdr.delivery",
"fields": [
{
"name": "qualifier",
"type": "string"
},
{
"name": "value",
"type": "string"
}
]
}
}
]
},
{
"name": "hazmatClasses",
"type": [
"null",
{
"type": "array",
"items": {
"type": "enum",
"name": "Hazmat",
"namespace": "com.pb.fdr.delivery",
"symbols": [
"Explosives",
"Gases",
"FlammableCombustibleLiquids",
"FlammableSolids",
"OxidizingSubstancesOrganicPeroxides",
"ToxicSubstancesAndInfectiousSubstances",
"RadioactiveMaterial",
"Corrosives",
"ORMD",
"ConsumerCommodities",
"MiscellaneousHazardousMaterials"
]
}
}
]
},
{
"name": "Subscription",
"type": [
"null",
{
"type": "record",
"name": "Subscription",
"namespace": "com.pb.fdr.delivery",
"fields": [
{
"name": "sendSms",
"type": [
"null",
"boolean"
]
},
{
"name": "sendEmail",
"type": [
"null",
"boolean"
]
},
{
"name": "events",
"type": {
"type": "array",
"items": {
"type": "enum",
"name": "Events",
"namespace": "com.pb.fdr.delivery",
"symbols": [
"Delivered",
"InTransit"
]
}
}
},
{
"name": "emailAddress",
"type": "string"
},
{
"name": "phoneNumber",
"type": "string"
}
]
}
]
}
]
}
Re: Schema parses in C# Avro lib but not in Kafka Schema registry
(assume that it is the java lib)
Posted by Patrick Farry <pa...@gmail.com>.
Thanks Ryan and Bryan.
Turned out it was user error :(. One of our guys made a tweak to the schema
before trying to upload it.
On Tue, Aug 6, 2019, 2:20 AM Ryan Skraba <ry...@skraba.com> wrote:
> Hello! I successfully managed to load the schema into the confluent
> schema-registry version 5.3.0 (containing Avro 1.8.1), using the docker
> quickstart [1] and the command line:
>
> I just tested that the load worked -- I didn't try reading or writing
> binary.
>
> # Save the schema into registry.
> curl -X POST -H "Content-Type: application/vnd.schemaregistry.v1+json"
> --data "@PackageCreateInformationSchema.json"
> http://localhost:8081/subjects/PackageCreateInformation/versions
> # Fetch and check the schema from the registry.
> curl -X GET
> http://localhost:8081/subjects/PackageCreateInformation/versions/1
>
> The PackageCreateInformationSchema.json file is a bit weird, in the format
> {"schema": "your_schema_string_representation"} which takes a lot of
> escaping quotes (the exact contents I used follows if you want to
> reproduce).
>
> How are you loading the schema, and what version of the schema-registry
> are you using? Perhaps we can narrow it down to a specific avro version
> and see if we can reproduce it outside of the registry.
>
> Best regards, Ryan
>
> [1]
> https://docs.confluent.io/current/quickstart/cos-docker-quickstart.html
>
> PackageCreateInformationSchema.json
>
>
> {"schema":"{\"type\":\"record\",\"name\":\"PackageCreateInformation\",\"namespace\":\"com.pb.fdr.delivery\",\"fields\":[{\"name\":\"sendSms\",\"type\":[\"null\",\"boolean\"]},{\"name\":\"trackingId\",\"type\":\"string\"},{\"name\":\"packageId\",\"type\":\"string\"},{\"name\":\"clientFacilityId\",\"type\":\"string\"},{\"name\":\"sortCode\",\"type\":\"string\"},{\"name\":\"ngsFacilityId\",\"type\":\"string\"},{\"name\":\"merchantId\",\"type\":\"string\"},{\"name\":\"ClassOfService\",\"type\":{\"type\":\"enum\",\"name\":\"ClassesOfService\",\"namespace\":\"com.pb.fdr.delivery\",\"symbols\":[\"Ground\",\"Express\",\"FirstClass\",\"Priority\",\"BPM\",\"PBD3D\"]}},{\"name\":\"carrier\",\"type\":{\"type\":\"enum\",\"name\":\"Carriers\",\"namespace\":\"com.pb.fdr.delivery\",\"symbols\":[\"USPS\",\"DHL\",\"UPS\",\"FedEx\"]}},{\"name\":\"carrierClassOfService\",\"type\":{\"type\":\"enum\",\"name\":\"CarrierClassesOfService\",\"namespace\":\"com.pb.fdr.delivery\",\"symbols\":[\"ParcelSelect\",\"ParcelSelectLightweight\",\"FirstClass\",\"Priority\",\"BPM\"]}},{\"name\":\"weight\",\"type\":{\"type\":\"record\",\"name\":\"Weight\",\"namespace\":\"com.pb.fdr.delivery\",\"fields\":[{\"name\":\"unitOfMeasure\",\"type\":{\"type\":\"enum\",\"name\":\"WeightUnitOfMeasure\",\"namespace\":\"com.pb.fdr.delivery\",\"symbols\":[\"Pounds\",\"Ounces\",\"Kilograms\",\"Grams\"]}},{\"name\":\"measurmentValue\",\"type\":\"double\"}]}},{\"name\":\"dimensions\",\"type\":{\"type\":\"record\",\"name\":\"Dimensions\",\"namespace\":\"com.pb.fdr.delivery\",\"fields\":[{\"name\":\"length\",\"type\":{\"type\":\"record\",\"name\":\"Length\",\"namespace\":\"com.pb.fdr.delivery\",\"fields\":[{\"name\":\"unitOfMeasure\",\"type\":{\"type\":\"enum\",\"name\":\"LengthUnitOfMeasure\",\"namespace\":\"com.pb.fdr.delivery\",\"symbols\":[\"Inches\",\"Meters\",\"Centimeters\"]}},{\"name\":\"measurmentValue\",\"type\":\"double\"}]}},{\"name\":\"width\",\"type\":\"Length\"},{\"name\":\"height\",\"type\":\"Length\"},{\"name\":\"girth\",\"type\":\"Length\"},{\"name\":\"isRectangular\",\"type\":\"boolean\"}]}},{\"name\":\"shipToAddress\",\"type\":{\"type\":\"record\",\"name\":\"Address\",\"namespace\":\"com.pb.fdr.delivery\",\"fields\":[{\"name\":\"name\",\"type\":\"string\"},{\"name\":\"attention\",\"type\":[\"null\",\"string\"]},{\"name\":\"address1\",\"type\":\"string\"},{\"name\":\"address2\",\"type\":[\"null\",\"string\"]},{\"name\":\"city\",\"type\":[\"null\",\"string\"]},{\"name\":\"stateOrProvince\",\"type\":[\"null\",\"string\"]},{\"name\":\"country\",\"type\":[\"null\",\"string\"]},{\"name\":\"isResidential\",\"type\":[\"null\",\"boolean\"]}]}},{\"name\":\"returnAddress\",\"type\":\"Address\"},{\"name\":\"ReferenceNumbers\",\"type\":[\"null\",{\"type\":\"array\",\"items\":{\"type\":\"record\",\"name\":\"ReferenceNumbers\",\"namespace\":\"com.pb.fdr.delivery\",\"fields\":[{\"name\":\"qualifier\",\"type\":\"string\"},{\"name\":\"value\",\"type\":\"string\"}]}}]},{\"name\":\"hazmatClasses\",\"type\":[\"null\",{\"type\":\"array\",\"items\":{\"type\":\"enum\",\"name\":\"Hazmat\",\"namespace\":\"com.pb.fdr.delivery\",\"symbols\":[\"Explosives\",\"Gases\",\"FlammableCombustibleLiquids\",\"FlammableSolids\",\"OxidizingSubstancesOrganicPeroxides\",\"ToxicSubstancesAndInfectiousSubstances\",\"RadioactiveMaterial\",\"Corrosives\",\"ORMD\",\"ConsumerCommodities\",\"MiscellaneousHazardousMaterials\"]}}]},{\"name\":\"Subscription\",\"type\":[\"null\",{\"type\":\"record\",\"name\":\"Subscription\",\"namespace\":\"com.pb.fdr.delivery\",\"fields\":[{\"name\":\"sendSms\",\"type\":[\"null\",\"boolean\"]},{\"name\":\"sendEmail\",\"type\":[\"null\",\"boolean\"]},{\"name\":\"events\",\"type\":{\"type\":\"array\",\"items\":{\"type\":\"enum\",\"name\":\"Events\",\"namespace\":\"com.pb.fdr.delivery\",\"symbols\":[\"Delivered\",\"InTransit\"]}}},{\"name\":\"emailAddress\",\"type\":\"string\"},{\"name\":\"phoneNumber\",\"type\":\"string\"}]}]}]}"}
>
>
>
> On Tue, Aug 6, 2019 at 2:50 AM Brian Lachniet <bl...@gmail.com> wrote:
>
>> I don't see anything obviously wrong. I tried generating Java code from
>> this schema with *avro-tools *and had no problems.
>>
>> Unfortunately, I haven't had the opportunity to use the Confluent Schema
>> Registry yet. This comment
>> <https://github.com/confluentinc/schema-registry/issues/214#issuecomment-437514914>
>> seems to suggest that you might find more detailed error messages in the
>> schema registry server logs. Have you tried looking there?
>>
>> On Mon, Aug 5, 2019 at 7:10 PM Patrick Farry <pa...@gmail.com>
>> wrote:
>>
>>> The schema was generated from a protocol using the C# 1.9 code. When we
>>> try to load it into the Kafka schema registry we get an error that the
>>> schema is invalid.
>>>
>>> The only feedback we get is “Length is not valid”. If we remove Length
>>> then the error says that “Address is not valid” - so presumably there is
>>> something wrong referencing previously defined named types.
>>>
>>> Is there anything obviously wrong?
>>>
>>>
>>> {
>>>
>>> "type": "record",
>>>
>>> "name": "PackageCreateInformation",
>>>
>>> "namespace": "com.pb.fdr.delivery",
>>>
>>> "fields": [
>>>
>>> {
>>>
>>> "name": "sendSms",
>>>
>>> "type": [
>>>
>>> "null",
>>>
>>> "boolean"
>>>
>>> ]
>>>
>>> },
>>>
>>> {
>>>
>>> "name": "trackingId",
>>>
>>> "type": "string"
>>>
>>> },
>>>
>>> {
>>>
>>> "name": "packageId",
>>>
>>> "type": "string"
>>>
>>> },
>>>
>>> {
>>>
>>> "name": "clientFacilityId",
>>>
>>> "type": "string"
>>>
>>> },
>>>
>>> {
>>>
>>> "name": "sortCode",
>>>
>>> "type": "string"
>>>
>>> },
>>>
>>> {
>>>
>>> "name": "ngsFacilityId",
>>>
>>> "type": "string"
>>>
>>> },
>>>
>>> {
>>>
>>> "name": "merchantId",
>>>
>>> "type": "string"
>>>
>>> },
>>>
>>> {
>>>
>>> "name": "ClassOfService",
>>>
>>> "type": {
>>>
>>> "type": "enum",
>>>
>>> "name": "ClassesOfService",
>>>
>>> "namespace": "com.pb.fdr.delivery",
>>>
>>> "symbols": [
>>>
>>> "Ground",
>>>
>>> "Express",
>>>
>>> "FirstClass",
>>>
>>> "Priority",
>>>
>>> "BPM",
>>>
>>> "PBD3D"
>>>
>>> ]
>>>
>>> }
>>>
>>> },
>>>
>>> {
>>>
>>> "name": "carrier",
>>>
>>> "type": {
>>>
>>> "type": "enum",
>>>
>>> "name": "Carriers",
>>>
>>> "namespace": "com.pb.fdr.delivery",
>>>
>>> "symbols": [
>>>
>>> "USPS",
>>>
>>> "DHL",
>>>
>>> "UPS",
>>>
>>> "FedEx"
>>>
>>> ]
>>>
>>> }
>>>
>>> },
>>>
>>> {
>>>
>>> "name": "carrierClassOfService",
>>>
>>> "type": {
>>>
>>> "type": "enum",
>>>
>>> "name": "CarrierClassesOfService",
>>>
>>> "namespace": "com.pb.fdr.delivery",
>>>
>>> "symbols": [
>>>
>>> "ParcelSelect",
>>>
>>> "ParcelSelectLightweight",
>>>
>>> "FirstClass",
>>>
>>> "Priority",
>>>
>>> "BPM"
>>>
>>> ]
>>>
>>> }
>>>
>>> },
>>>
>>> {
>>>
>>> "name": "weight",
>>>
>>> "type": {
>>>
>>> "type": "record",
>>>
>>> "name": "Weight",
>>>
>>> "namespace": "com.pb.fdr.delivery",
>>>
>>> "fields": [
>>>
>>> {
>>>
>>> "name": "unitOfMeasure",
>>>
>>> "type": {
>>>
>>> "type": "enum",
>>>
>>> "name": "WeightUnitOfMeasure",
>>>
>>> "namespace": "com.pb.fdr.delivery",
>>>
>>> "symbols": [
>>>
>>> "Pounds",
>>>
>>> "Ounces",
>>>
>>> "Kilograms",
>>>
>>> "Grams"
>>>
>>> ]
>>>
>>> }
>>>
>>> },
>>>
>>> {
>>>
>>> "name": "measurmentValue",
>>>
>>> "type": "double"
>>>
>>> }
>>>
>>> ]
>>>
>>> }
>>>
>>> },
>>>
>>> {
>>>
>>> "name": "dimensions",
>>>
>>> "type": {
>>>
>>> "type": "record",
>>>
>>> "name": "Dimensions",
>>>
>>> "namespace": "com.pb.fdr.delivery",
>>>
>>> "fields": [
>>>
>>> {
>>>
>>> "name": "length",
>>>
>>> "type": {
>>>
>>> "type": "record",
>>>
>>> "name": "Length",
>>>
>>> "namespace": "com.pb.fdr.delivery",
>>>
>>> "fields": [
>>>
>>> {
>>>
>>> "name": "unitOfMeasure",
>>>
>>> "type": {
>>>
>>> "type": "enum",
>>>
>>> "name": "LengthUnitOfMeasure",
>>>
>>> "namespace": "com.pb.fdr.delivery",
>>>
>>> "symbols": [
>>>
>>> "Inches",
>>>
>>> "Meters",
>>>
>>> "Centimeters"
>>>
>>> ]
>>>
>>> }
>>>
>>> },
>>>
>>> {
>>>
>>> "name": "measurmentValue",
>>>
>>> "type": "double"
>>>
>>> }
>>>
>>> ]
>>>
>>> }
>>>
>>> },
>>>
>>> {
>>>
>>> "name": "width",
>>>
>>> "type": "Length"
>>>
>>> },
>>>
>>> {
>>>
>>> "name": "height",
>>>
>>> "type": "Length"
>>>
>>> },
>>>
>>> {
>>>
>>> "name": "girth",
>>>
>>> "type": "Length"
>>>
>>> <div style="box-sizing:inherit;margin-left:32px;padding:0
>>>
>>
Re: Schema parses in C# Avro lib but not in Kafka Schema registry
(assume that it is the java lib)
Posted by Ryan Skraba <ry...@skraba.com>.
Hello! I successfully managed to load the schema into the confluent
schema-registry version 5.3.0 (containing Avro 1.8.1), using the docker
quickstart [1] and the command line:
I just tested that the load worked -- I didn't try reading or writing
binary.
# Save the schema into registry.
curl -X POST -H "Content-Type: application/vnd.schemaregistry.v1+json"
--data "@PackageCreateInformationSchema.json"
http://localhost:8081/subjects/PackageCreateInformation/versions
# Fetch and check the schema from the registry.
curl -X GET
http://localhost:8081/subjects/PackageCreateInformation/versions/1
The PackageCreateInformationSchema.json file is a bit weird, in the format
{"schema": "your_schema_string_representation"} which takes a lot of
escaping quotes (the exact contents I used follows if you want to
reproduce).
How are you loading the schema, and what version of the schema-registry are
you using? Perhaps we can narrow it down to a specific avro version and
see if we can reproduce it outside of the registry.
Best regards, Ryan
[1] https://docs.confluent.io/current/quickstart/cos-docker-quickstart.html
PackageCreateInformationSchema.json
{"schema":"{\"type\":\"record\",\"name\":\"PackageCreateInformation\",\"namespace\":\"com.pb.fdr.delivery\",\"fields\":[{\"name\":\"sendSms\",\"type\":[\"null\",\"boolean\"]},{\"name\":\"trackingId\",\"type\":\"string\"},{\"name\":\"packageId\",\"type\":\"string\"},{\"name\":\"clientFacilityId\",\"type\":\"string\"},{\"name\":\"sortCode\",\"type\":\"string\"},{\"name\":\"ngsFacilityId\",\"type\":\"string\"},{\"name\":\"merchantId\",\"type\":\"string\"},{\"name\":\"ClassOfService\",\"type\":{\"type\":\"enum\",\"name\":\"ClassesOfService\",\"namespace\":\"com.pb.fdr.delivery\",\"symbols\":[\"Ground\",\"Express\",\"FirstClass\",\"Priority\",\"BPM\",\"PBD3D\"]}},{\"name\":\"carrier\",\"type\":{\"type\":\"enum\",\"name\":\"Carriers\",\"namespace\":\"com.pb.fdr.delivery\",\"symbols\":[\"USPS\",\"DHL\",\"UPS\",\"FedEx\"]}},{\"name\":\"carrierClassOfService\",\"type\":{\"type\":\"enum\",\"name\":\"CarrierClassesOfService\",\"namespace\":\"com.pb.fdr.delivery\",\"symbols\":[\"ParcelSelect\",\"ParcelSelectLightweight\",\"FirstClass\",\"Priority\",\"BPM\"]}},{\"name\":\"weight\",\"type\":{\"type\":\"record\",\"name\":\"Weight\",\"namespace\":\"com.pb.fdr.delivery\",\"fields\":[{\"name\":\"unitOfMeasure\",\"type\":{\"type\":\"enum\",\"name\":\"WeightUnitOfMeasure\",\"namespace\":\"com.pb.fdr.delivery\",\"symbols\":[\"Pounds\",\"Ounces\",\"Kilograms\",\"Grams\"]}},{\"name\":\"measurmentValue\",\"type\":\"double\"}]}},{\"name\":\"dimensions\",\"type\":{\"type\":\"record\",\"name\":\"Dimensions\",\"namespace\":\"com.pb.fdr.delivery\",\"fields\":[{\"name\":\"length\",\"type\":{\"type\":\"record\",\"name\":\"Length\",\"namespace\":\"com.pb.fdr.delivery\",\"fields\":[{\"name\":\"unitOfMeasure\",\"type\":{\"type\":\"enum\",\"name\":\"LengthUnitOfMeasure\",\"namespace\":\"com.pb.fdr.delivery\",\"symbols\":[\"Inches\",\"Meters\",\"Centimeters\"]}},{\"name\":\"measurmentValue\",\"type\":\"double\"}]}},{\"name\":\"width\",\"type\":\"Length\"},{\"name\":\"height\",\"type\":\"Length\"},{\"name\":\"girth\",\"type\":\"Length\"},{\"name\":\"isRectangular\",\"type\":\"boolean\"}]}},{\"name\":\"shipToAddress\",\"type\":{\"type\":\"record\",\"name\":\"Address\",\"namespace\":\"com.pb.fdr.delivery\",\"fields\":[{\"name\":\"name\",\"type\":\"string\"},{\"name\":\"attention\",\"type\":[\"null\",\"string\"]},{\"name\":\"address1\",\"type\":\"string\"},{\"name\":\"address2\",\"type\":[\"null\",\"string\"]},{\"name\":\"city\",\"type\":[\"null\",\"string\"]},{\"name\":\"stateOrProvince\",\"type\":[\"null\",\"string\"]},{\"name\":\"country\",\"type\":[\"null\",\"string\"]},{\"name\":\"isResidential\",\"type\":[\"null\",\"boolean\"]}]}},{\"name\":\"returnAddress\",\"type\":\"Address\"},{\"name\":\"ReferenceNumbers\",\"type\":[\"null\",{\"type\":\"array\",\"items\":{\"type\":\"record\",\"name\":\"ReferenceNumbers\",\"namespace\":\"com.pb.fdr.delivery\",\"fields\":[{\"name\":\"qualifier\",\"type\":\"string\"},{\"name\":\"value\",\"type\":\"string\"}]}}]},{\"name\":\"hazmatClasses\",\"type\":[\"null\",{\"type\":\"array\",\"items\":{\"type\":\"enum\",\"name\":\"Hazmat\",\"namespace\":\"com.pb.fdr.delivery\",\"symbols\":[\"Explosives\",\"Gases\",\"FlammableCombustibleLiquids\",\"FlammableSolids\",\"OxidizingSubstancesOrganicPeroxides\",\"ToxicSubstancesAndInfectiousSubstances\",\"RadioactiveMaterial\",\"Corrosives\",\"ORMD\",\"ConsumerCommodities\",\"MiscellaneousHazardousMaterials\"]}}]},{\"name\":\"Subscription\",\"type\":[\"null\",{\"type\":\"record\",\"name\":\"Subscription\",\"namespace\":\"com.pb.fdr.delivery\",\"fields\":[{\"name\":\"sendSms\",\"type\":[\"null\",\"boolean\"]},{\"name\":\"sendEmail\",\"type\":[\"null\",\"boolean\"]},{\"name\":\"events\",\"type\":{\"type\":\"array\",\"items\":{\"type\":\"enum\",\"name\":\"Events\",\"namespace\":\"com.pb.fdr.delivery\",\"symbols\":[\"Delivered\",\"InTransit\"]}}},{\"name\":\"emailAddress\",\"type\":\"string\"},{\"name\":\"phoneNumber\",\"type\":\"string\"}]}]}]}"}
On Tue, Aug 6, 2019 at 2:50 AM Brian Lachniet <bl...@gmail.com> wrote:
> I don't see anything obviously wrong. I tried generating Java code from
> this schema with *avro-tools *and had no problems.
>
> Unfortunately, I haven't had the opportunity to use the Confluent Schema
> Registry yet. This comment
> <https://github.com/confluentinc/schema-registry/issues/214#issuecomment-437514914>
> seems to suggest that you might find more detailed error messages in the
> schema registry server logs. Have you tried looking there?
>
> On Mon, Aug 5, 2019 at 7:10 PM Patrick Farry <pa...@gmail.com>
> wrote:
>
>> The schema was generated from a protocol using the C# 1.9 code. When we
>> try to load it into the Kafka schema registry we get an error that the
>> schema is invalid.
>>
>> The only feedback we get is “Length is not valid”. If we remove Length
>> then the error says that “Address is not valid” - so presumably there is
>> something wrong referencing previously defined named types.
>>
>> Is there anything obviously wrong?
>>
>>
>> {
>>
>> "type": "record",
>>
>> "name": "PackageCreateInformation",
>>
>> "namespace": "com.pb.fdr.delivery",
>>
>> "fields": [
>>
>> {
>>
>> "name": "sendSms",
>>
>> "type": [
>>
>> "null",
>>
>> "boolean"
>>
>> ]
>>
>> },
>>
>> {
>>
>> "name": "trackingId",
>>
>> "type": "string"
>>
>> },
>>
>> {
>>
>> "name": "packageId",
>>
>> "type": "string"
>>
>> },
>>
>> {
>>
>> "name": "clientFacilityId",
>>
>> "type": "string"
>>
>> },
>>
>> {
>>
>> "name": "sortCode",
>>
>> "type": "string"
>>
>> },
>>
>> {
>>
>> "name": "ngsFacilityId",
>>
>> "type": "string"
>>
>> },
>>
>> {
>>
>> "name": "merchantId",
>>
>> "type": "string"
>>
>> },
>>
>> {
>>
>> "name": "ClassOfService",
>>
>> "type": {
>>
>> "type": "enum",
>>
>> "name": "ClassesOfService",
>>
>> "namespace": "com.pb.fdr.delivery",
>>
>> "symbols": [
>>
>> "Ground",
>>
>> "Express",
>>
>> "FirstClass",
>>
>> "Priority",
>>
>> "BPM",
>>
>> "PBD3D"
>>
>> ]
>>
>> }
>>
>> },
>>
>> {
>>
>> "name": "carrier",
>>
>> "type": {
>>
>> "type": "enum",
>>
>> "name": "Carriers",
>>
>> "namespace": "com.pb.fdr.delivery",
>>
>> "symbols": [
>>
>> "USPS",
>>
>> "DHL",
>>
>> "UPS",
>>
>> "FedEx"
>>
>> ]
>>
>> }
>>
>> },
>>
>> {
>>
>> "name": "carrierClassOfService",
>>
>> "type": {
>>
>> "type": "enum",
>>
>> "name": "CarrierClassesOfService",
>>
>> "namespace": "com.pb.fdr.delivery",
>>
>> "symbols": [
>>
>> "ParcelSelect",
>>
>> "ParcelSelectLightweight",
>>
>> "FirstClass",
>>
>> "Priority",
>>
>> "BPM"
>>
>> ]
>>
>> }
>>
>> },
>>
>> {
>>
>> "name": "weight",
>>
>> "type": {
>>
>> "type": "record",
>>
>> "name": "Weight",
>>
>> "namespace": "com.pb.fdr.delivery",
>>
>> "fields": [
>>
>> {
>>
>> "name": "unitOfMeasure",
>>
>> "type": {
>>
>> "type": "enum",
>>
>> "name": "WeightUnitOfMeasure",
>>
>> "namespace": "com.pb.fdr.delivery",
>>
>> "symbols": [
>>
>> "Pounds",
>>
>> "Ounces",
>>
>> "Kilograms",
>>
>> "Grams"
>>
>> ]
>>
>> }
>>
>> },
>>
>> {
>>
>> "name": "measurmentValue",
>>
>> "type": "double"
>>
>> }
>>
>> ]
>>
>> }
>>
>> },
>>
>> {
>>
>> "name": "dimensions",
>>
>> "type": {
>>
>> "type": "record",
>>
>> "name": "Dimensions",
>>
>> "namespace": "com.pb.fdr.delivery",
>>
>> "fields": [
>>
>> {
>>
>> "name": "length",
>>
>> "type": {
>>
>> "type": "record",
>>
>> "name": "Length",
>>
>> "namespace": "com.pb.fdr.delivery",
>>
>> "fields": [
>>
>> {
>>
>> "name": "unitOfMeasure",
>>
>> "type": {
>>
>> "type": "enum",
>>
>> "name": "LengthUnitOfMeasure",
>>
>> "namespace": "com.pb.fdr.delivery",
>>
>> "symbols": [
>>
>> "Inches",
>>
>> "Meters",
>>
>> "Centimeters"
>>
>> ]
>>
>> }
>>
>> },
>>
>> {
>>
>> "name": "measurmentValue",
>>
>> "type": "double"
>>
>> }
>>
>> ]
>>
>> }
>>
>> },
>>
>> {
>>
>> "name": "width",
>>
>> "type": "Length"
>>
>> },
>>
>> {
>>
>> "name": "height",
>>
>> "type": "Length"
>>
>> },
>>
>> {
>>
>> "name": "girth",
>>
>> "type": "Length"
>>
>> },
>>
>> {
>>
>> "name": "isRectangular",
>>
>> "type": "boolean"
>>
>> }
>>
>> ]
>>
>> }
>>
>> },
>>
>> {
>>
>> "name": "shipToAddress",
>>
>> "type": {
>>
>> "type": "record",
>>
>> "name": "Address",
>>
>> "namespace": "com.pb.fdr.delivery",
>>
>> "fields": [
>>
>> {
>>
>> "name": "name",
>>
>> "type": "string"
>>
>> },
>>
>> {
>>
>> "name": "attention",
>>
>> "type": [
>>
>> "null",
>>
>> "string"
>>
>> ]
>>
>> },
>>
>> {
>>
>> "name": "address1",
>>
>> "type": "string"
>>
>> },
>>
>> {
>>
>> "name": "address2",
>>
>> "type": [
>>
>> "null",
>>
>> "string"
>>
>> ]
>>
>> },
>>
>> {
>>
>> "name": "city",
>>
>> "type": [
>>
>> "null",
>>
>> "string"
>>
>> ]
>>
>> },
>>
>> {
>>
>> "name": "stateOrProvince",
>>
>> "type": [
>>
>> "null",
>>
>> "string"
>>
>> ]
>>
>> },
>>
>> {
>>
>> "name": "country",
>>
>> "type": [
>>
>> "null",
>>
>> "string"
>>
>> ]
>>
>> },
>>
>> {
>>
>> "name": "isResidential",
>>
>> "type": [
>>
>> "null",
>>
>> "boolean"
>>
>> ]
>>
>> }
>>
>> ]
>>
>> }
>>
>> },
>>
>> {
>>
>> "name": "returnAddress",
>>
>> "type": "Address"
>>
>> },
>>
>> {
>>
>> "name": "ReferenceNumbers",
>>
>> "type": [
>>
>> "null",
>>
>> {
>>
>> "type": "array",
>>
>> "items": {
>>
>> "type": "record",
>>
>> "name": "ReferenceNumbers",
>>
>> "namespace": "com.pb.fdr.delivery",
>>
>> "fields": [
>>
>> {
>>
>> "name": "qualifier",
>>
>> "type": "string"
>>
>> },
>>
>> {
>>
>> "name": "value",
>>
>> "type": "string"
>>
>> }
>>
>> ]
>>
>> }
>>
>> }
>>
>> ]
>>
>> },
>>
>> {
>>
>> "name": "hazmatClasses",
>>
>> "type": [
>>
>> "null",
>>
>> {
>>
>> "type": "array",
>>
>> "items": {
>>
>> "type": "enum",
>>
>> "name": "Hazmat",
>>
>> "namespace": "com.pb.fdr.delivery",
>>
>> "symbols": [
>>
>> "Explosives",
>>
>> "Gases",
>>
>> "FlammableCombustibleLiquids",
>>
>> "FlammableSolids",
>>
>> "OxidizingSubstancesOrganicPeroxides",
>>
>> "ToxicSubstancesAndInfectiousSubstances",
>>
>> "RadioactiveMaterial",
>>
>> "Corrosives",
>>
>> "ORMD",
>>
>> "ConsumerCommodities",
>>
>> "MiscellaneousHazardousMaterials"
>>
>> ]
>>
>> }
>>
>> }
>>
>> ]
>>
>> },
>>
>> {
>>
>> "name": "Subscription",
>>
>> "type": [
>>
>> "null",
>>
>> {
>>
>> "type": "record",
>>
>> "name": "Subscription",
>>
>> "namespace": "com.pb.fdr.delivery",
>>
>> "fields": [
>>
>> {
>>
>> "name": "sendSms",
>>
>> "type": [
>>
>> "null",
>>
>> "boolean"
>>
>> ]
>>
>> },
>>
>> {
>>
>> "name": "sendEmail",
>>
>> "type": [
>>
>> "null",
>>
>> "boolean"
>>
>> ]
>>
>> },
>>
>> {
>>
>> "name": "events",
>>
>> "type": {
>>
>> "type": "array",
>>
>> "items": {
>>
>> "type": "enum",
>>
>> "name": "Events",
>>
>> "namespace": "com.pb.fdr.delivery",
>>
>> "symbols": [
>>
>> "Delivered",
>>
>> "InTransit"
>>
>> ]
>>
>> }
>>
>> }
>>
>> },
>>
>> {
>>
>> "name": "emailAddress",
>>
>> "type": "string"
>>
>> },
>>
>> {
>>
>> "name": "phoneNumber",
>>
>> "type": "string"
>>
>> }
>>
>> ]
>>
>> }
>>
>> ]
>>
>> }
>>
>> ]
>>
>> }
>>
>>
>>
>
> --
>
> [image: 51b630b05e01a6d5134ccfd520f547c4.png]
>
> Brian Lachniet
>
> Software Engineer
>
> E: blachniet@gmail.com | blachniet.com <http://www.blachniet.com>
>
> <https://twitter.com/blachniet> <http://www.linkedin.com/in/blachniet>
>
Re: Schema parses in C# Avro lib but not in Kafka Schema registry
(assume that it is the java lib)
Posted by Brian Lachniet <bl...@gmail.com>.
I don't see anything obviously wrong. I tried generating Java code from
this schema with *avro-tools *and had no problems.
Unfortunately, I haven't had the opportunity to use the Confluent Schema
Registry yet. This comment
<https://github.com/confluentinc/schema-registry/issues/214#issuecomment-437514914>
seems to suggest that you might find more detailed error messages in the
schema registry server logs. Have you tried looking there?
On Mon, Aug 5, 2019 at 7:10 PM Patrick Farry <pa...@gmail.com>
wrote:
> The schema was generated from a protocol using the C# 1.9 code. When we
> try to load it into the Kafka schema registry we get an error that the
> schema is invalid.
>
> The only feedback we get is “Length is not valid”. If we remove Length
> then the error says that “Address is not valid” - so presumably there is
> something wrong referencing previously defined named types.
>
> Is there anything obviously wrong?
>
>
> {
>
> "type": "record",
>
> "name": "PackageCreateInformation",
>
> "namespace": "com.pb.fdr.delivery",
>
> "fields": [
>
> {
>
> "name": "sendSms",
>
> "type": [
>
> "null",
>
> "boolean"
>
> ]
>
> },
>
> {
>
> "name": "trackingId",
>
> "type": "string"
>
> },
>
> {
>
> "name": "packageId",
>
> "type": "string"
>
> },
>
> {
>
> "name": "clientFacilityId",
>
> "type": "string"
>
> },
>
> {
>
> "name": "sortCode",
>
> "type": "string"
>
> },
>
> {
>
> "name": "ngsFacilityId",
>
> "type": "string"
>
> },
>
> {
>
> "name": "merchantId",
>
> "type": "string"
>
> },
>
> {
>
> "name": "ClassOfService",
>
> "type": {
>
> "type": "enum",
>
> "name": "ClassesOfService",
>
> "namespace": "com.pb.fdr.delivery",
>
> "symbols": [
>
> "Ground",
>
> "Express",
>
> "FirstClass",
>
> "Priority",
>
> "BPM",
>
> "PBD3D"
>
> ]
>
> }
>
> },
>
> {
>
> "name": "carrier",
>
> "type": {
>
> "type": "enum",
>
> "name": "Carriers",
>
> "namespace": "com.pb.fdr.delivery",
>
> "symbols": [
>
> "USPS",
>
> "DHL",
>
> "UPS",
>
> "FedEx"
>
> ]
>
> }
>
> },
>
> {
>
> "name": "carrierClassOfService",
>
> "type": {
>
> "type": "enum",
>
> "name": "CarrierClassesOfService",
>
> "namespace": "com.pb.fdr.delivery",
>
> "symbols": [
>
> "ParcelSelect",
>
> "ParcelSelectLightweight",
>
> "FirstClass",
>
> "Priority",
>
> "BPM"
>
> ]
>
> }
>
> },
>
> {
>
> "name": "weight",
>
> "type": {
>
> "type": "record",
>
> "name": "Weight",
>
> "namespace": "com.pb.fdr.delivery",
>
> "fields": [
>
> {
>
> "name": "unitOfMeasure",
>
> "type": {
>
> "type": "enum",
>
> "name": "WeightUnitOfMeasure",
>
> "namespace": "com.pb.fdr.delivery",
>
> "symbols": [
>
> "Pounds",
>
> "Ounces",
>
> "Kilograms",
>
> "Grams"
>
> ]
>
> }
>
> },
>
> {
>
> "name": "measurmentValue",
>
> "type": "double"
>
> }
>
> ]
>
> }
>
> },
>
> {
>
> "name": "dimensions",
>
> "type": {
>
> "type": "record",
>
> "name": "Dimensions",
>
> "namespace": "com.pb.fdr.delivery",
>
> "fields": [
>
> {
>
> "name": "length",
>
> "type": {
>
> "type": "record",
>
> "name": "Length",
>
> "namespace": "com.pb.fdr.delivery",
>
> "fields": [
>
> {
>
> "name": "unitOfMeasure",
>
> "type": {
>
> "type": "enum",
>
> "name": "LengthUnitOfMeasure",
>
> "namespace": "com.pb.fdr.delivery",
>
> "symbols": [
>
> "Inches",
>
> "Meters",
>
> "Centimeters"
>
> ]
>
> }
>
> },
>
> {
>
> "name": "measurmentValue",
>
> "type": "double"
>
> }
>
> ]
>
> }
>
> },
>
> {
>
> "name": "width",
>
> "type": "Length"
>
> },
>
> {
>
> "name": "height",
>
> "type": "Length"
>
> },
>
> {
>
> "name": "girth",
>
> "type": "Length"
>
> },
>
> {
>
> "name": "isRectangular",
>
> "type": "boolean"
>
> }
>
> ]
>
> }
>
> },
>
> {
>
> "name": "shipToAddress",
>
> "type": {
>
> "type": "record",
>
> "name": "Address",
>
> "namespace": "com.pb.fdr.delivery",
>
> "fields": [
>
> {
>
> "name": "name",
>
> "type": "string"
>
> },
>
> {
>
> "name": "attention",
>
> "type": [
>
> "null",
>
> "string"
>
> ]
>
> },
>
> {
>
> "name": "address1",
>
> "type": "string"
>
> },
>
> {
>
> "name": "address2",
>
> "type": [
>
> "null",
>
> "string"
>
> ]
>
> },
>
> {
>
> "name": "city",
>
> "type": [
>
> "null",
>
> "string"
>
> ]
>
> },
>
> {
>
> "name": "stateOrProvince",
>
> "type": [
>
> "null",
>
> "string"
>
> ]
>
> },
>
> {
>
> "name": "country",
>
> "type": [
>
> "null",
>
> "string"
>
> ]
>
> },
>
> {
>
> "name": "isResidential",
>
> "type": [
>
> "null",
>
> "boolean"
>
> ]
>
> }
>
> ]
>
> }
>
> },
>
> {
>
> "name": "returnAddress",
>
> "type": "Address"
>
> },
>
> {
>
> "name": "ReferenceNumbers",
>
> "type": [
>
> "null",
>
> {
>
> "type": "array",
>
> "items": {
>
> "type": "record",
>
> "name": "ReferenceNumbers",
>
> "namespace": "com.pb.fdr.delivery",
>
> "fields": [
>
> {
>
> "name": "qualifier",
>
> "type": "string"
>
> },
>
> {
>
> "name": "value",
>
> "type": "string"
>
> }
>
> ]
>
> }
>
> }
>
> ]
>
> },
>
> {
>
> "name": "hazmatClasses",
>
> "type": [
>
> "null",
>
> {
>
> "type": "array",
>
> "items": {
>
> "type": "enum",
>
> "name": "Hazmat",
>
> "namespace": "com.pb.fdr.delivery",
>
> "symbols": [
>
> "Explosives",
>
> "Gases",
>
> "FlammableCombustibleLiquids",
>
> "FlammableSolids",
>
> "OxidizingSubstancesOrganicPeroxides",
>
> "ToxicSubstancesAndInfectiousSubstances",
>
> "RadioactiveMaterial",
>
> "Corrosives",
>
> "ORMD",
>
> "ConsumerCommodities",
>
> "MiscellaneousHazardousMaterials"
>
> ]
>
> }
>
> }
>
> ]
>
> },
>
> {
>
> "name": "Subscription",
>
> "type": [
>
> "null",
>
> {
>
> "type": "record",
>
> "name": "Subscription",
>
> "namespace": "com.pb.fdr.delivery",
>
> "fields": [
>
> {
>
> "name": "sendSms",
>
> "type": [
>
> "null",
>
> "boolean"
>
> ]
>
> },
>
> {
>
> "name": "sendEmail",
>
> "type": [
>
> "null",
>
> "boolean"
>
> ]
>
> },
>
> {
>
> "name": "events",
>
> "type": {
>
> "type": "array",
>
> "items": {
>
> "type": "enum",
>
> "name": "Events",
>
> "namespace": "com.pb.fdr.delivery",
>
> "symbols": [
>
> "Delivered",
>
> "InTransit"
>
> ]
>
> }
>
> }
>
> },
>
> {
>
> "name": "emailAddress",
>
> "type": "string"
>
> },
>
> {
>
> "name": "phoneNumber",
>
> "type": "string"
>
> }
>
> ]
>
> }
>
> ]
>
> }
>
> ]
>
> }
>
>
>
--
[image: 51b630b05e01a6d5134ccfd520f547c4.png]
Brian Lachniet
Software Engineer
E: blachniet@gmail.com | blachniet.com <http://www.blachniet.com>
<https://twitter.com/blachniet> <http://www.linkedin.com/in/blachniet>