You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@avro.apache.org by Siva <sb...@gmail.com> on 2016/02/18 01:21:59 UTC
fromJson is failing with null as uniontype
Hi Everyone,
I m new to avro, running into issues if a type is combined with "null",
like ["null","int"] or ["null", "string"].
I have a schema like below
{
"type":"record","namespace":"tutorialspoint",
"name":"empdetails","fields":[ {
"name":"experience",
"type":["null","string"],"default":null
},
{ "name":"age", "type":"int" }
]
}
Below is the json dataset.
{"experience" : "da", "age": 50}
java -jar avro-tools-1.7.7.jar fromjson --schema-file test.avsc test.json >
test.avro
If I have "null" value in "experience" column it goes though, but if it has
some string it is giving below error. Similar error with int types as well
(VALUE_NUMBER_INT).
Exception in thread "main" org.apache.avro.AvroTypeException: Expected
start-union. Got VALUE_STRING
at org.apache.avro.io.JsonDecoder.error(JsonDecoder.java:697)
at org.apache.avro.io.JsonDecoder.readIndex(JsonDecoder.java:441)
at
org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:290)
at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
I have columns with strings or nulls in json, is there a work around to
this error without changing the json data?
Thanks in advance.
Thanks,
Sivakumar Bhavanari.
Re: fromJson is failing with null as uniontype
Posted by Svante Karlsson <sv...@csi.se>.
The problem is that avro has it's own representation of union encoding so
your experience would encode to {"int": 50}.
In a recent project we ended up writing a slightly modded json parser to be
able to use avro schemas on existing json rest calls.
2016-02-29 9:20 GMT+01:00 Chris Miller <cm...@gmail.com>:
> Did you ever figure this out? I was having the same problem.
>
>
> --
> Chris Miller
>
> On Fri, Feb 19, 2016 at 2:53 AM, Siva <sb...@gmail.com> wrote:
>
>> Can someone help on this? Is anyone faced similar issue?
>>
>> Thanks,
>> Sivakumar Bhavanari.
>>
>> On Wed, Feb 17, 2016 at 4:21 PM, Siva <sb...@gmail.com> wrote:
>>
>>> Hi Everyone,
>>>
>>> I m new to avro, running into issues if a type is combined with "null",
>>> like ["null","int"] or ["null", "string"].
>>>
>>> I have a schema like below
>>>
>>> {
>>> "type":"record","namespace":"tutorialspoint",
>>> "name":"empdetails","fields":[ {
>>> "name":"experience",
>>> "type":["null","string"],"default":null
>>> },
>>> { "name":"age", "type":"int" }
>>> ]
>>> }
>>>
>>> Below is the json dataset.
>>>
>>> {"experience" : "da", "age": 50}
>>>
>>> java -jar avro-tools-1.7.7.jar fromjson --schema-file test.avsc
>>> test.json > test.avro
>>>
>>> If I have "null" value in "experience" column it goes though, but if it
>>> has some string it is giving below error. Similar error with int types as
>>> well (VALUE_NUMBER_INT).
>>>
>>> Exception in thread "main" org.apache.avro.AvroTypeException: Expected
>>> start-union. Got VALUE_STRING
>>> at org.apache.avro.io.JsonDecoder.error(JsonDecoder.java:697)
>>> at org.apache.avro.io.JsonDecoder.readIndex(JsonDecoder.java:441)
>>> at
>>> org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:290)
>>> at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
>>>
>>> I have columns with strings or nulls in json, is there a work around to
>>> this error without changing the json data?
>>>
>>> Thanks in advance.
>>>
>>> Thanks,
>>> Sivakumar Bhavanari.
>>>
>>
>>
>
Re: fromJson is failing with null as uniontype
Posted by Chris Miller <cm...@gmail.com>.
Did you ever figure this out? I was having the same problem.
--
Chris Miller
On Fri, Feb 19, 2016 at 2:53 AM, Siva <sb...@gmail.com> wrote:
> Can someone help on this? Is anyone faced similar issue?
>
> Thanks,
> Sivakumar Bhavanari.
>
> On Wed, Feb 17, 2016 at 4:21 PM, Siva <sb...@gmail.com> wrote:
>
>> Hi Everyone,
>>
>> I m new to avro, running into issues if a type is combined with "null",
>> like ["null","int"] or ["null", "string"].
>>
>> I have a schema like below
>>
>> {
>> "type":"record","namespace":"tutorialspoint",
>> "name":"empdetails","fields":[ {
>> "name":"experience",
>> "type":["null","string"],"default":null
>> },
>> { "name":"age", "type":"int" }
>> ]
>> }
>>
>> Below is the json dataset.
>>
>> {"experience" : "da", "age": 50}
>>
>> java -jar avro-tools-1.7.7.jar fromjson --schema-file test.avsc test.json
>> > test.avro
>>
>> If I have "null" value in "experience" column it goes though, but if it
>> has some string it is giving below error. Similar error with int types as
>> well (VALUE_NUMBER_INT).
>>
>> Exception in thread "main" org.apache.avro.AvroTypeException: Expected
>> start-union. Got VALUE_STRING
>> at org.apache.avro.io.JsonDecoder.error(JsonDecoder.java:697)
>> at org.apache.avro.io.JsonDecoder.readIndex(JsonDecoder.java:441)
>> at
>> org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:290)
>> at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
>>
>> I have columns with strings or nulls in json, is there a work around to
>> this error without changing the json data?
>>
>> Thanks in advance.
>>
>> Thanks,
>> Sivakumar Bhavanari.
>>
>
>
Re: fromJson is failing with null as uniontype
Posted by Siva <sb...@gmail.com>.
Can someone help on this? Is anyone faced similar issue?
Thanks,
Sivakumar Bhavanari.
On Wed, Feb 17, 2016 at 4:21 PM, Siva <sb...@gmail.com> wrote:
> Hi Everyone,
>
> I m new to avro, running into issues if a type is combined with "null",
> like ["null","int"] or ["null", "string"].
>
> I have a schema like below
>
> {
> "type":"record","namespace":"tutorialspoint",
> "name":"empdetails","fields":[ {
> "name":"experience",
> "type":["null","string"],"default":null
> },
> { "name":"age", "type":"int" }
> ]
> }
>
> Below is the json dataset.
>
> {"experience" : "da", "age": 50}
>
> java -jar avro-tools-1.7.7.jar fromjson --schema-file test.avsc test.json
> > test.avro
>
> If I have "null" value in "experience" column it goes though, but if it
> has some string it is giving below error. Similar error with int types as
> well (VALUE_NUMBER_INT).
>
> Exception in thread "main" org.apache.avro.AvroTypeException: Expected
> start-union. Got VALUE_STRING
> at org.apache.avro.io.JsonDecoder.error(JsonDecoder.java:697)
> at org.apache.avro.io.JsonDecoder.readIndex(JsonDecoder.java:441)
> at
> org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:290)
> at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
>
> I have columns with strings or nulls in json, is there a work around to
> this error without changing the json data?
>
> Thanks in advance.
>
> Thanks,
> Sivakumar Bhavanari.
>