You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by tl <tl...@rat.io> on 2016/03/02 16:23:09 UTC

NullPointerException from Json.ObjectWriter

Hi,

this is a bug report. I assume that this is _not_ the right place to report bugs but I really can’t find where to file it on [0]. If somebody can show me where to click there (or anywhere else) I’ll happily do that. Until then I’ll have at least posted it here (and off my chest).


Avro: 
NullPointerException from Json.ObjectWriter


Environment:
avro 1.8.0, jackson-core-asl 1.9.13, jackson-mapper-asl 1.9.13, Java 7, Mac OS X 10.11.3


Description:
The complete, slightly more involved code is on [1], especially [2], but the relevant parts are as follows:

// parsing the schema
Schema.Parser parser = new Schema.Parser();
Schema schema = parser.parse(new File("schema.avsc")) ;
// setting up the encoder and driver
Json.ObjectWriter jsonDatumWriter = new Json.ObjectWriter();
OutputStream output = new FileOutputStream(new File("output.json"));
Encoder encoder = EncoderFactory.get().jsonEncoder(schema, output, pretty);
// writing
System.out.println(payload); // reassuring test the payload is intact
jsonDatumWriter.write(payload, encoder);


The console will print out a nice JSON string (the payload), followed by the following exception:
Exception in thread "main" java.lang.NullPointerException
at org.apache.avro.data.Json.write(Json.java:183)
at org.apache.avro.data.Json.writeObject(Json.java:272)
at org.apache.avro.data.Json.access$000(Json.java:48)
at org.apache.avro.data.Json$ObjectWriter.write(Json.java:122)
at converTor.WriterObject.append(WriterObject.java:59)
at converTor.ConverTor.main(ConverTor.java:251)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)



My beginners guess is that the source of the problem is a call of asToken() in org.codehaus.jackson.JsonNode, which is abstract.


Regards,
Thomas


[0] https://issues.apache.org/jira/browse/avro/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel
[1] https://github.com/tomlurge/converTor
[2] https://github.com/tomlurge/converTor/blob/master/src/converTor/WriterObject.java

Re: NullPointerException from Json.ObjectWriter

Posted by tl <tl...@rat.io>.
> On 04.03.2016, at 21:53, Wing Yew Poon <wy...@cloudera.com> wrote:
> 
>>> 
>>> https://issues.apache.org/jira/browse/AVRO
>> 
>> That’s the page, but where to klick at? Where’s the link to filing a bug report?
>> If I click on "Create" and then "Create Service Desk Request" (which seems like the most fitting category) I’m presented with the following choices: "Kylin", "Atlas", "Apache Infrastructure", "Ranger". None of them seem right.
>> 
> It seems that the Apache JIRA is rather unintuitive. Once you're
> logged in, you click on the Create button, not select the dropdown
> next to the Create button. Then you should be able to create an issue.

OMG… thanks!

> - Wing Yew Poon





< he not busy being born is busy dying >

Re: NullPointerException from Json.ObjectWriter

Posted by Wing Yew Poon <wy...@cloudera.com>.
>>
>>  https://issues.apache.org/jira/browse/AVRO
>
> That’s the page, but where to klick at? Where’s the link to filing a bug report?
> If I click on "Create" and then "Create Service Desk Request" (which seems like the most fitting category) I’m presented with the following choices: "Kylin", "Atlas", "Apache Infrastructure", "Ranger". None of them seem right.
>
It seems that the Apache JIRA is rather unintuitive. Once you're
logged in, you click on the Create button, not select the dropdown
next to the Create button. Then you should be able to create an issue.

- Wing Yew Poon

Re: NullPointerException from Json.ObjectWriter

Posted by tl <tl...@rat.io>.
> On 04.03.2016, at 18:36, Ryan Blue <rb...@netflix.com.INVALID> wrote:
> 
> Thanks for the detailed report, Thomas. Sending an e-mail here is fine, but
> you can also open issues in the tracker here:
> 
>  https://issues.apache.org/jira/browse/AVRO

That’s the page, but where to klick at? Where’s the link to filing a bug report?
If I click on "Create" and then "Create Service Desk Request" (which seems like the most fitting category) I’m presented with the following choices: "Kylin", "Atlas", "Apache Infrastructure", "Ranger". None of them seem right.

> Could you send the schema you're using? It may be that your schema doesn't
> allow null, but you're passing one in. I'm not too familiar with the JSON
> encoder so that will help me take a look.

See below.

> By the way, I recommend using the
> binary encoder to the JSON encoder, but it's up to you.

I need to output JSON, Avro and Parquet. Avro and Parquet are working already, only JSON throws the NullPointerException.

Thanks for looking into this!
Thomas




{
  "type" : "record",
  "name" : "Torperf",
  "namespace" : "converTor.avro.classes.torperf",
  "fields" : [ {
    "name" : "descriptor_type",
    "type" : "string",
    "default" : "torperf 1.0"
  }, {
    "name" : "source",
    "type" : [ "null", "string" ],
    "doc" : "metrics-lib/TorperfResult: String getSource()"
  }, {
    "name" : "filesize",
    "type" : [ "null", "int" ],
    "doc" : "metrics-lib/TorperfResult: int getFileSize()"
  }, {
    "name" : "start",
    "type" : [ "null", "long" ],
    "doc" : "metrics-lib/TorperfResult: long getStartMillis()"
  }, {
    "name" : "socket",
    "type" : [ "null", "long" ],
    "doc" : "metrics-lib/TorperfResult: long getSocketMillis()"
  }, {
    "name" : "connect",
    "type" : [ "null", "long" ],
    "doc" : "metrics-lib/TorperfResult: long getConnectMillis()"
  }, {
    "name" : "negotiate",
    "type" : [ "null", "long" ],
    "doc" : "metrics-lib/TorperfResult: long getNegotiateMillis()"
  }, {
    "name" : "request",
    "type" : [ "null", "long" ],
    "doc" : "metrics-lib/TorperfResult: long getRequestMillis()"
  }, {
    "name" : "response",
    "type" : [ "null", "long" ],
    "doc" : "metrics-lib/TorperfResult: long getResponseMillis()"
  }, {
    "name" : "datarequest",
    "type" : [ "null", "long" ],
    "doc" : "metrics-lib/TorperfResult: long getDataRequestMillis()"
  }, {
    "name" : "dataresponse",
    "type" : [ "null", "long" ],
    "doc" : "metrics-lib/TorperfResult: long getDataResponseMillis()"
  }, {
    "name" : "datacomplete",
    "type" : [ "null", "long" ],
    "doc" : "metrics-lib/TorperfResult: long getDataCompleteMillis()"
  }, {
    "name" : "writebytes",
    "type" : [ "null", "int" ],
    "doc" : "metrics-lib/TorperfResult: int getWriteBytes()"
  }, {
    "name" : "readbytes",
    "type" : [ "null", "int" ],
    "doc" : "metrics-lib/TorperfResult: int getReadBytes()"
  }, {
    "name" : "didtimeout",
    "type" : [ "null", "boolean" ],
    "doc" : "metrics-lib/TorperfResult: Boolean didTimeout()"
  }, {
    "name" : "dataperc10",
    "type" : [ "null", "long" ],
    "doc" : "metrics-lib/TorperfResult: long SortedMap<Integer, Long> getDataPercentiles().get(10)"
  }, {
    "name" : "dataperc20",
    "type" : [ "null", "long" ],
    "doc" : "metrics-lib/TorperfResult: long SortedMap<Integer, Long> getDataPercentiles().get(20)"
  }, {
    "name" : "dataperc30",
    "type" : [ "null", "long" ],
    "doc" : "metrics-lib/TorperfResult: long SortedMap<Integer, Long> getDataPercentiles().get(30)"
  }, {
    "name" : "dataperc40",
    "type" : [ "null", "long" ],
    "doc" : "metrics-lib/TorperfResult: long SortedMap<Integer, Long> getDataPercentiles().get(40)"
  }, {
    "name" : "dataperc50",
    "type" : [ "null", "long" ],
    "doc" : "metrics-lib/TorperfResult: long SortedMap<Integer, Long> getDataPercentiles().get(50)"
  }, {
    "name" : "dataperc60",
    "type" : [ "null", "long" ],
    "doc" : "metrics-lib/TorperfResult: long SortedMap<Integer, Long> getDataPercentiles().get(60)"
  }, {
    "name" : "dataperc70",
    "type" : [ "null", "long" ],
    "doc" : "metrics-lib/TorperfResult: long SortedMap<Integer, Long> getDataPercentiles().get(70)"
  }, {
    "name" : "dataperc80",
    "type" : [ "null", "long" ],
    "doc" : "metrics-lib/TorperfResult: long SortedMap<Integer, Long> getDataPercentiles().get(80)"
  }, {
    "name" : "dataperc90",
    "type" : [ "null", "long" ],
    "doc" : "metrics-lib/TorperfResult: long SortedMap<Integer, Long> getDataPercentiles().get(90)"
  }, {
    "name" : "launch",
    "type" : [ "null", "long" ],
    "doc" : "metrics-lib/TorperfResult: long getLaunchMillis()()"
  }, {
    "name" : "used_at",
    "type" : [ "null", "long" ],
    "doc" : "metrics-lib/TorperfResult: long getUsedAtMillis()"
  }, {
    "name" : "path",
    "type" : [ "null", {
      "type" : "array",
      "items" : "string"
    } ],
    "doc" : "metrics-lib/TorperfResult: List<String> getPath()"
  }, {
    "name" : "buildtimes",
    "type" : [ "null", {
      "type" : "array",
      "items" : "long"
    } ],
    "doc" : "metrics-lib/TorperfResult: List<Long> getBuildTimes()"
  }, {
    "name" : "timeout",
    "type" : [ "null", "long" ],
    "doc" : "metrics-lib/TorperfResult: long getTimeout()"
  }, {
    "name" : "quantile",
    "type" : [ "null", "double" ],
    "doc" : "metrics-lib/TorperfResult: double getQuantile()"
  }, {
    "name" : "circ_id",
    "type" : [ "null", "int" ],
    "doc" : "metrics-lib/TorperfResult: int getCircId()"
  }, {
    "name" : "used_by",
    "type" : [ "null", "int" ],
    "doc" : "metrics-lib/TorperfResult: int getUsedBy()"
  } ],
  "aliases" : [ "torperfResult" ]
}














> rb
> 
> On Wed, Mar 2, 2016 at 7:23 AM, tl <tl...@rat.io> wrote:
> 
>> Hi,
>> 
>> this is a bug report. I assume that this is _not_ the right place to
>> report bugs but I really can’t find where to file it on [0]. If somebody
>> can show me where to click there (or anywhere else) I’ll happily do that.
>> Until then I’ll have at least posted it here (and off my chest).
>> 
>> 
>> Avro:
>> NullPointerException from Json.ObjectWriter
>> 
>> 
>> Environment:
>> avro 1.8.0, jackson-core-asl 1.9.13, jackson-mapper-asl 1.9.13, Java 7,
>> Mac OS X 10.11.3
>> 
>> 
>> Description:
>> The complete, slightly more involved code is on [1], especially [2], but
>> the relevant parts are as follows:
>> 
>> // parsing the schema
>> Schema.Parser parser = new Schema.Parser();
>> Schema schema = parser.parse(new File("schema.avsc")) ;
>> // setting up the encoder and driver
>> Json.ObjectWriter jsonDatumWriter = new Json.ObjectWriter();
>> OutputStream output = new FileOutputStream(new File("output.json"));
>> Encoder encoder = EncoderFactory.get().jsonEncoder(schema, output, pretty);
>> // writing
>> System.out.println(payload); // reassuring test the payload is intact
>> jsonDatumWriter.write(payload, encoder);
>> 
>> 
>> The console will print out a nice JSON string (the payload), followed by
>> the following exception:
>> Exception in thread "main" java.lang.NullPointerException
>> at org.apache.avro.data.Json.write(Json.java:183)
>> at org.apache.avro.data.Json.writeObject(Json.java:272)
>> at org.apache.avro.data.Json.access$000(Json.java:48)
>> at org.apache.avro.data.Json$ObjectWriter.write(Json.java:122)
>> at converTor.WriterObject.append(WriterObject.java:59)
>> at converTor.ConverTor.main(ConverTor.java:251)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:606)
>> at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
>> 
>> 
>> 
>> My beginners guess is that the source of the problem is a call of
>> asToken() in org.codehaus.jackson.JsonNode, which is abstract.
>> 
>> 
>> Regards,
>> Thomas
>> 
>> 
>> [0]
>> https://issues.apache.org/jira/browse/avro/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel
>> [1] https://github.com/tomlurge/converTor
>> [2]
>> https://github.com/tomlurge/converTor/blob/master/src/converTor/WriterObject.java
> 
> 
> 
> 
> -- 
> Ryan Blue
> Software Engineer
> Netflix





< he not busy being born is busy dying >

Re: NullPointerException from Json.ObjectWriter

Posted by Ryan Blue <rb...@netflix.com.INVALID>.
Thanks for the detailed report, Thomas. Sending an e-mail here is fine, but
you can also open issues in the tracker here:

  https://issues.apache.org/jira/browse/AVRO

Could you send the schema you're using? It may be that your schema doesn't
allow null, but you're passing one in. I'm not too familiar with the JSON
encoder so that will help me take a look. By the way, I recommend using the
binary encoder to the JSON encoder, but it's up to you.

rb

On Wed, Mar 2, 2016 at 7:23 AM, tl <tl...@rat.io> wrote:

> Hi,
>
> this is a bug report. I assume that this is _not_ the right place to
> report bugs but I really can’t find where to file it on [0]. If somebody
> can show me where to click there (or anywhere else) I’ll happily do that.
> Until then I’ll have at least posted it here (and off my chest).
>
>
> Avro:
> NullPointerException from Json.ObjectWriter
>
>
> Environment:
> avro 1.8.0, jackson-core-asl 1.9.13, jackson-mapper-asl 1.9.13, Java 7,
> Mac OS X 10.11.3
>
>
> Description:
> The complete, slightly more involved code is on [1], especially [2], but
> the relevant parts are as follows:
>
> // parsing the schema
> Schema.Parser parser = new Schema.Parser();
> Schema schema = parser.parse(new File("schema.avsc")) ;
> // setting up the encoder and driver
> Json.ObjectWriter jsonDatumWriter = new Json.ObjectWriter();
> OutputStream output = new FileOutputStream(new File("output.json"));
> Encoder encoder = EncoderFactory.get().jsonEncoder(schema, output, pretty);
> // writing
> System.out.println(payload); // reassuring test the payload is intact
> jsonDatumWriter.write(payload, encoder);
>
>
> The console will print out a nice JSON string (the payload), followed by
> the following exception:
> Exception in thread "main" java.lang.NullPointerException
> at org.apache.avro.data.Json.write(Json.java:183)
> at org.apache.avro.data.Json.writeObject(Json.java:272)
> at org.apache.avro.data.Json.access$000(Json.java:48)
> at org.apache.avro.data.Json$ObjectWriter.write(Json.java:122)
> at converTor.WriterObject.append(WriterObject.java:59)
> at converTor.ConverTor.main(ConverTor.java:251)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
>
>
>
> My beginners guess is that the source of the problem is a call of
> asToken() in org.codehaus.jackson.JsonNode, which is abstract.
>
>
> Regards,
> Thomas
>
>
> [0]
> https://issues.apache.org/jira/browse/avro/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel
> [1] https://github.com/tomlurge/converTor
> [2]
> https://github.com/tomlurge/converTor/blob/master/src/converTor/WriterObject.java




-- 
Ryan Blue
Software Engineer
Netflix