You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@nifi.apache.org by "Joseph Percivall (JIRA)" <ji...@apache.org> on 2015/12/04 20:14:11 UTC
[jira] [Created] (NIFI-1253) ConvertJSONToAvro doesn't fully
validate schema before scheduling
Joseph Percivall created NIFI-1253:
--------------------------------------
Summary: ConvertJSONToAvro doesn't fully validate schema before scheduling
Key: NIFI-1253
URL: https://issues.apache.org/jira/browse/NIFI-1253
Project: Apache NiFi
Issue Type: Improvement
Reporter: Joseph Percivall
Priority: Minor
ConvertJSONToAvro takes in a JSON file and has an Avro schema as a parameter. You can create a "valid" schema (as verified here[1]) but once the processor is triggered it encounters an IllegalArgumentException, stack trace below. I used this schema:
{noformat}
{
"name" :"results",
"type" : "array",
"items" :
{ "name" : "user",
"type" : "record",
"fields" : [
{"name": "gender", "type" : "string"},
{
"name" : "name",
"type" : {
"name": "name",
"type" : "record",
"fields" : [
{"name" : "title", "type" : "string"},
{"name" : "first", "type" : "string"},
{"name" : "last", "type" : "string"}
]
}
}
]
}
}
{noformat}
{noformat}
java.lang.IllegalArgumentException: Schemas for JSON files should be record
at org.kitesdk.shaded.com.google.common.base.Preconditions.checkArgument(Preconditions.java:88) ~[na:na]
at org.kitesdk.data.spi.filesystem.JSONFileReader.initialize(JSONFileReader.java:84) ~[na:na]
at org.apache.nifi.processors.kite.ConvertJSONToAvro$1.process(ConvertJSONToAvro.java:144) ~[na:na]
at org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2171) ~[nifi-framework-core-0.4.0-SNAPSHOT.jar:0.4.0-SNAPSHOT]
at org.apache.nifi.processors.kite.ConvertJSONToAvro.onTrigger(ConvertJSONToAvro.java:139) ~[na:na]
at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) ~[nifi-api-0.4.0-SNAPSHOT.jar:0.4.0-SNAPSHOT]
at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1146) ~[nifi-framework-core-0.4.0-SNAPSHOT.jar:0.4.0-SNAPSHOT]
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139) [nifi-framework-core-0.4.0-SNAPSHOT.jar:0.4.0-SNAPSHOT]
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49) [nifi-framework-core-0.4.0-SNAPSHOT.jar:0.4.0-SNAPSHOT]
at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119) [nifi-framework-core-0.4.0-SNAPSHOT.jar:0.4.0-SNAPSHOT]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_60]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_60]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_60]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_60]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_60]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_60]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60]
{noformat}
[1] https://json-schema-validator.herokuapp.com/avro.jsp
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)