You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by l vic <lv...@gmail.com> on 2018/08/30 08:51:21 UTC
Nifi RecordReader for Json?
I need to save two different json messages according to json schemas
available for each to different relational database tables.
I saw this blog:
https://blogs.apache.org/nifi/entry/record-oriented-data-with-nifi
with example using CSVRecordReader for csv->json transformation.
but what would be RecordReader for schema-based transformation from json?
Is this a valid approach, or what would be best approach to solve this
problem?
I am using: nifi-1.7.1-RC1...
Thank you,
Re: Nifi RecordReader for Json?
Posted by Joe Witt <jo...@gmail.com>.
hello v
there are some excellent blogs on
nifi record reader
and related items. you definitely want to check them out to understand how
it works.
https://blogs.apache.org/nifi/entry/record-oriented-data-with-nifi
thanks
On Thu, Aug 30, 2018, 8:48 AM l vic <lv...@gmail.com> wrote:
> So, where's JsonTreeReader? I am on nifi-1.7.1-RC1 and i don't see it in
> the list of available processors...
> Thanks,
> V
>
> On Thu, Aug 30, 2018 at 5:31 AM Sivaprasanna <si...@gmail.com>
> wrote:
>
>> Hi. Just like CSVRecordReader, we have record reader service for JSON.
>> It's called JsonTreeReader. You can use AvroSchemaRegistry and provide an
>> Avro schema (usually generated through InferAvroSchema processor) for your
>> JSON. Refer:
>> https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-record-serialization-services-nar/1.7.1/org.apache.nifi.json.JsonTreeReader/index.html
>>
>> -
>> Sivaprasanna
>>
>> On Thu, 30 Aug 2018 at 2:21 PM, l vic <lv...@gmail.com> wrote:
>>
>>> I need to save two different json messages according to json schemas
>>> available for each to different relational database tables.
>>> I saw this blog:
>>> https://blogs.apache.org/nifi/entry/record-oriented-data-with-nifi
>>> with example using CSVRecordReader for csv->json transformation.
>>> but what would be RecordReader for schema-based transformation from
>>> json? Is this a valid approach, or what would be best approach to solve
>>> this problem?
>>> I am using: nifi-1.7.1-RC1...
>>> Thank you,
>>>
>>
Re: Nifi RecordReader for Json?
Posted by l vic <lv...@gmail.com>.
What would be the right processor to use JsonTreeReader to get json
attributes as name/value pairs and to write them into the file?
Thanks again,
On Thu, Aug 30, 2018 at 2:09 PM Otto Fowler <ot...@gmail.com> wrote:
> What he said.
>
>
> On August 30, 2018 at 13:31:54, Joe Witt (joe.witt@gmail.com) wrote:
>
> Wow!
>
> My apologies for the really bad response I gave linking to the same
> article you mentioned. I should be more careful when
> reading/responding on the phone!
>
> Thanks
> On Thu, Aug 30, 2018 at 11:01 AM Matt Burgess <ma...@apache.org>
> wrote:
> >
> > V,
> >
> > Currently NiFi does not support specifying a schema in JSONSchema
> > format, you'll want to convert that to an Avro schema for use in
> > JsonTreeReader. I don't know JSONSchema that well so I'm not sure if
> > that "stats" schema is supposed to be included in the outgoing object.
> > I ran it through a utility-under-development [1] and got the following
> > Avro schema out:
> >
> >
> {"type":"record","name":"record0","fields":[{"name":"create_date","type":"long"},{"name":"id","type":"string"}]}
>
> >
> > This doesn't add the "stats" record and specifies "create_date" as a
> > long versus a BigInteger. I think you might want to use an Avro
> > logical type of "decimal" [2] depending on what the value is in the
> > actual JSON object:
> >
> > {"type":"record","name":"record0","fields":[
> > {"name":"create_date","type": {"type": "bytes","logicalType":
> > "decimal","precision": 12,"scale": 0}},
> > {"name":"id","type":"string"}
> > ]}
> >
> > If you have a stats object present, this might work:
> >
> > {"type":"record","name":"record0","fields":[
> > {"name": "stats", "type" :
> >
> {"type":"record","name":"statsRecord","fields":[{"name":"id","type":"string"},{"name":"bin_qualifier","type":"string"}]}},
>
> > {"name":"create_date","type": {"type": "bytes","logicalType":
> > "decimal","precision": 12,"scale": 0}},
> > {"name":"id","type":"string"}
> > ]}
> >
> > I didn't validate or try these so there may be typos or other
> > (hopefully minor) mistakes.
> >
> > Regards,
> > Matt
> >
> > [1] https://github.com/fge/json-schema-avro
> > [2] https://avro.apache.org/docs/1.8.2/spec.html#Decimal
> >
> > On Thu, Aug 30, 2018 at 9:54 AM l vic <lv...@gmail.com> wrote:
> > >
> > > I have json file for the schema that looks like the following:
> > >
> > > {
> > > "$schema": "http://json-schema.org/draft-04/schema#",
> > > "definitions": {
> > > "stats": {
> > > "type": "object",
> > > "additionalProperties": false,
> > > "properties": {
> > > "id": {
> > > "type": "string"
> > > },
> > > "bin_qualifier": {
> > > "type": "string"
> > > }
> > > }
> > > }
> > > },
> > > "additionalProperties": false,
> > > "description": "attributes",
> > > "type": "object",
> > > "properties": {
> > > "id": {
> > > "type": "string",
> > > "required": true,
> > > },
> > > "create_date": {
> > > "type": "integer",
> > > "javaType": "java.math.BigInteger",
> > > "required": true
> > > }
> > > }
> > > }
> > >
> > >
> > > How can I add this schema for JsonTreeReader?
> > >
> > > On Thu, Aug 30, 2018 at 9:02 AM Otto Fowler <ot...@gmail.com>
> wrote:
> > >>
> > >> The record readers are services, that processors use.
> > >> When you use a *Record* processor, you will have to select a Reader
> and a Writer Service, or create one ( which you can do through the UI ).
> > >> https://blogs.apache.org/nifi/entry/record-oriented-data-with-nifi
> > >>
> > >>
> > >> On August 30, 2018 at 08:48:08, l vic (lvic4594@gmail.com) wrote:
> > >>
> > >> So, where's JsonTreeReader? I am on nifi-1.7.1-RC1 and i don't see it
> in the list of available processors...
> > >> Thanks,
> > >> V
> > >>
> > >> On Thu, Aug 30, 2018 at 5:31 AM Sivaprasanna <
> sivaprasanna246@gmail.com> wrote:
> > >>>
> > >>> Hi. Just like CSVRecordReader, we have record reader service for
> JSON. It's called JsonTreeReader. You can use AvroSchemaRegistry and
> provide an Avro schema (usually generated through InferAvroSchema
> processor) for your JSON. Refer:
> https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-record-serialization-services-nar/1.7.1/org.apache.nifi.json.JsonTreeReader/index.html
> > >>>
> > >>> -
> > >>> Sivaprasanna
> > >>>
> > >>> On Thu, 30 Aug 2018 at 2:21 PM, l vic <lv...@gmail.com> wrote:
> > >>>>
> > >>>> I need to save two different json messages according to json
> schemas available for each to different relational database tables.
> > >>>> I saw this blog:
> > >>>> https://blogs.apache.org/nifi/entry/record-oriented-data-with-nifi
> > >>>> with example using CSVRecordReader for csv->json transformation.
> > >>>> but what would be RecordReader for schema-based transformation from
> json? Is this a valid approach, or what would be best approach to solve
> this problem?
> > >>>> I am using: nifi-1.7.1-RC1...
> > >>>> Thank you,
>
>
Re: Nifi RecordReader for Json?
Posted by Otto Fowler <ot...@gmail.com>.
What he said.
On August 30, 2018 at 13:31:54, Joe Witt (joe.witt@gmail.com) wrote:
Wow!
My apologies for the really bad response I gave linking to the same
article you mentioned. I should be more careful when
reading/responding on the phone!
Thanks
On Thu, Aug 30, 2018 at 11:01 AM Matt Burgess <ma...@apache.org> wrote:
>
> V,
>
> Currently NiFi does not support specifying a schema in JSONSchema
> format, you'll want to convert that to an Avro schema for use in
> JsonTreeReader. I don't know JSONSchema that well so I'm not sure if
> that "stats" schema is supposed to be included in the outgoing object.
> I ran it through a utility-under-development [1] and got the following
> Avro schema out:
>
>
{"type":"record","name":"record0","fields":[{"name":"create_date","type":"long"},{"name":"id","type":"string"}]}
>
> This doesn't add the "stats" record and specifies "create_date" as a
> long versus a BigInteger. I think you might want to use an Avro
> logical type of "decimal" [2] depending on what the value is in the
> actual JSON object:
>
> {"type":"record","name":"record0","fields":[
> {"name":"create_date","type": {"type": "bytes","logicalType":
> "decimal","precision": 12,"scale": 0}},
> {"name":"id","type":"string"}
> ]}
>
> If you have a stats object present, this might work:
>
> {"type":"record","name":"record0","fields":[
> {"name": "stats", "type" :
>
{"type":"record","name":"statsRecord","fields":[{"name":"id","type":"string"},{"name":"bin_qualifier","type":"string"}]}},
> {"name":"create_date","type": {"type": "bytes","logicalType":
> "decimal","precision": 12,"scale": 0}},
> {"name":"id","type":"string"}
> ]}
>
> I didn't validate or try these so there may be typos or other
> (hopefully minor) mistakes.
>
> Regards,
> Matt
>
> [1] https://github.com/fge/json-schema-avro
> [2] https://avro.apache.org/docs/1.8.2/spec.html#Decimal
>
> On Thu, Aug 30, 2018 at 9:54 AM l vic <lv...@gmail.com> wrote:
> >
> > I have json file for the schema that looks like the following:
> >
> > {
> > "$schema": "http://json-schema.org/draft-04/schema#",
> > "definitions": {
> > "stats": {
> > "type": "object",
> > "additionalProperties": false,
> > "properties": {
> > "id": {
> > "type": "string"
> > },
> > "bin_qualifier": {
> > "type": "string"
> > }
> > }
> > }
> > },
> > "additionalProperties": false,
> > "description": "attributes",
> > "type": "object",
> > "properties": {
> > "id": {
> > "type": "string",
> > "required": true,
> > },
> > "create_date": {
> > "type": "integer",
> > "javaType": "java.math.BigInteger",
> > "required": true
> > }
> > }
> > }
> >
> >
> > How can I add this schema for JsonTreeReader?
> >
> > On Thu, Aug 30, 2018 at 9:02 AM Otto Fowler <ot...@gmail.com>
wrote:
> >>
> >> The record readers are services, that processors use.
> >> When you use a *Record* processor, you will have to select a Reader
and a Writer Service, or create one ( which you can do through the UI ).
> >> https://blogs.apache.org/nifi/entry/record-oriented-data-with-nifi
> >>
> >>
> >> On August 30, 2018 at 08:48:08, l vic (lvic4594@gmail.com) wrote:
> >>
> >> So, where's JsonTreeReader? I am on nifi-1.7.1-RC1 and i don't see it
in the list of available processors...
> >> Thanks,
> >> V
> >>
> >> On Thu, Aug 30, 2018 at 5:31 AM Sivaprasanna <si...@gmail.com>
wrote:
> >>>
> >>> Hi. Just like CSVRecordReader, we have record reader service for
JSON. It's called JsonTreeReader. You can use AvroSchemaRegistry and
provide an Avro schema (usually generated through InferAvroSchema
processor) for your JSON. Refer:
https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-record-serialization-services-nar/1.7.1/org.apache.nifi.json.JsonTreeReader/index.html
> >>>
> >>> -
> >>> Sivaprasanna
> >>>
> >>> On Thu, 30 Aug 2018 at 2:21 PM, l vic <lv...@gmail.com> wrote:
> >>>>
> >>>> I need to save two different json messages according to json schemas
available for each to different relational database tables.
> >>>> I saw this blog:
> >>>> https://blogs.apache.org/nifi/entry/record-oriented-data-with-nifi
> >>>> with example using CSVRecordReader for csv->json transformation.
> >>>> but what would be RecordReader for schema-based transformation from
json? Is this a valid approach, or what would be best approach to solve
this problem?
> >>>> I am using: nifi-1.7.1-RC1...
> >>>> Thank you,
Re: Nifi RecordReader for Json?
Posted by Joe Witt <jo...@gmail.com>.
Wow!
My apologies for the really bad response I gave linking to the same
article you mentioned. I should be more careful when
reading/responding on the phone!
Thanks
On Thu, Aug 30, 2018 at 11:01 AM Matt Burgess <ma...@apache.org> wrote:
>
> V,
>
> Currently NiFi does not support specifying a schema in JSONSchema
> format, you'll want to convert that to an Avro schema for use in
> JsonTreeReader. I don't know JSONSchema that well so I'm not sure if
> that "stats" schema is supposed to be included in the outgoing object.
> I ran it through a utility-under-development [1] and got the following
> Avro schema out:
>
> {"type":"record","name":"record0","fields":[{"name":"create_date","type":"long"},{"name":"id","type":"string"}]}
>
> This doesn't add the "stats" record and specifies "create_date" as a
> long versus a BigInteger. I think you might want to use an Avro
> logical type of "decimal" [2] depending on what the value is in the
> actual JSON object:
>
> {"type":"record","name":"record0","fields":[
> {"name":"create_date","type": {"type": "bytes","logicalType":
> "decimal","precision": 12,"scale": 0}},
> {"name":"id","type":"string"}
> ]}
>
> If you have a stats object present, this might work:
>
> {"type":"record","name":"record0","fields":[
> {"name": "stats", "type" :
> {"type":"record","name":"statsRecord","fields":[{"name":"id","type":"string"},{"name":"bin_qualifier","type":"string"}]}},
> {"name":"create_date","type": {"type": "bytes","logicalType":
> "decimal","precision": 12,"scale": 0}},
> {"name":"id","type":"string"}
> ]}
>
> I didn't validate or try these so there may be typos or other
> (hopefully minor) mistakes.
>
> Regards,
> Matt
>
> [1] https://github.com/fge/json-schema-avro
> [2] https://avro.apache.org/docs/1.8.2/spec.html#Decimal
>
> On Thu, Aug 30, 2018 at 9:54 AM l vic <lv...@gmail.com> wrote:
> >
> > I have json file for the schema that looks like the following:
> >
> > {
> > "$schema": "http://json-schema.org/draft-04/schema#",
> > "definitions": {
> > "stats": {
> > "type": "object",
> > "additionalProperties": false,
> > "properties": {
> > "id": {
> > "type": "string"
> > },
> > "bin_qualifier": {
> > "type": "string"
> > }
> > }
> > }
> > },
> > "additionalProperties": false,
> > "description": "attributes",
> > "type": "object",
> > "properties": {
> > "id": {
> > "type": "string",
> > "required": true,
> > },
> > "create_date": {
> > "type": "integer",
> > "javaType": "java.math.BigInteger",
> > "required": true
> > }
> > }
> > }
> >
> >
> > How can I add this schema for JsonTreeReader?
> >
> > On Thu, Aug 30, 2018 at 9:02 AM Otto Fowler <ot...@gmail.com> wrote:
> >>
> >> The record readers are services, that processors use.
> >> When you use a *Record* processor, you will have to select a Reader and a Writer Service, or create one ( which you can do through the UI ).
> >> https://blogs.apache.org/nifi/entry/record-oriented-data-with-nifi
> >>
> >>
> >> On August 30, 2018 at 08:48:08, l vic (lvic4594@gmail.com) wrote:
> >>
> >> So, where's JsonTreeReader? I am on nifi-1.7.1-RC1 and i don't see it in the list of available processors...
> >> Thanks,
> >> V
> >>
> >> On Thu, Aug 30, 2018 at 5:31 AM Sivaprasanna <si...@gmail.com> wrote:
> >>>
> >>> Hi. Just like CSVRecordReader, we have record reader service for JSON. It's called JsonTreeReader. You can use AvroSchemaRegistry and provide an Avro schema (usually generated through InferAvroSchema processor) for your JSON. Refer: https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-record-serialization-services-nar/1.7.1/org.apache.nifi.json.JsonTreeReader/index.html
> >>>
> >>> -
> >>> Sivaprasanna
> >>>
> >>> On Thu, 30 Aug 2018 at 2:21 PM, l vic <lv...@gmail.com> wrote:
> >>>>
> >>>> I need to save two different json messages according to json schemas available for each to different relational database tables.
> >>>> I saw this blog:
> >>>> https://blogs.apache.org/nifi/entry/record-oriented-data-with-nifi
> >>>> with example using CSVRecordReader for csv->json transformation.
> >>>> but what would be RecordReader for schema-based transformation from json? Is this a valid approach, or what would be best approach to solve this problem?
> >>>> I am using: nifi-1.7.1-RC1...
> >>>> Thank you,
Re: Nifi RecordReader for Json?
Posted by Matt Burgess <ma...@apache.org>.
V,
Currently NiFi does not support specifying a schema in JSONSchema
format, you'll want to convert that to an Avro schema for use in
JsonTreeReader. I don't know JSONSchema that well so I'm not sure if
that "stats" schema is supposed to be included in the outgoing object.
I ran it through a utility-under-development [1] and got the following
Avro schema out:
{"type":"record","name":"record0","fields":[{"name":"create_date","type":"long"},{"name":"id","type":"string"}]}
This doesn't add the "stats" record and specifies "create_date" as a
long versus a BigInteger. I think you might want to use an Avro
logical type of "decimal" [2] depending on what the value is in the
actual JSON object:
{"type":"record","name":"record0","fields":[
{"name":"create_date","type": {"type": "bytes","logicalType":
"decimal","precision": 12,"scale": 0}},
{"name":"id","type":"string"}
]}
If you have a stats object present, this might work:
{"type":"record","name":"record0","fields":[
{"name": "stats", "type" :
{"type":"record","name":"statsRecord","fields":[{"name":"id","type":"string"},{"name":"bin_qualifier","type":"string"}]}},
{"name":"create_date","type": {"type": "bytes","logicalType":
"decimal","precision": 12,"scale": 0}},
{"name":"id","type":"string"}
]}
I didn't validate or try these so there may be typos or other
(hopefully minor) mistakes.
Regards,
Matt
[1] https://github.com/fge/json-schema-avro
[2] https://avro.apache.org/docs/1.8.2/spec.html#Decimal
On Thu, Aug 30, 2018 at 9:54 AM l vic <lv...@gmail.com> wrote:
>
> I have json file for the schema that looks like the following:
>
> {
> "$schema": "http://json-schema.org/draft-04/schema#",
> "definitions": {
> "stats": {
> "type": "object",
> "additionalProperties": false,
> "properties": {
> "id": {
> "type": "string"
> },
> "bin_qualifier": {
> "type": "string"
> }
> }
> }
> },
> "additionalProperties": false,
> "description": "attributes",
> "type": "object",
> "properties": {
> "id": {
> "type": "string",
> "required": true,
> },
> "create_date": {
> "type": "integer",
> "javaType": "java.math.BigInteger",
> "required": true
> }
> }
> }
>
>
> How can I add this schema for JsonTreeReader?
>
> On Thu, Aug 30, 2018 at 9:02 AM Otto Fowler <ot...@gmail.com> wrote:
>>
>> The record readers are services, that processors use.
>> When you use a *Record* processor, you will have to select a Reader and a Writer Service, or create one ( which you can do through the UI ).
>> https://blogs.apache.org/nifi/entry/record-oriented-data-with-nifi
>>
>>
>> On August 30, 2018 at 08:48:08, l vic (lvic4594@gmail.com) wrote:
>>
>> So, where's JsonTreeReader? I am on nifi-1.7.1-RC1 and i don't see it in the list of available processors...
>> Thanks,
>> V
>>
>> On Thu, Aug 30, 2018 at 5:31 AM Sivaprasanna <si...@gmail.com> wrote:
>>>
>>> Hi. Just like CSVRecordReader, we have record reader service for JSON. It's called JsonTreeReader. You can use AvroSchemaRegistry and provide an Avro schema (usually generated through InferAvroSchema processor) for your JSON. Refer: https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-record-serialization-services-nar/1.7.1/org.apache.nifi.json.JsonTreeReader/index.html
>>>
>>> -
>>> Sivaprasanna
>>>
>>> On Thu, 30 Aug 2018 at 2:21 PM, l vic <lv...@gmail.com> wrote:
>>>>
>>>> I need to save two different json messages according to json schemas available for each to different relational database tables.
>>>> I saw this blog:
>>>> https://blogs.apache.org/nifi/entry/record-oriented-data-with-nifi
>>>> with example using CSVRecordReader for csv->json transformation.
>>>> but what would be RecordReader for schema-based transformation from json? Is this a valid approach, or what would be best approach to solve this problem?
>>>> I am using: nifi-1.7.1-RC1...
>>>> Thank you,
Re: Nifi RecordReader for Json?
Posted by l vic <lv...@gmail.com>.
I have json file for the schema that looks like the following:
{
"$schema": "http://json-schema.org/draft-04/schema#",
"definitions": {
"stats": {
"type": "object",
"additionalProperties": false,
"properties": {
"id": {
"type": "string"
},
"bin_qualifier": {
"type": "string"
}
}
}
},
"additionalProperties": false,
"description": "attributes",
"type": "object",
"properties": {
"id": {
"type": "string",
"required": true,
},
"create_date": {
"type": "integer",
"javaType": "java.math.BigInteger",
"required": true
}
}
}
How can I add this schema for JsonTreeReader?
On Thu, Aug 30, 2018 at 9:02 AM Otto Fowler <ot...@gmail.com> wrote:
> The record readers are services, that processors use.
> When you use a *Record* processor, you will have to select a Reader and a
> Writer Service, or create one ( which you can do through the UI ).
> https://blogs.apache.org/nifi/entry/record-oriented-data-with-nifi
>
>
> On August 30, 2018 at 08:48:08, l vic (lvic4594@gmail.com) wrote:
>
> So, where's JsonTreeReader? I am on nifi-1.7.1-RC1 and i don't see it in
> the list of available processors...
> Thanks,
> V
>
> On Thu, Aug 30, 2018 at 5:31 AM Sivaprasanna <si...@gmail.com>
> wrote:
>
>> Hi. Just like CSVRecordReader, we have record reader service for JSON.
>> It's called JsonTreeReader. You can use AvroSchemaRegistry and provide an
>> Avro schema (usually generated through InferAvroSchema processor) for your
>> JSON. Refer:
>> https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-record-serialization-services-nar/1.7.1/org.apache.nifi.json.JsonTreeReader/index.html
>>
>> -
>> Sivaprasanna
>>
>> On Thu, 30 Aug 2018 at 2:21 PM, l vic <lv...@gmail.com> wrote:
>>
>>> I need to save two different json messages according to json schemas
>>> available for each to different relational database tables.
>>> I saw this blog:
>>> https://blogs.apache.org/nifi/entry/record-oriented-data-with-nifi
>>> with example using CSVRecordReader for csv->json transformation.
>>> but what would be RecordReader for schema-based transformation from
>>> json? Is this a valid approach, or what would be best approach to solve
>>> this problem?
>>> I am using: nifi-1.7.1-RC1...
>>> Thank you,
>>>
>>
Re: Nifi RecordReader for Json?
Posted by Otto Fowler <ot...@gmail.com>.
The record readers are services, that processors use.
When you use a *Record* processor, you will have to select a Reader and a
Writer Service, or create one ( which you can do through the UI ).
https://blogs.apache.org/nifi/entry/record-oriented-data-with-nifi
On August 30, 2018 at 08:48:08, l vic (lvic4594@gmail.com) wrote:
So, where's JsonTreeReader? I am on nifi-1.7.1-RC1 and i don't see it in
the list of available processors...
Thanks,
V
On Thu, Aug 30, 2018 at 5:31 AM Sivaprasanna <si...@gmail.com>
wrote:
> Hi. Just like CSVRecordReader, we have record reader service for JSON.
> It's called JsonTreeReader. You can use AvroSchemaRegistry and provide an
> Avro schema (usually generated through InferAvroSchema processor) for your
> JSON. Refer:
> https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-record-serialization-services-nar/1.7.1/org.apache.nifi.json.JsonTreeReader/index.html
>
> -
> Sivaprasanna
>
> On Thu, 30 Aug 2018 at 2:21 PM, l vic <lv...@gmail.com> wrote:
>
>> I need to save two different json messages according to json schemas
>> available for each to different relational database tables.
>> I saw this blog:
>> https://blogs.apache.org/nifi/entry/record-oriented-data-with-nifi
>> with example using CSVRecordReader for csv->json transformation.
>> but what would be RecordReader for schema-based transformation from json?
>> Is this a valid approach, or what would be best approach to solve this
>> problem?
>> I am using: nifi-1.7.1-RC1...
>> Thank you,
>>
>
Re: Nifi RecordReader for Json?
Posted by l vic <lv...@gmail.com>.
So, where's JsonTreeReader? I am on nifi-1.7.1-RC1 and i don't see it in
the list of available processors...
Thanks,
V
On Thu, Aug 30, 2018 at 5:31 AM Sivaprasanna <si...@gmail.com>
wrote:
> Hi. Just like CSVRecordReader, we have record reader service for JSON.
> It's called JsonTreeReader. You can use AvroSchemaRegistry and provide an
> Avro schema (usually generated through InferAvroSchema processor) for your
> JSON. Refer:
> https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-record-serialization-services-nar/1.7.1/org.apache.nifi.json.JsonTreeReader/index.html
>
> -
> Sivaprasanna
>
> On Thu, 30 Aug 2018 at 2:21 PM, l vic <lv...@gmail.com> wrote:
>
>> I need to save two different json messages according to json schemas
>> available for each to different relational database tables.
>> I saw this blog:
>> https://blogs.apache.org/nifi/entry/record-oriented-data-with-nifi
>> with example using CSVRecordReader for csv->json transformation.
>> but what would be RecordReader for schema-based transformation from json?
>> Is this a valid approach, or what would be best approach to solve this
>> problem?
>> I am using: nifi-1.7.1-RC1...
>> Thank you,
>>
>
Re: Nifi RecordReader for Json?
Posted by Sivaprasanna <si...@gmail.com>.
Hi. Just like CSVRecordReader, we have record reader service for JSON. It's
called JsonTreeReader. You can use AvroSchemaRegistry and provide an Avro
schema (usually generated through InferAvroSchema processor) for your JSON.
Refer:
https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-record-serialization-services-nar/1.7.1/org.apache.nifi.json.JsonTreeReader/index.html
-
Sivaprasanna
On Thu, 30 Aug 2018 at 2:21 PM, l vic <lv...@gmail.com> wrote:
> I need to save two different json messages according to json schemas
> available for each to different relational database tables.
> I saw this blog:
> https://blogs.apache.org/nifi/entry/record-oriented-data-with-nifi
> with example using CSVRecordReader for csv->json transformation.
> but what would be RecordReader for schema-based transformation from json?
> Is this a valid approach, or what would be best approach to solve this
> problem?
> I am using: nifi-1.7.1-RC1...
> Thank you,
>