You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by ericsm <ec...@sessionm.com> on 2017/12/13 16:26:02 UTC

How to generate json with array from rows

Hi,

I'm receiving an orders csv, with a single order spanning multiple rows (one
item per row):
order_number, name
123, item_1
123, item_2

I need to convert this to a json, whose simplified format is:
{
"order_number": "123",
"items": [
{ "name": "item_1" },
{ "name": "item_2" }
]
}

Currently to do this I'm passing the entire order CSV into an ExecuteScript
processor, but would like to know if there is a more 'nifi' way to do this.
It seems the first step would be to get nifi to recognize an arbitrary
number of rows as the same record (based on changing order_number), but I
don't see an obvious way to do that.

Thanks in advance,

Eric



--
Sent from: http://apache-nifi-users-list.2361937.n4.nabble.com/

Re: How to generate json with array from rows

Posted by Andy LoPresto <al...@apache.org>.
Eric,

I’d recommend using the ConvertRecord processor and the “record” paradigm introduced in Apache NiFi 1.2.0. You can read all about it in a series of blog posts, but essentially you would use a ConvertRecord processor with a CsvRecordReader controller service to translate the incoming rows into a generic internal “record” representation in NiFi, and a JsonRecordSetWriter to then output those records in JSON format. A simple follow-on ReplaceText can add the wrapping syntax around the output. I do believe there is a future effort to also provide a configuration in the JSON writer to indicate whether each record should be written as a standalone JSON document (existing behavior) or as an element in an array (what you are looking for but not currently offered).

[1] https://blogs.apache.org/nifi/entry/record-oriented-data-with-nifi
[2] https://blogs.apache.org/nifi/entry/real-time-sql-on-event
[3] https://bryanbende.com/development/2017/06/20/apache-nifi-records-and-schema-registries

Andy LoPresto
alopresto@apache.org
alopresto.apache@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Dec 13, 2017, at 8:26 AM, ericsm <ec...@sessionm.com> wrote:
> 
> Hi,
> 
> I'm receiving an orders csv, with a single order spanning multiple rows (one
> item per row):
> order_number, name
> 123, item_1
> 123, item_2
> 
> I need to convert this to a json, whose simplified format is:
> {
> "order_number": "123",
> "items": [
> { "name": "item_1" },
> { "name": "item_2" }
> ]
> }
> 
> Currently to do this I'm passing the entire order CSV into an ExecuteScript
> processor, but would like to know if there is a more 'nifi' way to do this.
> It seems the first step would be to get nifi to recognize an arbitrary
> number of rows as the same record (based on changing order_number), but I
> don't see an obvious way to do that.
> 
> Thanks in advance,
> 
> Eric
> 
> 
> 
> --
> Sent from: http://apache-nifi-users-list.2361937.n4.nabble.com/