Data Manipulation Language for Avro, JSON, CSV, etc

[Toivo Adams <to...@gmail.com>]

Hi

from user mailing list:
nifi processor to parse+update the current json on the fly
NIFI-361
Sometimes I miss DML in Java code.
I mean something like SQL
Maybe we can use DML in NiFi?
Some universal DML for any data? Be it Avro, JSON, CSV, etc.
DML should handle any tabular like data.
And not only filter, update single table, but also join, merge multiple
tables.
Apache Calcite looks very promising.

thanks
toivo






--
View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/Data-Manipulation-Language-for-Avro-JSON-CSV-etc-tp8967.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

Re: Data Manipulation Language for Avro, JSON, CSV, etc

[Toivo Adams <to...@gmail.com>]

Hi
Working CSV processor is now done.
SQL select statement can be used to manipulate CSV data.
Input, CSV file:

ID:int,AMOUNT1:float,AMOUNT2:float,AMOUNT3:float
008, 10.05, 15.45, 89.99
100, 20.25, 25.25, 45.25
200, 34.05, 25.05, 75.05

select ID, AMOUNT1+AMOUNT2+AMOUNT3 as TOTAL from CSV.A where ID=100

result:
ID      TOTAL
100  90.75

https://issues.apache.org/jira/browse/NIFI-1280
Please review.
Any feedback is highly welcome.

Thanks
toivo




--
View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/Data-Manipulation-Language-for-Avro-JSON-CSV-etc-tp8967p10080.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.