You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "Claudiu Stanciu (JIRA)" <ji...@apache.org> on 2017/07/14 10:42:00 UTC
[jira] [Comment Edited] (NIFI-2726) Create a CSVToOrc Processor
[ https://issues.apache.org/jira/browse/NIFI-2726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16087165#comment-16087165 ]
Claudiu Stanciu edited comment on NIFI-2726 at 7/14/17 10:41 AM:
-----------------------------------------------------------------
Hi, as mentioned in https://github.com/apache/nifi/pull/1964
With the release of Apache NiFi 1.2.0 and 1.3.0, the record paradigm has been introduced to better support protocol/format/schema conversion. This moves away from the ConvertXToY type processors in favor of providing custom record reader and recordset writer controller service implementations which can be leveraged and re-used by LookupRecord, ConvertRecord, PartitionRecord, QueryRecord, MergeRecord, UpdateRecord, etc. There is already a JsonTreeReader, JsonPathReader, and JsonRecordSetWriter available, so if you were to provide an OrcReader and OrcRecordSetWriter, you would be able to perform bidirectional conversion of JSON to/from Orc, along with Grok, CSV, Avro, freeform text, and scripted implementations.
was (Author: claudiu-stanciu):
As mentioned in https://github.com/apache/nifi/pull/1964
With the release of Apache NiFi 1.2.0 and 1.3.0, the record paradigm has been introduced to better support protocol/format/schema conversion. This moves away from the ConvertXToY type processors in favor of providing custom record reader and recordset writer controller service implementations which can be leveraged and re-used by LookupRecord, ConvertRecord, PartitionRecord, QueryRecord, MergeRecord, UpdateRecord, etc. There is already a JsonTreeReader, JsonPathReader, and JsonRecordSetWriter available, so if you were to provide an OrcReader and OrcRecordSetWriter, you would be able to perform bidirectional conversion of JSON to/from Orc, along with Grok, CSV, Avro, freeform text, and scripted implementations.
> Create a CSVToOrc Processor
> ---------------------------
>
> Key: NIFI-2726
> URL: https://issues.apache.org/jira/browse/NIFI-2726
> Project: Apache NiFi
> Issue Type: Improvement
> Affects Versions: 1.0.0
> Reporter: Andrew Psaltis
>
> Today the only way to get to ORC is via Avro. If my data comes in as CSV then I have to transform it to another format before going to ORC. It would be cleaner if the data can go direct to ORC from CSV
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)