You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by Eric Chaves <er...@uolet.com> on 2019/09/18 13:58:45 UTC
Validate CSV/Records by name instead of position
Hi folks,
Is it possible to validate fields/columns in Record or CSV by its name
instead of it's position? For example I have a record with two mandatory
fields and some optional fields but they may be on different position on
each ingested file. Should I use a script or there is already a processor
that could help me out with those?
Regards,
Re: Validate CSV/Records by name instead of position
Posted by Eric Chaves <er...@uolet.com>.
Awesome, thanks for the detailed steps!
Em qua, 18 de set de 2019 às 11:41, Jerry Vinokurov <gr...@gmail.com>
escreveu:
> This certainly works. You can create a schema registry and define an Avro
> schema listing your fields. Then make sure that when you set up the reader,
> it's configured to read the header so that it knows which fields go where
> in the record, set up the mode of the schema access to read from the
> registry you created, and then set the name of the actual schema, which is
> a property on the schema registry. This will correctly validate your CSV
> for you.
>
> On Wed, Sep 18, 2019 at 9:59 AM Eric Chaves <er...@uolet.com> wrote:
>
>> Hi folks,
>>
>> Is it possible to validate fields/columns in Record or CSV by its name
>> instead of it's position? For example I have a record with two mandatory
>> fields and some optional fields but they may be on different position on
>> each ingested file. Should I use a script or there is already a processor
>> that could help me out with those?
>>
>> Regards,
>>
>
>
> --
> http://www.google.com/profiles/grapesmoker
>
Re: Validate CSV/Records by name instead of position
Posted by Jerry Vinokurov <gr...@gmail.com>.
This certainly works. You can create a schema registry and define an Avro
schema listing your fields. Then make sure that when you set up the reader,
it's configured to read the header so that it knows which fields go where
in the record, set up the mode of the schema access to read from the
registry you created, and then set the name of the actual schema, which is
a property on the schema registry. This will correctly validate your CSV
for you.
On Wed, Sep 18, 2019 at 9:59 AM Eric Chaves <er...@uolet.com> wrote:
> Hi folks,
>
> Is it possible to validate fields/columns in Record or CSV by its name
> instead of it's position? For example I have a record with two mandatory
> fields and some optional fields but they may be on different position on
> each ingested file. Should I use a script or there is already a processor
> that could help me out with those?
>
> Regards,
>
--
http://www.google.com/profiles/grapesmoker