You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by Shashi <ss...@gmail.com> on 2016/12/07 12:59:31 UTC
ValidateCSV
Hi Team,
i have a file with header , footer and delimited with '|'.
Assit me on how to provide schema in ValidateCSV processor to validate my
header and footer.
Regards,
Shashi.
--
View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/ValidateCSV-tp14153.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.
Re: ValidateCSV
Posted by Pierre Villard <pi...@gmail.com>.
Shashi,
When right-clicking on a processor, you can go in "usage" to display
processor's documentation. Additionally, this documentation is also
published on the internet. For this processor you have:
https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.ValidateCsv/index.html
On this page you have a link "additional details" that links to a page with
some examples. Hope that helps.
Also, out of curiosity, what kind of header/footer do you envision? do you
have examples in mind?
When you say :
---
File header - service_ID|control_ID|call_date|Delta|gamma|Vega
File footer- Footer|201608230122|437
We want columns to be validated as we validate in Oracle before we insert
data into Oracle tables.
---
What validation are you expecting?
Pierre
2016-12-09 14:39 GMT+01:00 Shashi <ss...@gmail.com>:
> Hi Pierre,
>
> Thanks for the reply.
>
> could you please help me in giving one example.
>
> how to provide schema for validation in schema property of ValidateCsv.
> That helps me to parse my files
>
> Meanwhile i will raise jira ticket to include to validate header and footer
> in validateCSV.
>
> Thanks very much for your cooperation.
>
> Regards,
> Shashi.
>
>
>
> --
> View this message in context: http://apache-nifi-developer-
> list.39713.n7.nabble.com/ValidateCSV-tp14153p14177.html
> Sent from the Apache NiFi Developer List mailing list archive at
> Nabble.com.
>
Re: ValidateCSV
Posted by Shashi <ss...@gmail.com>.
Hi Pierre,
Thanks for the reply.
could you please help me in giving one example.
how to provide schema for validation in schema property of ValidateCsv.
That helps me to parse my files
Meanwhile i will raise jira ticket to include to validate header and footer
in validateCSV.
Thanks very much for your cooperation.
Regards,
Shashi.
--
View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/ValidateCSV-tp14153p14177.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.
Re: ValidateCSV
Posted by Pierre Villard <pi...@gmail.com>.
Hi Shashi,
As James pointed it out, the ValidateCSV processor does have a property
"Header" that says "my CSV data contains a header line, please ignore it
while validating data". That's all. There is nothing to validate the header
itself. And there is nothing about footer handling.
Using other processors, you may want to filter out the header and the
footer and have a specific handling for those lines. Does that make sense?
Feel free to raise JIRA if you think ValidateCSV processor should offer
better support for header/footer.
Pierre
2016-12-08 2:23 GMT+01:00 Shashi <ss...@gmail.com>:
> Hi James,
>
> Thanks for the reply.
>
> In validateCSV processor ,header property has two options like true and
> false.
>
> True - header is considered
> False- header is ignored
>
> I am going with "true" option to validate my file header by providing
> schema in schema property.
>
> Usecase:
>
> File header - service_ID|control_ID|call_date|Delta|gamma|Vega
>
> File footer- Footer|201608230122|437
>
> We want columns to be validated as we validate in Oracle before we insert
> data into Oracle tables.
>
> Regards,
> Shashi.
>
> On Dec 7, 2016 11:33 PM, "James Wing [via Apache NiFi Developer List]" <
> ml-node+s39713n14163h99@n7.nabble.com> wrote:
>
> > Shashi,
> >
> > The ValidateCsv processor's "Header" property allows the header to be
> > ignored in validation, but it does not have a special feature to validate
> > the header itself. I do not believe there is a similar feature for
> > ignoring a footer.
> >
> > Can you expand a bit more on your use case and what kind of validation
> you
> > want to do?
> >
> > Thanks,
> >
> > James
> >
> > On Wed, Dec 7, 2016 at 4:59 AM, Shashi <[hidden email]
> > <http:///user/SendEmail.jtp?type=node&node=14163&i=0>> wrote:
> >
> > > Hi Team,
> > >
> > > i have a file with header , footer and delimited with '|'.
> > >
> > > Assit me on how to provide schema in ValidateCSV processor to validate
> > my
> > > header and footer.
> > >
> > >
> > >
> > > Regards,
> > > Shashi.
> > >
> > >
> > >
> > > --
> > > View this message in context: http://apache-nifi-developer-
> > > list.39713.n7.nabble.com/ValidateCSV-tp14153.html
> > > Sent from the Apache NiFi Developer List mailing list archive at
> > > Nabble.com.
> > >
> >
> >
> > ------------------------------
> > If you reply to this email, your message will be added to the discussion
> > below:
> > http://apache-nifi-developer-list.39713.n7.nabble.com/
> > ValidateCSV-tp14153p14163.html
> > To unsubscribe from ValidateCSV, click here
> > <http://apache-nifi-developer-list.39713.n7.nabble.com/
> template/NamlServlet.jtp?macro=unsubscribe_by_code&node=14153&code=
> c3NoYXNoaWRoYXI4ODhAZ21haWwuY29tfDE0MTUzfDE1NDg3MjgyMjc=>
> > .
> > NAML
> > <http://apache-nifi-developer-list.39713.n7.nabble.com/
> template/NamlServlet.jtp?macro=macro_viewer&id=instant_
> html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.
> BasicNamespace-nabble.view.web.template.NabbleNamespace-
> nabble.view.web.template.NodeNamespace&breadcrumbs=
> notify_subscribers%21nabble%3Aemail.naml-instant_emails%
> 21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
> >
>
>
>
>
> --
> View this message in context: http://apache-nifi-developer-
> list.39713.n7.nabble.com/ValidateCSV-tp14153p14164.html
> Sent from the Apache NiFi Developer List mailing list archive at
> Nabble.com.
>
Re: ValidateCSV
Posted by Shashi <ss...@gmail.com>.
Hi James,
Thanks for the reply.
In validateCSV processor ,header property has two options like true and
false.
True - header is considered
False- header is ignored
I am going with "true" option to validate my file header by providing
schema in schema property.
Usecase:
File header - service_ID|control_ID|call_date|Delta|gamma|Vega
File footer- Footer|201608230122|437
We want columns to be validated as we validate in Oracle before we insert
data into Oracle tables.
Regards,
Shashi.
On Dec 7, 2016 11:33 PM, "James Wing [via Apache NiFi Developer List]" <
ml-node+s39713n14163h99@n7.nabble.com> wrote:
> Shashi,
>
> The ValidateCsv processor's "Header" property allows the header to be
> ignored in validation, but it does not have a special feature to validate
> the header itself. I do not believe there is a similar feature for
> ignoring a footer.
>
> Can you expand a bit more on your use case and what kind of validation you
> want to do?
>
> Thanks,
>
> James
>
> On Wed, Dec 7, 2016 at 4:59 AM, Shashi <[hidden email]
> <http:///user/SendEmail.jtp?type=node&node=14163&i=0>> wrote:
>
> > Hi Team,
> >
> > i have a file with header , footer and delimited with '|'.
> >
> > Assit me on how to provide schema in ValidateCSV processor to validate
> my
> > header and footer.
> >
> >
> >
> > Regards,
> > Shashi.
> >
> >
> >
> > --
> > View this message in context: http://apache-nifi-developer-
> > list.39713.n7.nabble.com/ValidateCSV-tp14153.html
> > Sent from the Apache NiFi Developer List mailing list archive at
> > Nabble.com.
> >
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
> http://apache-nifi-developer-list.39713.n7.nabble.com/
> ValidateCSV-tp14153p14163.html
> To unsubscribe from ValidateCSV, click here
> <http://apache-nifi-developer-list.39713.n7.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=14153&code=c3NoYXNoaWRoYXI4ODhAZ21haWwuY29tfDE0MTUzfDE1NDg3MjgyMjc=>
> .
> NAML
> <http://apache-nifi-developer-list.39713.n7.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/ValidateCSV-tp14153p14164.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.
Re: ValidateCSV
Posted by James Wing <jv...@gmail.com>.
Shashi,
The ValidateCsv processor's "Header" property allows the header to be
ignored in validation, but it does not have a special feature to validate
the header itself. I do not believe there is a similar feature for
ignoring a footer.
Can you expand a bit more on your use case and what kind of validation you
want to do?
Thanks,
James
On Wed, Dec 7, 2016 at 4:59 AM, Shashi <ss...@gmail.com> wrote:
> Hi Team,
>
> i have a file with header , footer and delimited with '|'.
>
> Assit me on how to provide schema in ValidateCSV processor to validate my
> header and footer.
>
>
>
> Regards,
> Shashi.
>
>
>
> --
> View this message in context: http://apache-nifi-developer-
> list.39713.n7.nabble.com/ValidateCSV-tp14153.html
> Sent from the Apache NiFi Developer List mailing list archive at
> Nabble.com.
>