You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by raghav130593 <ra...@gmail.com> on 2016/11/13 22:19:06 UTC
Uploading a csv file into an RDBMS table quickly like LOAD DATA
I receive tab delimited CSV files from an email and I want to insert them
into a Teradata DB table. Right now, I am going through a sequence of tasks
where I SplitText->extractText -> ReplaceText(where I create the SQL
statement) -> putSQL. i am creating a row by row SQL insert here and there
are thousands of records which does cause a slow insertion process. Is there
anyway where I can do something like the LOAD DATA INFILE command like in
MySQL where the csv data is loaded into the table at a very high speed. Is
there anyway where I can merge this SQL command with the incoming csv file
in NiFi?
--
View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/Uploading-a-csv-file-into-an-RDBMS-table-quickly-like-LOAD-DATA-tp13878.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.
Re: Uploading a csv file into an RDBMS table quickly like LOAD DATA
Posted by Andrew Grande <ap...@gmail.com>.
You could e.g. use NiFi to get the csv file, put it on a file system and
invoke the native mysql loader command via ExecuteStreamCommand.
Andrew
On Sun, Nov 13, 2016, 6:57 PM raghav130593 <ra...@gmail.com> wrote:
> I receive tab delimited CSV files from an email and I want to insert them
> into a Teradata DB table. Right now, I am going through a sequence of tasks
> where I SplitText->extractText -> ReplaceText(where I create the SQL
> statement) -> putSQL. i am creating a row by row SQL insert here and there
> are thousands of records which does cause a slow insertion process. Is
> there
> anyway where I can do something like the LOAD DATA INFILE command like in
> MySQL where the csv data is loaded into the table at a very high speed. Is
> there anyway where I can merge this SQL command with the incoming csv file
> in NiFi?
>
>
>
> --
> View this message in context:
> http://apache-nifi-developer-list.39713.n7.nabble.com/Uploading-a-csv-file-into-an-RDBMS-table-quickly-like-LOAD-DATA-tp13878.html
> Sent from the Apache NiFi Developer List mailing list archive at
> Nabble.com.
>