You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by Gennady Kondratov <ge...@outlook.com> on 2023/10/06 16:06:56 UTC
How to Connect Apache NiFi to PostgreSql and using nifi insert records from csv file to database
How to Connect Apache NiFi to PostgreSql and using nifi insert records from csv file to database/
i'd like to ask you how to get it or where looking for about it
Re: How to Connect Apache NiFi to PostgreSql and using nifi insert records from csv file to database
Posted by Matt Burgess <ma...@apache.org>.
Gennady,
That is a great explanation by Lars, also note that by default GetFile
will remove the CSV from your system. If you want to keep it but only
fetch it once, use ListFile -> FetchFile rather than GetFile.
Regards,
Matt
On Fri, Oct 6, 2023 at 12:44 PM Lars Winderling
<la...@posteo.de> wrote:
>
> Hi Gennady,
>
> you should set up a jdbc connection pool controller service and connect it to your postgres instance. Then you will need a csv record reader controller service, configured with the correct csv format. Eventually, put an ExecuteSQLRecord processor onto the canvas. It should use the connection pool and the csv reader. Check the remaining properties of the processor for more config options. Cf. also https://nifi.apache.org/docs/nifi-docs/html/user-guide.html.
> Of course, you will need some processor to get the csv files in the first place. For local files, use GetFile. So that depends on the source of your files. When the correct processor is found, connect it to the sql processor (success relationship).
> When using GetFile, you will need to put a GenerateFlowfile processor in front of it, and connect them. When starting it, it will produce a flowfile. The GetFile proc will obtain the data, the sql processor will insert it into the db.
> Make sure to configure error relationships as well by e.g. terminating them (just to get you started).
> I hope, you will manage to go on from there. The docs are pretty extensive, as is the user mailing list archive.
>
> Best, Lars
>
> On 6 October 2023 18:06:56 CEST, Gennady Kondratov <ge...@outlook.com> wrote:
> >How to Connect Apache NiFi to PostgreSql and using nifi insert records from csv file to database/
> >i'd like to ask you how to get it or where looking for about it
> >
Re: How to Connect Apache NiFi to PostgreSql and using nifi insert records from csv file to database
Posted by Lars Winderling <la...@posteo.de>.
Hi Gennady,
you should set up a jdbc connection pool controller service and connect it to your postgres instance. Then you will need a csv record reader controller service, configured with the correct csv format. Eventually, put an ExecuteSQLRecord processor onto the canvas. It should use the connection pool and the csv reader. Check the remaining properties of the processor for more config options. Cf. also https://nifi.apache.org/docs/nifi-docs/html/user-guide.html.
Of course, you will need some processor to get the csv files in the first place. For local files, use GetFile. So that depends on the source of your files. When the correct processor is found, connect it to the sql processor (success relationship).
When using GetFile, you will need to put a GenerateFlowfile processor in front of it, and connect them. When starting it, it will produce a flowfile. The GetFile proc will obtain the data, the sql processor will insert it into the db.
Make sure to configure error relationships as well by e.g. terminating them (just to get you started).
I hope, you will manage to go on from there. The docs are pretty extensive, as is the user mailing list archive.
Best, Lars
On 6 October 2023 18:06:56 CEST, Gennady Kondratov <ge...@outlook.com> wrote:
>How to Connect Apache NiFi to PostgreSql and using nifi insert records from csv file to database/
>i'd like to ask you how to get it or where looking for about it
>