You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by KhajaAsmath Mohammed <md...@gmail.com> on 2019/10/07 00:43:56 UTC

NIFI - Fetchfile - Execute SQL - Put Database Record

HI,

I have requirement of read a file from fileserver and delete all records
from Database if filename is present in database. Next step is to load data
from this file.

This is more like delete/insert. I cannot do upserts because the new file
can have more/less records after correction from the source.

Does anyone have idea to this?

I tried Fetchfile ->ExecuteSQL- PutDatabse.
I will loose the file in this case after executeSQL.

Have idea of placing funnel in between and do fork but delete might take
time before file starts processing.

Thanks,
Asmath

Re: NIFI - Fetchfile - Execute SQL - Put Database Record

Posted by Koji Kawamura <ij...@gmail.com>.
Hi Asmath,

How about using PutSQL?
FetchFile -> PutSQL -> PutDatabaseRecord
You can specify a SQL statement at PutSQL 'SQL Statement' property,
using FlowFile attribute.
For example, delete from tbl where file_name = '${filename}'
This way, the FlowFile content can be passed to PutDatabaseRecord as is.

Thanks,
Koji

On Mon, Oct 7, 2019 at 9:44 AM KhajaAsmath Mohammed
<md...@gmail.com> wrote:
>
> HI,
>
> I have requirement of read a file from fileserver and delete all records from Database if filename is present in database. Next step is to load data from this file.
>
> This is more like delete/insert. I cannot do upserts because the new file can have more/less records after correction from the source.
>
> Does anyone have idea to this?
>
> I tried Fetchfile ->ExecuteSQL- PutDatabse.
> I will loose the file in this case after executeSQL.
>
> Have idea of placing funnel in between and do fork but delete might take time before file starts processing.
>
> Thanks,
> Asmath
>