Backfilling in Nfi

["Chirthani, Deepak Reddy" <C-...@charter.com>]

Hello,

Is Nifi good at Backfilling the data? Let's say that I have a dataflow which is cron scheduled to run every 15 mins. Lets say that the source was down and the dataflow did not pulled any data for a couple of runs and after some time the source is up and running. If this happens frequently, how do you handle backfilling in Nifi? Please let me know if there is any automated way of doing this?

Thanks
E-MAIL CONFIDENTIALITY NOTICE: 
The contents of this e-mail message and any attachments are intended solely for the addressee(s) and may contain confidential and/or legally privileged information. If you are not the intended recipient of this message or if this message has been addressed to you in error, please immediately alert the sender by reply e-mail and then delete this message and any attachments. If you are not the intended recipient, you are notified that any use, dissemination, distribution, copying, or storage of this message or any attachment is strictly prohibited.

Re: Backfilling in Nfi

[Mark Payne <ma...@hotmail.com>]

Deepak,

NiFi, as a framework / application schedules the processor to run when you told it to. So every 15 minutes in this case. What the processor does then is dependent entirely on the processor. What processors in particular are you dealing with?

Thanks
-Mark

Sent from my iPhone

> On Jan 11, 2022, at 6:02 PM, Chirthani, Deepak Reddy <C-...@charter.com> wrote:
> 
> Hello,
> 
> Is Nifi good at Backfilling the data? Let's say that I have a dataflow which is cron scheduled to run every 15 mins. Lets say that the source was down and the dataflow did not pulled any data for a couple of runs and after some time the source is up and running. If this happens frequently, how do you handle backfilling in Nifi? Please let me know if there is any automated way of doing this?
> 
> Thanks
> E-MAIL CONFIDENTIALITY NOTICE: 
> The contents of this e-mail message and any attachments are intended solely for the addressee(s) and may contain confidential and/or legally privileged information. If you are not the intended recipient of this message or if this message has been addressed to you in error, please immediately alert the sender by reply e-mail and then delete this message and any attachments. If you are not the intended recipient, you are notified that any use, dissemination, distribution, copying, or storage of this message or any attachment is strictly prohibited.