Dataflow Design Help

[Jennifer Barnabee <je...@gmail.com>]

Hi All,
I was wondering what mechanisms folks might have used in their dataflows to
handle a sort of return-to-sender type scenario. The idea is that you use
NiFi to receive your data and check it for certain must-haves. Then, for
any data that doesn't have key items, you reject it in an automated way to
tell the sender to fix it and resend it. Say, for example, it's a
registration of some sort for a person, but their email address is missing.

Right now, we are doing this rejection report manually. In addition to
handling our incoming data feeds and splitting them/routing them as needed,
we'd also like to add this checking/rejection automation.

Has anyone dealt with this? Would you be willing to describe how you did it?

Thanks.
-Jenn

Re: Dataflow Design Help

[Jennifer Barnabee <je...@gmail.com>]

Hi Joe,
Thanks. Yes, this part is all clear. The part that makes it messier is that I think the rejection reports are sent back by email, and that's what the data providers are expecting. We could use PutEmail, but I just didn't know if that was the most graceful way to do this. I'm still learning more about the existing process, though. I think it may vary by data provider. Thanks again!
-Jenn

Sent from my iPad

> On Sep 8, 2015, at 8:44 PM, joepercivall@yahoo.com wrote:
> 
> Hey Jennifer,
> 
> How are you sending back the rejection to the user?
> 
> If you're just 'Post'ing to HTTP.  You could use the RouteOnAttribute and PostHTTP standard processors to first verify the email attribute exists. If it doesn't then route that flowfile to the PostHTTP a processor and post a HTTP response back. 
> 
> Hope that helps,
> Joe
> Sent from my phone
> 
>> On Sep 8, 2015, at 8:22 PM, Jennifer Barnabee <je...@gmail.com> wrote:
>> 
>> Hi All,
>> I was wondering what mechanisms folks might have used in their dataflows to handle a sort of return-to-sender type scenario. The idea is that you use NiFi to receive your data and check it for certain must-haves. Then, for any data that doesn't have key items, you reject it in an automated way to tell the sender to fix it and resend it. Say, for example, it's a registration of some sort for a person, but their email address is missing. 
>> 
>> Right now, we are doing this rejection report manually. In addition to handling our incoming data feeds and splitting them/routing them as needed, we'd also like to add this checking/rejection automation.
>> 
>> Has anyone dealt with this? Would you be willing to describe how you did it?
>> 
>> Thanks.
>> -Jenn
>> 
>> 

Re: Dataflow Design Help

[jo...@yahoo.com]

Hey Jennifer,

How are you sending back the rejection to the user?

If you're just 'Post'ing to HTTP.  You could use the RouteOnAttribute and PostHTTP standard processors to first verify the email attribute exists. If it doesn't then route that flowfile to the PostHTTP a processor and post a HTTP response back. 

Hope that helps,
Joe
Sent from my phone

> On Sep 8, 2015, at 8:22 PM, Jennifer Barnabee <je...@gmail.com> wrote:
> 
> Hi All,
> I was wondering what mechanisms folks might have used in their dataflows to handle a sort of return-to-sender type scenario. The idea is that you use NiFi to receive your data and check it for certain must-haves. Then, for any data that doesn't have key items, you reject it in an automated way to tell the sender to fix it and resend it. Say, for example, it's a registration of some sort for a person, but their email address is missing. 
> 
> Right now, we are doing this rejection report manually. In addition to handling our incoming data feeds and splitting them/routing them as needed, we'd also like to add this checking/rejection automation.
> 
> Has anyone dealt with this? Would you be willing to describe how you did it?
> 
> Thanks.
> -Jenn
> 
>