question

[kant kodali <ka...@gmail.com>]

Hi Guys,

I have the following problem and I am wondering If Nifi can help in the
following scenario. It is not a hard problem but if Nifi can help out of
the box that would be great.

Here is my message flow


​


I have a system U that captures all the messages from A to D and assume
system U never fails.

All messages from A to D are sent in an asynchronous way.
The components such as  A,B,C, D can fail at any time and for simplicity
lets say only one can fail at a time. If so system U will take charge and
emulate as a failed component such that the message flow from A to D will
always be successful and it would look like as though no component failed.

Can I leverage Nifi to do this?

Thanks,
kant

Re: question

[Lee Laim <le...@gmail.com>]

Hi Kant,

At a high level, Yes, NiFi can facilitate this type of data processing. As
for the specific details of your flow there may be challenges described in
the thread below[1].  All-in-all, I'm confident that NiFi has facilities to
address challenges when they surface, The built-in DetectDuplicate
processor is an example of this[2].


[1] https://community.hortonworks.com/questions/55454/can-nifi-promise-each-
of-the-flowfiles-can-be-proc.html

[2] https://nifi.apache.org/docs/nifi-docs/components/org.
apache.nifi.processors.standard.DetectDuplicate/

Thanks,
Lee

On Mon, Dec 19, 2016 at 4:30 PM, kant kodali <ka...@gmail.com> wrote:

> Also forgot to mention if a failed component comes back and try to send a
> message as U to its successor then there will be duplicates right so I
> would like to avoid duplicates and I wonder if Nifi can help me filter
> duplicates.
>
> Thanks!
>
> On Mon, Dec 19, 2016 at 3:23 PM, kant kodali <ka...@gmail.com> wrote:
>
>> Hi Guys,
>>
>> I have the following problem and I am wondering If Nifi can help in the
>> following scenario. It is not a hard problem but if Nifi can help out of
>> the box that would be great.
>>
>> Here is my message flow
>>
>>
>> ​
>>
>>
>> I have a system U that captures all the messages from A to D and assume
>> system U never fails.
>>
>> All messages from A to D are sent in an asynchronous way.
>> The components such as  A,B,C, D can fail at any time and for simplicity
>> lets say only one can fail at a time. If so system U will take charge and
>> emulate as a failed component such that the message flow from A to D will
>> always be successful and it would look like as though no component failed.
>>
>> Can I leverage Nifi to do this?
>>
>> Thanks,
>> kant
>>
>>
>>
>

Re: question

[kant kodali <ka...@gmail.com>]

Also forgot to mention if a failed component comes back and try to send a
message as U to its successor then there will be duplicates right so I
would like to avoid duplicates and I wonder if Nifi can help me filter
duplicates.

Thanks!

On Mon, Dec 19, 2016 at 3:23 PM, kant kodali <ka...@gmail.com> wrote:

> Hi Guys,
>
> I have the following problem and I am wondering If Nifi can help in the
> following scenario. It is not a hard problem but if Nifi can help out of
> the box that would be great.
>
> Here is my message flow
>
>
> ​
>
>
> I have a system U that captures all the messages from A to D and assume
> system U never fails.
>
> All messages from A to D are sent in an asynchronous way.
> The components such as  A,B,C, D can fail at any time and for simplicity
> lets say only one can fail at a time. If so system U will take charge and
> emulate as a failed component such that the message flow from A to D will
> always be successful and it would look like as though no component failed.
>
> Can I leverage Nifi to do this?
>
> Thanks,
> kant
>
>
>