Design question - code smell?

[David Gallagher <dg...@CleverDevices.com>]

Hi - I'm working on a flow with lots of processors and tons of reasons that something could fail (failure to match schema, bad return codes, file not found, etc). The intent is to send these failures to a ProduceAMQP processor with an appropriate routing key and error message. It makes the flow look terrible, though - no matter what I can think of, some processor out there is going to have multiple inbound connections and if I use a single Producer, it's going to look like a porcupine. Is there a pattern out there that I should be looking at for how to handle this scenario?


Thanks,


Dave

Re: Design question - code smell?

[David Gallagher <dg...@CleverDevices.com>]

Thanks! Just wanted to make sure I wasn't doing something really naive. I have a few funnels in my procedure now, it doesn't look so cluttered.

________________________________
From: Bryan Bende <bb...@gmail.com>
Sent: Wednesday, April 24, 2019 12:35:31 PM
To: users@nifi.apache.org
Subject: Re: Design question - code smell?

I think having a lot of error paths converge to a standard error
handling part of the flow is a common pattern.

It might help to converge different parts using funnels along the way
so that it doesn't end up with a hundred connections into the one
processor at the end, but its really just a preference.

On Wed, Apr 24, 2019 at 12:16 PM David Gallagher
<dg...@cleverdevices.com> wrote:
>
> Hi - I'm working on a flow with lots of processors and tons of reasons that something could fail (failure to match schema, bad return codes, file not found, etc). The intent is to send these failures to a ProduceAMQP processor with an appropriate routing key and error message. It makes the flow look terrible, though - no matter what I can think of, some processor out there is going to have multiple inbound connections and if I use a single Producer, it's going to look like a porcupine. Is there a pattern out there that I should be looking at for how to handle this scenario?
>
>
> Thanks,
>
>
> Dave

Re: Design question - code smell?

[Bryan Bende <bb...@gmail.com>]

I think having a lot of error paths converge to a standard error
handling part of the flow is a common pattern.

It might help to converge different parts using funnels along the way
so that it doesn't end up with a hundred connections into the one
processor at the end, but its really just a preference.

On Wed, Apr 24, 2019 at 12:16 PM David Gallagher
<dg...@cleverdevices.com> wrote:
>
> Hi - I'm working on a flow with lots of processors and tons of reasons that something could fail (failure to match schema, bad return codes, file not found, etc). The intent is to send these failures to a ProduceAMQP processor with an appropriate routing key and error message. It makes the flow look terrible, though - no matter what I can think of, some processor out there is going to have multiple inbound connections and if I use a single Producer, it's going to look like a porcupine. Is there a pattern out there that I should be looking at for how to handle this scenario?
>
>
> Thanks,
>
>
> Dave