Fail-fast

[Toivo Adams <to...@gmail.com>]

We have normal processing groups and special error handling group.
If during any step something will go wrong, processing of all items by this
id will be aborted.
Error handling group will be invoked which will do some cleanup.

Failure relationships are used to route to error handling group.

But sometimes it’s tricky to catch all errors.
Error message may go to log and I can catch error message using log
appender.
But is it possible to somehow send message to error handling group.
It seems without ProcessSession its not possible to create FlowFile and
possibly its not possible to send FlowFile to error handling group anyway
(without relationship).

It should be possible to use sockets to send JSON message to error handling
group (using some processor which will listen socket).

Even using Reporting task, how can I invoke error handling group?

Am I completely of the track?

Also, using a lot of Failure relationships clutters flow and it becomes
badly readable.
So getting completely rid of Failure relationships is very welcome.

I know plans to add wormholes but this possibly requires a lot planning and
work.

Thanks
Toivo




--
View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/Fail-fast-tp16456.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

Re: Fail-fast

[Joe Witt <jo...@gmail.com>]

Toivo

The flow can become complex to read when there are a number of
relationships used from any given processor.  This is certainly true
of certain error handling cases.  I do believe the wormholes and
versioned flows will help this considerably.

Another idea for such cases would be to let user select a given
process group/input port directly from the processor configuration
dialogue.  In this approach this means there would be no lines at all
for error handling on the graph unless the user wanted them and
instead they could just set the target right on the processor dialogue
rather than via drawing lines.  It might be sufficient though to get
the wormholes done and see how much nicer that makes flows.

Thanks
joe

On Tue, Jul 18, 2017 at 9:21 AM, Toivo Adams <to...@gmail.com> wrote:
> We have normal processing groups and special error handling group.
> If during any step something will go wrong, processing of all items by this
> id will be aborted.
> Error handling group will be invoked which will do some cleanup.
>
> Failure relationships are used to route to error handling group.
>
> But sometimes it’s tricky to catch all errors.
> Error message may go to log and I can catch error message using log
> appender.
> But is it possible to somehow send message to error handling group.
> It seems without ProcessSession its not possible to create FlowFile and
> possibly its not possible to send FlowFile to error handling group anyway
> (without relationship).
>
> It should be possible to use sockets to send JSON message to error handling
> group (using some processor which will listen socket).
>
> Even using Reporting task, how can I invoke error handling group?
>
> Am I completely of the track?
>
> Also, using a lot of Failure relationships clutters flow and it becomes
> badly readable.
> So getting completely rid of Failure relationships is very welcome.
>
> I know plans to add wormholes but this possibly requires a lot planning and
> work.
>
> Thanks
> Toivo
>
>
>
>
> --
> View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/Fail-fast-tp16456.html
> Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.