Wait for child file to process successfully

[BD International <b....@gmail.com>]

Hello,

I've got a data flow which picks up a zip file and uses UnpackContent to
extract the contents. The subsequent files are them converted to json and
stored in a database.

I would like to store the original zip file and only delete the file once
all the extracted files have been stored correctly, has anyone else come
across a way to do this?

Thanks in advance,

Brian

On 30 Nov 2016 14:28, "Aldrin Piri" <al...@gmail.com> wrote:

Hi Andreas,

1)  There is nothing from a framework perspective that provides this.
However, a typical option is to make use of an attribute from an upstream
processor to help categorize and handle the data.  Attributes written vary
from processor to processor or can be explicitly set/updated using the
UpdateAttribute processor.
2)  This is also something that is universally handled across the framework
through processors.  Some processors, such as InvokeHTTP and, I believe,
those for AWS, do set such properties when a failure happens.  What you are
attempting to do though seems like it might be a good enhancement to add to
the processor and, frankly, a reasonable request to also work toward
providing more universally across components in the application.  For the
time being, however, your UpdateAttribute approach is the best option at
this juncture.

Would you mind opening up a JIRA issue so we can discuss this a bit more
and evaluate trying to extend such functionality in a standardized way?


On Wed, Nov 30, 2016 at 9:08 AM, Andreas Petter (External) <
andreas.petter.external@telefonica.com> wrote:

> Hello everybody,
>
>
>
> I have 2 questions:
>
> 1.       Is there some way to find out through which relationship/queue a
> FlowFile walked into a processor, in the onTrigger-Method?
>
> 2.       Is there a generic way how errors (e.g. Exceptions) are
> propagated with FlowFiles to subsequent processors?
>
>
>
> Background Story:
>
> I am writing a failure processor which handles failure events from
> FetchSFTP outgoing relationships, writing some flowfile attributes into a
> database and performing some further tasks to cope with the error. Now I
> would like to know through which of the three failure-reporting-relationships
> the FlowFile came along and get some generic failure information (e.g. the
> Exception). Right now I am adding 3 UpdateAttribute processors which each
> add an attribute identifying the relationship (and thereby the type of
> error). Maybe there is a better way to do this? I am using NiFi 1.0.
>
>
>
> Thank you very much for any help you might provide.
>
> Kind regards,
>
> Andreas Petter
>