Adding Failure Information to AbstractRecordProcessor

[Shawn Weeks <sw...@weeksconsulting.us>]

I’d like to get some feedback on an enhancement that I’m looking at doing. I need a way for a custom Record Reader to pass information downstream on parse failures so that they can be logged for futhur investigation by a human. I’d like to have AbstractRecordProcessor add an attribute record.error.message on the flow file before it’s router to failure. Does anyone know of any issues this may cause related to security or otherwise or have a better way of handling it?

Thanks
Shawn

Re: Adding Failure Information to AbstractRecordProcessor

[Matt Burgess <ma...@apache.org>]

Shawn,

We've done this ad hoc in other places when it made sense to do so.
Maybe we'd want to avoid adding any record values to the error
message, although since the attribute is associated with the flow file
content I'm not sure that's necessary from a security perspective. The
only other thing I'd mention is to keep the error message as concise
as possible while still providing helpful information, for the purpose
of keeping the attribute on the small side.

Regards,
Matt

On Wed, Feb 12, 2020 at 4:54 PM Shawn Weeks <sw...@weeksconsulting.us> wrote:
>
> I’d like to get some feedback on an enhancement that I’m looking at doing. I need a way for a custom Record Reader to pass information downstream on parse failures so that they can be logged for futhur investigation by a human. I’d like to have AbstractRecordProcessor add an attribute record.error.message on the flow file before it’s router to failure. Does anyone know of any issues this may cause related to security or otherwise or have a better way of handling it?
>
> Thanks
> Shawn