[GitHub] [nifi] pvillard31 commented on pull request #5530: NIFI-9341 Adding record reader for CEF events

[GitBox <gi...@apache.org>]

pvillard31 commented on pull request #5530:
URL: https://github.com/apache/nifi/pull/5530#issuecomment-1007342283


   Hey @simonbence - I gave this another try and I'm a bit concerned on how this would work when processing a flow file containing a lot of CEF logs but with one line not being correctly formatted according to CEF specifications. Right now, if a single line cannot be parsed, the whole flow file would fail (when using a processor such as ConvertRecord). Can we think of a solution to always successfully parse a line but parse it in a specific way so that filtering between "good" and "bad" records can be done? I'm wondering if, similarly to the property RAW_FIELD, we could have an INVALID_FIELD, and if specified, the line would not be parsed but a record with a single field containing the raw line would be produced. This way, when using a processor like QueryRecord processor, it'd be super easy to do the conversion from CEF to JSON while also filtering bad and good records in two distinct flow files. Thoughts?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@nifi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org