[GitHub] [incubator-hop] sramazzina opened a new pull request #1244: HOP 3504

[GitBox <gi...@apache.org>]

sramazzina opened a new pull request #1244:
URL: https://github.com/apache/incubator-hop/pull/1244


   This PR is to let KafkaConsumer manage unexpected exception generated in the called sub pipeline by records injected from the kafka queue. This possibility is given only in case of batchSize = 1 because only in that case we have a feeling on which is the failing record and we can easily use the error path to let that record come out from the called sub pipeline. As soon as the records comes out, it is committed in kafka and we have the possibility to dump it somewhere else for further investigation. 
   
   The push to the error path of the failing record gives 2 benefits:
   
   - because of the commit it will free the kafka queue from a record that is stopping other records from being processed
   - the pipeline is not stopped because of the unexpected unhandled exception 
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org