Streams error handling

[Mike Gould <mi...@gmail.com>]

Hi
Are there any better error handling options for Kafka streams in java.

Any errors in the serdes will break the stream.  The suggested
implementation is to use the byte[] serde and do the deserialisation in a
map operation.  However this isn't ideal either as there's no great way to
handle exceptions.
My current tactics are to use flatMap in place of map everywhere and return
empySet on error. Unfortunately this means the error has to be handled
directly in the function where it happened and can only be handled as a
side effect.

It seems to me that this could be done better. Maybe the *Mapper interfaces
could allow specific checked exceptions. These could be handled by specific
downstream KStream.mapException() steps which might e.g. Put an error
response on another stream branch.
Alternatively could it be made easier to return something like an Either
from the Mappers with a the addition of few extra methods on KStream.

Unless there's a better error handling pattern which I've entirely missed?

Thanks
MIkeG

-- 
 - MikeG
http://en.wikipedia.org/wiki/Common_misconceptions
<http://en.wikipedia.org/wiki/Special:Random>