mapWithState: multiple operations on the same stream

[Udo Fholl <ud...@gmail.com>]

Hi,

I have a stream which I need to process events and send them to another
service and then remove the key from the state. I'm storing state because I
some events are delayed.

My current approach is to consolidate the state, store it with a
mapWithState invocation. Then rather than using a foreachRDD to send it I
use a transform invocation. I do that in order to mark the event as failed
or successfully sent. The service that I'm sending data to, is quite faulty.

What would be the best approach for this workflow:
* Obtain data
* Consolidate state
* Try to send
* If it succeeds remove from state otherwise keep it in the state

Thanks.

Best regards,
Udo.