[GitHub] [pulsar] KIC added a comment to the discussion: Pulsar functions should be able to return none and multiple values

[GitBox <gi...@apache.org>]

GitHub user KIC added a comment to the discussion: Pulsar functions should be able to return none and multiple values

@jerrypeng thanks for the clarification, I did not get that returning null is an option :-) 

regarding returning multiple events, I am not saying it is impossible but it is a bit trickier then that:

> This method returns a CompletableFuture. You can always wait for the CompletableFuture to complete before updating the state. If there is a send failure, throw an exception, and in EFFECTIVELY_ONCE, the function instance will restart it self and replay the last message. 

Imagine if you have 10 futures, 5 completed, 5 failed
You did not store the state (you would only if all futures completed)
You throw an exception
Function restarts (EFFECTIVELY_ONCE)

Now you have the same sate as the one you have had but not stored (function needs to be deterministic). But you have sent 5 of 10 events already, so you need to know that you only have to send messages 6 - 10 plus finally store the state. This could be done by querying the last message/event on the target topic. And this would even hold if only the state store would fail. Because in such a case you throw an exception at stateStore(), in the redo call of the function you see 10 of 10 messages were sent, so you just store the state.

However I just think it would be much more user friendly if this is all handled by the function caller and you could optimize accordingly while the project evolves.

GitHub link: https://github.com/apache/pulsar/discussions/19657#discussioncomment-5137527

----
This is an automatically sent email for commits@pulsar.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@pulsar.apache.org