[GitHub] jberstler commented on issue #234: requeue messages to kafka if subject is throttled

[GitBox <gi...@apache.org>]

jberstler commented on issue #234: requeue messages to kafka if subject is throttled
URL: https://github.com/apache/incubator-openwhisk-package-kafka/issues/234#issuecomment-362605056
 
 
   Like @csantanapr, I find it a little hard to believe we should expect customers to set up a separate topic for us, as well as allow us write access to it, in order to handle the case where OpenWhisk cannot (or will not) handle the rate at which they are producing messages. This is not to mention asking them to configure their triggers to make use of this new topic, which adds even more configuration burden and potential confusion when creating the trigger.
   
   Like @abaruni, I think the matter could be much more simply handled by allowing a longer retry period - possibly configurable by the trigger creator. Because the message contents are kept in memory, this retry period could, at least in theory, extend even beyond the customer's own Kafka retention policy - so long as the provider instance didn't lose the contents of its memory (say, through a reboot). I'm not sure I see a benefit to adjusting the exponential nature of the retry behavior. Once you start waiting > 60 seconds between retries, I don't think the exact value of how long you wait is really all that significant.
   
   I also think it is reasonable that if the customer wants to ensure that no messages are lost due to throttling, that they increase their cluster's message retention period in addition to configuring the new retry timeout parameter on their trigger.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services