You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by "Chris Riccomini (JIRA)" <ji...@apache.org> on 2015/03/23 20:45:53 UTC
[jira] [Commented] (SAMZA-610) Fail container (without send retry)
when Kafka producer throws a RetriableException
[ https://issues.apache.org/jira/browse/SAMZA-610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376489#comment-14376489 ]
Chris Riccomini commented on SAMZA-610:
---------------------------------------
bq. Currently, we retry from the Samza layer, which should not be necessary.
In addition to being unnecessary, it's actually dangerous, since retrying means that we could lose ordering. We might want to open a KAFKA JIRA to make it so that we can retry without losing ordering (by taking all queued messages *out* of the IO thread's queue, retrying the failed message, then appending the buffered messages *back* into the queue).
> Fail container (without send retry) when Kafka producer throws a RetriableException
> -----------------------------------------------------------------------------------
>
> Key: SAMZA-610
> URL: https://issues.apache.org/jira/browse/SAMZA-610
> Project: Samza
> Issue Type: Bug
> Reporter: Navina Ramesh
>
> With the new Java-based Kafka producer, we get 2 kinds of exceptions - RetriableException & Non-RetriableException.
> We have set max.retries to INT_MAX. This means that if the producer throws a RetriableException, it has already retried the send request. At the Samza level, we just have to fail the container in order to avoid re-ordering of messages. Currently, we retry from the Samza layer, which should not be necessary.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)