You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@samza.apache.org by "Yi Pan (Data Infrastructure) (JIRA)" <ji...@apache.org> on 2018/03/05 23:44:00 UTC

[jira] [Created] (SAMZA-1600) Changelog topics for batch jobs should not use combination of delete+compact cleanup policy in Kafka

Yi Pan (Data Infrastructure) created SAMZA-1600:
---------------------------------------------------

             Summary: Changelog topics for batch jobs should not use combination of delete+compact cleanup policy in Kafka
                 Key: SAMZA-1600
                 URL: https://issues.apache.org/jira/browse/SAMZA-1600
             Project: Samza
          Issue Type: Bug
            Reporter: Yi Pan (Data Infrastructure)


We have encountered a bug in Kafka 0.11 that when a topic is configured to use a combination of cleanup policy delete+compact, the broker enters a deadlock situation.

Currently, we set the delete+compact policy for changelog topics in batch jobs to reduce the storage requirement and the additional GC effort required. However, before the above mentioned Kafka bug is fixed, we will disable this feature for now.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)