You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by "Shanthoosh Venkataraman (Jira)" <ji...@apache.org> on 2020/01/15 00:37:00 UTC
[jira] [Created] (SAMZA-2434) Fix the coordinator steam creation
workflow
Shanthoosh Venkataraman created SAMZA-2434:
----------------------------------------------
Summary: Fix the coordinator steam creation workflow
Key: SAMZA-2434
URL: https://issues.apache.org/jira/browse/SAMZA-2434
Project: Samza
Issue Type: Bug
Affects Versions: 1.0, 1.1, 1.2, 1.3
Reporter: Shanthoosh Venkataraman
Assignee: Shanthoosh Venkataraman
Symptom: Coordinator kafka topic of a samza job currently gets created with cleanup.policy set to 'delete' instead of 'compact' for certain cases. This potentially leads to losing all the metadata(JobModel/Config) stored in coordinator stream for some samza jobs. This behavior was observed for hundreds of samza jobs within LI and would affect any job using recent samza-1.2 or samza-1.3 release in OSS.
Cause: The control-flow in KafkaSystemAdmin.createStream to build the coordinator stream spec swallows the essential kafka-topic configurations and passes along the empty configuration bag to the kafka-broker when creating the coordinator stream. This issue was introduced in SAMZA-1868.
Changes: Fix the topic-creation control-flow for the coordinator topics and generate the correct topic-configurations.
Tests: Added unit tests to validate that the expected topic configuration bag was generated for the coordinator stream topics.
API Changes: None
Upgrade Instructions: None
Usage Instructions: None
--
This message was sent by Atlassian Jira
(v8.3.4#803005)