You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by "Shanthoosh Venkataraman (Jira)" <ji...@apache.org> on 2020/01/15 00:37:00 UTC

[jira] [Created] (SAMZA-2434) Fix the coordinator steam creation workflow

Shanthoosh Venkataraman created SAMZA-2434:
----------------------------------------------

             Summary: Fix the coordinator steam creation workflow
                 Key: SAMZA-2434
                 URL: https://issues.apache.org/jira/browse/SAMZA-2434
             Project: Samza
          Issue Type: Bug
    Affects Versions: 1.0, 1.1, 1.2, 1.3
            Reporter: Shanthoosh Venkataraman
            Assignee: Shanthoosh Venkataraman


Symptom: Coordinator kafka topic of a samza job currently gets created with cleanup.policy set to 'delete' instead of 'compact' for certain cases. This potentially leads to losing all the metadata(JobModel/Config) stored in coordinator stream for some samza jobs. This behavior was observed for hundreds of samza jobs within LI and would affect any job using recent samza-1.2 or samza-1.3 release in OSS.

Cause: The control-flow in KafkaSystemAdmin.createStream to build the coordinator stream spec swallows the essential kafka-topic configurations and passes along the empty configuration bag to the kafka-broker when creating the coordinator stream. This issue was introduced in SAMZA-1868.

Changes: Fix the topic-creation control-flow for the coordinator topics and generate the correct topic-configurations.

Tests: Added unit tests to validate that the expected topic configuration bag was generated for the coordinator stream topics.

API Changes: None

Upgrade Instructions: None

Usage Instructions: None
 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)