[jira] [Created] (STORM-631) Simplifying kafka connector code

["Parth Brahmbhatt (JIRA)" <ji...@apache.org>]

Parth Brahmbhatt created STORM-631:
--------------------------------------

             Summary: Simplifying kafka connector code
                 Key: STORM-631
                 URL: https://issues.apache.org/jira/browse/STORM-631
             Project: Apache Storm
          Issue Type: Bug
          Components: storm-kafka
            Reporter: Parth Brahmbhatt
            Assignee: Parth Brahmbhatt


The current kafka connector was written when kafka did not support any APIs to expose its internal structures. We already have STORM-590 to modify the kafka connector to new kafka meta APIS. 

The current codebase uses some marker interfaces to support reading kafka partition/broker info from zookeeper or for users to specify the mapping them self. Adding one more layer that reads this info using kakfa APIs would reduce readability. Because we are using marker interfaces there are bunch of places that has if statements with instanceof checks and each one creates a separate code path to trace. We should delete these interfaces and their implementation in favor of a single way to get kafka topic information, using kafka APIs. This will be a backward incompatible change but should make future changes and usage easy.

Current kafka connector also has 2 separate configs for trident and core topologies. This configurations extend from a common interface but the extensions are not interchangeable. I think it is intuitive to have a single config with sane defaults for both core and trident spouts.








--
This message was sent by Atlassian JIRA
(v6.3.4#6332)