[jira] [Commented] (KAFKA-8342) Admin tool to setup Kafka Stream topology (internal) topics

["Patrik Kleindl (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-8342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16836232#comment-16836232 ] 

Patrik Kleindl commented on KAFKA-8342:
---------------------------------------

This sounds very interesting, I hope it is ok if I add our use case and some thoughts:

Our current goal was to minimize deployment order dependencies so we don't have to mind deploying an application (mostly streams, but normal producers/consumers too) and have all relevant topics created before startup.

The internal topics are handled by Kafka Streams itself, but as we are moving to a more restricted approach we will be in the situation described above.

Our solution was to add the definition of topics for each application as a configuration artifact and share the dependencies via maven.

On startup (or triggered via a command-line call) this configuration is parsed and the topics are set up as needed.

We do this for all "named" topics including input and output topics and could add the internal topics if needed.

If I understand the approach above correctly this would mean that a user would have to export the topology (either the toString() version or something more beautiful) and pass it to the tool for processing.

One thing I would like to add to the use case is the interaction with the schema registry, as we are using Avro the creation of the schemas for the internal topics and a necessary pain too as I expect this to be restricted in a similar way as topic creation.

> Admin tool to setup Kafka Stream topology (internal) topics
> -----------------------------------------------------------
>
>                 Key: KAFKA-8342
>                 URL: https://issues.apache.org/jira/browse/KAFKA-8342
>             Project: Kafka
>          Issue Type: New Feature
>            Reporter: Boyang Chen
>            Priority: Major
>              Labels: newbie
>
> We have seen customers who need to deploy their application to production environment but don't have access to create changelog and repartition topics. They need to ask admin team to manually create those topics before proceeding to start the actual stream job. We could add an admin tool to help them go through the process quicker by providing a command that could
>  # Read through current stream topology
>  # Create corresponding topics as needed, even including output topics.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)