[jira] [Updated] (KAFKA-1932) kafka topic (creation) templates

["Ahmet AKYOL (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-1932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ahmet AKYOL updated KAFKA-1932:
-------------------------------
    Description: 
AFAIK, the only way to create a Kafka topic (without using the default settings) is using the provided bash script.

Even though, a client support could be nice, I would prefer to see a template mechanism similar to [Elasticsearch Index Templates|http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-templates.html] . 

What I have in my mind is very simple and adding something like this into server properties :

template.name=pattern,numOfReplica,NumberOfPartition

and pattern can only contain "*" meaning starts with, ends with or contains.

example:
template.logtopics=*_log,2,20
template.loaders=*_loader,1,5

so,when some producer sends a message to a topic for the first time which ends with "_logs" , then, kafka can use above settings.

thanks in advance

update:

On second thought, maybe a command like kafka-create-template.sh could be more practical for cluster deployments, rather than adding to server.properties. Kafka internally registers this to ZK.

About use cases, I can understand an opposing argument like creating many topics is not a good design decision. Besides, my point is not to create so many topics, just to automate an important process by giving the responsibility to Kafka.

  was:
AFAIK, the only way to create a Kafka topic (without using the default settings) is using the provided bash script.

Even though, a client support could be nice, I would prefer to see a template mechanism similar to [Elasticsearch Index Templates|http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-templates.html] . 

What I have in my mind is very simple and adding something like this into server properties :

template.name=pattern,numOfReplica,NumberOfPartition

and pattern can only contain "*" meaning starts with, ends with or contains.

example:
template.logtopics=*_log,2,20
template.loaders=*_loader,1,5

so when the first time some producer sends a message to a topic that ends with "_logs" , kafka can use above settings.

thanks in advance


> kafka topic (creation) templates
> --------------------------------
>
>                 Key: KAFKA-1932
>                 URL: https://issues.apache.org/jira/browse/KAFKA-1932
>             Project: Kafka
>          Issue Type: Wish
>            Reporter: Ahmet AKYOL
>
> AFAIK, the only way to create a Kafka topic (without using the default settings) is using the provided bash script.
> Even though, a client support could be nice, I would prefer to see a template mechanism similar to [Elasticsearch Index Templates|http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-templates.html] . 
> What I have in my mind is very simple and adding something like this into server properties :
> template.name=pattern,numOfReplica,NumberOfPartition
> and pattern can only contain "*" meaning starts with, ends with or contains.
> example:
> template.logtopics=*_log,2,20
> template.loaders=*_loader,1,5
> so,when some producer sends a message to a topic for the first time which ends with "_logs" , then, kafka can use above settings.
> thanks in advance
> update:
> On second thought, maybe a command like kafka-create-template.sh could be more practical for cluster deployments, rather than adding to server.properties. Kafka internally registers this to ZK.
> About use cases, I can understand an opposing argument like creating many topics is not a good design decision. Besides, my point is not to create so many topics, just to automate an important process by giving the responsibility to Kafka.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)