[jira] [Comment Edited] (KAFKA-6248) Enable configuration of internal topics of Kafka Streams applications

["Bill Bejeck (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-6248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16260892#comment-16260892 ] 

Bill Bejeck edited comment on KAFKA-6248 at 11/21/17 8:59 PM:
--------------------------------------------------------------

Hi [~timvanlaer] 

Take a look at https://cwiki.apache.org/confluence/display/KAFKA/KIP-173%3A+Add+prefix+to+StreamsConfig+to+enable+setting+default+internal+topic+configs included in  Kafka Streams as of release 1.0, which allows you to configure internal topics using the {{topic.}} prefix.  

Note this is a global setting for all internal topics and the config needs to be a valid property from {{org.apache.kafka.common.config.TopicConfig}}

HTH,
Bill


was (Author: bbejeck):
Hi [~timvanlaer] 

Take a look at [KIP-173|https://cwiki.apache.org/confluence/display/KAFKA/KIP-173%3A+Add+prefix+to+StreamsConfig+to+enable+setting+default+internal+topic+configs.] included in  Kafka Streams as of release 1.0, which allows you to configure internal topics using the {{topic.}} prefix.  

Note this is a global setting for all internal topics and the config needs to be a valid property from {{org.apache.kafka.common.config.TopicConfig}}

HTH,
Bill

> Enable configuration of internal topics of Kafka Streams applications
> ---------------------------------------------------------------------
>
>                 Key: KAFKA-6248
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6248
>             Project: Kafka
>          Issue Type: Improvement
>          Components: streams
>            Reporter: Tim Van Laer
>            Priority: Minor
>
> In the current implementation of Kafka Streams, it is not possible to set custom configuration to internal topics (e.g. max.message.bytes, retention.ms...). It would be nice if a developer can set some specific configuration. 
> E.g. if you want to store messages bigger than 1MiB in a state store, you have to alter the corresponding changelog topic with a max.message.bytes setting. 
> The workaround is to create the 'internal' topics upfront using the correct naming convention so Kafka Streams will use the explicitly defined topics as if they are internal. 
> An alternative is to alter the internal topics after the Kafka Streams application is started and has created its internal topics. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)