[jira] [Updated] (KAFKA-4607) Kafka Streams allows you to provide strings with illegal characters for internal topic names

["Nikki Thean (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-4607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Nikki Thean updated KAFKA-4607:
-------------------------------
    Description: 
When using the aggregate function of the Kafka Streams DSL, I supplied the function with a name for the underlying state store that contained whitespace. The stream application ran with no error message but seemed to be hanging. 

After a lot of troubleshooting, I looked in the logs on the Kafka cluster and noticed that it was complaining about illegal characters in the log line. It turns out that an internal changelog topic had been created that contained whitespace and this was causing both Kafka and ZK to choke. I was unable to use the delete-topic command line utility as it also threw the illegal character error. In the end I had to manually delete the topics out of ZK (using an open source ZK tool because the ZK delete command was choking on any strings with spaces) and do some stuff with the offset logs.

I am not well versed in the Kafka or Kafka Streams code, but from looking through it, I notice that [InternalTopicManager|https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/processor/internals/InternalTopicManager.java#L170] does not seem to validate the names of auto-generated internal topics beyond [checking for null topic names|https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/processor/internals/InternalTopicConfig.java#L38] rather than using Topic.validate.

  was:
When using the aggregate function of the Kafka Streams DSL, I supplied the function with a name for the underlying state store that contained whitespace. The stream application ran with no error message but seemed to be hanging. 

After a lot of troubleshooting, I looked in the logs on the Kafka cluster and noticed that it was complaining about illegal characters in the log line. It turns out that an internal changelog topic had been created that contained whitespace and this was causing both Kafka and ZK to choke. I was unable to use the delete-topic command line utility as it also threw the illegal character error. In the end I had to manually delete the topics out of ZK and do some stuff with the offset logs.

I am not well versed in the Kafka or Kafka Streams code, but from looking through it, I notice that [InternalTopicManager|https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/processor/internals/InternalTopicManager.java#L170] does not seem to validate the names of auto-generated internal topics beyond [checking for null topic names|https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/processor/internals/InternalTopicConfig.java#L38] rather than using Topic.validate.


> Kafka Streams allows you to provide strings with illegal characters for internal topic names
> --------------------------------------------------------------------------------------------
>
>                 Key: KAFKA-4607
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4607
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: Nikki Thean
>            Priority: Minor
>
> When using the aggregate function of the Kafka Streams DSL, I supplied the function with a name for the underlying state store that contained whitespace. The stream application ran with no error message but seemed to be hanging. 
> After a lot of troubleshooting, I looked in the logs on the Kafka cluster and noticed that it was complaining about illegal characters in the log line. It turns out that an internal changelog topic had been created that contained whitespace and this was causing both Kafka and ZK to choke. I was unable to use the delete-topic command line utility as it also threw the illegal character error. In the end I had to manually delete the topics out of ZK (using an open source ZK tool because the ZK delete command was choking on any strings with spaces) and do some stuff with the offset logs.
> I am not well versed in the Kafka or Kafka Streams code, but from looking through it, I notice that [InternalTopicManager|https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/processor/internals/InternalTopicManager.java#L170] does not seem to validate the names of auto-generated internal topics beyond [checking for null topic names|https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/processor/internals/InternalTopicConfig.java#L38] rather than using Topic.validate.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)