[jira] [Commented] (KAFKA-6806) Unable to validate sink connectors without "topics" component which is not required

["Randall Hauch (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-6806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445930#comment-16445930 ] 

Randall Hauch commented on KAFKA-6806:
--------------------------------------

In AK 1.1 we added the ability for sink connectors to define topics via regular expressions (via {{topics.regex}}) instead of an explicit list of topic names (via {{topics}}). That change requires that sink connector configurations define either {{topics.regex}} or {{topics}}, but not both.

In particular, the [PR|https://github.com/apache/kafka/pull/4251/files] for KAFKA-6253 added [the few lines|https://github.com/apache/kafka/pull/4251/files#diff-7da33fbb8807de8e9ae0426dc0c14c10R70] that now require either {{topics.regex}} or {{topics}}. Prior to this change, there was only {{topic}} and it had an empty string as the default and thus was not required.

So, while it is true that sink connector configurations could be submitted and validated without a {{topic}} property (since there was a default), when that sink connector was deployed Connect would [throw an exception if the {{topic}} property value was empty|https://github.com/apache/kafka/blob/1.0.0/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSinkTask.java#L260-L262]. That means that the connector could be validated and deployed, but would fail to do anything useful.

> Unable to validate sink connectors without "topics" component which is not required
> -----------------------------------------------------------------------------------
>
>                 Key: KAFKA-6806
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6806
>             Project: Kafka
>          Issue Type: Bug
>          Components: KafkaConnect
>    Affects Versions: 1.1.0
>         Environment: CP4.1., Centos7
>            Reporter: Ivan Majnarić
>            Priority: Major
>
> The bug is happening when you try to create new connector through for example kafka-connect-ui.
> While both source and sink connectors were able to be validated through REST without "topics" as add-on with "connector.class" like this:
> {code:java}
> PUT / http://connect-url:8083/connector-plugins/com.datamountaineer.streamreactor.connect.cassandra.sink.CassandraSinkConnector/config/validate
> {
>     "connector.class": "com.datamountaineer.streamreactor.connect.cassandra.sink.CassandraSinkConnector",
> }{code}
> In the new version of CP4.1 you still can validate *source connectors* but not *sink connectors*. If you want to validate sink connectors you need to add to request -> "topics" config, like:
> {code:java}
> PUT / http://connect-url:8083/connector-plugins/com.datamountaineer.streamreactor.connect.cassandra.sink.CassandraSinkConnector/config/validate
> {
>     "connector.class": "com.datamountaineer.streamreactor.connect.cassandra.sink.CassandraSinkConnector",
>     "topics": "test-topic"
> }{code}
> So there is a little missmatch of the ways how to validate connectors which I think happened accidentally.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)