[jira] [Created] (KAFKA-13327) Preflight validations of connectors leads to 500 responses

["Chris Egerton (Jira)" <ji...@apache.org>]

Chris Egerton created KAFKA-13327:
-------------------------------------

             Summary: Preflight validations of connectors leads to 500 responses
                 Key: KAFKA-13327
                 URL: https://issues.apache.org/jira/browse/KAFKA-13327
             Project: Kafka
          Issue Type: Bug
          Components: KafkaConnect
            Reporter: Chris Egerton
            Assignee: Chris Egerton


The Connect framework performs some preflight validations for all connectors that are created in addition to allowing connectors to define their own custom validation logic by providing a {{ConfigDef}} object in [Connector::config|https://kafka.apache.org/30/javadoc/org/apache/kafka/connect/connector/Connector.html#config()] and performing multi-property validation in [Connector::validate|https://kafka.apache.org/30/javadoc/org/apache/kafka/connect/connector/Connector.html#validate(java.util.Map)].

When performed correctly, this validation information is surfaced to the user in the form of a [ConfigInfos|https://github.com/apache/kafka/blob/4eb386f6e060e12e1940c0d780987e3a7c438d74/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/rest/entities/ConfigInfos.java] object containing a list of [config objects|https://github.com/apache/kafka/blob/4eb386f6e060e12e1940c0d780987e3a7c438d74/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/rest/entities/ConfigInfo.java#L42-L45] whose [values|https://github.com/apache/kafka/blob/4eb386f6e060e12e1940c0d780987e3a7c438d74/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/rest/entities/ConfigInfo.java#L42-L45] contain one or more [error messages|https://github.com/apache/kafka/blob/4eb386f6e060e12e1940c0d780987e3a7c438d74/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/rest/entities/ConfigValueInfo.java#L61-L64]. This can be used as the response for a REST request to PUT /connector-plugins/\{connectorType}/config/validate and allows programmatic UIs to render error messages for every invalid property to the user.

However, some validations performed by the Connect framework do not follow this pattern and instead result in a 500 response being returned to the user. For example, logic specific to sink connectors (see [AbstractHerder|https://github.com/apache/kafka/blob/4eb386f6e060e12e1940c0d780987e3a7c438d74/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/AbstractHerder.java#L436] and [SinkConnectorConfig|https://github.com/apache/kafka/blob/4eb386f6e060e12e1940c0d780987e3a7c438d74/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/SinkConnectorConfig.java#L88-L125]) simply throws an exception instead of documenting the error with the offending property and returning it in a standard response.

 

We should correct this logic wherever possible so that configurations that are not fatally invalid (i.e., may have invalid properties but can still be translated into a meaningful {{ConfigInfos}} response object) do not cause a 500 response to be returned to the user.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)