You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2017/06/21 10:39:00 UTC
[jira] [Commented] (KAFKA-5475) Connector config validation REST
API endpoint not including fields for transformations
[ https://issues.apache.org/jira/browse/KAFKA-5475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16057313#comment-16057313 ]
ASF GitHub Bot commented on KAFKA-5475:
---------------------------------------
GitHub user kkonstantine opened a pull request:
https://github.com/apache/kafka/pull/3394
KAFKA-5475: Connector config validation needs to include tranformation types
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/kkonstantine/kafka KAFKA-5475-Connector-config-validation-REST-API-endpoint-not-including-fields-for-transformations
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/3394.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3394
----
commit 10c4ef7b8785119f42eb48dac237fe017c93b753
Author: Konstantine Karantasis <ko...@confluent.io>
Date: 2017-06-21T10:36:39Z
KAFKA-5475: Connector config validation needs to include tranformation types.
----
> Connector config validation REST API endpoint not including fields for transformations
> --------------------------------------------------------------------------------------
>
> Key: KAFKA-5475
> URL: https://issues.apache.org/jira/browse/KAFKA-5475
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
> Reporter: Ewen Cheslack-Postava
> Priority: Blocker
> Fix For: 0.11.0.0
>
>
> An issue with how embedded transformation configurations are included seems to have been introduced during 0.11.0.0. We are no longer seeing the `transforms.<xform>.type` being included in the validation output.
> {code}
> curl -X PUT -H "Content-Type: application/json" --data '{"connector.class": "org.apache.kafka.connect.file.FileStreamSourceConnector", "transforms": "foo,bar"}' http://localhost:8083/connector-plugins/org.apache.kafka.connect.file.FileStreamSourceConnector/config/validate | jq
> % Total % Received % Xferd Average Speed Time Time Time Current
> Dload Upload Total Spent Left Speed
> 100 3428 100 3325 100 103 344k 10917 --:--:-- --:--:-- --:--:-- 360k
> {
> "name": "org.apache.kafka.connect.file.FileStreamSourceConnector",
> "error_count": 1,
> "groups": [
> "Common",
> "Transforms"
> ],
> "configs": [
> {
> "definition": {
> "name": "value.converter",
> "type": "CLASS",
> "required": false,
> "default_value": null,
> "importance": "LOW",
> "documentation": "Converter class used to convert between Kafka Connect format and the serialized form that is written to Kafka. This controls the format of the values in messages written to or read from Kafka, and since this is independent of connectors it allows any connector to work with any serialization format. Examples of common formats include JSON and Avro.",
> "group": "Common",
> "width": "SHORT",
> "display_name": "Value converter class",
> "dependents": [],
> "order": 5
> },
> "value": {
> "name": "value.converter",
> "value": null,
> "recommended_values": [],
> "errors": [],
> "visible": true
> }
> },
> {
> "definition": {
> "name": "name",
> "type": "STRING",
> "required": true,
> "default_value": null,
> "importance": "HIGH",
> "documentation": "Globally unique name to use for this connector.",
> "group": "Common",
> "width": "MEDIUM",
> "display_name": "Connector name",
> "dependents": [],
> "order": 1
> },
> "value": {
> "name": "name",
> "value": null,
> "recommended_values": [],
> "errors": [
> "Missing required configuration \"name\" which has no default value."
> ],
> "visible": true
> }
> },
> {
> "definition": {
> "name": "tasks.max",
> "type": "INT",
> "required": false,
> "default_value": "1",
> "importance": "HIGH",
> "documentation": "Maximum number of tasks to use for this connector.",
> "group": "Common",
> "width": "SHORT",
> "display_name": "Tasks max",
> "dependents": [],
> "order": 3
> },
> "value": {
> "name": "tasks.max",
> "value": "1",
> "recommended_values": [],
> "errors": [],
> "visible": true
> }
> },
> {
> "definition": {
> "name": "connector.class",
> "type": "STRING",
> "required": true,
> "default_value": null,
> "importance": "HIGH",
> "documentation": "Name or alias of the class for this connector. Must be a subclass of org.apache.kafka.connect.connector.Connector. If the connector is org.apache.kafka.connect.file.FileStreamSinkConnector, you can either specify this full name, or use \"FileStreamSink\" or \"FileStreamSinkConnector\" to make the configuration a bit shorter",
> "group": "Common",
> "width": "LONG",
> "display_name": "Connector class",
> "dependents": [],
> "order": 2
> },
> "value": {
> "name": "connector.class",
> "value": "org.apache.kafka.connect.file.FileStreamSourceConnector",
> "recommended_values": [],
> "errors": [],
> "visible": true
> }
> },
> {
> "definition": {
> "name": "key.converter",
> "type": "CLASS",
> "required": false,
> "default_value": null,
> "importance": "LOW",
> "documentation": "Converter class used to convert between Kafka Connect format and the serialized form that is written to Kafka. This controls the format of the keys in messages written to or read from Kafka, and since this is independent of connectors it allows any connector to work with any serialization format. Examples of common formats include JSON and Avro.",
> "group": "Common",
> "width": "SHORT",
> "display_name": "Key converter class",
> "dependents": [],
> "order": 4
> },
> "value": {
> "name": "key.converter",
> "value": null,
> "recommended_values": [],
> "errors": [],
> "visible": true
> }
> },
> {
> "definition": {
> "name": "transforms",
> "type": "LIST",
> "required": false,
> "default_value": null,
> "importance": "LOW",
> "documentation": "Aliases for the transformations to be applied to records.",
> "group": "Transforms",
> "width": "LONG",
> "display_name": "Transforms",
> "dependents": [],
> "order": 6
> },
> "value": {
> "name": "transforms",
> "value": "foo,bar",
> "recommended_values": [],
> "errors": [],
> "visible": true
> }
> }
> ]
> }
> {code}
> In this example, we should have seen `transforms.foo.type` and `transforms.bar.type` included as additional configs.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)