You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by GitBox <gi...@apache.org> on 2022/01/21 14:56:17 UTC
[GitHub] [camel-kafka-connector] nervgh opened a new issue #1323: Do Camel Kafka Connectors support "Dead Letter Queue" and "Connect Reporter" features of Kafka Connect?
nervgh opened a new issue #1323:
URL: https://github.com/apache/camel-kafka-connector/issues/1323
I'm curious about supporting of **Dead Letter Queue** and **Connect Reporter** features of Kafka Connect. Do Camel Kafka Connectors support of them?
### Dead Letter Queue (feature)
> ### [KIP-610: Error Reporting in Sink Connectors](https://cwiki.apache.org/confluence/display/KAFKA/KIP-610%3A+Error+Reporting+in+Sink+Connectors)
>
> Extends an Error Reporting capability for sink connectors. Previously, error handling in Kafka Connect included functionality such as retrying, logging, and sending errant records to a dead letter queue. However, the dead letter queue functionality from [KIP-298](https://cwiki.apache.org/confluence/display/KAFKA/KIP-298%3A+Error+Handling+in+Connect) only supports error reporting within contexts of the transform operation, and key, value, and header converter operation. After records are sent to the connector for processing, there is no support for dead letter queue or error reporting functionality. [KIP-610](https://cwiki.apache.org/confluence/display/KAFKA/KIP-610%3A+Error+Reporting+in+Sink+Connectors) allows sink connectors to report individual records as being problematic, and they will be sent to the DLQ. For more information, see [Dead Letter Queue](https://docs.confluent.io/platform/6.0.0/connect/concepts.html#dead-letter-queues).
- https://docs.confluent.io/platform/6.0.0/release-notes/index.html#kconnect
- https://github.com/university-of-auckland/kafka-http-sink-connector/pull/8
### Connect Reporter (feature)
> The Kafka Connect Reporter submits the result of a sink operation to a reporter topic. After successfully sinking a record or following an error condition, the Connect Reporter is called to submit the result report. The report is constructed to include details about how the original record was handled along with additional information about the sink event. These records are written to configurable success and error topics for further consumption. The following is an example of the basic Connect Reporter configuration properties added to a sink connector configuration:
>
> ```properties
> reporter.bootstrap.servers=localhost:9092
> reporter.result.topic.name=success-responses
> reporter.result.topic.replication.factor=1
> reporter.error.topic.name=error-responses
> reporter.error.topic.replication.factor=1
> ```
- https://docs.confluent.io/home/connect/self-managed/userguide.html#kconnect-reporter
- #1296
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] valdar closed issue #1323: Do Camel Kafka Connectors support "Dead Letter Queue" and "Connect Reporter" features of Kafka Connect?
Posted by GitBox <gi...@apache.org>.
valdar closed issue #1323:
URL: https://github.com/apache/camel-kafka-connector/issues/1323
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] valdar commented on issue #1323: Do Camel Kafka Connectors support "Dead Letter Queue" and "Connect Reporter" features of Kafka Connect?
Posted by GitBox <gi...@apache.org>.
valdar commented on issue #1323:
URL: https://github.com/apache/camel-kafka-connector/issues/1323#issuecomment-1020576164
They should be supported. The work was done mostly in #835
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] nervgh commented on issue #1323: Do Camel Kafka Connectors support "Dead Letter Queue" and "Connect Reporter" features of Kafka Connect?
Posted by GitBox <gi...@apache.org>.
nervgh commented on issue #1323:
URL: https://github.com/apache/camel-kafka-connector/issues/1323#issuecomment-1021974563
@valdar thank you for your reply about error handling with DLQ! :+1:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] valdar commented on issue #1323: Do Camel Kafka Connectors support "Dead Letter Queue" and "Connect Reporter" features of Kafka Connect?
Posted by GitBox <gi...@apache.org>.
valdar commented on issue #1323:
URL: https://github.com/apache/camel-kafka-connector/issues/1323#issuecomment-1022440923
@nervgh I am closing this, feel free to reopen it or open other issues if something does not work as expected.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] valdar commented on issue #1323: Do Camel Kafka Connectors support "Dead Letter Queue" and "Connect Reporter" features of Kafka Connect?
Posted by GitBox <gi...@apache.org>.
valdar commented on issue #1323:
URL: https://github.com/apache/camel-kafka-connector/issues/1323#issuecomment-1020576164
They should be supported. The work was done mostly in #835
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org