You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by GitBox <gi...@apache.org> on 2020/12/23 15:31:00 UTC
[GitHub] [camel] ppalaga opened a new pull request #4815: Add Consuming Avro messages from Kafka using the schema-registry exam…
ppalaga opened a new pull request #4815:
URL: https://github.com/apache/camel/pull/4815
…ple to Kafka component docs
This follows up the discussion in https://github.com/apache/camel-quarkus/issues/1960 .
Could you please review and perhaps also add more details @tstuber?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel] oscerd merged pull request #4815: Add Consuming Avro messages from Kafka using the schema-registry exam…
Posted by GitBox <gi...@apache.org>.
oscerd merged pull request #4815:
URL: https://github.com/apache/camel/pull/4815
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel] ppalaga commented on a change in pull request #4815: Add Consuming Avro messages from Kafka using the schema-registry exam…
Posted by GitBox <gi...@apache.org>.
ppalaga commented on a change in pull request #4815:
URL: https://github.com/apache/camel/pull/4815#discussion_r548480346
##########
File path: catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/kafka-component.adoc
##########
@@ -580,3 +580,23 @@ from("kafka:my_topic?headerFilterStrategy=#myStrategy")
`myStrategy` object should be subclass of `HeaderFilterStrategy` and must be placed in the Camel registry, either manually or by registration as a bean in Spring/Blueprint, as it is `CamelContext` aware.
include::camel-spring-boot::page$kafka-starter.adoc[]
+
+== Consuming Avro messages from Kafka using the schema-registry
+*Since Camel 3.0.0*
+
+This route applies the schema and converts the Avro messages to POJOs for further processing:
+
+[source,java]
+----
+from(kafka("avro-events")
+ .brokers("localhost:9092")
+ .schemaRegistryURL("http://localhost:8081")
+ .specificAvroReader(true)
+ .valueDeserializer("io.confluent.kafka.serializers.KafkaAvroDeserializer")
+ .resolve(getContext()))
Review comment:
OK, let me remove resolve()
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel] ppalaga commented on a change in pull request #4815: Add Consuming Avro messages from Kafka using the schema-registry exam…
Posted by GitBox <gi...@apache.org>.
ppalaga commented on a change in pull request #4815:
URL: https://github.com/apache/camel/pull/4815#discussion_r548480963
##########
File path: catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/kafka-component.adoc
##########
@@ -580,3 +580,23 @@ from("kafka:my_topic?headerFilterStrategy=#myStrategy")
`myStrategy` object should be subclass of `HeaderFilterStrategy` and must be placed in the Camel registry, either manually or by registration as a bean in Spring/Blueprint, as it is `CamelContext` aware.
include::camel-spring-boot::page$kafka-starter.adoc[]
+
+== Consuming Avro messages from Kafka using the schema-registry
+*Since Camel 3.0.0*
+
+This route applies the schema and converts the Avro messages to POJOs for further processing:
+
+[source,java]
+----
+from(kafka("avro-events")
+ .brokers("localhost:9092")
+ .schemaRegistryURL("http://localhost:8081")
+ .specificAvroReader(true)
+ .valueDeserializer("io.confluent.kafka.serializers.KafkaAvroDeserializer")
+ .resolve(getContext()))
Review comment:
Fixed in d5154d9
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel] tstuber commented on pull request #4815: Add Consuming Avro messages from Kafka using the schema-registry exam…
Posted by GitBox <gi...@apache.org>.
tstuber commented on pull request #4815:
URL: https://github.com/apache/camel/pull/4815#issuecomment-751867526
I added a complete example of reading & writing to Kafka using avro format (and confluent schema registry):
https://github.com/tstuber/camel-quarkus-kafka-schema-registry
@ppalaga does that help you for documenting? Otherwise, the changes in your PR look good to me and I think it will help other people
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel] oscerd commented on pull request #4815: Add Consuming Avro messages from Kafka using the schema-registry exam…
Posted by GitBox <gi...@apache.org>.
oscerd commented on pull request #4815:
URL: https://github.com/apache/camel/pull/4815#issuecomment-753966350
Lets merge this and we'll do the same for other schema registries.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel] tstuber commented on a change in pull request #4815: Add Consuming Avro messages from Kafka using the schema-registry exam…
Posted by GitBox <gi...@apache.org>.
tstuber commented on a change in pull request #4815:
URL: https://github.com/apache/camel/pull/4815#discussion_r549489157
##########
File path: catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/kafka-component.adoc
##########
@@ -580,3 +580,23 @@ from("kafka:my_topic?headerFilterStrategy=#myStrategy")
`myStrategy` object should be subclass of `HeaderFilterStrategy` and must be placed in the Camel registry, either manually or by registration as a bean in Spring/Blueprint, as it is `CamelContext` aware.
include::camel-spring-boot::page$kafka-starter.adoc[]
+
+== Consuming Avro messages from Kafka using the schema-registry
+*Since Camel 3.0.0*
+
+This route applies the schema and converts the Avro messages to POJOs for further processing:
+
+[source,java]
+----
+from(kafka("avro-events")
+ .brokers("localhost:9092")
+ .schemaRegistryURL("http://localhost:8081")
+ .specificAvroReader(true)
+ .valueDeserializer("io.confluent.kafka.serializers.KafkaAvroDeserializer")
Review comment:
Actually, the adjusted code misses now a closing bracket at the end: `.valueDeserializer("io.confluent.kafka.serializers.KafkaAvroDeserializer"))`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel] ppalaga commented on a change in pull request #4815: Add Consuming Avro messages from Kafka using the schema-registry exam…
Posted by GitBox <gi...@apache.org>.
ppalaga commented on a change in pull request #4815:
URL: https://github.com/apache/camel/pull/4815#discussion_r548480168
##########
File path: catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/kafka-component.adoc
##########
@@ -580,3 +580,23 @@ from("kafka:my_topic?headerFilterStrategy=#myStrategy")
`myStrategy` object should be subclass of `HeaderFilterStrategy` and must be placed in the Camel registry, either manually or by registration as a bean in Spring/Blueprint, as it is `CamelContext` aware.
include::camel-spring-boot::page$kafka-starter.adoc[]
+
+== Consuming Avro messages from Kafka using the schema-registry
+*Since Camel 3.0.0*
+
+This route applies the schema and converts the Avro messages to POJOs for further processing:
+
+[source,java]
+----
+from(kafka("avro-events")
+ .brokers("localhost:9092")
+ .schemaRegistryURL("http://localhost:8081")
+ .specificAvroReader(true)
+ .valueDeserializer("io.confluent.kafka.serializers.KafkaAvroDeserializer")
Review comment:
I am all for improving it, but please note that I am totally lacking on the required expertise. All I did in this PR is just summing up the info gathered by @tstuber in https://github.com/apache/camel-quarkus/issues/1960 .
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel] davsclaus commented on a change in pull request #4815: Add Consuming Avro messages from Kafka using the schema-registry exam…
Posted by GitBox <gi...@apache.org>.
davsclaus commented on a change in pull request #4815:
URL: https://github.com/apache/camel/pull/4815#discussion_r548423651
##########
File path: catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/kafka-component.adoc
##########
@@ -580,3 +580,23 @@ from("kafka:my_topic?headerFilterStrategy=#myStrategy")
`myStrategy` object should be subclass of `HeaderFilterStrategy` and must be placed in the Camel registry, either manually or by registration as a bean in Spring/Blueprint, as it is `CamelContext` aware.
include::camel-spring-boot::page$kafka-starter.adoc[]
+
+== Consuming Avro messages from Kafka using the schema-registry
+*Since Camel 3.0.0*
+
+This route applies the schema and converts the Avro messages to POJOs for further processing:
+
+[source,java]
+----
+from(kafka("avro-events")
+ .brokers("localhost:9092")
+ .schemaRegistryURL("http://localhost:8081")
+ .specificAvroReader(true)
+ .valueDeserializer("io.confluent.kafka.serializers.KafkaAvroDeserializer")
Review comment:
Is it only confluent that has avra deseriazlier and this implies you are using confluent schema registry. Can we make this example also work with other schema registries? And add some more notes on this? And btw the example is using endpoint-dsl which we should mention.
##########
File path: catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/kafka-component.adoc
##########
@@ -580,3 +580,23 @@ from("kafka:my_topic?headerFilterStrategy=#myStrategy")
`myStrategy` object should be subclass of `HeaderFilterStrategy` and must be placed in the Camel registry, either manually or by registration as a bean in Spring/Blueprint, as it is `CamelContext` aware.
include::camel-spring-boot::page$kafka-starter.adoc[]
+
+== Consuming Avro messages from Kafka using the schema-registry
+*Since Camel 3.0.0*
+
+This route applies the schema and converts the Avro messages to POJOs for further processing:
+
+[source,java]
+----
+from(kafka("avro-events")
+ .brokers("localhost:9092")
+ .schemaRegistryURL("http://localhost:8081")
+ .specificAvroReader(true)
+ .valueDeserializer("io.confluent.kafka.serializers.KafkaAvroDeserializer")
+ .resolve(getContext()))
Review comment:
You would not call resolve when using endpoint-dsl creating routes
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org