You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Yoshimo <yo...@protonmail.com.INVALID> on 2018/12/04 07:59:42 UTC
Kafka component Spring boot auto configuration
Hello Camel users,
I am trying to auto configure the Kafka component using spring boot, but the configuration is not working. More specifically I am following this page:
https://github.com/apache/camel/blob/master/components/camel-kafka/src/main/docs/kafka-component.adoc
and I am trying to set this configuration in my yaml file:
camel.component.kafka.configuration.value-deserializer
However the default deserializer is used when I consume from the Kafka topic. If I specify the deserializer in the Kafka route then it works as expected. Also I have defined the configuration camel.springboot.name in my yaml file and I can see that it is picked up normally. Are there any configurations that enable this functionality and I am missing?
Thank you,
Yiannis
Re: Kafka component Spring boot auto configuration
Posted by Yoshimo <yo...@protonmail.com.INVALID>.
Hello Claus,
thank you for looking into this.
Camel: 2.23.0
Kafka: 2.0.0
SB: 2.1.0.RELEASE
I am also attaching my pom if you want to have a look.
Thank you,
Yiannis
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Wednesday, 5 December 2018 12:15, Claus Ibsen <cl...@gmail.com> wrote:
> Hi
>
> What version of Camel, Kafka and SB are you using?
>
> On Tue, Dec 4, 2018 at 8:59 AM Yoshimo yoshimo@protonmail.com.invalid wrote:
>
> > Hello Camel users,
> > I am trying to auto configure the Kafka component using spring boot, but the configuration is not working. More specifically I am following this page:
> > https://github.com/apache/camel/blob/master/components/camel-kafka/src/main/docs/kafka-component.adoc
> > and I am trying to set this configuration in my yaml file:
> > camel.component.kafka.configuration.value-deserializer
> > However the default deserializer is used when I consume from the Kafka topic. If I specify the deserializer in the Kafka route then it works as expected. Also I have defined the configuration camel.springboot.name in my yaml file and I can see that it is picked up normally. Are there any configurations that enable this functionality and I am missing?
> > Thank you,
> > Yiannis
>
> --
>
> Claus Ibsen
>
> ------------
>
> http://davsclaus.com @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2
Re: Kafka component Spring boot auto configuration
Posted by Yoshimo <yo...@protonmail.com.INVALID>.
Indeed, I added it and it works! Thank you very much!
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Thursday, 6 December 2018 11:03, Luca Burgazzoli <lb...@gmail.com> wrote:
> I think you only need to add camel-kafka-starter as a dependency to
> make it working.
> Every component has a related starter that does the spring-boot magic
>
> -----------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Luca Burgazzoli
> On Thu, Dec 6, 2018 at 9:54 AM Yoshimo yoshimo@protonmail.com.invalid wrote:
>
> > I have created a demo project which demonstrates the issue. The issue is in application.properties file and in TestRoute class.
> > https://drive.google.com/file/d/1PDZj5tNXqqr0EjB07wluLiwy4ACK1eiP/view?usp=sharing
> > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> > On Thursday, 6 December 2018 10:41, Yoshimo yoshimo@protonmail.com wrote:
> >
> > > I have attached a demo project which demonstrates the issue. The issue is in application.properties file and in TestRoute class.
> > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> > > On Wednesday, 5 December 2018 14:33, Yoshimo yoshimo@protonmail.com.INVALID wrote:
> > >
> > > > Hey Claus,
> > > > I think that you misread my previous email. The value.deserializer = class org.apache.kafka.common.serialization.StringDeserializer line is the logging output of the Kafka consumer being created by Camel when I try adding the Spring Boot configuration.
> > > > What I actually add to my yaml file is the following:
> > > > camel.component.kafka.configuration.value-deserializer: com.openbet.platform.pricepublisher.serde.SportsEventDeserializer
> > > > The above is being ignored and the Kafka consumer that gets created by Camel has the default deserialer.
> > > > Thank you
> > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> > > > On Wednesday, 5 December 2018 14:17, Claus Ibsen claus.ibsen@gmail.com wrote:
> > > >
> > > > > On Wed, Dec 5, 2018 at 12:56 PM Yoshimo yoshimo@protonmail.com wrote:
> > > > >
> > > > > > Here is what I add to my yaml file:
> > > > > > camel.component.kafka.configuration.value-deserializer: com.openbet.platform.pricepublisher.serde.SportsEventDeserializer
> > > > > > and here is my route:
> > > > > >
> > > > > > from("kafka:prices?brokers={{kafka.servers}}")
> > > > > > .convertBodyTo(DBModel.class)
> > > > > > ......
> > > > > >
> > > > > >
> > > > > > With this setup when I start the app I can see in the logs the Kafka configuration in the consumer:
> > > > > > value.deserializer = class org.apache.kafka.common.serialization.StringDeserializer
> > > > >
> > > > > Try with
> > > > > value.deserializer = org.apache.kafka.common.serialization.StringDeserializer
> > > > >
> > > > > > and I get exceptions while deserializing. However if I remove the config from the yaml file and update my route to be:
> > > > > > kafka:prices?brokers={{kafka.servers}}&valueDeserializer=com.openbet.platform.pricepublisher.serde.SportsEventDeserializer
> > > > > > then I see the following in the Kafka consumer config:
> > > > > > value.deserializer = class com.openbet.platform.pricepublisher.serde.SportsEventDeserializer
> > > > > > and the app works as expected. Just for reference the rest of my yaml file is:
> > > > > > kafka:
> > > > > > sportsTopic: sports
> > > > > > pricesTopic: prices
> > > > > > servers: localhost:25445
> > > > > > streams:
> > > > > > "[application.id]": price-publisher
> > > > > > "[bootstrap.servers]": ${kafka.servers}
> > > > > > "[default.key.serde]": com.openbet.platform.pricepublisher.serde.ActivityKeySerDe
> > > > > > "[default.value.serde]": com.openbet.platform.pricepublisher.serde.ActivityValueSerDe
> > > > > > Thank you,
> > > > > > Yiannis
> > > > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> > > > > > On Wednesday, 5 December 2018 13:30, Claus Ibsen claus.ibsen@gmail.com wrote:
> > > > > >
> > > > > > > Hi
> > > > > > > Can you post your yaml configuration of how you try to configure this
> > > > > > > kafka value deserializer
> > > > > > > On Wed, Dec 5, 2018 at 11:36 AM Yoshimo yoshimo@protonmail.com wrote:
> > > > > > >
> > > > > > > > Hello Claus,
> > > > > > > > thank you for looking into this.
> > > > > > > > Camel: 2.23.0
> > > > > > > > Kafka: 2.0.0
> > > > > > > > SB: 2.1.0.RELEASE
> > > > > > > > I am also attaching my pom if you want to have a look.
> > > > > > > > Thank you,
> > > > > > > > Yiannis
> > > > > > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> > > > > > > > On Wednesday, 5 December 2018 12:15, Claus Ibsen claus.ibsen@gmail.com wrote:
> > > > > > > >
> > > > > > > > > Hi
> > > > > > > > > What version of Camel, Kafka and SB are you using?
> > > > > > > > > On Tue, Dec 4, 2018 at 8:59 AM Yoshimo yoshimo@protonmail.com.invalid wrote:
> > > > > > > > >
> > > > > > > > > > Hello Camel users,
> > > > > > > > > > I am trying to auto configure the Kafka component using spring boot, but the configuration is not working. More specifically I am following this page:
> > > > > > > > > > https://github.com/apache/camel/blob/master/components/camel-kafka/src/main/docs/kafka-component.adoc
> > > > > > > > > > and I am trying to set this configuration in my yaml file:
> > > > > > > > > > camel.component.kafka.configuration.value-deserializer
> > > > > > > > > > However the default deserializer is used when I consume from the Kafka topic. If I specify the deserializer in the Kafka route then it works as expected. Also I have defined the configuration camel.springboot.name in my yaml file and I can see that it is picked up normally. Are there any configurations that enable this functionality and I am missing?
> > > > > > > > > > Thank you,
> > > > > > > > > > Yiannis
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Claus Ibsen
> > > > > > > > > http://davsclaus.com @davsclaus
> > > > > > > > > Camel in Action 2: https://www.manning.com/ibsen2
> > > > > > >
> > > > > > > --
> > > > > > > Claus Ibsen
> > > > > > > http://davsclaus.com @davsclaus
> > > > > > > Camel in Action 2: https://www.manning.com/ibsen2
> > > > >
> > > > > --
> > > > > Claus Ibsen
> > > > > http://davsclaus.com @davsclaus
> > > > > Camel in Action 2: https://www.manning.com/ibsen2
Re: Kafka component Spring boot auto configuration
Posted by Luca Burgazzoli <lb...@gmail.com>.
I think you only need to add camel-kafka-starter as a dependency to
make it working.
Every component has a related starter that does the spring-boot magic
---
Luca Burgazzoli
On Thu, Dec 6, 2018 at 9:54 AM Yoshimo <yo...@protonmail.com.invalid> wrote:
>
> I have created a demo project which demonstrates the issue. The issue is in application.properties file and in TestRoute class.
>
> https://drive.google.com/file/d/1PDZj5tNXqqr0EjB07wluLiwy4ACK1eiP/view?usp=sharing
>
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Thursday, 6 December 2018 10:41, Yoshimo <yo...@protonmail.com> wrote:
>
> > I have attached a demo project which demonstrates the issue. The issue is in application.properties file and in TestRoute class.
> >
> > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> > On Wednesday, 5 December 2018 14:33, Yoshimo yoshimo@protonmail.com.INVALID wrote:
> >
> > > Hey Claus,
> > > I think that you misread my previous email. The value.deserializer = class org.apache.kafka.common.serialization.StringDeserializer line is the logging output of the Kafka consumer being created by Camel when I try adding the Spring Boot configuration.
> > > What I actually add to my yaml file is the following:
> > > camel.component.kafka.configuration.value-deserializer: com.openbet.platform.pricepublisher.serde.SportsEventDeserializer
> > > The above is being ignored and the Kafka consumer that gets created by Camel has the default deserialer.
> > > Thank you
> > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> > > On Wednesday, 5 December 2018 14:17, Claus Ibsen claus.ibsen@gmail.com wrote:
> > >
> > > > On Wed, Dec 5, 2018 at 12:56 PM Yoshimo yoshimo@protonmail.com wrote:
> > > >
> > > > > Here is what I add to my yaml file:
> > > > > camel.component.kafka.configuration.value-deserializer: com.openbet.platform.pricepublisher.serde.SportsEventDeserializer
> > > > > and here is my route:
> > > > >
> > > > > from("kafka:prices?brokers={{kafka.servers}}")
> > > > > .convertBodyTo(DBModel.class)
> > > > > ......
> > > > >
> > > > >
> > > > > With this setup when I start the app I can see in the logs the Kafka configuration in the consumer:
> > > > > value.deserializer = class org.apache.kafka.common.serialization.StringDeserializer
> > > >
> > > > Try with
> > > > value.deserializer = org.apache.kafka.common.serialization.StringDeserializer
> > > >
> > > > > and I get exceptions while deserializing. However if I remove the config from the yaml file and update my route to be:
> > > > > kafka:prices?brokers={{kafka.servers}}&valueDeserializer=com.openbet.platform.pricepublisher.serde.SportsEventDeserializer
> > > > > then I see the following in the Kafka consumer config:
> > > > > value.deserializer = class com.openbet.platform.pricepublisher.serde.SportsEventDeserializer
> > > > > and the app works as expected. Just for reference the rest of my yaml file is:
> > > > > kafka:
> > > > > sportsTopic: sports
> > > > > pricesTopic: prices
> > > > > servers: localhost:25445
> > > > > streams:
> > > > > "[application.id]": price-publisher
> > > > > "[bootstrap.servers]": ${kafka.servers}
> > > > > "[default.key.serde]": com.openbet.platform.pricepublisher.serde.ActivityKeySerDe
> > > > > "[default.value.serde]": com.openbet.platform.pricepublisher.serde.ActivityValueSerDe
> > > > > Thank you,
> > > > > Yiannis
> > > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> > > > > On Wednesday, 5 December 2018 13:30, Claus Ibsen claus.ibsen@gmail.com wrote:
> > > > >
> > > > > > Hi
> > > > > > Can you post your yaml configuration of how you try to configure this
> > > > > > kafka value deserializer
> > > > > > On Wed, Dec 5, 2018 at 11:36 AM Yoshimo yoshimo@protonmail.com wrote:
> > > > > >
> > > > > > > Hello Claus,
> > > > > > > thank you for looking into this.
> > > > > > > Camel: 2.23.0
> > > > > > > Kafka: 2.0.0
> > > > > > > SB: 2.1.0.RELEASE
> > > > > > > I am also attaching my pom if you want to have a look.
> > > > > > > Thank you,
> > > > > > > Yiannis
> > > > > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> > > > > > > On Wednesday, 5 December 2018 12:15, Claus Ibsen claus.ibsen@gmail.com wrote:
> > > > > > >
> > > > > > > > Hi
> > > > > > > > What version of Camel, Kafka and SB are you using?
> > > > > > > > On Tue, Dec 4, 2018 at 8:59 AM Yoshimo yoshimo@protonmail.com.invalid wrote:
> > > > > > > >
> > > > > > > > > Hello Camel users,
> > > > > > > > > I am trying to auto configure the Kafka component using spring boot, but the configuration is not working. More specifically I am following this page:
> > > > > > > > > https://github.com/apache/camel/blob/master/components/camel-kafka/src/main/docs/kafka-component.adoc
> > > > > > > > > and I am trying to set this configuration in my yaml file:
> > > > > > > > > camel.component.kafka.configuration.value-deserializer
> > > > > > > > > However the default deserializer is used when I consume from the Kafka topic. If I specify the deserializer in the Kafka route then it works as expected. Also I have defined the configuration camel.springboot.name in my yaml file and I can see that it is picked up normally. Are there any configurations that enable this functionality and I am missing?
> > > > > > > > > Thank you,
> > > > > > > > > Yiannis
> > > > > > > >
> > > > > > > > --
> > > > > > > > Claus Ibsen
> > > > > > > > http://davsclaus.com @davsclaus
> > > > > > > > Camel in Action 2: https://www.manning.com/ibsen2
> > > > > >
> > > > > > --
> > > > > > Claus Ibsen
> > > > > > http://davsclaus.com @davsclaus
> > > > > > Camel in Action 2: https://www.manning.com/ibsen2
> > > >
> > > > --
> > > > Claus Ibsen
> > > > http://davsclaus.com @davsclaus
> > > > Camel in Action 2: https://www.manning.com/ibsen2
>
>
Re: Kafka component Spring boot auto configuration
Posted by Yoshimo <yo...@protonmail.com.INVALID>.
I have created a demo project which demonstrates the issue. The issue is in application.properties file and in TestRoute class.
https://drive.google.com/file/d/1PDZj5tNXqqr0EjB07wluLiwy4ACK1eiP/view?usp=sharing
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Thursday, 6 December 2018 10:41, Yoshimo <yo...@protonmail.com> wrote:
> I have attached a demo project which demonstrates the issue. The issue is in application.properties file and in TestRoute class.
>
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Wednesday, 5 December 2018 14:33, Yoshimo yoshimo@protonmail.com.INVALID wrote:
>
> > Hey Claus,
> > I think that you misread my previous email. The value.deserializer = class org.apache.kafka.common.serialization.StringDeserializer line is the logging output of the Kafka consumer being created by Camel when I try adding the Spring Boot configuration.
> > What I actually add to my yaml file is the following:
> > camel.component.kafka.configuration.value-deserializer: com.openbet.platform.pricepublisher.serde.SportsEventDeserializer
> > The above is being ignored and the Kafka consumer that gets created by Camel has the default deserialer.
> > Thank you
> > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> > On Wednesday, 5 December 2018 14:17, Claus Ibsen claus.ibsen@gmail.com wrote:
> >
> > > On Wed, Dec 5, 2018 at 12:56 PM Yoshimo yoshimo@protonmail.com wrote:
> > >
> > > > Here is what I add to my yaml file:
> > > > camel.component.kafka.configuration.value-deserializer: com.openbet.platform.pricepublisher.serde.SportsEventDeserializer
> > > > and here is my route:
> > > >
> > > > from("kafka:prices?brokers={{kafka.servers}}")
> > > > .convertBodyTo(DBModel.class)
> > > > ......
> > > >
> > > >
> > > > With this setup when I start the app I can see in the logs the Kafka configuration in the consumer:
> > > > value.deserializer = class org.apache.kafka.common.serialization.StringDeserializer
> > >
> > > Try with
> > > value.deserializer = org.apache.kafka.common.serialization.StringDeserializer
> > >
> > > > and I get exceptions while deserializing. However if I remove the config from the yaml file and update my route to be:
> > > > kafka:prices?brokers={{kafka.servers}}&valueDeserializer=com.openbet.platform.pricepublisher.serde.SportsEventDeserializer
> > > > then I see the following in the Kafka consumer config:
> > > > value.deserializer = class com.openbet.platform.pricepublisher.serde.SportsEventDeserializer
> > > > and the app works as expected. Just for reference the rest of my yaml file is:
> > > > kafka:
> > > > sportsTopic: sports
> > > > pricesTopic: prices
> > > > servers: localhost:25445
> > > > streams:
> > > > "[application.id]": price-publisher
> > > > "[bootstrap.servers]": ${kafka.servers}
> > > > "[default.key.serde]": com.openbet.platform.pricepublisher.serde.ActivityKeySerDe
> > > > "[default.value.serde]": com.openbet.platform.pricepublisher.serde.ActivityValueSerDe
> > > > Thank you,
> > > > Yiannis
> > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> > > > On Wednesday, 5 December 2018 13:30, Claus Ibsen claus.ibsen@gmail.com wrote:
> > > >
> > > > > Hi
> > > > > Can you post your yaml configuration of how you try to configure this
> > > > > kafka value deserializer
> > > > > On Wed, Dec 5, 2018 at 11:36 AM Yoshimo yoshimo@protonmail.com wrote:
> > > > >
> > > > > > Hello Claus,
> > > > > > thank you for looking into this.
> > > > > > Camel: 2.23.0
> > > > > > Kafka: 2.0.0
> > > > > > SB: 2.1.0.RELEASE
> > > > > > I am also attaching my pom if you want to have a look.
> > > > > > Thank you,
> > > > > > Yiannis
> > > > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> > > > > > On Wednesday, 5 December 2018 12:15, Claus Ibsen claus.ibsen@gmail.com wrote:
> > > > > >
> > > > > > > Hi
> > > > > > > What version of Camel, Kafka and SB are you using?
> > > > > > > On Tue, Dec 4, 2018 at 8:59 AM Yoshimo yoshimo@protonmail.com.invalid wrote:
> > > > > > >
> > > > > > > > Hello Camel users,
> > > > > > > > I am trying to auto configure the Kafka component using spring boot, but the configuration is not working. More specifically I am following this page:
> > > > > > > > https://github.com/apache/camel/blob/master/components/camel-kafka/src/main/docs/kafka-component.adoc
> > > > > > > > and I am trying to set this configuration in my yaml file:
> > > > > > > > camel.component.kafka.configuration.value-deserializer
> > > > > > > > However the default deserializer is used when I consume from the Kafka topic. If I specify the deserializer in the Kafka route then it works as expected. Also I have defined the configuration camel.springboot.name in my yaml file and I can see that it is picked up normally. Are there any configurations that enable this functionality and I am missing?
> > > > > > > > Thank you,
> > > > > > > > Yiannis
> > > > > > >
> > > > > > > --
> > > > > > > Claus Ibsen
> > > > > > > http://davsclaus.com @davsclaus
> > > > > > > Camel in Action 2: https://www.manning.com/ibsen2
> > > > >
> > > > > --
> > > > > Claus Ibsen
> > > > > http://davsclaus.com @davsclaus
> > > > > Camel in Action 2: https://www.manning.com/ibsen2
> > >
> > > --
> > > Claus Ibsen
> > > http://davsclaus.com @davsclaus
> > > Camel in Action 2: https://www.manning.com/ibsen2
Re: Kafka component Spring boot auto configuration
Posted by Yoshimo <yo...@protonmail.com.INVALID>.
Hey Claus,
I think that you misread my previous email. The value.deserializer = class org.apache.kafka.common.serialization.StringDeserializer line is the logging output of the Kafka consumer being created by Camel when I try adding the Spring Boot configuration.
What I actually add to my yaml file is the following:
camel.component.kafka.configuration.value-deserializer: com.openbet.platform.pricepublisher.serde.SportsEventDeserializer
The above is being ignored and the Kafka consumer that gets created by Camel has the default deserialer.
Thank you
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Wednesday, 5 December 2018 14:17, Claus Ibsen <cl...@gmail.com> wrote:
> On Wed, Dec 5, 2018 at 12:56 PM Yoshimo yoshimo@protonmail.com wrote:
>
> > Here is what I add to my yaml file:
> > camel.component.kafka.configuration.value-deserializer: com.openbet.platform.pricepublisher.serde.SportsEventDeserializer
> > and here is my route:
> >
> > from("kafka:prices?brokers={{kafka.servers}}")
> > .convertBodyTo(DBModel.class)
> > ......
> >
> >
> > With this setup when I start the app I can see in the logs the Kafka configuration in the consumer:
> > value.deserializer = class org.apache.kafka.common.serialization.StringDeserializer
>
> Try with
>
> value.deserializer = org.apache.kafka.common.serialization.StringDeserializer
>
> > and I get exceptions while deserializing. However if I remove the config from the yaml file and update my route to be:
> > kafka:prices?brokers={{kafka.servers}}&valueDeserializer=com.openbet.platform.pricepublisher.serde.SportsEventDeserializer
> > then I see the following in the Kafka consumer config:
> > value.deserializer = class com.openbet.platform.pricepublisher.serde.SportsEventDeserializer
> > and the app works as expected. Just for reference the rest of my yaml file is:
> > kafka:
> > sportsTopic: sports
> > pricesTopic: prices
> > servers: localhost:25445
> > streams:
> > "[application.id]": price-publisher
> > "[bootstrap.servers]": ${kafka.servers}
> > "[default.key.serde]": com.openbet.platform.pricepublisher.serde.ActivityKeySerDe
> > "[default.value.serde]": com.openbet.platform.pricepublisher.serde.ActivityValueSerDe
> > Thank you,
> > Yiannis
> > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> > On Wednesday, 5 December 2018 13:30, Claus Ibsen claus.ibsen@gmail.com wrote:
> >
> > > Hi
> > > Can you post your yaml configuration of how you try to configure this
> > > kafka value deserializer
> > > On Wed, Dec 5, 2018 at 11:36 AM Yoshimo yoshimo@protonmail.com wrote:
> > >
> > > > Hello Claus,
> > > > thank you for looking into this.
> > > > Camel: 2.23.0
> > > > Kafka: 2.0.0
> > > > SB: 2.1.0.RELEASE
> > > > I am also attaching my pom if you want to have a look.
> > > > Thank you,
> > > > Yiannis
> > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> > > > On Wednesday, 5 December 2018 12:15, Claus Ibsen claus.ibsen@gmail.com wrote:
> > > >
> > > > > Hi
> > > > > What version of Camel, Kafka and SB are you using?
> > > > > On Tue, Dec 4, 2018 at 8:59 AM Yoshimo yoshimo@protonmail.com.invalid wrote:
> > > > >
> > > > > > Hello Camel users,
> > > > > > I am trying to auto configure the Kafka component using spring boot, but the configuration is not working. More specifically I am following this page:
> > > > > > https://github.com/apache/camel/blob/master/components/camel-kafka/src/main/docs/kafka-component.adoc
> > > > > > and I am trying to set this configuration in my yaml file:
> > > > > > camel.component.kafka.configuration.value-deserializer
> > > > > > However the default deserializer is used when I consume from the Kafka topic. If I specify the deserializer in the Kafka route then it works as expected. Also I have defined the configuration camel.springboot.name in my yaml file and I can see that it is picked up normally. Are there any configurations that enable this functionality and I am missing?
> > > > > > Thank you,
> > > > > > Yiannis
> > > > >
> > > > > --
> > > > > Claus Ibsen
> > > > > http://davsclaus.com @davsclaus
> > > > > Camel in Action 2: https://www.manning.com/ibsen2
> > >
> > > --
> > > Claus Ibsen
> > >
> > > http://davsclaus.com @davsclaus
> > > Camel in Action 2: https://www.manning.com/ibsen2
>
> --
>
> Claus Ibsen
>
> ------------
>
> http://davsclaus.com @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2
Re: Kafka component Spring boot auto configuration
Posted by Claus Ibsen <cl...@gmail.com>.
On Wed, Dec 5, 2018 at 12:56 PM Yoshimo <yo...@protonmail.com> wrote:
>
> Here is what I add to my yaml file:
>
> camel.component.kafka.configuration.value-deserializer: com.openbet.platform.pricepublisher.serde.SportsEventDeserializer
>
> and here is my route:
>
> from("kafka:prices?brokers={{kafka.servers}}")
> .convertBodyTo(DBModel.class)
> ......
>
> With this setup when I start the app I can see in the logs the Kafka configuration in the consumer:
>
> value.deserializer = class org.apache.kafka.common.serialization.StringDeserializer
>
Try with
value.deserializer = org.apache.kafka.common.serialization.StringDeserializer
> and I get exceptions while deserializing. However if I remove the config from the yaml file and update my route to be:
>
> kafka:prices?brokers={{kafka.servers}}&valueDeserializer=com.openbet.platform.pricepublisher.serde.SportsEventDeserializer
>
> then I see the following in the Kafka consumer config:
>
> value.deserializer = class com.openbet.platform.pricepublisher.serde.SportsEventDeserializer
>
> and the app works as expected. Just for reference the rest of my yaml file is:
>
> kafka:
> sportsTopic: sports
> pricesTopic: prices
> servers: localhost:25445
> streams:
> "[application.id]": price-publisher
> "[bootstrap.servers]": ${kafka.servers}
> "[default.key.serde]": com.openbet.platform.pricepublisher.serde.ActivityKeySerDe
> "[default.value.serde]": com.openbet.platform.pricepublisher.serde.ActivityValueSerDe
>
> Thank you,
> Yiannis
>
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Wednesday, 5 December 2018 13:30, Claus Ibsen <cl...@gmail.com> wrote:
>
> > Hi
> >
> > Can you post your yaml configuration of how you try to configure this
> > kafka value deserializer
> > On Wed, Dec 5, 2018 at 11:36 AM Yoshimo yoshimo@protonmail.com wrote:
> >
> > > Hello Claus,
> > > thank you for looking into this.
> > > Camel: 2.23.0
> > > Kafka: 2.0.0
> > > SB: 2.1.0.RELEASE
> > > I am also attaching my pom if you want to have a look.
> > > Thank you,
> > > Yiannis
> > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> > > On Wednesday, 5 December 2018 12:15, Claus Ibsen claus.ibsen@gmail.com wrote:
> > >
> > > > Hi
> > > > What version of Camel, Kafka and SB are you using?
> > > > On Tue, Dec 4, 2018 at 8:59 AM Yoshimo yoshimo@protonmail.com.invalid wrote:
> > > >
> > > > > Hello Camel users,
> > > > > I am trying to auto configure the Kafka component using spring boot, but the configuration is not working. More specifically I am following this page:
> > > > > https://github.com/apache/camel/blob/master/components/camel-kafka/src/main/docs/kafka-component.adoc
> > > > > and I am trying to set this configuration in my yaml file:
> > > > > camel.component.kafka.configuration.value-deserializer
> > > > > However the default deserializer is used when I consume from the Kafka topic. If I specify the deserializer in the Kafka route then it works as expected. Also I have defined the configuration camel.springboot.name in my yaml file and I can see that it is picked up normally. Are there any configurations that enable this functionality and I am missing?
> > > > > Thank you,
> > > > > Yiannis
> > > >
> > > > --
> > > > Claus Ibsen
> > > >
> > > > http://davsclaus.com @davsclaus
> > > > Camel in Action 2: https://www.manning.com/ibsen2
> >
> > --
> >
> > Claus Ibsen
> >
> > ------------
> >
> > http://davsclaus.com @davsclaus
> > Camel in Action 2: https://www.manning.com/ibsen2
>
>
--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
Re: Kafka component Spring boot auto configuration
Posted by Yoshimo <yo...@protonmail.com.INVALID>.
Here is what I add to my yaml file:
camel.component.kafka.configuration.value-deserializer: com.openbet.platform.pricepublisher.serde.SportsEventDeserializer
and here is my route:
from("kafka:prices?brokers={{kafka.servers}}")
.convertBodyTo(DBModel.class)
......
With this setup when I start the app I can see in the logs the Kafka configuration in the consumer:
value.deserializer = class org.apache.kafka.common.serialization.StringDeserializer
and I get exceptions while deserializing. However if I remove the config from the yaml file and update my route to be:
kafka:prices?brokers={{kafka.servers}}&valueDeserializer=com.openbet.platform.pricepublisher.serde.SportsEventDeserializer
then I see the following in the Kafka consumer config:
value.deserializer = class com.openbet.platform.pricepublisher.serde.SportsEventDeserializer
and the app works as expected. Just for reference the rest of my yaml file is:
kafka:
sportsTopic: sports
pricesTopic: prices
servers: localhost:25445
streams:
"[application.id]": price-publisher
"[bootstrap.servers]": ${kafka.servers}
"[default.key.serde]": com.openbet.platform.pricepublisher.serde.ActivityKeySerDe
"[default.value.serde]": com.openbet.platform.pricepublisher.serde.ActivityValueSerDe
Thank you,
Yiannis
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Wednesday, 5 December 2018 13:30, Claus Ibsen <cl...@gmail.com> wrote:
> Hi
>
> Can you post your yaml configuration of how you try to configure this
> kafka value deserializer
> On Wed, Dec 5, 2018 at 11:36 AM Yoshimo yoshimo@protonmail.com wrote:
>
> > Hello Claus,
> > thank you for looking into this.
> > Camel: 2.23.0
> > Kafka: 2.0.0
> > SB: 2.1.0.RELEASE
> > I am also attaching my pom if you want to have a look.
> > Thank you,
> > Yiannis
> > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> > On Wednesday, 5 December 2018 12:15, Claus Ibsen claus.ibsen@gmail.com wrote:
> >
> > > Hi
> > > What version of Camel, Kafka and SB are you using?
> > > On Tue, Dec 4, 2018 at 8:59 AM Yoshimo yoshimo@protonmail.com.invalid wrote:
> > >
> > > > Hello Camel users,
> > > > I am trying to auto configure the Kafka component using spring boot, but the configuration is not working. More specifically I am following this page:
> > > > https://github.com/apache/camel/blob/master/components/camel-kafka/src/main/docs/kafka-component.adoc
> > > > and I am trying to set this configuration in my yaml file:
> > > > camel.component.kafka.configuration.value-deserializer
> > > > However the default deserializer is used when I consume from the Kafka topic. If I specify the deserializer in the Kafka route then it works as expected. Also I have defined the configuration camel.springboot.name in my yaml file and I can see that it is picked up normally. Are there any configurations that enable this functionality and I am missing?
> > > > Thank you,
> > > > Yiannis
> > >
> > > --
> > > Claus Ibsen
> > >
> > > http://davsclaus.com @davsclaus
> > > Camel in Action 2: https://www.manning.com/ibsen2
>
> --
>
> Claus Ibsen
>
> ------------
>
> http://davsclaus.com @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2
Re: Kafka component Spring boot auto configuration
Posted by Claus Ibsen <cl...@gmail.com>.
Hi
What version of Camel, Kafka and SB are you using?
On Tue, Dec 4, 2018 at 8:59 AM Yoshimo <yo...@protonmail.com.invalid> wrote:
>
> Hello Camel users,
>
> I am trying to auto configure the Kafka component using spring boot, but the configuration is not working. More specifically I am following this page:
>
> https://github.com/apache/camel/blob/master/components/camel-kafka/src/main/docs/kafka-component.adoc
>
> and I am trying to set this configuration in my yaml file:
> camel.component.kafka.configuration.value-deserializer
>
> However the default deserializer is used when I consume from the Kafka topic. If I specify the deserializer in the Kafka route then it works as expected. Also I have defined the configuration camel.springboot.name in my yaml file and I can see that it is picked up normally. Are there any configurations that enable this functionality and I am missing?
>
> Thank you,
> Yiannis
--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2