You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Martin Pelikán <pe...@gmail.com> on 2023/01/05 16:12:22 UTC
Objects not propagated to Camel Kafka 3.18.4
Hi Camel experts,
I am struggling with a Kafka component and passing in parameters via
additionalProperties. I am not able to propagate an Object. I tested
only Endpoint DSL. According to the Camel docs and APIs, I am allowed
to pass in an instance of Map<String, Object>
Camel version: 3.18.4
----Pseudocode-----
@Configuration
Class SpringConfig
@Bean(name = "autowiredMap")
public Map<String, Object> autowiredMap() {
var map = new HashMap<String, Object>();
map.put("testStringKey3", "testStringValue3");
map.put("testObjectKey3", new Object());
}
-------
Class MainRoute extends RouteBuilder()
@Autowired
@Qualifier("autowiredMap")
Map<String, Object> autowiredMap;
var map = new HashMap<String, Object>();
map.put("testStringKey1", "testStringValue1");
map.put("testObjectKey1", new Object());
from(kafka(…basic configuration works)
.schemaRegistryURL(http://original.com)
.additionalProperties(map)
.additionalProperties("testStringKey2", "testStringValue2")
.additionalProperties("testObjectKey1", new Object())
.additionalProperties("schema.registry.url", http://overwritten.com)
.additionalProperties(autowiredMap)...
Expected:
All properties from all additionalProperties methods are propagated to
the Kafka Properties configuration object.
In reality, Properties will contain the following:
schema.registry.url=http://overwritten.com
testStringKey1=testStringValue1
testStringKey2=testStringValue2
Non-string values are not propagated at all. Values from autowired Map
are not propagated at all. Autowired map is treated as
additionalProperties.autowiredMap=HashMap@123456
I can see those String values being added to the Kafka Endpoint URI:
kafka://additionalProperties.testStringKey1=xxx&additionalProperties.testStringKey2=xxx&additionalProperties.schema.registr.url=xxx..
Is it a bug in Camel or did I understand the documentation wrong? I
can prepare a test case later
Best regards,
Martin
Re: Objects not propagated to Camel Kafka 3.18.4
Posted by Claus Ibsen <cl...@gmail.com>.
I created a JIRA to not forget
https://issues.apache.org/jira/browse/CAMEL-18938
On Fri, Jan 6, 2023 at 10:06 AM Claus Ibsen <cl...@gmail.com> wrote:
> You can also use string uri instead of endpoint dsl, then you can refer to
> the via kafka:....?additionalProperties=#myMapBeanId
>
> So we may need for endpoint dsl to include a fluent builder for Map that
> is a string value to use for bean reference lookup
>
> On Fri, Jan 6, 2023 at 9:31 AM Claus Ibsen <cl...@gmail.com> wrote:
>
>>
>>
>> On Fri, Jan 6, 2023 at 9:17 AM Martin Pelikán <pe...@gmail.com> wrote:
>>
>>> Hi Claus,
>>>
>>> I tested your approach but wasn't successful. Reference is stored in
>>> Properties as text. I confirmed bean is registered in Spring context
>>> with the proper BeanID.
>>>
>>> @Bean(name = "tokenCredential")
>>> public TokenCredential tokenCredential() {...create an instance}
>>>
>>> in debugger (simplified) ->
>>> camelContext.registry.lookupByName("tokenCredential") -> returns valid
>>> object
>>> .....
>>> .additionalProperties("schema.registry.credential", "#tokenCredential")
>>>
>>> results into
>>> "schema.registry.credential" = "#tokenCredential"
>>>
>>> I guess the hash reference notation works only for Camel-specific
>>> fields, like
>>> .headerDeserializer("#customKafkaHeaderDeserializer") or
>>> .headerFilterStrategy("#customHeaderFilterStrategy") // verified, works
>>>
>>> Any ideas?
>>>
>>
>> Okay so we should make this possible for additional properties as well.
>> You are welcome to create a JIRA.
>>
>>
>>
>>>
>>> čt 5. 1. 2023 v 18:22 odesílatel Claus Ibsen <cl...@gmail.com>
>>> napsal:
>>> >
>>> > You cannot use complex object types, only string literal, numbers,
>>> booleans
>>> > etc.
>>> >
>>> > So what you need to do is to create those objects with some bean id
>>> (spring
>>> > boot @Bean stuff) and then refer to them via their bean id.
>>> >
>>> > Something ala
>>> >
>>> >
>>> > .additionallProperties("xxx", "#myBeanId")
>>> >
>>> >
>>> >
>>> >
>>> > On Thu, Jan 5, 2023 at 6:10 PM Martin Pelikán <pe...@gmail.com>
>>> wrote:
>>> >
>>> > > My case is related to connecting to Azure EventHubs Avro Schema
>>> Registry.
>>> > >
>>> > > TokenCredential cred = new
>>> > >
>>> > >
>>> ClientSecretCredentialBuilder().tenantId("xxx").clientId("xxx").clientSecret("xx").build();
>>> > > props.put("schema.registry.credential", cred);
>>> > >
>>> > > Another case that comes to my mind is with OAUTHBEARER
>>> authentication.
>>> > > Users could supply instantiated AuthenticateCallbackHandler class to
>>> > > the sasl.login.callback.handler.class property.
>>> > >
>>> > > TokenCredential as part of Properties works with
>>> > > org.apache.kafka.clients.consumer.KafkaConsumer.
>>> > >
>>> > > čt 5. 1. 2023 v 17:59 odesílatel Claus Ibsen <cl...@gmail.com>
>>> > > napsal:
>>> > > >
>>> > > > Whats the real world use-case for additional properties as Object
>>> or Map.
>>> > > > What would kafka understand and use these for?
>>> > > >
>>> > > > On Thu, Jan 5, 2023 at 5:12 PM Martin Pelikán <pe...@gmail.com>
>>> > > wrote:
>>> > > >
>>> > > > > Hi Camel experts,
>>> > > > >
>>> > > > > I am struggling with a Kafka component and passing in parameters
>>> via
>>> > > > > additionalProperties. I am not able to propagate an Object. I
>>> tested
>>> > > > > only Endpoint DSL. According to the Camel docs and APIs, I am
>>> allowed
>>> > > > > to pass in an instance of Map<String, Object>
>>> > > > >
>>> > > > > Camel version: 3.18.4
>>> > > > >
>>> > > > > ----Pseudocode-----
>>> > > > >
>>> > > > > @Configuration
>>> > > > > Class SpringConfig
>>> > > > >
>>> > > > > @Bean(name = "autowiredMap")
>>> > > > > public Map<String, Object> autowiredMap() {
>>> > > > > var map = new HashMap<String, Object>();
>>> > > > > map.put("testStringKey3", "testStringValue3");
>>> > > > > map.put("testObjectKey3", new Object());
>>> > > > > }
>>> > > > >
>>> > > > > -------
>>> > > > >
>>> > > > > Class MainRoute extends RouteBuilder()
>>> > > > >
>>> > > > > @Autowired
>>> > > > > @Qualifier("autowiredMap")
>>> > > > > Map<String, Object> autowiredMap;
>>> > > > >
>>> > > > > var map = new HashMap<String, Object>();
>>> > > > > map.put("testStringKey1", "testStringValue1");
>>> > > > > map.put("testObjectKey1", new Object());
>>> > > > >
>>> > > > > from(kafka(…basic configuration works)
>>> > > > > .schemaRegistryURL(http://original.com)
>>> > > > > .additionalProperties(map)
>>> > > > > .additionalProperties("testStringKey2", "testStringValue2")
>>> > > > > .additionalProperties("testObjectKey1", new Object())
>>> > > > > .additionalProperties("schema.registry.url",
>>> > > http://overwritten.com)
>>> > > > > .additionalProperties(autowiredMap)...
>>> > > > >
>>> > > > > Expected:
>>> > > > >
>>> > > > > All properties from all additionalProperties methods are
>>> propagated to
>>> > > > > the Kafka Properties configuration object.
>>> > > > >
>>> > > > > In reality, Properties will contain the following:
>>> > > > > schema.registry.url=http://overwritten.com
>>> > > > > testStringKey1=testStringValue1
>>> > > > > testStringKey2=testStringValue2
>>> > > > >
>>> > > > > Non-string values are not propagated at all. Values from
>>> autowired Map
>>> > > > > are not propagated at all. Autowired map is treated as
>>> > > > > additionalProperties.autowiredMap=HashMap@123456
>>> > > > >
>>> > > > > I can see those String values being added to the Kafka Endpoint
>>> URI:
>>> > > > >
>>> > > > >
>>> > >
>>> kafka://additionalProperties.testStringKey1=xxx&additionalProperties.testStringKey2=xxx&additionalProperties.schema.registr.url=xxx..
>>> > > > >
>>> > > > > Is it a bug in Camel or did I understand the documentation
>>> wrong? I
>>> > > > > can prepare a test case later
>>> > > > >
>>> > > > > Best regards,
>>> > > > >
>>> > > > > Martin
>>> > > > >
>>> > > >
>>> > > >
>>> > > > --
>>> > > > Claus Ibsen
>>> > > > -----------------
>>> > > > @davsclaus
>>> > > > Camel in Action 2: https://www.manning.com/ibsen2
>>> > >
>>> >
>>> >
>>> > --
>>> > Claus Ibsen
>>> > -----------------
>>> > @davsclaus
>>> > Camel in Action 2: https://www.manning.com/ibsen2
>>>
>>
>>
>> --
>> Claus Ibsen
>> -----------------
>> @davsclaus
>> Camel in Action 2: https://www.manning.com/ibsen2
>>
>
>
> --
> Claus Ibsen
> -----------------
> @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2
>
--
Claus Ibsen
-----------------
@davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
Re: Objects not propagated to Camel Kafka 3.18.4
Posted by Claus Ibsen <cl...@gmail.com>.
You can also use string uri instead of endpoint dsl, then you can refer to
the via kafka:....?additionalProperties=#myMapBeanId
So we may need for endpoint dsl to include a fluent builder for Map that is
a string value to use for bean reference lookup
On Fri, Jan 6, 2023 at 9:31 AM Claus Ibsen <cl...@gmail.com> wrote:
>
>
> On Fri, Jan 6, 2023 at 9:17 AM Martin Pelikán <pe...@gmail.com> wrote:
>
>> Hi Claus,
>>
>> I tested your approach but wasn't successful. Reference is stored in
>> Properties as text. I confirmed bean is registered in Spring context
>> with the proper BeanID.
>>
>> @Bean(name = "tokenCredential")
>> public TokenCredential tokenCredential() {...create an instance}
>>
>> in debugger (simplified) ->
>> camelContext.registry.lookupByName("tokenCredential") -> returns valid
>> object
>> .....
>> .additionalProperties("schema.registry.credential", "#tokenCredential")
>>
>> results into
>> "schema.registry.credential" = "#tokenCredential"
>>
>> I guess the hash reference notation works only for Camel-specific fields,
>> like
>> .headerDeserializer("#customKafkaHeaderDeserializer") or
>> .headerFilterStrategy("#customHeaderFilterStrategy") // verified, works
>>
>> Any ideas?
>>
>
> Okay so we should make this possible for additional properties as well.
> You are welcome to create a JIRA.
>
>
>
>>
>> čt 5. 1. 2023 v 18:22 odesílatel Claus Ibsen <cl...@gmail.com>
>> napsal:
>> >
>> > You cannot use complex object types, only string literal, numbers,
>> booleans
>> > etc.
>> >
>> > So what you need to do is to create those objects with some bean id
>> (spring
>> > boot @Bean stuff) and then refer to them via their bean id.
>> >
>> > Something ala
>> >
>> >
>> > .additionallProperties("xxx", "#myBeanId")
>> >
>> >
>> >
>> >
>> > On Thu, Jan 5, 2023 at 6:10 PM Martin Pelikán <pe...@gmail.com>
>> wrote:
>> >
>> > > My case is related to connecting to Azure EventHubs Avro Schema
>> Registry.
>> > >
>> > > TokenCredential cred = new
>> > >
>> > >
>> ClientSecretCredentialBuilder().tenantId("xxx").clientId("xxx").clientSecret("xx").build();
>> > > props.put("schema.registry.credential", cred);
>> > >
>> > > Another case that comes to my mind is with OAUTHBEARER authentication.
>> > > Users could supply instantiated AuthenticateCallbackHandler class to
>> > > the sasl.login.callback.handler.class property.
>> > >
>> > > TokenCredential as part of Properties works with
>> > > org.apache.kafka.clients.consumer.KafkaConsumer.
>> > >
>> > > čt 5. 1. 2023 v 17:59 odesílatel Claus Ibsen <cl...@gmail.com>
>> > > napsal:
>> > > >
>> > > > Whats the real world use-case for additional properties as Object
>> or Map.
>> > > > What would kafka understand and use these for?
>> > > >
>> > > > On Thu, Jan 5, 2023 at 5:12 PM Martin Pelikán <pe...@gmail.com>
>> > > wrote:
>> > > >
>> > > > > Hi Camel experts,
>> > > > >
>> > > > > I am struggling with a Kafka component and passing in parameters
>> via
>> > > > > additionalProperties. I am not able to propagate an Object. I
>> tested
>> > > > > only Endpoint DSL. According to the Camel docs and APIs, I am
>> allowed
>> > > > > to pass in an instance of Map<String, Object>
>> > > > >
>> > > > > Camel version: 3.18.4
>> > > > >
>> > > > > ----Pseudocode-----
>> > > > >
>> > > > > @Configuration
>> > > > > Class SpringConfig
>> > > > >
>> > > > > @Bean(name = "autowiredMap")
>> > > > > public Map<String, Object> autowiredMap() {
>> > > > > var map = new HashMap<String, Object>();
>> > > > > map.put("testStringKey3", "testStringValue3");
>> > > > > map.put("testObjectKey3", new Object());
>> > > > > }
>> > > > >
>> > > > > -------
>> > > > >
>> > > > > Class MainRoute extends RouteBuilder()
>> > > > >
>> > > > > @Autowired
>> > > > > @Qualifier("autowiredMap")
>> > > > > Map<String, Object> autowiredMap;
>> > > > >
>> > > > > var map = new HashMap<String, Object>();
>> > > > > map.put("testStringKey1", "testStringValue1");
>> > > > > map.put("testObjectKey1", new Object());
>> > > > >
>> > > > > from(kafka(…basic configuration works)
>> > > > > .schemaRegistryURL(http://original.com)
>> > > > > .additionalProperties(map)
>> > > > > .additionalProperties("testStringKey2", "testStringValue2")
>> > > > > .additionalProperties("testObjectKey1", new Object())
>> > > > > .additionalProperties("schema.registry.url",
>> > > http://overwritten.com)
>> > > > > .additionalProperties(autowiredMap)...
>> > > > >
>> > > > > Expected:
>> > > > >
>> > > > > All properties from all additionalProperties methods are
>> propagated to
>> > > > > the Kafka Properties configuration object.
>> > > > >
>> > > > > In reality, Properties will contain the following:
>> > > > > schema.registry.url=http://overwritten.com
>> > > > > testStringKey1=testStringValue1
>> > > > > testStringKey2=testStringValue2
>> > > > >
>> > > > > Non-string values are not propagated at all. Values from
>> autowired Map
>> > > > > are not propagated at all. Autowired map is treated as
>> > > > > additionalProperties.autowiredMap=HashMap@123456
>> > > > >
>> > > > > I can see those String values being added to the Kafka Endpoint
>> URI:
>> > > > >
>> > > > >
>> > >
>> kafka://additionalProperties.testStringKey1=xxx&additionalProperties.testStringKey2=xxx&additionalProperties.schema.registr.url=xxx..
>> > > > >
>> > > > > Is it a bug in Camel or did I understand the documentation wrong?
>> I
>> > > > > can prepare a test case later
>> > > > >
>> > > > > Best regards,
>> > > > >
>> > > > > Martin
>> > > > >
>> > > >
>> > > >
>> > > > --
>> > > > Claus Ibsen
>> > > > -----------------
>> > > > @davsclaus
>> > > > Camel in Action 2: https://www.manning.com/ibsen2
>> > >
>> >
>> >
>> > --
>> > Claus Ibsen
>> > -----------------
>> > @davsclaus
>> > Camel in Action 2: https://www.manning.com/ibsen2
>>
>
>
> --
> Claus Ibsen
> -----------------
> @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2
>
--
Claus Ibsen
-----------------
@davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
Re: Objects not propagated to Camel Kafka 3.18.4
Posted by Claus Ibsen <cl...@gmail.com>.
On Fri, Jan 6, 2023 at 9:17 AM Martin Pelikán <pe...@gmail.com> wrote:
> Hi Claus,
>
> I tested your approach but wasn't successful. Reference is stored in
> Properties as text. I confirmed bean is registered in Spring context
> with the proper BeanID.
>
> @Bean(name = "tokenCredential")
> public TokenCredential tokenCredential() {...create an instance}
>
> in debugger (simplified) ->
> camelContext.registry.lookupByName("tokenCredential") -> returns valid
> object
> .....
> .additionalProperties("schema.registry.credential", "#tokenCredential")
>
> results into
> "schema.registry.credential" = "#tokenCredential"
>
> I guess the hash reference notation works only for Camel-specific fields,
> like
> .headerDeserializer("#customKafkaHeaderDeserializer") or
> .headerFilterStrategy("#customHeaderFilterStrategy") // verified, works
>
> Any ideas?
>
Okay so we should make this possible for additional properties as well. You
are welcome to create a JIRA.
>
> čt 5. 1. 2023 v 18:22 odesílatel Claus Ibsen <cl...@gmail.com>
> napsal:
> >
> > You cannot use complex object types, only string literal, numbers,
> booleans
> > etc.
> >
> > So what you need to do is to create those objects with some bean id
> (spring
> > boot @Bean stuff) and then refer to them via their bean id.
> >
> > Something ala
> >
> >
> > .additionallProperties("xxx", "#myBeanId")
> >
> >
> >
> >
> > On Thu, Jan 5, 2023 at 6:10 PM Martin Pelikán <pe...@gmail.com>
> wrote:
> >
> > > My case is related to connecting to Azure EventHubs Avro Schema
> Registry.
> > >
> > > TokenCredential cred = new
> > >
> > >
> ClientSecretCredentialBuilder().tenantId("xxx").clientId("xxx").clientSecret("xx").build();
> > > props.put("schema.registry.credential", cred);
> > >
> > > Another case that comes to my mind is with OAUTHBEARER authentication.
> > > Users could supply instantiated AuthenticateCallbackHandler class to
> > > the sasl.login.callback.handler.class property.
> > >
> > > TokenCredential as part of Properties works with
> > > org.apache.kafka.clients.consumer.KafkaConsumer.
> > >
> > > čt 5. 1. 2023 v 17:59 odesílatel Claus Ibsen <cl...@gmail.com>
> > > napsal:
> > > >
> > > > Whats the real world use-case for additional properties as Object or
> Map.
> > > > What would kafka understand and use these for?
> > > >
> > > > On Thu, Jan 5, 2023 at 5:12 PM Martin Pelikán <pe...@gmail.com>
> > > wrote:
> > > >
> > > > > Hi Camel experts,
> > > > >
> > > > > I am struggling with a Kafka component and passing in parameters
> via
> > > > > additionalProperties. I am not able to propagate an Object. I
> tested
> > > > > only Endpoint DSL. According to the Camel docs and APIs, I am
> allowed
> > > > > to pass in an instance of Map<String, Object>
> > > > >
> > > > > Camel version: 3.18.4
> > > > >
> > > > > ----Pseudocode-----
> > > > >
> > > > > @Configuration
> > > > > Class SpringConfig
> > > > >
> > > > > @Bean(name = "autowiredMap")
> > > > > public Map<String, Object> autowiredMap() {
> > > > > var map = new HashMap<String, Object>();
> > > > > map.put("testStringKey3", "testStringValue3");
> > > > > map.put("testObjectKey3", new Object());
> > > > > }
> > > > >
> > > > > -------
> > > > >
> > > > > Class MainRoute extends RouteBuilder()
> > > > >
> > > > > @Autowired
> > > > > @Qualifier("autowiredMap")
> > > > > Map<String, Object> autowiredMap;
> > > > >
> > > > > var map = new HashMap<String, Object>();
> > > > > map.put("testStringKey1", "testStringValue1");
> > > > > map.put("testObjectKey1", new Object());
> > > > >
> > > > > from(kafka(…basic configuration works)
> > > > > .schemaRegistryURL(http://original.com)
> > > > > .additionalProperties(map)
> > > > > .additionalProperties("testStringKey2", "testStringValue2")
> > > > > .additionalProperties("testObjectKey1", new Object())
> > > > > .additionalProperties("schema.registry.url",
> > > http://overwritten.com)
> > > > > .additionalProperties(autowiredMap)...
> > > > >
> > > > > Expected:
> > > > >
> > > > > All properties from all additionalProperties methods are
> propagated to
> > > > > the Kafka Properties configuration object.
> > > > >
> > > > > In reality, Properties will contain the following:
> > > > > schema.registry.url=http://overwritten.com
> > > > > testStringKey1=testStringValue1
> > > > > testStringKey2=testStringValue2
> > > > >
> > > > > Non-string values are not propagated at all. Values from autowired
> Map
> > > > > are not propagated at all. Autowired map is treated as
> > > > > additionalProperties.autowiredMap=HashMap@123456
> > > > >
> > > > > I can see those String values being added to the Kafka Endpoint
> URI:
> > > > >
> > > > >
> > >
> kafka://additionalProperties.testStringKey1=xxx&additionalProperties.testStringKey2=xxx&additionalProperties.schema.registr.url=xxx..
> > > > >
> > > > > Is it a bug in Camel or did I understand the documentation wrong? I
> > > > > can prepare a test case later
> > > > >
> > > > > Best regards,
> > > > >
> > > > > Martin
> > > > >
> > > >
> > > >
> > > > --
> > > > Claus Ibsen
> > > > -----------------
> > > > @davsclaus
> > > > Camel in Action 2: https://www.manning.com/ibsen2
> > >
> >
> >
> > --
> > Claus Ibsen
> > -----------------
> > @davsclaus
> > Camel in Action 2: https://www.manning.com/ibsen2
>
--
Claus Ibsen
-----------------
@davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
Re: Objects not propagated to Camel Kafka 3.18.4
Posted by Martin Pelikán <pe...@gmail.com>.
Hi Claus,
I tested your approach but wasn't successful. Reference is stored in
Properties as text. I confirmed bean is registered in Spring context
with the proper BeanID.
@Bean(name = "tokenCredential")
public TokenCredential tokenCredential() {...create an instance}
in debugger (simplified) ->
camelContext.registry.lookupByName("tokenCredential") -> returns valid
object
.....
.additionalProperties("schema.registry.credential", "#tokenCredential")
results into
"schema.registry.credential" = "#tokenCredential"
I guess the hash reference notation works only for Camel-specific fields, like
.headerDeserializer("#customKafkaHeaderDeserializer") or
.headerFilterStrategy("#customHeaderFilterStrategy") // verified, works
Any ideas?
čt 5. 1. 2023 v 18:22 odesílatel Claus Ibsen <cl...@gmail.com> napsal:
>
> You cannot use complex object types, only string literal, numbers, booleans
> etc.
>
> So what you need to do is to create those objects with some bean id (spring
> boot @Bean stuff) and then refer to them via their bean id.
>
> Something ala
>
>
> .additionallProperties("xxx", "#myBeanId")
>
>
>
>
> On Thu, Jan 5, 2023 at 6:10 PM Martin Pelikán <pe...@gmail.com> wrote:
>
> > My case is related to connecting to Azure EventHubs Avro Schema Registry.
> >
> > TokenCredential cred = new
> >
> > ClientSecretCredentialBuilder().tenantId("xxx").clientId("xxx").clientSecret("xx").build();
> > props.put("schema.registry.credential", cred);
> >
> > Another case that comes to my mind is with OAUTHBEARER authentication.
> > Users could supply instantiated AuthenticateCallbackHandler class to
> > the sasl.login.callback.handler.class property.
> >
> > TokenCredential as part of Properties works with
> > org.apache.kafka.clients.consumer.KafkaConsumer.
> >
> > čt 5. 1. 2023 v 17:59 odesílatel Claus Ibsen <cl...@gmail.com>
> > napsal:
> > >
> > > Whats the real world use-case for additional properties as Object or Map.
> > > What would kafka understand and use these for?
> > >
> > > On Thu, Jan 5, 2023 at 5:12 PM Martin Pelikán <pe...@gmail.com>
> > wrote:
> > >
> > > > Hi Camel experts,
> > > >
> > > > I am struggling with a Kafka component and passing in parameters via
> > > > additionalProperties. I am not able to propagate an Object. I tested
> > > > only Endpoint DSL. According to the Camel docs and APIs, I am allowed
> > > > to pass in an instance of Map<String, Object>
> > > >
> > > > Camel version: 3.18.4
> > > >
> > > > ----Pseudocode-----
> > > >
> > > > @Configuration
> > > > Class SpringConfig
> > > >
> > > > @Bean(name = "autowiredMap")
> > > > public Map<String, Object> autowiredMap() {
> > > > var map = new HashMap<String, Object>();
> > > > map.put("testStringKey3", "testStringValue3");
> > > > map.put("testObjectKey3", new Object());
> > > > }
> > > >
> > > > -------
> > > >
> > > > Class MainRoute extends RouteBuilder()
> > > >
> > > > @Autowired
> > > > @Qualifier("autowiredMap")
> > > > Map<String, Object> autowiredMap;
> > > >
> > > > var map = new HashMap<String, Object>();
> > > > map.put("testStringKey1", "testStringValue1");
> > > > map.put("testObjectKey1", new Object());
> > > >
> > > > from(kafka(…basic configuration works)
> > > > .schemaRegistryURL(http://original.com)
> > > > .additionalProperties(map)
> > > > .additionalProperties("testStringKey2", "testStringValue2")
> > > > .additionalProperties("testObjectKey1", new Object())
> > > > .additionalProperties("schema.registry.url",
> > http://overwritten.com)
> > > > .additionalProperties(autowiredMap)...
> > > >
> > > > Expected:
> > > >
> > > > All properties from all additionalProperties methods are propagated to
> > > > the Kafka Properties configuration object.
> > > >
> > > > In reality, Properties will contain the following:
> > > > schema.registry.url=http://overwritten.com
> > > > testStringKey1=testStringValue1
> > > > testStringKey2=testStringValue2
> > > >
> > > > Non-string values are not propagated at all. Values from autowired Map
> > > > are not propagated at all. Autowired map is treated as
> > > > additionalProperties.autowiredMap=HashMap@123456
> > > >
> > > > I can see those String values being added to the Kafka Endpoint URI:
> > > >
> > > >
> > kafka://additionalProperties.testStringKey1=xxx&additionalProperties.testStringKey2=xxx&additionalProperties.schema.registr.url=xxx..
> > > >
> > > > Is it a bug in Camel or did I understand the documentation wrong? I
> > > > can prepare a test case later
> > > >
> > > > Best regards,
> > > >
> > > > Martin
> > > >
> > >
> > >
> > > --
> > > Claus Ibsen
> > > -----------------
> > > @davsclaus
> > > Camel in Action 2: https://www.manning.com/ibsen2
> >
>
>
> --
> Claus Ibsen
> -----------------
> @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2
Re: Objects not propagated to Camel Kafka 3.18.4
Posted by Claus Ibsen <cl...@gmail.com>.
You cannot use complex object types, only string literal, numbers, booleans
etc.
So what you need to do is to create those objects with some bean id (spring
boot @Bean stuff) and then refer to them via their bean id.
Something ala
.additionallProperties("xxx", "#myBeanId")
On Thu, Jan 5, 2023 at 6:10 PM Martin Pelikán <pe...@gmail.com> wrote:
> My case is related to connecting to Azure EventHubs Avro Schema Registry.
>
> TokenCredential cred = new
>
> ClientSecretCredentialBuilder().tenantId("xxx").clientId("xxx").clientSecret("xx").build();
> props.put("schema.registry.credential", cred);
>
> Another case that comes to my mind is with OAUTHBEARER authentication.
> Users could supply instantiated AuthenticateCallbackHandler class to
> the sasl.login.callback.handler.class property.
>
> TokenCredential as part of Properties works with
> org.apache.kafka.clients.consumer.KafkaConsumer.
>
> čt 5. 1. 2023 v 17:59 odesílatel Claus Ibsen <cl...@gmail.com>
> napsal:
> >
> > Whats the real world use-case for additional properties as Object or Map.
> > What would kafka understand and use these for?
> >
> > On Thu, Jan 5, 2023 at 5:12 PM Martin Pelikán <pe...@gmail.com>
> wrote:
> >
> > > Hi Camel experts,
> > >
> > > I am struggling with a Kafka component and passing in parameters via
> > > additionalProperties. I am not able to propagate an Object. I tested
> > > only Endpoint DSL. According to the Camel docs and APIs, I am allowed
> > > to pass in an instance of Map<String, Object>
> > >
> > > Camel version: 3.18.4
> > >
> > > ----Pseudocode-----
> > >
> > > @Configuration
> > > Class SpringConfig
> > >
> > > @Bean(name = "autowiredMap")
> > > public Map<String, Object> autowiredMap() {
> > > var map = new HashMap<String, Object>();
> > > map.put("testStringKey3", "testStringValue3");
> > > map.put("testObjectKey3", new Object());
> > > }
> > >
> > > -------
> > >
> > > Class MainRoute extends RouteBuilder()
> > >
> > > @Autowired
> > > @Qualifier("autowiredMap")
> > > Map<String, Object> autowiredMap;
> > >
> > > var map = new HashMap<String, Object>();
> > > map.put("testStringKey1", "testStringValue1");
> > > map.put("testObjectKey1", new Object());
> > >
> > > from(kafka(…basic configuration works)
> > > .schemaRegistryURL(http://original.com)
> > > .additionalProperties(map)
> > > .additionalProperties("testStringKey2", "testStringValue2")
> > > .additionalProperties("testObjectKey1", new Object())
> > > .additionalProperties("schema.registry.url",
> http://overwritten.com)
> > > .additionalProperties(autowiredMap)...
> > >
> > > Expected:
> > >
> > > All properties from all additionalProperties methods are propagated to
> > > the Kafka Properties configuration object.
> > >
> > > In reality, Properties will contain the following:
> > > schema.registry.url=http://overwritten.com
> > > testStringKey1=testStringValue1
> > > testStringKey2=testStringValue2
> > >
> > > Non-string values are not propagated at all. Values from autowired Map
> > > are not propagated at all. Autowired map is treated as
> > > additionalProperties.autowiredMap=HashMap@123456
> > >
> > > I can see those String values being added to the Kafka Endpoint URI:
> > >
> > >
> kafka://additionalProperties.testStringKey1=xxx&additionalProperties.testStringKey2=xxx&additionalProperties.schema.registr.url=xxx..
> > >
> > > Is it a bug in Camel or did I understand the documentation wrong? I
> > > can prepare a test case later
> > >
> > > Best regards,
> > >
> > > Martin
> > >
> >
> >
> > --
> > Claus Ibsen
> > -----------------
> > @davsclaus
> > Camel in Action 2: https://www.manning.com/ibsen2
>
--
Claus Ibsen
-----------------
@davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
Re: Objects not propagated to Camel Kafka 3.18.4
Posted by Martin Pelikán <pe...@gmail.com>.
My case is related to connecting to Azure EventHubs Avro Schema Registry.
TokenCredential cred = new
ClientSecretCredentialBuilder().tenantId("xxx").clientId("xxx").clientSecret("xx").build();
props.put("schema.registry.credential", cred);
Another case that comes to my mind is with OAUTHBEARER authentication.
Users could supply instantiated AuthenticateCallbackHandler class to
the sasl.login.callback.handler.class property.
TokenCredential as part of Properties works with
org.apache.kafka.clients.consumer.KafkaConsumer.
čt 5. 1. 2023 v 17:59 odesílatel Claus Ibsen <cl...@gmail.com> napsal:
>
> Whats the real world use-case for additional properties as Object or Map.
> What would kafka understand and use these for?
>
> On Thu, Jan 5, 2023 at 5:12 PM Martin Pelikán <pe...@gmail.com> wrote:
>
> > Hi Camel experts,
> >
> > I am struggling with a Kafka component and passing in parameters via
> > additionalProperties. I am not able to propagate an Object. I tested
> > only Endpoint DSL. According to the Camel docs and APIs, I am allowed
> > to pass in an instance of Map<String, Object>
> >
> > Camel version: 3.18.4
> >
> > ----Pseudocode-----
> >
> > @Configuration
> > Class SpringConfig
> >
> > @Bean(name = "autowiredMap")
> > public Map<String, Object> autowiredMap() {
> > var map = new HashMap<String, Object>();
> > map.put("testStringKey3", "testStringValue3");
> > map.put("testObjectKey3", new Object());
> > }
> >
> > -------
> >
> > Class MainRoute extends RouteBuilder()
> >
> > @Autowired
> > @Qualifier("autowiredMap")
> > Map<String, Object> autowiredMap;
> >
> > var map = new HashMap<String, Object>();
> > map.put("testStringKey1", "testStringValue1");
> > map.put("testObjectKey1", new Object());
> >
> > from(kafka(…basic configuration works)
> > .schemaRegistryURL(http://original.com)
> > .additionalProperties(map)
> > .additionalProperties("testStringKey2", "testStringValue2")
> > .additionalProperties("testObjectKey1", new Object())
> > .additionalProperties("schema.registry.url", http://overwritten.com)
> > .additionalProperties(autowiredMap)...
> >
> > Expected:
> >
> > All properties from all additionalProperties methods are propagated to
> > the Kafka Properties configuration object.
> >
> > In reality, Properties will contain the following:
> > schema.registry.url=http://overwritten.com
> > testStringKey1=testStringValue1
> > testStringKey2=testStringValue2
> >
> > Non-string values are not propagated at all. Values from autowired Map
> > are not propagated at all. Autowired map is treated as
> > additionalProperties.autowiredMap=HashMap@123456
> >
> > I can see those String values being added to the Kafka Endpoint URI:
> >
> > kafka://additionalProperties.testStringKey1=xxx&additionalProperties.testStringKey2=xxx&additionalProperties.schema.registr.url=xxx..
> >
> > Is it a bug in Camel or did I understand the documentation wrong? I
> > can prepare a test case later
> >
> > Best regards,
> >
> > Martin
> >
>
>
> --
> Claus Ibsen
> -----------------
> @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2
Re: Objects not propagated to Camel Kafka 3.18.4
Posted by Claus Ibsen <cl...@gmail.com>.
Whats the real world use-case for additional properties as Object or Map.
What would kafka understand and use these for?
On Thu, Jan 5, 2023 at 5:12 PM Martin Pelikán <pe...@gmail.com> wrote:
> Hi Camel experts,
>
> I am struggling with a Kafka component and passing in parameters via
> additionalProperties. I am not able to propagate an Object. I tested
> only Endpoint DSL. According to the Camel docs and APIs, I am allowed
> to pass in an instance of Map<String, Object>
>
> Camel version: 3.18.4
>
> ----Pseudocode-----
>
> @Configuration
> Class SpringConfig
>
> @Bean(name = "autowiredMap")
> public Map<String, Object> autowiredMap() {
> var map = new HashMap<String, Object>();
> map.put("testStringKey3", "testStringValue3");
> map.put("testObjectKey3", new Object());
> }
>
> -------
>
> Class MainRoute extends RouteBuilder()
>
> @Autowired
> @Qualifier("autowiredMap")
> Map<String, Object> autowiredMap;
>
> var map = new HashMap<String, Object>();
> map.put("testStringKey1", "testStringValue1");
> map.put("testObjectKey1", new Object());
>
> from(kafka(…basic configuration works)
> .schemaRegistryURL(http://original.com)
> .additionalProperties(map)
> .additionalProperties("testStringKey2", "testStringValue2")
> .additionalProperties("testObjectKey1", new Object())
> .additionalProperties("schema.registry.url", http://overwritten.com)
> .additionalProperties(autowiredMap)...
>
> Expected:
>
> All properties from all additionalProperties methods are propagated to
> the Kafka Properties configuration object.
>
> In reality, Properties will contain the following:
> schema.registry.url=http://overwritten.com
> testStringKey1=testStringValue1
> testStringKey2=testStringValue2
>
> Non-string values are not propagated at all. Values from autowired Map
> are not propagated at all. Autowired map is treated as
> additionalProperties.autowiredMap=HashMap@123456
>
> I can see those String values being added to the Kafka Endpoint URI:
>
> kafka://additionalProperties.testStringKey1=xxx&additionalProperties.testStringKey2=xxx&additionalProperties.schema.registr.url=xxx..
>
> Is it a bug in Camel or did I understand the documentation wrong? I
> can prepare a test case later
>
> Best regards,
>
> Martin
>
--
Claus Ibsen
-----------------
@davsclaus
Camel in Action 2: https://www.manning.com/ibsen2