You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by ac...@apache.org on 2023/10/19 11:25:23 UTC
[camel-kamelets] branch main updated (ddc23c1f -> ea7e5eec)
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/camel-kamelets.git
from ddc23c1f Updated CHANGELOG.md
new 8465717f Create a Specialized Kamelet starting from Kafka Source using Azure Schema Registry - Added the Credential bean
new 86bed97d Create a Specialized Kamelet starting from Kafka Source using Azure Schema Registry - Added Kamelet
new 04d138e8 Create a Specialized Kamelet starting from Kafka Source using Azure Schema Registry - Added Kamelet
new e6d1e998 Create a Specialized Kamelet starting from Kafka Source using Azure Schema Registry - Added Kamelet
new cff1988d Create a Specialized Kamelet starting from Kafka Source using Azure Schema Registry - Added Kamelet
new cf19379d Create a Specialized Kamelet starting from Kafka Source using Azure Schema Registry - Added Kamelet
new b7673180 Create a Specialized Kamelet starting from Kafka Source using Azure Schema Registry - Added Kamelet
new ea7e5eec Create a Specialized Kamelet starting from Kafka Source using Azure Schema Registry - Added Kamelet
The 8 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
docs/modules/ROOT/nav.adoc | 1 +
...kafka-azure-schema-registry-source.kamelet.yaml | 54 ++++++++++++++--------
.../registry/DefaultAzureCredentialWrapper.java} | 26 ++++++-----
...afka-azure-schema-registry-source.kamelet.yaml} | 54 ++++++++++++++--------
script/validator/validator.go | 2 +-
5 files changed, 89 insertions(+), 48 deletions(-)
copy library/camel-kamelets/src/main/resources/kamelets/kafka-source.kamelet.yaml => kamelets/kafka-azure-schema-registry-source.kamelet.yaml (81%)
copy library/camel-kamelets-utils/src/main/java/org/apache/camel/kamelets/utils/{transform/kafka/ManualCommit.java => kafka/registry/DefaultAzureCredentialWrapper.java} (54%)
copy library/camel-kamelets/src/main/resources/kamelets/{kafka-source.kamelet.yaml => kafka-azure-schema-registry-source.kamelet.yaml} (81%)
[camel-kamelets] 07/08: Create a Specialized Kamelet starting from Kafka Source using Azure Schema Registry - Added Kamelet
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel-kamelets.git
commit b76731800b89547bf78fe3da927b802d24a5b142
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Thu Oct 19 13:02:38 2023 +0200
Create a Specialized Kamelet starting from Kafka Source using Azure Schema Registry - Added Kamelet
Signed-off-by: Andrea Cosentino <an...@gmail.com>
---
kamelets/kafka-azure-schema-registry-source.kamelet.yaml | 2 +-
.../resources/kamelets/kafka-azure-schema-registry-source.kamelet.yaml | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/kamelets/kafka-azure-schema-registry-source.kamelet.yaml b/kamelets/kafka-azure-schema-registry-source.kamelet.yaml
index 52c44885..568fdb7e 100644
--- a/kamelets/kafka-azure-schema-registry-source.kamelet.yaml
+++ b/kamelets/kafka-azure-schema-registry-source.kamelet.yaml
@@ -154,7 +154,7 @@ spec:
brokers: "{{bootstrapServers}}"
securityProtocol: "{{securityProtocol}}"
saslMechanism: "{{saslMechanism}}"
- saslJaasConfig: 'org.apache.kafka.common.security.plain.PlainLoginModule required username="$ConnectionString" password={{connectionstring}};'
+ saslJaasConfig: 'org.apache.kafka.common.security.plain.PlainLoginModule required username="$ConnectionString" password={{password}};'
autoCommitEnable: "{{autoCommitEnable}}"
allowManualCommit: "{{allowManualCommit}}"
pollOnError: "{{pollOnError}}"
diff --git a/library/camel-kamelets/src/main/resources/kamelets/kafka-azure-schema-registry-source.kamelet.yaml b/library/camel-kamelets/src/main/resources/kamelets/kafka-azure-schema-registry-source.kamelet.yaml
index 52c44885..568fdb7e 100644
--- a/library/camel-kamelets/src/main/resources/kamelets/kafka-azure-schema-registry-source.kamelet.yaml
+++ b/library/camel-kamelets/src/main/resources/kamelets/kafka-azure-schema-registry-source.kamelet.yaml
@@ -154,7 +154,7 @@ spec:
brokers: "{{bootstrapServers}}"
securityProtocol: "{{securityProtocol}}"
saslMechanism: "{{saslMechanism}}"
- saslJaasConfig: 'org.apache.kafka.common.security.plain.PlainLoginModule required username="$ConnectionString" password={{connectionstring}};'
+ saslJaasConfig: 'org.apache.kafka.common.security.plain.PlainLoginModule required username="$ConnectionString" password={{password}};'
autoCommitEnable: "{{autoCommitEnable}}"
allowManualCommit: "{{allowManualCommit}}"
pollOnError: "{{pollOnError}}"
[camel-kamelets] 06/08: Create a Specialized Kamelet starting from Kafka Source using Azure Schema Registry - Added Kamelet
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel-kamelets.git
commit cf19379d4fd82335516833f8493ecaa26bfb81cc
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Thu Oct 19 12:51:20 2023 +0200
Create a Specialized Kamelet starting from Kafka Source using Azure Schema Registry - Added Kamelet
Signed-off-by: Andrea Cosentino <an...@gmail.com>
---
.../resources/kamelets/kafka-azure-schema-registry-source.kamelet.yaml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/library/camel-kamelets/src/main/resources/kamelets/kafka-azure-schema-registry-source.kamelet.yaml b/library/camel-kamelets/src/main/resources/kamelets/kafka-azure-schema-registry-source.kamelet.yaml
index 705d57ba..52c44885 100644
--- a/library/camel-kamelets/src/main/resources/kamelets/kafka-azure-schema-registry-source.kamelet.yaml
+++ b/library/camel-kamelets/src/main/resources/kamelets/kafka-azure-schema-registry-source.kamelet.yaml
@@ -36,7 +36,7 @@ spec:
required:
- topic
- bootstrapServers
- - azureSchemaRegistryUrl
+ - azureRegistryUrl
- password
type: object
properties:
[camel-kamelets] 02/08: Create a Specialized Kamelet starting from Kafka Source using Azure Schema Registry - Added Kamelet
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel-kamelets.git
commit 86bed97dd8c169fa05e54649cccc8d01d710a5d1
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Thu Oct 19 12:48:10 2023 +0200
Create a Specialized Kamelet starting from Kafka Source using Azure Schema Registry - Added Kamelet
Signed-off-by: Andrea Cosentino <an...@gmail.com>
---
...kafka-azure-schema-registry-source.kamelet.yaml | 171 +++++++++++++++++++++
...kafka-azure-schema-registry-source.kamelet.yaml | 171 +++++++++++++++++++++
2 files changed, 342 insertions(+)
diff --git a/kamelets/kafka-azure-schema-registry-source.kamelet.yaml b/kamelets/kafka-azure-schema-registry-source.kamelet.yaml
new file mode 100644
index 00000000..e95856c5
--- /dev/null
+++ b/kamelets/kafka-azure-schema-registry-source.kamelet.yaml
@@ -0,0 +1,171 @@
+# ---------------------------------------------------------------------------
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ---------------------------------------------------------------------------
+apiVersion: camel.apache.org/v1
+kind: Kamelet
+metadata:
+ name: kafka-azure-schema-registry-source
+ annotations:
+ camel.apache.org/kamelet.support.level: "Preview"
+ camel.apache.org/catalog.version: "4.1.0-SNAPSHOT"
+ camel.apache.org/kamelet.icon: "data:image/svg+xml;base64,PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0idXRmLTgiPz4NCjwhLS0gR2VuZXJhdG9yOiBBZG9iZSBJbGx1c3RyYXRvciAxOS4wLjAsIFNWRyBFeHBvcnQgUGx1Zy1JbiAuIFNWRyBWZXJzaW9uOiA2LjAwIEJ1aWxkIDApICAtLT4NCjxzdmcgdmVyc2lvbj0iMS4xIiBpZD0iTGF5ZXJfMSIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIiB4bWxuczp4bGluaz0iaHR0cDovL3d3dy53My5vcmcvMTk5OS94bGluayIgeD0iMHB4IiB5PSIwcHgiDQoJIHZpZXdCb3g9IjAgMCA1MDAgNTAwIiBzdHlsZT0iZW5hYmxlLWJhY2tncm91bmQ6bmV3IDAgMCA1MD [...]
+ camel.apache.org/provider: "Apache Software Foundation"
+ camel.apache.org/kamelet.group: "Kafka"
+ camel.apache.org/kamelet.namespace: "Kafka"
+ camel.apache.org/keda.type: "kafka"
+ labels:
+ camel.apache.org/kamelet.type: "source"
+spec:
+ definition:
+ title: "Azure Kafka through Eventhubs with Azure Schema Registry Source"
+ description: |-
+ Receive data from Kafka topics on Azure Eventhubs on combined with Azure Schema Registry.
+ required:
+ - topic
+ - bootstrapServers
+ - apicurioRegistryUrl
+ type: object
+ properties:
+ topic:
+ title: Topic Names
+ description: Comma separated list of Kafka topic names
+ type: string
+ x-descriptors:
+ - urn:keda:metadata:topic
+ - urn:keda:required
+ bootstrapServers:
+ title: Bootstrap Servers
+ description: Comma separated list of Kafka Broker URLs
+ type: string
+ x-descriptors:
+ - urn:keda:metadata:bootstrapServers
+ - urn:keda:required
+ securityProtocol:
+ title: Security Protocol
+ description: Protocol used to communicate with brokers. SASL_PLAINTEXT, PLAINTEXT, SASL_SSL and SSL are supported
+ type: string
+ default: SASL_SSL
+ saslMechanism:
+ title: SASL Mechanism
+ description: The Simple Authentication and Security Layer (SASL) Mechanism used.
+ type: string
+ default: PLAIN
+ password:
+ title: Password
+ description: Password to authenticate to kafka
+ type: string
+ format: password
+ x-descriptors:
+ - urn:alm:descriptor:com.tectonic.ui:password
+ - urn:camel:group:credentials
+ - urn:keda:authentication:password
+ - urn:keda:required
+ autoCommitEnable:
+ title: Auto Commit Enable
+ description: If true, periodically commit to ZooKeeper the offset of messages already fetched by the consumer
+ type: boolean
+ default: true
+ x-descriptors:
+ - 'urn:alm:descriptor:com.tectonic.ui:checkbox'
+ allowManualCommit:
+ title: Allow Manual Commit
+ description: Whether to allow doing manual commits
+ type: boolean
+ default: false
+ x-descriptors:
+ - 'urn:alm:descriptor:com.tectonic.ui:checkbox'
+ pollOnError:
+ title: Poll On Error Behavior
+ description: What to do if kafka threw an exception while polling for new messages. There are 5 enums and the value can be one of DISCARD, ERROR_HANDLER, RECONNECT, RETRY, STOP
+ type: string
+ default: "ERROR_HANDLER"
+ autoOffsetReset:
+ title: Auto Offset Reset
+ description: What to do when there is no initial offset. There are 3 enums and the value can be one of latest, earliest, none
+ type: string
+ default: "latest"
+ x-descriptors:
+ - urn:keda:metadata:offsetResetPolicy
+ consumerGroup:
+ title: Consumer Group
+ description: A string that uniquely identifies the group of consumers to which this source belongs
+ type: string
+ example: "my-group-id"
+ x-descriptors:
+ - urn:keda:metadata:consumerGroup
+ - urn:keda:required
+ deserializeHeaders:
+ title: Automatically Deserialize Headers
+ description: When enabled the Kamelet source will deserialize all message headers to String representation.
+ type: boolean
+ x-descriptors:
+ - 'urn:alm:descriptor:com.tectonic.ui:checkbox'
+ default: true
+ valueDeserializer:
+ title: Value Deserializer
+ description: Deserializer class for value that implements the Deserializer interface.
+ type: string
+ default: "com.microsoft.azure.schemaregistry.kafka.avro.KafkaAvroDeserializer"
+ azureRegistryUrl:
+ title: Apicurio Registry URL
+ description: The Apicurio Schema Registry URL
+ type: string
+ x-descriptors:
+ - urn:keda:metadata:bootstrapServers
+ - urn:keda:required
+ specificAvroValueType:
+ title: Specific Avro Value Type
+ description: The Specific Type Avro will have to deal with
+ type: string
+ example: "com.example.Order"
+ dependencies:
+ - "mvn:org.apache.camel.kamelets:camel-kamelets-utils:4.1.0-SNAPSHOT"
+ - "camel:kafka"
+ - "camel:core"
+ - "camel:kamelet"
+ - "mvn:com.microsoft.azure:azure-schemaregistry-kafka-avro:1.1.1"
+ - "mvn:com.azure:azure-data-schemaregistry-apacheavro:1.1.4"
+ - "mvn:com.azure:azure-identity:1.9.0"
+ template:
+ beans:
+ - name: defaultAzureCredential
+ type: "#class:org.apache.camel.kamelets.utils.kafka.registry.DefaultAzureCredentialWrapper"
+ - name: kafkaHeaderDeserializer
+ type: "#class:org.apache.camel.kamelets.utils.serialization.kafka.KafkaHeaderDeserializer"
+ properties:
+ enabled: '{{deserializeHeaders}}'
+ from:
+ uri: "kafka:{{topic}}"
+ parameters:
+ brokers: "{{bootstrapServers}}"
+ securityProtocol: "{{securityProtocol}}"
+ saslMechanism: "{{saslMechanism}}"
+ saslJaasConfig: 'org.apache.kafka.common.security.plain.PlainLoginModule required username="$ConnectionString" password={{connectionstring}};'
+ autoCommitEnable: "{{autoCommitEnable}}"
+ allowManualCommit: "{{allowManualCommit}}"
+ pollOnError: "{{pollOnError}}"
+ autoOffsetReset: "{{autoOffsetReset}}"
+ groupId: "{{?consumerGroup}}"
+ valueDeserializer: "{{valueDeserializer}}"
+ additionalProperties.schema.registry.url: "{{azureRegistryUrl}}"
+ additionalProperties.schema.group: avro
+ additionalProperties.schema.registry.credential: '#bean:defaultAzureCredential'
+ additionalProperties.specific.avro.value.type: '#valueAs(java.lang.Class):{{specificAvroValueType}}'
+ additionalProperties.specific.avro.reader: '#valueAs(boolean):true'
+ steps:
+ - process:
+ ref: "{{kafkaHeaderDeserializer}}"
+ - to: "kamelet:sink"
diff --git a/library/camel-kamelets/src/main/resources/kamelets/kafka-azure-schema-registry-source.kamelet.yaml b/library/camel-kamelets/src/main/resources/kamelets/kafka-azure-schema-registry-source.kamelet.yaml
new file mode 100644
index 00000000..e95856c5
--- /dev/null
+++ b/library/camel-kamelets/src/main/resources/kamelets/kafka-azure-schema-registry-source.kamelet.yaml
@@ -0,0 +1,171 @@
+# ---------------------------------------------------------------------------
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ---------------------------------------------------------------------------
+apiVersion: camel.apache.org/v1
+kind: Kamelet
+metadata:
+ name: kafka-azure-schema-registry-source
+ annotations:
+ camel.apache.org/kamelet.support.level: "Preview"
+ camel.apache.org/catalog.version: "4.1.0-SNAPSHOT"
+ camel.apache.org/kamelet.icon: "data:image/svg+xml;base64,PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0idXRmLTgiPz4NCjwhLS0gR2VuZXJhdG9yOiBBZG9iZSBJbGx1c3RyYXRvciAxOS4wLjAsIFNWRyBFeHBvcnQgUGx1Zy1JbiAuIFNWRyBWZXJzaW9uOiA2LjAwIEJ1aWxkIDApICAtLT4NCjxzdmcgdmVyc2lvbj0iMS4xIiBpZD0iTGF5ZXJfMSIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIiB4bWxuczp4bGluaz0iaHR0cDovL3d3dy53My5vcmcvMTk5OS94bGluayIgeD0iMHB4IiB5PSIwcHgiDQoJIHZpZXdCb3g9IjAgMCA1MDAgNTAwIiBzdHlsZT0iZW5hYmxlLWJhY2tncm91bmQ6bmV3IDAgMCA1MD [...]
+ camel.apache.org/provider: "Apache Software Foundation"
+ camel.apache.org/kamelet.group: "Kafka"
+ camel.apache.org/kamelet.namespace: "Kafka"
+ camel.apache.org/keda.type: "kafka"
+ labels:
+ camel.apache.org/kamelet.type: "source"
+spec:
+ definition:
+ title: "Azure Kafka through Eventhubs with Azure Schema Registry Source"
+ description: |-
+ Receive data from Kafka topics on Azure Eventhubs on combined with Azure Schema Registry.
+ required:
+ - topic
+ - bootstrapServers
+ - apicurioRegistryUrl
+ type: object
+ properties:
+ topic:
+ title: Topic Names
+ description: Comma separated list of Kafka topic names
+ type: string
+ x-descriptors:
+ - urn:keda:metadata:topic
+ - urn:keda:required
+ bootstrapServers:
+ title: Bootstrap Servers
+ description: Comma separated list of Kafka Broker URLs
+ type: string
+ x-descriptors:
+ - urn:keda:metadata:bootstrapServers
+ - urn:keda:required
+ securityProtocol:
+ title: Security Protocol
+ description: Protocol used to communicate with brokers. SASL_PLAINTEXT, PLAINTEXT, SASL_SSL and SSL are supported
+ type: string
+ default: SASL_SSL
+ saslMechanism:
+ title: SASL Mechanism
+ description: The Simple Authentication and Security Layer (SASL) Mechanism used.
+ type: string
+ default: PLAIN
+ password:
+ title: Password
+ description: Password to authenticate to kafka
+ type: string
+ format: password
+ x-descriptors:
+ - urn:alm:descriptor:com.tectonic.ui:password
+ - urn:camel:group:credentials
+ - urn:keda:authentication:password
+ - urn:keda:required
+ autoCommitEnable:
+ title: Auto Commit Enable
+ description: If true, periodically commit to ZooKeeper the offset of messages already fetched by the consumer
+ type: boolean
+ default: true
+ x-descriptors:
+ - 'urn:alm:descriptor:com.tectonic.ui:checkbox'
+ allowManualCommit:
+ title: Allow Manual Commit
+ description: Whether to allow doing manual commits
+ type: boolean
+ default: false
+ x-descriptors:
+ - 'urn:alm:descriptor:com.tectonic.ui:checkbox'
+ pollOnError:
+ title: Poll On Error Behavior
+ description: What to do if kafka threw an exception while polling for new messages. There are 5 enums and the value can be one of DISCARD, ERROR_HANDLER, RECONNECT, RETRY, STOP
+ type: string
+ default: "ERROR_HANDLER"
+ autoOffsetReset:
+ title: Auto Offset Reset
+ description: What to do when there is no initial offset. There are 3 enums and the value can be one of latest, earliest, none
+ type: string
+ default: "latest"
+ x-descriptors:
+ - urn:keda:metadata:offsetResetPolicy
+ consumerGroup:
+ title: Consumer Group
+ description: A string that uniquely identifies the group of consumers to which this source belongs
+ type: string
+ example: "my-group-id"
+ x-descriptors:
+ - urn:keda:metadata:consumerGroup
+ - urn:keda:required
+ deserializeHeaders:
+ title: Automatically Deserialize Headers
+ description: When enabled the Kamelet source will deserialize all message headers to String representation.
+ type: boolean
+ x-descriptors:
+ - 'urn:alm:descriptor:com.tectonic.ui:checkbox'
+ default: true
+ valueDeserializer:
+ title: Value Deserializer
+ description: Deserializer class for value that implements the Deserializer interface.
+ type: string
+ default: "com.microsoft.azure.schemaregistry.kafka.avro.KafkaAvroDeserializer"
+ azureRegistryUrl:
+ title: Apicurio Registry URL
+ description: The Apicurio Schema Registry URL
+ type: string
+ x-descriptors:
+ - urn:keda:metadata:bootstrapServers
+ - urn:keda:required
+ specificAvroValueType:
+ title: Specific Avro Value Type
+ description: The Specific Type Avro will have to deal with
+ type: string
+ example: "com.example.Order"
+ dependencies:
+ - "mvn:org.apache.camel.kamelets:camel-kamelets-utils:4.1.0-SNAPSHOT"
+ - "camel:kafka"
+ - "camel:core"
+ - "camel:kamelet"
+ - "mvn:com.microsoft.azure:azure-schemaregistry-kafka-avro:1.1.1"
+ - "mvn:com.azure:azure-data-schemaregistry-apacheavro:1.1.4"
+ - "mvn:com.azure:azure-identity:1.9.0"
+ template:
+ beans:
+ - name: defaultAzureCredential
+ type: "#class:org.apache.camel.kamelets.utils.kafka.registry.DefaultAzureCredentialWrapper"
+ - name: kafkaHeaderDeserializer
+ type: "#class:org.apache.camel.kamelets.utils.serialization.kafka.KafkaHeaderDeserializer"
+ properties:
+ enabled: '{{deserializeHeaders}}'
+ from:
+ uri: "kafka:{{topic}}"
+ parameters:
+ brokers: "{{bootstrapServers}}"
+ securityProtocol: "{{securityProtocol}}"
+ saslMechanism: "{{saslMechanism}}"
+ saslJaasConfig: 'org.apache.kafka.common.security.plain.PlainLoginModule required username="$ConnectionString" password={{connectionstring}};'
+ autoCommitEnable: "{{autoCommitEnable}}"
+ allowManualCommit: "{{allowManualCommit}}"
+ pollOnError: "{{pollOnError}}"
+ autoOffsetReset: "{{autoOffsetReset}}"
+ groupId: "{{?consumerGroup}}"
+ valueDeserializer: "{{valueDeserializer}}"
+ additionalProperties.schema.registry.url: "{{azureRegistryUrl}}"
+ additionalProperties.schema.group: avro
+ additionalProperties.schema.registry.credential: '#bean:defaultAzureCredential'
+ additionalProperties.specific.avro.value.type: '#valueAs(java.lang.Class):{{specificAvroValueType}}'
+ additionalProperties.specific.avro.reader: '#valueAs(boolean):true'
+ steps:
+ - process:
+ ref: "{{kafkaHeaderDeserializer}}"
+ - to: "kamelet:sink"
[camel-kamelets] 01/08: Create a Specialized Kamelet starting from Kafka Source using Azure Schema Registry - Added the Credential bean
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel-kamelets.git
commit 8465717f745a5d200ffaa1c07ce9b4c385ca42ee
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Thu Oct 19 12:47:37 2023 +0200
Create a Specialized Kamelet starting from Kafka Source using Azure Schema Registry - Added the Credential bean
Signed-off-by: Andrea Cosentino <an...@gmail.com>
---
.../registry/DefaultAzureCredentialWrapper.java | 37 ++++++++++++++++++++++
1 file changed, 37 insertions(+)
diff --git a/library/camel-kamelets-utils/src/main/java/org/apache/camel/kamelets/utils/kafka/registry/DefaultAzureCredentialWrapper.java b/library/camel-kamelets-utils/src/main/java/org/apache/camel/kamelets/utils/kafka/registry/DefaultAzureCredentialWrapper.java
new file mode 100644
index 00000000..dd0dcdc5
--- /dev/null
+++ b/library/camel-kamelets-utils/src/main/java/org/apache/camel/kamelets/utils/kafka/registry/DefaultAzureCredentialWrapper.java
@@ -0,0 +1,37 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.kamelets.utils.kafka.registry;
+
+import com.azure.core.credential.AccessToken;
+import com.azure.core.credential.TokenCredential;
+import com.azure.core.credential.TokenRequestContext;
+import com.azure.identity.DefaultAzureCredentialBuilder;
+import reactor.core.publisher.Mono;
+
+public class DefaultAzureCredentialWrapper implements TokenCredential {
+
+ private final TokenCredential credential;
+
+ public DefaultAzureCredentialWrapper() {
+ this.credential = new DefaultAzureCredentialBuilder().build();
+ }
+
+ @Override
+ public Mono<AccessToken> getToken(TokenRequestContext tokenRequestContext) {
+ return this.credential.getToken(tokenRequestContext);
+ }
+}
[camel-kamelets] 04/08: Create a Specialized Kamelet starting from Kafka Source using Azure Schema Registry - Added Kamelet
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel-kamelets.git
commit e6d1e998a88a844345ea5439aaa4e429ea984cbd
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Thu Oct 19 12:49:58 2023 +0200
Create a Specialized Kamelet starting from Kafka Source using Azure Schema Registry - Added Kamelet
Signed-off-by: Andrea Cosentino <an...@gmail.com>
---
.../kamelets/kafka-azure-schema-registry-source.kamelet.yaml | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/library/camel-kamelets/src/main/resources/kamelets/kafka-azure-schema-registry-source.kamelet.yaml b/library/camel-kamelets/src/main/resources/kamelets/kafka-azure-schema-registry-source.kamelet.yaml
index e95856c5..705d57ba 100644
--- a/library/camel-kamelets/src/main/resources/kamelets/kafka-azure-schema-registry-source.kamelet.yaml
+++ b/library/camel-kamelets/src/main/resources/kamelets/kafka-azure-schema-registry-source.kamelet.yaml
@@ -32,11 +32,12 @@ spec:
definition:
title: "Azure Kafka through Eventhubs with Azure Schema Registry Source"
description: |-
- Receive data from Kafka topics on Azure Eventhubs on combined with Azure Schema Registry.
+ Receive data from Kafka topics on Azure Eventhubs combined with Azure Schema Registry.
required:
- topic
- bootstrapServers
- - apicurioRegistryUrl
+ - azureSchemaRegistryUrl
+ - password
type: object
properties:
topic:
@@ -120,7 +121,7 @@ spec:
type: string
default: "com.microsoft.azure.schemaregistry.kafka.avro.KafkaAvroDeserializer"
azureRegistryUrl:
- title: Apicurio Registry URL
+ title: Azure Schema Registry URL
description: The Apicurio Schema Registry URL
type: string
x-descriptors:
[camel-kamelets] 05/08: Create a Specialized Kamelet starting from Kafka Source using Azure Schema Registry - Added Kamelet
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel-kamelets.git
commit cff1988d68a7e799f5e00ac1feee7589fe54378f
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Thu Oct 19 12:50:32 2023 +0200
Create a Specialized Kamelet starting from Kafka Source using Azure Schema Registry - Added Kamelet
Signed-off-by: Andrea Cosentino <an...@gmail.com>
---
kamelets/kafka-azure-schema-registry-source.kamelet.yaml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/kamelets/kafka-azure-schema-registry-source.kamelet.yaml b/kamelets/kafka-azure-schema-registry-source.kamelet.yaml
index 705d57ba..52c44885 100644
--- a/kamelets/kafka-azure-schema-registry-source.kamelet.yaml
+++ b/kamelets/kafka-azure-schema-registry-source.kamelet.yaml
@@ -36,7 +36,7 @@ spec:
required:
- topic
- bootstrapServers
- - azureSchemaRegistryUrl
+ - azureRegistryUrl
- password
type: object
properties:
[camel-kamelets] 03/08: Create a Specialized Kamelet starting from Kafka Source using Azure Schema Registry - Added Kamelet
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel-kamelets.git
commit 04d138e83d6fc194926145ac91c1f1a9d303026a
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Thu Oct 19 12:49:33 2023 +0200
Create a Specialized Kamelet starting from Kafka Source using Azure Schema Registry - Added Kamelet
Signed-off-by: Andrea Cosentino <an...@gmail.com>
---
docs/modules/ROOT/nav.adoc | 1 +
kamelets/kafka-azure-schema-registry-source.kamelet.yaml | 7 ++++---
2 files changed, 5 insertions(+), 3 deletions(-)
diff --git a/docs/modules/ROOT/nav.adoc b/docs/modules/ROOT/nav.adoc
index 258bdec5..227dd455 100644
--- a/docs/modules/ROOT/nav.adoc
+++ b/docs/modules/ROOT/nav.adoc
@@ -124,6 +124,7 @@
* xref:json-serialize-action.adoc[]
* xref:jsonata-action.adoc[]
* xref:kafka-apicurio-registry-not-secured-source.adoc[]
+* xref:kafka-azure-schema-registry-source.adoc[]
* xref:kafka-manual-commit-action.adoc[]
* xref:kafka-not-secured-sink.adoc[]
* xref:kafka-not-secured-source.adoc[]
diff --git a/kamelets/kafka-azure-schema-registry-source.kamelet.yaml b/kamelets/kafka-azure-schema-registry-source.kamelet.yaml
index e95856c5..705d57ba 100644
--- a/kamelets/kafka-azure-schema-registry-source.kamelet.yaml
+++ b/kamelets/kafka-azure-schema-registry-source.kamelet.yaml
@@ -32,11 +32,12 @@ spec:
definition:
title: "Azure Kafka through Eventhubs with Azure Schema Registry Source"
description: |-
- Receive data from Kafka topics on Azure Eventhubs on combined with Azure Schema Registry.
+ Receive data from Kafka topics on Azure Eventhubs combined with Azure Schema Registry.
required:
- topic
- bootstrapServers
- - apicurioRegistryUrl
+ - azureSchemaRegistryUrl
+ - password
type: object
properties:
topic:
@@ -120,7 +121,7 @@ spec:
type: string
default: "com.microsoft.azure.schemaregistry.kafka.avro.KafkaAvroDeserializer"
azureRegistryUrl:
- title: Apicurio Registry URL
+ title: Azure Schema Registry URL
description: The Apicurio Schema Registry URL
type: string
x-descriptors:
[camel-kamelets] 08/08: Create a Specialized Kamelet starting from Kafka Source using Azure Schema Registry - Added Kamelet
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel-kamelets.git
commit ea7e5eec1954d28b5a63521fe2b881d4175a4526
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Thu Oct 19 13:04:14 2023 +0200
Create a Specialized Kamelet starting from Kafka Source using Azure Schema Registry - Added Kamelet
Signed-off-by: Andrea Cosentino <an...@gmail.com>
---
script/validator/validator.go | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/script/validator/validator.go b/script/validator/validator.go
index 35784476..cdc44142 100644
--- a/script/validator/validator.go
+++ b/script/validator/validator.go
@@ -399,7 +399,7 @@ func listKamelets(dir string) []KameletInfo {
func verifyUsedParams(kamelets []KameletInfo) (errors []error) {
for _, k := range kamelets {
- if k.FileName != "../../kamelets/azure-storage-blob-source.kamelet.yaml" && k.FileName != "../../kamelets/aws-s3-cdc-source.kamelet.yaml" && k.FileName != "../../kamelets/set-kafka-key-action.kamelet.yaml" && k.FileName != "../../kamelets/azure-storage-blob-cdc-source.kamelet.yaml" && k.FileName != "../../kamelets/google-storage-cdc-source.kamelet.yaml" && k.FileName != "../../kamelets/elasticsearch-search-source.kamelet.yaml" && k.FileName != "../../kamelets/opensearch-search-source.k [...]
+ if k.FileName != "../../kamelets/azure-storage-blob-source.kamelet.yaml" && k.FileName != "../../kamelets/aws-s3-cdc-source.kamelet.yaml" && k.FileName != "../../kamelets/set-kafka-key-action.kamelet.yaml" && k.FileName != "../../kamelets/azure-storage-blob-cdc-source.kamelet.yaml" && k.FileName != "../../kamelets/google-storage-cdc-source.kamelet.yaml" && k.FileName != "../../kamelets/elasticsearch-search-source.kamelet.yaml" && k.FileName != "../../kamelets/opensearch-search-source.k [...]
used := getUsedParams(k.Kamelet)
declared := getDeclaredParams(k.Kamelet)
for p := range used {