You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by ac...@apache.org on 2021/01/18 13:22:43 UTC
[camel-kafka-connector] branch aws2-kinesis-smt created (now
e93bef8)
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a change to branch aws2-kinesis-smt
in repository https://gitbox.apache.org/repos/asf/camel-kafka-connector.git.
at e93bef8 Regen documentation
This branch includes the following new commits:
new edc3e19 AWS2-Kinesis connector: Add a transformation to extract only the data from a source stream
new e93bef8 Regen documentation
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
[camel-kafka-connector] 01/02: AWS2-Kinesis connector: Add a
transformation to extract only the data from a source stream
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch aws2-kinesis-smt
in repository https://gitbox.apache.org/repos/asf/camel-kafka-connector.git
commit edc3e1986abc2dda2f84a9f6dfccdcc082c9aa1f
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Mon Jan 18 13:57:09 2021 +0100
AWS2-Kinesis connector: Add a transformation to extract only the data from a source stream
---
.../transformers/KinesisRecordDataTransforms.java | 71 ++++++++++++++++++++++
1 file changed, 71 insertions(+)
diff --git a/connectors/camel-aws2-kinesis-kafka-connector/src/main/java/org/apache/camel/kafkaconnector/aws2kinesis/transformers/KinesisRecordDataTransforms.java b/connectors/camel-aws2-kinesis-kafka-connector/src/main/java/org/apache/camel/kafkaconnector/aws2kinesis/transformers/KinesisRecordDataTransforms.java
new file mode 100644
index 0000000..8cb2016
--- /dev/null
+++ b/connectors/camel-aws2-kinesis-kafka-connector/src/main/java/org/apache/camel/kafkaconnector/aws2kinesis/transformers/KinesisRecordDataTransforms.java
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.kafkaconnector.aws2kinesis.transformers;
+
+import java.util.Map;
+
+import org.apache.camel.kafkaconnector.utils.SchemaHelper;
+import org.apache.kafka.common.config.ConfigDef;
+import org.apache.kafka.connect.connector.ConnectRecord;
+import org.apache.kafka.connect.transforms.Transformation;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import software.amazon.awssdk.services.kinesis.model.Record;
+
+public class KinesisRecordDataTransforms <R extends ConnectRecord<R>> implements Transformation<R> {
+ public static final String FIELD_KEY_CONFIG = "key";
+ public static final ConfigDef CONFIG_DEF = new ConfigDef()
+ .define(FIELD_KEY_CONFIG, ConfigDef.Type.STRING, null, ConfigDef.Importance.MEDIUM,
+ "Transforms Kinesis Data to String");
+
+ private static final Logger LOG = LoggerFactory.getLogger(KinesisRecordDataTransforms.class);
+
+ @Override
+ public R apply(R r) {
+ Object value = r.value();
+
+ if (value instanceof Record) {
+ LOG.debug("Converting record from Data to String");
+ Record message = (Record) r.value();
+
+ String payload = new String(message.data().asByteArray());
+ return r.newRecord(r.topic(), r.kafkaPartition(), null, r.key(),
+ SchemaHelper.buildSchemaBuilderForType(payload), payload, r.timestamp());
+
+ } else {
+ LOG.debug("Unexpected message type: {}", r.value().getClass());
+
+ return r;
+ }
+ }
+
+ @Override
+ public ConfigDef config() {
+ return CONFIG_DEF;
+ }
+
+ @Override
+ public void close() {
+
+ }
+
+ @Override
+ public void configure(Map<String, ?> map) {
+
+ }
+}
[camel-kafka-connector] 02/02: Regen documentation
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch aws2-kinesis-smt
in repository https://gitbox.apache.org/repos/asf/camel-kafka-connector.git
commit e93bef8d7a474fc96718bcc6d77a4aeb9c04f0c8
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Mon Jan 18 14:12:02 2021 +0100
Regen documentation
---
.../generated/resources/connectors/camel-aws2-kinesis-sink.json | 5 ++++-
.../resources/connectors/camel-aws2-kinesis-source.json | 5 ++++-
.../src/generated/resources/camel-aws2-kinesis-sink.json | 5 ++++-
.../src/generated/resources/camel-aws2-kinesis-source.json | 5 ++++-
.../src/main/docs/camel-aws2-kinesis-kafka-sink-connector.adoc | 9 ++++++++-
.../src/main/docs/camel-aws2-kinesis-kafka-source-connector.adoc | 9 ++++++++-
.../connectors/camel-aws2-kinesis-kafka-sink-connector.adoc | 9 ++++++++-
.../connectors/camel-aws2-kinesis-kafka-source-connector.adoc | 9 ++++++++-
8 files changed, 48 insertions(+), 8 deletions(-)
diff --git a/camel-kafka-connector-catalog/src/generated/resources/connectors/camel-aws2-kinesis-sink.json b/camel-kafka-connector-catalog/src/generated/resources/connectors/camel-aws2-kinesis-sink.json
index ae413db..9ece36d 100644
--- a/camel-kafka-connector-catalog/src/generated/resources/connectors/camel-aws2-kinesis-sink.json
+++ b/camel-kafka-connector-catalog/src/generated/resources/connectors/camel-aws2-kinesis-sink.json
@@ -171,5 +171,8 @@
"priority": "MEDIUM",
"required": "false"
}
- }
+ },
+ "transforms": [
+ "org.apache.camel.kafkaconnector.aws2kinesis.transformers.KinesisRecordDataTransforms"
+ ]
}
\ No newline at end of file
diff --git a/camel-kafka-connector-catalog/src/generated/resources/connectors/camel-aws2-kinesis-source.json b/camel-kafka-connector-catalog/src/generated/resources/connectors/camel-aws2-kinesis-source.json
index 7eb0fb2..b45d0a3 100644
--- a/camel-kafka-connector-catalog/src/generated/resources/connectors/camel-aws2-kinesis-source.json
+++ b/camel-kafka-connector-catalog/src/generated/resources/connectors/camel-aws2-kinesis-source.json
@@ -403,5 +403,8 @@
"priority": "MEDIUM",
"required": "false"
}
- }
+ },
+ "transforms": [
+ "org.apache.camel.kafkaconnector.aws2kinesis.transformers.KinesisRecordDataTransforms"
+ ]
}
\ No newline at end of file
diff --git a/connectors/camel-aws2-kinesis-kafka-connector/src/generated/resources/camel-aws2-kinesis-sink.json b/connectors/camel-aws2-kinesis-kafka-connector/src/generated/resources/camel-aws2-kinesis-sink.json
index ae413db..9ece36d 100644
--- a/connectors/camel-aws2-kinesis-kafka-connector/src/generated/resources/camel-aws2-kinesis-sink.json
+++ b/connectors/camel-aws2-kinesis-kafka-connector/src/generated/resources/camel-aws2-kinesis-sink.json
@@ -171,5 +171,8 @@
"priority": "MEDIUM",
"required": "false"
}
- }
+ },
+ "transforms": [
+ "org.apache.camel.kafkaconnector.aws2kinesis.transformers.KinesisRecordDataTransforms"
+ ]
}
\ No newline at end of file
diff --git a/connectors/camel-aws2-kinesis-kafka-connector/src/generated/resources/camel-aws2-kinesis-source.json b/connectors/camel-aws2-kinesis-kafka-connector/src/generated/resources/camel-aws2-kinesis-source.json
index 7eb0fb2..b45d0a3 100644
--- a/connectors/camel-aws2-kinesis-kafka-connector/src/generated/resources/camel-aws2-kinesis-source.json
+++ b/connectors/camel-aws2-kinesis-kafka-connector/src/generated/resources/camel-aws2-kinesis-source.json
@@ -403,5 +403,8 @@
"priority": "MEDIUM",
"required": "false"
}
- }
+ },
+ "transforms": [
+ "org.apache.camel.kafkaconnector.aws2kinesis.transformers.KinesisRecordDataTransforms"
+ ]
}
\ No newline at end of file
diff --git a/connectors/camel-aws2-kinesis-kafka-connector/src/main/docs/camel-aws2-kinesis-kafka-sink-connector.adoc b/connectors/camel-aws2-kinesis-kafka-connector/src/main/docs/camel-aws2-kinesis-kafka-sink-connector.adoc
index 9c9ca7c..6e515f8 100644
--- a/connectors/camel-aws2-kinesis-kafka-connector/src/main/docs/camel-aws2-kinesis-kafka-sink-connector.adoc
+++ b/connectors/camel-aws2-kinesis-kafka-connector/src/main/docs/camel-aws2-kinesis-kafka-sink-connector.adoc
@@ -63,10 +63,17 @@ The camel-aws2-kinesis sink connector has no converters out of the box.
-The camel-aws2-kinesis sink connector has no transforms out of the box.
+The camel-aws2-kinesis sink connector supports 0 transforms out of the box, which are listed below.
+[source,java]
+----
+
+org.apache.camel.kafkaconnector.aws2kinesis.transformers.KinesisRecordDataTransforms
+
+----
+
The camel-aws2-kinesis sink connector has no aggregation strategies out of the box.
diff --git a/connectors/camel-aws2-kinesis-kafka-connector/src/main/docs/camel-aws2-kinesis-kafka-source-connector.adoc b/connectors/camel-aws2-kinesis-kafka-connector/src/main/docs/camel-aws2-kinesis-kafka-source-connector.adoc
index 0c1c6c4..974a79c 100644
--- a/connectors/camel-aws2-kinesis-kafka-connector/src/main/docs/camel-aws2-kinesis-kafka-source-connector.adoc
+++ b/connectors/camel-aws2-kinesis-kafka-connector/src/main/docs/camel-aws2-kinesis-kafka-source-connector.adoc
@@ -91,10 +91,17 @@ The camel-aws2-kinesis source connector has no converters out of the box.
-The camel-aws2-kinesis source connector has no transforms out of the box.
+The camel-aws2-kinesis source connector supports 0 transforms out of the box, which are listed below.
+[source,java]
+----
+
+org.apache.camel.kafkaconnector.aws2kinesis.transformers.KinesisRecordDataTransforms
+
+----
+
The camel-aws2-kinesis source connector has no aggregation strategies out of the box.
diff --git a/docs/modules/ROOT/pages/connectors/camel-aws2-kinesis-kafka-sink-connector.adoc b/docs/modules/ROOT/pages/connectors/camel-aws2-kinesis-kafka-sink-connector.adoc
index 9c9ca7c..6e515f8 100644
--- a/docs/modules/ROOT/pages/connectors/camel-aws2-kinesis-kafka-sink-connector.adoc
+++ b/docs/modules/ROOT/pages/connectors/camel-aws2-kinesis-kafka-sink-connector.adoc
@@ -63,10 +63,17 @@ The camel-aws2-kinesis sink connector has no converters out of the box.
-The camel-aws2-kinesis sink connector has no transforms out of the box.
+The camel-aws2-kinesis sink connector supports 0 transforms out of the box, which are listed below.
+[source,java]
+----
+
+org.apache.camel.kafkaconnector.aws2kinesis.transformers.KinesisRecordDataTransforms
+
+----
+
The camel-aws2-kinesis sink connector has no aggregation strategies out of the box.
diff --git a/docs/modules/ROOT/pages/connectors/camel-aws2-kinesis-kafka-source-connector.adoc b/docs/modules/ROOT/pages/connectors/camel-aws2-kinesis-kafka-source-connector.adoc
index 0c1c6c4..974a79c 100644
--- a/docs/modules/ROOT/pages/connectors/camel-aws2-kinesis-kafka-source-connector.adoc
+++ b/docs/modules/ROOT/pages/connectors/camel-aws2-kinesis-kafka-source-connector.adoc
@@ -91,10 +91,17 @@ The camel-aws2-kinesis source connector has no converters out of the box.
-The camel-aws2-kinesis source connector has no transforms out of the box.
+The camel-aws2-kinesis source connector supports 0 transforms out of the box, which are listed below.
+[source,java]
+----
+
+org.apache.camel.kafkaconnector.aws2kinesis.transformers.KinesisRecordDataTransforms
+
+----
+
The camel-aws2-kinesis source connector has no aggregation strategies out of the box.