You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by ac...@apache.org on 2022/11/30 10:36:32 UTC
[camel-kamelets] branch 1188 created (now d5962237)
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a change to branch 1188
in repository https://gitbox.apache.org/repos/asf/camel-kamelets.git
at d5962237 Add a Google Cloud Storage CDC Source Kamelet
This branch includes the following new commits:
new a80958b3 Add a Google Cloud Storage CDC Source Kamelet
new 26888462 Add a Google Cloud Storage CDC Source Kamelet
new 0f79bc87 Add a Google Cloud Storage CDC Source Kamelet
new cbe49f7c Add a Google Cloud Storage CDC Source Kamelet
new 07fd61dd Add a Google Cloud Storage CDC Source Kamelet - Docs
new 5f03f06f Add a Google Cloud Storage CDC Source Kamelet - Docs
new 3be56abf Add a Google Cloud Storage CDC Source Kamelet
new cf5989e4 Add a Google Cloud Storage CDC Source Kamelet
new d5962237 Add a Google Cloud Storage CDC Source Kamelet
The 9 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
[camel-kamelets] 08/09: Add a Google Cloud Storage CDC Source Kamelet
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch 1188
in repository https://gitbox.apache.org/repos/asf/camel-kamelets.git
commit cf5989e4ada0f5b638110009b17124cfdec375e4
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Wed Nov 30 11:31:23 2022 +0100
Add a Google Cloud Storage CDC Source Kamelet
Signed-off-by: Andrea Cosentino <an...@gmail.com>
---
.../src/main/resources/kamelets/google-storage-cdc-source.kamelet.yaml | 1 +
1 file changed, 1 insertion(+)
diff --git a/library/camel-kamelets/src/main/resources/kamelets/google-storage-cdc-source.kamelet.yaml b/library/camel-kamelets/src/main/resources/kamelets/google-storage-cdc-source.kamelet.yaml
index aa1bd513..f55543ea 100644
--- a/library/camel-kamelets/src/main/resources/kamelets/google-storage-cdc-source.kamelet.yaml
+++ b/library/camel-kamelets/src/main/resources/kamelets/google-storage-cdc-source.kamelet.yaml
@@ -91,6 +91,7 @@ spec:
- "camel:google-pubsub"
- "camel:google-storage"
- "camel:jackson"
+ - "camel:core"
template:
from:
uri: "google-pubsub://{{projectId}}:{{subscriptionName}}"
[camel-kamelets] 01/09: Add a Google Cloud Storage CDC Source Kamelet
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch 1188
in repository https://gitbox.apache.org/repos/asf/camel-kamelets.git
commit a80958b3cce223f536e38f33e2ef2591c22d87c3
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Wed Nov 30 11:18:25 2022 +0100
Add a Google Cloud Storage CDC Source Kamelet
Signed-off-by: Andrea Cosentino <an...@gmail.com>
---
kamelets/google-storage-cdc-source.kamelet.yaml | 123 ++++++++++++++++++++++++
1 file changed, 123 insertions(+)
diff --git a/kamelets/google-storage-cdc-source.kamelet.yaml b/kamelets/google-storage-cdc-source.kamelet.yaml
new file mode 100644
index 00000000..1e7e48ab
--- /dev/null
+++ b/kamelets/google-storage-cdc-source.kamelet.yaml
@@ -0,0 +1,123 @@
+# ---------------------------------------------------------------------------
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ---------------------------------------------------------------------------
+apiVersion: camel.apache.org/v1alpha1
+kind: Kamelet
+metadata:
+ name: google-storage-cdc-source
+ annotations:
+ camel.apache.org/kamelet.support.level: "Stable"
+ camel.apache.org/catalog.version: "main-SNAPSHOT"
+ camel.apache.org/kamelet.icon: "data:image/svg+xml;base64,PHN2ZyBoZWlnaHQ9IjI1MDAiIHZpZXdCb3g9Ii0xLjYzMzIzNTQzIDcuMDMyNjA5MzMgMTMxLjI2NTc0NjgyIDExNC40MzkzOTA2NyIgd2lkdGg9IjI1MDAiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyIgeG1sbnM6eGxpbms9Imh0dHA6Ly93d3cudzMub3JnLzE5OTkveGxpbmsiPjxsaW5lYXJHcmFkaWVudCBpZD0iYSIgZ3JhZGllbnRVbml0cz0idXNlclNwYWNlT25Vc2UiIHgxPSI2NCIgeDI9IjY0IiB5MT0iNy4wMzQiIHkyPSIxMjAuNzg5Ij48c3RvcCBvZmZzZXQ9IjAiIHN0b3AtY29sb3I9IiM0Mzg3ZmQiLz48c3RvcCBvZmZzZXQ9IjEiIHN0b3 [...]
+ camel.apache.org/provider: "Apache Software Foundation"
+ camel.apache.org/kamelet.group: "Google Storage"
+ labels:
+ camel.apache.org/kamelet.type: "source"
+spec:
+ definition:
+ title: "Google Pubsub Source"
+ description: |-
+ Consume messages from Google Cloud Pub/Sub.
+ required:
+ - projectId
+ - subscriptionName
+ - serviceAccountKey
+ - bucketNameOrArn
+ type: object
+ properties:
+ projectId:
+ title: Project Id
+ description: The Google Cloud Pub/Sub Project ID.
+ type: string
+ subscriptionName:
+ title: Subscription Name
+ description: The subscription name.
+ type: string
+ serviceAccountKey:
+ title: Service Account Key
+ description: The service account key to use as credentials for the Pub/Sub publisher/subscriber. You must encode this value in base64.
+ type: binary
+ x-descriptors:
+ - urn:camel:group:credentials
+ synchronousPull:
+ title: Synchronous Pull
+ description: Specifies to synchronously pull batches of messages.
+ type: boolean
+ x-descriptors:
+ - 'urn:alm:descriptor:com.tectonic.ui:checkbox'
+ default: false
+ maxMessagesPerPoll:
+ title: Max Messages Per Poll
+ description: The maximum number of messages to receive from the server in a single API call.
+ type: integer
+ default: 1
+ concurrentConsumers:
+ title: Concurrent Consumers
+ description: The number of parallel streams to consume from the subscription.
+ type: integer
+ default: 1
+ bucketNameOrArn:
+ title: Bucket Name Or ARN
+ description: The Google Cloud Storage bucket name or Bucket Amazon Resource Name (ARN).
+ type: string
+ getObject:
+ title: Get Object in Bucket
+ description: >-
+ If getObject is enabled, then the file created in the Bucket will be
+ get and returned as body, if not only the event will be returned as body.
+ type: boolean
+ x-descriptors:
+ - 'urn:alm:descriptor:com.tectonic.ui:checkbox'
+ default: false
+ dependencies:
+ - "camel:kamelet"
+ - "camel:google-pubsub"
+ - "camel:google-storage"
+ - "camel:jackson"
+ template:
+ from:
+ uri: "google-pubsub://{{projectId}}:{{subscriptionName}}"
+ parameters:
+ serviceAccountKey: "base64:{{serviceAccountKey}}"
+ synchronousPull: "{{synchronousPull}}"
+ maxMessagesPerPoll: "{{maxMessagesPerPoll}}"
+ concurrentConsumers: "{{concurrentConsumers}}"
+ steps:
+ - choice:
+ precondition: true
+ when:
+ - simple: '${properties:getObject:true}'
+ steps:
+ - set-header:
+ name: attributesStorage
+ simple: ${headers.CamelGooglePubsub.Attributes}
+ - log: "${headers.attributesStorage}"
+ - set-property:
+ name: google-storage-event-type
+ simple: "${headers.attributesStorage[eventType]}"
+ - set-property:
+ name: google-storage-object-id
+ simple: "${headers.attributesStorage[objectId]}"
+ - set-property:
+ name: google-storage-bucket-id
+ simple: "${headers.attributesStorage[bucketId]}"
+ - choice:
+ when:
+ - simple: '${exchangeProperty.google-storage-event-type} == "OBJECT_FINALIZE" && ${exchangeProperty.google-storage-bucket-id} == "{{bucketNameOrArn}}"'
+ steps:
+ - toD: >-
+ google-storage:{{bucketNameOrArn}}?serviceAccountKey=base64:{{serviceAccountKey}}&operation=getObject&objectName=${exchangeProperty.google-storage-object-id}
+ - to: 'kamelet:sink'
[camel-kamelets] 03/09: Add a Google Cloud Storage CDC Source Kamelet
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch 1188
in repository https://gitbox.apache.org/repos/asf/camel-kamelets.git
commit 0f79bc87eab3f69305dcfe8e0c1ca95268ce0642
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Wed Nov 30 11:26:34 2022 +0100
Add a Google Cloud Storage CDC Source Kamelet
Signed-off-by: Andrea Cosentino <an...@gmail.com>
---
kamelets/google-storage-cdc-source.kamelet.yaml | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/kamelets/google-storage-cdc-source.kamelet.yaml b/kamelets/google-storage-cdc-source.kamelet.yaml
index 1e7e48ab..aa1bd513 100644
--- a/kamelets/google-storage-cdc-source.kamelet.yaml
+++ b/kamelets/google-storage-cdc-source.kamelet.yaml
@@ -28,9 +28,13 @@ metadata:
camel.apache.org/kamelet.type: "source"
spec:
definition:
- title: "Google Pubsub Source"
+ title: "Google Storage CDC Source"
description: |-
- Consume messages from Google Cloud Pub/Sub.
+ Receive data from Google Pubsub reporting events related to a Google Storage bucket.
+
+ Service account key is the basic method for authenticating to the Google Pubsub and Google Storage.
+
+ To use this Kamelet you'll need to set up Events notification for your Google Storage Bucket.
required:
- projectId
- subscriptionName
[camel-kamelets] 07/09: Add a Google Cloud Storage CDC Source Kamelet
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch 1188
in repository https://gitbox.apache.org/repos/asf/camel-kamelets.git
commit 3be56abf0a2c34ce4f46f49592d40b33f76b4533
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Wed Nov 30 11:30:24 2022 +0100
Add a Google Cloud Storage CDC Source Kamelet
Signed-off-by: Andrea Cosentino <an...@gmail.com>
---
kamelets/google-storage-cdc-source.kamelet.yaml | 1 +
1 file changed, 1 insertion(+)
diff --git a/kamelets/google-storage-cdc-source.kamelet.yaml b/kamelets/google-storage-cdc-source.kamelet.yaml
index aa1bd513..f55543ea 100644
--- a/kamelets/google-storage-cdc-source.kamelet.yaml
+++ b/kamelets/google-storage-cdc-source.kamelet.yaml
@@ -91,6 +91,7 @@ spec:
- "camel:google-pubsub"
- "camel:google-storage"
- "camel:jackson"
+ - "camel:core"
template:
from:
uri: "google-pubsub://{{projectId}}:{{subscriptionName}}"
[camel-kamelets] 04/09: Add a Google Cloud Storage CDC Source Kamelet
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch 1188
in repository https://gitbox.apache.org/repos/asf/camel-kamelets.git
commit cbe49f7c0bef02504871b1826a68ba3df4774e2a
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Wed Nov 30 11:28:25 2022 +0100
Add a Google Cloud Storage CDC Source Kamelet
Signed-off-by: Andrea Cosentino <an...@gmail.com>
---
.../resources/kamelets/google-storage-cdc-source.kamelet.yaml | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/library/camel-kamelets/src/main/resources/kamelets/google-storage-cdc-source.kamelet.yaml b/library/camel-kamelets/src/main/resources/kamelets/google-storage-cdc-source.kamelet.yaml
index 1e7e48ab..aa1bd513 100644
--- a/library/camel-kamelets/src/main/resources/kamelets/google-storage-cdc-source.kamelet.yaml
+++ b/library/camel-kamelets/src/main/resources/kamelets/google-storage-cdc-source.kamelet.yaml
@@ -28,9 +28,13 @@ metadata:
camel.apache.org/kamelet.type: "source"
spec:
definition:
- title: "Google Pubsub Source"
+ title: "Google Storage CDC Source"
description: |-
- Consume messages from Google Cloud Pub/Sub.
+ Receive data from Google Pubsub reporting events related to a Google Storage bucket.
+
+ Service account key is the basic method for authenticating to the Google Pubsub and Google Storage.
+
+ To use this Kamelet you'll need to set up Events notification for your Google Storage Bucket.
required:
- projectId
- subscriptionName
[camel-kamelets] 06/09: Add a Google Cloud Storage CDC Source Kamelet - Docs
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch 1188
in repository https://gitbox.apache.org/repos/asf/camel-kamelets.git
commit 5f03f06f6890ca56d6b751413ae8515032fc6841
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Wed Nov 30 11:29:36 2022 +0100
Add a Google Cloud Storage CDC Source Kamelet - Docs
Signed-off-by: Andrea Cosentino <an...@gmail.com>
---
docs/modules/ROOT/nav.adoc | 206 +++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 206 insertions(+)
diff --git a/docs/modules/ROOT/nav.adoc b/docs/modules/ROOT/nav.adoc
index f6db3169..8c2f44a5 100644
--- a/docs/modules/ROOT/nav.adoc
+++ b/docs/modules/ROOT/nav.adoc
@@ -1,2 +1,208 @@
// THIS FILE IS AUTOMATICALLY GENERATED: DO NOT EDIT
+* xref:avro-deserialize-action.adoc[]
+* xref:avro-serialize-action.adoc[]
+* xref:aws-cloudtrail-source.adoc[]
+* xref:aws-cloudwatch-sink.adoc[]
+* xref:aws-ddb-sink.adoc[]
+* xref:aws-ddb-streams-source.adoc[]
+* xref:aws-ec2-sink.adoc[]
+* xref:aws-eventbridge-sink.adoc[]
+* xref:aws-kinesis-firehose-sink.adoc[]
+* xref:aws-kinesis-sink.adoc[]
+* xref:aws-kinesis-source.adoc[]
+* xref:aws-lambda-sink.adoc[]
+* xref:aws-redshift-sink.adoc[]
+* xref:aws-redshift-source.adoc[]
+* xref:aws-s3-cdc-source.adoc[]
+* xref:aws-s3-sink.adoc[]
+* xref:aws-s3-source.adoc[]
+* xref:aws-s3-streaming-upload-sink.adoc[]
+* xref:aws-secrets-manager-sink.adoc[]
+* xref:aws-ses-sink.adoc[]
+* xref:aws-sns-fifo-sink.adoc[]
+* xref:aws-sns-sink.adoc[]
+* xref:aws-sqs-batch-sink.adoc[]
+* xref:aws-sqs-fifo-sink.adoc[]
+* xref:aws-sqs-sink.adoc[]
+* xref:aws-sqs-source.adoc[]
+* xref:aws-translate-action.adoc[]
+* xref:azure-cosmosdb-source.adoc[]
+* xref:azure-eventhubs-sink.adoc[]
+* xref:azure-eventhubs-source.adoc[]
+* xref:azure-functions-sink.adoc[]
+* xref:azure-servicebus-sink.adoc[]
+* xref:azure-servicebus-source.adoc[]
+* xref:azure-storage-blob-cdc-source.adoc[]
+* xref:azure-storage-blob-changefeed-source.adoc[]
+* xref:azure-storage-blob-sink.adoc[]
+* xref:azure-storage-blob-source.adoc[]
+* xref:azure-storage-queue-sink.adoc[]
+* xref:azure-storage-queue-source.adoc[]
+* xref:beer-source.adoc[]
+* xref:bitcoin-source.adoc[]
+* xref:caffeine-action.adoc[]
+* xref:cassandra-sink.adoc[]
+* xref:cassandra-source.adoc[]
+* xref:ceph-sink.adoc[]
+* xref:ceph-source.adoc[]
+* xref:chuck-norris-source.adoc[]
+* xref:chunk-template-action.adoc[]
+* xref:couchbase-sink.adoc[]
+* xref:cron-source.adoc[]
+* xref:delay-action.adoc[]
+* xref:dns-dig-action.adoc[]
+* xref:dns-ip-action.adoc[]
+* xref:dns-lookup-action.adoc[]
+* xref:drop-header-action.adoc[]
+* xref:drop-headers-action.adoc[]
+* xref:dropbox-sink.adoc[]
+* xref:dropbox-source.adoc[]
+* xref:earthquake-source.adoc[]
+* xref:elasticsearch-index-sink.adoc[]
+* xref:elasticsearch-search-source.adoc[]
+* xref:exec-sink.adoc[]
+* xref:extract-field-action.adoc[]
+* xref:fhir-source.adoc[]
+* xref:file-watch-source.adoc[]
+* xref:freemarker-template-action.adoc[]
+* xref:ftp-sink.adoc[]
+* xref:ftp-source.adoc[]
+* xref:ftps-sink.adoc[]
+* xref:ftps-source.adoc[]
+* xref:github-commit-source.adoc[]
+* xref:github-event-source.adoc[]
+* xref:github-pullrequest-comment-source.adoc[]
+* xref:github-pullrequest-source.adoc[]
+* xref:github-tag-source.adoc[]
+* xref:google-bigquery-sink.adoc[]
+* xref:google-calendar-source.adoc[]
+* xref:google-functions-sink.adoc[]
+* xref:google-mail-source.adoc[]
+* xref:google-pubsub-sink.adoc[]
+* xref:google-pubsub-source.adoc[]
+* xref:google-sheets-source.adoc[]
+* xref:google-storage-cdc-source.adoc[]
+* xref:google-storage-sink.adoc[]
+* xref:google-storage-source.adoc[]
+* xref:has-header-filter-action.adoc[]
+* xref:header-matches-filter-action.adoc[]
+* xref:hoist-field-action.adoc[]
+* xref:http-secured-sink.adoc[]
+* xref:http-secured-source.adoc[]
+* xref:http-sink.adoc[]
+* xref:http-source.adoc[]
+* xref:infinispan-sink.adoc[]
+* xref:infinispan-source.adoc[]
+* xref:insert-field-action.adoc[]
+* xref:insert-header-action.adoc[]
+* xref:is-tombstone-filter-action.adoc[]
+* xref:jira-add-comment-sink.adoc[]
+* xref:jira-add-issue-sink.adoc[]
+* xref:jira-oauth-source.adoc[]
+* xref:jira-source.adoc[]
+* xref:jira-transition-issue-sink.adoc[]
+* xref:jira-update-issue-sink.adoc[]
+* xref:jms-amqp-10-sink.adoc[]
+* xref:jms-amqp-10-source.adoc[]
+* xref:jms-apache-activemq-sink.adoc[]
+* xref:jms-apache-activemq-source.adoc[]
+* xref:jms-apache-artemis-sink.adoc[]
+* xref:jms-apache-artemis-source.adoc[]
+* xref:jms-ibm-mq-sink.adoc[]
+* xref:jms-ibm-mq-source.adoc[]
+* xref:jolt-transformation-action.adoc[]
+* xref:jslt-action.adoc[]
+* xref:json-deserialize-action.adoc[]
+* xref:json-patch-action.adoc[]
+* xref:json-schema-validator-action.adoc[]
+* xref:json-serialize-action.adoc[]
+* xref:jsonata-action.adoc[]
+* xref:kafka-manual-commit-action.adoc[]
+* xref:kafka-not-secured-sink.adoc[]
+* xref:kafka-not-secured-source.adoc[]
+* xref:kafka-sink.adoc[]
+* xref:kafka-source.adoc[]
+* xref:kafka-ssl-sink.adoc[]
+* xref:kafka-ssl-source.adoc[]
+* xref:kubernetes-namespaces-source.adoc[]
+* xref:kubernetes-nodes-source.adoc[]
+* xref:kubernetes-pods-source.adoc[]
+* xref:log-action.adoc[]
+* xref:log-sink.adoc[]
+* xref:mail-imap-source.adoc[]
+* xref:mail-sink.adoc[]
+* xref:mariadb-sink.adoc[]
+* xref:mariadb-source.adoc[]
+* xref:mask-field-action.adoc[]
+* xref:message-timestamp-router-action.adoc[]
+* xref:minio-sink.adoc[]
+* xref:minio-source.adoc[]
+* xref:mongodb-changes-stream-source.adoc[]
+* xref:mongodb-sink.adoc[]
+* xref:mongodb-source.adoc[]
+* xref:mqtt-sink.adoc[]
+* xref:mqtt-source.adoc[]
+* xref:mqtt5-sink.adoc[]
+* xref:mqtt5-source.adoc[]
+* xref:mustache-template-action.adoc[]
+* xref:mvel-template-action.adoc[]
+* xref:mysql-sink.adoc[]
+* xref:mysql-source.adoc[]
+* xref:nats-sink.adoc[]
+* xref:nats-source.adoc[]
+* xref:nominatim-geocode-action.adoc[]
+* xref:ogcapi-features-action.adoc[]
+* xref:openai-classification-action.adoc[]
+* xref:openai-completion-action.adoc[]
+* xref:oracle-database-sink.adoc[]
+* xref:oracle-database-source.adoc[]
+* xref:pdf-action.adoc[]
+* xref:postgresql-sink.adoc[]
+* xref:postgresql-source.adoc[]
+* xref:predicate-filter-action.adoc[]
+* xref:protobuf-deserialize-action.adoc[]
+* xref:protobuf-serialize-action.adoc[]
+* xref:pulsar-sink.adoc[]
+* xref:pulsar-source.adoc[]
+* xref:rabbitmq-source.adoc[]
+* xref:redis-sink.adoc[]
+* xref:redis-source.adoc[]
+* xref:regex-router-action.adoc[]
+* xref:replace-field-action.adoc[]
+* xref:rest-openapi-sink.adoc[]
+* xref:salesforce-create-sink.adoc[]
+* xref:salesforce-delete-sink.adoc[]
+* xref:salesforce-source.adoc[]
+* xref:salesforce-update-sink.adoc[]
+* xref:scp-sink.adoc[]
+* xref:sftp-sink.adoc[]
+* xref:sftp-source.adoc[]
+* xref:slack-sink.adoc[]
+* xref:slack-source.adoc[]
+* xref:solr-sink.adoc[]
+* xref:solr-source.adoc[]
+* xref:splunk-hec-sink.adoc[]
+* xref:splunk-sink.adoc[]
+* xref:splunk-source.adoc[]
+* xref:sqlserver-sink.adoc[]
+* xref:sqlserver-source.adoc[]
+* xref:ssh-sink.adoc[]
+* xref:ssh-source.adoc[]
+* xref:string-template-action.adoc[]
+* xref:telegram-sink.adoc[]
+* xref:telegram-source.adoc[]
+* xref:throttle-action.adoc[]
+* xref:timer-source.adoc[]
+* xref:timestamp-router-action.adoc[]
+* xref:topic-name-matches-filter-action.adoc[]
+* xref:twitter-directmessage-source.adoc[]
+* xref:twitter-search-source.adoc[]
+* xref:twitter-timeline-source.adoc[]
+* xref:value-to-key-action.adoc[]
+* xref:velocity-template-action.adoc[]
+* xref:webhook-source.adoc[]
+* xref:websocket-source.adoc[]
+* xref:wttrin-source.adoc[]
+* xref:xj-identity-action.adoc[]
+* xref:xj-template-action.adoc[]
// THIS FILE IS AUTOMATICALLY GENERATED: DO NOT EDIT
[camel-kamelets] 05/09: Add a Google Cloud Storage CDC Source Kamelet - Docs
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch 1188
in repository https://gitbox.apache.org/repos/asf/camel-kamelets.git
commit 07fd61ddca8a5ed9ef04dfb11be0c44055060708
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Wed Nov 30 11:28:49 2022 +0100
Add a Google Cloud Storage CDC Source Kamelet - Docs
Signed-off-by: Andrea Cosentino <an...@gmail.com>
---
docs/modules/ROOT/nav.adoc | 205 ---------------------------------------------
1 file changed, 205 deletions(-)
diff --git a/docs/modules/ROOT/nav.adoc b/docs/modules/ROOT/nav.adoc
index 6a7ea6ec..f6db3169 100644
--- a/docs/modules/ROOT/nav.adoc
+++ b/docs/modules/ROOT/nav.adoc
@@ -1,207 +1,2 @@
// THIS FILE IS AUTOMATICALLY GENERATED: DO NOT EDIT
-* xref:avro-deserialize-action.adoc[]
-* xref:avro-serialize-action.adoc[]
-* xref:aws-cloudtrail-source.adoc[]
-* xref:aws-cloudwatch-sink.adoc[]
-* xref:aws-ddb-sink.adoc[]
-* xref:aws-ddb-streams-source.adoc[]
-* xref:aws-ec2-sink.adoc[]
-* xref:aws-eventbridge-sink.adoc[]
-* xref:aws-kinesis-firehose-sink.adoc[]
-* xref:aws-kinesis-sink.adoc[]
-* xref:aws-kinesis-source.adoc[]
-* xref:aws-lambda-sink.adoc[]
-* xref:aws-redshift-sink.adoc[]
-* xref:aws-redshift-source.adoc[]
-* xref:aws-s3-cdc-source.adoc[]
-* xref:aws-s3-sink.adoc[]
-* xref:aws-s3-source.adoc[]
-* xref:aws-s3-streaming-upload-sink.adoc[]
-* xref:aws-secrets-manager-sink.adoc[]
-* xref:aws-ses-sink.adoc[]
-* xref:aws-sns-fifo-sink.adoc[]
-* xref:aws-sns-sink.adoc[]
-* xref:aws-sqs-batch-sink.adoc[]
-* xref:aws-sqs-fifo-sink.adoc[]
-* xref:aws-sqs-sink.adoc[]
-* xref:aws-sqs-source.adoc[]
-* xref:aws-translate-action.adoc[]
-* xref:azure-cosmosdb-source.adoc[]
-* xref:azure-eventhubs-sink.adoc[]
-* xref:azure-eventhubs-source.adoc[]
-* xref:azure-functions-sink.adoc[]
-* xref:azure-servicebus-sink.adoc[]
-* xref:azure-servicebus-source.adoc[]
-* xref:azure-storage-blob-cdc-source.adoc[]
-* xref:azure-storage-blob-changefeed-source.adoc[]
-* xref:azure-storage-blob-sink.adoc[]
-* xref:azure-storage-blob-source.adoc[]
-* xref:azure-storage-queue-sink.adoc[]
-* xref:azure-storage-queue-source.adoc[]
-* xref:beer-source.adoc[]
-* xref:bitcoin-source.adoc[]
-* xref:caffeine-action.adoc[]
-* xref:cassandra-sink.adoc[]
-* xref:cassandra-source.adoc[]
-* xref:ceph-sink.adoc[]
-* xref:ceph-source.adoc[]
-* xref:chuck-norris-source.adoc[]
-* xref:chunk-template-action.adoc[]
-* xref:couchbase-sink.adoc[]
-* xref:cron-source.adoc[]
-* xref:delay-action.adoc[]
-* xref:dns-dig-action.adoc[]
-* xref:dns-ip-action.adoc[]
-* xref:dns-lookup-action.adoc[]
-* xref:drop-header-action.adoc[]
-* xref:drop-headers-action.adoc[]
-* xref:dropbox-sink.adoc[]
-* xref:dropbox-source.adoc[]
-* xref:earthquake-source.adoc[]
-* xref:elasticsearch-index-sink.adoc[]
-* xref:elasticsearch-search-source.adoc[]
-* xref:exec-sink.adoc[]
-* xref:extract-field-action.adoc[]
-* xref:fhir-source.adoc[]
-* xref:file-watch-source.adoc[]
-* xref:freemarker-template-action.adoc[]
-* xref:ftp-sink.adoc[]
-* xref:ftp-source.adoc[]
-* xref:ftps-sink.adoc[]
-* xref:ftps-source.adoc[]
-* xref:github-commit-source.adoc[]
-* xref:github-event-source.adoc[]
-* xref:github-pullrequest-comment-source.adoc[]
-* xref:github-pullrequest-source.adoc[]
-* xref:github-tag-source.adoc[]
-* xref:google-bigquery-sink.adoc[]
-* xref:google-calendar-source.adoc[]
-* xref:google-functions-sink.adoc[]
-* xref:google-mail-source.adoc[]
-* xref:google-pubsub-sink.adoc[]
-* xref:google-pubsub-source.adoc[]
-* xref:google-sheets-source.adoc[]
-* xref:google-storage-sink.adoc[]
-* xref:google-storage-source.adoc[]
-* xref:has-header-filter-action.adoc[]
-* xref:header-matches-filter-action.adoc[]
-* xref:hoist-field-action.adoc[]
-* xref:http-secured-sink.adoc[]
-* xref:http-secured-source.adoc[]
-* xref:http-sink.adoc[]
-* xref:http-source.adoc[]
-* xref:infinispan-sink.adoc[]
-* xref:infinispan-source.adoc[]
-* xref:insert-field-action.adoc[]
-* xref:insert-header-action.adoc[]
-* xref:is-tombstone-filter-action.adoc[]
-* xref:jira-add-comment-sink.adoc[]
-* xref:jira-add-issue-sink.adoc[]
-* xref:jira-oauth-source.adoc[]
-* xref:jira-source.adoc[]
-* xref:jira-transition-issue-sink.adoc[]
-* xref:jira-update-issue-sink.adoc[]
-* xref:jms-amqp-10-sink.adoc[]
-* xref:jms-amqp-10-source.adoc[]
-* xref:jms-apache-activemq-sink.adoc[]
-* xref:jms-apache-activemq-source.adoc[]
-* xref:jms-apache-artemis-sink.adoc[]
-* xref:jms-apache-artemis-source.adoc[]
-* xref:jms-ibm-mq-sink.adoc[]
-* xref:jms-ibm-mq-source.adoc[]
-* xref:jolt-transformation-action.adoc[]
-* xref:jslt-action.adoc[]
-* xref:json-deserialize-action.adoc[]
-* xref:json-patch-action.adoc[]
-* xref:json-schema-validator-action.adoc[]
-* xref:json-serialize-action.adoc[]
-* xref:jsonata-action.adoc[]
-* xref:kafka-manual-commit-action.adoc[]
-* xref:kafka-not-secured-sink.adoc[]
-* xref:kafka-not-secured-source.adoc[]
-* xref:kafka-sink.adoc[]
-* xref:kafka-source.adoc[]
-* xref:kafka-ssl-sink.adoc[]
-* xref:kafka-ssl-source.adoc[]
-* xref:kubernetes-namespaces-source.adoc[]
-* xref:kubernetes-nodes-source.adoc[]
-* xref:kubernetes-pods-source.adoc[]
-* xref:log-action.adoc[]
-* xref:log-sink.adoc[]
-* xref:mail-imap-source.adoc[]
-* xref:mail-sink.adoc[]
-* xref:mariadb-sink.adoc[]
-* xref:mariadb-source.adoc[]
-* xref:mask-field-action.adoc[]
-* xref:message-timestamp-router-action.adoc[]
-* xref:minio-sink.adoc[]
-* xref:minio-source.adoc[]
-* xref:mongodb-changes-stream-source.adoc[]
-* xref:mongodb-sink.adoc[]
-* xref:mongodb-source.adoc[]
-* xref:mqtt-sink.adoc[]
-* xref:mqtt-source.adoc[]
-* xref:mqtt5-sink.adoc[]
-* xref:mqtt5-source.adoc[]
-* xref:mustache-template-action.adoc[]
-* xref:mvel-template-action.adoc[]
-* xref:mysql-sink.adoc[]
-* xref:mysql-source.adoc[]
-* xref:nats-sink.adoc[]
-* xref:nats-source.adoc[]
-* xref:nominatim-geocode-action.adoc[]
-* xref:ogcapi-features-action.adoc[]
-* xref:openai-classification-action.adoc[]
-* xref:openai-completion-action.adoc[]
-* xref:oracle-database-sink.adoc[]
-* xref:oracle-database-source.adoc[]
-* xref:pdf-action.adoc[]
-* xref:postgresql-sink.adoc[]
-* xref:postgresql-source.adoc[]
-* xref:predicate-filter-action.adoc[]
-* xref:protobuf-deserialize-action.adoc[]
-* xref:protobuf-serialize-action.adoc[]
-* xref:pulsar-sink.adoc[]
-* xref:pulsar-source.adoc[]
-* xref:rabbitmq-source.adoc[]
-* xref:redis-sink.adoc[]
-* xref:redis-source.adoc[]
-* xref:regex-router-action.adoc[]
-* xref:replace-field-action.adoc[]
-* xref:rest-openapi-sink.adoc[]
-* xref:salesforce-create-sink.adoc[]
-* xref:salesforce-delete-sink.adoc[]
-* xref:salesforce-source.adoc[]
-* xref:salesforce-update-sink.adoc[]
-* xref:scp-sink.adoc[]
-* xref:sftp-sink.adoc[]
-* xref:sftp-source.adoc[]
-* xref:slack-sink.adoc[]
-* xref:slack-source.adoc[]
-* xref:solr-sink.adoc[]
-* xref:solr-source.adoc[]
-* xref:splunk-hec-sink.adoc[]
-* xref:splunk-sink.adoc[]
-* xref:splunk-source.adoc[]
-* xref:sqlserver-sink.adoc[]
-* xref:sqlserver-source.adoc[]
-* xref:ssh-sink.adoc[]
-* xref:ssh-source.adoc[]
-* xref:string-template-action.adoc[]
-* xref:telegram-sink.adoc[]
-* xref:telegram-source.adoc[]
-* xref:throttle-action.adoc[]
-* xref:timer-source.adoc[]
-* xref:timestamp-router-action.adoc[]
-* xref:topic-name-matches-filter-action.adoc[]
-* xref:twitter-directmessage-source.adoc[]
-* xref:twitter-search-source.adoc[]
-* xref:twitter-timeline-source.adoc[]
-* xref:value-to-key-action.adoc[]
-* xref:velocity-template-action.adoc[]
-* xref:webhook-source.adoc[]
-* xref:websocket-source.adoc[]
-* xref:wttrin-source.adoc[]
-* xref:xj-identity-action.adoc[]
-* xref:xj-template-action.adoc[]
// THIS FILE IS AUTOMATICALLY GENERATED: DO NOT EDIT
[camel-kamelets] 02/09: Add a Google Cloud Storage CDC Source Kamelet
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch 1188
in repository https://gitbox.apache.org/repos/asf/camel-kamelets.git
commit 26888462b97a5b58a4bc7ab9feab65408b3c19cf
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Wed Nov 30 11:19:37 2022 +0100
Add a Google Cloud Storage CDC Source Kamelet
Signed-off-by: Andrea Cosentino <an...@gmail.com>
---
.../google-storage-cdc-source.kamelet.yaml | 123 +++++++++++++++++++++
1 file changed, 123 insertions(+)
diff --git a/library/camel-kamelets/src/main/resources/kamelets/google-storage-cdc-source.kamelet.yaml b/library/camel-kamelets/src/main/resources/kamelets/google-storage-cdc-source.kamelet.yaml
new file mode 100644
index 00000000..1e7e48ab
--- /dev/null
+++ b/library/camel-kamelets/src/main/resources/kamelets/google-storage-cdc-source.kamelet.yaml
@@ -0,0 +1,123 @@
+# ---------------------------------------------------------------------------
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ---------------------------------------------------------------------------
+apiVersion: camel.apache.org/v1alpha1
+kind: Kamelet
+metadata:
+ name: google-storage-cdc-source
+ annotations:
+ camel.apache.org/kamelet.support.level: "Stable"
+ camel.apache.org/catalog.version: "main-SNAPSHOT"
+ camel.apache.org/kamelet.icon: "data:image/svg+xml;base64,PHN2ZyBoZWlnaHQ9IjI1MDAiIHZpZXdCb3g9Ii0xLjYzMzIzNTQzIDcuMDMyNjA5MzMgMTMxLjI2NTc0NjgyIDExNC40MzkzOTA2NyIgd2lkdGg9IjI1MDAiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyIgeG1sbnM6eGxpbms9Imh0dHA6Ly93d3cudzMub3JnLzE5OTkveGxpbmsiPjxsaW5lYXJHcmFkaWVudCBpZD0iYSIgZ3JhZGllbnRVbml0cz0idXNlclNwYWNlT25Vc2UiIHgxPSI2NCIgeDI9IjY0IiB5MT0iNy4wMzQiIHkyPSIxMjAuNzg5Ij48c3RvcCBvZmZzZXQ9IjAiIHN0b3AtY29sb3I9IiM0Mzg3ZmQiLz48c3RvcCBvZmZzZXQ9IjEiIHN0b3 [...]
+ camel.apache.org/provider: "Apache Software Foundation"
+ camel.apache.org/kamelet.group: "Google Storage"
+ labels:
+ camel.apache.org/kamelet.type: "source"
+spec:
+ definition:
+ title: "Google Pubsub Source"
+ description: |-
+ Consume messages from Google Cloud Pub/Sub.
+ required:
+ - projectId
+ - subscriptionName
+ - serviceAccountKey
+ - bucketNameOrArn
+ type: object
+ properties:
+ projectId:
+ title: Project Id
+ description: The Google Cloud Pub/Sub Project ID.
+ type: string
+ subscriptionName:
+ title: Subscription Name
+ description: The subscription name.
+ type: string
+ serviceAccountKey:
+ title: Service Account Key
+ description: The service account key to use as credentials for the Pub/Sub publisher/subscriber. You must encode this value in base64.
+ type: binary
+ x-descriptors:
+ - urn:camel:group:credentials
+ synchronousPull:
+ title: Synchronous Pull
+ description: Specifies to synchronously pull batches of messages.
+ type: boolean
+ x-descriptors:
+ - 'urn:alm:descriptor:com.tectonic.ui:checkbox'
+ default: false
+ maxMessagesPerPoll:
+ title: Max Messages Per Poll
+ description: The maximum number of messages to receive from the server in a single API call.
+ type: integer
+ default: 1
+ concurrentConsumers:
+ title: Concurrent Consumers
+ description: The number of parallel streams to consume from the subscription.
+ type: integer
+ default: 1
+ bucketNameOrArn:
+ title: Bucket Name Or ARN
+ description: The Google Cloud Storage bucket name or Bucket Amazon Resource Name (ARN).
+ type: string
+ getObject:
+ title: Get Object in Bucket
+ description: >-
+ If getObject is enabled, then the file created in the Bucket will be
+ get and returned as body, if not only the event will be returned as body.
+ type: boolean
+ x-descriptors:
+ - 'urn:alm:descriptor:com.tectonic.ui:checkbox'
+ default: false
+ dependencies:
+ - "camel:kamelet"
+ - "camel:google-pubsub"
+ - "camel:google-storage"
+ - "camel:jackson"
+ template:
+ from:
+ uri: "google-pubsub://{{projectId}}:{{subscriptionName}}"
+ parameters:
+ serviceAccountKey: "base64:{{serviceAccountKey}}"
+ synchronousPull: "{{synchronousPull}}"
+ maxMessagesPerPoll: "{{maxMessagesPerPoll}}"
+ concurrentConsumers: "{{concurrentConsumers}}"
+ steps:
+ - choice:
+ precondition: true
+ when:
+ - simple: '${properties:getObject:true}'
+ steps:
+ - set-header:
+ name: attributesStorage
+ simple: ${headers.CamelGooglePubsub.Attributes}
+ - log: "${headers.attributesStorage}"
+ - set-property:
+ name: google-storage-event-type
+ simple: "${headers.attributesStorage[eventType]}"
+ - set-property:
+ name: google-storage-object-id
+ simple: "${headers.attributesStorage[objectId]}"
+ - set-property:
+ name: google-storage-bucket-id
+ simple: "${headers.attributesStorage[bucketId]}"
+ - choice:
+ when:
+ - simple: '${exchangeProperty.google-storage-event-type} == "OBJECT_FINALIZE" && ${exchangeProperty.google-storage-bucket-id} == "{{bucketNameOrArn}}"'
+ steps:
+ - toD: >-
+ google-storage:{{bucketNameOrArn}}?serviceAccountKey=base64:{{serviceAccountKey}}&operation=getObject&objectName=${exchangeProperty.google-storage-object-id}
+ - to: 'kamelet:sink'
[camel-kamelets] 09/09: Add a Google Cloud Storage CDC Source Kamelet
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch 1188
in repository https://gitbox.apache.org/repos/asf/camel-kamelets.git
commit d59622379fdf12ba1635de49f40b59c5493796ac
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Wed Nov 30 11:33:03 2022 +0100
Add a Google Cloud Storage CDC Source Kamelet
Signed-off-by: Andrea Cosentino <an...@gmail.com>
---
script/validator/validator.go | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/script/validator/validator.go b/script/validator/validator.go
index 42b5e4b4..c880695c 100644
--- a/script/validator/validator.go
+++ b/script/validator/validator.go
@@ -410,7 +410,7 @@ func listKamelets(dir string) []KameletInfo {
func verifyUsedParams(kamelets []KameletInfo) (errors []error) {
for _, k := range kamelets {
- if (k.FileName != "../../kamelets/azure-storage-blob-source.kamelet.yaml" && k.FileName != "../../kamelets/aws-s3-cdc-source.kamelet.yaml" && k.FileName != "../../kamelets/azure-storage-blob-cdc-source.kamelet.yaml") {
+ if (k.FileName != "../../kamelets/azure-storage-blob-source.kamelet.yaml" && k.FileName != "../../kamelets/aws-s3-cdc-source.kamelet.yaml" && k.FileName != "../../kamelets/azure-storage-blob-cdc-source.kamelet.yaml" && k.FileName != "../../kamelets/google-storage-cdc-source.kamelet.yaml") {
used := getUsedParams(k.Kamelet)
declared := getDeclaredParams(k.Kamelet)
for p := range used {