You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by ac...@apache.org on 2022/11/30 10:48:11 UTC
[camel-kamelets] branch main updated (38fb581a -> bb204b90)
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/camel-kamelets.git
from 38fb581a Updated CHANGELOG.md
new 6068ff26 Add a Google Cloud Storage CDC Source Kamelet
new 0e571447 Add a Google Cloud Storage CDC Source Kamelet
new 227b204b Add a Google Cloud Storage CDC Source Kamelet
new 08134595 Add a Google Cloud Storage CDC Source Kamelet
new 324c5687 Add a Google Cloud Storage CDC Source Kamelet - Docs
new ca275e5f Add a Google Cloud Storage CDC Source Kamelet - Docs
new e69205a6 Add a Google Cloud Storage CDC Source Kamelet
new 788f58cd Add a Google Cloud Storage CDC Source Kamelet
new 9b07be89 Add a Google Cloud Storage CDC Source Kamelet
new c57fec1f Add a Google Cloud Storage CDC Source Kamelet
new bb204b90 Add a Google Cloud Storage CDC Source Kamelet
The 11 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
docs/modules/ROOT/nav.adoc | 1 +
.../google-storage-cdc-source.kamelet.yaml | 58 +++++++++++++++++++---
...yaml => google-storage-cdc-source.kamelet.yaml} | 58 +++++++++++++++++++---
script/validator/validator.go | 2 +-
4 files changed, 106 insertions(+), 13 deletions(-)
copy library/camel-kamelets/src/main/resources/kamelets/google-pubsub-source.kamelet.yaml => kamelets/google-storage-cdc-source.kamelet.yaml (64%)
copy library/camel-kamelets/src/main/resources/kamelets/{google-pubsub-source.kamelet.yaml => google-storage-cdc-source.kamelet.yaml} (64%)
[camel-kamelets] 05/11: Add a Google Cloud Storage CDC Source Kamelet - Docs
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel-kamelets.git
commit 324c5687734426d999519d09cea925f7f49e174b
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Wed Nov 30 11:28:49 2022 +0100
Add a Google Cloud Storage CDC Source Kamelet - Docs
Signed-off-by: Andrea Cosentino <an...@gmail.com>
---
docs/modules/ROOT/nav.adoc | 205 ---------------------------------------------
1 file changed, 205 deletions(-)
diff --git a/docs/modules/ROOT/nav.adoc b/docs/modules/ROOT/nav.adoc
index 6a7ea6ec..f6db3169 100644
--- a/docs/modules/ROOT/nav.adoc
+++ b/docs/modules/ROOT/nav.adoc
@@ -1,207 +1,2 @@
// THIS FILE IS AUTOMATICALLY GENERATED: DO NOT EDIT
-* xref:avro-deserialize-action.adoc[]
-* xref:avro-serialize-action.adoc[]
-* xref:aws-cloudtrail-source.adoc[]
-* xref:aws-cloudwatch-sink.adoc[]
-* xref:aws-ddb-sink.adoc[]
-* xref:aws-ddb-streams-source.adoc[]
-* xref:aws-ec2-sink.adoc[]
-* xref:aws-eventbridge-sink.adoc[]
-* xref:aws-kinesis-firehose-sink.adoc[]
-* xref:aws-kinesis-sink.adoc[]
-* xref:aws-kinesis-source.adoc[]
-* xref:aws-lambda-sink.adoc[]
-* xref:aws-redshift-sink.adoc[]
-* xref:aws-redshift-source.adoc[]
-* xref:aws-s3-cdc-source.adoc[]
-* xref:aws-s3-sink.adoc[]
-* xref:aws-s3-source.adoc[]
-* xref:aws-s3-streaming-upload-sink.adoc[]
-* xref:aws-secrets-manager-sink.adoc[]
-* xref:aws-ses-sink.adoc[]
-* xref:aws-sns-fifo-sink.adoc[]
-* xref:aws-sns-sink.adoc[]
-* xref:aws-sqs-batch-sink.adoc[]
-* xref:aws-sqs-fifo-sink.adoc[]
-* xref:aws-sqs-sink.adoc[]
-* xref:aws-sqs-source.adoc[]
-* xref:aws-translate-action.adoc[]
-* xref:azure-cosmosdb-source.adoc[]
-* xref:azure-eventhubs-sink.adoc[]
-* xref:azure-eventhubs-source.adoc[]
-* xref:azure-functions-sink.adoc[]
-* xref:azure-servicebus-sink.adoc[]
-* xref:azure-servicebus-source.adoc[]
-* xref:azure-storage-blob-cdc-source.adoc[]
-* xref:azure-storage-blob-changefeed-source.adoc[]
-* xref:azure-storage-blob-sink.adoc[]
-* xref:azure-storage-blob-source.adoc[]
-* xref:azure-storage-queue-sink.adoc[]
-* xref:azure-storage-queue-source.adoc[]
-* xref:beer-source.adoc[]
-* xref:bitcoin-source.adoc[]
-* xref:caffeine-action.adoc[]
-* xref:cassandra-sink.adoc[]
-* xref:cassandra-source.adoc[]
-* xref:ceph-sink.adoc[]
-* xref:ceph-source.adoc[]
-* xref:chuck-norris-source.adoc[]
-* xref:chunk-template-action.adoc[]
-* xref:couchbase-sink.adoc[]
-* xref:cron-source.adoc[]
-* xref:delay-action.adoc[]
-* xref:dns-dig-action.adoc[]
-* xref:dns-ip-action.adoc[]
-* xref:dns-lookup-action.adoc[]
-* xref:drop-header-action.adoc[]
-* xref:drop-headers-action.adoc[]
-* xref:dropbox-sink.adoc[]
-* xref:dropbox-source.adoc[]
-* xref:earthquake-source.adoc[]
-* xref:elasticsearch-index-sink.adoc[]
-* xref:elasticsearch-search-source.adoc[]
-* xref:exec-sink.adoc[]
-* xref:extract-field-action.adoc[]
-* xref:fhir-source.adoc[]
-* xref:file-watch-source.adoc[]
-* xref:freemarker-template-action.adoc[]
-* xref:ftp-sink.adoc[]
-* xref:ftp-source.adoc[]
-* xref:ftps-sink.adoc[]
-* xref:ftps-source.adoc[]
-* xref:github-commit-source.adoc[]
-* xref:github-event-source.adoc[]
-* xref:github-pullrequest-comment-source.adoc[]
-* xref:github-pullrequest-source.adoc[]
-* xref:github-tag-source.adoc[]
-* xref:google-bigquery-sink.adoc[]
-* xref:google-calendar-source.adoc[]
-* xref:google-functions-sink.adoc[]
-* xref:google-mail-source.adoc[]
-* xref:google-pubsub-sink.adoc[]
-* xref:google-pubsub-source.adoc[]
-* xref:google-sheets-source.adoc[]
-* xref:google-storage-sink.adoc[]
-* xref:google-storage-source.adoc[]
-* xref:has-header-filter-action.adoc[]
-* xref:header-matches-filter-action.adoc[]
-* xref:hoist-field-action.adoc[]
-* xref:http-secured-sink.adoc[]
-* xref:http-secured-source.adoc[]
-* xref:http-sink.adoc[]
-* xref:http-source.adoc[]
-* xref:infinispan-sink.adoc[]
-* xref:infinispan-source.adoc[]
-* xref:insert-field-action.adoc[]
-* xref:insert-header-action.adoc[]
-* xref:is-tombstone-filter-action.adoc[]
-* xref:jira-add-comment-sink.adoc[]
-* xref:jira-add-issue-sink.adoc[]
-* xref:jira-oauth-source.adoc[]
-* xref:jira-source.adoc[]
-* xref:jira-transition-issue-sink.adoc[]
-* xref:jira-update-issue-sink.adoc[]
-* xref:jms-amqp-10-sink.adoc[]
-* xref:jms-amqp-10-source.adoc[]
-* xref:jms-apache-activemq-sink.adoc[]
-* xref:jms-apache-activemq-source.adoc[]
-* xref:jms-apache-artemis-sink.adoc[]
-* xref:jms-apache-artemis-source.adoc[]
-* xref:jms-ibm-mq-sink.adoc[]
-* xref:jms-ibm-mq-source.adoc[]
-* xref:jolt-transformation-action.adoc[]
-* xref:jslt-action.adoc[]
-* xref:json-deserialize-action.adoc[]
-* xref:json-patch-action.adoc[]
-* xref:json-schema-validator-action.adoc[]
-* xref:json-serialize-action.adoc[]
-* xref:jsonata-action.adoc[]
-* xref:kafka-manual-commit-action.adoc[]
-* xref:kafka-not-secured-sink.adoc[]
-* xref:kafka-not-secured-source.adoc[]
-* xref:kafka-sink.adoc[]
-* xref:kafka-source.adoc[]
-* xref:kafka-ssl-sink.adoc[]
-* xref:kafka-ssl-source.adoc[]
-* xref:kubernetes-namespaces-source.adoc[]
-* xref:kubernetes-nodes-source.adoc[]
-* xref:kubernetes-pods-source.adoc[]
-* xref:log-action.adoc[]
-* xref:log-sink.adoc[]
-* xref:mail-imap-source.adoc[]
-* xref:mail-sink.adoc[]
-* xref:mariadb-sink.adoc[]
-* xref:mariadb-source.adoc[]
-* xref:mask-field-action.adoc[]
-* xref:message-timestamp-router-action.adoc[]
-* xref:minio-sink.adoc[]
-* xref:minio-source.adoc[]
-* xref:mongodb-changes-stream-source.adoc[]
-* xref:mongodb-sink.adoc[]
-* xref:mongodb-source.adoc[]
-* xref:mqtt-sink.adoc[]
-* xref:mqtt-source.adoc[]
-* xref:mqtt5-sink.adoc[]
-* xref:mqtt5-source.adoc[]
-* xref:mustache-template-action.adoc[]
-* xref:mvel-template-action.adoc[]
-* xref:mysql-sink.adoc[]
-* xref:mysql-source.adoc[]
-* xref:nats-sink.adoc[]
-* xref:nats-source.adoc[]
-* xref:nominatim-geocode-action.adoc[]
-* xref:ogcapi-features-action.adoc[]
-* xref:openai-classification-action.adoc[]
-* xref:openai-completion-action.adoc[]
-* xref:oracle-database-sink.adoc[]
-* xref:oracle-database-source.adoc[]
-* xref:pdf-action.adoc[]
-* xref:postgresql-sink.adoc[]
-* xref:postgresql-source.adoc[]
-* xref:predicate-filter-action.adoc[]
-* xref:protobuf-deserialize-action.adoc[]
-* xref:protobuf-serialize-action.adoc[]
-* xref:pulsar-sink.adoc[]
-* xref:pulsar-source.adoc[]
-* xref:rabbitmq-source.adoc[]
-* xref:redis-sink.adoc[]
-* xref:redis-source.adoc[]
-* xref:regex-router-action.adoc[]
-* xref:replace-field-action.adoc[]
-* xref:rest-openapi-sink.adoc[]
-* xref:salesforce-create-sink.adoc[]
-* xref:salesforce-delete-sink.adoc[]
-* xref:salesforce-source.adoc[]
-* xref:salesforce-update-sink.adoc[]
-* xref:scp-sink.adoc[]
-* xref:sftp-sink.adoc[]
-* xref:sftp-source.adoc[]
-* xref:slack-sink.adoc[]
-* xref:slack-source.adoc[]
-* xref:solr-sink.adoc[]
-* xref:solr-source.adoc[]
-* xref:splunk-hec-sink.adoc[]
-* xref:splunk-sink.adoc[]
-* xref:splunk-source.adoc[]
-* xref:sqlserver-sink.adoc[]
-* xref:sqlserver-source.adoc[]
-* xref:ssh-sink.adoc[]
-* xref:ssh-source.adoc[]
-* xref:string-template-action.adoc[]
-* xref:telegram-sink.adoc[]
-* xref:telegram-source.adoc[]
-* xref:throttle-action.adoc[]
-* xref:timer-source.adoc[]
-* xref:timestamp-router-action.adoc[]
-* xref:topic-name-matches-filter-action.adoc[]
-* xref:twitter-directmessage-source.adoc[]
-* xref:twitter-search-source.adoc[]
-* xref:twitter-timeline-source.adoc[]
-* xref:value-to-key-action.adoc[]
-* xref:velocity-template-action.adoc[]
-* xref:webhook-source.adoc[]
-* xref:websocket-source.adoc[]
-* xref:wttrin-source.adoc[]
-* xref:xj-identity-action.adoc[]
-* xref:xj-template-action.adoc[]
// THIS FILE IS AUTOMATICALLY GENERATED: DO NOT EDIT
[camel-kamelets] 04/11: Add a Google Cloud Storage CDC Source Kamelet
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel-kamelets.git
commit 08134595e655252d55b8cd04d3b78288135a74d2
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Wed Nov 30 11:28:25 2022 +0100
Add a Google Cloud Storage CDC Source Kamelet
Signed-off-by: Andrea Cosentino <an...@gmail.com>
---
.../resources/kamelets/google-storage-cdc-source.kamelet.yaml | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/library/camel-kamelets/src/main/resources/kamelets/google-storage-cdc-source.kamelet.yaml b/library/camel-kamelets/src/main/resources/kamelets/google-storage-cdc-source.kamelet.yaml
index 1e7e48ab..aa1bd513 100644
--- a/library/camel-kamelets/src/main/resources/kamelets/google-storage-cdc-source.kamelet.yaml
+++ b/library/camel-kamelets/src/main/resources/kamelets/google-storage-cdc-source.kamelet.yaml
@@ -28,9 +28,13 @@ metadata:
camel.apache.org/kamelet.type: "source"
spec:
definition:
- title: "Google Pubsub Source"
+ title: "Google Storage CDC Source"
description: |-
- Consume messages from Google Cloud Pub/Sub.
+ Receive data from Google Pubsub reporting events related to a Google Storage bucket.
+
+ Service account key is the basic method for authenticating to the Google Pubsub and Google Storage.
+
+ To use this Kamelet you'll need to set up Events notification for your Google Storage Bucket.
required:
- projectId
- subscriptionName
[camel-kamelets] 09/11: Add a Google Cloud Storage CDC Source Kamelet
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel-kamelets.git
commit 9b07be89e0cc70c97c6afdf1a8d3623a7d6cde4d
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Wed Nov 30 11:33:03 2022 +0100
Add a Google Cloud Storage CDC Source Kamelet
Signed-off-by: Andrea Cosentino <an...@gmail.com>
---
script/validator/validator.go | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/script/validator/validator.go b/script/validator/validator.go
index 42b5e4b4..c880695c 100644
--- a/script/validator/validator.go
+++ b/script/validator/validator.go
@@ -410,7 +410,7 @@ func listKamelets(dir string) []KameletInfo {
func verifyUsedParams(kamelets []KameletInfo) (errors []error) {
for _, k := range kamelets {
- if (k.FileName != "../../kamelets/azure-storage-blob-source.kamelet.yaml" && k.FileName != "../../kamelets/aws-s3-cdc-source.kamelet.yaml" && k.FileName != "../../kamelets/azure-storage-blob-cdc-source.kamelet.yaml") {
+ if (k.FileName != "../../kamelets/azure-storage-blob-source.kamelet.yaml" && k.FileName != "../../kamelets/aws-s3-cdc-source.kamelet.yaml" && k.FileName != "../../kamelets/azure-storage-blob-cdc-source.kamelet.yaml" && k.FileName != "../../kamelets/google-storage-cdc-source.kamelet.yaml") {
used := getUsedParams(k.Kamelet)
declared := getDeclaredParams(k.Kamelet)
for p := range used {
[camel-kamelets] 03/11: Add a Google Cloud Storage CDC Source Kamelet
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel-kamelets.git
commit 227b204bd1a368b131f197788f5ae6632d2783b2
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Wed Nov 30 11:26:34 2022 +0100
Add a Google Cloud Storage CDC Source Kamelet
Signed-off-by: Andrea Cosentino <an...@gmail.com>
---
kamelets/google-storage-cdc-source.kamelet.yaml | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/kamelets/google-storage-cdc-source.kamelet.yaml b/kamelets/google-storage-cdc-source.kamelet.yaml
index 1e7e48ab..aa1bd513 100644
--- a/kamelets/google-storage-cdc-source.kamelet.yaml
+++ b/kamelets/google-storage-cdc-source.kamelet.yaml
@@ -28,9 +28,13 @@ metadata:
camel.apache.org/kamelet.type: "source"
spec:
definition:
- title: "Google Pubsub Source"
+ title: "Google Storage CDC Source"
description: |-
- Consume messages from Google Cloud Pub/Sub.
+ Receive data from Google Pubsub reporting events related to a Google Storage bucket.
+
+ Service account key is the basic method for authenticating to the Google Pubsub and Google Storage.
+
+ To use this Kamelet you'll need to set up Events notification for your Google Storage Bucket.
required:
- projectId
- subscriptionName
[camel-kamelets] 11/11: Add a Google Cloud Storage CDC Source Kamelet
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel-kamelets.git
commit bb204b9083a686bdc7fa74fdab1cf693d8b37fc8
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Wed Nov 30 11:39:40 2022 +0100
Add a Google Cloud Storage CDC Source Kamelet
Signed-off-by: Andrea Cosentino <an...@gmail.com>
---
.../src/main/resources/kamelets/google-storage-cdc-source.kamelet.yaml | 2 ++
1 file changed, 2 insertions(+)
diff --git a/library/camel-kamelets/src/main/resources/kamelets/google-storage-cdc-source.kamelet.yaml b/library/camel-kamelets/src/main/resources/kamelets/google-storage-cdc-source.kamelet.yaml
index f55543ea..b3c9c4b3 100644
--- a/library/camel-kamelets/src/main/resources/kamelets/google-storage-cdc-source.kamelet.yaml
+++ b/library/camel-kamelets/src/main/resources/kamelets/google-storage-cdc-source.kamelet.yaml
@@ -35,6 +35,8 @@ spec:
Service account key is the basic method for authenticating to the Google Pubsub and Google Storage.
To use this Kamelet you'll need to set up Events notification for your Google Storage Bucket.
+
+ You must encode the `serviceAccountKey` property before you pass it as a parameter.
required:
- projectId
- subscriptionName
[camel-kamelets] 01/11: Add a Google Cloud Storage CDC Source Kamelet
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel-kamelets.git
commit 6068ff269eb8523f4778a8e9cc1a5e665718938f
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Wed Nov 30 11:18:25 2022 +0100
Add a Google Cloud Storage CDC Source Kamelet
Signed-off-by: Andrea Cosentino <an...@gmail.com>
---
kamelets/google-storage-cdc-source.kamelet.yaml | 123 ++++++++++++++++++++++++
1 file changed, 123 insertions(+)
diff --git a/kamelets/google-storage-cdc-source.kamelet.yaml b/kamelets/google-storage-cdc-source.kamelet.yaml
new file mode 100644
index 00000000..1e7e48ab
--- /dev/null
+++ b/kamelets/google-storage-cdc-source.kamelet.yaml
@@ -0,0 +1,123 @@
+# ---------------------------------------------------------------------------
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ---------------------------------------------------------------------------
+apiVersion: camel.apache.org/v1alpha1
+kind: Kamelet
+metadata:
+ name: google-storage-cdc-source
+ annotations:
+ camel.apache.org/kamelet.support.level: "Stable"
+ camel.apache.org/catalog.version: "main-SNAPSHOT"
+ camel.apache.org/kamelet.icon: "data:image/svg+xml;base64,PHN2ZyBoZWlnaHQ9IjI1MDAiIHZpZXdCb3g9Ii0xLjYzMzIzNTQzIDcuMDMyNjA5MzMgMTMxLjI2NTc0NjgyIDExNC40MzkzOTA2NyIgd2lkdGg9IjI1MDAiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyIgeG1sbnM6eGxpbms9Imh0dHA6Ly93d3cudzMub3JnLzE5OTkveGxpbmsiPjxsaW5lYXJHcmFkaWVudCBpZD0iYSIgZ3JhZGllbnRVbml0cz0idXNlclNwYWNlT25Vc2UiIHgxPSI2NCIgeDI9IjY0IiB5MT0iNy4wMzQiIHkyPSIxMjAuNzg5Ij48c3RvcCBvZmZzZXQ9IjAiIHN0b3AtY29sb3I9IiM0Mzg3ZmQiLz48c3RvcCBvZmZzZXQ9IjEiIHN0b3 [...]
+ camel.apache.org/provider: "Apache Software Foundation"
+ camel.apache.org/kamelet.group: "Google Storage"
+ labels:
+ camel.apache.org/kamelet.type: "source"
+spec:
+ definition:
+ title: "Google Pubsub Source"
+ description: |-
+ Consume messages from Google Cloud Pub/Sub.
+ required:
+ - projectId
+ - subscriptionName
+ - serviceAccountKey
+ - bucketNameOrArn
+ type: object
+ properties:
+ projectId:
+ title: Project Id
+ description: The Google Cloud Pub/Sub Project ID.
+ type: string
+ subscriptionName:
+ title: Subscription Name
+ description: The subscription name.
+ type: string
+ serviceAccountKey:
+ title: Service Account Key
+ description: The service account key to use as credentials for the Pub/Sub publisher/subscriber. You must encode this value in base64.
+ type: binary
+ x-descriptors:
+ - urn:camel:group:credentials
+ synchronousPull:
+ title: Synchronous Pull
+ description: Specifies to synchronously pull batches of messages.
+ type: boolean
+ x-descriptors:
+ - 'urn:alm:descriptor:com.tectonic.ui:checkbox'
+ default: false
+ maxMessagesPerPoll:
+ title: Max Messages Per Poll
+ description: The maximum number of messages to receive from the server in a single API call.
+ type: integer
+ default: 1
+ concurrentConsumers:
+ title: Concurrent Consumers
+ description: The number of parallel streams to consume from the subscription.
+ type: integer
+ default: 1
+ bucketNameOrArn:
+ title: Bucket Name Or ARN
+ description: The Google Cloud Storage bucket name or Bucket Amazon Resource Name (ARN).
+ type: string
+ getObject:
+ title: Get Object in Bucket
+ description: >-
+ If getObject is enabled, then the file created in the Bucket will be
+ get and returned as body, if not only the event will be returned as body.
+ type: boolean
+ x-descriptors:
+ - 'urn:alm:descriptor:com.tectonic.ui:checkbox'
+ default: false
+ dependencies:
+ - "camel:kamelet"
+ - "camel:google-pubsub"
+ - "camel:google-storage"
+ - "camel:jackson"
+ template:
+ from:
+ uri: "google-pubsub://{{projectId}}:{{subscriptionName}}"
+ parameters:
+ serviceAccountKey: "base64:{{serviceAccountKey}}"
+ synchronousPull: "{{synchronousPull}}"
+ maxMessagesPerPoll: "{{maxMessagesPerPoll}}"
+ concurrentConsumers: "{{concurrentConsumers}}"
+ steps:
+ - choice:
+ precondition: true
+ when:
+ - simple: '${properties:getObject:true}'
+ steps:
+ - set-header:
+ name: attributesStorage
+ simple: ${headers.CamelGooglePubsub.Attributes}
+ - log: "${headers.attributesStorage}"
+ - set-property:
+ name: google-storage-event-type
+ simple: "${headers.attributesStorage[eventType]}"
+ - set-property:
+ name: google-storage-object-id
+ simple: "${headers.attributesStorage[objectId]}"
+ - set-property:
+ name: google-storage-bucket-id
+ simple: "${headers.attributesStorage[bucketId]}"
+ - choice:
+ when:
+ - simple: '${exchangeProperty.google-storage-event-type} == "OBJECT_FINALIZE" && ${exchangeProperty.google-storage-bucket-id} == "{{bucketNameOrArn}}"'
+ steps:
+ - toD: >-
+ google-storage:{{bucketNameOrArn}}?serviceAccountKey=base64:{{serviceAccountKey}}&operation=getObject&objectName=${exchangeProperty.google-storage-object-id}
+ - to: 'kamelet:sink'
[camel-kamelets] 08/11: Add a Google Cloud Storage CDC Source Kamelet
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel-kamelets.git
commit 788f58cd181b708ff7c4dabacd7e2c226b6b9574
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Wed Nov 30 11:31:23 2022 +0100
Add a Google Cloud Storage CDC Source Kamelet
Signed-off-by: Andrea Cosentino <an...@gmail.com>
---
.../src/main/resources/kamelets/google-storage-cdc-source.kamelet.yaml | 1 +
1 file changed, 1 insertion(+)
diff --git a/library/camel-kamelets/src/main/resources/kamelets/google-storage-cdc-source.kamelet.yaml b/library/camel-kamelets/src/main/resources/kamelets/google-storage-cdc-source.kamelet.yaml
index aa1bd513..f55543ea 100644
--- a/library/camel-kamelets/src/main/resources/kamelets/google-storage-cdc-source.kamelet.yaml
+++ b/library/camel-kamelets/src/main/resources/kamelets/google-storage-cdc-source.kamelet.yaml
@@ -91,6 +91,7 @@ spec:
- "camel:google-pubsub"
- "camel:google-storage"
- "camel:jackson"
+ - "camel:core"
template:
from:
uri: "google-pubsub://{{projectId}}:{{subscriptionName}}"
[camel-kamelets] 02/11: Add a Google Cloud Storage CDC Source Kamelet
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel-kamelets.git
commit 0e5714473397efe66d6c42e295d8bbb8cfc2ea71
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Wed Nov 30 11:19:37 2022 +0100
Add a Google Cloud Storage CDC Source Kamelet
Signed-off-by: Andrea Cosentino <an...@gmail.com>
---
.../google-storage-cdc-source.kamelet.yaml | 123 +++++++++++++++++++++
1 file changed, 123 insertions(+)
diff --git a/library/camel-kamelets/src/main/resources/kamelets/google-storage-cdc-source.kamelet.yaml b/library/camel-kamelets/src/main/resources/kamelets/google-storage-cdc-source.kamelet.yaml
new file mode 100644
index 00000000..1e7e48ab
--- /dev/null
+++ b/library/camel-kamelets/src/main/resources/kamelets/google-storage-cdc-source.kamelet.yaml
@@ -0,0 +1,123 @@
+# ---------------------------------------------------------------------------
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ---------------------------------------------------------------------------
+apiVersion: camel.apache.org/v1alpha1
+kind: Kamelet
+metadata:
+ name: google-storage-cdc-source
+ annotations:
+ camel.apache.org/kamelet.support.level: "Stable"
+ camel.apache.org/catalog.version: "main-SNAPSHOT"
+ camel.apache.org/kamelet.icon: "data:image/svg+xml;base64,PHN2ZyBoZWlnaHQ9IjI1MDAiIHZpZXdCb3g9Ii0xLjYzMzIzNTQzIDcuMDMyNjA5MzMgMTMxLjI2NTc0NjgyIDExNC40MzkzOTA2NyIgd2lkdGg9IjI1MDAiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyIgeG1sbnM6eGxpbms9Imh0dHA6Ly93d3cudzMub3JnLzE5OTkveGxpbmsiPjxsaW5lYXJHcmFkaWVudCBpZD0iYSIgZ3JhZGllbnRVbml0cz0idXNlclNwYWNlT25Vc2UiIHgxPSI2NCIgeDI9IjY0IiB5MT0iNy4wMzQiIHkyPSIxMjAuNzg5Ij48c3RvcCBvZmZzZXQ9IjAiIHN0b3AtY29sb3I9IiM0Mzg3ZmQiLz48c3RvcCBvZmZzZXQ9IjEiIHN0b3 [...]
+ camel.apache.org/provider: "Apache Software Foundation"
+ camel.apache.org/kamelet.group: "Google Storage"
+ labels:
+ camel.apache.org/kamelet.type: "source"
+spec:
+ definition:
+ title: "Google Pubsub Source"
+ description: |-
+ Consume messages from Google Cloud Pub/Sub.
+ required:
+ - projectId
+ - subscriptionName
+ - serviceAccountKey
+ - bucketNameOrArn
+ type: object
+ properties:
+ projectId:
+ title: Project Id
+ description: The Google Cloud Pub/Sub Project ID.
+ type: string
+ subscriptionName:
+ title: Subscription Name
+ description: The subscription name.
+ type: string
+ serviceAccountKey:
+ title: Service Account Key
+ description: The service account key to use as credentials for the Pub/Sub publisher/subscriber. You must encode this value in base64.
+ type: binary
+ x-descriptors:
+ - urn:camel:group:credentials
+ synchronousPull:
+ title: Synchronous Pull
+ description: Specifies to synchronously pull batches of messages.
+ type: boolean
+ x-descriptors:
+ - 'urn:alm:descriptor:com.tectonic.ui:checkbox'
+ default: false
+ maxMessagesPerPoll:
+ title: Max Messages Per Poll
+ description: The maximum number of messages to receive from the server in a single API call.
+ type: integer
+ default: 1
+ concurrentConsumers:
+ title: Concurrent Consumers
+ description: The number of parallel streams to consume from the subscription.
+ type: integer
+ default: 1
+ bucketNameOrArn:
+ title: Bucket Name Or ARN
+ description: The Google Cloud Storage bucket name or Bucket Amazon Resource Name (ARN).
+ type: string
+ getObject:
+ title: Get Object in Bucket
+ description: >-
+ If getObject is enabled, then the file created in the Bucket will be
+ get and returned as body, if not only the event will be returned as body.
+ type: boolean
+ x-descriptors:
+ - 'urn:alm:descriptor:com.tectonic.ui:checkbox'
+ default: false
+ dependencies:
+ - "camel:kamelet"
+ - "camel:google-pubsub"
+ - "camel:google-storage"
+ - "camel:jackson"
+ template:
+ from:
+ uri: "google-pubsub://{{projectId}}:{{subscriptionName}}"
+ parameters:
+ serviceAccountKey: "base64:{{serviceAccountKey}}"
+ synchronousPull: "{{synchronousPull}}"
+ maxMessagesPerPoll: "{{maxMessagesPerPoll}}"
+ concurrentConsumers: "{{concurrentConsumers}}"
+ steps:
+ - choice:
+ precondition: true
+ when:
+ - simple: '${properties:getObject:true}'
+ steps:
+ - set-header:
+ name: attributesStorage
+ simple: ${headers.CamelGooglePubsub.Attributes}
+ - log: "${headers.attributesStorage}"
+ - set-property:
+ name: google-storage-event-type
+ simple: "${headers.attributesStorage[eventType]}"
+ - set-property:
+ name: google-storage-object-id
+ simple: "${headers.attributesStorage[objectId]}"
+ - set-property:
+ name: google-storage-bucket-id
+ simple: "${headers.attributesStorage[bucketId]}"
+ - choice:
+ when:
+ - simple: '${exchangeProperty.google-storage-event-type} == "OBJECT_FINALIZE" && ${exchangeProperty.google-storage-bucket-id} == "{{bucketNameOrArn}}"'
+ steps:
+ - toD: >-
+ google-storage:{{bucketNameOrArn}}?serviceAccountKey=base64:{{serviceAccountKey}}&operation=getObject&objectName=${exchangeProperty.google-storage-object-id}
+ - to: 'kamelet:sink'
[camel-kamelets] 06/11: Add a Google Cloud Storage CDC Source Kamelet - Docs
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel-kamelets.git
commit ca275e5ff44882c8131d9acf3d9163aea4b36194
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Wed Nov 30 11:29:36 2022 +0100
Add a Google Cloud Storage CDC Source Kamelet - Docs
Signed-off-by: Andrea Cosentino <an...@gmail.com>
---
docs/modules/ROOT/nav.adoc | 206 +++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 206 insertions(+)
diff --git a/docs/modules/ROOT/nav.adoc b/docs/modules/ROOT/nav.adoc
index f6db3169..8c2f44a5 100644
--- a/docs/modules/ROOT/nav.adoc
+++ b/docs/modules/ROOT/nav.adoc
@@ -1,2 +1,208 @@
// THIS FILE IS AUTOMATICALLY GENERATED: DO NOT EDIT
+* xref:avro-deserialize-action.adoc[]
+* xref:avro-serialize-action.adoc[]
+* xref:aws-cloudtrail-source.adoc[]
+* xref:aws-cloudwatch-sink.adoc[]
+* xref:aws-ddb-sink.adoc[]
+* xref:aws-ddb-streams-source.adoc[]
+* xref:aws-ec2-sink.adoc[]
+* xref:aws-eventbridge-sink.adoc[]
+* xref:aws-kinesis-firehose-sink.adoc[]
+* xref:aws-kinesis-sink.adoc[]
+* xref:aws-kinesis-source.adoc[]
+* xref:aws-lambda-sink.adoc[]
+* xref:aws-redshift-sink.adoc[]
+* xref:aws-redshift-source.adoc[]
+* xref:aws-s3-cdc-source.adoc[]
+* xref:aws-s3-sink.adoc[]
+* xref:aws-s3-source.adoc[]
+* xref:aws-s3-streaming-upload-sink.adoc[]
+* xref:aws-secrets-manager-sink.adoc[]
+* xref:aws-ses-sink.adoc[]
+* xref:aws-sns-fifo-sink.adoc[]
+* xref:aws-sns-sink.adoc[]
+* xref:aws-sqs-batch-sink.adoc[]
+* xref:aws-sqs-fifo-sink.adoc[]
+* xref:aws-sqs-sink.adoc[]
+* xref:aws-sqs-source.adoc[]
+* xref:aws-translate-action.adoc[]
+* xref:azure-cosmosdb-source.adoc[]
+* xref:azure-eventhubs-sink.adoc[]
+* xref:azure-eventhubs-source.adoc[]
+* xref:azure-functions-sink.adoc[]
+* xref:azure-servicebus-sink.adoc[]
+* xref:azure-servicebus-source.adoc[]
+* xref:azure-storage-blob-cdc-source.adoc[]
+* xref:azure-storage-blob-changefeed-source.adoc[]
+* xref:azure-storage-blob-sink.adoc[]
+* xref:azure-storage-blob-source.adoc[]
+* xref:azure-storage-queue-sink.adoc[]
+* xref:azure-storage-queue-source.adoc[]
+* xref:beer-source.adoc[]
+* xref:bitcoin-source.adoc[]
+* xref:caffeine-action.adoc[]
+* xref:cassandra-sink.adoc[]
+* xref:cassandra-source.adoc[]
+* xref:ceph-sink.adoc[]
+* xref:ceph-source.adoc[]
+* xref:chuck-norris-source.adoc[]
+* xref:chunk-template-action.adoc[]
+* xref:couchbase-sink.adoc[]
+* xref:cron-source.adoc[]
+* xref:delay-action.adoc[]
+* xref:dns-dig-action.adoc[]
+* xref:dns-ip-action.adoc[]
+* xref:dns-lookup-action.adoc[]
+* xref:drop-header-action.adoc[]
+* xref:drop-headers-action.adoc[]
+* xref:dropbox-sink.adoc[]
+* xref:dropbox-source.adoc[]
+* xref:earthquake-source.adoc[]
+* xref:elasticsearch-index-sink.adoc[]
+* xref:elasticsearch-search-source.adoc[]
+* xref:exec-sink.adoc[]
+* xref:extract-field-action.adoc[]
+* xref:fhir-source.adoc[]
+* xref:file-watch-source.adoc[]
+* xref:freemarker-template-action.adoc[]
+* xref:ftp-sink.adoc[]
+* xref:ftp-source.adoc[]
+* xref:ftps-sink.adoc[]
+* xref:ftps-source.adoc[]
+* xref:github-commit-source.adoc[]
+* xref:github-event-source.adoc[]
+* xref:github-pullrequest-comment-source.adoc[]
+* xref:github-pullrequest-source.adoc[]
+* xref:github-tag-source.adoc[]
+* xref:google-bigquery-sink.adoc[]
+* xref:google-calendar-source.adoc[]
+* xref:google-functions-sink.adoc[]
+* xref:google-mail-source.adoc[]
+* xref:google-pubsub-sink.adoc[]
+* xref:google-pubsub-source.adoc[]
+* xref:google-sheets-source.adoc[]
+* xref:google-storage-cdc-source.adoc[]
+* xref:google-storage-sink.adoc[]
+* xref:google-storage-source.adoc[]
+* xref:has-header-filter-action.adoc[]
+* xref:header-matches-filter-action.adoc[]
+* xref:hoist-field-action.adoc[]
+* xref:http-secured-sink.adoc[]
+* xref:http-secured-source.adoc[]
+* xref:http-sink.adoc[]
+* xref:http-source.adoc[]
+* xref:infinispan-sink.adoc[]
+* xref:infinispan-source.adoc[]
+* xref:insert-field-action.adoc[]
+* xref:insert-header-action.adoc[]
+* xref:is-tombstone-filter-action.adoc[]
+* xref:jira-add-comment-sink.adoc[]
+* xref:jira-add-issue-sink.adoc[]
+* xref:jira-oauth-source.adoc[]
+* xref:jira-source.adoc[]
+* xref:jira-transition-issue-sink.adoc[]
+* xref:jira-update-issue-sink.adoc[]
+* xref:jms-amqp-10-sink.adoc[]
+* xref:jms-amqp-10-source.adoc[]
+* xref:jms-apache-activemq-sink.adoc[]
+* xref:jms-apache-activemq-source.adoc[]
+* xref:jms-apache-artemis-sink.adoc[]
+* xref:jms-apache-artemis-source.adoc[]
+* xref:jms-ibm-mq-sink.adoc[]
+* xref:jms-ibm-mq-source.adoc[]
+* xref:jolt-transformation-action.adoc[]
+* xref:jslt-action.adoc[]
+* xref:json-deserialize-action.adoc[]
+* xref:json-patch-action.adoc[]
+* xref:json-schema-validator-action.adoc[]
+* xref:json-serialize-action.adoc[]
+* xref:jsonata-action.adoc[]
+* xref:kafka-manual-commit-action.adoc[]
+* xref:kafka-not-secured-sink.adoc[]
+* xref:kafka-not-secured-source.adoc[]
+* xref:kafka-sink.adoc[]
+* xref:kafka-source.adoc[]
+* xref:kafka-ssl-sink.adoc[]
+* xref:kafka-ssl-source.adoc[]
+* xref:kubernetes-namespaces-source.adoc[]
+* xref:kubernetes-nodes-source.adoc[]
+* xref:kubernetes-pods-source.adoc[]
+* xref:log-action.adoc[]
+* xref:log-sink.adoc[]
+* xref:mail-imap-source.adoc[]
+* xref:mail-sink.adoc[]
+* xref:mariadb-sink.adoc[]
+* xref:mariadb-source.adoc[]
+* xref:mask-field-action.adoc[]
+* xref:message-timestamp-router-action.adoc[]
+* xref:minio-sink.adoc[]
+* xref:minio-source.adoc[]
+* xref:mongodb-changes-stream-source.adoc[]
+* xref:mongodb-sink.adoc[]
+* xref:mongodb-source.adoc[]
+* xref:mqtt-sink.adoc[]
+* xref:mqtt-source.adoc[]
+* xref:mqtt5-sink.adoc[]
+* xref:mqtt5-source.adoc[]
+* xref:mustache-template-action.adoc[]
+* xref:mvel-template-action.adoc[]
+* xref:mysql-sink.adoc[]
+* xref:mysql-source.adoc[]
+* xref:nats-sink.adoc[]
+* xref:nats-source.adoc[]
+* xref:nominatim-geocode-action.adoc[]
+* xref:ogcapi-features-action.adoc[]
+* xref:openai-classification-action.adoc[]
+* xref:openai-completion-action.adoc[]
+* xref:oracle-database-sink.adoc[]
+* xref:oracle-database-source.adoc[]
+* xref:pdf-action.adoc[]
+* xref:postgresql-sink.adoc[]
+* xref:postgresql-source.adoc[]
+* xref:predicate-filter-action.adoc[]
+* xref:protobuf-deserialize-action.adoc[]
+* xref:protobuf-serialize-action.adoc[]
+* xref:pulsar-sink.adoc[]
+* xref:pulsar-source.adoc[]
+* xref:rabbitmq-source.adoc[]
+* xref:redis-sink.adoc[]
+* xref:redis-source.adoc[]
+* xref:regex-router-action.adoc[]
+* xref:replace-field-action.adoc[]
+* xref:rest-openapi-sink.adoc[]
+* xref:salesforce-create-sink.adoc[]
+* xref:salesforce-delete-sink.adoc[]
+* xref:salesforce-source.adoc[]
+* xref:salesforce-update-sink.adoc[]
+* xref:scp-sink.adoc[]
+* xref:sftp-sink.adoc[]
+* xref:sftp-source.adoc[]
+* xref:slack-sink.adoc[]
+* xref:slack-source.adoc[]
+* xref:solr-sink.adoc[]
+* xref:solr-source.adoc[]
+* xref:splunk-hec-sink.adoc[]
+* xref:splunk-sink.adoc[]
+* xref:splunk-source.adoc[]
+* xref:sqlserver-sink.adoc[]
+* xref:sqlserver-source.adoc[]
+* xref:ssh-sink.adoc[]
+* xref:ssh-source.adoc[]
+* xref:string-template-action.adoc[]
+* xref:telegram-sink.adoc[]
+* xref:telegram-source.adoc[]
+* xref:throttle-action.adoc[]
+* xref:timer-source.adoc[]
+* xref:timestamp-router-action.adoc[]
+* xref:topic-name-matches-filter-action.adoc[]
+* xref:twitter-directmessage-source.adoc[]
+* xref:twitter-search-source.adoc[]
+* xref:twitter-timeline-source.adoc[]
+* xref:value-to-key-action.adoc[]
+* xref:velocity-template-action.adoc[]
+* xref:webhook-source.adoc[]
+* xref:websocket-source.adoc[]
+* xref:wttrin-source.adoc[]
+* xref:xj-identity-action.adoc[]
+* xref:xj-template-action.adoc[]
// THIS FILE IS AUTOMATICALLY GENERATED: DO NOT EDIT
[camel-kamelets] 10/11: Add a Google Cloud Storage CDC Source Kamelet
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel-kamelets.git
commit c57fec1fab9195e024643e15f7f0baa9284eef64
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Wed Nov 30 11:39:10 2022 +0100
Add a Google Cloud Storage CDC Source Kamelet
Signed-off-by: Andrea Cosentino <an...@gmail.com>
---
kamelets/google-storage-cdc-source.kamelet.yaml | 2 ++
1 file changed, 2 insertions(+)
diff --git a/kamelets/google-storage-cdc-source.kamelet.yaml b/kamelets/google-storage-cdc-source.kamelet.yaml
index f55543ea..b3c9c4b3 100644
--- a/kamelets/google-storage-cdc-source.kamelet.yaml
+++ b/kamelets/google-storage-cdc-source.kamelet.yaml
@@ -35,6 +35,8 @@ spec:
Service account key is the basic method for authenticating to the Google Pubsub and Google Storage.
To use this Kamelet you'll need to set up Events notification for your Google Storage Bucket.
+
+ You must encode the `serviceAccountKey` property before you pass it as a parameter.
required:
- projectId
- subscriptionName
[camel-kamelets] 07/11: Add a Google Cloud Storage CDC Source Kamelet
Posted by ac...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
acosentino pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel-kamelets.git
commit e69205a614c5cf9f8a5cb083c9ff5267b993fcdd
Author: Andrea Cosentino <an...@gmail.com>
AuthorDate: Wed Nov 30 11:30:24 2022 +0100
Add a Google Cloud Storage CDC Source Kamelet
Signed-off-by: Andrea Cosentino <an...@gmail.com>
---
kamelets/google-storage-cdc-source.kamelet.yaml | 1 +
1 file changed, 1 insertion(+)
diff --git a/kamelets/google-storage-cdc-source.kamelet.yaml b/kamelets/google-storage-cdc-source.kamelet.yaml
index aa1bd513..f55543ea 100644
--- a/kamelets/google-storage-cdc-source.kamelet.yaml
+++ b/kamelets/google-storage-cdc-source.kamelet.yaml
@@ -91,6 +91,7 @@ spec:
- "camel:google-pubsub"
- "camel:google-storage"
- "camel:jackson"
+ - "camel:core"
template:
from:
uri: "google-pubsub://{{projectId}}:{{subscriptionName}}"