You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by GitBox <gi...@apache.org> on 2021/03/30 03:21:46 UTC

[GitHub] [camel-kamelets] claudio4j opened a new pull request #107: AWS S3 Sink Kamelet

claudio4j opened a new pull request #107:
URL: https://github.com/apache/camel-kamelets/pull/107


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [camel-kamelets] claudio4j commented on a change in pull request #107: AWS S3 Sink Kamelet

Posted by GitBox <gi...@apache.org>.
claudio4j commented on a change in pull request #107:
URL: https://github.com/apache/camel-kamelets/pull/107#discussion_r606270149



##########
File path: aws-s3-sink.kamelet.yaml
##########
@@ -0,0 +1,95 @@
+apiVersion: camel.apache.org/v1alpha1
+kind: Kamelet
+metadata:
+  name: aws-s3-sink
+  annotations:
+    camel.apache.org/kamelet.icon: "data:image/svg+xml;base64,PHN2ZyB2ZXJzaW9uPSIxLjEiIGlkPSJMYXllcl8xIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHg9IjAiIHk9IjAiIHZpZXdCb3g9IjAgMCAyNDguMiAzMDAiIHhtbDpzcGFjZT0icHJlc2VydmUiPjxzdHlsZT4uc3QyOHtmaWxsOiM4YzMxMjN9LnN0Mjl7ZmlsbDojZTA1MjQzfTwvc3R5bGU+PHBhdGggY2xhc3M9InN0MjgiIGQ9Ik0yMCA1Mi4xTDAgNjJ2MTc1LjVsMjAgOS45LjEtLjFWNTIuMmwtLjEtLjEiLz48cGF0aCBjbGFzcz0ic3QyOSIgZD0iTTEyNyAyMjJMMjAgMjQ3LjVWNTIuMUwxMjcgNzd2MTQ1Ii8+PHBhdGggY2xhc3M9InN0MjgiIGQ9Ik03OC43IDE4Mi4xbDQ1LjQgNS44LjMtLjcuMy03NC40LS41LS42LTQ1LjQgNS43LS4xIDY0LjIiLz48cGF0aCBjbGFzcz0ic3QyOCIgZD0iTTEyNC4xIDIyMi4zbDEwNC4xIDI1LjIuMi0uM1Y1Mi4xbC0uMi0uMi0xMDQuMSAyNS40djE0NSIvPjxwYXRoIGNsYXNzPSJzdDI5IiBkPSJNMTY5LjUgMTgyLjFsLTQ1LjQgNS44di03NS43bDQ1LjQgNS43djY0LjIiLz48cGF0aCBkPSJNMTY5LjUgODYuOWwtNDUuNCA4LjMtNDUuNC04LjNMMTI0IDc1bDQ1LjUgMTEuOSIgZmlsbD0iIzVlMWYxOCIvPjxwYXRoIGQ9Ik0xNjkuNSAyMTMuMWwtNDUuNC04LjMtNDUuNCA4LjMgNDUuMyAxMi43IDQ1LjUtMTIuNyIgZmlsbD0iI2YyYjBhOSIvPjxwYXRoIGNsYXNzPSJzdDI4IiB
 kPSJNNzguNyA4Ni45bDQ1LjQtMTEuMi40LS4xVi4zbC0uNC0uMy00NS40IDIyLjd2NjQuMiIvPjxwYXRoIGNsYXNzPSJzdDI5IiBkPSJNMTY5LjUgODYuOWwtNDUuNC0xMS4yVjBsNDUuNCAyMi43djY0LjIiLz48cGF0aCBjbGFzcz0ic3QyOCIgZD0iTTEyNC4xIDMwMGwtNDUuNC0yMi43di02NC4ybDQ1LjQgMTEuMi43LjgtLjIgNzMuNi0uNSAxLjMiLz48cGF0aCBjbGFzcz0ic3QyOSIgZD0iTTEyNC4xIDMwMGw0NS40LTIyLjd2LTY0LjJsLTQ1LjQgMTEuMlYzMDBNMjI4LjIgNTIuMWwyMCAxMHYxNzUuNWwtMjAgMTBWNTIuMSIvPjwvc3ZnPg=="
+    camel.apache.org/provider: "Apache Software Foundation"
+  labels:
+    camel.apache.org/kamelet.type: "sink"
+spec:
+  definition:
+    title: "AWS S3 Sink"
+    description: |-
+      Upload data to AWS S3.
+
+      The Kamelet expects the following headers to be set:
+
+      - `file` / `ce-file`: as the file name to upload
+
+      If the header won't be set the exchange ID will be used as file name.
+    required:
+      - bucketNameOrArn
+      - accessKey
+      - secretKey
+      - region
+    properties:
+      bucketNameOrArn:
+        title: Bucket Name
+        description: The S3 Bucket name or ARN.
+        type: string
+      accessKey:
+        title: Access Key
+        description: The access key obtained from AWS.
+        type: string
+        x-descriptors:
+        - urn:alm:descriptor:com.tectonic.ui:password
+      secretKey:
+        title: Secret Key
+        description: The secret key obtained from AWS.
+        type: string
+        x-descriptors:
+        - urn:alm:descriptor:com.tectonic.ui:password
+      region:
+        title: AWS Region
+        description: The AWS region to connect to.
+        type: string
+        example: eu-west-1
+      overrideEndpoint:
+        title: Override Endpoint
+        description: Set the need for overidding the endpoint. This option needs to be used in combination with uriEndpointOverride.
+        type: boolean
+        default: false
+      uriEndpointOverride:
+        title: Override Endpoint URI
+        description: Set the overriding uri endpoint. This option needs to be used in combination with overrideEndpoint option.
+        type: string
+        example: "http://another-s3-endpoint:9000"
+      autoCreateBucket:
+        title: Autocreate bucket
+        description: Setting the autocreation of the S3 bucket bucketName.
+        type: boolean
+        x-descriptors:
+        - 'urn:alm:descriptor:com.tectonic.ui:checkbox'
+        default: false
+  flow:
+    from:
+      uri: "kamelet:source"
+      steps:
+      - choice:
+          when:
+          - simple: "${header[file]}"
+            steps:
+            - set-header:
+                name: CamelAwsS3Key
+                simple: "${header[file]}"
+          - simple: "${header[ce-file]}"
+            steps:
+            - set-header:
+                name: CamelAwsS3Key
+                simple: "${header[ce-file]}"
+          otherwise:
+            steps:
+            - set-header:
+                name: CamelAwsS3Key
+                simple: "${exchangeId}"
+      - log: "s3 body - ${body}"

Review comment:
       Ahh, just forgot to remove it before pushing.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [camel-kamelets] claudio4j commented on pull request #107: AWS S3 Sink Kamelet

Posted by GitBox <gi...@apache.org>.
claudio4j commented on pull request #107:
URL: https://github.com/apache/camel-kamelets/pull/107#issuecomment-810206341


   In camel-dropbox if there is no header to designate the filename, it uses the exchange id, but in camel-aws2-s3 there is no such default for filename. So, in this case I think requiring the filename is appropriate isn't ? Or should we set the exchange id if there is no CamelAwsS3Key header ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [camel-kamelets] oscerd commented on a change in pull request #107: AWS S3 Sink Kamelet

Posted by GitBox <gi...@apache.org>.
oscerd commented on a change in pull request #107:
URL: https://github.com/apache/camel-kamelets/pull/107#discussion_r606068266



##########
File path: aws-s3-sink.kamelet.yaml
##########
@@ -0,0 +1,95 @@
+apiVersion: camel.apache.org/v1alpha1
+kind: Kamelet
+metadata:
+  name: aws-s3-sink
+  annotations:
+    camel.apache.org/kamelet.icon: "data:image/svg+xml;base64,PHN2ZyB2ZXJzaW9uPSIxLjEiIGlkPSJMYXllcl8xIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHg9IjAiIHk9IjAiIHZpZXdCb3g9IjAgMCAyNDguMiAzMDAiIHhtbDpzcGFjZT0icHJlc2VydmUiPjxzdHlsZT4uc3QyOHtmaWxsOiM4YzMxMjN9LnN0Mjl7ZmlsbDojZTA1MjQzfTwvc3R5bGU+PHBhdGggY2xhc3M9InN0MjgiIGQ9Ik0yMCA1Mi4xTDAgNjJ2MTc1LjVsMjAgOS45LjEtLjFWNTIuMmwtLjEtLjEiLz48cGF0aCBjbGFzcz0ic3QyOSIgZD0iTTEyNyAyMjJMMjAgMjQ3LjVWNTIuMUwxMjcgNzd2MTQ1Ii8+PHBhdGggY2xhc3M9InN0MjgiIGQ9Ik03OC43IDE4Mi4xbDQ1LjQgNS44LjMtLjcuMy03NC40LS41LS42LTQ1LjQgNS43LS4xIDY0LjIiLz48cGF0aCBjbGFzcz0ic3QyOCIgZD0iTTEyNC4xIDIyMi4zbDEwNC4xIDI1LjIuMi0uM1Y1Mi4xbC0uMi0uMi0xMDQuMSAyNS40djE0NSIvPjxwYXRoIGNsYXNzPSJzdDI5IiBkPSJNMTY5LjUgMTgyLjFsLTQ1LjQgNS44di03NS43bDQ1LjQgNS43djY0LjIiLz48cGF0aCBkPSJNMTY5LjUgODYuOWwtNDUuNCA4LjMtNDUuNC04LjNMMTI0IDc1bDQ1LjUgMTEuOSIgZmlsbD0iIzVlMWYxOCIvPjxwYXRoIGQ9Ik0xNjkuNSAyMTMuMWwtNDUuNC04LjMtNDUuNCA4LjMgNDUuMyAxMi43IDQ1LjUtMTIuNyIgZmlsbD0iI2YyYjBhOSIvPjxwYXRoIGNsYXNzPSJzdDI4IiB
 kPSJNNzguNyA4Ni45bDQ1LjQtMTEuMi40LS4xVi4zbC0uNC0uMy00NS40IDIyLjd2NjQuMiIvPjxwYXRoIGNsYXNzPSJzdDI5IiBkPSJNMTY5LjUgODYuOWwtNDUuNC0xMS4yVjBsNDUuNCAyMi43djY0LjIiLz48cGF0aCBjbGFzcz0ic3QyOCIgZD0iTTEyNC4xIDMwMGwtNDUuNC0yMi43di02NC4ybDQ1LjQgMTEuMi43LjgtLjIgNzMuNi0uNSAxLjMiLz48cGF0aCBjbGFzcz0ic3QyOSIgZD0iTTEyNC4xIDMwMGw0NS40LTIyLjd2LTY0LjJsLTQ1LjQgMTEuMlYzMDBNMjI4LjIgNTIuMWwyMCAxMHYxNzUuNWwtMjAgMTBWNTIuMSIvPjwvc3ZnPg=="
+    camel.apache.org/provider: "Apache Software Foundation"
+  labels:
+    camel.apache.org/kamelet.type: "sink"
+spec:
+  definition:
+    title: "AWS S3 Sink"
+    description: |-
+      Upload data to AWS S3.
+
+      The Kamelet expects the following headers to be set:
+
+      - `file` / `ce-file`: as the file name to upload
+
+      If the header won't be set the exchange ID will be used as file name.
+    required:
+      - bucketNameOrArn
+      - accessKey
+      - secretKey
+      - region
+    properties:
+      bucketNameOrArn:
+        title: Bucket Name
+        description: The S3 Bucket name or ARN.
+        type: string
+      accessKey:
+        title: Access Key
+        description: The access key obtained from AWS.
+        type: string
+        x-descriptors:
+        - urn:alm:descriptor:com.tectonic.ui:password
+      secretKey:
+        title: Secret Key
+        description: The secret key obtained from AWS.
+        type: string
+        x-descriptors:
+        - urn:alm:descriptor:com.tectonic.ui:password
+      region:
+        title: AWS Region
+        description: The AWS region to connect to.
+        type: string
+        example: eu-west-1
+      overrideEndpoint:
+        title: Override Endpoint
+        description: Set the need for overidding the endpoint. This option needs to be used in combination with uriEndpointOverride.
+        type: boolean
+        default: false
+      uriEndpointOverride:
+        title: Override Endpoint URI
+        description: Set the overriding uri endpoint. This option needs to be used in combination with overrideEndpoint option.
+        type: string
+        example: "http://another-s3-endpoint:9000"
+      autoCreateBucket:
+        title: Autocreate bucket
+        description: Setting the autocreation of the S3 bucket bucketName.
+        type: boolean
+        x-descriptors:
+        - 'urn:alm:descriptor:com.tectonic.ui:checkbox'
+        default: false
+  flow:
+    from:
+      uri: "kamelet:source"
+      steps:
+      - choice:
+          when:
+          - simple: "${header[file]}"
+            steps:
+            - set-header:
+                name: CamelAwsS3Key
+                simple: "${header[file]}"
+          - simple: "${header[ce-file]}"
+            steps:
+            - set-header:
+                name: CamelAwsS3Key
+                simple: "${header[ce-file]}"
+          otherwise:
+            steps:
+            - set-header:
+                name: CamelAwsS3Key
+                simple: "${exchangeId}"
+      - log: "s3 body - ${body}"

Review comment:
       I don't think this is needed, eventually a user can add a step with a log sink in the middle of a binding.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [camel-kamelets] oscerd commented on pull request #107: AWS S3 Sink Kamelet

Posted by GitBox <gi...@apache.org>.
oscerd commented on pull request #107:
URL: https://github.com/apache/camel-kamelets/pull/107#issuecomment-810210394


   > In camel-dropbox if there is no header to designate the filename, it uses the exchange id, but in camel-aws2-s3 there is no such default for filename. So, in this case I think requiring the filename is appropriate isn't ? Or should we set the exchange id if there is no CamelAwsS3Key header ?
   
   I meant to say the same style as dropbox, by adding an otherwise step to set the header as exchange id. Like in the minio sink connector for example: https://github.com/apache/camel-kamelets/blob/master/minio-sink.kamelet.yaml#L60


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [camel-kamelets] nicolaferraro merged pull request #107: AWS S3 Sink Kamelet

Posted by GitBox <gi...@apache.org>.
nicolaferraro merged pull request #107:
URL: https://github.com/apache/camel-kamelets/pull/107


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org