You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by GitBox <gi...@apache.org> on 2022/04/18 11:44:39 UTC

[GitHub] [camel-kafka-connector] anuragp89 opened a new issue, #1357: camel kafka connector - s3 Sink

anuragp89 opened a new issue, #1357:
URL: https://github.com/apache/camel-kafka-connector/issues/1357

   Hi Team,
   
   I have setup camel kafka s3 sink connector 1.0.0 build and it's working fine. But I am not able to find any configuration to define the names of the file to be uploaded to S3 bucket. Below is my connector configuration:
   
   apiVersion: kafka.strimzi.io/v1beta2
   kind: KafkaConnector
   metadata:
     name: camel-aws-s3-sink-connector
     labels:
       strimzi.io/cluster: my-connect-cluster
   spec:
     class: org.apache.camel.kafkaconnector.awss3sink.CamelAwss3sinkSinkConnector
     tasksMax: 1
     config:
       topics: iot    
       camel.kamelet.aws-s3-sink.accessKey: **********
       camel.kamelet.aws-s3-sink.bucketNameOrArn: camel-kafkaconnect-s3
       camel.kamelet.aws-s3-sink.region: us-west-2
       camel.kamelet.aws-s3-sink.secretKey: ***********
       key.converter: org.apache.kafka.connect.storage.StringConverter
       value.converter: org.apache.kafka.connect.storage.StringConverter
       errors.log.enable: true
       errors.log.include.messages: true
   
   
   I can see that in 0.11 camel-aws2-s3-kafka-connector build , there is a configuration called "camel.sink.endpoint.keyName" which can be used to define the custom names of the files being uploaded to S3.  Is the custom file name feature not available in 1.0 build or I am missing something. Please suggest.
   
   Thanks,
   Anurag


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [camel-kafka-connector] oscerd commented on issue #1357: camel kafka connector - s3 Sink

Posted by GitBox <gi...@apache.org>.
oscerd commented on issue #1357:
URL: https://github.com/apache/camel-kafka-connector/issues/1357#issuecomment-1102088669

   If you specify something like /aaa/bbb/ccc/filename.txt as the CamelAwsS3Key, the file will be uploaded in the specific folder.
   
   What you get as camel-kafka-connector on 1.0.x, is listed here as kamelets: https://github.com/apache/camel-kamelets/tree/main/kamelets
   
   Sink and source kamelets are the building blocks for camel kafka connector. If you want to see something supported, you could open an issues at camel-kamelets.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [camel-kafka-connector] anuragp89 commented on issue #1357: camel kafka connector - s3 Sink

Posted by GitBox <gi...@apache.org>.
anuragp89 commented on issue #1357:
URL: https://github.com/apache/camel-kafka-connector/issues/1357#issuecomment-1101967705

   Hi Andrea,
   
   Thanks for the response. In addition to file name, if the connector configuration could add flexibility to upload files in a specific folder in the bucket (similar to prefix configuration in Source connector), that would be really helpful. I believe I should go with using 0.11 aws s3 sink connector build for now. 
   Just out of curiosity, there are feature rich connectors and extended configuration options in 0.11x release, will they be eventually migrated to 1.x.
   
   Thanks,
   Anurag
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [camel-kafka-connector] oscerd commented on issue #1357: camel kafka connector - s3 Sink

Posted by GitBox <gi...@apache.org>.
oscerd commented on issue #1357:
URL: https://github.com/apache/camel-kafka-connector/issues/1357#issuecomment-1101358366

   You need to pass an header in your kafka record named "CamelHeader.CamelAwsS3Key" with the file name to be used. The keyname field has been added to the Kamelet used in 1.0.x release, but it's not released yet in camel kafka connector, so you'll have to wait.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [camel-kafka-connector] oscerd closed issue #1357: camel kafka connector - s3 Sink

Posted by GitBox <gi...@apache.org>.
oscerd closed issue #1357: camel kafka connector - s3 Sink
URL: https://github.com/apache/camel-kafka-connector/issues/1357


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [camel-kafka-connector] anuragp89 commented on issue #1357: camel kafka connector - s3 Sink

Posted by GitBox <gi...@apache.org>.
anuragp89 commented on issue #1357:
URL: https://github.com/apache/camel-kafka-connector/issues/1357#issuecomment-1102121520

   Thanks Andrea, this is very helpful.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org