You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by GitBox <gi...@apache.org> on 2020/07/29 08:01:12 UTC
[GitHub] [camel-kafka-connector] rbaumgar opened a new issue #338: aws2S3connector connected to minio ClassCastException
rbaumgar opened a new issue #338:
URL: https://github.com/apache/camel-kafka-connector/issues/338
try to use CamelAws2s3SourceConnector to connect to another S3 provider (minio)
when I tyr to upload a simple text file to my s2 bucket -> error
KafkaConnector
============
apiVersion: kafka.strimzi.io/v1alpha1
kind: KafkaConnector
metadata:
name: aws2-s3-source-connector
labels:
strimzi.io/cluster: my-connect-cluster
spec:
class: org.apache.camel.kafkaconnector.aws2s3.CamelAws2s3SourceConnector
config:
camel.component.aws2-s3.configuration.access-key: minio
camel.component.aws2-s3.configuration.secret-key: minio123
camel.component.aws2-s3.configuration.region: EU_WEST_1
camel.source.endpoint.autocloseBody: false
camel.source.endpoint.overrideEndpoint: true
camel.source.endpoint.uriEndpointOverride: http://minio-server
camel.source.maxPollDuration: 10000
camel.source.path.bucketNameOrArn: camel-connector-test
key.converter: org.apache.kafka.connect.storage.StringConverter
topics: s3-topic
value.converter: org.apache.camel.kafkaconnector.aws2s3.converters.S3ObjectConverter
tasksMax: 1
Log
===
2020-07-28 12:35:44,903 INFO Setting initial properties in Camel context: [{camel.source.endpoint.autocloseBody=false, connector.class=org.apache.camel.kafkaconnector.aws2s3.CamelAws2s3SourceConnector, camel.source.maxPollDuration=5000, camel.source.endpoint.runLoggingLevel=TRACE, camel.component.aws2-s3.configuration.region=EU_WEST_1, topics=s3-topic, tasks.max=1, camel.source.endpoint.overrideEndpoint=true, camel.source.endpoint.uriEndpointOverride=http://minio-server, camel.source.component=aws2-s3, task.class=org.apache.camel.kafkaconnector.aws2s3.CamelAws2s3SourceTask, camel.source.path.bucketNameOrArn=camel-connector-test, camel.component.aws2-s3.configuration.access-key=minio, name=aws2-s3-source-connector, camel.component.aws2-s3.configuration.secret-key=minio123, value.converter=org.apache.camel.kafkaconnector.aws2s3.converters.S3ObjectConverter, key.converter=org.apache.kafka.connect.storage.StringConverter}] (org.apache.camel.kafkaconnector.utils.CamelMainSupport) [task
-thread-aws2-s3-source-connector-0]
2020-07-28 12:35:44,913 INFO Creating Camel route from(aws2-s3://camel-connector-test?autocloseBody=false&overrideEndpoint=true&runLoggingLevel=TRACE&uriEndpointOverride=http://minio-server).to(direct:end?pollingConsumerQueueSize=1000&pollingConsumerBlockTimeout=0&pollingConsumerBlockWhenFull=true) (org.apache.camel.kafkaconnector.utils.CamelMainSupport) [task-thread-aws2-s3-source-connector-0]
2020-07-28 12:35:44,916 INFO [Producer clientId=connector-producer-aws2-s3-source-connector-0] Cluster ID: k_EXjSBzQvmhHkadWFBbcg (org.apache.kafka.clients.Metadata) [kafka-producer-network-thread | connector-producer-aws2-s3-source-connector-0]
2020-07-28 12:35:44,922 INFO Starting CamelContext (org.apache.camel.kafkaconnector.utils.CamelMainSupport) [task-thread-aws2-s3-source-connector-0]
2020-07-28 12:35:44,923 INFO Using properties from: classpath:application.properties;optional=true (org.apache.camel.main.BaseMainSupport) [pool-14-thread-1]
2020-07-28 12:35:44,928 INFO Auto-configuration summary: (org.apache.camel.main.BaseMainSupport) [pool-14-thread-1]
2020-07-28 12:35:44,928 INFO camel.component.aws2-s3.configuration.region=EU_WEST_1 (org.apache.camel.main.BaseMainSupport) [pool-14-thread-1]
2020-07-28 12:35:44,928 INFO camel.component.aws2-s3.configuration.accessKey=minio (org.apache.camel.main.BaseMainSupport) [pool-14-thread-1]
2020-07-28 12:35:44,928 INFO camel.component.aws2-s3.configuration.secretKey=minio123 (org.apache.camel.main.BaseMainSupport) [pool-14-thread-1]
2020-07-28 12:35:44,929 INFO No additional Camel XML routes discovered from: classpath:camel/*.xml (org.apache.camel.main.DefaultRoutesCollector) [pool-14-thread-1]
2020-07-28 12:35:44,929 INFO No additional Camel XML rests discovered from: classpath:camel-rest/*.xml (org.apache.camel.main.DefaultRoutesCollector) [pool-14-thread-1]
2020-07-28 12:35:44,938 INFO Apache Camel 3.3.0 (CamelContext: camel-4) is starting (org.apache.camel.impl.engine.AbstractCamelContext) [pool-14-thread-1]
2020-07-28 12:35:44,938 INFO StreamCaching is not in use. If using streams then its recommended to enable stream caching. See more details at http://camel.apache.org/stream-caching.html (org.apache.camel.impl.engine.AbstractCamelContext) [pool-14-thread-1]
2020-07-28 12:35:44,952 INFO Route: route4 started and consuming from: aws2-s3://camel-connector-test (org.apache.camel.impl.engine.AbstractCamelContext) [pool-14-thread-1]
2020-07-28 12:35:44,952 INFO Total 1 routes, of which 1 are started (org.apache.camel.impl.engine.AbstractCamelContext) [pool-14-thread-1]
2020-07-28 12:35:44,952 INFO Apache Camel 3.3.0 (CamelContext: camel-4) started in 0.014 seconds (org.apache.camel.impl.engine.AbstractCamelContext) [pool-14-thread-1]
2020-07-28 12:35:44,953 INFO CamelContext started (org.apache.camel.kafkaconnector.utils.CamelMainSupport) [task-thread-aws2-s3-source-connector-0]
2020-07-28 12:35:44,953 INFO CamelSourceTask connector task started (org.apache.camel.kafkaconnector.CamelSourceTask) [task-thread-aws2-s3-source-connector-0]
2020-07-28 12:35:44,953 INFO WorkerSourceTask{id=aws2-s3-source-connector-0} Source task finished initialization and start (org.apache.kafka.connect.runtime.WorkerSourceTask) [task-thread-aws2-s3-source-connector-0]
2020-07-28 12:35:45,968 INFO WorkerSourceTask{id=aws2-s3-source-connector-0} Committing offsets (org.apache.kafka.connect.runtime.WorkerSourceTask) [task-thread-aws2-s3-source-connector-0]
2020-07-28 12:35:45,969 INFO WorkerSourceTask{id=aws2-s3-source-connector-0} flushing 0 outstanding messages for offset commit (org.apache.kafka.connect.runtime.WorkerSourceTask) [task-thread-aws2-s3-source-connector-0]
2020-07-28 12:35:45,969 ERROR WorkerSourceTask{id=aws2-s3-source-connector-0} Task threw an uncaught and unrecoverable exception (org.apache.kafka.connect.runtime.WorkerTask) [task-thread-aws2-s3-source-connector-0]
org.apache.kafka.connect.errors.ConnectException: Tolerance exceeded in error handler
at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndHandleError(RetryWithToleranceOperator.java:178)
at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execute(RetryWithToleranceOperator.java:104)
at org.apache.kafka.connect.runtime.WorkerSourceTask.convertTransformedRecord(WorkerSourceTask.java:295)
at org.apache.kafka.connect.runtime.WorkerSourceTask.sendRecords(WorkerSourceTask.java:321)
at org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:245)
at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:184)
at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:234)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to software.amazon.awssdk.core.ResponseInputStream
at org.apache.camel.kafkaconnector.aws2s3.converters.S3ObjectConverter.fromConnectData(S3ObjectConverter.java:37)
at org.apache.kafka.connect.storage.Converter.fromConnectData(Converter.java:63)
at org.apache.kafka.connect.runtime.WorkerSourceTask.lambda$convertTransformedRecord$2(WorkerSourceTask.java:295)
at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndRetry(RetryWithToleranceOperator.java:128)
at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndHandleError(RetryWithToleranceOperator.java:162)
... 11 more
2020-07-28 12:35:45,969 ERROR WorkerSourceTask{id=aws2-s3-source-connector-0} Task is being killed and will not recover until manually restarted (org.apache.kafka.connect.runtime.WorkerTask) [task-thread-aws2-s3-source-connector-0]
2020-07-28 12:35:45,969 INFO Stopping CamelSourceTask connector task (org.apache.camel.kafkaconnector.CamelSourceTask) [task-thread-aws2-s3-source-connector-0]
2020-07-28 12:35:45,969 INFO Stopping CamelContext (org.apache.camel.kafkaconnector.utils.CamelMainSupport) [task-thread-aws2-s3-source-connector-0]
2020-07-28 12:35:45,969 INFO Apache Camel 3.3.0 (CamelContext: camel-4) is shutting down (org.apache.camel.impl.engine.AbstractCamelContext) [task-thread-aws2-s3-source-connector-0]
2020-07-28 12:35:45,969 INFO Starting to graceful shutdown 1 routes (timeout 45 seconds) (org.apache.camel.impl.engine.DefaultShutdownStrategy) [task-thread-aws2-s3-source-connector-0]
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] oscerd commented on issue #338: aws2S3connector connected to minio ClassCastException
Posted by GitBox <gi...@apache.org>.
oscerd commented on issue #338:
URL: https://github.com/apache/camel-kafka-connector/issues/338#issuecomment-665019554
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] rbaumgar commented on issue #338: aws2S3connector connected to minio ClassCastException
Posted by GitBox <gi...@apache.org>.
rbaumgar commented on issue #338:
URL: https://github.com/apache/camel-kafka-connector/issues/338#issuecomment-665023421
Andrea,
thx! works like a charm. I did not found this example...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] rbaumgar closed issue #338: aws2S3connector connected to minio ClassCastException
Posted by GitBox <gi...@apache.org>.
rbaumgar closed issue #338:
URL: https://github.com/apache/camel-kafka-connector/issues/338
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org