You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by GitBox <gi...@apache.org> on 2021/02/03 13:26:56 UTC
[GitHub] [camel-kafka-connector] h4gen opened a new issue #969: Convert NettyChannelBufferStreamCache from NettyHttpSource not converted to string
h4gen opened a new issue #969:
URL: https://github.com/apache/camel-kafka-connector/issues/969
Hi everybody,
This Issue was originally posted by me in [Zulip](https://camel.zulipchat.com/#narrow/stream/257303-camel-kafka-connector/topic/Convert.20NettyChannelBufferStreamCache.20from.20NettyHttpSource.3F/near/225001495)
I am currently trying to accept text or json messages via the CamelNettyhttpSourceConnector component.
I use the following configuration:
````
{
"connector.class": "org.apache.camel.kafkaconnector.nettyhttp.CamelNettyhttpSourceConnector",
"camel.source.endpoint.receiveBufferSize": "10",
"camel.source.path.host": "0.0.0.0",
"camel.source.path.protocol": "http",
"tasks.max": "1",
"topics": "test",
"camel.source.path.port": "8081",
"name": "http",
"value.converter": "org.apache.kafka.connect.storage.StringConverter",
"key.converter": "org.apache.kafka.connect.storage.StringConverter"
}
````
I send messages to the connector with:
````
curl -H "Content-Type: text/plain" -X POST -d '{"username":"xyz","password":"xyz"}' http://localhost:8081
````
or
````
curl -H "Content-Type: application/json" -X POST -d '{"username":"xyz","password":"xyz"}' http://localhost:8081
````
The messages end up in my topic "test" but there is just a bunch of `org.apache.camel.component.netty.http.NettyChannelBufferStreamCache` Objects or sometimes even empty objects.
The error occurs with version `0.7.0` and `0.7.1`. I just downloaded the tar archives from the components page in the respecive versions and extracted them into my kafka connect connector folder (Cloudera CDP 7.1)
Thank you very much.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] valdar commented on issue #969: Convert NettyChannelBufferStreamCache from NettyHttpSource not converted to string
Posted by GitBox <gi...@apache.org>.
valdar commented on issue #969:
URL: https://github.com/apache/camel-kafka-connector/issues/969#issuecomment-796621681
@h4gen this should be fixed now in `master` so will be available starting form version `0.9.0`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] valdar edited a comment on issue #969: Convert NettyChannelBufferStreamCache from NettyHttpSource not converted to string
Posted by GitBox <gi...@apache.org>.
valdar edited a comment on issue #969:
URL: https://github.com/apache/camel-kafka-connector/issues/969#issuecomment-772564703
@h4gen yes my bad, wrong config. Try:
```
transforms: tostring
transforms.tostring.type: org.apache.camel.kafkaconnector.transforms.CamelTypeConverterTransform$Value
transforms.tostring.target.type: java.lang.String
```
note the added `$Value` in the type.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] h4gen commented on issue #969: Convert NettyChannelBufferStreamCache from NettyHttpSource not converted to string
Posted by GitBox <gi...@apache.org>.
h4gen commented on issue #969:
URL: https://github.com/apache/camel-kafka-connector/issues/969#issuecomment-796622702
@valdar Thank you very much for taking care of this! Looking forward to try it out :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] h4gen commented on issue #969: Convert NettyChannelBufferStreamCache from NettyHttpSource not converted to string
Posted by GitBox <gi...@apache.org>.
h4gen commented on issue #969:
URL: https://github.com/apache/camel-kafka-connector/issues/969#issuecomment-772574361
@valdar thanks for the fast reply. Now the configuration was accepted and the connector deployed. However when I send messages to the connector i get the following:
````
org.apache.kafka.connect.errors.ConnectException: Tolerance exceeded in error handler at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndHandleError(RetryWithToleranceOperator.java:178) at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execute(RetryWithToleranceOperator.java:104) at org.apache.kafka.connect.runtime.TransformationChain.apply(TransformationChain.java:50) at org.apache.kafka.connect.runtime.WorkerSourceTask.sendRecords(WorkerSourceTask.java:315) at org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:240) at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:177) at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:227) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoo
lExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.kafka.connect.errors.DataException: CamelTypeConverter was not able to convert value `null` to target type of `String` at org.apache.camel.kafkaconnector.transforms.CamelTypeConverterTransform.convertValueWithCamelTypeConverter(CamelTypeConverterTransform.java:57) at org.apache.camel.kafkaconnector.transforms.CamelTypeConverterTransform.apply(CamelTypeConverterTransform.java:47) at org.apache.kafka.connect.runtime.TransformationChain.lambda$apply$0(TransformationChain.java:50) at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndRetry(RetryWithToleranceOperator.java:128) at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndHandleError(RetryWithToleranceOperator.java:162) ... 11 more
````
Interestingly messages are forwarded to my topic but are just "empty". Does this make sense?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] valdar commented on issue #969: Convert NettyChannelBufferStreamCache from NettyHttpSource not converted to string
Posted by GitBox <gi...@apache.org>.
valdar commented on issue #969:
URL: https://github.com/apache/camel-kafka-connector/issues/969#issuecomment-772564703
@h4gen yes my bad, wrong config. Try:
```
transforms: tostring
transforms.tostring.type: org.apache.camel.kafkaconnector.transforms.CamelTypeConverterTransform$Value
transforms.tostring.target.type: java.lang.String
```
note the added `$Value` in the type.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] valdar commented on issue #969: Convert NettyChannelBufferStreamCache from NettyHttpSource not converted to string
Posted by GitBox <gi...@apache.org>.
valdar commented on issue #969:
URL: https://github.com/apache/camel-kafka-connector/issues/969#issuecomment-772537658
@h4gen can you try to add this to your configuration:
```
transforms: tostring
transforms.tostring.type: org.apache.camel.kafkaconnector.transforms.CamelTypeConverterTransform
transforms.tostring.target.type: java.lang.String
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] oscerd commented on issue #969: Convert NettyChannelBufferStreamCache from NettyHttpSource not converted to string
Posted by GitBox <gi...@apache.org>.
oscerd commented on issue #969:
URL: https://github.com/apache/camel-kafka-connector/issues/969#issuecomment-772506152
Thanks for reporting. We need to provide a converter or smt, with mechanism to convert from stream to string.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] h4gen commented on issue #969: Convert NettyChannelBufferStreamCache from NettyHttpSource not converted to string
Posted by GitBox <gi...@apache.org>.
h4gen commented on issue #969:
URL: https://github.com/apache/camel-kafka-connector/issues/969#issuecomment-772542731
Hi @valdar!
Thanks for the suggestion. I added the snippet to the configuration, leading to the following configuration in total:
````
{
"connector.class": "org.apache.camel.kafkaconnector.nettyhttp.CamelNettyhttpSourceConnector",
"camel.source.endpoint.receiveBufferSize": "10",
"camel.source.path.host": "0.0.0.0",
"camel.source.path.protocol": "http",
"tasks.max": "1",
"topics": "test",
"camel.source.path.port": "8081",
"name": "http",
"value.converter": "org.apache.kafka.connect.storage.StringConverter",
"key.converter": "org.apache.kafka.connect.storage.StringConverter",
"transforms": "tostring",
"transforms.tostring.type": "org.apache.camel.kafkaconnector.transforms.CamelTypeConverterTransform",
"transforms.tostring.target.type": "java.lang.String"
}
````
Now I receive the following Error when i try to deploy the connector:
````
transforms.tostring.type: Invalid value class org.apache.camel.kafkaconnector.transforms.CamelTypeConverterTransform for configuration transforms.tostring.type: Error getting config definition from Transformation: null
````
Thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] valdar commented on issue #969: Convert NettyChannelBufferStreamCache from NettyHttpSource not converted to string
Posted by GitBox <gi...@apache.org>.
valdar commented on issue #969:
URL: https://github.com/apache/camel-kafka-connector/issues/969#issuecomment-796625099
@h4gen actually it should be in `0.7.3` too, that should be released sometime next week :tada:
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] oscerd closed issue #969: Convert NettyChannelBufferStreamCache from NettyHttpSource not converted to string
Posted by GitBox <gi...@apache.org>.
oscerd closed issue #969:
URL: https://github.com/apache/camel-kafka-connector/issues/969
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] h4gen commented on issue #969: Convert NettyChannelBufferStreamCache from NettyHttpSource not converted to string
Posted by GitBox <gi...@apache.org>.
h4gen commented on issue #969:
URL: https://github.com/apache/camel-kafka-connector/issues/969#issuecomment-811078351
Hey @valdar: Was this included in `0.7.3` ? Because I just tried to use the `0.7.3` component and the error is still there :(
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] h4gen edited a comment on issue #969: Convert NettyChannelBufferStreamCache from NettyHttpSource not converted to string
Posted by GitBox <gi...@apache.org>.
h4gen edited a comment on issue #969:
URL: https://github.com/apache/camel-kafka-connector/issues/969#issuecomment-772542731
Hi @valdar!
Thanks for the suggestion. I added the snippet to the configuration, leading to the following configuration in total:
````
{
"connector.class": "org.apache.camel.kafkaconnector.nettyhttp.CamelNettyhttpSourceConnector",
"camel.source.endpoint.receiveBufferSize": "10",
"camel.source.path.host": "0.0.0.0",
"camel.source.path.protocol": "http",
"tasks.max": "1",
"topics": "test",
"camel.source.path.port": "8081",
"name": "http",
"value.converter": "org.apache.kafka.connect.storage.StringConverter",
"key.converter": "org.apache.kafka.connect.storage.StringConverter",
"transforms": "tostring",
"transforms.tostring.type": "org.apache.camel.kafkaconnector.transforms.CamelTypeConverterTransform",
"transforms.tostring.target.type": "java.lang.String"
}
````
Now I receive the following Error when i try to deploy the connector:
````
transforms.tostring.type: Invalid value class org.apache.camel.kafkaconnector.transforms.CamelTypeConverterTransform for configuration transforms.tostring.type: Error getting config definition from Transformation: null
````
I assume this was not the expected behavior? :)
Thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org