You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by GitBox <gi...@apache.org> on 2022/01/25 16:03:02 UTC
[GitHub] [camel-kafka-connector] Jelibo opened a new issue #1332: CameHeader value is wrongly interpreted as BigDecimal and causes ClassCastException
Jelibo opened a new issue #1332:
URL: https://github.com/apache/camel-kafka-connector/issues/1332
using `camel-kafka-connector 0.9.0`
using sink connector `CamelAws2s3SinkConnector`
Kafka header `CamelHeader.Id: l1kt74gl9w00` is logged as:
`ConnectHeader(key=CamelHeader.Id, value= l1kt74gl9w00, schema=Schema{STRING})`
Kafka header `CamelHeader.Id: 69203120e780` is logged as:
`ConnectHeader(key=CamelHeader.Id, value=6.9203120E+787, schema=Schema{org.apache.kafka.connect.data.Decimal:BYTES})`
causing the task to fail:
```
org.apache.kafka.connect.errors.ConnectException: Exiting WorkerSinkTask due to unrecoverable exception.
at org.apache.kafka.connect.runtime.WorkerSinkTask.deliverMessages(WorkerSinkTask.java:614)
at org.apache.kafka.connect.runtime.WorkerSinkTask.poll(WorkerSinkTask.java:329)
at org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:232)
at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:201)
at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:189)
at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:238)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.ClassCastException
```
```
Caused by: java.lang.ClassCastException: class java.math.BigDecimal cannot be cast to class [B (java.math.BigDecimal and [B are in module java.base of loader 'bootstrap')
at org.apache.camel.kafkaconnector.CamelSinkTask.mapHeader(CamelSinkTask.java:233)
at org.apache.camel.kafkaconnector.CamelSinkTask.put(CamelSinkTask.java:184)
at org.apache.kafka.connect.runtime.WorkerSinkTask.deliverMessages(WorkerSinkTask.java:586)
... 10 more
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] Jelibo commented on issue #1332: CameHeader value is wrongly interpreted as BigDecimal and causes ClassCastException
Posted by GitBox <gi...@apache.org>.
Jelibo commented on issue #1332:
URL: https://github.com/apache/camel-kafka-connector/issues/1332#issuecomment-1023429518
the records are produced by `org.apache.camel.kafkaconnector.rabbitmq.CamelRabbitmqSourceConnector`
there is no fine-grain config per header?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] valdar commented on issue #1332: CameHeader value is wrongly interpreted as BigDecimal and causes ClassCastException
Posted by GitBox <gi...@apache.org>.
valdar commented on issue #1332:
URL: https://github.com/apache/camel-kafka-connector/issues/1332#issuecomment-1025585617
> >
>
> I meant there is no config option how headers are handled and I have no control over it between these two connectors. I can only disable header by pattern.
Well you can plugin a custom HeaderConverter.
It is a bug though, I think I have a fix but releasing it for 0.11.0 and 1.0.0 will take a bit...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] Jelibo commented on issue #1332: CameHeader value is wrongly interpreted as BigDecimal and causes ClassCastException
Posted by GitBox <gi...@apache.org>.
Jelibo commented on issue #1332:
URL: https://github.com/apache/camel-kafka-connector/issues/1332#issuecomment-1025577418
> > there is no fine-grain config per header?
>
> What do you mean?
I meant there is no config option how headers are handled and I have no control over it between these two connectors. I can only disable header by pattern.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] valdar closed issue #1332: CameHeader value is wrongly interpreted as BigDecimal and causes ClassCastException
Posted by GitBox <gi...@apache.org>.
valdar closed issue #1332:
URL: https://github.com/apache/camel-kafka-connector/issues/1332
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] valdar commented on issue #1332: CameHeader value is wrongly interpreted as BigDecimal and causes ClassCastException
Posted by GitBox <gi...@apache.org>.
valdar commented on issue #1332:
URL: https://github.com/apache/camel-kafka-connector/issues/1332#issuecomment-1023261522
Hello, how are the records that the sink consume produced?
It seems that the header does not adhere with the schema declared:
`schema=Schema{org.apache.kafka.connect.data.Decimal:BYTES}` suggest that the value should be a bigdecimal encoded as a byte, so to retrieve it one should do something on the line of: `Decimal.toLogical(schema, (byte[]) header.value())` see [here](https://github.com/apache/camel-kafka-connector/blob/camel-kafka-connector-0.9.0/core/src/main/java/org/apache/camel/kafkaconnector/CamelSinkTask.java#L233). Unfortunately `header.value()` returns a `BigDecimal` instead of a `byte[]` as is declared it should in the provided schema...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] valdar edited a comment on issue #1332: CameHeader value is wrongly interpreted as BigDecimal and causes ClassCastException
Posted by GitBox <gi...@apache.org>.
valdar edited a comment on issue #1332:
URL: https://github.com/apache/camel-kafka-connector/issues/1332#issuecomment-1023261522
Hello @Jeliboi,
how are the records that the sink consume produced?
It seems that the header does not adhere with the schema declared:
`schema=Schema{org.apache.kafka.connect.data.Decimal:BYTES}` suggest that the value should be a bigdecimal encoded as a byte, so to retrieve it one should do something on the line of: `Decimal.toLogical(schema, (byte[]) header.value())` see [here](https://github.com/apache/camel-kafka-connector/blob/camel-kafka-connector-0.9.0/core/src/main/java/org/apache/camel/kafkaconnector/CamelSinkTask.java#L233). Unfortunately `header.value()` returns a `BigDecimal` instead of a `byte[]` as is declared it should in the provided schema...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] valdar commented on issue #1332: CameHeader value is wrongly interpreted as BigDecimal and causes ClassCastException
Posted by GitBox <gi...@apache.org>.
valdar commented on issue #1332:
URL: https://github.com/apache/camel-kafka-connector/issues/1332#issuecomment-1023471020
> the records are produced by `org.apache.camel.kafkaconnector.rabbitmq.CamelRabbitmqSourceConnector`
> there is no fine-grain config per header?
Which version is this connector?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] Jelibo commented on issue #1332: CameHeader value is wrongly interpreted as BigDecimal and causes ClassCastException
Posted by GitBox <gi...@apache.org>.
Jelibo commented on issue #1332:
URL: https://github.com/apache/camel-kafka-connector/issues/1332#issuecomment-1024005871
Should be the same version `0.9.0`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] valdar edited a comment on issue #1332: CameHeader value is wrongly interpreted as BigDecimal and causes ClassCastException
Posted by GitBox <gi...@apache.org>.
valdar edited a comment on issue #1332:
URL: https://github.com/apache/camel-kafka-connector/issues/1332#issuecomment-1023261522
Hello @Jelibo,
how are the records that the sink consume produced?
It seems that the header does not adhere with the schema declared:
`schema=Schema{org.apache.kafka.connect.data.Decimal:BYTES}` suggest that the value should be a bigdecimal encoded as a byte, so to retrieve it one should do something on the line of: `Decimal.toLogical(schema, (byte[]) header.value())` see [here](https://github.com/apache/camel-kafka-connector/blob/camel-kafka-connector-0.9.0/core/src/main/java/org/apache/camel/kafkaconnector/CamelSinkTask.java#L233). Unfortunately `header.value()` returns a `BigDecimal` instead of a `byte[]` as is declared it should in the provided schema...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] Jelibo edited a comment on issue #1332: CameHeader value is wrongly interpreted as BigDecimal and causes ClassCastException
Posted by GitBox <gi...@apache.org>.
Jelibo edited a comment on issue #1332:
URL: https://github.com/apache/camel-kafka-connector/issues/1332#issuecomment-1024005871
It is the same version as sink - `0.9.0`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] valdar commented on issue #1332: CameHeader value is wrongly interpreted as BigDecimal and causes ClassCastException
Posted by GitBox <gi...@apache.org>.
valdar commented on issue #1332:
URL: https://github.com/apache/camel-kafka-connector/issues/1332#issuecomment-1025573989
After a little bit of investigation, this is actually a bug in how bigdeciaml headers are handled in conjunction with the default `org.apache.kafka.connect.storage.SimpleHeaderConverter`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-kafka-connector] valdar commented on issue #1332: CameHeader value is wrongly interpreted as BigDecimal and causes ClassCastException
Posted by GitBox <gi...@apache.org>.
valdar commented on issue #1332:
URL: https://github.com/apache/camel-kafka-connector/issues/1332#issuecomment-1024054141
> there is no fine-grain config per header?
What do you mean?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org