You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by GitBox <gi...@apache.org> on 2022/07/01 08:43:21 UTC
[GitHub] [incubator-seatunnel] immustard opened a new issue, #2105: [Bug] [KafkaTableStream] Kafka Source
immustard opened a new issue, #2105:
URL: https://github.com/apache/incubator-seatunnel/issues/2105
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/incubator-seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues.
### What happened
source is KafkaTableStream and the schema uses the example. but the flink job error : com.fasterxml.jackson.databind.node.ArrayNode cannot be cast to com.fasterxml.jackson.databind.node.ObjectNode
### SeaTunnel Version
v2.1.2
### SeaTunnel Config
```conf
env {
execution.parallelism=1
}
source {
KafkaTableStream {
consumer.bootstrap.servers = "192.168.10.102:9092"
consumer.group.id = "seatunnel-learn"
topics = test_csv
result_table_name = test
format.type = csv
schema = "[{\"field\":\"name\",\"type\":\"string\"},{\"field\":\"age\",\"type\":\"int\"}]"
format.field-delimiter = ";"
format.allow-comments = "true"
format.ignore-parse-errors = "true"
}
}
transform {
sql {
sql = "select name, age from test where age > 18"
}
}
sink {
Kafka {
topics = "test_sink"
producer.bootstrap.server = "192.168.10.102:9092"
}
}
```
### Running Command
```shell
./bin/start-seatunnel-flink.sh --config ./config/kafkaETL.conf
```
### Error Exception
```log
Exception in thread "main" java.lang.RuntimeException: Execute Flink task error
at org.apache.seatunnel.core.flink.command.FlinkTaskExecuteCommand.execute(FlinkTaskExecuteCommand.java:84)
at org.apache.seatunnel.core.base.Seatunnel.run(Seatunnel.java:39)
at org.apache.seatunnel.example.flink.LocalFlinkExample.main(LocalFlinkExample.java:40)
Caused by: java.lang.RuntimeException: String json deserialization exception.
at org.apache.seatunnel.common.utils.JsonUtils.parseObject(JsonUtils.java:242)
at org.apache.seatunnel.flink.kafka.source.KafkaTableStream.prepare(KafkaTableStream.java:114)
at org.apache.seatunnel.flink.kafka.source.KafkaTableStream.prepare(KafkaTableStream.java:51)
at org.apache.seatunnel.core.base.command.BaseTaskExecuteCommand.lambda$prepare$0(BaseTaskExecuteCommand.java:67)
at java.util.ArrayList.forEach(ArrayList.java:1259)
at org.apache.seatunnel.core.base.command.BaseTaskExecuteCommand.prepare(BaseTaskExecuteCommand.java:67)
at org.apache.seatunnel.core.flink.command.FlinkTaskExecuteCommand.execute(FlinkTaskExecuteCommand.java:80)
... 2 more
Caused by: java.lang.ClassCastException: com.fasterxml.jackson.databind.node.ArrayNode cannot be cast to com.fasterxml.jackson.databind.node.ObjectNode
at org.apache.seatunnel.common.utils.JsonUtils.parseObject(JsonUtils.java:239)
... 8 more
```
### Flink or Spark Version
Flink 1.13.6
### Java or Scala Version
Java 1.8
scala 2.12
### Screenshots
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Hisoka-X commented on issue #2105: [Bug] [KafkaTableStream] Kafka Source
Posted by GitBox <gi...@apache.org>.
Hisoka-X commented on issue #2105:
URL: https://github.com/apache/incubator-seatunnel/issues/2105#issuecomment-1175719221
Did you try use list schema in 2.1.1, is it work fine?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] immustard commented on issue #2105: [Bug] [KafkaTableStream] Kafka Source
Posted by GitBox <gi...@apache.org>.
immustard commented on issue #2105:
URL: https://github.com/apache/incubator-seatunnel/issues/2105#issuecomment-1174778015
您好，您的邮件已经收到。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Hisoka-X commented on issue #2105: [Bug] [KafkaTableStream] Kafka Source
Posted by GitBox <gi...@apache.org>.
Hisoka-X commented on issue #2105:
URL: https://github.com/apache/incubator-seatunnel/issues/2105#issuecomment-1175710541
I got, the reason is we replace fastjson by jackson, so if your schema is list not object, the convert will report exception, I will fix it. Thanks for report.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] VictorZeng commented on issue #2105: [Bug] [KafkaTableStream] Kafka Source
Posted by GitBox <gi...@apache.org>.
VictorZeng commented on issue #2105:
URL: https://github.com/apache/incubator-seatunnel/issues/2105#issuecomment-1181511385
> I got, the reason is we replace fastjson by jackson, so if your schema is list not object, the convert will report exception, I will fix it. Thanks for report.
Hi, Why use Jackson to replace fastjson?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] san346596324 commented on issue #2105: [Bug] [KafkaTableStream] Kafka Source
Posted by GitBox <gi...@apache.org>.
san346596324 commented on issue #2105:
URL: https://github.com/apache/incubator-seatunnel/issues/2105#issuecomment-1174777672
+1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] immustard commented on issue #2105: [Bug] [KafkaTableStream] Kafka Source
Posted by GitBox <gi...@apache.org>.
immustard commented on issue #2105:
URL: https://github.com/apache/incubator-seatunnel/issues/2105#issuecomment-1175649685
> +1,2.11 no problem
yep, it's no problem in v2.1.1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Hisoka-X closed issue #2105: [Bug] [KafkaTableStream] Kafka Source
Posted by GitBox <gi...@apache.org>.
Hisoka-X closed issue #2105: [Bug] [KafkaTableStream] Kafka Source
URL: https://github.com/apache/incubator-seatunnel/issues/2105
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Hisoka-X commented on issue #2105: [Bug] [KafkaTableStream] Kafka Source
Posted by GitBox <gi...@apache.org>.
Hisoka-X commented on issue #2105:
URL: https://github.com/apache/incubator-seatunnel/issues/2105#issuecomment-1181640499
Close by #2168
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] Hisoka-X commented on issue #2105: [Bug] [KafkaTableStream] Kafka Source
Posted by GitBox <gi...@apache.org>.
Hisoka-X commented on issue #2105:
URL: https://github.com/apache/incubator-seatunnel/issues/2105#issuecomment-1181523372
> > I got, the reason is we replace fastjson by jackson, so if your schema is list not object, the convert will report exception, I will fix it. Thanks for report.
>
> Hi, Why use Jackson to replace fastjson?
Different json frameworks have been used in the project before. For unification, we chose jackson.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] immustard commented on issue #2105: [Bug] [KafkaTableStream] Kafka Source
Posted by GitBox <gi...@apache.org>.
immustard commented on issue #2105:
URL: https://github.com/apache/incubator-seatunnel/issues/2105#issuecomment-1177100908
> Did you try use list schema in 2.1.1, is it work fine?
yes, it's work fine in 2.1.1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] immustard commented on issue #2105: [Bug] [KafkaTableStream] Kafka Source
Posted by GitBox <gi...@apache.org>.
immustard commented on issue #2105:
URL: https://github.com/apache/incubator-seatunnel/issues/2105#issuecomment-1274224656
您好，您的邮件已经收到。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org