You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by GitBox <gi...@apache.org> on 2022/01/12 02:43:55 UTC
[GitHub] [incubator-seatunnel] JustinMars1 opened a new issue #1024: [Bug] [mysql sink] 解析kafka数据写入mysql失败但未打印错误信息
JustinMars1 opened a new issue #1024:
URL: https://github.com/apache/incubator-seatunnel/issues/1024
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/incubator-seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues.
### What happened
解析kafka数据写入mysql失败但未打印错误信息
1.MySQL建表语句（test数据库）
CREATE TABLE `nn` (
`id` int(11) DEFAULT NULL,
`name` varchar(255) DEFAULT NULL,
`created` date DEFAULT NULL,
`modified` date DEFAULT NULL,
`yn` int(11) DEFAULT NULL
)；
2.创建kafka生产者，生产信息如下
>1,justin
>2.mars
3.使用打印信息的方式数据是显示的
如
sink {
console {}
}
输出信息如下
+---+------+
|id |name |
+---+------+
|1 |justin|
+---+------+
4.使用mysql配置信息，未写入数据库，也为打印错误日志
### SeaTunnel Version
waterdrop-dist-2.0.4-2.11.8
### SeaTunnel Config
```conf
env {
# seatunnel defined streaming batch duration in seconds
spark.streaming.batchDuration = 5
spark.app.name = "seatunnel"
spark.ui.port = 13000
}
source {
kafkaStream {
topics = "seatunnel"
consumer.bootstrap.servers = "bigdata2:9092"
consumer.group.id = "seatunnel_group_mysql1"
auto.offset.reset = "latest"
}
}
transform {
split {
source_field = "raw_message"
delimiter = ","
fields = ["id","name"]
result_table_name = "mars"
}
sql {
sql = "select id,name from mars",
table_name = "mars"
}
}
sink {
Mysql {
save_mode = "update",
truncate = true,
url = "jdbc:mysql://ip:3306/test",
user= "root",
password = "*****",
dbtable = "mars",
customUpdateStmt = "INSERT INTO test.nn (id, name, created, modified, yn) values(?, ?, now(), now(), 1) ON DUPLICATE KEY UPDATE id = IFNULL(VALUES (id), id), name = IFNULL(VALUES (name), name)"
}
}
```
### Running Command
```shell
./bin/start-waterdrop-spark.sh --master local[4] --deploy-mode client --config ./config/sparkkafkamysql.conf
```
### Error Exception
```log
未打印错误信息，但也为写入mysql数据库
```
### Flink or Spark Version
_No response_
### Java or Scala Version
jdk 1.8
### Screenshots
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] JustinMars1 commented on issue #1024: [Bug] [mysql sink] Parse Kafka data and write it to MySQL. Data writing fails and no error message is printed
Posted by GitBox <gi...@apache.org>.
JustinMars1 commented on issue #1024:
URL: https://github.com/apache/incubator-seatunnel/issues/1024#issuecomment-1010585193
> For better communication, please describe your problem in English, I will close this issue, you can reopen it when you are done updating.
already edit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] CalvinKirs closed issue #1024: [Bug] [mysql sink] 解析kafka数据写入mysql失败但未打印错误信息
Posted by GitBox <gi...@apache.org>.
CalvinKirs closed issue #1024:
URL: https://github.com/apache/incubator-seatunnel/issues/1024
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] JustinMars1 edited a comment on issue #1024: [Bug] [mysql sink] Parse Kafka data and write it to MySQL. Data writing fails and no error message is printed
Posted by GitBox <gi...@apache.org>.
JustinMars1 edited a comment on issue #1024:
URL: https://github.com/apache/incubator-seatunnel/issues/1024#issuecomment-1010575795
use dbtable = "nn" the same problem
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] CalvinKirs commented on issue #1024: [Bug] [mysql sink] 解析kafka数据写入mysql失败但未打印错误信息
Posted by GitBox <gi...@apache.org>.
CalvinKirs commented on issue #1024:
URL: https://github.com/apache/incubator-seatunnel/issues/1024#issuecomment-1010578344
For better communication, please describe your problem in English, I will close this issue, you can reopen it when you are done updating.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] JustinMars1 commented on issue #1024: [Bug] [mysql sink] 解析kafka数据写入mysql失败但未打印错误信息
Posted by GitBox <gi...@apache.org>.
JustinMars1 commented on issue #1024:
URL: https://github.com/apache/incubator-seatunnel/issues/1024#issuecomment-1010575795
dbtable = "mars"使用nn,也是数据未入库
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org