You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "WeirsWesley (via GitHub)" <gi...@apache.org> on 2023/06/08 02:05:57 UTC
[GitHub] [seatunnel] WeirsWesley opened a new issue, #4902: [Bug] [Transform-v2 mysql to mysql] Sink[0]-Jdbc-test_user source table name [test_user] can not be found
WeirsWesley opened a new issue, #4902:
URL: https://github.com/apache/seatunnel/issues/4902
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues.
### What happened
One transform can succeed. Two more have something wrong.
`{
"env": {
"job.mode": "BATCH"
},
"source": [{
"driver": "com.mysql.cj.jdbc.Driver",
"password": "*",
"plugin_name": "jdbc",
"query": "select * from test_user",
"url": "jdbc:mysql://192.168.*.*:3306/test_source",
"user": "root"
}],
"transform": [ {
"after_loc": 4,
"before_loc": 6,
"cover_key": "#",
"dest_field": "id_card",
"plugin_name": "CoverHAT",
"result_table_name": "test_user_bak",
"source_table_name": "test_user",
"src_field": "id_card"
}],
"sink": [{
"driver": "com.mysql.cj.jdbc.Driver",
"password": "*",
"plugin_name": "jdbc",
"query": "insert into test_user_bak values(?,?,?,?,?)",
"url": "jdbc:mysql://192.168.*.*:3306/test_target",
"user": "root"
}]
}`
One transform like this can succeed. But two more need source_table_name in sink and result_table_name in source. Error reported when running after adding.
I am certain that the table exists.
### SeaTunnel Version
2.3.1
### SeaTunnel Config
```conf
{
"env": {
"job.mode": "BATCH"
},
"source": [{
"driver": "com.mysql.cj.jdbc.Driver",
"password": "*",
"plugin_name": "jdbc",
"query": "select * from test_user",
"url": "jdbc:mysql://192.168.*.*:3306/test_source",
"user": "root",
"result_table_name": "test_user_bak"
}],
"transform": [{
"cover_key": "*",
"dest_field": "remark",
"end_loc": 9,
"plugin_name": "CoverBody",
"result_table_name": "test_user_bak",
"source_table_name": "test_user",
"src_field": "remark",
"start_loc": 4
}, {
"cover_key": "*",
"dest_field": "mail",
"plugin_name": "CoverBefore",
"result_table_name": "test_user_bak",
"source_table_name": "test_user",
"special_key": "@",
"src_field": "mail"
}, {
"after_loc": 4,
"before_loc": 6,
"cover_key": "#",
"dest_field": "id_card",
"plugin_name": "CoverHAT",
"result_table_name": "test_user_bak",
"source_table_name": "test_user",
"src_field": "id_card"
}],
"sink": [{
"driver": "com.mysql.cj.jdbc.Driver",
"password": "*",
"plugin_name": "jdbc",
"query": "insert into test_user_bak values(?,?,?,?,?)",
"url": "jdbc:mysql://192.168.*.*:3306/test_target",
"user": "root",
"source_table_name": "test_user"
}]
}
```
### Running Command
```shell
/opt/apache-seatunnel-incubating-2.3.1/bin/seatunnel.sh --config /opt/configTemp/86003.json -e local
```
### Error Exception
```log
2023-06-08 08:47:30,344 ERROR org.apache.seatunnel.core.starter.SeaTunnel -
===============================================================================
2023-06-08 08:47:30,344 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Fatal Error,
2023-06-08 08:47:30,344 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Please submit bug report in https://github.com/apache/incubator-seatunnel/issues
2023-06-08 08:47:30,344 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Reason:SeaTunnel job executed failed
2023-06-08 08:47:30,345 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Exception StackTrace:org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:181)
at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34)
Caused by: org.apache.seatunnel.engine.common.exception.JobDefineCheckException: Sink[0]-Jdbc-test_user source table name [test_user] can not be found
at org.apache.seatunnel.engine.core.parse.JobConfigParser.sourceAnalyze(JobConfigParser.java:243)
at org.apache.seatunnel.engine.core.parse.JobConfigParser.complexAnalyze(JobConfigParser.java:219)
at org.apache.seatunnel.engine.core.parse.JobConfigParser.parse(JobConfigParser.java:134)
at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parse(MultipleTableJobConfigParser.java:112)
at org.apache.seatunnel.engine.client.job.JobExecutionEnvironment.getLogicalDag(JobExecutionEnvironment.java:155)
at org.apache.seatunnel.engine.client.job.JobExecutionEnvironment.execute(JobExecutionEnvironment.java:147)
at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:140)
... 2 more
2023-06-08 08:47:30,346 ERROR org.apache.seatunnel.core.starter.SeaTunnel -
===============================================================================
Exception in thread "main" org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:181)
at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34)
Caused by: org.apache.seatunnel.engine.common.exception.JobDefineCheckException: Sink[0]-Jdbc-test_user source table name [test_user] can not be found
at org.apache.seatunnel.engine.core.parse.JobConfigParser.sourceAnalyze(JobConfigParser.java:243)
at org.apache.seatunnel.engine.core.parse.JobConfigParser.complexAnalyze(JobConfigParser.java:219)
at org.apache.seatunnel.engine.core.parse.JobConfigParser.parse(JobConfigParser.java:134)
at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parse(MultipleTableJobConfigParser.java:112)
at org.apache.seatunnel.engine.client.job.JobExecutionEnvironment.getLogicalDag(JobExecutionEnvironment.java:155)
at org.apache.seatunnel.engine.client.job.JobExecutionEnvironment.execute(JobExecutionEnvironment.java:147)
at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:140)
... 2 more
```
### Flink or Spark Version
_No response_
### Java or Scala Version
1.8
### Screenshots
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [seatunnel] WeirsWesley closed issue #4902: [Bug] [Transform-v2 mysql to mysql] Sink[0]-Jdbc-test_user source table name [test_user] can not be found
Posted by "WeirsWesley (via GitHub)" <gi...@apache.org>.
WeirsWesley closed issue #4902: [Bug] [Transform-v2 mysql to mysql] Sink[0]-Jdbc-test_user source table name [test_user] can not be found
URL: https://github.com/apache/seatunnel/issues/4902
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [seatunnel] zhilinli123 commented on issue #4902: [Bug] [Transform-v2 mysql to mysql] Sink[0]-Jdbc-test_user source table name [test_user] can not be found
Posted by "zhilinli123 (via GitHub)" <gi...@apache.org>.
zhilinli123 commented on issue #4902:
URL: https://github.com/apache/seatunnel/issues/4902#issuecomment-1582084647
Can you send your conf file
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org