You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "WeirsWesley (via GitHub)" <gi...@apache.org> on 2023/06/08 02:05:57 UTC

[GitHub] [seatunnel] WeirsWesley opened a new issue, #4902: [Bug] [Transform-v2 mysql to mysql] Sink[0]-Jdbc-test_user source table name [test_user] can not be found

WeirsWesley opened a new issue, #4902:
URL: https://github.com/apache/seatunnel/issues/4902

   ### Search before asking
   
   - [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues.
   
   
   ### What happened
   
   One transform can succeed.  Two more have something wrong.  
   
   `{
       "env": {
           "job.mode": "BATCH"
       },
       "source": [{
           "driver": "com.mysql.cj.jdbc.Driver",
           "password": "*",
           "plugin_name": "jdbc",
           "query": "select * from test_user",
           "url": "jdbc:mysql://192.168.*.*:3306/test_source",
           "user": "root"
       }],
       "transform": [ {
           "after_loc": 4,
           "before_loc": 6,
           "cover_key": "#",
           "dest_field": "id_card",
           "plugin_name": "CoverHAT",
           "result_table_name": "test_user_bak",
           "source_table_name": "test_user",
           "src_field": "id_card"
       }],
       "sink": [{
           "driver": "com.mysql.cj.jdbc.Driver",
           "password": "*",
           "plugin_name": "jdbc",
           "query": "insert into test_user_bak values(?,?,?,?,?)",
           "url": "jdbc:mysql://192.168.*.*:3306/test_target",
           "user": "root"
       }]
   }`
   
   One transform like this can succeed.  But two more need source_table_name in sink and result_table_name in source. Error reported when running after adding.
   
   I am certain that the table exists.
   
   ### SeaTunnel Version
   
   2.3.1
   
   ### SeaTunnel Config
   
   ```conf
   {
       "env": {
           "job.mode": "BATCH"
       },
       "source": [{
           "driver": "com.mysql.cj.jdbc.Driver",
           "password": "*",
           "plugin_name": "jdbc",
           "query": "select * from test_user",
           "url": "jdbc:mysql://192.168.*.*:3306/test_source",
           "user": "root",
           "result_table_name": "test_user_bak"
       }],
       "transform": [{
           "cover_key": "*",
           "dest_field": "remark",
           "end_loc": 9,
           "plugin_name": "CoverBody",
           "result_table_name": "test_user_bak",
           "source_table_name": "test_user",
           "src_field": "remark",
           "start_loc": 4
       }, {
           "cover_key": "*",
           "dest_field": "mail",
           "plugin_name": "CoverBefore",
           "result_table_name": "test_user_bak",
           "source_table_name": "test_user",
           "special_key": "@",
           "src_field": "mail"
       }, {
           "after_loc": 4,
           "before_loc": 6,
           "cover_key": "#",
           "dest_field": "id_card",
           "plugin_name": "CoverHAT",
           "result_table_name": "test_user_bak",
           "source_table_name": "test_user",
           "src_field": "id_card"
       }],
       "sink": [{
           "driver": "com.mysql.cj.jdbc.Driver",
           "password": "*",
           "plugin_name": "jdbc",
           "query": "insert into test_user_bak values(?,?,?,?,?)",
           "url": "jdbc:mysql://192.168.*.*:3306/test_target",
           "user": "root",
           "source_table_name": "test_user"
       }]
   }
   ```
   
   
   ### Running Command
   
   ```shell
   /opt/apache-seatunnel-incubating-2.3.1/bin/seatunnel.sh --config /opt/configTemp/86003.json -e local
   ```
   
   
   ### Error Exception
   
   ```log
   2023-06-08 08:47:30,344 ERROR org.apache.seatunnel.core.starter.SeaTunnel -
   
   ===============================================================================
   
   
   2023-06-08 08:47:30,344 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Fatal Error,
   
   2023-06-08 08:47:30,344 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Please submit bug report in https://github.com/apache/incubator-seatunnel/issues
   
   2023-06-08 08:47:30,344 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Reason:SeaTunnel job executed failed
   
   2023-06-08 08:47:30,345 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Exception StackTrace:org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
           at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:181)
           at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
           at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34)
   Caused by: org.apache.seatunnel.engine.common.exception.JobDefineCheckException: Sink[0]-Jdbc-test_user source table name [test_user] can not be found
           at org.apache.seatunnel.engine.core.parse.JobConfigParser.sourceAnalyze(JobConfigParser.java:243)
           at org.apache.seatunnel.engine.core.parse.JobConfigParser.complexAnalyze(JobConfigParser.java:219)
           at org.apache.seatunnel.engine.core.parse.JobConfigParser.parse(JobConfigParser.java:134)
           at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parse(MultipleTableJobConfigParser.java:112)
           at org.apache.seatunnel.engine.client.job.JobExecutionEnvironment.getLogicalDag(JobExecutionEnvironment.java:155)
           at org.apache.seatunnel.engine.client.job.JobExecutionEnvironment.execute(JobExecutionEnvironment.java:147)
           at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:140)
           ... 2 more
   
   2023-06-08 08:47:30,346 ERROR org.apache.seatunnel.core.starter.SeaTunnel -
   ===============================================================================
   
   
   
   Exception in thread "main" org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
           at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:181)
           at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
           at org.apache.seatunnel.core.starter.seatunnel.SeaTunnelClient.main(SeaTunnelClient.java:34)
   Caused by: org.apache.seatunnel.engine.common.exception.JobDefineCheckException: Sink[0]-Jdbc-test_user source table name [test_user] can not be found
           at org.apache.seatunnel.engine.core.parse.JobConfigParser.sourceAnalyze(JobConfigParser.java:243)
           at org.apache.seatunnel.engine.core.parse.JobConfigParser.complexAnalyze(JobConfigParser.java:219)
           at org.apache.seatunnel.engine.core.parse.JobConfigParser.parse(JobConfigParser.java:134)
           at org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser.parse(MultipleTableJobConfigParser.java:112)
           at org.apache.seatunnel.engine.client.job.JobExecutionEnvironment.getLogicalDag(JobExecutionEnvironment.java:155)
           at org.apache.seatunnel.engine.client.job.JobExecutionEnvironment.execute(JobExecutionEnvironment.java:147)
           at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:140)
           ... 2 more
   ```
   
   
   ### Flink or Spark Version
   
   _No response_
   
   ### Java or Scala Version
   
   1.8
   
   ### Screenshots
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] WeirsWesley closed issue #4902: [Bug] [Transform-v2 mysql to mysql] Sink[0]-Jdbc-test_user source table name [test_user] can not be found

Posted by "WeirsWesley (via GitHub)" <gi...@apache.org>.
WeirsWesley closed issue #4902: [Bug] [Transform-v2 mysql to mysql] Sink[0]-Jdbc-test_user source table name [test_user] can not be found
URL: https://github.com/apache/seatunnel/issues/4902


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] zhilinli123 commented on issue #4902: [Bug] [Transform-v2 mysql to mysql] Sink[0]-Jdbc-test_user source table name [test_user] can not be found

Posted by "zhilinli123 (via GitHub)" <gi...@apache.org>.
zhilinli123 commented on issue #4902:
URL: https://github.com/apache/seatunnel/issues/4902#issuecomment-1582084647

   Can you send your conf file
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org