You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "taohaozhi1129 (via GitHub)" <gi...@apache.org> on 2024/04/29 09:56:46 UTC
[I] [Bug] [Source Jdbc] Source Jdbc Error [seatunnel]
taohaozhi1129 opened a new issue, #6775:
URL: https://github.com/apache/seatunnel/issues/6775
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues.
### What happened
When synchronizing MySQL data, the order of the fields in the SELECT statement after the query must match the order of the fields in the MySQL table. Otherwise, an error will be thrown.
### SeaTunnel Version
2.3.4
### SeaTunnel Config
```conf
env {
parallelism = 2
job.mode = "BATCH"
checkpoint.interval = 10000
}
source {
Jdbc {
url = "jdbc:mysql://localhost:3306/test"
driver = "com.mysql.cj.jdbc.Driver"
user = "admin"
password = "123456"
connection_check_timeout_sec = 10
table_path = "test.pet_agreement"
query = "select id as id,apply_no as apply_no,pet_id as pet_id,pet_code as pet_code,pet_name as pet_name,pet_sex as pet_sex,base_id as base_id,base_name as base_name,tenant_addr as tenant_addr,adopt_name as adopt_name,phone as phone,adopt_card as adopt_card,adopt_address as adopt_address,user_id as user_id,nick_name as nick_name,adopt_date as adopt_date,sign_date as sign_date,status as status,type as type,variety_code as variety_code,variety_name as variety_name,type_code as type_code,type_name as type_name,url as url,tenant_id as tenant_id,create_by as create_by,update_by as update_by,create_time as create_time,update_time as update_time,del_flag as del_flag,remark as remark,sign_link as sign_link from pet_agreement"
}
}
sink {
Doris {
fenodes = "10.1.1.1:8030"
username = "root"
password = "123456"
sink.label-prefix = "st"
database = "test"
table = "ods_ipbu_pet_pet_agreement_d_f"
sink.enable-2pc = true
doris.config = {"format":"json","read_json_by_line":true}
data_save_mode = "DROP_DATA"
}
}
```
### Running Command
```shell
seatunnel run job
```
### Error Exception
```log
2024-04-29 17:47:18,011 ERROR org.apache.seatunnel.core.starter.SeaTunnel - Exception StackTrace:org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:199)
at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
at org.apache.seatunnel.example.engine.SeaTunnelEngineExample.main(SeaTunnelEngineExample.java:43)
Caused by: org.apache.seatunnel.engine.common.exception.SeaTunnelEngineException: org.apache.seatunnel.connectors.seatunnel.jdbc.exception.JdbcConnectorException: ErrorCode:[COMMON-08], ErrorDescription:[Sql operation failed, such as (execute,addBatch,close) etc...] - Couldn't read data - Cannot determine value type from string 'APL1669256132977766400'
at org.apache.seatunnel.connectors.seatunnel.jdbc.internal.JdbcInputFormat.nextRecord(JdbcInputFormat.java:156)
at org.apache.seatunnel.connectors.seatunnel.jdbc.source.JdbcSourceReader.pollNext(JdbcSourceReader.java:69)
at org.apache.seatunnel.engine.server.task.flow.SourceFlowLifeCycle.collect(SourceFlowLifeCycle.java:150)
at org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask.collect(SourceSeaTunnelTask.java:116)
at org.apache.seatunnel.engine.server.task.SeaTunnelTask.stateProcess(SeaTunnelTask.java:168)
at org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask.call(SourceSeaTunnelTask.java:121)
at org.apache.seatunnel.engine.server.TaskExecutionService$BlockingWorker.run(TaskExecutionService.java:643)
at org.apache.seatunnel.engine.server.TaskExecutionService$NamedTaskWrapper.run(TaskExecutionService.java:944)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.sql.SQLDataException: Cannot determine value type from string 'APL1669256132977766400'
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:115)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:98)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:90)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:64)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:74)
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:96)
at com.mysql.cj.jdbc.result.ResultSetImpl.getObject(ResultSetImpl.java:1431)
at com.mysql.cj.jdbc.result.ResultSetImpl.getLong(ResultSetImpl.java:856)
at org.apache.seatunnel.connectors.seatunnel.jdbc.utils.JdbcUtils.getLong(JdbcUtils.java:66)
at org.apache.seatunnel.connectors.seatunnel.jdbc.internal.converter.AbstractJdbcRowConverter.toInternal(AbstractJdbcRowConverter.java:74)
at org.apache.seatunnel.connectors.seatunnel.jdbc.internal.JdbcInputFormat.nextRecord(JdbcInputFormat.java:146)
... 12 more
Caused by: com.mysql.cj.exceptions.DataConversionException: Cannot determine value type from string 'APL1669256132977766400'
at com.mysql.cj.result.AbstractNumericValueFactory.createFromBytes(AbstractNumericValueFactory.java:65)
at com.mysql.cj.protocol.a.MysqlTextValueDecoder.decodeByteArray(MysqlTextValueDecoder.java:143)
at com.mysql.cj.protocol.result.AbstractResultsetRow.decodeAndCreateReturnValue(AbstractResultsetRow.java:135)
at com.mysql.cj.protocol.result.AbstractResultsetRow.getValueFromBytes(AbstractResultsetRow.java:243)
at com.mysql.cj.protocol.a.result.TextBufferRow.getValue(TextBufferRow.java:132)
at com.mysql.cj.jdbc.result.ResultSetImpl.getObject(ResultSetImpl.java:1329)
... 16 more
at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:191)
... 2 more
2024-04-29 17:47:18,011 ERROR org.apache.seatunnel.core.starter.SeaTunnel -
===============================================================================
Exception in thread "main" org.apache.seatunnel.core.starter.exception.CommandExecuteException: SeaTunnel job executed failed
at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:199)
at org.apache.seatunnel.core.starter.SeaTunnel.run(SeaTunnel.java:40)
at org.apache.seatunnel.example.engine.SeaTunnelEngineExample.main(SeaTunnelEngineExample.java:43)
Caused by: org.apache.seatunnel.engine.common.exception.SeaTunnelEngineException: org.apache.seatunnel.connectors.seatunnel.jdbc.exception.JdbcConnectorException: ErrorCode:[COMMON-08], ErrorDescription:[Sql operation failed, such as (execute,addBatch,close) etc...] - Couldn't read data - Cannot determine value type from string 'APL1669256132977766400'
at org.apache.seatunnel.connectors.seatunnel.jdbc.internal.JdbcInputFormat.nextRecord(JdbcInputFormat.java:156)
at org.apache.seatunnel.connectors.seatunnel.jdbc.source.JdbcSourceReader.pollNext(JdbcSourceReader.java:69)
at org.apache.seatunnel.engine.server.task.flow.SourceFlowLifeCycle.collect(SourceFlowLifeCycle.java:150)
at org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask.collect(SourceSeaTunnelTask.java:116)
at org.apache.seatunnel.engine.server.task.SeaTunnelTask.stateProcess(SeaTunnelTask.java:168)
at org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask.call(SourceSeaTunnelTask.java:121)
at org.apache.seatunnel.engine.server.TaskExecutionService$BlockingWorker.run(TaskExecutionService.java:643)
at org.apache.seatunnel.engine.server.TaskExecutionService$NamedTaskWrapper.run(TaskExecutionService.java:944)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.sql.SQLDataException: Cannot determine value type from string 'APL1669256132977766400'
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:115)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:98)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:90)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:64)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:74)
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:96)
at com.mysql.cj.jdbc.result.ResultSetImpl.getObject(ResultSetImpl.java:1431)
at com.mysql.cj.jdbc.result.ResultSetImpl.getLong(ResultSetImpl.java:856)
at org.apache.seatunnel.connectors.seatunnel.jdbc.utils.JdbcUtils.getLong(JdbcUtils.java:66)
at org.apache.seatunnel.connectors.seatunnel.jdbc.internal.converter.AbstractJdbcRowConverter.toInternal(AbstractJdbcRowConverter.java:74)
at org.apache.seatunnel.connectors.seatunnel.jdbc.internal.JdbcInputFormat.nextRecord(JdbcInputFormat.java:146)
... 12 more
Caused by: com.mysql.cj.exceptions.DataConversionException: Cannot determine value type from string 'APL1669256132977766400'
at com.mysql.cj.result.AbstractNumericValueFactory.createFromBytes(AbstractNumericValueFactory.java:65)
at com.mysql.cj.protocol.a.MysqlTextValueDecoder.decodeByteArray(MysqlTextValueDecoder.java:143)
at com.mysql.cj.protocol.result.AbstractResultsetRow.decodeAndCreateReturnValue(AbstractResultsetRow.java:135)
at com.mysql.cj.protocol.result.AbstractResultsetRow.getValueFromBytes(AbstractResultsetRow.java:243)
at com.mysql.cj.protocol.a.result.TextBufferRow.getValue(TextBufferRow.java:132)
at com.mysql.cj.jdbc.result.ResultSetImpl.getObject(ResultSetImpl.java:1329)
... 16 more
at org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand.execute(ClientExecuteCommand.java:191)
... 2 more
```
### Zeta or Flink or Spark Version
_No response_
### Java or Scala Version
_No response_
### Screenshots
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] [Source Jdbc] Source Jdbc Error [seatunnel]
Posted by "Carl-Zhou-CN (via GitHub)" <gi...@apache.org>.
Carl-Zhou-CN closed issue #6775: [Bug] [Source Jdbc] Source Jdbc Error
URL: https://github.com/apache/seatunnel/issues/6775
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org