You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@rocketmq.apache.org by GitBox <gi...@apache.org> on 2022/08/23 13:06:44 UTC
[GitHub] [rocketmq-connect] Oliverwqcwrw opened a new issue, #268: JdbcSourceTask start failed
Oliverwqcwrw opened a new issue, #268:
URL: https://github.com/apache/rocketmq-connect/issues/268
**BUG REPORT**
1. Please describe the issue you observed:
- What did you do (The steps to reproduce)?
start runtime and JdbcSourceConnector
my request param
>
{
"connector-class":"org.apache.rocketmq.connect.jdbc.connector.JdbcSourceConnector",
"max-task":"2",
"connection.url":"jdbc:mysql://localhost:3306",
"connection.user":"root",
"connection.password":"root",
"table.whitelist":"nacos.config_info",
"mode": "incrementing",
"incrementing.column.name":"id",
"timestamp.initial": -1,
"source-record-converter":"org.apache.rocketmq.connect.runtime.converter.record.json.JsonConverter",
"connect-topicname":"configInfo",
"value-converter":"org.apache.rocketmq.connect.runtime.converter.record.json.JsonConverter",
"key-converter":"org.apache.rocketmq.connect.runtime.converter.record.json.JsonConverter"
}
the error log
>
2022-08-23 20:41:16 ERROR StateMachineService - Execution exception , {}
java.util.concurrent.ExecutionException: io.openmessaging.connector.api.errors.ConnectException: Failed trying to validate that columns used for offsets are NOT NULL
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:206)
at org.apache.rocketmq.connect.runtime.connectorwrapper.Worker.checkErrorTasks(Worker.java:448)
at org.apache.rocketmq.connect.runtime.connectorwrapper.Worker.maintainTaskState(Worker.java:338)
at org.apache.rocketmq.connect.runtime.connectorwrapper.Worker$StateMachineService.run(Worker.java:710)
at java.lang.Thread.run(Thread.java:748)
Caused by: io.openmessaging.connector.api.errors.ConnectException: Failed trying to validate that columns used for offsets are NOT NULL
at org.apache.rocketmq.connect.jdbc.source.offset.SourceOffsetCompute.validateNonNullable(SourceOffsetCompute.java:237)
at org.apache.rocketmq.connect.jdbc.source.offset.SourceOffsetCompute.initOffsetValues(SourceOffsetCompute.java:151)
at org.apache.rocketmq.connect.jdbc.source.offset.SourceOffsetCompute.initOffset(SourceOffsetCompute.java:119)
at org.apache.rocketmq.connect.jdbc.connector.JdbcSourceTask.start(JdbcSourceTask.java:177)
at org.apache.rocketmq.connect.runtime.connectorwrapper.WorkerSourceTask.initializeAndStart(WorkerSourceTask.java:453)
at org.apache.rocketmq.connect.runtime.connectorwrapper.WorkerTask.doInitializeAndStart(WorkerTask.java:91)
at org.apache.rocketmq.connect.runtime.connectorwrapper.WorkerTask.doRun(WorkerTask.java:167)
at org.apache.rocketmq.connect.runtime.connectorwrapper.WorkerTask.run(WorkerTask.java:186)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
at java.util.concurrent.FutureTask.run(FutureTask.java)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
... 1 common frames omitted
Caused by: java.sql.SQLSyntaxErrorException: Table 'b3log_symphony.config_info' doesn't exist
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:120)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)
at com.mysql.cj.jdbc.StatementImpl.executeQuery(StatementImpl.java:1218)
at com.mysql.cj.jdbc.DatabaseMetaData$7.forEach(DatabaseMetaData.java:2980)
at com.mysql.cj.jdbc.DatabaseMetaData$7.forEach(DatabaseMetaData.java:2968)
at com.mysql.cj.jdbc.IterateBlock.doForAll(IterateBlock.java:56)
at com.mysql.cj.jdbc.DatabaseMetaData.getPrimaryKeys(DatabaseMetaData.java:3021)
at org.apache.rocketmq.connect.jdbc.dialect.impl.GenericDatabaseDialect.primaryKeyColumns(GenericDatabaseDialect.java:745)
at org.apache.rocketmq.connect.jdbc.dialect.impl.GenericDatabaseDialect.describeColumns(GenericDatabaseDialect.java:567)
at org.apache.rocketmq.connect.jdbc.dialect.impl.GenericDatabaseDialect.describeColumns(GenericDatabaseDialect.java:548)
at org.apache.rocketmq.connect.jdbc.source.offset.SourceOffsetCompute.validateNonNullable(SourceOffsetCompute.java:207)
... 14 common frames omitted
I debug and find that the problem is that the `mysqL-connector-Java` dependency does not concatenate the database name we passed when concatenating the SQL
com.mysql.cj.jdbc.DatabaseMetaData#getPrimaryKeys
<img width="995" alt="image" src="https://user-images.githubusercontent.com/31796988/186163967-737be08a-6356-4e57-855f-b883a1da13dd.png">
<img width="1017" alt="image" src="https://user-images.githubusercontent.com/31796988/186163997-e7f1775b-4fb8-4538-b8e3-cb2dc6fa8cc9.png">
<img width="947" alt="image" src="https://user-images.githubusercontent.com/31796988/186164038-5cd11807-7560-4085-b00c-6121753e2ce4.png">
I tried to upgrade to 8.30 without this problem, but there are other problems, I am not sure whether it is my environment or a mysql dependency problem
- What is expected to see?
start JdbcSourceTask success
- What did you see instead?
start JdbcSourceTask failed
2. Please tell us about your environment:
system: Mac OS 12.5.1
connect: master
4. Other information (e.g. detailed explanation, logs, related issues, suggestions on how to fix, etc):
If there is something wrong with my environment or configuration, please let me know. Thanks :)
If it is a dependency problem, we can upgrade to 8.30 and fix the other problems later
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-connect] sunxiaojian commented on issue #268: JdbcSourceTask start failed
Posted by GitBox <gi...@apache.org>.
sunxiaojian commented on issue #268:
URL: https://github.com/apache/rocketmq-connect/issues/268#issuecomment-1225095662
>
OK, I'll try it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-connect] sunxiaojian commented on issue #268: JdbcSourceTask start failed
Posted by GitBox <gi...@apache.org>.
sunxiaojian commented on issue #268:
URL: https://github.com/apache/rocketmq-connect/issues/268#issuecomment-1225078693
Check your table structure and describe the table structure information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-connect] ShannonDing closed issue #268: JdbcSourceTask start failed
Posted by GitBox <gi...@apache.org>.
ShannonDing closed issue #268: JdbcSourceTask start failed
URL: https://github.com/apache/rocketmq-connect/issues/268
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-connect] sunxiaojian commented on issue #268: JdbcSourceTask start failed
Posted by GitBox <gi...@apache.org>.
sunxiaojian commented on issue #268:
URL: https://github.com/apache/rocketmq-connect/issues/268#issuecomment-1225168951
I tried it. There is no problem. Is this configuration correct?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-connect] Oliverwqcwrw commented on issue #268: JdbcSourceTask start failed
Posted by GitBox <gi...@apache.org>.
Oliverwqcwrw commented on issue #268:
URL: https://github.com/apache/rocketmq-connect/issues/268#issuecomment-1225352737
5.1.49 no problem, I'm ready to upgrade to 8.0.30 because 5.1.49 has some vulnerabilities
[CVE-2022-21363](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-21363)
[CVE-2019-2692](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-2692)
[CVE-2018-3258](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-3258)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [rocketmq-connect] Oliverwqcwrw commented on issue #268: JdbcSourceTask start failed
Posted by GitBox <gi...@apache.org>.
Oliverwqcwrw commented on issue #268:
URL: https://github.com/apache/rocketmq-connect/issues/268#issuecomment-1225094472
Thanks for your reply @sunxiaojian
My table structure:
>
-- auto-generated definition
create table config_info
(
id bigint auto_increment comment 'id'
primary key,
data_id varchar(255) not null comment 'data_id',
group_id varchar(255) null,
content longtext not null comment 'content',
md5 varchar(32) null comment 'md5',
gmt_create datetime default CURRENT_TIMESTAMP not null comment '创建时间',
gmt_modified datetime default CURRENT_TIMESTAMP not null comment '修改时间',
src_user text null comment 'source user',
src_ip varchar(50) null comment 'source ip',
app_name varchar(128) null,
tenant_id varchar(128) default '' null comment '租户字段',
c_desc varchar(256) null,
c_use varchar(64) null,
effect varchar(64) null,
type varchar(64) null,
c_schema text null,
constraint uk_configinfo_datagrouptenant
unique (data_id, group_id, tenant_id)
)
comment 'config_info' collate = utf8_bin;
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org