You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@flink.apache.org by "mdzz9527 (via GitHub)" <gi...@apache.org> on 2024/05/29 10:00:34 UTC
[I] update operation does not take effect [flink-cdc]
mdzz9527 opened a new issue, #3378:
URL: https://github.com/apache/flink-cdc/issues/3378
### Please don't check the box below
- [X] I'm aware that bugs and new features should be reported on Apache Jira or Flink mailing list instead of here
### Again, please don't check the box below
- [X] I'm aware that new issues on GitHub will be ignored and automatically closed.
### 请不要勾选以下选项
- [X] 我已知悉缺陷和新功能需要在 Apache Jira 或 Flink 邮件列表中反馈，而不是在这里创建新 issue。
### 也请不要勾选以下选项
- [X] 我已知悉 GitHub 上的新 issue 会被忽略且自动关闭。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@flink.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] update operation does not take effect [flink-cdc]
Posted by "GlassesCXY (via GitHub)" <gi...@apache.org>.
GlassesCXY commented on issue #3378:
URL: https://github.com/apache/flink-cdc/issues/3378#issuecomment-2507020181
解决了吗
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] update operation does not take effect [flink-cdc]
Posted by "mdzz9527 (via GitHub)" <gi...@apache.org>.
mdzz9527 commented on issue #3378:
URL: https://github.com/apache/flink-cdc/issues/3378#issuecomment-2137045429
1. 环境 flink 1.17.2
2. 模式 yarn-session flink SQL
3. 使用flink-sql-connector-cdc-oracle-3.1.0flink-connector-jdbc-3.1.2-1.17
4. 表都有主键
oracle 的权限配置严格按照 flink cdc 文档配置 https://nightlies.apache.org/flink/flink-cdc-docs-release-3.1/docs/connectors/flink-sources/oracle-cdc/
5. SQL 配置如下,可以实现数据全量同步,新增数据同步,删除数据同步,无法同步数据update操作,现象:update 不会删除旧数据,同时会新增一条 ID = 0 的,且只包含变更信息的错误数据;后续测试发现:不管update 多少条数据,也只是不断更新 这条ID=0 的数据
6. SQL 如下
`SET
execution.checkpointing.interval = 10min;
create table products (
ID int,
NAME string,
DESCRIPTION string,
WEIGHT float,
primary key (ID) not enforced
) with (
'connector' = 'oracle-cdc',
'hostname' = '10.2.0.79',
'port' = '1521',
'username' = 'flinkuser',
'password' = 'flinkpw',
'database-name' = 'ORCL',
'schema-name' = 'FLINKUSER',
'table-name' = 'PRODUCTS',
'debezium.log.mining.strategy' = 'online_catalog',
'debezium.log.mining.continuous.mine' = 'true',
'connection.pool.size' = '20',
'connect.max-retries' = '3',
'scan.snapshot.fetch.size' = '1024',
'scan.incremental.snapshot.chunk.size' = '8096',
'scan.incremental.snapshot.enabled' = 'true',
'debezium.snapshot.mode' = 'initial'
-- initial
-- schema_only
);
create table tg_products (
ID decimal(9, 0),
NAME string,
DESCRIPTION string,
WEIGHT float,
primary key (ID) not enforced
) with (
'connector' = 'jdbc',
'url' = 'jdbc:mysql://1xxxxxxxxxx:33061/transit_data_oms',
'username' = 'xxxxxxxxx',
'password' = 'xxxxxx',
'table-name' = 'products'
);
insert into
tg_products select * from products;
`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] update operation does not take effect [flink-cdc]
Posted by "mdzz9527 (via GitHub)" <gi...@apache.org>.
mdzz9527 commented on issue #3378:
URL: https://github.com/apache/flink-cdc/issues/3378#issuecomment-2137100987
flink1.18.1 也稳定复现,
[jobmanager_log.txt](https://github.com/apache/flink-cdc/files/15483037/jobmanager_log.txt)
这是刚才在flink1.18.1出现这个问题的日志
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] update operation does not take effect [flink-cdc]
Posted by "mdzz9527 (via GitHub)" <gi...@apache.org>.
mdzz9527 commented on issue #3378:
URL: https://github.com/apache/flink-cdc/issues/3378#issuecomment-2137053341
1. oracle version:
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
PL/SQL Release 11.2.0.1.0 - Production
CORE 11.2.0.1.0 Production
TNS for 64-bit Windows: Version 11.2.0.1.0 - Production
NLSRTL Version 11.2.0.1.0 - Production
2. mysql verison : 5.7.41
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] update operation does not take effect [flink-cdc]
Posted by "mdzz9527 (via GitHub)" <gi...@apache.org>.
mdzz9527 commented on issue #3378:
URL: https://github.com/apache/flink-cdc/issues/3378#issuecomment-2137048597
其他依赖
1. xdb-19.3.0.0.jar
2. ojdbc8-19.3.0.0.jar
3. mysql-connector-java-8.0.27.jar
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] update operation does not take effect [flink-cdc]
Posted by "leonardBang (via GitHub)" <gi...@apache.org>.
leonardBang commented on issue #3378:
URL: https://github.com/apache/flink-cdc/issues/3378#issuecomment-2137244301
Thanks @mdzz9527 , could you report the issue in [Flink JIRA](https://issues.apache.org/jira/projects/FLINK/summary) ？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] update operation does not take effect [flink-cdc]
Posted by "leonardBang (via GitHub)" <gi...@apache.org>.
leonardBang closed issue #3378: update operation does not take effect
URL: https://github.com/apache/flink-cdc/issues/3378
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org