You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "nmz0324 (via GitHub)" <gi...@apache.org> on 2023/05/05 01:23:51 UTC
[GitHub] [incubator-seatunnel] nmz0324 opened a new issue, #4704: [Bug] The mysql-cdc delete operation has been parsed into insert Doris
nmz0324 opened a new issue, #4704:
URL: https://github.com/apache/incubator-seatunnel/issues/4704
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/incubator-seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues.
### What happened
mysql:
CREATE DATABASE emp_1;
USE emp_1;
CREATE TABLE employees_1 (
emp_no INT NOT NULL,
birth_date DATE NOT NULL,
first_name VARCHAR(14) NOT NULL,
last_name VARCHAR(16) NOT NULL,
gender ENUM ('M','F') NOT NULL,
hire_date DATE NOT NULL,
PRIMARY KEY (emp_no)
);
INSERT INTO `employees_1` VALUES
(10001,'1953-09-02','Georgi','Facello','M','1986-06-26'),
(10002,'1964-06-02','Bezalel','Simmel','F','1985-11-21'),
(10003,'1959-12-03','Parto','Bamford','M','1986-08-28'),
(10004,'1954-05-01','Chirstian','Koblick','M','1986-12-01'),
(10005,'1955-01-21','Kyoichi','Maliniak','M','1989-09-12'),
(10006,'1953-04-20','Anneke','Preusig','F','1989-06-02'),
(10007,'1957-05-23','Tzvetan','Zielinski','F','1989-02-10'),
(10008,'1958-02-19','Saniya','Kalloufi','M','1994-09-15'),
(10009,'1952-04-19','Sumant','Peac','F','1985-02-18'),
(10010,'1963-06-01','Duangkaew','Piveteau','F','1989-08-24'),
(10011,'1953-11-07','Mary','Sluis','F','1990-01-22'),
(10012,'1960-10-04','Patricio','Bridgland','M','1992-12-18'),
(10013,'1963-06-07','Eberhardt','Terkki','M','1985-10-20'),
(10014,'1956-02-12','Berni','Genin','M','1987-03-11'),
(10015,'1959-08-19','Guoxiang','Nooteboom','M','1987-07-02'),
(10016,'1961-05-02','Kazuhito','Cappelletti','M','1995-01-27'),
(10017,'1958-07-06','Cristinel','Bouloucos','F','1993-08-03'),
(10018,'1954-06-19','Kazuhide','Peha','F','1987-04-03'),
(10019,'1953-01-23','Lillian','Haddadi','M','1999-04-30'),
(10020,'1952-12-24','Mayuko','Warwick','M','1991-01-26'),
(10021,'1960-02-20','Ramzi','Erde','M','1988-02-10'),
(10022,'1952-07-08','Shahaf','Famili','M','1995-08-22'),
(10023,'1953-09-29','Bojan','Montemayor','F','1989-12-17'),
(10024,'1958-09-05','Suzette','Pettey','F','1997-05-19'),
(10025,'1958-10-31','Prasadram','Heyers','M','1987-08-17'),
(10026,'1953-04-03','Yongqiao','Berztiss','M','1995-03-20'),
(10027,'1962-07-10','Divier','Reistad','F','1989-07-07'),
(10028,'1963-11-26','Domenick','Tempesti','M','1991-10-22'),
(10029,'1956-12-13','Otmar','Herbst','M','1985-11-20'),
(10030,'1958-07-14','Elvis','Demeyer','M','1994-02-17'),
(10031,'1959-01-27','Karsten','Joslin','M','1991-09-01'),
(10032,'1960-08-09','Jeong','Reistad','F','1990-06-20'),
(10033,'1956-11-14','Arif','Merlo','M','1987-03-18'),
(10034,'1962-12-29','Bader','Swan','M','1988-09-21'),
(10035,'1953-02-08','Alain','Chappelet','M','1988-09-05'),
(10036,'1959-08-10','Adamantios','Portugali','M','1992-01-03');
doris :
CREATE TABLE employees_1 (
emp_no INT NOT NULL,
birth_date DATE NOT NULL,
first_name VARCHAR(14) NOT NULL,
last_name VARCHAR(16) NOT NULL,
gender ENUM ('M','F') NOT NULL,
hire_date DATE NOT NULL,
PRIMARY KEY (emp_no)
);
### SeaTunnel Version
2.3.1
### SeaTunnel Config
```conf
env {
execution.parallelism = 2
job.mode = "STREAMING"
checkpoint.interval = 2000
}
source {
MySQL-CDC {
result_table_name = "MySQL-CDC"
parallelism = 1
server-id = 5656
username = "root"
password = "root"
table-names = ["emp_1.employees_1"]
base-url = "jdbc:mysql://192.168.15.73:3306/emp_1"
}
}transform {
}
sink
{
Doris {
fenodes = "192.168.15.73:18030"
username = root
password = "root"
table.identifier = "demo.employees_1"
sink.enable-2pc = "true"
sink.label-prefix = "test_json_9"
sink.enable-delete = "true"
doris.config = {
format="json"
read_json_by_line="true"
}
}
}
```
### Running Command
```shell
sh seatunnel.sh --config /opt/seatunnel/apache-seatunnel-incubating-2.3.1/config/mysql_cdc2doris.conf -e local
```
### Error Exception
```log
mysql 删除数据 delete from emp_1.employees_1 where emp_no=10011
doris 里面插入了这条数据；
```
### Flink or Spark Version
no
### Java or Scala Version
no
### Screenshots
no
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] nmz0324 commented on issue #4704: [Bug] The mysql-cdc delete operation has been parsed into insert Doris
Posted by "nmz0324 (via GitHub)" <gi...@apache.org>.
nmz0324 commented on issue #4704:
URL: https://github.com/apache/incubator-seatunnel/issues/4704#issuecomment-1535589903
doris 1.2.2 MySQL deleted, Doris did not synchronize deletion
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] nmz0324 closed issue #4704: [Bug] The mysql-cdc delete operation has been parsed into insert Doris
Posted by "nmz0324 (via GitHub)" <gi...@apache.org>.
nmz0324 closed issue #4704: [Bug] The mysql-cdc delete operation has been parsed into insert Doris
URL: https://github.com/apache/incubator-seatunnel/issues/4704
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] zy-kkk commented on issue #4704: [Bug] The mysql-cdc delete operation has been parsed into insert Doris
Posted by "zy-kkk (via GitHub)" <gi...@apache.org>.
zy-kkk commented on issue #4704:
URL: https://github.com/apache/incubator-seatunnel/issues/4704#issuecomment-1535937605
> doris 1.2.2 MySQL deleted, Doris did not synchronize deletion
please update doris version to 1.2.4-1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] nmz0324 commented on issue #4704: [Bug] The mysql-cdc delete operation has been parsed into insert Doris
Posted by "nmz0324 (via GitHub)" <gi...@apache.org>.
nmz0324 commented on issue #4704:
URL: https://github.com/apache/incubator-seatunnel/issues/4704#issuecomment-1537950267
ok
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org