You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@streampark.apache.org by "szk108 (via GitHub)" <gi...@apache.org> on 2023/07/17 01:50:04 UTC
[GitHub] [incubator-streampark] szk108 opened a new issue, #2861: [Bug] Bug titleDoes Streampark not support jdbc connector to implement upsert to insert into MySQL and PGSQL?
szk108 opened a new issue, #2861:
URL: https://github.com/apache/incubator-streampark/issues/2861
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/incubator-streampark/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues.
### Java Version
java 8
streampark 2.0.0
flink 1.16.1
mysql 5.7
postgresql 9.4
### Scala Version
2.12.x
### StreamPark Version
2.0.0
### Flink Version
1.16.1
### deploy mode
kubernetes-application
### What happened
I saw that the official website of flink already supports jdbc to connect to mysql or postgresql, and can implement upsert to write data to the two databases, but streampark keeps reporting syntax errors？
### Error Exception
```log
i write a test to sink mysql and postgresql,table 'test_out' sink to mysql and 'test_customer_output' sink to postgresql.but SQL parse like don`t support to upsert.
errors:
Caused by; org apache finktable api5glParserException: SOL parse faled. Encountered "on" atline 27 column 49. Was eypecting one of E0 BCEP. "ETCH" NTESECT”.".......
```
### Screenshots
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!(您是否要贡献这个PR?)
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@streampark.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-streampark] wolfboys commented on issue #2861: [Bug] Bug titleDoes Streampark not support jdbc connector to implement upsert to insert into MySQL and PGSQL?
Posted by "wolfboys (via GitHub)" <gi...@apache.org>.
wolfboys commented on issue #2861:
URL: https://github.com/apache/incubator-streampark/issues/2861#issuecomment-1637363429
I see this syntax in your Flink SQL. I'm not sure if this is a supported syntax in Flink SQL. Are you sure this is a supported syntax in Flink SQL?
```
insert into test out values ('a', 'b'),('a' ,"c) on conflict(product id) do update set product name excluded.product name;
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@streampark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] Bug titleDoes Streampark not support jdbc connector to implement upsert to insert into MySQL and PGSQL? [incubator-streampark]
Posted by "wolfboys (via GitHub)" <gi...@apache.org>.
wolfboys closed issue #2861: [Bug] Bug titleDoes Streampark not support jdbc connector to implement upsert to insert into MySQL and PGSQL?
URL: https://github.com/apache/incubator-streampark/issues/2861
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@streampark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-streampark] szk108 commented on issue #2861: [Bug] Bug titleDoes Streampark not support jdbc connector to implement upsert to insert into MySQL and PGSQL?
Posted by "szk108 (via GitHub)" <gi...@apache.org>.
szk108 commented on issue #2861:
URL: https://github.com/apache/incubator-streampark/issues/2861#issuecomment-1637732220
@ziqiang-wang @wolfboys i can see this statement on the official website of flink . https://nightlies.apache.org/flink/flink-docs-release-1.16/docs/connectors/table/jdbc/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@streampark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-streampark] ziqiang-wang commented on issue #2861: [Bug] Bug titleDoes Streampark not support jdbc connector to implement upsert to insert into MySQL and PGSQL?
Posted by "ziqiang-wang (via GitHub)" <gi...@apache.org>.
ziqiang-wang commented on issue #2861:
URL: https://github.com/apache/incubator-streampark/issues/2861#issuecomment-1637627373
>
`on duplicate key update`
this is mysql's statement, not the flink sql
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@streampark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] Bug titleDoes Streampark not support jdbc connector to implement upsert to insert into MySQL and PGSQL? [incubator-streampark]
Posted by "DingPengfei (via GitHub)" <gi...@apache.org>.
DingPengfei commented on issue #2861:
URL: https://github.com/apache/incubator-streampark/issues/2861#issuecomment-1905622212
any pr solve this issue?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@streampark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] Bug titleDoes Streampark not support jdbc connector to implement upsert to insert into MySQL and PGSQL? [incubator-streampark]
Posted by "wolfboys (via GitHub)" <gi...@apache.org>.
wolfboys commented on issue #2861:
URL: https://github.com/apache/incubator-streampark/issues/2861#issuecomment-1844401900
Answered
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@streampark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-streampark] szk108 commented on issue #2861: [Bug] Bug titleDoes Streampark not support jdbc connector to implement upsert to insert into MySQL and PGSQL?
Posted by "szk108 (via GitHub)" <gi...@apache.org>.
szk108 commented on issue #2861:
URL: https://github.com/apache/incubator-streampark/issues/2861#issuecomment-1637285649
If I just insert data normally, an error will still be reported. It seems that the default is upsert writing, but this upsert mode streampark does not support it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@streampark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-streampark] szk108 commented on issue #2861: [Bug] Bug titleDoes Streampark not support jdbc connector to implement upsert to insert into MySQL and PGSQL?
Posted by "szk108 (via GitHub)" <gi...@apache.org>.
szk108 commented on issue #2861:
URL: https://github.com/apache/incubator-streampark/issues/2861#issuecomment-1637352600
CREATE TABLE test_out(
product_id varchar PRIMARY KEY not enforced,
product_name varchar
)with(
'connector' = 'jdbc',
'driver= 'org.postgresql.Driver",
'url' ='jdbc:postgresql://gp.....',
'username' = 'xxxxxxx',
'password' = "xxxxxxx",
'table-name'='test_out'
);
CREATE TABLE test_customer_output(
'id' int PRIMARY KEY not enforced,
start_time timestamp ,
card_id varchar(20) ,
event varchar(20),
'day' int,
amount decimal(16,2)
) with (
'connector' = 'jdbc',
'url’= "jdbc:mysql://xxxxx
'username'= xxxxxx,
'password’= xxxxx,
'table-name” = 'test_customer_output'
);
--测试upsert功能
-- insert into test_out values ('a','b'),('a','c') on conflict(product_id) do update set product_name = excluded.product_name';
-- insert into test_customer_output values(348,'2023-04-28 09:15:31.0',aa','bb',27,1818),(348,'2023-04-28 09:15:31.0','aaa','cct',27,1818)
-- on duplicate key update
-- id= values(id),
-- start_time = values(start time),
-- card_id=values(card id),
-- eyent = values(event),
-- 'day'=values('day'),
-- amount = values(amount)
-- ;
--测试正常插入数据按主键更新
insert into testout values ('a','b'),('a',c');
insert into test customer output values(348,'2023-04-28 09:15:31.0', 'aa' ,'bbb',27,1818),(348,'2023-04-28 09:15:31.0','aa','ccc',27,1818);
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@streampark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-streampark] szk108 commented on issue #2861: [Bug] Bug titleDoes Streampark not support jdbc connector to implement upsert to insert into MySQL and PGSQL?
Posted by "szk108 (via GitHub)" <gi...@apache.org>.
szk108 commented on issue #2861:
URL: https://github.com/apache/incubator-streampark/issues/2861#issuecomment-1637304619
Here is my SQL
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@streampark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org