You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "gaoyan1998 (via GitHub)" <gi...@apache.org> on 2024/08/17 07:35:51 UTC

[PR] [Mysql Pipline] Support mysql pipline rate limit [flink-cdc]

gaoyan1998 opened a new pull request, #3547:
URL: https://github.com/apache/flink-cdc/pull/3547

   This PR  Support  speed rate, 
   Prevent too much data in the initial stage from causing pressure downstream
   
   usageļ¼š
   ```yaml
   source:
     type: mysql
     ....
     ....
   sink:
     ...
     ....
   pipeline:
     name: Sync MySQL Database to Doris
     parallelism: 3
   # Limit the maximum number of data items per second on average
     rate.limit: 3
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [Mysql Pipline] Support mysql pipline rate limit [flink-cdc]

Posted by "gaoyan1998 (via GitHub)" <gi...@apache.org>.
gaoyan1998 commented on PR #3547:
URL: https://github.com/apache/flink-cdc/pull/3547#issuecomment-2303594608

   > Thanks @gaoyan1998 for the contribution, FLIP 27 Source API also has a basic rate limiting design, Would you consider develop base on that one?
   
   yes i can do it. Can you give me a link? 
   There are too many jira lists and I can't find it


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [Mysql Pipline] Support mysql pipline rate limit [flink-cdc]

Posted by "gaoyan1998 (via GitHub)" <gi...@apache.org>.
gaoyan1998 commented on PR #3547:
URL: https://github.com/apache/flink-cdc/pull/3547#issuecomment-2348856963

   @yuxiqian  @leonardBang  Thanks for guidance, the new pr is #3607. I will closed this pr.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [Mysql Pipline] Support mysql pipline rate limit [flink-cdc]

Posted by "gaoyan1998 (via GitHub)" <gi...@apache.org>.
gaoyan1998 closed pull request #3547: [Mysql Pipline] Support mysql pipline rate limit
URL: https://github.com/apache/flink-cdc/pull/3547


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [Mysql Pipline] Support mysql pipline rate limit [flink-cdc]

Posted by "leonardBang (via GitHub)" <gi...@apache.org>.
leonardBang commented on PR #3547:
URL: https://github.com/apache/flink-cdc/pull/3547#issuecomment-2296584623

   Thanks @gaoyan1998 for the contribution, FLIP 27 Source API also has  a basic rate limiting design, Would you consider develop base on that one?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [Mysql Pipline] Support mysql pipline rate limit [flink-cdc]

Posted by "yuxiqian (via GitHub)" <gi...@apache.org>.
yuxiqian commented on PR #3547:
URL: https://github.com/apache/flink-cdc/pull/3547#issuecomment-2303600839

   Will [FLIP-238](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=217385972) be helpful when implementing rate limited source?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org