You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Piotr Nowojski (JIRA)" <ji...@apache.org> on 2018/06/28 08:14:00 UTC
[jira] [Closed] (FLINK-9560) RateLimiting for FileSystem
[ https://issues.apache.org/jira/browse/FLINK-9560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Piotr Nowojski closed FLINK-9560.
---------------------------------
Resolution: Unresolved
Closing because of the way how S3 file systems are internally implemented. For details check [~etienne.carriere@datadome.co] comment above.
> RateLimiting for FileSystem
> ---------------------------
>
> Key: FLINK-9560
> URL: https://issues.apache.org/jira/browse/FLINK-9560
> Project: Flink
> Issue Type: Improvement
> Components: FileSystem
> Affects Versions: 1.5.0
> Reporter: Etienne CARRIERE
> Assignee: Etienne CARRIERE
> Priority: Major
> Labels: pull-request-available
>
> *Pain*: On our system, we see that during checkpoint , all the bandwidth is take to send the checkpoint to object storage (s3 in our case)
> *Proposal* : After the creation of some limitation on Filesystem (mostly number of connections with the tickets FLINK-8125/FLINK-8198/FLINK-9468), I propose to add ratelimiting "per Filesystem" .
> *Proposal of implementation* : Modify LimitedConnectionsFileSystem to add a ratelimiter on both Input and OutputStream.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)