You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "swy (JIRA)" <ji...@apache.org> on 2018/06/15 15:01:00 UTC
[jira] [Updated] (FLINK-9597) Flink fail to scale!

     [ https://issues.apache.org/jira/browse/FLINK-9597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

swy updated FLINK-9597:
-----------------------
    Description: 
Hi, we found that our Flink application with simple logic, which using process function is not scale-able when scale from 8 parallelism onward even though with sufficient resources. Below it the result where is capped at around 250k TPS. No matter how we tune the parallelism of the operators it just not work, same to increase source parallelism.

Please refer to "scaleNotWork.png",
1. fixed source parallelism 4, other operators parallelism 8
2. fixed source parallelism 4, other operators parallelism 16
3. fixed source parallelism 4, other operators parallelism 32
4. fixed source parallelism 6, other operators parallelism 8
5. fixed source parallelism 6, other operators parallelism 16
6. fixed source parallelism 6, other operators parallelism 32
7. fixed source parallelism 6, other operators parallelism 64



  was:
Hi, we found that our Flink application with simple logic, which using process function is not scale-able when scale from 8 parallelism onward even though with sufficient resources. Below it the result where is capped at around 250k TPS. No matter how we tune the parallelism of the operators it just not work, same to increase source parallelism.

Please refer to "scaleNotWork.png"
1. fixed source parallelism 4, other operators parallelism 8
1. fixed source parallelism 4, other operators parallelism 8
1. fixed source parallelism 4, other operators parallelism 8
1. fixed source parallelism 4, other operators parallelism 8
1. fixed source parallelism 4, other operators parallelism 8




> Flink fail to scale!
> --------------------
>
>                 Key: FLINK-9597
>                 URL: https://issues.apache.org/jira/browse/FLINK-9597
>             Project: Flink
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.5.0
>            Reporter: swy
>            Priority: Major
>         Attachments: scaleNotWork.png
>
>
> Hi, we found that our Flink application with simple logic, which using process function is not scale-able when scale from 8 parallelism onward even though with sufficient resources. Below it the result where is capped at around 250k TPS. No matter how we tune the parallelism of the operators it just not work, same to increase source parallelism.
> Please refer to "scaleNotWork.png",
> 1. fixed source parallelism 4, other operators parallelism 8
> 2. fixed source parallelism 4, other operators parallelism 16
> 3. fixed source parallelism 4, other operators parallelism 32
> 4. fixed source parallelism 6, other operators parallelism 8
> 5. fixed source parallelism 6, other operators parallelism 16
> 6. fixed source parallelism 6, other operators parallelism 32
> 7. fixed source parallelism 6, other operators parallelism 64



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)