[jira] [Created] (FLINK-9442) Flink Scaling not working

["Yow Siew Wai (JIRA)" <ji...@apache.org>]

Yow Siew Wai created FLINK-9442:
-----------------------------------

             Summary: Flink Scaling not working
                 Key: FLINK-9442
                 URL: https://issues.apache.org/jira/browse/FLINK-9442
             Project: Flink
          Issue Type: Bug
            Reporter: Yow Siew Wai


Hi,

 

We are in the middle of testing scaling ability of Flink. But we found that scaling not working, no matter increase more slot or increase number of Task Manager. We would expect a linear, if not close-to-linear scaling performance but the result even show degradation. Appreciated any comments.

 

Test Details,

 

-VMWare vsphere

-Just a simple pass through test,

    - auto gen source 3mil records, each 1000k in size, parallelism=1

    - source pass into next map operator, which just return the same record, and sent counter to statsD, parallelism is in cases = 2,4,6

- 3 TM, total 6 slots(2/TM) each JM/TM has 32 vCPU, 100GB memory

- Result:

      - 2 slots: 26 seconds, 3mil/26=115k TPS

      - 4 slots: 23 seconds, 3mil/23=130k TPS

      - 6 slots: 22 seconds, 3mil/22=136k TPS

 

As shown the scaling is almost nothing. Any clue? Thanks.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)