[jira] [Created] (IGNITE-12751) callAsync(jobs, rdc) performance degrades quadratically as jobs.size() grows

["Ilya Kasnacheev (Jira)" <ji...@apache.org>]

Ilya Kasnacheev created IGNITE-12751:
----------------------------------------

             Summary: callAsync(jobs, rdc) performance degrades quadratically as jobs.size() grows
                 Key: IGNITE-12751
                 URL: https://issues.apache.org/jira/browse/IGNITE-12751
             Project: Ignite
          Issue Type: Bug
          Components: compute
    Affects Versions: 2.7.6
            Reporter: Ilya Kasnacheev
         Attachments: word-count-reproducer.zip

Please consider attached reproducer and linked report.

{{compute.callAsync(jobs, reducer);
Result [res=33, tookMs=81, jobs=5] //warm up
Result [res=99, tookMs=21, jobs=15]
Result [res=330, tookMs=22, jobs=50]
Result [res=990, tookMs=57, jobs=150]
Result [res=3300, tookMs=146, jobs=500]
Result [res=9900, tookMs=231, jobs=1500]
Result [res=33000, tookMs=840, jobs=5000]
Result [res=99000, tookMs=6965, jobs=15000]
Result [res=330000, tookMs=118394, jobs=50000]}}

As soon jobs.size() grows past 5000, performance begins to degrade quadratically.

I don't expect that it will be completely linear, but I would assume that it should stay linear-ish until size() hits at least 100000, given that we see clusters of 100 nodes and it's not unthinkable to expect 1000 jobs to be run on each node. 5000 jobs (which still give OK performance) / 100 nodes is just 50 jobs per node, which becomes limiting factor.

Linked question also mentions OOM event, which may be caused of intermediate storage of (N^2) data.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)