[jira] [Commented] (IGNITE-3018) Cache affinity calculation is slow with large nodes number

["Taras Ledkov (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IGNITE-3018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15785367#comment-15785367 ] 

Taras Ledkov commented on IGNITE-3018:
--------------------------------------

I've implemented two realization of the rendezvous affinity function:
- One is the full compatible with current and contains several performance fix (the recommendations for this fix are described at the IGNITE-3905 by Andrey & Alex). 
- Another is based on 32/64 bits Wang/Jenkins hash and build-in hashCode() for ClusterNode hash object for instead of MD5 (see above).

Also *Fair* affinity function is benchmarking together.

The results look like {{FairAffinityFunction}} provides the best performance for large nodes count.

Benchmark case: create base topology contains N nodes and change topology: add/remove one node.

|| Nodes ||      Fair          ||    Rendezvous (old)       ||    Rendezvous (compat)   ||     FastRendezvous   ||
|*10*    | 9.7 ms +/- 21.020 ms |  5.6 ms +/- 3.551 ms       | 5.3 ms +/- 3.674 ms      | 1.4 ms +/- 1.391 ms    |
|*100*   | 1.1 ms +/- 2.035 ms  |  52.3 ms +/- 9.569 ms      | 37.0 ms +/- 8.756 ms     | 5.2 ms +/- 3.468 ms    |
|*200*   | 0.9 ms +/- 1.202 ms  |  105.8 ms +/- 8.682 ms     | 75.1 ms +/- 12.888 ms    | 9.0 ms +/- 4.026 ms    |
|*300*   | 1.5 ms +/- 2.194 ms  |  163.8 ms +/- 13.147 ms    | 111.8 ms +/- 14.388 ms   | 14.1 ms +/- 6.257 ms   |
|*400*   | 2.0 ms +/- 2.447 ms  |  222.2 ms +/- 15.378 ms    | 146.9 ms +/- 12.616 ms   | 18.9 ms +/- 7.656 ms   |
|*500*   | 1.1 ms +/- 1.010 ms  |  283.5 ms +/- 15.329 ms    | 184.5 ms +/- 12.529 ms   | 22.5 ms +/- 7.070 ms   |
|*600*   | 4.3 ms +/- 2.887 ms  |  345.2 ms +/- 19.667 ms    | 223.9 ms +/- 15.834 ms   | 28.2 ms +/- 8.213 ms   |



> Cache affinity calculation is slow with large nodes number
> ----------------------------------------------------------
>
>                 Key: IGNITE-3018
>                 URL: https://issues.apache.org/jira/browse/IGNITE-3018
>             Project: Ignite
>          Issue Type: Bug
>          Components: cache
>            Reporter: Semen Boikov
>            Assignee: Taras Ledkov
>             Fix For: 2.0
>
>         Attachments: 003.png, 064.png, 100.png, 128.png, 200.png, 300.png, 400.png, 500.png, 600.png
>
>
> With large number of cache server nodes (> 200)  RendezvousAffinityFunction and FairAffinityFunction work pretty slow .
> For RendezvousAffinityFunction.assignPartitions can take hundredes of milliseconds, for FairAffinityFunction it can take seconds.
> For RendezvousAffinityFunction most time is spent in MD5 hash calculation and nodes list sorting. As optimization we can try to cache {partion, node} MD5 hash or try another hash function. Also several minor optimizations are possible (avoid unncecessary allocations, only one thread local 'get', etc).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)