You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Larry Xiao <xi...@sjtu.edu.cn> on 2014/09/28 05:25:39 UTC
PageRank execution imbalance, might hurt performance by 6x
Hi all!
I'm running PageRank on GraphX, and I find on some tasks on one machine
can spend 5~6 times more time than on others, others are perfectly
balance (around 1 second to finish).
And since time for a stage (iteration) is determined by the slowest
task, the performance is undesirable.
I don't know if there's any internals that might make execution
unstable? Like scheduling, garbage collection …
A stage for mapPartitions at GraphImpl.scala:409
> in mapReduceTriplets
> 408 // Map and combine.
> 409 val preAgg = view.edges.partitionsRDD.mapPartitions(_.flatMap {
> 410 case (pid, edgePartition) =>
> 411 // Choose scan method
looks like this:
Tasks
Index ID Attempt Status Locality Level Executor Launch Time
Duration ▴ GC Time Accumulators Input Shuffle Read Write Time
Shuffle Write Errors
21 787 0 SUCCESS PROCESS_LOCAL brick0 2014/09/28 03:04:42 7 s
333.3 MB (memory) 4.9 MB 1 ms 652.3 KB
0 768 0 SUCCESS PROCESS_LOCAL brick2 2014/09/28 03:04:42 7 s
531.5 MB (memory) 8.0 MB 2 ms 1321.5 KB
9 775 0 SUCCESS PROCESS_LOCAL brick0 2014/09/28 03:04:42 6 s
270.4 MB (memory) 4.1 MB 1 ms 659.3 KB
15 781 0 SUCCESS PROCESS_LOCAL brick0 2014/09/28 03:04:42 6 s
272.7 MB (memory) 4.3 MB 1 ms 658.9 KB
3 769 0 SUCCESS PROCESS_LOCAL brick0 2014/09/28 03:04:42 6 s
285.5 MB (memory) 4.4 MB 1 ms 658.5 KB
6 774 0 SUCCESS PROCESS_LOCAL brick2 2014/09/28 03:04:42 6 s
346.8 MB (memory) 4.6 MB 1 ms 657.0 KB
12 780 0 SUCCESS PROCESS_LOCAL brick2 2014/09/28 03:04:42 6 s
313.2 MB (memory) 4.4 MB 1 ms 645.5 KB
18 786 0 SUCCESS PROCESS_LOCAL brick2 2014/09/28 03:04:42 6 s
281.7 MB (memory) 4.2 MB 1 ms 660.1 KB
1 771 0 SUCCESS PROCESS_LOCAL brick3 2014/09/28 03:04:42 2 s
339.1 MB (memory) 5.1 MB 1 ms 657.4 KB
7 777 0 SUCCESS PROCESS_LOCAL brick3 2014/09/28 03:04:42 2 s
322.8 MB (memory) 4.9 MB 1 ms 654.5 KB
13 783 0 SUCCESS PROCESS_LOCAL brick3 2014/09/28 03:04:42 2 s
279.8 MB (memory) 4.6 MB 1 ms 655.4 KB
19 789 0 SUCCESS PROCESS_LOCAL brick3 2014/09/28 03:04:42 2 s
268.4 MB (memory) 4.4 MB 1 ms 658.5 KB
16 784 0 SUCCESS PROCESS_LOCAL brick4 2014/09/28 03:04:42 1 s
339.1 MB (memory) 5.1 MB 1 ms 660.1 KB
11 776 0 SUCCESS PROCESS_LOCAL brick1 2014/09/28 03:04:42 1 s
341.0 MB (memory) 5.3 MB 1 ms 655.4 KB
2 773 0 SUCCESS PROCESS_LOCAL brick5 2014/09/28 03:04:42 1 s
320.9 MB (memory) 4.9 MB 1 ms 655.3 KB
22 790 0 SUCCESS PROCESS_LOCAL brick4 2014/09/28 03:04:42 1 s
301.7 MB (memory) 4.9 MB 1 ms 659.5 KB
17 782 0 SUCCESS PROCESS_LOCAL brick1 2014/09/28 03:04:42 1 s
317.1 MB (memory) 5.2 MB 1 ms 653.7 KB
23 788 0 SUCCESS PROCESS_LOCAL brick1 2014/09/28 03:04:42 1 s
268.3 MB (memory) 4.9 MB 1 ms 664.3 KB
8 779 0 SUCCESS PROCESS_LOCAL brick5 2014/09/28 03:04:42 1 s
291.3 MB (memory) 4.6 MB 1 ms 660.6 KB
20 791 0 SUCCESS PROCESS_LOCAL brick5 2014/09/28 03:04:42 1 s
272.7 MB (memory) 4.5 MB 1 ms 661.7 KB
10 778 0 SUCCESS PROCESS_LOCAL brick4 2014/09/28 03:04:42 1 s
276.5 MB (memory) 4.4 MB 1 ms 656.4 KB
4 772 0 SUCCESS PROCESS_LOCAL brick4 2014/09/28 03:04:42 1 s
260.8 MB (memory) 4.4 MB 1 ms 661.7 KB
14 785 0 SUCCESS PROCESS_LOCAL brick5 2014/09/28 03:04:42 1 s
262.7 MB (memory) 4.3 MB 1 ms 651.7 KB
5 770 0 SUCCESS PROCESS_LOCAL brick1 2014/09/28 03:04:42 1 s
276.5 MB (memory) 4.7 MB 1 ms 655.1 KB
Thanks!
Larry