[GitHub] storm issue #2293: STORM-2231 Fix multi-threads issue on executor send queue

[HeartSaVioR <gi...@git.apache.org>]

Github user HeartSaVioR commented on the issue:

    https://github.com/apache/storm/pull/2293
  
    Attached perf. test numbers.
    
    TVL, rate 100000, topology.max.spout.pending=5000
    
    > before patch
    
    uptime:  241 acked: 3,015,580 acked/sec: 100,519.33 failed:        0 99%:      48,562,175 99.9%:      68,943,871 min:       6,598,656 max:     113,115,135 mean:   16,473,567.56 stddev:    7,837,977.69 user:    209,500 sys:     31,370 gc:      4,059 mem:     603.67
    uptime:  241 acked: 3,015,560 acked/sec: 100,518.67 failed:        0 99%:      59,047,935 99.9%:     118,292,479 min:       6,529,024 max:     190,054,399 mean:   16,693,789.11 stddev:   10,039,823.68 user:    210,070 sys:     31,890 gc:      3,942 mem:     696.79
    uptime:  240 acked: 3,013,360 acked/sec: 100,445.33 failed:        0 99%:      43,450,367 99.9%:      60,948,479 min:       6,533,120 max:      95,354,879 mean:   16,021,801.15 stddev:    6,956,089.85 user:    209,140 sys:     32,040 gc:      3,699 mem:     565.77
    
    > after patch
    
    uptime:  240 acked: 3,013,520 acked/sec: 100,450.67 failed:        0 99%:      70,975,487 99.9%:     126,418,943 min:       6,533,120 max:     209,321,983 mean:   17,371,349.66 stddev:   11,903,746.03 user:    212,520 sys:     30,080 gc:      4,248 mem:     680.16
    uptime:  240 acked: 3,014,140 acked/sec: 100,471.33 failed:        0 99%:      87,425,023 99.9%:     148,504,575 min:       6,488,064 max:     240,517,119 mean:   18,471,646.86 stddev:   14,432,363.00 user:    213,850 sys:     30,490 gc:      4,334 mem:     656.84
    uptime:  240 acked: 3,015,380 acked/sec: 100,512.67 failed:        0 99%:      49,119,231 99.9%:      77,791,231 min:       6,656,000 max:     132,579,327 mean:   16,009,039.03 stddev:    7,829,363.14 user:    206,440 sys:     31,640 gc:      3,828 mem:   1,066.89
    
    It seems to be clear performance hit which is expected. 
    
    So maybe, 2 cents, if we have consensus that the use case is minor (initiating poll to user@/dev@, or any other approaches), we can consider about not supporting thread-safety on output collector for the benefit of major use cases.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---