[GitHub] storm issue #1591: STORM-1038: Upgrade netty to 4.x in 1.x-branch

[hsun-cnnxty <gi...@git.apache.org>]

Github user hsun-cnnxty commented on the issue:

    https://github.com/apache/storm/pull/1591
  
    Performance test results are attached here.  
    [perf_compare_netty_3vs4_1.x-branch.zip](https://github.com/apache/storm/files/392165/perf_compare_netty_3vs4_1.x-branch.zip)
    
    Baseline: current head of 1.x-branch (e55684b164fe99823a4ae3562b62867a26aaba97)
    Netty4:   57fbccc7e0d710ccaf04a5a828f0ff4cf29ec855
    
    In addition to default configuration, 
    io.netty.noPreferDirect=false
    io.netty.allocator.type=pooled
    
    following configurations are also tested for netty 4.x
    
    -unpool
    io.netty.allocator.type=unpooled. Ask Netty to not use pooled allocation for buffer
    
    -nodirect
    io.netty.noPreferDirect=true.  Ask Netty to not use off-heap buffer allocation
    
    -unpool-nodirect
    io.netty.noPreferDirect=true
    io.netty.allocator.type=unpooled
    
    Due to the capacity of my laptop, I tested following loads: 2k, 5k, 10k.  At 10k, the CPU usage reaches above 85% and failures started to appear at startup. 
    
    The performance data seems to suggest
    - no big difference is observed comparing before/after netty upgrades
    I suspect only a small percentage of traffic will cross worker instances (jvm) so Netty does not play a big role in performance measured
    
    - netty 4.x consumes a little more CPU and memory
    The small memory increment may be simply b/c netty 4.x pulled in more classes.
    
    - netty 4.x has lower CPU consumption on GC
    This is especially visible with the default configuration where off-heap buffer allocation is used.
    
     
    
    
    
    
    
    



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---