[jira] [Assigned] (TS-3016) High CPU in 5.0

["Sudheer Vinukonda (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/TS-3016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sudheer Vinukonda reassigned TS-3016:
-------------------------------------

    Assignee: Sudheer Vinukonda

> High CPU in 5.0
> ---------------
>
>                 Key: TS-3016
>                 URL: https://issues.apache.org/jira/browse/TS-3016
>             Project: Traffic Server
>          Issue Type: Bug
>          Components: Core
>            Reporter: Sudheer Vinukonda
>            Assignee: Sudheer Vinukonda
>         Attachments: TS-3016.diff
>
>
> After 5.0 roll out, our production systems are noticing much higher cpu compared to pre-upgrade ats4 cpu usage. 
> For example, on some of our hosts running 8 hosts, below is the comparison before and after the upgade:
> {code}
> cpu %util: 88.7% (version: ats5)
> cpu %util: 70% (version: ats4)
> {code}
> Running perf top on traffic_server shows most CPU spent in SSL api calls, initially causing us to suspect any SSL related changes. However, after some analysis, it looks like the issue might be caused by the changes made in TS-1365. After reverting the changes in TS-1365, the CPU usage comes back to pre-upgrade levels again.
> {code}
> ats5:
> ------
> Samples: 374K of event 'cycles', Event count (approx.): 29126701697
>  26.44%  libcrypto.so.1.0.1e   [.] bn_sqr4x_mont
>  16.26%  libcrypto.so.1.0.1e   [.] bn_mul_mont  <------
>   6.74%  libcrypto.so.1.0.1e   [.] bn_mul4x_mont_gather5
>   4.95%  libcrypto.so.1.0.1e   [.] BN_usub
>   1.54%  libcrypto.so.1.0.1e   [.] sha256_block_data_order
>   1.37%  libcrypto.so.1.0.1e   [.] BN_mod_mul_montgomery
> 80.89 libcrypto.so.1.0.1e
> 7.09 traffic_server
> 5.02 [kernel]
> 2.71 libc-2.12.so
> 0.59 libpthread-2.12.so
> 0.42 libssl.so.1.0.1e
> 0.32 libtsutil.so.5
> 0.32 [bnx2]
> 0.02 libstdc++.so.6.0.13
> 0.02 libpcre.so.0.0.1
> 0.01 quick_filter.so
> 0.01 libtcl8.5.so
> 0.01 librt-2.12.so
> 0.01 [kernel].vsyscall_fn
> 0.01 [kernel].vsyscall_1
> 0 regex_remap.so
> 0 libresolv-2.12.so
> 0 libgtlocip.so.1.7.4.B86
> 0 header_rewrite.so
> 0 conf_remap.so
> 0 [sd_mod]
> 0 [mpt2sas]
> 0 [kernel].vsyscall_0
> 0 [jbd]
> 0 [ipv6]
> 0 [ext3]
> 0 [dm_mod]
> ats4:
> -----
> Samples: 249K of event 'cycles', Event count (approx.): 18282595621
>  39.34%  libcrypto.so.1.0.1e   [.] bn_sqr4x_mont
>  10.23%  libcrypto.so.1.0.1e   [.] bn_mul4x_mont_gather5
>   6.25%  libcrypto.so.1.0.1e   [.] bn_mul_mont  <------
>   2.19%  libcrypto.so.1.0.1e   [.] sha256_block_data_order
>   2.03%  libcrypto.so.1.0.1e   [.] BN_usub
>   1.50%  [kernel]              [k] find_busiest_group
>   1.08%  libcrypto.so.1.0.1e   [.] sha1_block_data_order_ssse3
> 80.93 libcrypto.so.1.0.1e
> 6.9 [kernel]
> 4.96 traffic_server
> 3 libc-2.12.so
> 0.84 libpthread-2.12.so
> 0.81 libssl.so.1.0.1e
> 0.38 [bnx2]
> 0.32 libtsutil.so.4
> 0.02 libtcl8.5.so
> 0.02 librt-2.12.so
> 0.02 libpcre.so.0.0.1
> 0.02 [kernel].vsyscall_fn
> 0.01 quick_filter.so
> 0.01 [kernel].vsyscall_1
> 0.01 [kernel].vsyscall_0
> 0 regex_remap.so
> 0 libstdc++.so.6.0.13
> 0 libresolv-2.12.so
> 0 header_rewrite.so
> 0 conf_remap.so
> 0 [sd_mod]
> 0 [mpt2sas]
> 0 [jbd]
> 0 [ipv6]
> 0 [ext3]
> 0 [dm_mod]
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)