You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@trafficserver.apache.org by "Leif Hedstrom (JIRA)" <ji...@apache.org> on 2016/08/18 00:43:20 UTC
[jira] [Updated] (TS-4742) FATAL: ClusterCache.cc:1759: failed
assert `tc->tunnel_cont == tc`
[ https://issues.apache.org/jira/browse/TS-4742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Leif Hedstrom updated TS-4742:
------------------------------
Fix Version/s: sometime
> FATAL: ClusterCache.cc:1759: failed assert `tc->tunnel_cont == tc`
> ------------------------------------------------------------------
>
> Key: TS-4742
> URL: https://issues.apache.org/jira/browse/TS-4742
> Project: Traffic Server
> Issue Type: Bug
> Components: Clustering
> Reporter: Paul Stengel
> Fix For: sometime
>
>
> We're running TS in a two-server cluster right now. The clustering works well and is functioning as you'd expect. However, every so often, TS crashes with the following backtrace:
> {noformat}
> FATAL: ClusterCache.cc:1759: failed assert `tc->tunnel_cont == tc`
> traffic_server: using root directory '/usr'
> traffic_server: Aborted (Signal sent by tkill() 13260 114)
> traffic_server - STACK TRACE:
> /usr/bin/traffic_server(_Z19crash_logger_invokeiP9siginfo_tPv+0xa0)[0x55bbaed08f00]
> /lib/x86_64-linux-gnu/libpthread.so.0(+0x113d0)[0x2b45c2ff13d0]
> /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x38)[0x2b45c3cd7418]
> /lib/x86_64-linux-gnu/libc.so.6(abort+0x16a)[0x2b45c3cd901a]
> /usr/lib/trafficserver/libtsutil.so.6(+0x2430f)[0x2b45c1c8f30f]
> /usr/lib/trafficserver/libtsutil.so.6(+0x243a9)[0x2b45c1c8f3a9]
> /usr/lib/trafficserver/libtsutil.so.6(+0x22595)[0x2b45c1c8d595]
> /usr/bin/traffic_server(+0x2ee7e8)[0x55bbaef207e8]
> /usr/bin/traffic_server(_ZN12OneWayTunnel10startEventEiPv+0x16f)[0x55bbaeecd9df]
> /usr/bin/traffic_server(_ZN14ClusterHandler32cluster_signal_and_update_lockedEiP18ClusterVConnectionP17ClusterVConnState+0x2f)[0x55bbaef05abf]
> /usr/bin/traffic_server(_ZN14ClusterHandler20valid_for_data_writeEP18ClusterVConnection+0x8fa)[0x55bbaeefff1a]
> /usr/bin/traffic_server(_ZN14ClusterHandler23build_write_descriptorsEv+0x171)[0x55bbaef00091]
> /usr/bin/traffic_server(_ZN14ClusterHandler13process_writeElb+0x578)[0x55bbaef00868]
> /usr/bin/traffic_server(_ZN14ClusterHandler16mainClusterEventEiP5Event+0x195)[0x55bbaef00ae5]
> /usr/bin/traffic_server(_ZN12ClusterState10IOCompleteEv+0xc7)[0x55bbaef04be7]
> /usr/bin/traffic_server(_ZN12ClusterState16doIO_write_eventEiPv+0x119)[0x55bbaef04fd9]
> /usr/bin/traffic_server(_Z15write_to_net_ioP10NetHandlerP18UnixNetVConnectionP7EThread+0x1191)[0x55bbaefabfd1]
> /usr/bin/traffic_server(_ZN10NetHandler12mainNetEventEiP5Event+0x328)[0x55bbaef98ad8]
> /usr/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x92)[0x55bbaefdd362]
> /usr/bin/traffic_server(_ZN7EThread7executeEv+0x679)[0x55bbaefde079]
> /usr/bin/traffic_server(+0x3aae49)[0x55bbaefdce49]
> /lib/x86_64-linux-gnu/libpthread.so.0(+0x76fa)[0x2b45c2fe76fa]
> /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x2b45c3da8b5d]
> {noformat}
> We've not found a way to reproduce the crash. Here are the diags.log entries that show when the server starts back up (indicating a crash and recovery):
> {noformat}
> [Aug 11 07:23:46.245] Server {0x2afc9cf458c0} NOTE: <Main.cc:1902 (main)> traffic server running
> [Aug 11 11:14:10.272] Server {0x2ba545a318c0} NOTE: <Main.cc:1902 (main)> traffic server running
> [Aug 11 12:08:51.188] Server {0x2b887a0228c0} NOTE: <Main.cc:1902 (main)> traffic server running
> [Aug 11 14:57:23.900] Server {0x2aad8422b8c0} NOTE: <Main.cc:1902 (main)> traffic server running
> [Aug 11 14:58:17.844] Server {0x2b66aad508c0} NOTE: <Main.cc:1902 (main)> traffic server running
> [Aug 11 15:08:38.752] Server {0x2b09a50cb8c0} NOTE: <Main.cc:1902 (main)> traffic server running
> [Aug 11 15:08:51.001] Server {0x2aee8a7c88c0} NOTE: <Main.cc:1902 (main)> traffic server running
> [Aug 11 15:10:46.796] Server {0x2ace4544e8c0} NOTE: <Main.cc:1902 (main)> traffic server running
> [Aug 11 15:25:27.599] Server {0x2b90b94788c0} NOTE: <Main.cc:1902 (main)> traffic server running
> [Aug 11 15:26:13.642] Server {0x2b814d2088c0} NOTE: <Main.cc:1902 (main)> traffic server running
> [Aug 11 16:03:31.729] Server {0x2b259ff968c0} NOTE: <Main.cc:1902 (main)> traffic server running
> [Aug 11 16:03:44.057] Server {0x2b1cf99198c0} NOTE: <Main.cc:1902 (main)> traffic server running
> [Aug 11 16:15:27.307] Server {0x2b2333b1d8c0} NOTE: <Main.cc:1902 (main)> traffic server running
> [Aug 11 16:39:56.205] Server {0x2b92f018f8c0} NOTE: <Main.cc:1902 (main)> traffic server running
> [Aug 11 16:40:07.535] Server {0x2aad138ab8c0} NOTE: <Main.cc:1902 (main)> traffic server running
> [Aug 11 16:40:17.971] Server {0x2b94640748c0} NOTE: <Main.cc:1902 (main)> traffic server running
> [Aug 11 16:41:07.280] Server {0x2b45c1a798c0} NOTE: <Main.cc:1902 (main)> traffic server running
> [Aug 11 17:01:28.328] Server {0x2adec40e68c0} NOTE: <Main.cc:1902 (main)> traffic server running
> [Aug 11 17:06:17.853] Server {0x2aef987758c0} NOTE: <Main.cc:1902 (main)> traffic server running
> {noformat}
> {noformat}
> root@gateway-us-east-1:/var/log/trafficserver$ traffic_server -V
> traffic_server: using root directory '/usr'
> Apache Traffic Server - traffic_server - 6.2.0 - (build # 080418 on Aug 4 2016 at 18:36:55)
> {noformat}
> {noformat}
> root@gateway-us-east-1:~$ cat /usr/lib/os-release
> NAME="Ubuntu"
> VERSION="16.04.1 LTS (Xenial Xerus)"
> ID=ubuntu
> ID_LIKE=debian
> PRETTY_NAME="Ubuntu 16.04.1 LTS"
> VERSION_ID="16.04"
> HOME_URL="http://www.ubuntu.com/"
> SUPPORT_URL="http://help.ubuntu.com/"
> BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
> UBUNTU_CODENAME=xenial
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)