You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@trafficserver.apache.org by GitBox <gi...@apache.org> on 2021/09/16 18:23:45 UTC
[GitHub] [trafficserver] meeramn opened a new issue #8329: Crash while using next hop strategy
meeramn opened a new issue #8329:
URL: https://github.com/apache/trafficserver/issues/8329
Strategies yaml config got loaded but seeing crash after sending curl request.
Please find the configuration below.
**cat remap.config**
map /nexthop-test/ http://httpbin.org/ @strategy=nexthop-test-strategy
**cat strategies.yaml**
hosts:
- &nexthop-test-host
host: test-host.xyz.com
protocol:
- scheme: http
port: 80
- scheme: https
port: 443
groups:
- &nexthop-test-group
- <<: *nexthop-test-host
weight: 1.5
strategies:
- strategy: 'nexthop-test-strategy'
policy: consistent_hash
hash_key: path
go_direct: false
groups:
- *nexthop-test-group
scheme: http
failover:
ring_mode: alternate_ring
response_codes:
- 404
markdown_codes:
- 503
health_check:
- passive
**Curl**
curl -vvv -o /dev/null http://localhost/nexthop-test/get
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* About to connect() to localhost port 80 (#0)
* Trying ::1...
* Connected to localhost (::1) port 80 (#0)
> GET /nexthop-test/get HTTP/1.1
> User-Agent: curl/7.29.0
> Host: localhost
> Accept: */*
>
0 0 0 0 0 0 0 0 --:--:-- 0:00:34 --:--:-- 0* Empty reply from server
0 0 0 0 0 0 0 0 --:--:-- 0:00:34 --:--:-- 0
* Connection #0 to host localhost left intact
curl: (52) Empty reply from server
**Stack trace**
traffic_server: received signal 11 (Segmentation fault)
traffic_server - STACK TRACE:
/opt/bazinga/bin/traffic_server(_Z19crash_logger_invokeiP9siginfo_tPv+0x94)[0x55edd0cc3524]
/lib64/libpthread.so.0(+0xf630)[0x7f8c4352f630]
/opt/bazinga/lib/trafficserver/libtscore.so.9(_ZN17ATSConsistentHash6lookupEPKcPNSt3__114__map_iteratorINS2_15__tree_iteratorINS2_12__value_typeImP21ATSConsistentHashNodeEEPNS2_11__tree_nodeIS8_PvEElEEEEPbP9ATSHash64+0xf3)[0x7f8c44a177d3]
/opt/bazinga/bin/traffic_server(_ZN21NextHopConsistentHash11findNextHopEP13tsapi_httptxnPvl+0x87f)[0x55edd0dc7e3f]
/opt/bazinga/bin/traffic_server(+0x254617)[0x55edd0d93617]
/opt/bazinga/bin/traffic_server(+0x2387be)[0x55edd0d777be]
/opt/bazinga/bin/traffic_server(_ZN12HttpTransact27handle_response_from_parentEPNS_5StateE+0xbaa)[0x55edd0d85f0a]
/opt/bazinga/bin/traffic_server(_ZN12HttpTransact14HandleResponseEPNS_5StateE+0x4a5)[0x55edd0d7adc5]
/opt/bazinga/bin/traffic_server(_ZN6HttpSM25handle_server_setup_errorEiPv+0x3a1)[0x55edd0d52491]
/opt/bazinga/bin/traffic_server(_ZN6HttpSM33state_read_server_response_headerEiPv+0x540)[0x55edd0d51e90]
/opt/bazinga/bin/traffic_server(_ZN6HttpSM12main_handlerEiPv+0x1ac)[0x55edd0d4099c]
/opt/bazinga/bin/traffic_server(+0x409883)[0x55edd0f48883]
/opt/bazinga/bin/traffic_server(_ZN18UnixNetVConnection11net_read_ioEP10NetHandlerP7EThread+0xa94)[0x55edd0f47574]
/opt/bazinga/bin/traffic_server(_ZN10NetHandler18process_ready_listEv+0xa3)[0x55edd0f38793]
/opt/bazinga/bin/traffic_server(_ZN10NetHandler15waitForActivityEl+0x54e)[0x55edd0f38e6e]
/opt/bazinga/bin/traffic_server(_ZThn64_N10NetHandler15waitForActivityEl+0xd)[0x55edd0f38f3d]
/opt/bazinga/bin/traffic_server(_ZN7EThread15execute_regularEv+0x55f)[0x55edd0f68dff]
/opt/bazinga/bin/traffic_server(_ZN7EThread7executeEv+0x179)[0x55edd0f69079]
/opt/bazinga/bin/traffic_server(+0x428556)[0x55edd0f67556]
/lib64/libpthread.so.0(+0x7ea5)[0x7f8c43527ea5]
/lib64/libc.so.6(clone+0x6d)[0x7f8c42d288dd]
Segmentation fault
[Complete Logs](https://gist.github.com/meeramn/ab4add102e2900df9e81c7344168c332)
[GDB](https://gist.github.com/meeramn/9f5f3dcf02dcbecdda3f5dedd195c447)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficserver.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] randall commented on issue #8329: Crash while using next hop strategy
Posted by GitBox <gi...@apache.org>.
randall commented on issue #8329:
URL: https://github.com/apache/trafficserver/issues/8329#issuecomment-923241999
We build with llvm 12, so maybe a difference in how gcc/llvm handles things.
Full stack of crashed thread:
```
#0 0x00007febf7b221d9 in waitpid () from /lib64/libpthread.so.0
#1 0x00005639c1c44504 in crash_logger_invoke(int, siginfo_t*, void*) (signo=11, info=0x7febeff05b70, ctx=0x7febeff05a40)
at traffic_server/Crash.cc:168
#2 <signal handler called>
#3 __tree_min<std::__tree_node_base<void *> *> (__x=0x600) at /opt/bazinga/include/c++/v1/__tree:147
#4 __tree_next_iter<std::__tree_end_node<std::__tree_node_base<void *> *> *, std::__tree_node_base<void *> *> (
__x=0x7febf5789fb0) at /opt/bazinga/include/c++/v1/__tree:183
#5 operator++ (this=<optimized out>) at /opt/bazinga/include/c++/v1/__tree:842
#6 operator++ (this=<optimized out>) at /opt/bazinga/include/c++/v1/map:804
#7 operator++ (this=<optimized out>) at /opt/bazinga/include/c++/v1/map:809
#8 ATSConsistentHash::lookup(char const*, std::__1::__map_iterator<std::__1::__tree_iterator<std::__1::__value_type<unsigned long, ATSConsistentHashNode*>, std::__1::__tree_node<std::__1::__value_type<unsigned long, ATSConsistentHashNode*>, void*>*, long> >*, bool*, ATSHash64*) (this=0x7febf5789f98, url=0x0, i=0x7feb4fd33f68, w=<optimized out>, h=<optimized out>)
at ConsistentHash.cc:109
#9 0x00005639c1d48e3f in chash_lookup (ring=..., hash_key=298391872744178962, iter=0x7feb4fd33f68,
wrapped=0x7febeff068bf, hash=0x7febeff068c0, hash_init=<optimized out>, mapWrapped=<optimized out>,
sm_id=<optimized out>) at NextHopConsistentHash.cc:48
#10 NextHopConsistentHash::findNextHop(tsapi_httptxn*, void*, long) (this=0x7febf59ff018, txnp=0x7feb4fd33500,
ih=<optimized out>, now=0) at NextHopConsistentHash.cc:337
#11 0x00005639c1d14617 in nextParent(HttpTransact::State*) (s=0x7feb4fd33600) at HttpTransact.cc:249
#12 0x00005639c1cf87be in find_server_and_update_current_info(HttpTransact::State*) (s=0x7feb4fd33600)
at HttpTransact.cc:584
#13 0x00005639c1d06f0a in HttpTransact::handle_response_from_parent(HttpTransact::State*) (s=0x7feb4fd33600)
at HttpTransact.cc:3665
#14 0x00005639c1cfbdc5 in HttpTransact::HandleResponse(HttpTransact::State*) (s=0x7feb4fd33600) at HttpTransact.cc:3462
#15 0x00005639c1cd3491 in call_transact_and_set_next_state (this=0x7feb4fd33500, f=<optimized out>) at HttpSM.cc:7391
#16 HttpSM::handle_server_setup_error(int, void*) (this=0x7feb4fd33500, event=<optimized out>, data=<optimized out>)
at HttpSM.cc:5790
#17 0x00005639c1cd2e90 in HttpSM::state_read_server_response_header(int, void*) (this=0x7feb4fd33500, event=104,
data=0x7febe9742e60) at HttpSM.cc:2063
#18 0x00005639c1cc199c in HttpSM::main_handler(int, void*) (this=0x7feb4fd33500, event=104, data=0x7febe9742e60)
at HttpSM.cc:2715
#19 0x00005639c1ec9883 in handleEvent (this=<optimized out>, event=104, data=0x7feb4fd33f68)
at /usr/src/debug/bazinga-trafficserver-9.1.1/iocore/eventsystem/I_Continuation.h:219
#20 read_signal_and_update(int, UnixNetVConnection*) (event=104, vc=0x7febe9742c80) at UnixNetVConnection.cc:83
#21 0x00005639c1ec8574 in read_signal_done (event=104, nh=0x7febf5211d50, vc=0x7febe9742c80) at UnixNetVConnection.cc:150
#22 read_from_net (nh=<optimized out>, vc=<optimized out>, thread=<optimized out>) at UnixNetVConnection.cc:292
#23 UnixNetVConnection::net_read_io(NetHandler*, EThread*) (this=<optimized out>, nh=0x7febf5211d50,
lthread=<optimized out>) at UnixNetVConnection.cc:921
#24 0x00005639c1eb9793 in NetHandler::process_ready_list() (this=0x7febf5211d50) at UnixNet.cc:415
#25 0x00005639c1eb9e6e in NetHandler::waitForActivity(long) (this=<optimized out>, timeout=<optimized out>)
at UnixNet.cc:552
#26 0x00005639c1eb9f3d in non-virtual thunk to NetHandler::waitForActivity(long) ()
#27 0x00005639c1ee9dff in EThread::execute_regular() (this=0x7febf520dbc0) at UnixEThread.cc:303
#28 0x00005639c1eea079 in EThread::execute() (this=0x7febf520dbc0) at UnixEThread.cc:364
#29 0x00005639c1ee8556 in spawn_thread_internal(void*) (a=0x7febf62519e0) at Thread.cc:92
#30 0x00007febf7b1aea5 in start_thread () from /lib64/libpthread.so.0
#31 0x00007febf731b8dd in clone () from /lib64/libc.so.6
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficserver.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] jrushford commented on issue #8329: Crash while using next hop strategy
Posted by GitBox <gi...@apache.org>.
jrushford commented on issue #8329:
URL: https://github.com/apache/trafficserver/issues/8329#issuecomment-921918237
@meeramn I've tried your exact configuration and curl command on 9.1.x. I've tried the config and am unable to duplicate the crash. I've tried different failure scenarios with the parent, unreachable, trafficserver shutdown, etc and am not able to duplicate the crash. Everything is working as expected.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficserver.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] jrushford closed issue #8329: Crash while using next hop strategy
Posted by GitBox <gi...@apache.org>.
jrushford closed issue #8329:
URL: https://github.com/apache/trafficserver/issues/8329
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficserver.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] randall edited a comment on issue #8329: Crash while using next hop strategy
Posted by GitBox <gi...@apache.org>.
randall edited a comment on issue #8329:
URL: https://github.com/apache/trafficserver/issues/8329#issuecomment-923241999
We build with llvm 12, so maybe a difference in how gcc/llvm handles things.
Full stack of crashed thread:
```
#0 0x00007febf7b221d9 in waitpid () from /lib64/libpthread.so.0
#1 0x00005639c1c44504 in crash_logger_invoke(int, siginfo_t*, void*) (signo=11, info=0x7febeff05b70, ctx=0x7febeff05a40)
at traffic_server/Crash.cc:168
#2 <signal handler called>
#3 __tree_min<std::__tree_node_base<void *> *> (__x=0x600) at /opt/ats/include/c++/v1/__tree:147
#4 __tree_next_iter<std::__tree_end_node<std::__tree_node_base<void *> *> *, std::__tree_node_base<void *> *> (
__x=0x7febf5789fb0) at /opt/ats/include/c++/v1/__tree:183
#5 operator++ (this=<optimized out>) at /opt/ats/include/c++/v1/__tree:842
#6 operator++ (this=<optimized out>) at /opt/ats/include/c++/v1/map:804
#7 operator++ (this=<optimized out>) at /opt/ats/include/c++/v1/map:809
#8 ATSConsistentHash::lookup(char const*, std::__1::__map_iterator<std::__1::__tree_iterator<std::__1::__value_type<unsigned long, ATSConsistentHashNode*>, std::__1::__tree_node<std::__1::__value_type<unsigned long, ATSConsistentHashNode*>, void*>*, long> >*, bool*, ATSHash64*) (this=0x7febf5789f98, url=0x0, i=0x7feb4fd33f68, w=<optimized out>, h=<optimized out>)
at ConsistentHash.cc:109
#9 0x00005639c1d48e3f in chash_lookup (ring=..., hash_key=298391872744178962, iter=0x7feb4fd33f68,
wrapped=0x7febeff068bf, hash=0x7febeff068c0, hash_init=<optimized out>, mapWrapped=<optimized out>,
sm_id=<optimized out>) at NextHopConsistentHash.cc:48
#10 NextHopConsistentHash::findNextHop(tsapi_httptxn*, void*, long) (this=0x7febf59ff018, txnp=0x7feb4fd33500,
ih=<optimized out>, now=0) at NextHopConsistentHash.cc:337
#11 0x00005639c1d14617 in nextParent(HttpTransact::State*) (s=0x7feb4fd33600) at HttpTransact.cc:249
#12 0x00005639c1cf87be in find_server_and_update_current_info(HttpTransact::State*) (s=0x7feb4fd33600)
at HttpTransact.cc:584
#13 0x00005639c1d06f0a in HttpTransact::handle_response_from_parent(HttpTransact::State*) (s=0x7feb4fd33600)
at HttpTransact.cc:3665
#14 0x00005639c1cfbdc5 in HttpTransact::HandleResponse(HttpTransact::State*) (s=0x7feb4fd33600) at HttpTransact.cc:3462
#15 0x00005639c1cd3491 in call_transact_and_set_next_state (this=0x7feb4fd33500, f=<optimized out>) at HttpSM.cc:7391
#16 HttpSM::handle_server_setup_error(int, void*) (this=0x7feb4fd33500, event=<optimized out>, data=<optimized out>)
at HttpSM.cc:5790
#17 0x00005639c1cd2e90 in HttpSM::state_read_server_response_header(int, void*) (this=0x7feb4fd33500, event=104,
data=0x7febe9742e60) at HttpSM.cc:2063
#18 0x00005639c1cc199c in HttpSM::main_handler(int, void*) (this=0x7feb4fd33500, event=104, data=0x7febe9742e60)
at HttpSM.cc:2715
#19 0x00005639c1ec9883 in handleEvent (this=<optimized out>, event=104, data=0x7feb4fd33f68)
at /usr/src/debug/ats-trafficserver-9.1.1/iocore/eventsystem/I_Continuation.h:219
#20 read_signal_and_update(int, UnixNetVConnection*) (event=104, vc=0x7febe9742c80) at UnixNetVConnection.cc:83
#21 0x00005639c1ec8574 in read_signal_done (event=104, nh=0x7febf5211d50, vc=0x7febe9742c80) at UnixNetVConnection.cc:150
#22 read_from_net (nh=<optimized out>, vc=<optimized out>, thread=<optimized out>) at UnixNetVConnection.cc:292
#23 UnixNetVConnection::net_read_io(NetHandler*, EThread*) (this=<optimized out>, nh=0x7febf5211d50,
lthread=<optimized out>) at UnixNetVConnection.cc:921
#24 0x00005639c1eb9793 in NetHandler::process_ready_list() (this=0x7febf5211d50) at UnixNet.cc:415
#25 0x00005639c1eb9e6e in NetHandler::waitForActivity(long) (this=<optimized out>, timeout=<optimized out>)
at UnixNet.cc:552
#26 0x00005639c1eb9f3d in non-virtual thunk to NetHandler::waitForActivity(long) ()
#27 0x00005639c1ee9dff in EThread::execute_regular() (this=0x7febf520dbc0) at UnixEThread.cc:303
#28 0x00005639c1eea079 in EThread::execute() (this=0x7febf520dbc0) at UnixEThread.cc:364
#29 0x00005639c1ee8556 in spawn_thread_internal(void*) (a=0x7febf62519e0) at Thread.cc:92
#30 0x00007febf7b1aea5 in start_thread () from /lib64/libpthread.so.0
#31 0x00007febf731b8dd in clone () from /lib64/libc.so.6
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficserver.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] jrushford commented on issue #8329: Crash while using next hop strategy
Posted by GitBox <gi...@apache.org>.
jrushford commented on issue #8329:
URL: https://github.com/apache/trafficserver/issues/8329#issuecomment-921951522
@meeramn I can re-create the crash when I'm running trafficserver on macOS. I cannot duplicate it on a linux build with devtoolset-9
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficserver.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] jrushford closed issue #8329: Crash while using next hop strategy
Posted by GitBox <gi...@apache.org>.
jrushford closed issue #8329:
URL: https://github.com/apache/trafficserver/issues/8329
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficserver.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org