You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@trafficserver.apache.org by Susan Hinrichs <sh...@oath.com> on 2018/11/16 14:25:21 UTC
Re: Traffic server suddenly drop the packets and crashed
Are you exhausting ephemeral ports? What is the output of "ss -s" on your
Traffic Server machine? That is quite easy exhaust all of the ephemeral
ports for a single load testing client machine between active and time-wait
connections.
Is Traffic Server generating a core or a stack trace? What version of
Traffic Server?
On Fri, Nov 16, 2018 at 1:42 AM Vasanth Mathivanan <
vasanth.m@evolutiondigital.com> wrote:
> Hi ,
>
> We are testing 1000 client access one by one load test after 700 clients
> dropping the packets like “connection timeout” or “Connection reset by
> peer ” errors .
>
>
>
> Ram and Cpus resources are everything is fine .Ethernet speed also 20gb/s
> both client and traffic server . Origin server has 2gb/s speed between
> ATS and Origin server .While running 700 requests in origin server content
> will be receiving speed less than 10Mb/s then traffic server and client
> getting more than 1Gb/s speed .
>
>
>
> We are also monitoring that resources everything running fine but crashed
> the traffic server after 700 requests .
>
>
>
> Note: We are created Java player , We will give the no of requests its
> call the particular URL based on given requests .
>
>
>
> __Vasanth
>
>
>
> Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for
> Windows 10
>
>
>
Re: Traffic server suddenly drop the packets and crashed
Posted by Susan Hinrichs <sh...@oath.com>.
You probably want to extend your ip_local_port range to 1025 to 65535. I
assume you have an application that requires the reserved ports.
When you run your test, what is the output of "ss -s" ? Is the number of
ports in TIME_WAIT really high?
On Fri, Nov 16, 2018 at 8:43 AM Vasanth Mathivanan <
vasanth.m@evolutiondigital.com> wrote:
> Yes , its generating core dump file in traffic server root path and we
> are using ATS.6.2 in centos 6.5.
>
> traffic.out logs
>
> *“ATAL: InkAPI.cc:1879: failed assert `sdk_sanity_check_mbuffer(bufp) ==
> TS_SUCCESS`*
>
> *traffic_server: using root directory '/opt/trafficserver'*
>
> *traffic_server: received signal 6 (Aborted)*
>
> *traffic_server - STACK TRACE: *
>
>
> */opt/trafficserver/bin/traffic_server(_Z19crash_logger_invokeiP7siginfoPv+0x8e)[0x4ac5ee]*
>
> */lib64/libpthread.so.0[0x3030e0f7e0]*
>
> */lib64/libc.so.6(gsignal+0x35)[0x3030a32495]*
>
> */lib64/libc.so.6(abort+0x175)[0x3030a33c75]*
>
> */opt/trafficserver/lib/libtsutil.so.6(+0x28228)[0x2b5cb60dc228]*
>
> */opt/trafficserver/lib/libtsutil.so.6(+0x282bc)[0x2b5cb60dc2bc]*
>
> */opt/trafficserver/lib/libtsutil.so.6(+0x26595)[0x2b5cb60da595]*
>
> */opt/trafficserver/bin/traffic_server(TSHandleMLocRelease+0x3a)[0x4c33ca]*
>
>
> */opt/trafficserver/libexec/trafficserver/stale_while_revalidate.so(+0x38c9)[0x2b5e9eaa48c9]*
>
>
> */opt/trafficserver/bin/traffic_server(_ZN15INKContInternal12handle_eventEiPv+0xdb)[0x4c12ab]*
>
>
> */opt/trafficserver/bin/traffic_server(_ZN6HttpSM17state_api_calloutEiPv+0x1b7)[0x5ac967]*
>
>
> */opt/trafficserver/bin/traffic_server(_ZN6HttpSM14set_next_stateEv+0x65b)[0x5b226b]*
>
>
> */opt/trafficserver/bin/traffic_server(_ZN6HttpSM22state_cache_open_writeEiPv+0x20e)[0x5a3cde]*
>
>
> */opt/trafficserver/bin/traffic_server(_ZN6HttpSM12main_handlerEiPv+0xca)[0x5b3f3a]*
>
>
> */opt/trafficserver/bin/traffic_server(_ZN11HttpCacheSM22state_cache_open_writeEiPv+0x281)[0x58a481]*
>
>
> */opt/trafficserver/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x120)[0x766f00]*
>
>
> */opt/trafficserver/bin/traffic_server(_ZN7EThread7executeEv+0x811)[0x767bd1]*
>
> */opt/trafficserver/bin/traffic_server[0x76698a]*
>
> */lib64/libpthread.so.0[0x3030e07aa1]*
>
> */lib64/libc.so.6(clone+0x6d)[0x3030ae8bdd]”*
>
>
>
> No I did not set ephemeral ports .
>
>
> Had default values already .
>
> net.ipv4.ip_local_port_range = 32768 60999
>
> net.ipv4.ip_local_reserved_ports = ###this one is empty
>
>
>
> So its set by manually .
>
> net.ipv4.ip_local_reserved_ports = 30000-31000
>
>
>
> Is this ok ? .
>
>
>
>
> __Vasanth
>
> Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for
> Windows 10
>
>
>
> *From: *Susan Hinrichs <sh...@oath.com>
> *Sent: *Friday, November 16, 2018 7:55 PM
> *To: *users@trafficserver.apache.org
> *Subject: *Re: Traffic server suddenly drop the packets and crashed
>
>
> Are you exhausting ephemeral ports? What is the output of "ss -s" on
> your Traffic Server machine? That is quite easy exhaust all of the
> ephemeral ports for a single load testing client machine between active and
> time-wait connections.
>
> Is Traffic Server generating a core or a stack trace? What version of
> Traffic Server?
>
> On Fri, Nov 16, 2018 at 1:42 AM Vasanth Mathivanan <
> vasanth.m@evolutiondigital.com> wrote:
>
>> Hi ,
>>
>> We are testing 1000 client access one by one load test after 700 clients
>> dropping the packets like “connection timeout” or “Connection reset by
>> peer ” errors .
>>
>>
>>
>> Ram and Cpus resources are everything is fine .Ethernet speed also 20gb/s
>> both client and traffic server . Origin server has 2gb/s speed between
>> ATS and Origin server .While running 700 requests in origin server content
>> will be receiving speed less than 10Mb/s then traffic server and client
>> getting more than 1Gb/s speed .
>>
>>
>>
>> We are also monitoring that resources everything running fine but crashed
>> the traffic server after 700 requests .
>>
>>
>>
>> Note: We are created Java player , We will give the no of requests its
>> call the particular URL based on given requests .
>>
>>
>>
>> __Vasanth
>>
>>
>>
>> Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for
>> Windows 10
>>
>>
>>
>
Re: Traffic server suddenly drop the packets and crashed
Posted by Vasanth Mathivanan <va...@evolutiondigital.com>.
@Alan that plugin is enabled but that config file created as empty there is no parameters mentioned.
@Leif Collapsed forwarding enabled in remap config .Do you have new version of ATS for red hat 6 (el6) ? If you have could you pls share ?
@susan Ok I will change the local port range .i checked while running the TIME_WAIT connections are more than 12000
__Vasanth
Sent from my iPhone
On 17-Nov-2018, at 6:31 AM, Leif Hedstrom <zw...@apache.org>> wrote:
Yes, I’d highly recommend against using this plugin. Not only is it broken in many ways, it’s no longer supported in new versions of ATS. And, it really doesn’t do you much good anyways, I’d look at the collapsed forwarding plugin(s) instead.
Cheers,
— Leif
On Nov 17, 2018, at 1:01 AM, Alan Carroll <so...@oath.com>> wrote:
Looks like a problem with the "stale-while-revalidate" plugin. I know that's been problematic. I'm not sure what the current state is.
On Fri, Nov 16, 2018 at 8:43 AM Vasanth Mathivanan <va...@evolutiondigital.com>> wrote:
Yes , its generating core dump file in traffic server root path and we are using ATS.6.2 in centos 6.5.
traffic.out logs
“ATAL: InkAPI.cc:1879<http://inkapi.cc:1879>: failed assert `sdk_sanity_check_mbuffer(bufp) == TS_SUCCESS`
traffic_server: using root directory '/opt/trafficserver'
traffic_server: received signal 6 (Aborted)
traffic_server - STACK TRACE:
/opt/trafficserver/bin/traffic_server(_Z19crash_logger_invokeiP7siginfoPv+0x8e)[0x4ac5ee]
/lib64/libpthread.so.0[0x3030e0f7e0]
/lib64/libc.so.6(gsignal+0x35)[0x3030a32495]
/lib64/libc.so.6(abort+0x175)[0x3030a33c75]
/opt/trafficserver/lib/libtsutil.so.6(+0x28228)[0x2b5cb60dc228]
/opt/trafficserver/lib/libtsutil.so.6(+0x282bc)[0x2b5cb60dc2bc]
/opt/trafficserver/lib/libtsutil.so.6(+0x26595)[0x2b5cb60da595]
/opt/trafficserver/bin/traffic_server(TSHandleMLocRelease+0x3a)[0x4c33ca]
/opt/trafficserver/libexec/trafficserver/stale_while_revalidate.so(+0x38c9)[0x2b5e9eaa48c9]
/opt/trafficserver/bin/traffic_server(_ZN15INKContInternal12handle_eventEiPv+0xdb)[0x4c12ab]
/opt/trafficserver/bin/traffic_server(_ZN6HttpSM17state_api_calloutEiPv+0x1b7)[0x5ac967]
/opt/trafficserver/bin/traffic_server(_ZN6HttpSM14set_next_stateEv+0x65b)[0x5b226b]
/opt/trafficserver/bin/traffic_server(_ZN6HttpSM22state_cache_open_writeEiPv+0x20e)[0x5a3cde]
/opt/trafficserver/bin/traffic_server(_ZN6HttpSM12main_handlerEiPv+0xca)[0x5b3f3a]
/opt/trafficserver/bin/traffic_server(_ZN11HttpCacheSM22state_cache_open_writeEiPv+0x281)[0x58a481]
/opt/trafficserver/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x120)[0x766f00]
/opt/trafficserver/bin/traffic_server(_ZN7EThread7executeEv+0x811)[0x767bd1]
/opt/trafficserver/bin/traffic_server[0x76698a]
/lib64/libpthread.so.0[0x3030e07aa1]
/lib64/libc.so.6(clone+0x6d)[0x3030ae8bdd]”
No I did not set ephemeral ports .
Had default values already .
net.ipv4.ip_local_port_range = 32768 60999
net.ipv4.ip_local_reserved_ports = ###this one is empty
So its set by manually .
net.ipv4.ip_local_reserved_ports = 30000-31000
Is this ok ? .
__Vasanth
Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10
From: Susan Hinrichs<ma...@oath.com>
Sent: Friday, November 16, 2018 7:55 PM
To: users@trafficserver.apache.org<ma...@trafficserver.apache.org>
Subject: Re: Traffic server suddenly drop the packets and crashed
Are you exhausting ephemeral ports? What is the output of "ss -s" on your Traffic Server machine? That is quite easy exhaust all of the ephemeral ports for a single load testing client machine between active and time-wait connections.
Is Traffic Server generating a core or a stack trace? What version of Traffic Server?
On Fri, Nov 16, 2018 at 1:42 AM Vasanth Mathivanan <va...@evolutiondigital.com>> wrote:
Hi ,
We are testing 1000 client access one by one load test after 700 clients dropping the packets like “connection timeout” or “Connection reset by peer ” errors .
Ram and Cpus resources are everything is fine .Ethernet speed also 20gb/s both client and traffic server . Origin server has 2gb/s speed between ATS and Origin server .While running 700 requests in origin server content will be receiving speed less than 10Mb/s then traffic server and client getting more than 1Gb/s speed .
We are also monitoring that resources everything running fine but crashed the traffic server after 700 requests .
Note: We are created Java player , We will give the no of requests its call the particular URL based on given requests .
__Vasanth
Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10
--
Beware the fisherman who's casting out his line in to a dried up riverbed.
Oh don't try to tell him 'cause he won't believe. Throw some bread to the ducks instead.
It's easier that way. - Genesis : Duke : VI 25-28
Re: Traffic server suddenly drop the packets and crashed
Posted by Leif Hedstrom <zw...@apache.org>.
Yes, I’d highly recommend against using this plugin. Not only is it broken in many ways, it’s no longer supported in new versions of ATS. And, it really doesn’t do you much good anyways, I’d look at the collapsed forwarding plugin(s) instead.
Cheers,
— Leif
> On Nov 17, 2018, at 1:01 AM, Alan Carroll <so...@oath.com> wrote:
>
> Looks like a problem with the "stale-while-revalidate" plugin. I know that's been problematic. I'm not sure what the current state is.
>
>> On Fri, Nov 16, 2018 at 8:43 AM Vasanth Mathivanan <va...@evolutiondigital.com> wrote:
>> Yes , its generating core dump file in traffic server root path and we are using ATS.6.2 in centos 6.5.
>>
>> traffic.out logs
>>
>> “ATAL: InkAPI.cc:1879: failed assert `sdk_sanity_check_mbuffer(bufp) == TS_SUCCESS`
>>
>> traffic_server: using root directory '/opt/trafficserver'
>>
>> traffic_server: received signal 6 (Aborted)
>>
>> traffic_server - STACK TRACE:
>>
>> /opt/trafficserver/bin/traffic_server(_Z19crash_logger_invokeiP7siginfoPv+0x8e)[0x4ac5ee]
>>
>> /lib64/libpthread.so.0[0x3030e0f7e0]
>>
>> /lib64/libc.so.6(gsignal+0x35)[0x3030a32495]
>>
>> /lib64/libc.so.6(abort+0x175)[0x3030a33c75]
>>
>> /opt/trafficserver/lib/libtsutil.so.6(+0x28228)[0x2b5cb60dc228]
>>
>> /opt/trafficserver/lib/libtsutil.so.6(+0x282bc)[0x2b5cb60dc2bc]
>>
>> /opt/trafficserver/lib/libtsutil.so.6(+0x26595)[0x2b5cb60da595]
>>
>> /opt/trafficserver/bin/traffic_server(TSHandleMLocRelease+0x3a)[0x4c33ca]
>>
>> /opt/trafficserver/libexec/trafficserver/stale_while_revalidate.so(+0x38c9)[0x2b5e9eaa48c9]
>>
>> /opt/trafficserver/bin/traffic_server(_ZN15INKContInternal12handle_eventEiPv+0xdb)[0x4c12ab]
>>
>> /opt/trafficserver/bin/traffic_server(_ZN6HttpSM17state_api_calloutEiPv+0x1b7)[0x5ac967]
>>
>> /opt/trafficserver/bin/traffic_server(_ZN6HttpSM14set_next_stateEv+0x65b)[0x5b226b]
>>
>> /opt/trafficserver/bin/traffic_server(_ZN6HttpSM22state_cache_open_writeEiPv+0x20e)[0x5a3cde]
>>
>> /opt/trafficserver/bin/traffic_server(_ZN6HttpSM12main_handlerEiPv+0xca)[0x5b3f3a]
>>
>> /opt/trafficserver/bin/traffic_server(_ZN11HttpCacheSM22state_cache_open_writeEiPv+0x281)[0x58a481]
>>
>> /opt/trafficserver/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x120)[0x766f00]
>>
>> /opt/trafficserver/bin/traffic_server(_ZN7EThread7executeEv+0x811)[0x767bd1]
>>
>> /opt/trafficserver/bin/traffic_server[0x76698a]
>>
>> /lib64/libpthread.so.0[0x3030e07aa1]
>>
>> /lib64/libc.so.6(clone+0x6d)[0x3030ae8bdd]”
>>
>>
>>
>> No I did not set ephemeral ports .
>>
>>
>> Had default values already .
>>
>> net.ipv4.ip_local_port_range = 32768 60999
>>
>> net.ipv4.ip_local_reserved_ports = ###this one is empty
>>
>>
>>
>> So its set by manually .
>>
>> net.ipv4.ip_local_reserved_ports = 30000-31000
>>
>>
>>
>> Is this ok ? .
>>
>>
>>
>>
>> __Vasanth
>>
>> Sent from Mail for Windows 10
>>
>>
>>
>> From: Susan Hinrichs
>> Sent: Friday, November 16, 2018 7:55 PM
>> To: users@trafficserver.apache.org
>> Subject: Re: Traffic server suddenly drop the packets and crashed
>>
>>
>>
>> Are you exhausting ephemeral ports? What is the output of "ss -s" on your Traffic Server machine? That is quite easy exhaust all of the ephemeral ports for a single load testing client machine between active and time-wait connections.
>>
>> Is Traffic Server generating a core or a stack trace? What version of Traffic Server?
>>
>>> On Fri, Nov 16, 2018 at 1:42 AM Vasanth Mathivanan <va...@evolutiondigital.com> wrote:
>>> Hi ,
>>>
>>> We are testing 1000 client access one by one load test after 700 clients dropping the packets like “connection timeout” or “Connection reset by peer ” errors .
>>>
>>>
>>>
>>> Ram and Cpus resources are everything is fine .Ethernet speed also 20gb/s both client and traffic server . Origin server has 2gb/s speed between ATS and Origin server .While running 700 requests in origin server content will be receiving speed less than 10Mb/s then traffic server and client getting more than 1Gb/s speed .
>>>
>>>
>>>
>>> We are also monitoring that resources everything running fine but crashed the traffic server after 700 requests .
>>>
>>>
>>>
>>> Note: We are created Java player , We will give the no of requests its call the particular URL based on given requests .
>>>
>>>
>>>
>>> __Vasanth
>>>
>>>
>>>
>>> Sent from Mail for Windows 10
>>>
>>>
>>>
>
>
> --
> Beware the fisherman who's casting out his line in to a dried up riverbed.
> Oh don't try to tell him 'cause he won't believe. Throw some bread to the ducks instead.
> It's easier that way. - Genesis : Duke : VI 25-28
Re: Traffic server suddenly drop the packets and crashed
Posted by Alan Carroll <so...@oath.com>.
Looks like a problem with the "stale-while-revalidate" plugin. I know
that's been problematic. I'm not sure what the current state is.
On Fri, Nov 16, 2018 at 8:43 AM Vasanth Mathivanan <
vasanth.m@evolutiondigital.com> wrote:
> Yes , its generating core dump file in traffic server root path and we
> are using ATS.6.2 in centos 6.5.
>
> traffic.out logs
>
> *“ATAL: InkAPI.cc:1879: failed assert `sdk_sanity_check_mbuffer(bufp) ==
> TS_SUCCESS`*
>
> *traffic_server: using root directory '/opt/trafficserver'*
>
> *traffic_server: received signal 6 (Aborted)*
>
> *traffic_server - STACK TRACE: *
>
>
> */opt/trafficserver/bin/traffic_server(_Z19crash_logger_invokeiP7siginfoPv+0x8e)[0x4ac5ee]*
>
> */lib64/libpthread.so.0[0x3030e0f7e0]*
>
> */lib64/libc.so.6(gsignal+0x35)[0x3030a32495]*
>
> */lib64/libc.so.6(abort+0x175)[0x3030a33c75]*
>
> */opt/trafficserver/lib/libtsutil.so.6(+0x28228)[0x2b5cb60dc228]*
>
> */opt/trafficserver/lib/libtsutil.so.6(+0x282bc)[0x2b5cb60dc2bc]*
>
> */opt/trafficserver/lib/libtsutil.so.6(+0x26595)[0x2b5cb60da595]*
>
> */opt/trafficserver/bin/traffic_server(TSHandleMLocRelease+0x3a)[0x4c33ca]*
>
>
> */opt/trafficserver/libexec/trafficserver/stale_while_revalidate.so(+0x38c9)[0x2b5e9eaa48c9]*
>
>
> */opt/trafficserver/bin/traffic_server(_ZN15INKContInternal12handle_eventEiPv+0xdb)[0x4c12ab]*
>
>
> */opt/trafficserver/bin/traffic_server(_ZN6HttpSM17state_api_calloutEiPv+0x1b7)[0x5ac967]*
>
>
> */opt/trafficserver/bin/traffic_server(_ZN6HttpSM14set_next_stateEv+0x65b)[0x5b226b]*
>
>
> */opt/trafficserver/bin/traffic_server(_ZN6HttpSM22state_cache_open_writeEiPv+0x20e)[0x5a3cde]*
>
>
> */opt/trafficserver/bin/traffic_server(_ZN6HttpSM12main_handlerEiPv+0xca)[0x5b3f3a]*
>
>
> */opt/trafficserver/bin/traffic_server(_ZN11HttpCacheSM22state_cache_open_writeEiPv+0x281)[0x58a481]*
>
>
> */opt/trafficserver/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x120)[0x766f00]*
>
>
> */opt/trafficserver/bin/traffic_server(_ZN7EThread7executeEv+0x811)[0x767bd1]*
>
> */opt/trafficserver/bin/traffic_server[0x76698a]*
>
> */lib64/libpthread.so.0[0x3030e07aa1]*
>
> */lib64/libc.so.6(clone+0x6d)[0x3030ae8bdd]”*
>
>
>
> No I did not set ephemeral ports .
>
>
> Had default values already .
>
> net.ipv4.ip_local_port_range = 32768 60999
>
> net.ipv4.ip_local_reserved_ports = ###this one is empty
>
>
>
> So its set by manually .
>
> net.ipv4.ip_local_reserved_ports = 30000-31000
>
>
>
> Is this ok ? .
>
>
>
>
> __Vasanth
>
> Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for
> Windows 10
>
>
>
> *From: *Susan Hinrichs <sh...@oath.com>
> *Sent: *Friday, November 16, 2018 7:55 PM
> *To: *users@trafficserver.apache.org
> *Subject: *Re: Traffic server suddenly drop the packets and crashed
>
>
> Are you exhausting ephemeral ports? What is the output of "ss -s" on
> your Traffic Server machine? That is quite easy exhaust all of the
> ephemeral ports for a single load testing client machine between active and
> time-wait connections.
>
> Is Traffic Server generating a core or a stack trace? What version of
> Traffic Server?
>
> On Fri, Nov 16, 2018 at 1:42 AM Vasanth Mathivanan <
> vasanth.m@evolutiondigital.com> wrote:
>
>> Hi ,
>>
>> We are testing 1000 client access one by one load test after 700 clients
>> dropping the packets like “connection timeout” or “Connection reset by
>> peer ” errors .
>>
>>
>>
>> Ram and Cpus resources are everything is fine .Ethernet speed also 20gb/s
>> both client and traffic server . Origin server has 2gb/s speed between
>> ATS and Origin server .While running 700 requests in origin server content
>> will be receiving speed less than 10Mb/s then traffic server and client
>> getting more than 1Gb/s speed .
>>
>>
>>
>> We are also monitoring that resources everything running fine but crashed
>> the traffic server after 700 requests .
>>
>>
>>
>> Note: We are created Java player , We will give the no of requests its
>> call the particular URL based on given requests .
>>
>>
>>
>> __Vasanth
>>
>>
>>
>> Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for
>> Windows 10
>>
>>
>>
>
--
*Beware the fisherman who's casting out his line in to a dried up riverbed.*
*Oh don't try to tell him 'cause he won't believe. Throw some bread to the
ducks instead.*
*It's easier that way. *- Genesis : Duke : VI 25-28
RE: Traffic server suddenly drop the packets and crashed
Posted by Vasanth Mathivanan <va...@evolutiondigital.com>.
Yes , its generating core dump file in traffic server root path and we are using ATS.6.2 in centos 6.5.
traffic.out logs
“ATAL: InkAPI.cc:1879: failed assert `sdk_sanity_check_mbuffer(bufp) == TS_SUCCESS`
traffic_server: using root directory '/opt/trafficserver'
traffic_server: received signal 6 (Aborted)
traffic_server - STACK TRACE:
/opt/trafficserver/bin/traffic_server(_Z19crash_logger_invokeiP7siginfoPv+0x8e)[0x4ac5ee]
/lib64/libpthread.so.0[0x3030e0f7e0]
/lib64/libc.so.6(gsignal+0x35)[0x3030a32495]
/lib64/libc.so.6(abort+0x175)[0x3030a33c75]
/opt/trafficserver/lib/libtsutil.so.6(+0x28228)[0x2b5cb60dc228]
/opt/trafficserver/lib/libtsutil.so.6(+0x282bc)[0x2b5cb60dc2bc]
/opt/trafficserver/lib/libtsutil.so.6(+0x26595)[0x2b5cb60da595]
/opt/trafficserver/bin/traffic_server(TSHandleMLocRelease+0x3a)[0x4c33ca]
/opt/trafficserver/libexec/trafficserver/stale_while_revalidate.so(+0x38c9)[0x2b5e9eaa48c9]
/opt/trafficserver/bin/traffic_server(_ZN15INKContInternal12handle_eventEiPv+0xdb)[0x4c12ab]
/opt/trafficserver/bin/traffic_server(_ZN6HttpSM17state_api_calloutEiPv+0x1b7)[0x5ac967]
/opt/trafficserver/bin/traffic_server(_ZN6HttpSM14set_next_stateEv+0x65b)[0x5b226b]
/opt/trafficserver/bin/traffic_server(_ZN6HttpSM22state_cache_open_writeEiPv+0x20e)[0x5a3cde]
/opt/trafficserver/bin/traffic_server(_ZN6HttpSM12main_handlerEiPv+0xca)[0x5b3f3a]
/opt/trafficserver/bin/traffic_server(_ZN11HttpCacheSM22state_cache_open_writeEiPv+0x281)[0x58a481]
/opt/trafficserver/bin/traffic_server(_ZN7EThread13process_eventEP5Eventi+0x120)[0x766f00]
/opt/trafficserver/bin/traffic_server(_ZN7EThread7executeEv+0x811)[0x767bd1]
/opt/trafficserver/bin/traffic_server[0x76698a]
/lib64/libpthread.so.0[0x3030e07aa1]
/lib64/libc.so.6(clone+0x6d)[0x3030ae8bdd]”
No I did not set ephemeral ports .
Had default values already .
net.ipv4.ip_local_port_range = 32768 60999
net.ipv4.ip_local_reserved_ports = ###this one is empty
So its set by manually .
net.ipv4.ip_local_reserved_ports = 30000-31000
Is this ok ? .
__Vasanth
Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10
From: Susan Hinrichs<ma...@oath.com>
Sent: Friday, November 16, 2018 7:55 PM
To: users@trafficserver.apache.org<ma...@trafficserver.apache.org>
Subject: Re: Traffic server suddenly drop the packets and crashed
Are you exhausting ephemeral ports? What is the output of "ss -s" on your Traffic Server machine? That is quite easy exhaust all of the ephemeral ports for a single load testing client machine between active and time-wait connections.
Is Traffic Server generating a core or a stack trace? What version of Traffic Server?
On Fri, Nov 16, 2018 at 1:42 AM Vasanth Mathivanan <va...@evolutiondigital.com>> wrote:
Hi ,
We are testing 1000 client access one by one load test after 700 clients dropping the packets like “connection timeout” or “Connection reset by peer ” errors .
Ram and Cpus resources are everything is fine .Ethernet speed also 20gb/s both client and traffic server . Origin server has 2gb/s speed between ATS and Origin server .While running 700 requests in origin server content will be receiving speed less than 10Mb/s then traffic server and client getting more than 1Gb/s speed .
We are also monitoring that resources everything running fine but crashed the traffic server after 700 requests .
Note: We are created Java player , We will give the no of requests its call the particular URL based on given requests .
__Vasanth
Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10