You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@trafficserver.apache.org by "Spidey01 (via GitHub)" <gi...@apache.org> on 2023/10/03 15:37:30 UTC
Re: [I] transaction timeouts with proxy.config.http.cache.open_write_fail_action set to '5' [trafficserver]
Spidey01 commented on issue #8614:
URL: https://github.com/apache/trafficserver/issues/8614#issuecomment-1745232464
This morning, I tried reproducing this with ATS 9.2.1 and it just works as expected - issue does not reproduce.
```
==> /var/log/trafficserver/access.log <==
{"localtimestamp": "03/Oct/2023:11:16:00 -0400", "client_ip": "10.211.55.2", "cache_response": "TCP_MISS", "cache_response_code": "200", "request_page": "http://cache-vm/MHawk/rolling-buffer/itcrowd/index.m3u8", "server_response": "200", "bytes": 158, "user_agent": "thundering-herd.sh/horse2", "sessionId": "01_2329856082", "origin_response_time": 2, "client_response_time": 56, "ssl_connection": 0, "referrer": "-"}
{"localtimestamp": "03/Oct/2023:11:16:00 -0400", "client_ip": "10.211.55.2", "cache_response": "TCP_MISS", "cache_response_code": "200", "request_page": "http://cache-vm/MHawk/rolling-buffer/itcrowd/index.m3u8", "server_response": "200", "bytes": 158, "user_agent": "thundering-herd.sh/horse1", "sessionId": "01_2329856081", "origin_response_time": 1, "client_response_time": 68, "ssl_connection": 0, "referrer": "-"}
{"localtimestamp": "03/Oct/2023:11:16:00 -0400", "client_ip": "10.211.55.2", "cache_response": "TCP_HIT", "cache_response_code": "200", "request_page": "http://cache-vm/MHawk/rolling-buffer/itcrowd/index.m3u8", "server_response": "000", "bytes": 158, "user_agent": "thundering-herd.sh/horse4", "sessionId": "01_2329856084", "origin_response_time": -1, "client_response_time": 145, "ssl_connection": 0, "referrer": "-"}
{"localtimestamp": "03/Oct/2023:11:16:00 -0400", "client_ip": "10.211.55.2", "cache_response": "TCP_MEM_HIT", "cache_response_code": "200", "request_page": "http://cache-vm/MHawk/rolling-buffer/itcrowd/index.m3u8", "server_response": "000", "bytes": 158, "user_agent": "thundering-herd.sh/horse3", "sessionId": "01_2329856083", "origin_response_time": -1, "client_response_time": 194, "ssl_connection": 0, "referrer": "-"}
```
I'm not sure if this is related to bug fixes between 9.1.3 and 9.2.1, or changes to other timeout settings. My VM's read-while writer related settings haven't changed since my previous comment, but my timeout settings relative to the default have since been simplified.
```
grep timeout records.config
CONFIG proxy.config.http.connect_attempts_timeout INT 1
CONFIG proxy.config.http.keep_alive_no_activity_timeout_in INT 115
CONFIG proxy.config.http.transaction_active_timeout_in INT 40
CONFIG proxy.config.http.transaction_active_timeout_out INT 2
CONFIG proxy.config.http.transaction_no_activity_timeout_out INT 2
CONFIG proxy.config.http.background_fill_active_timeout INT 0
```
For reference, here's an updated copy of thundering-herd.sh, not that it should make a difference in reproducibility.
```sh
#!/bin/sh
default_rate=""
rate=$default_rate
lead_rate="$rate"
default_maxtime=9999999
maxtime=$default_maxtime
lead_maxtime=$maxtime
output=""
lead_output=$output
stride=0
usage() {
echo "$(basename $0) [-m MAXTIME | -M MAXTIME] [-r RATE | -R RATE] URL SIZE INTERVAL"
echo
echo " -m MAXTIME Set the curl --max-time for each horse. Default is ${maxtime}."
echo " -M MAXTIME Set the curl --max-time for lead horse (overrides -m)."
echo " -r RATE Set a curl --limit-rate for each horse. Default is not add --limit-rate."
echo " -R RATE Set a curl --limit-rate for lead horse (overrides -r)."
echo " -o OUTPUT Set a curl --output for each horse. The horse number will be appended to OUTPUT."
echo " -O OUTPUT Set a curl --output for lead horse. The horse number will be appended to OUTPUT."
echo " -s STRIDE Modify every horse settings by stride."
echo " E.g., -s 3 => every 3rd horse gets -r, -m; rest get defaults."
echo " Note that -o is always set, because curl treats binary data to terminal as an abort."
echo
echo "*WARNING* the lead horse in this context is client one. Which may NOT be the connection that the server chooses as the lead horse due to other constraints such as thread scheduling, TCP setup, etc"
echo
echo "URL: the URL to hit from each client."
echo "SIZE: herd size, how many clients."
echo "INTERVAL: stagger horses by sleep INTERVAL"
}
while getopts "r:R:m:M:o:O:s:h" arg; do
case "$arg" in
m)
maxtime=$OPTARG
;;
M)
lead_maxtime=$OPTARG
;;
r)
rate=$OPTARG
;;
R)
lead_rate=$OPTARG
;;
o)
output=$OPTARG
;;
O)
lead_output=$OPTARG
;;
s)
stride=$OPTARG
;;
h)
usage
exit
;;
\?)
[ -n "$OPTARG" ] && echo "Invalid option: -$OPTARG"
exit $OPTERR
;;
esac
done
shift $(($OPTIND - 1))
url="$1"
size="$2"
interval="$3"
echo "url: $url size: $size interval: $interval maxtime: $maxtime"
[ -z $url -o -z $size ] && echo "must specify url and size" && usage && exit
i=0
while [ $i -lt $size ]; do
i=$(($i + 1))
echo "start client $i"
target_maxtime=
target_rate=
[ -n "$output" ] && target_output="--output $output"
if [ $stride -eq 0 ]; then
# Apply to every horse.
[ -n "$maxtime" ] && target_maxtime="--max-time $maxtime"
[ -n "$rate" ] && target_rate="--limit-rate $rate"
elif [ $stride -gt 0 ]; then
if [ $(($i % $stride)) -eq 0 ]; then
# Apply every stride'th horse.
echo "$stride 'th horse"
[ -n "$maxtime" ] && target_maxtime="--max-time $maxtime"
[ -n "$rate" ] && target_rate="--limit-rate $rate"
else
target_maxtime=""
target_rate=""
fi
fi
[ -n "$lead_maxtime" -a $i -eq 1 ] && target_maxtime="--max-time $lead_maxtime"
[ -n "$lead_rate" -a $i -eq 1 ] && target_rate="--limit-rate $lead_rate"
[ -n "$lead_output" -a $i -eq 1 ] && target_output="--output $lead_output"
user_agent="-A thundering-herd.sh/horse$i"
echo "horse $i target_maxtime: $target_maxtime target_rate: $target_rate stride: $stride user_agent: $user_agent"
# continue
case $(uname -s) in
Darwin)
# macOS uses BSD style script.
script /tmp/curl.$i.out curl $target_output $target_maxtime $target_rate $user_agent -v "$url" -L &
;;
Linux)
# Assume util-linux style script.
script -c "curl $target_output $target_maxtime $target_rate $user_agent -v '$url' -L" /tmp/curl.$i.out &
;;
esac
[ -n "$interval" ] && sleep $interval
done
# Be nice to terminal.
wait
echo
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficserver.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org