[I] rate_limit_sni_expiry AuTest is flaky: the holder races FIFO removal [trafficserver]

["moonchen (via GitHub)" <gi...@apache.org>]

moonchen opened a new issue, #13714:
URL: https://github.com/apache/trafficserver/issues/13714

   ## Summary
   
   `rate_limit_sni_expiry` can fail before exercising queue expiry: the helper removes the holder's FIFO before the background process opens it. Without the holder occupying the single slot, the next connection is admitted immediately and the expected `too old` message never appears.
   
   ## Observed
   
   [AuTest build 43964](https://ci.trafficserver.apache.org/job/Github_Builds/job/autest/43964/consoleFull), shard 2of4 for #13712 at `5168392148496eb83305e61fffcab3edbcf68e21`, finished with 143 passed, 1 failed, and 5 skipped:
   
   ```text
   file .../ts/log/traffic.out : a queued connection was expired - Failed
       Reason: ... did not contains expression: "too old"
   ```
   
   The [saved sandbox](https://ci.trafficserver.apache.org/job/Github_Builds/job/autest/43964/artifact/output/13712/sandbox.tar.gz) contains this in the client's `stream.stderr.txt`:
   
   ```text
   .../rate_limit_sni_expiry_client.sh: line 57: /tmp/rl_holder.MoWFrK/fifo: No such file or directory
   ```
   
   `traffic.out` shows only two connections reserving and releasing the slot, at 18:21:53 and 18:21:58. Neither queues. ATS exits normally, the underflow/abort assertion passes, and the helper still prints `rate_limit-expiry-done` and returns 0.
   
   ## Cause
   
   The [holder setup](https://github.com/apache/trafficserver/blob/5168392148496eb83305e61fffcab3edbcf68e21/tests/gold_tests/pluginTest/rate_limit/rate_limit_sni_expiry_client.sh#L53-L60) does this:
   
   ```bash
   mkfifo "$fifo"
   ${OSSL} <"$fifo" >/dev/null 2>&1 &
   exec 3<>"$fifo"
   rm -rf "$fifo_dir"
   sleep 3
   ```
   
   The parent's read/write open of the FIFO does not wait for the background child to open its read end. If the parent removes the directory first, the child's redirection fails and the holder never connects. The later sleep cannot recover from that failure, and the helper does not check the holder's status.
   
   ## Suggested fix
   
   Keep the FIFO available until the holder has opened it, or establish the required descriptors before launching the holder. Wait for the holder to reserve the slot before starting the queued connection, and report holder startup failures directly. Preserve the assertion that expiry was actually exercised.
   
   ## Notes
   
   Related to #13679, which covered the separate `rate_limit_sni_queue` test's short handshake window. This failure is in `rate_limit_sni_expiry` and includes a concrete FIFO-open error. Found while checking CI for #13712; that PR does not change this helper.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@trafficserver.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] rate_limit_sni_expiry AuTest is flaky: the holder races FIFO removal [trafficserver]

["moonchen (via GitHub)" <gi...@apache.org>]

moonchen closed issue #13714: rate_limit_sni_expiry AuTest is flaky: the holder races FIFO removal
URL: https://github.com/apache/trafficserver/issues/13714


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@trafficserver.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org