You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@trafficserver.apache.org by "masaori335 (via GitHub)" <gi...@apache.org> on 2024/09/24 06:10:37 UTC
[PR] Use BRAVO rwlock for ReplaceablePtr [trafficserver]
masaori335 opened a new pull request, #11786:
URL: https://github.com/apache/trafficserver/pull/11786
# Problem
My pthread mutex lock profiler says a mutex in the `Cache::open_read` is heavy on cache hit benchmark.
```
mutex b'[unknown]' ::: wait time 680245.52us ::: hold time 139069.39us ::: enter count 28602 ::: try-lock failure count 0
b'std::__1::mutex::lock()+0x6 [libc++.so.1.0]' (7f86927927a6)
b'Cache::open_read(Continuation*, ts::CryptoHash const*, HTTPHdr*, HttpConfigAccessor const*, CacheFragType, char const*, int)+0x48 [traffic_server]' (55bab493e338)
b'HttpSM::do_cache_lookup_and_read()+0x2ce [traffic_server]' (55bab47ea71e)
b'HttpSM::state_read_client_request_header(int, void*)+0xdf3 [traffic_server]' (55bab47ccea3)
b'HttpSM::main_handler(int, void*)+0x133 [traffic_server]' (55bab47cb813)
b'HttpSM::state_add_to_list(int, void*)+0x66 [traffic_server]' (55bab47cb8e6)
b'HttpSM::attach_client_session(ProxyTransaction*)+0x528 [traffic_server]' (55bab47cbf98)
b'Http1ClientSession::new_transaction()+0x57 [traffic_server]' (55bab47a8a67)
b'Http1ClientSession::state_keep_alive(int, void*)+0xf9 [traffic_server]' (55bab47a86f9)
b'read_signal_and_update(int, UnixNetVConnection*) [clone .llvm.3458165247859049333]+0x17a [traffic_server]' (55bab4a5501a)
b'UnixNetVConnection::net_read_io(NetHandler*, EThread*)+0x6c1 [traffic_server]' (55bab4a53fd1)
b'NetHandler::process_ready_list()+0x221 [traffic_server]' (55bab4a43e21)
b'NetHandler::waitForActivity(long)+0xdc [traffic_server]' (55bab4a4436c)
b'non-virtual thunk to NetHandler::waitForActivity(long)+0xd [traffic_server]' (55bab4a443ed)
b'EThread::execute_regular()+0x59a [traffic_server]' (55bab4a83e9a)
b'EThread::execute()+0x15d [traffic_server]' (55bab4a8448d)
b'spawn_thread_internal(void*) [clone .llvm.16199673917350461532]+0x57 [traffic_server]' (55bab4a82007)
b'start_thread+0x2d2 [libc.so.6]' (7f8692289c02)
```
I think this is a `ReplaceablePtr` in the `Cache::key_to_stripe`. It uses `std::shared_mutex` and it doesn't scale well on multi-thread.
https://github.com/apache/trafficserver/blob/c20661125764e6472763d586cd736184782865cb/src/iocore/cache/Cache.cc#L750-L753
# Approach
Replace `std::shared_mutex` with `ts::bravo::shared_mutex`. With this patch, the heavy mutex is gone from profiler and benchmark show improvement.
Protocol | Test (with this patch) | Control (without this patch) | Diff
----|----|----|-----
HTTP/1.1 on TCP | 743,879 | 624,634 | 1.191
HTTP/2 on TLS | 539,059 | 518,023 | 1.041
HTTP/1.1 on TLS | 629,364 | 567,375 | 1.109
Some conditions of the benchmark is below.
- 63 threads
- 5 volumes
- 100 URLs
- 100 % cache hit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@trafficserver.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Use BRAVO rwlock for ReplaceablePtr [trafficserver]
Posted by "masaori335 (via GitHub)" <gi...@apache.org>.
masaori335 commented on PR #11786:
URL: https://github.com/apache/trafficserver/pull/11786#issuecomment-2376062185
[approve ci rocky]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@trafficserver.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Use BRAVO rwlock for ReplaceablePtr [trafficserver]
Posted by "masaori335 (via GitHub)" <gi...@apache.org>.
masaori335 commented on PR #11786:
URL: https://github.com/apache/trafficserver/pull/11786#issuecomment-2370457190
ASan detected something. I'm checking it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@trafficserver.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org