You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@trafficserver.apache.org by GitBox <gi...@apache.org> on 2022/04/16 13:13:28 UTC
[GitHub] [trafficserver] amoghyermalkar123 opened a new issue, #8672: What's the point of ttl-in-cache ?
amoghyermalkar123 opened a new issue, #8672:
URL: https://github.com/apache/trafficserver/issues/8672
I have the following cache.config configuration:
```
url_regex=subdomain.domain.com.* method=PUT method=POST method=PATCH method=DELETE method=OPTIONS method=GET method=HEAD ttl-in-cache=15s
```
i have a one child one parent tiered ATS architecture.
When I request a content under the specified domain, it caches beyond the mentioned ttl-in-cache value, i.e. in this case more than 15 seconds. I tried quite some different combinations of the configuration from cache.config but nothing has worked so far. The content is always cached beyond the mentioned time frame.
I have tried matching on the domains, hosts tried using that with revalidate timer, nothing seems to work.
One thing to note I have cached only one image under this domain.
I confirm how long its cached for by the Age header and by verifying via logs and everytime it shows TCP_MEM_HIT.
I also tried purging all caches, making the first request, waiting for around 10-20 minutes before hitting the second request but same result is what I am seeing.
After all this, I tried going through the CLFUS algorithm, a brief look into the cache architecture to try to understand how TTL is taken under consideration, but to no avail. I only found one reference to ttl in cache architecture that to while cache reading.
In the RAM cache document there is no mention of ttl.
What am I missing in all these cases? I thought maybe the TTL might be for disk cache, but for ATS to even go to disk IO would be when the object is removed from RAM which is not happening even after hours.
The CLFUS algorithm is extrememly sophisticated to even consider ttl-in-cache, so I am not understanding if I am missing something? Are there some opinionated ways being followed here? Have I missed a document to go through?
I understood that cache is a cycle? so maybe after the cache is full eviction begins? if so, whats the point of ttl-in-cache configuration from cache.config?
Where exactly does the ttl-in-cache come into play?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficserver.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] amoghyermalkar123 commented on issue #8672: What's the point of ttl-in-cache ?
Posted by GitBox <gi...@apache.org>.
amoghyermalkar123 commented on issue #8672:
URL: https://github.com/apache/trafficserver/issues/8672#issuecomment-1104728012
Understood.
The problem solved itself when we realized that the configured origin had an inconsistent manifest and was later resolved. As for cache.config, as the doc says that it's an anti pattern and we should rely on origin folks with their max-age header, we'll build our solution in accordance.
@mlibbey thank you so much for sticking with me as I tried wrapping my head around this. Really appreciate the help.
Closing out the issue :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficserver.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] amoghyermalkar123 commented on issue #8672: What's the point of ttl-in-cache ?
Posted by GitBox <gi...@apache.org>.
amoghyermalkar123 commented on issue #8672:
URL: https://github.com/apache/trafficserver/issues/8672#issuecomment-1104176670
Hey, so these are the logs for a transaction where the origin server does gives us max-age header
```
[Apr 20 16:37:47.499] [ET_NET 16] DEBUG: <HttpTransact.cc:6378 (is_response_valid)> (http_trans) [2] [is_response_valid] No errors in response
[Apr 20 16:37:47.499] [ET_NET 16] DEBUG: <HttpTransact.cc:3338 (handle_response_from_parent)> (http_trans) [2] [hrfp] connection alive
[Apr 20 16:37:47.499] [ET_NET 16] DEBUG: <HttpTransact.cc:3761 (handle_forward_server_connection_open)> (http_trans) [2] [handle_forward_server_connection_open] (hfsco)
[Apr 20 16:37:47.500] [ET_NET 16] DEBUG: <HttpTransact.cc:3762 (handle_forward_server_connection_open)> (http_seq) [2] [HttpTransact::handle_server_connection_open]
[Apr 20 16:37:47.500] [ET_NET 16] DEBUG: <HttpSM.cc:4162 (do_hostdb_update_if_necessary)> (http) server info = 45.126.189.10:443
[Apr 20 16:37:47.500] [ET_NET 16] DEBUG: <HttpTransact.cc:3849 (handle_forward_server_connection_open)> (http_trans) [2] [hfsco] cache action: CACHE_DO_WRITE
[Apr 20 16:37:47.501] [ET_NET 16] DEBUG: <HttpTransact.cc:3978 (handle_cache_operation_on_forward_server_response)> (http_trans) [2] [handle_cache_operation_on_forward_server_response] (hcoofsr)
[Apr 20 16:37:47.501] [ET_NET 16] DEBUG: <HttpTransact.cc:3979 (handle_cache_operation_on_forward_server_response)> (http_seq) [2] [handle_cache_operation_on_forward_server_response]
[Apr 20 16:37:47.501] [ET_NET 16] DEBUG: <HttpTransact.cc:6058 (is_response_cacheable)> (http_trans) [2] [is_response_cacheable] client permits storing
[Apr 20 16:37:47.501] [ET_NET 16] DEBUG: <HttpTransact.cc:6081 (is_response_cacheable)> (http_trans) [2] [is_response_cacheable] last_modified, expires, or max-age is required
[Apr 20 16:37:47.502] [ET_NET 16] DEBUG: <HttpTransact.cc:3988 (handle_cache_operation_on_forward_server_response)> (http_trans) [2] [hcoofsr] response is not cacheable
[Apr 20 16:37:47.502] [ET_NET 16] DEBUG: <HttpTransact.cc:4131 (handle_cache_operation_on_forward_server_response)> (http_trans) [2] [hcoofsr] response code: 200
[Apr 20 16:37:47.503] [ET_NET 16] DEBUG: <HttpTransactHeaders.cc:438 (calculate_document_age)> (http_age) [calculate_document_age] age_value: 5049
[Apr 20 16:37:47.503] [ET_NET 16] DEBUG: <HttpTransactHeaders.cc:439 (calculate_document_age)> (http_age) [calculate_document_age] date_value: 1650467618
[Apr 20 16:37:47.503] [ET_NET 16] DEBUG: <HttpTransactHeaders.cc:440 (calculate_document_age)> (http_age) [calculate_document_age] response_time: 1650472667
[Apr 20 16:37:47.504] [ET_NET 16] DEBUG: <HttpTransactHeaders.cc:441 (calculate_document_age)> (http_age) [calculate_document_age] now: 1650472667
[Apr 20 16:37:47.504] [ET_NET 16] DEBUG: <HttpTransactHeaders.cc:442 (calculate_document_age)> (http_age) [calculate_document_age] now (fixed): 1650472667
[Apr 20 16:37:47.504] [ET_NET 16] DEBUG: <HttpTransactHeaders.cc:443 (calculate_document_age)> (http_age) [calculate_document_age] apparent_age: 5049
[Apr 20 16:37:47.505] [ET_NET 16] DEBUG: <HttpTransactHeaders.cc:444 (calculate_document_age)> (http_age) [calculate_document_age] corrected_received_age: 5049
[Apr 20 16:37:47.505] [ET_NET 16] DEBUG: <HttpTransactHeaders.cc:445 (calculate_document_age)> (http_age) [calculate_document_age] response_delay: 0
[Apr 20 16:37:47.505] [ET_NET 16] DEBUG: <HttpTransactHeaders.cc:446 (calculate_document_age)> (http_age) [calculate_document_age] corrected_initial_age: 5049
[Apr 20 16:37:47.506] [ET_NET 16] DEBUG: <HttpTransactHeaders.cc:447 (calculate_document_age)> (http_age) [calculate_document_age] resident_time: 0
[Apr 20 16:37:47.506] [ET_NET 16] DEBUG: <HttpTransactHeaders.cc:448 (calculate_document_age)> (http_age) [calculate_document_age] current_age: 5049
```
As you can see the ATS says the response is not cacheable. Whereas,
`< cache-control: max-age=21600` we get this header from origin. We have configured ATS to not ignore these headers as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficserver.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] amoghyermalkar123 commented on issue #8672: What's the point of ttl-in-cache ?
Posted by GitBox <gi...@apache.org>.
amoghyermalkar123 commented on issue #8672:
URL: https://github.com/apache/trafficserver/issues/8672#issuecomment-1104177578
Please let me know if you need more information to help.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficserver.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] mlibbey commented on issue #8672: What's the point of ttl-in-cache ?
Posted by GitBox <gi...@apache.org>.
mlibbey commented on issue #8672:
URL: https://github.com/apache/trafficserver/issues/8672#issuecomment-1100672315
Can you show the curl response with all headers? If you are running the x-debug plug-in, the x-cache header would be nice to see.
Can you say a little more about what you mean by ‘keeps in cache indefinitely’ (for instance, if max-age is 10, and you fetch after 11 sec, it’s stale?
We don’t use cache.config… so when we need to say don’t cache something either we use a header_rewrite like above setting Cache-control to ‘no-store’ or turning the cache off all together for a remap.
There is a whole developer guide… and a cache section in there too. Perhaps better if you suggest sections that you expected to answer x but didn’t find that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficserver.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] amoghyermalkar123 commented on issue #8672: What's the point of ttl-in-cache ?
Posted by GitBox <gi...@apache.org>.
amoghyermalkar123 commented on issue #8672:
URL: https://github.com/apache/trafficserver/issues/8672#issuecomment-1101427577
for cache.config
url_regex=domain.com/* method=GET method=HEAD revalidate=1s
and max-age header from origin server came in as 2 seconds. We have configured ATS to not
ignore cache control max-age header as well.
(via following config - proxy.config.http.cache.ignore_client_cc_max_age)
so now - with the following request sequences, seperated by a few seconds apart -
1
```
curl -v -H 'X-Debug: X-Cache' https://domain.com/
<X-Cache: miss, Miss from origin
```
2
```
curl -v -H 'X-Debug: X-Cache' https://domain.com/
<X-Cache: hit-stale, Miss from origin
```
3
```
curl -v -H 'X-Debug: X-Cache' https://domain.com/
<X-Cache: hit-fresh, Miss from origin
```
4 - this is a few minutes apart
```
curl -v -H 'X-Debug: X-Cache' https://domain.com/
<X-Cache: hit-fresh, Miss from origin
```
We have a live HLS file, which is updated every 2 seconds, but due to ATS's current nature playback cannot proceed since ATS constantly gives the same response instead of going to the origin server and fetching the latest `index.m3u8` file.
Please note that the `index.m3u8` file lies at the same path and is updated at the same path every time, hence the origin server url stays the same every time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficserver.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] mlibbey commented on issue #8672: What's the point of ttl-in-cache ?
Posted by GitBox <gi...@apache.org>.
mlibbey commented on issue #8672:
URL: https://github.com/apache/trafficserver/issues/8672#issuecomment-1101940555
- If the origin is setting cache control, why have a cache.config line?
- between request 1 and 2, the object went stale, which is what you want. On that request, the origin would have seen some sort of request (most likely a INM or IMS conditional), and presumably a 304 Not modified return.
- Is the Origin's etag/Last-Modified header changing as the content changes? Once the content changes on the origin, and you send it a INM/IMS request, what does the origin respond with?
(fun with curl ... I like `curl -D- -o/dev/null 'http...' ` as a nicer way to show headers. I most frequently use curl -IXGET 'http...' to issue a get, nicely show the headers, but abort before getting the body.)
FWIW, the community has several production HLS use cases with ATS -- the case you are describing is highly used.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficserver.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] amoghyermalkar123 commented on issue #8672: What's the point of ttl-in-cache ?
Posted by GitBox <gi...@apache.org>.
amoghyermalkar123 commented on issue #8672:
URL: https://github.com/apache/trafficserver/issues/8672#issuecomment-1100673862
@mlibbey I haven't used the x-debug plugin, please give me some time as I re-test and get back to you on this thread.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficserver.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] amoghyermalkar123 commented on issue #8672: What's the point of ttl-in-cache ?
Posted by GitBox <gi...@apache.org>.
amoghyermalkar123 commented on issue #8672:
URL: https://github.com/apache/trafficserver/issues/8672#issuecomment-1100661204
So I'm currently using an origin server that responds with max-age header, but ATS still does not seem to respect the header value and keeps the object in cache for indefinite amount of time. For a little context, we are using cache.config, if this file is getting in the way please let me know the same.
I'm also wondering whether there is any configuration that would allow ATS to completely override the Cache-Control headers, in which case I will need some directions as to how I can disable it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficserver.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [trafficserver] amoghyermalkar123 closed issue #8672: What's the point of ttl-in-cache ?
Posted by GitBox <gi...@apache.org>.
amoghyermalkar123 closed issue #8672: What's the point of ttl-in-cache ?
URL: https://github.com/apache/trafficserver/issues/8672
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@trafficserver.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org