You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@trafficserver.apache.org by Veiko Kukk <ve...@gmail.com> on 2017/12/21 10:40:02 UTC
How to purge all cached negative responses
Hi,
We had configuration mistake that enforced Cache-Contro: max-age=157784630
also to negative responses that then got cached.
Now, after fixing config, we need to purge all those objects from cache.
It's even good enough if I could get list of objects with certain HTTP
status code, then I could write script that purges those objects one by one.
How to do mass purge based on HTTP status code or just get list of objects
based on HTTP status code?
Veiko
Re: How to purge all cached negative responses
Posted by Sudheer Vinukonda <su...@yahoo.com>.
> On Dec 21, 2017, at 7:54 AM, Sudheer Vinukonda <su...@yahoo.com> wrote:
>
>
>
>> On Dec 21, 2017, at 7:43 AM, Miles Libbey <ml...@apache.org> wrote:
>>
>> How about a header_rewrite like
>> If status>399
>> Turn off cache.
>
> It’s too “late” to turn off cache at this point (using header_rewrite with the hooks it currently allows)
>
>>
>> You might just be better off dropping the entire cache or changing the cache generation number for that remap ruleset.
>> miles
>>
>>> On Thursday, December 21, 2017, Sudheer Vinukonda <su...@yahoo.com> wrote:
>>> Another option is to write a simple plugin that hooks at Cache Lookup Complete state to overwrite the cache status from hit to stale for negative responses (status > 399). Unfortunately, I think header rewrite doesn’t expose the cache lookup Complete hooks, so this has to be written as a plugin (perhaps, might be simple with Lua?).
>>>
>>> This approach won’t purge these objects instantly, but prevents them from returning to clients from the cache and they eventually get removed automatically due to cyclic cache.
>>>
>>> Thanks,
>>>
>>> Sudheer
>>>
>>>> On Dec 21, 2017, at 6:39 AM, David Carlin <dc...@oath.com> wrote:
>>>>
>>>> I'm suggesting finding the failed requests from the logs, and purging those URLs so future requests work.
>>>>
>>>> David
>>>>
>>>>> On Thu, Dec 21, 2017 at 9:20 AM, Veiko Kukk <ve...@gmail.com> wrote:
>>>>> Thank you for your answer.
>>>>>
>>>>> Sure I can find those failures from logs, but it's no good because, well, they have then already failed for client.
>>>>> I've read about cache inspector, but it does not seem to be able to filter based on HTTP status code.
>>>>>
>>>>> Veiko
>>>>>
>>>>>
>>>>> 2017-12-21 14:21 GMT+02:00 David Carlin <dc...@oath.com>:
>>>>>> Can you grab the list of objects from the log files? Thats only thing I can think of.
>>>>>>
>>>>>> The cache inspector exists, but I've never had any luck with it. I think our cache is too large for it:
>>>>>>
>>>>>> https://docs.trafficserver.apache.org/en/latest/admin-guide/storage/index.en.html#inspecting-the-cache
>>>>>>
>>>>>> David
>>>>>>
>>>>>>> On Thu, Dec 21, 2017 at 5:40 AM, Veiko Kukk <ve...@gmail.com> wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> We had configuration mistake that enforced Cache-Contro: max-age=157784630 also to negative responses that then got cached.
>>>>>>>
>>>>>>> Now, after fixing config, we need to purge all those objects from cache. It's even good enough if I could get list of objects with certain HTTP status code, then I could write script that purges those objects one by one.
>>>>>>>
>>>>>>> How to do mass purge based on HTTP status code or just get list of objects based on HTTP status code?
>>>>>>>
>>>>>>>
>>>>>>> Veiko
>>>>>>>
>>>>>>
>>>>>
>>>>
Re: How to purge all cached negative responses
Posted by Sudheer Vinukonda <su...@yahoo.com>.
> On Dec 21, 2017, at 7:43 AM, Miles Libbey <ml...@apache.org> wrote:
>
> How about a header_rewrite like
> If status>399
> Turn off cache.
It’s too “late” to turn off cache at this point.
>
> You might just be better off dropping the entire cache or changing the cache generation number for that remap ruleset.
> miles
>
>> On Thursday, December 21, 2017, Sudheer Vinukonda <su...@yahoo.com> wrote:
>> Another option is to write a simple plugin that hooks at Cache Lookup Complete state to overwrite the cache status from hit to stale for negative responses (status > 399). Unfortunately, I think header rewrite doesn’t expose the cache lookup Complete hooks, so this has to be written as a plugin (perhaps, might be simple with Lua?).
>>
>> This approach won’t purge these objects instantly, but prevents them from returning to clients from the cache and they eventually get removed automatically due to cyclic cache.
>>
>> Thanks,
>>
>> Sudheer
>>
>>> On Dec 21, 2017, at 6:39 AM, David Carlin <dc...@oath.com> wrote:
>>>
>>> I'm suggesting finding the failed requests from the logs, and purging those URLs so future requests work.
>>>
>>> David
>>>
>>>> On Thu, Dec 21, 2017 at 9:20 AM, Veiko Kukk <ve...@gmail.com> wrote:
>>>> Thank you for your answer.
>>>>
>>>> Sure I can find those failures from logs, but it's no good because, well, they have then already failed for client.
>>>> I've read about cache inspector, but it does not seem to be able to filter based on HTTP status code.
>>>>
>>>> Veiko
>>>>
>>>>
>>>> 2017-12-21 14:21 GMT+02:00 David Carlin <dc...@oath.com>:
>>>>> Can you grab the list of objects from the log files? Thats only thing I can think of.
>>>>>
>>>>> The cache inspector exists, but I've never had any luck with it. I think our cache is too large for it:
>>>>>
>>>>> https://docs.trafficserver.apache.org/en/latest/admin-guide/storage/index.en.html#inspecting-the-cache
>>>>>
>>>>> David
>>>>>
>>>>>> On Thu, Dec 21, 2017 at 5:40 AM, Veiko Kukk <ve...@gmail.com> wrote:
>>>>>> Hi,
>>>>>>
>>>>>> We had configuration mistake that enforced Cache-Contro: max-age=157784630 also to negative responses that then got cached.
>>>>>>
>>>>>> Now, after fixing config, we need to purge all those objects from cache. It's even good enough if I could get list of objects with certain HTTP status code, then I could write script that purges those objects one by one.
>>>>>>
>>>>>> How to do mass purge based on HTTP status code or just get list of objects based on HTTP status code?
>>>>>>
>>>>>>
>>>>>> Veiko
>>>>>>
>>>>>
>>>>
>>>
Re: How to purge all cached negative responses
Posted by Miles Libbey <ml...@apache.org>.
How about a header_rewrite like
If status>399
Turn off cache.
You might just be better off dropping the entire cache or changing the
cache generation number for that remap ruleset.
miles
On Thursday, December 21, 2017, Sudheer Vinukonda <
sudheervinukonda@yahoo.com> wrote:
> Another option is to write a simple plugin that hooks at Cache Lookup
> Complete state to overwrite the cache status from hit to stale for negative
> responses (status > 399). Unfortunately, I think header rewrite doesn’t
> expose the cache lookup Complete hooks, so this has to be written as a
> plugin (perhaps, might be simple with Lua?).
>
> This approach won’t purge these objects instantly, but prevents them from
> returning to clients from the cache and they eventually get removed
> automatically due to cyclic cache.
>
> Thanks,
>
> Sudheer
>
> On Dec 21, 2017, at 6:39 AM, David Carlin <dc...@oath.com> wrote:
>
> I'm suggesting finding the failed requests from the logs, and purging
> those URLs so future requests work.
>
> David
>
> On Thu, Dec 21, 2017 at 9:20 AM, Veiko Kukk <ve...@gmail.com> wrote:
>
>> Thank you for your answer.
>>
>> Sure I can find those failures from logs, but it's no good because, well,
>> they have then already failed for client.
>> I've read about cache inspector, but it does not seem to be able to
>> filter based on HTTP status code.
>>
>> Veiko
>>
>>
>> 2017-12-21 14:21 GMT+02:00 David Carlin <dc...@oath.com>:
>>
>>> Can you grab the list of objects from the log files? Thats only thing
>>> I can think of.
>>>
>>> The cache inspector exists, but I've never had any luck with it. I
>>> think our cache is too large for it:
>>>
>>> https://docs.trafficserver.apache.org/en/latest/admin-guide/
>>> storage/index.en.html#inspecting-the-cache
>>>
>>> David
>>>
>>> On Thu, Dec 21, 2017 at 5:40 AM, Veiko Kukk <ve...@gmail.com>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> We had configuration mistake that enforced Cache-Contro:
>>>> max-age=157784630 also to negative responses that then got cached.
>>>>
>>>> Now, after fixing config, we need to purge all those objects from
>>>> cache. It's even good enough if I could get list of objects with certain
>>>> HTTP status code, then I could write script that purges those objects one
>>>> by one.
>>>>
>>>> How to do mass purge based on HTTP status code or just get list of
>>>> objects based on HTTP status code?
>>>>
>>>>
>>>> Veiko
>>>>
>>>>
>>>
>>
>
Re: How to purge all cached negative responses
Posted by Sudheer Vinukonda <su...@yahoo.com>.
Another option is to write a simple plugin that hooks at Cache Lookup Complete state to overwrite the cache status from hit to stale for negative responses (status > 399). Unfortunately, I think header rewrite doesn’t expose the cache lookup Complete hooks, so this has to be written as a plugin (perhaps, might be simple with Lua?).
This approach won’t purge these objects instantly, but prevents them from returning to clients from the cache and they eventually get removed automatically due to cyclic cache.
Thanks,
Sudheer
> On Dec 21, 2017, at 6:39 AM, David Carlin <dc...@oath.com> wrote:
>
> I'm suggesting finding the failed requests from the logs, and purging those URLs so future requests work.
>
> David
>
>> On Thu, Dec 21, 2017 at 9:20 AM, Veiko Kukk <ve...@gmail.com> wrote:
>> Thank you for your answer.
>>
>> Sure I can find those failures from logs, but it's no good because, well, they have then already failed for client.
>> I've read about cache inspector, but it does not seem to be able to filter based on HTTP status code.
>>
>> Veiko
>>
>>
>> 2017-12-21 14:21 GMT+02:00 David Carlin <dc...@oath.com>:
>>> Can you grab the list of objects from the log files? Thats only thing I can think of.
>>>
>>> The cache inspector exists, but I've never had any luck with it. I think our cache is too large for it:
>>>
>>> https://docs.trafficserver.apache.org/en/latest/admin-guide/storage/index.en.html#inspecting-the-cache
>>>
>>> David
>>>
>>>> On Thu, Dec 21, 2017 at 5:40 AM, Veiko Kukk <ve...@gmail.com> wrote:
>>>> Hi,
>>>>
>>>> We had configuration mistake that enforced Cache-Contro: max-age=157784630 also to negative responses that then got cached.
>>>>
>>>> Now, after fixing config, we need to purge all those objects from cache. It's even good enough if I could get list of objects with certain HTTP status code, then I could write script that purges those objects one by one.
>>>>
>>>> How to do mass purge based on HTTP status code or just get list of objects based on HTTP status code?
>>>>
>>>>
>>>> Veiko
>>>>
>>>
>>
>
Re: How to purge all cached negative responses
Posted by David Carlin <dc...@oath.com>.
I'm suggesting finding the failed requests from the logs, and purging those
URLs so future requests work.
David
On Thu, Dec 21, 2017 at 9:20 AM, Veiko Kukk <ve...@gmail.com> wrote:
> Thank you for your answer.
>
> Sure I can find those failures from logs, but it's no good because, well,
> they have then already failed for client.
> I've read about cache inspector, but it does not seem to be able to filter
> based on HTTP status code.
>
> Veiko
>
>
> 2017-12-21 14:21 GMT+02:00 David Carlin <dc...@oath.com>:
>
>> Can you grab the list of objects from the log files? Thats only thing I
>> can think of.
>>
>> The cache inspector exists, but I've never had any luck with it. I think
>> our cache is too large for it:
>>
>> https://docs.trafficserver.apache.org/en/latest/admin-guide/
>> storage/index.en.html#inspecting-the-cache
>>
>> David
>>
>> On Thu, Dec 21, 2017 at 5:40 AM, Veiko Kukk <ve...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> We had configuration mistake that enforced Cache-Contro:
>>> max-age=157784630 also to negative responses that then got cached.
>>>
>>> Now, after fixing config, we need to purge all those objects from cache.
>>> It's even good enough if I could get list of objects with certain HTTP
>>> status code, then I could write script that purges those objects one by one.
>>>
>>> How to do mass purge based on HTTP status code or just get list of
>>> objects based on HTTP status code?
>>>
>>>
>>> Veiko
>>>
>>>
>>
>
Re: How to purge all cached negative responses
Posted by Veiko Kukk <ve...@gmail.com>.
Thank you for your answer.
Sure I can find those failures from logs, but it's no good because, well,
they have then already failed for client.
I've read about cache inspector, but it does not seem to be able to filter
based on HTTP status code.
Veiko
2017-12-21 14:21 GMT+02:00 David Carlin <dc...@oath.com>:
> Can you grab the list of objects from the log files? Thats only thing I
> can think of.
>
> The cache inspector exists, but I've never had any luck with it. I think
> our cache is too large for it:
>
> https://docs.trafficserver.apache.org/en/latest/admin-
> guide/storage/index.en.html#inspecting-the-cache
>
> David
>
> On Thu, Dec 21, 2017 at 5:40 AM, Veiko Kukk <ve...@gmail.com> wrote:
>
>> Hi,
>>
>> We had configuration mistake that enforced Cache-Contro:
>> max-age=157784630 also to negative responses that then got cached.
>>
>> Now, after fixing config, we need to purge all those objects from cache.
>> It's even good enough if I could get list of objects with certain HTTP
>> status code, then I could write script that purges those objects one by one.
>>
>> How to do mass purge based on HTTP status code or just get list of
>> objects based on HTTP status code?
>>
>>
>> Veiko
>>
>>
>
Re: How to purge all cached negative responses
Posted by David Carlin <dc...@oath.com>.
Can you grab the list of objects from the log files? Thats only thing I
can think of.
The cache inspector exists, but I've never had any luck with it. I think
our cache is too large for it:
https://docs.trafficserver.apache.org/en/latest/admin-guide/storage/index.en.html#inspecting-the-cache
David
On Thu, Dec 21, 2017 at 5:40 AM, Veiko Kukk <ve...@gmail.com> wrote:
> Hi,
>
> We had configuration mistake that enforced Cache-Contro: max-age=157784630
> also to negative responses that then got cached.
>
> Now, after fixing config, we need to purge all those objects from cache.
> It's even good enough if I could get list of objects with certain HTTP
> status code, then I could write script that purges those objects one by one.
>
> How to do mass purge based on HTTP status code or just get list of objects
> based on HTTP status code?
>
>
> Veiko
>
>