You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@trafficcontrol.apache.org by GitBox <gi...@apache.org> on 2020/11/12 22:24:13 UTC

[GitHub] [trafficcontrol] mitchell852 opened a new issue #5279: ORT/atstccfg: regex_revalidate.config should never have 1+ rule for the same pattern

mitchell852 opened a new issue #5279:
URL: https://github.com/apache/trafficcontrol/issues/5279

## I'm submitting a ...
- improvement request (usability, performance, tech debt, etc.)

## Traffic Control components affected ...
- Traffic Ops ORT

## Current behavior:
TO/TP allows users to create invalidation requests (aka jobs) with a TTL for an asset. That request will be added to the regex_revalidate.config file created by ORT/atstccfg and looks like such:

```
Path: /opt/trafficserver/etc/trafficserver/regex_revalidate.config

^^ Imagine that rule was created on 11/12 with a one week TTL (11/19). That means the rule will stay in regex_revalidate.config until it is removed on 11/19. However, assume a request comes in on 11/13 for http://my.origin.net/foo.png, the rule will be exercised once and ONLY once and the rule will essentially be "dead".

TO also allows users to create invalidation requests (aka jobs) for duplicate assets (or pattern) and TO will warn the user when doing so per #5092 which will result in a regex_revalidate.config file that looks like this:

```
Path: /opt/trafficserver/etc/trafficserver/regex_revalidate.config

# DO NOT EDIT - Generated for cdn-1 by Traffic Ops (https://trafficops.domain.tld/) on Thu Nov 12 21:07:18 UTC 2020
http://my.origin.net/foo.png 1605823198 <-- this rule was created on 11/12 with a one week TTL (11/19)
http://my.origin.net/foo.png 1606082398 <-- this rule was created on 11/15 with a one week TTL (11/22)
```

^^ In this case, the 1st rule is basically "dead" as it has already been exercised and wont' be again, however, because of the way the ATS regex revalidate plugin works, it will only respect the first rule and any subsequent ruies for the same asset (or pattern) will simply be ignored until the first rule is removed entirely when it expires (11/19). This means the 2nd rule CANNOT run before 11/19.

## New behavior:
ORT/atstccfg should only create ONE rule for a given asset/pattern. It should take the last or latest rule created in the jobs table for that pattern.

For example if the jobs table has multiple rows for asset_url=http://my.origin.net/foo.png, regex_revalidate.config should only include ONE entry for http://my.origin.net/foo.png. It should
- not include expired rules where the TTL has passed (current behavior)
- only include the rule with the latest entered_time

This would have the effect of allowing users to update rules.

## Minimal reproduction of the problem with instructions:
1. Make a request to a cache server for a given asset. Make another request and ensure that it is returned from cache.
2. In TP or TO API, create an invalidation requests for that asset with TTL=24h. Make a request to the applicable cache for that asset and notice a cache miss as the content was invalidated and fetched from the origin.
3. In TP or TO API, create an invalidation requests for that SAME asset with TTL=24h. Make a request to the applicable cache for that asset and notice a cache HIT when the 2nd rule should have forced an invalidation. This is because the first rule still exists in regex_revalidate.config and is blocking any subsequent rules for the same asset/pattern.

<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

https://apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org