You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@trafficserver.apache.org by zw...@apache.org on 2020/03/09 15:18:37 UTC
[trafficserver] 01/02: cache_range_request plugin: add support for
opt X-CRR-IMS reval header
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
commit 517af474e18f27dc4e65b6a0dc28288452b0b09b
Author: Brian Olsen <br...@comcast.com>
AuthorDate: Wed Nov 20 18:35:14 2019 +0000
cache_range_request plugin: add support for opt X-CRR-IMS reval header
(cherry picked from commit 964336fd3ee66360b9cf274a19ec69a056ac6dcd)
---
.../plugins/cache_range_requests.en.rst | 168 +++++++++
doc/admin-guide/plugins/index.en.rst | 13 +-
plugins/cache_range_requests/README | 45 ---
plugins/cache_range_requests/README.md | 80 ++++
.../cache_range_requests/cache_range_requests.cc | 420 +++++++++++++--------
.../cache_range_requests.test.py | 42 ++-
.../cache_range_requests_ims.test.py | 141 +++++++
.../cache_range_requests/gold/frange.stdout.gold | 2 +-
.../cache_range_requests/gold/full.stdout.gold | 2 +-
.../cache_range_requests/gold/inner.stdout.gold | 2 +-
.../cache_range_requests/gold/last.stdout.gold | 2 +-
.../cache_range_requests/gold/pselect.stdout.gold | 2 +-
12 files changed, 702 insertions(+), 217 deletions(-)
diff --git a/doc/admin-guide/plugins/cache_range_requests.en.rst b/doc/admin-guide/plugins/cache_range_requests.en.rst
new file mode 100644
index 0000000..5d33f7c
--- /dev/null
+++ b/doc/admin-guide/plugins/cache_range_requests.en.rst
@@ -0,0 +1,168 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+
+.. include:: ../../common.defs
+
+.. _admin-plugins-cache-range-requests:
+
+
+Cache Range Requests Plugin
+***************************
+
+Description
+===========
+
+Most origin servers support HTTP/1.1 range requests (rfc 7233).
+ATS internally handles range request caching in one of 2 ways:
+
+* Don't cache range requests.
+* Only server range requests from a wholly cached object.
+
+This plugin allows you to remap individual range requests so that they
+are stored as individual objects in the ATS cache when subsequent range
+requests are likely to use the same range. This spreads range requests
+over multiple stripes thereby reducing I/O wait and system load averages.
+
+:program:`cache_range_requests` reads the range request header byte range
+value and then creates a new ``cache key URL`` using the original request
+url with the range value appended to it. The range header is removed
+where appropriate from the requests and the origin server response code
+is changed from a 206 to a 200 to insure that the object is written to
+cache using the new cache key url. The response code sent to the client
+will be changed back to a 206 and all requests to the origin server will
+contain the range header so that the correct response is received.
+
+The :program:`cache_range_requests` plugin by itself has no logic to
+efficiently manage overlapping ranges. It is best to use this plugin
+in conjunction with a smart client that only requests predetermined
+non overlapping cache ranges (request blocking) or as a helper for the
+:program:`slice` plugin.
+
+Only requests which contain the ``Range: <units>=`` GET header
+will be served by the :program:`cache_range_requests` plugin.
+
+If/when ATS implements partial object caching this plugin will
+become deprecated.
+
+*NOTE* Given a multi range request the :program:`cache_range_requests`
+only processes the first range and ignores the rest.
+
+How to run the plugin
+=====================
+
+The plugin can run as a global plugin (a single global instance configured
+using :file:`plugin.config`) or as per-remap plugin (a separate instance
+configured per remap rule in :file:`remap.config`).
+
+Global instance
+---------------
+
+.. code::
+
+ $ cat plugin.config
+ cache_range_request.so
+
+
+Per-remap instance
+------------------
+
+.. code::
+
+ $cat remap.config
+ map http://www.example.com http://www.origin.com \
+ @plugin=cache_range_requests.so
+
+
+If both global and per-remap instance are used the per-remap configuration
+would take precedence (per-remap configuration would be applied and the
+global configuration ignored).
+
+Plugin options
+==============
+
+
+Parent Selection as Cache Key
+-----------------------------
+
+.. option:: --ps-cachekey
+.. option:: -p
+
+Without this option parent selection is based solely on the hash of a
+URL Path a URL is requested from the same upstream parent cache listed
+in parent.config
+
+
+With this option parent selection is based on the full ``cache key URL``
+which includes information about the partial content range. In this mode,
+all requests (include partial content) will use consistent hashing method
+for parent selection.
+
+
+X-CRR-IMS header support
+------------------------
+
+.. option:: --consider-ims
+.. option:: -c
+
+To support slice plugin self healing an option to force revalidation
+after cache lookup complete was added. This option is triggered by a
+special header:
+
+.. code::
+
+ X-CRR-IMS: Tue, 19 Nov 2019 13:26:45 GMT
+
+When this header is provided and a `cache hit fresh` is encoutered the
+``Date`` header of the object in cache is compared to this header date
+value. If the cache date is *less* than this IMS date then the object
+is marked as STALE and an appropriate If-Modified-Since or If-Match
+request along with this X-CRR-IMS header is passed up to the parent.
+
+In order for this to properly work in a CDN each cache in the
+chain *SHOULD* also contain a remap rule with the
+:program:`cache_range_requests` plugin with this option set.
+
+
+Configuration examples
+======================
+
+Global plugin
+-------------
+
+.. code::
+
+ cache_range_requests.so --ps-cachekey --consider-ims
+
+or
+
+.. code::
+
+ cache_range_requests.so -p -c
+
+Remap plugin
+------------
+
+.. code::
+
+ map http://ats http://parent @plugin=cache_range_requests.so @pparam=--ps-cachekey @pparam=--consider-ims
+
+or
+
+.. code::
+
+ map http://ats http://parent @plugin=cache_range_requests.so @pparam=-p @pparam=-c
diff --git a/doc/admin-guide/plugins/index.en.rst b/doc/admin-guide/plugins/index.en.rst
index 48e9825..48a3d5b 100644
--- a/doc/admin-guide/plugins/index.en.rst
+++ b/doc/admin-guide/plugins/index.en.rst
@@ -50,6 +50,7 @@ Plugins that are considered stable are installed by default in |TS| releases.
Background Fetch <background_fetch.en>
Cache Key Manipulation <cachekey.en>
Cache Promotion Policies <cache_promote.en>
+ Cache Range Requests <cache_range_requests.en>
Combo Handler <combo_handler.en>
Configuration Remap <conf_remap.en>
Cookie Remap <cookie_remap.en>
@@ -63,7 +64,6 @@ Plugins that are considered stable are installed by default in |TS| releases.
Regex Remap <regex_remap.en>
Regex Revalidate <regex_revalidate.en>
Remap Purge <remap_purge.en>
- Slice <slice.en>
Stats over HTTP <stats_over_http.en>
TCPInfo <tcpinfo.en>
XDebug <xdebug.en>
@@ -83,6 +83,9 @@ Plugins that are considered stable are installed by default in |TS| releases.
:doc:`Cache Promotion Policies <cache_promote.en>`
Allows for control over which assets should be written to cache, or not.
+:doc:`Cache Range Requests <cache_range_requests.en>`
+ Cache ranges by adding the range request header to the cache key.
+
:doc:`Combo Handler <combo_handler.en>`
Provides an intelligent way to combine multiple URLs into a single URL, and have Apache Traffic Server combine the components into one response.
@@ -159,7 +162,7 @@ directory of the |TS| source tree. Experimental plugins can be compiled by passi
Multiplexer <multiplexer.en>
MySQL Remap <mysql_remap.en>
Signed URLs <url_sig.en>
- Slicer <slice.en>
+ Slice <slice.en>
SSL Headers <sslheaders.en>
SSL Session Reuse <ssl_session_reuse.en>
System Statistics <system_stats.en>
@@ -223,10 +226,10 @@ directory of the |TS| source tree. Experimental plugins can be compiled by passi
:doc:`Signed URLs <url_sig.en>`
Adds support for verifying URL signatures for incoming requests to either deny or redirect access.
-:doc:`Slicer <slice.en>`
+:doc:`Slice <slice.en>`
Slice full file or range based requests into deterministic chunks, allowing large files to be
- spread across multiple cache stripes. Allows range requests to be satisfied by stitching these
- chunks together.
+ spread across multiple cache stripes. Allows arbitrary range requests to be satisfied by stitching
+ these chunks together.
:doc:`SSL Session Reuse <ssl_session_reuse.en>`
Coordinates Session ID and ticket based TLS session resumption between a group of ATS machines.
diff --git a/plugins/cache_range_requests/README b/plugins/cache_range_requests/README
deleted file mode 100644
index 5495f68..0000000
--- a/plugins/cache_range_requests/README
+++ /dev/null
@@ -1,45 +0,0 @@
-
-Thousands of range requests for a very large object in the traffic server cache
-are likely to increase system load averages due to I/O wait as objects are stored
-on a single stripe or disk drive.
-
-This plugin allows you to remap individual range requests so that they are stored
-as individual objects in the ATS cache when subsequent range requests are likely
-to use the same range. This spreads range requests over multiple stripes thereby
-reducing I/O wait and system load averages.
-
-This plugin reads the range request header byte range value and then creates a
-new cache key url using the original request url with the range value appended
-to it. The range header is removed where appropriate from the requests and the
-origin server response code is changed from a 206 to a 200 to insure that the
-object is written to cache using the new cache key url. The response code sent
-to the client will be changed back to a 206 and all requests to the origin server
-will contain the range header so that the correct response is received.
-
-Configuration:
-
- Add @plugin=cache_range_requests.so to your remap.config rules.
-
- Or for a global plugin where all range requests are processed,
- Add cache_range_requests.so to the plugin.config
-
-Parent Selection Mode (consistent-hash only):
-
- default: Parent selection is based solely on the hash of a URL Path
- In this mode, all partial content of a URL is requested from the same
- upstream parent cache listed in parent.config
-
- cache_key_url: Parent selection is based on the full cache_key_url which
- includes information about the partial content range.
- In this mode, all requests (include partial content) will use
- consistent hashing method for parent selection.
-
- To enable cache_key_url parent select mode, the following param must be set:
-
- Global Plugin (plugin.config):
-
- cache_range_requests.so ps_mode:cache_key_url
-
- Remap Plugin (remap.config):
-
- <from-url> <to-url> @plugin=cache_range_requests.so @pparam=ps_mode:cache_key_url
diff --git a/plugins/cache_range_requests/README.md b/plugins/cache_range_requests/README.md
new file mode 100644
index 0000000..ef567a7
--- /dev/null
+++ b/plugins/cache_range_requests/README.md
@@ -0,0 +1,80 @@
+
+Thousands of range requests for a very large object in the traffic server
+cache are likely to increase system load averages due to I/O wait as
+objects are stored on a single stripe or disk drive.
+
+This plugin allows you to remap individual range requests so that they
+are stored as individual objects in the ATS cache when subsequent range
+requests are likely to use the same range. This spreads range requests
+over multiple stripes thereby reducing I/O wait and system load averages.
+
+This plugin reads the range request header byte range value and then
+creates a new cache key url using the original request url with the range
+value appended to it. The range header is removed where appropriate
+from the requests and the origin server response code is changed from
+a 206 to a 200 to insure that the object is written to cache using the
+new cache key url. The response code sent to the client will be changed
+back to a 206 and all requests to the origin server will contain the
+range header so that the correct response is received.
+
+Configuration:
+
+ Add @plugin=cache_range_requests.so to your remap.config rules.
+
+ Or for a global plugin where all range requests are processed,
+ Add cache_range_requests.so to the plugin.config
+
+Parent Selection Mode (consistent-hash only):
+
+ default: Parent selection is based solely on the hash of a URL Path
+ In this mode, all partial content of a URL is requested
+ from the same upstream parent cache listed in parent.config
+
+ Cache_key_url: Parent selection is based on the full cache_key_url
+ which includes information about the partial content
+ range. In this mode, all requests (include partial
+ content) will use consistent hashing method for
+ parent selection.
+
+ To enable cache_key_url parent select mode, the following param must be set:
+
+ Global Plugin (plugin.config):
+
+ cache_range_requests.so -p
+ cache_range_requests.so --ps-cachekey
+
+ Remap Plugin (remap.config):
+
+ <from-url> <to-url> @plugin=cache_range_requests.so @pparam=--ps-cachekey
+ <from-url> <to-url> @plugin=cache_range_requests.so @pparam=-p
+
+X-CRR-IMS header support
+
+ To support slice plugin self healing an option to force
+ revalidation after cache lookup complete was added. This option
+ is triggered by a special header:
+
+ This optional header looks like:
+
+ X-CRR-IMS: Tue, 19 Nov 2019 13:26:45 GMT
+
+ If the cache lookup was a cache hit and the cache header date
+ is *less* than this header value then the cache state is switched
+ from FRESH to STALE which results in If-Modified-Since or
+ If-Match request being passed to the parent.
+
+ In order for this option to be enabled the the following parameter
+ must be set:
+
+ Global Plugin (plugin.config):
+
+ cache_range_requests.so --consider-ims
+ cache_range_requests.so -c
+
+ Remap Plugin (remap.config):
+
+ <from-url> <to-url> @plugin=cache_range_requests.so @pparam=--consider-ims
+ <from-url> <to-url> @plugin=cache_range_requests.so @pparam=-c
+
+ Consider using the header_rewrite plugin to protect the parent
+ from using this option as an attack vector against an origin.
diff --git a/plugins/cache_range_requests/cache_range_requests.cc b/plugins/cache_range_requests/cache_range_requests.cc
index 60f512c..13381d2 100644
--- a/plugins/cache_range_requests/cache_range_requests.cc
+++ b/plugins/cache_range_requests/cache_range_requests.cc
@@ -26,68 +26,107 @@
* requests.
*/
-#include <cstdio>
-#include <cstring>
#include "ts/ts.h"
#include "ts/remap.h"
+#include <cstdio>
+#include <cstring>
+#include <getopt.h>
+#include <string>
+#include <string_view>
+
#define PLUGIN_NAME "cache_range_requests"
#define DEBUG_LOG(fmt, ...) TSDebug(PLUGIN_NAME, "[%s:%d] %s(): " fmt, __FILE__, __LINE__, __func__, ##__VA_ARGS__)
#define ERROR_LOG(fmt, ...) TSError("[%s:%d] %s(): " fmt, __FILE__, __LINE__, __func__, ##__VA_ARGS__)
+namespace
+{
typedef enum parent_select_mode {
PS_DEFAULT, // Default ATS parent selection mode
PS_CACHEKEY_URL, // Set parent selection url to cache_key url
} parent_select_mode_t;
struct pluginconfig {
- parent_select_mode_t ps_mode;
+ parent_select_mode_t ps_mode{PS_DEFAULT};
+ bool consider_ims_header{false};
};
struct txndata {
- char *range_value;
+ std::string range_value;
+ time_t ims_time{0};
};
-static int handle_read_request_header(TSCont, TSEvent, void *);
-static void range_header_check(TSHttpTxn txnp, struct pluginconfig *pc);
-static void handle_send_origin_request(TSCont, TSHttpTxn, struct txndata *);
-static void handle_client_send_response(TSHttpTxn, struct txndata *);
-static void handle_server_read_response(TSHttpTxn, struct txndata *);
-static int remove_header(TSMBuffer, TSMLoc, const char *, int);
-static bool set_header(TSMBuffer, TSMLoc, const char *, int, const char *, int);
-static int transaction_handler(TSCont, TSEvent, void *);
-static struct pluginconfig *create_pluginconfig(int argc, const char *argv[]);
-static void delete_pluginconfig(struct pluginconfig *);
+// Header for optional revalidation
+constexpr std::string_view X_IMS_HEADER = {"X-CRR-IMS"};
// pluginconfig struct (global plugin only)
-static struct pluginconfig *gPluginConfig = nullptr;
+pluginconfig *gPluginConfig = {nullptr};
+
+int handle_read_request_header(TSCont, TSEvent, void *);
+void range_header_check(TSHttpTxn, pluginconfig *const);
+void handle_send_origin_request(TSCont, TSHttpTxn, txndata *const);
+void handle_client_send_response(TSHttpTxn, txndata *const);
+void handle_server_read_response(TSHttpTxn, txndata *const);
+int remove_header(TSMBuffer, TSMLoc, const char *, int);
+bool set_header(TSMBuffer, TSMLoc, const char *, int, const char *, int);
+int transaction_handler(TSCont, TSEvent, void *);
+struct pluginconfig *create_pluginconfig(int argc, char *const argv[]);
+void delete_pluginconfig(pluginconfig *const);
/**
* Creates pluginconfig data structure
* Sets default parent url selection mode
* Walk plugin argument list and updates config
*/
-static struct pluginconfig *
-create_pluginconfig(int argc, const char *argv[])
+pluginconfig *
+create_pluginconfig(int argc, char *const argv[])
{
- struct pluginconfig *pc = nullptr;
+ DEBUG_LOG("Number of arguments: %d", argc);
+ for (int index = 0; index < argc; ++index) {
+ DEBUG_LOG("args[%d] = %s", index, argv[index]);
+ }
- pc = static_cast<struct pluginconfig *>(TSmalloc(sizeof(struct pluginconfig)));
+ pluginconfig *const pc = new pluginconfig;
if (nullptr == pc) {
ERROR_LOG("Can't allocate pluginconfig");
return nullptr;
}
- // Plugin uses default ATS selection (hash of URL path)
- pc->ps_mode = PS_DEFAULT;
+ static const struct option longopts[] = {
+ {const_cast<char *>("ps-cachekey"), no_argument, nullptr, 'p'},
+ {const_cast<char *>("consider-ims"), no_argument, nullptr, 'c'},
+ {nullptr, 0, nullptr, 0},
+ };
- // Walk through param list.
- for (int c = 0; c < argc; c++) {
- if (strcmp("ps_mode:cache_key_url", argv[c]) == 0) {
- pc->ps_mode = PS_CACHEKEY_URL;
+ // getopt assumes args start at '1'
+ ++argc;
+ --argv;
+
+ for (;;) {
+ int const opt = getopt_long(argc, argv, "", longopts, nullptr);
+ if (-1 == opt) {
break;
}
+
+ switch (opt) {
+ case 'p': {
+ DEBUG_LOG("Plugin modifies parent selection key");
+ pc->ps_mode = PS_CACHEKEY_URL;
+ } break;
+ case 'c': {
+ DEBUG_LOG("Plugin considers the '%.*s' header", (int)X_IMS_HEADER.size(), X_IMS_HEADER.data());
+ pc->consider_ims_header = true;
+ } break;
+ default: {
+ } break;
+ }
+ }
+
+ // Backwards compatibility
+ if (optind < argc && 0 == strcmp("ps_mode:cache_key_url", argv[optind])) {
+ DEBUG_LOG("Plugin modifies parent selection key (deprecated)");
+ pc->ps_mode = PS_CACHEKEY_URL;
}
return pc;
@@ -96,21 +135,19 @@ create_pluginconfig(int argc, const char *argv[])
/**
* Destroy pluginconfig data structure.
*/
-static void
-delete_pluginconfig(struct pluginconfig *pc)
+void
+delete_pluginconfig(pluginconfig *const pc)
{
if (nullptr != pc) {
DEBUG_LOG("Delete struct pluginconfig");
- TSfree(pc);
- pc = nullptr;
+ delete pc;
}
}
/**
* Entry point when used as a global plugin.
- *
*/
-static int
+int
handle_read_request_header(TSCont txn_contp, TSEvent event, void *edata)
{
TSHttpTxn txnp = static_cast<TSHttpTxn>(edata);
@@ -131,35 +168,36 @@ handle_read_request_header(TSCont txn_contp, TSEvent event, void *edata)
* 3. Schedules TS_HTTP_SEND_REQUEST_HDR_HOOK, TS_HTTP_SEND_RESPONSE_HDR_HOOK,
* and TS_HTTP_TXN_CLOSE_HOOK for further processing.
*/
-static void
-range_header_check(TSHttpTxn txnp, struct pluginconfig *pc)
+void
+range_header_check(TSHttpTxn txnp, pluginconfig *const pc)
{
char cache_key_url[8192] = {0};
char *req_url;
int length, url_length, cache_key_url_length;
- struct txndata *txn_state;
- TSMBuffer hdr_bufp;
- TSMLoc req_hdrs = nullptr;
- TSMLoc loc = nullptr;
+ txndata *txn_state;
+ TSMBuffer hdr_buf;
+ TSMLoc hdr_loc = nullptr;
+ TSMLoc loc = nullptr;
TSCont txn_contp;
- if (TS_SUCCESS == TSHttpTxnClientReqGet(txnp, &hdr_bufp, &req_hdrs)) {
- loc = TSMimeHdrFieldFind(hdr_bufp, req_hdrs, TS_MIME_FIELD_RANGE, TS_MIME_LEN_RANGE);
+ if (TS_SUCCESS == TSHttpTxnClientReqGet(txnp, &hdr_buf, &hdr_loc)) {
+ loc = TSMimeHdrFieldFind(hdr_buf, hdr_loc, TS_MIME_FIELD_RANGE, TS_MIME_LEN_RANGE);
if (TS_NULL_MLOC != loc) {
- const char *hdr_value = TSMimeHdrFieldValueStringGet(hdr_bufp, req_hdrs, loc, 0, &length);
+ const char *hdr_value = TSMimeHdrFieldValueStringGet(hdr_buf, hdr_loc, loc, 0, &length);
+ TSHandleMLocRelease(hdr_buf, hdr_loc, loc);
+
if (!hdr_value || length <= 0) {
DEBUG_LOG("Not a range request.");
} else {
if (nullptr == (txn_contp = TSContCreate(static_cast<TSEventFunc>(transaction_handler), nullptr))) {
ERROR_LOG("failed to create the transaction handler continuation.");
} else {
- txn_state = static_cast<struct txndata *>(TSmalloc(sizeof(struct txndata)));
- txn_state->range_value = TSstrndup(hdr_value, length);
- DEBUG_LOG("length: %d, txn_state->range_value: %s", length, txn_state->range_value);
- txn_state->range_value[length] = '\0'; // workaround for bug in core
-
+ txn_state = new txndata;
+ std::string &rv = txn_state->range_value;
+ rv.assign(hdr_value, length);
+ DEBUG_LOG("length: %d, txn_state->range_value: %s", length, rv.c_str());
req_url = TSHttpTxnEffectiveUrlStringGet(txnp, &url_length);
- cache_key_url_length = snprintf(cache_key_url, 8192, "%s-%s", req_url, txn_state->range_value);
+ cache_key_url_length = snprintf(cache_key_url, 8192, "%s-%s", req_url, rv.c_str());
DEBUG_LOG("Rewriting cache URL for %s to %s", req_url, cache_key_url);
if (req_url != nullptr) {
TSfree(req_url);
@@ -170,24 +208,38 @@ range_header_check(TSHttpTxn txnp, struct pluginconfig *pc)
DEBUG_LOG("failed to change the cache url to %s.", cache_key_url);
}
- // Optionally set the parent_selection_url to the cache_key url or path
- if (nullptr != pc && PS_DEFAULT != pc->ps_mode) {
- TSMLoc ps_loc = nullptr;
-
- if (PS_CACHEKEY_URL == pc->ps_mode) {
- const char *start = cache_key_url;
- const char *end = cache_key_url + cache_key_url_length;
- if (TS_SUCCESS == TSUrlCreate(hdr_bufp, &ps_loc) &&
- TS_PARSE_DONE == TSUrlParse(hdr_bufp, ps_loc, &start, end) && // This should always succeed.
- TS_SUCCESS == TSHttpTxnParentSelectionUrlSet(txnp, hdr_bufp, ps_loc)) {
- DEBUG_LOG("Set Parent Selection URL to cache_key_url: %s", cache_key_url);
- TSHandleMLocRelease(hdr_bufp, TS_NULL_MLOC, ps_loc);
+ if (nullptr != pc) {
+ // Optionally set the parent_selection_url to the cache_key url or path
+ if (PS_DEFAULT != pc->ps_mode) {
+ TSMLoc ps_loc = nullptr;
+
+ if (PS_CACHEKEY_URL == pc->ps_mode) {
+ const char *start = cache_key_url;
+ const char *end = cache_key_url + cache_key_url_length;
+ if (TS_SUCCESS == TSUrlCreate(hdr_buf, &ps_loc) &&
+ TS_PARSE_DONE == TSUrlParse(hdr_buf, ps_loc, &start, end) && // This should always succeed.
+ TS_SUCCESS == TSHttpTxnParentSelectionUrlSet(txnp, hdr_buf, ps_loc)) {
+ DEBUG_LOG("Set Parent Selection URL to cache_key_url: %s", cache_key_url);
+ TSHandleMLocRelease(hdr_buf, TS_NULL_MLOC, ps_loc);
+ }
+ }
+ }
+
+ // optionally consider an X-CRR-IMS header
+ if (pc->consider_ims_header) {
+ TSMLoc const imsloc = TSMimeHdrFieldFind(hdr_buf, hdr_loc, X_IMS_HEADER.data(), X_IMS_HEADER.size());
+ if (TS_NULL_MLOC != imsloc) {
+ time_t const itime = TSMimeHdrFieldValueDateGet(hdr_buf, hdr_loc, imsloc);
+ TSHandleMLocRelease(hdr_buf, hdr_loc, imsloc);
+ if (0 < itime) {
+ txn_state->ims_time = itime;
+ }
}
}
}
// remove the range request header.
- if (remove_header(hdr_bufp, req_hdrs, TS_MIME_FIELD_RANGE, TS_MIME_LEN_RANGE) > 0) {
+ if (remove_header(hdr_buf, hdr_loc, TS_MIME_FIELD_RANGE, TS_MIME_LEN_RANGE) > 0) {
DEBUG_LOG("Removed the Range: header from the request.");
}
@@ -196,13 +248,18 @@ range_header_check(TSHttpTxn txnp, struct pluginconfig *pc)
TSHttpTxnHookAdd(txnp, TS_HTTP_SEND_RESPONSE_HDR_HOOK, txn_contp);
TSHttpTxnHookAdd(txnp, TS_HTTP_TXN_CLOSE_HOOK, txn_contp);
DEBUG_LOG("Added TS_HTTP_SEND_REQUEST_HDR_HOOK, TS_HTTP_SEND_RESPONSE_HDR_HOOK, and TS_HTTP_TXN_CLOSE_HOOK");
+
+ if (0 < txn_state->ims_time) {
+ TSHttpTxnHookAdd(txnp, TS_HTTP_CACHE_LOOKUP_COMPLETE_HOOK, txn_contp);
+ DEBUG_LOG("Also Added TS_HTTP_CACHE_LOOKUP_COMPLETE_HOOK");
+ }
}
}
- TSHandleMLocRelease(hdr_bufp, req_hdrs, loc);
+ // TSHandleMLocRelease(hdr_buf, hdr_loc, loc);
} else {
DEBUG_LOG("no range request header.");
}
- TSHandleMLocRelease(hdr_bufp, TS_NULL_MLOC, req_hdrs);
+ TSHandleMLocRelease(hdr_buf, TS_NULL_MLOC, hdr_loc);
} else {
DEBUG_LOG("failed to retrieve the server request");
}
@@ -212,41 +269,44 @@ range_header_check(TSHttpTxn txnp, struct pluginconfig *pc)
* Restores the range request header if the request must be
* satisfied from the origin and schedules the TS_READ_RESPONSE_HDR_HOOK.
*/
-static void
-handle_send_origin_request(TSCont contp, TSHttpTxn txnp, struct txndata *txn_state)
+void
+handle_send_origin_request(TSCont contp, TSHttpTxn txnp, txndata *const txn_state)
{
- TSMBuffer hdr_bufp;
- TSMLoc req_hdrs = nullptr;
+ TSMBuffer hdr_buf;
+ TSMLoc hdr_loc = nullptr;
+
+ std::string const &rv = txn_state->range_value;
- if (TS_SUCCESS == TSHttpTxnServerReqGet(txnp, &hdr_bufp, &req_hdrs) && txn_state->range_value != nullptr) {
- if (set_header(hdr_bufp, req_hdrs, TS_MIME_FIELD_RANGE, TS_MIME_LEN_RANGE, txn_state->range_value,
- strlen(txn_state->range_value))) {
- DEBUG_LOG("Added range header: %s", txn_state->range_value);
+ if (TS_SUCCESS == TSHttpTxnServerReqGet(txnp, &hdr_buf, &hdr_loc) && !rv.empty()) {
+ if (set_header(hdr_buf, hdr_loc, TS_MIME_FIELD_RANGE, TS_MIME_LEN_RANGE, rv.data(), rv.length())) {
+ DEBUG_LOG("Added range header: %s", rv.c_str());
TSHttpTxnHookAdd(txnp, TS_HTTP_READ_RESPONSE_HDR_HOOK, contp);
}
}
- TSHandleMLocRelease(hdr_bufp, TS_NULL_MLOC, req_hdrs);
+ TSHandleMLocRelease(hdr_buf, TS_NULL_MLOC, hdr_loc);
}
/**
* Changes the response code back to a 206 Partial content before
* replying to the client that requested a range.
*/
-static void
-handle_client_send_response(TSHttpTxn txnp, struct txndata *txn_state)
+void
+handle_client_send_response(TSHttpTxn txnp, txndata *const txn_state)
{
bool partial_content_reason = false;
char *p;
int length;
- TSMBuffer response, hdr_bufp;
- TSMLoc resp_hdr, req_hdrs = nullptr;
+ TSMBuffer resp_buf = nullptr;
+ TSMBuffer req_buf = nullptr;
+ TSMLoc resp_loc = nullptr;
+ TSMLoc req_loc = nullptr;
- TSReturnCode result = TSHttpTxnClientRespGet(txnp, &response, &resp_hdr);
+ TSReturnCode result = TSHttpTxnClientRespGet(txnp, &resp_buf, &resp_loc);
DEBUG_LOG("result: %d", result);
if (TS_SUCCESS == result) {
- TSHttpStatus status = TSHttpHdrStatusGet(response, resp_hdr);
+ TSHttpStatus status = TSHttpHdrStatusGet(resp_buf, resp_loc);
// a cached result will have a TS_HTTP_OK with a 'Partial Content' reason
- if ((p = const_cast<char *>(TSHttpHdrReasonGet(response, resp_hdr, &length))) != nullptr) {
+ if ((p = const_cast<char *>(TSHttpHdrReasonGet(resp_buf, resp_loc, &length))) != nullptr) {
if ((length == 15) && (0 == strncasecmp(p, "Partial Content", length))) {
partial_content_reason = true;
}
@@ -254,23 +314,23 @@ handle_client_send_response(TSHttpTxn txnp, struct txndata *txn_state)
DEBUG_LOG("%d %.*s", status, length, p);
if (TS_HTTP_STATUS_OK == status && partial_content_reason) {
DEBUG_LOG("Got TS_HTTP_STATUS_OK.");
- TSHttpHdrStatusSet(response, resp_hdr, TS_HTTP_STATUS_PARTIAL_CONTENT);
+ TSHttpHdrStatusSet(resp_buf, resp_loc, TS_HTTP_STATUS_PARTIAL_CONTENT);
DEBUG_LOG("Set response header to TS_HTTP_STATUS_PARTIAL_CONTENT.");
}
}
+ std::string const &rv = txn_state->range_value;
// add the range request header back in so that range requests may be logged.
- if (TS_SUCCESS == TSHttpTxnClientReqGet(txnp, &hdr_bufp, &req_hdrs) && txn_state->range_value != nullptr) {
- if (set_header(hdr_bufp, req_hdrs, TS_MIME_FIELD_RANGE, TS_MIME_LEN_RANGE, txn_state->range_value,
- strlen(txn_state->range_value))) {
- DEBUG_LOG("added range header: %s", txn_state->range_value);
+ if (TS_SUCCESS == TSHttpTxnClientReqGet(txnp, &req_buf, &req_loc) && !rv.empty()) {
+ if (set_header(req_buf, req_loc, TS_MIME_FIELD_RANGE, TS_MIME_LEN_RANGE, rv.data(), rv.length())) {
+ DEBUG_LOG("added range header: %s", rv.c_str());
} else {
DEBUG_LOG("set_header() failed.");
}
} else {
DEBUG_LOG("failed to get Request Headers");
}
- TSHandleMLocRelease(response, TS_NULL_MLOC, resp_hdr);
- TSHandleMLocRelease(hdr_bufp, TS_NULL_MLOC, req_hdrs);
+ TSHandleMLocRelease(resp_buf, TS_NULL_MLOC, resp_loc);
+ TSHandleMLocRelease(req_buf, TS_NULL_MLOC, req_loc);
}
/**
@@ -278,20 +338,20 @@ handle_client_send_response(TSHttpTxn txnp, struct txndata *txn_state)
* the response code from a 206 Partial content to a 200 OK so that
* the response will be written to cache.
*/
-static void
-handle_server_read_response(TSHttpTxn txnp, struct txndata *txn_state)
+void
+handle_server_read_response(TSHttpTxn txnp, txndata *const txn_state)
{
- TSMBuffer response;
- TSMLoc resp_hdr;
+ TSMBuffer resp_buf = nullptr;
+ TSMLoc resp_loc = nullptr;
TSHttpStatus status;
- if (TS_SUCCESS == TSHttpTxnServerRespGet(txnp, &response, &resp_hdr)) {
- status = TSHttpHdrStatusGet(response, resp_hdr);
+ if (TS_SUCCESS == TSHttpTxnServerRespGet(txnp, &resp_buf, &resp_loc)) {
+ status = TSHttpHdrStatusGet(resp_buf, resp_loc);
if (TS_HTTP_STATUS_PARTIAL_CONTENT == status) {
DEBUG_LOG("Got TS_HTTP_STATUS_PARTIAL_CONTENT.");
- TSHttpHdrStatusSet(response, resp_hdr, TS_HTTP_STATUS_OK);
+ TSHttpHdrStatusSet(resp_buf, resp_loc, TS_HTTP_STATUS_OK);
DEBUG_LOG("Set response header to TS_HTTP_STATUS_OK.");
- bool cacheable = TSHttpTxnIsCacheable(txnp, nullptr, response);
+ bool cacheable = TSHttpTxnIsCacheable(txnp, nullptr, resp_buf);
DEBUG_LOG("range is cacheable: %d", cacheable);
} else if (TS_HTTP_STATUS_OK == status) {
DEBUG_LOG("The origin does not support range requests, attempting to disable cache write.");
@@ -302,7 +362,7 @@ handle_server_read_response(TSHttpTxn txnp, struct txndata *txn_state)
}
}
}
- TSHandleMLocRelease(response, TS_NULL_MLOC, resp_hdr);
+ TSHandleMLocRelease(resp_buf, TS_NULL_MLOC, resp_loc);
}
/**
@@ -311,18 +371,18 @@ handle_server_read_response(TSHttpTxn txnp, struct txndata *txn_state)
*
* From background_fetch.cc
*/
-static int
-remove_header(TSMBuffer bufp, TSMLoc hdr_loc, const char *header, int len)
+int
+remove_header(TSMBuffer buf, TSMLoc hdr_loc, const char *header, int len)
{
- TSMLoc field = TSMimeHdrFieldFind(bufp, hdr_loc, header, len);
+ TSMLoc field = TSMimeHdrFieldFind(buf, hdr_loc, header, len);
int cnt = 0;
while (field) {
- TSMLoc tmp = TSMimeHdrFieldNextDup(bufp, hdr_loc, field);
+ TSMLoc tmp = TSMimeHdrFieldNextDup(buf, hdr_loc, field);
++cnt;
- TSMimeHdrFieldDestroy(bufp, hdr_loc, field);
- TSHandleMLocRelease(bufp, hdr_loc, field);
+ TSMimeHdrFieldDestroy(buf, hdr_loc, field);
+ TSHandleMLocRelease(buf, hdr_loc, field);
field = tmp;
}
@@ -336,25 +396,25 @@ remove_header(TSMBuffer bufp, TSMLoc hdr_loc, const char *header, int len)
*
* From background_fetch.cc
*/
-static bool
-set_header(TSMBuffer bufp, TSMLoc hdr_loc, const char *header, int len, const char *val, int val_len)
+bool
+set_header(TSMBuffer buf, TSMLoc hdr_loc, const char *header, int len, const char *val, int val_len)
{
- if (!bufp || !hdr_loc || !header || len <= 0 || !val || val_len <= 0) {
+ if (!buf || !hdr_loc || !header || len <= 0 || !val || val_len <= 0) {
return false;
}
DEBUG_LOG("header: %s, len: %d, val: %s, val_len: %d", header, len, val, val_len);
bool ret = false;
- TSMLoc field_loc = TSMimeHdrFieldFind(bufp, hdr_loc, header, len);
+ TSMLoc field_loc = TSMimeHdrFieldFind(buf, hdr_loc, header, len);
if (!field_loc) {
// No existing header, so create one
- if (TS_SUCCESS == TSMimeHdrFieldCreateNamed(bufp, hdr_loc, header, len, &field_loc)) {
- if (TS_SUCCESS == TSMimeHdrFieldValueStringSet(bufp, hdr_loc, field_loc, -1, val, val_len)) {
- TSMimeHdrFieldAppend(bufp, hdr_loc, field_loc);
+ if (TS_SUCCESS == TSMimeHdrFieldCreateNamed(buf, hdr_loc, header, len, &field_loc)) {
+ if (TS_SUCCESS == TSMimeHdrFieldValueStringSet(buf, hdr_loc, field_loc, -1, val, val_len)) {
+ TSMimeHdrFieldAppend(buf, hdr_loc, field_loc);
ret = true;
}
- TSHandleMLocRelease(bufp, hdr_loc, field_loc);
+ TSHandleMLocRelease(buf, hdr_loc, field_loc);
}
} else {
TSMLoc tmp = nullptr;
@@ -363,14 +423,14 @@ set_header(TSMBuffer bufp, TSMLoc hdr_loc, const char *header, int len, const ch
while (field_loc) {
if (first) {
first = false;
- if (TS_SUCCESS == TSMimeHdrFieldValueStringSet(bufp, hdr_loc, field_loc, -1, val, val_len)) {
+ if (TS_SUCCESS == TSMimeHdrFieldValueStringSet(buf, hdr_loc, field_loc, -1, val, val_len)) {
ret = true;
}
} else {
- TSMimeHdrFieldDestroy(bufp, hdr_loc, field_loc);
+ TSMimeHdrFieldDestroy(buf, hdr_loc, field_loc);
}
- tmp = TSMimeHdrFieldNextDup(bufp, hdr_loc, field_loc);
- TSHandleMLocRelease(bufp, hdr_loc, field_loc);
+ tmp = TSMimeHdrFieldNextDup(buf, hdr_loc, field_loc);
+ TSHandleMLocRelease(buf, hdr_loc, field_loc);
field_loc = tmp;
}
}
@@ -378,6 +438,91 @@ set_header(TSMBuffer bufp, TSMLoc hdr_loc, const char *header, int len, const ch
return ret;
}
+time_t
+get_date_from_cached_hdr(TSHttpTxn txn)
+{
+ TSMBuffer buf;
+ TSMLoc hdr_loc, date_loc;
+ time_t date = 0;
+
+ if (TSHttpTxnCachedRespGet(txn, &buf, &hdr_loc) == TS_SUCCESS) {
+ date_loc = TSMimeHdrFieldFind(buf, hdr_loc, TS_MIME_FIELD_DATE, TS_MIME_LEN_DATE);
+ if (date_loc != TS_NULL_MLOC) {
+ date = TSMimeHdrFieldValueDateGet(buf, hdr_loc, date_loc);
+ TSHandleMLocRelease(buf, hdr_loc, date_loc);
+ }
+ TSHandleMLocRelease(buf, TS_NULL_MLOC, hdr_loc);
+ }
+
+ return date;
+}
+
+/**
+ * Handle a special IMS request.
+ */
+void
+handle_cache_lookup_complete(TSHttpTxn txnp, txndata *const txn_state)
+{
+ int cachestat;
+ if (TS_SUCCESS == TSHttpTxnCacheLookupStatusGet(txnp, &cachestat)) {
+ if (TS_CACHE_LOOKUP_HIT_FRESH == cachestat) {
+ time_t const ch_time = get_date_from_cached_hdr(txnp);
+ if (ch_time < txn_state->ims_time) {
+ TSHttpTxnCacheLookupStatusSet(txnp, TS_CACHE_LOOKUP_HIT_STALE);
+ if (TSIsDebugTagSet(PLUGIN_NAME)) {
+ int url_len = 0;
+ char *const req_url = TSHttpTxnEffectiveUrlStringGet(txnp, &url_len);
+ if (nullptr != req_url) {
+ std::string const &rv = txn_state->range_value;
+ DEBUG_LOG("Forced revalidate %.*s-%s", url_len, req_url, rv.c_str());
+
+ TSfree(req_url);
+ }
+ }
+ }
+ }
+ }
+}
+
+/**
+ * Transaction event handler.
+ */
+int
+transaction_handler(TSCont contp, TSEvent event, void *edata)
+{
+ TSHttpTxn txnp = static_cast<TSHttpTxn>(edata);
+ txndata *const txn_state = static_cast<txndata *>(TSContDataGet(contp));
+
+ switch (event) {
+ case TS_EVENT_HTTP_READ_RESPONSE_HDR:
+ handle_server_read_response(txnp, txn_state);
+ break;
+ case TS_EVENT_HTTP_SEND_REQUEST_HDR:
+ handle_send_origin_request(contp, txnp, txn_state);
+ break;
+ case TS_EVENT_HTTP_SEND_RESPONSE_HDR:
+ handle_client_send_response(txnp, txn_state);
+ break;
+ case TS_EVENT_HTTP_CACHE_LOOKUP_COMPLETE:
+ handle_cache_lookup_complete(txnp, txn_state);
+ break;
+ case TS_EVENT_HTTP_TXN_CLOSE:
+ if (txn_state != nullptr) {
+ TSContDataSet(contp, nullptr);
+ delete txn_state;
+ }
+ TSContDestroy(contp);
+ break;
+ default:
+ TSAssert(!"Unexpected event");
+ break;
+ }
+ TSHttpTxnReenable(txnp, TS_EVENT_HTTP_CONTINUE);
+ return 0;
+}
+
+} // namespace
+
/**
* Remap initialization.
*/
@@ -411,11 +556,10 @@ TSRemapNewInstance(int argc, char *argv[], void **ih, char * /*errbuf */, int /*
}
// Skip over the Remap params
- const char **plugin_argv = const_cast<const char **>(argv + 2);
- argc -= 2;
+ char *const *plugin_argv = const_cast<char *const *>(argv);
// Parse the argument list.
- *ih = create_pluginconfig(argc, plugin_argv);
+ *ih = static_cast<void *>(create_pluginconfig(argc - 2, plugin_argv + 2));
if (*ih == nullptr) {
ERROR_LOG("Can't create pluginconfig");
@@ -430,7 +574,7 @@ TSRemapNewInstance(int argc, char *argv[], void **ih, char * /*errbuf */, int /*
void
TSRemapDeleteInstance(void *ih)
{
- struct pluginconfig *pc = static_cast<struct pluginconfig *>(ih);
+ pluginconfig *const pc = static_cast<pluginconfig *>(ih);
if (nullptr != pc) {
delete_pluginconfig(pc);
@@ -443,7 +587,7 @@ TSRemapDeleteInstance(void *ih)
TSRemapStatus
TSRemapDoRemap(void *ih, TSHttpTxn txnp, TSRemapRequestInfo * /* rri */)
{
- struct pluginconfig *pc = static_cast<struct pluginconfig *>(ih);
+ pluginconfig *const pc = static_cast<pluginconfig *>(ih);
range_header_check(txnp, pc);
@@ -470,11 +614,9 @@ TSPluginInit(int argc, const char *argv[])
}
if (nullptr == gPluginConfig) {
- if (argc > 1) {
- // Skip ahead of first param (name of traffic server plugin shared object)
- const char **plugin_argv = const_cast<const char **>(argv + 1);
- argc -= 1;
- gPluginConfig = create_pluginconfig(argc, plugin_argv);
+ if (1 < argc) {
+ char *const *plugin_argv = const_cast<char *const *>(argv);
+ gPluginConfig = create_pluginconfig(argc - 1, plugin_argv + 1);
}
}
@@ -485,39 +627,3 @@ TSPluginInit(int argc, const char *argv[])
TSHttpHookAdd(TS_HTTP_READ_REQUEST_HDR_HOOK, txnp_cont);
}
}
-
-/**
- * Transaction event handler.
- */
-static int
-transaction_handler(TSCont contp, TSEvent event, void *edata)
-{
- TSHttpTxn txnp = static_cast<TSHttpTxn>(edata);
- struct txndata *txn_state = static_cast<struct txndata *>(TSContDataGet(contp));
-
- switch (event) {
- case TS_EVENT_HTTP_READ_RESPONSE_HDR:
- handle_server_read_response(txnp, txn_state);
- break;
- case TS_EVENT_HTTP_SEND_REQUEST_HDR:
- handle_send_origin_request(contp, txnp, txn_state);
- break;
- case TS_EVENT_HTTP_SEND_RESPONSE_HDR:
- handle_client_send_response(txnp, txn_state);
- break;
- case TS_EVENT_HTTP_TXN_CLOSE:
- if (txn_state != nullptr && txn_state->range_value != nullptr) {
- TSfree(txn_state->range_value);
- }
- if (txn_state != nullptr) {
- TSfree(txn_state);
- }
- TSContDestroy(contp);
- break;
- default:
- TSAssert(!"Unexpected event");
- break;
- }
- TSHttpTxnReenable(txnp, TS_EVENT_HTTP_CONTINUE);
- return 0;
-}
diff --git a/tests/gold_tests/pluginTest/cache_range_requests/cache_range_requests.test.py b/tests/gold_tests/pluginTest/cache_range_requests/cache_range_requests.test.py
index 49aaf6e..c46b127 100644
--- a/tests/gold_tests/pluginTest/cache_range_requests/cache_range_requests.test.py
+++ b/tests/gold_tests/pluginTest/cache_range_requests/cache_range_requests.test.py
@@ -76,7 +76,7 @@ res_full = {"headers":
"HTTP/1.1 200 OK\r\n" +
"Cache-Control: max-age=500\r\n" +
"Connection: close\r\n" +
- 'Etag: "772102f4-56f4bc1e6d417"\r\n' +
+ 'Etag: "path"\r\n' +
"Last-Modified: Sat, 23 Jun 2018 09:27:29 GMT\r\n" +
"\r\n",
"timestamp": "1469733493.993",
@@ -107,7 +107,7 @@ res_inner = {"headers":
"Cache-Control: max-age=500\r\n" +
"Content-Range: bytes {0}/{1}\r\n".format(inner_str, bodylen) +
"Connection: close\r\n" +
- 'Etag: "772102f4-56f4bc1e6d417"\r\n' +
+ 'Etag: "path"\r\n' +
"Last-Modified: Sat, 23 Jun 2018 09:27:29 GMT\r\n" +
"\r\n",
"timestamp": "1469733493.993",
@@ -135,7 +135,7 @@ res_frange = {"headers":
"Cache-Control: max-age=500\r\n" +
"Content-Range: bytes 0-{0}/{0}\r\n".format(bodylen) +
"Connection: close\r\n" +
- 'Etag: "772102f4-56f4bc1e6d417"\r\n' +
+ 'Etag: "path"\r\n' +
"Last-Modified: Sat, 23 Jun 2018 09:27:29 GMT\r\n" +
"\r\n",
"timestamp": "1469733493.993",
@@ -163,7 +163,7 @@ res_last = {"headers":
"Cache-Control: max-age=200\r\n" +
"Content-Range: bytes {0}-{1}/{1}\r\n".format(bodylen - 5, bodylen) +
"Connection: close\r\n" +
- 'Etag: "772102f4-56f4bc1e6d417"\r\n' +
+ 'Etag: "path"\r\n' +
"Last-Modified: Sat, 23 Jun 2018 09:27:29 GMT\r\n" +
"\r\n",
"timestamp": "1469733493.993",
@@ -191,7 +191,7 @@ res_pselect = {"headers":
"Cache-Control: max-age=200\r\n" +
"Content-Range: bytes {}/19\r\n".format(pselect_str) +
"Connection: close\r\n" +
- 'Etag: "772102f4-56f4bc1e6d417"\r\n' +
+ 'Etag: "path"\r\n' +
"Last-Modified: Sat, 23 Jun 2018 09:27:29 GMT\r\n" +
"\r\n",
"timestamp": "1469733493.993",
@@ -200,11 +200,30 @@ res_pselect = {"headers":
server.addResponse("sessionlog.json", req_pselect, res_pselect)
+req_psd = {"headers":
+ "GET /path HTTP/1.1\r\n" +
+ "Host: psd\r\n" +
+ "Accept: */*\r\n" +
+ "Range: bytes={}\r\n".format(pselect_str) +
+ "uuid: pselect\r\n" +
+ "\r\n",
+ "timestamp": "1469733493.993",
+ "body": ""
+}
+
+server.addResponse("sessionlog.json", req_psd, res_pselect)
+
# cache range requests plugin remap
ts.Disk.remap_config.AddLines([
'map http://www.example.com http://127.0.0.1:{}'.format(server.Variables.Port) +
' @plugin=cache_range_requests.so',
+
+ # parent select cache key option
'map http://parentselect http://127.0.0.1:{}'.format(server.Variables.Port) +
+ ' @plugin=cache_range_requests.so @pparam=--ps-cachekey',
+
+ # deprecated
+ 'map http://psd http://127.0.0.1:{}'.format(server.Variables.Port) +
' @plugin=cache_range_requests.so @pparam=ps_mode:cache_key_url',
])
@@ -338,3 +357,16 @@ ps.ReturnCode = 0
ps.Streams.stdout = "gold/pselect_none.stdout.gold"
tr.StillRunningAfter = ts
tr.StillRunningAfter = server
+
+# 11 Test - cache_key_url request -- deprecated
+tr = Test.AddTestRun("cache_key_url request - dprecated")
+ps = tr.Processes.Default
+ps.Command = curl_and_args + ' http://psd/path -r {} -H "uuid: pselect"'.format(pselect_str)
+ps.ReturnCode = 0
+ps.Streams.stdout = "gold/pselect.stdout.gold"
+ps.Streams.stdout.Content = Testers.ContainsExpression(
+ "X-ParentSelection-Key: .*-bytes=",
+ "expected bytes in parent selection key",
+)
+tr.StillRunningAfter = ts
+tr.StillRunningAfter = server
diff --git a/tests/gold_tests/pluginTest/cache_range_requests/cache_range_requests_ims.test.py b/tests/gold_tests/pluginTest/cache_range_requests/cache_range_requests_ims.test.py
new file mode 100644
index 0000000..f249700
--- /dev/null
+++ b/tests/gold_tests/pluginTest/cache_range_requests/cache_range_requests_ims.test.py
@@ -0,0 +1,141 @@
+'''
+'''
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import os
+import time
+
+Test.Summary = '''
+cache_range_requests X-CRR-IMS plugin test
+'''
+
+## Test description:
+# Preload the cache with the entire asset to be range requested.
+# Reload remap rule with cache_range_requests plugin
+# Request content through the cache_range_requests plugin
+
+Test.SkipUnless(
+ Condition.PluginExists('cache_range_requests.so'),
+ Condition.PluginExists('xdebug.so'),
+)
+Test.ContinueOnFail = False
+Test.testName = "cache_range_requests"
+
+# Define and configure ATS
+ts = Test.MakeATSProcess("ts", command="traffic_server")
+
+# Define and configure origin server
+server = Test.MakeOriginServer("server")
+
+# default root
+req_chk = {"headers":
+ "GET / HTTP/1.1\r\n" +
+ "Host: www.example.com\r\n" +
+ "uuid: none\r\n" +
+ "\r\n",
+ "timestamp": "1469733493.993",
+ "body": ""
+}
+
+res_chk = {"headers":
+ "HTTP/1.1 200 OK\r\n" +
+ "Connection: close\r\n" +
+ "\r\n",
+ "timestamp": "1469733493.993",
+ "body": ""
+}
+
+server.addResponse("sessionlog.json", req_chk, res_chk)
+
+body = "lets go surfin now"
+bodylen = len(body)
+
+req_full = {"headers":
+ "GET /path HTTP/1.1\r\n" +
+ "Host: www.example.com\r\n" +
+ "Accept: */*\r\n" +
+ "Range: bytes=0-\r\n" +
+ "\r\n",
+ "timestamp": "1469733493.993",
+ "body": ""
+}
+
+res_full = {"headers":
+ "HTTP/1.1 206 Partial Content\r\n" +
+ "Accept-Ranges: bytes\r\n" +
+ "Cache-Control: max-age=500\r\n" +
+ "Content-Range: bytes 0-{0}/{0}\r\n".format(bodylen) +
+ "Connection: close\r\n" +
+ 'Etag: "772102f4-56f4bc1e6d417"\r\n' +
+ "Last-Modified: Sat, 23 Jun 2018 09:27:29 GMT\r\n" +
+ "\r\n",
+ "timestamp": "1469733493.993",
+ "body": body
+}
+
+server.addResponse("sessionlog.json", req_full, res_full)
+
+# cache range requests plugin remap
+ts.Disk.remap_config.AddLine(
+ 'map http://www.example.com http://127.0.0.1:{}'.format(server.Variables.Port) +
+ ' @plugin=cache_range_requests.so @pparam=--consider-ims',
+)
+
+# cache debug
+ts.Disk.plugin_config.AddLine('xdebug.so')
+
+# minimal configuration
+ts.Disk.records_config.update({
+ 'proxy.config.diags.debug.enabled': 1,
+ 'proxy.config.diags.debug.tags': 'cache_range_requests',
+ 'proxy.config.http.cache.http': 1,
+ 'proxy.config.http.wait_for_cache': 1,
+})
+
+curl_and_args = 'curl -s -D /dev/stdout -o /dev/stderr -x localhost:{} -H "x-debug: x-cache"'.format(ts.Variables.port)
+
+# 0 Test - Fetch whole asset into cache
+tr = Test.AddTestRun("0- range cache load")
+ps = tr.Processes.Default
+ps.StartBefore(server, ready=When.PortOpen(server.Variables.Port))
+ps.StartBefore(Test.Processes.ts, ready=When.PortOpen(ts.Variables.port))
+ps.Command = curl_and_args + ' http://www.example.com/path -r 0-'
+ps.ReturnCode = 0
+ps.Streams.stdout.Content = Testers.ContainsExpression("X-Cache: miss", "expected cache miss for load")
+tr.StillRunningAfter = ts
+
+
+# set up the IMS date field (go in the future) RFC 2616
+futuretime = time.time() + 100 # seconds
+futurestr = time.strftime("%a, %d %b %Y %H:%M:%S GMT", time.gmtime(futuretime))
+
+# test inner range
+# 1 Test - Fetch range into cache
+tr = Test.AddTestRun("0- cache hit check")
+ps = tr.Processes.Default
+ps.Command = curl_and_args + ' http://www.example.com/path -r 0-'.format(futurestr)
+ps.ReturnCode = 0
+ps.Streams.stdout.Content = Testers.ContainsExpression("X-Cache: hit", "expected cache hit")
+tr.StillRunningAfter = ts
+
+# 2 Test - Ensure X-CRR-IMS header results in hit-stale
+tr = Test.AddTestRun("0- range X-CRR-IMS check")
+ps = tr.Processes.Default
+ps.Command = curl_and_args + ' http://www.example.com/path -r 0- -H "X-CRR-IMS: {}"'.format(futurestr)
+ps.ReturnCode = 0
+ps.Streams.stdout.Content = Testers.ContainsExpression("X-Cache: hit-stale", "expected cache hit-stale")
+tr.StillRunningAfter = ts
diff --git a/tests/gold_tests/pluginTest/cache_range_requests/gold/frange.stdout.gold b/tests/gold_tests/pluginTest/cache_range_requests/gold/frange.stdout.gold
index 6dae639..e108e5c 100644
--- a/tests/gold_tests/pluginTest/cache_range_requests/gold/frange.stdout.gold
+++ b/tests/gold_tests/pluginTest/cache_range_requests/gold/frange.stdout.gold
@@ -2,7 +2,7 @@ HTTP/1.1 206 Partial Content
Accept-Ranges: bytes
Cache-Control: max-age=500
Content-Range: bytes 0-18/18
-Etag: "772102f4-56f4bc1e6d417"
+Etag: "path"
Last-Modified: Sat, 23 Jun 2018 09:27:29 GMT
Content-Length: 18
Date: ``
diff --git a/tests/gold_tests/pluginTest/cache_range_requests/gold/full.stdout.gold b/tests/gold_tests/pluginTest/cache_range_requests/gold/full.stdout.gold
index a485d26..303f162 100644
--- a/tests/gold_tests/pluginTest/cache_range_requests/gold/full.stdout.gold
+++ b/tests/gold_tests/pluginTest/cache_range_requests/gold/full.stdout.gold
@@ -1,6 +1,6 @@
HTTP/1.1 200 OK
Cache-Control: max-age=500
-Etag: "772102f4-56f4bc1e6d417"
+Etag: "path"
Last-Modified: Sat, 23 Jun 2018 09:27:29 GMT
Content-Length: 18
Date: ``
diff --git a/tests/gold_tests/pluginTest/cache_range_requests/gold/inner.stdout.gold b/tests/gold_tests/pluginTest/cache_range_requests/gold/inner.stdout.gold
index 0628bfa..56aa6d1 100644
--- a/tests/gold_tests/pluginTest/cache_range_requests/gold/inner.stdout.gold
+++ b/tests/gold_tests/pluginTest/cache_range_requests/gold/inner.stdout.gold
@@ -2,7 +2,7 @@ HTTP/1.1 206 Partial Content
Accept-Ranges: bytes
Cache-Control: max-age=500
Content-Range: bytes 7-15/18
-Etag: "772102f4-56f4bc1e6d417"
+Etag: "path"
Last-Modified: Sat, 23 Jun 2018 09:27:29 GMT
Content-Length: 7
Date: ``
diff --git a/tests/gold_tests/pluginTest/cache_range_requests/gold/last.stdout.gold b/tests/gold_tests/pluginTest/cache_range_requests/gold/last.stdout.gold
index 5e2da00..45c17fa 100644
--- a/tests/gold_tests/pluginTest/cache_range_requests/gold/last.stdout.gold
+++ b/tests/gold_tests/pluginTest/cache_range_requests/gold/last.stdout.gold
@@ -2,7 +2,7 @@ HTTP/1.1 206 Partial Content
Accept-Ranges: bytes
Cache-Control: max-age=500
Content-Range: bytes 12-18/18
-Etag: "772102f4-56f4bc1e6d417"
+Etag: "path"
Last-Modified: Sat, 23 Jun 2018 09:27:29 GMT
Content-Length: 5
Date: ``
diff --git a/tests/gold_tests/pluginTest/cache_range_requests/gold/pselect.stdout.gold b/tests/gold_tests/pluginTest/cache_range_requests/gold/pselect.stdout.gold
index 11902d7..98165d4 100644
--- a/tests/gold_tests/pluginTest/cache_range_requests/gold/pselect.stdout.gold
+++ b/tests/gold_tests/pluginTest/cache_range_requests/gold/pselect.stdout.gold
@@ -2,7 +2,7 @@ HTTP/1.1 206 Partial Content
Accept-Ranges: bytes
Cache-Control: max-age=500
Content-Range: bytes 1-10/18
-Etag: "772102f4-56f4bc1e6d417"
+Etag: "path"
Last-Modified: Sat, 23 Jun 2018 09:27:29 GMT
Content-Length: 9
Date: ``