You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@trafficserver.apache.org by zw...@apache.org on 2021/02/03 23:45:55 UTC
[trafficserver] branch 9.1.x updated (3e222f5 -> de8c125)
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a change to branch 9.1.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git.
from 3e222f5 Do not write to the cache if the plugin decides not to write to the cache (#7461)
new 255cd5e Fix comment in include/tscore/Filenames.h. (#7457)
new 7249341 Fix heap use after free in DNSProcessor::getby() (#3871)
new cd434e7 Add synchronization between UDPNetProcessor::UDPBind in main Thread and initialize_thread_for_udp_net in ET_UDP Thread (#7407)
new 3ba9fbd Generalize SNI support (#6870)
new c1c8734 Do not provide a stale negative cache (#7422)
new f922261 CacheRead: clear dir entry if doc is found to be truncated (#7064)
new 2871625 Update to the new MicroServer 1.0.6 release (#7460)
new a7fed3e Fixing compress expectation for new microserver (#7463)
new e9c4b96 Cleans up duplicated TSOutboundConnectionMatchType definition (#7090)
new 9b1c40e Change atoi to atol, causing obvious issues on what needs to be int64's (#7466)
new 1094f76 Adjust to actually try a server address more than once (#7288)
new 8283e7e Updating to Proxy Verifier v2.0.0 (#7454)
new e743261 Cleanup: Add SNIRoutingType (#7453)
new 6b34b16 Proxy Verifier: Making use of delay directives for caching tests. (#7468)
new 1ae48be Cleanup: Get rid of MIMEFieldWrapper from HPACK encoding (#6520)
new ee755b5 Add command line utility to help convert remap plugin usage to ATS9. (#7426)
new 86226cd Update AuTest version update directions for pipenv (#7469)
new 2926a69 API to retrieve NoStore set by plugins (#7439)
new 58b6e32 drop use of BIO_f_base64 and EVP_PKEY_new_mac_key (#7106)
new de8c125 use std::unordered_map to store sessions (#7405)
The 20 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
ci/jenkins/bin/autest.sh | 26 +-
doc/appendices/command-line/cvtremappi.en.rst | 69 +++
include/ts/apidefs.h.in | 3 -
include/ts/ts.h | 7 +
include/tscore/Filenames.h | 2 +-
iocore/cache/CacheRead.cc | 6 +-
iocore/dns/DNS.cc | 2 +-
iocore/net/Makefile.am | 1 +
iocore/net/P_SNIActionPerformer.h | 48 +-
iocore/net/P_SSLNetVConnection.h | 71 ++-
iocore/net/SSLNetVConnection.cc | 13 +-
iocore/net/SSLSNIConfig.cc | 2 +-
iocore/net/SSLSessionCache.cc | 175 +++----
iocore/net/SSLSessionCache.h | 67 ++-
.../test_RecordsConfig.h => iocore/net/SSLTypes.h | 7 +-
iocore/net/SSLUtils.cc | 133 ++---
iocore/net/TLSSNISupport.cc | 145 ++++++
.../net/TLSSNISupport.h | 54 +-
iocore/net/YamlSNIConfig.cc | 9 +-
iocore/net/YamlSNIConfig.h | 7 +-
plugins/experimental/access_control/Makefile.inc | 4 +-
.../experimental/access_control/access_control.cc | 6 +-
plugins/experimental/access_control/headers.cc | 4 +-
.../access_control/unit_tests/test_utils.cc | 48 +-
plugins/experimental/access_control/utils.cc | 172 ++----
plugins/experimental/access_control/utils.h | 2 -
plugins/experimental/system_stats/system_stats.c | 4 +-
proxy/http/HttpProxyAPIEnums.h | 15 -
proxy/http/HttpProxyServerMain.cc | 13 +
proxy/http/HttpProxyServerMain.h | 4 +
proxy/http/HttpTransact.cc | 100 ++--
proxy/http2/HPACK.cc | 256 +++++----
proxy/http2/HPACK.h | 20 +-
proxy/http2/unit_tests/test_HpackIndexingTable.cc | 17 +-
src/traffic_server/InkAPI.cc | 9 +
src/traffic_server/traffic_server.cc | 7 +
tests/Pipfile | 8 +-
tests/gold_tests/autest-site/conditions.test.ext | 4 +
tests/gold_tests/autest-site/init.cli.ext | 20 +-
tests/gold_tests/autest-site/setup.cli.ext | 21 +-
.../autest-site/verifier_client.test.ext | 13 +-
.../autest-site/verifier_server.test.ext | 17 +-
.../cache/replay/cache-control-max-age.replay.yaml | 28 +
...negative-caching-300-second-timeout.replay.yaml | 4 +
.../replay/negative-caching-customized.replay.yaml | 12 +
.../replay/negative-caching-default.replay.yaml | 16 +
.../replay/negative-caching-disabled.replay.yaml | 17 +
.../replay/negative-caching-no-timeout.replay.yaml | 4 +
.../replay/negative-caching-timeout.replay.yaml | 4 +
tests/gold_tests/h2/gold/post_chunked.gold | 2 +-
tests/gold_tests/pluginTest/compress/compress.gold | 2 +-
tests/prepare_proxy_verifier.sh | 8 +-
tests/proxy-verifier-version.txt | 1 +
tools/cvtremappi | 575 +++++++++++++++++++++
tools/insnew | 67 +++
55 files changed, 1651 insertions(+), 700 deletions(-)
create mode 100644 doc/appendices/command-line/cvtremappi.en.rst
copy lib/records/test_RecordsConfig.h => iocore/net/SSLTypes.h (91%)
create mode 100644 iocore/net/TLSSNISupport.cc
copy proxy/private/SSLProxySession.h => iocore/net/TLSSNISupport.h (50%)
create mode 100644 tests/proxy-verifier-version.txt
create mode 100755 tools/cvtremappi
create mode 100755 tools/insnew
[trafficserver] 11/20: Adjust to actually try a server address more
than once (#7288)
Posted by zw...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 9.1.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
commit 1094f76a91568e8a57ddf954a17f4eecc00361c7
Author: Susan Hinrichs <sh...@yahoo-inc.com>
AuthorDate: Mon Feb 1 13:04:24 2021 -0600
Adjust to actually try a server address more than once (#7288)
---
proxy/http/HttpTransact.cc | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/proxy/http/HttpTransact.cc b/proxy/http/HttpTransact.cc
index 9b9e9ce..b89d7f4 100644
--- a/proxy/http/HttpTransact.cc
+++ b/proxy/http/HttpTransact.cc
@@ -3746,7 +3746,7 @@ HttpTransact::handle_response_from_server(State *s)
return CallOSDNSLookup(s);
} else if ((s->dns_info.srv_lookup_success || s->host_db_info.is_rr_elt()) &&
(s->txn_conf->connect_attempts_rr_retries > 0) &&
- (s->current.attempts % s->txn_conf->connect_attempts_rr_retries == 0)) {
+ ((s->current.attempts + 1) % s->txn_conf->connect_attempts_rr_retries == 0)) {
delete_server_rr_entry(s, max_connect_retries);
return;
} else {
[trafficserver] 20/20: use std::unordered_map to store sessions
(#7405)
Posted by zw...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 9.1.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
commit de8c12567dc3efbc451a2f3eb58f25eba9ebf47c
Author: Fei Deng <du...@gmail.com>
AuthorDate: Wed Feb 3 01:50:19 2021 -0600
use std::unordered_map to store sessions (#7405)
make the hash function better for session id
use std::shared_mutex to speed things up
disable session cache for tlsv1.3 sessions since it is not needed
use std::map to order sessions by time
---
iocore/net/SSLSessionCache.cc | 175 +++++++++++++++++++-----------------------
iocore/net/SSLSessionCache.h | 67 ++++++++++++----
iocore/net/SSLUtils.cc | 12 +++
3 files changed, 141 insertions(+), 113 deletions(-)
diff --git a/iocore/net/SSLSessionCache.cc b/iocore/net/SSLSessionCache.cc
index 17fb174..60ec19e 100644
--- a/iocore/net/SSLSessionCache.cc
+++ b/iocore/net/SSLSessionCache.cc
@@ -129,51 +129,50 @@ SSLSessionBucket::insertSession(const SSLSessionID &id, SSL_SESSION *sess, SSL *
Debug("ssl.session_cache", "Inserting session '%s' to bucket %p.", buf, this);
}
- MUTEX_TRY_LOCK(lock, mutex, this_ethread());
- if (!lock.is_locked()) {
+ Ptr<IOBufferData> buf;
+ Ptr<IOBufferData> buf_exdata;
+ size_t len_exdata = sizeof(ssl_session_cache_exdata);
+ buf = new_IOBufferData(buffer_size_to_index(len, MAX_BUFFER_SIZE_INDEX), MEMALIGNED);
+ ink_release_assert(static_cast<size_t>(buf->block_size()) >= len);
+ unsigned char *loc = reinterpret_cast<unsigned char *>(buf->data());
+ i2d_SSL_SESSION(sess, &loc);
+ buf_exdata = new_IOBufferData(buffer_size_to_index(len, MAX_BUFFER_SIZE_INDEX), MEMALIGNED);
+ ink_release_assert(static_cast<size_t>(buf_exdata->block_size()) >= len_exdata);
+ ssl_session_cache_exdata *exdata = reinterpret_cast<ssl_session_cache_exdata *>(buf_exdata->data());
+ // This could be moved to a function in charge of populating exdata
+ exdata->curve = (ssl == nullptr) ? 0 : SSLGetCurveNID(ssl);
+ ink_hrtime now = Thread::get_hrtime_updated();
+
+ ats_scoped_obj<SSLSession> ssl_session(new SSLSession(id, buf, len, buf_exdata, now));
+
+ std::unique_lock lock(mutex, std::try_to_lock);
+ if (!lock.owns_lock()) {
if (ssl_rsb) {
SSL_INCREMENT_DYN_STAT(ssl_session_cache_lock_contention);
}
if (SSLConfigParams::session_cache_skip_on_lock_contention) {
return;
}
- lock.acquire(this_ethread());
+ lock.lock();
}
PRINT_BUCKET("insertSession before")
- if (queue.size >= static_cast<int>(SSLConfigParams::session_cache_max_bucket_size)) {
+ if (bucket_data.size() >= SSLConfigParams::session_cache_max_bucket_size) {
if (ssl_rsb) {
SSL_INCREMENT_DYN_STAT(ssl_session_cache_eviction);
}
- removeOldestSession();
+ removeOldestSession(lock);
}
// Don't insert if it is already there
- SSLSession *node = queue.tail;
- while (node) {
- if (node->session_id == id) {
- return;
- }
- node = node->link.prev;
+ if (bucket_data.find(id) != bucket_data.end()) {
+ return;
}
- Ptr<IOBufferData> buf;
- Ptr<IOBufferData> buf_exdata;
- size_t len_exdata = sizeof(ssl_session_cache_exdata);
- buf = new_IOBufferData(buffer_size_to_index(len, MAX_BUFFER_SIZE_INDEX), MEMALIGNED);
- ink_release_assert(static_cast<size_t>(buf->block_size()) >= len);
- unsigned char *loc = reinterpret_cast<unsigned char *>(buf->data());
- i2d_SSL_SESSION(sess, &loc);
- buf_exdata = new_IOBufferData(buffer_size_to_index(len, MAX_BUFFER_SIZE_INDEX), MEMALIGNED);
- ink_release_assert(static_cast<size_t>(buf_exdata->block_size()) >= len_exdata);
- ssl_session_cache_exdata *exdata = reinterpret_cast<ssl_session_cache_exdata *>(buf_exdata->data());
- // This could be moved to a function in charge of populating exdata
- exdata->curve = (ssl == nullptr) ? 0 : SSLGetCurveNID(ssl);
-
- ats_scoped_obj<SSLSession> ssl_session(new SSLSession(id, buf, len, buf_exdata));
-
/* do the actual insert */
- queue.enqueue(ssl_session.release());
+ auto node = ssl_session.release();
+ bucket_data[id] = node;
+ bucket_data_ts[now] = node;
PRINT_BUCKET("insertSession after")
}
@@ -182,33 +181,26 @@ int
SSLSessionBucket::getSessionBuffer(const SSLSessionID &id, char *buffer, int &len)
{
int true_len = 0;
- MUTEX_TRY_LOCK(lock, mutex, this_ethread());
- if (!lock.is_locked()) {
+ std::shared_lock lock(mutex, std::try_to_lock);
+ if (!lock.owns_lock()) {
if (ssl_rsb) {
SSL_INCREMENT_DYN_STAT(ssl_session_cache_lock_contention);
}
if (SSLConfigParams::session_cache_skip_on_lock_contention) {
return true_len;
}
-
- lock.acquire(this_ethread());
+ lock.lock();
}
- // We work backwards because that's the most likely place we'll find our session...
- SSLSession *node = queue.tail;
- while (node) {
- if (node->session_id == id) {
- true_len = node->len_asn1_data;
- if (buffer) {
- const unsigned char *loc = reinterpret_cast<const unsigned char *>(node->asn1_data->data());
- if (true_len < len) {
- len = true_len;
- }
- memcpy(buffer, loc, len);
- return true_len;
- }
+ auto node = bucket_data.find(id);
+ if (buffer && node != bucket_data.end()) {
+ true_len = node->second->len_asn1_data;
+ const unsigned char *loc = reinterpret_cast<const unsigned char *>(node->second->asn1_data->data());
+ if (true_len < len) {
+ len = true_len;
}
- node = node->link.prev;
+ memcpy(buffer, loc, len);
+ return true_len;
}
return 0;
}
@@ -224,38 +216,31 @@ SSLSessionBucket::getSession(const SSLSessionID &id, SSL_SESSION **sess, ssl_ses
Debug("ssl.session_cache", "Looking for session with id '%s' in bucket %p", buf, this);
- MUTEX_TRY_LOCK(lock, mutex, this_ethread());
- if (!lock.is_locked()) {
+ std::shared_lock lock(mutex, std::try_to_lock);
+ if (!lock.owns_lock()) {
if (ssl_rsb) {
SSL_INCREMENT_DYN_STAT(ssl_session_cache_lock_contention);
}
if (SSLConfigParams::session_cache_skip_on_lock_contention) {
return false;
}
-
- lock.acquire(this_ethread());
+ lock.lock();
}
PRINT_BUCKET("getSession")
- // We work backwards because that's the most likely place we'll find our session...
- SSLSession *node = queue.tail;
- while (node) {
- if (node->session_id == id) {
- const unsigned char *loc = reinterpret_cast<const unsigned char *>(node->asn1_data->data());
- *sess = d2i_SSL_SESSION(nullptr, &loc, node->len_asn1_data);
- if (data != nullptr) {
- ssl_session_cache_exdata *exdata = reinterpret_cast<ssl_session_cache_exdata *>(node->extra_data->data());
- *data = exdata;
- }
-
- return true;
- }
- node = node->link.prev;
+ auto node = bucket_data.find(id);
+ if (node == bucket_data.end()) {
+ Debug("ssl.session_cache", "Session with id '%s' not found in bucket %p.", buf, this);
+ return false;
}
-
- Debug("ssl.session_cache", "Session with id '%s' not found in bucket %p.", buf, this);
- return false;
+ const unsigned char *loc = reinterpret_cast<const unsigned char *>(node->second->asn1_data->data());
+ *sess = d2i_SSL_SESSION(nullptr, &loc, node->second->len_asn1_data);
+ if (data != nullptr) {
+ ssl_session_cache_exdata *exdata = reinterpret_cast<ssl_session_cache_exdata *>(node->second->extra_data->data());
+ *data = exdata;
+ }
+ return true;
}
void inline SSLSessionBucket::print(const char *ref_str) const
@@ -266,53 +251,51 @@ void inline SSLSessionBucket::print(const char *ref_str) const
}
fprintf(stderr, "-------------- BUCKET %p (%s) ----------------\n", this, ref_str);
- fprintf(stderr, "Current Size: %d, Max Size: %zd\n", queue.size, SSLConfigParams::session_cache_max_bucket_size);
- fprintf(stderr, "Queue: \n");
+ fprintf(stderr, "Current Size: %ld, Max Size: %zd\n", bucket_data.size(), SSLConfigParams::session_cache_max_bucket_size);
+ fprintf(stderr, "Bucket: \n");
- SSLSession *node = queue.head;
- while (node) {
- char s_buf[2 * node->session_id.len + 1];
- node->session_id.toString(s_buf, sizeof(s_buf));
+ for (auto &x : bucket_data) {
+ char s_buf[2 * x.second->session_id.len + 1];
+ x.second->session_id.toString(s_buf, sizeof(s_buf));
fprintf(stderr, " %s\n", s_buf);
- node = node->link.next;
}
}
-void inline SSLSessionBucket::removeOldestSession()
+void inline SSLSessionBucket::removeOldestSession(const std::unique_lock<std::shared_mutex> &lock)
{
- // Caller must hold the bucket lock.
- ink_assert(this_ethread() == mutex->thread_holding);
+ // Caller must hold the bucket shared_mutex with unique_lock.
+ ink_assert(lock.owns_lock());
PRINT_BUCKET("removeOldestSession before")
- while (queue.head && queue.size >= static_cast<int>(SSLConfigParams::session_cache_max_bucket_size)) {
- SSLSession *old_head = queue.pop();
- if (is_debug_tag_set("ssl.session_cache")) {
- char buf[old_head->session_id.len * 2 + 1];
- old_head->session_id.toString(buf, sizeof(buf));
- Debug("ssl.session_cache", "Removing session '%s' from bucket %p because the bucket has size %d and max %zd", buf, this,
- (queue.size + 1), SSLConfigParams::session_cache_max_bucket_size);
- }
- delete old_head;
- }
+
+ auto node = bucket_data_ts.begin();
+ bucket_data.erase(node->second->session_id);
+ bucket_data_ts.erase(node);
+
PRINT_BUCKET("removeOldestSession after")
}
void
SSLSessionBucket::removeSession(const SSLSessionID &id)
{
- SCOPED_MUTEX_LOCK(lock, mutex, this_ethread()); // We can't bail on contention here because this session MUST be removed.
- SSLSession *node = queue.head;
- while (node) {
- if (node->session_id == id) {
- queue.remove(node);
- delete node;
- return;
- }
- node = node->link.next;
+ // We can't bail on contention here because this session MUST be removed.
+ std::unique_lock lock(mutex);
+
+ auto node = bucket_data.find(id);
+
+ PRINT_BUCKET("removeSession before")
+
+ if (node != bucket_data.end()) {
+ bucket_data_ts.erase(node->second->time_stamp);
+ bucket_data.erase(node);
}
+
+ PRINT_BUCKET("removeSession after")
+
+ return;
}
/* Session Bucket */
-SSLSessionBucket::SSLSessionBucket() : mutex(new_ProxyMutex()) {}
+SSLSessionBucket::SSLSessionBucket() {}
SSLSessionBucket::~SSLSessionBucket() {}
diff --git a/iocore/net/SSLSessionCache.h b/iocore/net/SSLSessionCache.h
index 44ba12d..05a5930 100644
--- a/iocore/net/SSLSessionCache.h
+++ b/iocore/net/SSLSessionCache.h
@@ -29,6 +29,8 @@
#include "P_SSLUtils.h"
#include "ts/apidefs.h"
#include <openssl/ssl.h>
+#include <mutex>
+#include <shared_mutex>
#define SSL_MAX_SESSION_SIZE 256
@@ -36,12 +38,21 @@ struct ssl_session_cache_exdata {
ssl_curve_id curve = 0;
};
+inline void
+hash_combine(uint64_t &seed, uint64_t hash)
+{
+ // using boost's version of hash combine, substituting magic number with a 64bit version
+ // https://www.boost.org/doc/libs/1_43_0/doc/html/hash/reference.html#boost.hash_combine
+ seed ^= hash + 0x9E3779B97F4A7C15 + (seed << 6) + (seed >> 2);
+}
+
struct SSLSessionID : public TSSslSessionID {
SSLSessionID(const unsigned char *s, size_t l)
{
len = l;
ink_release_assert(l <= sizeof(bytes));
memcpy(bytes, s, l);
+ hash();
}
SSLSessionID(const SSLSessionID &other)
@@ -50,6 +61,7 @@ struct SSLSessionID : public TSSslSessionID {
memcpy(bytes, other.bytes, other.len);
len = other.len;
+ hash();
}
bool
@@ -101,15 +113,33 @@ struct SSLSessionID : public TSSslSessionID {
uint64_t
hash() const
{
- // because the session ids should be uniformly random let's just use the last 64 bits as the hash.
- // The first bytes could be interpreted as a name, and so not random.
- if (len >= sizeof(uint64_t)) {
- return *reinterpret_cast<uint64_t *>(const_cast<char *>(bytes + len - sizeof(uint64_t)));
- } else if (len) {
- return static_cast<uint64_t>(bytes[0]);
- } else {
- return 0;
+ if (hash_value == 0) {
+ // because the session ids should be uniformly random, we can treat the bits as a hash value
+ // however we need to combine them if the length is longer than 64bits
+ if (len >= sizeof(uint64_t)) {
+ uint64_t seed = 0;
+ for (uint64_t i = 0; i < len; i += sizeof(uint64_t)) {
+ hash_combine(seed, static_cast<uint64_t>(bytes[i]));
+ }
+ hash_value = seed;
+ } else if (len) {
+ hash_value = static_cast<uint64_t>(bytes[0]);
+ } else {
+ hash_value = 0;
+ }
}
+ return hash_value;
+ }
+
+private:
+ mutable uint64_t hash_value = 0;
+};
+
+struct SSLSessionIDHash {
+ uint64_t
+ operator()(const SSLSessionID &id) const
+ {
+ return id.hash();
}
};
@@ -120,9 +150,11 @@ public:
Ptr<IOBufferData> asn1_data; /* this is the ASN1 representation of the SSL_CTX */
size_t len_asn1_data;
Ptr<IOBufferData> extra_data;
+ ink_hrtime time_stamp;
- SSLSession(const SSLSessionID &id, const Ptr<IOBufferData> &ssl_asn1_data, size_t len_asn1, Ptr<IOBufferData> &exdata)
- : session_id(id), asn1_data(ssl_asn1_data), len_asn1_data(len_asn1), extra_data(exdata)
+ SSLSession(const SSLSessionID &id, const Ptr<IOBufferData> &ssl_asn1_data, size_t len_asn1, Ptr<IOBufferData> &exdata,
+ ink_hrtime ts)
+ : session_id(id), asn1_data(ssl_asn1_data), len_asn1_data(len_asn1), extra_data(exdata), time_stamp(ts)
{
}
@@ -134,18 +166,19 @@ class SSLSessionBucket
public:
SSLSessionBucket();
~SSLSessionBucket();
- void insertSession(const SSLSessionID &, SSL_SESSION *ctx, SSL *ssl);
- bool getSession(const SSLSessionID &, SSL_SESSION **ctx, ssl_session_cache_exdata **data);
- int getSessionBuffer(const SSLSessionID &, char *buffer, int &len);
- void removeSession(const SSLSessionID &);
+ void insertSession(const SSLSessionID &sid, SSL_SESSION *sess, SSL *ssl);
+ bool getSession(const SSLSessionID &sid, SSL_SESSION **sess, ssl_session_cache_exdata **data);
+ int getSessionBuffer(const SSLSessionID &sid, char *buffer, int &len);
+ void removeSession(const SSLSessionID &sid);
private:
/* these method must be used while hold the lock */
void print(const char *) const;
- void removeOldestSession();
+ void removeOldestSession(const std::unique_lock<std::shared_mutex> &lock);
- Ptr<ProxyMutex> mutex;
- CountQueue<SSLSession> queue;
+ mutable std::shared_mutex mutex;
+ std::unordered_map<SSLSessionID, SSLSession *, SSLSessionIDHash> bucket_data;
+ std::map<ink_hrtime, SSLSession *> bucket_data_ts;
};
class SSLSessionCache
diff --git a/iocore/net/SSLUtils.cc b/iocore/net/SSLUtils.cc
index d34d6b8..1c2398a 100644
--- a/iocore/net/SSLUtils.cc
+++ b/iocore/net/SSLUtils.cc
@@ -191,6 +191,12 @@ ssl_get_cached_session(SSL *ssl, const unsigned char *id, int len, int *copy)
static int
ssl_new_cached_session(SSL *ssl, SSL_SESSION *sess)
{
+#ifdef TLS1_3_VERSION
+ if (SSL_SESSION_get_protocol_version(sess) == TLS1_3_VERSION) {
+ return 0;
+ }
+#endif
+
unsigned int len = 0;
const unsigned char *id = SSL_SESSION_get_id(sess, &len);
@@ -219,6 +225,12 @@ ssl_new_cached_session(SSL *ssl, SSL_SESSION *sess)
static void
ssl_rm_cached_session(SSL_CTX *ctx, SSL_SESSION *sess)
{
+#ifdef TLS1_3_VERSION
+ if (SSL_SESSION_get_protocol_version(sess) == TLS1_3_VERSION) {
+ return;
+ }
+#endif
+
unsigned int len = 0;
const unsigned char *id = SSL_SESSION_get_id(sess, &len);
SSLSessionID sid(id, len);
[trafficserver] 13/20: Cleanup: Add SNIRoutingType (#7453)
Posted by zw...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 9.1.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
commit e743261e0fa30ccf768f6346e58640ecc4f210ca
Author: Masaori Koshiba <ma...@apache.org>
AuthorDate: Tue Feb 2 07:45:29 2021 +0900
Cleanup: Add SNIRoutingType (#7453)
---
iocore/net/P_SNIActionPerformer.h | 19 +++++++-------
iocore/net/P_SSLNetVConnection.h | 54 ++++++++++++++++++++++++---------------
iocore/net/SSLSNIConfig.cc | 2 +-
iocore/net/SSLTypes.h | 31 ++++++++++++++++++++++
iocore/net/YamlSNIConfig.cc | 9 +++----
iocore/net/YamlSNIConfig.h | 7 ++---
6 files changed, 83 insertions(+), 39 deletions(-)
diff --git a/iocore/net/P_SNIActionPerformer.h b/iocore/net/P_SNIActionPerformer.h
index b255969..07d6c5c 100644
--- a/iocore/net/P_SNIActionPerformer.h
+++ b/iocore/net/P_SNIActionPerformer.h
@@ -31,10 +31,13 @@
#pragma once
#include "I_EventSystem.h"
-#include <vector>
#include "P_SSLNextProtocolAccept.h"
+#include "SSLTypes.h"
+
#include "tscore/ink_inet.h"
+#include <vector>
+
class ActionItem
{
public:
@@ -92,8 +95,7 @@ private:
class TunnelDestination : public ActionItem
{
public:
- TunnelDestination(const std::string_view &dest, bool decrypt, bool tls_upstream)
- : destination(dest), tunnel_decrypt(decrypt), tls_upstream(tls_upstream)
+ TunnelDestination(const std::string_view &dest, SNIRoutingType type) : destination(dest), type(type)
{
need_fix = (destination.find_first_of('$') != std::string::npos);
}
@@ -108,15 +110,16 @@ public:
// If needed, we will try to amend the tunnel destination.
if (ctx._fqdn_wildcard_captured_groups && need_fix) {
const auto &fixed_dst = replace_match_groups(destination, *ctx._fqdn_wildcard_captured_groups);
- ssl_netvc->set_tunnel_destination(fixed_dst, tunnel_decrypt, tls_upstream);
+ ssl_netvc->set_tunnel_destination(fixed_dst, type);
Debug("TunnelDestination", "Destination now is [%s], configured [%s]", fixed_dst.c_str(), destination.c_str());
} else {
- ssl_netvc->set_tunnel_destination(destination, tunnel_decrypt, tls_upstream);
+ ssl_netvc->set_tunnel_destination(destination, type);
}
- if (ssl_netvc->has_tunnel_destination() && !ssl_netvc->decrypt_tunnel()) {
+ if (type == SNIRoutingType::BLIND) {
ssl_netvc->attributes = HttpProxyPort::TRANSPORT_BLIND_TUNNEL;
}
}
+
return SSL_TLSEXT_ERR_OK;
}
@@ -187,10 +190,8 @@ private:
}
std::string destination;
-
- bool tunnel_decrypt;
+ SNIRoutingType type = SNIRoutingType::NONE;
bool need_fix;
- bool tls_upstream;
};
class VerifyClient : public ActionItem
diff --git a/iocore/net/P_SSLNetVConnection.h b/iocore/net/P_SSLNetVConnection.h
index 3a85970..419e092 100644
--- a/iocore/net/P_SSLNetVConnection.h
+++ b/iocore/net/P_SSLNetVConnection.h
@@ -330,25 +330,15 @@ public:
return tunnel_port;
}
- /* Returns true if this vc was configured for forward_route or partial_blind_route
- */
- bool
- decrypt_tunnel()
- {
- return has_tunnel_destination() && tunnel_decrypt;
- }
-
- /* Returns true if this vc was configured partial_blind_route
- */
- bool
- upstream_tls()
- {
- return has_tunnel_destination() && tls_upstream;
- }
+ bool decrypt_tunnel() const;
+ bool upstream_tls() const;
+ SNIRoutingType tunnel_type() const;
void
- set_tunnel_destination(const std::string_view &destination, bool decrypt, bool upstream_tls)
+ set_tunnel_destination(const std::string_view &destination, SNIRoutingType type)
{
+ _tunnel_type = type;
+
auto pos = destination.find(":");
if (nullptr != tunnel_host) {
ats_free(tunnel_host);
@@ -360,8 +350,6 @@ public:
tunnel_port = 0;
tunnel_host = ats_strndup(destination.data(), destination.length());
}
- tunnel_decrypt = decrypt;
- tls_upstream = upstream_tls;
}
int populate_protocol(std::string_view *results, int n) const override;
@@ -529,8 +517,7 @@ private:
int64_t redoWriteSize = 0;
char *tunnel_host = nullptr;
in_port_t tunnel_port = 0;
- bool tunnel_decrypt = false;
- bool tls_upstream = false;
+ SNIRoutingType _tunnel_type = SNIRoutingType::NONE;
X509_STORE_CTX *verify_cert = nullptr;
// Null-terminated string, or nullptr if there is no SNI server name.
@@ -548,3 +535,30 @@ private:
typedef int (SSLNetVConnection::*SSLNetVConnHandler)(int, void *);
extern ClassAllocator<SSLNetVConnection> sslNetVCAllocator;
+
+//
+// Inline Functions
+//
+inline SNIRoutingType
+SSLNetVConnection::tunnel_type() const
+{
+ return _tunnel_type;
+}
+
+/**
+ Returns true if this vc was configured for forward_route or partial_blind_route
+ */
+inline bool
+SSLNetVConnection::decrypt_tunnel() const
+{
+ return _tunnel_type == SNIRoutingType::FORWARD || _tunnel_type == SNIRoutingType::PARTIAL_BLIND;
+}
+
+/**
+ Returns true if this vc was configured partial_blind_route
+ */
+inline bool
+SSLNetVConnection::upstream_tls() const
+{
+ return _tunnel_type == SNIRoutingType::PARTIAL_BLIND;
+}
diff --git a/iocore/net/SSLSNIConfig.cc b/iocore/net/SSLSNIConfig.cc
index 9f38847..2f4379d 100644
--- a/iocore/net/SSLSNIConfig.cc
+++ b/iocore/net/SSLSNIConfig.cc
@@ -77,7 +77,7 @@ SNIConfigParams::loadSNIConfig()
ai->actions.push_back(std::make_unique<TLSValidProtocols>(item.protocol_mask));
}
if (item.tunnel_destination.length() > 0) {
- ai->actions.push_back(std::make_unique<TunnelDestination>(item.tunnel_destination, item.tunnel_decrypt, item.tls_upstream));
+ ai->actions.push_back(std::make_unique<TunnelDestination>(item.tunnel_destination, item.tunnel_type));
}
ai->actions.push_back(std::make_unique<SNI_IpAllow>(item.ip_allow, item.fqdn));
diff --git a/iocore/net/SSLTypes.h b/iocore/net/SSLTypes.h
new file mode 100644
index 0000000..977fa5a
--- /dev/null
+++ b/iocore/net/SSLTypes.h
@@ -0,0 +1,31 @@
+/** @file
+
+ A brief file description
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+
+#pragma once
+
+enum class SNIRoutingType {
+ NONE = 0,
+ BLIND,
+ FORWARD,
+ PARTIAL_BLIND,
+};
diff --git a/iocore/net/YamlSNIConfig.cc b/iocore/net/YamlSNIConfig.cc
index 0cd27d5..aa1d86b 100644
--- a/iocore/net/YamlSNIConfig.cc
+++ b/iocore/net/YamlSNIConfig.cc
@@ -207,16 +207,13 @@ template <> struct convert<YamlSNIConfig::Item> {
if (node[TS_tunnel_route]) {
item.tunnel_destination = node[TS_tunnel_route].as<std::string>();
- item.tunnel_decrypt = false;
- item.tls_upstream = false;
+ item.tunnel_type = SNIRoutingType::BLIND;
} else if (node[TS_forward_route]) {
item.tunnel_destination = node[TS_forward_route].as<std::string>();
- item.tunnel_decrypt = true;
- item.tls_upstream = false;
+ item.tunnel_type = SNIRoutingType::FORWARD;
} else if (node[TS_partial_blind_route]) {
item.tunnel_destination = node[TS_partial_blind_route].as<std::string>();
- item.tunnel_decrypt = true;
- item.tls_upstream = true;
+ item.tunnel_type = SNIRoutingType::PARTIAL_BLIND;
}
if (node[TS_verify_server_policy]) {
diff --git a/iocore/net/YamlSNIConfig.h b/iocore/net/YamlSNIConfig.h
index e51d173..1116953 100644
--- a/iocore/net/YamlSNIConfig.h
+++ b/iocore/net/YamlSNIConfig.h
@@ -26,6 +26,8 @@
#include <optional>
#include <memory>
+#include "SSLTypes.h"
+
#include "tscore/Errata.h"
#define TSDECL(id) constexpr char TS_##id[] = #id
@@ -76,10 +78,9 @@ struct YamlSNIConfig {
uint8_t verify_client_level = 255;
std::string verify_client_ca_file;
std::string verify_client_ca_dir;
- uint8_t host_sni_policy = 255;
+ uint8_t host_sni_policy = 255;
+ SNIRoutingType tunnel_type = SNIRoutingType::NONE;
std::string tunnel_destination;
- bool tunnel_decrypt = false;
- bool tls_upstream = false;
Policy verify_server_policy = Policy::UNSET;
Property verify_server_properties = Property::UNSET;
std::string client_cert;
[trafficserver] 12/20: Updating to Proxy Verifier v2.0.0 (#7454)
Posted by zw...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 9.1.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
commit 8283e7e8084f9cf0264e2ec9fe0c5baaa3e25009
Author: Brian Neradt <br...@gmail.com>
AuthorDate: Mon Feb 1 15:59:00 2021 -0600
Updating to Proxy Verifier v2.0.0 (#7454)
We will want to use Proxy Verifier v2.0.0 for it's delayed transaction
and HTTP/2 to server capabilities. This updates the AuTest extensions to
properly handle the new version.
---
ci/jenkins/bin/autest.sh | 26 ++++++++++++++++++++--
tests/gold_tests/autest-site/conditions.test.ext | 4 ++++
tests/gold_tests/autest-site/setup.cli.ext | 21 ++++++++++++++++-
.../autest-site/verifier_client.test.ext | 13 ++++++++---
.../autest-site/verifier_server.test.ext | 17 +++++++++-----
tests/prepare_proxy_verifier.sh | 8 +++----
tests/proxy-verifier-version.txt | 1 +
7 files changed, 74 insertions(+), 16 deletions(-)
diff --git a/ci/jenkins/bin/autest.sh b/ci/jenkins/bin/autest.sh
index dfe4b88..7ec18ee 100755
--- a/ci/jenkins/bin/autest.sh
+++ b/ci/jenkins/bin/autest.sh
@@ -46,6 +46,9 @@ URL="https://ci.trafficserver.apache.org/autest"
JOB_ID=${ghprbPullId:-${ATS_BRANCH:-master}}
AUSB="ausb-${JOB_ID}.${BUILD_NUMBER}"
SANDBOX="/var/tmp/${AUSB}"
+PROXY_VERIFIER_VERSIONS="/home/jenkins/proxy-verifier"
+PROXY_VERIFIER_VERSION_FILE="tests/proxy-verifier-version.txt"
+PROXY_VERIFIER_PREPARE="tests/prepare_proxy_verifier.sh"
# Optional settings
CCACHE=""
@@ -57,6 +60,7 @@ QUIC=""
CURL=""
AUTEST_DEBUG=""
AUTEST_VERBOSE=""
+PROXY_VERIFIER_ARGUMENT=""
[ "1" == "$enable_ccache" ] && CCACHE="--enable-ccache"
[ "1" == "$enable_werror" ] && WERROR="--enable-werror"
@@ -86,7 +90,8 @@ set -x
# Configure
autoreconf -if
-./configure --prefix="${INSTALL}" \
+./configure \
+ --prefix="${INSTALL}" \
--with-user=jenkins \
--enable-experimental-plugins \
--enable-example-plugins \
@@ -111,7 +116,24 @@ AUTEST="/usr/bin/autest"
[ ! -x ${AUTEST} ] && AUTEST="/usr/local/bin/autest"
set -x
-${AUTEST} -D ./tests/gold_tests --sandbox "$SANDBOX" --ats-bin "${INSTALL}/bin" $AUTEST_DEBUG $AUTEST_VERBOSE
+pv_version=""
+if [ -f "${PROXY_VERIFIER_VERSION_FILE}" ]; then
+ pv_version=`cat "${PROXY_VERIFIER_VERSION_FILE}"`
+elif [ -f "${PROXY_VERIFIER_PREPARE}" ]; then
+ pv_version=`awk -F'"' '/^pv_version/ {print $2}' "${PROXY_VERIFIER_PREPARE}"`
+fi
+if [ "x${pv_version}" != "x" ]; then
+ PROXY_VERIFIER_BIN="${PROXY_VERIFIER_VERSIONS}/${pv_version}/bin"
+ PROXY_VERIFIER_ARGUMENT="--proxy-verifier-bin ${PROXY_VERIFIER_BIN}"
+fi
+
+${AUTEST} \
+ -D ./tests/gold_tests \
+ --sandbox "$SANDBOX" \
+ --ats-bin "${INSTALL}/bin" \
+ $PROXY_VERIFIER_ARGUMENT \
+ $AUTEST_DEBUG \
+ $AUTEST_VERBOSE
status=$?
set +x
diff --git a/tests/gold_tests/autest-site/conditions.test.ext b/tests/gold_tests/autest-site/conditions.test.ext
index c9d3a0d..52af993 100644
--- a/tests/gold_tests/autest-site/conditions.test.ext
+++ b/tests/gold_tests/autest-site/conditions.test.ext
@@ -40,6 +40,10 @@ def HasCurlVersion(self, version):
return self.EnsureVersion(["curl", "--version"], min_version=version)
+def HasProxyVerifierVersion(self, version):
+ return self.EnsureVersion(["verifier-client", "--version"], min_version=version)
+
+
def HasCurlFeature(self, feature):
def default(output):
diff --git a/tests/gold_tests/autest-site/setup.cli.ext b/tests/gold_tests/autest-site/setup.cli.ext
index 14b14fd..e752fa0 100644
--- a/tests/gold_tests/autest-site/setup.cli.ext
+++ b/tests/gold_tests/autest-site/setup.cli.ext
@@ -23,6 +23,8 @@ import pprint
import subprocess
import shutil
+PROXY_VERIFIER_VERSION_FILENAME = 'proxy-verifier-version.txt'
+
test_root = dirname(dirname(AutestSitePath))
repo_root = dirname(test_root)
@@ -38,6 +40,9 @@ else:
host.WriteVerbose(['ats'], "Test build root: {}:".format(ENV['BUILD_ROOT']))
+proxy_verifer_version_file = os.path.join(test_root, PROXY_VERIFIER_VERSION_FILENAME)
+proxy_verifer_version = open(proxy_verifer_version_file, "rt").read().strip()
+
if Arguments.proxy_verifier_bin is not None:
ENV['VERIFIER_BIN'] = Arguments.proxy_verifier_bin
host.WriteVerbose(
@@ -47,7 +52,7 @@ if Arguments.proxy_verifier_bin is not None:
else:
# No Verifier bin path was specified. First see if a Proxy Verifier was
# unpacked as a part of preparing for this test.
- unpack_bin = os.path.join(test_root, 'proxy-verifier', 'unpack', 'bin')
+ unpack_bin = os.path.join(test_root, 'proxy-verifier', 'unpack', proxy_verifer_version, 'bin')
if os.path.exists(os.path.join(unpack_bin, 'verifier-client')):
ENV['VERIFIER_BIN'] = unpack_bin
host.WriteVerbose(
@@ -68,6 +73,20 @@ else:
host.WriteError("Could not find Proxy Verifier binaries. "
"Try running: ", prepare_proxy_verifier_path)
+required_pv_version = Version(proxy_verifer_version[1:])
+verifier_client = os.path.join(ENV['VERIFIER_BIN'], 'verifier-client')
+pv_version_out = subprocess.check_output([verifier_client, "--version"])
+pv_version = Version(pv_version_out.decode("utf-8").split()[1])
+if pv_version < required_pv_version:
+ host.WriteError(
+ f"Proxy Verifier at {verifier_client} is too old. "
+ f"Version required: {required_pv_version}, version found: {pv_version}")
+else:
+ host.WriteVerbose(
+ ['ats'],
+ f"Proxy Verifier at {verifier_client} has version: {pv_version}")
+
+
if ENV['ATS_BIN'] is not None:
# Add variables for Tests
traffic_layout = os.path.join(ENV['ATS_BIN'], "traffic_layout")
diff --git a/tests/gold_tests/autest-site/verifier_client.test.ext b/tests/gold_tests/autest-site/verifier_client.test.ext
index 41ef41a..9ff5c03 100755
--- a/tests/gold_tests/autest-site/verifier_client.test.ext
+++ b/tests/gold_tests/autest-site/verifier_client.test.ext
@@ -24,7 +24,7 @@ from verifier_common import create_address_argument
def _configure_client(obj, process, name, replay_path, http_ports=None,
https_ports=None, keys=None, ssl_cert='', ca_cert='',
- other_args='--verbose diag'):
+ verbose=True, other_args=''):
"""
Configure the process for running the verifier-client.
@@ -61,12 +61,14 @@ def _configure_client(obj, process, name, replay_path, http_ports=None,
if not http_ports:
http_ports = [8080]
if http_ports:
+ command += "--connect-http "
command += create_address_argument(http_ports)
command += " "
if not https_ports:
https_ports = [4443]
if https_ports:
+ command += "--connect-https "
command += create_address_argument(https_ports)
command += " "
@@ -94,6 +96,9 @@ def _configure_client(obj, process, name, replay_path, http_ports=None,
process.Setup.Copy(ca_cert, run_ca_cert, CopyLogic.SoftFiles)
command += ' --ca-certs "{}" '.format(run_ca_cert)
+ if verbose:
+ command += ' --verbose diag '
+
if other_args:
command += " {}".format(other_args)
@@ -110,7 +115,7 @@ def _configure_client(obj, process, name, replay_path, http_ports=None,
def AddVerifierClientProcess(run, name, replay_path, http_ports=None,
https_ports=None, keys=None, ssl_cert='', ca_cert='',
- other_args='--verbose diag'):
+ verbose=True, other_args=''):
"""
Set the Default process of the test run to a verifier-client Process.
@@ -134,6 +139,8 @@ def AddVerifierClientProcess(run, name, replay_path, http_ports=None,
ca_cert: (path) The location of the CA for HTTPS encryption. If this
is not provided a stock CA will be used.
+ verbose: (bool) Whether to enable debug level logging.
+
other_args: (str) Any other arbitrary options to pass to verifier-client.
Returns:
@@ -143,7 +150,7 @@ def AddVerifierClientProcess(run, name, replay_path, http_ports=None,
p = run.Processes.Default
_configure_client(run, p, name, replay_path, http_ports, https_ports,
- keys, ssl_cert, ca_cert, other_args)
+ keys, ssl_cert, ca_cert, verbose, other_args)
return p
diff --git a/tests/gold_tests/autest-site/verifier_server.test.ext b/tests/gold_tests/autest-site/verifier_server.test.ext
index 3fd7f44..eab23ab 100755
--- a/tests/gold_tests/autest-site/verifier_server.test.ext
+++ b/tests/gold_tests/autest-site/verifier_server.test.ext
@@ -23,7 +23,7 @@ from verifier_common import create_address_argument
def _configure_server(obj, process, name, replay_path, http_ports=None, https_ports=None,
- ssl_cert='', ca_cert='', other_args='--verbose diag'):
+ ssl_cert='', ca_cert='', verbose=True, other_args=''):
"""
Configure the provided process to run a verifier-server command.
@@ -55,7 +55,7 @@ def _configure_server(obj, process, name, replay_path, http_ports=None, https_po
process.Variables['http_port'] = http_ports[0]
if len(http_ports) > 0:
- command += "--listen "
+ command += "--listen-http "
command += create_address_argument(http_ports)
command += " "
@@ -100,6 +100,9 @@ def _configure_server(obj, process, name, replay_path, http_ports=None, https_po
process.Setup.Copy(replay_path, run_replay_path, CopyLogic.SoftFiles)
command += "{} ".format(run_replay_path)
+ if verbose:
+ command += ' --verbose diag '
+
if other_args:
command += "{} ".format(other_args)
process.Command = command
@@ -114,7 +117,7 @@ def _configure_server(obj, process, name, replay_path, http_ports=None, https_po
def MakeVerifierServerProcess(test, name, replay_path, http_ports=None,
https_ports=None, ssl_cert='', ca_cert='',
- other_args='--verbose diag'):
+ verbose=True, other_args=''):
"""
Create a verifier-server process for the Test.
@@ -140,6 +143,8 @@ def MakeVerifierServerProcess(test, name, replay_path, http_ports=None,
ca_cert: (path) The location of the CA for HTTPS encryption. If this
is not provided a stock CA will be used.
+ verbose: (bool) Whether to enable debug level logging.
+
other_args: (str) Any other arbitrary options to pass to verifier-server.
Raises:
@@ -148,13 +153,13 @@ def MakeVerifierServerProcess(test, name, replay_path, http_ports=None,
"""
server = test.Processes.Process(name)
_configure_server(test, server, name, replay_path, http_ports, https_ports,
- ssl_cert, ca_cert, other_args)
+ ssl_cert, ca_cert, verbose, other_args)
return server
def AddVerifierServerProcess(run, name, replay_path, http_ports=None,
https_ports=None, ssl_cert='', ca_cert='',
- other_args='--verbose diag'):
+ verbose=True, other_args=''):
"""
Create a verifier-server process and configure it for the given TestRun.
@@ -169,7 +174,7 @@ def AddVerifierServerProcess(run, name, replay_path, http_ports=None,
server = run.Processes.Process(name)
_configure_server(run, server, name, replay_path, http_ports, https_ports,
- ssl_cert, ca_cert, other_args)
+ ssl_cert, ca_cert, verbose, other_args)
client = run.Processes.Default
client.StartBefore(server)
diff --git a/tests/prepare_proxy_verifier.sh b/tests/prepare_proxy_verifier.sh
index 6690b84..06a2faa 100755
--- a/tests/prepare_proxy_verifier.sh
+++ b/tests/prepare_proxy_verifier.sh
@@ -18,16 +18,16 @@
# limitations under the License.
ROOT=${ROOT:-$(cd $(dirname $0) && git rev-parse --show-toplevel)}
+pv_name="proxy-verifier"
+pv_version=`cat ${ROOT}/tests/proxy-verifier-version.txt`
pv_top_dir="${ROOT}/tests/proxy-verifier"
-pv_unpack_dir="${pv_top_dir}/unpack"
+pv_unpack_dir="${pv_top_dir}/unpack/${pv_version}"
bin_dir="${pv_unpack_dir}/bin"
-pv_name="proxy-verifier"
-pv_version="v1.10.0"
pv_dir="${pv_name}-${pv_version}"
pv_tar_filename="${pv_dir}.tar.gz"
pv_tar="${pv_top_dir}/${pv_tar_filename}"
pv_tar_url="https://ci.trafficserver.apache.org/bintray/${pv_tar_filename}"
-expected_sha1="f9ad11942f9098733e0286112a3c17975ebed363"
+expected_sha1="f644ac9838ba74e674b9efe7e7b3d244037faf21"
pv_client="${bin_dir}/verifier-client"
pv_server="${bin_dir}/verifier-server"
TAR=${TAR:-tar}
diff --git a/tests/proxy-verifier-version.txt b/tests/proxy-verifier-version.txt
new file mode 100644
index 0000000..46b105a
--- /dev/null
+++ b/tests/proxy-verifier-version.txt
@@ -0,0 +1 @@
+v2.0.0
[trafficserver] 19/20: drop use of BIO_f_base64 and
EVP_PKEY_new_mac_key (#7106)
Posted by zw...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 9.1.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
commit 58b6e32895f6a0342201e509149d0a3c7af1c10c
Author: Randall Meyer <rr...@apache.org>
AuthorDate: Tue Feb 2 18:42:25 2021 -0800
drop use of BIO_f_base64 and EVP_PKEY_new_mac_key (#7106)
BIO_f_base64 is not available in BoringSSL plus we have our own Base64
methods. With this change, the estimation values change, but shouldn't
effect actual runtime.
And BoringSSL suggests using the HMAC* functions in place of EVP_PKEY_new_mac_key
---
plugins/experimental/access_control/Makefile.inc | 4 +-
.../experimental/access_control/access_control.cc | 6 +-
plugins/experimental/access_control/headers.cc | 4 +-
.../access_control/unit_tests/test_utils.cc | 48 +++---
plugins/experimental/access_control/utils.cc | 172 ++++++---------------
plugins/experimental/access_control/utils.h | 2 -
6 files changed, 83 insertions(+), 153 deletions(-)
diff --git a/plugins/experimental/access_control/Makefile.inc b/plugins/experimental/access_control/Makefile.inc
index 3fefd4f..92d799f 100644
--- a/plugins/experimental/access_control/Makefile.inc
+++ b/plugins/experimental/access_control/Makefile.inc
@@ -28,11 +28,11 @@ experimental_access_control_access_control_la_SOURCES = \
check_PROGRAMS += experimental/access_control/test_access_control
experimental_access_control_test_access_control_CPPFLAGS = $(AM_CPPFLAGS) -I$(abs_top_srcdir)/tests/include -DACCESS_CONTROL_UNIT_TEST
-experimental_access_control_test_access_control_LDADD = $(OPENSSL_LIBS)
+experimental_access_control_test_access_control_LDADD = $(OPENSSL_LIBS) $(top_builddir)/src/tscore/libtscore.la
+
experimental_access_control_test_access_control_SOURCES = \
experimental/access_control/unit_tests/test_access_control.cc \
experimental/access_control/unit_tests/test_utils.cc \
experimental/access_control/access_control.cc \
experimental/access_control/common.cc \
experimental/access_control/utils.cc
-
diff --git a/plugins/experimental/access_control/access_control.cc b/plugins/experimental/access_control/access_control.cc
index 3f71c0f..9cb9495 100644
--- a/plugins/experimental/access_control/access_control.cc
+++ b/plugins/experimental/access_control/access_control.cc
@@ -358,7 +358,7 @@ static const std::map<String, String> _digestAlgosMap = createStaticDigestAlgoMa
* @param hf Hash Function (HF) [optional]
* @param secret secret
* @param message input message
- * @param messageLen input message lenght
+ * @param messageLen input message length
* @param buffer output buffer for storing the message digest
* @param len output buffer length
* @return number of characters actually written to the output buffer.
@@ -437,7 +437,7 @@ accessTokenStatusToString(const AccessTokenStatus &state)
s = "INVALID_FIELD_VALUE";
break;
case INVALID_VERSION:
- s = "UNSUPORTED_VERSION";
+ s = "UNSUPPORTED_VERSION";
break;
case INVALID_SECRET:
s = "NO_SECRET_SPECIFIED";
@@ -461,7 +461,7 @@ accessTokenStatusToString(const AccessTokenStatus &state)
s = "INVALID_KEYID";
break;
case INVALID_HASH_FUNCTION:
- s = "UNSUPORTED_HASH_FUNCTION";
+ s = "UNSUPPORTED_HASH_FUNCTION";
break;
default:
s = "";
diff --git a/plugins/experimental/access_control/headers.cc b/plugins/experimental/access_control/headers.cc
index ddc64e4..a22e0ff 100644
--- a/plugins/experimental/access_control/headers.cc
+++ b/plugins/experimental/access_control/headers.cc
@@ -80,7 +80,7 @@ headerExist(TSMBuffer bufp, TSMLoc hdrLoc, const char *header, int headerlen)
* @param header header name
* @param headerlen header name length
* @param value buffer for the value
- * @param valuelen lenght of the buffer for the value
+ * @param valuelen length of the buffer for the value
* @return pointer to the string with the value.
*/
char *
@@ -129,7 +129,7 @@ getHeader(TSMBuffer bufp, TSMLoc hdrLoc, const char *header, int headerlen, char
* @param header header name
* @param headerlen header name len
* @param value the new value
- * @param valuelen lenght of the value
+ * @param valuelen length of the value
* @return true - OK, false - failed
*/
bool
diff --git a/plugins/experimental/access_control/unit_tests/test_utils.cc b/plugins/experimental/access_control/unit_tests/test_utils.cc
index 9cc69aa..bbad959 100644
--- a/plugins/experimental/access_control/unit_tests/test_utils.cc
+++ b/plugins/experimental/access_control/unit_tests/test_utils.cc
@@ -26,34 +26,33 @@
#include "../common.h"
/*********************************************************************************************************************
- * Base64 related test
- * @note the purpose of these test is not to test Base64 itself since it is provided by openssl library,
- * the idea is to test the usage and some corner cases.
+ * Base64 related tests
+ * @note the purpose of these tests is to test the usage and some corner cases.
********************************************************************************************************************/
TEST_CASE("Base64: estimate buffer size needed to encode a message", "[Base64][access_control][utility]")
{
size_t encodedLen;
- /* Test with a zero decoded message lenght */
+ /* Test with a zero decoded message length */
encodedLen = cryptoBase64EncodedSize(0);
- CHECK(0 == encodedLen);
+ CHECK(4 == encodedLen);
/* Test with a random non-zero decoded message length */
encodedLen = cryptoBase64EncodedSize(64);
- CHECK(88 == encodedLen);
+ CHECK(89 == encodedLen);
/* Test the space for padding. Size of encoding that would result in 2 x "=" padding */
encodedLen = cryptoBase64EncodedSize(strlen("176a1620e31b14782ba2b66de3edc5b3cb19630475b2ce2ee292d5fd0fe41c3abc"));
- CHECK(88 == encodedLen);
+ CHECK(92 == encodedLen);
/* Test the space for padding. Size of encoding that would result in 1 x "=" padding */
encodedLen = cryptoBase64EncodedSize(strlen("176a1620e31b14782ba2b66de3edc5b3cb19630475b2ce2ee292d5fd0fe41c3ab"));
- CHECK(88 == encodedLen);
+ CHECK(90 == encodedLen);
/* Test the space for padding. Size of encoding that would result in no padding */
encodedLen = cryptoBase64EncodedSize(strlen("176a1620e31b14782ba2b66de3edc5b3cb19630475b2ce2ee292d5fd0fe41c3a"));
- CHECK(88 == encodedLen);
+ CHECK(89 == encodedLen);
}
TEST_CASE("Base64: estimate buffer size needed to decode a message", "[Base64][access_control][utility]")
@@ -64,24 +63,24 @@ TEST_CASE("Base64: estimate buffer size needed to decode a message", "[Base64][a
/* Padding with 2 x '=' */
encoded = "MTc2YTE2MjBlMzFiMTQ3ODJiYTJiNjZkZTNlZGM1YjNjYjE5NjMwNDc1YjJjZTJlZTI5MmQ1ZmQwZmU0MWMzYQ==";
encodedLen = cryptoBase64DecodeSize(encoded, strlen(encoded));
- CHECK(64 == encodedLen);
+ CHECK(66 == encodedLen);
/* Padding with 1 x '=' */
encoded = "MTc2YTE2MjBlMzFiMTQ3ODJiYTJiNjZkZTNlZGM1YjNjYjE5NjMwNDc1YjJjZTJlZTI5MmQ1ZmQwZmU0MWMzYWI=";
encodedLen = cryptoBase64DecodeSize(encoded, strlen(encoded));
- CHECK(65 == encodedLen);
+ CHECK(66 == encodedLen);
/* Padding with 0 x "=" */
encoded = "MTc2YTE2MjBlMzFiMTQ3ODJiYTJiNjZkZTNlZGM1YjNjYjE5NjMwNDc1YjJjZTJlZTI5MmQ1ZmQwZmU0MWMzYWJj";
encodedLen = cryptoBase64DecodeSize(encoded, strlen(encoded));
CHECK(66 == encodedLen);
- /* Test empty encoded message caclulation */
+ /* Test empty encoded message calculation */
encoded = "";
encodedLen = cryptoBase64DecodeSize(encoded, strlen(encoded));
CHECK(0 == encodedLen);
- /* Test empty encoded message caclulation */
+ /* Test empty encoded message calculation */
encoded = nullptr;
encodedLen = cryptoBase64DecodeSize(encoded, 0);
CHECK(0 == encodedLen);
@@ -94,16 +93,17 @@ TEST_CASE("Base64: quick encode / decode", "[Base64][access_control][utility]")
CHECK(64 == messageLen);
size_t encodedMessageEstimatedLen = cryptoBase64EncodedSize(messageLen);
- CHECK(88 == encodedMessageEstimatedLen);
+ CHECK(89 == encodedMessageEstimatedLen);
char encodedMessage[encodedMessageEstimatedLen];
+ // now encode message into encodedMessage
size_t encodedMessageLen = cryptoBase64Encode(message, messageLen, encodedMessage, encodedMessageEstimatedLen);
CHECK(88 == encodedMessageLen);
CHECK(0 == strncmp(encodedMessage, "MTc2YTE2MjBlMzFiMTQ3ODJiYTJiNjZkZTNlZGM1YjNjYjE5NjMwNDc1YjJjZTJlZTI5MmQ1ZmQwZmU0MWMzYQ==",
encodedMessageLen));
size_t decodedMessageEstimatedLen = cryptoBase64DecodeSize(encodedMessage, encodedMessageLen);
- CHECK(64 == decodedMessageEstimatedLen);
+ CHECK(66 == decodedMessageEstimatedLen);
char decodedMessage[encodedMessageEstimatedLen];
size_t decodedMessageLen = cryptoBase64Decode(encodedMessage, encodedMessageLen, decodedMessage, encodedMessageLen);
@@ -180,8 +180,9 @@ TEST_CASE("Base64: quick encode / decode with '+', '/' and various paddings", "[
CHECK(0 == strncmp(encodedMessage, encoded[i], encodedMessageLen));
/* Decode */
- size_t decodedMessageEstimatedLen = cryptoBase64DecodeSize(encodedMessage, encodedMessageLen);
- CHECK(strlen(decoded[i]) == decodedMessageEstimatedLen);
+ // Keep test around incase our implementation's estimation gets better
+ // size_t decodedMessageEstimatedLen = cryptoBase64DecodeSize(encodedMessage, encodedMessageLen);
+ // CHECK(strlen(decoded[i]) == decodedMessageEstimatedLen);
char decodedMessage[encodedMessageEstimatedLen];
size_t decodedMessageLen = cryptoBase64Decode(encodedMessage, encodedMessageLen, decodedMessage, encodedMessageLen);
CHECK(strlen(decoded[i]) == decodedMessageLen);
@@ -191,7 +192,6 @@ TEST_CASE("Base64: quick encode / decode with '+', '/' and various paddings", "[
/*********************************************************************************************************************
* Modified Base64 related test
- * @note since Modified Base64 function use Base64 which use openssl the idea is to test the modification itself
* (for more info see the comment in the implementation) + some corner cases.
********************************************************************************************************************/
@@ -215,6 +215,9 @@ TEST_CASE("Base64: modified encode / decode with '+', '/' and various paddings",
char decodedMessage[encodedMessageEstimatedLen];
size_t decodedMessageLen =
cryptoModifiedBase64Decode(encodedMessage, encodedMessageLen, decodedMessage, decodedMessageEstimatedLen);
+ CAPTURE(i);
+ CAPTURE(decoded[i]);
+ CAPTURE(std::string(decodedMessage));
CHECK(strlen(decoded[i]) == decodedMessageLen);
CHECK(0 == strncmp(decodedMessage, message, messageLen));
}
@@ -222,7 +225,6 @@ TEST_CASE("Base64: modified encode / decode with '+', '/' and various paddings",
/*********************************************************************************************************************
* Digest calculation related test
- * @note since Modified Base64 function use Base64 which use openssl the idea is to test the modification itself
* (for more info see the comment in the implementation) + some corner cases.
********************************************************************************************************************/
@@ -236,10 +238,9 @@ TEST_CASE("HMAC Digest: test various supported/unsupported types", "[MAC][access
char out[MAX_MSGDIGEST_BUFFER_SIZE];
char hexOut[MAX_MSGDIGEST_BUFFER_SIZE];
- StringList types = {"MD4", "MD5", "RIPEMD160", "SHA1", "SHA224", "SHA256", "SHA384", "SHA512"};
+ StringList types = {"MD4", "MD5", "SHA1", "SHA224", "SHA256", "SHA384", "SHA512"};
StringList digests = {"6b3057137a6e17613883ac25a628b1b3",
"820117c62fa161804efb3743cc838b81",
- "ccf3230972bcf229fb3b16741495c74a72bbdd14",
"0e3dfdfb04a3dfcd4d195cb1a5e4186feab2e0c1",
"00a6f43962e2b35cb2491f81d59ef2268309c8cde744891188c9b855",
"149333e1db61f9a18a91a13aca0370b89cec4c546360b85530ae2da97b7b1cb9",
@@ -247,6 +248,11 @@ TEST_CASE("HMAC Digest: test various supported/unsupported types", "[MAC][access
"e075c8b0637bc4fb82cdca66a2b72e3c1734f4f78c803e5db7ca879f85f16b2e057fa62bdd09eef5bbea562990d52a671927033056"
"314c19092263f753ecd019"};
+#ifndef OPENSSL_IS_BORINGSSL // RIPEMD160 is not available on BoringSSL?
+ types.push_back("RIPEMD160");
+ digests.push_back("ccf3230972bcf229fb3b16741495c74a72bbdd14");
+#endif
+
StringList::iterator digestIter = digests.begin();
for (String digestType : types) {
size_t outLen = cryptoMessageDigestGet(digestType.c_str(), data.c_str(), data.length(), key.c_str(), key.length(), out,
diff --git a/plugins/experimental/access_control/utils.cc b/plugins/experimental/access_control/utils.cc
index 6476d56..cd212d1 100644
--- a/plugins/experimental/access_control/utils.cc
+++ b/plugins/experimental/access_control/utils.cc
@@ -22,42 +22,15 @@
* @see utils.h
*/
-#include <cerrno> /* errno */
-#include <climits> /* LONG_MIN, LONG_MAX */
-#include <openssl/bio.h> /* BIO I/O abstraction */
-#include <openssl/buffer.h> /* buf_mem_st */
-#include <openssl/err.h> /* ERR_get_error() and ERR_error_string_n() */
+#include <cerrno> /* errno */
+#include <openssl/err.h> /* ERR_get_error() and ERR_error_string_n() */
+#include <openssl/hmac.h>
#include <openssl/crypto.h>
#include "common.h"
#include "utils.h"
-
-/**
- * @brief Parse a counted string containing a long integer
- *
- * @param s ptr to the counted string
- * @param lenght character count
- * @param val where the long integer to be stored
- * @return true - success, false - failed to parse
- */
-bool
-parseStrLong(const char *s, size_t length, long &val)
-{
- // Make an extra copy since strtol expects NULL-terminated strings.
- char str[length + 1];
- strncpy(str, s, length);
- str[length] = 0;
-
- errno = 0;
- char *temp;
- val = strtol(str, &temp, 0);
-
- if (temp == str || *temp != '\0' || ((val == LONG_MIN || val == LONG_MAX) && errno == ERANGE)) {
- AccessControlError("Could not convert '%s' to a long integer and leftover string is: '%s'", str, temp);
- return false;
- }
- return true;
-}
+#include "tscore/ink_base64.h"
+#include "ink_autoconf.h"
/* ******* Encoding/Decoding functions ******* */
@@ -247,54 +220,48 @@ size_t
cryptoMessageDigestGet(const char *digestType, const char *data, size_t dataLen, const char *key, size_t keyLen, char *out,
size_t outLen)
{
- EVP_MD_CTX *ctx = nullptr;
+#ifndef HAVE_HMAC_CTX_NEW
+ HMAC_CTX ctx[1];
+#else
+ HMAC_CTX *ctx;
+#endif
+
const EVP_MD *md = nullptr;
- EVP_PKEY *pkey = nullptr;
- size_t len = outLen;
+ unsigned int len = 0;
char buffer[256];
- size_t result = 0;
- if (!(ctx = EVP_MD_CTX_create())) {
- AccessControlError("failed to create EVP message digest context: %s", cryptoErrStr(buffer, sizeof(buffer)));
- } else {
-#ifndef OPENSSL_IS_BORINGSSL
- if (!(pkey = EVP_PKEY_new_mac_key(EVP_PKEY_HMAC, nullptr, reinterpret_cast<const unsigned char *>(key), keyLen))) {
+ if (!(md = EVP_get_digestbyname(digestType))) {
+ AccessControlError("unknown digest name '%s'", digestType);
+ return 0;
+ }
+
+#ifndef HAVE_HMAC_CTX_NEW
+ HMAC_CTX_init(ctx);
#else
- if (false) {
+ ctx = HMAC_CTX_new();
#endif
- AccessControlError("failed to create EVP private key. %s", cryptoErrStr(buffer, sizeof(buffer)));
- EVP_MD_CTX_destroy(ctx);
- } else {
- do {
- if (!(md = EVP_get_digestbyname(digestType))) {
- AccessControlError("failed to get digest by name %s. %s", digestType, cryptoErrStr(buffer, sizeof(buffer)));
- break;
- }
-
- if (1 != EVP_DigestSignInit(ctx, nullptr, md, nullptr, pkey)) {
- AccessControlError("failed to set up signing context. %s", cryptoErrStr(buffer, sizeof(buffer)));
- break;
- }
-
- if (1 != EVP_DigestSignUpdate(ctx, data, dataLen)) {
- AccessControlError("failed to update the signing hash. %s", cryptoErrStr(buffer, sizeof(buffer)));
- break;
- }
-
- if (1 != EVP_DigestSignFinal(ctx, reinterpret_cast<unsigned char *>(out), &len)) {
- AccessControlError("failed to finalize the signing hash. %s", cryptoErrStr(buffer, sizeof(buffer)));
- }
-
- /* success */
- result = len;
- } while (false);
-
- EVP_PKEY_free(pkey);
- EVP_MD_CTX_destroy(ctx);
- }
+ if (!HMAC_Init_ex(ctx, key, keyLen, md, NULL)) {
+ AccessControlError("failed to create EVP message digest context: %s", cryptoErrStr(buffer, sizeof(buffer)));
+ goto err;
}
- return result;
+ if (!HMAC_Update(ctx, reinterpret_cast<const unsigned char *>(data), dataLen)) {
+ AccessControlError("failed to update the signing hash: %s", cryptoErrStr(buffer, sizeof(buffer)));
+ goto err;
+ }
+
+ if (!HMAC_Final(ctx, reinterpret_cast<unsigned char *>(out), &len)) {
+ AccessControlError("failed to finalize the signing hash: %s", cryptoErrStr(buffer, sizeof(buffer)));
+ goto err;
+ }
+
+err:
+#ifndef HAVE_HMAC_CTX_NEW
+ HMAC_CTX_cleanup(ctx);
+#else
+ HMAC_CTX_free(ctx);
+#endif
+ return len;
}
/**
@@ -329,7 +296,7 @@ cryptoMessageDigestEqual(const char *md1, size_t md1Len, const char *md2, size_t
size_t
cryptoBase64EncodedSize(size_t decodedSize)
{
- return (((4 * decodedSize) / 3) + 3) & ~3;
+ return ATS_BASE64_ENCODE_DSTLEN(decodedSize);
}
/**
@@ -341,22 +308,7 @@ cryptoBase64EncodedSize(size_t decodedSize)
size_t
cryptoBase64DecodeSize(const char *encoded, size_t encodedLen)
{
- if (nullptr == encoded || 0 == encodedLen) {
- return 0;
- }
-
- size_t padding = 0;
- const char *end = encoded + encodedLen;
-
- if ('=' == *(--end)) {
- padding++;
- }
-
- if ('=' == *(--end)) {
- padding++;
- }
-
- return (3 * encodedLen) / 4 - padding;
+ return ATS_BASE64_DECODE_DSTLEN(encodedLen);
}
/**
@@ -375,27 +327,11 @@ cryptoBase64Encode(const char *in, size_t inLen, char *out, size_t outLen)
return 0;
}
- BIO *head, *b64, *bmem;
- BUF_MEM *bptr;
- size_t len = 0;
-
- head = b64 = BIO_new(BIO_f_base64());
- if (nullptr != b64) {
- BIO_set_flags(b64, BIO_FLAGS_BASE64_NO_NL);
- bmem = BIO_new(BIO_s_mem());
- if (nullptr != bmem) {
- head = BIO_push(b64, bmem);
-
- BIO_write(head, in, inLen);
- (void)BIO_flush(head);
-
- BIO_get_mem_ptr(head, &bptr);
- len = bptr->length < outLen ? bptr->length : outLen;
- strncpy(out, bptr->data, len);
- }
- BIO_free_all(head);
+ if (!ats_base64_encode(in, inLen, out, outLen, &outLen)) {
+ return 0;
}
- return len;
+
+ return outLen;
}
/**
@@ -414,21 +350,11 @@ cryptoBase64Decode(const char *in, size_t inLen, char *out, size_t outLen)
return 0;
}
- BIO *head, *bmem, *b64;
- size_t len = 0;
-
- head = b64 = BIO_new(BIO_f_base64());
- if (nullptr != b64) {
- BIO_set_flags(b64, BIO_FLAGS_BASE64_NO_NL);
- bmem = BIO_new_mem_buf((void *)in, inLen);
- if (nullptr != bmem) {
- head = BIO_push(b64, bmem);
- len = BIO_read(head, out, outLen);
- }
- BIO_free_all(head);
+ if (!ats_base64_decode(in, inLen, reinterpret_cast<unsigned char *>(out), outLen, &outLen)) {
+ return 0;
}
- return len;
+ return outLen;
}
/**
diff --git a/plugins/experimental/access_control/utils.h b/plugins/experimental/access_control/utils.h
index 6c821df..ebdb5f2 100644
--- a/plugins/experimental/access_control/utils.h
+++ b/plugins/experimental/access_control/utils.h
@@ -30,8 +30,6 @@
#define MAX_MSGDIGEST_BUFFER_SIZE EVP_MAX_MD_SIZE
-bool parseStrLong(const char *s, size_t len, long &val);
-
/* ******* Encoding/Decoding functions ******* */
size_t hexEncode(const char *in, size_t inLen, char *out, size_t outLen);
[trafficserver] 03/20: Add synchronization between
UDPNetProcessor::UDPBind in main Thread and initialize_thread_for_udp_net
in ET_UDP Thread (#7407)
Posted by zw...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 9.1.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
commit cd434e7f4e0277d134a8a1cbc38337308e7d9e87
Author: YuanYingdong <19...@qq.com>
AuthorDate: Fri Jan 29 07:16:26 2021 +0800
Add synchronization between UDPNetProcessor::UDPBind in main Thread and initialize_thread_for_udp_net in ET_UDP Thread (#7407)
Co-authored-by: yingdongyuan <yi...@deeproute.ai>
---
proxy/http/HttpProxyServerMain.cc | 13 +++++++++++++
proxy/http/HttpProxyServerMain.h | 4 ++++
src/traffic_server/traffic_server.cc | 7 +++++++
3 files changed, 24 insertions(+)
diff --git a/proxy/http/HttpProxyServerMain.cc b/proxy/http/HttpProxyServerMain.cc
index 2a75537..0da59e0 100644
--- a/proxy/http/HttpProxyServerMain.cc
+++ b/proxy/http/HttpProxyServerMain.cc
@@ -56,6 +56,10 @@ std::mutex proxyServerMutex;
std::condition_variable proxyServerCheck;
bool et_net_threads_ready = false;
+std::mutex etUdpMutex;
+std::condition_variable etUdpCheck;
+bool et_udp_threads_ready = false;
+
extern int num_of_net_threads;
extern int num_accept_threads;
@@ -322,6 +326,15 @@ init_HttpProxyServer()
lock.unlock();
proxyServerCheck.notify_one();
}
+
+#if TS_USE_QUIC == 1
+ if (eventProcessor.has_tg_started(ET_UDP)) {
+ std::unique_lock<std::mutex> lock(etUdpMutex);
+ et_udp_threads_ready = true;
+ lock.unlock();
+ etUdpCheck.notify_one();
+ }
+#endif
}
void
diff --git a/proxy/http/HttpProxyServerMain.h b/proxy/http/HttpProxyServerMain.h
index 1de9ced..46f59aa 100644
--- a/proxy/http/HttpProxyServerMain.h
+++ b/proxy/http/HttpProxyServerMain.h
@@ -52,3 +52,7 @@ NetProcessor::AcceptOptions make_net_accept_options(const HttpProxyPort *port, u
extern std::mutex proxyServerMutex;
extern std::condition_variable proxyServerCheck;
extern bool et_net_threads_ready;
+
+extern std::mutex etUdpMutex;
+extern std::condition_variable etUdpCheck;
+extern bool et_udp_threads_ready;
diff --git a/src/traffic_server/traffic_server.cc b/src/traffic_server/traffic_server.cc
index 4f54e98..c251698 100644
--- a/src/traffic_server/traffic_server.cc
+++ b/src/traffic_server/traffic_server.cc
@@ -2098,6 +2098,7 @@ main(int /* argc ATS_UNUSED */, const char **argv)
}
if (num_of_udp_threads) {
udpNet.start(num_of_udp_threads, stacksize);
+ eventProcessor.thread_group[ET_UDP]._afterStartCallback = init_HttpProxyServer;
}
// Initialize Response Body Factory
@@ -2135,6 +2136,12 @@ main(int /* argc ATS_UNUSED */, const char **argv)
proxyServerCheck.wait(lock, [] { return et_net_threads_ready; });
}
+#if TS_USE_QUIC == 1
+ if (num_of_udp_threads) {
+ std::unique_lock<std::mutex> lock(etUdpMutex);
+ etUdpCheck.wait(lock, [] { return et_udp_threads_ready; });
+ }
+#endif
// Delay only if config value set and flag value is zero
// (-1 => cache already initialized)
if (delay_p && ink_atomic_cas(&delay_listen_for_cache, 0, 1)) {
[trafficserver] 14/20: Proxy Verifier: Making use of delay
directives for caching tests. (#7468)
Posted by zw...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 9.1.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
commit 6b34b1614b2075197b8bb831d99a3b2f7cd98486
Author: Brian Neradt <br...@gmail.com>
AuthorDate: Mon Feb 1 17:20:04 2021 -0600
Proxy Verifier: Making use of delay directives for caching tests. (#7468)
The non-Proxy Verifier tests handle transactions more slowly than Proxy
Verifier since they spin up a new AuTest curl process for each one.
Proxy Verifier, on the other hand, runs all the transactions in a replay
file in one process. It has been observed that certain tests that rely
upon caching between transactions can get false negatives because a
subsequent transaction occurs before the IO for caching completes. This
can also result in false positives too where the inverse happens: the
ATS process may be incorrectly trying to cache something it shouldn't,
but the test misses it because the subsequent transaction happens too
quickly.
This makes use of Proxy Verifier's delay directive to slow down certain
transactions to give enough time for any IO to complete.
---
.../cache/replay/cache-control-max-age.replay.yaml | 28 ++++++++++++++++++++++
...negative-caching-300-second-timeout.replay.yaml | 4 ++++
.../replay/negative-caching-customized.replay.yaml | 12 ++++++++++
.../replay/negative-caching-default.replay.yaml | 16 +++++++++++++
.../replay/negative-caching-disabled.replay.yaml | 17 +++++++++++++
.../replay/negative-caching-no-timeout.replay.yaml | 4 ++++
.../replay/negative-caching-timeout.replay.yaml | 4 ++++
7 files changed, 85 insertions(+)
diff --git a/tests/gold_tests/cache/replay/cache-control-max-age.replay.yaml b/tests/gold_tests/cache/replay/cache-control-max-age.replay.yaml
index ba6305d..8e88e71 100644
--- a/tests/gold_tests/cache/replay/cache-control-max-age.replay.yaml
+++ b/tests/gold_tests/cache/replay/cache-control-max-age.replay.yaml
@@ -34,6 +34,10 @@ meta:
fields:
- [ Host, example.com ]
+ # Add a delay so ATS has time to finish any caching IO for the previous
+ # transaction.
+ delay: 50ms
+
- request_for_zero_max_age: &request_for_zero_max_age
client-request:
method: "GET"
@@ -44,6 +48,10 @@ meta:
fields:
- [ Host, example.com ]
+ # Add a delay so ATS has time to finish any caching IO for the previous
+ # transaction.
+ delay: 50ms
+
- request_for_negative_max_age: &request_for_negative_max_age
client-request:
method: "GET"
@@ -54,6 +62,10 @@ meta:
fields:
- [ Host, example.com ]
+ # Add a delay so ATS has time to finish any caching IO for the previous
+ # transaction.
+ delay: 50ms
+
- request_for_non_number_max_age: &request_for_non_number_max_age
client-request:
method: "GET"
@@ -64,6 +76,10 @@ meta:
fields:
- [ Host, example.com ]
+ # Add a delay so ATS has time to finish any caching IO for the previous
+ # transaction.
+ delay: 50ms
+
- 200_ok_response: &200_ok_response
server-response:
status: 200
@@ -249,6 +265,10 @@ sessions:
- [ Host, example.com ]
- [ Cache-Control, max-age=300 ]
+ # Add a delay so ATS has time to finish any caching IO for the previous
+ # transaction.
+ delay: 50ms
+
# This should be replied to out of the cache, therefore this 400 response
# should not make it to the client.
server-response:
@@ -296,6 +316,10 @@ sessions:
- [ Host, example.com ]
- [ Cache-Control, max-age=0 ]
+ # Add a delay so ATS has time to finish any caching IO for the previous
+ # transaction.
+ delay: 50ms
+
# This should go through to the server because the 0 max-age
# directive in the request should make ATS consider it stale.
server-response:
@@ -345,6 +369,10 @@ sessions:
- [ Host, example.com ]
- [ Cache-Control, max-age=-300 ]
+ # Add a delay so ATS has time to finish any caching IO for the previous
+ # transaction.
+ delay: 50ms
+
# This should go through to the server because the negative max-age
# directive in the request should make ATS consider it stale.
server-response:
diff --git a/tests/gold_tests/cache/replay/negative-caching-300-second-timeout.replay.yaml b/tests/gold_tests/cache/replay/negative-caching-300-second-timeout.replay.yaml
index 999b8cd..536c7a6 100644
--- a/tests/gold_tests/cache/replay/negative-caching-300-second-timeout.replay.yaml
+++ b/tests/gold_tests/cache/replay/negative-caching-300-second-timeout.replay.yaml
@@ -36,6 +36,10 @@ meta:
fields:
- [ Host, example.com ]
+ # Add a delay so ATS has time to finish any caching IO for the previous
+ # transaction.
+ delay: 50ms
+
sessions:
- transactions:
diff --git a/tests/gold_tests/cache/replay/negative-caching-customized.replay.yaml b/tests/gold_tests/cache/replay/negative-caching-customized.replay.yaml
index 6d51bd3..3b12d46 100644
--- a/tests/gold_tests/cache/replay/negative-caching-customized.replay.yaml
+++ b/tests/gold_tests/cache/replay/negative-caching-customized.replay.yaml
@@ -72,6 +72,10 @@ sessions:
fields:
- [ Host, example.com ]
+ # Add a delay so ATS has time to finish any caching IO for the previous
+ # transaction.
+ delay: 50ms
+
# Since 404 responses are customized to not be cached, this will go
# through.
<<: *200_response
@@ -115,6 +119,10 @@ sessions:
fields:
- [ Host, example.com ]
+ # Add a delay so ATS has time to finish any caching IO for the previous
+ # transaction.
+ delay: 50ms
+
# By customization, the 400 will be cached and this will not go through.
<<: *200_response
@@ -151,6 +159,10 @@ sessions:
fields:
- [ Host, example.com ]
+ # Add a delay so ATS has time to finish any caching IO for the previous
+ # transaction.
+ delay: 50ms
+
# This should not go to the server since the 200 response is cached.
server-response:
status: 400
diff --git a/tests/gold_tests/cache/replay/negative-caching-default.replay.yaml b/tests/gold_tests/cache/replay/negative-caching-default.replay.yaml
index f06f30c..0ab66f5 100644
--- a/tests/gold_tests/cache/replay/negative-caching-default.replay.yaml
+++ b/tests/gold_tests/cache/replay/negative-caching-default.replay.yaml
@@ -69,6 +69,10 @@ sessions:
fields:
- [ Host, example.com ]
+ # Add a delay so ATS has time to finish any caching IO for the previous
+ # transaction.
+ delay: 50ms
+
# 404 responses should be cached when negative caching is enabled, so this
# should not get to the server. But if it does, return a 200 so the test
# knows that something went wrong.
@@ -113,6 +117,10 @@ sessions:
fields:
- [ Host, example.com ]
+ # Add a delay so ATS has time to finish any caching IO for the previous
+ # transaction.
+ delay: 50ms
+
# 400 responses should not be cached. Verify this goes to the server
# by returning and expecting a 200 response.
<<: *200_response
@@ -149,6 +157,10 @@ sessions:
fields:
- [ Host, example.com ]
+ # Add a delay so ATS has time to finish any caching IO for the previous
+ # transaction.
+ delay: 50ms
+
# This should not go to the server, but if it does return a 400 so we catch
# it.
server-response:
@@ -197,6 +209,10 @@ sessions:
fields:
- [ Host, example.com ]
+ # Add a delay so ATS has time to finish any caching IO for the previous
+ # transaction.
+ delay: 50ms
+
# 405 responses should not be cached. Verify this goes to the server
# by returning and expecting a 200 response.
<<: *200_response
diff --git a/tests/gold_tests/cache/replay/negative-caching-disabled.replay.yaml b/tests/gold_tests/cache/replay/negative-caching-disabled.replay.yaml
index 1ef338a..d09362a 100644
--- a/tests/gold_tests/cache/replay/negative-caching-disabled.replay.yaml
+++ b/tests/gold_tests/cache/replay/negative-caching-disabled.replay.yaml
@@ -32,6 +32,10 @@ meta:
fields:
- [ Host, example.com ]
+ # Add a delay so ATS has time to finish any caching IO for the previous
+ # transaction.
+ delay: 50ms
+
- request_for_path_404: &request_for_path_404
client-request:
method: "GET"
@@ -42,6 +46,10 @@ meta:
fields:
- [ Host, example.com ]
+ # Add a delay so ATS has time to finish any caching IO for the previous
+ # transaction.
+ delay: 50ms
+
- request_for_no_cache_control_response: &request_for_no_cache_control_response
client-request:
method: "GET"
@@ -52,6 +60,11 @@ meta:
fields:
- [ Host, example.com ]
+ # Add a delay so ATS has time to finish any caching IO for the previous
+ # transaction.
+ delay: 50ms
+
+
- request_for_404_with_cc: &request_for_404_with_cc
client-request:
method: "GET"
@@ -62,6 +75,10 @@ meta:
fields:
- [ Host, example.com ]
+ # Add a delay so ATS has time to finish any caching IO for the previous
+ # transaction.
+ delay: 50ms
+
sessions:
- transactions:
diff --git a/tests/gold_tests/cache/replay/negative-caching-no-timeout.replay.yaml b/tests/gold_tests/cache/replay/negative-caching-no-timeout.replay.yaml
index 6f15278..634b141 100644
--- a/tests/gold_tests/cache/replay/negative-caching-no-timeout.replay.yaml
+++ b/tests/gold_tests/cache/replay/negative-caching-no-timeout.replay.yaml
@@ -39,6 +39,10 @@ sessions:
fields:
- [ Host, example.com ]
+ # Add a delay so ATS has time to finish any caching IO for the previous
+ # transaction.
+ delay: 50ms
+
# This should not go to the server. Verify we get the cached 404 response
# instead of this new 403 response.
server-response:
diff --git a/tests/gold_tests/cache/replay/negative-caching-timeout.replay.yaml b/tests/gold_tests/cache/replay/negative-caching-timeout.replay.yaml
index 02cf1e9..1c1e01e 100644
--- a/tests/gold_tests/cache/replay/negative-caching-timeout.replay.yaml
+++ b/tests/gold_tests/cache/replay/negative-caching-timeout.replay.yaml
@@ -74,6 +74,10 @@ sessions:
fields:
- [ Host, example.com ]
+ # Add a delay so ATS has time to finish any caching IO for the previous
+ # transaction.
+ delay: 50ms
+
# 403 responses should be cached when negative caching is enabled, so this
# should not get to the server. But if it does, return a 404 so the test
# knows that something went wrong.
[trafficserver] 07/20: Update to the new MicroServer 1.0.6 release
(#7460)
Posted by zw...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 9.1.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
commit 28716256b29261a57ad39559ac2e4bbb954f2d80
Author: Brian Neradt <br...@gmail.com>
AuthorDate: Fri Jan 29 13:50:03 2021 -0600
Update to the new MicroServer 1.0.6 release (#7460)
The new MicroServer 1.0.6 release has the following:
1. Fixes a bug in response body generation to POST requests.
2. Implements the handling of non-existent parts of keys in
transactions.
---
tests/Pipfile | 2 +-
tests/gold_tests/h2/gold/post_chunked.gold | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/tests/Pipfile b/tests/Pipfile
index e40e3b2..727ea1e 100644
--- a/tests/Pipfile
+++ b/tests/Pipfile
@@ -32,7 +32,7 @@ dnslib = "*"
requests = "*"
gunicorn = "*"
httpbin = "*"
-microserver = ">=1.0.5"
+microserver = ">=1.0.6"
jsonschema = "*"
python-jose = "*"
diff --git a/tests/gold_tests/h2/gold/post_chunked.gold b/tests/gold_tests/h2/gold/post_chunked.gold
index 0ff06d1..ad47100 100644
--- a/tests/gold_tests/h2/gold/post_chunked.gold
+++ b/tests/gold_tests/h2/gold/post_chunked.gold
@@ -1 +1 @@
-abbbbbbbbb
\ No newline at end of file
+0123456789
\ No newline at end of file
[trafficserver] 04/20: Generalize SNI support (#6870)
Posted by zw...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 9.1.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
commit 3ba9fbdb97db59b9266048743c8673baeaf2a718
Author: Masakazu Kitajo <ma...@apache.org>
AuthorDate: Fri Jan 29 09:08:19 2021 +0900
Generalize SNI support (#6870)
* Introduce SNISupport to generalize callback function
This makes callback functions for SNI usable on QUIC connections.
* Move code for SNI actions from the callback functions to SNIActionPerformers
---
iocore/net/Makefile.am | 1 +
iocore/net/P_SNIActionPerformer.h | 31 ++++----
iocore/net/P_SSLNetVConnection.h | 17 +++--
iocore/net/SSLNetVConnection.cc | 13 ++--
iocore/net/SSLUtils.cc | 121 +++++++++----------------------
iocore/net/TLSSNISupport.cc | 145 ++++++++++++++++++++++++++++++++++++++
iocore/net/TLSSNISupport.h | 61 ++++++++++++++++
7 files changed, 273 insertions(+), 116 deletions(-)
diff --git a/iocore/net/Makefile.am b/iocore/net/Makefile.am
index 0209c3a..34d9fff 100644
--- a/iocore/net/Makefile.am
+++ b/iocore/net/Makefile.am
@@ -185,6 +185,7 @@ libinknet_a_SOURCES = \
SSLUtils.cc \
OCSPStapling.cc \
TLSSessionResumptionSupport.cc \
+ TLSSNISupport.cc \
UDPIOEvent.cc \
UnixConnection.cc \
UnixNet.cc \
diff --git a/iocore/net/P_SNIActionPerformer.h b/iocore/net/P_SNIActionPerformer.h
index cd0cd6e..b255969 100644
--- a/iocore/net/P_SNIActionPerformer.h
+++ b/iocore/net/P_SNIActionPerformer.h
@@ -49,7 +49,7 @@ public:
std::optional<std::vector<std::string>> _fqdn_wildcard_captured_groups;
};
- virtual int SNIAction(Continuation *cont, const Context &ctx) const = 0;
+ virtual int SNIAction(TLSSNISupport *snis, const Context &ctx) const = 0;
/**
This method tests whether this action would have been triggered by a
@@ -72,9 +72,9 @@ public:
~ControlH2() override {}
int
- SNIAction(Continuation *cont, const Context &ctx) const override
+ SNIAction(TLSSNISupport *snis, const Context &ctx) const override
{
- auto ssl_vc = dynamic_cast<SSLNetVConnection *>(cont);
+ auto ssl_vc = dynamic_cast<SSLNetVConnection *>(snis);
if (ssl_vc) {
if (!enable_h2) {
ssl_vc->disableProtocol(TS_ALPN_PROTOCOL_INDEX_HTTP_2_0);
@@ -100,10 +100,10 @@ public:
~TunnelDestination() override {}
int
- SNIAction(Continuation *cont, const Context &ctx) const override
+ SNIAction(TLSSNISupport *snis, const Context &ctx) const override
{
// Set the netvc option?
- SSLNetVConnection *ssl_netvc = dynamic_cast<SSLNetVConnection *>(cont);
+ SSLNetVConnection *ssl_netvc = dynamic_cast<SSLNetVConnection *>(snis);
if (ssl_netvc) {
// If needed, we will try to amend the tunnel destination.
if (ctx._fqdn_wildcard_captured_groups && need_fix) {
@@ -113,6 +113,9 @@ public:
} else {
ssl_netvc->set_tunnel_destination(destination, tunnel_decrypt, tls_upstream);
}
+ if (ssl_netvc->has_tunnel_destination() && !ssl_netvc->decrypt_tunnel()) {
+ ssl_netvc->attributes = HttpProxyPort::TRANSPORT_BLIND_TUNNEL;
+ }
}
return SSL_TLSEXT_ERR_OK;
}
@@ -201,9 +204,9 @@ public:
VerifyClient(const char *param, std::string_view file, std::string_view dir) : VerifyClient(atoi(param), file, dir) {}
~VerifyClient() override;
int
- SNIAction(Continuation *cont, const Context &ctx) const override
+ SNIAction(TLSSNISupport *snis, const Context &ctx) const override
{
- auto ssl_vc = dynamic_cast<SSLNetVConnection *>(cont);
+ auto ssl_vc = dynamic_cast<SSLNetVConnection *>(snis);
Debug("ssl_sni", "action verify param %d", this->mode);
setClientCertLevel(ssl_vc->ssl, this->mode);
ssl_vc->set_ca_cert_file(ca_file, ca_dir);
@@ -232,7 +235,7 @@ public:
HostSniPolicy(uint8_t param) : policy(param) {}
~HostSniPolicy() override {}
int
- SNIAction(Continuation *cont, const Context &ctx) const override
+ SNIAction(TLSSNISupport *snis, const Context &ctx) const override
{
// On action this doesn't do anything
return SSL_TLSEXT_ERR_OK;
@@ -261,14 +264,14 @@ public:
TLSValidProtocols() : protocol_mask(max_mask) {}
TLSValidProtocols(unsigned long protocols) : unset(false), protocol_mask(protocols) {}
int
- SNIAction(Continuation *cont, const Context & /* ctx */) const override
+ SNIAction(TLSSNISupport *snis, const Context & /* ctx */) const override
{
if (!unset) {
- auto ssl_vc = dynamic_cast<SSLNetVConnection *>(cont);
+ auto ssl_vc = dynamic_cast<SSLNetVConnection *>(snis);
Debug("ssl_sni", "TLSValidProtocol param 0%x", static_cast<unsigned int>(this->protocol_mask));
- ssl_vc->protocol_mask_set = true;
- ssl_vc->protocol_mask = protocol_mask;
+ ssl_vc->set_valid_tls_protocols(protocol_mask, TLSValidProtocols::max_mask);
}
+
return SSL_TLSEXT_ERR_OK;
}
};
@@ -301,14 +304,14 @@ public:
} // end function SNI_IpAllow
int
- SNIAction(Continuation *cont, const Context &ctx) const override
+ SNIAction(TLSSNISupport *snis, const Context &ctx) const override
{
// i.e, ip filtering is not required
if (ip_map.count() == 0) {
return SSL_TLSEXT_ERR_OK;
}
- auto ssl_vc = dynamic_cast<SSLNetVConnection *>(cont);
+ auto ssl_vc = dynamic_cast<SSLNetVConnection *>(snis);
auto ip = ssl_vc->get_remote_endpoint();
// check the allowed ips
diff --git a/iocore/net/P_SSLNetVConnection.h b/iocore/net/P_SSLNetVConnection.h
index da47678..3a85970 100644
--- a/iocore/net/P_SSLNetVConnection.h
+++ b/iocore/net/P_SSLNetVConnection.h
@@ -48,6 +48,7 @@
#include "P_UnixNet.h"
#include "P_ALPNSupport.h"
#include "TLSSessionResumptionSupport.h"
+#include "TLSSNISupport.h"
#include "P_SSLUtils.h"
#include "P_SSLConfig.h"
@@ -93,7 +94,7 @@ enum SSLHandshakeStatus { SSL_HANDSHAKE_ONGOING, SSL_HANDSHAKE_DONE, SSL_HANDSHA
// A VConnection for a network socket.
//
//////////////////////////////////////////////////////////////////
-class SSLNetVConnection : public UnixNetVConnection, public ALPNSupport, public TLSSessionResumptionSupport
+class SSLNetVConnection : public UnixNetVConnection, public ALPNSupport, public TLSSessionResumptionSupport, public TLSSNISupport
{
typedef UnixNetVConnection super; ///< Parent type.
@@ -384,11 +385,9 @@ public:
const char *
get_server_name() const override
{
- return _serverName.get() ? _serverName.get() : "";
+ return _get_sni_server_name() ? _get_sni_server_name() : "";
}
- void set_server_name(std::string_view name);
-
bool
support_sni() const override
{
@@ -473,6 +472,13 @@ public:
return _ca_cert_dir.get();
}
+ void
+ set_valid_tls_protocols(unsigned long proto_mask, unsigned long max_mask)
+ {
+ SSL_set_options(this->ssl, proto_mask);
+ SSL_clear_options(this->ssl, max_mask & ~proto_mask);
+ }
+
protected:
const IpEndpoint &
_getLocalEndpoint() override
@@ -480,6 +486,8 @@ protected:
return local_addr;
}
+ void _fire_ssl_servername_event() override;
+
private:
std::string_view map_tls_protocol_to_tag(const char *proto_string) const;
bool update_rbio(bool move_to_socket);
@@ -526,7 +534,6 @@ private:
X509_STORE_CTX *verify_cert = nullptr;
// Null-terminated string, or nullptr if there is no SNI server name.
- std::unique_ptr<char[]> _serverName;
std::unique_ptr<char[]> _ca_cert_file;
std::unique_ptr<char[]> _ca_cert_dir;
diff --git a/iocore/net/SSLNetVConnection.cc b/iocore/net/SSLNetVConnection.cc
index 1f525df..f4eaaa8 100644
--- a/iocore/net/SSLNetVConnection.cc
+++ b/iocore/net/SSLNetVConnection.cc
@@ -215,6 +215,7 @@ SSLNetVConnection::_bindSSLObject()
{
SSLNetVCAttach(this->ssl, this);
TLSSessionResumptionSupport::bind(this->ssl, this);
+ TLSSNISupport::bind(this->ssl, this);
}
void
@@ -222,6 +223,7 @@ SSLNetVConnection::_unbindSSLObject()
{
SSLNetVCDetach(this->ssl);
TLSSessionResumptionSupport::unbind(this->ssl);
+ TLSSNISupport::unbind(this->ssl);
}
static void
@@ -928,7 +930,6 @@ SSLNetVConnection::do_io_close(int lerrno)
void
SSLNetVConnection::clear()
{
- _serverName.reset();
_ca_cert_file.reset();
_ca_cert_dir.reset();
@@ -938,6 +939,7 @@ SSLNetVConnection::clear()
}
ALPNSupport::clear();
TLSSessionResumptionSupport::clear();
+ TLSSNISupport::_clear();
sslHandshakeStatus = SSL_HANDSHAKE_ONGOING;
sslHandshakeBeginTime = 0;
@@ -1918,14 +1920,9 @@ SSLNetVConnection::protocol_contains(std::string_view prefix) const
}
void
-SSLNetVConnection::set_server_name(std::string_view name)
+SSLNetVConnection::_fire_ssl_servername_event()
{
- if (name.size()) {
- char *n = new char[name.size() + 1];
- std::memcpy(n, name.data(), name.size());
- n[name.size()] = '\0';
- _serverName.reset(n);
- }
+ this->callHooks(TS_EVENT_SSL_SERVERNAME);
}
void
diff --git a/iocore/net/SSLUtils.cc b/iocore/net/SSLUtils.cc
index 54a9d35..d34d6b8 100644
--- a/iocore/net/SSLUtils.cc
+++ b/iocore/net/SSLUtils.cc
@@ -347,76 +347,31 @@ ssl_verify_client_callback(int preverify_ok, X509_STORE_CTX *ctx)
return preverify_ok;
}
-static int
-PerformAction(Continuation *cont, const char *servername)
-{
- SNIConfig::scoped_config params;
- if (const auto &actions = params->get(servername); !actions.first) {
- Debug("ssl_sni", "%s not available in the map", servername);
- } else {
- for (auto &&item : *actions.first) {
- auto ret = item->SNIAction(cont, actions.second);
- if (ret != SSL_TLSEXT_ERR_OK) {
- return ret;
- }
- }
- }
- return SSL_TLSEXT_ERR_OK;
-}
-
#if TS_USE_HELLO_CB
// Pausable callback
static int
ssl_client_hello_callback(SSL *s, int *al, void *arg)
{
- SSLNetVConnection *netvc = SSLNetVCAccess(s);
- const char *servername = nullptr;
- const unsigned char *p;
- size_t remaining, len;
-
- if (!netvc || netvc->ssl != s) {
- Debug("ssl.error", "ssl_client_hello_callback call back on stale netvc");
+ TLSSNISupport *snis = TLSSNISupport::getInstance(s);
+ if (snis) {
+ snis->on_client_hello(s, al, arg);
+ int ret = snis->perform_sni_action();
+ if (ret != SSL_TLSEXT_ERR_OK) {
+ return SSL_CLIENT_HELLO_ERROR;
+ }
+ } else {
+ // This error suggests either of these:
+ // 1) Call back on unsupported netvc -- Don't register callback unnecessarily
+ // 2) Call back on stale netvc
+ Debug("ssl.error", "ssl_client_hello_callback was called unexpectedly");
return SSL_CLIENT_HELLO_ERROR;
}
- // Parse the server name if the get extension call succeeds and there are more than 2 bytes to parse
- if (SSL_client_hello_get0_ext(s, TLSEXT_TYPE_server_name, &p, &remaining) && remaining > 2) {
- // Parse to get to the name, originally from test/handshake_helper.c in openssl tree
- /* Extract the length of the supplied list of names. */
- len = *(p++) << 8;
- len += *(p++);
- if (len + 2 == remaining) {
- remaining = len;
- /*
- * The list in practice only has a single element, so we only consider
- * the first one.
- */
- if (*p++ == TLSEXT_NAMETYPE_host_name) {
- remaining--;
- /* Now we can finally pull out the byte array with the actual hostname. */
- if (remaining > 2) {
- len = *(p++) << 8;
- len += *(p++);
- if (len + 2 <= remaining) {
- servername = reinterpret_cast<const char *>(p);
- }
- }
- }
- }
- }
- if (servername) {
- netvc->set_server_name(std::string_view(servername, len));
- }
- int ret = PerformAction(netvc, netvc->get_server_name());
- if (ret != SSL_TLSEXT_ERR_OK) {
+ SSLNetVConnection *netvc = SSLNetVCAccess(s);
+ if (!netvc || netvc->ssl != s) {
+ Debug("ssl.error", "ssl_client_hello_callback call back on stale netvc");
return SSL_CLIENT_HELLO_ERROR;
}
- if (netvc->has_tunnel_destination() && !netvc->decrypt_tunnel()) {
- netvc->attributes = HttpProxyPort::TRANSPORT_BLIND_TUNNEL;
- }
- if (netvc->protocol_mask_set) {
- setTLSValidProtocols(s, netvc->protocol_mask, TLSValidProtocols::max_mask);
- }
bool reenabled = netvc->callHooks(TS_EVENT_SSL_CLIENT_HELLO);
@@ -473,32 +428,26 @@ ssl_cert_callback(SSL *ssl, void * /*arg*/)
* Cannot stop this callback. Always reeneabled
*/
static int
-ssl_servername_callback(SSL *ssl, int * /* ad */, void * /*arg*/)
+ssl_servername_callback(SSL *ssl, int *al, void *arg)
{
- SSLNetVConnection *netvc = SSLNetVCAccess(ssl);
-
- if (!netvc || netvc->ssl != ssl) {
- Debug("ssl.error", "ssl_servername_callback call back on stale netvc");
- return SSL_TLSEXT_ERR_ALERT_FATAL;
- }
-
- netvc->callHooks(TS_EVENT_SSL_SERVERNAME);
-
- const char *name = SSL_get_servername(ssl, TLSEXT_NAMETYPE_host_name);
- if (name) {
- netvc->set_server_name(name);
- }
-
+ TLSSNISupport *snis = TLSSNISupport::getInstance(ssl);
+ if (snis) {
+ snis->on_servername(ssl, al, arg);
#if !TS_USE_HELLO_CB
- // Only call the SNI actions here if not already performed in the HELLO_CB
- int ret = PerformAction(netvc, netvc->get_server_name());
- if (ret != SSL_TLSEXT_ERR_OK) {
- return SSL_TLSEXT_ERR_ALERT_FATAL;
- }
+ // Only call the SNI actions here if not already performed in the HELLO_CB
+ int ret = snis->perform_sni_action();
+ if (ret != SSL_TLSEXT_ERR_OK) {
+ return SSL_TLSEXT_ERR_ALERT_FATAL;
+ }
#endif
- if (netvc->has_tunnel_destination() && !netvc->decrypt_tunnel()) {
- netvc->attributes = HttpProxyPort::TRANSPORT_BLIND_TUNNEL;
+ } else {
+ // This error suggests either of these:
+ // 1) Call back on unsupported netvc -- Don't register callback unnecessarily
+ // 2) Call back on stale netvc
+ Debug("ssl.error", "ssl_servername_callback was called unexpectedly");
+ return SSL_TLSEXT_ERR_ALERT_FATAL;
}
+
return SSL_TLSEXT_ERR_OK;
}
@@ -912,6 +861,7 @@ SSLInitializeLibrary()
ssl_vc_index = SSL_get_ex_new_index(0, (void *)"NetVC index", nullptr, nullptr, nullptr);
TLSSessionResumptionSupport::initialize();
+ TLSSNISupport::initialize();
open_ssl_initialized = true;
}
@@ -1125,13 +1075,6 @@ SSLMultiCertConfigLoader::_set_handshake_callbacks(SSL_CTX *ctx)
}
void
-setTLSValidProtocols(SSL *ssl, unsigned long proto_mask, unsigned long max_mask)
-{
- SSL_set_options(ssl, proto_mask);
- SSL_clear_options(ssl, max_mask & ~proto_mask);
-}
-
-void
setClientCertLevel(SSL *ssl, uint8_t certLevel)
{
SSLConfig::scoped_config params;
diff --git a/iocore/net/TLSSNISupport.cc b/iocore/net/TLSSNISupport.cc
new file mode 100644
index 0000000..a7b883f
--- /dev/null
+++ b/iocore/net/TLSSNISupport.cc
@@ -0,0 +1,145 @@
+/** @file
+
+ SNISupport.cc provides implmentations for SNISupport methods
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+#include "TLSSNISupport.h"
+#include "tscore/ink_assert.h"
+#include "tscore/Diags.h"
+#include "P_SSLSNI.h"
+
+int TLSSNISupport::_ex_data_index = -1;
+
+void
+TLSSNISupport::initialize()
+{
+ ink_assert(_ex_data_index == -1);
+ if (_ex_data_index == -1) {
+ _ex_data_index = SSL_get_ex_new_index(0, (void *)"TLSSNISupport index", nullptr, nullptr, nullptr);
+ }
+}
+
+TLSSNISupport *
+TLSSNISupport::getInstance(SSL *ssl)
+{
+ return static_cast<TLSSNISupport *>(SSL_get_ex_data(ssl, _ex_data_index));
+}
+
+void
+TLSSNISupport::bind(SSL *ssl, TLSSNISupport *snis)
+{
+ SSL_set_ex_data(ssl, _ex_data_index, snis);
+}
+
+void
+TLSSNISupport::unbind(SSL *ssl)
+{
+ SSL_set_ex_data(ssl, _ex_data_index, nullptr);
+}
+
+int
+TLSSNISupport::perform_sni_action()
+{
+ const char *servername = this->_get_sni_server_name();
+ SNIConfig::scoped_config params;
+ if (const auto &actions = params->get(servername); !actions.first) {
+ Debug("ssl_sni", "%s not available in the map", servername);
+ } else {
+ for (auto &&item : *actions.first) {
+ auto ret = item->SNIAction(this, actions.second);
+ if (ret != SSL_TLSEXT_ERR_OK) {
+ return ret;
+ }
+ }
+ }
+ return SSL_TLSEXT_ERR_OK;
+}
+
+#if TS_USE_HELLO_CB
+void
+TLSSNISupport::on_client_hello(SSL *ssl, int *al, void *arg)
+{
+ const char *servername = nullptr;
+ const unsigned char *p;
+ size_t remaining, len;
+ // Parse the server name if the get extension call succeeds and there are more than 2 bytes to parse
+ if (SSL_client_hello_get0_ext(ssl, TLSEXT_TYPE_server_name, &p, &remaining) && remaining > 2) {
+ // Parse to get to the name, originally from test/handshake_helper.c in openssl tree
+ /* Extract the length of the supplied list of names. */
+ len = *(p++) << 8;
+ len += *(p++);
+ if (len + 2 == remaining) {
+ remaining = len;
+ /*
+ * The list in practice only has a single element, so we only consider
+ * the first one.
+ */
+ if (*p++ == TLSEXT_NAMETYPE_host_name) {
+ remaining--;
+ /* Now we can finally pull out the byte array with the actual hostname. */
+ if (remaining > 2) {
+ len = *(p++) << 8;
+ len += *(p++);
+ if (len + 2 <= remaining) {
+ servername = reinterpret_cast<const char *>(p);
+ }
+ }
+ }
+ }
+ }
+ if (servername) {
+ this->_set_sni_server_name(std::string_view(servername, len));
+ }
+}
+#endif
+
+void
+TLSSNISupport::on_servername(SSL *ssl, int *al, void *arg)
+{
+ this->_fire_ssl_servername_event();
+
+ const char *name = SSL_get_servername(ssl, TLSEXT_NAMETYPE_host_name);
+ if (name) {
+ this->_set_sni_server_name(name);
+ }
+}
+
+void
+TLSSNISupport::_clear()
+{
+ _sni_server_name.reset();
+}
+
+const char *
+TLSSNISupport::_get_sni_server_name() const
+{
+ return _sni_server_name.get() ? _sni_server_name.get() : "";
+}
+
+void
+TLSSNISupport::_set_sni_server_name(std::string_view name)
+{
+ if (name.size()) {
+ char *n = new char[name.size() + 1];
+ std::memcpy(n, name.data(), name.size());
+ n[name.size()] = '\0';
+ _sni_server_name.reset(n);
+ }
+}
diff --git a/iocore/net/TLSSNISupport.h b/iocore/net/TLSSNISupport.h
new file mode 100644
index 0000000..8257202
--- /dev/null
+++ b/iocore/net/TLSSNISupport.h
@@ -0,0 +1,61 @@
+/** @file
+
+ TLSSNISupport implements common methods and members to
+ support protocols for Server Name Indication
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+#pragma once
+
+#include <string_view>
+#include <memory>
+#include <openssl/ssl.h>
+#include "tscore/ink_config.h"
+
+class TLSSNISupport
+{
+public:
+ virtual ~TLSSNISupport() = default;
+
+ static void initialize();
+ static TLSSNISupport *getInstance(SSL *ssl);
+ static void bind(SSL *ssl, TLSSNISupport *snis);
+ static void unbind(SSL *ssl);
+
+ int perform_sni_action();
+ // Callback functions for OpenSSL libraries
+#if TS_USE_HELLO_CB
+ void on_client_hello(SSL *ssl, int *al, void *arg);
+#endif
+ void on_servername(SSL *ssl, int *al, void *arg);
+
+protected:
+ virtual void _fire_ssl_servername_event() = 0;
+
+ void _clear();
+ const char *_get_sni_server_name() const;
+
+private:
+ static int _ex_data_index;
+
+ // Null-terminated string, or nullptr if there is no SNI server name.
+ std::unique_ptr<char[]> _sni_server_name;
+
+ void _set_sni_server_name(std::string_view name);
+};
[trafficserver] 06/20: CacheRead: clear dir entry if doc is found
to be truncated (#7064)
Posted by zw...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 9.1.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
commit f922261b934c5766a60c0cddc330bbec30673dfc
Author: Brian Olsen <bn...@gmail.com>
AuthorDate: Fri Jan 29 06:17:40 2021 -0700
CacheRead: clear dir entry if doc is found to be truncated (#7064)
---
iocore/cache/CacheRead.cc | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/iocore/cache/CacheRead.cc b/iocore/cache/CacheRead.cc
index 6c89987..7cf8064 100644
--- a/iocore/cache/CacheRead.cc
+++ b/iocore/cache/CacheRead.cc
@@ -563,7 +563,6 @@ CacheVC::openReadReadDone(int event, Event *e)
VC_SCHED_LOCK_RETRY();
}
if (event == AIO_EVENT_DONE && !io.ok()) {
- dir_delete(&earliest_key, vol, &earliest_dir);
goto Lerror;
}
if (last_collision && // no missed lock
@@ -621,10 +620,11 @@ Lerror : {
if (request.valid()) {
int url_length;
const char *url_text = request.url_get()->string_get_ref(&url_length);
- Warning("Document %s truncated, url[%.*s]", earliest_key.toHexStr(tmpstring), url_length, url_text);
+ Warning("Document %s truncated, url[%.*s] .. clearing", earliest_key.toHexStr(tmpstring), url_length, url_text);
} else {
- Warning("Document %s truncated", earliest_key.toHexStr(tmpstring));
+ Warning("Document %s truncated .. clearing", earliest_key.toHexStr(tmpstring));
}
+ dir_delete(&earliest_key, vol, &earliest_dir);
return calluser(VC_EVENT_ERROR);
}
Ldone:
[trafficserver] 05/20: Do not provide a stale negative cache (#7422)
Posted by zw...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 9.1.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
commit c1c873405a8f426fc763d16a5fcd407c2d4be63c
Author: Kazuhiko <ka...@fdiary.net>
AuthorDate: Fri Jan 29 06:08:02 2021 +0100
Do not provide a stale negative cache (#7422)
---
proxy/http/HttpTransact.cc | 98 ++++++++++++++++++++++++----------------------
1 file changed, 51 insertions(+), 47 deletions(-)
diff --git a/proxy/http/HttpTransact.cc b/proxy/http/HttpTransact.cc
index 3f1f852..9b9e9ce 100644
--- a/proxy/http/HttpTransact.cc
+++ b/proxy/http/HttpTransact.cc
@@ -4347,61 +4347,65 @@ HttpTransact::handle_cache_operation_on_forward_server_response(State *s)
server_response_code == HTTP_STATUS_BAD_GATEWAY || server_response_code == HTTP_STATUS_SERVICE_UNAVAILABLE) &&
s->cache_info.action == CACHE_DO_UPDATE && s->txn_conf->negative_revalidating_enabled &&
is_stale_cache_response_returnable(s)) {
- TxnDebug("http_trans", "[hcoofsr] negative revalidating: revalidate stale object and serve from cache");
-
- s->cache_info.object_store.create();
- s->cache_info.object_store.request_set(&s->hdr_info.client_request);
- s->cache_info.object_store.response_set(s->cache_info.object_read->response_get());
- base_response = s->cache_info.object_store.response_get();
- time_t exp_time = s->txn_conf->negative_revalidating_lifetime + ink_local_time();
- base_response->set_expires(exp_time);
-
- SET_VIA_STRING(VIA_CACHE_FILL_ACTION, VIA_CACHE_UPDATED);
- HTTP_INCREMENT_DYN_STAT(http_cache_updates_stat);
-
- // unset Cache-control: "need-revalidate-once" (if it's set)
- // This directive is used internally by T.S. to invalidate
- // documents so that an invalidated document needs to be
- // revalidated again.
- base_response->unset_cooked_cc_need_revalidate_once();
-
- if (is_request_conditional(&s->hdr_info.client_request) &&
- HttpTransactCache::match_response_to_request_conditionals(&s->hdr_info.client_request,
- s->cache_info.object_read->response_get(),
- s->response_received_time) == HTTP_STATUS_NOT_MODIFIED) {
- s->next_action = SM_ACTION_INTERNAL_CACHE_UPDATE_HEADERS;
- client_response_code = HTTP_STATUS_NOT_MODIFIED;
- } else {
- if (s->method == HTTP_WKSIDX_HEAD) {
- s->cache_info.action = CACHE_DO_UPDATE;
- s->next_action = SM_ACTION_INTERNAL_CACHE_NOOP;
+ HTTPStatus cached_response_code = s->cache_info.object_read->response_get()->status_get();
+ if (!(cached_response_code == HTTP_STATUS_INTERNAL_SERVER_ERROR || cached_response_code == HTTP_STATUS_GATEWAY_TIMEOUT ||
+ cached_response_code == HTTP_STATUS_BAD_GATEWAY || cached_response_code == HTTP_STATUS_SERVICE_UNAVAILABLE)) {
+ TxnDebug("http_trans", "[hcoofsr] negative revalidating: revalidate stale object and serve from cache");
+
+ s->cache_info.object_store.create();
+ s->cache_info.object_store.request_set(&s->hdr_info.client_request);
+ s->cache_info.object_store.response_set(s->cache_info.object_read->response_get());
+ base_response = s->cache_info.object_store.response_get();
+ time_t exp_time = s->txn_conf->negative_revalidating_lifetime + ink_local_time();
+ base_response->set_expires(exp_time);
+
+ SET_VIA_STRING(VIA_CACHE_FILL_ACTION, VIA_CACHE_UPDATED);
+ HTTP_INCREMENT_DYN_STAT(http_cache_updates_stat);
+
+ // unset Cache-control: "need-revalidate-once" (if it's set)
+ // This directive is used internally by T.S. to invalidate
+ // documents so that an invalidated document needs to be
+ // revalidated again.
+ base_response->unset_cooked_cc_need_revalidate_once();
+
+ if (is_request_conditional(&s->hdr_info.client_request) &&
+ HttpTransactCache::match_response_to_request_conditionals(&s->hdr_info.client_request,
+ s->cache_info.object_read->response_get(),
+ s->response_received_time) == HTTP_STATUS_NOT_MODIFIED) {
+ s->next_action = SM_ACTION_INTERNAL_CACHE_UPDATE_HEADERS;
+ client_response_code = HTTP_STATUS_NOT_MODIFIED;
} else {
- s->cache_info.action = CACHE_DO_SERVE_AND_UPDATE;
- s->next_action = SM_ACTION_SERVE_FROM_CACHE;
+ if (s->method == HTTP_WKSIDX_HEAD) {
+ s->cache_info.action = CACHE_DO_UPDATE;
+ s->next_action = SM_ACTION_INTERNAL_CACHE_NOOP;
+ } else {
+ s->cache_info.action = CACHE_DO_SERVE_AND_UPDATE;
+ s->next_action = SM_ACTION_SERVE_FROM_CACHE;
+ }
+
+ client_response_code = s->cache_info.object_read->response_get()->status_get();
}
- client_response_code = s->cache_info.object_read->response_get()->status_get();
- }
+ ink_assert(base_response->valid());
- ink_assert(base_response->valid());
+ if (client_response_code == HTTP_STATUS_NOT_MODIFIED) {
+ ink_assert(GET_VIA_STRING(VIA_CLIENT_REQUEST) != VIA_CLIENT_SIMPLE);
+ SET_VIA_STRING(VIA_CLIENT_REQUEST, VIA_CLIENT_IMS);
+ SET_VIA_STRING(VIA_PROXY_RESULT, VIA_PROXY_NOT_MODIFIED);
+ } else {
+ SET_VIA_STRING(VIA_PROXY_RESULT, VIA_PROXY_SERVED);
+ }
- if (client_response_code == HTTP_STATUS_NOT_MODIFIED) {
- ink_assert(GET_VIA_STRING(VIA_CLIENT_REQUEST) != VIA_CLIENT_SIMPLE);
- SET_VIA_STRING(VIA_CLIENT_REQUEST, VIA_CLIENT_IMS);
- SET_VIA_STRING(VIA_PROXY_RESULT, VIA_PROXY_NOT_MODIFIED);
- } else {
- SET_VIA_STRING(VIA_PROXY_RESULT, VIA_PROXY_SERVED);
- }
+ ink_assert(client_response_code != HTTP_STATUS_NONE);
- ink_assert(client_response_code != HTTP_STATUS_NONE);
+ if (s->next_action == SM_ACTION_SERVE_FROM_CACHE && s->state_machine->do_transform_open()) {
+ set_header_for_transform(s, base_response);
+ } else {
+ build_response(s, base_response, &s->hdr_info.client_response, s->client_info.http_version, client_response_code);
+ }
- if (s->next_action == SM_ACTION_SERVE_FROM_CACHE && s->state_machine->do_transform_open()) {
- set_header_for_transform(s, base_response);
- } else {
- build_response(s, base_response, &s->hdr_info.client_response, s->client_info.http_version, client_response_code);
+ return;
}
-
- return;
}
s->next_action = SM_ACTION_SERVER_READ;
[trafficserver] 18/20: API to retrieve NoStore set by plugins
(#7439)
Posted by zw...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 9.1.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
commit 2926a69e034c5302b02199fe639081b11cd3966d
Author: Vijay Mamidi <vi...@yahoo.com>
AuthorDate: Tue Feb 2 14:48:42 2021 -0800
API to retrieve NoStore set by plugins (#7439)
---
include/ts/ts.h | 7 +++++++
src/traffic_server/InkAPI.cc | 9 +++++++++
2 files changed, 16 insertions(+)
diff --git a/include/ts/ts.h b/include/ts/ts.h
index 5fef637..4d3b4c2 100644
--- a/include/ts/ts.h
+++ b/include/ts/ts.h
@@ -1380,6 +1380,13 @@ tsapi void TSHttpTxnReqCacheableSet(TSHttpTxn txnp, int flag);
*/
tsapi TSReturnCode TSHttpTxnServerRespNoStoreSet(TSHttpTxn txnp, int flag);
+/** Get flag indicating whether or not to cache the server response for
+ given TSHttpTxn
+ @param txnp The transaction whose server response you do not want to store.
+
+ @return TS_SUCCESS.
+*/
+tsapi bool TSHttpTxnServerRespNoStoreGet(TSHttpTxn txnp);
tsapi TSReturnCode TSFetchPageRespGet(TSHttpTxn txnp, TSMBuffer *bufp, TSMLoc *offset);
tsapi char *TSFetchRespGet(TSHttpTxn txnp, int *length);
tsapi TSReturnCode TSHttpTxnCacheLookupStatusGet(TSHttpTxn txnp, int *lookup_status);
diff --git a/src/traffic_server/InkAPI.cc b/src/traffic_server/InkAPI.cc
index 0d6c20c..50e2fdd 100644
--- a/src/traffic_server/InkAPI.cc
+++ b/src/traffic_server/InkAPI.cc
@@ -5659,6 +5659,15 @@ TSHttpTxnServerRespNoStoreSet(TSHttpTxn txnp, int flag)
return TS_SUCCESS;
}
+bool
+TSHttpTxnServerRespNoStoreGet(TSHttpTxn txnp)
+{
+ sdk_assert(sdk_sanity_check_txn(txnp) == TS_SUCCESS);
+
+ HttpTransact::State *s = &(((HttpSM *)txnp)->t_state);
+ return s->api_server_response_no_store;
+}
+
TSReturnCode
TSHttpTxnServerRespIgnore(TSHttpTxn txnp)
{
[trafficserver] 15/20: Cleanup: Get rid of MIMEFieldWrapper from
HPACK encoding (#6520)
Posted by zw...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 9.1.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
commit 1ae48be5f5970d6af2df944f8c3200c354a05715
Author: Masaori Koshiba <ma...@apache.org>
AuthorDate: Tue Feb 2 13:17:58 2021 +0900
Cleanup: Get rid of MIMEFieldWrapper from HPACK encoding (#6520)
---
proxy/http2/HPACK.cc | 256 ++++++++++------------
proxy/http2/HPACK.h | 20 +-
proxy/http2/unit_tests/test_HpackIndexingTable.cc | 17 +-
3 files changed, 135 insertions(+), 158 deletions(-)
diff --git a/proxy/http2/HPACK.cc b/proxy/http2/HPACK.cc
index df8d212..81ed795 100644
--- a/proxy/http2/HPACK.cc
+++ b/proxy/http2/HPACK.cc
@@ -24,6 +24,7 @@
#include "HPACK.h"
#include "tscpp/util/LocalBuffer.h"
+#include "tscpp/util/TextView.h"
namespace
{
@@ -98,76 +99,71 @@ typedef enum {
TS_HPACK_STATIC_TABLE_ENTRY_NUM
} TS_HPACK_STATIC_TABLE_ENTRY;
-struct StaticTable {
- StaticTable(const char *n, const char *v) : name(n), value(v), name_size(strlen(name)), value_size(strlen(value)) {}
- const char *name;
- const char *value;
- const int name_size;
- const int value_size;
-};
-
-static const StaticTable STATIC_TABLE[] = {{"", ""},
- {":authority", ""},
- {":method", "GET"},
- {":method", "POST"},
- {":path", "/"},
- {":path", "/index.html"},
- {":scheme", "http"},
- {":scheme", "https"},
- {":status", "200"},
- {":status", "204"},
- {":status", "206"},
- {":status", "304"},
- {":status", "400"},
- {":status", "404"},
- {":status", "500"},
- {"accept-charset", ""},
- {"accept-encoding", "gzip, deflate"},
- {"accept-language", ""},
- {"accept-ranges", ""},
- {"accept", ""},
- {"access-control-allow-origin", ""},
- {"age", ""},
- {"allow", ""},
- {"authorization", ""},
- {"cache-control", ""},
- {"content-disposition", ""},
- {"content-encoding", ""},
- {"content-language", ""},
- {"content-length", ""},
- {"content-location", ""},
- {"content-range", ""},
- {"content-type", ""},
- {"cookie", ""},
- {"date", ""},
- {"etag", ""},
- {"expect", ""},
- {"expires", ""},
- {"from", ""},
- {"host", ""},
- {"if-match", ""},
- {"if-modified-since", ""},
- {"if-none-match", ""},
- {"if-range", ""},
- {"if-unmodified-since", ""},
- {"last-modified", ""},
- {"link", ""},
- {"location", ""},
- {"max-forwards", ""},
- {"proxy-authenticate", ""},
- {"proxy-authorization", ""},
- {"range", ""},
- {"referer", ""},
- {"refresh", ""},
- {"retry-after", ""},
- {"server", ""},
- {"set-cookie", ""},
- {"strict-transport-security", ""},
- {"transfer-encoding", ""},
- {"user-agent", ""},
- {"vary", ""},
- {"via", ""},
- {"www-authenticate", ""}};
+constexpr HpackHeaderField STATIC_TABLE[] = {{"", ""},
+ {":authority", ""},
+ {":method", "GET"},
+ {":method", "POST"},
+ {":path", "/"},
+ {":path", "/index.html"},
+ {":scheme", "http"},
+ {":scheme", "https"},
+ {":status", "200"},
+ {":status", "204"},
+ {":status", "206"},
+ {":status", "304"},
+ {":status", "400"},
+ {":status", "404"},
+ {":status", "500"},
+ {"accept-charset", ""},
+ {"accept-encoding", "gzip, deflate"},
+ {"accept-language", ""},
+ {"accept-ranges", ""},
+ {"accept", ""},
+ {"access-control-allow-origin", ""},
+ {"age", ""},
+ {"allow", ""},
+ {"authorization", ""},
+ {"cache-control", ""},
+ {"content-disposition", ""},
+ {"content-encoding", ""},
+ {"content-language", ""},
+ {"content-length", ""},
+ {"content-location", ""},
+ {"content-range", ""},
+ {"content-type", ""},
+ {"cookie", ""},
+ {"date", ""},
+ {"etag", ""},
+ {"expect", ""},
+ {"expires", ""},
+ {"from", ""},
+ {"host", ""},
+ {"if-match", ""},
+ {"if-modified-since", ""},
+ {"if-none-match", ""},
+ {"if-range", ""},
+ {"if-unmodified-since", ""},
+ {"last-modified", ""},
+ {"link", ""},
+ {"location", ""},
+ {"max-forwards", ""},
+ {"proxy-authenticate", ""},
+ {"proxy-authorization", ""},
+ {"range", ""},
+ {"referer", ""},
+ {"refresh", ""},
+ {"retry-after", ""},
+ {"server", ""},
+ {"set-cookie", ""},
+ {"strict-transport-security", ""},
+ {"transfer-encoding", ""},
+ {"user-agent", ""},
+ {"vary", ""},
+ {"via", ""},
+ {"www-authenticate", ""}};
+
+constexpr std::string_view HPACK_HDR_FIELD_COOKIE = STATIC_TABLE[TS_HPACK_STATIC_TABLE_COOKIE].name;
+constexpr std::string_view HPACK_HDR_FIELD_AUTHORIZATION = STATIC_TABLE[TS_HPACK_STATIC_TABLE_AUTHORIZATION].name;
/**
Threshold for total HdrHeap size which used by HPAK Dynamic Table.
@@ -227,19 +223,18 @@ hpack_parse_field_type(uint8_t ftype)
namespace HpackStaticTable
{
HpackLookupResult
- lookup(const char *name, int name_len, const char *value, int value_len)
+ lookup(const HpackHeaderField &header)
{
HpackLookupResult result;
for (unsigned int index = 1; index < TS_HPACK_STATIC_TABLE_ENTRY_NUM; ++index) {
- const char *table_name = STATIC_TABLE[index].name;
- int table_name_len = STATIC_TABLE[index].name_size;
- const char *table_value = STATIC_TABLE[index].value;
- int table_value_len = STATIC_TABLE[index].value_size;
+ std::string_view name = STATIC_TABLE[index].name;
+ std::string_view value = STATIC_TABLE[index].value;
+ // TODO: replace `strcasecmp` with `memcmp`
// Check whether name (and value) are matched
- if (ptr_len_casecmp(name, name_len, table_name, table_name_len) == 0) {
- if ((value_len == table_value_len) && (memcmp(value, table_value, value_len) == 0)) {
+ if (strcasecmp(header.name, name) == 0) {
+ if (memcmp(header.value, value) == 0) {
result.index = index;
result.index_type = HpackIndex::STATIC;
result.match_type = HpackMatch::EXACT;
@@ -261,19 +256,10 @@ namespace HpackStaticTable
// HpackIndexingTable
//
HpackLookupResult
-HpackIndexingTable::lookup(const MIMEFieldWrapper &field) const
-{
- int target_name_len = 0, target_value_len = 0;
- const char *target_name = field.name_get(&target_name_len);
- const char *target_value = field.value_get(&target_value_len);
- return lookup(target_name, target_name_len, target_value, target_value_len);
-}
-
-HpackLookupResult
-HpackIndexingTable::lookup(const char *name, int name_len, const char *value, int value_len) const
+HpackIndexingTable::lookup(const HpackHeaderField &header) const
{
// static table
- HpackLookupResult result = HpackStaticTable::lookup(name, name_len, value, value_len);
+ HpackLookupResult result = HpackStaticTable::lookup(header);
// if result is not EXACT match, lookup dynamic table
if (result.match_type == HpackMatch::EXACT) {
@@ -281,8 +267,7 @@ HpackIndexingTable::lookup(const char *name, int name_len, const char *value, in
}
// dynamic table
- if (HpackLookupResult dt_result = this->_dynamic_table.lookup(name, name_len, value, value_len);
- dt_result.match_type == HpackMatch::EXACT) {
+ if (HpackLookupResult dt_result = this->_dynamic_table.lookup(header); dt_result.match_type == HpackMatch::EXACT) {
return dt_result;
}
@@ -299,8 +284,8 @@ HpackIndexingTable::get_header_field(uint32_t index, MIMEFieldWrapper &field) co
if (index < TS_HPACK_STATIC_TABLE_ENTRY_NUM) {
// static table
- field.name_set(STATIC_TABLE[index].name, STATIC_TABLE[index].name_size);
- field.value_set(STATIC_TABLE[index].value, STATIC_TABLE[index].value_size);
+ field.name_set(STATIC_TABLE[index].name.data(), STATIC_TABLE[index].name.size());
+ field.value_set(STATIC_TABLE[index].value.data(), STATIC_TABLE[index].value.size());
} else if (index < TS_HPACK_STATIC_TABLE_ENTRY_NUM + _dynamic_table.length()) {
// dynamic table
const MIMEField *m_field = _dynamic_table.get_header_field(index - TS_HPACK_STATIC_TABLE_ENTRY_NUM);
@@ -322,9 +307,9 @@ HpackIndexingTable::get_header_field(uint32_t index, MIMEFieldWrapper &field) co
}
void
-HpackIndexingTable::add_header_field(const MIMEField *field)
+HpackIndexingTable::add_header_field(const HpackHeaderField &header)
{
- _dynamic_table.add_header_field(field);
+ _dynamic_table.add_header_field(header);
}
uint32_t
@@ -376,12 +361,9 @@ HpackDynamicTable::get_header_field(uint32_t index) const
}
void
-HpackDynamicTable::add_header_field(const MIMEField *field)
+HpackDynamicTable::add_header_field(const HpackHeaderField &header)
{
- int name_len, value_len;
- const char *name = field->name_get(&name_len);
- const char *value = field->value_get(&value_len);
- uint32_t header_size = ADDITIONAL_OCTETS + name_len + value_len;
+ uint32_t header_size = ADDITIONAL_OCTETS + header.name.size() + header.value.size();
if (header_size > _maximum_size) {
// [RFC 7541] 4.4. Entry Eviction When Adding New Entries
@@ -403,31 +385,28 @@ HpackDynamicTable::add_header_field(const MIMEField *field)
this->_current_size += header_size;
this->_evict_overflowed_entries();
- MIMEField *new_field = this->_mhdr->field_create(name, name_len);
- new_field->value_set(this->_mhdr->m_heap, this->_mhdr->m_mime, value, value_len);
+ MIMEField *new_field = this->_mhdr->field_create(header.name.data(), header.name.size());
+ new_field->value_set(this->_mhdr->m_heap, this->_mhdr->m_mime, header.value.data(), header.value.size());
this->_mhdr->field_attach(new_field);
this->_headers.push_front(new_field);
}
}
HpackLookupResult
-HpackDynamicTable::lookup(const char *name, int name_len, const char *value, int value_len) const
+HpackDynamicTable::lookup(const HpackHeaderField &header) const
{
HpackLookupResult result;
const unsigned int entry_num = TS_HPACK_STATIC_TABLE_ENTRY_NUM + this->length();
for (unsigned int index = TS_HPACK_STATIC_TABLE_ENTRY_NUM; index < entry_num; ++index) {
- const MIMEField *m_field = this->get_header_field(index - TS_HPACK_STATIC_TABLE_ENTRY_NUM);
-
- int table_name_len = 0;
- const char *table_name = m_field->name_get(&table_name_len);
-
- int table_value_len = 0;
- const char *table_value = m_field->value_get(&table_value_len);
+ const MIMEField *m_field = this->_headers.at(index - TS_HPACK_STATIC_TABLE_ENTRY_NUM);
+ std::string_view name = m_field->name_get();
+ std::string_view value = m_field->value_get();
+ // TODO: replace `strcasecmp` with `memcmp`
// Check whether name (and value) are matched
- if (ptr_len_casecmp(name, name_len, table_name, table_name_len) == 0) {
- if ((value_len == table_value_len) && (memcmp(value, table_value, value_len) == 0)) {
+ if (strcasecmp(header.name, name) == 0) {
+ if (memcmp(header.value, value) == 0) {
result.index = index;
result.index_type = HpackIndex::DYNAMIC;
result.match_type = HpackMatch::EXACT;
@@ -559,7 +538,7 @@ encode_indexed_header_field(uint8_t *buf_start, const uint8_t *buf_end, uint32_t
}
int64_t
-encode_literal_header_field_with_indexed_name(uint8_t *buf_start, const uint8_t *buf_end, const MIMEFieldWrapper &header,
+encode_literal_header_field_with_indexed_name(uint8_t *buf_start, const uint8_t *buf_end, const HpackHeaderField &header,
uint32_t index, HpackIndexingTable &indexing_table, HpackField type)
{
uint8_t *p = buf_start;
@@ -570,7 +549,7 @@ encode_literal_header_field_with_indexed_name(uint8_t *buf_start, const uint8_t
switch (type) {
case HpackField::INDEXED_LITERAL:
- indexing_table.add_header_field(header.field_get());
+ indexing_table.add_header_field(header);
prefix = 6;
flag = 0x40;
break;
@@ -601,20 +580,18 @@ encode_literal_header_field_with_indexed_name(uint8_t *buf_start, const uint8_t
p += len;
// Value String
- int value_len;
- const char *value = header.value_get(&value_len);
- len = xpack_encode_string(p, buf_end, value, value_len);
+ len = xpack_encode_string(p, buf_end, header.value.data(), header.value.size());
if (len == -1) {
return -1;
}
p += len;
- Debug("hpack_encode", "Encoded field: %d: %.*s", index, value_len, value);
+ Debug("hpack_encode", "Encoded field: %d: %.*s", index, static_cast<int>(header.value.size()), header.value.data());
return p - buf_start;
}
int64_t
-encode_literal_header_field_with_new_name(uint8_t *buf_start, const uint8_t *buf_end, const MIMEFieldWrapper &header,
+encode_literal_header_field_with_new_name(uint8_t *buf_start, const uint8_t *buf_end, const HpackHeaderField &header,
HpackIndexingTable &indexing_table, HpackField type)
{
uint8_t *p = buf_start;
@@ -625,7 +602,7 @@ encode_literal_header_field_with_new_name(uint8_t *buf_start, const uint8_t *buf
switch (type) {
case HpackField::INDEXED_LITERAL:
- indexing_table.add_header_field(header.field_get());
+ indexing_table.add_header_field(header);
flag = 0x40;
break;
case HpackField::NOINDEX_LITERAL:
@@ -644,12 +621,11 @@ encode_literal_header_field_with_new_name(uint8_t *buf_start, const uint8_t *buf
// Convert field name to lower case to follow HTTP2 spec.
// This conversion is needed because WKSs in MIMEFields is old fashioned
- int name_len;
- const char *original_name = header.name_get(&name_len);
+ int name_len = header.name.size();
+ const char *original_name = header.name.data();
ts::LocalBuffer<char> local_buffer(name_len);
char *lower_name = local_buffer.data();
-
for (int i = 0; i < name_len; i++) {
lower_name[i] = ParseRules::ink_tolower(original_name[i]);
}
@@ -662,16 +638,16 @@ encode_literal_header_field_with_new_name(uint8_t *buf_start, const uint8_t *buf
p += len;
// Value String
- int value_len;
- const char *value = header.value_get(&value_len);
- len = xpack_encode_string(p, buf_end, value, value_len);
+ len = xpack_encode_string(p, buf_end, header.value.data(), header.value.size());
if (len == -1) {
return -1;
}
p += len;
- Debug("hpack_encode", "Encoded field: %.*s: %.*s", name_len, lower_name, value_len, value);
+ Debug("hpack_encode", "Encoded field: %.*s: %.*s", name_len, lower_name, static_cast<int>(header.value.size()),
+ header.value.data());
+
return p - buf_start;
}
@@ -794,7 +770,8 @@ decode_literal_header_field(MIMEFieldWrapper &header, const uint8_t *buf_start,
// Incremental Indexing adds header to header table as new entry
if (isIncremental) {
- indexing_table.add_header_field(header.field_get());
+ const MIMEField *field = header.field_get();
+ indexing_table.add_header_field({field->name_get(), field->value_get()});
}
// Print decoded header field
@@ -930,14 +907,13 @@ hpack_encode_header_block(HpackIndexingTable &indexing_table, uint8_t *out_buf,
{
uint8_t *cursor = out_buf;
const uint8_t *const out_buf_end = out_buf + out_buf_len;
- int64_t written = 0;
ink_assert(http_hdr_type_get(hdr->m_http) != HTTP_TYPE_UNKNOWN);
// Update dynamic table size
if (maximum_table_size >= 0) {
indexing_table.update_maximum_size(maximum_table_size);
- written = encode_dynamic_table_size_update(cursor, out_buf_end, maximum_table_size);
+ int64_t written = encode_dynamic_table_size_update(cursor, out_buf_end, maximum_table_size);
if (written == HPACK_ERROR_COMPRESSION_ERROR) {
return HPACK_ERROR_COMPRESSION_ERROR;
}
@@ -946,22 +922,25 @@ hpack_encode_header_block(HpackIndexingTable &indexing_table, uint8_t *out_buf,
MIMEFieldIter field_iter;
for (MIMEField *field = hdr->iter_get_first(&field_iter); field != nullptr; field = hdr->iter_get_next(&field_iter)) {
- HpackField field_type;
- MIMEFieldWrapper header(field, hdr->m_heap, hdr->m_http->m_fields_impl);
- int name_len;
- int value_len;
- const char *name = header.name_get(&name_len);
- header.value_get(&value_len);
+ std::string_view name = field->name_get();
+ std::string_view value = field->value_get();
+
// Choose field representation (See RFC7541 7.1.3)
// - Authorization header obviously should not be indexed
// - Short Cookie header should not be indexed because of low entropy
- if ((ptr_len_casecmp(name, name_len, MIME_FIELD_COOKIE, MIME_LEN_COOKIE) == 0 && value_len < 20) ||
- (ptr_len_casecmp(name, name_len, MIME_FIELD_AUTHORIZATION, MIME_LEN_AUTHORIZATION) == 0)) {
+ HpackField field_type;
+ // TODO: replace `strcasecmp` with `memcmp`
+ if ((value.size() < 20 && strcasecmp(name, HPACK_HDR_FIELD_COOKIE) == 0) ||
+ (strcasecmp(name, HPACK_HDR_FIELD_AUTHORIZATION) == 0)) {
field_type = HpackField::NEVERINDEX_LITERAL;
} else {
field_type = HpackField::INDEXED_LITERAL;
}
+
+ HpackHeaderField header{name, value};
const HpackLookupResult result = indexing_table.lookup(header);
+
+ int64_t written = 0;
switch (result.match_type) {
case HpackMatch::NONE:
written = encode_literal_header_field_with_new_name(cursor, out_buf_end, header, indexing_table, field_type);
@@ -974,10 +953,9 @@ hpack_encode_header_block(HpackIndexingTable &indexing_table, uint8_t *out_buf,
written = encode_indexed_header_field(cursor, out_buf_end, result.index);
break;
default:
- // Does it happen?
- written = 0;
break;
}
+
if (written == HPACK_ERROR_COMPRESSION_ERROR) {
return HPACK_ERROR_COMPRESSION_ERROR;
}
diff --git a/proxy/http2/HPACK.h b/proxy/http2/HPACK.h
index ffe67eb..8c538ef 100644
--- a/proxy/http2/HPACK.h
+++ b/proxy/http2/HPACK.h
@@ -29,6 +29,7 @@
#include "../hdrs/XPACK.h"
#include <deque>
+#include <string_view>
// It means that any header field can be compressed/decompressed by ATS
const static int HPACK_ERROR_COMPRESSION_ERROR = -1;
@@ -61,6 +62,11 @@ struct HpackLookupResult {
HpackMatch match_type = HpackMatch::NONE;
};
+struct HpackHeaderField {
+ std::string_view name;
+ std::string_view value;
+};
+
class MIMEFieldWrapper
{
public:
@@ -113,9 +119,9 @@ public:
HpackDynamicTable &operator=(const HpackDynamicTable &) = delete;
const MIMEField *get_header_field(uint32_t index) const;
- void add_header_field(const MIMEField *field);
+ void add_header_field(const HpackHeaderField &header);
- HpackLookupResult lookup(const char *name, int name_len, const char *value, int value_len) const;
+ HpackLookupResult lookup(const HpackHeaderField &header) const;
uint32_t maximum_size() const;
uint32_t size() const;
void update_maximum_size(uint32_t new_size);
@@ -145,11 +151,10 @@ public:
HpackIndexingTable(HpackIndexingTable &) = delete;
HpackIndexingTable &operator=(const HpackIndexingTable &) = delete;
- HpackLookupResult lookup(const MIMEFieldWrapper &field) const;
- HpackLookupResult lookup(const char *name, int name_len, const char *value, int value_len) const;
+ HpackLookupResult lookup(const HpackHeaderField &header) const;
int get_header_field(uint32_t index, MIMEFieldWrapper &header_field) const;
- void add_header_field(const MIMEField *field);
+ void add_header_field(const HpackHeaderField &header);
uint32_t maximum_size() const;
uint32_t size() const;
void update_maximum_size(uint32_t new_size);
@@ -160,10 +165,11 @@ private:
// Low level interfaces
int64_t encode_indexed_header_field(uint8_t *buf_start, const uint8_t *buf_end, uint32_t index);
-int64_t encode_literal_header_field_with_indexed_name(uint8_t *buf_start, const uint8_t *buf_end, const MIMEFieldWrapper &header,
+int64_t encode_literal_header_field_with_indexed_name(uint8_t *buf_start, const uint8_t *buf_end, const HpackHeaderField &header,
uint32_t index, HpackIndexingTable &indexing_table, HpackField type);
-int64_t encode_literal_header_field_with_new_name(uint8_t *buf_start, const uint8_t *buf_end, const MIMEFieldWrapper &header,
+int64_t encode_literal_header_field_with_new_name(uint8_t *buf_start, const uint8_t *buf_end, const HpackHeaderField &header,
HpackIndexingTable &indexing_table, HpackField type);
+
int64_t decode_indexed_header_field(MIMEFieldWrapper &header, const uint8_t *buf_start, const uint8_t *buf_end,
HpackIndexingTable &indexing_table);
int64_t decode_literal_header_field(MIMEFieldWrapper &header, const uint8_t *buf_start, const uint8_t *buf_end,
diff --git a/proxy/http2/unit_tests/test_HpackIndexingTable.cc b/proxy/http2/unit_tests/test_HpackIndexingTable.cc
index ff470c7..d750015 100644
--- a/proxy/http2/unit_tests/test_HpackIndexingTable.cc
+++ b/proxy/http2/unit_tests/test_HpackIndexingTable.cc
@@ -198,19 +198,13 @@ TEST_CASE("HPACK low level APIs", "[hpack]")
for (unsigned int i = 9; i < sizeof(literal_test_case) / sizeof(literal_test_case[0]); i++) {
memset(buf, 0, BUFSIZE_FOR_REGRESSION_TEST);
- ats_scoped_obj<HTTPHdr> headers(new HTTPHdr);
- headers->create(HTTP_TYPE_RESPONSE);
- MIMEField *field = mime_field_create(headers->m_heap, headers->m_http->m_fields_impl);
- MIMEFieldWrapper header(field, headers->m_heap, headers->m_http->m_fields_impl);
+ HpackHeaderField header{literal_test_case[i].raw_name, literal_test_case[i].raw_value};
- header.name_set(literal_test_case[i].raw_name, strlen(literal_test_case[i].raw_name));
- header.value_set(literal_test_case[i].raw_value, strlen(literal_test_case[i].raw_value));
if (literal_test_case[i].index > 0) {
len =
encode_literal_header_field_with_indexed_name(buf, buf + BUFSIZE_FOR_REGRESSION_TEST, header,
literal_test_case[i].index, indexing_table, literal_test_case[i].type);
} else {
- header.name_set(literal_test_case[i].raw_name, strlen(literal_test_case[i].raw_name));
len = encode_literal_header_field_with_new_name(buf, buf + BUFSIZE_FOR_REGRESSION_TEST, header, indexing_table,
literal_test_case[i].type);
}
@@ -383,15 +377,14 @@ TEST_CASE("HPACK high level APIs", "[hpack]")
j++) {
const char *expected_name = dynamic_table_response_test_case[i][j].name;
const char *expected_value = dynamic_table_response_test_case[i][j].value;
- int expected_name_len = strlen(expected_name);
- int expected_value_len = strlen(expected_value);
- if (expected_name_len == 0) {
+ if (strlen(expected_name) == 0) {
break;
}
- HpackLookupResult lookupResult =
- indexing_table.lookup(expected_name, expected_name_len, expected_value, expected_value_len);
+ HpackHeaderField expected_header{expected_name, expected_value};
+ HpackLookupResult lookupResult = indexing_table.lookup(expected_header);
+
CHECK(lookupResult.match_type == HpackMatch::EXACT);
CHECK(lookupResult.index_type == HpackIndex::DYNAMIC);
[trafficserver] 02/20: Fix heap use after free in
DNSProcessor::getby() (#3871)
Posted by zw...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 9.1.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
commit 7249341703d20abe078551f7bf00c7a043bb492f
Author: Oknet <xu...@skyguard.com.cn>
AuthorDate: Fri Jan 29 07:05:38 2021 +0800
Fix heap use after free in DNSProcessor::getby() (#3871)
---
iocore/dns/DNS.cc | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/iocore/dns/DNS.cc b/iocore/dns/DNS.cc
index 4bd0550..95a913b 100644
--- a/iocore/dns/DNS.cc
+++ b/iocore/dns/DNS.cc
@@ -1225,7 +1225,7 @@ DNSEntry::mainEvent(int event, Event *e)
} else {
Debug("dns", "adding first to collapsing queue");
dnsH->entries.enqueue(this);
- write_dns(dnsH);
+ dnsProcessor.thread->schedule_imm(dnsH);
}
return EVENT_DONE;
}
[trafficserver] 16/20: Add command line utility to help convert
remap plugin usage to ATS9. (#7426)
Posted by zw...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 9.1.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
commit ee755b5adeb06437d4a0d03887a9bbe167fe4aa2
Author: Walt Karas <wk...@verizonmedia.com>
AuthorDate: Tue Feb 2 11:49:57 2021 -0600
Add command line utility to help convert remap plugin usage to ATS9. (#7426)
---
doc/appendices/command-line/cvtremappi.en.rst | 69 ++++
tools/cvtremappi | 575 ++++++++++++++++++++++++++
tools/insnew | 67 +++
3 files changed, 711 insertions(+)
diff --git a/doc/appendices/command-line/cvtremappi.en.rst b/doc/appendices/command-line/cvtremappi.en.rst
new file mode 100644
index 0000000..264d474
--- /dev/null
+++ b/doc/appendices/command-line/cvtremappi.en.rst
@@ -0,0 +1,69 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+.. include:: ../../common.defs
+
+.. _cvtremappi:
+
+cvtremappi
+**********
+
+Description
+===========
+
+To help convert your remapping configuration from pre-ATS9 to ATS9 and later. It may be useful if you use any
+of the core plugins regex_remap.so, header_rewrite.so or gzip.so. (For this script to work, the python3
+command has to be in your path.) You can specify where your remap configuration file is with the option:
+
+--filepath FILEPATH
+
+If this parameter is omitted, it defaults to ``./remap.config`` . The script will make necessary modifications
+to this file, and any files it includes with ``.include`` . It will change `@plugin=gzip.so` to its new name,
+`@plugin=compress.so` . When regex_remap.so is invoked as the first remap plugin, it will add the parameter
+@pparam=pristine . (This makes it work the same as in pre-9 ATS, where the request URL is the pre-remapping
+URL for the first plugin for a remap rule.) When `header_rewrite.so` is used as a remap plugin, no changes
+are needed in the remap configuration line invoking it. However, changes may be necessary to the
+configuration files passed to it as parameters. If a header rewrite configuration file is used for both the
+invocation of header rewrite as the first plugin for remap rules, and for other invocations, it may be
+necessary to generate two new versions of it. In these cases, the prefix `1st-` is added to file's name,
+for the version used with header rewrite as the first plugin. If you prefer that a different prefix be added,
+you can specify it with this option:
+
+--prefix PREFIX
+
+If you are also using header rewrite as a global plugin, you should also provide the filepath of the global
+plugin configuration file with this option:
+
+--plugin PLUGIN
+
+(Note that, if the PLUGIN filepath is relative, it should be relative to the directory containing the remap
+configuration file, not relative to the directory the script is run from. Note also that, if relative paths
+for include files for header rewrite config files appear in the configuration files, they are assumed to be
+relative to the directory containing the remap configuration file.)
+
+Header rewrite previously had some logic that has been eliminated in ATS9. If a line in a header rewrite
+configuration file relies on this deprecated logic, an error message will be output to standard error. The
+text `ERROR:` will be prepended to the line in the configuration file causing the error.
+
+The script writes, one per line, a list of the files it is changing or creating to the standard output. But
+both new and changed files will be written into entirely new files with the suffix `.new` added to the filepath.
+For example, if `remap.config` is changed by the script, it will put the changed version of the file in
+`remap.config.new` . This gives you a chance to review the changes the script has made. You can then put the
+changed files into effect with the tool script `insnew`. This script reads a list of filepaths, one per line,
+from the standard input. For each filepath `FP`, if it specifies an existing file, it will rename it to
+`FP.old`. It will then rename the file `FP.new` to `FP`. This second script should be run from the same
+current directory as the first script was run from.
diff --git a/tools/cvtremappi b/tools/cvtremappi
new file mode 100755
index 0000000..290a3b5
--- /dev/null
+++ b/tools/cvtremappi
@@ -0,0 +1,575 @@
+#!/usr/bin/env python3
+
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Automatically convert remapping configuration for use of header_rewrite.so, regex_remap.so and gzip.so
+# plugins from pre-ATS9 to ATS9 and later. (See docs.trafficserver.apache.org, Command Line Utilities
+# Appendix.)
+
+import argparse
+
+BACKSLASH = "\\"
+
+parser = argparse.ArgumentParser(
+ prog="cvt7to9",
+ description= "Convert remap configuration from ATS7 to ATS9"
+)
+parser.add_argument("--filepath", default="remap.config", help="path specifier of remap config file")
+parser.add_argument("--prefix", default="1st-", help="prefix for new header_rewrite config files")
+parser.add_argument(
+ "--plugin", default=None, help="path specifier (relative to FILEPATH) of (global) plugins config file"
+)
+args = parser.parse_args()
+
+import sys
+
+def eprint(*args, **kwargs):
+ print(*args, file=sys.stderr, **kwargs)
+
+# Prefix to add to file relative pathspecs before opening file.
+#
+pathspec_prefix = None
+
+import os
+import copy
+
+# Remap or header rewrite config file.
+#
+class CfgFile:
+ def __init__(self, param):
+ if isinstance(param, CfgFile):
+ # Making copy of header rewrite config file to use when header rewrite is first plugin for remap
+ # rule.
+ #
+ self.changed = True
+ d = os.path.dirname(param.pathspec)
+ if (d != "") and (d != "/"):
+ d += "/"
+ self.pathspec = d + args.prefix + os.path.basename(param.pathspec)
+ self.lines = copy.copy(param.lines)
+
+ else:
+ # Opening existing config file, param is pathspec.
+ #
+ self.changed = False
+ self.pathspec = param
+ ps = param
+ if ps[:1] != "/":
+ ps = pathspec_prefix + ps
+ try:
+ fd = open(ps, "r")
+ except:
+ eprint(f"fatal error: failure opening {ps} for reading")
+ sys.exit(1)
+
+ try:
+ self.lines = fd.readlines()
+ except:
+ eprint(f"fatal error: failure reading {ps}")
+ sys.exit(1)
+
+ try:
+ fd.close()
+ except:
+ eprint(f"fatal error: failure closing {ps}")
+ sys.exit(1)
+
+ # Write out new version of file if it's contents are new. ".new" is appended for the pathspec to get the
+ # pathspec of the new file.
+ #
+ def close(self):
+ if self.changed:
+ ps = self.pathspec
+ if ps[:1] != "/":
+ ps = pathspec_prefix + ps
+ print(ps)
+ ps += ".new"
+ try:
+ fd = open(ps, "w")
+ except:
+ eprint(f"fatal error: failure opening {ps} for writing")
+ sys.exit(1)
+
+ for ln in self.lines:
+ if ln != None:
+ try:
+ fd.write(ln)
+ except:
+ eprint(f"fatal error: failure writing {ps}")
+ sys.exit(1)
+
+ try:
+ fd.close()
+ except:
+ eprint(f"fatal error: failure closing {ps}")
+ sys.exit(1)
+
+# A generic object.
+#
+class Obj:
+ def __init__(self):
+ pass
+
+# A dictionary that is a mapping from pathspecs for header rewrite config files to generic objects. If the
+# object has a "first" attribute, it is a list of 2-tuples. The first tuple entry is a reference to the
+# the CfgFile object for a remap config file. The second tuple entry is the index into the list of lines,
+# with the index of the line where header_rewrite.so is the first remap plugin, and the header rewrite
+# config file is a parameter to it. If "other" is an attribute of the generic object, the config file is
+# used as a parameter to calls to a global instance of header_rewrite.so, or instances of header_rewrite.so on
+# a remap rule as the second or later remap plugin. (The value of the "other" attribute is always None because
+# the value doesn't matter.)
+#
+header_rewrite_cfgs = dict()
+
+# Read in (global) header rewrite config file pathspecs from plugin.config file.
+#
+def handle_global(gbl_pathspec):
+
+ if gbl_pathspec[:1] != "/":
+ gbl_pathspec = pathspec_prefix + gbl_pathspec
+
+ try:
+ fd = open(gbl_pathspec, "r")
+ except:
+ eprint(f"fatal error: failure opening {gbl_pathspec} for reading")
+ sys.exit(1)
+
+ try:
+ lines = fd.readlines()
+ except:
+ eprint(f"fatal error: failure reading {gbl_pathspec}")
+ sys.exit(1)
+
+ try:
+ fd.close()
+ except:
+ eprint(f"fatal error: failure closing {gbl_pathspec}")
+ sys.exit(1)
+
+ ln_num = 0
+ while ln_num < len(lines):
+ ln = lines[ln_num]
+
+ # Join continuation lines to the first line.
+ #
+ num_ln_joined = 1
+ while ((ln_num + num_ln_joined) < len(lines)) and (ln[-2:] == (BACKSLASH + "\n")):
+ ln[:-2] += " " + lines[ln_num + num_ln_joined]
+ num_ln_joined += 1
+
+ ofst = ln.find("#")
+ if ofst >= 0:
+ # Remove comment.
+ #
+ ln = ln[:ofst]
+
+ ln = ln.split()
+
+ if (len(ln) > 0) and (ln[0] == "header_rewrite.so"):
+ for param in ln[1:]:
+ hr_obj = Obj()
+ hr_obj.other = None
+ header_rewrite_cfgs[param] = hr_obj
+ break
+
+ ln_num += num_ln_joined
+
+# A list of CfgFile instances for the main remap config file and any include files it contains, directly or
+# indirectly.
+#
+remap_cfgs = []
+
+# Handle the remap config file, and call this recursively to handle include files.
+#
+def handle_remap(filepath):
+
+ def skip_white(a_str):
+ if a_str[:1].isspace():
+ return 1
+ elif a_str.startswith(BACKSLASH + "\n"):
+ return 2
+ return 0
+
+ rc = CfgFile(filepath)
+
+ remap_cfgs.append(rc)
+
+ ln_num = 0
+ while ln_num < len(rc.lines):
+ ln = rc.lines[ln_num]
+
+ # Join continuation lines to the first line, and make them None in the rc.lines array.
+ #
+ num_ln_joined = 1
+ while ((ln_num + num_ln_joined) < len(rc.lines)) and (ln[-2:] == (BACKSLASH + "\n")):
+ ln += rc.lines[ln_num + num_ln_joined]
+ rc.lines[ln_num + num_ln_joined] = None
+ num_ln_joined += 1
+
+ rc.lines[ln_num] = ln
+
+ len_content = ln.find("#")
+ if len_content < 0:
+ len_content = len(ln) - 1
+
+ # Only process lines with content.
+ #
+ if (len_content > 0) and not ln[:len_content].isspace():
+ ofst = ln.find(".include")
+ if (ofst == 0) or ((ofst > 0) and ln[:ofst].isspace()):
+ ofst += len(".include")
+ start_ps = -1
+ while ofst < len_content:
+ sw = skip_white(ln[ofst:])
+ if sw == 0:
+ if start_ps < 0:
+ start_ps = ofst
+ ofst += 1
+ else:
+ if start_ps >= 0:
+ handle_remap(ln[start_ps:ofst])
+ start_ps = -1
+ ofst += sw
+
+ if start_ps >= 0:
+ handle_remap(ln[start_ps:len_content])
+ else:
+ # First, gzip.so -> compress.so
+ #
+ lnc = ln[:len_content]
+ lncr = lnc.replace("@plugin=gzip.so", "@plugin=compress.so")
+ if lncr != lnc:
+ ln = lncr + ln[len_content:]
+ len_content = len(lncr)
+ rc.lines[ln_num] = ln
+ rc.changed = True
+
+ ofst = ln[:len_content].find("@plugin=")
+ if ofst >= 0:
+ # Assuming it's some sort of remap line since it's got @plugin.
+ #
+ ofst += len("@plugin=")
+
+ new_ln = None
+
+ if ln[ofst:].startswith("header_rewrite.so"):
+ ofst += len("header_rewrite.so")
+
+ while ofst < len_content:
+ ofst2 = ln[ofst:len_content].find("@")
+ if ofst2 < 0:
+ break;
+
+ ofst += ofst2
+
+ if not ln[ofst:].startswith("@pparam="):
+ break
+
+ ofst += len("@pparam=")
+
+ ofst2 = ofst
+ while ofst2 < len_content:
+ sw = skip_white(ln[ofst2:])
+ if sw != 0:
+ break
+ ofst2 += 1
+
+ hr_pathspec = ln[ofst:ofst2]
+ ofst = ofst2
+
+ if hr_pathspec not in header_rewrite_cfgs:
+ hr_obj = Obj()
+ header_rewrite_cfgs[hr_pathspec] = hr_obj;
+ else:
+ hr_obj = header_rewrite_cfgs[hr_pathspec]
+
+ if not hasattr(hr_obj, "first"):
+ hr_obj.first = []
+
+ hr_obj.first.append((rc, ln_num))
+
+ elif ln[ofst:].startswith("regex_remap.so"):
+ ofst += len("regex_remap.so")
+
+ ofst2 = ln[ofst:len_content].find("@plugin=")
+
+ if ofst2 < 0:
+ ofst2 = len_content
+ else:
+ ofst2 += ofst
+
+ if ((ln[ofst:ofst2].find("@pparam=pristine") < 0) and
+ (ln[ofst:ofst2].find("@pparam=no-pristine") < 0)):
+
+ new_ln = ln[:ofst2]
+
+ if not new_ln[-1].isspace():
+ new_ln += " "
+
+ new_ln += "@pparam=pristine"
+
+ if not ln[ofst2].isspace():
+ new_ln += " "
+
+ ofst = len(new_ln)
+
+ new_ln += ln[ofst2:]
+
+ else:
+ ofst = ofst2
+
+ if new_ln:
+ rc.lines[ln_num] = new_ln
+ rc.changed = True
+ len_content += len(new_ln) - len(ln)
+ ln = new_ln
+
+ # Handle it if header rewrite is called as a second or later plugin.
+ #
+ while ofst < len_content:
+ ofst2 = ln[ofst:len_content].find("@plugin=header_rewrite.so")
+
+ if ofst2 < 0:
+ break
+
+ ofst += ofst2 + len("@plugin=header_rewrite.so")
+
+ while ofst < len_content:
+ ofst2 = ln[ofst:len_content].find("@")
+ if ofst2 < 0:
+ break
+
+ ofst += ofst2 + 1
+
+ if not ln[ofst:].startswith("pparam="):
+ ofst -= 1
+ break
+
+ ofst += len("pparam=")
+
+ ofst2 = ofst
+ while ofst2 < len_content:
+ sw = skip_white(ln[ofst2:])
+ if sw != 0:
+ break
+ ofst2 += 1
+
+ hr_pathspec = ln[ofst:ofst2]
+ ofst = ofst2
+
+ if hr_pathspec not in header_rewrite_cfgs:
+ hr_obj = Obj()
+ header_rewrite_cfgs[hr_pathspec] = hr_obj;
+ else:
+ hr_obj = header_rewrite_cfgs[hr_pathspec]
+
+ hr_obj.other = None # Only the existence of this attr matters, not its value.
+
+ ln_num += num_ln_joined
+
+def handle_header_rewrite(pathspec, obj):
+
+ # This class manages changes to a header rewrite config file.
+ #
+ class HRCfg:
+ # 'pathspec' is the key in header_rewrite_cfgs and 'obj' is the generic object value.
+ #
+ def __init__(self, pathspec, obj):
+ self._base = CfgFile(pathspec)
+ self._obj = obj
+
+ def has_first(self):
+ return hasattr(self._obj, "first")
+
+ def has_other(self):
+ return hasattr(self._obj, "other")
+
+ def get_lines(self):
+ return self._base.lines
+
+ # Assumes has_first is true.
+ #
+ def chg_ln_first(self, ln_num, ln):
+ if not hasattr(self, "_first"):
+ if self.has_other():
+ # Must make two versions of header rewrite config file.
+ #
+ self._first = CfgFile(self._base)
+ else:
+ self._first = self._base
+ self._first.changed = True
+
+ self._first.lines[ln_num] = ln
+
+ # Assumes has_other is true.
+ #
+ def chg_ln_cmn(self, ln_num, ln):
+ if hasattr(self, "_first") and not (self._first is self._base):
+ self._first.lines[ln_num] = ln
+
+ self._base.lines[ln_num] = ln
+ self._base.changed = True
+
+ def close(self):
+ self._base.close()
+ if hasattr(self, "_first") and not (self._first is self._base):
+ self._first.close()
+
+ old_pparam = "@pparam=" + self._base.pathspec
+
+ # Backpatch remap configuration files with pathspec of new header rewrite config file for call
+ # to header rewrite as first remap plugin.
+ #
+ for pair in self._obj.first:
+ rc = pair[0]
+ rc.changed = True
+ ln = rc.lines[pair[1]]
+ ofst = ln.find(old_pparam) + len("@pparam=")
+ rc.lines[pair[1]] = ln[:ofst] + self._first.pathspec + ln[ofst + len(self._base.pathspec):]
+
+ # Returns a pair: length of string before # comment, and a flag indicating if %< (beginning of a variable
+ # exapansion) was in any of the double-quoted strings.
+ #
+ def get_content_len(s):
+ # States.
+ #
+ COPYING = 0
+ ESCAPE_COPYING = 1
+ IN_QUOTED = 2
+ ESCAPE_IN_QUOTED = 3
+ PERCENT_IN_QUOTED = 4
+
+ state = COPYING
+ ofst = 0
+ variable_expansion = False
+
+ while ofst < (len(s) - 1):
+ if state == COPYING:
+ if s[ofst] == "#":
+ # Start of comment.
+ #
+ return (ofst, variable_expansion)
+ elif s[ofst] == '"':
+ state = IN_QUOTED
+ else:
+ if s[ofst] == BACKSLASH:
+ state = ESCAPE_COPYING
+
+ elif state == IN_QUOTED:
+ if s[ofst] == BACKSLASH:
+ state = ESCAPE_IN_QUOTED
+ elif s[ofst] == '"':
+ state = COPYING
+ elif s[ofst] == '%':
+ state = PERCENT_IN_QUOTED
+
+ elif state == ESCAPE_COPYING:
+ state = COPYING
+
+ elif state == ESCAPE_IN_QUOTED:
+ if s[ofst] == '%':
+ state = PERCENT_IN_QUOTED
+ else:
+ state = IN_QUOTED
+
+ elif state == PERCENT_IN_QUOTED:
+ if s[ofst] == '<':
+ variable_expansion = True
+ state = IN_QUOTED
+ elif s[ofst] == BACKSLASH:
+ state = ESCAPE_IN_QUOTED
+ elif s[ofst] == '"':
+ state = COPYING
+ elif s[ofst] == '%':
+ state = PERCENT_IN_QUOTED
+ else:
+ state = IN_QUOTED
+
+ ofst += 1
+
+ return (ofst, variable_expansion)
+
+ hrc = HRCfg(pathspec, obj)
+ lines = hrc.get_lines()
+ ln_num = 0
+ while ln_num < len(lines):
+ ln = lines[ln_num]
+ prepend_error = False
+
+ len_content, variable_expansion = get_content_len(ln)
+
+ if variable_expansion:
+ eprint(
+ f"error: {pathspec}, line {ln_num + 1}: variable expansions cannot be automatically converted"
+ )
+ prepend_error = True
+
+ lnc = ln[:len_content]
+ lnc = lnc.replace("%{CLIENT-IP}", "%{INBOUND:REMOTE-ADDR}")
+ lnc = lnc.replace("%{INCOMING-PORT}", "%{INBOUND:LOCAL-PORT}")
+ lnc = lnc.replace("%{PATH}", "%{URL:PATH}")
+ lnc = lnc.replace("%{QUERY}", "%{URL:QUERY}")
+
+ if hrc.has_other() and (prepend_error or (lnc != ln[:len_content])):
+ if prepend_error:
+ hrc.chg_ln_cmn(ln_num, "ERROR: " + lnc + ln[len_content:])
+ else:
+ hrc.chg_ln_cmn(ln_num, lnc + ln[len_content:])
+
+ if hrc.has_first():
+ lnc_save = lnc
+ ofst = lnc.find("set-destination")
+ if (ofst >= 0):
+ if (ofst == 0) or lnc[:ofst].isspace():
+ ofst += len("set-destination")
+ ofst2 = lnc[ofst:].find("PATH")
+ if (ofst2 > 0) and lnc[ofst:(ofst + ofst2)].isspace():
+ eprint(
+ f"error: {pathspec}, line {ln_num + 1}: the functionality of set-destination PATH" +
+ " for the first remap plugin does not exist in ATS9"
+ )
+ prepend_error = True
+
+ lnc = lnc.replace("%{URL:", "%{CLIENT-URL:")
+ if lnc != lnc_save or prepend_error:
+ if prepend_error:
+ hrc.chg_ln_first(ln_num, "ERROR: " + lnc + ln[len_content:])
+ else:
+ hrc.chg_ln_first(ln_num, lnc + ln[len_content:])
+
+ ln_num += 1
+
+ hrc.close()
+
+pathspec_prefix = os.path.dirname(args.filepath)
+if (pathspec_prefix != "") and (pathspec_prefix != "/"):
+ pathspec_prefix += "/"
+
+if not (args.plugin is None):
+ handle_global(args.plugin)
+
+handle_remap(os.path.basename(args.filepath))
+
+for pathspec in header_rewrite_cfgs:
+ handle_header_rewrite(pathspec, header_rewrite_cfgs[pathspec])
+
+# loop through remap_cfgs and close each one.
+for rc in remap_cfgs:
+ rc.close()
+
+sys.exit(0)
diff --git a/tools/insnew b/tools/insnew
new file mode 100755
index 0000000..94b1c8d
--- /dev/null
+++ b/tools/insnew
@@ -0,0 +1,67 @@
+#!/usr/bin/env bash
+
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# For each filespec FS given (one per line) on stdin, rename FS (if it exists) to FS.old, then rename FS.new to
+# FS.
+
+if [[ "$*" = "--help" ]] ; then
+ echo "For each filespec FS read from stdin, rename FS.new to FS. If FS already exists, rename to FS.old" >&2
+ exit 0
+fi
+
+while read F
+do
+python3 - "$F" << THE_END
+
+import sys
+import os
+
+def eprint(*args, **kwargs):
+ print(*args, file=sys.stderr, **kwargs)
+
+def rename_file(old_path, new_path):
+ try:
+ os.rename(old_path, new_path)
+ except:
+ eprint(f"fatal error: failure renaming {old_path} to {new_path}\n")
+ sys.exit(1)
+
+def rename_to_old(pathspec):
+ op = pathspec + ".old"
+
+ if os.access(op, os.F_OK):
+ last = 2
+ while os.access(op + ".{}".format(last), os.F_OK):
+ last += 1
+ while last > 2:
+ rename_file(op + ".{}".format(last - 1), op + ".{}".format(last))
+ last -= 1
+ rename_file(op, op + ".2")
+
+ rename_file(pathspec, op)
+
+filespec = sys.argv[1]
+if os.access(filespec, os.F_OK):
+ rename_to_old(filespec)
+rename_file(filespec + ".new", filespec)
+
+sys.exit(0)
+
+THE_END
+done
[trafficserver] 09/20: Cleans up duplicated
TSOutboundConnectionMatchType definition (#7090)
Posted by zw...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 9.1.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
commit e9c4b96b4720885a3d91f1f7f75163a4f39d8fb2
Author: Randall Meyer <rr...@apache.org>
AuthorDate: Mon Feb 1 09:54:30 2021 -0800
Cleans up duplicated TSOutboundConnectionMatchType definition (#7090)
---
include/ts/apidefs.h.in | 3 ---
proxy/http/HttpProxyAPIEnums.h | 15 ---------------
2 files changed, 18 deletions(-)
diff --git a/include/ts/apidefs.h.in b/include/ts/apidefs.h.in
index 27e1522..e8ee795 100644
--- a/include/ts/apidefs.h.in
+++ b/include/ts/apidefs.h.in
@@ -598,8 +598,6 @@ typedef enum {
AFTER_BODY,
} TSFetchWakeUpOptions;
-#ifndef _HTTP_PROXY_API_ENUMS_H_
-#define _HTTP_PROXY_API_ENUMS_H_
/// Values for per server outbound connection tracking group definition.
/// See proxy.config.http.per_server.match
typedef enum {
@@ -608,7 +606,6 @@ typedef enum {
TS_SERVER_OUTBOUND_MATCH_HOST,
TS_SERVER_OUTBOUND_MATCH_BOTH
} TSOutboundConnectionMatchType;
-#endif
/* librecords types */
diff --git a/proxy/http/HttpProxyAPIEnums.h b/proxy/http/HttpProxyAPIEnums.h
index d27b1ab..cd2dfa3 100644
--- a/proxy/http/HttpProxyAPIEnums.h
+++ b/proxy/http/HttpProxyAPIEnums.h
@@ -56,18 +56,3 @@ typedef enum {
TS_SERVER_SESSION_SHARING_POOL_THREAD,
TS_SERVER_SESSION_SHARING_POOL_HYBRID
} TSServerSessionSharingPoolType;
-
-// This is use to signal apidefs.h to not define these again.
-#ifndef _HTTP_PROXY_API_ENUMS_H_
-#define _HTTP_PROXY_API_ENUMS_H_
-
-/// Values for per server outbound connection tracking group definition.
-/// See proxy.config.http.per_server.match
-typedef enum {
- TS_SERVER_OUTBOUND_MATCH_IP,
- TS_SERVER_OUTBOUND_MATCH_PORT,
- TS_SERVER_OUTBOUND_MATCH_HOST,
- TS_SERVER_OUTBOUND_MATCH_BOTH
-} TSOutboundConnectionMatchType;
-
-#endif
[trafficserver] 10/20: Change atoi to atol,
causing obvious issues on what needs to be int64's (#7466)
Posted by zw...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 9.1.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
commit 9b1c40e148ff2f102797ceca5c0f37ed655c5642
Author: Evan Zelkowitz <ez...@apache.org>
AuthorDate: Mon Feb 1 11:41:57 2021 -0700
Change atoi to atol, causing obvious issues on what needs to be int64's (#7466)
The system stats plugin is currently using atoi and ints. These need to be longs to match int64
---
plugins/experimental/system_stats/system_stats.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/plugins/experimental/system_stats/system_stats.c b/plugins/experimental/system_stats/system_stats.c
index 7f7a42d..f8b5104 100644
--- a/plugins/experimental/system_stats/system_stats.c
+++ b/plugins/experimental/system_stats/system_stats.c
@@ -112,7 +112,7 @@ getFile(const char *filename, char *buffer, int bufferSize)
}
static void
-statSet(const char *name, int value, TSMutex stat_creation_mutex)
+statSet(const char *name, long value, TSMutex stat_creation_mutex)
{
int stat_id = statAdd(name, TS_RECORDDATATYPE_INT, stat_creation_mutex);
if (stat_id != TS_ERROR) {
@@ -153,7 +153,7 @@ setNetStat(TSMutex stat_creation_mutex, const char *interface, const char *entry
if (getFile(&sysfs_name[0], &data[0], sizeof(data)) < 0) {
TSDebug(DEBUG_TAG, "Error reading file %s", sysfs_name);
} else {
- statSet(stat_name, atoi(data), stat_creation_mutex);
+ statSet(stat_name, atol(data), stat_creation_mutex);
}
}
[trafficserver] 08/20: Fixing compress expectation for new
microserver (#7463)
Posted by zw...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 9.1.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
commit a7fed3ea70960056c86033acec671507e7fc83de
Author: Brian Neradt <br...@gmail.com>
AuthorDate: Fri Jan 29 19:17:22 2021 -0600
Fixing compress expectation for new microserver (#7463)
The new 1.0.6 microserver generates a body such that the content length
for it is 71 instead of 30 for one of the tests. Updating the gold file
to match this expectation.
---
tests/gold_tests/pluginTest/compress/compress.gold | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tests/gold_tests/pluginTest/compress/compress.gold b/tests/gold_tests/pluginTest/compress/compress.gold
index 5067978..fc4444b 100644
--- a/tests/gold_tests/pluginTest/compress/compress.gold
+++ b/tests/gold_tests/pluginTest/compress/compress.gold
@@ -184,5 +184,5 @@
< Content-Type: text/javascript
< Content-Encoding: gzip
< Vary: Accept-Encoding
-< Content-Length: 30
+< Content-Length: 71
===
[trafficserver] 01/20: Fix comment in include/tscore/Filenames.h.
(#7457)
Posted by zw...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 9.1.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
commit 255cd5ec91da47ccacf2280a1807e372d12acc1c
Author: Walt Karas <wk...@verizonmedia.com>
AuthorDate: Thu Jan 28 15:13:47 2021 -0600
Fix comment in include/tscore/Filenames.h. (#7457)
---
include/tscore/Filenames.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/include/tscore/Filenames.h b/include/tscore/Filenames.h
index 6fe0eb8..60d8441 100644
--- a/include/tscore/Filenames.h
+++ b/include/tscore/Filenames.h
@@ -30,7 +30,7 @@ namespace filename
constexpr const char *VOLUME = "volume.config";
constexpr const char *PLUGIN = "plugin.config";
- // These still need to have their corrensponding records.config settings remove
+ // These still need to have their corresponding records.config settings removed.
constexpr const char *LOGGING = "logging.yaml";
constexpr const char *CACHE = "cache.config";
constexpr const char *IP_ALLOW = "ip_allow.yaml";
[trafficserver] 17/20: Update AuTest version update directions for
pipenv (#7469)
Posted by zw...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
zwoop pushed a commit to branch 9.1.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git
commit 86226cde88db4f2056589e1db88887c989f20c4c
Author: Brian Neradt <br...@gmail.com>
AuthorDate: Tue Feb 2 13:22:11 2021 -0600
Update AuTest version update directions for pipenv (#7469)
The current AuTest extension describes how to update AuTest for pip,
not the currently used pipenv environment. This updates those directions
for pipenv.
This also adds a version check for microserver analogous to the version
check for AuTest.
---
tests/Pipfile | 6 ++++++
tests/gold_tests/autest-site/init.cli.ext | 20 ++++++++++++++------
2 files changed, 20 insertions(+), 6 deletions(-)
diff --git a/tests/Pipfile b/tests/Pipfile
index 727ea1e..a41b53b 100644
--- a/tests/Pipfile
+++ b/tests/Pipfile
@@ -24,7 +24,10 @@ autopep8 = "*"
pyflakes = "*"
[packages]
+
+# Keep init.cli.ext updated with this required autest version.
autest = "==1.8.1"
+
traffic-replay = "*" # this should install TRLib, MicroServer, MicroDNS, Traffic-Replay
hyper = "*"
dnslib = "*"
@@ -32,7 +35,10 @@ dnslib = "*"
requests = "*"
gunicorn = "*"
httpbin = "*"
+
+# Keep init.cli.ext updated with this required microserver version.
microserver = ">=1.0.6"
+
jsonschema = "*"
python-jose = "*"
diff --git a/tests/gold_tests/autest-site/init.cli.ext b/tests/gold_tests/autest-site/init.cli.ext
index 77678f1..505d63e 100644
--- a/tests/gold_tests/autest-site/init.cli.ext
+++ b/tests/gold_tests/autest-site/init.cli.ext
@@ -17,21 +17,29 @@
# limitations under the License.
import sys
+import microserver
if sys.version_info < (3, 6, 0):
host.WriteError(
"You need python 3.6 or later to run these tests\n", show_stack=False)
-autest_version = "1.8.1"
-if AuTestVersion() < autest_version:
+needed_autest_version = "1.8.1"
+found_autest_version = AuTestVersion()
+if AuTestVersion() < needed_autest_version:
host.WriteError(
- "Tests need AuTest version {needed_version} or better, found version {found_version}\n"
- "Please update AuTest:\n pip install --upgrade autest\n".format(
- needed_version=autest_version,
- found_version=AuTestVersion()),
+ f"Tests need AuTest version {needed_autest_version} or better, found version {found_autest_version}\n"
+ "Please update AuTest:\n pipenv --rm && pipenv install\n",
show_stack=False)
+needed_microserver_version = "1.0.6"
+found_microserver_version = microserver.__version__
+if found_microserver_version < needed_microserver_version:
+ host.WriteError(
+ f"Tests need a MicroServer version {needed_microserver_version} or better, found version {found_microserver_version}\n"
+ "Please update MicroServer:\n pipenv --rm && pipenv install\n",
+ show_stack=False)
+
Settings.path_argument(["--ats-bin"],
required=True,
help="A user provided directory to ATS bin")