You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@trafficserver.apache.org by ericcarlschwartz <gi...@git.apache.org> on 2016/08/06 18:20:19 UTC
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
GitHub user ericcarlschwartz opened a pull request:
https://github.com/apache/trafficserver/pull/843
[TS-4723] ATS CARP Plugin
JIRA: https://issues.apache.org/jira/browse/TS-4723
This is the code for our ATS CARP plugin along with some tests.
Two caveats with this code:
1. The makefile is all wrong. I'll definitely fix this before we get it merged. It's just something you can checkout if you want to build on your own machine. I'm not super familiar with automake, so I didn't add this to the Makefile.am in the parent directory or write my own Makefile.am and Makefile.in for this. If someone could send me a doc or help me do that, I'd love to learn.
2. There are some simple unit tests included here along with an api stub for running them. I've noticed the other experimental plugins don't have tests with them. If you'd like me to drop them, I can. Figured I'd include for now.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ericcarlschwartz/trafficserver TS-4723
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/trafficserver/pull/843.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #843
----
commit caed7922f3aa19c362178aee8e8e995dc62f51e1
Author: Eric Schwartz <es...@goesfoes.corp.gq1.yahoo.com>
Date: 2016-08-06T18:10:41Z
[TS-4723] ATS CARP Plugin
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by SolidWallOfCode <gi...@git.apache.org>.
Github user SolidWallOfCode commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r74255626
--- Diff: plugins/experimental/carp/CarpConfigPool.cc ---
@@ -0,0 +1,218 @@
+/** @file
+
+ Manage a list of CARP configurations
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+#include <sstream>
+
+#include "Common.h"
+#include "CarpConfigPool.h"
+#include "UrlComponents.h"
+#include "CarpHost.h"
+
+#include <netdb.h>
+#include <ts/ts.h>
+
+using namespace std;
+
+/*******************************************************************/
+CarpConfigPool::CarpConfigPool()
+{
+ _globalHash = NULL;
+ _globalConfig = NULL;
+}
+
+/*******************************************************************/
+CarpConfigPool::~CarpConfigPool()
+{
+ for(CarpConfigListIter it=_configList.begin(); it != _configList.end(); it++) {
+ it->second->_config->stop();
+ delete it->second;
+ }
+}
+/*******************************************************************/
+static int initCarpConfigAndHash(CarpConfigAndHash * cch, string sFilename) {
+ cch->_configPath = sFilename;
+ cch->_config = new CarpConfig();
+ TSAssert(cch->_config);
+
+ if (!cch->_config->loadConfig(sFilename)) {
+ return -1;
--- End diff --
Leak?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by ericcarlschwartz <gi...@git.apache.org>.
Github user ericcarlschwartz commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r74290371
--- Diff: plugins/experimental/carp/CarpConfig.cc ---
@@ -0,0 +1,581 @@
+/** @file
+
+ Loads the CARP configuration
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+
+//////////////////////////////////////////////////////////////
+// Read CARP configuration file
+// [Servers]
+// host1.yahoo.com:4080 weight=2 # port 4080 on host1.yahoo.com with weight factor of 2
+// host2.yahoo.com # port 80 on host2.yahoo.com with (default) weight factor of 1
+//
+// [Values]
+// healthcheck={host}:8001/status.html
+// healthfreq=30
+// global=on
+//
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <memory.h>
+#include <ts/ts.h>
+#include <sys/time.h>
+
+#include <sstream>
+
+#include "CarpConfig.h"
+#include "Common.h"
+#include "CarpConfigPool.h"
+
+using namespace std;
+
+#define DEFAULT_HEALTH_CHECK_FREQ 30 // 30 second period for health checks
+#define DEFAULT_HEALTH_CHECK_PORT 80 // default to makeing healthcheck requests against port 80
+#define DEFAULT_CONFIG_RELOAD_FREQ 30 // 30 seconds used in TSContSchedule
+#define DEFAULT_PORT 80 // default to makeing requests against port 80
+#define DEFAULT_WEIGHT 1 // default weight
+#define DEFAULT_SCHEME "http"
+#define DEFAULT_REPLICATION_FACTOR 1
+
+// config section headers
+static const char *const SECTION_SERVERS_STR = "[Servers]";
+static const char *const SECTION_VALUES_STR = "[Values]";
+
+// key strings
+static const char *const KEY_HEALTHCHECK_STR = "healthcheck";
+static const char *const KEY_HEALTHFREQ_STR = "healthfreq";
+static const char *const KEY_RELOADFREQ_STR = "reloadfreq";
+static const char *const KEY_HCTIMEOUT_STR = "hctimeout";
+static const char *const KEY_BLACKLIST_STR = "blacklist";
+static const char *const KEY_WHITELIST_STR = "whitelist";
+static const char *const KEY_MODE_STR = "mode";
+static const char *const KEY_ALLOWFWDPORT_STR = "allowfwdport";
+static const char *const KEY_REPLICATIONFACTOR_STR = "replicationfactor";
+
+// parameter strings
+static const char *const WEIGHT_EQUALS_STRING = "weight=";
+static const char *const GROUP_EQUALS_STRING = "group=";
+static const char *const KEY_MODE_PREREMAP_STR = "pre-remap";
+static const char *const KEY_MODE_POSTREMAP_STR = "post-remap";
+
+
+/**********************************************************/
+bool
+getInt(char** pptr, int *val)
+{
+ bool bReturn = false;
+ int v=0;
+
+ char* ptr = *pptr;
+ // skip white space if any
+ while (*ptr && isspace(*ptr)) ++ptr;
+ // get digits
+ if (*ptr) {
+ while (*ptr && isdigit(*ptr)) {
+ v *= 10;
+ v += (*ptr)-'0';
+ ++ptr;
+ bReturn = true;
+ }
+ }
+ if (bReturn) {
+ *pptr = ptr;
+ *val = v;
+ }
+
+ return bReturn;
+}
+
+/**********************************************************/
+// [http[s]://]host[:port]/path
+bool
+getHostAndPort(char** pptr,string* sHost,int* iPort, string* sScheme)
+{
+ bool bReturn = false;
+ char* ptr = *pptr;
+
+ *iPort = 80;
+ *sScheme = DEFAULT_SCHEME;
+ if(strncmp(*pptr,"https",5) == 0) {
+ *iPort = 443;
+ *sScheme = "https";
+ }
+
+ //skip leading white space
+ while (*ptr && isspace(*ptr)) ++ptr;
+
+ if (*ptr) { // validate not end of string
+ if(strncmp(ptr,"http://",7) == 0) {
+ ptr += 7;
+ } else if(strncmp(ptr,"https://",8) == 0) {
+ ptr += 8;
+ }
+
+ char* ptemp = ptr; // start of host
+ //find white space or ':' or '/'
+ while (*ptr && !isspace(*ptr) && *ptr != ':' && *ptr != '/') ++ptr;
--- End diff --
No, this was all written in a pre IPv6 world/is pretty old code. I could definitely extend at some point can talk to Josh about it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by SolidWallOfCode <gi...@git.apache.org>.
Github user SolidWallOfCode commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r74263649
--- Diff: plugins/experimental/carp/CarpHashAlgorithm.cc ---
@@ -0,0 +1,396 @@
+/** @file
+
+ Implements the CARP hash algorithm
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+#include <ts/ts.h>
+#include <stdio.h>
+#include <memory.h>
+#include <list>
+
+#include <iostream>
+#include <sstream>
+
+#include "Common.h"
+#include "CarpHashAlgorithm.h"
+
+using namespace std;
+
+/*****************************************************************/
+void
+HashNode::dump(string& s)
+{
+ stringstream ss;
+ string sSockaddr;
+ getStringFromSockaddr(reinterpret_cast<const struct sockaddr *>( &forwardAddr), sSockaddr);
+
+ ss << scheme << "://" << name << ":"<<listenPort<<" ("<<sSockaddr<<") weight:"<<weight<< (_status ? string(" UP ") : string(" DOWN "));
+ if(_statusTime) {
+ ss << "(" << time(NULL)-_statusTime<< "s ago in "<< _statusLatencyMs << "mS)";
+ }
+ ss << " hits:" << _hits;
+ ss << " carp_noforwarded:" << _carp_noforwarded;
+ ss << " carp_forwarded:" << _carp_forwarded;
+ ss << endl;
+ s += ss.str();
+}
+
+/*****************************************************************/
+void
+HashAlgorithm::addHost(std::string name, unsigned int port, std::string scheme, double weight, bool self, struct sockaddr_storage fwdAddr)
+{
+ HashNode* node = new HashNode(name, port, scheme, weight, self, fwdAddr);
+ TSAssert(NULL != node);
+ addHost(node);
+}
+
+/*****************************************************************/
+void
+HashAlgorithm::addHost(HashNode* node)
+{
+ if (!node) return;
+ _hostList.push_back(node);
+}
+
+/*****************************************************************/
+HashNode*
+HashAlgorithm::findStatusByNameAndPort(const string& name, unsigned int port,
+ size_t* index) {
+ /*
+ * Todo: This use loop to find the corresponding HashNode
+ * But the HttpClient and the Hash was related, so we
+ * could use more easier method to write the status
+ */
+ for (size_t ptr = 0; ptr < _hostList.size(); ptr++) {
+ if (_hostList[ptr]->listenPort == port
+ && _hostList[ptr]->name.compare(name) == 0) { // found it
+ if (index) {
+ *index = ptr;
+ }
+ return _hostList[ptr];
+ }
+ }
+ return NULL;
+}
+
+size_t
+HashAlgorithm::findHashNodeIndex(HashNode *node) {
+ for (size_t ptr = 0; ptr < _hostList.size(); ptr++) {
+ if ( _hostList[ptr] == node) {
+ return ptr;
+ }
+ }
+ return -1;
+}
+
+/*****************************************************************/
+void
+HashAlgorithm::setStatus(const string& name, unsigned int port, bool status, time_t time, uint64_t latencyMs)
+{
+ TSDebug(DEBUG_TAG_INIT, "HashAlgorithm::setStatus name=%s status=%d", name.c_str(), status);
+
+ HashNode* node = findStatusByNameAndPort(name,port);
+ if(node) {
+ node->setStatus(status,time,latencyMs);
+ } else {
+ TSError("Carp internal error setStatus host %s not found",name.c_str());
+ }
+}
+
+void
+HashAlgorithm::setStatus(HashNode * node, bool status, time_t time, uint64_t latencyMs)
+{
+ TSDebug(DEBUG_TAG_INIT, "HashAlgorithm::setStatus name=%s status=%d", node->name.c_str(), status);
+
+// HashNode* node = findStatusByNameAndPort(name,port);
+ if(node) {
--- End diff --
Why is `node` checked if it's passed in? That seems more like an assert or shouldn't be checked at all.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by SolidWallOfCode <gi...@git.apache.org>.
Github user SolidWallOfCode commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r75853615
--- Diff: plugins/experimental/carp/carp.cc ---
@@ -0,0 +1,713 @@
+/** @file
+
+ A brief file description
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+
+#include <errno.h>
+
+#include <string>
+#include <sstream>
+#include <stdlib.h>
+#include <stdio.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+#include <unistd.h>
+
+#include <memory.h>
+
+#include <ts/ts.h>
+
+#include "Common.h"
+#include "CarpConfig.h"
+#include "CarpConfigPool.h"
+#include "CarpHashAlgorithm.h"
+#include "UrlComponents.h"
+
+using namespace std;
+
+CarpConfigPool* g_CarpConfigPool = NULL;
+int g_carpSelectedHostArgIndex = 0;
+TSTextLogObject g_logObject = NULL;
+
+const char *logFileName = "carp";
+
+//////////////////////////////////////////////////////////////////////////////
+//////////////////////////////////////////////////////////////////////////////
+/*
+ check for our carp routed header, dump status if requested
+ */
+static int
+processCarpRoutedHeader(TSHttpTxn txnp, TSMBuffer bufp, TSMLoc hdr_loc)
+{
+ string value;
+ if (getHeader(bufp, hdr_loc, CARP_ROUTED_HEADER, value)) { // if found header
+ if (value.compare("1") == 0) { // is loop prevention value
+ TSDebug(DEBUG_TAG_HOOK, "Found %s header with loop prevention value, not forwarding again", CARP_ROUTED_HEADER.c_str());
+ return 0;
+ } else if (value.compare("dump") == 0) { // is dump status request
+ TSDebug(DEBUG_TAG_HOOK, "Found %s header with dump request", CARP_ROUTED_HEADER.c_str());
+ string status;
+ g_CarpConfigPool->getGlobalHashAlgo()->dump(status);
+ TSHttpTxnSetHttpRetStatus(txnp, TS_HTTP_STATUS_MULTI_STATUS);
+ TSHttpTxnErrorBodySet(txnp, TSstrdup(status.c_str()), status.length(), NULL);
+ return -1;
+ }
+ TSDebug(DEBUG_TAG_HOOK, "Found %s header with unknown value of %s, ignoring", CARP_ROUTED_HEADER.c_str(), value.c_str());
+ removeHeader(bufp, hdr_loc, CARP_ROUTED_HEADER);
+ }
+ return 1; // all OK
+}
+
+static bool
+checkListForSelf(std::vector<HashNode *> list)
+{
+ for (size_t k = 0; k < list.size(); k++) {
+ if (list[k]->isSelf) return true;
+ }
+ return false;
+}
+
+/**
+ bIsPOSTRemap = false --- Hash request and forward to peer
+ bIsPOSTRemap = true --- hash request, extract OS sockaddr, insert forwarding header, forward
+ */
+static int
+handleRequestProcessing(TSCont contp, TSEvent event, void *edata, bool bIsPOSTRemap)
+{
+ TSHttpTxn txnp = (TSHttpTxn) edata;
+ TSMBuffer bufp;
+ TSMLoc hdr_loc;
+ TSMLoc url_loc;
+
+ // get the client request so we can get URL and add header
+ if (TSHttpTxnClientReqGet(txnp, &bufp, &hdr_loc) != TS_SUCCESS) {
+ TSError("carp couldn't get request headers");
+ return -1;
+ }
+
+ int method_len;
+ const char *method = TSHttpHdrMethodGet(bufp, hdr_loc, &method_len);
+ if (NULL == method) {
+ TSError("carp couldn't get http method");
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return -1;
+ }
+ if (((method_len == TS_HTTP_LEN_DELETE) && (strncasecmp(method, TS_HTTP_METHOD_DELETE, TS_HTTP_LEN_DELETE) == 0)) ||
+ ((method_len == TS_HTTP_LEN_PURGE) && (strncasecmp(method, TS_HTTP_METHOD_PURGE, TS_HTTP_LEN_PURGE) == 0))) {
+ TSDebug(DEBUG_TAG_HOOK, "Request method is '%s' so not routing request", string(method,method_len).c_str());
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ }
+
+ if (TSHttpHdrUrlGet(bufp, hdr_loc, &url_loc) != TS_SUCCESS) {
+ TSError("carp couldn't get url");
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return -1;
+ }
+ // if has carp loop prevention header, do not remap
+ int iTemp = processCarpRoutedHeader(txnp, bufp, hdr_loc);
+ if(iTemp <= 0) { // if dump or do not remap
+ // check origin client request's scheme for premap mode
+ if (!bIsPOSTRemap) {
+ string oriScheme;
+ if (!getHeader(bufp, hdr_loc, CARP_PREMAP_SCHEME, oriScheme)) {
+ TSDebug(DEBUG_TAG_HOOK, "couldn't get '%s' header", CARP_PREMAP_SCHEME.c_str());
+ } else {
+ bool isHttps = (oriScheme == TS_URL_SCHEME_HTTPS);
+
+ if (isHttps) {
+ TSUrlSchemeSet(bufp, url_loc, TS_URL_SCHEME_HTTPS, TS_URL_LEN_HTTPS);
+ } else {
+ TSUrlSchemeSet(bufp, url_loc, TS_URL_SCHEME_HTTP, TS_URL_LEN_HTTP);
+ }
+
+ removeHeader(bufp, hdr_loc, CARP_STATUS_HEADER);
+ removeHeader(bufp, hdr_loc, CARP_ROUTED_HEADER);
+ removeHeader(bufp, hdr_loc, CARP_PREMAP_SCHEME.c_str());
+ TSDebug(DEBUG_TAG_HOOK, "Set client request's scheme to %s through %s header",
+ isHttps?"https":"http", CARP_PREMAP_SCHEME.c_str());
+ }
+ } else {
+ removeHeader(bufp, hdr_loc, CARP_ROUTED_HEADER);
+ }
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return iTemp;
+ }
+
+ UrlComponents reqUrl;
+ reqUrl.populate(bufp, url_loc);
+ // the url ONLY used to determine the cache owner
+ string sUrl;
+
+ if (!bIsPOSTRemap) { // if pre-remap, then host not in URL so get from header
+ string sHost;
+ if (!getHeader(bufp, hdr_loc, TS_MIME_FIELD_HOST, sHost)) {
+ TSDebug(DEBUG_TAG_HOOK, "Could not find host header, ignoring it");
+ }
+ reqUrl.setHost(sHost);
--- End diff --
Shouldn't this be in `else` if it's being ignored?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by SolidWallOfCode <gi...@git.apache.org>.
Github user SolidWallOfCode commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r74256344
--- Diff: plugins/experimental/carp/CarpConfigPool.cc ---
@@ -0,0 +1,218 @@
+/** @file
+
+ Manage a list of CARP configurations
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+#include <sstream>
+
+#include "Common.h"
+#include "CarpConfigPool.h"
+#include "UrlComponents.h"
+#include "CarpHost.h"
+
+#include <netdb.h>
+#include <ts/ts.h>
+
+using namespace std;
+
+/*******************************************************************/
+CarpConfigPool::CarpConfigPool()
+{
+ _globalHash = NULL;
+ _globalConfig = NULL;
+}
+
+/*******************************************************************/
+CarpConfigPool::~CarpConfigPool()
+{
+ for(CarpConfigListIter it=_configList.begin(); it != _configList.end(); it++) {
+ it->second->_config->stop();
+ delete it->second;
+ }
+}
+/*******************************************************************/
+static int initCarpConfigAndHash(CarpConfigAndHash * cch, string sFilename) {
+ cch->_configPath = sFilename;
+ cch->_config = new CarpConfig();
+ TSAssert(cch->_config);
+
+ if (!cch->_config->loadConfig(sFilename)) {
+ return -1;
+ }
+
+ cch->_hashAlgo = new CarpHashAlgorithm(cch->_config);
+
+ TSAssert(cch->_hashAlgo);
+
+ CarpHostList* hostList = cch->_config->getHostList();
+ // add hosts, etc to hash algo
+ char szServerName[256];
+ *szServerName = 0;
+
+ if (!gethostname(szServerName, 255)) { // success!
+ TSDebug(DEBUG_TAG_INIT, "using %s as server name to detect 'self'",
+ szServerName);
+ }
+ char buf[1024];
+ struct hostent self, *selfhe = getHostIp(string(szServerName), &self, buf,
+ sizeof(buf));
+
+ for (CarpHostListIter i = hostList->begin(); i != hostList->end(); i++) {
+ HashNode *hashNode;
+ bool bSelf = false;
+ if (NULL != selfhe) { // check for 'self'
+ // include hostname and port
+ bSelf = isSelf((*i)->getName(), (*i)->getPort(), selfhe);
+ }
+ if (cch->_config->getHealthCheckPort() == -1) { // did they specify 'PORT'?
+ (*i)->setHealthCheckPort((*i)->getPort()); // set HC port from server spec'd port
+ } else {
+ (*i)->setHealthCheckPort(cch->_config->getHealthCheckPort()); // set HC port
+ }
+ string sHCUrl = cch->_config->getHealthCheckUrl();
+ size_t pos = sHCUrl.find("{port}");
+ if (pos != string::npos) { // need to replace '{port}' with servers port
+ stringstream ss;
+ ss << (*i)->getPort();
+ sHCUrl.replace(pos, 6, ss.str()); // 6 = strlen of '{port}'
+ }
+ pos = sHCUrl.find("{host}");
+ if (pos != string::npos) {
+ sHCUrl.replace(pos, 6, (*i)->getName());
+ }
+ (*i)->setHealthCheckUrl(sHCUrl); // set HC Url
+
+ // Look up host and create addr struct for healthchecks
+ char hBuf[1024];
+ struct hostent host, *hosthe = getHostIp((*i)->getName(), &host, hBuf,
+ sizeof(hBuf));
+ if (hosthe) {
+ // convert hostent to sockaddr_in structure
+ sockaddr_in hIn;
+ memcpy(&hIn.sin_addr, hosthe->h_addr_list[0], hosthe->h_length);
+ hIn.sin_port = htons((*i)->getHealthCheckPort()); // set port
+ if (hosthe->h_length == 4) { // size match IPv4? assume such
--- End diff --
Something is wrong here - checking the length is not robust.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by SolidWallOfCode <gi...@git.apache.org>.
Github user SolidWallOfCode commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r75854917
--- Diff: plugins/experimental/carp/carp.cc ---
@@ -0,0 +1,713 @@
+/** @file
+
+ A brief file description
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+
+#include <errno.h>
+
+#include <string>
+#include <sstream>
+#include <stdlib.h>
+#include <stdio.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+#include <unistd.h>
+
+#include <memory.h>
+
+#include <ts/ts.h>
+
+#include "Common.h"
+#include "CarpConfig.h"
+#include "CarpConfigPool.h"
+#include "CarpHashAlgorithm.h"
+#include "UrlComponents.h"
+
+using namespace std;
+
+CarpConfigPool* g_CarpConfigPool = NULL;
+int g_carpSelectedHostArgIndex = 0;
+TSTextLogObject g_logObject = NULL;
+
+const char *logFileName = "carp";
+
+//////////////////////////////////////////////////////////////////////////////
+//////////////////////////////////////////////////////////////////////////////
+/*
+ check for our carp routed header, dump status if requested
+ */
+static int
+processCarpRoutedHeader(TSHttpTxn txnp, TSMBuffer bufp, TSMLoc hdr_loc)
+{
+ string value;
+ if (getHeader(bufp, hdr_loc, CARP_ROUTED_HEADER, value)) { // if found header
+ if (value.compare("1") == 0) { // is loop prevention value
+ TSDebug(DEBUG_TAG_HOOK, "Found %s header with loop prevention value, not forwarding again", CARP_ROUTED_HEADER.c_str());
+ return 0;
+ } else if (value.compare("dump") == 0) { // is dump status request
+ TSDebug(DEBUG_TAG_HOOK, "Found %s header with dump request", CARP_ROUTED_HEADER.c_str());
+ string status;
+ g_CarpConfigPool->getGlobalHashAlgo()->dump(status);
+ TSHttpTxnSetHttpRetStatus(txnp, TS_HTTP_STATUS_MULTI_STATUS);
+ TSHttpTxnErrorBodySet(txnp, TSstrdup(status.c_str()), status.length(), NULL);
+ return -1;
+ }
+ TSDebug(DEBUG_TAG_HOOK, "Found %s header with unknown value of %s, ignoring", CARP_ROUTED_HEADER.c_str(), value.c_str());
+ removeHeader(bufp, hdr_loc, CARP_ROUTED_HEADER);
+ }
+ return 1; // all OK
+}
+
+static bool
+checkListForSelf(std::vector<HashNode *> list)
+{
+ for (size_t k = 0; k < list.size(); k++) {
+ if (list[k]->isSelf) return true;
+ }
+ return false;
+}
+
+/**
+ bIsPOSTRemap = false --- Hash request and forward to peer
+ bIsPOSTRemap = true --- hash request, extract OS sockaddr, insert forwarding header, forward
+ */
+static int
+handleRequestProcessing(TSCont contp, TSEvent event, void *edata, bool bIsPOSTRemap)
+{
+ TSHttpTxn txnp = (TSHttpTxn) edata;
+ TSMBuffer bufp;
+ TSMLoc hdr_loc;
+ TSMLoc url_loc;
+
+ // get the client request so we can get URL and add header
+ if (TSHttpTxnClientReqGet(txnp, &bufp, &hdr_loc) != TS_SUCCESS) {
+ TSError("carp couldn't get request headers");
+ return -1;
+ }
+
+ int method_len;
+ const char *method = TSHttpHdrMethodGet(bufp, hdr_loc, &method_len);
+ if (NULL == method) {
+ TSError("carp couldn't get http method");
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return -1;
+ }
+ if (((method_len == TS_HTTP_LEN_DELETE) && (strncasecmp(method, TS_HTTP_METHOD_DELETE, TS_HTTP_LEN_DELETE) == 0)) ||
+ ((method_len == TS_HTTP_LEN_PURGE) && (strncasecmp(method, TS_HTTP_METHOD_PURGE, TS_HTTP_LEN_PURGE) == 0))) {
+ TSDebug(DEBUG_TAG_HOOK, "Request method is '%s' so not routing request", string(method,method_len).c_str());
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ }
+
+ if (TSHttpHdrUrlGet(bufp, hdr_loc, &url_loc) != TS_SUCCESS) {
+ TSError("carp couldn't get url");
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return -1;
+ }
+ // if has carp loop prevention header, do not remap
+ int iTemp = processCarpRoutedHeader(txnp, bufp, hdr_loc);
+ if(iTemp <= 0) { // if dump or do not remap
+ // check origin client request's scheme for premap mode
+ if (!bIsPOSTRemap) {
+ string oriScheme;
+ if (!getHeader(bufp, hdr_loc, CARP_PREMAP_SCHEME, oriScheme)) {
+ TSDebug(DEBUG_TAG_HOOK, "couldn't get '%s' header", CARP_PREMAP_SCHEME.c_str());
+ } else {
+ bool isHttps = (oriScheme == TS_URL_SCHEME_HTTPS);
+
+ if (isHttps) {
+ TSUrlSchemeSet(bufp, url_loc, TS_URL_SCHEME_HTTPS, TS_URL_LEN_HTTPS);
+ } else {
+ TSUrlSchemeSet(bufp, url_loc, TS_URL_SCHEME_HTTP, TS_URL_LEN_HTTP);
+ }
+
+ removeHeader(bufp, hdr_loc, CARP_STATUS_HEADER);
+ removeHeader(bufp, hdr_loc, CARP_ROUTED_HEADER);
+ removeHeader(bufp, hdr_loc, CARP_PREMAP_SCHEME.c_str());
+ TSDebug(DEBUG_TAG_HOOK, "Set client request's scheme to %s through %s header",
+ isHttps?"https":"http", CARP_PREMAP_SCHEME.c_str());
+ }
+ } else {
+ removeHeader(bufp, hdr_loc, CARP_ROUTED_HEADER);
+ }
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return iTemp;
+ }
+
+ UrlComponents reqUrl;
+ reqUrl.populate(bufp, url_loc);
+ // the url ONLY used to determine the cache owner
+ string sUrl;
+
+ if (!bIsPOSTRemap) { // if pre-remap, then host not in URL so get from header
+ string sHost;
+ if (!getHeader(bufp, hdr_loc, TS_MIME_FIELD_HOST, sHost)) {
+ TSDebug(DEBUG_TAG_HOOK, "Could not find host header, ignoring it");
+ }
+ reqUrl.setHost(sHost);
+
+ //[YTSATS-836] heuristically ignore the scheme and port when calculate cache owner
+ UrlComponents normalizedUrl = reqUrl;
+ normalizedUrl.setScheme(CARP_SCHEME_FOR_HASH);
+ normalizedUrl.setPort(CARP_PORT_FOR_HASH);
+ normalizedUrl.construct(sUrl);
+ } else {
+ reqUrl.construct(sUrl);
+ }
+
+
+ if (g_CarpConfigPool->getGlobalConfig()->hasWhiteList()) {
+ string sCarpable;
+ if (!getHeader(bufp, hdr_loc, CARPABLE_HEADER, sCarpable)) { // if no carpable header check whitelist
+ if (!g_CarpConfigPool->getGlobalConfig()->isWhiteListed(reqUrl.getHost())) { // if white list exists, then host must be present
+ TSDebug(DEBUG_TAG_HOOK, "Host '%s' is not whitelisted, not going through carp", reqUrl.getHost().c_str());
+ TSHandleMLocRelease(bufp, hdr_loc, url_loc);
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ } else {
+ TSDebug(DEBUG_TAG_HOOK, "Found host (%s) whitelisted, routing...",reqUrl.getHost().c_str());
+ }
+ } else { // found carpable header, make sure it's 1
+ if (sCarpable.compare("1") != 0) { // carpable header present but not 0, be strict and do not forward request
+ TSDebug(DEBUG_TAG_HOOK, "Carpable (%s) present but value not acceptable (%s)", CARPABLE_HEADER.c_str(), sCarpable.c_str());
+ TSHandleMLocRelease(bufp, hdr_loc, url_loc);
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ } else {
+ TSDebug(DEBUG_TAG_HOOK, "Found Carpable header, routing...");
+ }
+ }
+ } else { // no whitelist so blacklist could be used
+ if (g_CarpConfigPool->getGlobalConfig()->isBlackListed(reqUrl.getHost())) { // if host black listed, do not carp
+ TSDebug(DEBUG_TAG_HOOK, "Host '%s' is blacklisted, not going through carp", reqUrl.getHost().c_str());
+ TSHandleMLocRelease(bufp, hdr_loc, url_loc);
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ }
+ }
+
+ TSDebug(DEBUG_TAG_HOOK, "URL to hash with=%s", sUrl.c_str());
+
+ // get nodeList and select node to forward to
+ std::vector<HashNode *> nodeList = g_CarpConfigPool->getGlobalHashAlgo()->getRemapProxyList(sUrl);
+ bool bAddHeaderResult=false;
+ bool bAddForwardedResult = false;
+ HashNode* node = NULL;
+ bool bIsOwner = false;
+
+ if (nodeList.size() == 0) { // no hosts available to forward to
+ TSDebug(DEBUG_TAG_HOOK, "no hosts available to forward to, will handle locally");
+ goto done;
+ } else {
+ node = nodeList[0];
+ }
+
+ bIsOwner = checkListForSelf(nodeList);
+ for (size_t k = 0; k < nodeList.size(); k++) {
+ TSDebug(DEBUG_TAG_HOOK, "nodeList host %d name is %s", static_cast<int>(k), nodeList[k]->name.c_str());
+ }
+
+ //handle forwarding
+ TSDebug(DEBUG_TAG_HOOK, "forwarding to '%s' (isSelf=%d)", node->name.c_str(), node->isSelf);
+ if (!node->isSelf) { // carp does not forward if we choose ourself
+ node->carpForward();
+ TSDebug(DEBUG_TAG_HOOK, "carp forwarded to %s.", node->name.c_str());
+ // insert carp loop prevention header
+ bAddHeaderResult = addHeader(bufp, hdr_loc, CARP_ROUTED_HEADER, string("1"));
+ if (!bAddHeaderResult) {
+ TSError("Carp, error inserting '%s' header", CARP_ROUTED_HEADER.c_str());
+ }
+ bAddForwardedResult = addHeader(bufp, hdr_loc, CARP_STATUS_HEADER, string(CARP_FORWARDED));
+ if (!bAddForwardedResult) {
+ TSError("Carp, error inserting '%s' header", CARP_STATUS_HEADER.c_str());
+ }
+
+ if (bIsPOSTRemap) { // if post remap, get remapped/OS Server Addr and add as header
+ const struct sockaddr* sa = TSHttpTxnServerAddrGet(txnp);
+ // const struct sockaddr* sa = TSHttpTxnNextHopAddrGet(txnp);
+ if (sa) { // sanity check
+ struct sockaddr_storage ss;
+ memcpy(static_cast<void *> (&ss), sa, sizeof (sockaddr_storage));
--- End diff --
How do you know `sa` is `sizeof sockaddr_storage` long?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by SolidWallOfCode <gi...@git.apache.org>.
Github user SolidWallOfCode commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r75869437
--- Diff: plugins/experimental/carp/CarpConfigPool.cc ---
@@ -0,0 +1,218 @@
+/** @file
+
+ Manage a list of CARP configurations
--- End diff --
Is there really a list of CARP configurations? If so, is it used anywhere?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by ericcarlschwartz <gi...@git.apache.org>.
Github user ericcarlschwartz commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r75941557
--- Diff: plugins/experimental/carp/CarpConfig.cc ---
@@ -0,0 +1,581 @@
+/** @file
+
+ Loads the CARP configuration
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+
+//////////////////////////////////////////////////////////////
+// Read CARP configuration file
+// [Servers]
+// host1.yahoo.com:4080 weight=2 # port 4080 on host1.yahoo.com with weight factor of 2
+// host2.yahoo.com # port 80 on host2.yahoo.com with (default) weight factor of 1
+//
+// [Values]
+// healthcheck={host}:8001/status.html
+// healthfreq=30
+// global=on
+//
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <memory.h>
+#include <ts/ts.h>
+#include <sys/time.h>
+
+#include <sstream>
+
+#include "CarpConfig.h"
+#include "Common.h"
+#include "CarpConfigPool.h"
+
+using namespace std;
+
+#define DEFAULT_HEALTH_CHECK_FREQ 30 // 30 second period for health checks
+#define DEFAULT_HEALTH_CHECK_PORT 80 // default to makeing healthcheck requests against port 80
+#define DEFAULT_CONFIG_RELOAD_FREQ 30 // 30 seconds used in TSContSchedule
+#define DEFAULT_PORT 80 // default to makeing requests against port 80
+#define DEFAULT_WEIGHT 1 // default weight
+#define DEFAULT_SCHEME "http"
+#define DEFAULT_REPLICATION_FACTOR 1
+
+// config section headers
+static const char *const SECTION_SERVERS_STR = "[Servers]";
+static const char *const SECTION_VALUES_STR = "[Values]";
+
+// key strings
+static const char *const KEY_HEALTHCHECK_STR = "healthcheck";
+static const char *const KEY_HEALTHFREQ_STR = "healthfreq";
+static const char *const KEY_RELOADFREQ_STR = "reloadfreq";
+static const char *const KEY_HCTIMEOUT_STR = "hctimeout";
+static const char *const KEY_BLACKLIST_STR = "blacklist";
+static const char *const KEY_WHITELIST_STR = "whitelist";
+static const char *const KEY_MODE_STR = "mode";
+static const char *const KEY_ALLOWFWDPORT_STR = "allowfwdport";
+static const char *const KEY_REPLICATIONFACTOR_STR = "replicationfactor";
+
+// parameter strings
+static const char *const WEIGHT_EQUALS_STRING = "weight=";
+static const char *const GROUP_EQUALS_STRING = "group=";
+static const char *const KEY_MODE_PREREMAP_STR = "pre-remap";
+static const char *const KEY_MODE_POSTREMAP_STR = "post-remap";
+
+
+/**********************************************************/
+bool
+getInt(char** pptr, int *val)
+{
+ bool bReturn = false;
+ int v=0;
+
+ char* ptr = *pptr;
+ // skip white space if any
+ while (*ptr && isspace(*ptr)) ++ptr;
+ // get digits
+ if (*ptr) {
+ while (*ptr && isdigit(*ptr)) {
+ v *= 10;
+ v += (*ptr)-'0';
+ ++ptr;
+ bReturn = true;
+ }
+ }
+ if (bReturn) {
+ *pptr = ptr;
+ *val = v;
+ }
+
+ return bReturn;
+}
+
+/**********************************************************/
+// [http[s]://]host[:port]/path
+bool
+getHostAndPort(char** pptr,string* sHost,int* iPort, string* sScheme)
+{
+ bool bReturn = false;
+ char* ptr = *pptr;
+
+ *iPort = 80;
+ *sScheme = DEFAULT_SCHEME;
+ if(strncmp(*pptr,"https",5) == 0) {
+ *iPort = 443;
+ *sScheme = "https";
+ }
+
+ //skip leading white space
+ while (*ptr && isspace(*ptr)) ++ptr;
--- End diff --
I addressed this by always skipping the leading white space.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by SolidWallOfCode <gi...@git.apache.org>.
Github user SolidWallOfCode commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r75868057
--- Diff: plugins/experimental/carp/carp.cc ---
@@ -0,0 +1,713 @@
+/** @file
+
+ A brief file description
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+
+#include <errno.h>
+
+#include <string>
+#include <sstream>
+#include <stdlib.h>
+#include <stdio.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+#include <unistd.h>
+
+#include <memory.h>
+
+#include <ts/ts.h>
+
+#include "Common.h"
+#include "CarpConfig.h"
+#include "CarpConfigPool.h"
+#include "CarpHashAlgorithm.h"
+#include "UrlComponents.h"
+
+using namespace std;
+
+CarpConfigPool* g_CarpConfigPool = NULL;
+int g_carpSelectedHostArgIndex = 0;
+TSTextLogObject g_logObject = NULL;
+
+const char *logFileName = "carp";
+
+//////////////////////////////////////////////////////////////////////////////
+//////////////////////////////////////////////////////////////////////////////
+/*
+ check for our carp routed header, dump status if requested
+ */
+static int
+processCarpRoutedHeader(TSHttpTxn txnp, TSMBuffer bufp, TSMLoc hdr_loc)
+{
+ string value;
+ if (getHeader(bufp, hdr_loc, CARP_ROUTED_HEADER, value)) { // if found header
+ if (value.compare("1") == 0) { // is loop prevention value
+ TSDebug(DEBUG_TAG_HOOK, "Found %s header with loop prevention value, not forwarding again", CARP_ROUTED_HEADER.c_str());
+ return 0;
+ } else if (value.compare("dump") == 0) { // is dump status request
+ TSDebug(DEBUG_TAG_HOOK, "Found %s header with dump request", CARP_ROUTED_HEADER.c_str());
+ string status;
+ g_CarpConfigPool->getGlobalHashAlgo()->dump(status);
+ TSHttpTxnSetHttpRetStatus(txnp, TS_HTTP_STATUS_MULTI_STATUS);
+ TSHttpTxnErrorBodySet(txnp, TSstrdup(status.c_str()), status.length(), NULL);
+ return -1;
+ }
+ TSDebug(DEBUG_TAG_HOOK, "Found %s header with unknown value of %s, ignoring", CARP_ROUTED_HEADER.c_str(), value.c_str());
+ removeHeader(bufp, hdr_loc, CARP_ROUTED_HEADER);
+ }
+ return 1; // all OK
+}
+
+static bool
+checkListForSelf(std::vector<HashNode *> list)
+{
+ for (size_t k = 0; k < list.size(); k++) {
+ if (list[k]->isSelf) return true;
+ }
+ return false;
+}
+
+/**
+ bIsPOSTRemap = false --- Hash request and forward to peer
+ bIsPOSTRemap = true --- hash request, extract OS sockaddr, insert forwarding header, forward
+ */
+static int
+handleRequestProcessing(TSCont contp, TSEvent event, void *edata, bool bIsPOSTRemap)
+{
+ TSHttpTxn txnp = (TSHttpTxn) edata;
+ TSMBuffer bufp;
+ TSMLoc hdr_loc;
+ TSMLoc url_loc;
+
+ // get the client request so we can get URL and add header
+ if (TSHttpTxnClientReqGet(txnp, &bufp, &hdr_loc) != TS_SUCCESS) {
+ TSError("carp couldn't get request headers");
+ return -1;
+ }
+
+ int method_len;
+ const char *method = TSHttpHdrMethodGet(bufp, hdr_loc, &method_len);
+ if (NULL == method) {
+ TSError("carp couldn't get http method");
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return -1;
+ }
+ if (((method_len == TS_HTTP_LEN_DELETE) && (strncasecmp(method, TS_HTTP_METHOD_DELETE, TS_HTTP_LEN_DELETE) == 0)) ||
+ ((method_len == TS_HTTP_LEN_PURGE) && (strncasecmp(method, TS_HTTP_METHOD_PURGE, TS_HTTP_LEN_PURGE) == 0))) {
+ TSDebug(DEBUG_TAG_HOOK, "Request method is '%s' so not routing request", string(method,method_len).c_str());
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ }
+
+ if (TSHttpHdrUrlGet(bufp, hdr_loc, &url_loc) != TS_SUCCESS) {
+ TSError("carp couldn't get url");
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return -1;
+ }
+ // if has carp loop prevention header, do not remap
+ int iTemp = processCarpRoutedHeader(txnp, bufp, hdr_loc);
+ if(iTemp <= 0) { // if dump or do not remap
+ // check origin client request's scheme for premap mode
+ if (!bIsPOSTRemap) {
+ string oriScheme;
+ if (!getHeader(bufp, hdr_loc, CARP_PREMAP_SCHEME, oriScheme)) {
+ TSDebug(DEBUG_TAG_HOOK, "couldn't get '%s' header", CARP_PREMAP_SCHEME.c_str());
+ } else {
+ bool isHttps = (oriScheme == TS_URL_SCHEME_HTTPS);
+
+ if (isHttps) {
+ TSUrlSchemeSet(bufp, url_loc, TS_URL_SCHEME_HTTPS, TS_URL_LEN_HTTPS);
+ } else {
+ TSUrlSchemeSet(bufp, url_loc, TS_URL_SCHEME_HTTP, TS_URL_LEN_HTTP);
+ }
+
+ removeHeader(bufp, hdr_loc, CARP_STATUS_HEADER);
+ removeHeader(bufp, hdr_loc, CARP_ROUTED_HEADER);
+ removeHeader(bufp, hdr_loc, CARP_PREMAP_SCHEME.c_str());
+ TSDebug(DEBUG_TAG_HOOK, "Set client request's scheme to %s through %s header",
+ isHttps?"https":"http", CARP_PREMAP_SCHEME.c_str());
+ }
+ } else {
+ removeHeader(bufp, hdr_loc, CARP_ROUTED_HEADER);
+ }
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return iTemp;
+ }
+
+ UrlComponents reqUrl;
+ reqUrl.populate(bufp, url_loc);
+ // the url ONLY used to determine the cache owner
+ string sUrl;
+
+ if (!bIsPOSTRemap) { // if pre-remap, then host not in URL so get from header
+ string sHost;
+ if (!getHeader(bufp, hdr_loc, TS_MIME_FIELD_HOST, sHost)) {
+ TSDebug(DEBUG_TAG_HOOK, "Could not find host header, ignoring it");
+ }
+ reqUrl.setHost(sHost);
+
+ //[YTSATS-836] heuristically ignore the scheme and port when calculate cache owner
+ UrlComponents normalizedUrl = reqUrl;
+ normalizedUrl.setScheme(CARP_SCHEME_FOR_HASH);
+ normalizedUrl.setPort(CARP_PORT_FOR_HASH);
+ normalizedUrl.construct(sUrl);
+ } else {
+ reqUrl.construct(sUrl);
+ }
+
+
+ if (g_CarpConfigPool->getGlobalConfig()->hasWhiteList()) {
+ string sCarpable;
+ if (!getHeader(bufp, hdr_loc, CARPABLE_HEADER, sCarpable)) { // if no carpable header check whitelist
+ if (!g_CarpConfigPool->getGlobalConfig()->isWhiteListed(reqUrl.getHost())) { // if white list exists, then host must be present
+ TSDebug(DEBUG_TAG_HOOK, "Host '%s' is not whitelisted, not going through carp", reqUrl.getHost().c_str());
+ TSHandleMLocRelease(bufp, hdr_loc, url_loc);
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ } else {
+ TSDebug(DEBUG_TAG_HOOK, "Found host (%s) whitelisted, routing...",reqUrl.getHost().c_str());
+ }
+ } else { // found carpable header, make sure it's 1
+ if (sCarpable.compare("1") != 0) { // carpable header present but not 0, be strict and do not forward request
+ TSDebug(DEBUG_TAG_HOOK, "Carpable (%s) present but value not acceptable (%s)", CARPABLE_HEADER.c_str(), sCarpable.c_str());
+ TSHandleMLocRelease(bufp, hdr_loc, url_loc);
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ } else {
+ TSDebug(DEBUG_TAG_HOOK, "Found Carpable header, routing...");
+ }
+ }
+ } else { // no whitelist so blacklist could be used
+ if (g_CarpConfigPool->getGlobalConfig()->isBlackListed(reqUrl.getHost())) { // if host black listed, do not carp
+ TSDebug(DEBUG_TAG_HOOK, "Host '%s' is blacklisted, not going through carp", reqUrl.getHost().c_str());
+ TSHandleMLocRelease(bufp, hdr_loc, url_loc);
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ }
+ }
+
+ TSDebug(DEBUG_TAG_HOOK, "URL to hash with=%s", sUrl.c_str());
+
+ // get nodeList and select node to forward to
+ std::vector<HashNode *> nodeList = g_CarpConfigPool->getGlobalHashAlgo()->getRemapProxyList(sUrl);
+ bool bAddHeaderResult=false;
+ bool bAddForwardedResult = false;
+ HashNode* node = NULL;
+ bool bIsOwner = false;
+
+ if (nodeList.size() == 0) { // no hosts available to forward to
+ TSDebug(DEBUG_TAG_HOOK, "no hosts available to forward to, will handle locally");
+ goto done;
+ } else {
+ node = nodeList[0];
+ }
+
+ bIsOwner = checkListForSelf(nodeList);
+ for (size_t k = 0; k < nodeList.size(); k++) {
+ TSDebug(DEBUG_TAG_HOOK, "nodeList host %d name is %s", static_cast<int>(k), nodeList[k]->name.c_str());
+ }
+
+ //handle forwarding
+ TSDebug(DEBUG_TAG_HOOK, "forwarding to '%s' (isSelf=%d)", node->name.c_str(), node->isSelf);
+ if (!node->isSelf) { // carp does not forward if we choose ourself
+ node->carpForward();
+ TSDebug(DEBUG_TAG_HOOK, "carp forwarded to %s.", node->name.c_str());
+ // insert carp loop prevention header
+ bAddHeaderResult = addHeader(bufp, hdr_loc, CARP_ROUTED_HEADER, string("1"));
+ if (!bAddHeaderResult) {
+ TSError("Carp, error inserting '%s' header", CARP_ROUTED_HEADER.c_str());
+ }
+ bAddForwardedResult = addHeader(bufp, hdr_loc, CARP_STATUS_HEADER, string(CARP_FORWARDED));
+ if (!bAddForwardedResult) {
+ TSError("Carp, error inserting '%s' header", CARP_STATUS_HEADER.c_str());
+ }
+
+ if (bIsPOSTRemap) { // if post remap, get remapped/OS Server Addr and add as header
+ const struct sockaddr* sa = TSHttpTxnServerAddrGet(txnp);
+ // const struct sockaddr* sa = TSHttpTxnNextHopAddrGet(txnp);
+ if (sa) { // sanity check
+ struct sockaddr_storage ss;
+ memcpy(static_cast<void *> (&ss), sa, sizeof (sockaddr_storage));
+ if ((reinterpret_cast<sockaddr_in *> (&ss))->sin_port == 0) { // set port from client request URL
+ (reinterpret_cast<sockaddr_in *> (&ss))->sin_port = htons(reqUrl.getPort());
+ }
+
+ string sTemp;
+ getStringFromSockaddr(reinterpret_cast<const sockaddr *> (&ss), sTemp);
+ TSDebug(DEBUG_TAG_HOOK, "Inserting forward header with sockaddr:%s", sTemp.c_str());
+ string sSockaddr;
+ sSockaddr.reserve(32 + sizeof (sockaddr_storage) * 2);
+ for (unsigned int i = 0; i<sizeof (sockaddr_storage); i++) {
+ char val[8];
+ sprintf(val, "%02X", reinterpret_cast<const unsigned char *> (&ss)[i]);
+ sSockaddr += val;
+ }
+ sSockaddr += "/" + reqUrl.getScheme();
+ // insert carp forwarding header
+ bool bAddFwdHeaderResult = addHeader(bufp, hdr_loc, CARP_FORWARD_HEADER, sSockaddr);
+ if (!bAddFwdHeaderResult) {
+ TSError("Carp, error inserting '%s' header", CARP_FORWARD_HEADER.c_str());
+ }
+ }
+ } else { // for premap mode
+ string sScheme = reqUrl.getScheme();
+
+ if (!addHeader(bufp, hdr_loc, CARP_PREMAP_SCHEME, sScheme)) {
+ TSError("Carp, error inserting '%s' header in premap mode", CARP_PREMAP_SCHEME.c_str());
+ } else {
+ TSDebug(DEBUG_TAG_HOOK, "Insert client request scheme %s in premap mode", sScheme.c_str());
+ }
+ }
+ // set origin server/destination
+ // TSHttpTxnConfigIntSet(txnp, TS_CONFIG_HTTP_SHARE_SERVER_SESSIONS, 0); // disable conn sharing due to bug when used with TSHttpTxnServerAddrSet
+ if (TSHttpTxnServerAddrSet(txnp, reinterpret_cast<const struct sockaddr *> (&node->forwardAddr)) != TS_SUCCESS) {
+ TSDebug(DEBUG_TAG_HOOK, "Error calling TSHttpTxnServerAddrSet");
+ } else {
+ // set scheme appropriately based on destination
+ TSDebug(DEBUG_TAG_HOOK, "Setting scheme to '%s'", node->getSchemeString());
+ TSUrlSchemeSet(bufp, url_loc, node->getSchemeString(), -1);
+ if (!bIsPOSTRemap) { // since we are forwarding, do not remap request and do not cache result
+ TSSkipRemappingSet(txnp, true);
+ }
+ TSHttpTxnArgSet(txnp, g_carpSelectedHostArgIndex, static_cast<void *> (node));
+ if (!bIsOwner) {
+ TSHttpTxnServerRespNoStoreSet(txnp, 1); // do not cache the response
+ } else {
+ TSHttpTxnServerRespNoStoreSet(txnp, 0); // we are replicate owner, cache response
+ }
+ }
+ } else {
+ node->carpNoForward();
+ TSDebug(DEBUG_TAG_HOOK, "carp forwarded to self.");
+ }
+
+done :
+ // done w/buffers, release them
+ TSHandleMLocRelease(bufp, hdr_loc, url_loc);
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+}
+
+/**
+ * Convert ASCII hex digit to value of hex digit
+ * @param ch
+ * @return
+ */
+static unsigned char
+getValueOfHex(unsigned char ch)
+{
+ ch -= '0';
+ if(ch > 9) ch -= 7; // 'A' - ':' = 7
+ return ch;
+}
+
+/**
+ Process request pre-remap while running post-remap mode
+ if forwarding header present and rules met, disable remap processing, forward to OS
+ */
+static int
+handleForwardRequestProcessing(TSCont contp, TSEvent event, void *edata)
+{
+ // we only want to do anything here if CARP_ROUTED_HEADER present
+ // and CARP_FORWARD_HEADER is present
+ // and incoming request is from port getAllowedForwardPort()
+ TSHttpTxn txnp = (TSHttpTxn) edata;
+ TSMBuffer bufp;
+ TSMLoc hdr_loc;
+
+ // get the client request
+ if (TSHttpTxnClientReqGet(txnp, &bufp, &hdr_loc) != TS_SUCCESS) {
+ TSError("carp couldn't get request headers");
+ return -1;
+ }
+
+ // if has carp loop prevention header, do not remap
+ int iTemp = processCarpRoutedHeader(txnp, bufp, hdr_loc);
+ if(iTemp == -1 || iTemp == 1) { // is dump request or routed header not present
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return (iTemp == 1) ? 0 : iTemp; // return 0 if routed header not present
+ }
+
+ // check incoming port number first
+ const struct sockaddr_in* sa = reinterpret_cast<const sockaddr_in *>(TSHttpTxnIncomingAddrGet(txnp));
+ if(!sa) {
+ TSDebug(DEBUG_TAG_HOOK, "TSHttpTxnIncomingAddrGet() returned NULL");
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ }
+
+ int iPort = ntohs(sa->sin_port);
+ if(iPort != g_CarpConfigPool->getGlobalConfig()->getAllowedForwardPort()) {
+ TSDebug(DEBUG_TAG_HOOK, "Allowed forward port does not match. Incoming request on %d, but configured for %d.",iPort, g_CarpConfigPool->getGlobalConfig()->getAllowedForwardPort());
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ }
+
+ TSDebug(DEBUG_TAG_HOOK, "Incoming requests port number validated (%d), continuing", iPort);
+
+ string value;
+ if (getHeader(bufp, hdr_loc, CARP_ROUTED_HEADER, value)) { // if found header make sure is valid loop prevention header
+ if (value.compare("1") != 0) { // is not expected value, leave
+ TSDebug(DEBUG_TAG_HOOK, "Carp routed header not loop prevention value");
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ }
+ } else { // routed header not there, get out of here
+ TSDebug(DEBUG_TAG_HOOK, "Carp routed header not present");
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ }
+
+ if (!getHeader(bufp, hdr_loc, CARP_FORWARD_HEADER, value)) { // forward header not found, get out of here
+ TSDebug(DEBUG_TAG_HOOK, "Carp forward header not present");
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ }
+
+ bool bIsHttps = false;
+ size_t hexStringEnd = value.find("/");
+ if(hexStringEnd == string::npos) {
+ hexStringEnd = value.length();
+ } else { // header contains '/http' or '/https'
+ bIsHttps = (value.find("https",hexStringEnd) != string::npos) ? true : false;
+ }
+
+ struct sockaddr_storage sas;
+ char *ptr = reinterpret_cast<char *> (&sas);
+ for (unsigned int i = 0; i < hexStringEnd; i += 2) {
+ unsigned char ch = (getValueOfHex(value[i]) << 4) + getValueOfHex(value[i + 1]);
+ ptr[i >> 1] = ch;
+ }
+
+ string sTemp;
+ getStringFromSockaddr(reinterpret_cast<const sockaddr *> (&sas), sTemp);
+ TSDebug(DEBUG_TAG_HOOK, "Extracted sockaddr from forward header:%s", sTemp.c_str());
+
+// TSHttpTxnConfigIntSet(txnp, TS_CONFIG_HTTP_SHARE_SERVER_SESSIONS, 0); // disable conn sharing due to bug when used with TSHttpTxnServerAddrSet
+ if (TSHttpTxnServerAddrSet(txnp, reinterpret_cast<const struct sockaddr *> (&sas)) != TS_SUCCESS) {
+ TSDebug(DEBUG_TAG_HOOK, "Error calling TSHttpTxnServerAddrSet");
+ return -1;
+ }
+
+ // set scheme for connection to OS (ats will use the scheme of the carp client that connected if we don't do this)
+ // solves issue when inbound client connection scheme does not match carp peer connection scheme
+ TSMLoc url_loc;
+ if (TSHttpHdrUrlGet(bufp, hdr_loc, &url_loc) != TS_SUCCESS) {
+ TSError("carp couldn't get url");
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return -1;
+ }
+
+ if(bIsHttps) {
+ TSUrlSchemeSet(bufp, url_loc, TS_URL_SCHEME_HTTPS, TS_URL_LEN_HTTPS);
+ } else {
+ TSUrlSchemeSet(bufp, url_loc, TS_URL_SCHEME_HTTP, TS_URL_LEN_HTTP);
+ }
+
+ // request is ready for OS, do not remap
+ TSSkipRemappingSet(txnp, true);
+ removeHeader(bufp, hdr_loc, CARP_FORWARD_HEADER); // remove our header
+ removeHeader(bufp, hdr_loc, CARP_STATUS_HEADER);
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ TSDebug(DEBUG_TAG_HOOK, "Carp life should be good");
+ return 0;
+}
+
+/**
+ Process request post-remap while running post-remap mode
+ hash request, extract OS sockaddr, insert forwarding header, forward
+ */
+static int
+handlePostRemapRequestProcessing(TSCont contp, TSEvent event, void *edata)
+{
+ return handleRequestProcessing(contp, event, edata, true);
+}
+
+/**
+ This function is called to process origin responses to detect bad ppers
+ */
+static int
+handleResponseProcessing(TSCont contp, TSEvent event, void *edata)
+{
+ TSHttpTxn txnp = (TSHttpTxn) edata;
+
+ HashNode *node = static_cast<HashNode *> (TSHttpTxnArgGet(txnp, g_carpSelectedHostArgIndex));
+ if (node) { // if was forwarded by carp
+ bool bDown = false;
+ TSServerState conState = TSHttpTxnServerStateGet(txnp);
+ TSDebug(DEBUG_TAG_HOOK, "TSHttpTxnServerStateGet(txnp)=%d",conState);
+
+ // the following list of "bad" server states taken from HttpTransact::is_response_valid(State* s, HTTPHdr* incoming_response)
+ switch(conState) {
+ case TS_SRVSTATE_CONNECTION_ERROR :
+ TSDebug(DEBUG_TAG_HOOK, "Connection error");
+ bDown = true;
+ break;
+ case TS_SRVSTATE_CONNECTION_CLOSED :
+ TSDebug(DEBUG_TAG_HOOK, "Connection closed");
+ bDown = true;
+ break;
+ case TS_SRVSTATE_ACTIVE_TIMEOUT :
+ TSDebug(DEBUG_TAG_HOOK, "Active timeout");
+ bDown = true;
+ break;
+ case TS_SRVSTATE_INACTIVE_TIMEOUT :
+ TSDebug(DEBUG_TAG_HOOK, "Inactive timeout");
+ bDown = true;
+ break;
+ case TS_SRVSTATE_OPEN_RAW_ERROR :
+ TSDebug(DEBUG_TAG_HOOK, "Open raw error");
+ bDown = true;
+ break;
+ case TS_SRVSTATE_PARSE_ERROR :
+ TSDebug(DEBUG_TAG_HOOK, "Parse error");
+ bDown = true;
+ break;
+ case TS_SRVSTATE_CONGEST_CONTROL_CONGESTED_ON_F :
+ TSDebug(DEBUG_TAG_HOOK, "Congest control congested on F");
+ bDown = true;
+ break;
+ case TS_SRVSTATE_CONGEST_CONTROL_CONGESTED_ON_M :
+ TSDebug(DEBUG_TAG_HOOK, "Congest control congested on M");
+ bDown = true;
+ break;
+ default: // assume all other cases ok
+ break;
+ }
+ if(bDown) {
+ g_CarpConfigPool->getGlobalHashAlgo()->setStatus(node->name,node->listenPort,false,time(NULL),0);
+ TSDebug(DEBUG_TAG_HOOK, "marking %s as down",node->name.c_str());
+ }
+ } else {
+ TSMLoc hdr_loc;
+ TSMBuffer bufp;
+
+ if (TSHttpTxnClientReqGet(txnp, &bufp, &hdr_loc) != TS_SUCCESS) {
+ TSError("carp couldn't get request headers");
+ return -1;
+ }
+
+ string value;
+ if (getHeader(bufp, hdr_loc, CARP_STATUS_HEADER, value)) { //if found header
+ if((value.compare(CARP_FORWARDED) == 0)) {
+ removeHeader(bufp, hdr_loc, CARP_STATUS_HEADER);
+ }
+ }
+
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+
+ TSDebug(DEBUG_TAG_HOOK, "request not routed by carp");
+ }
+ return 0;
+}
+
+/**
+ This function is called on every request (when global is enabled)
+ */
+int
+carpPluginHook(TSCont contp, TSEvent event, void *edata)
+{
+ int iReturn = 0;
+ TSHttpTxn txnp = (TSHttpTxn) edata;
+
+ // handle the event for correct state
+ switch (event) {
+
+ case TS_EVENT_HTTP_READ_REQUEST_HDR:
+ TSDebug(DEBUG_TAG_HOOK, "event TS_EVENT_HTTP_READ_REQUEST_HDR");
+ if (g_CarpConfigPool->getGlobalConfig()->getMode() == CarpConfig::PRE) { // pre-remap mode
+ iReturn = handleRequestProcessing(contp, event, edata, false);
+ } else { // post remap mode
+ // check for forward header
+ iReturn = handleForwardRequestProcessing(contp, event, edata);
+ }
+ break;
+
+ case TS_EVENT_HTTP_OS_DNS: // only used in POST-REMAP mode
--- End diff --
Should this case use the `TS_HTTP_POST_REMAP` hook?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by SolidWallOfCode <gi...@git.apache.org>.
Github user SolidWallOfCode commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r75879815
--- Diff: plugins/experimental/carp/UrlComponents.h ---
@@ -0,0 +1,219 @@
+/** @file
+
+ URL helper
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+#ifndef __URLCOMPONENTS_H__
+#define __URLCOMPONENTS_H__ 1
+
+#include <cstdio>
+///////////////////////////////////////////////////////////////////////////////
+// Class holding one request URL's component.
+//
+
+struct UrlComponents
+{
+
+ UrlComponents()
+ : _port(0)
+ {
+ }
+
+ void populate(TSMBuffer bufp, TSMLoc urlLoc)
+ {
+ int scheme_len;
+ int host_len;
+ int path_len;
+ int query_len;
+ int matrix_len;
+
+ const char* scheme = TSUrlSchemeGet(bufp, urlLoc, &scheme_len);
+ const char* host = TSUrlHostGet(bufp, urlLoc, &host_len);
+ const char* path = TSUrlPathGet(bufp, urlLoc, &path_len);
+ const char* query = TSUrlHttpQueryGet(bufp, urlLoc, &query_len);
+ const char* matrix = TSUrlHttpParamsGet(bufp, urlLoc, &matrix_len);
+ _port = TSUrlPortGet(bufp, urlLoc);
+
+ _scheme.assign(scheme, scheme_len);
+ _host.assign(host, host_len);
+ _path.assign(path, path_len);
+ _query.assign(query, query_len);
+ _matrix.assign(matrix, matrix_len);
+
+ }
+
+ // get entire url (e.g.http://host/path?query)
+ void construct(std::string & url)
+ {
+ // schemeExtra= :// = 3
+ // portExtra = :xxxxx = 6
+ // just in case extra = 32
+ size_t iLen = _scheme.size() + _host.size() + _path.size() + _query.size() + _matrix.size() + 3 + 6 + 32;
+ url.reserve(iLen);
+
+ size_t pos = _host.find(":");
+ if (pos != std::string::npos) {
+ // Strip the embeded port
+ // If the port is non-standard it will be added back below
+ _host.resize(pos);
+ }
+
+ const int bitAddPort = 1;
+ const int bitAddQuery = 1 << 1;
+ const int bitAddMatrix = 1 << 2;
+ int bitField = bitAddPort; // add port by default
+ if ((_scheme.compare("http") == 0 && _port == 80) || (_scheme.compare("https") == 0 && _port == 443) ) bitField &= ~bitAddPort;
+ if (_query.size() != 0) bitField |= bitAddQuery;
+ if (_matrix.size() != 0) bitField |= bitAddMatrix;
+
+ switch (bitField) {
+ case 0: // default port, no query, no matrix
+ url = _scheme + "://" + _host + "/" + _path;
+ break;
+
+ case bitAddPort:
+ { // port, no query, no matrix
+ char sTemp[10];
+ url = _scheme + "://" + _host + ":";
+ sprintf(sTemp, "%d", _port);
+ url += sTemp;
+ url += "/" + _path;
+ break;
+ }
+
+ case bitAddQuery: // default port, with query, no matrix
+ url = _scheme + "://" + _host + "/" + _path + "?" + _query;
+ break;
+ case bitAddQuery | bitAddMatrix: // default port, with query, with matrix (even possible?)
+ url = _scheme + "://" + _host + "/" + _path + ";" + _matrix + "?" + _query;
+ break;
+
+ case bitAddMatrix: // default port, no query, with matrix
+ url = _scheme + "://" + _host + "/" + _path + ";" + _matrix;
+ break;
+
+ case bitAddPort | bitAddQuery: // port, with query, no matrix
+ { //port, with query, with matrix (even possible?)
+ char sTemp[10];
+ url = _scheme + "://" + _host + ":";
+ sprintf(sTemp, "%d", _port);
+ url += sTemp;
+ url += "/" + _path + "?" + _query;
+ break;
+ }
+
+ case bitAddPort | bitAddQuery | bitAddMatrix:
+ { //port, with query, with matrix (even possible?)
+ char sTemp[10];
+ url = _scheme + "://" + _host + ":";
+ sprintf(sTemp, "%d", _port);
+ url += sTemp;
+ url += "/" + _path + ";" + _matrix + "?" + _query;
+ break;
+ }
+
+ case bitAddPort | bitAddMatrix:
+ { // port, no query, with matrix
+ char sTemp[10];
+ url = _scheme + "://" + _host + ":";
+ sprintf(sTemp, "%d", _port);
+ url += sTemp;
+ url += "/" + _path + ";" + _matrix;
+ break;
+ }
+ }
+ }
+
+ // get path w/query or matrix
+ void getCompletePathString(std::string& p)
+ {
+ // schemeExtra= :// = 3
+ // portExtra = :xxxxx = 6
+ // just in case extra = 32
+ size_t iLen = _path.size() + _query.size() + _matrix.size() + 3 + 6 + 32;
+ p.reserve(iLen);
+
+ int bitField=0;
+ const int bitAddQuery = 1 << 1;
+ const int bitAddMatrix = 1 << 2;
+ if (_query.size() != 0) bitField |= bitAddQuery;
+ if (_matrix.size() != 0) bitField |= bitAddMatrix;
+
+ switch (bitField) {
--- End diff --
Can this be unified with the previous method, rather than duplicated?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by ericcarlschwartz <gi...@git.apache.org>.
Github user ericcarlschwartz commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r74290124
--- Diff: plugins/experimental/carp/CarpConfig.cc ---
@@ -0,0 +1,581 @@
+/** @file
+
+ Loads the CARP configuration
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+
+//////////////////////////////////////////////////////////////
+// Read CARP configuration file
+// [Servers]
+// host1.yahoo.com:4080 weight=2 # port 4080 on host1.yahoo.com with weight factor of 2
+// host2.yahoo.com # port 80 on host2.yahoo.com with (default) weight factor of 1
+//
+// [Values]
+// healthcheck={host}:8001/status.html
+// healthfreq=30
+// global=on
+//
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <memory.h>
+#include <ts/ts.h>
+#include <sys/time.h>
+
+#include <sstream>
+
+#include "CarpConfig.h"
+#include "Common.h"
+#include "CarpConfigPool.h"
+
+using namespace std;
+
+#define DEFAULT_HEALTH_CHECK_FREQ 30 // 30 second period for health checks
+#define DEFAULT_HEALTH_CHECK_PORT 80 // default to makeing healthcheck requests against port 80
+#define DEFAULT_CONFIG_RELOAD_FREQ 30 // 30 seconds used in TSContSchedule
+#define DEFAULT_PORT 80 // default to makeing requests against port 80
+#define DEFAULT_WEIGHT 1 // default weight
+#define DEFAULT_SCHEME "http"
+#define DEFAULT_REPLICATION_FACTOR 1
+
+// config section headers
+static const char *const SECTION_SERVERS_STR = "[Servers]";
+static const char *const SECTION_VALUES_STR = "[Values]";
+
+// key strings
+static const char *const KEY_HEALTHCHECK_STR = "healthcheck";
+static const char *const KEY_HEALTHFREQ_STR = "healthfreq";
+static const char *const KEY_RELOADFREQ_STR = "reloadfreq";
+static const char *const KEY_HCTIMEOUT_STR = "hctimeout";
+static const char *const KEY_BLACKLIST_STR = "blacklist";
+static const char *const KEY_WHITELIST_STR = "whitelist";
+static const char *const KEY_MODE_STR = "mode";
+static const char *const KEY_ALLOWFWDPORT_STR = "allowfwdport";
+static const char *const KEY_REPLICATIONFACTOR_STR = "replicationfactor";
+
+// parameter strings
+static const char *const WEIGHT_EQUALS_STRING = "weight=";
+static const char *const GROUP_EQUALS_STRING = "group=";
+static const char *const KEY_MODE_PREREMAP_STR = "pre-remap";
+static const char *const KEY_MODE_POSTREMAP_STR = "post-remap";
+
+
+/**********************************************************/
+bool
+getInt(char** pptr, int *val)
--- End diff --
Think it's done with this function because it basically grabs the next integer and advances the pointer and can then be called again with the new pointer to get the next one.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by SolidWallOfCode <gi...@git.apache.org>.
Github user SolidWallOfCode commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r75863547
--- Diff: plugins/experimental/carp/carp.cc ---
@@ -0,0 +1,713 @@
+/** @file
+
+ A brief file description
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+
+#include <errno.h>
+
+#include <string>
+#include <sstream>
+#include <stdlib.h>
+#include <stdio.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+#include <unistd.h>
+
+#include <memory.h>
+
+#include <ts/ts.h>
+
+#include "Common.h"
+#include "CarpConfig.h"
+#include "CarpConfigPool.h"
+#include "CarpHashAlgorithm.h"
+#include "UrlComponents.h"
+
+using namespace std;
+
+CarpConfigPool* g_CarpConfigPool = NULL;
+int g_carpSelectedHostArgIndex = 0;
+TSTextLogObject g_logObject = NULL;
+
+const char *logFileName = "carp";
+
+//////////////////////////////////////////////////////////////////////////////
+//////////////////////////////////////////////////////////////////////////////
+/*
+ check for our carp routed header, dump status if requested
+ */
+static int
+processCarpRoutedHeader(TSHttpTxn txnp, TSMBuffer bufp, TSMLoc hdr_loc)
+{
+ string value;
+ if (getHeader(bufp, hdr_loc, CARP_ROUTED_HEADER, value)) { // if found header
+ if (value.compare("1") == 0) { // is loop prevention value
+ TSDebug(DEBUG_TAG_HOOK, "Found %s header with loop prevention value, not forwarding again", CARP_ROUTED_HEADER.c_str());
+ return 0;
+ } else if (value.compare("dump") == 0) { // is dump status request
+ TSDebug(DEBUG_TAG_HOOK, "Found %s header with dump request", CARP_ROUTED_HEADER.c_str());
+ string status;
+ g_CarpConfigPool->getGlobalHashAlgo()->dump(status);
+ TSHttpTxnSetHttpRetStatus(txnp, TS_HTTP_STATUS_MULTI_STATUS);
+ TSHttpTxnErrorBodySet(txnp, TSstrdup(status.c_str()), status.length(), NULL);
+ return -1;
+ }
+ TSDebug(DEBUG_TAG_HOOK, "Found %s header with unknown value of %s, ignoring", CARP_ROUTED_HEADER.c_str(), value.c_str());
+ removeHeader(bufp, hdr_loc, CARP_ROUTED_HEADER);
+ }
+ return 1; // all OK
+}
+
+static bool
+checkListForSelf(std::vector<HashNode *> list)
+{
+ for (size_t k = 0; k < list.size(); k++) {
+ if (list[k]->isSelf) return true;
+ }
+ return false;
+}
+
+/**
+ bIsPOSTRemap = false --- Hash request and forward to peer
+ bIsPOSTRemap = true --- hash request, extract OS sockaddr, insert forwarding header, forward
+ */
+static int
+handleRequestProcessing(TSCont contp, TSEvent event, void *edata, bool bIsPOSTRemap)
+{
+ TSHttpTxn txnp = (TSHttpTxn) edata;
+ TSMBuffer bufp;
+ TSMLoc hdr_loc;
+ TSMLoc url_loc;
+
+ // get the client request so we can get URL and add header
+ if (TSHttpTxnClientReqGet(txnp, &bufp, &hdr_loc) != TS_SUCCESS) {
+ TSError("carp couldn't get request headers");
+ return -1;
+ }
+
+ int method_len;
+ const char *method = TSHttpHdrMethodGet(bufp, hdr_loc, &method_len);
+ if (NULL == method) {
+ TSError("carp couldn't get http method");
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return -1;
+ }
+ if (((method_len == TS_HTTP_LEN_DELETE) && (strncasecmp(method, TS_HTTP_METHOD_DELETE, TS_HTTP_LEN_DELETE) == 0)) ||
+ ((method_len == TS_HTTP_LEN_PURGE) && (strncasecmp(method, TS_HTTP_METHOD_PURGE, TS_HTTP_LEN_PURGE) == 0))) {
+ TSDebug(DEBUG_TAG_HOOK, "Request method is '%s' so not routing request", string(method,method_len).c_str());
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ }
+
+ if (TSHttpHdrUrlGet(bufp, hdr_loc, &url_loc) != TS_SUCCESS) {
+ TSError("carp couldn't get url");
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return -1;
+ }
+ // if has carp loop prevention header, do not remap
+ int iTemp = processCarpRoutedHeader(txnp, bufp, hdr_loc);
+ if(iTemp <= 0) { // if dump or do not remap
+ // check origin client request's scheme for premap mode
+ if (!bIsPOSTRemap) {
+ string oriScheme;
+ if (!getHeader(bufp, hdr_loc, CARP_PREMAP_SCHEME, oriScheme)) {
+ TSDebug(DEBUG_TAG_HOOK, "couldn't get '%s' header", CARP_PREMAP_SCHEME.c_str());
+ } else {
+ bool isHttps = (oriScheme == TS_URL_SCHEME_HTTPS);
+
+ if (isHttps) {
+ TSUrlSchemeSet(bufp, url_loc, TS_URL_SCHEME_HTTPS, TS_URL_LEN_HTTPS);
+ } else {
+ TSUrlSchemeSet(bufp, url_loc, TS_URL_SCHEME_HTTP, TS_URL_LEN_HTTP);
+ }
+
+ removeHeader(bufp, hdr_loc, CARP_STATUS_HEADER);
+ removeHeader(bufp, hdr_loc, CARP_ROUTED_HEADER);
+ removeHeader(bufp, hdr_loc, CARP_PREMAP_SCHEME.c_str());
+ TSDebug(DEBUG_TAG_HOOK, "Set client request's scheme to %s through %s header",
+ isHttps?"https":"http", CARP_PREMAP_SCHEME.c_str());
+ }
+ } else {
+ removeHeader(bufp, hdr_loc, CARP_ROUTED_HEADER);
+ }
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return iTemp;
+ }
+
+ UrlComponents reqUrl;
+ reqUrl.populate(bufp, url_loc);
+ // the url ONLY used to determine the cache owner
+ string sUrl;
+
+ if (!bIsPOSTRemap) { // if pre-remap, then host not in URL so get from header
+ string sHost;
+ if (!getHeader(bufp, hdr_loc, TS_MIME_FIELD_HOST, sHost)) {
+ TSDebug(DEBUG_TAG_HOOK, "Could not find host header, ignoring it");
+ }
+ reqUrl.setHost(sHost);
+
+ //[YTSATS-836] heuristically ignore the scheme and port when calculate cache owner
+ UrlComponents normalizedUrl = reqUrl;
+ normalizedUrl.setScheme(CARP_SCHEME_FOR_HASH);
+ normalizedUrl.setPort(CARP_PORT_FOR_HASH);
+ normalizedUrl.construct(sUrl);
+ } else {
+ reqUrl.construct(sUrl);
+ }
+
+
+ if (g_CarpConfigPool->getGlobalConfig()->hasWhiteList()) {
+ string sCarpable;
+ if (!getHeader(bufp, hdr_loc, CARPABLE_HEADER, sCarpable)) { // if no carpable header check whitelist
+ if (!g_CarpConfigPool->getGlobalConfig()->isWhiteListed(reqUrl.getHost())) { // if white list exists, then host must be present
+ TSDebug(DEBUG_TAG_HOOK, "Host '%s' is not whitelisted, not going through carp", reqUrl.getHost().c_str());
+ TSHandleMLocRelease(bufp, hdr_loc, url_loc);
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ } else {
+ TSDebug(DEBUG_TAG_HOOK, "Found host (%s) whitelisted, routing...",reqUrl.getHost().c_str());
+ }
+ } else { // found carpable header, make sure it's 1
+ if (sCarpable.compare("1") != 0) { // carpable header present but not 0, be strict and do not forward request
+ TSDebug(DEBUG_TAG_HOOK, "Carpable (%s) present but value not acceptable (%s)", CARPABLE_HEADER.c_str(), sCarpable.c_str());
+ TSHandleMLocRelease(bufp, hdr_loc, url_loc);
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ } else {
+ TSDebug(DEBUG_TAG_HOOK, "Found Carpable header, routing...");
+ }
+ }
+ } else { // no whitelist so blacklist could be used
+ if (g_CarpConfigPool->getGlobalConfig()->isBlackListed(reqUrl.getHost())) { // if host black listed, do not carp
+ TSDebug(DEBUG_TAG_HOOK, "Host '%s' is blacklisted, not going through carp", reqUrl.getHost().c_str());
+ TSHandleMLocRelease(bufp, hdr_loc, url_loc);
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ }
+ }
+
+ TSDebug(DEBUG_TAG_HOOK, "URL to hash with=%s", sUrl.c_str());
+
+ // get nodeList and select node to forward to
+ std::vector<HashNode *> nodeList = g_CarpConfigPool->getGlobalHashAlgo()->getRemapProxyList(sUrl);
+ bool bAddHeaderResult=false;
+ bool bAddForwardedResult = false;
+ HashNode* node = NULL;
+ bool bIsOwner = false;
+
+ if (nodeList.size() == 0) { // no hosts available to forward to
+ TSDebug(DEBUG_TAG_HOOK, "no hosts available to forward to, will handle locally");
+ goto done;
+ } else {
+ node = nodeList[0];
+ }
+
+ bIsOwner = checkListForSelf(nodeList);
+ for (size_t k = 0; k < nodeList.size(); k++) {
+ TSDebug(DEBUG_TAG_HOOK, "nodeList host %d name is %s", static_cast<int>(k), nodeList[k]->name.c_str());
+ }
+
+ //handle forwarding
+ TSDebug(DEBUG_TAG_HOOK, "forwarding to '%s' (isSelf=%d)", node->name.c_str(), node->isSelf);
+ if (!node->isSelf) { // carp does not forward if we choose ourself
+ node->carpForward();
+ TSDebug(DEBUG_TAG_HOOK, "carp forwarded to %s.", node->name.c_str());
+ // insert carp loop prevention header
+ bAddHeaderResult = addHeader(bufp, hdr_loc, CARP_ROUTED_HEADER, string("1"));
+ if (!bAddHeaderResult) {
+ TSError("Carp, error inserting '%s' header", CARP_ROUTED_HEADER.c_str());
+ }
+ bAddForwardedResult = addHeader(bufp, hdr_loc, CARP_STATUS_HEADER, string(CARP_FORWARDED));
+ if (!bAddForwardedResult) {
+ TSError("Carp, error inserting '%s' header", CARP_STATUS_HEADER.c_str());
+ }
+
+ if (bIsPOSTRemap) { // if post remap, get remapped/OS Server Addr and add as header
+ const struct sockaddr* sa = TSHttpTxnServerAddrGet(txnp);
+ // const struct sockaddr* sa = TSHttpTxnNextHopAddrGet(txnp);
+ if (sa) { // sanity check
+ struct sockaddr_storage ss;
+ memcpy(static_cast<void *> (&ss), sa, sizeof (sockaddr_storage));
+ if ((reinterpret_cast<sockaddr_in *> (&ss))->sin_port == 0) { // set port from client request URL
+ (reinterpret_cast<sockaddr_in *> (&ss))->sin_port = htons(reqUrl.getPort());
+ }
+
+ string sTemp;
+ getStringFromSockaddr(reinterpret_cast<const sockaddr *> (&ss), sTemp);
+ TSDebug(DEBUG_TAG_HOOK, "Inserting forward header with sockaddr:%s", sTemp.c_str());
+ string sSockaddr;
+ sSockaddr.reserve(32 + sizeof (sockaddr_storage) * 2);
+ for (unsigned int i = 0; i<sizeof (sockaddr_storage); i++) {
+ char val[8];
+ sprintf(val, "%02X", reinterpret_cast<const unsigned char *> (&ss)[i]);
+ sSockaddr += val;
+ }
+ sSockaddr += "/" + reqUrl.getScheme();
+ // insert carp forwarding header
+ bool bAddFwdHeaderResult = addHeader(bufp, hdr_loc, CARP_FORWARD_HEADER, sSockaddr);
+ if (!bAddFwdHeaderResult) {
+ TSError("Carp, error inserting '%s' header", CARP_FORWARD_HEADER.c_str());
+ }
+ }
+ } else { // for premap mode
+ string sScheme = reqUrl.getScheme();
+
+ if (!addHeader(bufp, hdr_loc, CARP_PREMAP_SCHEME, sScheme)) {
+ TSError("Carp, error inserting '%s' header in premap mode", CARP_PREMAP_SCHEME.c_str());
+ } else {
+ TSDebug(DEBUG_TAG_HOOK, "Insert client request scheme %s in premap mode", sScheme.c_str());
+ }
+ }
+ // set origin server/destination
+ // TSHttpTxnConfigIntSet(txnp, TS_CONFIG_HTTP_SHARE_SERVER_SESSIONS, 0); // disable conn sharing due to bug when used with TSHttpTxnServerAddrSet
+ if (TSHttpTxnServerAddrSet(txnp, reinterpret_cast<const struct sockaddr *> (&node->forwardAddr)) != TS_SUCCESS) {
+ TSDebug(DEBUG_TAG_HOOK, "Error calling TSHttpTxnServerAddrSet");
+ } else {
+ // set scheme appropriately based on destination
+ TSDebug(DEBUG_TAG_HOOK, "Setting scheme to '%s'", node->getSchemeString());
+ TSUrlSchemeSet(bufp, url_loc, node->getSchemeString(), -1);
+ if (!bIsPOSTRemap) { // since we are forwarding, do not remap request and do not cache result
+ TSSkipRemappingSet(txnp, true);
+ }
+ TSHttpTxnArgSet(txnp, g_carpSelectedHostArgIndex, static_cast<void *> (node));
+ if (!bIsOwner) {
+ TSHttpTxnServerRespNoStoreSet(txnp, 1); // do not cache the response
+ } else {
+ TSHttpTxnServerRespNoStoreSet(txnp, 0); // we are replicate owner, cache response
+ }
+ }
+ } else {
+ node->carpNoForward();
+ TSDebug(DEBUG_TAG_HOOK, "carp forwarded to self.");
+ }
+
+done :
+ // done w/buffers, release them
+ TSHandleMLocRelease(bufp, hdr_loc, url_loc);
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+}
+
+/**
+ * Convert ASCII hex digit to value of hex digit
+ * @param ch
+ * @return
+ */
+static unsigned char
+getValueOfHex(unsigned char ch)
+{
+ ch -= '0';
+ if(ch > 9) ch -= 7; // 'A' - ':' = 7
+ return ch;
+}
+
+/**
+ Process request pre-remap while running post-remap mode
+ if forwarding header present and rules met, disable remap processing, forward to OS
+ */
+static int
+handleForwardRequestProcessing(TSCont contp, TSEvent event, void *edata)
+{
+ // we only want to do anything here if CARP_ROUTED_HEADER present
+ // and CARP_FORWARD_HEADER is present
+ // and incoming request is from port getAllowedForwardPort()
+ TSHttpTxn txnp = (TSHttpTxn) edata;
+ TSMBuffer bufp;
+ TSMLoc hdr_loc;
+
+ // get the client request
+ if (TSHttpTxnClientReqGet(txnp, &bufp, &hdr_loc) != TS_SUCCESS) {
+ TSError("carp couldn't get request headers");
+ return -1;
+ }
+
+ // if has carp loop prevention header, do not remap
+ int iTemp = processCarpRoutedHeader(txnp, bufp, hdr_loc);
+ if(iTemp == -1 || iTemp == 1) { // is dump request or routed header not present
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return (iTemp == 1) ? 0 : iTemp; // return 0 if routed header not present
+ }
+
+ // check incoming port number first
+ const struct sockaddr_in* sa = reinterpret_cast<const sockaddr_in *>(TSHttpTxnIncomingAddrGet(txnp));
+ if(!sa) {
+ TSDebug(DEBUG_TAG_HOOK, "TSHttpTxnIncomingAddrGet() returned NULL");
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ }
+
+ int iPort = ntohs(sa->sin_port);
+ if(iPort != g_CarpConfigPool->getGlobalConfig()->getAllowedForwardPort()) {
+ TSDebug(DEBUG_TAG_HOOK, "Allowed forward port does not match. Incoming request on %d, but configured for %d.",iPort, g_CarpConfigPool->getGlobalConfig()->getAllowedForwardPort());
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ }
+
+ TSDebug(DEBUG_TAG_HOOK, "Incoming requests port number validated (%d), continuing", iPort);
+
+ string value;
+ if (getHeader(bufp, hdr_loc, CARP_ROUTED_HEADER, value)) { // if found header make sure is valid loop prevention header
+ if (value.compare("1") != 0) { // is not expected value, leave
+ TSDebug(DEBUG_TAG_HOOK, "Carp routed header not loop prevention value");
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ }
+ } else { // routed header not there, get out of here
+ TSDebug(DEBUG_TAG_HOOK, "Carp routed header not present");
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ }
+
+ if (!getHeader(bufp, hdr_loc, CARP_FORWARD_HEADER, value)) { // forward header not found, get out of here
--- End diff --
Wasn't this just checked?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by ericcarlschwartz <gi...@git.apache.org>.
Github user ericcarlschwartz commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r75918715
--- Diff: plugins/experimental/carp/carp.cc ---
@@ -0,0 +1,713 @@
+/** @file
+
+ A brief file description
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+
+#include <errno.h>
+
+#include <string>
+#include <sstream>
+#include <stdlib.h>
+#include <stdio.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+#include <unistd.h>
+
+#include <memory.h>
+
+#include <ts/ts.h>
+
+#include "Common.h"
+#include "CarpConfig.h"
+#include "CarpConfigPool.h"
+#include "CarpHashAlgorithm.h"
+#include "UrlComponents.h"
+
+using namespace std;
+
+CarpConfigPool* g_CarpConfigPool = NULL;
+int g_carpSelectedHostArgIndex = 0;
+TSTextLogObject g_logObject = NULL;
+
+const char *logFileName = "carp";
+
+//////////////////////////////////////////////////////////////////////////////
+//////////////////////////////////////////////////////////////////////////////
+/*
+ check for our carp routed header, dump status if requested
+ */
+static int
+processCarpRoutedHeader(TSHttpTxn txnp, TSMBuffer bufp, TSMLoc hdr_loc)
+{
+ string value;
+ if (getHeader(bufp, hdr_loc, CARP_ROUTED_HEADER, value)) { // if found header
+ if (value.compare("1") == 0) { // is loop prevention value
+ TSDebug(DEBUG_TAG_HOOK, "Found %s header with loop prevention value, not forwarding again", CARP_ROUTED_HEADER.c_str());
+ return 0;
+ } else if (value.compare("dump") == 0) { // is dump status request
+ TSDebug(DEBUG_TAG_HOOK, "Found %s header with dump request", CARP_ROUTED_HEADER.c_str());
+ string status;
+ g_CarpConfigPool->getGlobalHashAlgo()->dump(status);
+ TSHttpTxnSetHttpRetStatus(txnp, TS_HTTP_STATUS_MULTI_STATUS);
+ TSHttpTxnErrorBodySet(txnp, TSstrdup(status.c_str()), status.length(), NULL);
+ return -1;
+ }
+ TSDebug(DEBUG_TAG_HOOK, "Found %s header with unknown value of %s, ignoring", CARP_ROUTED_HEADER.c_str(), value.c_str());
+ removeHeader(bufp, hdr_loc, CARP_ROUTED_HEADER);
+ }
+ return 1; // all OK
+}
+
+static bool
+checkListForSelf(std::vector<HashNode *> list)
+{
+ for (size_t k = 0; k < list.size(); k++) {
+ if (list[k]->isSelf) return true;
+ }
+ return false;
+}
+
+/**
+ bIsPOSTRemap = false --- Hash request and forward to peer
+ bIsPOSTRemap = true --- hash request, extract OS sockaddr, insert forwarding header, forward
+ */
+static int
+handleRequestProcessing(TSCont contp, TSEvent event, void *edata, bool bIsPOSTRemap)
+{
+ TSHttpTxn txnp = (TSHttpTxn) edata;
+ TSMBuffer bufp;
+ TSMLoc hdr_loc;
+ TSMLoc url_loc;
+
+ // get the client request so we can get URL and add header
+ if (TSHttpTxnClientReqGet(txnp, &bufp, &hdr_loc) != TS_SUCCESS) {
+ TSError("carp couldn't get request headers");
+ return -1;
+ }
+
+ int method_len;
+ const char *method = TSHttpHdrMethodGet(bufp, hdr_loc, &method_len);
+ if (NULL == method) {
+ TSError("carp couldn't get http method");
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return -1;
+ }
+ if (((method_len == TS_HTTP_LEN_DELETE) && (strncasecmp(method, TS_HTTP_METHOD_DELETE, TS_HTTP_LEN_DELETE) == 0)) ||
+ ((method_len == TS_HTTP_LEN_PURGE) && (strncasecmp(method, TS_HTTP_METHOD_PURGE, TS_HTTP_LEN_PURGE) == 0))) {
+ TSDebug(DEBUG_TAG_HOOK, "Request method is '%s' so not routing request", string(method,method_len).c_str());
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ }
+
+ if (TSHttpHdrUrlGet(bufp, hdr_loc, &url_loc) != TS_SUCCESS) {
+ TSError("carp couldn't get url");
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return -1;
+ }
+ // if has carp loop prevention header, do not remap
+ int iTemp = processCarpRoutedHeader(txnp, bufp, hdr_loc);
+ if(iTemp <= 0) { // if dump or do not remap
+ // check origin client request's scheme for premap mode
+ if (!bIsPOSTRemap) {
+ string oriScheme;
+ if (!getHeader(bufp, hdr_loc, CARP_PREMAP_SCHEME, oriScheme)) {
+ TSDebug(DEBUG_TAG_HOOK, "couldn't get '%s' header", CARP_PREMAP_SCHEME.c_str());
+ } else {
+ bool isHttps = (oriScheme == TS_URL_SCHEME_HTTPS);
+
+ if (isHttps) {
+ TSUrlSchemeSet(bufp, url_loc, TS_URL_SCHEME_HTTPS, TS_URL_LEN_HTTPS);
+ } else {
+ TSUrlSchemeSet(bufp, url_loc, TS_URL_SCHEME_HTTP, TS_URL_LEN_HTTP);
+ }
+
+ removeHeader(bufp, hdr_loc, CARP_STATUS_HEADER);
+ removeHeader(bufp, hdr_loc, CARP_ROUTED_HEADER);
--- End diff --
Will do. I'm going to do a big clang-format of this after addressing your comments because the one I thought I did doesn't seem to have worked.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by zwoop <gi...@git.apache.org>.
Github user zwoop commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r74176854
--- Diff: plugins/experimental/carp/CarpConfig.cc ---
@@ -0,0 +1,581 @@
+/** @file
+
+ Loads the CARP configuration
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+
+//////////////////////////////////////////////////////////////
+// Read CARP configuration file
+// [Servers]
+// host1.yahoo.com:4080 weight=2 # port 4080 on host1.yahoo.com with weight factor of 2
+// host2.yahoo.com # port 80 on host2.yahoo.com with (default) weight factor of 1
+//
+// [Values]
+// healthcheck={host}:8001/status.html
+// healthfreq=30
+// global=on
+//
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <memory.h>
+#include <ts/ts.h>
+#include <sys/time.h>
+
+#include <sstream>
+
+#include "CarpConfig.h"
+#include "Common.h"
+#include "CarpConfigPool.h"
+
+using namespace std;
+
+#define DEFAULT_HEALTH_CHECK_FREQ 30 // 30 second period for health checks
+#define DEFAULT_HEALTH_CHECK_PORT 80 // default to makeing healthcheck requests against port 80
+#define DEFAULT_CONFIG_RELOAD_FREQ 30 // 30 seconds used in TSContSchedule
+#define DEFAULT_PORT 80 // default to makeing requests against port 80
+#define DEFAULT_WEIGHT 1 // default weight
+#define DEFAULT_SCHEME "http"
+#define DEFAULT_REPLICATION_FACTOR 1
+
+// config section headers
+static const char *const SECTION_SERVERS_STR = "[Servers]";
+static const char *const SECTION_VALUES_STR = "[Values]";
+
+// key strings
+static const char *const KEY_HEALTHCHECK_STR = "healthcheck";
+static const char *const KEY_HEALTHFREQ_STR = "healthfreq";
+static const char *const KEY_RELOADFREQ_STR = "reloadfreq";
+static const char *const KEY_HCTIMEOUT_STR = "hctimeout";
+static const char *const KEY_BLACKLIST_STR = "blacklist";
+static const char *const KEY_WHITELIST_STR = "whitelist";
+static const char *const KEY_MODE_STR = "mode";
+static const char *const KEY_ALLOWFWDPORT_STR = "allowfwdport";
+static const char *const KEY_REPLICATIONFACTOR_STR = "replicationfactor";
+
+// parameter strings
+static const char *const WEIGHT_EQUALS_STRING = "weight=";
+static const char *const GROUP_EQUALS_STRING = "group=";
+static const char *const KEY_MODE_PREREMAP_STR = "pre-remap";
+static const char *const KEY_MODE_POSTREMAP_STR = "post-remap";
+
+
+/**********************************************************/
+bool
+getInt(char** pptr, int *val)
+{
+ bool bReturn = false;
+ int v=0;
+
+ char* ptr = *pptr;
+ // skip white space if any
+ while (*ptr && isspace(*ptr)) ++ptr;
+ // get digits
+ if (*ptr) {
+ while (*ptr && isdigit(*ptr)) {
+ v *= 10;
+ v += (*ptr)-'0';
+ ++ptr;
+ bReturn = true;
+ }
+ }
+ if (bReturn) {
+ *pptr = ptr;
+ *val = v;
+ }
+
+ return bReturn;
+}
+
+/**********************************************************/
+// [http[s]://]host[:port]/path
+bool
+getHostAndPort(char** pptr,string* sHost,int* iPort, string* sScheme)
+{
+ bool bReturn = false;
+ char* ptr = *pptr;
+
+ *iPort = 80;
+ *sScheme = DEFAULT_SCHEME;
+ if(strncmp(*pptr,"https",5) == 0) {
+ *iPort = 443;
+ *sScheme = "https";
+ }
+
--- End diff --
What amc is saying is to use TS_URL_SCHEME_HTTPS and TS_URL_LEN_HTTPS.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver issue #843: [TS-4723] ATS CARP Plugin
Posted by bryancall <gi...@git.apache.org>.
Github user bryancall commented on the issue:
https://github.com/apache/trafficserver/pull/843
We can reopen this if needed after the Denver meetup on CARP and Parent Selection.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by SolidWallOfCode <gi...@git.apache.org>.
Github user SolidWallOfCode commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r75853501
--- Diff: plugins/experimental/carp/carp.cc ---
@@ -0,0 +1,713 @@
+/** @file
+
+ A brief file description
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+
+#include <errno.h>
+
+#include <string>
+#include <sstream>
+#include <stdlib.h>
+#include <stdio.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+#include <unistd.h>
+
+#include <memory.h>
+
+#include <ts/ts.h>
+
+#include "Common.h"
+#include "CarpConfig.h"
+#include "CarpConfigPool.h"
+#include "CarpHashAlgorithm.h"
+#include "UrlComponents.h"
+
+using namespace std;
+
+CarpConfigPool* g_CarpConfigPool = NULL;
+int g_carpSelectedHostArgIndex = 0;
+TSTextLogObject g_logObject = NULL;
+
+const char *logFileName = "carp";
+
+//////////////////////////////////////////////////////////////////////////////
+//////////////////////////////////////////////////////////////////////////////
+/*
+ check for our carp routed header, dump status if requested
+ */
+static int
+processCarpRoutedHeader(TSHttpTxn txnp, TSMBuffer bufp, TSMLoc hdr_loc)
+{
+ string value;
+ if (getHeader(bufp, hdr_loc, CARP_ROUTED_HEADER, value)) { // if found header
+ if (value.compare("1") == 0) { // is loop prevention value
+ TSDebug(DEBUG_TAG_HOOK, "Found %s header with loop prevention value, not forwarding again", CARP_ROUTED_HEADER.c_str());
+ return 0;
+ } else if (value.compare("dump") == 0) { // is dump status request
+ TSDebug(DEBUG_TAG_HOOK, "Found %s header with dump request", CARP_ROUTED_HEADER.c_str());
+ string status;
+ g_CarpConfigPool->getGlobalHashAlgo()->dump(status);
+ TSHttpTxnSetHttpRetStatus(txnp, TS_HTTP_STATUS_MULTI_STATUS);
+ TSHttpTxnErrorBodySet(txnp, TSstrdup(status.c_str()), status.length(), NULL);
+ return -1;
+ }
+ TSDebug(DEBUG_TAG_HOOK, "Found %s header with unknown value of %s, ignoring", CARP_ROUTED_HEADER.c_str(), value.c_str());
+ removeHeader(bufp, hdr_loc, CARP_ROUTED_HEADER);
+ }
+ return 1; // all OK
+}
+
+static bool
+checkListForSelf(std::vector<HashNode *> list)
+{
+ for (size_t k = 0; k < list.size(); k++) {
+ if (list[k]->isSelf) return true;
+ }
+ return false;
+}
+
+/**
+ bIsPOSTRemap = false --- Hash request and forward to peer
+ bIsPOSTRemap = true --- hash request, extract OS sockaddr, insert forwarding header, forward
+ */
+static int
+handleRequestProcessing(TSCont contp, TSEvent event, void *edata, bool bIsPOSTRemap)
+{
+ TSHttpTxn txnp = (TSHttpTxn) edata;
+ TSMBuffer bufp;
+ TSMLoc hdr_loc;
+ TSMLoc url_loc;
+
+ // get the client request so we can get URL and add header
+ if (TSHttpTxnClientReqGet(txnp, &bufp, &hdr_loc) != TS_SUCCESS) {
+ TSError("carp couldn't get request headers");
+ return -1;
+ }
+
+ int method_len;
+ const char *method = TSHttpHdrMethodGet(bufp, hdr_loc, &method_len);
+ if (NULL == method) {
+ TSError("carp couldn't get http method");
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return -1;
+ }
+ if (((method_len == TS_HTTP_LEN_DELETE) && (strncasecmp(method, TS_HTTP_METHOD_DELETE, TS_HTTP_LEN_DELETE) == 0)) ||
+ ((method_len == TS_HTTP_LEN_PURGE) && (strncasecmp(method, TS_HTTP_METHOD_PURGE, TS_HTTP_LEN_PURGE) == 0))) {
+ TSDebug(DEBUG_TAG_HOOK, "Request method is '%s' so not routing request", string(method,method_len).c_str());
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ }
+
+ if (TSHttpHdrUrlGet(bufp, hdr_loc, &url_loc) != TS_SUCCESS) {
+ TSError("carp couldn't get url");
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return -1;
+ }
+ // if has carp loop prevention header, do not remap
+ int iTemp = processCarpRoutedHeader(txnp, bufp, hdr_loc);
+ if(iTemp <= 0) { // if dump or do not remap
+ // check origin client request's scheme for premap mode
+ if (!bIsPOSTRemap) {
+ string oriScheme;
+ if (!getHeader(bufp, hdr_loc, CARP_PREMAP_SCHEME, oriScheme)) {
+ TSDebug(DEBUG_TAG_HOOK, "couldn't get '%s' header", CARP_PREMAP_SCHEME.c_str());
+ } else {
+ bool isHttps = (oriScheme == TS_URL_SCHEME_HTTPS);
+
+ if (isHttps) {
+ TSUrlSchemeSet(bufp, url_loc, TS_URL_SCHEME_HTTPS, TS_URL_LEN_HTTPS);
+ } else {
+ TSUrlSchemeSet(bufp, url_loc, TS_URL_SCHEME_HTTP, TS_URL_LEN_HTTP);
+ }
+
+ removeHeader(bufp, hdr_loc, CARP_STATUS_HEADER);
+ removeHeader(bufp, hdr_loc, CARP_ROUTED_HEADER);
--- End diff --
Fix indentation.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by SolidWallOfCode <gi...@git.apache.org>.
Github user SolidWallOfCode commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r74271671
--- Diff: plugins/experimental/carp/CarpHashAlgorithm.cc ---
@@ -0,0 +1,396 @@
+/** @file
+
+ Implements the CARP hash algorithm
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+#include <ts/ts.h>
+#include <stdio.h>
+#include <memory.h>
+#include <list>
+
+#include <iostream>
+#include <sstream>
+
+#include "Common.h"
+#include "CarpHashAlgorithm.h"
+
+using namespace std;
+
+/*****************************************************************/
+void
+HashNode::dump(string& s)
+{
+ stringstream ss;
+ string sSockaddr;
+ getStringFromSockaddr(reinterpret_cast<const struct sockaddr *>( &forwardAddr), sSockaddr);
+
+ ss << scheme << "://" << name << ":"<<listenPort<<" ("<<sSockaddr<<") weight:"<<weight<< (_status ? string(" UP ") : string(" DOWN "));
+ if(_statusTime) {
+ ss << "(" << time(NULL)-_statusTime<< "s ago in "<< _statusLatencyMs << "mS)";
+ }
+ ss << " hits:" << _hits;
+ ss << " carp_noforwarded:" << _carp_noforwarded;
+ ss << " carp_forwarded:" << _carp_forwarded;
+ ss << endl;
+ s += ss.str();
+}
+
+/*****************************************************************/
+void
+HashAlgorithm::addHost(std::string name, unsigned int port, std::string scheme, double weight, bool self, struct sockaddr_storage fwdAddr)
+{
+ HashNode* node = new HashNode(name, port, scheme, weight, self, fwdAddr);
+ TSAssert(NULL != node);
+ addHost(node);
+}
+
+/*****************************************************************/
+void
+HashAlgorithm::addHost(HashNode* node)
+{
+ if (!node) return;
+ _hostList.push_back(node);
+}
+
+/*****************************************************************/
+HashNode*
+HashAlgorithm::findStatusByNameAndPort(const string& name, unsigned int port,
+ size_t* index) {
+ /*
+ * Todo: This use loop to find the corresponding HashNode
+ * But the HttpClient and the Hash was related, so we
+ * could use more easier method to write the status
+ */
+ for (size_t ptr = 0; ptr < _hostList.size(); ptr++) {
+ if (_hostList[ptr]->listenPort == port
+ && _hostList[ptr]->name.compare(name) == 0) { // found it
+ if (index) {
+ *index = ptr;
+ }
+ return _hostList[ptr];
+ }
+ }
+ return NULL;
+}
+
+size_t
+HashAlgorithm::findHashNodeIndex(HashNode *node) {
+ for (size_t ptr = 0; ptr < _hostList.size(); ptr++) {
+ if ( _hostList[ptr] == node) {
+ return ptr;
+ }
+ }
+ return -1;
+}
+
+/*****************************************************************/
+void
+HashAlgorithm::setStatus(const string& name, unsigned int port, bool status, time_t time, uint64_t latencyMs)
+{
+ TSDebug(DEBUG_TAG_INIT, "HashAlgorithm::setStatus name=%s status=%d", name.c_str(), status);
+
+ HashNode* node = findStatusByNameAndPort(name,port);
+ if(node) {
+ node->setStatus(status,time,latencyMs);
+ } else {
+ TSError("Carp internal error setStatus host %s not found",name.c_str());
+ }
+}
+
+void
+HashAlgorithm::setStatus(HashNode * node, bool status, time_t time, uint64_t latencyMs)
+{
+ TSDebug(DEBUG_TAG_INIT, "HashAlgorithm::setStatus name=%s status=%d", node->name.c_str(), status);
+
+// HashNode* node = findStatusByNameAndPort(name,port);
+ if(node) {
+ node->setStatus(status,time,latencyMs);
+ } else {
+ TSError("Carp internal error setStatus host %s not found",node->name.c_str());
+ }
+}
+
+/*****************************************************************/
+HashAlgorithm::~HashAlgorithm()
+{
+ for (size_t ptr = 0; ptr < _hostList.size(); ptr++) {
+ delete _hostList[ptr];
+ }
+}
+
+/*****************************************************************/
+void
+HashAlgorithm::dump(string& s)
+{
+ _config->dump(s);
+
+ stringstream ss;
+ ss << "Hash Algo stats:" << endl;
+ s += ss.str();
+
+ for (size_t ptr = 0; ptr < _hostList.size(); ptr++) {
+ ss.str("");
+ ss.clear();
+ ss << ptr << "-";
+ s += ss.str();
+ _hostList[ptr]->dump(s);
+ }
+}
+
+/*****************************************************************/
+void
+CarpHashAlgorithm::algoInit()
+{
+ // calculate hash for each host
+ for (size_t ptr = 0; ptr < _hostList.size(); ptr++) {
+ std::string cname;
+ char buf[64];
+ memset(buf, 0, 64);
+ snprintf(buf, 64, "%s:%d", _hostList[ptr]->name.c_str(), _hostList[ptr]->listenPort);
+ cname = buf;
+
+ _hostList[ptr]->hash = _calculateHash(cname);
+ }
+
+ // calculate loadmultiplier
+ _calculateLoadMultiplier();
+
+ for (size_t ptr = 0; ptr < _hostList.size(); ptr++) {
+ TSDebug(DEBUG_TAG_INIT, "algoInit host=%s port=%d hash=0x%x weight=%g loadFac=%g loadMult=%g isSelf=%d status=%d",
+ _hostList[ptr]->name.c_str(),
+ _hostList[ptr]->listenPort,
+ _hostList[ptr]->hash,
+ _hostList[ptr]->weight,
+ _hostList[ptr]->loadFactor,
+ _hostList[ptr]->loadMultiplier,
+ _hostList[ptr]->isSelf,
+ _hostList[ptr]->getStatus());
+ }
+}
+
+/*****************************************************************/
+std::vector<HashNode*>
+CarpHashAlgorithm::getRemapProxyList(const std::string& url)
+{
+ std::vector<HashNode *> hn=_selectReplicateNodes(url);
+ return hn;
+}
+
+/*****************************************************************/
+HashNode*
+CarpHashAlgorithm::getRemapProxy(const std::string& url)
+{
+ HashNode *hn=_selectNode(url);
+ if(hn) {
+ hn->hit();
+ }
+ return hn;
+}
+
+
+/*****************************************************************/
+unsigned int
+CarpHashAlgorithm::_calculateHash(const std::string& hostname)
+{
+ // check input data
+ if (hostname.empty())
+ return 0;
+
+ unsigned int hash = 0;
--- End diff --
Does the hash output have a well defined size? That is, is it specifically a 32 bit or 64 bit (for example) hash?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by SolidWallOfCode <gi...@git.apache.org>.
Github user SolidWallOfCode commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r75874806
--- Diff: plugins/experimental/carp/Common.cc ---
@@ -0,0 +1,364 @@
+
+/** @file
+
+ Loads the CARP configuration
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+#include "Common.h"
+
+#include <arpa/inet.h>
+#include <stdlib.h>
+#include <stdio.h>
+#include <errno.h>
+#include <netdb.h>
+#include <string.h>
+#include <memory.h>
+
+using namespace std;
+
+/************************************************************************/
+void stringExplode(string str, string separator, vector<string>* results)
+{
+ size_t found;
+ found = str.find_first_of(separator);
+ while (found != std::string::npos)
+ {
+ if (found > 0)
+ {
+ results->push_back(str.substr(0, found));
+ }
+ str = str.substr(found + 1);
+ found = str.find_first_of(separator);
+ }
+ if (str.length() > 0)
+ {
+ results->push_back(str);
+ }
+}
+
+/************************************************************************/
+/*
+ Parse /proc/{pid}/net/tcp
+ sl local_address rem_address st tx_queue rx_queue tr tm->when retrnsmt uid timeout inode
+ 0: 00000000:036B 00000000:0000 0A 00000000:00000000 00:00000000 00000000 0 0 22272 1 ffff8801b7ecb700 299 0 0 2 -1
+ 1: 00000000:08AE 00000000:0000 0A 00000000:00000000 00:00000000 00000000 0 0 10621 1 ffff8801b7b9a080 299 0 0 2 -1
+ 2: 00000000:006F 00000000:0000 0A 00000000:00000000 00:00000000 00000000 0 0 22046 1 ffff8801b7b9b400 299 0 0 2 -1
+ Parse /proc/{pid}/net/tcp6
+ * sl local_address remote_address st tx_queue rx_queue tr tm->when retrnsmt uid timeout inode
+ 0: 00000000000000000000000000000000:08AE 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000 0 0 10619 1 ffff8801b266d780 299 0 0 2 -1
+ 1: 00000000000000000000000000000000:006F 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000 0 0 22051 1 ffff8801b3a51880 299 0 0 2 -1
+ 2: 00000000000000000000000000000000:0016 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000 0 0 10656 1 ffff8801b266d040 299 0 0 2 -1
+ 3: 00000000000000000000000000000000:8E77 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000 0 0 22300 1 ffff8801b2f5c780 299 0 0 2 -1
+
+ looking for local_address:PORT wher st = LISTEN (0x0A)
+ * ./include/net/tcp_states.h
+ * enum {
+ TCP_ESTABLISHED = 1,
+ TCP_SYN_SENT,
+ TCP_SYN_RECV,
+ TCP_FIN_WAIT1,
+ TCP_FIN_WAIT2,
+ TCP_TIME_WAIT,
+ TCP_CLOSE,
+ TCP_CLOSE_WAIT,
+ TCP_LAST_ACK,
+ TCP_LISTEN,
+ TCP_CLOSING,
+
+ TCP_MAX_STATES
+ */
+
+/************************************************************************/
+
+/* used internally */
+bool
+scanProcFileForPort(string sFilename, string sPid, unsigned int iPort)
+{
+ bool bMatch = false;
+ TSFile file;
+ int iLine = 0;
+ bool bParseDone = false;
+
+ // attempt to open file
+ TSDebug(DEBUG_TAG_INIT, "Trying to open proc file @ %s to determine listening ports", sFilename.c_str());
+ file = TSfopen(sFilename.c_str(), "r");
+ if (file == NULL) {
+ TSError("Failed to open proc tcp file of %s. Error=%s", sFilename.c_str(), strerror(errno));
+ return false;
+ }
+
+ TSDebug(DEBUG_TAG_INIT, "Successfully opened %s file", sFilename.c_str());
+
+ char buffer[1024];
+ memset(buffer, 0, sizeof (buffer));
+
+ while (TSfgets(file, buffer, sizeof (buffer) - 1) != NULL && !bParseDone) {
+ char *eol = strstr(buffer, "\r\n");
+ if (!eol)
+ eol = strchr(buffer, '\n');
+ if (eol)
+ *eol = 0; // remove ending LF or CRLF
+
+ // TSDebug(DEBUG_TAG_INIT, "Parsing line: %s", buffer);
+ ++iLine;
+ if (iLine == 1) { // ignore header line
+ continue;
+ }
+ vector<string> vParts;
+ stringExplode(string(buffer), string(" "), &vParts);
+ // part 3 is status
+ unsigned int iStatus = 0;
+ sscanf(vParts[3].c_str(), "%x", &iStatus);
+ if (iStatus == TCP_LISTEN) {
+ vector<string> vLocalParts;
+ stringExplode(vParts[1], string(":"), &vLocalParts); // part 1 is localaddr
+ unsigned int iLPort = 0;
+ sscanf(vLocalParts[1].c_str(), "%x", &iLPort);
+ TSDebug(DEBUG_TAG_INIT, "Found listening port %d", iLPort);
+ if (iLPort == iPort) {
+ string sLabel = string("socket:[" + vParts[9] + "]");
+ // find inode in /proc/{pid}/fd/
+ string sDir = "/proc/" + string(sPid) + "/fd";
+
+ DIR *dp;
+ struct dirent *dirp;
+ if ((dp = opendir(sDir.c_str())) == NULL) {
+ TSDebug(DEBUG_TAG_INIT, "Failed to open directory %s, %s", sDir.c_str(), strerror(errno));
+ continue;
+ }
+
+ while ((dirp = readdir(dp)) != NULL && !bParseDone) {
+ //TSDebug(DEBUG_TAG_INIT, "File in %s = %s inode=%d", sDir.c_str(),dirp->d_name,dirp->d_ino);
+ string sPath = sDir + "/" + dirp->d_name;
+ char sBuf[256];
+ ssize_t iCount = readlink(sPath.c_str(), sBuf, sizeof (sBuf));
+ if (iCount > 0) {
+ string sLink(sBuf, iCount);
+ if (sLink.compare(sLabel) == 0) {
+ TSDebug(DEBUG_TAG_INIT, "Found that port %d is opened for listening by pid %s", iLPort, sPid.c_str());
+ bParseDone = true;
+ bMatch = true;
+ }
+ }
+ }
+ closedir(dp);
+ }
+ }
+ /*
+ for(int i=0;i<vParts.size();i++)
+ {
+ TSDebug(DEBUG_TAG_INIT, "part[%d]='%s'", i, vParts[i].c_str());
+ }
+ */
+ }
+
+ TSfclose(file);
+
+ return bMatch;
+}
+
+/************************************************************************/
+bool
+isPortSelf(unsigned int iPort)
+{
+ pid_t pid = getpid();
+ char sPid[10];
+ bool bMatch = false;
+
+ string sFileName;
+ // open file @ /proc/{pid}/net/tcp
+
+
+ sprintf(sPid,"%d",pid);
+
+ // look for IPv4 listener first
+ sFileName = "/proc/" + string(sPid) + "/net/tcp";
+ bMatch = scanProcFileForPort(sFileName,sPid, iPort);
+ if(!bMatch) { // did not find IPv4 listener, check for IPv6
+ sFileName = "/proc/" + string(sPid) + "/net/tcp6";
+ bMatch = scanProcFileForPort(sFileName,sPid, iPort);
+ }
+ return bMatch;
+}
+
+/************************************************************************/
+struct hostent *
+getHostIp(string hName, struct hostent *h, char *buf, int buflen)
+{
+ int res, err;
+ struct hostent *hp = NULL;
+
+ res = gethostbyname_r(hName.c_str(), h, buf, buflen, &hp, &err);
+ if ((res == 0) && (hp != NULL)) {
+ return hp;
+ } else {
+ TSDebug(DEBUG_TAG_INIT, "gethostbyname_r failed for %s. Error=%s", hName.c_str(), strerror(err));
+ }
+ return NULL;
+}
+
+
+/************************************************************************/
+/*
+ * *** WARNING *** You will need to run the carp plugin with traffic_manager for it to detect
+ * itself and to forward directly to the origin. It will not work by running traffic_server directly!
+ */
+bool
+isSelf(string sName, int iPort, struct hostent *pSelf)
+{
+ bool bMatch = false;
+ struct hostent other, *pOther;
+ char buf[1024];
+ pOther = getHostIp(sName,&other,buf, sizeof(buf));
+ //uint32_t self_addr = htonl(*(uint32_t *)pSelf->h_addr_list);
+ //TSDebug(DEBUG_TAG_INIT, "isSelf SELF h_addrtype=%d h_length=%d ip=%s",
+ // pSelf->h_addrtype, pSelf->h_length, inet_ntoa(*(struct in_addr*)&self_addr));
+ if(pOther) {
+ if( (pOther->h_addrtype != pSelf->h_addrtype) || // check basics are same
+ (pOther->h_length != pSelf->h_length)) {
+ return false;
+ }
+
+ for (int i = 0; pOther->h_addr_list[i] != NULL; i++) { // loop through each in other
+ for (int j = 0; pSelf->h_addr_list[j] != NULL; j++) { // loop through self
+ // uint32_t other_addr = htonl(*(uint32_t *)pOther->h_addr_list);
+ // TSDebug(DEBUG_TAG_INIT, "isSelf OTHER h_addrtype=%d h_length=%d ip=%s",
+ // pOther->h_addrtype, pOther->h_length, inet_ntoa(*(struct in_addr*)&other_addr));
+
+ if(memcmp(pSelf->h_addr_list[j], pOther->h_addr_list[i], pOther->h_length) == 0) {
+ // check for matching ports
+ // for self, look at the ports we are listening on.
+ bMatch = isPortSelf(iPort);
+ TSDebug(DEBUG_TAG_INIT, "port matched %s", bMatch ? "true":"false");
+ }
+ }
+ }
+ }
+ return bMatch;
+}
+
+/************************************************************************/
+bool
+addHeader(TSMBuffer& reqp, TSMLoc& hdr_loc, string header, string value)
+{
+ bool bReturn = false;
+ if (value.size() <= 0) {
+ TSDebug(DEBUG_TAG_HOOK, "\tWould set header %s to an empty value, skipping", header.c_str());
+ } else {
+ TSMLoc new_field;
+
+ if (TS_SUCCESS == TSMimeHdrFieldCreateNamed(reqp, hdr_loc, header.data(), header.size(), &new_field)) {
+ if (TS_SUCCESS == TSMimeHdrFieldValueStringInsert(reqp, hdr_loc, new_field, -1, value.data(), value.size())) {
+ if (TS_SUCCESS == TSMimeHdrFieldAppend(reqp, hdr_loc, new_field)) {
+ TSDebug(DEBUG_TAG_HOOK, "\tAdded header %s: %s", header.c_str(), value.c_str());
+ bReturn = true;
+ }
+ }
+ TSHandleMLocRelease(reqp, hdr_loc, new_field);
+ }
+ }
+ return bReturn;
+}
+
+/************************************************************************/
+bool
+getHeader(TSMBuffer& reqp, TSMLoc& hdr_loc, const std::string header, std::string& value)
+{
+ bool bReturn = false;
+ TSMLoc fieldLoc = TSMimeHdrFieldFind(reqp, hdr_loc, header.data(), header.size());
+
+ if (fieldLoc && (fieldLoc != NULL)) {
+ const char *str;
+ int strLen = 0;
+ str = TSMimeHdrFieldValueStringGet(reqp, hdr_loc, fieldLoc, 0, &strLen);
+ if (str && strLen > 0) {
+ value.assign(str, strLen);
+ bReturn = true;
+ } else {
+ bReturn = false;
+ }
+ TSHandleMLocRelease(reqp, hdr_loc, fieldLoc);
+ }
+ return bReturn;
+}
+
+/************************************************************************/
+bool
+removeHeader(TSMBuffer& reqp, TSMLoc& hdr_loc, string header)
+{
+ bool bReturn = false;
+ TSMLoc fieldLoc = TSMimeHdrFieldFind(reqp, hdr_loc, header.data(), header.size());
+
+ if (fieldLoc && (fieldLoc != NULL)) {
+ if(TSMimeHdrFieldRemove(reqp, hdr_loc, fieldLoc) == TS_SUCCESS) {
+ if(TSMimeHdrFieldDestroy(reqp, hdr_loc, fieldLoc) == TS_SUCCESS) {
+ TSDebug(DEBUG_TAG_HOOK, "\tRemoved header %s", header.c_str());
+ bReturn = true;
+ }
+ }
+ TSHandleMLocRelease(reqp, hdr_loc, fieldLoc);
+ }
+ return bReturn;
+}
+
+
+bool
+setHeader(TSMBuffer& reqp, TSMLoc& hdr_loc, const std::string header, const std::string value)
+{
+ bool bReturn = false;
+ TSMLoc fieldLoc = TSMimeHdrFieldFind(reqp, hdr_loc, header.data(), header.size());
+
+ if (fieldLoc && (fieldLoc != NULL)) {
+ if (TSMimeHdrFieldValueStringSet(reqp, hdr_loc, fieldLoc, 0, value.data(), value.length()) == TS_SUCCESS) {
+ TSDebug(DEBUG_TAG_HOOK, "\tSet header %s to %s", header.c_str(), value.c_str());
+ bReturn = true;
+ } else {
+ bReturn = false;
+ }
+ TSHandleMLocRelease(reqp, hdr_loc, fieldLoc);
+ }
+ return bReturn;
+}
+
+/************************************************************************/
+//Convert a struct sockaddr address to a string, IPv4 and IPv6:
+bool
+getStringFromSockaddr(const struct sockaddr *sa, string& s)
+{
+ char str[INET6_ADDRSTRLEN];
+ memset(str,sizeof(str),0);
+ switch (sa->sa_family) {
+ case AF_INET:
+ inet_ntop(AF_INET, &(((struct sockaddr_in *) sa)->sin_addr), str, sizeof(str));
+ break;
+
+ case AF_INET6:
+ inet_ntop(AF_INET6, &(((struct sockaddr_in6 *) sa)->sin6_addr), str, sizeof(str));
+ break;
+
+ default:
+ s.assign("Unknown");
+ return false;
+ }
+ s.assign(str);
+ sprintf(str,":%d",ntohs(((struct sockaddr_in *) sa)->sin_port));
--- End diff --
You might want to use brackets around IPv6 addresses so the port colon and the internal IPv6 colons do not get mixed up.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by ericcarlschwartz <gi...@git.apache.org>.
Github user ericcarlschwartz commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r76106996
--- Diff: plugins/experimental/carp/CarpConfig.cc ---
@@ -0,0 +1,581 @@
+/** @file
+
+ Loads the CARP configuration
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+
+//////////////////////////////////////////////////////////////
+// Read CARP configuration file
+// [Servers]
+// host1.yahoo.com:4080 weight=2 # port 4080 on host1.yahoo.com with weight factor of 2
+// host2.yahoo.com # port 80 on host2.yahoo.com with (default) weight factor of 1
+//
+// [Values]
+// healthcheck={host}:8001/status.html
+// healthfreq=30
+// global=on
+//
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <memory.h>
+#include <ts/ts.h>
+#include <sys/time.h>
+
+#include <sstream>
+
+#include "CarpConfig.h"
+#include "Common.h"
+#include "CarpConfigPool.h"
+
+using namespace std;
+
+#define DEFAULT_HEALTH_CHECK_FREQ 30 // 30 second period for health checks
+#define DEFAULT_HEALTH_CHECK_PORT 80 // default to makeing healthcheck requests against port 80
+#define DEFAULT_CONFIG_RELOAD_FREQ 30 // 30 seconds used in TSContSchedule
+#define DEFAULT_PORT 80 // default to makeing requests against port 80
+#define DEFAULT_WEIGHT 1 // default weight
+#define DEFAULT_SCHEME "http"
+#define DEFAULT_REPLICATION_FACTOR 1
+
+// config section headers
+static const char *const SECTION_SERVERS_STR = "[Servers]";
+static const char *const SECTION_VALUES_STR = "[Values]";
+
+// key strings
+static const char *const KEY_HEALTHCHECK_STR = "healthcheck";
+static const char *const KEY_HEALTHFREQ_STR = "healthfreq";
+static const char *const KEY_RELOADFREQ_STR = "reloadfreq";
+static const char *const KEY_HCTIMEOUT_STR = "hctimeout";
+static const char *const KEY_BLACKLIST_STR = "blacklist";
+static const char *const KEY_WHITELIST_STR = "whitelist";
+static const char *const KEY_MODE_STR = "mode";
+static const char *const KEY_ALLOWFWDPORT_STR = "allowfwdport";
+static const char *const KEY_REPLICATIONFACTOR_STR = "replicationfactor";
+
+// parameter strings
+static const char *const WEIGHT_EQUALS_STRING = "weight=";
+static const char *const GROUP_EQUALS_STRING = "group=";
+static const char *const KEY_MODE_PREREMAP_STR = "pre-remap";
+static const char *const KEY_MODE_POSTREMAP_STR = "post-remap";
+
+
+/**********************************************************/
+bool
+getInt(char** pptr, int *val)
--- End diff --
Kept the wrapper function for the bool return value and pointer update but uses strtol internally.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by SolidWallOfCode <gi...@git.apache.org>.
Github user SolidWallOfCode commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r75870527
--- Diff: plugins/experimental/carp/CarpHashAlgorithm.cc ---
@@ -0,0 +1,396 @@
+/** @file
+
+ Implements the CARP hash algorithm
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+#include <ts/ts.h>
+#include <stdio.h>
+#include <memory.h>
+#include <list>
+
+#include <iostream>
+#include <sstream>
+
+#include "Common.h"
+#include "CarpHashAlgorithm.h"
+
+using namespace std;
+
+/*****************************************************************/
+void
+HashNode::dump(string& s)
+{
+ stringstream ss;
+ string sSockaddr;
+ getStringFromSockaddr(reinterpret_cast<const struct sockaddr *>( &forwardAddr), sSockaddr);
+
+ ss << scheme << "://" << name << ":"<<listenPort<<" ("<<sSockaddr<<") weight:"<<weight<< (_status ? string(" UP ") : string(" DOWN "));
+ if(_statusTime) {
+ ss << "(" << time(NULL)-_statusTime<< "s ago in "<< _statusLatencyMs << "mS)";
+ }
+ ss << " hits:" << _hits;
+ ss << " carp_noforwarded:" << _carp_noforwarded;
+ ss << " carp_forwarded:" << _carp_forwarded;
+ ss << endl;
+ s += ss.str();
+}
+
+/*****************************************************************/
+void
+HashAlgorithm::addHost(std::string name, unsigned int port, std::string scheme, double weight, bool self, struct sockaddr_storage fwdAddr)
+{
+ HashNode* node = new HashNode(name, port, scheme, weight, self, fwdAddr);
+ TSAssert(NULL != node);
+ addHost(node);
+}
+
+/*****************************************************************/
+void
+HashAlgorithm::addHost(HashNode* node)
+{
+ if (!node) return;
+ _hostList.push_back(node);
+}
+
+/*****************************************************************/
+HashNode*
+HashAlgorithm::findStatusByNameAndPort(const string& name, unsigned int port,
+ size_t* index) {
+ /*
+ * Todo: This use loop to find the corresponding HashNode
+ * But the HttpClient and the Hash was related, so we
+ * could use more easier method to write the status
+ */
+ for (size_t ptr = 0; ptr < _hostList.size(); ptr++) {
+ if (_hostList[ptr]->listenPort == port
+ && _hostList[ptr]->name.compare(name) == 0) { // found it
+ if (index) {
+ *index = ptr;
+ }
+ return _hostList[ptr];
+ }
+ }
+ return NULL;
+}
+
+size_t
+HashAlgorithm::findHashNodeIndex(HashNode *node) {
+ for (size_t ptr = 0; ptr < _hostList.size(); ptr++) {
+ if ( _hostList[ptr] == node) {
+ return ptr;
+ }
+ }
+ return -1;
+}
+
+/*****************************************************************/
+void
+HashAlgorithm::setStatus(const string& name, unsigned int port, bool status, time_t time, uint64_t latencyMs)
+{
+ TSDebug(DEBUG_TAG_INIT, "HashAlgorithm::setStatus name=%s status=%d", name.c_str(), status);
+
+ HashNode* node = findStatusByNameAndPort(name,port);
+ if(node) {
+ node->setStatus(status,time,latencyMs);
+ } else {
+ TSError("Carp internal error setStatus host %s not found",name.c_str());
+ }
+}
+
+void
+HashAlgorithm::setStatus(HashNode * node, bool status, time_t time, uint64_t latencyMs)
+{
+ TSDebug(DEBUG_TAG_INIT, "HashAlgorithm::setStatus name=%s status=%d", node->name.c_str(), status);
+
+// HashNode* node = findStatusByNameAndPort(name,port);
+ if(node) {
+ node->setStatus(status,time,latencyMs);
+ } else {
+ TSError("Carp internal error setStatus host %s not found",node->name.c_str());
+ }
+}
+
+/*****************************************************************/
+HashAlgorithm::~HashAlgorithm()
+{
+ for (size_t ptr = 0; ptr < _hostList.size(); ptr++) {
+ delete _hostList[ptr];
+ }
+}
+
+/*****************************************************************/
+void
+HashAlgorithm::dump(string& s)
+{
+ _config->dump(s);
+
+ stringstream ss;
+ ss << "Hash Algo stats:" << endl;
+ s += ss.str();
+
+ for (size_t ptr = 0; ptr < _hostList.size(); ptr++) {
+ ss.str("");
+ ss.clear();
+ ss << ptr << "-";
+ s += ss.str();
+ _hostList[ptr]->dump(s);
+ }
+}
+
+/*****************************************************************/
+void
+CarpHashAlgorithm::algoInit()
+{
+ // calculate hash for each host
+ for (size_t ptr = 0; ptr < _hostList.size(); ptr++) {
+ std::string cname;
+ char buf[64];
+ memset(buf, 0, 64);
+ snprintf(buf, 64, "%s:%d", _hostList[ptr]->name.c_str(), _hostList[ptr]->listenPort);
+ cname = buf;
+
+ _hostList[ptr]->hash = _calculateHash(cname);
+ }
+
+ // calculate loadmultiplier
+ _calculateLoadMultiplier();
+
+ for (size_t ptr = 0; ptr < _hostList.size(); ptr++) {
+ TSDebug(DEBUG_TAG_INIT, "algoInit host=%s port=%d hash=0x%x weight=%g loadFac=%g loadMult=%g isSelf=%d status=%d",
+ _hostList[ptr]->name.c_str(),
+ _hostList[ptr]->listenPort,
+ _hostList[ptr]->hash,
+ _hostList[ptr]->weight,
+ _hostList[ptr]->loadFactor,
+ _hostList[ptr]->loadMultiplier,
+ _hostList[ptr]->isSelf,
+ _hostList[ptr]->getStatus());
+ }
+}
+
+/*****************************************************************/
+std::vector<HashNode*>
+CarpHashAlgorithm::getRemapProxyList(const std::string& url)
+{
+ std::vector<HashNode *> hn=_selectReplicateNodes(url);
+ return hn;
+}
+
+/*****************************************************************/
+HashNode*
+CarpHashAlgorithm::getRemapProxy(const std::string& url)
+{
+ HashNode *hn=_selectNode(url);
+ if(hn) {
+ hn->hit();
+ }
+ return hn;
+}
+
+
+/*****************************************************************/
+unsigned int
+CarpHashAlgorithm::_calculateHash(const std::string& hostname)
+{
+ // check input data
+ if (hostname.empty())
+ return 0;
+
+ unsigned int hash = 0;
+
+ // calculate MemberProxy_Hash
+ for (size_t ptr = 0; ptr < hostname.length(); ptr++)
+ hash += ROTATE_LEFT(hash, 19) + hostname.c_str()[ptr];
+
+ hash += hash * 0x62531965;
+ hash = ROTATE_LEFT(hash, 21);
+ // return
+ return hash;
+}
+
+/*****************************************************************/
+void
+CarpHashAlgorithm::_calculateLoadMultiplier()
+{
+ /*
+ *
+ * 3.3. Load Factor
+
+ Support for array members with differing HTTP processing & caching
+ capacity is achieved by multiplying each of the combined hash values
+ by a Load Factor Multiplier.
+
+ The Load Factor Multiplier for an individual member is calculated by
+ taking each member's relative Load Factor and applying the
+ following formula:
+
+ The Load Factor Multiplier must be calculated from the smallest
+ P_k to the largest P_k. The sum of all P_k's must be 1.
+
+ For each proxy server 1,...,K, the Load Factor Multiplier, X_k, is
+ calculated iteratively as follows:
+
+ All X_n values are 32 bit floating point numbers.
+
+ X_1 = pow ((K*p_1), (1/K))
+
+ X_k = ([K-k+1] * [P_k - P_{k-1}])/(X_1 * X_2 * ... * X_{k-1})
+ X_k += pow ((X_{k-1}, {K-k+1})
+ X_k = pow (X_k, {1/(K-k+1)})
+
+ where:
+
+ X_k = Load Factor Multiplier for proxy k
+ K = number of proxies in an array
+ P_k = relative percent of the load that proxy k should handle
+
+ This is then combined with the previously computed hashes as
+
+ Resultant_value = Combined_Hash * X_k
+
+
+ * recalculate loadfactor for each proxy [borrow it from nginx-upstream-carp module]
--- End diff --
How is this different from the previous formula in this comment?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by SolidWallOfCode <gi...@git.apache.org>.
Github user SolidWallOfCode commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r74175735
--- Diff: plugins/experimental/carp/CarpConfig.cc ---
@@ -0,0 +1,581 @@
+/** @file
+
+ Loads the CARP configuration
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+
+//////////////////////////////////////////////////////////////
+// Read CARP configuration file
+// [Servers]
+// host1.yahoo.com:4080 weight=2 # port 4080 on host1.yahoo.com with weight factor of 2
+// host2.yahoo.com # port 80 on host2.yahoo.com with (default) weight factor of 1
+//
+// [Values]
+// healthcheck={host}:8001/status.html
+// healthfreq=30
+// global=on
+//
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <memory.h>
+#include <ts/ts.h>
+#include <sys/time.h>
+
+#include <sstream>
+
+#include "CarpConfig.h"
+#include "Common.h"
+#include "CarpConfigPool.h"
+
+using namespace std;
+
+#define DEFAULT_HEALTH_CHECK_FREQ 30 // 30 second period for health checks
+#define DEFAULT_HEALTH_CHECK_PORT 80 // default to makeing healthcheck requests against port 80
+#define DEFAULT_CONFIG_RELOAD_FREQ 30 // 30 seconds used in TSContSchedule
+#define DEFAULT_PORT 80 // default to makeing requests against port 80
+#define DEFAULT_WEIGHT 1 // default weight
+#define DEFAULT_SCHEME "http"
+#define DEFAULT_REPLICATION_FACTOR 1
+
+// config section headers
+static const char *const SECTION_SERVERS_STR = "[Servers]";
+static const char *const SECTION_VALUES_STR = "[Values]";
+
+// key strings
+static const char *const KEY_HEALTHCHECK_STR = "healthcheck";
+static const char *const KEY_HEALTHFREQ_STR = "healthfreq";
+static const char *const KEY_RELOADFREQ_STR = "reloadfreq";
+static const char *const KEY_HCTIMEOUT_STR = "hctimeout";
+static const char *const KEY_BLACKLIST_STR = "blacklist";
+static const char *const KEY_WHITELIST_STR = "whitelist";
+static const char *const KEY_MODE_STR = "mode";
+static const char *const KEY_ALLOWFWDPORT_STR = "allowfwdport";
+static const char *const KEY_REPLICATIONFACTOR_STR = "replicationfactor";
+
+// parameter strings
+static const char *const WEIGHT_EQUALS_STRING = "weight=";
+static const char *const GROUP_EQUALS_STRING = "group=";
+static const char *const KEY_MODE_PREREMAP_STR = "pre-remap";
+static const char *const KEY_MODE_POSTREMAP_STR = "post-remap";
+
+
+/**********************************************************/
+bool
+getInt(char** pptr, int *val)
+{
+ bool bReturn = false;
+ int v=0;
+
+ char* ptr = *pptr;
+ // skip white space if any
+ while (*ptr && isspace(*ptr)) ++ptr;
+ // get digits
+ if (*ptr) {
+ while (*ptr && isdigit(*ptr)) {
+ v *= 10;
+ v += (*ptr)-'0';
+ ++ptr;
+ bReturn = true;
+ }
+ }
+ if (bReturn) {
+ *pptr = ptr;
+ *val = v;
+ }
+
+ return bReturn;
+}
+
+/**********************************************************/
+// [http[s]://]host[:port]/path
+bool
+getHostAndPort(char** pptr,string* sHost,int* iPort, string* sScheme)
+{
+ bool bReturn = false;
+ char* ptr = *pptr;
+
+ *iPort = 80;
+ *sScheme = DEFAULT_SCHEME;
+ if(strncmp(*pptr,"https",5) == 0) {
--- End diff --
Strings like `https` should be compile time constants so that you don't need to use magic constants like `5` later.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by ericcarlschwartz <gi...@git.apache.org>.
Github user ericcarlschwartz commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r74291077
--- Diff: plugins/experimental/carp/CarpHashAlgorithm.cc ---
@@ -0,0 +1,396 @@
+/** @file
+
+ Implements the CARP hash algorithm
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+#include <ts/ts.h>
+#include <stdio.h>
+#include <memory.h>
+#include <list>
+
+#include <iostream>
+#include <sstream>
+
+#include "Common.h"
+#include "CarpHashAlgorithm.h"
+
+using namespace std;
+
+/*****************************************************************/
+void
+HashNode::dump(string& s)
+{
+ stringstream ss;
+ string sSockaddr;
+ getStringFromSockaddr(reinterpret_cast<const struct sockaddr *>( &forwardAddr), sSockaddr);
+
+ ss << scheme << "://" << name << ":"<<listenPort<<" ("<<sSockaddr<<") weight:"<<weight<< (_status ? string(" UP ") : string(" DOWN "));
+ if(_statusTime) {
+ ss << "(" << time(NULL)-_statusTime<< "s ago in "<< _statusLatencyMs << "mS)";
+ }
+ ss << " hits:" << _hits;
+ ss << " carp_noforwarded:" << _carp_noforwarded;
+ ss << " carp_forwarded:" << _carp_forwarded;
+ ss << endl;
+ s += ss.str();
+}
+
+/*****************************************************************/
+void
+HashAlgorithm::addHost(std::string name, unsigned int port, std::string scheme, double weight, bool self, struct sockaddr_storage fwdAddr)
+{
+ HashNode* node = new HashNode(name, port, scheme, weight, self, fwdAddr);
+ TSAssert(NULL != node);
+ addHost(node);
+}
+
+/*****************************************************************/
+void
+HashAlgorithm::addHost(HashNode* node)
+{
+ if (!node) return;
+ _hostList.push_back(node);
+}
+
+/*****************************************************************/
+HashNode*
+HashAlgorithm::findStatusByNameAndPort(const string& name, unsigned int port,
+ size_t* index) {
+ /*
+ * Todo: This use loop to find the corresponding HashNode
+ * But the HttpClient and the Hash was related, so we
+ * could use more easier method to write the status
+ */
+ for (size_t ptr = 0; ptr < _hostList.size(); ptr++) {
+ if (_hostList[ptr]->listenPort == port
+ && _hostList[ptr]->name.compare(name) == 0) { // found it
+ if (index) {
+ *index = ptr;
+ }
+ return _hostList[ptr];
+ }
+ }
+ return NULL;
+}
+
+size_t
+HashAlgorithm::findHashNodeIndex(HashNode *node) {
+ for (size_t ptr = 0; ptr < _hostList.size(); ptr++) {
+ if ( _hostList[ptr] == node) {
+ return ptr;
+ }
+ }
+ return -1;
+}
+
+/*****************************************************************/
+void
+HashAlgorithm::setStatus(const string& name, unsigned int port, bool status, time_t time, uint64_t latencyMs)
+{
+ TSDebug(DEBUG_TAG_INIT, "HashAlgorithm::setStatus name=%s status=%d", name.c_str(), status);
+
+ HashNode* node = findStatusByNameAndPort(name,port);
+ if(node) {
+ node->setStatus(status,time,latencyMs);
+ } else {
+ TSError("Carp internal error setStatus host %s not found",name.c_str());
+ }
+}
+
+void
+HashAlgorithm::setStatus(HashNode * node, bool status, time_t time, uint64_t latencyMs)
+{
+ TSDebug(DEBUG_TAG_INIT, "HashAlgorithm::setStatus name=%s status=%d", node->name.c_str(), status);
+
+// HashNode* node = findStatusByNameAndPort(name,port);
+ if(node) {
+ node->setStatus(status,time,latencyMs);
+ } else {
+ TSError("Carp internal error setStatus host %s not found",node->name.c_str());
+ }
+}
+
+/*****************************************************************/
+HashAlgorithm::~HashAlgorithm()
+{
+ for (size_t ptr = 0; ptr < _hostList.size(); ptr++) {
+ delete _hostList[ptr];
+ }
+}
+
+/*****************************************************************/
+void
+HashAlgorithm::dump(string& s)
+{
+ _config->dump(s);
+
+ stringstream ss;
+ ss << "Hash Algo stats:" << endl;
+ s += ss.str();
+
+ for (size_t ptr = 0; ptr < _hostList.size(); ptr++) {
+ ss.str("");
+ ss.clear();
+ ss << ptr << "-";
+ s += ss.str();
+ _hostList[ptr]->dump(s);
+ }
+}
+
+/*****************************************************************/
+void
+CarpHashAlgorithm::algoInit()
+{
+ // calculate hash for each host
+ for (size_t ptr = 0; ptr < _hostList.size(); ptr++) {
+ std::string cname;
+ char buf[64];
+ memset(buf, 0, 64);
+ snprintf(buf, 64, "%s:%d", _hostList[ptr]->name.c_str(), _hostList[ptr]->listenPort);
+ cname = buf;
+
+ _hostList[ptr]->hash = _calculateHash(cname);
+ }
+
+ // calculate loadmultiplier
+ _calculateLoadMultiplier();
+
+ for (size_t ptr = 0; ptr < _hostList.size(); ptr++) {
+ TSDebug(DEBUG_TAG_INIT, "algoInit host=%s port=%d hash=0x%x weight=%g loadFac=%g loadMult=%g isSelf=%d status=%d",
+ _hostList[ptr]->name.c_str(),
+ _hostList[ptr]->listenPort,
+ _hostList[ptr]->hash,
+ _hostList[ptr]->weight,
+ _hostList[ptr]->loadFactor,
+ _hostList[ptr]->loadMultiplier,
+ _hostList[ptr]->isSelf,
+ _hostList[ptr]->getStatus());
+ }
+}
+
+/*****************************************************************/
+std::vector<HashNode*>
+CarpHashAlgorithm::getRemapProxyList(const std::string& url)
+{
+ std::vector<HashNode *> hn=_selectReplicateNodes(url);
--- End diff --
that's better
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by SolidWallOfCode <gi...@git.apache.org>.
Github user SolidWallOfCode commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r74178543
--- Diff: plugins/experimental/carp/CarpConfig.cc ---
@@ -0,0 +1,581 @@
+/** @file
+
+ Loads the CARP configuration
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+
+//////////////////////////////////////////////////////////////
+// Read CARP configuration file
+// [Servers]
+// host1.yahoo.com:4080 weight=2 # port 4080 on host1.yahoo.com with weight factor of 2
+// host2.yahoo.com # port 80 on host2.yahoo.com with (default) weight factor of 1
+//
+// [Values]
+// healthcheck={host}:8001/status.html
+// healthfreq=30
+// global=on
+//
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <memory.h>
+#include <ts/ts.h>
+#include <sys/time.h>
+
+#include <sstream>
+
+#include "CarpConfig.h"
+#include "Common.h"
+#include "CarpConfigPool.h"
+
+using namespace std;
+
+#define DEFAULT_HEALTH_CHECK_FREQ 30 // 30 second period for health checks
+#define DEFAULT_HEALTH_CHECK_PORT 80 // default to makeing healthcheck requests against port 80
+#define DEFAULT_CONFIG_RELOAD_FREQ 30 // 30 seconds used in TSContSchedule
+#define DEFAULT_PORT 80 // default to makeing requests against port 80
+#define DEFAULT_WEIGHT 1 // default weight
+#define DEFAULT_SCHEME "http"
+#define DEFAULT_REPLICATION_FACTOR 1
+
+// config section headers
+static const char *const SECTION_SERVERS_STR = "[Servers]";
+static const char *const SECTION_VALUES_STR = "[Values]";
+
+// key strings
+static const char *const KEY_HEALTHCHECK_STR = "healthcheck";
+static const char *const KEY_HEALTHFREQ_STR = "healthfreq";
+static const char *const KEY_RELOADFREQ_STR = "reloadfreq";
+static const char *const KEY_HCTIMEOUT_STR = "hctimeout";
+static const char *const KEY_BLACKLIST_STR = "blacklist";
+static const char *const KEY_WHITELIST_STR = "whitelist";
+static const char *const KEY_MODE_STR = "mode";
+static const char *const KEY_ALLOWFWDPORT_STR = "allowfwdport";
+static const char *const KEY_REPLICATIONFACTOR_STR = "replicationfactor";
+
+// parameter strings
+static const char *const WEIGHT_EQUALS_STRING = "weight=";
+static const char *const GROUP_EQUALS_STRING = "group=";
+static const char *const KEY_MODE_PREREMAP_STR = "pre-remap";
+static const char *const KEY_MODE_POSTREMAP_STR = "post-remap";
+
+
+/**********************************************************/
+bool
+getInt(char** pptr, int *val)
+{
+ bool bReturn = false;
+ int v=0;
+
+ char* ptr = *pptr;
+ // skip white space if any
+ while (*ptr && isspace(*ptr)) ++ptr;
+ // get digits
+ if (*ptr) {
+ while (*ptr && isdigit(*ptr)) {
+ v *= 10;
+ v += (*ptr)-'0';
+ ++ptr;
+ bReturn = true;
+ }
+ }
+ if (bReturn) {
+ *pptr = ptr;
+ *val = v;
+ }
+
+ return bReturn;
+}
+
+/**********************************************************/
+// [http[s]://]host[:port]/path
+bool
+getHostAndPort(char** pptr,string* sHost,int* iPort, string* sScheme)
+{
+ bool bReturn = false;
+ char* ptr = *pptr;
+
+ *iPort = 80;
+ *sScheme = DEFAULT_SCHEME;
+ if(strncmp(*pptr,"https",5) == 0) {
+ *iPort = 443;
+ *sScheme = "https";
+ }
+
+ //skip leading white space
+ while (*ptr && isspace(*ptr)) ++ptr;
+
+ if (*ptr) { // validate not end of string
+ if(strncmp(ptr,"http://",7) == 0) {
+ ptr += 7;
+ } else if(strncmp(ptr,"https://",8) == 0) {
+ ptr += 8;
+ }
+
+ char* ptemp = ptr; // start of host
+ //find white space or ':' or '/'
+ while (*ptr && !isspace(*ptr) && *ptr != ':' && *ptr != '/') ++ptr;
+
+ *sHost = string(ptemp, (ptr - ptemp));
+ // skip white space (if any) after host
+ while (*ptr && isspace(*ptr)) ++ptr;
+ bReturn = true;
+
+ if (*ptr) {// have more to parse
+ // need to get port number?
+ if (*ptr == ':') {
+ ++ptr;
+ if (!getInt(&ptr, iPort)) {
+ // could be our special 'PORT' value, check for that
+ if(!strncmp(ptr,"{port}",6)) { // yes, is '{port}'
+ *iPort = -1;
+ bReturn = true;
+ } else { // really was an error
+ TSError("carp: error parsing port number from '%s'", *pptr);
+ bReturn = false;
+ }
+ } else {
+ // if port number is 443, treat the scheme as https
+ if (*iPort == 443) {
+ *sScheme = "https";
+ }
+ }
+ }
+ }
+ }
+ if(bReturn) {
+ *pptr = ptr;
+ }
+
+ return bReturn;
+}
+/**********************************************************/
+CarpConfig::CarpConfig()
+{
+ _healthCheckPort = DEFAULT_HEALTH_CHECK_PORT;
+ _healthCheckFreq = DEFAULT_HEALTH_CHECK_FREQ;
+ _configCheckFreq = DEFAULT_CONFIG_RELOAD_FREQ;
+ _healthCheckTimeout = DEFAULT_HEALTH_CHECK_TIMEOUT;
+ _setExit = 0;
+ _mode = PRE;
+ _allowForwardPort = 0;
+ _replicationFactor = DEFAULT_REPLICATION_FACTOR;
+ _nGroups = 0;
+}
+
+/**********************************************************/
+CarpConfig::~CarpConfig()
+{
+ for(size_t ptr = 0; ptr < _servers.size(); ptr++) {
+ delete _servers[ptr];
+ }
+
+ for(size_t ptr = 0; ptr < _httpClients.size(); ptr++) {
+ delete _httpClients[ptr];
+ }
+}
+
+/**********************************************************/
+bool
+CarpConfig::loadConfig(string filename)
+{
+ TSFile file;
+ GroupCountList group_counts;
+ GroupCountListIter group_counts_it;
+
+ // attempt to open config file assuming full path provided
+ TSDebug(DEBUG_TAG_INIT, "Trying to open config file in this path: %s", filename.c_str());
+ file = TSfopen(filename.c_str(), "r");
+ if (file == NULL) {
+ TSError("Failed to open carp config file %s", filename.c_str());
+ return false;
+ }
+
+ TSDebug(DEBUG_TAG_INIT, "Successfully opened config file");
+
+ char buffer[1024];
+ memset(buffer, 0, sizeof(buffer));
+
+ enum CONFIG_SECTION {
+ CFG_NONE_SECTION = 0,
+ CFG_SERVER_SECTION,
+ CFG_VALUES_SECTION
+ };
+ int cfg_section = CFG_NONE_SECTION;
+ bool done_parsing = false;
+ while (TSfgets(file, buffer, sizeof(buffer) - 1) != NULL && !done_parsing) {
+
+ char *eol = 0;
+ // make sure line was not bigger than buffer
+ if ((eol = strchr(buffer, '\n')) == NULL && (eol = strstr(buffer, "\r\n")) == NULL) {
+ TSError("carp config line was too long, did not get a good line in cfg, skipping, line: %s", buffer);
+ memset(buffer, 0, sizeof(buffer));
+ continue;
+ }
+
+ // make sure line has something useful on it
+ if (eol - buffer < 2 || buffer[0] == '#' || isspace(buffer[0])) {
+ memset(buffer, 0, sizeof(buffer));
+ continue;
+ }
+
+ // remove ending CR/LF
+ *eol = 0;
+
+ // check if we are changing sections
+ if (strncasecmp(buffer, SECTION_SERVERS_STR, strlen(SECTION_SERVERS_STR)) == 0) {
+ cfg_section = CFG_SERVER_SECTION;
+ TSDebug(DEBUG_TAG_INIT, "Parsing [Servers] section");
+ continue;
+ } else if (strncasecmp(buffer, SECTION_VALUES_STR, strlen(SECTION_VALUES_STR)) == 0) {
+ cfg_section = CFG_VALUES_SECTION;
+ TSDebug(DEBUG_TAG_INIT, "Parsing [Values] section");
+ continue;
+ }
+
+ //TSDebug(DEBUG_TAG_INIT, "config line input:'%s'", buffer);
+
+ switch (cfg_section) {
+ case CFG_SERVER_SECTION:
+ {
+ string sHost;
+ int iPort = DEFAULT_PORT;
+ int iWeight = DEFAULT_WEIGHT;
+ int iGroup = DEFAULT_GROUP;
+ string sScheme = DEFAULT_SCHEME;
+ bool bSuccess = true;
+
+ char* ptr = buffer;
+
+ if(!getHostAndPort(&ptr,&sHost,&iPort, &sScheme)) {
+ TSError("carp: error parsing port number from '%s'", ptr);
+ }
+
+ while (*ptr) {
+ // skip white space (if any)
+ while (*ptr && isspace(*ptr)) ++ptr;
+
+ if (*ptr) { // next we could find weight= or group=
+ if (strncmp(ptr, WEIGHT_EQUALS_STRING, strlen(WEIGHT_EQUALS_STRING)) == 0) {
+ ptr += strlen(WEIGHT_EQUALS_STRING);
+ if (!getInt(&ptr, &iWeight)) {
+ TSError("carp: error parsing weight value from '%s'", buffer);
+ bSuccess = false;
+ continue;
+ }
+ } else if (strncmp(ptr, GROUP_EQUALS_STRING, strlen(GROUP_EQUALS_STRING)) == 0) {
+ ptr += strlen(GROUP_EQUALS_STRING);
+ if (!getInt(&ptr, &iGroup)) {
+ TSError("carp: error parsing group value from '%s'", buffer);
+ bSuccess = false;
+ continue;
+ }
+ } else {
+ TSError("carp: error parsing from line '%s'", buffer);
+ // malformed entry, skip to next space
+ while (*ptr && !isspace(*ptr)) ++ptr;
+ bSuccess = false;
+ continue;
+ }
+ }
+ }
+
+ if (!bSuccess) continue;
+
+ group_counts_it = group_counts.find(iGroup);
+ if (group_counts_it != group_counts.end()) {
+ group_counts[iGroup]++;
+ } else {
+ _nGroups++;
+ group_counts[iGroup] = 1;
+ }
+
+ TSDebug(DEBUG_TAG_INIT, "Host = %s, port=%d, weight=%d, group=%d", sHost.c_str(), iPort, iWeight, iGroup);
+ CarpHost* host=new CarpHost(sHost, iPort, sScheme, iWeight, iGroup);
+ TSAssert(host != NULL);
+ // store the parsed data
+ addHost(host);
+ break;
+ }
+ case CFG_VALUES_SECTION:
+ {
+ char* ptr = buffer;
+
+ //skip leading white space
+ while (*ptr && isspace(*ptr)) ++ptr;
+
+ //find end of key
+ while (*ptr && !isspace(*ptr) && *ptr != '=') ++ptr;
+
+ string sKey = string(buffer, (ptr - buffer));
+
+ // skip white space (if any) after key
+ while (*ptr && isspace(*ptr)) ++ptr;
+ if (*ptr != '=') {
+ TSError("carp: expecting '=' after key in line '%s'", buffer);
+ continue;
+ }
+
+ // skip '=' and white space (if any) after '='
+ ++ptr;
+ while (*ptr && isspace(*ptr)) ++ptr;
+
+ char* ptemp = ptr; // get start of value
+
+ // find end of value
+ while (*ptr && !isspace(*ptr)) ++ptr;
+ string sValue = string(ptemp, ptr - ptemp);
+
+ TSDebug(DEBUG_TAG_INIT, "Key=%s Value=%s", sKey.c_str(), sValue.c_str());
+
+ char* pTemp = (char *)sValue.c_str();
+ if (sKey.compare(KEY_HEALTHCHECK_STR) == 0) {
+ string sH;
+ int iP;
+ string scheme;
+ _healthCheckUrl = string(pTemp);
+ if (!getHostAndPort(&pTemp, &sH, &iP, &scheme)) {
+ TSError("carp: error parsing host and/or port number from '%s'", buffer);
+ }
+
+ // store the parsed data
+ _healthCheckPort = iP;
+ TSDebug(DEBUG_TAG_INIT, "healthcheck Url=%s port=%d", _healthCheckUrl.c_str(), _healthCheckPort);
+
+ } else if (sKey.compare(KEY_HEALTHFREQ_STR) == 0) {
+ int iFreq;
+ if (!getInt(&pTemp, &iFreq)) {
+ TSError("carp: error parsing number from '%s'", buffer);
+ } else {
+ TSDebug(DEBUG_TAG_INIT, "healthcheck freq=%d", iFreq);
+ // store the parsed data
+ _healthCheckFreq = iFreq;
+ }
+ } else if (sKey.compare(KEY_HCTIMEOUT_STR) == 0) {
+ int iFreq;
+ if (!getInt(&pTemp, &iFreq)) {
+ TSError("carp: error parsing number from '%s'", buffer);
+ } else {
+ TSDebug(DEBUG_TAG_INIT, "healthcheck timeout value=%d", iFreq);
+ // store the parsed data
+ _healthCheckTimeout = iFreq;
+ }
+ } else if (sKey.compare(KEY_RELOADFREQ_STR) == 0) {
+ int lFreq;
+ if (!getInt(&pTemp, &lFreq)) {
+ TSError("carp: error parsing number from '%s'", buffer);
+ } else {
+ TSDebug(DEBUG_TAG_INIT, "config reload freq=%d", lFreq);
+ _configCheckFreq = lFreq;
+ }
+ } else if (sKey.compare(KEY_BLACKLIST_STR) == 0) {
+ vector<string> results;
+ stringExplode(sValue, string(","), &results);
+ for (vector<string>::iterator it = results.begin(); it != results.end();
+ it++) {
+ TSDebug(DEBUG_TAG_INIT, "Adding blacklist hostname %s",
+ (*it).c_str());
+ _blackList.insert(*it);
+ }
+ } else if (sKey.compare(KEY_WHITELIST_STR) == 0) {
+ vector<string> results;
+ stringExplode(sValue, string(","), &results);
+ for(vector<string>::iterator it=results.begin(); it != results.end(); it++) {
+ TSDebug(DEBUG_TAG_INIT, "Adding whitelist hostname %s",(*it).c_str() );
+ _whiteList.insert(*it);
+ }
+ } else if (sKey.compare(KEY_MODE_STR) == 0) {
+ if(sValue.compare(KEY_MODE_PREREMAP_STR) == 0) {
+ _mode = PRE;
+ } else if(sValue.compare(KEY_MODE_POSTREMAP_STR) == 0) {
+ _mode = POST;
+ } else {
+ TSError("carp: invalid mode in '%s'", buffer);
+ }
+ } else if (sKey.compare(KEY_ALLOWFWDPORT_STR) == 0) {
+ int iPort;
+ if (!getInt(&pTemp, &iPort)) {
+ TSError("carp: error parsing number from '%s'", buffer);
+ }
+ else {
+ TSDebug(DEBUG_TAG_INIT, "Allow forwarding port=%d", iPort);
+ // store the parsed data
+ _allowForwardPort = iPort;
+ }
+ } else if (sKey.compare(KEY_REPLICATIONFACTOR_STR) == 0) {
+ int iFactor;
+ if (!getInt(&pTemp, &iFactor)) {
+ TSError("carp: error parsing number from '%s'", buffer);
+ }
+ else {
+ TSDebug(DEBUG_TAG_INIT, "Replication factor=%d", iFactor);
+ // store the parsed data
+ _replicationFactor = iFactor;
+ }
+
+ } else
+ TSError("carp found bad setting under Values section '%s'", buffer);
+ break;
+ }
+ default:
+ // ignore bad cfg_section
+ TSDebug(DEBUG_TAG_INIT, "hit default in switch, ignoring extra input '%s'",buffer);
+ break;
+ };
+ } //while
+
+ if (_healthCheckTimeout > _healthCheckFreq - 1 ) {
+ _healthCheckTimeout = _healthCheckFreq - 1;
+ }
+
+ TSfclose(file);
+
+ if (_blackList.size() && _whiteList.size() ) {
+ TSError("Carp configured with both blacklist and whitelist, blacklist will be ignored");
+ }
+
+ if (_nGroups > _replicationFactor) {
+ TSError("Too many groups configured! Failing config.");
+ return false;
+ } else {
+ TSDebug(DEBUG_TAG_INIT, "Group Config is as follows:");
+ for (map<int, int>::const_iterator it = group_counts.begin(); it != group_counts.end(); it++) {
+ TSDebug(DEBUG_TAG_INIT, "Group %d has %d members.", it->first, it->second);
+ _group_count_list[it->first] = it->second;
+ }
+ }
+
+ return true;
+}
+
+/**********************************************************/
+void
+CarpConfig::addHost(CarpHost* host) {
+ _servers.push_back(host);
+}
+
+void
+CarpConfig::addHealthCheckClient(HttpFetch *client) {
+ client->setHealthcheckTimeout(_healthCheckTimeout);
+ _httpClients.push_back(client);
+
+}
+
+void
+CarpConfig::setPath(string path) {
+ _configPath = path;
+}
+
+string
+CarpConfig::getPath() {
+ return _configPath;
+}
+
+/**********************************************************/
+void *
+CarpConfigHealthCheckThreadStart(void* data)
+{
+ CarpConfigAndHash* cch = static_cast<CarpConfigAndHash *> (data);
+ TSAssert(cch);
+ return cch->_config->run(cch->_hashAlgo);
+}
+
+/**********************************************************/
+bool
+CarpConfig::isBlackListed(const string& sHost)
+{
+ if(!_blackList.size()) return false;
+ return (_blackList.find(sHost) != _blackList.end());
--- End diff --
Not clear it's worth checking for an empty list, as `find` will do that as well.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by bryancall <gi...@git.apache.org>.
Github user bryancall closed the pull request at:
https://github.com/apache/trafficserver/pull/843
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by SolidWallOfCode <gi...@git.apache.org>.
Github user SolidWallOfCode commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r75863130
--- Diff: plugins/experimental/carp/carp.cc ---
@@ -0,0 +1,713 @@
+/** @file
+
+ A brief file description
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+
+#include <errno.h>
+
+#include <string>
+#include <sstream>
+#include <stdlib.h>
+#include <stdio.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+#include <unistd.h>
+
+#include <memory.h>
+
+#include <ts/ts.h>
+
+#include "Common.h"
+#include "CarpConfig.h"
+#include "CarpConfigPool.h"
+#include "CarpHashAlgorithm.h"
+#include "UrlComponents.h"
+
+using namespace std;
+
+CarpConfigPool* g_CarpConfigPool = NULL;
+int g_carpSelectedHostArgIndex = 0;
+TSTextLogObject g_logObject = NULL;
+
+const char *logFileName = "carp";
+
+//////////////////////////////////////////////////////////////////////////////
+//////////////////////////////////////////////////////////////////////////////
+/*
+ check for our carp routed header, dump status if requested
+ */
+static int
+processCarpRoutedHeader(TSHttpTxn txnp, TSMBuffer bufp, TSMLoc hdr_loc)
+{
+ string value;
+ if (getHeader(bufp, hdr_loc, CARP_ROUTED_HEADER, value)) { // if found header
+ if (value.compare("1") == 0) { // is loop prevention value
+ TSDebug(DEBUG_TAG_HOOK, "Found %s header with loop prevention value, not forwarding again", CARP_ROUTED_HEADER.c_str());
+ return 0;
+ } else if (value.compare("dump") == 0) { // is dump status request
+ TSDebug(DEBUG_TAG_HOOK, "Found %s header with dump request", CARP_ROUTED_HEADER.c_str());
+ string status;
+ g_CarpConfigPool->getGlobalHashAlgo()->dump(status);
+ TSHttpTxnSetHttpRetStatus(txnp, TS_HTTP_STATUS_MULTI_STATUS);
+ TSHttpTxnErrorBodySet(txnp, TSstrdup(status.c_str()), status.length(), NULL);
+ return -1;
+ }
+ TSDebug(DEBUG_TAG_HOOK, "Found %s header with unknown value of %s, ignoring", CARP_ROUTED_HEADER.c_str(), value.c_str());
+ removeHeader(bufp, hdr_loc, CARP_ROUTED_HEADER);
+ }
+ return 1; // all OK
+}
+
+static bool
+checkListForSelf(std::vector<HashNode *> list)
+{
+ for (size_t k = 0; k < list.size(); k++) {
+ if (list[k]->isSelf) return true;
+ }
+ return false;
+}
+
+/**
+ bIsPOSTRemap = false --- Hash request and forward to peer
+ bIsPOSTRemap = true --- hash request, extract OS sockaddr, insert forwarding header, forward
+ */
+static int
+handleRequestProcessing(TSCont contp, TSEvent event, void *edata, bool bIsPOSTRemap)
+{
+ TSHttpTxn txnp = (TSHttpTxn) edata;
+ TSMBuffer bufp;
+ TSMLoc hdr_loc;
+ TSMLoc url_loc;
+
+ // get the client request so we can get URL and add header
+ if (TSHttpTxnClientReqGet(txnp, &bufp, &hdr_loc) != TS_SUCCESS) {
+ TSError("carp couldn't get request headers");
+ return -1;
+ }
+
+ int method_len;
+ const char *method = TSHttpHdrMethodGet(bufp, hdr_loc, &method_len);
+ if (NULL == method) {
+ TSError("carp couldn't get http method");
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return -1;
+ }
+ if (((method_len == TS_HTTP_LEN_DELETE) && (strncasecmp(method, TS_HTTP_METHOD_DELETE, TS_HTTP_LEN_DELETE) == 0)) ||
+ ((method_len == TS_HTTP_LEN_PURGE) && (strncasecmp(method, TS_HTTP_METHOD_PURGE, TS_HTTP_LEN_PURGE) == 0))) {
+ TSDebug(DEBUG_TAG_HOOK, "Request method is '%s' so not routing request", string(method,method_len).c_str());
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ }
+
+ if (TSHttpHdrUrlGet(bufp, hdr_loc, &url_loc) != TS_SUCCESS) {
+ TSError("carp couldn't get url");
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return -1;
+ }
+ // if has carp loop prevention header, do not remap
+ int iTemp = processCarpRoutedHeader(txnp, bufp, hdr_loc);
+ if(iTemp <= 0) { // if dump or do not remap
+ // check origin client request's scheme for premap mode
+ if (!bIsPOSTRemap) {
+ string oriScheme;
+ if (!getHeader(bufp, hdr_loc, CARP_PREMAP_SCHEME, oriScheme)) {
+ TSDebug(DEBUG_TAG_HOOK, "couldn't get '%s' header", CARP_PREMAP_SCHEME.c_str());
+ } else {
+ bool isHttps = (oriScheme == TS_URL_SCHEME_HTTPS);
+
+ if (isHttps) {
+ TSUrlSchemeSet(bufp, url_loc, TS_URL_SCHEME_HTTPS, TS_URL_LEN_HTTPS);
+ } else {
+ TSUrlSchemeSet(bufp, url_loc, TS_URL_SCHEME_HTTP, TS_URL_LEN_HTTP);
+ }
+
+ removeHeader(bufp, hdr_loc, CARP_STATUS_HEADER);
+ removeHeader(bufp, hdr_loc, CARP_ROUTED_HEADER);
+ removeHeader(bufp, hdr_loc, CARP_PREMAP_SCHEME.c_str());
+ TSDebug(DEBUG_TAG_HOOK, "Set client request's scheme to %s through %s header",
+ isHttps?"https":"http", CARP_PREMAP_SCHEME.c_str());
+ }
+ } else {
+ removeHeader(bufp, hdr_loc, CARP_ROUTED_HEADER);
+ }
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return iTemp;
+ }
+
+ UrlComponents reqUrl;
+ reqUrl.populate(bufp, url_loc);
+ // the url ONLY used to determine the cache owner
+ string sUrl;
+
+ if (!bIsPOSTRemap) { // if pre-remap, then host not in URL so get from header
+ string sHost;
+ if (!getHeader(bufp, hdr_loc, TS_MIME_FIELD_HOST, sHost)) {
+ TSDebug(DEBUG_TAG_HOOK, "Could not find host header, ignoring it");
+ }
+ reqUrl.setHost(sHost);
+
+ //[YTSATS-836] heuristically ignore the scheme and port when calculate cache owner
+ UrlComponents normalizedUrl = reqUrl;
+ normalizedUrl.setScheme(CARP_SCHEME_FOR_HASH);
+ normalizedUrl.setPort(CARP_PORT_FOR_HASH);
+ normalizedUrl.construct(sUrl);
+ } else {
+ reqUrl.construct(sUrl);
+ }
+
+
+ if (g_CarpConfigPool->getGlobalConfig()->hasWhiteList()) {
+ string sCarpable;
+ if (!getHeader(bufp, hdr_loc, CARPABLE_HEADER, sCarpable)) { // if no carpable header check whitelist
+ if (!g_CarpConfigPool->getGlobalConfig()->isWhiteListed(reqUrl.getHost())) { // if white list exists, then host must be present
+ TSDebug(DEBUG_TAG_HOOK, "Host '%s' is not whitelisted, not going through carp", reqUrl.getHost().c_str());
+ TSHandleMLocRelease(bufp, hdr_loc, url_loc);
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ } else {
+ TSDebug(DEBUG_TAG_HOOK, "Found host (%s) whitelisted, routing...",reqUrl.getHost().c_str());
+ }
+ } else { // found carpable header, make sure it's 1
+ if (sCarpable.compare("1") != 0) { // carpable header present but not 0, be strict and do not forward request
+ TSDebug(DEBUG_TAG_HOOK, "Carpable (%s) present but value not acceptable (%s)", CARPABLE_HEADER.c_str(), sCarpable.c_str());
+ TSHandleMLocRelease(bufp, hdr_loc, url_loc);
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ } else {
+ TSDebug(DEBUG_TAG_HOOK, "Found Carpable header, routing...");
+ }
+ }
+ } else { // no whitelist so blacklist could be used
+ if (g_CarpConfigPool->getGlobalConfig()->isBlackListed(reqUrl.getHost())) { // if host black listed, do not carp
+ TSDebug(DEBUG_TAG_HOOK, "Host '%s' is blacklisted, not going through carp", reqUrl.getHost().c_str());
+ TSHandleMLocRelease(bufp, hdr_loc, url_loc);
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ }
+ }
+
+ TSDebug(DEBUG_TAG_HOOK, "URL to hash with=%s", sUrl.c_str());
+
+ // get nodeList and select node to forward to
+ std::vector<HashNode *> nodeList = g_CarpConfigPool->getGlobalHashAlgo()->getRemapProxyList(sUrl);
+ bool bAddHeaderResult=false;
+ bool bAddForwardedResult = false;
+ HashNode* node = NULL;
+ bool bIsOwner = false;
+
+ if (nodeList.size() == 0) { // no hosts available to forward to
+ TSDebug(DEBUG_TAG_HOOK, "no hosts available to forward to, will handle locally");
+ goto done;
+ } else {
+ node = nodeList[0];
+ }
+
+ bIsOwner = checkListForSelf(nodeList);
+ for (size_t k = 0; k < nodeList.size(); k++) {
+ TSDebug(DEBUG_TAG_HOOK, "nodeList host %d name is %s", static_cast<int>(k), nodeList[k]->name.c_str());
+ }
+
+ //handle forwarding
+ TSDebug(DEBUG_TAG_HOOK, "forwarding to '%s' (isSelf=%d)", node->name.c_str(), node->isSelf);
+ if (!node->isSelf) { // carp does not forward if we choose ourself
+ node->carpForward();
+ TSDebug(DEBUG_TAG_HOOK, "carp forwarded to %s.", node->name.c_str());
+ // insert carp loop prevention header
+ bAddHeaderResult = addHeader(bufp, hdr_loc, CARP_ROUTED_HEADER, string("1"));
+ if (!bAddHeaderResult) {
+ TSError("Carp, error inserting '%s' header", CARP_ROUTED_HEADER.c_str());
+ }
+ bAddForwardedResult = addHeader(bufp, hdr_loc, CARP_STATUS_HEADER, string(CARP_FORWARDED));
+ if (!bAddForwardedResult) {
+ TSError("Carp, error inserting '%s' header", CARP_STATUS_HEADER.c_str());
+ }
+
+ if (bIsPOSTRemap) { // if post remap, get remapped/OS Server Addr and add as header
+ const struct sockaddr* sa = TSHttpTxnServerAddrGet(txnp);
+ // const struct sockaddr* sa = TSHttpTxnNextHopAddrGet(txnp);
+ if (sa) { // sanity check
+ struct sockaddr_storage ss;
+ memcpy(static_cast<void *> (&ss), sa, sizeof (sockaddr_storage));
+ if ((reinterpret_cast<sockaddr_in *> (&ss))->sin_port == 0) { // set port from client request URL
+ (reinterpret_cast<sockaddr_in *> (&ss))->sin_port = htons(reqUrl.getPort());
+ }
+
+ string sTemp;
+ getStringFromSockaddr(reinterpret_cast<const sockaddr *> (&ss), sTemp);
+ TSDebug(DEBUG_TAG_HOOK, "Inserting forward header with sockaddr:%s", sTemp.c_str());
+ string sSockaddr;
+ sSockaddr.reserve(32 + sizeof (sockaddr_storage) * 2);
+ for (unsigned int i = 0; i<sizeof (sockaddr_storage); i++) {
+ char val[8];
+ sprintf(val, "%02X", reinterpret_cast<const unsigned char *> (&ss)[i]);
+ sSockaddr += val;
+ }
+ sSockaddr += "/" + reqUrl.getScheme();
+ // insert carp forwarding header
+ bool bAddFwdHeaderResult = addHeader(bufp, hdr_loc, CARP_FORWARD_HEADER, sSockaddr);
+ if (!bAddFwdHeaderResult) {
+ TSError("Carp, error inserting '%s' header", CARP_FORWARD_HEADER.c_str());
+ }
+ }
+ } else { // for premap mode
+ string sScheme = reqUrl.getScheme();
+
+ if (!addHeader(bufp, hdr_loc, CARP_PREMAP_SCHEME, sScheme)) {
+ TSError("Carp, error inserting '%s' header in premap mode", CARP_PREMAP_SCHEME.c_str());
+ } else {
+ TSDebug(DEBUG_TAG_HOOK, "Insert client request scheme %s in premap mode", sScheme.c_str());
+ }
+ }
+ // set origin server/destination
+ // TSHttpTxnConfigIntSet(txnp, TS_CONFIG_HTTP_SHARE_SERVER_SESSIONS, 0); // disable conn sharing due to bug when used with TSHttpTxnServerAddrSet
+ if (TSHttpTxnServerAddrSet(txnp, reinterpret_cast<const struct sockaddr *> (&node->forwardAddr)) != TS_SUCCESS) {
+ TSDebug(DEBUG_TAG_HOOK, "Error calling TSHttpTxnServerAddrSet");
+ } else {
+ // set scheme appropriately based on destination
+ TSDebug(DEBUG_TAG_HOOK, "Setting scheme to '%s'", node->getSchemeString());
+ TSUrlSchemeSet(bufp, url_loc, node->getSchemeString(), -1);
+ if (!bIsPOSTRemap) { // since we are forwarding, do not remap request and do not cache result
+ TSSkipRemappingSet(txnp, true);
+ }
+ TSHttpTxnArgSet(txnp, g_carpSelectedHostArgIndex, static_cast<void *> (node));
+ if (!bIsOwner) {
+ TSHttpTxnServerRespNoStoreSet(txnp, 1); // do not cache the response
+ } else {
+ TSHttpTxnServerRespNoStoreSet(txnp, 0); // we are replicate owner, cache response
+ }
+ }
+ } else {
+ node->carpNoForward();
+ TSDebug(DEBUG_TAG_HOOK, "carp forwarded to self.");
+ }
+
+done :
+ // done w/buffers, release them
+ TSHandleMLocRelease(bufp, hdr_loc, url_loc);
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+}
+
+/**
+ * Convert ASCII hex digit to value of hex digit
+ * @param ch
+ * @return
+ */
+static unsigned char
+getValueOfHex(unsigned char ch)
+{
+ ch -= '0';
+ if(ch > 9) ch -= 7; // 'A' - ':' = 7
--- End diff --
Really? Use a magic constant then explain it, instead of just letting the compiler do the constant folding? Also, this doesn't work for lower case.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by SolidWallOfCode <gi...@git.apache.org>.
Github user SolidWallOfCode commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r74178621
--- Diff: plugins/experimental/carp/CarpConfig.cc ---
@@ -0,0 +1,581 @@
+/** @file
+
+ Loads the CARP configuration
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+
+//////////////////////////////////////////////////////////////
+// Read CARP configuration file
+// [Servers]
+// host1.yahoo.com:4080 weight=2 # port 4080 on host1.yahoo.com with weight factor of 2
+// host2.yahoo.com # port 80 on host2.yahoo.com with (default) weight factor of 1
+//
+// [Values]
+// healthcheck={host}:8001/status.html
+// healthfreq=30
+// global=on
+//
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <memory.h>
+#include <ts/ts.h>
+#include <sys/time.h>
+
+#include <sstream>
+
+#include "CarpConfig.h"
+#include "Common.h"
+#include "CarpConfigPool.h"
+
+using namespace std;
+
+#define DEFAULT_HEALTH_CHECK_FREQ 30 // 30 second period for health checks
+#define DEFAULT_HEALTH_CHECK_PORT 80 // default to makeing healthcheck requests against port 80
+#define DEFAULT_CONFIG_RELOAD_FREQ 30 // 30 seconds used in TSContSchedule
+#define DEFAULT_PORT 80 // default to makeing requests against port 80
+#define DEFAULT_WEIGHT 1 // default weight
+#define DEFAULT_SCHEME "http"
+#define DEFAULT_REPLICATION_FACTOR 1
+
+// config section headers
+static const char *const SECTION_SERVERS_STR = "[Servers]";
+static const char *const SECTION_VALUES_STR = "[Values]";
+
+// key strings
+static const char *const KEY_HEALTHCHECK_STR = "healthcheck";
+static const char *const KEY_HEALTHFREQ_STR = "healthfreq";
+static const char *const KEY_RELOADFREQ_STR = "reloadfreq";
+static const char *const KEY_HCTIMEOUT_STR = "hctimeout";
+static const char *const KEY_BLACKLIST_STR = "blacklist";
+static const char *const KEY_WHITELIST_STR = "whitelist";
+static const char *const KEY_MODE_STR = "mode";
+static const char *const KEY_ALLOWFWDPORT_STR = "allowfwdport";
+static const char *const KEY_REPLICATIONFACTOR_STR = "replicationfactor";
+
+// parameter strings
+static const char *const WEIGHT_EQUALS_STRING = "weight=";
+static const char *const GROUP_EQUALS_STRING = "group=";
+static const char *const KEY_MODE_PREREMAP_STR = "pre-remap";
+static const char *const KEY_MODE_POSTREMAP_STR = "post-remap";
+
+
+/**********************************************************/
+bool
+getInt(char** pptr, int *val)
+{
+ bool bReturn = false;
+ int v=0;
+
+ char* ptr = *pptr;
+ // skip white space if any
+ while (*ptr && isspace(*ptr)) ++ptr;
+ // get digits
+ if (*ptr) {
+ while (*ptr && isdigit(*ptr)) {
+ v *= 10;
+ v += (*ptr)-'0';
+ ++ptr;
+ bReturn = true;
+ }
+ }
+ if (bReturn) {
+ *pptr = ptr;
+ *val = v;
+ }
+
+ return bReturn;
+}
+
+/**********************************************************/
+// [http[s]://]host[:port]/path
+bool
+getHostAndPort(char** pptr,string* sHost,int* iPort, string* sScheme)
+{
+ bool bReturn = false;
+ char* ptr = *pptr;
+
+ *iPort = 80;
+ *sScheme = DEFAULT_SCHEME;
+ if(strncmp(*pptr,"https",5) == 0) {
+ *iPort = 443;
+ *sScheme = "https";
+ }
+
+ //skip leading white space
+ while (*ptr && isspace(*ptr)) ++ptr;
+
+ if (*ptr) { // validate not end of string
+ if(strncmp(ptr,"http://",7) == 0) {
+ ptr += 7;
+ } else if(strncmp(ptr,"https://",8) == 0) {
+ ptr += 8;
+ }
+
+ char* ptemp = ptr; // start of host
+ //find white space or ':' or '/'
+ while (*ptr && !isspace(*ptr) && *ptr != ':' && *ptr != '/') ++ptr;
+
+ *sHost = string(ptemp, (ptr - ptemp));
+ // skip white space (if any) after host
+ while (*ptr && isspace(*ptr)) ++ptr;
+ bReturn = true;
+
+ if (*ptr) {// have more to parse
+ // need to get port number?
+ if (*ptr == ':') {
+ ++ptr;
+ if (!getInt(&ptr, iPort)) {
+ // could be our special 'PORT' value, check for that
+ if(!strncmp(ptr,"{port}",6)) { // yes, is '{port}'
+ *iPort = -1;
+ bReturn = true;
+ } else { // really was an error
+ TSError("carp: error parsing port number from '%s'", *pptr);
+ bReturn = false;
+ }
+ } else {
+ // if port number is 443, treat the scheme as https
+ if (*iPort == 443) {
+ *sScheme = "https";
+ }
+ }
+ }
+ }
+ }
+ if(bReturn) {
+ *pptr = ptr;
+ }
+
+ return bReturn;
+}
+/**********************************************************/
+CarpConfig::CarpConfig()
+{
+ _healthCheckPort = DEFAULT_HEALTH_CHECK_PORT;
+ _healthCheckFreq = DEFAULT_HEALTH_CHECK_FREQ;
+ _configCheckFreq = DEFAULT_CONFIG_RELOAD_FREQ;
+ _healthCheckTimeout = DEFAULT_HEALTH_CHECK_TIMEOUT;
+ _setExit = 0;
+ _mode = PRE;
+ _allowForwardPort = 0;
+ _replicationFactor = DEFAULT_REPLICATION_FACTOR;
+ _nGroups = 0;
+}
+
+/**********************************************************/
+CarpConfig::~CarpConfig()
+{
+ for(size_t ptr = 0; ptr < _servers.size(); ptr++) {
+ delete _servers[ptr];
+ }
+
+ for(size_t ptr = 0; ptr < _httpClients.size(); ptr++) {
+ delete _httpClients[ptr];
+ }
+}
+
+/**********************************************************/
+bool
+CarpConfig::loadConfig(string filename)
+{
+ TSFile file;
+ GroupCountList group_counts;
+ GroupCountListIter group_counts_it;
+
+ // attempt to open config file assuming full path provided
+ TSDebug(DEBUG_TAG_INIT, "Trying to open config file in this path: %s", filename.c_str());
+ file = TSfopen(filename.c_str(), "r");
+ if (file == NULL) {
+ TSError("Failed to open carp config file %s", filename.c_str());
+ return false;
+ }
+
+ TSDebug(DEBUG_TAG_INIT, "Successfully opened config file");
+
+ char buffer[1024];
+ memset(buffer, 0, sizeof(buffer));
+
+ enum CONFIG_SECTION {
+ CFG_NONE_SECTION = 0,
+ CFG_SERVER_SECTION,
+ CFG_VALUES_SECTION
+ };
+ int cfg_section = CFG_NONE_SECTION;
+ bool done_parsing = false;
+ while (TSfgets(file, buffer, sizeof(buffer) - 1) != NULL && !done_parsing) {
+
+ char *eol = 0;
+ // make sure line was not bigger than buffer
+ if ((eol = strchr(buffer, '\n')) == NULL && (eol = strstr(buffer, "\r\n")) == NULL) {
+ TSError("carp config line was too long, did not get a good line in cfg, skipping, line: %s", buffer);
+ memset(buffer, 0, sizeof(buffer));
+ continue;
+ }
+
+ // make sure line has something useful on it
+ if (eol - buffer < 2 || buffer[0] == '#' || isspace(buffer[0])) {
+ memset(buffer, 0, sizeof(buffer));
+ continue;
+ }
+
+ // remove ending CR/LF
+ *eol = 0;
+
+ // check if we are changing sections
+ if (strncasecmp(buffer, SECTION_SERVERS_STR, strlen(SECTION_SERVERS_STR)) == 0) {
+ cfg_section = CFG_SERVER_SECTION;
+ TSDebug(DEBUG_TAG_INIT, "Parsing [Servers] section");
+ continue;
+ } else if (strncasecmp(buffer, SECTION_VALUES_STR, strlen(SECTION_VALUES_STR)) == 0) {
+ cfg_section = CFG_VALUES_SECTION;
+ TSDebug(DEBUG_TAG_INIT, "Parsing [Values] section");
+ continue;
+ }
+
+ //TSDebug(DEBUG_TAG_INIT, "config line input:'%s'", buffer);
+
+ switch (cfg_section) {
+ case CFG_SERVER_SECTION:
+ {
+ string sHost;
+ int iPort = DEFAULT_PORT;
+ int iWeight = DEFAULT_WEIGHT;
+ int iGroup = DEFAULT_GROUP;
+ string sScheme = DEFAULT_SCHEME;
+ bool bSuccess = true;
+
+ char* ptr = buffer;
+
+ if(!getHostAndPort(&ptr,&sHost,&iPort, &sScheme)) {
+ TSError("carp: error parsing port number from '%s'", ptr);
+ }
+
+ while (*ptr) {
+ // skip white space (if any)
+ while (*ptr && isspace(*ptr)) ++ptr;
+
+ if (*ptr) { // next we could find weight= or group=
+ if (strncmp(ptr, WEIGHT_EQUALS_STRING, strlen(WEIGHT_EQUALS_STRING)) == 0) {
+ ptr += strlen(WEIGHT_EQUALS_STRING);
+ if (!getInt(&ptr, &iWeight)) {
+ TSError("carp: error parsing weight value from '%s'", buffer);
+ bSuccess = false;
+ continue;
+ }
+ } else if (strncmp(ptr, GROUP_EQUALS_STRING, strlen(GROUP_EQUALS_STRING)) == 0) {
+ ptr += strlen(GROUP_EQUALS_STRING);
+ if (!getInt(&ptr, &iGroup)) {
+ TSError("carp: error parsing group value from '%s'", buffer);
+ bSuccess = false;
+ continue;
+ }
+ } else {
+ TSError("carp: error parsing from line '%s'", buffer);
+ // malformed entry, skip to next space
+ while (*ptr && !isspace(*ptr)) ++ptr;
+ bSuccess = false;
+ continue;
+ }
+ }
+ }
+
+ if (!bSuccess) continue;
+
+ group_counts_it = group_counts.find(iGroup);
+ if (group_counts_it != group_counts.end()) {
+ group_counts[iGroup]++;
+ } else {
+ _nGroups++;
+ group_counts[iGroup] = 1;
+ }
+
+ TSDebug(DEBUG_TAG_INIT, "Host = %s, port=%d, weight=%d, group=%d", sHost.c_str(), iPort, iWeight, iGroup);
+ CarpHost* host=new CarpHost(sHost, iPort, sScheme, iWeight, iGroup);
+ TSAssert(host != NULL);
+ // store the parsed data
+ addHost(host);
+ break;
+ }
+ case CFG_VALUES_SECTION:
+ {
+ char* ptr = buffer;
+
+ //skip leading white space
+ while (*ptr && isspace(*ptr)) ++ptr;
+
+ //find end of key
+ while (*ptr && !isspace(*ptr) && *ptr != '=') ++ptr;
+
+ string sKey = string(buffer, (ptr - buffer));
+
+ // skip white space (if any) after key
+ while (*ptr && isspace(*ptr)) ++ptr;
+ if (*ptr != '=') {
+ TSError("carp: expecting '=' after key in line '%s'", buffer);
+ continue;
+ }
+
+ // skip '=' and white space (if any) after '='
+ ++ptr;
+ while (*ptr && isspace(*ptr)) ++ptr;
+
+ char* ptemp = ptr; // get start of value
+
+ // find end of value
+ while (*ptr && !isspace(*ptr)) ++ptr;
+ string sValue = string(ptemp, ptr - ptemp);
+
+ TSDebug(DEBUG_TAG_INIT, "Key=%s Value=%s", sKey.c_str(), sValue.c_str());
+
+ char* pTemp = (char *)sValue.c_str();
+ if (sKey.compare(KEY_HEALTHCHECK_STR) == 0) {
+ string sH;
+ int iP;
+ string scheme;
+ _healthCheckUrl = string(pTemp);
+ if (!getHostAndPort(&pTemp, &sH, &iP, &scheme)) {
+ TSError("carp: error parsing host and/or port number from '%s'", buffer);
+ }
+
+ // store the parsed data
+ _healthCheckPort = iP;
+ TSDebug(DEBUG_TAG_INIT, "healthcheck Url=%s port=%d", _healthCheckUrl.c_str(), _healthCheckPort);
+
+ } else if (sKey.compare(KEY_HEALTHFREQ_STR) == 0) {
+ int iFreq;
+ if (!getInt(&pTemp, &iFreq)) {
+ TSError("carp: error parsing number from '%s'", buffer);
+ } else {
+ TSDebug(DEBUG_TAG_INIT, "healthcheck freq=%d", iFreq);
+ // store the parsed data
+ _healthCheckFreq = iFreq;
+ }
+ } else if (sKey.compare(KEY_HCTIMEOUT_STR) == 0) {
+ int iFreq;
+ if (!getInt(&pTemp, &iFreq)) {
+ TSError("carp: error parsing number from '%s'", buffer);
+ } else {
+ TSDebug(DEBUG_TAG_INIT, "healthcheck timeout value=%d", iFreq);
+ // store the parsed data
+ _healthCheckTimeout = iFreq;
+ }
+ } else if (sKey.compare(KEY_RELOADFREQ_STR) == 0) {
+ int lFreq;
+ if (!getInt(&pTemp, &lFreq)) {
+ TSError("carp: error parsing number from '%s'", buffer);
+ } else {
+ TSDebug(DEBUG_TAG_INIT, "config reload freq=%d", lFreq);
+ _configCheckFreq = lFreq;
+ }
+ } else if (sKey.compare(KEY_BLACKLIST_STR) == 0) {
+ vector<string> results;
+ stringExplode(sValue, string(","), &results);
+ for (vector<string>::iterator it = results.begin(); it != results.end();
+ it++) {
+ TSDebug(DEBUG_TAG_INIT, "Adding blacklist hostname %s",
+ (*it).c_str());
+ _blackList.insert(*it);
+ }
+ } else if (sKey.compare(KEY_WHITELIST_STR) == 0) {
+ vector<string> results;
+ stringExplode(sValue, string(","), &results);
+ for(vector<string>::iterator it=results.begin(); it != results.end(); it++) {
+ TSDebug(DEBUG_TAG_INIT, "Adding whitelist hostname %s",(*it).c_str() );
+ _whiteList.insert(*it);
+ }
+ } else if (sKey.compare(KEY_MODE_STR) == 0) {
+ if(sValue.compare(KEY_MODE_PREREMAP_STR) == 0) {
+ _mode = PRE;
+ } else if(sValue.compare(KEY_MODE_POSTREMAP_STR) == 0) {
+ _mode = POST;
+ } else {
+ TSError("carp: invalid mode in '%s'", buffer);
+ }
+ } else if (sKey.compare(KEY_ALLOWFWDPORT_STR) == 0) {
+ int iPort;
+ if (!getInt(&pTemp, &iPort)) {
+ TSError("carp: error parsing number from '%s'", buffer);
+ }
+ else {
+ TSDebug(DEBUG_TAG_INIT, "Allow forwarding port=%d", iPort);
+ // store the parsed data
+ _allowForwardPort = iPort;
+ }
+ } else if (sKey.compare(KEY_REPLICATIONFACTOR_STR) == 0) {
+ int iFactor;
+ if (!getInt(&pTemp, &iFactor)) {
+ TSError("carp: error parsing number from '%s'", buffer);
+ }
+ else {
+ TSDebug(DEBUG_TAG_INIT, "Replication factor=%d", iFactor);
+ // store the parsed data
+ _replicationFactor = iFactor;
+ }
+
+ } else
+ TSError("carp found bad setting under Values section '%s'", buffer);
+ break;
+ }
+ default:
+ // ignore bad cfg_section
+ TSDebug(DEBUG_TAG_INIT, "hit default in switch, ignoring extra input '%s'",buffer);
+ break;
+ };
+ } //while
+
+ if (_healthCheckTimeout > _healthCheckFreq - 1 ) {
+ _healthCheckTimeout = _healthCheckFreq - 1;
+ }
+
+ TSfclose(file);
+
+ if (_blackList.size() && _whiteList.size() ) {
+ TSError("Carp configured with both blacklist and whitelist, blacklist will be ignored");
+ }
+
+ if (_nGroups > _replicationFactor) {
+ TSError("Too many groups configured! Failing config.");
+ return false;
+ } else {
+ TSDebug(DEBUG_TAG_INIT, "Group Config is as follows:");
+ for (map<int, int>::const_iterator it = group_counts.begin(); it != group_counts.end(); it++) {
+ TSDebug(DEBUG_TAG_INIT, "Group %d has %d members.", it->first, it->second);
+ _group_count_list[it->first] = it->second;
+ }
+ }
+
+ return true;
+}
+
+/**********************************************************/
+void
+CarpConfig::addHost(CarpHost* host) {
+ _servers.push_back(host);
+}
+
+void
+CarpConfig::addHealthCheckClient(HttpFetch *client) {
+ client->setHealthcheckTimeout(_healthCheckTimeout);
+ _httpClients.push_back(client);
+
+}
+
+void
+CarpConfig::setPath(string path) {
+ _configPath = path;
+}
+
+string
+CarpConfig::getPath() {
+ return _configPath;
+}
+
+/**********************************************************/
+void *
+CarpConfigHealthCheckThreadStart(void* data)
+{
+ CarpConfigAndHash* cch = static_cast<CarpConfigAndHash *> (data);
+ TSAssert(cch);
+ return cch->_config->run(cch->_hashAlgo);
+}
+
+/**********************************************************/
+bool
+CarpConfig::isBlackListed(const string& sHost)
+{
+ if(!_blackList.size()) return false;
+ return (_blackList.find(sHost) != _blackList.end());
+}
+
+/**********************************************************/
+bool
+CarpConfig::isWhiteListed(const string& sHost)
+{
+ if(!_whiteList.size()) return false;
+ return (_whiteList.find(sHost) != _whiteList.end());
+}
+
+/**********************************************************/
+/*
+ perform healthchecks on the hosts and mark them up/down
+*/
+
+void*
+CarpConfig::run(HashAlgorithm *hash) {
+
+ // every httpClient would send a health request to each peer
+ // would be nice to not wait and just 'go' when the server is 100% up
+ sleep(5);
--- End diff --
Is there some lifecycle hook that would be useful for this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by ericcarlschwartz <gi...@git.apache.org>.
Github user ericcarlschwartz commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r74289748
--- Diff: plugins/experimental/carp/CarpConfig.cc ---
@@ -0,0 +1,581 @@
+/** @file
+
+ Loads the CARP configuration
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+
+//////////////////////////////////////////////////////////////
+// Read CARP configuration file
+// [Servers]
+// host1.yahoo.com:4080 weight=2 # port 4080 on host1.yahoo.com with weight factor of 2
+// host2.yahoo.com # port 80 on host2.yahoo.com with (default) weight factor of 1
+//
+// [Values]
+// healthcheck={host}:8001/status.html
+// healthfreq=30
+// global=on
+//
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <memory.h>
+#include <ts/ts.h>
+#include <sys/time.h>
+
+#include <sstream>
+
+#include "CarpConfig.h"
+#include "Common.h"
+#include "CarpConfigPool.h"
+
+using namespace std;
+
+#define DEFAULT_HEALTH_CHECK_FREQ 30 // 30 second period for health checks
+#define DEFAULT_HEALTH_CHECK_PORT 80 // default to makeing healthcheck requests against port 80
+#define DEFAULT_CONFIG_RELOAD_FREQ 30 // 30 seconds used in TSContSchedule
+#define DEFAULT_PORT 80 // default to makeing requests against port 80
+#define DEFAULT_WEIGHT 1 // default weight
+#define DEFAULT_SCHEME "http"
+#define DEFAULT_REPLICATION_FACTOR 1
+
+// config section headers
+static const char *const SECTION_SERVERS_STR = "[Servers]";
+static const char *const SECTION_VALUES_STR = "[Values]";
+
+// key strings
+static const char *const KEY_HEALTHCHECK_STR = "healthcheck";
+static const char *const KEY_HEALTHFREQ_STR = "healthfreq";
+static const char *const KEY_RELOADFREQ_STR = "reloadfreq";
+static const char *const KEY_HCTIMEOUT_STR = "hctimeout";
+static const char *const KEY_BLACKLIST_STR = "blacklist";
+static const char *const KEY_WHITELIST_STR = "whitelist";
+static const char *const KEY_MODE_STR = "mode";
+static const char *const KEY_ALLOWFWDPORT_STR = "allowfwdport";
+static const char *const KEY_REPLICATIONFACTOR_STR = "replicationfactor";
+
+// parameter strings
+static const char *const WEIGHT_EQUALS_STRING = "weight=";
+static const char *const GROUP_EQUALS_STRING = "group=";
+static const char *const KEY_MODE_PREREMAP_STR = "pre-remap";
+static const char *const KEY_MODE_POSTREMAP_STR = "post-remap";
+
+
+/**********************************************************/
+bool
+getInt(char** pptr, int *val)
+{
+ bool bReturn = false;
+ int v=0;
+
+ char* ptr = *pptr;
+ // skip white space if any
+ while (*ptr && isspace(*ptr)) ++ptr;
+ // get digits
+ if (*ptr) {
+ while (*ptr && isdigit(*ptr)) {
+ v *= 10;
+ v += (*ptr)-'0';
+ ++ptr;
+ bReturn = true;
+ }
+ }
+ if (bReturn) {
+ *pptr = ptr;
+ *val = v;
+ }
+
+ return bReturn;
+}
+
+/**********************************************************/
+// [http[s]://]host[:port]/path
+bool
+getHostAndPort(char** pptr,string* sHost,int* iPort, string* sScheme)
+{
+ bool bReturn = false;
+ char* ptr = *pptr;
+
+ *iPort = 80;
+ *sScheme = DEFAULT_SCHEME;
+ if(strncmp(*pptr,"https",5) == 0) {
--- End diff --
ah definitely
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by SolidWallOfCode <gi...@git.apache.org>.
Github user SolidWallOfCode commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r74256058
--- Diff: plugins/experimental/carp/CarpConfigPool.cc ---
@@ -0,0 +1,218 @@
+/** @file
+
+ Manage a list of CARP configurations
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+#include <sstream>
+
+#include "Common.h"
+#include "CarpConfigPool.h"
+#include "UrlComponents.h"
+#include "CarpHost.h"
+
+#include <netdb.h>
+#include <ts/ts.h>
+
+using namespace std;
+
+/*******************************************************************/
+CarpConfigPool::CarpConfigPool()
+{
+ _globalHash = NULL;
+ _globalConfig = NULL;
+}
+
+/*******************************************************************/
+CarpConfigPool::~CarpConfigPool()
+{
+ for(CarpConfigListIter it=_configList.begin(); it != _configList.end(); it++) {
+ it->second->_config->stop();
+ delete it->second;
+ }
+}
+/*******************************************************************/
+static int initCarpConfigAndHash(CarpConfigAndHash * cch, string sFilename) {
+ cch->_configPath = sFilename;
+ cch->_config = new CarpConfig();
+ TSAssert(cch->_config);
+
+ if (!cch->_config->loadConfig(sFilename)) {
+ return -1;
+ }
+
+ cch->_hashAlgo = new CarpHashAlgorithm(cch->_config);
+
+ TSAssert(cch->_hashAlgo);
+
+ CarpHostList* hostList = cch->_config->getHostList();
+ // add hosts, etc to hash algo
+ char szServerName[256];
+ *szServerName = 0;
+
+ if (!gethostname(szServerName, 255)) { // success!
+ TSDebug(DEBUG_TAG_INIT, "using %s as server name to detect 'self'",
+ szServerName);
+ }
+ char buf[1024];
+ struct hostent self, *selfhe = getHostIp(string(szServerName), &self, buf,
+ sizeof(buf));
+
+ for (CarpHostListIter i = hostList->begin(); i != hostList->end(); i++) {
+ HashNode *hashNode;
+ bool bSelf = false;
+ if (NULL != selfhe) { // check for 'self'
+ // include hostname and port
+ bSelf = isSelf((*i)->getName(), (*i)->getPort(), selfhe);
+ }
+ if (cch->_config->getHealthCheckPort() == -1) { // did they specify 'PORT'?
+ (*i)->setHealthCheckPort((*i)->getPort()); // set HC port from server spec'd port
+ } else {
+ (*i)->setHealthCheckPort(cch->_config->getHealthCheckPort()); // set HC port
+ }
+ string sHCUrl = cch->_config->getHealthCheckUrl();
+ size_t pos = sHCUrl.find("{port}");
+ if (pos != string::npos) { // need to replace '{port}' with servers port
+ stringstream ss;
+ ss << (*i)->getPort();
+ sHCUrl.replace(pos, 6, ss.str()); // 6 = strlen of '{port}'
+ }
+ pos = sHCUrl.find("{host}");
+ if (pos != string::npos) {
+ sHCUrl.replace(pos, 6, (*i)->getName());
+ }
+ (*i)->setHealthCheckUrl(sHCUrl); // set HC Url
+
+ // Look up host and create addr struct for healthchecks
+ char hBuf[1024];
+ struct hostent host, *hosthe = getHostIp((*i)->getName(), &host, hBuf,
+ sizeof(hBuf));
+ if (hosthe) {
+ // convert hostent to sockaddr_in structure
+ sockaddr_in hIn;
+ memcpy(&hIn.sin_addr, hosthe->h_addr_list[0], hosthe->h_length);
+ hIn.sin_port = htons((*i)->getHealthCheckPort()); // set port
+ if (hosthe->h_length == 4) { // size match IPv4? assume such
--- End diff --
If it's really an IP address the family member of the sockaddr should be checked, not the length.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by SolidWallOfCode <gi...@git.apache.org>.
Github user SolidWallOfCode commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r74176123
--- Diff: plugins/experimental/carp/CarpConfig.cc ---
@@ -0,0 +1,581 @@
+/** @file
+
+ Loads the CARP configuration
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+
+//////////////////////////////////////////////////////////////
+// Read CARP configuration file
+// [Servers]
+// host1.yahoo.com:4080 weight=2 # port 4080 on host1.yahoo.com with weight factor of 2
+// host2.yahoo.com # port 80 on host2.yahoo.com with (default) weight factor of 1
+//
+// [Values]
+// healthcheck={host}:8001/status.html
+// healthfreq=30
+// global=on
+//
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <memory.h>
+#include <ts/ts.h>
+#include <sys/time.h>
+
+#include <sstream>
+
+#include "CarpConfig.h"
+#include "Common.h"
+#include "CarpConfigPool.h"
+
+using namespace std;
+
+#define DEFAULT_HEALTH_CHECK_FREQ 30 // 30 second period for health checks
+#define DEFAULT_HEALTH_CHECK_PORT 80 // default to makeing healthcheck requests against port 80
+#define DEFAULT_CONFIG_RELOAD_FREQ 30 // 30 seconds used in TSContSchedule
+#define DEFAULT_PORT 80 // default to makeing requests against port 80
+#define DEFAULT_WEIGHT 1 // default weight
+#define DEFAULT_SCHEME "http"
+#define DEFAULT_REPLICATION_FACTOR 1
+
+// config section headers
+static const char *const SECTION_SERVERS_STR = "[Servers]";
+static const char *const SECTION_VALUES_STR = "[Values]";
+
+// key strings
+static const char *const KEY_HEALTHCHECK_STR = "healthcheck";
+static const char *const KEY_HEALTHFREQ_STR = "healthfreq";
+static const char *const KEY_RELOADFREQ_STR = "reloadfreq";
+static const char *const KEY_HCTIMEOUT_STR = "hctimeout";
+static const char *const KEY_BLACKLIST_STR = "blacklist";
+static const char *const KEY_WHITELIST_STR = "whitelist";
+static const char *const KEY_MODE_STR = "mode";
+static const char *const KEY_ALLOWFWDPORT_STR = "allowfwdport";
+static const char *const KEY_REPLICATIONFACTOR_STR = "replicationfactor";
+
+// parameter strings
+static const char *const WEIGHT_EQUALS_STRING = "weight=";
+static const char *const GROUP_EQUALS_STRING = "group=";
+static const char *const KEY_MODE_PREREMAP_STR = "pre-remap";
+static const char *const KEY_MODE_POSTREMAP_STR = "post-remap";
+
+
+/**********************************************************/
+bool
+getInt(char** pptr, int *val)
+{
+ bool bReturn = false;
+ int v=0;
+
+ char* ptr = *pptr;
+ // skip white space if any
+ while (*ptr && isspace(*ptr)) ++ptr;
+ // get digits
+ if (*ptr) {
+ while (*ptr && isdigit(*ptr)) {
+ v *= 10;
+ v += (*ptr)-'0';
+ ++ptr;
+ bReturn = true;
+ }
+ }
+ if (bReturn) {
+ *pptr = ptr;
+ *val = v;
+ }
+
+ return bReturn;
+}
+
+/**********************************************************/
+// [http[s]://]host[:port]/path
+bool
+getHostAndPort(char** pptr,string* sHost,int* iPort, string* sScheme)
+{
+ bool bReturn = false;
+ char* ptr = *pptr;
+
+ *iPort = 80;
+ *sScheme = DEFAULT_SCHEME;
+ if(strncmp(*pptr,"https",5) == 0) {
+ *iPort = 443;
+ *sScheme = "https";
+ }
+
+ //skip leading white space
+ while (*ptr && isspace(*ptr)) ++ptr;
+
+ if (*ptr) { // validate not end of string
+ if(strncmp(ptr,"http://",7) == 0) {
+ ptr += 7;
+ } else if(strncmp(ptr,"https://",8) == 0) {
+ ptr += 8;
+ }
+
+ char* ptemp = ptr; // start of host
+ //find white space or ':' or '/'
+ while (*ptr && !isspace(*ptr) && *ptr != ':' && *ptr != '/') ++ptr;
+
+ *sHost = string(ptemp, (ptr - ptemp));
+ // skip white space (if any) after host
+ while (*ptr && isspace(*ptr)) ++ptr;
+ bReturn = true;
+
+ if (*ptr) {// have more to parse
+ // need to get port number?
+ if (*ptr == ':') {
+ ++ptr;
+ if (!getInt(&ptr, iPort)) {
+ // could be our special 'PORT' value, check for that
+ if(!strncmp(ptr,"{port}",6)) { // yes, is '{port}'
+ *iPort = -1;
+ bReturn = true;
+ } else { // really was an error
+ TSError("carp: error parsing port number from '%s'", *pptr);
+ bReturn = false;
+ }
+ } else {
+ // if port number is 443, treat the scheme as https
+ if (*iPort == 443) {
+ *sScheme = "https";
+ }
+ }
+ }
+ }
+ }
+ if(bReturn) {
+ *pptr = ptr;
+ }
+
+ return bReturn;
+}
+/**********************************************************/
+CarpConfig::CarpConfig()
+{
+ _healthCheckPort = DEFAULT_HEALTH_CHECK_PORT;
+ _healthCheckFreq = DEFAULT_HEALTH_CHECK_FREQ;
+ _configCheckFreq = DEFAULT_CONFIG_RELOAD_FREQ;
+ _healthCheckTimeout = DEFAULT_HEALTH_CHECK_TIMEOUT;
+ _setExit = 0;
+ _mode = PRE;
+ _allowForwardPort = 0;
+ _replicationFactor = DEFAULT_REPLICATION_FACTOR;
+ _nGroups = 0;
+}
+
+/**********************************************************/
+CarpConfig::~CarpConfig()
+{
+ for(size_t ptr = 0; ptr < _servers.size(); ptr++) {
+ delete _servers[ptr];
+ }
+
+ for(size_t ptr = 0; ptr < _httpClients.size(); ptr++) {
+ delete _httpClients[ptr];
+ }
+}
+
+/**********************************************************/
+bool
+CarpConfig::loadConfig(string filename)
+{
+ TSFile file;
+ GroupCountList group_counts;
+ GroupCountListIter group_counts_it;
+
+ // attempt to open config file assuming full path provided
--- End diff --
So the carp config file can't be accessed via a relative path based in the TS config file directory?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by SolidWallOfCode <gi...@git.apache.org>.
Github user SolidWallOfCode commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r75874123
--- Diff: plugins/experimental/carp/Common.cc ---
@@ -0,0 +1,364 @@
+
+/** @file
+
+ Loads the CARP configuration
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+#include "Common.h"
+
+#include <arpa/inet.h>
+#include <stdlib.h>
+#include <stdio.h>
+#include <errno.h>
+#include <netdb.h>
+#include <string.h>
+#include <memory.h>
+
+using namespace std;
+
+/************************************************************************/
+void stringExplode(string str, string separator, vector<string>* results)
+{
+ size_t found;
+ found = str.find_first_of(separator);
+ while (found != std::string::npos)
+ {
+ if (found > 0)
+ {
+ results->push_back(str.substr(0, found));
+ }
+ str = str.substr(found + 1);
+ found = str.find_first_of(separator);
+ }
+ if (str.length() > 0)
+ {
+ results->push_back(str);
+ }
+}
+
+/************************************************************************/
+/*
+ Parse /proc/{pid}/net/tcp
+ sl local_address rem_address st tx_queue rx_queue tr tm->when retrnsmt uid timeout inode
+ 0: 00000000:036B 00000000:0000 0A 00000000:00000000 00:00000000 00000000 0 0 22272 1 ffff8801b7ecb700 299 0 0 2 -1
+ 1: 00000000:08AE 00000000:0000 0A 00000000:00000000 00:00000000 00000000 0 0 10621 1 ffff8801b7b9a080 299 0 0 2 -1
+ 2: 00000000:006F 00000000:0000 0A 00000000:00000000 00:00000000 00000000 0 0 22046 1 ffff8801b7b9b400 299 0 0 2 -1
+ Parse /proc/{pid}/net/tcp6
+ * sl local_address remote_address st tx_queue rx_queue tr tm->when retrnsmt uid timeout inode
+ 0: 00000000000000000000000000000000:08AE 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000 0 0 10619 1 ffff8801b266d780 299 0 0 2 -1
+ 1: 00000000000000000000000000000000:006F 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000 0 0 22051 1 ffff8801b3a51880 299 0 0 2 -1
+ 2: 00000000000000000000000000000000:0016 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000 0 0 10656 1 ffff8801b266d040 299 0 0 2 -1
+ 3: 00000000000000000000000000000000:8E77 00000000000000000000000000000000:0000 0A 00000000:00000000 00:00000000 00000000 0 0 22300 1 ffff8801b2f5c780 299 0 0 2 -1
+
+ looking for local_address:PORT wher st = LISTEN (0x0A)
+ * ./include/net/tcp_states.h
+ * enum {
+ TCP_ESTABLISHED = 1,
+ TCP_SYN_SENT,
+ TCP_SYN_RECV,
+ TCP_FIN_WAIT1,
+ TCP_FIN_WAIT2,
+ TCP_TIME_WAIT,
+ TCP_CLOSE,
+ TCP_CLOSE_WAIT,
+ TCP_LAST_ACK,
+ TCP_LISTEN,
+ TCP_CLOSING,
+
+ TCP_MAX_STATES
+ */
+
+/************************************************************************/
+
+/* used internally */
+bool
+scanProcFileForPort(string sFilename, string sPid, unsigned int iPort)
+{
+ bool bMatch = false;
+ TSFile file;
+ int iLine = 0;
+ bool bParseDone = false;
+
+ // attempt to open file
+ TSDebug(DEBUG_TAG_INIT, "Trying to open proc file @ %s to determine listening ports", sFilename.c_str());
+ file = TSfopen(sFilename.c_str(), "r");
+ if (file == NULL) {
+ TSError("Failed to open proc tcp file of %s. Error=%s", sFilename.c_str(), strerror(errno));
+ return false;
+ }
+
+ TSDebug(DEBUG_TAG_INIT, "Successfully opened %s file", sFilename.c_str());
+
+ char buffer[1024];
+ memset(buffer, 0, sizeof (buffer));
+
+ while (TSfgets(file, buffer, sizeof (buffer) - 1) != NULL && !bParseDone) {
+ char *eol = strstr(buffer, "\r\n");
+ if (!eol)
+ eol = strchr(buffer, '\n');
+ if (eol)
+ *eol = 0; // remove ending LF or CRLF
+
+ // TSDebug(DEBUG_TAG_INIT, "Parsing line: %s", buffer);
+ ++iLine;
+ if (iLine == 1) { // ignore header line
+ continue;
+ }
+ vector<string> vParts;
+ stringExplode(string(buffer), string(" "), &vParts);
+ // part 3 is status
+ unsigned int iStatus = 0;
+ sscanf(vParts[3].c_str(), "%x", &iStatus);
+ if (iStatus == TCP_LISTEN) {
+ vector<string> vLocalParts;
+ stringExplode(vParts[1], string(":"), &vLocalParts); // part 1 is localaddr
+ unsigned int iLPort = 0;
+ sscanf(vLocalParts[1].c_str(), "%x", &iLPort);
+ TSDebug(DEBUG_TAG_INIT, "Found listening port %d", iLPort);
+ if (iLPort == iPort) {
+ string sLabel = string("socket:[" + vParts[9] + "]");
+ // find inode in /proc/{pid}/fd/
+ string sDir = "/proc/" + string(sPid) + "/fd";
+
+ DIR *dp;
+ struct dirent *dirp;
+ if ((dp = opendir(sDir.c_str())) == NULL) {
+ TSDebug(DEBUG_TAG_INIT, "Failed to open directory %s, %s", sDir.c_str(), strerror(errno));
+ continue;
+ }
+
+ while ((dirp = readdir(dp)) != NULL && !bParseDone) {
+ //TSDebug(DEBUG_TAG_INIT, "File in %s = %s inode=%d", sDir.c_str(),dirp->d_name,dirp->d_ino);
+ string sPath = sDir + "/" + dirp->d_name;
+ char sBuf[256];
+ ssize_t iCount = readlink(sPath.c_str(), sBuf, sizeof (sBuf));
+ if (iCount > 0) {
+ string sLink(sBuf, iCount);
+ if (sLink.compare(sLabel) == 0) {
+ TSDebug(DEBUG_TAG_INIT, "Found that port %d is opened for listening by pid %s", iLPort, sPid.c_str());
+ bParseDone = true;
+ bMatch = true;
+ }
+ }
+ }
+ closedir(dp);
+ }
+ }
+ /*
+ for(int i=0;i<vParts.size();i++)
+ {
+ TSDebug(DEBUG_TAG_INIT, "part[%d]='%s'", i, vParts[i].c_str());
+ }
+ */
+ }
+
+ TSfclose(file);
+
+ return bMatch;
+}
+
+/************************************************************************/
+bool
+isPortSelf(unsigned int iPort)
+{
+ pid_t pid = getpid();
+ char sPid[10];
+ bool bMatch = false;
+
+ string sFileName;
+ // open file @ /proc/{pid}/net/tcp
+
+
+ sprintf(sPid,"%d",pid);
+
+ // look for IPv4 listener first
+ sFileName = "/proc/" + string(sPid) + "/net/tcp";
+ bMatch = scanProcFileForPort(sFileName,sPid, iPort);
+ if(!bMatch) { // did not find IPv4 listener, check for IPv6
+ sFileName = "/proc/" + string(sPid) + "/net/tcp6";
+ bMatch = scanProcFileForPort(sFileName,sPid, iPort);
+ }
+ return bMatch;
+}
+
+/************************************************************************/
+struct hostent *
+getHostIp(string hName, struct hostent *h, char *buf, int buflen)
+{
+ int res, err;
+ struct hostent *hp = NULL;
+
+ res = gethostbyname_r(hName.c_str(), h, buf, buflen, &hp, &err);
+ if ((res == 0) && (hp != NULL)) {
+ return hp;
+ } else {
+ TSDebug(DEBUG_TAG_INIT, "gethostbyname_r failed for %s. Error=%s", hName.c_str(), strerror(err));
+ }
+ return NULL;
+}
+
+
+/************************************************************************/
+/*
+ * *** WARNING *** You will need to run the carp plugin with traffic_manager for it to detect
+ * itself and to forward directly to the origin. It will not work by running traffic_server directly!
+ */
--- End diff --
Why? Because `traffic_manager` opens the ports and so only if it is running will the port scan work? In that case, why not wait until `TS_LIFECYCLE_PORTS_READY_HOOK`?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by ericcarlschwartz <gi...@git.apache.org>.
Github user ericcarlschwartz commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r75919006
--- Diff: plugins/experimental/carp/carp.cc ---
@@ -0,0 +1,713 @@
+/** @file
+
+ A brief file description
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+
+#include <errno.h>
+
+#include <string>
+#include <sstream>
+#include <stdlib.h>
+#include <stdio.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+#include <unistd.h>
+
+#include <memory.h>
+
+#include <ts/ts.h>
+
+#include "Common.h"
+#include "CarpConfig.h"
+#include "CarpConfigPool.h"
+#include "CarpHashAlgorithm.h"
+#include "UrlComponents.h"
+
+using namespace std;
+
+CarpConfigPool* g_CarpConfigPool = NULL;
+int g_carpSelectedHostArgIndex = 0;
+TSTextLogObject g_logObject = NULL;
+
+const char *logFileName = "carp";
+
+//////////////////////////////////////////////////////////////////////////////
+//////////////////////////////////////////////////////////////////////////////
+/*
+ check for our carp routed header, dump status if requested
+ */
+static int
+processCarpRoutedHeader(TSHttpTxn txnp, TSMBuffer bufp, TSMLoc hdr_loc)
+{
+ string value;
+ if (getHeader(bufp, hdr_loc, CARP_ROUTED_HEADER, value)) { // if found header
+ if (value.compare("1") == 0) { // is loop prevention value
+ TSDebug(DEBUG_TAG_HOOK, "Found %s header with loop prevention value, not forwarding again", CARP_ROUTED_HEADER.c_str());
+ return 0;
+ } else if (value.compare("dump") == 0) { // is dump status request
+ TSDebug(DEBUG_TAG_HOOK, "Found %s header with dump request", CARP_ROUTED_HEADER.c_str());
+ string status;
+ g_CarpConfigPool->getGlobalHashAlgo()->dump(status);
+ TSHttpTxnSetHttpRetStatus(txnp, TS_HTTP_STATUS_MULTI_STATUS);
+ TSHttpTxnErrorBodySet(txnp, TSstrdup(status.c_str()), status.length(), NULL);
+ return -1;
+ }
+ TSDebug(DEBUG_TAG_HOOK, "Found %s header with unknown value of %s, ignoring", CARP_ROUTED_HEADER.c_str(), value.c_str());
+ removeHeader(bufp, hdr_loc, CARP_ROUTED_HEADER);
+ }
+ return 1; // all OK
+}
+
+static bool
+checkListForSelf(std::vector<HashNode *> list)
+{
+ for (size_t k = 0; k < list.size(); k++) {
+ if (list[k]->isSelf) return true;
+ }
+ return false;
+}
+
+/**
+ bIsPOSTRemap = false --- Hash request and forward to peer
+ bIsPOSTRemap = true --- hash request, extract OS sockaddr, insert forwarding header, forward
+ */
+static int
+handleRequestProcessing(TSCont contp, TSEvent event, void *edata, bool bIsPOSTRemap)
+{
+ TSHttpTxn txnp = (TSHttpTxn) edata;
+ TSMBuffer bufp;
+ TSMLoc hdr_loc;
+ TSMLoc url_loc;
+
+ // get the client request so we can get URL and add header
+ if (TSHttpTxnClientReqGet(txnp, &bufp, &hdr_loc) != TS_SUCCESS) {
+ TSError("carp couldn't get request headers");
+ return -1;
+ }
+
+ int method_len;
+ const char *method = TSHttpHdrMethodGet(bufp, hdr_loc, &method_len);
+ if (NULL == method) {
+ TSError("carp couldn't get http method");
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return -1;
+ }
+ if (((method_len == TS_HTTP_LEN_DELETE) && (strncasecmp(method, TS_HTTP_METHOD_DELETE, TS_HTTP_LEN_DELETE) == 0)) ||
+ ((method_len == TS_HTTP_LEN_PURGE) && (strncasecmp(method, TS_HTTP_METHOD_PURGE, TS_HTTP_LEN_PURGE) == 0))) {
+ TSDebug(DEBUG_TAG_HOOK, "Request method is '%s' so not routing request", string(method,method_len).c_str());
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ }
+
+ if (TSHttpHdrUrlGet(bufp, hdr_loc, &url_loc) != TS_SUCCESS) {
+ TSError("carp couldn't get url");
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return -1;
+ }
+ // if has carp loop prevention header, do not remap
+ int iTemp = processCarpRoutedHeader(txnp, bufp, hdr_loc);
+ if(iTemp <= 0) { // if dump or do not remap
+ // check origin client request's scheme for premap mode
+ if (!bIsPOSTRemap) {
+ string oriScheme;
+ if (!getHeader(bufp, hdr_loc, CARP_PREMAP_SCHEME, oriScheme)) {
+ TSDebug(DEBUG_TAG_HOOK, "couldn't get '%s' header", CARP_PREMAP_SCHEME.c_str());
+ } else {
+ bool isHttps = (oriScheme == TS_URL_SCHEME_HTTPS);
+
+ if (isHttps) {
+ TSUrlSchemeSet(bufp, url_loc, TS_URL_SCHEME_HTTPS, TS_URL_LEN_HTTPS);
+ } else {
+ TSUrlSchemeSet(bufp, url_loc, TS_URL_SCHEME_HTTP, TS_URL_LEN_HTTP);
+ }
+
+ removeHeader(bufp, hdr_loc, CARP_STATUS_HEADER);
+ removeHeader(bufp, hdr_loc, CARP_ROUTED_HEADER);
+ removeHeader(bufp, hdr_loc, CARP_PREMAP_SCHEME.c_str());
+ TSDebug(DEBUG_TAG_HOOK, "Set client request's scheme to %s through %s header",
+ isHttps?"https":"http", CARP_PREMAP_SCHEME.c_str());
+ }
+ } else {
+ removeHeader(bufp, hdr_loc, CARP_ROUTED_HEADER);
+ }
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return iTemp;
+ }
+
+ UrlComponents reqUrl;
+ reqUrl.populate(bufp, url_loc);
+ // the url ONLY used to determine the cache owner
+ string sUrl;
+
+ if (!bIsPOSTRemap) { // if pre-remap, then host not in URL so get from header
+ string sHost;
+ if (!getHeader(bufp, hdr_loc, TS_MIME_FIELD_HOST, sHost)) {
+ TSDebug(DEBUG_TAG_HOOK, "Could not find host header, ignoring it");
+ }
+ reqUrl.setHost(sHost);
+
+ //[YTSATS-836] heuristically ignore the scheme and port when calculate cache owner
+ UrlComponents normalizedUrl = reqUrl;
+ normalizedUrl.setScheme(CARP_SCHEME_FOR_HASH);
+ normalizedUrl.setPort(CARP_PORT_FOR_HASH);
+ normalizedUrl.construct(sUrl);
+ } else {
+ reqUrl.construct(sUrl);
+ }
+
+
+ if (g_CarpConfigPool->getGlobalConfig()->hasWhiteList()) {
+ string sCarpable;
+ if (!getHeader(bufp, hdr_loc, CARPABLE_HEADER, sCarpable)) { // if no carpable header check whitelist
+ if (!g_CarpConfigPool->getGlobalConfig()->isWhiteListed(reqUrl.getHost())) { // if white list exists, then host must be present
+ TSDebug(DEBUG_TAG_HOOK, "Host '%s' is not whitelisted, not going through carp", reqUrl.getHost().c_str());
+ TSHandleMLocRelease(bufp, hdr_loc, url_loc);
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ } else {
+ TSDebug(DEBUG_TAG_HOOK, "Found host (%s) whitelisted, routing...",reqUrl.getHost().c_str());
+ }
+ } else { // found carpable header, make sure it's 1
+ if (sCarpable.compare("1") != 0) { // carpable header present but not 0, be strict and do not forward request
+ TSDebug(DEBUG_TAG_HOOK, "Carpable (%s) present but value not acceptable (%s)", CARPABLE_HEADER.c_str(), sCarpable.c_str());
+ TSHandleMLocRelease(bufp, hdr_loc, url_loc);
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ } else {
+ TSDebug(DEBUG_TAG_HOOK, "Found Carpable header, routing...");
+ }
+ }
+ } else { // no whitelist so blacklist could be used
+ if (g_CarpConfigPool->getGlobalConfig()->isBlackListed(reqUrl.getHost())) { // if host black listed, do not carp
+ TSDebug(DEBUG_TAG_HOOK, "Host '%s' is blacklisted, not going through carp", reqUrl.getHost().c_str());
+ TSHandleMLocRelease(bufp, hdr_loc, url_loc);
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ }
+ }
+
+ TSDebug(DEBUG_TAG_HOOK, "URL to hash with=%s", sUrl.c_str());
+
+ // get nodeList and select node to forward to
+ std::vector<HashNode *> nodeList = g_CarpConfigPool->getGlobalHashAlgo()->getRemapProxyList(sUrl);
+ bool bAddHeaderResult=false;
+ bool bAddForwardedResult = false;
+ HashNode* node = NULL;
+ bool bIsOwner = false;
+
+ if (nodeList.size() == 0) { // no hosts available to forward to
+ TSDebug(DEBUG_TAG_HOOK, "no hosts available to forward to, will handle locally");
+ goto done;
+ } else {
+ node = nodeList[0];
+ }
+
+ bIsOwner = checkListForSelf(nodeList);
+ for (size_t k = 0; k < nodeList.size(); k++) {
+ TSDebug(DEBUG_TAG_HOOK, "nodeList host %d name is %s", static_cast<int>(k), nodeList[k]->name.c_str());
+ }
+
+ //handle forwarding
+ TSDebug(DEBUG_TAG_HOOK, "forwarding to '%s' (isSelf=%d)", node->name.c_str(), node->isSelf);
+ if (!node->isSelf) { // carp does not forward if we choose ourself
+ node->carpForward();
+ TSDebug(DEBUG_TAG_HOOK, "carp forwarded to %s.", node->name.c_str());
+ // insert carp loop prevention header
+ bAddHeaderResult = addHeader(bufp, hdr_loc, CARP_ROUTED_HEADER, string("1"));
+ if (!bAddHeaderResult) {
+ TSError("Carp, error inserting '%s' header", CARP_ROUTED_HEADER.c_str());
+ }
+ bAddForwardedResult = addHeader(bufp, hdr_loc, CARP_STATUS_HEADER, string(CARP_FORWARDED));
+ if (!bAddForwardedResult) {
+ TSError("Carp, error inserting '%s' header", CARP_STATUS_HEADER.c_str());
+ }
+
+ if (bIsPOSTRemap) { // if post remap, get remapped/OS Server Addr and add as header
+ const struct sockaddr* sa = TSHttpTxnServerAddrGet(txnp);
+ // const struct sockaddr* sa = TSHttpTxnNextHopAddrGet(txnp);
+ if (sa) { // sanity check
+ struct sockaddr_storage ss;
+ memcpy(static_cast<void *> (&ss), sa, sizeof (sockaddr_storage));
+ if ((reinterpret_cast<sockaddr_in *> (&ss))->sin_port == 0) { // set port from client request URL
+ (reinterpret_cast<sockaddr_in *> (&ss))->sin_port = htons(reqUrl.getPort());
+ }
+
+ string sTemp;
+ getStringFromSockaddr(reinterpret_cast<const sockaddr *> (&ss), sTemp);
+ TSDebug(DEBUG_TAG_HOOK, "Inserting forward header with sockaddr:%s", sTemp.c_str());
+ string sSockaddr;
+ sSockaddr.reserve(32 + sizeof (sockaddr_storage) * 2);
+ for (unsigned int i = 0; i<sizeof (sockaddr_storage); i++) {
+ char val[8];
+ sprintf(val, "%02X", reinterpret_cast<const unsigned char *> (&ss)[i]);
+ sSockaddr += val;
+ }
+ sSockaddr += "/" + reqUrl.getScheme();
+ // insert carp forwarding header
+ bool bAddFwdHeaderResult = addHeader(bufp, hdr_loc, CARP_FORWARD_HEADER, sSockaddr);
+ if (!bAddFwdHeaderResult) {
+ TSError("Carp, error inserting '%s' header", CARP_FORWARD_HEADER.c_str());
+ }
+ }
+ } else { // for premap mode
+ string sScheme = reqUrl.getScheme();
+
+ if (!addHeader(bufp, hdr_loc, CARP_PREMAP_SCHEME, sScheme)) {
+ TSError("Carp, error inserting '%s' header in premap mode", CARP_PREMAP_SCHEME.c_str());
+ } else {
+ TSDebug(DEBUG_TAG_HOOK, "Insert client request scheme %s in premap mode", sScheme.c_str());
+ }
+ }
+ // set origin server/destination
+ // TSHttpTxnConfigIntSet(txnp, TS_CONFIG_HTTP_SHARE_SERVER_SESSIONS, 0); // disable conn sharing due to bug when used with TSHttpTxnServerAddrSet
+ if (TSHttpTxnServerAddrSet(txnp, reinterpret_cast<const struct sockaddr *> (&node->forwardAddr)) != TS_SUCCESS) {
+ TSDebug(DEBUG_TAG_HOOK, "Error calling TSHttpTxnServerAddrSet");
+ } else {
+ // set scheme appropriately based on destination
+ TSDebug(DEBUG_TAG_HOOK, "Setting scheme to '%s'", node->getSchemeString());
+ TSUrlSchemeSet(bufp, url_loc, node->getSchemeString(), -1);
+ if (!bIsPOSTRemap) { // since we are forwarding, do not remap request and do not cache result
+ TSSkipRemappingSet(txnp, true);
+ }
+ TSHttpTxnArgSet(txnp, g_carpSelectedHostArgIndex, static_cast<void *> (node));
+ if (!bIsOwner) {
+ TSHttpTxnServerRespNoStoreSet(txnp, 1); // do not cache the response
+ } else {
+ TSHttpTxnServerRespNoStoreSet(txnp, 0); // we are replicate owner, cache response
--- End diff --
Is there a better API I should be using? I remember seeing multiple APIs for this and thought this was the right one.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by SolidWallOfCode <gi...@git.apache.org>.
Github user SolidWallOfCode commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r75871332
--- Diff: plugins/experimental/carp/CarpHost.h ---
@@ -0,0 +1,141 @@
+/** @file
+
+ Loads the CARP configuration
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+#ifndef __CARPHOST_H__
+#define __CARPHOST_H__ 1
+
+#define DEFAULT_GROUP 1
+
+#include <string>
+#include <netinet/in.h>
+
+class CarpHost
+{
+public:
+
+ CarpHost(std::string& name, int port, std::string& scheme, int weight, int group = DEFAULT_GROUP) : _port(port),
+ _name(name), _scheme(scheme), _weight(weight), _group(group)
+ {
+ _healthCheckAddr.ss_family=AF_UNSPEC;
+ };
+
+ ~CarpHost()
+ {
+
+ };
+
+ int getPort()
+ {
+ return _port;
+ };
+
+ void setPort(int p)
+ {
+ _port = p;
+ };
+
+ const std::string& getName()
+ {
+ return _name;
+ };
+
+ void setName(std::string& n)
+ {
+ _name = n;
+ };
+
+ const std::string& getScheme()
+ {
+ return _scheme;
+ }
+
+ void setScheme(std::string& scheme)
+ {
+ _scheme = scheme;
+ }
+
+ const struct sockaddr_storage* getHealthCheckAddr()
--- End diff --
`sockaddr_storage` is not something that should be passed around, either as a return value or an parameter type. Use `sockaddr*`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by SolidWallOfCode <gi...@git.apache.org>.
Github user SolidWallOfCode commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r75879116
--- Diff: plugins/experimental/carp/HttpFetch.cc ---
@@ -0,0 +1,439 @@
+/** @file
+
+ Limited URL fetcher..
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+
+#define __STDC_LIMIT_MACROS // need INT64_MAX definition
+#include <stdint.h>
+#include <sys/time.h>
+
+#include "HttpFetch.h"
+#include "UrlComponents.h"
+#include "Common.h"
+
+using std::string;
+
+/**********************************************************/
+// just 'pass through' and call object's handlEvent fn
+
+static int
+handleHttpFetchIOEvents(TSCont cont, TSEvent event, void *edata)
+{
+ HttpFetch *fetchObj = static_cast<HttpFetch *> (TSContDataGet(cont));
+ if (NULL == fetchObj) {
+ TSDebug(DEBUG_FETCH_TAG, "handleHttpFetchEvents continuation data NULL");
+ TSAssert(fetchObj);
+ }
+ return fetchObj->handleIOEvent(cont, event, edata);
+}
+
+/**********************************************************/
+HttpFetch::HttpFetch(const std::string &url, HashAlgorithm *hashAlgo,
+ HashNode *hashNode,const char *method)
+{
+ TSMBuffer bufp;
+ _url = url;
+ _respInfo = NULL;
+ _reqInfo = NULL;
+ _hashAlgo = hashAlgo;
+ _hashNode = hashNode;
+ _hcTimeoutSecond = DEFAULT_HEALTH_CHECK_TIMEOUT;
+
+ bufp = TSMBufferCreate();
+
+ if (bufp != NULL) {
+ TSMLoc urlp;
+ if (TSUrlCreate(bufp, &urlp) == TS_SUCCESS) {
+ const char *start = url.data();
+ if (TSUrlParse(bufp, urlp, &start, start + url.length()) == TS_PARSE_DONE) {
+ UrlComponents reqUrl;
+ string sPath;
+ string sHost;
+ reqUrl.populate(bufp, urlp);
+ reqUrl.getCompletePathString(sPath);
+ reqUrl.getCompleteHostString(sHost);
+ _request = string(method) + " " + sPath + " HTTP/1.0\r\nHost: " + sHost + "\r\n";
+ _request += CARP_ROUTED_HEADER + ": 1\r\n";
+ _request += "\r\n";
+ }
+ TSHandleMLocRelease(bufp, NULL, urlp);
+ }
+ }
+ TSMBufferDestroy(bufp);
+ TSDebug(DEBUG_FETCH_TAG, "HttpFetch assembled this request %s", _request.c_str());
+ _responseStatus=TS_HTTP_STATUS_NONE;
+ _ready = true;
+}
+
+/**********************************************************/
+HttpFetch::~HttpFetch()
+{
+}
+/**********************************************************/
+void
+HttpFetch::setHealthcheckTimeout(int timeout) {
+ _hcTimeoutSecond = timeout;
+}
+/**********************************************************/
+void
+HttpFetch::makeAsyncRequest(struct sockaddr const* serverAddr)
+{
+ _ready = false;
+ __sync_synchronize();
+ _result = UNKNOWN;
+ TSCont fetchCont = TSContCreate(handleHttpFetchIOEvents, TSMutexCreate());
+ //TSCont fetchCont = TSContCreate(handleHttpFetchIOEvents, NULL);
+ TSContDataSet(fetchCont, static_cast<void *> (this));
+
+ // save server addr
+ memmove((void *) &_serverAddr, (void *) serverAddr, sizeof (struct sockaddr));
+
+ // initiate request
+ TSDebug(DEBUG_FETCH_TAG, "TSNetConnect()");
+
+ struct timeval tvStart;
+ gettimeofday(&tvStart, NULL);
+ _startTime = tvStart.tv_sec*1000 + tvStart.tv_usec/1000;
+
+ struct sockaddr_in tempServerAddr;
+ memmove((void *) &tempServerAddr, (void *) &_serverAddr, sizeof (struct sockaddr));
+ TSDebug(DEBUG_FETCH_TAG, "serverAddr: %s:%d", inet_ntoa(tempServerAddr.sin_addr), ntohs(tempServerAddr.sin_port) );
+ _hcTimeout = TSContSchedule(fetchCont, _hcTimeoutSecond * 1000, TS_THREAD_POOL_DEFAULT);
+ _connAction = TSNetConnect(fetchCont, (const struct sockaddr *) &_serverAddr);
+
+}
+
+/**********************************************************/
+void
+HttpFetch::parseResponse()
+{
+ TSIOBufferBlock block;
+ TSParseResult pr = TS_PARSE_CONT;
+ int64_t avail = 0;
+ char *start = NULL;
+ char *initialStart = NULL;
+
+ TSDebug(DEBUG_FETCH_TAG, "Entering parse_response");
+
+ block = TSIOBufferReaderStart(_respIOBufReader);
+
+ if (!_respInfo->headerParsed) {
+ while ((pr == TS_PARSE_CONT) && (block != NULL)) {
+ initialStart = start = (char *) TSIOBufferBlockReadStart(block, _respIOBufReader, &avail);
+ if (avail > 0) {
+ pr = TSHttpHdrParseResp(_respInfo->parser, _respInfo->buf, _respInfo->http_hdr_loc, (const char **) &start, (const char *) (start + avail));
+ _responseHeaders.append(initialStart, start - initialStart);
+ }
+ block = TSIOBufferBlockNext(block);
+ }
+ // update avail
+ if (start && initialStart) {
+ avail -= (start - initialStart);
+ }
+ if (pr != TS_PARSE_CONT) {
+ _respInfo->status = TSHttpHdrStatusGet(_respInfo->buf, _respInfo->http_hdr_loc);
+ _responseStatus = _respInfo->status;
+ _respInfo->headerParsed = true;
+ TSDebug(DEBUG_FETCH_TAG, "HTTP Status: %d", _respInfo->status);
+ }
+ }
+
+ if (_respInfo->headerParsed) {
+ if (avail && start) { // if bytes left from header parsing, get those
+ _responseBody.append(start, avail);
+ if(block) {
+ block = TSIOBufferBlockNext(block);
+ }
+ }
+
+ while (block != NULL) {
+ start = (char *) TSIOBufferBlockReadStart(block, _respIOBufReader, &avail);
+ if (avail > 0) {
+ _responseBody.append(start, avail);
+ }
+ block = TSIOBufferBlockNext(block);
+ }
+ }
+ TSDebug(DEBUG_FETCH_TAG, "Leaving parseResponse");
+}
+
+
+/**********************************************************/
+int
+HttpFetch::handleIOEvent(TSCont cont, TSEvent event, void *edata)
+{
+ int64_t avail;
+ bool bCleanUp = false; // free everything when transaction is complete
+
+
+ TSDebug(DEBUG_FETCH_TAG, "Entering handleIOEvent");
+
+ switch (event) {
+ case TS_EVENT_NET_CONNECT: // connected to server
+ TSDebug(DEBUG_FETCH_TAG, "Connected (maybe)");
+ _connAction = NULL;
+ _respInfo = createResponseInfo();
+ _reqInfo = createRequestInfo();
+
+ _reqIOBuf = TSIOBufferCreate();
+ _reqIOBufReader = TSIOBufferReaderAlloc(_reqIOBuf);
+ _respIOBuf = TSIOBufferCreate();
+ _respIOBufReader = TSIOBufferReaderAlloc(_respIOBuf);
+
+ TSHttpHdrPrint(_reqInfo->buf, _reqInfo->http_hdr_loc, _reqIOBuf);
+ TSIOBufferWrite(_reqIOBuf, "\r\n", 2);
+
+ _vConn = static_cast<TSVConn> (edata); // get connection
+
+ _rVIO = TSVConnRead(_vConn, cont, _respIOBuf, INT64_MAX);
+
+ TSDebug(DEBUG_FETCH_TAG, "Writing %ld bytes", TSIOBufferReaderAvail(_reqIOBufReader));
+ _wVIO = TSVConnWrite(_vConn, cont, _reqIOBufReader, TSIOBufferReaderAvail(_reqIOBufReader));
+ break;
+ case TS_EVENT_NET_CONNECT_FAILED:
+ TSDebug(DEBUG_FETCH_TAG, "Connect failed");
+ _connAction = NULL;
+ TSActionCancel(_hcTimeout);
+ _hcTimeout = NULL;
+ _result = FAILURE;
+ break;
+ case TS_EVENT_ERROR:
+ TSDebug(DEBUG_FETCH_TAG, "Error event");
--- End diff --
Cancel `_hcTimeout`?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by SolidWallOfCode <gi...@git.apache.org>.
Github user SolidWallOfCode commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r74175897
--- Diff: plugins/experimental/carp/CarpConfig.cc ---
@@ -0,0 +1,581 @@
+/** @file
+
+ Loads the CARP configuration
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+
+//////////////////////////////////////////////////////////////
+// Read CARP configuration file
+// [Servers]
+// host1.yahoo.com:4080 weight=2 # port 4080 on host1.yahoo.com with weight factor of 2
+// host2.yahoo.com # port 80 on host2.yahoo.com with (default) weight factor of 1
+//
+// [Values]
+// healthcheck={host}:8001/status.html
+// healthfreq=30
+// global=on
+//
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <memory.h>
+#include <ts/ts.h>
+#include <sys/time.h>
+
+#include <sstream>
+
+#include "CarpConfig.h"
+#include "Common.h"
+#include "CarpConfigPool.h"
+
+using namespace std;
+
+#define DEFAULT_HEALTH_CHECK_FREQ 30 // 30 second period for health checks
+#define DEFAULT_HEALTH_CHECK_PORT 80 // default to makeing healthcheck requests against port 80
+#define DEFAULT_CONFIG_RELOAD_FREQ 30 // 30 seconds used in TSContSchedule
+#define DEFAULT_PORT 80 // default to makeing requests against port 80
+#define DEFAULT_WEIGHT 1 // default weight
+#define DEFAULT_SCHEME "http"
+#define DEFAULT_REPLICATION_FACTOR 1
+
+// config section headers
+static const char *const SECTION_SERVERS_STR = "[Servers]";
+static const char *const SECTION_VALUES_STR = "[Values]";
+
+// key strings
+static const char *const KEY_HEALTHCHECK_STR = "healthcheck";
+static const char *const KEY_HEALTHFREQ_STR = "healthfreq";
+static const char *const KEY_RELOADFREQ_STR = "reloadfreq";
+static const char *const KEY_HCTIMEOUT_STR = "hctimeout";
+static const char *const KEY_BLACKLIST_STR = "blacklist";
+static const char *const KEY_WHITELIST_STR = "whitelist";
+static const char *const KEY_MODE_STR = "mode";
+static const char *const KEY_ALLOWFWDPORT_STR = "allowfwdport";
+static const char *const KEY_REPLICATIONFACTOR_STR = "replicationfactor";
+
+// parameter strings
+static const char *const WEIGHT_EQUALS_STRING = "weight=";
+static const char *const GROUP_EQUALS_STRING = "group=";
+static const char *const KEY_MODE_PREREMAP_STR = "pre-remap";
+static const char *const KEY_MODE_POSTREMAP_STR = "post-remap";
+
+
+/**********************************************************/
+bool
+getInt(char** pptr, int *val)
+{
+ bool bReturn = false;
+ int v=0;
+
+ char* ptr = *pptr;
+ // skip white space if any
+ while (*ptr && isspace(*ptr)) ++ptr;
+ // get digits
+ if (*ptr) {
+ while (*ptr && isdigit(*ptr)) {
+ v *= 10;
+ v += (*ptr)-'0';
+ ++ptr;
+ bReturn = true;
+ }
+ }
+ if (bReturn) {
+ *pptr = ptr;
+ *val = v;
+ }
+
+ return bReturn;
+}
+
+/**********************************************************/
+// [http[s]://]host[:port]/path
+bool
+getHostAndPort(char** pptr,string* sHost,int* iPort, string* sScheme)
+{
+ bool bReturn = false;
+ char* ptr = *pptr;
+
+ *iPort = 80;
+ *sScheme = DEFAULT_SCHEME;
+ if(strncmp(*pptr,"https",5) == 0) {
+ *iPort = 443;
+ *sScheme = "https";
+ }
+
+ //skip leading white space
+ while (*ptr && isspace(*ptr)) ++ptr;
--- End diff --
AFAICT, "https://" and " https://" will behave differently because the port is set to 443 only if the string immediately begins with "https". Is that intended? Why check that twice in any case, instead of handling the port in the check just below?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by SolidWallOfCode <gi...@git.apache.org>.
Github user SolidWallOfCode commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r75860439
--- Diff: plugins/experimental/carp/carp.cc ---
@@ -0,0 +1,713 @@
+/** @file
+
+ A brief file description
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+
+#include <errno.h>
+
+#include <string>
+#include <sstream>
+#include <stdlib.h>
+#include <stdio.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+#include <unistd.h>
+
+#include <memory.h>
+
+#include <ts/ts.h>
+
+#include "Common.h"
+#include "CarpConfig.h"
+#include "CarpConfigPool.h"
+#include "CarpHashAlgorithm.h"
+#include "UrlComponents.h"
+
+using namespace std;
+
+CarpConfigPool* g_CarpConfigPool = NULL;
+int g_carpSelectedHostArgIndex = 0;
+TSTextLogObject g_logObject = NULL;
+
+const char *logFileName = "carp";
+
+//////////////////////////////////////////////////////////////////////////////
+//////////////////////////////////////////////////////////////////////////////
+/*
+ check for our carp routed header, dump status if requested
+ */
+static int
+processCarpRoutedHeader(TSHttpTxn txnp, TSMBuffer bufp, TSMLoc hdr_loc)
+{
+ string value;
+ if (getHeader(bufp, hdr_loc, CARP_ROUTED_HEADER, value)) { // if found header
+ if (value.compare("1") == 0) { // is loop prevention value
+ TSDebug(DEBUG_TAG_HOOK, "Found %s header with loop prevention value, not forwarding again", CARP_ROUTED_HEADER.c_str());
+ return 0;
+ } else if (value.compare("dump") == 0) { // is dump status request
+ TSDebug(DEBUG_TAG_HOOK, "Found %s header with dump request", CARP_ROUTED_HEADER.c_str());
+ string status;
+ g_CarpConfigPool->getGlobalHashAlgo()->dump(status);
+ TSHttpTxnSetHttpRetStatus(txnp, TS_HTTP_STATUS_MULTI_STATUS);
+ TSHttpTxnErrorBodySet(txnp, TSstrdup(status.c_str()), status.length(), NULL);
+ return -1;
+ }
+ TSDebug(DEBUG_TAG_HOOK, "Found %s header with unknown value of %s, ignoring", CARP_ROUTED_HEADER.c_str(), value.c_str());
+ removeHeader(bufp, hdr_loc, CARP_ROUTED_HEADER);
+ }
+ return 1; // all OK
+}
+
+static bool
+checkListForSelf(std::vector<HashNode *> list)
+{
+ for (size_t k = 0; k < list.size(); k++) {
+ if (list[k]->isSelf) return true;
+ }
+ return false;
+}
+
+/**
+ bIsPOSTRemap = false --- Hash request and forward to peer
+ bIsPOSTRemap = true --- hash request, extract OS sockaddr, insert forwarding header, forward
+ */
+static int
+handleRequestProcessing(TSCont contp, TSEvent event, void *edata, bool bIsPOSTRemap)
+{
+ TSHttpTxn txnp = (TSHttpTxn) edata;
+ TSMBuffer bufp;
+ TSMLoc hdr_loc;
+ TSMLoc url_loc;
+
+ // get the client request so we can get URL and add header
+ if (TSHttpTxnClientReqGet(txnp, &bufp, &hdr_loc) != TS_SUCCESS) {
+ TSError("carp couldn't get request headers");
+ return -1;
+ }
+
+ int method_len;
+ const char *method = TSHttpHdrMethodGet(bufp, hdr_loc, &method_len);
+ if (NULL == method) {
+ TSError("carp couldn't get http method");
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return -1;
+ }
+ if (((method_len == TS_HTTP_LEN_DELETE) && (strncasecmp(method, TS_HTTP_METHOD_DELETE, TS_HTTP_LEN_DELETE) == 0)) ||
+ ((method_len == TS_HTTP_LEN_PURGE) && (strncasecmp(method, TS_HTTP_METHOD_PURGE, TS_HTTP_LEN_PURGE) == 0))) {
+ TSDebug(DEBUG_TAG_HOOK, "Request method is '%s' so not routing request", string(method,method_len).c_str());
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ }
+
+ if (TSHttpHdrUrlGet(bufp, hdr_loc, &url_loc) != TS_SUCCESS) {
+ TSError("carp couldn't get url");
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return -1;
+ }
+ // if has carp loop prevention header, do not remap
+ int iTemp = processCarpRoutedHeader(txnp, bufp, hdr_loc);
+ if(iTemp <= 0) { // if dump or do not remap
+ // check origin client request's scheme for premap mode
+ if (!bIsPOSTRemap) {
+ string oriScheme;
+ if (!getHeader(bufp, hdr_loc, CARP_PREMAP_SCHEME, oriScheme)) {
+ TSDebug(DEBUG_TAG_HOOK, "couldn't get '%s' header", CARP_PREMAP_SCHEME.c_str());
+ } else {
+ bool isHttps = (oriScheme == TS_URL_SCHEME_HTTPS);
+
+ if (isHttps) {
+ TSUrlSchemeSet(bufp, url_loc, TS_URL_SCHEME_HTTPS, TS_URL_LEN_HTTPS);
+ } else {
+ TSUrlSchemeSet(bufp, url_loc, TS_URL_SCHEME_HTTP, TS_URL_LEN_HTTP);
+ }
+
+ removeHeader(bufp, hdr_loc, CARP_STATUS_HEADER);
+ removeHeader(bufp, hdr_loc, CARP_ROUTED_HEADER);
+ removeHeader(bufp, hdr_loc, CARP_PREMAP_SCHEME.c_str());
+ TSDebug(DEBUG_TAG_HOOK, "Set client request's scheme to %s through %s header",
+ isHttps?"https":"http", CARP_PREMAP_SCHEME.c_str());
+ }
+ } else {
+ removeHeader(bufp, hdr_loc, CARP_ROUTED_HEADER);
+ }
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return iTemp;
+ }
+
+ UrlComponents reqUrl;
+ reqUrl.populate(bufp, url_loc);
+ // the url ONLY used to determine the cache owner
+ string sUrl;
+
+ if (!bIsPOSTRemap) { // if pre-remap, then host not in URL so get from header
+ string sHost;
+ if (!getHeader(bufp, hdr_loc, TS_MIME_FIELD_HOST, sHost)) {
+ TSDebug(DEBUG_TAG_HOOK, "Could not find host header, ignoring it");
+ }
+ reqUrl.setHost(sHost);
+
+ //[YTSATS-836] heuristically ignore the scheme and port when calculate cache owner
+ UrlComponents normalizedUrl = reqUrl;
+ normalizedUrl.setScheme(CARP_SCHEME_FOR_HASH);
+ normalizedUrl.setPort(CARP_PORT_FOR_HASH);
+ normalizedUrl.construct(sUrl);
+ } else {
+ reqUrl.construct(sUrl);
+ }
+
+
+ if (g_CarpConfigPool->getGlobalConfig()->hasWhiteList()) {
+ string sCarpable;
+ if (!getHeader(bufp, hdr_loc, CARPABLE_HEADER, sCarpable)) { // if no carpable header check whitelist
+ if (!g_CarpConfigPool->getGlobalConfig()->isWhiteListed(reqUrl.getHost())) { // if white list exists, then host must be present
+ TSDebug(DEBUG_TAG_HOOK, "Host '%s' is not whitelisted, not going through carp", reqUrl.getHost().c_str());
+ TSHandleMLocRelease(bufp, hdr_loc, url_loc);
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ } else {
+ TSDebug(DEBUG_TAG_HOOK, "Found host (%s) whitelisted, routing...",reqUrl.getHost().c_str());
+ }
+ } else { // found carpable header, make sure it's 1
+ if (sCarpable.compare("1") != 0) { // carpable header present but not 0, be strict and do not forward request
+ TSDebug(DEBUG_TAG_HOOK, "Carpable (%s) present but value not acceptable (%s)", CARPABLE_HEADER.c_str(), sCarpable.c_str());
+ TSHandleMLocRelease(bufp, hdr_loc, url_loc);
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ } else {
+ TSDebug(DEBUG_TAG_HOOK, "Found Carpable header, routing...");
+ }
+ }
+ } else { // no whitelist so blacklist could be used
+ if (g_CarpConfigPool->getGlobalConfig()->isBlackListed(reqUrl.getHost())) { // if host black listed, do not carp
+ TSDebug(DEBUG_TAG_HOOK, "Host '%s' is blacklisted, not going through carp", reqUrl.getHost().c_str());
+ TSHandleMLocRelease(bufp, hdr_loc, url_loc);
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ }
+ }
+
+ TSDebug(DEBUG_TAG_HOOK, "URL to hash with=%s", sUrl.c_str());
+
+ // get nodeList and select node to forward to
+ std::vector<HashNode *> nodeList = g_CarpConfigPool->getGlobalHashAlgo()->getRemapProxyList(sUrl);
+ bool bAddHeaderResult=false;
+ bool bAddForwardedResult = false;
+ HashNode* node = NULL;
+ bool bIsOwner = false;
+
+ if (nodeList.size() == 0) { // no hosts available to forward to
+ TSDebug(DEBUG_TAG_HOOK, "no hosts available to forward to, will handle locally");
+ goto done;
+ } else {
+ node = nodeList[0];
+ }
+
+ bIsOwner = checkListForSelf(nodeList);
+ for (size_t k = 0; k < nodeList.size(); k++) {
+ TSDebug(DEBUG_TAG_HOOK, "nodeList host %d name is %s", static_cast<int>(k), nodeList[k]->name.c_str());
+ }
+
+ //handle forwarding
+ TSDebug(DEBUG_TAG_HOOK, "forwarding to '%s' (isSelf=%d)", node->name.c_str(), node->isSelf);
+ if (!node->isSelf) { // carp does not forward if we choose ourself
+ node->carpForward();
+ TSDebug(DEBUG_TAG_HOOK, "carp forwarded to %s.", node->name.c_str());
+ // insert carp loop prevention header
+ bAddHeaderResult = addHeader(bufp, hdr_loc, CARP_ROUTED_HEADER, string("1"));
+ if (!bAddHeaderResult) {
+ TSError("Carp, error inserting '%s' header", CARP_ROUTED_HEADER.c_str());
+ }
+ bAddForwardedResult = addHeader(bufp, hdr_loc, CARP_STATUS_HEADER, string(CARP_FORWARDED));
+ if (!bAddForwardedResult) {
+ TSError("Carp, error inserting '%s' header", CARP_STATUS_HEADER.c_str());
+ }
+
+ if (bIsPOSTRemap) { // if post remap, get remapped/OS Server Addr and add as header
+ const struct sockaddr* sa = TSHttpTxnServerAddrGet(txnp);
+ // const struct sockaddr* sa = TSHttpTxnNextHopAddrGet(txnp);
+ if (sa) { // sanity check
+ struct sockaddr_storage ss;
+ memcpy(static_cast<void *> (&ss), sa, sizeof (sockaddr_storage));
+ if ((reinterpret_cast<sockaddr_in *> (&ss))->sin_port == 0) { // set port from client request URL
+ (reinterpret_cast<sockaddr_in *> (&ss))->sin_port = htons(reqUrl.getPort());
+ }
+
+ string sTemp;
+ getStringFromSockaddr(reinterpret_cast<const sockaddr *> (&ss), sTemp);
+ TSDebug(DEBUG_TAG_HOOK, "Inserting forward header with sockaddr:%s", sTemp.c_str());
+ string sSockaddr;
+ sSockaddr.reserve(32 + sizeof (sockaddr_storage) * 2);
+ for (unsigned int i = 0; i<sizeof (sockaddr_storage); i++) {
+ char val[8];
+ sprintf(val, "%02X", reinterpret_cast<const unsigned char *> (&ss)[i]);
+ sSockaddr += val;
+ }
+ sSockaddr += "/" + reqUrl.getScheme();
+ // insert carp forwarding header
+ bool bAddFwdHeaderResult = addHeader(bufp, hdr_loc, CARP_FORWARD_HEADER, sSockaddr);
+ if (!bAddFwdHeaderResult) {
+ TSError("Carp, error inserting '%s' header", CARP_FORWARD_HEADER.c_str());
+ }
+ }
+ } else { // for premap mode
+ string sScheme = reqUrl.getScheme();
+
+ if (!addHeader(bufp, hdr_loc, CARP_PREMAP_SCHEME, sScheme)) {
+ TSError("Carp, error inserting '%s' header in premap mode", CARP_PREMAP_SCHEME.c_str());
+ } else {
+ TSDebug(DEBUG_TAG_HOOK, "Insert client request scheme %s in premap mode", sScheme.c_str());
+ }
+ }
+ // set origin server/destination
+ // TSHttpTxnConfigIntSet(txnp, TS_CONFIG_HTTP_SHARE_SERVER_SESSIONS, 0); // disable conn sharing due to bug when used with TSHttpTxnServerAddrSet
+ if (TSHttpTxnServerAddrSet(txnp, reinterpret_cast<const struct sockaddr *> (&node->forwardAddr)) != TS_SUCCESS) {
+ TSDebug(DEBUG_TAG_HOOK, "Error calling TSHttpTxnServerAddrSet");
+ } else {
+ // set scheme appropriately based on destination
+ TSDebug(DEBUG_TAG_HOOK, "Setting scheme to '%s'", node->getSchemeString());
+ TSUrlSchemeSet(bufp, url_loc, node->getSchemeString(), -1);
+ if (!bIsPOSTRemap) { // since we are forwarding, do not remap request and do not cache result
+ TSSkipRemappingSet(txnp, true);
+ }
+ TSHttpTxnArgSet(txnp, g_carpSelectedHostArgIndex, static_cast<void *> (node));
+ if (!bIsOwner) {
+ TSHttpTxnServerRespNoStoreSet(txnp, 1); // do not cache the response
+ } else {
+ TSHttpTxnServerRespNoStoreSet(txnp, 0); // we are replicate owner, cache response
--- End diff --
Be aware this doesn't force a cache, it inhibits the inhibition of caching. It's a messed up API and there's an outstanding bug (TS-3426) on fixing it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by SolidWallOfCode <gi...@git.apache.org>.
Github user SolidWallOfCode commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r74176237
--- Diff: plugins/experimental/carp/CarpConfig.cc ---
@@ -0,0 +1,581 @@
+/** @file
+
+ Loads the CARP configuration
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+
+//////////////////////////////////////////////////////////////
+// Read CARP configuration file
+// [Servers]
+// host1.yahoo.com:4080 weight=2 # port 4080 on host1.yahoo.com with weight factor of 2
+// host2.yahoo.com # port 80 on host2.yahoo.com with (default) weight factor of 1
+//
+// [Values]
+// healthcheck={host}:8001/status.html
+// healthfreq=30
+// global=on
+//
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <memory.h>
+#include <ts/ts.h>
+#include <sys/time.h>
+
+#include <sstream>
+
+#include "CarpConfig.h"
+#include "Common.h"
+#include "CarpConfigPool.h"
+
+using namespace std;
+
+#define DEFAULT_HEALTH_CHECK_FREQ 30 // 30 second period for health checks
+#define DEFAULT_HEALTH_CHECK_PORT 80 // default to makeing healthcheck requests against port 80
+#define DEFAULT_CONFIG_RELOAD_FREQ 30 // 30 seconds used in TSContSchedule
+#define DEFAULT_PORT 80 // default to makeing requests against port 80
+#define DEFAULT_WEIGHT 1 // default weight
+#define DEFAULT_SCHEME "http"
+#define DEFAULT_REPLICATION_FACTOR 1
+
+// config section headers
+static const char *const SECTION_SERVERS_STR = "[Servers]";
+static const char *const SECTION_VALUES_STR = "[Values]";
+
+// key strings
+static const char *const KEY_HEALTHCHECK_STR = "healthcheck";
+static const char *const KEY_HEALTHFREQ_STR = "healthfreq";
+static const char *const KEY_RELOADFREQ_STR = "reloadfreq";
+static const char *const KEY_HCTIMEOUT_STR = "hctimeout";
+static const char *const KEY_BLACKLIST_STR = "blacklist";
+static const char *const KEY_WHITELIST_STR = "whitelist";
+static const char *const KEY_MODE_STR = "mode";
+static const char *const KEY_ALLOWFWDPORT_STR = "allowfwdport";
+static const char *const KEY_REPLICATIONFACTOR_STR = "replicationfactor";
+
+// parameter strings
+static const char *const WEIGHT_EQUALS_STRING = "weight=";
+static const char *const GROUP_EQUALS_STRING = "group=";
+static const char *const KEY_MODE_PREREMAP_STR = "pre-remap";
+static const char *const KEY_MODE_POSTREMAP_STR = "post-remap";
+
+
+/**********************************************************/
+bool
+getInt(char** pptr, int *val)
+{
+ bool bReturn = false;
+ int v=0;
+
+ char* ptr = *pptr;
+ // skip white space if any
+ while (*ptr && isspace(*ptr)) ++ptr;
+ // get digits
+ if (*ptr) {
+ while (*ptr && isdigit(*ptr)) {
+ v *= 10;
+ v += (*ptr)-'0';
+ ++ptr;
+ bReturn = true;
+ }
+ }
+ if (bReturn) {
+ *pptr = ptr;
+ *val = v;
+ }
+
+ return bReturn;
+}
+
+/**********************************************************/
+// [http[s]://]host[:port]/path
+bool
+getHostAndPort(char** pptr,string* sHost,int* iPort, string* sScheme)
+{
+ bool bReturn = false;
+ char* ptr = *pptr;
+
+ *iPort = 80;
+ *sScheme = DEFAULT_SCHEME;
+ if(strncmp(*pptr,"https",5) == 0) {
+ *iPort = 443;
+ *sScheme = "https";
+ }
+
+ //skip leading white space
+ while (*ptr && isspace(*ptr)) ++ptr;
+
+ if (*ptr) { // validate not end of string
+ if(strncmp(ptr,"http://",7) == 0) {
+ ptr += 7;
+ } else if(strncmp(ptr,"https://",8) == 0) {
+ ptr += 8;
+ }
+
+ char* ptemp = ptr; // start of host
+ //find white space or ':' or '/'
+ while (*ptr && !isspace(*ptr) && *ptr != ':' && *ptr != '/') ++ptr;
+
+ *sHost = string(ptemp, (ptr - ptemp));
+ // skip white space (if any) after host
+ while (*ptr && isspace(*ptr)) ++ptr;
+ bReturn = true;
+
+ if (*ptr) {// have more to parse
+ // need to get port number?
+ if (*ptr == ':') {
+ ++ptr;
+ if (!getInt(&ptr, iPort)) {
+ // could be our special 'PORT' value, check for that
+ if(!strncmp(ptr,"{port}",6)) { // yes, is '{port}'
+ *iPort = -1;
+ bReturn = true;
+ } else { // really was an error
+ TSError("carp: error parsing port number from '%s'", *pptr);
+ bReturn = false;
+ }
+ } else {
+ // if port number is 443, treat the scheme as https
+ if (*iPort == 443) {
+ *sScheme = "https";
+ }
+ }
+ }
+ }
+ }
+ if(bReturn) {
+ *pptr = ptr;
+ }
+
+ return bReturn;
+}
+/**********************************************************/
+CarpConfig::CarpConfig()
+{
+ _healthCheckPort = DEFAULT_HEALTH_CHECK_PORT;
+ _healthCheckFreq = DEFAULT_HEALTH_CHECK_FREQ;
+ _configCheckFreq = DEFAULT_CONFIG_RELOAD_FREQ;
+ _healthCheckTimeout = DEFAULT_HEALTH_CHECK_TIMEOUT;
+ _setExit = 0;
+ _mode = PRE;
+ _allowForwardPort = 0;
+ _replicationFactor = DEFAULT_REPLICATION_FACTOR;
+ _nGroups = 0;
+}
+
+/**********************************************************/
+CarpConfig::~CarpConfig()
+{
+ for(size_t ptr = 0; ptr < _servers.size(); ptr++) {
+ delete _servers[ptr];
+ }
+
+ for(size_t ptr = 0; ptr < _httpClients.size(); ptr++) {
+ delete _httpClients[ptr];
+ }
+}
+
+/**********************************************************/
+bool
+CarpConfig::loadConfig(string filename)
+{
+ TSFile file;
+ GroupCountList group_counts;
+ GroupCountListIter group_counts_it;
+
+ // attempt to open config file assuming full path provided
+ TSDebug(DEBUG_TAG_INIT, "Trying to open config file in this path: %s", filename.c_str());
+ file = TSfopen(filename.c_str(), "r");
+ if (file == NULL) {
+ TSError("Failed to open carp config file %s", filename.c_str());
+ return false;
+ }
+
+ TSDebug(DEBUG_TAG_INIT, "Successfully opened config file");
+
+ char buffer[1024];
+ memset(buffer, 0, sizeof(buffer));
+
+ enum CONFIG_SECTION {
+ CFG_NONE_SECTION = 0,
+ CFG_SERVER_SECTION,
+ CFG_VALUES_SECTION
+ };
+ int cfg_section = CFG_NONE_SECTION;
+ bool done_parsing = false;
+ while (TSfgets(file, buffer, sizeof(buffer) - 1) != NULL && !done_parsing) {
+
+ char *eol = 0;
+ // make sure line was not bigger than buffer
+ if ((eol = strchr(buffer, '\n')) == NULL && (eol = strstr(buffer, "\r\n")) == NULL) {
--- End diff --
How could the string "\r\n" be found if the character "\n" wasn't?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by SolidWallOfCode <gi...@git.apache.org>.
Github user SolidWallOfCode commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r75859721
--- Diff: plugins/experimental/carp/carp.cc ---
@@ -0,0 +1,713 @@
+/** @file
+
+ A brief file description
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+
+#include <errno.h>
+
+#include <string>
+#include <sstream>
+#include <stdlib.h>
+#include <stdio.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+#include <unistd.h>
+
+#include <memory.h>
+
+#include <ts/ts.h>
+
+#include "Common.h"
+#include "CarpConfig.h"
+#include "CarpConfigPool.h"
+#include "CarpHashAlgorithm.h"
+#include "UrlComponents.h"
+
+using namespace std;
+
+CarpConfigPool* g_CarpConfigPool = NULL;
+int g_carpSelectedHostArgIndex = 0;
+TSTextLogObject g_logObject = NULL;
+
+const char *logFileName = "carp";
+
+//////////////////////////////////////////////////////////////////////////////
+//////////////////////////////////////////////////////////////////////////////
+/*
+ check for our carp routed header, dump status if requested
+ */
+static int
+processCarpRoutedHeader(TSHttpTxn txnp, TSMBuffer bufp, TSMLoc hdr_loc)
+{
+ string value;
+ if (getHeader(bufp, hdr_loc, CARP_ROUTED_HEADER, value)) { // if found header
+ if (value.compare("1") == 0) { // is loop prevention value
+ TSDebug(DEBUG_TAG_HOOK, "Found %s header with loop prevention value, not forwarding again", CARP_ROUTED_HEADER.c_str());
+ return 0;
+ } else if (value.compare("dump") == 0) { // is dump status request
+ TSDebug(DEBUG_TAG_HOOK, "Found %s header with dump request", CARP_ROUTED_HEADER.c_str());
+ string status;
+ g_CarpConfigPool->getGlobalHashAlgo()->dump(status);
+ TSHttpTxnSetHttpRetStatus(txnp, TS_HTTP_STATUS_MULTI_STATUS);
+ TSHttpTxnErrorBodySet(txnp, TSstrdup(status.c_str()), status.length(), NULL);
+ return -1;
+ }
+ TSDebug(DEBUG_TAG_HOOK, "Found %s header with unknown value of %s, ignoring", CARP_ROUTED_HEADER.c_str(), value.c_str());
+ removeHeader(bufp, hdr_loc, CARP_ROUTED_HEADER);
+ }
+ return 1; // all OK
+}
+
+static bool
+checkListForSelf(std::vector<HashNode *> list)
+{
+ for (size_t k = 0; k < list.size(); k++) {
+ if (list[k]->isSelf) return true;
+ }
+ return false;
+}
+
+/**
+ bIsPOSTRemap = false --- Hash request and forward to peer
+ bIsPOSTRemap = true --- hash request, extract OS sockaddr, insert forwarding header, forward
+ */
+static int
+handleRequestProcessing(TSCont contp, TSEvent event, void *edata, bool bIsPOSTRemap)
+{
+ TSHttpTxn txnp = (TSHttpTxn) edata;
+ TSMBuffer bufp;
+ TSMLoc hdr_loc;
+ TSMLoc url_loc;
+
+ // get the client request so we can get URL and add header
+ if (TSHttpTxnClientReqGet(txnp, &bufp, &hdr_loc) != TS_SUCCESS) {
+ TSError("carp couldn't get request headers");
+ return -1;
+ }
+
+ int method_len;
+ const char *method = TSHttpHdrMethodGet(bufp, hdr_loc, &method_len);
+ if (NULL == method) {
+ TSError("carp couldn't get http method");
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return -1;
+ }
+ if (((method_len == TS_HTTP_LEN_DELETE) && (strncasecmp(method, TS_HTTP_METHOD_DELETE, TS_HTTP_LEN_DELETE) == 0)) ||
+ ((method_len == TS_HTTP_LEN_PURGE) && (strncasecmp(method, TS_HTTP_METHOD_PURGE, TS_HTTP_LEN_PURGE) == 0))) {
+ TSDebug(DEBUG_TAG_HOOK, "Request method is '%s' so not routing request", string(method,method_len).c_str());
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ }
+
+ if (TSHttpHdrUrlGet(bufp, hdr_loc, &url_loc) != TS_SUCCESS) {
+ TSError("carp couldn't get url");
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return -1;
+ }
+ // if has carp loop prevention header, do not remap
+ int iTemp = processCarpRoutedHeader(txnp, bufp, hdr_loc);
+ if(iTemp <= 0) { // if dump or do not remap
+ // check origin client request's scheme for premap mode
+ if (!bIsPOSTRemap) {
+ string oriScheme;
+ if (!getHeader(bufp, hdr_loc, CARP_PREMAP_SCHEME, oriScheme)) {
+ TSDebug(DEBUG_TAG_HOOK, "couldn't get '%s' header", CARP_PREMAP_SCHEME.c_str());
+ } else {
+ bool isHttps = (oriScheme == TS_URL_SCHEME_HTTPS);
+
+ if (isHttps) {
+ TSUrlSchemeSet(bufp, url_loc, TS_URL_SCHEME_HTTPS, TS_URL_LEN_HTTPS);
+ } else {
+ TSUrlSchemeSet(bufp, url_loc, TS_URL_SCHEME_HTTP, TS_URL_LEN_HTTP);
+ }
+
+ removeHeader(bufp, hdr_loc, CARP_STATUS_HEADER);
+ removeHeader(bufp, hdr_loc, CARP_ROUTED_HEADER);
+ removeHeader(bufp, hdr_loc, CARP_PREMAP_SCHEME.c_str());
+ TSDebug(DEBUG_TAG_HOOK, "Set client request's scheme to %s through %s header",
+ isHttps?"https":"http", CARP_PREMAP_SCHEME.c_str());
+ }
+ } else {
+ removeHeader(bufp, hdr_loc, CARP_ROUTED_HEADER);
+ }
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return iTemp;
+ }
+
+ UrlComponents reqUrl;
+ reqUrl.populate(bufp, url_loc);
+ // the url ONLY used to determine the cache owner
+ string sUrl;
+
+ if (!bIsPOSTRemap) { // if pre-remap, then host not in URL so get from header
+ string sHost;
+ if (!getHeader(bufp, hdr_loc, TS_MIME_FIELD_HOST, sHost)) {
+ TSDebug(DEBUG_TAG_HOOK, "Could not find host header, ignoring it");
+ }
+ reqUrl.setHost(sHost);
+
+ //[YTSATS-836] heuristically ignore the scheme and port when calculate cache owner
+ UrlComponents normalizedUrl = reqUrl;
+ normalizedUrl.setScheme(CARP_SCHEME_FOR_HASH);
+ normalizedUrl.setPort(CARP_PORT_FOR_HASH);
+ normalizedUrl.construct(sUrl);
+ } else {
+ reqUrl.construct(sUrl);
+ }
+
+
+ if (g_CarpConfigPool->getGlobalConfig()->hasWhiteList()) {
+ string sCarpable;
+ if (!getHeader(bufp, hdr_loc, CARPABLE_HEADER, sCarpable)) { // if no carpable header check whitelist
+ if (!g_CarpConfigPool->getGlobalConfig()->isWhiteListed(reqUrl.getHost())) { // if white list exists, then host must be present
+ TSDebug(DEBUG_TAG_HOOK, "Host '%s' is not whitelisted, not going through carp", reqUrl.getHost().c_str());
+ TSHandleMLocRelease(bufp, hdr_loc, url_loc);
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ } else {
+ TSDebug(DEBUG_TAG_HOOK, "Found host (%s) whitelisted, routing...",reqUrl.getHost().c_str());
+ }
+ } else { // found carpable header, make sure it's 1
+ if (sCarpable.compare("1") != 0) { // carpable header present but not 0, be strict and do not forward request
+ TSDebug(DEBUG_TAG_HOOK, "Carpable (%s) present but value not acceptable (%s)", CARPABLE_HEADER.c_str(), sCarpable.c_str());
+ TSHandleMLocRelease(bufp, hdr_loc, url_loc);
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ } else {
+ TSDebug(DEBUG_TAG_HOOK, "Found Carpable header, routing...");
+ }
+ }
+ } else { // no whitelist so blacklist could be used
+ if (g_CarpConfigPool->getGlobalConfig()->isBlackListed(reqUrl.getHost())) { // if host black listed, do not carp
+ TSDebug(DEBUG_TAG_HOOK, "Host '%s' is blacklisted, not going through carp", reqUrl.getHost().c_str());
+ TSHandleMLocRelease(bufp, hdr_loc, url_loc);
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ }
+ }
+
+ TSDebug(DEBUG_TAG_HOOK, "URL to hash with=%s", sUrl.c_str());
+
+ // get nodeList and select node to forward to
+ std::vector<HashNode *> nodeList = g_CarpConfigPool->getGlobalHashAlgo()->getRemapProxyList(sUrl);
+ bool bAddHeaderResult=false;
+ bool bAddForwardedResult = false;
+ HashNode* node = NULL;
+ bool bIsOwner = false;
+
+ if (nodeList.size() == 0) { // no hosts available to forward to
+ TSDebug(DEBUG_TAG_HOOK, "no hosts available to forward to, will handle locally");
+ goto done;
+ } else {
+ node = nodeList[0];
+ }
+
+ bIsOwner = checkListForSelf(nodeList);
+ for (size_t k = 0; k < nodeList.size(); k++) {
+ TSDebug(DEBUG_TAG_HOOK, "nodeList host %d name is %s", static_cast<int>(k), nodeList[k]->name.c_str());
+ }
+
+ //handle forwarding
+ TSDebug(DEBUG_TAG_HOOK, "forwarding to '%s' (isSelf=%d)", node->name.c_str(), node->isSelf);
+ if (!node->isSelf) { // carp does not forward if we choose ourself
+ node->carpForward();
+ TSDebug(DEBUG_TAG_HOOK, "carp forwarded to %s.", node->name.c_str());
+ // insert carp loop prevention header
+ bAddHeaderResult = addHeader(bufp, hdr_loc, CARP_ROUTED_HEADER, string("1"));
+ if (!bAddHeaderResult) {
+ TSError("Carp, error inserting '%s' header", CARP_ROUTED_HEADER.c_str());
+ }
+ bAddForwardedResult = addHeader(bufp, hdr_loc, CARP_STATUS_HEADER, string(CARP_FORWARDED));
+ if (!bAddForwardedResult) {
+ TSError("Carp, error inserting '%s' header", CARP_STATUS_HEADER.c_str());
+ }
+
+ if (bIsPOSTRemap) { // if post remap, get remapped/OS Server Addr and add as header
+ const struct sockaddr* sa = TSHttpTxnServerAddrGet(txnp);
+ // const struct sockaddr* sa = TSHttpTxnNextHopAddrGet(txnp);
+ if (sa) { // sanity check
+ struct sockaddr_storage ss;
+ memcpy(static_cast<void *> (&ss), sa, sizeof (sockaddr_storage));
+ if ((reinterpret_cast<sockaddr_in *> (&ss))->sin_port == 0) { // set port from client request URL
+ (reinterpret_cast<sockaddr_in *> (&ss))->sin_port = htons(reqUrl.getPort());
+ }
+
+ string sTemp;
+ getStringFromSockaddr(reinterpret_cast<const sockaddr *> (&ss), sTemp);
--- End diff --
This is so messed up I don't know where to start.
* As noted above, `sa` is certain to not be as long as a `sockaddr_storage` this prints out effectively random bytes to be shipped to the peer, presuming it doesn't just ASAN crash first.
* Since in other places it is clear only IPv4 is supported, why not use `sockaddr_in`? What's the benefit of using `soockaddr_storage`?
* Why bother with `ss` at all? AFAICT it's used only to set the port and since it's just printed to the string anyway, why not just print the port explicitly?
* Why not use `sTemp`? Or have the debug message use `val`. Having two different string conversions just for a debug output seems overly complex.
* In preference to either, why not use `inet_ntoa` or `inet_ntop`?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by SolidWallOfCode <gi...@git.apache.org>.
Github user SolidWallOfCode commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r74476684
--- Diff: plugins/experimental/carp/CarpConfig.cc ---
@@ -0,0 +1,581 @@
+/** @file
+
+ Loads the CARP configuration
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+
+//////////////////////////////////////////////////////////////
+// Read CARP configuration file
+// [Servers]
+// host1.yahoo.com:4080 weight=2 # port 4080 on host1.yahoo.com with weight factor of 2
+// host2.yahoo.com # port 80 on host2.yahoo.com with (default) weight factor of 1
+//
+// [Values]
+// healthcheck={host}:8001/status.html
+// healthfreq=30
+// global=on
+//
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <memory.h>
+#include <ts/ts.h>
+#include <sys/time.h>
+
+#include <sstream>
+
+#include "CarpConfig.h"
+#include "Common.h"
+#include "CarpConfigPool.h"
+
+using namespace std;
+
+#define DEFAULT_HEALTH_CHECK_FREQ 30 // 30 second period for health checks
+#define DEFAULT_HEALTH_CHECK_PORT 80 // default to makeing healthcheck requests against port 80
+#define DEFAULT_CONFIG_RELOAD_FREQ 30 // 30 seconds used in TSContSchedule
+#define DEFAULT_PORT 80 // default to makeing requests against port 80
+#define DEFAULT_WEIGHT 1 // default weight
+#define DEFAULT_SCHEME "http"
+#define DEFAULT_REPLICATION_FACTOR 1
+
+// config section headers
+static const char *const SECTION_SERVERS_STR = "[Servers]";
+static const char *const SECTION_VALUES_STR = "[Values]";
+
+// key strings
+static const char *const KEY_HEALTHCHECK_STR = "healthcheck";
+static const char *const KEY_HEALTHFREQ_STR = "healthfreq";
+static const char *const KEY_RELOADFREQ_STR = "reloadfreq";
+static const char *const KEY_HCTIMEOUT_STR = "hctimeout";
+static const char *const KEY_BLACKLIST_STR = "blacklist";
+static const char *const KEY_WHITELIST_STR = "whitelist";
+static const char *const KEY_MODE_STR = "mode";
+static const char *const KEY_ALLOWFWDPORT_STR = "allowfwdport";
+static const char *const KEY_REPLICATIONFACTOR_STR = "replicationfactor";
+
+// parameter strings
+static const char *const WEIGHT_EQUALS_STRING = "weight=";
+static const char *const GROUP_EQUALS_STRING = "group=";
+static const char *const KEY_MODE_PREREMAP_STR = "pre-remap";
+static const char *const KEY_MODE_POSTREMAP_STR = "post-remap";
+
+
+/**********************************************************/
+bool
+getInt(char** pptr, int *val)
--- End diff --
You can do that with `strtol`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by SolidWallOfCode <gi...@git.apache.org>.
Github user SolidWallOfCode commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r75875430
--- Diff: plugins/experimental/carp/HttpFetch.cc ---
@@ -0,0 +1,439 @@
+/** @file
+
+ Limited URL fetcher..
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+
+#define __STDC_LIMIT_MACROS // need INT64_MAX definition
+#include <stdint.h>
+#include <sys/time.h>
+
+#include "HttpFetch.h"
+#include "UrlComponents.h"
+#include "Common.h"
+
+using std::string;
+
+/**********************************************************/
+// just 'pass through' and call object's handlEvent fn
+
+static int
+handleHttpFetchIOEvents(TSCont cont, TSEvent event, void *edata)
+{
+ HttpFetch *fetchObj = static_cast<HttpFetch *> (TSContDataGet(cont));
+ if (NULL == fetchObj) {
+ TSDebug(DEBUG_FETCH_TAG, "handleHttpFetchEvents continuation data NULL");
+ TSAssert(fetchObj);
+ }
+ return fetchObj->handleIOEvent(cont, event, edata);
+}
+
+/**********************************************************/
+HttpFetch::HttpFetch(const std::string &url, HashAlgorithm *hashAlgo,
+ HashNode *hashNode,const char *method)
+{
+ TSMBuffer bufp;
+ _url = url;
+ _respInfo = NULL;
+ _reqInfo = NULL;
+ _hashAlgo = hashAlgo;
+ _hashNode = hashNode;
+ _hcTimeoutSecond = DEFAULT_HEALTH_CHECK_TIMEOUT;
+
+ bufp = TSMBufferCreate();
+
+ if (bufp != NULL) {
+ TSMLoc urlp;
+ if (TSUrlCreate(bufp, &urlp) == TS_SUCCESS) {
+ const char *start = url.data();
+ if (TSUrlParse(bufp, urlp, &start, start + url.length()) == TS_PARSE_DONE) {
+ UrlComponents reqUrl;
+ string sPath;
+ string sHost;
+ reqUrl.populate(bufp, urlp);
+ reqUrl.getCompletePathString(sPath);
+ reqUrl.getCompleteHostString(sHost);
+ _request = string(method) + " " + sPath + " HTTP/1.0\r\nHost: " + sHost + "\r\n";
+ _request += CARP_ROUTED_HEADER + ": 1\r\n";
+ _request += "\r\n";
+ }
+ TSHandleMLocRelease(bufp, NULL, urlp);
+ }
+ }
+ TSMBufferDestroy(bufp);
+ TSDebug(DEBUG_FETCH_TAG, "HttpFetch assembled this request %s", _request.c_str());
+ _responseStatus=TS_HTTP_STATUS_NONE;
+ _ready = true;
+}
+
+/**********************************************************/
+HttpFetch::~HttpFetch()
+{
+}
+/**********************************************************/
+void
+HttpFetch::setHealthcheckTimeout(int timeout) {
+ _hcTimeoutSecond = timeout;
+}
+/**********************************************************/
+void
+HttpFetch::makeAsyncRequest(struct sockaddr const* serverAddr)
+{
+ _ready = false;
+ __sync_synchronize();
+ _result = UNKNOWN;
+ TSCont fetchCont = TSContCreate(handleHttpFetchIOEvents, TSMutexCreate());
+ //TSCont fetchCont = TSContCreate(handleHttpFetchIOEvents, NULL);
+ TSContDataSet(fetchCont, static_cast<void *> (this));
+
+ // save server addr
+ memmove((void *) &_serverAddr, (void *) serverAddr, sizeof (struct sockaddr));
--- End diff --
This seems guaranteed to fail. `sizeof (struct sockaddr)` will be smaller than any actual payload containing member of that family so no address data will be copied.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by SolidWallOfCode <gi...@git.apache.org>.
Github user SolidWallOfCode commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r75854284
--- Diff: plugins/experimental/carp/carp.cc ---
@@ -0,0 +1,713 @@
+/** @file
+
+ A brief file description
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+
+#include <errno.h>
+
+#include <string>
+#include <sstream>
+#include <stdlib.h>
+#include <stdio.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+#include <unistd.h>
+
+#include <memory.h>
+
+#include <ts/ts.h>
+
+#include "Common.h"
+#include "CarpConfig.h"
+#include "CarpConfigPool.h"
+#include "CarpHashAlgorithm.h"
+#include "UrlComponents.h"
+
+using namespace std;
+
+CarpConfigPool* g_CarpConfigPool = NULL;
+int g_carpSelectedHostArgIndex = 0;
+TSTextLogObject g_logObject = NULL;
+
+const char *logFileName = "carp";
+
+//////////////////////////////////////////////////////////////////////////////
+//////////////////////////////////////////////////////////////////////////////
+/*
+ check for our carp routed header, dump status if requested
+ */
+static int
+processCarpRoutedHeader(TSHttpTxn txnp, TSMBuffer bufp, TSMLoc hdr_loc)
+{
+ string value;
+ if (getHeader(bufp, hdr_loc, CARP_ROUTED_HEADER, value)) { // if found header
+ if (value.compare("1") == 0) { // is loop prevention value
+ TSDebug(DEBUG_TAG_HOOK, "Found %s header with loop prevention value, not forwarding again", CARP_ROUTED_HEADER.c_str());
+ return 0;
+ } else if (value.compare("dump") == 0) { // is dump status request
+ TSDebug(DEBUG_TAG_HOOK, "Found %s header with dump request", CARP_ROUTED_HEADER.c_str());
+ string status;
+ g_CarpConfigPool->getGlobalHashAlgo()->dump(status);
+ TSHttpTxnSetHttpRetStatus(txnp, TS_HTTP_STATUS_MULTI_STATUS);
+ TSHttpTxnErrorBodySet(txnp, TSstrdup(status.c_str()), status.length(), NULL);
+ return -1;
+ }
+ TSDebug(DEBUG_TAG_HOOK, "Found %s header with unknown value of %s, ignoring", CARP_ROUTED_HEADER.c_str(), value.c_str());
+ removeHeader(bufp, hdr_loc, CARP_ROUTED_HEADER);
+ }
+ return 1; // all OK
+}
+
+static bool
+checkListForSelf(std::vector<HashNode *> list)
+{
+ for (size_t k = 0; k < list.size(); k++) {
+ if (list[k]->isSelf) return true;
+ }
+ return false;
+}
+
+/**
+ bIsPOSTRemap = false --- Hash request and forward to peer
+ bIsPOSTRemap = true --- hash request, extract OS sockaddr, insert forwarding header, forward
+ */
+static int
+handleRequestProcessing(TSCont contp, TSEvent event, void *edata, bool bIsPOSTRemap)
+{
+ TSHttpTxn txnp = (TSHttpTxn) edata;
+ TSMBuffer bufp;
+ TSMLoc hdr_loc;
+ TSMLoc url_loc;
+
+ // get the client request so we can get URL and add header
+ if (TSHttpTxnClientReqGet(txnp, &bufp, &hdr_loc) != TS_SUCCESS) {
+ TSError("carp couldn't get request headers");
+ return -1;
+ }
+
+ int method_len;
+ const char *method = TSHttpHdrMethodGet(bufp, hdr_loc, &method_len);
+ if (NULL == method) {
+ TSError("carp couldn't get http method");
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return -1;
+ }
+ if (((method_len == TS_HTTP_LEN_DELETE) && (strncasecmp(method, TS_HTTP_METHOD_DELETE, TS_HTTP_LEN_DELETE) == 0)) ||
+ ((method_len == TS_HTTP_LEN_PURGE) && (strncasecmp(method, TS_HTTP_METHOD_PURGE, TS_HTTP_LEN_PURGE) == 0))) {
+ TSDebug(DEBUG_TAG_HOOK, "Request method is '%s' so not routing request", string(method,method_len).c_str());
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ }
+
+ if (TSHttpHdrUrlGet(bufp, hdr_loc, &url_loc) != TS_SUCCESS) {
+ TSError("carp couldn't get url");
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return -1;
+ }
+ // if has carp loop prevention header, do not remap
+ int iTemp = processCarpRoutedHeader(txnp, bufp, hdr_loc);
+ if(iTemp <= 0) { // if dump or do not remap
+ // check origin client request's scheme for premap mode
+ if (!bIsPOSTRemap) {
+ string oriScheme;
+ if (!getHeader(bufp, hdr_loc, CARP_PREMAP_SCHEME, oriScheme)) {
+ TSDebug(DEBUG_TAG_HOOK, "couldn't get '%s' header", CARP_PREMAP_SCHEME.c_str());
+ } else {
+ bool isHttps = (oriScheme == TS_URL_SCHEME_HTTPS);
+
+ if (isHttps) {
+ TSUrlSchemeSet(bufp, url_loc, TS_URL_SCHEME_HTTPS, TS_URL_LEN_HTTPS);
+ } else {
+ TSUrlSchemeSet(bufp, url_loc, TS_URL_SCHEME_HTTP, TS_URL_LEN_HTTP);
+ }
+
+ removeHeader(bufp, hdr_loc, CARP_STATUS_HEADER);
+ removeHeader(bufp, hdr_loc, CARP_ROUTED_HEADER);
+ removeHeader(bufp, hdr_loc, CARP_PREMAP_SCHEME.c_str());
+ TSDebug(DEBUG_TAG_HOOK, "Set client request's scheme to %s through %s header",
+ isHttps?"https":"http", CARP_PREMAP_SCHEME.c_str());
+ }
+ } else {
+ removeHeader(bufp, hdr_loc, CARP_ROUTED_HEADER);
+ }
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return iTemp;
+ }
+
+ UrlComponents reqUrl;
+ reqUrl.populate(bufp, url_loc);
+ // the url ONLY used to determine the cache owner
+ string sUrl;
+
+ if (!bIsPOSTRemap) { // if pre-remap, then host not in URL so get from header
+ string sHost;
+ if (!getHeader(bufp, hdr_loc, TS_MIME_FIELD_HOST, sHost)) {
+ TSDebug(DEBUG_TAG_HOOK, "Could not find host header, ignoring it");
+ }
+ reqUrl.setHost(sHost);
+
+ //[YTSATS-836] heuristically ignore the scheme and port when calculate cache owner
+ UrlComponents normalizedUrl = reqUrl;
+ normalizedUrl.setScheme(CARP_SCHEME_FOR_HASH);
+ normalizedUrl.setPort(CARP_PORT_FOR_HASH);
+ normalizedUrl.construct(sUrl);
+ } else {
+ reqUrl.construct(sUrl);
+ }
+
+
+ if (g_CarpConfigPool->getGlobalConfig()->hasWhiteList()) {
+ string sCarpable;
+ if (!getHeader(bufp, hdr_loc, CARPABLE_HEADER, sCarpable)) { // if no carpable header check whitelist
+ if (!g_CarpConfigPool->getGlobalConfig()->isWhiteListed(reqUrl.getHost())) { // if white list exists, then host must be present
+ TSDebug(DEBUG_TAG_HOOK, "Host '%s' is not whitelisted, not going through carp", reqUrl.getHost().c_str());
+ TSHandleMLocRelease(bufp, hdr_loc, url_loc);
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ } else {
+ TSDebug(DEBUG_TAG_HOOK, "Found host (%s) whitelisted, routing...",reqUrl.getHost().c_str());
+ }
+ } else { // found carpable header, make sure it's 1
+ if (sCarpable.compare("1") != 0) { // carpable header present but not 0, be strict and do not forward request
+ TSDebug(DEBUG_TAG_HOOK, "Carpable (%s) present but value not acceptable (%s)", CARPABLE_HEADER.c_str(), sCarpable.c_str());
+ TSHandleMLocRelease(bufp, hdr_loc, url_loc);
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ } else {
+ TSDebug(DEBUG_TAG_HOOK, "Found Carpable header, routing...");
+ }
+ }
+ } else { // no whitelist so blacklist could be used
+ if (g_CarpConfigPool->getGlobalConfig()->isBlackListed(reqUrl.getHost())) { // if host black listed, do not carp
+ TSDebug(DEBUG_TAG_HOOK, "Host '%s' is blacklisted, not going through carp", reqUrl.getHost().c_str());
+ TSHandleMLocRelease(bufp, hdr_loc, url_loc);
+ TSHandleMLocRelease(bufp, TS_NULL_MLOC, hdr_loc);
+ return 0;
+ }
+ }
+
+ TSDebug(DEBUG_TAG_HOOK, "URL to hash with=%s", sUrl.c_str());
+
+ // get nodeList and select node to forward to
+ std::vector<HashNode *> nodeList = g_CarpConfigPool->getGlobalHashAlgo()->getRemapProxyList(sUrl);
--- End diff --
Does this need to be copied?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by ericcarlschwartz <gi...@git.apache.org>.
Github user ericcarlschwartz commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r75949804
--- Diff: plugins/experimental/carp/CarpConfig.cc ---
@@ -0,0 +1,581 @@
+/** @file
+
+ Loads the CARP configuration
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+
+//////////////////////////////////////////////////////////////
+// Read CARP configuration file
+// [Servers]
+// host1.yahoo.com:4080 weight=2 # port 4080 on host1.yahoo.com with weight factor of 2
+// host2.yahoo.com # port 80 on host2.yahoo.com with (default) weight factor of 1
+//
+// [Values]
+// healthcheck={host}:8001/status.html
+// healthfreq=30
+// global=on
+//
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <memory.h>
+#include <ts/ts.h>
+#include <sys/time.h>
+
+#include <sstream>
+
+#include "CarpConfig.h"
+#include "Common.h"
+#include "CarpConfigPool.h"
+
+using namespace std;
+
+#define DEFAULT_HEALTH_CHECK_FREQ 30 // 30 second period for health checks
+#define DEFAULT_HEALTH_CHECK_PORT 80 // default to makeing healthcheck requests against port 80
+#define DEFAULT_CONFIG_RELOAD_FREQ 30 // 30 seconds used in TSContSchedule
+#define DEFAULT_PORT 80 // default to makeing requests against port 80
+#define DEFAULT_WEIGHT 1 // default weight
+#define DEFAULT_SCHEME "http"
+#define DEFAULT_REPLICATION_FACTOR 1
+
+// config section headers
+static const char *const SECTION_SERVERS_STR = "[Servers]";
+static const char *const SECTION_VALUES_STR = "[Values]";
+
+// key strings
+static const char *const KEY_HEALTHCHECK_STR = "healthcheck";
+static const char *const KEY_HEALTHFREQ_STR = "healthfreq";
+static const char *const KEY_RELOADFREQ_STR = "reloadfreq";
+static const char *const KEY_HCTIMEOUT_STR = "hctimeout";
+static const char *const KEY_BLACKLIST_STR = "blacklist";
+static const char *const KEY_WHITELIST_STR = "whitelist";
+static const char *const KEY_MODE_STR = "mode";
+static const char *const KEY_ALLOWFWDPORT_STR = "allowfwdport";
+static const char *const KEY_REPLICATIONFACTOR_STR = "replicationfactor";
+
+// parameter strings
+static const char *const WEIGHT_EQUALS_STRING = "weight=";
+static const char *const GROUP_EQUALS_STRING = "group=";
+static const char *const KEY_MODE_PREREMAP_STR = "pre-remap";
+static const char *const KEY_MODE_POSTREMAP_STR = "post-remap";
+
+
+/**********************************************************/
+bool
+getInt(char** pptr, int *val)
+{
+ bool bReturn = false;
+ int v=0;
+
+ char* ptr = *pptr;
+ // skip white space if any
+ while (*ptr && isspace(*ptr)) ++ptr;
+ // get digits
+ if (*ptr) {
+ while (*ptr && isdigit(*ptr)) {
+ v *= 10;
+ v += (*ptr)-'0';
+ ++ptr;
+ bReturn = true;
+ }
+ }
+ if (bReturn) {
+ *pptr = ptr;
+ *val = v;
+ }
+
+ return bReturn;
+}
+
+/**********************************************************/
+// [http[s]://]host[:port]/path
+bool
+getHostAndPort(char** pptr,string* sHost,int* iPort, string* sScheme)
+{
+ bool bReturn = false;
+ char* ptr = *pptr;
+
+ *iPort = 80;
+ *sScheme = DEFAULT_SCHEME;
+ if(strncmp(*pptr,"https",5) == 0) {
+ *iPort = 443;
+ *sScheme = "https";
+ }
+
+ //skip leading white space
+ while (*ptr && isspace(*ptr)) ++ptr;
+
+ if (*ptr) { // validate not end of string
+ if(strncmp(ptr,"http://",7) == 0) {
+ ptr += 7;
+ } else if(strncmp(ptr,"https://",8) == 0) {
+ ptr += 8;
+ }
+
+ char* ptemp = ptr; // start of host
+ //find white space or ':' or '/'
+ while (*ptr && !isspace(*ptr) && *ptr != ':' && *ptr != '/') ++ptr;
+
+ *sHost = string(ptemp, (ptr - ptemp));
+ // skip white space (if any) after host
+ while (*ptr && isspace(*ptr)) ++ptr;
+ bReturn = true;
+
+ if (*ptr) {// have more to parse
+ // need to get port number?
+ if (*ptr == ':') {
+ ++ptr;
+ if (!getInt(&ptr, iPort)) {
+ // could be our special 'PORT' value, check for that
+ if(!strncmp(ptr,"{port}",6)) { // yes, is '{port}'
+ *iPort = -1;
+ bReturn = true;
+ } else { // really was an error
+ TSError("carp: error parsing port number from '%s'", *pptr);
+ bReturn = false;
+ }
+ } else {
+ // if port number is 443, treat the scheme as https
+ if (*iPort == 443) {
+ *sScheme = "https";
+ }
+ }
+ }
+ }
+ }
+ if(bReturn) {
+ *pptr = ptr;
+ }
+
+ return bReturn;
+}
+/**********************************************************/
+CarpConfig::CarpConfig()
+{
+ _healthCheckPort = DEFAULT_HEALTH_CHECK_PORT;
+ _healthCheckFreq = DEFAULT_HEALTH_CHECK_FREQ;
+ _configCheckFreq = DEFAULT_CONFIG_RELOAD_FREQ;
+ _healthCheckTimeout = DEFAULT_HEALTH_CHECK_TIMEOUT;
+ _setExit = 0;
+ _mode = PRE;
+ _allowForwardPort = 0;
+ _replicationFactor = DEFAULT_REPLICATION_FACTOR;
+ _nGroups = 0;
+}
+
+/**********************************************************/
+CarpConfig::~CarpConfig()
+{
+ for(size_t ptr = 0; ptr < _servers.size(); ptr++) {
+ delete _servers[ptr];
+ }
+
+ for(size_t ptr = 0; ptr < _httpClients.size(); ptr++) {
+ delete _httpClients[ptr];
+ }
+}
+
+/**********************************************************/
+bool
+CarpConfig::loadConfig(string filename)
+{
+ TSFile file;
+ GroupCountList group_counts;
+ GroupCountListIter group_counts_it;
+
+ // attempt to open config file assuming full path provided
+ TSDebug(DEBUG_TAG_INIT, "Trying to open config file in this path: %s", filename.c_str());
+ file = TSfopen(filename.c_str(), "r");
+ if (file == NULL) {
+ TSError("Failed to open carp config file %s", filename.c_str());
+ return false;
+ }
+
+ TSDebug(DEBUG_TAG_INIT, "Successfully opened config file");
+
+ char buffer[1024];
+ memset(buffer, 0, sizeof(buffer));
+
+ enum CONFIG_SECTION {
+ CFG_NONE_SECTION = 0,
+ CFG_SERVER_SECTION,
+ CFG_VALUES_SECTION
+ };
+ int cfg_section = CFG_NONE_SECTION;
+ bool done_parsing = false;
+ while (TSfgets(file, buffer, sizeof(buffer) - 1) != NULL && !done_parsing) {
+
+ char *eol = 0;
+ // make sure line was not bigger than buffer
+ if ((eol = strchr(buffer, '\n')) == NULL && (eol = strstr(buffer, "\r\n")) == NULL) {
+ TSError("carp config line was too long, did not get a good line in cfg, skipping, line: %s", buffer);
+ memset(buffer, 0, sizeof(buffer));
+ continue;
+ }
+
+ // make sure line has something useful on it
+ if (eol - buffer < 2 || buffer[0] == '#' || isspace(buffer[0])) {
+ memset(buffer, 0, sizeof(buffer));
+ continue;
+ }
+
+ // remove ending CR/LF
+ *eol = 0;
+
+ // check if we are changing sections
+ if (strncasecmp(buffer, SECTION_SERVERS_STR, strlen(SECTION_SERVERS_STR)) == 0) {
+ cfg_section = CFG_SERVER_SECTION;
+ TSDebug(DEBUG_TAG_INIT, "Parsing [Servers] section");
+ continue;
+ } else if (strncasecmp(buffer, SECTION_VALUES_STR, strlen(SECTION_VALUES_STR)) == 0) {
+ cfg_section = CFG_VALUES_SECTION;
+ TSDebug(DEBUG_TAG_INIT, "Parsing [Values] section");
+ continue;
+ }
+
+ //TSDebug(DEBUG_TAG_INIT, "config line input:'%s'", buffer);
+
+ switch (cfg_section) {
+ case CFG_SERVER_SECTION:
+ {
+ string sHost;
+ int iPort = DEFAULT_PORT;
+ int iWeight = DEFAULT_WEIGHT;
+ int iGroup = DEFAULT_GROUP;
+ string sScheme = DEFAULT_SCHEME;
+ bool bSuccess = true;
+
+ char* ptr = buffer;
+
+ if(!getHostAndPort(&ptr,&sHost,&iPort, &sScheme)) {
+ TSError("carp: error parsing port number from '%s'", ptr);
+ }
+
+ while (*ptr) {
+ // skip white space (if any)
+ while (*ptr && isspace(*ptr)) ++ptr;
+
+ if (*ptr) { // next we could find weight= or group=
+ if (strncmp(ptr, WEIGHT_EQUALS_STRING, strlen(WEIGHT_EQUALS_STRING)) == 0) {
+ ptr += strlen(WEIGHT_EQUALS_STRING);
+ if (!getInt(&ptr, &iWeight)) {
+ TSError("carp: error parsing weight value from '%s'", buffer);
+ bSuccess = false;
+ continue;
+ }
+ } else if (strncmp(ptr, GROUP_EQUALS_STRING, strlen(GROUP_EQUALS_STRING)) == 0) {
+ ptr += strlen(GROUP_EQUALS_STRING);
+ if (!getInt(&ptr, &iGroup)) {
+ TSError("carp: error parsing group value from '%s'", buffer);
+ bSuccess = false;
+ continue;
+ }
+ } else {
+ TSError("carp: error parsing from line '%s'", buffer);
+ // malformed entry, skip to next space
+ while (*ptr && !isspace(*ptr)) ++ptr;
+ bSuccess = false;
+ continue;
+ }
+ }
+ }
+
+ if (!bSuccess) continue;
+
+ group_counts_it = group_counts.find(iGroup);
+ if (group_counts_it != group_counts.end()) {
+ group_counts[iGroup]++;
+ } else {
+ _nGroups++;
+ group_counts[iGroup] = 1;
+ }
+
+ TSDebug(DEBUG_TAG_INIT, "Host = %s, port=%d, weight=%d, group=%d", sHost.c_str(), iPort, iWeight, iGroup);
+ CarpHost* host=new CarpHost(sHost, iPort, sScheme, iWeight, iGroup);
+ TSAssert(host != NULL);
+ // store the parsed data
+ addHost(host);
+ break;
+ }
+ case CFG_VALUES_SECTION:
+ {
+ char* ptr = buffer;
+
+ //skip leading white space
+ while (*ptr && isspace(*ptr)) ++ptr;
+
+ //find end of key
+ while (*ptr && !isspace(*ptr) && *ptr != '=') ++ptr;
+
+ string sKey = string(buffer, (ptr - buffer));
+
+ // skip white space (if any) after key
+ while (*ptr && isspace(*ptr)) ++ptr;
+ if (*ptr != '=') {
+ TSError("carp: expecting '=' after key in line '%s'", buffer);
+ continue;
+ }
+
+ // skip '=' and white space (if any) after '='
+ ++ptr;
+ while (*ptr && isspace(*ptr)) ++ptr;
+
+ char* ptemp = ptr; // get start of value
+
+ // find end of value
+ while (*ptr && !isspace(*ptr)) ++ptr;
+ string sValue = string(ptemp, ptr - ptemp);
+
+ TSDebug(DEBUG_TAG_INIT, "Key=%s Value=%s", sKey.c_str(), sValue.c_str());
+
+ char* pTemp = (char *)sValue.c_str();
+ if (sKey.compare(KEY_HEALTHCHECK_STR) == 0) {
+ string sH;
+ int iP;
+ string scheme;
+ _healthCheckUrl = string(pTemp);
+ if (!getHostAndPort(&pTemp, &sH, &iP, &scheme)) {
+ TSError("carp: error parsing host and/or port number from '%s'", buffer);
+ }
+
+ // store the parsed data
+ _healthCheckPort = iP;
+ TSDebug(DEBUG_TAG_INIT, "healthcheck Url=%s port=%d", _healthCheckUrl.c_str(), _healthCheckPort);
+
+ } else if (sKey.compare(KEY_HEALTHFREQ_STR) == 0) {
+ int iFreq;
+ if (!getInt(&pTemp, &iFreq)) {
+ TSError("carp: error parsing number from '%s'", buffer);
+ } else {
+ TSDebug(DEBUG_TAG_INIT, "healthcheck freq=%d", iFreq);
+ // store the parsed data
+ _healthCheckFreq = iFreq;
+ }
+ } else if (sKey.compare(KEY_HCTIMEOUT_STR) == 0) {
+ int iFreq;
+ if (!getInt(&pTemp, &iFreq)) {
+ TSError("carp: error parsing number from '%s'", buffer);
+ } else {
+ TSDebug(DEBUG_TAG_INIT, "healthcheck timeout value=%d", iFreq);
+ // store the parsed data
+ _healthCheckTimeout = iFreq;
+ }
+ } else if (sKey.compare(KEY_RELOADFREQ_STR) == 0) {
+ int lFreq;
+ if (!getInt(&pTemp, &lFreq)) {
+ TSError("carp: error parsing number from '%s'", buffer);
+ } else {
+ TSDebug(DEBUG_TAG_INIT, "config reload freq=%d", lFreq);
+ _configCheckFreq = lFreq;
+ }
+ } else if (sKey.compare(KEY_BLACKLIST_STR) == 0) {
+ vector<string> results;
+ stringExplode(sValue, string(","), &results);
+ for (vector<string>::iterator it = results.begin(); it != results.end();
+ it++) {
+ TSDebug(DEBUG_TAG_INIT, "Adding blacklist hostname %s",
+ (*it).c_str());
+ _blackList.insert(*it);
+ }
+ } else if (sKey.compare(KEY_WHITELIST_STR) == 0) {
+ vector<string> results;
+ stringExplode(sValue, string(","), &results);
+ for(vector<string>::iterator it=results.begin(); it != results.end(); it++) {
+ TSDebug(DEBUG_TAG_INIT, "Adding whitelist hostname %s",(*it).c_str() );
+ _whiteList.insert(*it);
+ }
+ } else if (sKey.compare(KEY_MODE_STR) == 0) {
+ if(sValue.compare(KEY_MODE_PREREMAP_STR) == 0) {
+ _mode = PRE;
+ } else if(sValue.compare(KEY_MODE_POSTREMAP_STR) == 0) {
+ _mode = POST;
+ } else {
+ TSError("carp: invalid mode in '%s'", buffer);
+ }
+ } else if (sKey.compare(KEY_ALLOWFWDPORT_STR) == 0) {
+ int iPort;
+ if (!getInt(&pTemp, &iPort)) {
+ TSError("carp: error parsing number from '%s'", buffer);
+ }
+ else {
+ TSDebug(DEBUG_TAG_INIT, "Allow forwarding port=%d", iPort);
+ // store the parsed data
+ _allowForwardPort = iPort;
+ }
+ } else if (sKey.compare(KEY_REPLICATIONFACTOR_STR) == 0) {
+ int iFactor;
+ if (!getInt(&pTemp, &iFactor)) {
+ TSError("carp: error parsing number from '%s'", buffer);
+ }
+ else {
+ TSDebug(DEBUG_TAG_INIT, "Replication factor=%d", iFactor);
+ // store the parsed data
+ _replicationFactor = iFactor;
+ }
+
+ } else
+ TSError("carp found bad setting under Values section '%s'", buffer);
+ break;
+ }
+ default:
+ // ignore bad cfg_section
+ TSDebug(DEBUG_TAG_INIT, "hit default in switch, ignoring extra input '%s'",buffer);
+ break;
+ };
+ } //while
+
+ if (_healthCheckTimeout > _healthCheckFreq - 1 ) {
+ _healthCheckTimeout = _healthCheckFreq - 1;
+ }
+
+ TSfclose(file);
+
+ if (_blackList.size() && _whiteList.size() ) {
+ TSError("Carp configured with both blacklist and whitelist, blacklist will be ignored");
+ }
+
+ if (_nGroups > _replicationFactor) {
+ TSError("Too many groups configured! Failing config.");
+ return false;
+ } else {
+ TSDebug(DEBUG_TAG_INIT, "Group Config is as follows:");
+ for (map<int, int>::const_iterator it = group_counts.begin(); it != group_counts.end(); it++) {
+ TSDebug(DEBUG_TAG_INIT, "Group %d has %d members.", it->first, it->second);
+ _group_count_list[it->first] = it->second;
+ }
+ }
+
+ return true;
+}
+
+/**********************************************************/
+void
+CarpConfig::addHost(CarpHost* host) {
+ _servers.push_back(host);
+}
+
+void
+CarpConfig::addHealthCheckClient(HttpFetch *client) {
+ client->setHealthcheckTimeout(_healthCheckTimeout);
+ _httpClients.push_back(client);
+
+}
+
+void
+CarpConfig::setPath(string path) {
+ _configPath = path;
+}
+
+string
+CarpConfig::getPath() {
+ return _configPath;
+}
+
+/**********************************************************/
+void *
+CarpConfigHealthCheckThreadStart(void* data)
+{
+ CarpConfigAndHash* cch = static_cast<CarpConfigAndHash *> (data);
+ TSAssert(cch);
+ return cch->_config->run(cch->_hashAlgo);
+}
+
+/**********************************************************/
+bool
+CarpConfig::isBlackListed(const string& sHost)
+{
+ if(!_blackList.size()) return false;
+ return (_blackList.find(sHost) != _blackList.end());
+}
+
+/**********************************************************/
+bool
+CarpConfig::isWhiteListed(const string& sHost)
+{
+ if(!_whiteList.size()) return false;
+ return (_whiteList.find(sHost) != _whiteList.end());
+}
+
+/**********************************************************/
+/*
+ perform healthchecks on the hosts and mark them up/down
+*/
+
+void*
+CarpConfig::run(HashAlgorithm *hash) {
+
+ // every httpClient would send a health request to each peer
+ // would be nice to not wait and just 'go' when the server is 100% up
+ sleep(5);
--- End diff --
TS_LIFECYCLE_PORTS_READY_HOOK, perhaps?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by SolidWallOfCode <gi...@git.apache.org>.
Github user SolidWallOfCode commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r74271405
--- Diff: plugins/experimental/carp/CarpHashAlgorithm.cc ---
@@ -0,0 +1,396 @@
+/** @file
+
+ Implements the CARP hash algorithm
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+#include <ts/ts.h>
+#include <stdio.h>
+#include <memory.h>
+#include <list>
+
+#include <iostream>
+#include <sstream>
+
+#include "Common.h"
+#include "CarpHashAlgorithm.h"
+
+using namespace std;
+
+/*****************************************************************/
+void
+HashNode::dump(string& s)
+{
+ stringstream ss;
+ string sSockaddr;
+ getStringFromSockaddr(reinterpret_cast<const struct sockaddr *>( &forwardAddr), sSockaddr);
+
+ ss << scheme << "://" << name << ":"<<listenPort<<" ("<<sSockaddr<<") weight:"<<weight<< (_status ? string(" UP ") : string(" DOWN "));
+ if(_statusTime) {
+ ss << "(" << time(NULL)-_statusTime<< "s ago in "<< _statusLatencyMs << "mS)";
+ }
+ ss << " hits:" << _hits;
+ ss << " carp_noforwarded:" << _carp_noforwarded;
+ ss << " carp_forwarded:" << _carp_forwarded;
+ ss << endl;
+ s += ss.str();
+}
+
+/*****************************************************************/
+void
+HashAlgorithm::addHost(std::string name, unsigned int port, std::string scheme, double weight, bool self, struct sockaddr_storage fwdAddr)
+{
+ HashNode* node = new HashNode(name, port, scheme, weight, self, fwdAddr);
+ TSAssert(NULL != node);
+ addHost(node);
+}
+
+/*****************************************************************/
+void
+HashAlgorithm::addHost(HashNode* node)
+{
+ if (!node) return;
+ _hostList.push_back(node);
+}
+
+/*****************************************************************/
+HashNode*
+HashAlgorithm::findStatusByNameAndPort(const string& name, unsigned int port,
+ size_t* index) {
+ /*
+ * Todo: This use loop to find the corresponding HashNode
+ * But the HttpClient and the Hash was related, so we
+ * could use more easier method to write the status
+ */
+ for (size_t ptr = 0; ptr < _hostList.size(); ptr++) {
+ if (_hostList[ptr]->listenPort == port
+ && _hostList[ptr]->name.compare(name) == 0) { // found it
+ if (index) {
+ *index = ptr;
+ }
+ return _hostList[ptr];
+ }
+ }
+ return NULL;
+}
+
+size_t
+HashAlgorithm::findHashNodeIndex(HashNode *node) {
+ for (size_t ptr = 0; ptr < _hostList.size(); ptr++) {
+ if ( _hostList[ptr] == node) {
+ return ptr;
+ }
+ }
+ return -1;
+}
+
+/*****************************************************************/
+void
+HashAlgorithm::setStatus(const string& name, unsigned int port, bool status, time_t time, uint64_t latencyMs)
+{
+ TSDebug(DEBUG_TAG_INIT, "HashAlgorithm::setStatus name=%s status=%d", name.c_str(), status);
+
+ HashNode* node = findStatusByNameAndPort(name,port);
+ if(node) {
+ node->setStatus(status,time,latencyMs);
+ } else {
+ TSError("Carp internal error setStatus host %s not found",name.c_str());
+ }
+}
+
+void
+HashAlgorithm::setStatus(HashNode * node, bool status, time_t time, uint64_t latencyMs)
+{
+ TSDebug(DEBUG_TAG_INIT, "HashAlgorithm::setStatus name=%s status=%d", node->name.c_str(), status);
+
+// HashNode* node = findStatusByNameAndPort(name,port);
+ if(node) {
+ node->setStatus(status,time,latencyMs);
+ } else {
+ TSError("Carp internal error setStatus host %s not found",node->name.c_str());
+ }
+}
+
+/*****************************************************************/
+HashAlgorithm::~HashAlgorithm()
+{
+ for (size_t ptr = 0; ptr < _hostList.size(); ptr++) {
+ delete _hostList[ptr];
+ }
+}
+
+/*****************************************************************/
+void
+HashAlgorithm::dump(string& s)
+{
+ _config->dump(s);
+
+ stringstream ss;
+ ss << "Hash Algo stats:" << endl;
+ s += ss.str();
+
+ for (size_t ptr = 0; ptr < _hostList.size(); ptr++) {
+ ss.str("");
+ ss.clear();
+ ss << ptr << "-";
+ s += ss.str();
+ _hostList[ptr]->dump(s);
+ }
+}
+
+/*****************************************************************/
+void
+CarpHashAlgorithm::algoInit()
+{
+ // calculate hash for each host
+ for (size_t ptr = 0; ptr < _hostList.size(); ptr++) {
+ std::string cname;
+ char buf[64];
+ memset(buf, 0, 64);
+ snprintf(buf, 64, "%s:%d", _hostList[ptr]->name.c_str(), _hostList[ptr]->listenPort);
+ cname = buf;
+
+ _hostList[ptr]->hash = _calculateHash(cname);
+ }
+
+ // calculate loadmultiplier
+ _calculateLoadMultiplier();
+
+ for (size_t ptr = 0; ptr < _hostList.size(); ptr++) {
+ TSDebug(DEBUG_TAG_INIT, "algoInit host=%s port=%d hash=0x%x weight=%g loadFac=%g loadMult=%g isSelf=%d status=%d",
+ _hostList[ptr]->name.c_str(),
+ _hostList[ptr]->listenPort,
+ _hostList[ptr]->hash,
+ _hostList[ptr]->weight,
+ _hostList[ptr]->loadFactor,
+ _hostList[ptr]->loadMultiplier,
+ _hostList[ptr]->isSelf,
+ _hostList[ptr]->getStatus());
+ }
+}
+
+/*****************************************************************/
+std::vector<HashNode*>
+CarpHashAlgorithm::getRemapProxyList(const std::string& url)
+{
+ std::vector<HashNode *> hn=_selectReplicateNodes(url);
--- End diff --
`return _selectReplicateNodes(url);` ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by SolidWallOfCode <gi...@git.apache.org>.
Github user SolidWallOfCode commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r74178454
--- Diff: plugins/experimental/carp/CarpConfig.cc ---
@@ -0,0 +1,581 @@
+/** @file
+
+ Loads the CARP configuration
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+
+//////////////////////////////////////////////////////////////
+// Read CARP configuration file
+// [Servers]
+// host1.yahoo.com:4080 weight=2 # port 4080 on host1.yahoo.com with weight factor of 2
+// host2.yahoo.com # port 80 on host2.yahoo.com with (default) weight factor of 1
+//
+// [Values]
+// healthcheck={host}:8001/status.html
+// healthfreq=30
+// global=on
+//
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <memory.h>
+#include <ts/ts.h>
+#include <sys/time.h>
+
+#include <sstream>
+
+#include "CarpConfig.h"
+#include "Common.h"
+#include "CarpConfigPool.h"
+
+using namespace std;
+
+#define DEFAULT_HEALTH_CHECK_FREQ 30 // 30 second period for health checks
+#define DEFAULT_HEALTH_CHECK_PORT 80 // default to makeing healthcheck requests against port 80
+#define DEFAULT_CONFIG_RELOAD_FREQ 30 // 30 seconds used in TSContSchedule
+#define DEFAULT_PORT 80 // default to makeing requests against port 80
+#define DEFAULT_WEIGHT 1 // default weight
+#define DEFAULT_SCHEME "http"
+#define DEFAULT_REPLICATION_FACTOR 1
+
+// config section headers
+static const char *const SECTION_SERVERS_STR = "[Servers]";
+static const char *const SECTION_VALUES_STR = "[Values]";
+
+// key strings
+static const char *const KEY_HEALTHCHECK_STR = "healthcheck";
+static const char *const KEY_HEALTHFREQ_STR = "healthfreq";
+static const char *const KEY_RELOADFREQ_STR = "reloadfreq";
+static const char *const KEY_HCTIMEOUT_STR = "hctimeout";
+static const char *const KEY_BLACKLIST_STR = "blacklist";
+static const char *const KEY_WHITELIST_STR = "whitelist";
+static const char *const KEY_MODE_STR = "mode";
+static const char *const KEY_ALLOWFWDPORT_STR = "allowfwdport";
+static const char *const KEY_REPLICATIONFACTOR_STR = "replicationfactor";
+
+// parameter strings
+static const char *const WEIGHT_EQUALS_STRING = "weight=";
+static const char *const GROUP_EQUALS_STRING = "group=";
+static const char *const KEY_MODE_PREREMAP_STR = "pre-remap";
+static const char *const KEY_MODE_POSTREMAP_STR = "post-remap";
+
+
+/**********************************************************/
+bool
+getInt(char** pptr, int *val)
+{
+ bool bReturn = false;
+ int v=0;
+
+ char* ptr = *pptr;
+ // skip white space if any
+ while (*ptr && isspace(*ptr)) ++ptr;
+ // get digits
+ if (*ptr) {
+ while (*ptr && isdigit(*ptr)) {
+ v *= 10;
+ v += (*ptr)-'0';
+ ++ptr;
+ bReturn = true;
+ }
+ }
+ if (bReturn) {
+ *pptr = ptr;
+ *val = v;
+ }
+
+ return bReturn;
+}
+
+/**********************************************************/
+// [http[s]://]host[:port]/path
+bool
+getHostAndPort(char** pptr,string* sHost,int* iPort, string* sScheme)
+{
+ bool bReturn = false;
+ char* ptr = *pptr;
+
+ *iPort = 80;
+ *sScheme = DEFAULT_SCHEME;
+ if(strncmp(*pptr,"https",5) == 0) {
+ *iPort = 443;
+ *sScheme = "https";
+ }
+
+ //skip leading white space
+ while (*ptr && isspace(*ptr)) ++ptr;
+
+ if (*ptr) { // validate not end of string
+ if(strncmp(ptr,"http://",7) == 0) {
+ ptr += 7;
+ } else if(strncmp(ptr,"https://",8) == 0) {
+ ptr += 8;
+ }
+
+ char* ptemp = ptr; // start of host
+ //find white space or ':' or '/'
+ while (*ptr && !isspace(*ptr) && *ptr != ':' && *ptr != '/') ++ptr;
+
+ *sHost = string(ptemp, (ptr - ptemp));
+ // skip white space (if any) after host
+ while (*ptr && isspace(*ptr)) ++ptr;
+ bReturn = true;
+
+ if (*ptr) {// have more to parse
+ // need to get port number?
+ if (*ptr == ':') {
+ ++ptr;
+ if (!getInt(&ptr, iPort)) {
+ // could be our special 'PORT' value, check for that
+ if(!strncmp(ptr,"{port}",6)) { // yes, is '{port}'
+ *iPort = -1;
+ bReturn = true;
+ } else { // really was an error
+ TSError("carp: error parsing port number from '%s'", *pptr);
+ bReturn = false;
+ }
+ } else {
+ // if port number is 443, treat the scheme as https
+ if (*iPort == 443) {
+ *sScheme = "https";
+ }
+ }
+ }
+ }
+ }
+ if(bReturn) {
+ *pptr = ptr;
+ }
+
+ return bReturn;
+}
+/**********************************************************/
+CarpConfig::CarpConfig()
+{
+ _healthCheckPort = DEFAULT_HEALTH_CHECK_PORT;
+ _healthCheckFreq = DEFAULT_HEALTH_CHECK_FREQ;
+ _configCheckFreq = DEFAULT_CONFIG_RELOAD_FREQ;
+ _healthCheckTimeout = DEFAULT_HEALTH_CHECK_TIMEOUT;
+ _setExit = 0;
+ _mode = PRE;
+ _allowForwardPort = 0;
+ _replicationFactor = DEFAULT_REPLICATION_FACTOR;
+ _nGroups = 0;
+}
+
+/**********************************************************/
+CarpConfig::~CarpConfig()
+{
+ for(size_t ptr = 0; ptr < _servers.size(); ptr++) {
+ delete _servers[ptr];
+ }
+
+ for(size_t ptr = 0; ptr < _httpClients.size(); ptr++) {
+ delete _httpClients[ptr];
+ }
+}
+
+/**********************************************************/
+bool
+CarpConfig::loadConfig(string filename)
+{
+ TSFile file;
+ GroupCountList group_counts;
+ GroupCountListIter group_counts_it;
+
+ // attempt to open config file assuming full path provided
+ TSDebug(DEBUG_TAG_INIT, "Trying to open config file in this path: %s", filename.c_str());
+ file = TSfopen(filename.c_str(), "r");
+ if (file == NULL) {
+ TSError("Failed to open carp config file %s", filename.c_str());
+ return false;
+ }
+
+ TSDebug(DEBUG_TAG_INIT, "Successfully opened config file");
+
+ char buffer[1024];
+ memset(buffer, 0, sizeof(buffer));
+
+ enum CONFIG_SECTION {
+ CFG_NONE_SECTION = 0,
+ CFG_SERVER_SECTION,
+ CFG_VALUES_SECTION
+ };
+ int cfg_section = CFG_NONE_SECTION;
+ bool done_parsing = false;
+ while (TSfgets(file, buffer, sizeof(buffer) - 1) != NULL && !done_parsing) {
+
+ char *eol = 0;
+ // make sure line was not bigger than buffer
+ if ((eol = strchr(buffer, '\n')) == NULL && (eol = strstr(buffer, "\r\n")) == NULL) {
+ TSError("carp config line was too long, did not get a good line in cfg, skipping, line: %s", buffer);
+ memset(buffer, 0, sizeof(buffer));
+ continue;
+ }
+
+ // make sure line has something useful on it
+ if (eol - buffer < 2 || buffer[0] == '#' || isspace(buffer[0])) {
+ memset(buffer, 0, sizeof(buffer));
+ continue;
+ }
+
+ // remove ending CR/LF
+ *eol = 0;
+
+ // check if we are changing sections
+ if (strncasecmp(buffer, SECTION_SERVERS_STR, strlen(SECTION_SERVERS_STR)) == 0) {
+ cfg_section = CFG_SERVER_SECTION;
+ TSDebug(DEBUG_TAG_INIT, "Parsing [Servers] section");
+ continue;
+ } else if (strncasecmp(buffer, SECTION_VALUES_STR, strlen(SECTION_VALUES_STR)) == 0) {
+ cfg_section = CFG_VALUES_SECTION;
+ TSDebug(DEBUG_TAG_INIT, "Parsing [Values] section");
+ continue;
+ }
+
+ //TSDebug(DEBUG_TAG_INIT, "config line input:'%s'", buffer);
+
+ switch (cfg_section) {
+ case CFG_SERVER_SECTION:
+ {
+ string sHost;
+ int iPort = DEFAULT_PORT;
+ int iWeight = DEFAULT_WEIGHT;
+ int iGroup = DEFAULT_GROUP;
+ string sScheme = DEFAULT_SCHEME;
+ bool bSuccess = true;
+
+ char* ptr = buffer;
+
+ if(!getHostAndPort(&ptr,&sHost,&iPort, &sScheme)) {
+ TSError("carp: error parsing port number from '%s'", ptr);
+ }
+
+ while (*ptr) {
+ // skip white space (if any)
+ while (*ptr && isspace(*ptr)) ++ptr;
+
+ if (*ptr) { // next we could find weight= or group=
+ if (strncmp(ptr, WEIGHT_EQUALS_STRING, strlen(WEIGHT_EQUALS_STRING)) == 0) {
+ ptr += strlen(WEIGHT_EQUALS_STRING);
+ if (!getInt(&ptr, &iWeight)) {
+ TSError("carp: error parsing weight value from '%s'", buffer);
+ bSuccess = false;
+ continue;
+ }
+ } else if (strncmp(ptr, GROUP_EQUALS_STRING, strlen(GROUP_EQUALS_STRING)) == 0) {
+ ptr += strlen(GROUP_EQUALS_STRING);
+ if (!getInt(&ptr, &iGroup)) {
+ TSError("carp: error parsing group value from '%s'", buffer);
+ bSuccess = false;
+ continue;
+ }
+ } else {
+ TSError("carp: error parsing from line '%s'", buffer);
+ // malformed entry, skip to next space
+ while (*ptr && !isspace(*ptr)) ++ptr;
+ bSuccess = false;
+ continue;
+ }
+ }
+ }
+
+ if (!bSuccess) continue;
+
+ group_counts_it = group_counts.find(iGroup);
+ if (group_counts_it != group_counts.end()) {
+ group_counts[iGroup]++;
+ } else {
+ _nGroups++;
+ group_counts[iGroup] = 1;
+ }
+
+ TSDebug(DEBUG_TAG_INIT, "Host = %s, port=%d, weight=%d, group=%d", sHost.c_str(), iPort, iWeight, iGroup);
+ CarpHost* host=new CarpHost(sHost, iPort, sScheme, iWeight, iGroup);
+ TSAssert(host != NULL);
+ // store the parsed data
+ addHost(host);
+ break;
+ }
+ case CFG_VALUES_SECTION:
+ {
+ char* ptr = buffer;
+
+ //skip leading white space
+ while (*ptr && isspace(*ptr)) ++ptr;
+
+ //find end of key
+ while (*ptr && !isspace(*ptr) && *ptr != '=') ++ptr;
+
+ string sKey = string(buffer, (ptr - buffer));
+
+ // skip white space (if any) after key
+ while (*ptr && isspace(*ptr)) ++ptr;
+ if (*ptr != '=') {
+ TSError("carp: expecting '=' after key in line '%s'", buffer);
+ continue;
+ }
+
+ // skip '=' and white space (if any) after '='
+ ++ptr;
+ while (*ptr && isspace(*ptr)) ++ptr;
+
+ char* ptemp = ptr; // get start of value
+
+ // find end of value
+ while (*ptr && !isspace(*ptr)) ++ptr;
+ string sValue = string(ptemp, ptr - ptemp);
+
+ TSDebug(DEBUG_TAG_INIT, "Key=%s Value=%s", sKey.c_str(), sValue.c_str());
+
+ char* pTemp = (char *)sValue.c_str();
+ if (sKey.compare(KEY_HEALTHCHECK_STR) == 0) {
+ string sH;
+ int iP;
+ string scheme;
+ _healthCheckUrl = string(pTemp);
+ if (!getHostAndPort(&pTemp, &sH, &iP, &scheme)) {
+ TSError("carp: error parsing host and/or port number from '%s'", buffer);
+ }
+
+ // store the parsed data
+ _healthCheckPort = iP;
+ TSDebug(DEBUG_TAG_INIT, "healthcheck Url=%s port=%d", _healthCheckUrl.c_str(), _healthCheckPort);
+
+ } else if (sKey.compare(KEY_HEALTHFREQ_STR) == 0) {
+ int iFreq;
+ if (!getInt(&pTemp, &iFreq)) {
+ TSError("carp: error parsing number from '%s'", buffer);
+ } else {
+ TSDebug(DEBUG_TAG_INIT, "healthcheck freq=%d", iFreq);
+ // store the parsed data
+ _healthCheckFreq = iFreq;
+ }
+ } else if (sKey.compare(KEY_HCTIMEOUT_STR) == 0) {
+ int iFreq;
+ if (!getInt(&pTemp, &iFreq)) {
+ TSError("carp: error parsing number from '%s'", buffer);
+ } else {
+ TSDebug(DEBUG_TAG_INIT, "healthcheck timeout value=%d", iFreq);
+ // store the parsed data
+ _healthCheckTimeout = iFreq;
+ }
+ } else if (sKey.compare(KEY_RELOADFREQ_STR) == 0) {
+ int lFreq;
+ if (!getInt(&pTemp, &lFreq)) {
+ TSError("carp: error parsing number from '%s'", buffer);
+ } else {
+ TSDebug(DEBUG_TAG_INIT, "config reload freq=%d", lFreq);
+ _configCheckFreq = lFreq;
+ }
+ } else if (sKey.compare(KEY_BLACKLIST_STR) == 0) {
+ vector<string> results;
+ stringExplode(sValue, string(","), &results);
+ for (vector<string>::iterator it = results.begin(); it != results.end();
+ it++) {
+ TSDebug(DEBUG_TAG_INIT, "Adding blacklist hostname %s",
+ (*it).c_str());
+ _blackList.insert(*it);
+ }
+ } else if (sKey.compare(KEY_WHITELIST_STR) == 0) {
+ vector<string> results;
+ stringExplode(sValue, string(","), &results);
+ for(vector<string>::iterator it=results.begin(); it != results.end(); it++) {
+ TSDebug(DEBUG_TAG_INIT, "Adding whitelist hostname %s",(*it).c_str() );
+ _whiteList.insert(*it);
+ }
+ } else if (sKey.compare(KEY_MODE_STR) == 0) {
+ if(sValue.compare(KEY_MODE_PREREMAP_STR) == 0) {
+ _mode = PRE;
+ } else if(sValue.compare(KEY_MODE_POSTREMAP_STR) == 0) {
+ _mode = POST;
+ } else {
+ TSError("carp: invalid mode in '%s'", buffer);
+ }
+ } else if (sKey.compare(KEY_ALLOWFWDPORT_STR) == 0) {
+ int iPort;
+ if (!getInt(&pTemp, &iPort)) {
+ TSError("carp: error parsing number from '%s'", buffer);
+ }
+ else {
+ TSDebug(DEBUG_TAG_INIT, "Allow forwarding port=%d", iPort);
+ // store the parsed data
+ _allowForwardPort = iPort;
+ }
+ } else if (sKey.compare(KEY_REPLICATIONFACTOR_STR) == 0) {
+ int iFactor;
+ if (!getInt(&pTemp, &iFactor)) {
+ TSError("carp: error parsing number from '%s'", buffer);
+ }
+ else {
+ TSDebug(DEBUG_TAG_INIT, "Replication factor=%d", iFactor);
+ // store the parsed data
+ _replicationFactor = iFactor;
+ }
+
+ } else
+ TSError("carp found bad setting under Values section '%s'", buffer);
+ break;
+ }
+ default:
+ // ignore bad cfg_section
+ TSDebug(DEBUG_TAG_INIT, "hit default in switch, ignoring extra input '%s'",buffer);
+ break;
+ };
+ } //while
+
+ if (_healthCheckTimeout > _healthCheckFreq - 1 ) {
+ _healthCheckTimeout = _healthCheckFreq - 1;
--- End diff --
Should probably generate a diagnostic in this case.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by SolidWallOfCode <gi...@git.apache.org>.
Github user SolidWallOfCode commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r74262675
--- Diff: plugins/experimental/carp/CarpHashAlgorithm.cc ---
@@ -0,0 +1,396 @@
+/** @file
+
+ Implements the CARP hash algorithm
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+#include <ts/ts.h>
+#include <stdio.h>
+#include <memory.h>
+#include <list>
+
+#include <iostream>
+#include <sstream>
+
+#include "Common.h"
+#include "CarpHashAlgorithm.h"
+
+using namespace std;
+
+/*****************************************************************/
+void
+HashNode::dump(string& s)
+{
+ stringstream ss;
+ string sSockaddr;
+ getStringFromSockaddr(reinterpret_cast<const struct sockaddr *>( &forwardAddr), sSockaddr);
+
+ ss << scheme << "://" << name << ":"<<listenPort<<" ("<<sSockaddr<<") weight:"<<weight<< (_status ? string(" UP ") : string(" DOWN "));
+ if(_statusTime) {
+ ss << "(" << time(NULL)-_statusTime<< "s ago in "<< _statusLatencyMs << "mS)";
+ }
+ ss << " hits:" << _hits;
+ ss << " carp_noforwarded:" << _carp_noforwarded;
+ ss << " carp_forwarded:" << _carp_forwarded;
+ ss << endl;
+ s += ss.str();
+}
+
+/*****************************************************************/
+void
+HashAlgorithm::addHost(std::string name, unsigned int port, std::string scheme, double weight, bool self, struct sockaddr_storage fwdAddr)
--- End diff --
Pass `string` by `const&` where possible.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by SolidWallOfCode <gi...@git.apache.org>.
Github user SolidWallOfCode commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r75878885
--- Diff: plugins/experimental/carp/HttpFetch.cc ---
@@ -0,0 +1,439 @@
+/** @file
+
+ Limited URL fetcher..
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+
+#define __STDC_LIMIT_MACROS // need INT64_MAX definition
+#include <stdint.h>
+#include <sys/time.h>
+
+#include "HttpFetch.h"
+#include "UrlComponents.h"
+#include "Common.h"
+
+using std::string;
+
+/**********************************************************/
+// just 'pass through' and call object's handlEvent fn
+
+static int
+handleHttpFetchIOEvents(TSCont cont, TSEvent event, void *edata)
+{
+ HttpFetch *fetchObj = static_cast<HttpFetch *> (TSContDataGet(cont));
+ if (NULL == fetchObj) {
+ TSDebug(DEBUG_FETCH_TAG, "handleHttpFetchEvents continuation data NULL");
+ TSAssert(fetchObj);
+ }
+ return fetchObj->handleIOEvent(cont, event, edata);
+}
+
+/**********************************************************/
+HttpFetch::HttpFetch(const std::string &url, HashAlgorithm *hashAlgo,
+ HashNode *hashNode,const char *method)
+{
+ TSMBuffer bufp;
+ _url = url;
+ _respInfo = NULL;
+ _reqInfo = NULL;
+ _hashAlgo = hashAlgo;
+ _hashNode = hashNode;
+ _hcTimeoutSecond = DEFAULT_HEALTH_CHECK_TIMEOUT;
+
+ bufp = TSMBufferCreate();
+
+ if (bufp != NULL) {
+ TSMLoc urlp;
+ if (TSUrlCreate(bufp, &urlp) == TS_SUCCESS) {
+ const char *start = url.data();
+ if (TSUrlParse(bufp, urlp, &start, start + url.length()) == TS_PARSE_DONE) {
+ UrlComponents reqUrl;
+ string sPath;
+ string sHost;
+ reqUrl.populate(bufp, urlp);
+ reqUrl.getCompletePathString(sPath);
+ reqUrl.getCompleteHostString(sHost);
+ _request = string(method) + " " + sPath + " HTTP/1.0\r\nHost: " + sHost + "\r\n";
+ _request += CARP_ROUTED_HEADER + ": 1\r\n";
+ _request += "\r\n";
+ }
+ TSHandleMLocRelease(bufp, NULL, urlp);
+ }
+ }
+ TSMBufferDestroy(bufp);
+ TSDebug(DEBUG_FETCH_TAG, "HttpFetch assembled this request %s", _request.c_str());
+ _responseStatus=TS_HTTP_STATUS_NONE;
+ _ready = true;
+}
+
+/**********************************************************/
+HttpFetch::~HttpFetch()
+{
+}
+/**********************************************************/
+void
+HttpFetch::setHealthcheckTimeout(int timeout) {
+ _hcTimeoutSecond = timeout;
+}
+/**********************************************************/
+void
+HttpFetch::makeAsyncRequest(struct sockaddr const* serverAddr)
+{
+ _ready = false;
+ __sync_synchronize();
+ _result = UNKNOWN;
+ TSCont fetchCont = TSContCreate(handleHttpFetchIOEvents, TSMutexCreate());
+ //TSCont fetchCont = TSContCreate(handleHttpFetchIOEvents, NULL);
+ TSContDataSet(fetchCont, static_cast<void *> (this));
+
+ // save server addr
+ memmove((void *) &_serverAddr, (void *) serverAddr, sizeof (struct sockaddr));
+
+ // initiate request
+ TSDebug(DEBUG_FETCH_TAG, "TSNetConnect()");
+
+ struct timeval tvStart;
+ gettimeofday(&tvStart, NULL);
+ _startTime = tvStart.tv_sec*1000 + tvStart.tv_usec/1000;
+
+ struct sockaddr_in tempServerAddr;
+ memmove((void *) &tempServerAddr, (void *) &_serverAddr, sizeof (struct sockaddr));
--- End diff --
Type mismatch here too - `sockaddr` vs `sockaddr_in`. Further, why copy to `tempServerAddr`?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by SolidWallOfCode <gi...@git.apache.org>.
Github user SolidWallOfCode commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r74262139
--- Diff: plugins/experimental/carp/CarpConfigPool.cc ---
@@ -0,0 +1,218 @@
+/** @file
+
+ Manage a list of CARP configurations
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+#include <sstream>
+
+#include "Common.h"
+#include "CarpConfigPool.h"
+#include "UrlComponents.h"
+#include "CarpHost.h"
+
+#include <netdb.h>
+#include <ts/ts.h>
+
+using namespace std;
+
+/*******************************************************************/
+CarpConfigPool::CarpConfigPool()
+{
+ _globalHash = NULL;
+ _globalConfig = NULL;
+}
+
+/*******************************************************************/
+CarpConfigPool::~CarpConfigPool()
+{
+ for(CarpConfigListIter it=_configList.begin(); it != _configList.end(); it++) {
+ it->second->_config->stop();
+ delete it->second;
+ }
+}
+/*******************************************************************/
+static int initCarpConfigAndHash(CarpConfigAndHash * cch, string sFilename) {
+ cch->_configPath = sFilename;
+ cch->_config = new CarpConfig();
+ TSAssert(cch->_config);
+
+ if (!cch->_config->loadConfig(sFilename)) {
+ return -1;
+ }
+
+ cch->_hashAlgo = new CarpHashAlgorithm(cch->_config);
+
+ TSAssert(cch->_hashAlgo);
+
+ CarpHostList* hostList = cch->_config->getHostList();
+ // add hosts, etc to hash algo
+ char szServerName[256];
+ *szServerName = 0;
+
+ if (!gethostname(szServerName, 255)) { // success!
+ TSDebug(DEBUG_TAG_INIT, "using %s as server name to detect 'self'",
+ szServerName);
+ }
+ char buf[1024];
+ struct hostent self, *selfhe = getHostIp(string(szServerName), &self, buf,
+ sizeof(buf));
+
+ for (CarpHostListIter i = hostList->begin(); i != hostList->end(); i++) {
+ HashNode *hashNode;
+ bool bSelf = false;
+ if (NULL != selfhe) { // check for 'self'
+ // include hostname and port
+ bSelf = isSelf((*i)->getName(), (*i)->getPort(), selfhe);
+ }
+ if (cch->_config->getHealthCheckPort() == -1) { // did they specify 'PORT'?
+ (*i)->setHealthCheckPort((*i)->getPort()); // set HC port from server spec'd port
+ } else {
+ (*i)->setHealthCheckPort(cch->_config->getHealthCheckPort()); // set HC port
+ }
+ string sHCUrl = cch->_config->getHealthCheckUrl();
+ size_t pos = sHCUrl.find("{port}");
+ if (pos != string::npos) { // need to replace '{port}' with servers port
+ stringstream ss;
+ ss << (*i)->getPort();
+ sHCUrl.replace(pos, 6, ss.str()); // 6 = strlen of '{port}'
+ }
+ pos = sHCUrl.find("{host}");
+ if (pos != string::npos) {
+ sHCUrl.replace(pos, 6, (*i)->getName());
+ }
+ (*i)->setHealthCheckUrl(sHCUrl); // set HC Url
+
+ // Look up host and create addr struct for healthchecks
+ char hBuf[1024];
+ struct hostent host, *hosthe = getHostIp((*i)->getName(), &host, hBuf,
+ sizeof(hBuf));
+ if (hosthe) {
+ // convert hostent to sockaddr_in structure
+ sockaddr_in hIn;
+ memcpy(&hIn.sin_addr, hosthe->h_addr_list[0], hosthe->h_length);
+ hIn.sin_port = htons((*i)->getHealthCheckPort()); // set port
+ if (hosthe->h_length == 4) { // size match IPv4? assume such
+ hIn.sin_family = AF_INET;
+ } else { // assume IPv6
+ hIn.sin_family = AF_INET6;
+ }
+ (*i)->setHealthCheckAddr(
+ reinterpret_cast<struct sockaddr_storage &>(hIn));
+ hIn.sin_port = htons((*i)->getPort());
+ hashNode = new HashNode((*i)->getName(), (*i)->getPort(),
+ (*i)->getScheme(), (*i)->getWeight(), bSelf,
+ reinterpret_cast<struct sockaddr_storage &>(hIn),
+ (*i)->getGroup());
+ cch->_hashAlgo->addHost(hashNode);
+ } else {
+ //Config error or dns error. Should not continue
+ TSError("carp: error get peer address of host '%s'", (*i)->getName().c_str());
+ return -1;
+ }
+
+ HttpFetch *f = new HttpFetch(sHCUrl, cch->_hashAlgo, hashNode);
+ cch->_config->addHealthCheckClient(f);
+ }
+ string diag;
+ cch->_config->dump(diag);
+ TSDebug(DEBUG_TAG_INIT, "Carp Configuration\n%s", diag.c_str());
+
+ // tell algo we are done configuring it
+ cch->_hashAlgo->algoInit();
+
+ return 1;
+}
+
+/*******************************************************************/
+int
+cleanHandler(TSCont cont, TSEvent event, void *edata) {
+ CarpConfigAndHash * cch = (CarpConfigAndHash *)TSContDataGet(cont);
+ delete cch;
+ TSContDestroy(cont);
+
+ return 1;
+}
+
+/*******************************************************************/
+CarpConfigAndHash*
+CarpConfigPool::processConfigFile(string sFilename,bool isGlobal)
+{
+
+ CarpConfigAndHash* cch = _configList[sFilename];
+
+ if ( NULL == cch ) { // new config file
+ cch = new CarpConfigAndHash();
+ _configList[sFilename] = cch;
+ TSDebug(DEBUG_TAG_INIT, "processing new config file '%s'", sFilename.c_str());
+ if (initCarpConfigAndHash(cch, sFilename) < 0 ) {
+ return NULL;
+ }
+
+ if(isGlobal) { // extract global setting(s) from this config file and save locally
+ _globalHash = cch->_hashAlgo;
+ _globalConfig = cch->_config;
+ }
+ // create and start health watcher thread
+ cch->_thread = TSThreadCreate( CarpConfigHealthCheckThreadStart,static_cast<void *> (cch) );
+ } else { // config reload
+
+ TSDebug(DEBUG_TAG_HEALTH, "Reload the config file '%s'", sFilename.c_str());
+ CarpConfigAndHash *newCCH = new CarpConfigAndHash();
+ CarpConfigAndHash *oldCCH = cch;
+ if (initCarpConfigAndHash(newCCH, sFilename) < 0 ) {
+ return NULL;
+ }
+ /*
+ * Find the status of current Host in previous HashAlgo, and
+ * then assign the value to the new HashAlgo.
+ */
+ size_t index;
+ vector<CarpHost*> *list = newCCH->_config->getHostList();
+ for (unsigned int i = 0; i < list->size(); i++) {
+ string name = (*list)[i]->getName();
--- End diff --
Any reason for this to not be `string const&`?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by SolidWallOfCode <gi...@git.apache.org>.
Github user SolidWallOfCode commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r73973367
--- Diff: plugins/experimental/carp/CarpConfig.cc ---
@@ -0,0 +1,581 @@
+/** @file
+
+ Loads the CARP configuration
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+
+//////////////////////////////////////////////////////////////
+// Read CARP configuration file
+// [Servers]
+// host1.yahoo.com:4080 weight=2 # port 4080 on host1.yahoo.com with weight factor of 2
+// host2.yahoo.com # port 80 on host2.yahoo.com with (default) weight factor of 1
+//
+// [Values]
+// healthcheck={host}:8001/status.html
+// healthfreq=30
+// global=on
+//
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <memory.h>
+#include <ts/ts.h>
+#include <sys/time.h>
+
+#include <sstream>
+
+#include "CarpConfig.h"
+#include "Common.h"
+#include "CarpConfigPool.h"
+
+using namespace std;
+
+#define DEFAULT_HEALTH_CHECK_FREQ 30 // 30 second period for health checks
+#define DEFAULT_HEALTH_CHECK_PORT 80 // default to makeing healthcheck requests against port 80
+#define DEFAULT_CONFIG_RELOAD_FREQ 30 // 30 seconds used in TSContSchedule
+#define DEFAULT_PORT 80 // default to makeing requests against port 80
+#define DEFAULT_WEIGHT 1 // default weight
+#define DEFAULT_SCHEME "http"
+#define DEFAULT_REPLICATION_FACTOR 1
+
+// config section headers
+static const char *const SECTION_SERVERS_STR = "[Servers]";
+static const char *const SECTION_VALUES_STR = "[Values]";
+
+// key strings
+static const char *const KEY_HEALTHCHECK_STR = "healthcheck";
+static const char *const KEY_HEALTHFREQ_STR = "healthfreq";
+static const char *const KEY_RELOADFREQ_STR = "reloadfreq";
+static const char *const KEY_HCTIMEOUT_STR = "hctimeout";
+static const char *const KEY_BLACKLIST_STR = "blacklist";
+static const char *const KEY_WHITELIST_STR = "whitelist";
+static const char *const KEY_MODE_STR = "mode";
+static const char *const KEY_ALLOWFWDPORT_STR = "allowfwdport";
+static const char *const KEY_REPLICATIONFACTOR_STR = "replicationfactor";
+
+// parameter strings
+static const char *const WEIGHT_EQUALS_STRING = "weight=";
+static const char *const GROUP_EQUALS_STRING = "group=";
+static const char *const KEY_MODE_PREREMAP_STR = "pre-remap";
+static const char *const KEY_MODE_POSTREMAP_STR = "post-remap";
+
+
+/**********************************************************/
+bool
+getInt(char** pptr, int *val)
+{
+ bool bReturn = false;
+ int v=0;
+
+ char* ptr = *pptr;
+ // skip white space if any
+ while (*ptr && isspace(*ptr)) ++ptr;
+ // get digits
+ if (*ptr) {
+ while (*ptr && isdigit(*ptr)) {
+ v *= 10;
+ v += (*ptr)-'0';
+ ++ptr;
+ bReturn = true;
+ }
+ }
+ if (bReturn) {
+ *pptr = ptr;
+ *val = v;
+ }
+
+ return bReturn;
+}
+
+/**********************************************************/
+// [http[s]://]host[:port]/path
+bool
+getHostAndPort(char** pptr,string* sHost,int* iPort, string* sScheme)
+{
+ bool bReturn = false;
+ char* ptr = *pptr;
+
+ *iPort = 80;
+ *sScheme = DEFAULT_SCHEME;
+ if(strncmp(*pptr,"https",5) == 0) {
+ *iPort = 443;
+ *sScheme = "https";
+ }
+
+ //skip leading white space
+ while (*ptr && isspace(*ptr)) ++ptr;
+
+ if (*ptr) { // validate not end of string
+ if(strncmp(ptr,"http://",7) == 0) {
+ ptr += 7;
+ } else if(strncmp(ptr,"https://",8) == 0) {
+ ptr += 8;
+ }
+
+ char* ptemp = ptr; // start of host
+ //find white space or ':' or '/'
+ while (*ptr && !isspace(*ptr) && *ptr != ':' && *ptr != '/') ++ptr;
--- End diff --
Raw IPv6 addresses aren't going to work, then.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by SolidWallOfCode <gi...@git.apache.org>.
Github user SolidWallOfCode commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r75871198
--- Diff: plugins/experimental/carp/CarpHost.h ---
@@ -0,0 +1,141 @@
+/** @file
+
+ Loads the CARP configuration
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+#ifndef __CARPHOST_H__
+#define __CARPHOST_H__ 1
+
+#define DEFAULT_GROUP 1
+
+#include <string>
+#include <netinet/in.h>
+
+class CarpHost
+{
+public:
+
+ CarpHost(std::string& name, int port, std::string& scheme, int weight, int group = DEFAULT_GROUP) : _port(port),
+ _name(name), _scheme(scheme), _weight(weight), _group(group)
+ {
+ _healthCheckAddr.ss_family=AF_UNSPEC;
+ };
+
+ ~CarpHost()
+ {
+
+ };
+
+ int getPort()
+ {
+ return _port;
+ };
+
+ void setPort(int p)
+ {
+ _port = p;
+ };
+
+ const std::string& getName()
+ {
+ return _name;
+ };
+
+ void setName(std::string& n)
--- End diff --
`std::string const& n`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] trafficserver pull request #843: [TS-4723] ATS CARP Plugin
Posted by SolidWallOfCode <gi...@git.apache.org>.
Github user SolidWallOfCode commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/843#discussion_r74175678
--- Diff: plugins/experimental/carp/CarpConfig.cc ---
@@ -0,0 +1,581 @@
+/** @file
+
+ Loads the CARP configuration
+
+ @section license License
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ */
+
+//////////////////////////////////////////////////////////////
+// Read CARP configuration file
+// [Servers]
+// host1.yahoo.com:4080 weight=2 # port 4080 on host1.yahoo.com with weight factor of 2
+// host2.yahoo.com # port 80 on host2.yahoo.com with (default) weight factor of 1
+//
+// [Values]
+// healthcheck={host}:8001/status.html
+// healthfreq=30
+// global=on
+//
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <memory.h>
+#include <ts/ts.h>
+#include <sys/time.h>
+
+#include <sstream>
+
+#include "CarpConfig.h"
+#include "Common.h"
+#include "CarpConfigPool.h"
+
+using namespace std;
+
+#define DEFAULT_HEALTH_CHECK_FREQ 30 // 30 second period for health checks
+#define DEFAULT_HEALTH_CHECK_PORT 80 // default to makeing healthcheck requests against port 80
+#define DEFAULT_CONFIG_RELOAD_FREQ 30 // 30 seconds used in TSContSchedule
+#define DEFAULT_PORT 80 // default to makeing requests against port 80
+#define DEFAULT_WEIGHT 1 // default weight
+#define DEFAULT_SCHEME "http"
+#define DEFAULT_REPLICATION_FACTOR 1
+
+// config section headers
+static const char *const SECTION_SERVERS_STR = "[Servers]";
+static const char *const SECTION_VALUES_STR = "[Values]";
+
+// key strings
+static const char *const KEY_HEALTHCHECK_STR = "healthcheck";
+static const char *const KEY_HEALTHFREQ_STR = "healthfreq";
+static const char *const KEY_RELOADFREQ_STR = "reloadfreq";
+static const char *const KEY_HCTIMEOUT_STR = "hctimeout";
+static const char *const KEY_BLACKLIST_STR = "blacklist";
+static const char *const KEY_WHITELIST_STR = "whitelist";
+static const char *const KEY_MODE_STR = "mode";
+static const char *const KEY_ALLOWFWDPORT_STR = "allowfwdport";
+static const char *const KEY_REPLICATIONFACTOR_STR = "replicationfactor";
+
+// parameter strings
+static const char *const WEIGHT_EQUALS_STRING = "weight=";
+static const char *const GROUP_EQUALS_STRING = "group=";
+static const char *const KEY_MODE_PREREMAP_STR = "pre-remap";
+static const char *const KEY_MODE_POSTREMAP_STR = "post-remap";
+
+
+/**********************************************************/
+bool
+getInt(char** pptr, int *val)
--- End diff --
Couldn't this be done more reliably with `strtol`?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---