You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@trafficserver.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2016/08/16 02:22:21 UTC
[jira] [Work logged] (TS-4042) Add feature to buffer request body
before making downstream requests
[ https://issues.apache.org/jira/browse/TS-4042?focusedWorklogId=26482&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-26482 ]
ASF GitHub Bot logged work on TS-4042:
--------------------------------------
Author: ASF GitHub Bot
Created on: 16/Aug/16 02:22
Start Date: 16/Aug/16 02:22
Worklog Time Spent: 10m
Work Description: Github user bryancall commented on a diff in the pull request:
https://github.com/apache/trafficserver/pull/351#discussion_r74867804
--- Diff: plugins/request_buffer/request_buffer.cc ---
@@ -0,0 +1,119 @@
+/* request_buffer.cc - Plugin to enable request buffer for the given transaction.
+ */
+
+#include <ctype.h>
+#include <stdio.h>
+#include <string.h>
+#include <time.h>
+#include <string>
+
+#include "ts/ts.h"
+#include "ts/ink_defs.h"
+
+#define PLUGIN_NAME "request_buffer"
+
+static const int MIN_BYTE_PER_SEC = 1000;
+static int TXN_INDEX_ARG_TIME;
+struct TimeRecord {
+ timespec start_time;
+ TimeRecord() { clock_gettime(CLOCK_MONOTONIC, &start_time); }
+};
+bool
+is_post_request(TSHttpTxn txnp)
+{
+ const char *method;
+ int method_len;
+ TSMLoc req_loc;
+ TSMBuffer req_bufp;
+ if (TSHttpTxnClientReqGet(txnp, &req_bufp, &req_loc) == TS_ERROR) {
+ TSError("Error while retrieving client request header\n");
+ return false;
+ }
+ method = TSHttpHdrMethodGet(req_bufp, req_loc, &method_len);
+ if (static_cast<size_t>(method_len) != strlen(TS_HTTP_METHOD_POST) || strncasecmp(method, TS_HTTP_METHOD_POST, method_len) != 0) {
+ TSHandleMLocRelease(req_bufp, TS_NULL_MLOC, req_loc);
+ return false;
+ }
+ TSHandleMLocRelease(req_bufp, TS_NULL_MLOC, req_loc);
+ return true;
+}
+bool
+reached_min_speed(TSHttpTxn txnp, int body_len)
+{
+ TimeRecord *timeRecord = (TimeRecord *)TSHttpTxnArgGet(txnp, TXN_INDEX_ARG_TIME);
+ timespec now_time;
+ clock_gettime(CLOCK_MONOTONIC, &now_time);
+ double time_diff_in_sec =
+ (now_time.tv_sec - timeRecord->start_time.tv_sec) + 1e-9 * (now_time.tv_nsec - timeRecord->start_time.tv_nsec);
+ TSDebug("http", "time_diff_in_sec = %f, body_len = %d, date_rate = %f\n", time_diff_in_sec, body_len,
+ body_len / time_diff_in_sec);
+ return body_len / time_diff_in_sec >= MIN_BYTE_PER_SEC;
+}
+static int
+hook_handler(TSCont contp, TSEvent event, void *edata)
+{
+ TSHttpTxn txnp = (TSHttpTxn)(edata);
+ if (event == TS_EVENT_HTTP_READ_REQUEST_HDR && is_post_request(txnp)) {
+ // enable the request body buffering
+ TSHttpTxnConfigIntSet(txnp, TS_CONFIG_HTTP_REQUEST_BUFFER_ENABLED, 1);
+
+ // save the start time for calculating the data rate
+ TimeRecord *timeRecord = new TimeRecord();
+ TSHttpTxnArgSet(txnp, TXN_INDEX_ARG_TIME, static_cast<void *>(timeRecord));
+
+ TSHttpTxnHookAdd(txnp, TS_HTTP_REQUEST_BUFFER_READ_HOOK, TSContCreate(hook_handler, TSMutexCreate()));
+ TSHttpTxnHookAdd(txnp, TS_HTTP_REQUEST_BUFFER_READ_COMPLETE_HOOK, TSContCreate(hook_handler, TSMutexCreate()));
+ TSHttpTxnHookAdd(txnp, TS_HTTP_TXN_CLOSE_HOOK, TSContCreate(hook_handler, TSMutexCreate()));
+ } else if (event == TS_EVENT_HTTP_REQUEST_BUFFER_READ || event == TS_EVENT_HTTP_REQUEST_BUFFER_COMPLETE) {
+ int64_t ret_len = TSHttpTxnClientReqBodyBytesGet(txnp);
+ if (event == TS_EVENT_HTTP_REQUEST_BUFFER_READ && !reached_min_speed(txnp, ret_len)) {
+ TSError("[hook_handler] Error : reached_min_speed checking failed\n");
+ TSHttpTxnReenable(txnp, TS_EVENT_ERROR);
+ return 0;
+ }
+
+ // get the received request body
+ TSIOBufferReader buffer_reader = TSHttpTxnGetClientRequestBufferReader(txnp);
+ int64_t read_avail = TSIOBufferReaderAvail(buffer_reader);
+ if (read_avail) {
+ char *body = (char *)TSmalloc(sizeof(char) * read_avail);
+ int64_t consumed = 0;
+ int64_t data_len = 0;
+ const char *char_data = NULL;
+ TSIOBufferBlock block = TSIOBufferReaderStart(buffer_reader);
+ while (block != NULL) {
+ char_data = TSIOBufferBlockReadStart(block, buffer_reader, &data_len);
+ memcpy(body + consumed, char_data, data_len);
+ consumed += data_len;
+ block = TSIOBufferBlockNext(block);
+ }
+ // play with the body
--- End diff --
Why are you copying the body if you aren't doing anything with it?
Issue Time Tracking
-------------------
Worklog Id: (was: 26482)
Time Spent: 10m
Remaining Estimate: 0h
> Add feature to buffer request body before making downstream requests
> --------------------------------------------------------------------
>
> Key: TS-4042
> URL: https://issues.apache.org/jira/browse/TS-4042
> Project: Traffic Server
> Issue Type: Improvement
> Components: Core, CPP API, TS API
> Reporter: Brian Geffon
> Assignee: Brian Geffon
> Fix For: 7.0.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> We need a way to examine the request body without making a downstream request, this feature has many use cases including:
> - Ability to buffer the body and ensure a full post is received before committing downstream resources.
> - Ability to choose an origin based on request body
> - Ability to do request content filtering such as a WAF might provide before the origin is involved.
> Today you have two options to inspect a request body:
> 1) Transformations: the problem with transformations is that you only start receiving the request bytes after a sink has been established, which in this case is the downstream origin.
> 2) Create an intercept and use fetch apis to then send the downstream request: while this technically works it turns out to be a ton of code and is in general pretty problematic, we actually tried this approach for a while and had nothing but problems with it.
> We feel it would be ideal if we could intercept the body without breaking the normal ATS state flow. There used to exist code (and it's still in the core just #ifdefed out) to drain the request body. I use that code as the basis for this request buffering code. We added APIs to both the C and C++ APIs so that this request buffering can be enabled from a plugin and the plugin can inspect the body as chunks arrive or when it's complete. We've included an example plugin that will error a transaction if a minimum rate of transfer is not maintained.
> I'm confident that this feature will bring plenty of questions / feedback, so let's get that party started.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)