[jira] [Updated] (TS-4042) Add feature to buffer request body before making downstream requests

["Brian Geffon (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/TS-4042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Brian Geffon updated TS-4042:
-----------------------------
    Description: 
We need a way to examine the request body without making a downstream request, this feature has many use cases including:
  - Ability to buffer the body and ensure a full post is received before committing downstream resources.
  - Ability to choose an origin based on request body
  - Ability to do request content filtering such as a WAF might provide before the origin is involved.

Today you have two options to inspect a request body:
  1) Transformations: the problem with transformations is that you only start receiving the request bytes after a sink has been established, which in this case is the downstream origin.
  2) Create an intercept and use fetch apis to then send the downstream request: while this technically works it turns out to be a ton of code and is in general pretty problematic, we actually tried this approach for a while and had nothing but problems with it.

We feel it would be ideal if we could intercept the body without breaking the normal ATS state flow. There used to exist code (and it's still in the core just #ifdefed out) to drain the request body. I use that code as the basis for this request buffering code. We added APIs to both the C and C++ APIs so that this request buffering can be enabled from a plugin and the plugin can inspect the body as chunks arrive or when it's complete. We've included an example plugin that will error a transaction if a minimum rate of transfer is not maintained.

I'm confident that this feature will bring plenty of questions / feedback, so let's get that party started.

> Add feature to buffer request body before making downstream requests
> --------------------------------------------------------------------
>
>                 Key: TS-4042
>                 URL: https://issues.apache.org/jira/browse/TS-4042
>             Project: Traffic Server
>          Issue Type: Improvement
>          Components: Core, CPP API, TS API
>            Reporter: Brian Geffon
>            Assignee: Brian Geffon
>
> We need a way to examine the request body without making a downstream request, this feature has many use cases including:
>   - Ability to buffer the body and ensure a full post is received before committing downstream resources.
>   - Ability to choose an origin based on request body
>   - Ability to do request content filtering such as a WAF might provide before the origin is involved.
> Today you have two options to inspect a request body:
>   1) Transformations: the problem with transformations is that you only start receiving the request bytes after a sink has been established, which in this case is the downstream origin.
>   2) Create an intercept and use fetch apis to then send the downstream request: while this technically works it turns out to be a ton of code and is in general pretty problematic, we actually tried this approach for a while and had nothing but problems with it.
> We feel it would be ideal if we could intercept the body without breaking the normal ATS state flow. There used to exist code (and it's still in the core just #ifdefed out) to drain the request body. I use that code as the basis for this request buffering code. We added APIs to both the C and C++ APIs so that this request buffering can be enabled from a plugin and the plugin can inspect the body as chunks arrive or when it's complete. We've included an example plugin that will error a transaction if a minimum rate of transfer is not maintained.
> I'm confident that this feature will bring plenty of questions / feedback, so let's get that party started.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)