You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@trafficserver.apache.org by "Alan M. Carroll" <am...@network-geographics.com> on 2014/05/23 22:39:00 UTC
API proposal
TSHttpConnectWithPluginID
=========================
Allows the plugin to initiate an http connection. This will tag the HTTP state machine with extra data that can be accessed by the logging interface.
Synopsis
--------
`#include <ts/ts.h>`
.. c:function:: TSVConn TSHttpConnectWithPluginId(sockaddr const *addr, char const* tag, int64_t id)
Description
-----------
This call attempts to create an HTTP state machine and a virtual connection to that state machine. This is more efficient than using :c:func:`TSNetConnect` because it avoids using the operating system stack via the loopback interface.
*addr*
This is the network address of the target of the connection. This includes the port which should be stored in the :c:type:`sockaddr` structure.
*tag*
This is a tag that is passed through to the HTTP state machine. It must be a persistent string that has a lifetime longer than the connection. It is accessible via the log field ``pitag``. This is intended as a class or type identifier that is consistent across all connections for this plugin. In effect, the name of the plugin. This can be ``NULL``.
*id*
This is a numeric identifier that is passed through to the HTTP state machine. It is accessible via the log field ``piid``. This is intended as a connection identifier and should be distinct for every call to ``TSHttpConnectWithPluginID``. The easist mechanism is to define a plugin global value and increment it for each connection. The value ``0`` is reserved to mean "not set" and can be used as a default if this functionality is not needed.
The virtual connection returned as the ``TSCVonn`` is API equivalent to a network virtual connection both to the plugin and to internal mechanisms. Data is read and written to the connection (and thence to the target system) by reading and writing on this virtual connection.
The combination of tag and id is intended to enable correlation in log post processing. The tag identifies the connection as related to the plugin and the id can be used in conjuction with plugin generated logs to correlate the log records.
This is also intended to be useful for debugging by tagging not just the HTTP state machine but also the connection virtual connections (if of type ``PluginVC``) with correlated data.
.. topic:: Example
The SPDY plugin uses this to correlate client sessions with SPDY streams. Each client connection is assigned a distinct numeric identifier. This is passed as the *id* to ``TSHttpConnectWithPluginId``. The name is selected to be the NPN string for the client session protocol, e.g. "spdy/3" or "spdy/3.1". Log post processing can then count the number of connections for the various supported protocols and the number of streams / sessions / transactions for each real client connection to Traffic Server.
Re: API proposal
Posted by James Peach <jp...@apache.org>.
On May 27, 2014, at 6:08 AM, Alan M. Carroll <am...@network-geographics.com> wrote:
> James,
>
> Monday, May 26, 2014, 9:19:27 PM, you wrote:
>
>> Hi Alan,
>
>> So AFAICT this is not a true superset of the TSClientProtoStack API. FWIW, I also think that removing that API should have been reviewed.
>
> This proposal is not intended as a replacement for TSClientProtoStack, but to solve a different problem (essentially logging for SPDY, although done in a way that provides utility for other plugins) that wasn't solved by TSClientProtoStack.
>
> For TSClientProtoStack, the problem was that given other structural changes to the SPDY implementation, the API was broken and couldn't be fixed. In particular, changing the way it was handled by the NetVCs removed the data chain needed to provide the information.
So how do you log TCP+SPDY vs. TCP+TLS+SPDY?
J
Re: API proposal
Posted by "Alan M. Carroll" <am...@network-geographics.com>.
James,
Monday, May 26, 2014, 9:19:27 PM, you wrote:
> Hi Alan,
> So AFAICT this is not a true superset of the TSClientProtoStack API. FWIW, I also think that removing that API should have been reviewed.
This proposal is not intended as a replacement for TSClientProtoStack, but to solve a different problem (essentially logging for SPDY, although done in a way that provides utility for other plugins) that wasn't solved by TSClientProtoStack.
For TSClientProtoStack, the problem was that given other structural changes to the SPDY implementation, the API was broken and couldn't be fixed. In particular, changing the way it was handled by the NetVCs removed the data chain needed to provide the information.
Re: API proposal
Posted by James Peach <jp...@apache.org>.
Hi Alan,
So AFAICT this is not a true superset of the TSClientProtoStack API. FWIW, I also think that removing that API should have been reviewed.
TSClientProtoStack would distinguish between requests triggered by SPDY over TLS and raw SPDY, as well as HTTP vs. HTTPS. I don't know whether that capability is still desired, but it's worth pointing the difference. Unless this API chains the new VC up to a parent VC and logs the full chain? I know you proposed doing that ...
On May 23, 2014, at 1:39 PM, Alan M. Carroll <am...@network-geographics.com> wrote:
> TSHttpConnectWithPluginID
> =========================
>
> Allows the plugin to initiate an http connection. This will tag the HTTP state machine with extra data that can be accessed by the logging interface.
>
>
> Synopsis
> --------
>
> `#include <ts/ts.h>`
>
> .. c:function:: TSVConn TSHttpConnectWithPluginId(sockaddr const *addr, char const* tag, int64_t id)
>
>
> Description
> -----------
>
> This call attempts to create an HTTP state machine and a virtual connection to that state machine. This is more efficient than using :c:func:`TSNetConnect` because it avoids using the operating system stack via the loopback interface.
>
> *addr*
> This is the network address of the target of the connection. This includes the port which should be stored in the :c:type:`sockaddr` structure.
>
> *tag*
> This is a tag that is passed through to the HTTP state machine. It must be a persistent string that has a lifetime longer than the connection. It is accessible via the log field ``pitag``. This is intended as a class or type identifier that is consistent across all connections for this plugin. In effect, the name of the plugin. This can be ``NULL``.
>
> *id*
> This is a numeric identifier that is passed through to the HTTP state machine. It is accessible via the log field ``piid``. This is intended as a connection identifier and should be distinct for every call to ``TSHttpConnectWithPluginID``. The easist mechanism is to define a plugin global value and increment it for each connection. The value ``0`` is reserved to mean "not set" and can be used as a default if this functionality is not needed.
If you increment a global counter, how would you correlate logs across a Traffic Server restart? You'll end up with duplicate transaction IDs; so how is a plugin supposed to handle that?
> The virtual connection returned as the ``TSCVonn`` is API equivalent to a network virtual connection both to the plugin and to internal mechanisms. Data is read and written to the connection (and thence to the target system) by reading and writing on this virtual connection.
>
> The combination of tag and id is intended to enable correlation in log post processing. The tag identifies the connection as related to the plugin and the id can be used in conjuction with plugin generated logs to correlate the log records.
How would this work in practice? It's common for a plugin to hold a request up while it makes sub-requets. How would this let you correlate the sub-requests with the original client-originated request?
> This is also intended to be useful for debugging by tagging not just the HTTP state machine but also the connection virtual connections (if of type ``PluginVC``) with correlated data.
>
> .. topic:: Example
>
> The SPDY plugin uses this to correlate client sessions with SPDY streams. Each client connection is assigned a distinct numeric identifier. This is passed as the *id* to ``TSHttpConnectWithPluginId``. The name is selected to be the NPN string for the client session protocol, e.g. "spdy/3" or "spdy/3.1". Log post processing can then count the number of connections for the various supported protocols and the number of streams / sessions / transactions for each real client connection to Traffic Server.
>
Re: API proposal
Posted by "Alan M. Carroll" <am...@network-geographics.com>.
William,
I've thought about that. I think this is at the limits of what I this is reasonable for argument lists. If we want more (and we would want at least transparency) I would rather go with an options structure which can contain a much larger number of options. This is a bit ugly in C, but I think it could be livable.
OutbouncConnectOptions opt;
TSOutboundConnectOptionsInit(&opt);
opt.id = 17;
opt.tag = "my-super-plugin";
opt.transparent = true; /* or 1? */
opt.something = some-other-value;
struct sockaddr addr; /* initialize somehow */
opt.target = &addr;
TSVConn vc = TSHttpConnectWithOpptions(opt);
The key thing is that TSOutboundConnectionOptionsInit() would set everything to default values that could be ignored so you only have to set the options that you care about.
Friday, May 23, 2014, 8:46:42 PM, you wrote:
> This sounds very useful. (We did something hacky with the address to get similar functionality just inside a plugin, this would be much cleaner.)
> I wonder if it should take some extra optional arguments to allow doing transparent stuff too. (So it would be a complete super-set.)
RE: API proposal
Posted by "Bardwell, William" <wb...@akamai.com>.
This sounds very useful. (We did something hacky with the address to get similar functionality just inside a plugin, this would be much cleaner.)
I wonder if it should take some extra optional arguments to allow doing transparent stuff too. (So it would be a complete super-set.)
-William Bardwell
________________________________________
From: Alan M. Carroll [amc@network-geographics.com]
Sent: Friday, May 23, 2014 4:39 PM
To: dev@trafficserver.apache.org
Subject: API proposal
TSHttpConnectWithPluginID
=========================
Allows the plugin to initiate an http connection. This will tag the HTTP state machine with extra data that can be accessed by the logging interface.
Synopsis
--------
`#include <ts/ts.h>`
.. c:function:: TSVConn TSHttpConnectWithPluginId(sockaddr const *addr, char const* tag, int64_t id)
Description
-----------
This call attempts to create an HTTP state machine and a virtual connection to that state machine. This is more efficient than using :c:func:`TSNetConnect` because it avoids using the operating system stack via the loopback interface.
*addr*
This is the network address of the target of the connection. This includes the port which should be stored in the :c:type:`sockaddr` structure.
*tag*
This is a tag that is passed through to the HTTP state machine. It must be a persistent string that has a lifetime longer than the connection. It is accessible via the log field ``pitag``. This is intended as a class or type identifier that is consistent across all connections for this plugin. In effect, the name of the plugin. This can be ``NULL``.
*id*
This is a numeric identifier that is passed through to the HTTP state machine. It is accessible via the log field ``piid``. This is intended as a connection identifier and should be distinct for every call to ``TSHttpConnectWithPluginID``. The easist mechanism is to define a plugin global value and increment it for each connection. The value ``0`` is reserved to mean "not set" and can be used as a default if this functionality is not needed.
The virtual connection returned as the ``TSCVonn`` is API equivalent to a network virtual connection both to the plugin and to internal mechanisms. Data is read and written to the connection (and thence to the target system) by reading and writing on this virtual connection.
The combination of tag and id is intended to enable correlation in log post processing. The tag identifies the connection as related to the plugin and the id can be used in conjuction with plugin generated logs to correlate the log records.
This is also intended to be useful for debugging by tagging not just the HTTP state machine but also the connection virtual connections (if of type ``PluginVC``) with correlated data.
.. topic:: Example
The SPDY plugin uses this to correlate client sessions with SPDY streams. Each client connection is assigned a distinct numeric identifier. This is passed as the *id* to ``TSHttpConnectWithPluginId``. The name is selected to be the NPN string for the client session protocol, e.g. "spdy/3" or "spdy/3.1". Log post processing can then count the number of connections for the various supported protocols and the number of streams / sessions / transactions for each real client connection to Traffic Server.