[trafficserver] branch 9.0.x updated: Traffic Dump documentation for post_process.py (#7161)

[zw...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

zwoop pushed a commit to branch 9.0.x
in repository https://gitbox.apache.org/repos/asf/trafficserver.git


The following commit(s) were added to refs/heads/9.0.x by this push:
     new 83bf7ce  Traffic Dump documentation for post_process.py (#7161)
83bf7ce is described below

commit 83bf7ce55a25bf168d8e62144d9a38b7d72b3366
Author: Brian Neradt <br...@gmail.com>
AuthorDate: Thu Sep 3 14:32:55 2020 -0500

    Traffic Dump documentation for post_process.py (#7161)
    
    This makes some tweaks to the Traffic Dump documentation wording and
    adds information about post_process.py
    
    (cherry picked from commit 22e25a2a93ad9e7968b5223f0679edce5c7e8e4b)
---
 doc/admin-guide/plugins/traffic_dump.en.rst | 46 +++++++++++++++++++----------
 1 file changed, 30 insertions(+), 16 deletions(-)

diff --git a/doc/admin-guide/plugins/traffic_dump.en.rst b/doc/admin-guide/plugins/traffic_dump.en.rst
index acf6717..9b21343 100644
--- a/doc/admin-guide/plugins/traffic_dump.en.rst
+++ b/doc/admin-guide/plugins/traffic_dump.en.rst
@@ -27,46 +27,60 @@ Traffic Dump Plugin
 Description
 ===========
 
-``Traffic Dump`` captures session traffic and writes to a replay file :ts:git:`tests/tools/traffic-replay/replay_schema.json` for each captured session. It then can be used to replay traffic for testing purpose.
+``Traffic Dump`` captures traffic for a set of sampled sessions and records this traffic to replay files according to the :ts:git:`tests/tools/traffic-replay/replay_schema.json` schema. These replay files can be used to replay traffic for testing purposes. Further, since the traffic is written decrypted, they can be used to conveniently analyze HTTP traffic going through ATS.
 
 Plugin Configuration
 ====================
 .. program:: traffic_dump.so
 
-``Traffic Dump`` is a global plugin and is configured via :file:`plugin.config`.
+``traffic_dump.so``
+  ``Traffic Dump`` is a global plugin and is configured via :file:`plugin.config`.
+
    .. option:: --logdir <path_to_dump>
 
-   (`required`) - specifies the directory for writing all dump files. If path is relative, it is relative to the Traffic Server directory. The plugin will use first three characters of the client's IP to create subdirs in an attempt to spread dumps evenly and avoid too many files in a single directory.
+   (`required`) - Specifies the directory for writing all dump files. If the path is relative it is considered relative to the Traffic Server directory. The plugin will use the first three characters of the client's IP to create subdirs in an attempt to spread dumps evenly and avoid too many files in a single directory.
 
    .. option:: --sample <N>
 
-   (`required`) - specifies the sampling ratio N. Traffic Dump will capture every one out of N sessions. This ratio can also be changed via traffic_ctl without restarting ATS.
+   (`required`) - Specifies the sampling ratio N. Traffic Dump will capture every one out of N sessions. This ratio can also be changed via ``traffic_ctl`` without restarting ATS.
 
    .. option:: --limit <N>
 
-   (`required`) - specifies the max disk usage N bytes (approximate). Traffic Dump will stop capturing new sessions once disk usage exceeds this limit.
+   (`required`) - Specifies the max disk usage to N bytes (approximate). Traffic Dump will stop capturing new sessions once disk usage exceeds this limit.
 
    .. option:: --sensitive-fields <field1,field2,...,fieldn>
 
-   (`optional`) - a comma separated list of HTTP case-insensitive field names whose values are considered sensitive information. Traffic Dump will not dump the incoming field values for any of these fields but will instead dump a generic value for them of the same length as the original. If this option is not used, a default list of "Cookie,Set-Cookie" is used. Providing this option overwrites that default list with whatever values the user provides. Pass a quoted empty string as the arg [...]
+   (`optional`) - A comma separated list of HTTP case-insensitive field names whose values are considered sensitive information. Traffic Dump will not dump the incoming field values for any of these fields but will instead dump a generic value for them of the same length as the original. If this option is not used, a default list of "Cookie,Set-Cookie" is used. Providing this option overwrites that default list with whatever values the user provides. Pass a quoted empty string as the arg [...]
 
    .. option:: --sni-filter <SNI_Name>
 
-   (`optional`) - an SNI with which to filter sessions. Only HTTPS sessions with the provided SNI will be dumped. The sample option will apply a sampling rate to these filtered sessions. Thus, with a sample value of 2, 1/2 of all sessions with the specified SNI will be dumped.
+   (`optional`) - An SNI with which to filter sessions. Only HTTPS sessions with the provided SNI will be dumped. The sample option will apply a sampling rate to these filtered sessions. Thus, with a sample value of 2, 1/2 of all sessions with the specified SNI will be dumped.
 
 ``traffic_ctl`` <command>
+  ``Traffic Dump`` can be dynamically configured via ``traffic_ctl``.
+
    * ``traffic_ctl plugin msg traffic_dump.sample N`` - changes the sampling ratio N as mentioned above.
    * ``traffic_ctl plugin msg traffic_dump.reset`` - resets the disk usage counter.
    * ``traffic_ctl plugin msg traffic_dump.limit N`` - changes the max disk usage to N bytes as mentioned above.
 
 Replay Format
 =============
-This format contains traffic data including:
-
-* Each session and transactions in the session.
-* Timestamps.
-* The four sets of headers (user agent request, proxy request, origin server response, proxy response).
-* The protocol stack for the user agent.
-* The transaction count for the outbound session.
-* The content block sizes.
-* See schema here: :ts:git:`tests/tools/lib/replay_schema.json`
+This replay files contain the following information:
+
+* Each sampled session and all the transactions for those sessions.
+* Timestamps of each transaction.
+* The four sets of HTTP message headers (user agent request, proxy request, origin server response, proxy response).
+* The protocol stacks for the user agent and the origin server connections.
+* The number of body bytes for each message.
+
+For details, see the schema: :ts:git:`tests/tools/lib/replay_schema.json`
+
+Post Processing
+===============
+Traffic Dump comes with a post processing script located at :ts:git:`plugins/experimental/traffic_dump/post_process.py`. This filters out incomplete sessions and transactions and merges a specifiable amount of sessions from a client into single replay files. It also optionally formats the single-line JSON files into a human readable format. It takes the following arguments:
+
+* The first positional argument is the input directory containing the replay files captured by traffic_dump as described above.
+* The second positional argument is the output directory containing the processed replay files.
+* ``-n`` is an optional argument specifying how many sessions will be attempted to be merged into single replay files. The default is 10.
+
+There are other options. Use ``--help`` for a description of these.