You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@trafficserver.apache.org by js...@apache.org on 2016/02/17 00:01:27 UTC
trafficserver git commit: TS-2695: docs: howto on basics with
logstash and circonus
Repository: trafficserver
Updated Branches:
refs/heads/master a17db36fb -> 0aabfb0bf
TS-2695: docs: howto on basics with logstash and circonus
Project: http://git-wip-us.apache.org/repos/asf/trafficserver/repo
Commit: http://git-wip-us.apache.org/repos/asf/trafficserver/commit/0aabfb0b
Tree: http://git-wip-us.apache.org/repos/asf/trafficserver/tree/0aabfb0b
Diff: http://git-wip-us.apache.org/repos/asf/trafficserver/diff/0aabfb0b
Branch: refs/heads/master
Commit: 0aabfb0bf7c63f8a2dd062b703cecb3c7f6eced8
Parents: a17db36
Author: Jon Sime <js...@apache.org>
Authored: Tue Feb 16 22:59:36 2016 +0000
Committer: Jon Sime <js...@apache.org>
Committed: Tue Feb 16 23:00:59 2016 +0000
----------------------------------------------------------------------
.../monitoring/third-party/circonus.en.rst | 125 ++++++++++++
.../monitoring/third-party/index.en.rst | 15 +-
.../monitoring/third-party/logstash.en.rst | 198 +++++++++++++++++++
.../admin/monitor/circonus/check-config-1.png | Bin 0 -> 160451 bytes
.../admin/monitor/circonus/check-config-2.png | Bin 0 -> 49277 bytes
.../admin/monitor/circonus/check-config-3.png | Bin 0 -> 54633 bytes
.../admin/monitor/circonus/metric-grid.png | Bin 0 -> 83094 bytes
.../admin/monitor/circonus/new-check-button.png | Bin 0 -> 10076 bytes
8 files changed, 335 insertions(+), 3 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/trafficserver/blob/0aabfb0b/doc/admin-guide/monitoring/monitoring/third-party/circonus.en.rst
----------------------------------------------------------------------
diff --git a/doc/admin-guide/monitoring/monitoring/third-party/circonus.en.rst b/doc/admin-guide/monitoring/monitoring/third-party/circonus.en.rst
new file mode 100644
index 0000000..27ae6d2
--- /dev/null
+++ b/doc/admin-guide/monitoring/monitoring/third-party/circonus.en.rst
@@ -0,0 +1,125 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+.. include:: ../../../../common.defs
+
+.. _admin-monitoring-circonus:
+
+Circonus
+********
+
+`Circonus <http://www.circonus.com/>`_ is a commercial systems monitoring and
+analytics product, available in hosted (SaaS) and on-site configurations.
+
+While it does not currently offer a module specific to |ATS| statistics, it is
+very easy to add monitoring by combining the |TS| plugin :ref:`Stats Over HTTP
+<admin-plugins-stats-over-http>` with Circonus's JSON metrics collector.
+
+Setup
+=====
+
+The following steps assume that you already have an active Circonus account on
+their hosted service, or that you already have an on-site installation. If you
+do not, that must be set up first. Trial accounts are available and will
+suffice for this guide.
+
+Enable Stats Over HTTP
+----------------------
+
+Follow the directions in the :ref:`admin-plugins-stats-over-http` chapter to
+configure the plugin. At a bare minimum, you will need to add the plugin to
+:file:`plugin.config`. For this guide, we will assume the following entry::
+
+ stats_over_http.so _statistics
+
+This enables the plugin and makes |TS| statistics available at the path
+``/_statistics``. For production environments, you are strongly encouraged to
+use a different (and obfuscated) path for your statistics since they reveal
+many internal details of your caching proxy. The plugin documentation provides
+more details.
+
+Create Circonus Check
+---------------------
+
+For these steps, we will assume that |TS| is listening on port ``80`` at the
+domain ``trafficserver.example.com``. You should adjust these to match your
+environment.
+
+#. Begin the new check creation process from within your Circonus account by
+ clicking the *New Check* button near the top-right of the checks screen.
+
+ .. image:: ../../../../static/images/admin/monitor/circonus/new-check-button.png
+ :alt: Circonus New Check button
+ :align: center
+
+#. For the check type, you should select *JSON* under the *Custom* list, and
+ then choose the *Pull* type. The broker you choose to use is entirely up to
+ you and will depend largely on whether you are using on-site Circonus or the
+ hosted service, as well as the geographic location of your |TS| instance(s).
+
+ .. image:: ../../../../static/images/admin/monitor/circonus/check-config-1.png
+ :alt: Choosing a check type
+ :align: center
+
+#. Click on *Configure Check* to proceed.
+
+#. Click on the *Show Advanced Configuration* link to expand the check options.
+ Enter your |TS| instance's externally-accessible domain for *Host* and make
+ sure to set *URI* to the path at which the statistics are available. Adjust
+ any of the other options to match your environment if necessary (for this
+ guide, only *Host* and *URI* will need to be entered).
+
+ .. image:: ../../../../static/images/admin/monitor/circonus/check-config-2.png
+ :alt: Advanced check configuration
+ :align: center
+
+#. Click on *Test Check* to proceed.
+
+#. If all is well, you should see a long list (several hundred entries) of |TS|
+ statistics, with their associated values. If you do not, verify that |TS| is
+ running and use ``curl`` to manually fetch the statistics data from your
+ server.
+
+ .. image:: ../../../../static/images/admin/monitor/circonus/check-config-3.png
+ :alt: Check test
+ :align: center
+
+#. You may want to limit the metrics you actually track from |TS|, since so
+ many are made available. If so, simply uncheck those you aren't interested
+ in from the list. If you only want to track a few metrics, click
+ *Select None* and then check the ones you want. You can filter the results
+ to easily find metrics in the list.
+
+#. Save your new check.
+
+#. Verify the collected metrics in Circonus by opening your new check and
+ clicking on *Metrics Grid* for an overview visualization of all the data
+ being collected.
+
+ .. image:: ../../../../static/images/admin/monitor/circonus/metric-grid.png
+ :alt: Circonus metric grid
+ :align: center
+
+Congratulations! You're now ready to begin setting up alerts, visualizations,
+and dashboards for your |TS| statistics. You can repeat the above steps for any
+additional |TS| instances you have running.
+
+Further Reading
+===============
+
+* `Official Circonus Website <http://www.circonus.com/>`_
+
http://git-wip-us.apache.org/repos/asf/trafficserver/blob/0aabfb0b/doc/admin-guide/monitoring/monitoring/third-party/index.en.rst
----------------------------------------------------------------------
diff --git a/doc/admin-guide/monitoring/monitoring/third-party/index.en.rst b/doc/admin-guide/monitoring/monitoring/third-party/index.en.rst
index fb52da9..2e947d6 100644
--- a/doc/admin-guide/monitoring/monitoring/third-party/index.en.rst
+++ b/doc/admin-guide/monitoring/monitoring/third-party/index.en.rst
@@ -22,11 +22,20 @@
Integrating Third-Party Tools
*****************************
+There are many monitoring and alerting systems available, too many for us to
+hope to cover every possible option here. We can, however, attempt to document
+using some of the more common options.
+
+Some of these third party monitoring applications and services are able to tap
+into the extensive list of |TS| statistics, others make use of |TS| log files,
+and yet others are aimed at simple health check reporting. An extensive service
+monitoring configuration may take advantage of more than one service to play to
+their complementary strengths. Which tool, or combination of tools, is right
+for your infrastructure will likely vary.
+
.. toctree::
- :maxdepth: 2
+ :maxdepth: 1
circonus.en
logstash.en
- nagios.en
- zabbix.en
http://git-wip-us.apache.org/repos/asf/trafficserver/blob/0aabfb0b/doc/admin-guide/monitoring/monitoring/third-party/logstash.en.rst
----------------------------------------------------------------------
diff --git a/doc/admin-guide/monitoring/monitoring/third-party/logstash.en.rst b/doc/admin-guide/monitoring/monitoring/third-party/logstash.en.rst
new file mode 100644
index 0000000..9ae39f2
--- /dev/null
+++ b/doc/admin-guide/monitoring/monitoring/third-party/logstash.en.rst
@@ -0,0 +1,198 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+.. include:: ../../../../common.defs
+
+.. _admin-monitoring-logstash:
+
+Logstash
+********
+
+`Logstash <https://www.elastic.co/products/logstash>`_ is a powerful, open
+source, unstructured data processing program that can accept text data from
+many different sources (directly over TCP/UDP, via Unix sockets, or by reading
+in files from disk for example), in many different formats and transform those
+inputs into structured, searchable documents.
+
+One of the most common use cases is to take text based system and application
+logs, extract individual fields (e.g. host names, error codes, timing data, and
+so on), and make the data available in Elasticsearch for searching and
+reporting.
+
+In this guide, we will cover the basics of getting your |TS| log data into
+Logstash. Going the next step and building fancy Kibana dashboards on top of
+that is currently left as an exercise for the reader.
+
+|TS| Log Formats
+================
+
+|TS| provides a very flexible set of logging outputs. Almost any format can be
+constructed. The full range of options is covered in the
+:ref:`admin-monitoring-logging` chapter.
+
+This guide will walk you through using the appropriate filters in Logstash for
+the common logging formats in |TS|. If you have constructed your own custom log
+formats, you will need to build upon these examples and refer to the Logstash
+documentation to produce custom filters capable of parsing your own formats.
+
+Logstash Input
+==============
+
+For the on-disk logs produced by |TS|, you will want to use Logstash's ``file``
+input plugin. Note that your logs must be in ``ASCII`` format, not ``binary``,
+for the plugin to work.
+
+Assuming that your |TS| event logs are named ``access-<rotationtimestamp>.log``
+and stored at ``/var/log/trafficserver/``, the following Logstash input
+configuration should work::
+
+ input {
+ file {
+ path => /var/log/trafficserver/access-*.log
+ }
+ }
+
+Logstash provides some additional tweaking options, which are explained in the
+`file plugin documentation <https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html>`_
+but the above provides the bare minimum required to have Logstash read log data
+from local disks.
+
+Logstash Filters
+================
+
+The `grok filter <https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html>`_
+in Logstash allows you to completely tailor the parsing of your source data and
+extract as many or as few fields as you like.
+
+Some patterns are already built and can be used very easily. If you have built
+custom log formats for |TS|, you may need to write your own patterns, however.
+
+Squid Compatible
+----------------
+
+The :ref:`admin-logging-format-squid` log format includes, unsurprisingly, a
+few useful fields for proxy servers. Using the following grok pattern will
+extract this information from your |TS| logs if you employ the Squid compatible
+log format::
+
+ filter {
+ grok {
+ match => { "message" => "%{NUMBER:timestamp} %{NUMBER:timetoserve} %{IPORHOST:clientip} %{WORD:cachecode}/%{NUMBER:response} %{NUMBER:bytes} %{WORD:verb} %{NOTSPACE:request} %{USER:auth} %{NOTSPACE:route} %{DATA:contenttype}" }
+ }
+ date {
+ match => [ "timestamp", "UNIX" ]
+ }
+ }
+
+The resulting structured document will contain the following fields:
+
+=========== ===================================================================
+Field Description
+=========== ===================================================================
+timestamp Date and time of the client request.
+timetoserve Time, in seconds, from initial client connection to |TS| until the
+ last byte has been sent back to client from |TS|.
+clientip Client IP address or hostname.
+cachecode :ref:`admin-monitoring-logging-cache-result-codes`.
+response HTTP response status code sent by |TS| to the client.
+bytes Length, in bytes, of the |TS| response to the client, including
+ headers.
+verb HTTP method (e.g. ``GET``, ``POST``, etc.) of the client request.
+request URL specified by the client request.
+auth Authentication username supplied by the client, if present.
+route Proxy hierarchy route; the route used by |TS| to retrieve the cache
+ object.
+contenttype Content type of the response.
+=========== ===================================================================
+
+Netscape Common
+---------------
+
+If your |TS| instance is already outputting :ref:`admin-logging-format-common`
+format logs, then Logstash's ``COMMONAPACHELOG`` pattern will handle your logs
+out of the box. Add the following filter block to your Logstash configuration::
+
+ filter {
+ grok {
+ match => { "message" => "%{COMMONAPACHELOG}" }
+ }
+ }
+
+This will produce a structured document for each log entry with the following
+fields:
+
+=========== ===================================================================
+Field Description
+=========== ===================================================================
+clientip Client IP address or hostname.
+ident Always a literal ``-`` character for |TS| logs.
+auth The authentication username for the client request. A ``-`` means
+ no authentication was required (or supplied).
+timestamp The date and time of the client request.
+verb HTTP method used for the request (e.g. ``GET``, ``POST``, etc.).
+request URL specified by the client request.
+httpversion HTTP version (e.g. ``1.1``) used by the client.
+rawrequest *See note below.*
+response HTTP status code used for |TS| response (not the origin's response
+ code).
+bytes Length of |TS| response to client, in bytes.
+=========== ===================================================================
+
+.. note::
+
+ ``rawrequest`` is populated when the usual ``"<verb> <request> http/<httpversion>"``
+ pattern was not matched. In that event, those three fields will be missing
+ from the document, and instead ``rawrequest`` will have the original string.
+
+Netscape Extended
+-----------------
+
+The following pattern adds to Common Apache to support the additional fields
+found in :ref:`admin-logging-format-extended`::
+
+ filter {
+ grok {
+ match => { "message" => "%{COMMONAPACHELOG} %{NUMBER:originstatus} %{NUMBER:originrespbytes} %{NUMBER:clientreqbytes} %{NUMBER:proxyreqbytes} %{NUMBER:clienthdrbytes} %{NUMBER:proxyresphdrbytes} %{NUMBER:proxyreqhdrbytes} %{NUMBER:originhdrbytes} %{NUMBER:timetoserve}" }
+ }
+ }
+
+Because this starts out with the ``COMMONAPACHELOG`` pattern, you will get all
+of the fields mentioned in `Netscape Common`_ above, as well as the following:
+
+================= =============================================================
+Field Description
+================= =============================================================
+originstatus HTTP status code returned by origin server.
+originrespbytes Body length, in bytes, of origin's response to |TS|.
+clientreqbytes Body length, in bytes, of client request to |TS|.
+proxyreqbytes Body length, in bytes, of |TS| request to origin.
+clienthdrbytes Header length, in bytes, of client request to |TS|.
+proxyresphdrbytes Header length, in bytes, of |TS| response to client.
+proxyreqhdrbytes Header length, in bytes, of |TS| request to origin.
+originhdrbytes Header length, in bytes, of origin's response to |TS|.
+timetoserve Time, in seconds, from initial client connection to |TS|
+ until the last byte has been sent back to client from |TS|.
+================= =============================================================
+
+Further Reading
+===============
+
+* `Logstash Documentation <https://www.elastic.co/guide/en/logstash/current/index.html>`_
+
+* `Grok Patterns <https://github.com/logstash-plugins/logstash-patterns-core/tree/master/patterns>`_
+
+* `Elasticsearch Documentation <https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html>`_
http://git-wip-us.apache.org/repos/asf/trafficserver/blob/0aabfb0b/doc/static/images/admin/monitor/circonus/check-config-1.png
----------------------------------------------------------------------
diff --git a/doc/static/images/admin/monitor/circonus/check-config-1.png b/doc/static/images/admin/monitor/circonus/check-config-1.png
new file mode 100644
index 0000000..be98f99
Binary files /dev/null and b/doc/static/images/admin/monitor/circonus/check-config-1.png differ
http://git-wip-us.apache.org/repos/asf/trafficserver/blob/0aabfb0b/doc/static/images/admin/monitor/circonus/check-config-2.png
----------------------------------------------------------------------
diff --git a/doc/static/images/admin/monitor/circonus/check-config-2.png b/doc/static/images/admin/monitor/circonus/check-config-2.png
new file mode 100644
index 0000000..afd7fe9
Binary files /dev/null and b/doc/static/images/admin/monitor/circonus/check-config-2.png differ
http://git-wip-us.apache.org/repos/asf/trafficserver/blob/0aabfb0b/doc/static/images/admin/monitor/circonus/check-config-3.png
----------------------------------------------------------------------
diff --git a/doc/static/images/admin/monitor/circonus/check-config-3.png b/doc/static/images/admin/monitor/circonus/check-config-3.png
new file mode 100644
index 0000000..dfd025a
Binary files /dev/null and b/doc/static/images/admin/monitor/circonus/check-config-3.png differ
http://git-wip-us.apache.org/repos/asf/trafficserver/blob/0aabfb0b/doc/static/images/admin/monitor/circonus/metric-grid.png
----------------------------------------------------------------------
diff --git a/doc/static/images/admin/monitor/circonus/metric-grid.png b/doc/static/images/admin/monitor/circonus/metric-grid.png
new file mode 100644
index 0000000..8cbb222
Binary files /dev/null and b/doc/static/images/admin/monitor/circonus/metric-grid.png differ
http://git-wip-us.apache.org/repos/asf/trafficserver/blob/0aabfb0b/doc/static/images/admin/monitor/circonus/new-check-button.png
----------------------------------------------------------------------
diff --git a/doc/static/images/admin/monitor/circonus/new-check-button.png b/doc/static/images/admin/monitor/circonus/new-check-button.png
new file mode 100644
index 0000000..a7107d2
Binary files /dev/null and b/doc/static/images/admin/monitor/circonus/new-check-button.png differ