You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@accumulo.apache.org by GitBox <gi...@apache.org> on 2018/03/24 16:47:57 UTC
[GitHub] mikewalch closed pull request #62: Added metrics setup instructions for Grafana
mikewalch closed pull request #62: Added metrics setup instructions for Grafana
URL: https://github.com/apache/accumulo-website/pull/62
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/_docs-2-0/administration/monitoring-metrics.md b/_docs-2-0/administration/monitoring-metrics.md
index 48e3a9d1..d97ec0e7 100644
--- a/_docs-2-0/administration/monitoring-metrics.md
+++ b/_docs-2-0/administration/monitoring-metrics.md
@@ -95,8 +95,13 @@ from growing without bound.
Each server process should have log messages from the Metrics2 library about the sinks that were created. Be sure to check
the Accumulo processes log files when debugging missing metrics output.
-For additional information on configuring Metrics2, visit the [Javadoc page for Metrics2](https://hadoop.apache.org/docs/current/api/org/apache/hadoop/metrics2/package-summary.html).
+Below are additional resources for configuring Metrics2:
+* [Javadoc page for Metrics2][metrics2-javadoc]
+* [Blog post][grafana-post] about configuring Grafana/InfluxDB to view Accumulo metrics2
+
+[grafana-post]: {{ site.baseurl }}/blog/2018/03/22/view-metrics-in-grafana/
+[metrics2-javadoc]: https://hadoop.apache.org/docs/current/api/org/apache/hadoop/metrics2/package-summary.html
[tracing]: {{page.docs_baseurl}}/administration/tracing
[monitor.ssl.keyStore]: {{ page.docs_baseurl }}/administration/properties#monitor_ssl_keyStore
[monitor.ssl.keyStorePassword]: {{ page.docs_baseurl }}/administration/properties#monitor_ssl_keystorePassword
diff --git a/_posts/blog/2018-03-22-view-metrics-in-grafana.md b/_posts/blog/2018-03-22-view-metrics-in-grafana.md
new file mode 100644
index 00000000..31657093
--- /dev/null
+++ b/_posts/blog/2018-03-22-view-metrics-in-grafana.md
@@ -0,0 +1,89 @@
+---
+title: "View Accumulo metrics in Grafana"
+---
+
+The [Accumulo monitor][monitor] provides the easiest way to view Accumulo metrics but it has two limitations:
+
+1. The overview page is limited to 10 graphs
+1. These graphs only show metrics for the past two hours
+
+The Accumulo monitor could be improved to be better a visualizaiton tool for metrics but this doesn't make
+sense when tools like [Grafana], [Graphite], and [Ganglia] exist and Accumulo can report metrics to these tools
+using [Hadoop Metrics2][metrics2]. While it's easy to configure Accumulo to send metrics, the hard part is
+setting up and configuring a metrics visualization tool (i.e Graphite, Ganglia, Grafana/InfluxDB) to collect
+and view these metrics.
+
+To ease this burden, this post describes how to send Accumulo metrics to [InfluxDB], a time series database,
+and make them viewable in [Grafana], a visualization tool.
+
+Below is a screenshot of Accumulo metrics in Grafana:
+
+<img src='/images/blog/201803_metrics/accumulo-metrics1.png' width="100%">
+<img src='/images/blog/201803_metrics/accumulo-metrics2.png' width="100%">
+
+## Set up metrics using Uno
+
+[Uno] can setup and configure InfluxDB/Grafana when it sets up Accumulo using the following command:
+
+```
+uno setup accumulo --with-metrics
+```
+
+Metrics from Accumulo will be immediately viewable in Grafana at [http://localhost:3000/](http://localhost:3000/).
+
+## Set up metrics manually
+
+1. Follow the standard installation instructions for [InfluxDB] and [Grafana]. As for versions,
+ the instructions below were written using InfluxDB v0.9.4.2 and Grafana v2.5.0.
+1. Add the following to your InfluxDB configuration to configure it accept metrics in Graphite
+ format from Accumulo. The configuration below contains templates that transform the Graphite
+ metrics into a format that is usable in InfluxDB.
+ ```
+ [[graphite]]
+ bind-address = ":2003"
+ enabled = true
+ database = "accumulo_metrics"
+ protocol = "tcp"
+ consistency-level = "one"
+ separator = "_"
+ batch-size = 1000
+ batch-pending = 5
+ batch-timeout = "1s"
+ templates = [
+ "accumulo.*.*.*.*.*.*.* measurement.measurement.measurement.d.e.f.host.measurement",
+ "accumulo.*.*.*.*.*.* measurement.measurement.measurement.d.e.host.measurement",
+ "accumulo.*.*.*.*.* measurement.measurement.measurement.d.host.measurement",
+ ]
+ ```
+1. Configure the Accumulo configuration file `hadoop-metrics2-accumulo.properties` to send Graphite
+ metrics to InfluxDB. Below is example configuration. Remember to replace `<INFLUXDB_HOST>` with
+ the actual host.
+ ```
+ *.period=30
+ accumulo.sink.graphite.class=org.apache.hadoop.metrics2.sink.GraphiteSink
+ accumulo.sink.graphite.server_host=<INFLUXDB_HOST>
+ accumulo.sink.graphite.server_port=2003
+ accumulo.sink.graphite.metrics_prefix=accumulo
+ ```
+ Make sure the reporting frequency is set to 30 sec (i.e `*.period=30`). This is required if you are
+ using the provided Grafana dashboards that is configured in the next step.
+1. Grafana needs to be configured to load dashboard JSON templates from a directory. Before restarting
+ Grafana, you should copy [this Accumulo dashboard template][dashboard] to the `dashboards/` directory
+ configured below.
+ ```
+ [dashboards.json]
+ enabled = true
+ path = <GRAFANA_HOME>/dashboards
+ ```
+1. If you restart Grafana, you will see the Accumulo dashboard configured but all of their charts will
+ be empty unless you have Accumulo running and configured to send data to InfluxDB. When you start
+ sending data, you may need to refresh the dashboard page in the browser to start viewing metrics.
+
+[Uno]: https://github.com/astralway/uno
+[metrics2]: /docs/2.0/administration/monitoring-metrics#hadoop-metrics2
+[Graphite]: https://graphiteapp.org/
+[Ganglia]: http://ganglia.sourceforge.net/
+[monitor]: http://0.0.0.0:4000/features/#screenshots
+[Grafana]: http://grafana.org/
+[InfluxDB]: https://influxdb.com/
+[dashboard]: https://raw.githubusercontent.com/astralway/uno/master/conf/grafana/accumulo-dashboard.json
diff --git a/images/blog/201803_metrics/accumulo-metrics1.png b/images/blog/201803_metrics/accumulo-metrics1.png
new file mode 100644
index 00000000..a46cf868
Binary files /dev/null and b/images/blog/201803_metrics/accumulo-metrics1.png differ
diff --git a/images/blog/201803_metrics/accumulo-metrics2.png b/images/blog/201803_metrics/accumulo-metrics2.png
new file mode 100644
index 00000000..b1929098
Binary files /dev/null and b/images/blog/201803_metrics/accumulo-metrics2.png differ
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services