You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@karaf.apache.org by jb...@apache.org on 2020/05/02 05:05:51 UTC
svn commit: r39232 - in /release/karaf/documentation/decanter: 2_x.html
2_x.pdf
Author: jbonofre
Date: Sat May 2 05:05:51 2020
New Revision: 39232
Log:
Update Decanter documentation
Modified:
release/karaf/documentation/decanter/2_x.html
release/karaf/documentation/decanter/2_x.pdf
Modified: release/karaf/documentation/decanter/2_x.html
==============================================================================
--- release/karaf/documentation/decanter/2_x.html (original)
+++ release/karaf/documentation/decanter/2_x.html Sat May 2 05:05:51 2020
@@ -2584,16 +2584,21 @@ table.CodeRay td.code>pre{padding:0}
<li><a href="#_activemq_jmx">1.2.7. ActiveMQ (JMX)</a></li>
<li><a href="#_camel_jmx">1.2.8. Camel (JMX)</a></li>
<li><a href="#_camel_tracer_notifier">1.2.9. Camel Tracer & Notifier</a></li>
-<li><a href="#_system">1.2.10. System</a></li>
-<li><a href="#_network_socket">1.2.11. Network socket</a></li>
-<li><a href="#_jms">1.2.12. JMS</a></li>
-<li><a href="#_mqtt">1.2.13. MQTT</a></li>
-<li><a href="#_kafka">1.2.14. Kafka</a></li>
-<li><a href="#_rest_servlet">1.2.15. Rest Servlet</a></li>
-<li><a href="#_soap">1.2.16. SOAP</a></li>
-<li><a href="#_dropwizard_metrics">1.2.17. Dropwizard Metrics</a></li>
-<li><a href="#_jdbc">1.2.18. JDBC</a></li>
-<li><a href="#_customizing_properties_in_collectors">1.2.19. Customizing properties in collectors</a></li>
+<li><a href="#_system_oshi">1.2.10. System (oshi)</a></li>
+<li><a href="#_system_script">1.2.11. System (script)</a></li>
+<li><a href="#_network_socket">1.2.12. Network socket</a></li>
+<li><a href="#_jms">1.2.13. JMS</a></li>
+<li><a href="#_mqtt">1.2.14. MQTT</a></li>
+<li><a href="#_kafka">1.2.15. Kafka</a></li>
+<li><a href="#_rest_servlet">1.2.16. Rest Servlet</a></li>
+<li><a href="#_soap">1.2.17. SOAP</a></li>
+<li><a href="#_dropwizard_metrics">1.2.18. Dropwizard Metrics</a></li>
+<li><a href="#_jdbc">1.2.19. JDBC</a></li>
+<li><a href="#_configadmin">1.2.20. ConfigAdmin</a></li>
+<li><a href="#_prometheus">1.2.21. Prometheus</a></li>
+<li><a href="#_redis">1.2.22. Redis</a></li>
+<li><a href="#_elasticsearch">1.2.23. Elasticsearch</a></li>
+<li><a href="#_customizing_properties_in_collectors">1.2.24. Customizing properties in collectors</a></li>
</ul>
</li>
<li><a href="#_appenders">1.3. Appenders</a>
@@ -2605,7 +2610,7 @@ table.CodeRay td.code>pre{padding:0}
<li><a href="#_jms_2">1.3.5. JMS</a></li>
<li><a href="#_camel">1.3.6. Camel</a></li>
<li><a href="#_kafka_2">1.3.7. Kafka</a></li>
-<li><a href="#_redis">1.3.8. Redis</a></li>
+<li><a href="#_redis_2">1.3.8. Redis</a></li>
<li><a href="#_mqtt_2">1.3.9. MQTT</a></li>
<li><a href="#_cassandra">1.3.10. Cassandra</a></li>
<li><a href="#_influxdb">1.3.11. InfluxDB</a></li>
@@ -2622,6 +2627,12 @@ table.CodeRay td.code>pre{padding:0}
<li><a href="#_alerters">1.4.2. Alerters</a></li>
</ul>
</li>
+<li><a href="#_processors">1.5. Processors</a>
+<ul class="sectlevel3">
+<li><a href="#_pass_through">1.5.1. Pass Through</a></li>
+<li><a href="#_aggregate">1.5.2. Aggregate</a></li>
+</ul>
+</li>
</ul>
</li>
<li><a href="#_developer_guide">2. Developer Guide</a>
@@ -2635,6 +2646,7 @@ table.CodeRay td.code>pre{padding:0}
</li>
<li><a href="#_custom_appender">2.3. Custom Appender</a></li>
<li><a href="#_custom_alerter">2.4. Custom Alerter</a></li>
+<li><a href="#_custom_processor">2.5. Custom Processor</a></li>
</ul>
</li>
</ul>
@@ -3415,7 +3427,77 @@ call your extender to populate extra pro
</div>
</div>
<div class="sect3">
-<h4 id="_system">1.2.10. System</h4>
+<h4 id="_system_oshi">1.2.10. System (oshi)</h4>
+<div class="paragraph">
+<p>The oshi collector is a system collector (polled) that periodically retrieve all details about the hardware and the operating system.</p>
+</div>
+<div class="paragraph">
+<p>This collector gets lot of details about the machine.</p>
+</div>
+<div class="paragraph">
+<p>The <code>decanter-collector-oshi</code> feature installs the oshi system collector:</p>
+</div>
+<div class="listingblock">
+<div class="content">
+<pre>karaf@root()> feature:install decanter-collector-oshi</pre>
+</div>
+</div>
+<div class="paragraph">
+<p>This feature installs a default <code>etc/org.apache.karaf.decanter.collector.oshi.cfg</code> configuration file containing:</p>
+</div>
+<div class="listingblock">
+<div class="content">
+<pre>################################################################################
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+################################################################################
+
+#
+# Decanter oshi (system) collector
+#
+
+# computerSystem=true
+# computerSystem.baseboard=true
+# computerSystem.firmware=true
+# memory=true
+# processors=true
+# processors.logical=true
+# displays=true
+# disks=true
+# disks.partitions=true
+# graphicsCards=true
+# networkIFs=true
+# powerSources=true
+# soundCards=true
+# sensors=true
+# usbDevices=true
+# operatingSystem=true
+# operatingSystem.fileSystems=true
+# operatingSystem.networkParams=true
+# operatingSystem.processes=true
+# operatingSystem.services=true</pre>
+</div>
+</div>
+<div class="paragraph">
+<p>By default, the oshi collector gets all details about the machine. You can filter what you want to harvest in this configuration file.</p>
+</div>
+</div>
+<div class="sect3">
+<h4 id="_system_script">1.2.11. System (script)</h4>
<div class="paragraph">
<p>The system collector is a polled collector (periodically executed by the Decanter Scheduler).</p>
</div>
@@ -3488,7 +3570,7 @@ call your extender to populate extra pro
</div>
</div>
<div class="sect3">
-<h4 id="_network_socket">1.2.11. Network socket</h4>
+<h4 id="_network_socket">1.2.12. Network socket</h4>
<div class="paragraph">
<p>The Decanter network socket collector listens for incoming messages coming from a remote network socket collector.</p>
</div>
@@ -3540,7 +3622,7 @@ sent by the client.</p>
</div>
</div>
<div class="sect3">
-<h4 id="_jms">1.2.12. JMS</h4>
+<h4 id="_jms">1.2.13. JMS</h4>
<div class="paragraph">
<p>The Decanter JMS collector consumes the data from a JMS queue or topic. It’s a way to aggregate collected data coming
from (several) remote machines.</p>
@@ -3596,7 +3678,7 @@ destination.type=queue
</div>
</div>
<div class="sect3">
-<h4 id="_mqtt">1.2.13. MQTT</h4>
+<h4 id="_mqtt">1.2.14. MQTT</h4>
<div class="paragraph">
<p>The Decanter MQTT collector receives collected messages from a MQTT broker. It’s a way to aggregate collected data coming
from (several) remote machines.</p>
@@ -3643,7 +3725,7 @@ topic=decanter</pre>
</div>
</div>
<div class="sect3">
-<h4 id="_kafka">1.2.14. Kafka</h4>
+<h4 id="_kafka">1.2.15. Kafka</h4>
<div class="paragraph">
<p>The Decanter Kafka collector receives collected messages from a Kafka broker. It’s a way to aggregate collected data coming
from (several) remote machines.</p>
@@ -3725,7 +3807,7 @@ from (several) remote machines.</p>
</div>
</div>
<div class="sect3">
-<h4 id="_rest_servlet">1.2.15. Rest Servlet</h4>
+<h4 id="_rest_servlet">1.2.16. Rest Servlet</h4>
<div class="paragraph">
<p>The Decanter Rest Servlet collector registers a servlet on the OSGi HTTP service (by default on <code>/decanter/collect</code>).</p>
</div>
@@ -3742,7 +3824,7 @@ from (several) remote machines.</p>
</div>
</div>
<div class="sect3">
-<h4 id="_soap">1.2.16. SOAP</h4>
+<h4 id="_soap">1.2.17. SOAP</h4>
<div class="paragraph">
<p>The Decanter SOAP collector periodically requests a SOAP service and returns the result (the SOAP Response, or error details if it failed).</p>
</div>
@@ -3785,7 +3867,7 @@ soap.request=</pre>
</div>
</div>
<div class="sect3">
-<h4 id="_dropwizard_metrics">1.2.17. Dropwizard Metrics</h4>
+<h4 id="_dropwizard_metrics">1.2.18. Dropwizard Metrics</h4>
<div class="paragraph">
<p>The Decanter Dropwizard Metrics collector get a <code>MetricSet</code> OSGi service and periodically get the metrics in the set.</p>
</div>
@@ -3803,12 +3885,12 @@ send to the Decanter dispatcher.</p>
</div>
</div>
<div class="sect3">
-<h4 id="_jdbc">1.2.18. JDBC</h4>
+<h4 id="_jdbc">1.2.19. JDBC</h4>
<div class="paragraph">
<p>The Decanter JDBC collector periodically executes a query on a database and sends the query result to the dispatcher.</p>
</div>
<div class="paragraph">
-<p>The <code>decanter-collector-jdbc</code> installs the JDBC collector:</p>
+<p>The <code>decanter-collector-jdbc</code> feature installs the JDBC collector:</p>
</div>
<div class="listingblock">
<div class="content">
@@ -3850,7 +3932,173 @@ create this datasource using the Karaf <
</div>
</div>
<div class="sect3">
-<h4 id="_customizing_properties_in_collectors">1.2.19. Customizing properties in collectors</h4>
+<h4 id="_configadmin">1.2.20. ConfigAdmin</h4>
+<div class="paragraph">
+<p>The Decanter ConfigAdmin collector listens for any configuration change and send the updated configuration to the dispatcher.</p>
+</div>
+<div class="paragraph">
+<p>The <code>decanter-collector-configadmin</code> feature installs the ConfigAdmin collector:</p>
+</div>
+<div class="listingblock">
+<div class="content">
+<pre>karaf@root()> feature:install decanter-collector-configadmin</pre>
+</div>
+</div>
+</div>
+<div class="sect3">
+<h4 id="_prometheus">1.2.21. Prometheus</h4>
+<div class="paragraph">
+<p>The Decanter Prometheus collector is able to periodically (scheduled collector) read Prometheus servlet output to create events sent in Decanter.</p>
+</div>
+<div class="paragraph">
+<p>The <code>decanter-collector-prometheus</code> feature installs the Prometheus collector:</p>
+</div>
+<div class="listingblock">
+<div class="content">
+<pre>karaf@root()> feature:install decanter-collector-prometheus</pre>
+</div>
+</div>
+<div class="paragraph">
+<p>The feature also installs the <code>etc/org.apache.karaf.decanter.collector.prometheus.cfg</code> configuration file containing:</p>
+</div>
+<div class="listingblock">
+<div class="content">
+<pre>prometheus.url=http://host/prometheus</pre>
+</div>
+</div>
+<div class="paragraph">
+<p>The <code>prometheus.url</code> property is mandatory and define the location of the Prometheus export servlet (that could be provided by the Decanter Prometheus appender for instance).</p>
+</div>
+</div>
+<div class="sect3">
+<h4 id="_redis">1.2.22. Redis</h4>
+<div class="paragraph">
+<p>The Decanter Redis collector is able to periodically (scheduled collector) read Redis Map to get key/value pairs.
+You can filter the keys you want thanks to key pattern.</p>
+</div>
+<div class="paragraph">
+<p>The <code>decanter-collector-redis</code> feature installs the Redis collector:</p>
+</div>
+<div class="listingblock">
+<div class="content">
+<pre>karaf@root()> feature:install decanter-collector-redis</pre>
+</div>
+</div>
+<div class="paragraph">
+<p>The feature also installs the <code>etc/org.apache.karaf.decanter.collector.redis.cfg</code> configuration file containing:</p>
+</div>
+<div class="listingblock">
+<div class="content">
+<pre>address=localhost:6379
+
+#
+# Define the connection mode.
+#Â Possible modes: Single (default), Master_Slave, Sentinel, Cluster
+#
+mode=Single
+
+#
+# Name of the Redis map
+# Default is Decanter
+#
+map=Decanter
+
+#
+#Â For Master_Slave mode, we define the location of the master
+# Default is localhost:6379
+#
+#masterAddress=localhost:6379
+
+#
+# For Sentinel model, define the name of the master
+# Default is myMaster
+#
+#masterName=myMaster
+
+#
+#Â For Cluster mode, define the scan interval of the nodes in the cluster
+# Default value is 2000 (2 seconds).
+#
+#scanInterval=2000
+
+#
+# Key pattern to looking for.
+# Default is *
+#
+#keyPattern=*</pre>
+</div>
+</div>
+<div class="paragraph">
+<p>You can configure the Redis connection (depending of the topology) and the key pattern in this configuration file.</p>
+</div>
+</div>
+<div class="sect3">
+<h4 id="_elasticsearch">1.2.23. Elasticsearch</h4>
+<div class="paragraph">
+<p>The Decanter Elasticsearch collector retrieves documents from Elasticsearch periodically (scheduled collector).
+By default, it harvests all documents in the given index, but you can also specify a query.</p>
+</div>
+<div class="paragraph">
+<p>The <code>decanter-collector-elasticsearch</code> feature installs the Elasticsearch collector:</p>
+</div>
+<div class="listingblock">
+<div class="content">
+<pre>karaf@root()> feature:install decanter-collector-elasticsearch</pre>
+</div>
+</div>
+<div class="paragraph">
+<p>The feature also install <code>etc/org.apache.karaf.decanter.collector.elasticsearch.cfg</code> configuration file containing:</p>
+</div>
+<div class="listingblock">
+<div class="content">
+<pre># HTTP address of the elasticsearch nodes (separated with comma)
+addresses=http://localhost:9200
+
+# Basic username and password authentication (no authentication by default)
+#username=user
+#password=password
+
+# Name of the index to request (decanter by default)
+#index=decanter
+
+# Query to request document (match all by default)
+#query=
+
+# Starting point for the document query (no from by default)
+#from=
+
+# Max number of documents retrieved (no max by default)
+#max=
+
+# Search timeout, in seconds (no timeout by default)
+#timeout=</pre>
+</div>
+</div>
+<div class="ulist">
+<ul>
+<li>
+<p><code>addresses</code> property is the location of the Elasticsearch instances. Default is <code>http://localhost:9200</code>.</p>
+</li>
+<li>
+<p><code>username</code> and <code>password</code> properties are used for authentication. They are <code>null</code> (no authentication) by default.</p>
+</li>
+<li>
+<p><code>index</code> property is the Elasticsearch index where to get documents. It’s <code>decanter</code> by default.</p>
+</li>
+<li>
+<p><code>query</code> property is a search query to use. Default is <code>null</code> meaning all documents in the index are harvested.</p>
+</li>
+<li>
+<p><code>from</code> and <code>max</code> properties are used to "square" the query. They are <code>null</code> by default.</p>
+</li>
+<li>
+<p><code>timeout</code> property limits the query execution. There’s no timeout by default.</p>
+</li>
+</ul>
+</div>
+</div>
+<div class="sect3">
+<h4 id="_customizing_properties_in_collectors">1.2.24. Customizing properties in collectors</h4>
<div class="paragraph">
<p>You can add, rename or remove properties collected by the collectors before sending it to the dispatcher.</p>
</div>
@@ -4342,7 +4590,7 @@ destination.uri=direct-vm:decanter</pre>
</div>
</div>
<div class="sect3">
-<h4 id="_redis">1.3.8. Redis</h4>
+<h4 id="_redis_2">1.3.8. Redis</h4>
<div class="paragraph">
<p>The Decanter Redis appender sends the data (collected by the collectors) to a Redis broker.</p>
</div>
@@ -4942,7 +5190,7 @@ ID â Servlet â�
</div>
</div>
<div class="sect4">
-<h5 id="_prometheus">Prometheus</h5>
+<h5 id="_prometheus_2">Prometheus</h5>
<div class="paragraph">
<p>The <code>decanter-appender-prometheus</code> feature collects and exposes metrics on prometheus:</p>
</div>
@@ -5385,6 +5633,94 @@ To "turn" an appender as an alerter, it
</div>
</div>
</div>
+<div class="sect2">
+<h3 id="_processors">1.5. Processors</h3>
+<div class="paragraph">
+<p>Decanter Processors are optional. They receive data from the collectors, apply a processing logic on the received event, and send a new event to the appenders.</p>
+</div>
+<div class="paragraph">
+<p>The processors are listening for incoming events on <code>decanter/collect/*</code> dispatcher topics and send processed events to <code>decanter/process/*</code> dispatcher topics.
+By default, the appenders are listening on <code>decanter/collect/*</code> topics. If you want to append processed events, you have to configure the appenders
+to listen on <code>decanter/process/*</code> topics. To do that, you just have to change appender configuration with:</p>
+</div>
+<div class="listingblock">
+<div class="content">
+<pre>event.topics=decanter/process/*</pre>
+</div>
+</div>
+<div class="paragraph">
+<p>It’s possible to "chain" processors thanks to the topics.
+For instance, you can have the first processor listening on <code>decanter/collect/*</code> topic (containing events coming from the collectors), and
+sending processed events to <code>decanter/process/first</code>. Then, a second processor can listen on <code>decanter/process/first</code> topic and send processed
+data to <code>decanter/process/second</code>. Finally, at the end of the chain, you have to configure the appenders to listen on
+<code>decanter/process/second</code>.</p>
+</div>
+<div class="sect3">
+<h4 id="_pass_through">1.5.1. Pass Through</h4>
+<div class="paragraph">
+<p>This processor doesn’t implement any concrete logic. It’s for the example how to implement a processor.</p>
+</div>
+<div class="paragraph">
+<p>You can install this processor using the <code>decanter-processor-passthrough</code> feature:</p>
+</div>
+<div class="listingblock">
+<div class="content">
+<pre>karaf@root()> feature:install decanter-processor-passthrough</pre>
+</div>
+</div>
+</div>
+<div class="sect3">
+<h4 id="_aggregate">1.5.2. Aggregate</h4>
+<div class="paragraph">
+<p>This processor "merges" several incoming events in a single one that is sent periodically.</p>
+</div>
+<div class="paragraph">
+<p>You can install this processor using the <code>decanter-processor-aggregate</code> feature:</p>
+</div>
+<div class="listingblock">
+<div class="content">
+<pre>karaf@root()> feature:install decanter-processor-aggregate</pre>
+</div>
+</div>
+<div class="paragraph">
+<p>By default, the "merged" event is sent every minute. You can change this using the <code>period</code> configuration.</p>
+</div>
+<div class="paragraph">
+<p>You can provisiong <code>etc/org.apache.karaf.decanter.processor.aggregate.cfg</code> configuration file with:</p>
+</div>
+<div class="listingblock">
+<div class="content">
+<pre>period=120 # this is the period in seconds
+target.topics=decanter/process/aggregate # that's the default target topic</pre>
+</div>
+</div>
+<div class="paragraph">
+<p>You can also decide if a known property is overwritten in the aggregator or appended.</p>
+</div>
+<div class="paragraph">
+<p>By default, properties are not overwritten, meaning that it’s prefixed by the event index in the aggregator:</p>
+</div>
+<div class="listingblock">
+<div class="content">
+<pre>0.foo=first
+0.other=bar
+1.foo=second
+1.other=bar</pre>
+</div>
+</div>
+<div class="paragraph">
+<p>In the processor <code>etc/org.apache.karaf.decanter.processor.aggregate.cfg</code> configuration file, you can enable <code>overwrite</code>:</p>
+</div>
+<div class="listingblock">
+<div class="content">
+<pre>overwrite=true</pre>
+</div>
+</div>
+<div class="paragraph">
+<p>Then, if a property already exist in the aggregator, its value will be overwritten by the new event value received in the aggregator.</p>
+</div>
+</div>
+</div>
</div>
</div>
<div class="sect1">
@@ -6031,12 +6367,21 @@ public class Activator implements Bundle
<p>Once built, you can enable this alerter by deploying the bundle in Karaf (using the deploy folder or the <code>bundle:install</code> command).</p>
</div>
</div>
+<div class="sect2">
+<h3 id="_custom_processor">2.5. Custom Processor</h3>
+<div class="paragraph">
+<p>A Decanter Processor is listening OSGi EventAdmin events from <code>decanter/collect/*</code> and send a new OSGi EventAdmin event to <code>decanter/process/*`</code></p>
+</div>
+<div class="paragraph">
+<p>To see how to implement your own processor, you can take a look on the pass through processor: <a href="https://github.com/apache/karaf-decanter/blob/master/processor/passthrough/src/main/java/org/apache/karaf/decanter/processor/passthrough/PassThroughProcessor.java" class="bare">https://github.com/apache/karaf-decanter/blob/master/processor/passthrough/src/main/java/org/apache/karaf/decanter/processor/passthrough/PassThroughProcessor.java</a></p>
+</div>
+</div>
</div>
</div>
</div>
<div id="footer">
<div id="footer-text">
-Last updated 2020-02-16 09:02:36 CET
+Last updated 2020-04-20 09:52:49 CEST
</div>
</div>
</body>
Modified: release/karaf/documentation/decanter/2_x.pdf
==============================================================================
Binary files release/karaf/documentation/decanter/2_x.pdf (original) and release/karaf/documentation/decanter/2_x.pdf Sat May 2 05:05:51 2020 differ