You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by le...@apache.org on 2020/07/28 12:33:27 UTC
[hudi] branch asf-site updated: [DOC][HUDI-1123] add doc for user
defined metrics reporter (#1879)
This is an automated email from the ASF dual-hosted git repository.
leesf pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/hudi.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 7b12147 [DOC][HUDI-1123] add doc for user defined metrics reporter (#1879)
7b12147 is described below
commit 7b12147f7b2cb35598ee20ecd8caa2f25de5db47
Author: zherenyu831 <52...@users.noreply.github.com>
AuthorDate: Tue Jul 28 21:33:13 2020 +0900
[DOC][HUDI-1123] add doc for user defined metrics reporter (#1879)
---
docs/_docs/2_4_configurations.cn.md | 10 +++++++
docs/_docs/2_4_configurations.md | 11 ++++++++
docs/_docs/2_8_metrics.md | 53 +++++++++++++++++++++++++++++++++++++
3 files changed, 74 insertions(+)
diff --git a/docs/_docs/2_4_configurations.cn.md b/docs/_docs/2_4_configurations.cn.md
index a75419e..b577990 100644
--- a/docs/_docs/2_4_configurations.cn.md
+++ b/docs/_docs/2_4_configurations.cn.md
@@ -519,6 +519,16 @@ Hudi提供了一个选项,可以通过将对该分区中的插入作为对现
属性: `hoodie.metrics.datadog.metric.tags` <br/>
<span style="color:grey">Datadog指标标签(逗号分隔),将和指标数据一并发送。</span>
+#### 用户自定义发送器
+
+##### on(metricsOn = false) {#on}
+属性: `hoodie.metrics.on` <br/>
+<span style="color:grey">打开或关闭发送指标。默认情况下处于关闭状态。</span>
+
+##### withReporterClass(className = "") {#withReporterClass}
+属性: `hoodie.metrics.reporter.class` <br/>
+<span style="color:grey">用于处理发送指标的用户自定义类,必须是AbstractUserDefinedMetricsReporter类的子类.</span>
+
### 内存配置
控制由Hudi内部执行的压缩和合并的内存使用情况
[withMemoryConfig](#withMemoryConfig) (HoodieMemoryConfig) <br/>
diff --git a/docs/_docs/2_4_configurations.md b/docs/_docs/2_4_configurations.md
index 6fc1049..627d148 100644
--- a/docs/_docs/2_4_configurations.md
+++ b/docs/_docs/2_4_configurations.md
@@ -483,6 +483,17 @@ Property: `hoodie.metrics.datadog.metric.host` <br/>
Property: `hoodie.metrics.datadog.metric.tags` <br/>
<span style="color:grey">Datadog metric tags (comma-delimited) to be sent along with metrics data.</span>
+#### USER DEFINED REPORTER
+
+##### on(metricsOn = false) {#on}
+`hoodie.metrics.on` <br/>
+<span style="color:grey">Turn on/off metrics reporting. off by default.</span>
+
+##### withReporterClass(className = "") {#withReporterClass}
+Property: `hoodie.metrics.reporter.class` <br/>
+<span style="color:grey">User-defined class used to report metrics, must be a subclass of AbstractUserDefinedMetricsReporter.</span>
+
+
### Memory configs
Controls memory usage for compaction and merges, performed internally by Hudi
[withMemoryConfig](#withMemoryConfig) (HoodieMemoryConfig) <br/>
diff --git a/docs/_docs/2_8_metrics.md b/docs/_docs/2_8_metrics.md
index e5043af..287053c 100644
--- a/docs/_docs/2_8_metrics.md
+++ b/docs/_docs/2_8_metrics.md
@@ -90,6 +90,59 @@ In this demo, we ran a `HoodieDeltaStreamer` job with `HoodieMetrics` turned on
* `<prefix>.<table name>.deltastreamer.duration`
* `<prefix>.<table name>.deltastreamer.hiveSyncDuration`
+
+### UserDefinedMetricsReporter
+
+Allows users to define a custom metrics reporter.
+
+#### Configurations
+The following is an example of `UserDefinedMetricsReporter`. More detailed configurations can be referenced [here](configurations.html#user-defined-reporter).
+
+```properties
+hoodie.metrics.on=true
+hoodie.metrics.reporter.class=test.TestUserDefinedMetricsReporter
+```
+
+#### Demo
+In this simple demo, TestMetricsReporter will print all gauges every 10 seconds
+
+```java
+public static class TestUserDefinedMetricsReporter
+ extends AbstractUserDefinedMetricsReporter {
+ private static final Logger log = LogManager.getLogger(DummyMetricsReporter.class);
+
+ private ScheduledExecutorService exec = Executors.newScheduledThreadPool(1, r -> {
+ Thread t = Executors.defaultThreadFactory().newThread(r);
+ t.setDaemon(true);
+ return t;
+ });
+
+ public TestUserDefinedMetricsReporter(Properties props, MetricRegistry registry) {
+ super(props, registry);
+ }
+
+ @Override
+ public void start() {
+ exec.schedule(this::report, 10, TimeUnit.SECONDS);
+ }
+
+ @Override
+ public void report() {
+ this.getRegistry().getGauges().forEach((key, value) ->
+ log.info("key: " + key + " value: " + value.getValue().toString()));
+ }
+
+ @Override
+ public Closeable getReporter() {
+ return null;
+ }
+
+ @Override
+ public void stop() {
+ exec.shutdown();
+ }
+}
+```
## HoodieMetrics