You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by le...@apache.org on 2020/07/28 12:33:27 UTC

[hudi] branch asf-site updated: [DOC][HUDI-1123] add doc for user defined metrics reporter (#1879)

This is an automated email from the ASF dual-hosted git repository.

leesf pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/hudi.git


The following commit(s) were added to refs/heads/asf-site by this push:
     new 7b12147  [DOC][HUDI-1123] add doc for user defined metrics reporter (#1879)
7b12147 is described below

commit 7b12147f7b2cb35598ee20ecd8caa2f25de5db47
Author: zherenyu831 <52...@users.noreply.github.com>
AuthorDate: Tue Jul 28 21:33:13 2020 +0900

    [DOC][HUDI-1123] add doc for user defined metrics reporter (#1879)
---
 docs/_docs/2_4_configurations.cn.md | 10 +++++++
 docs/_docs/2_4_configurations.md    | 11 ++++++++
 docs/_docs/2_8_metrics.md           | 53 +++++++++++++++++++++++++++++++++++++
 3 files changed, 74 insertions(+)

diff --git a/docs/_docs/2_4_configurations.cn.md b/docs/_docs/2_4_configurations.cn.md
index a75419e..b577990 100644
--- a/docs/_docs/2_4_configurations.cn.md
+++ b/docs/_docs/2_4_configurations.cn.md
@@ -519,6 +519,16 @@ Hudi提供了一个选项,可以通过将对该分区中的插入作为对现
 属性: `hoodie.metrics.datadog.metric.tags` <br/>
 <span style="color:grey">Datadog指标标签(逗号分隔),将和指标数据一并发送。</span>
 
+#### 用户自定义发送器
+
+##### on(metricsOn = false) {#on}
+属性: `hoodie.metrics.on` <br/>
+<span style="color:grey">打开或关闭发送指标。默认情况下处于关闭状态。</span>
+
+##### withReporterClass(className = "") {#withReporterClass}
+属性: `hoodie.metrics.reporter.class` <br/>
+<span style="color:grey">用于处理发送指标的用户自定义类,必须是AbstractUserDefinedMetricsReporter类的子类.</span>
+
 ### 内存配置
 控制由Hudi内部执行的压缩和合并的内存使用情况
 [withMemoryConfig](#withMemoryConfig) (HoodieMemoryConfig) <br/>
diff --git a/docs/_docs/2_4_configurations.md b/docs/_docs/2_4_configurations.md
index 6fc1049..627d148 100644
--- a/docs/_docs/2_4_configurations.md
+++ b/docs/_docs/2_4_configurations.md
@@ -483,6 +483,17 @@ Property: `hoodie.metrics.datadog.metric.host` <br/>
 Property: `hoodie.metrics.datadog.metric.tags` <br/>
 <span style="color:grey">Datadog metric tags (comma-delimited) to be sent along with metrics data.</span>
 
+#### USER DEFINED REPORTER
+
+##### on(metricsOn = false) {#on}
+`hoodie.metrics.on` <br/>
+<span style="color:grey">Turn on/off metrics reporting. off by default.</span>
+
+##### withReporterClass(className = "") {#withReporterClass}
+Property: `hoodie.metrics.reporter.class` <br/>
+<span style="color:grey">User-defined class used to report metrics, must be a subclass of AbstractUserDefinedMetricsReporter.</span>
+
+
 ### Memory configs
 Controls memory usage for compaction and merges, performed internally by Hudi
 [withMemoryConfig](#withMemoryConfig) (HoodieMemoryConfig) <br/>
diff --git a/docs/_docs/2_8_metrics.md b/docs/_docs/2_8_metrics.md
index e5043af..287053c 100644
--- a/docs/_docs/2_8_metrics.md
+++ b/docs/_docs/2_8_metrics.md
@@ -90,6 +90,59 @@ In this demo, we ran a `HoodieDeltaStreamer` job with `HoodieMetrics` turned on
 
  * `<prefix>.<table name>.deltastreamer.duration`
  * `<prefix>.<table name>.deltastreamer.hiveSyncDuration`
+ 
+### UserDefinedMetricsReporter
+
+Allows users to define a custom metrics reporter.
+
+#### Configurations
+The following is an example of `UserDefinedMetricsReporter`. More detailed configurations can be referenced [here](configurations.html#user-defined-reporter).
+
+```properties
+hoodie.metrics.on=true
+hoodie.metrics.reporter.class=test.TestUserDefinedMetricsReporter
+```
+
+#### Demo
+In this simple demo, TestMetricsReporter will print all gauges every 10 seconds
+
+```java
+public static class TestUserDefinedMetricsReporter 
+    extends AbstractUserDefinedMetricsReporter {
+  private static final Logger log = LogManager.getLogger(DummyMetricsReporter.class);
+
+  private ScheduledExecutorService exec = Executors.newScheduledThreadPool(1, r -> {
+      Thread t = Executors.defaultThreadFactory().newThread(r);
+      t.setDaemon(true);
+      return t;
+  });
+
+  public TestUserDefinedMetricsReporter(Properties props, MetricRegistry registry) {
+    super(props, registry);
+  }
+
+  @Override
+  public void start() {
+    exec.schedule(this::report, 10, TimeUnit.SECONDS);
+  }
+
+  @Override
+  public void report() {
+    this.getRegistry().getGauges().forEach((key, value) -> 
+      log.info("key: " + key + " value: " + value.getValue().toString()));
+  }
+
+  @Override
+  public Closeable getReporter() {
+    return null;
+  }
+
+  @Override
+  public void stop() {
+    exec.shutdown();
+  }
+}
+```
 
 ## HoodieMetrics