You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2020/05/08 06:37:59 UTC
[GitHub] [incubator-hudi] xushiyan opened a new pull request #1603: [HUDI-836] Add configs for Datadog metrics reporter
xushiyan opened a new pull request #1603:
URL: https://github.com/apache/incubator-hudi/pull/1603
Update configuration docs for https://github.com/apache/incubator-hudi/pull/1572
## Committer checklist
- [ ] Has a corresponding JIRA in PR title & commit
- [ ] Commit message is descriptive of the change
- [ ] CI is green
- [ ] Necessary doc changes done or have another open PR
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-hudi] xushiyan commented on pull request #1603: [HUDI-836] Add configs for Datadog metrics reporter
Posted by GitBox <gi...@apache.org>.
xushiyan commented on pull request #1603:
URL: https://github.com/apache/incubator-hudi/pull/1603#issuecomment-629712980
@yanghua Not sure if the merging is blocked by generating 0.5.3 docs as this is for 0.6.0 docs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-hudi] xushiyan commented on a change in pull request #1603: [HUDI-836] Add configs for Datadog metrics reporter
Posted by GitBox <gi...@apache.org>.
xushiyan commented on a change in pull request #1603:
URL: https://github.com/apache/incubator-hudi/pull/1603#discussion_r421969776
##########
File path: docs/_docs/2_4_configurations.md
##########
@@ -396,56 +396,90 @@ Property: `hoodie.compaction.daybased.target` <br/>
Property: `hoodie.compaction.payload.class` <br/>
<span style="color:grey">This needs to be same as class used during insert/upserts. Just like writing, compaction also uses the record payload class to merge records in the log against each other, merge again with the base file and produce the final record to be written after compaction.</span>
-
### Metrics configs
+
Enables reporting on Hudi metrics.
[withMetricsConfig](#withMetricsConfig) (HoodieMetricsConfig) <br/>
-<span style="color:grey">Hudi publishes metrics on every commit, clean, rollback etc.The following sections list the supported reporters.</span>
+<span style="color:grey">Hudi publishes metrics on every commit, clean, rollback etc. The following sections list the supported reporters.</span>
#### GRAPHITE
-##### on(metricsOn = true) {#on}
+##### withMetricsOn(metricsOn = true) {#withMetricsOn}
`hoodie.metrics.on` <br/>
<span style="color:grey">Turn sending metrics on/off. on by default.</span>
##### withReporterType(reporterType = GRAPHITE) {#withReporterType}
Property: `hoodie.metrics.reporter.type` <br/>
<span style="color:grey">Type of metrics reporter.</span>
-##### toGraphiteHost(host = localhost) {#toGraphiteHost}
+##### withGraphiteHost(host = localhost) {#withGraphiteHost}
Property: `hoodie.metrics.graphite.host` <br/>
<span style="color:grey">Graphite host to connect to</span>
-##### onGraphitePort(port = 4756) {#onGraphitePort}
+##### withGraphitePort(port = 4756) {#withGraphitePort}
Property: `hoodie.metrics.graphite.port` <br/>
<span style="color:grey">Graphite port to connect to</span>
-##### usePrefix(prefix = "") {#usePrefix}
+##### withGraphitePrefix(prefix = "") {#withGraphitePrefix}
Property: `hoodie.metrics.graphite.metric.prefix` <br/>
<span style="color:grey">Standard prefix applied to all metrics. This helps to add datacenter, environment information for e.g</span>
#### JMX
-##### on(metricsOn = true) {#on}
+##### withMetricsOn(metricsOn = true) {#withMetricsOn}
`hoodie.metrics.on` <br/>
<span style="color:grey">Turn sending metrics on/off. on by default.</span>
##### withReporterType(reporterType = JMX) {#withReporterType}
Property: `hoodie.metrics.reporter.type` <br/>
<span style="color:grey">Type of metrics reporter.</span>
-##### toJmxHost(host = localhost) {#toJmxHost}
+##### withJmxHost(host = localhost) {#withJmxHost}
Property: `hoodie.metrics.jmx.host` <br/>
<span style="color:grey">Jmx host to connect to</span>
-##### onJmxPort(port = 1000-5000) {#onJmxPort}
+##### withJmxPort(port = 1000-5000) {#withJmxPort}
Property: `hoodie.metrics.jmx.port` <br/>
<span style="color:grey">Jmx port to connect to</span>
-##### usePrefix(prefix = "") {#usePrefix}
Review comment:
No prefix settings for JMX hence removing this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-hudi] xushiyan commented on a change in pull request #1603: [HUDI-836] Add configs for Datadog metrics reporter
Posted by GitBox <gi...@apache.org>.
xushiyan commented on a change in pull request #1603:
URL: https://github.com/apache/incubator-hudi/pull/1603#discussion_r421969776
##########
File path: docs/_docs/2_4_configurations.md
##########
@@ -396,56 +396,90 @@ Property: `hoodie.compaction.daybased.target` <br/>
Property: `hoodie.compaction.payload.class` <br/>
<span style="color:grey">This needs to be same as class used during insert/upserts. Just like writing, compaction also uses the record payload class to merge records in the log against each other, merge again with the base file and produce the final record to be written after compaction.</span>
-
### Metrics configs
+
Enables reporting on Hudi metrics.
[withMetricsConfig](#withMetricsConfig) (HoodieMetricsConfig) <br/>
-<span style="color:grey">Hudi publishes metrics on every commit, clean, rollback etc.The following sections list the supported reporters.</span>
+<span style="color:grey">Hudi publishes metrics on every commit, clean, rollback etc. The following sections list the supported reporters.</span>
#### GRAPHITE
-##### on(metricsOn = true) {#on}
+##### withMetricsOn(metricsOn = true) {#withMetricsOn}
`hoodie.metrics.on` <br/>
<span style="color:grey">Turn sending metrics on/off. on by default.</span>
##### withReporterType(reporterType = GRAPHITE) {#withReporterType}
Property: `hoodie.metrics.reporter.type` <br/>
<span style="color:grey">Type of metrics reporter.</span>
-##### toGraphiteHost(host = localhost) {#toGraphiteHost}
+##### withGraphiteHost(host = localhost) {#withGraphiteHost}
Property: `hoodie.metrics.graphite.host` <br/>
<span style="color:grey">Graphite host to connect to</span>
-##### onGraphitePort(port = 4756) {#onGraphitePort}
+##### withGraphitePort(port = 4756) {#withGraphitePort}
Property: `hoodie.metrics.graphite.port` <br/>
<span style="color:grey">Graphite port to connect to</span>
-##### usePrefix(prefix = "") {#usePrefix}
+##### withGraphitePrefix(prefix = "") {#withGraphitePrefix}
Property: `hoodie.metrics.graphite.metric.prefix` <br/>
<span style="color:grey">Standard prefix applied to all metrics. This helps to add datacenter, environment information for e.g</span>
#### JMX
-##### on(metricsOn = true) {#on}
+##### withMetricsOn(metricsOn = true) {#withMetricsOn}
`hoodie.metrics.on` <br/>
<span style="color:grey">Turn sending metrics on/off. on by default.</span>
##### withReporterType(reporterType = JMX) {#withReporterType}
Property: `hoodie.metrics.reporter.type` <br/>
<span style="color:grey">Type of metrics reporter.</span>
-##### toJmxHost(host = localhost) {#toJmxHost}
+##### withJmxHost(host = localhost) {#withJmxHost}
Property: `hoodie.metrics.jmx.host` <br/>
<span style="color:grey">Jmx host to connect to</span>
-##### onJmxPort(port = 1000-5000) {#onJmxPort}
+##### withJmxPort(port = 1000-5000) {#withJmxPort}
Property: `hoodie.metrics.jmx.port` <br/>
<span style="color:grey">Jmx port to connect to</span>
-##### usePrefix(prefix = "") {#usePrefix}
Review comment:
No prefix settings for JMX hence removing this.
##########
File path: docs/_docs/2_4_configurations.md
##########
@@ -396,56 +396,90 @@ Property: `hoodie.compaction.daybased.target` <br/>
Property: `hoodie.compaction.payload.class` <br/>
<span style="color:grey">This needs to be same as class used during insert/upserts. Just like writing, compaction also uses the record payload class to merge records in the log against each other, merge again with the base file and produce the final record to be written after compaction.</span>
-
### Metrics configs
+
Enables reporting on Hudi metrics.
[withMetricsConfig](#withMetricsConfig) (HoodieMetricsConfig) <br/>
-<span style="color:grey">Hudi publishes metrics on every commit, clean, rollback etc.The following sections list the supported reporters.</span>
+<span style="color:grey">Hudi publishes metrics on every commit, clean, rollback etc. The following sections list the supported reporters.</span>
#### GRAPHITE
-##### on(metricsOn = true) {#on}
+##### withMetricsOn(metricsOn = true) {#withMetricsOn}
Review comment:
Trying to make `with` the same prefix for this builder pattern. IMO having different prepositions is just confusing.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-hudi] xushiyan commented on a change in pull request #1603: [HUDI-836] Add configs for Datadog metrics reporter
Posted by GitBox <gi...@apache.org>.
xushiyan commented on a change in pull request #1603:
URL: https://github.com/apache/incubator-hudi/pull/1603#discussion_r421980372
##########
File path: docs/_docs/2_4_configurations.md
##########
@@ -396,56 +396,90 @@ Property: `hoodie.compaction.daybased.target` <br/>
Property: `hoodie.compaction.payload.class` <br/>
<span style="color:grey">This needs to be same as class used during insert/upserts. Just like writing, compaction also uses the record payload class to merge records in the log against each other, merge again with the base file and produce the final record to be written after compaction.</span>
-
### Metrics configs
+
Enables reporting on Hudi metrics.
[withMetricsConfig](#withMetricsConfig) (HoodieMetricsConfig) <br/>
-<span style="color:grey">Hudi publishes metrics on every commit, clean, rollback etc.The following sections list the supported reporters.</span>
+<span style="color:grey">Hudi publishes metrics on every commit, clean, rollback etc. The following sections list the supported reporters.</span>
#### GRAPHITE
-##### on(metricsOn = true) {#on}
+##### withMetricsOn(metricsOn = false) {#withMetricsOn}
`hoodie.metrics.on` <br/>
-<span style="color:grey">Turn sending metrics on/off. on by default.</span>
+<span style="color:grey">Turn on/off metrics reporting. off by default.</span>
##### withReporterType(reporterType = GRAPHITE) {#withReporterType}
Property: `hoodie.metrics.reporter.type` <br/>
<span style="color:grey">Type of metrics reporter.</span>
-##### toGraphiteHost(host = localhost) {#toGraphiteHost}
+##### withGraphiteHost(host = localhost) {#withGraphiteHost}
Property: `hoodie.metrics.graphite.host` <br/>
<span style="color:grey">Graphite host to connect to</span>
-##### onGraphitePort(port = 4756) {#onGraphitePort}
+##### withGraphitePort(port = 4756) {#withGraphitePort}
Property: `hoodie.metrics.graphite.port` <br/>
<span style="color:grey">Graphite port to connect to</span>
-##### usePrefix(prefix = "") {#usePrefix}
+##### withGraphitePrefix(prefix = "") {#withGraphitePrefix}
Property: `hoodie.metrics.graphite.metric.prefix` <br/>
<span style="color:grey">Standard prefix applied to all metrics. This helps to add datacenter, environment information for e.g</span>
#### JMX
-##### on(metricsOn = true) {#on}
+##### withMetricsOn(metricsOn = false) {#withMetricsOn}
`hoodie.metrics.on` <br/>
-<span style="color:grey">Turn sending metrics on/off. on by default.</span>
+<span style="color:grey">Turn on/off metrics reporting. off by default.</span>
##### withReporterType(reporterType = JMX) {#withReporterType}
Property: `hoodie.metrics.reporter.type` <br/>
<span style="color:grey">Type of metrics reporter.</span>
-##### toJmxHost(host = localhost) {#toJmxHost}
+##### withJmxHost(host = localhost) {#withJmxHost}
Property: `hoodie.metrics.jmx.host` <br/>
<span style="color:grey">Jmx host to connect to</span>
-##### onJmxPort(port = 1000-5000) {#onJmxPort}
+##### withJmxPort(port = 1000-5000) {#withJmxPort}
Review comment:
Trying to make `with` the same prefix for this builder pattern. IMO having different prepositions is just confusing.
##########
File path: docs/_docs/2_4_configurations.md
##########
@@ -396,56 +396,90 @@ Property: `hoodie.compaction.daybased.target` <br/>
Property: `hoodie.compaction.payload.class` <br/>
<span style="color:grey">This needs to be same as class used during insert/upserts. Just like writing, compaction also uses the record payload class to merge records in the log against each other, merge again with the base file and produce the final record to be written after compaction.</span>
-
### Metrics configs
+
Enables reporting on Hudi metrics.
[withMetricsConfig](#withMetricsConfig) (HoodieMetricsConfig) <br/>
-<span style="color:grey">Hudi publishes metrics on every commit, clean, rollback etc.The following sections list the supported reporters.</span>
+<span style="color:grey">Hudi publishes metrics on every commit, clean, rollback etc. The following sections list the supported reporters.</span>
#### GRAPHITE
-##### on(metricsOn = true) {#on}
+##### withMetricsOn(metricsOn = false) {#withMetricsOn}
`hoodie.metrics.on` <br/>
-<span style="color:grey">Turn sending metrics on/off. on by default.</span>
+<span style="color:grey">Turn on/off metrics reporting. off by default.</span>
##### withReporterType(reporterType = GRAPHITE) {#withReporterType}
Property: `hoodie.metrics.reporter.type` <br/>
<span style="color:grey">Type of metrics reporter.</span>
-##### toGraphiteHost(host = localhost) {#toGraphiteHost}
+##### withGraphiteHost(host = localhost) {#withGraphiteHost}
Property: `hoodie.metrics.graphite.host` <br/>
<span style="color:grey">Graphite host to connect to</span>
-##### onGraphitePort(port = 4756) {#onGraphitePort}
+##### withGraphitePort(port = 4756) {#withGraphitePort}
Property: `hoodie.metrics.graphite.port` <br/>
<span style="color:grey">Graphite port to connect to</span>
-##### usePrefix(prefix = "") {#usePrefix}
+##### withGraphitePrefix(prefix = "") {#withGraphitePrefix}
Property: `hoodie.metrics.graphite.metric.prefix` <br/>
<span style="color:grey">Standard prefix applied to all metrics. This helps to add datacenter, environment information for e.g</span>
#### JMX
-##### on(metricsOn = true) {#on}
+##### withMetricsOn(metricsOn = false) {#withMetricsOn}
`hoodie.metrics.on` <br/>
-<span style="color:grey">Turn sending metrics on/off. on by default.</span>
+<span style="color:grey">Turn on/off metrics reporting. off by default.</span>
##### withReporterType(reporterType = JMX) {#withReporterType}
Property: `hoodie.metrics.reporter.type` <br/>
<span style="color:grey">Type of metrics reporter.</span>
-##### toJmxHost(host = localhost) {#toJmxHost}
+##### withJmxHost(host = localhost) {#withJmxHost}
Property: `hoodie.metrics.jmx.host` <br/>
<span style="color:grey">Jmx host to connect to</span>
-##### onJmxPort(port = 1000-5000) {#onJmxPort}
+##### withJmxPort(port = 1000-5000) {#withJmxPort}
Property: `hoodie.metrics.jmx.port` <br/>
<span style="color:grey">Jmx port to connect to</span>
-##### usePrefix(prefix = "") {#usePrefix}
-Property: `hoodie.metrics.jmx.metric.prefix` <br/>
-<span style="color:grey">Standard prefix applied to all metrics. This helps to add datacenter, environment information for e.g</span>
-
Review comment:
No prefix settings for JMX hence removing this.
##########
File path: docs/_docs/2_4_configurations.md
##########
@@ -396,56 +396,90 @@ Property: `hoodie.compaction.daybased.target` <br/>
Property: `hoodie.compaction.payload.class` <br/>
<span style="color:grey">This needs to be same as class used during insert/upserts. Just like writing, compaction also uses the record payload class to merge records in the log against each other, merge again with the base file and produce the final record to be written after compaction.</span>
-
### Metrics configs
+
Enables reporting on Hudi metrics.
[withMetricsConfig](#withMetricsConfig) (HoodieMetricsConfig) <br/>
-<span style="color:grey">Hudi publishes metrics on every commit, clean, rollback etc.The following sections list the supported reporters.</span>
+<span style="color:grey">Hudi publishes metrics on every commit, clean, rollback etc. The following sections list the supported reporters.</span>
#### GRAPHITE
-##### on(metricsOn = true) {#on}
+##### withMetricsOn(metricsOn = false) {#withMetricsOn}
`hoodie.metrics.on` <br/>
-<span style="color:grey">Turn sending metrics on/off. on by default.</span>
+<span style="color:grey">Turn on/off metrics reporting. off by default.</span>
Review comment:
This is false by default
##########
File path: docs/_docs/2_4_configurations.md
##########
@@ -396,56 +396,90 @@ Property: `hoodie.compaction.daybased.target` <br/>
Property: `hoodie.compaction.payload.class` <br/>
<span style="color:grey">This needs to be same as class used during insert/upserts. Just like writing, compaction also uses the record payload class to merge records in the log against each other, merge again with the base file and produce the final record to be written after compaction.</span>
-
### Metrics configs
+
Enables reporting on Hudi metrics.
[withMetricsConfig](#withMetricsConfig) (HoodieMetricsConfig) <br/>
-<span style="color:grey">Hudi publishes metrics on every commit, clean, rollback etc.The following sections list the supported reporters.</span>
+<span style="color:grey">Hudi publishes metrics on every commit, clean, rollback etc. The following sections list the supported reporters.</span>
#### GRAPHITE
-##### on(metricsOn = true) {#on}
+##### withMetricsOn(metricsOn = true) {#withMetricsOn}
Review comment:
Trying to make `with` the same prefix for this builder pattern. IMO having different prepositions is just confusing.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-hudi] xushiyan commented on pull request #1603: [HUDI-836] Add configs for Datadog metrics reporter
Posted by GitBox <gi...@apache.org>.
xushiyan commented on pull request #1603:
URL: https://github.com/apache/incubator-hudi/pull/1603#issuecomment-628304763
Note: blocked by #1572
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-hudi] xushiyan edited a comment on pull request #1603: [HUDI-836] Add configs for Datadog metrics reporter
Posted by GitBox <gi...@apache.org>.
xushiyan edited a comment on pull request #1603:
URL: https://github.com/apache/incubator-hudi/pull/1603#issuecomment-629712980
@yanghua This is ready for review now. But not sure if the merging is blocked by generating 0.5.3 docs as this is for 0.6.0 docs.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-hudi] yanghua commented on pull request #1603: [HUDI-836] Add configs for Datadog metrics reporter
Posted by GitBox <gi...@apache.org>.
yanghua commented on pull request #1603:
URL: https://github.com/apache/incubator-hudi/pull/1603#issuecomment-632999818
@xushiyan Since the reporter has been merged, I will review this PR soon.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-hudi] xushiyan commented on a change in pull request #1603: [HUDI-836] Add configs for Datadog metrics reporter
Posted by GitBox <gi...@apache.org>.
xushiyan commented on a change in pull request #1603:
URL: https://github.com/apache/incubator-hudi/pull/1603#discussion_r421980372
##########
File path: docs/_docs/2_4_configurations.md
##########
@@ -396,56 +396,90 @@ Property: `hoodie.compaction.daybased.target` <br/>
Property: `hoodie.compaction.payload.class` <br/>
<span style="color:grey">This needs to be same as class used during insert/upserts. Just like writing, compaction also uses the record payload class to merge records in the log against each other, merge again with the base file and produce the final record to be written after compaction.</span>
-
### Metrics configs
+
Enables reporting on Hudi metrics.
[withMetricsConfig](#withMetricsConfig) (HoodieMetricsConfig) <br/>
-<span style="color:grey">Hudi publishes metrics on every commit, clean, rollback etc.The following sections list the supported reporters.</span>
+<span style="color:grey">Hudi publishes metrics on every commit, clean, rollback etc. The following sections list the supported reporters.</span>
#### GRAPHITE
-##### on(metricsOn = true) {#on}
+##### withMetricsOn(metricsOn = false) {#withMetricsOn}
`hoodie.metrics.on` <br/>
-<span style="color:grey">Turn sending metrics on/off. on by default.</span>
+<span style="color:grey">Turn on/off metrics reporting. off by default.</span>
##### withReporterType(reporterType = GRAPHITE) {#withReporterType}
Property: `hoodie.metrics.reporter.type` <br/>
<span style="color:grey">Type of metrics reporter.</span>
-##### toGraphiteHost(host = localhost) {#toGraphiteHost}
+##### withGraphiteHost(host = localhost) {#withGraphiteHost}
Property: `hoodie.metrics.graphite.host` <br/>
<span style="color:grey">Graphite host to connect to</span>
-##### onGraphitePort(port = 4756) {#onGraphitePort}
+##### withGraphitePort(port = 4756) {#withGraphitePort}
Property: `hoodie.metrics.graphite.port` <br/>
<span style="color:grey">Graphite port to connect to</span>
-##### usePrefix(prefix = "") {#usePrefix}
+##### withGraphitePrefix(prefix = "") {#withGraphitePrefix}
Property: `hoodie.metrics.graphite.metric.prefix` <br/>
<span style="color:grey">Standard prefix applied to all metrics. This helps to add datacenter, environment information for e.g</span>
#### JMX
-##### on(metricsOn = true) {#on}
+##### withMetricsOn(metricsOn = false) {#withMetricsOn}
`hoodie.metrics.on` <br/>
-<span style="color:grey">Turn sending metrics on/off. on by default.</span>
+<span style="color:grey">Turn on/off metrics reporting. off by default.</span>
##### withReporterType(reporterType = JMX) {#withReporterType}
Property: `hoodie.metrics.reporter.type` <br/>
<span style="color:grey">Type of metrics reporter.</span>
-##### toJmxHost(host = localhost) {#toJmxHost}
+##### withJmxHost(host = localhost) {#withJmxHost}
Property: `hoodie.metrics.jmx.host` <br/>
<span style="color:grey">Jmx host to connect to</span>
-##### onJmxPort(port = 1000-5000) {#onJmxPort}
+##### withJmxPort(port = 1000-5000) {#withJmxPort}
Review comment:
Trying to make `with` the same prefix for this builder pattern. IMO having different prepositions is just confusing.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org