You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by GitBox <gi...@apache.org> on 2022/09/16 09:44:13 UTC
[GitHub] [skywalking] soander opened a new pull request, #9620: Support the telegraf receiver plugin module
soander opened a new pull request, #9620:
URL: https://github.com/apache/skywalking/pull/9620
<!--
β οΈ Please make sure to read this template first, pull requests that don't accord with this template
maybe closed without notice.
Texts surrounded by `<` and `>` are meant to be replaced by you, e.g. <framework name>, <issue number>.
Put an `x` in the `[ ]` to mark the item as CHECKED. `[x]`
-->
<!-- ==== π Remove this line WHEN AND ONLY WHEN you're fixing a bug, follow the checklist π ====
### Fix <bug description or the bug issue number or bug issue link>
- [ ] Add a unit test to verify that the fix works.
- [ ] Explain briefly why the bug exists and how to fix it.
==== π Remove this line WHEN AND ONLY WHEN you're fixing a bug, follow the checklist π ==== -->
<!-- ==== π Remove this line WHEN AND ONLY WHEN you're improving the performance, follow the checklist π ====
### Improve the performance of <class or module or ...>
- [ ] Add a benchmark for the improvement, refer to [the existing ones](https://github.com/apache/skywalking/blob/master/apm-commons/apm-datacarrier/src/test/java/org/apache/skywalking/apm/commons/datacarrier/LinkedArrayBenchmark.java)
- [ ] The benchmark result.
```text
<Paste the benchmark results here>
```
- [ ] Links/URLs to the theory proof or discussion articles/blogs. <links/URLs here>
==== π Remove this line WHEN AND ONLY WHEN you're improving the performance, follow the checklist π ==== -->
### <Support the telegraf receiver plugin module>
- [ ] If this is non-trivial feature, paste the links/URLs to the design doc.
- [ ] Update the documentation to include this new feature.
- [ ] Tests(including UT, IT, E2E) are added to verify the new feature.
- [X] If it's UI related, attach the screenshots below.
- [ ] If this pull request closes/resolves/fixes an existing issue, replace the issue number. Closes #<issue number>.
- [ ] Update the [`CHANGES` log](https://github.com/apache/skywalking/blob/master/docs/en/changes/changes.md).
The following screenshot shows the detail query information.
<img width="1090" alt="image" src="https://user-images.githubusercontent.com/79000735/190608100-7d9aaf1c-df59-47c4-ba39-7b8ecd31bce0.png">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] soander commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
soander commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r979170546
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/handler/TelegrafServiceHandler.java:
##########
@@ -0,0 +1,128 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.handler;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.google.common.collect.ImmutableMap;
+import com.linecorp.armeria.server.annotation.Post;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.skywalking.apm.network.common.v3.Commands;
+import org.apache.skywalking.oap.meter.analyzer.MetricConvert;
+import org.apache.skywalking.oap.meter.analyzer.dsl.Sample;
+import org.apache.skywalking.oap.meter.analyzer.dsl.SampleFamily;
+import org.apache.skywalking.oap.meter.analyzer.dsl.SampleFamilyBuilder;
+import org.apache.skywalking.oap.server.core.analysis.meter.MeterSystem;
+import org.apache.skywalking.oap.server.library.module.ModuleManager;
+import org.apache.skywalking.oap.server.receiver.telegraf.provider.config.TelegrafConfig;
+import org.apache.skywalking.oap.server.receiver.telegraf.provider.handler.pojo.TelegrafData;
+import org.apache.skywalking.oap.server.telemetry.TelemetryModule;
+import org.apache.skywalking.oap.server.telemetry.api.CounterMetrics;
+import org.apache.skywalking.oap.server.telemetry.api.HistogramMetrics;
+import org.apache.skywalking.oap.server.telemetry.api.MetricsCreator;
+import org.apache.skywalking.oap.server.telemetry.api.MetricsTag;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+@Slf4j
+public class TelegrafServiceHandler {
+
+ private final HistogramMetrics histogram;
+ private final CounterMetrics errorCounter;
+ private List<MetricConvert> metricConvert;
+
+ public TelegrafServiceHandler(ModuleManager moduleManager, MeterSystem meterSystem, List<TelegrafConfig> rules) {
+
+ this.metricConvert = rules.stream().map(r -> new MetricConvert(r, meterSystem)).collect(Collectors.toList());
+
+ final MetricsCreator metricsCreator = moduleManager.find(TelemetryModule.NAME)
+ .provider()
+ .getService(MetricsCreator.class);
+
+ histogram = metricsCreator.createHistogramMetric(
+ "telegraf_in_latency", "The process latency of telegraf data",
+ new MetricsTag.Keys("protocol"), new MetricsTag.Values("http")
+ );
+
+ errorCounter = metricsCreator.createCounter(
+ "telegraf_error_count", "The error number of telegraf analysis",
+ new MetricsTag.Keys("protocol"), new MetricsTag.Values("http")
+ );
+ }
+
+ /**
+ * Convert the TelegrafData object to meter {@link Sample}
+ **/
+ public List<Sample> convertTelegraf(TelegrafData telegrafData) {
+
+ List<Sample> sampleList = new ArrayList<>(Collections.emptyList());
+
+ Map<String, Object> fields = telegrafData.getFields();
+ String name = telegrafData.getName();
+ Map<String, String> tags = telegrafData.getTags();
+ ImmutableMap<String, String> immutableTags = ImmutableMap.copyOf(tags);
+ long timestamp = telegrafData.getTimestamp();
+
+ fields.forEach((key, value) -> {
+ Sample.SampleBuilder builder = Sample.builder();
+ if (value instanceof String) return;
+ Sample sample = builder.name(name + "_" + key)
+ .timestamp(timestamp * 1000L)
+ .value(((Number) value).doubleValue())
+ .labels(immutableTags).build();
+
+ sampleList.add(sample);
+ });
+ return sampleList;
+
+ }
+
+ @Post("/telegraf")
+ public Commands collectData(String jsonInfo) {
+ TelegrafData telegrafData;
+ try (HistogramMetrics.Timer ignored = histogram.createTimer()) {
+ List<Sample> allSamples = new ArrayList<>();
+ ObjectMapper mapper = new ObjectMapper();
+ JsonNode node = mapper.readTree(jsonInfo);
Review Comment:
> Not sure whether it's necessary that class `TelegrafData` should implement `RequestConverterFunction`, this is a standard POJO and the Armeria should be able to deserialize it correctly
I set the class `TelegrafData` like this:
The method shows this **error**.
After I removed the code `@RequestConverter(TelegrafData.class)`, the oap-server can be run, but the class `TelegrafServiceHandler` can't work, and pops up **error** info:
According to the Armeria document, the POJO class should implement `RequestConverterFunction` that can be used in the `@RequestConverter` and `@RequestObject` annotations. You can check the section **Conversion between an HTTP message and a Java object** in this document [https://armeria.dev/docs/server-annotated-service/](url) or the part **Write a request converter** in this tutorials [https://armeria.dev/tutorials/rest/blog/implement-create/](url).
Maybe the class `TelegrafData` can be optimized, but I think class `TelegrafData `must implement `RequestConverterFunction` for converting JSON object to JAVA object.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r1009078995
##########
apm-dist/src/main/assembly/binary.xml:
##########
@@ -69,6 +69,7 @@
<include>ui-initialized-templates/*/*.json</include>
<include>lal/*</include>
<include>log-mal-rules/*</include>
+ <include>telegraf-rules/*.yaml</include>
Review Comment:
```suggestion
<include>telegraf-rules/*</include>
```
Why do we need a `yaml` suffix? I assume there is no other files in the folder, right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] soander commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
soander commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r1013771541
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/handler/TelegrafServiceHandler.java:
##########
@@ -0,0 +1,129 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.handler;
+
+import com.google.common.collect.ImmutableMap;
+import com.linecorp.armeria.server.annotation.Post;
+import com.linecorp.armeria.server.annotation.RequestConverter;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.skywalking.apm.network.common.v3.Commands;
+import org.apache.skywalking.oap.meter.analyzer.MetricConvert;
+import org.apache.skywalking.oap.meter.analyzer.dsl.Sample;
+import org.apache.skywalking.oap.meter.analyzer.dsl.SampleFamily;
+import org.apache.skywalking.oap.meter.analyzer.dsl.SampleFamilyBuilder;
+import org.apache.skywalking.oap.meter.analyzer.prometheus.rule.Rule;
+import org.apache.skywalking.oap.server.core.analysis.meter.MeterSystem;
+import org.apache.skywalking.oap.server.library.module.ModuleManager;
+import org.apache.skywalking.oap.server.receiver.telegraf.provider.handler.pojo.TelegrafData;
+import org.apache.skywalking.oap.server.receiver.telegraf.provider.handler.pojo.TelegrafDatum;
+import org.apache.skywalking.oap.server.telemetry.TelemetryModule;
+import org.apache.skywalking.oap.server.telemetry.api.CounterMetrics;
+import org.apache.skywalking.oap.server.telemetry.api.HistogramMetrics;
+import org.apache.skywalking.oap.server.telemetry.api.MetricsCreator;
+import org.apache.skywalking.oap.server.telemetry.api.MetricsTag;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+@Slf4j
+public class TelegrafServiceHandler {
+
+ private final HistogramMetrics histogram;
+ private final CounterMetrics errorCounter;
+ private List<MetricConvert> metricConvert;
+
+ public TelegrafServiceHandler(ModuleManager moduleManager, MeterSystem meterSystem, List<Rule> rules) {
+
+ this.metricConvert = rules.stream().map(r -> new MetricConvert(r, meterSystem)).collect(Collectors.toList());
+
+ final MetricsCreator metricsCreator = moduleManager.find(TelemetryModule.NAME)
+ .provider()
+ .getService(MetricsCreator.class);
+
+ histogram = metricsCreator.createHistogramMetric(
+ "telegraf_in_latency", "The process latency of telegraf data",
+ new MetricsTag.Keys("protocol"), new MetricsTag.Values("http")
+ );
+
+ errorCounter = metricsCreator.createCounter(
+ "telegraf_error_count", "The error number of telegraf analysis",
+ new MetricsTag.Keys("protocol"), new MetricsTag.Values("http")
+ );
+ }
+
+ /**
+ * Convert the TelegrafData object to meter {@link Sample}
+ **/
+ public List<Sample> convertTelegraf(TelegrafDatum telegrafData) {
+
+ List<Sample> sampleList = new ArrayList<>();
+
+ Map<String, Object> fields = telegrafData.getFields();
+ String name = telegrafData.getName();
+ Map<String, String> tags = telegrafData.getTags();
+ ImmutableMap<String, String> immutableTags = ImmutableMap.copyOf(tags);
+ long timestamp = telegrafData.getTimestamp();
+
+ fields.forEach((key, value) -> {
+ if (value instanceof Number) {
+ Sample.SampleBuilder builder = Sample.builder();
+ Sample sample = builder.name(name + "_" + key)
+ .timestamp(timestamp * 1000L)
+ .value(((Number) value).doubleValue())
+ .labels(immutableTags).build();
+
+ sampleList.add(sample);
+ }
+ });
+ return sampleList;
+
+ }
+
+ public ImmutableMap<String, SampleFamily> convertSampleFamily(TelegrafData telegrafData) {
+ List<Sample> allSamples = new ArrayList<>();
+
+ List<TelegrafDatum> metrics = telegrafData.getMetrics();
+ for (TelegrafDatum m : metrics) {
+ List<Sample> samples = convertTelegraf(m);
+ allSamples.addAll(samples);
+ }
+
+ // Grouping all samples by their name, then build sampleFamily
Review Comment:
> This policy seems not right. The samples should be grouped by bulk/sampling ID. Because
>
> 1. Metrics in the same name with different timestamps should belong to different sample families.
> 2. Different metrics(name) collected in the same timestamp(one collecting) should belong to the same sample family. You could check the MAL doc, which mentioned metrics should be calculated through one formula, ref to https://skywalking.apache.org/docs/main/next/en/concepts-and-designs/mal/#binary-operators, `instance_trace_analysis_error_count / instance_trace_count`
I already fix the issue by grouping sample family by name and timestamp, and extra UTs have been tested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] liuhaoyang commented on pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
liuhaoyang commented on PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#issuecomment-1279767917
> Marked this as `no update`, as we can't have any feedback for two weeks. I am not sure whether this is still on going or still prepares for upstream review.
This PR are still being revised. I will help him to add the e2e tests ..
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] sonatype-lift[bot] commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
sonatype-lift[bot] commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r1000278596
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/config/TelegrafConfigs.java:
##########
@@ -0,0 +1,76 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.config;
+
+import lombok.extern.slf4j.Slf4j;
+import org.apache.skywalking.oap.server.library.module.ModuleStartException;
+import org.apache.skywalking.oap.server.library.util.CollectionUtils;
+import org.apache.skywalking.oap.server.library.util.ResourceUtils;
+import org.yaml.snakeyaml.Yaml;
+
+import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.FileReader;
+import java.io.IOException;
+import java.io.Reader;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Objects;
+import java.util.stream.Collectors;
+
+@Slf4j
+public class TelegrafConfigs {
+ public static List<TelegrafConfig> loadConfigs(String path, List<String> fileNames) throws ModuleStartException {
+ if (CollectionUtils.isEmpty(fileNames)) {
+ return Collections.emptyList();
+ }
+
+ File[] configs;
+ try {
+ configs = ResourceUtils.getPathFiles(path);
+ } catch (FileNotFoundException e) {
+ throw new ModuleStartException("Load telegraf configs failed", e);
+ }
+
+ return Arrays.stream(configs)
+ .filter(File::isFile)
+ .map(f -> {
+ String fileName = f.getName();
+ int dotIndex = fileName.lastIndexOf('.');
+ fileName = (dotIndex == -1) ? fileName : fileName.substring(0, dotIndex);
+ if (!fileNames.contains(fileName)) {
+ return null;
+ }
+ try (Reader r = new FileReader(f)) {
+ return new Yaml().loadAs(r, TelegrafConfig.class);
+ } catch (IOException e) {
+ log.warn("Reading file {} failed", f, e);
+ try {
+ throw new ModuleStartException(e.getMessage(), e);
+ } catch (ModuleStartException ex) {
+ ex.printStackTrace();
Review Comment:
*[CatchAndPrintStackTrace](https://errorprone.info/bugpattern/CatchAndPrintStackTrace):* Logging or rethrowing exceptions should usually be preferred to catching and calling printStackTrace
---
<details><summary><b>βΉοΈ Learn about @sonatype-lift commands</b></summary>
You can reply with the following commands. For example, reply with ***@sonatype-lift ignoreall*** to leave out all findings.
| **Command** | **Usage** |
| ------------- | ------------- |
| `@sonatype-lift ignore` | Leave out the above finding from this PR |
| `@sonatype-lift ignoreall` | Leave out all the existing findings from this PR |
| `@sonatype-lift exclude <file\|issue\|path\|tool>` | Exclude specified `file\|issue\|path\|tool` from Lift findings by updating your config.toml file |
**Note:** When talking to LiftBot, you need to **refresh** the page to see its response.
<sub>[Click here](https://github.com/apps/sonatype-lift/installations/new) to add LiftBot to another repo.</sub></details>
---
Was this a good recommendation?
[ [π Not relevant](https://www.sonatype.com/lift-comment-rating?comment=346699033&lift_comment_rating=1) ] - [ [π Won't fix](https://www.sonatype.com/lift-comment-rating?comment=346699033&lift_comment_rating=2) ] - [ [π Not critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=346699033&lift_comment_rating=3) ] - [ [π Critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=346699033&lift_comment_rating=4) ] - [ [π Critical, fixing now](https://www.sonatype.com/lift-comment-rating?comment=346699033&lift_comment_rating=5) ]
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/config/TelegrafConfigs.java:
##########
@@ -0,0 +1,76 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.config;
+
+import lombok.extern.slf4j.Slf4j;
+import org.apache.skywalking.oap.server.library.module.ModuleStartException;
+import org.apache.skywalking.oap.server.library.util.CollectionUtils;
+import org.apache.skywalking.oap.server.library.util.ResourceUtils;
+import org.yaml.snakeyaml.Yaml;
+
+import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.FileReader;
+import java.io.IOException;
+import java.io.Reader;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Objects;
+import java.util.stream.Collectors;
+
+@Slf4j
+public class TelegrafConfigs {
+ public static List<TelegrafConfig> loadConfigs(String path, List<String> fileNames) throws ModuleStartException {
+ if (CollectionUtils.isEmpty(fileNames)) {
+ return Collections.emptyList();
+ }
+
+ File[] configs;
+ try {
+ configs = ResourceUtils.getPathFiles(path);
+ } catch (FileNotFoundException e) {
+ throw new ModuleStartException("Load telegraf configs failed", e);
+ }
+
+ return Arrays.stream(configs)
+ .filter(File::isFile)
+ .map(f -> {
+ String fileName = f.getName();
+ int dotIndex = fileName.lastIndexOf('.');
+ fileName = (dotIndex == -1) ? fileName : fileName.substring(0, dotIndex);
+ if (!fileNames.contains(fileName)) {
+ return null;
+ }
+ try (Reader r = new FileReader(f)) {
Review Comment:
*[DefaultCharset](https://errorprone.info/bugpattern/DefaultCharset):* Implicit use of the platform default charset, which can result in differing behaviour between JVM executions or incorrect behavior if the encoding of the data source doesn't match expectations.
---
```suggestion
try (Reader r = Files.newBufferedReader(f.toPath(), Charset.defaultCharset())) {
```
---
<details><summary><b>βΉοΈ Learn about @sonatype-lift commands</b></summary>
You can reply with the following commands. For example, reply with ***@sonatype-lift ignoreall*** to leave out all findings.
| **Command** | **Usage** |
| ------------- | ------------- |
| `@sonatype-lift ignore` | Leave out the above finding from this PR |
| `@sonatype-lift ignoreall` | Leave out all the existing findings from this PR |
| `@sonatype-lift exclude <file\|issue\|path\|tool>` | Exclude specified `file\|issue\|path\|tool` from Lift findings by updating your config.toml file |
**Note:** When talking to LiftBot, you need to **refresh** the page to see its response.
<sub>[Click here](https://github.com/apps/sonatype-lift/installations/new) to add LiftBot to another repo.</sub></details>
---
Was this a good recommendation?
[ [π Not relevant](https://www.sonatype.com/lift-comment-rating?comment=346698917&lift_comment_rating=1) ] - [ [π Won't fix](https://www.sonatype.com/lift-comment-rating?comment=346698917&lift_comment_rating=2) ] - [ [π Not critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=346698917&lift_comment_rating=3) ] - [ [π Critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=346698917&lift_comment_rating=4) ] - [ [π Critical, fixing now](https://www.sonatype.com/lift-comment-rating?comment=346698917&lift_comment_rating=5) ]
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/config/TelegrafConfig.java:
##########
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.config;
+
+import lombok.Data;
+import lombok.NoArgsConstructor;
+import org.apache.skywalking.oap.meter.analyzer.MetricRuleConfig;
+
+import java.util.List;
+
+@Data
+public class TelegrafConfig implements MetricRuleConfig {
+
+ private String metricPrefix;
+ private String expSuffix;
+ private String expPrefix;
+ private String filter;
+ private String initExp;
+ private List<Rule> metricsRules;
+
+ @Data
+ @NoArgsConstructor
+ public static class Rule implements RuleConfig {
+ private String name;
+ private String exp;
Review Comment:
*[MissingOverride](https://errorprone.info/bugpattern/MissingOverride):* getExp implements method in RuleConfig; expected @Override
---
```suggestion
private String @Override exp;
```
---
<details><summary><b>βΉοΈ Learn about @sonatype-lift commands</b></summary>
You can reply with the following commands. For example, reply with ***@sonatype-lift ignoreall*** to leave out all findings.
| **Command** | **Usage** |
| ------------- | ------------- |
| `@sonatype-lift ignore` | Leave out the above finding from this PR |
| `@sonatype-lift ignoreall` | Leave out all the existing findings from this PR |
| `@sonatype-lift exclude <file\|issue\|path\|tool>` | Exclude specified `file\|issue\|path\|tool` from Lift findings by updating your config.toml file |
**Note:** When talking to LiftBot, you need to **refresh** the page to see its response.
<sub>[Click here](https://github.com/apps/sonatype-lift/installations/new) to add LiftBot to another repo.</sub></details>
---
Was this a good recommendation?
[ [π Not relevant](https://www.sonatype.com/lift-comment-rating?comment=346698802&lift_comment_rating=1) ] - [ [π Won't fix](https://www.sonatype.com/lift-comment-rating?comment=346698802&lift_comment_rating=2) ] - [ [π Not critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=346698802&lift_comment_rating=3) ] - [ [π Critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=346698802&lift_comment_rating=4) ] - [ [π Critical, fixing now](https://www.sonatype.com/lift-comment-rating?comment=346698802&lift_comment_rating=5) ]
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/config/TelegrafConfig.java:
##########
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.config;
+
+import lombok.Data;
+import lombok.NoArgsConstructor;
+import org.apache.skywalking.oap.meter.analyzer.MetricRuleConfig;
+
+import java.util.List;
+
+@Data
+public class TelegrafConfig implements MetricRuleConfig {
+
+ private String metricPrefix;
+ private String expSuffix;
+ private String expPrefix;
+ private String filter;
+ private String initExp;
+ private List<Rule> metricsRules;
+
+ @Data
+ @NoArgsConstructor
+ public static class Rule implements RuleConfig {
+ private String name;
Review Comment:
*[MissingOverride](https://errorprone.info/bugpattern/MissingOverride):* getName implements method in RuleConfig; expected @Override
---
```suggestion
private String @Override name;
```
---
<details><summary><b>βΉοΈ Learn about @sonatype-lift commands</b></summary>
You can reply with the following commands. For example, reply with ***@sonatype-lift ignoreall*** to leave out all findings.
| **Command** | **Usage** |
| ------------- | ------------- |
| `@sonatype-lift ignore` | Leave out the above finding from this PR |
| `@sonatype-lift ignoreall` | Leave out all the existing findings from this PR |
| `@sonatype-lift exclude <file\|issue\|path\|tool>` | Exclude specified `file\|issue\|path\|tool` from Lift findings by updating your config.toml file |
**Note:** When talking to LiftBot, you need to **refresh** the page to see its response.
<sub>[Click here](https://github.com/apps/sonatype-lift/installations/new) to add LiftBot to another repo.</sub></details>
---
Was this a good recommendation?
[ [π Not relevant](https://www.sonatype.com/lift-comment-rating?comment=346698807&lift_comment_rating=1) ] - [ [π Won't fix](https://www.sonatype.com/lift-comment-rating?comment=346698807&lift_comment_rating=2) ] - [ [π Not critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=346698807&lift_comment_rating=3) ] - [ [π Critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=346698807&lift_comment_rating=4) ] - [ [π Critical, fixing now](https://www.sonatype.com/lift-comment-rating?comment=346698807&lift_comment_rating=5) ]
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/config/TelegrafConfig.java:
##########
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.config;
+
+import lombok.Data;
+import lombok.NoArgsConstructor;
+import org.apache.skywalking.oap.meter.analyzer.MetricRuleConfig;
+
+import java.util.List;
+
+@Data
+public class TelegrafConfig implements MetricRuleConfig {
+
+ private String metricPrefix;
Review Comment:
*[MissingOverride](https://errorprone.info/bugpattern/MissingOverride):* getMetricPrefix implements method in MetricRuleConfig; expected @Override
---
```suggestion
private String @Override metricPrefix;
```
---
<details><summary><b>βΉοΈ Learn about @sonatype-lift commands</b></summary>
You can reply with the following commands. For example, reply with ***@sonatype-lift ignoreall*** to leave out all findings.
| **Command** | **Usage** |
| ------------- | ------------- |
| `@sonatype-lift ignore` | Leave out the above finding from this PR |
| `@sonatype-lift ignoreall` | Leave out all the existing findings from this PR |
| `@sonatype-lift exclude <file\|issue\|path\|tool>` | Exclude specified `file\|issue\|path\|tool` from Lift findings by updating your config.toml file |
**Note:** When talking to LiftBot, you need to **refresh** the page to see its response.
<sub>[Click here](https://github.com/apps/sonatype-lift/installations/new) to add LiftBot to another repo.</sub></details>
---
Was this a good recommendation?
[ [π Not relevant](https://www.sonatype.com/lift-comment-rating?comment=346698834&lift_comment_rating=1) ] - [ [π Won't fix](https://www.sonatype.com/lift-comment-rating?comment=346698834&lift_comment_rating=2) ] - [ [π Not critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=346698834&lift_comment_rating=3) ] - [ [π Critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=346698834&lift_comment_rating=4) ] - [ [π Critical, fixing now](https://www.sonatype.com/lift-comment-rating?comment=346698834&lift_comment_rating=5) ]
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/config/TelegrafConfig.java:
##########
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.config;
+
+import lombok.Data;
+import lombok.NoArgsConstructor;
+import org.apache.skywalking.oap.meter.analyzer.MetricRuleConfig;
+
+import java.util.List;
+
+@Data
+public class TelegrafConfig implements MetricRuleConfig {
+
+ private String metricPrefix;
+ private String expSuffix;
+ private String expPrefix;
+ private String filter;
Review Comment:
*[MissingOverride](https://errorprone.info/bugpattern/MissingOverride):* getFilter implements method in MetricRuleConfig; expected @Override
---
```suggestion
private String @Override filter;
```
---
<details><summary><b>βΉοΈ Learn about @sonatype-lift commands</b></summary>
You can reply with the following commands. For example, reply with ***@sonatype-lift ignoreall*** to leave out all findings.
| **Command** | **Usage** |
| ------------- | ------------- |
| `@sonatype-lift ignore` | Leave out the above finding from this PR |
| `@sonatype-lift ignoreall` | Leave out all the existing findings from this PR |
| `@sonatype-lift exclude <file\|issue\|path\|tool>` | Exclude specified `file\|issue\|path\|tool` from Lift findings by updating your config.toml file |
**Note:** When talking to LiftBot, you need to **refresh** the page to see its response.
<sub>[Click here](https://github.com/apps/sonatype-lift/installations/new) to add LiftBot to another repo.</sub></details>
---
Was this a good recommendation?
[ [π Not relevant](https://www.sonatype.com/lift-comment-rating?comment=346698840&lift_comment_rating=1) ] - [ [π Won't fix](https://www.sonatype.com/lift-comment-rating?comment=346698840&lift_comment_rating=2) ] - [ [π Not critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=346698840&lift_comment_rating=3) ] - [ [π Critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=346698840&lift_comment_rating=4) ] - [ [π Critical, fixing now](https://www.sonatype.com/lift-comment-rating?comment=346698840&lift_comment_rating=5) ]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r1000557641
##########
docs/en/setup/backend/backend-vm-monitoring.md:
##########
@@ -3,38 +3,63 @@ SkyWalking leverages Prometheus node-exporter to collect metrics data from the V
[OpenTelemetry receiver](opentelemetry-receiver.md) and into the [Meter System](./../../concepts-and-designs/meter.md).
VM entity as a `Service` in OAP and on the `Layer: OS_LINUX`.
+SkyWalking also provides InfluxDB Telegraf to receive VMs' metrics data by [Telegraf receiver](./telegraf-receiver.md).
+The telegraf receiver plugin receiver, process and convert the metrics, then it send converted metrics to [Meter System](./../../concepts-and-designs/meter.md).
+VM entity as a `Service` in OAP and on the `Layer: OS_LINUX`.
+
## Data flow
+**For OpenTelemetry receiver:**
1. The Prometheus node-exporter collects metrics data from the VMs.
2. The OpenTelemetry Collector fetches metrics from node-exporter via Prometheus Receiver and pushes metrics to the SkyWalking OAP Server via the OpenCensus gRPC Exporter or OpenTelemetry gRPC exporter.
3. The SkyWalking OAP Server parses the expression with [MAL](../../concepts-and-designs/mal.md) to filter/calculate/aggregate and store the results.
+**For Telegraf receiver:**
+1. The InfluxDB Telegraf [input plugins](https://docs.influxdata.com/telegraf/v1.24/plugins/) collects various metrics data from the VMs.
+2. The InfluxDB Telegraf send `JSON` format metrics by `HTTP` messages to Telegraf Receiver, then pushes converted metrics to the SkyWalking OAP Server [Meter System](./../../concepts-and-designs/meter.md).
+3. The SkyWalking OAP Server parses the expression with [MAL](../../concepts-and-designs/mal.md) to filter/calculate/aggregate and store the results.
## Setup
-
+**For OpenTelemetry receiver:**
1. Setup [Prometheus node-exporter](https://prometheus.io/docs/guides/node-exporter/).
2. Setup [OpenTelemetry Collector ](https://opentelemetry.io/docs/collector/). This is an example for OpenTelemetry Collector configuration [otel-collector-config.yaml](../../../../test/e2e-v2/cases/vm/prometheus-node-exporter/otel-collector-config.yaml).
3. Config SkyWalking [OpenTelemetry receiver](opentelemetry-receiver.md).
+**For Telegraf receiver:**
+1. Setup InfluxDB Telegraf's `telegraf.conf file` according to [Telegraf office document](https://docs.influxdata.com/telegraf/v1.24/).
+2. Setup InfluxDB Telegraf's `telegraf.conf file` specific rules according to [Telegraf receiver document](telegraf-receiver.md).
+3. Config SkyWalking [Telegraf receiver](telegraf-receiver.md).
+
## Supported Metrics
+**For OpenTelemetry receiver:**
Review Comment:
Metrics name for UI, or metrics name from telegraf?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] soander commented on pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
soander commented on PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#issuecomment-1258868199
> One important thing, please make sure your codes could pass all code checks, UT, and e2e tests before request reviewing. Otherwise, we may even read the wrong codes, which doesn't make sure to make many reviewers put time into a PR.
I'm sorry for the inconvenience. How can I do the code checks, UT, and e2e tests locally? Thank you for your guidanceοΌ
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r1009081084
##########
docs/en/setup/backend/telegraf-receiver.md:
##########
@@ -0,0 +1,45 @@
+# Telegraf receiver
Review Comment:
Metrics receiver doc should be hosted in the menu like this
<img width="350" alt="image" src="https://user-images.githubusercontent.com/5441976/198949306-a7e2e327-4458-4d15-88ac-36fbe0b1ce14.png">
You should update the menu.yml in the docs folder.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] kezhenxu94 commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
kezhenxu94 commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r979163885
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/handler/TelegrafServiceHandler.java:
##########
@@ -0,0 +1,128 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.handler;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.google.common.collect.ImmutableMap;
+import com.linecorp.armeria.server.annotation.Post;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.skywalking.apm.network.common.v3.Commands;
+import org.apache.skywalking.oap.meter.analyzer.MetricConvert;
+import org.apache.skywalking.oap.meter.analyzer.dsl.Sample;
+import org.apache.skywalking.oap.meter.analyzer.dsl.SampleFamily;
+import org.apache.skywalking.oap.meter.analyzer.dsl.SampleFamilyBuilder;
+import org.apache.skywalking.oap.server.core.analysis.meter.MeterSystem;
+import org.apache.skywalking.oap.server.library.module.ModuleManager;
+import org.apache.skywalking.oap.server.receiver.telegraf.provider.config.TelegrafConfig;
+import org.apache.skywalking.oap.server.receiver.telegraf.provider.handler.pojo.TelegrafData;
+import org.apache.skywalking.oap.server.telemetry.TelemetryModule;
+import org.apache.skywalking.oap.server.telemetry.api.CounterMetrics;
+import org.apache.skywalking.oap.server.telemetry.api.HistogramMetrics;
+import org.apache.skywalking.oap.server.telemetry.api.MetricsCreator;
+import org.apache.skywalking.oap.server.telemetry.api.MetricsTag;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+@Slf4j
+public class TelegrafServiceHandler {
+
+ private final HistogramMetrics histogram;
+ private final CounterMetrics errorCounter;
+ private List<MetricConvert> metricConvert;
+
+ public TelegrafServiceHandler(ModuleManager moduleManager, MeterSystem meterSystem, List<TelegrafConfig> rules) {
+
+ this.metricConvert = rules.stream().map(r -> new MetricConvert(r, meterSystem)).collect(Collectors.toList());
+
+ final MetricsCreator metricsCreator = moduleManager.find(TelemetryModule.NAME)
+ .provider()
+ .getService(MetricsCreator.class);
+
+ histogram = metricsCreator.createHistogramMetric(
+ "telegraf_in_latency", "The process latency of telegraf data",
+ new MetricsTag.Keys("protocol"), new MetricsTag.Values("http")
+ );
+
+ errorCounter = metricsCreator.createCounter(
+ "telegraf_error_count", "The error number of telegraf analysis",
+ new MetricsTag.Keys("protocol"), new MetricsTag.Values("http")
+ );
+ }
+
+ /**
+ * Convert the TelegrafData object to meter {@link Sample}
+ **/
+ public List<Sample> convertTelegraf(TelegrafData telegrafData) {
+
+ List<Sample> sampleList = new ArrayList<>(Collections.emptyList());
+
+ Map<String, Object> fields = telegrafData.getFields();
+ String name = telegrafData.getName();
+ Map<String, String> tags = telegrafData.getTags();
+ ImmutableMap<String, String> immutableTags = ImmutableMap.copyOf(tags);
+ long timestamp = telegrafData.getTimestamp();
+
+ fields.forEach((key, value) -> {
+ Sample.SampleBuilder builder = Sample.builder();
+ if (value instanceof String) return;
+ Sample sample = builder.name(name + "_" + key)
+ .timestamp(timestamp * 1000L)
+ .value(((Number) value).doubleValue())
+ .labels(immutableTags).build();
+
+ sampleList.add(sample);
+ });
+ return sampleList;
+
+ }
+
+ @Post("/telegraf")
+ public Commands collectData(String jsonInfo) {
+ TelegrafData telegrafData;
+ try (HistogramMetrics.Timer ignored = histogram.createTimer()) {
+ List<Sample> allSamples = new ArrayList<>();
+ ObjectMapper mapper = new ObjectMapper();
+ JsonNode node = mapper.readTree(jsonInfo);
Review Comment:
Not sure whether it's necessary that class `TelegrafData` should implement `RequestConverterFunction`, this is a standard POJO and the Armeria should be able to deserialize it correctly
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r979607620
##########
docs/en/changes/changes.md:
##########
@@ -20,6 +20,7 @@
in `no-init` mode.
* Make sure instance list ordered in TTL processor to avoid TTL timer never runs.
* Support monitoring PostgreSQL slow SQLs.
+* Support Telegraf receiver plugin module.
Review Comment:
```suggestion
* Support Telegraf receiver plugin module.
* Support monitoring Linux OS through Telegraf.
```
Also, you should update two docs, and update change logs accordingly too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#issuecomment-1257553578
One important thing, please make sure your codes could pass all code checks, UT, and e2e tests before request reviewing. Otherwise, we may even read the wrong codes, which doesn't make sure to make many reviewers put time into a PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] liuhaoyang commented on pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
liuhaoyang commented on PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#issuecomment-1296078804
> Has your mentor reviewed, checked and tested this?
Yes. I have reviewed and performed the tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r1009078644
##########
oap-server/server-starter/src/main/resources/ui-initialized-templates/os_linux/linux-service.json:
##########
@@ -156,10 +156,12 @@
"showYAxis": true
},
"metrics": [
- "meter_vm_filesystem_percentage"
+ "meter_vm_filesystem_percentage",
+ "meter_vm_filesystem_percentage_telegraf"
],
"metricTypes": [
- "readLabeledMetricsValues"
+ "readLabeledMetricsValues",
+ "readMetricsValues"
Review Comment:
@Fine0830 I noticed this kind of hybrid setting, one for labeled metrics, the other for a single value. Is this supported in the plan?
@soander Please post screenshot about how it looks like in OpenTelemetry and Telegraf cases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] kezhenxu94 commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
kezhenxu94 commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r1013521496
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/TelegrafReceiverProvider.java:
##########
@@ -0,0 +1,110 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider;
+
+import com.google.common.base.Splitter;
+import com.linecorp.armeria.common.HttpMethod;
+import org.apache.skywalking.oap.meter.analyzer.prometheus.rule.Rule;
+import org.apache.skywalking.oap.meter.analyzer.prometheus.rule.Rules;
+import org.apache.skywalking.oap.server.core.CoreModule;
+import org.apache.skywalking.oap.server.core.analysis.meter.MeterSystem;
+import org.apache.skywalking.oap.server.core.server.HTTPHandlerRegister;
+import org.apache.skywalking.oap.server.library.module.ModuleDefine;
+import org.apache.skywalking.oap.server.library.module.ModuleProvider;
+import org.apache.skywalking.oap.server.library.module.ModuleStartException;
+import org.apache.skywalking.oap.server.library.module.ServiceNotProvidedException;
+import org.apache.skywalking.oap.server.library.util.CollectionUtils;
+import org.apache.skywalking.oap.server.library.util.StringUtil;
+import org.apache.skywalking.oap.server.receiver.sharing.server.SharingServerModule;
+import org.apache.skywalking.oap.server.receiver.telegraf.module.TelegrafReceiverModule;
+import org.apache.skywalking.oap.server.receiver.telegraf.provider.handler.TelegrafServiceHandler;
+
+import java.io.IOException;
+import java.util.Collections;
+import java.util.List;
+
+public class TelegrafReceiverProvider extends ModuleProvider {
+ private List<Rule> configs;
+ private TelegrafModuleConfig moduleConfig;
+
+ public TelegrafReceiverProvider() {
+ this.moduleConfig = new TelegrafModuleConfig();
+ }
+
Review Comment:
No need
```suggestion
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] soander commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
soander commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r973544221
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/handler/TelegrafServiceHandler.java:
##########
@@ -0,0 +1,128 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.handler;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.google.common.collect.ImmutableMap;
+import com.linecorp.armeria.server.annotation.Post;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.skywalking.apm.network.common.v3.Commands;
+import org.apache.skywalking.oap.meter.analyzer.MetricConvert;
+import org.apache.skywalking.oap.meter.analyzer.dsl.Sample;
+import org.apache.skywalking.oap.meter.analyzer.dsl.SampleFamily;
+import org.apache.skywalking.oap.meter.analyzer.dsl.SampleFamilyBuilder;
+import org.apache.skywalking.oap.server.core.analysis.meter.MeterSystem;
+import org.apache.skywalking.oap.server.library.module.ModuleManager;
+import org.apache.skywalking.oap.server.receiver.telegraf.provider.config.TelegrafConfig;
+import org.apache.skywalking.oap.server.receiver.telegraf.provider.handler.pojo.TelegrafData;
+import org.apache.skywalking.oap.server.telemetry.TelemetryModule;
+import org.apache.skywalking.oap.server.telemetry.api.CounterMetrics;
+import org.apache.skywalking.oap.server.telemetry.api.HistogramMetrics;
+import org.apache.skywalking.oap.server.telemetry.api.MetricsCreator;
+import org.apache.skywalking.oap.server.telemetry.api.MetricsTag;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+@Slf4j
+public class TelegrafServiceHandler {
+
+ private final HistogramMetrics histogram;
+ private final CounterMetrics errorCounter;
+ private List<MetricConvert> metricConvert;
+
+ public TelegrafServiceHandler(ModuleManager moduleManager, MeterSystem meterSystem, List<TelegrafConfig> rules) {
+
+ this.metricConvert = rules.stream().map(r -> new MetricConvert(r, meterSystem)).collect(Collectors.toList());
+
+ final MetricsCreator metricsCreator = moduleManager.find(TelemetryModule.NAME)
+ .provider()
+ .getService(MetricsCreator.class);
+
+ histogram = metricsCreator.createHistogramMetric(
+ "telegraf_in_latency", "The process latency of telegraf data",
+ new MetricsTag.Keys("protocol"), new MetricsTag.Values("http")
+ );
+
+ errorCounter = metricsCreator.createCounter(
+ "telegraf_error_count", "The error number of telegraf analysis",
+ new MetricsTag.Keys("protocol"), new MetricsTag.Values("http")
+ );
+ }
+
+ /**
+ * Convert the TelegrafData object to meter {@link Sample}
+ **/
+ public List<Sample> convertTelegraf(TelegrafData telegrafData) {
+
+ List<Sample> sampleList = new ArrayList<>(Collections.emptyList());
+
+ Map<String, Object> fields = telegrafData.getFields();
+ String name = telegrafData.getName();
+ Map<String, String> tags = telegrafData.getTags();
+ ImmutableMap<String, String> immutableTags = ImmutableMap.copyOf(tags);
+ long timestamp = telegrafData.getTimestamp();
+
+ fields.forEach((key, value) -> {
+ Sample.SampleBuilder builder = Sample.builder();
+ if (value instanceof String) return;
Review Comment:
> Hi, could you provide some JSON data samples? I want to know more details about the convert logic. Such as why the value might be String.
The following data is JSON data samples:
`{"metrics":[{"fields":{"available":6047739904,"available_percent":35.41215070500567,"total":17078149120,"used":11030409216,"used_percent":64.58784929499433},"name":"mem","tags":{"host":"DESKTOP-S9HLAV6"},"timestamp":1663391390},{"fields":{"usage_guest":0,"usage_guest_nice":0,"usage_idle":95.32710280373831,"usage_iowait":0,"usage_irq":0.3115264797507788,"usage_nice":0,"usage_softirq":0,"usage_steal":0,"usage_system":1.7133956386292835,"usage_user":2.64797507788162},"name":"cpu","tags":{"cpu":"cpu0","host":"DESKTOP-S9HLAV6"},"timestamp":1663391390}]}`
The convert logic is build a sample by iterating every pair of key and value. Then adding the single simple to the sample list. Maybe the value will never be String, so the line of code is probably useless.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wankai123 commented on pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
wankai123 commented on PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#issuecomment-1303299945
The others LGTM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r1012569228
##########
oap-server/server-starter/src/main/resources/telegraf-rules/vm.yaml:
##########
@@ -0,0 +1,56 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+expSuffix: service(['host'], Layer.OS_LINUX)
+metricPrefix: meter_vm
+metricsRules:
+
+ # cpu
+ - name: cpu_total_percentage
+ exp: cpu_usage_user.tagEqual('cpu' , 'cpu-total')
+ - name: cpu_average_used_telegraf
+ exp: cpu_usage_user.tagEqual('cpu' , 'cpu-total')
+ - name: cpu_load1
+ exp: system_load1
+ - name: cpu_load5
+ exp: system_load5
+ - name: cpu_load15
+ exp: system_load15
+
+ # memory
+ - name: memory_total
+ exp: mem_total
+ - name: memory_available
+ exp: mem_available
+ - name: memory_used
+ exp: mem_used
+
+ # swap
+ - name: memory_swap_free
+ exp: swap_free
+ - name: memory_swap_total
+ exp: cpu_swap_total
+ - name: memory_swap_percentage
+ exp: 100 - ((swap_free * 100) / cpu_swap_total)
+
+ #node filesystem
+ - name: filesystem_percentage_telegraf
+ exp: disk_used_percent
Review Comment:
Too much data could cause tests unstable because GHA only provides 3U8G to the whole test.
For the screenshot, the biggest issue is, SampleFamily should have a concept of the group for the bulk. We should not face one SampleFamily have metrics belonging to the same metric and labels, but with multiple timestamps.
Otherwise, the aggregation result would be strange and unpredictable.
Could you share how you process this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#issuecomment-1258873548
It is better you could ask for your mentor, as this is a Summer 2022 Program.
Generally, there are
- All tests are controlled through GitHub Action, https://github.com/apache/skywalking/tree/master/.github/workflows
- About e2e tests, https://skywalking.apache.org/docs/main/next/en/guides/readme/#end-to-end-tests-e2e
For merging this in upstream, testing is a very important, also time costing task. It may take more time than you wrote this feature. But this is how the open-source community works, in order to collaborate with people remotely.
Be patient.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] sonatype-lift[bot] commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
sonatype-lift[bot] commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r996724565
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/config/TelegrafConfigs.java:
##########
@@ -0,0 +1,76 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.config;
+
+import lombok.extern.slf4j.Slf4j;
+import org.apache.skywalking.oap.server.library.module.ModuleStartException;
+import org.apache.skywalking.oap.server.library.util.CollectionUtils;
+import org.apache.skywalking.oap.server.library.util.ResourceUtils;
+import org.yaml.snakeyaml.Yaml;
+
+import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.FileReader;
+import java.io.IOException;
+import java.io.Reader;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Objects;
+import java.util.stream.Collectors;
+
+@Slf4j
+public class TelegrafConfigs {
+ public static List<TelegrafConfig> loadConfigs(String path, List<String> fileNames) throws ModuleStartException {
+ if (CollectionUtils.isEmpty(fileNames)) {
+ return Collections.emptyList();
+ }
+
+ File[] configs;
+ try {
+ configs = ResourceUtils.getPathFiles(path);
+ } catch (FileNotFoundException e) {
+ throw new ModuleStartException("Load telegraf configs failed", e);
+ }
+
+ return Arrays.stream(configs)
+ .filter(File::isFile)
+ .map(f -> {
+ String fileName = f.getName();
+ int dotIndex = fileName.lastIndexOf('.');
+ fileName = (dotIndex == -1) ? fileName : fileName.substring(0, dotIndex);
+ if (!fileNames.contains(fileName)) {
+ return null;
+ }
+ try (Reader r = new FileReader(f)) {
+ return new Yaml().loadAs(r, TelegrafConfig.class);
+ } catch (IOException e) {
+ log.warn("Reading file {} failed", f, e);
+ try {
+ throw new ModuleStartException(e.getMessage(), e);
+ } catch (ModuleStartException ex) {
+ ex.printStackTrace();
Review Comment:
*[CatchAndPrintStackTrace](https://errorprone.info/bugpattern/CatchAndPrintStackTrace):* Logging or rethrowing exceptions should usually be preferred to catching and calling printStackTrace
---
<details><summary><b>βΉοΈ Learn about @sonatype-lift commands</b></summary>
You can reply with the following commands. For example, reply with ***@sonatype-lift ignoreall*** to leave out all findings.
| **Command** | **Usage** |
| ------------- | ------------- |
| `@sonatype-lift ignore` | Leave out the above finding from this PR |
| `@sonatype-lift ignoreall` | Leave out all the existing findings from this PR |
| `@sonatype-lift exclude <file\|issue\|path\|tool>` | Exclude specified `file\|issue\|path\|tool` from Lift findings by updating your config.toml file |
**Note:** When talking to LiftBot, you need to **refresh** the page to see its response.
<sub>[Click here](https://github.com/apps/sonatype-lift/installations/new) to add LiftBot to another repo.</sub></details>
---
Was this a good recommendation?
[ [π Not relevant](https://www.sonatype.com/lift-comment-rating?comment=345569012&lift_comment_rating=1) ] - [ [π Won't fix](https://www.sonatype.com/lift-comment-rating?comment=345569012&lift_comment_rating=2) ] - [ [π Not critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=345569012&lift_comment_rating=3) ] - [ [π Critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=345569012&lift_comment_rating=4) ] - [ [π Critical, fixing now](https://www.sonatype.com/lift-comment-rating?comment=345569012&lift_comment_rating=5) ]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] soander commented on pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
soander commented on PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#issuecomment-1285070081
> 4 files left for this PR? What happens?
I made a wrong push...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r1000257309
##########
docs/en/setup/backend/backend-vm-monitoring.md:
##########
@@ -3,38 +3,63 @@ SkyWalking leverages Prometheus node-exporter to collect metrics data from the V
[OpenTelemetry receiver](opentelemetry-receiver.md) and into the [Meter System](./../../concepts-and-designs/meter.md).
VM entity as a `Service` in OAP and on the `Layer: OS_LINUX`.
+SkyWalking also provides InfluxDB Telegraf to receive VMs' metrics data by [Telegraf receiver](./telegraf-receiver.md).
+The telegraf receiver plugin receiver, process and convert the metrics, then it send converted metrics to [Meter System](./../../concepts-and-designs/meter.md).
+VM entity as a `Service` in OAP and on the `Layer: OS_LINUX`.
+
## Data flow
+**For OpenTelemetry receiver:**
1. The Prometheus node-exporter collects metrics data from the VMs.
2. The OpenTelemetry Collector fetches metrics from node-exporter via Prometheus Receiver and pushes metrics to the SkyWalking OAP Server via the OpenCensus gRPC Exporter or OpenTelemetry gRPC exporter.
3. The SkyWalking OAP Server parses the expression with [MAL](../../concepts-and-designs/mal.md) to filter/calculate/aggregate and store the results.
+**For Telegraf receiver:**
+1. The InfluxDB Telegraf [input plugins](https://docs.influxdata.com/telegraf/v1.24/plugins/) collects various metrics data from the VMs.
+2. The InfluxDB Telegraf send `JSON` format metrics by `HTTP` messages to Telegraf Receiver, then pushes converted metrics to the SkyWalking OAP Server [Meter System](./../../concepts-and-designs/meter.md).
+3. The SkyWalking OAP Server parses the expression with [MAL](../../concepts-and-designs/mal.md) to filter/calculate/aggregate and store the results.
## Setup
-
+**For OpenTelemetry receiver:**
1. Setup [Prometheus node-exporter](https://prometheus.io/docs/guides/node-exporter/).
2. Setup [OpenTelemetry Collector ](https://opentelemetry.io/docs/collector/). This is an example for OpenTelemetry Collector configuration [otel-collector-config.yaml](../../../../test/e2e-v2/cases/vm/prometheus-node-exporter/otel-collector-config.yaml).
3. Config SkyWalking [OpenTelemetry receiver](opentelemetry-receiver.md).
+**For Telegraf receiver:**
+1. Setup InfluxDB Telegraf's `telegraf.conf file` according to [Telegraf office document](https://docs.influxdata.com/telegraf/v1.24/).
+2. Setup InfluxDB Telegraf's `telegraf.conf file` specific rules according to [Telegraf receiver document](telegraf-receiver.md).
+3. Config SkyWalking [Telegraf receiver](telegraf-receiver.md).
+
## Supported Metrics
+**For OpenTelemetry receiver:**
Review Comment:
If they are same, keep dataflow doc is fine, but you don't have to list two sheets for the same metrics. Right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] soander closed pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
soander closed pull request #9620: Support the telegraf receiver plugin module
URL: https://github.com/apache/skywalking/pull/9620
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r977061479
##########
oap-server/server-starter/src/main/resources/telegraf-rules/default.yaml:
##########
@@ -0,0 +1,50 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+expSuffix: service(['host'], Layer.OS_LINUX)
Review Comment:
If it can't for now, please let me know. Unless there are differences at aggregation function level, or downsampling. We should be able to.
Otherwise, this configurations(rules) should be in default release. Because there is no dashboard for it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] soander commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
soander commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r973548214
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/config/TelegrafConfigs.java:
##########
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.config;
+
+import lombok.extern.slf4j.Slf4j;
+import org.apache.skywalking.oap.server.library.module.ModuleStartException;
+import org.apache.skywalking.oap.server.library.util.CollectionUtils;
+import org.apache.skywalking.oap.server.library.util.ResourceUtils;
+import org.yaml.snakeyaml.Yaml;
+
+import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.FileReader;
+import java.io.IOException;
+import java.io.Reader;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Objects;
+import java.util.stream.Collectors;
+
+@Slf4j
+public class TelegrafConfigs {
+ public static List<TelegrafConfig> loadConfigs(String path, List<String> fileNames) throws ModuleStartException {
+ if (CollectionUtils.isEmpty(fileNames)) {
+ return Collections.emptyList();
+ }
+
+ File[] configs;
+ try {
+ configs = ResourceUtils.getPathFiles(path);
+ } catch (FileNotFoundException e) {
+ throw new ModuleStartException("Load telegraf configs failed", e);
+ }
+
+ return Arrays.stream(configs)
+ .filter(File::isFile)
+ .map(f -> {
+ String fileName = f.getName();
+ int dotIndex = fileName.lastIndexOf('.');
+ fileName = (dotIndex == -1) ? fileName : fileName.substring(0, dotIndex);
+ if (!fileNames.contains(fileName)) {
+ return null;
+ }
+ try (Reader r = new FileReader(f)) {
+ return new Yaml().loadAs(r, TelegrafConfig.class);
+ } catch (IOException e) {
+ log.warn("Reading file {} failed", f, e);
Review Comment:
> Please throw module start exception, this is a fatal error
I set the `loadConfigs` method throws the `ModuleStartException` , should I change it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] sonatype-lift[bot] commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
sonatype-lift[bot] commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r1000278856
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/config/TelegrafConfig.java:
##########
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.config;
+
+import lombok.Data;
+import lombok.NoArgsConstructor;
+import org.apache.skywalking.oap.meter.analyzer.MetricRuleConfig;
+
+import java.util.List;
+
+@Data
+public class TelegrafConfig implements MetricRuleConfig {
+
+ private String metricPrefix;
+ private String expSuffix;
+ private String expPrefix;
Review Comment:
*[MissingOverride](https://errorprone.info/bugpattern/MissingOverride):* getExpPrefix implements method in MetricRuleConfig; expected @Override
---
```suggestion
private String @Override expPrefix;
```
---
<details><summary><b>βΉοΈ Learn about @sonatype-lift commands</b></summary>
You can reply with the following commands. For example, reply with ***@sonatype-lift ignoreall*** to leave out all findings.
| **Command** | **Usage** |
| ------------- | ------------- |
| `@sonatype-lift ignore` | Leave out the above finding from this PR |
| `@sonatype-lift ignoreall` | Leave out all the existing findings from this PR |
| `@sonatype-lift exclude <file\|issue\|path\|tool>` | Exclude specified `file\|issue\|path\|tool` from Lift findings by updating your config.toml file |
**Note:** When talking to LiftBot, you need to **refresh** the page to see its response.
<sub>[Click here](https://github.com/apps/sonatype-lift/installations/new) to add LiftBot to another repo.</sub></details>
---
Was this a good recommendation?
[ [π Not relevant](https://www.sonatype.com/lift-comment-rating?comment=346698900&lift_comment_rating=1) ] - [ [π Won't fix](https://www.sonatype.com/lift-comment-rating?comment=346698900&lift_comment_rating=2) ] - [ [π Not critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=346698900&lift_comment_rating=3) ] - [ [π Critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=346698900&lift_comment_rating=4) ] - [ [π Critical, fixing now](https://www.sonatype.com/lift-comment-rating?comment=346698900&lift_comment_rating=5) ]
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/config/TelegrafConfig.java:
##########
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.config;
+
+import lombok.Data;
+import lombok.NoArgsConstructor;
+import org.apache.skywalking.oap.meter.analyzer.MetricRuleConfig;
+
+import java.util.List;
+
+@Data
+public class TelegrafConfig implements MetricRuleConfig {
+
+ private String metricPrefix;
+ private String expSuffix;
+ private String expPrefix;
+ private String filter;
+ private String initExp;
Review Comment:
*[MissingOverride](https://errorprone.info/bugpattern/MissingOverride):* getInitExp implements method in MetricRuleConfig; expected @Override
---
```suggestion
private String @Override initExp;
```
---
<details><summary><b>βΉοΈ Learn about @sonatype-lift commands</b></summary>
You can reply with the following commands. For example, reply with ***@sonatype-lift ignoreall*** to leave out all findings.
| **Command** | **Usage** |
| ------------- | ------------- |
| `@sonatype-lift ignore` | Leave out the above finding from this PR |
| `@sonatype-lift ignoreall` | Leave out all the existing findings from this PR |
| `@sonatype-lift exclude <file\|issue\|path\|tool>` | Exclude specified `file\|issue\|path\|tool` from Lift findings by updating your config.toml file |
**Note:** When talking to LiftBot, you need to **refresh** the page to see its response.
<sub>[Click here](https://github.com/apps/sonatype-lift/installations/new) to add LiftBot to another repo.</sub></details>
---
Was this a good recommendation?
[ [π Not relevant](https://www.sonatype.com/lift-comment-rating?comment=346698971&lift_comment_rating=1) ] - [ [π Won't fix](https://www.sonatype.com/lift-comment-rating?comment=346698971&lift_comment_rating=2) ] - [ [π Not critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=346698971&lift_comment_rating=3) ] - [ [π Critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=346698971&lift_comment_rating=4) ] - [ [π Critical, fixing now](https://www.sonatype.com/lift-comment-rating?comment=346698971&lift_comment_rating=5) ]
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/config/TelegrafConfig.java:
##########
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.config;
+
+import lombok.Data;
+import lombok.NoArgsConstructor;
+import org.apache.skywalking.oap.meter.analyzer.MetricRuleConfig;
+
+import java.util.List;
+
+@Data
+public class TelegrafConfig implements MetricRuleConfig {
+
+ private String metricPrefix;
+ private String expSuffix;
+ private String expPrefix;
+ private String filter;
+ private String initExp;
+ private List<Rule> metricsRules;
Review Comment:
*[MissingOverride](https://errorprone.info/bugpattern/MissingOverride):* getMetricsRules implements method in MetricRuleConfig; expected @Override
---
```suggestion
private List<Rule> @Override metricsRules;
```
---
<details><summary><b>βΉοΈ Learn about @sonatype-lift commands</b></summary>
You can reply with the following commands. For example, reply with ***@sonatype-lift ignoreall*** to leave out all findings.
| **Command** | **Usage** |
| ------------- | ------------- |
| `@sonatype-lift ignore` | Leave out the above finding from this PR |
| `@sonatype-lift ignoreall` | Leave out all the existing findings from this PR |
| `@sonatype-lift exclude <file\|issue\|path\|tool>` | Exclude specified `file\|issue\|path\|tool` from Lift findings by updating your config.toml file |
**Note:** When talking to LiftBot, you need to **refresh** the page to see its response.
<sub>[Click here](https://github.com/apps/sonatype-lift/installations/new) to add LiftBot to another repo.</sub></details>
---
Was this a good recommendation?
[ [π Not relevant](https://www.sonatype.com/lift-comment-rating?comment=346699000&lift_comment_rating=1) ] - [ [π Won't fix](https://www.sonatype.com/lift-comment-rating?comment=346699000&lift_comment_rating=2) ] - [ [π Not critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=346699000&lift_comment_rating=3) ] - [ [π Critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=346699000&lift_comment_rating=4) ] - [ [π Critical, fixing now](https://www.sonatype.com/lift-comment-rating?comment=346699000&lift_comment_rating=5) ]
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/config/TelegrafConfig.java:
##########
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.config;
+
+import lombok.Data;
+import lombok.NoArgsConstructor;
+import org.apache.skywalking.oap.meter.analyzer.MetricRuleConfig;
+
+import java.util.List;
+
+@Data
+public class TelegrafConfig implements MetricRuleConfig {
+
+ private String metricPrefix;
+ private String expSuffix;
Review Comment:
*[MissingOverride](https://errorprone.info/bugpattern/MissingOverride):* getExpSuffix implements method in MetricRuleConfig; expected @Override
---
```suggestion
private String @Override expSuffix;
```
---
<details><summary><b>βΉοΈ Learn about @sonatype-lift commands</b></summary>
You can reply with the following commands. For example, reply with ***@sonatype-lift ignoreall*** to leave out all findings.
| **Command** | **Usage** |
| ------------- | ------------- |
| `@sonatype-lift ignore` | Leave out the above finding from this PR |
| `@sonatype-lift ignoreall` | Leave out all the existing findings from this PR |
| `@sonatype-lift exclude <file\|issue\|path\|tool>` | Exclude specified `file\|issue\|path\|tool` from Lift findings by updating your config.toml file |
**Note:** When talking to LiftBot, you need to **refresh** the page to see its response.
<sub>[Click here](https://github.com/apps/sonatype-lift/installations/new) to add LiftBot to another repo.</sub></details>
---
Was this a good recommendation?
[ [π Not relevant](https://www.sonatype.com/lift-comment-rating?comment=346699044&lift_comment_rating=1) ] - [ [π Won't fix](https://www.sonatype.com/lift-comment-rating?comment=346699044&lift_comment_rating=2) ] - [ [π Not critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=346699044&lift_comment_rating=3) ] - [ [π Critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=346699044&lift_comment_rating=4) ] - [ [π Critical, fixing now](https://www.sonatype.com/lift-comment-rating?comment=346699044&lift_comment_rating=5) ]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] sonatype-lift[bot] commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
sonatype-lift[bot] commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r972876467
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/config/TelegrafConfig.java:
##########
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.config;
+
+import lombok.Data;
+import lombok.NoArgsConstructor;
+import org.apache.skywalking.oap.meter.analyzer.MetricRuleConfig;
+
+import java.util.List;
+
+@Data
+public class TelegrafConfig implements MetricRuleConfig {
+
+ private String metricPrefix;
Review Comment:
*[MissingOverride](https://errorprone.info/bugpattern/MissingOverride):* getMetricPrefix implements method in MetricRuleConfig; expected @Override
---
```suggestion
private String @Override metricPrefix;
```
---
<details><summary><b>βΉοΈ Learn about @sonatype-lift commands</b></summary>
You can reply with the following commands. For example, reply with ***@sonatype-lift ignoreall*** to leave out all findings.
| **Command** | **Usage** |
| ------------- | ------------- |
| `@sonatype-lift ignore` | Leave out the above finding from this PR |
| `@sonatype-lift ignoreall` | Leave out all the existing findings from this PR |
| `@sonatype-lift exclude <file\|issue\|path\|tool>` | Exclude specified `file\|issue\|path\|tool` from Lift findings by updating your config.toml file |
**Note:** When talking to LiftBot, you need to **refresh** the page to see its response.
<sub>[Click here](https://github.com/apps/sonatype-lift/installations/new) to add LiftBot to another repo.</sub></details>
---
Was this a good recommendation?
[ [π Not relevant](https://www.sonatype.com/lift-comment-rating?comment=333280509&lift_comment_rating=1) ] - [ [π Won't fix](https://www.sonatype.com/lift-comment-rating?comment=333280509&lift_comment_rating=2) ] - [ [π Not critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=333280509&lift_comment_rating=3) ] - [ [π Critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=333280509&lift_comment_rating=4) ] - [ [π Critical, fixing now](https://www.sonatype.com/lift-comment-rating?comment=333280509&lift_comment_rating=5) ]
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/config/TelegrafConfig.java:
##########
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.config;
+
+import lombok.Data;
+import lombok.NoArgsConstructor;
+import org.apache.skywalking.oap.meter.analyzer.MetricRuleConfig;
+
+import java.util.List;
+
+@Data
+public class TelegrafConfig implements MetricRuleConfig {
+
+ private String metricPrefix;
+ private String expSuffix;
+ private String expPrefix;
+ private String filter;
+ private String initExp;
Review Comment:
*[MissingOverride](https://errorprone.info/bugpattern/MissingOverride):* getInitExp implements method in MetricRuleConfig; expected @Override
---
```suggestion
private String @Override initExp;
```
---
<details><summary><b>βΉοΈ Learn about @sonatype-lift commands</b></summary>
You can reply with the following commands. For example, reply with ***@sonatype-lift ignoreall*** to leave out all findings.
| **Command** | **Usage** |
| ------------- | ------------- |
| `@sonatype-lift ignore` | Leave out the above finding from this PR |
| `@sonatype-lift ignoreall` | Leave out all the existing findings from this PR |
| `@sonatype-lift exclude <file\|issue\|path\|tool>` | Exclude specified `file\|issue\|path\|tool` from Lift findings by updating your config.toml file |
**Note:** When talking to LiftBot, you need to **refresh** the page to see its response.
<sub>[Click here](https://github.com/apps/sonatype-lift/installations/new) to add LiftBot to another repo.</sub></details>
---
Was this a good recommendation?
[ [π Not relevant](https://www.sonatype.com/lift-comment-rating?comment=333280641&lift_comment_rating=1) ] - [ [π Won't fix](https://www.sonatype.com/lift-comment-rating?comment=333280641&lift_comment_rating=2) ] - [ [π Not critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=333280641&lift_comment_rating=3) ] - [ [π Critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=333280641&lift_comment_rating=4) ] - [ [π Critical, fixing now](https://www.sonatype.com/lift-comment-rating?comment=333280641&lift_comment_rating=5) ]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] liuhaoyang commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
liuhaoyang commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r1011953830
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/test/java/org/apache/skywalking/oap/server/receiver/telegraf/JSONConvertTest.java:
##########
@@ -0,0 +1,126 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf;
+
+import com.google.common.collect.ImmutableMap;
+import lombok.Builder;
+import lombok.EqualsAndHashCode;
+import lombok.ToString;
+import lombok.Getter;
+import lombok.AllArgsConstructor;
+import org.apache.skywalking.oap.server.receiver.telegraf.provider.handler.TelegrafServiceHandler;
+import org.apache.skywalking.oap.server.receiver.telegraf.provider.handler.pojo.TelegrafDatum;
+import org.junit.Before;
+import org.junit.Ignore;
+import org.junit.Test;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import static org.junit.Assert.assertEquals;
+
+public class JSONConvertTest {
Review Comment:
> Could you check `ZabbixMetricsTest` and `ZabbixProtocolHandlerTest` about how to do UT? What we need to test is that, we have a JSON text(same format as telegraf reported, that could link to a doc), then, the UT calls some methods in your handler(or something here). Then we could verify the sample family with expectations.
>
> In the current one, you seem using a mock class to test another mock class.
>
> cc @kezhenxu94
I noticed that @soander added unit tests today, cc @wu-sheng
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r1008881524
##########
.mvn/wrapper/maven-wrapper.properties:
##########
@@ -5,14 +5,14 @@
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
-#
+#
# http://www.apache.org/licenses/LICENSE-2.0
-#
+#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
distributionUrl=https://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/3.8.4/apache-maven-3.8.4-bin.zip
-wrapperUrl=https://repo.maven.apache.org/maven2/org/apache/maven/wrapper/maven-wrapper/3.1.0/maven-wrapper-3.1.0.jar
+wrapperUrl=https://repo.maven.apache.org/maven2/org/apache/maven/wrapper/maven-wrapper/3.1.0/maven-wrapper-3.1.0.jar
Review Comment:
There are unexpected changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#issuecomment-1296065469
Has your mentor reviewed, checked and tested this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] soander commented on pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
soander commented on PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#issuecomment-1296405145
> I still noticed there are several mentioned things not been fixed.
>
> 1. There is no unit test to cover telegraf format converting. `TelegrafServiceHandler` is a HTTP handler. What it received is an HTTP text, what it looks like, and how it could be converted to SampleFamily should be well tested, including legal text or illegal text cases.
> 2. You don't explain what is the relationship between this VM monitoring and OpenTelemetry's Linux monitoring.
> 3. In [backend-vm-monitoring.md](https://github.com/apache/skywalking/pull/9620/files#diff-9b751b71ba658be11f59887b6462a97bd3675a181c3cd3fcf7d5c4e3d452c295), you mentioned, there are two ways to support VM monitoring. But they seem not the same, such as not using the same metric name. So, how the dashboard would work? In the document, there is not any document mentioning that.
>
> I still can't follow the whole picture of this feature implementation. Could any of you explain it? We can't deliver this to the community with this confusion. Especially, `telegraf-rules/vm.yaml` is included as a part of the official release.
>
> All default activated MAL scripts match with the dashboard settings, as well as the document description about the data flow.
1. I will write the HTTP text converting to SampleFamily unit test.
2. Telegraf and OpenTelemetry are two different metrics collection methods, both of them can collect linux system metrics and display it on the UI.
3. The metrics are collected by telegraf can also be displayed on the VM monitoring dashboard. The default telegraf MAL will convert the telegraf metrics into VM metrics.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] soander commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
soander commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r1013611043
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/handler/TelegrafServiceHandler.java:
##########
@@ -0,0 +1,129 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.handler;
+
+import com.google.common.collect.ImmutableMap;
+import com.linecorp.armeria.server.annotation.Post;
+import com.linecorp.armeria.server.annotation.RequestConverter;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.skywalking.apm.network.common.v3.Commands;
+import org.apache.skywalking.oap.meter.analyzer.MetricConvert;
+import org.apache.skywalking.oap.meter.analyzer.dsl.Sample;
+import org.apache.skywalking.oap.meter.analyzer.dsl.SampleFamily;
+import org.apache.skywalking.oap.meter.analyzer.dsl.SampleFamilyBuilder;
+import org.apache.skywalking.oap.meter.analyzer.prometheus.rule.Rule;
+import org.apache.skywalking.oap.server.core.analysis.meter.MeterSystem;
+import org.apache.skywalking.oap.server.library.module.ModuleManager;
+import org.apache.skywalking.oap.server.receiver.telegraf.provider.handler.pojo.TelegrafData;
+import org.apache.skywalking.oap.server.receiver.telegraf.provider.handler.pojo.TelegrafDatum;
+import org.apache.skywalking.oap.server.telemetry.TelemetryModule;
+import org.apache.skywalking.oap.server.telemetry.api.CounterMetrics;
+import org.apache.skywalking.oap.server.telemetry.api.HistogramMetrics;
+import org.apache.skywalking.oap.server.telemetry.api.MetricsCreator;
+import org.apache.skywalking.oap.server.telemetry.api.MetricsTag;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+@Slf4j
+public class TelegrafServiceHandler {
+
+ private final HistogramMetrics histogram;
+ private final CounterMetrics errorCounter;
+ private List<MetricConvert> metricConvert;
+
+ public TelegrafServiceHandler(ModuleManager moduleManager, MeterSystem meterSystem, List<Rule> rules) {
+
+ this.metricConvert = rules.stream().map(r -> new MetricConvert(r, meterSystem)).collect(Collectors.toList());
+
+ final MetricsCreator metricsCreator = moduleManager.find(TelemetryModule.NAME)
+ .provider()
+ .getService(MetricsCreator.class);
+
+ histogram = metricsCreator.createHistogramMetric(
+ "telegraf_in_latency", "The process latency of telegraf data",
+ new MetricsTag.Keys("protocol"), new MetricsTag.Values("http")
+ );
+
+ errorCounter = metricsCreator.createCounter(
+ "telegraf_error_count", "The error number of telegraf analysis",
+ new MetricsTag.Keys("protocol"), new MetricsTag.Values("http")
+ );
+ }
+
+ /**
+ * Convert the TelegrafData object to meter {@link Sample}
+ **/
+ public List<Sample> convertTelegraf(TelegrafDatum telegrafData) {
+
+ List<Sample> sampleList = new ArrayList<>();
+
+ Map<String, Object> fields = telegrafData.getFields();
+ String name = telegrafData.getName();
+ Map<String, String> tags = telegrafData.getTags();
+ ImmutableMap<String, String> immutableTags = ImmutableMap.copyOf(tags);
+ long timestamp = telegrafData.getTimestamp();
+
+ fields.forEach((key, value) -> {
+ if (value instanceof Number) {
+ Sample.SampleBuilder builder = Sample.builder();
+ Sample sample = builder.name(name + "_" + key)
+ .timestamp(timestamp * 1000L)
+ .value(((Number) value).doubleValue())
+ .labels(immutableTags).build();
+
+ sampleList.add(sample);
+ }
+ });
+ return sampleList;
+
+ }
+
+ public ImmutableMap<String, SampleFamily> convertSampleFamily(TelegrafData telegrafData) {
+ List<Sample> allSamples = new ArrayList<>();
+
+ List<TelegrafDatum> metrics = telegrafData.getMetrics();
+ for (TelegrafDatum m : metrics) {
+ List<Sample> samples = convertTelegraf(m);
+ allSamples.addAll(samples);
+ }
+
+ // Grouping all samples by their name, then build sampleFamily
Review Comment:
> This policy seems not right. The samples should be grouped by bulk/sampling ID. Because
>
> 1. Metrics in the same name with different timestamps should belong to different sample families.
> 2. Different metrics(name) collected in the same timestamp(one collecting) should belong to the same sample family. You could check the MAL doc, which mentioned metrics should be calculated through one formula, ref to https://skywalking.apache.org/docs/main/next/en/concepts-and-designs/mal/#binary-operators, `instance_trace_analysis_error_count / instance_trace_count`
Got it. I will modify this part logic.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#issuecomment-1278692953
Marked this as `no update`, as we can't have any feedback for two weeks. I am not sure whether this is still on going or still prepares for upstream review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] kezhenxu94 commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
kezhenxu94 commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r1009103857
##########
oap-server/server-starter/src/main/resources/telegraf-rules/vm.yaml:
##########
@@ -0,0 +1,56 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+expSuffix: service(['host'], Layer.OS_LINUX)
+metricPrefix: meter_vm
+metricsRules:
+
+ # cpu
+ - name: cpu_total_percentage
+ exp: cpu_usage_user.tagEqual('cpu' , 'cpu-total')
+ - name: cpu_average_used_telegraf
+ exp: cpu_usage_user.tagEqual('cpu' , 'cpu-total')
Review Comment:
The `exp` is exactly the same as `cpu_total_percentage`, may be it's something like `cpu_usage_user.tagEqual('cpu' , 'cpu-used')`? Can you recheck
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/config/TelegrafConfigs.java:
##########
@@ -0,0 +1,70 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.config;
+
+import lombok.extern.slf4j.Slf4j;
+import org.apache.skywalking.oap.server.library.module.ModuleStartException;
+import org.apache.skywalking.oap.server.library.util.CollectionUtils;
+import org.apache.skywalking.oap.server.library.util.ResourceUtils;
+import org.yaml.snakeyaml.Yaml;
+
+import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.FileReader;
+import java.io.IOException;
+import java.io.Reader;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Objects;
+import java.util.stream.Collectors;
+
+@Slf4j
+public class TelegrafConfigs {
Review Comment:
This file is duplicate of `org.apache.skywalking.oap.meter.analyzer.prometheus.rule.Rules`
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/config/TelegrafConfig.java:
##########
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.config;
+
+import lombok.Data;
+import lombok.NoArgsConstructor;
+import org.apache.skywalking.oap.meter.analyzer.MetricRuleConfig;
+
+import java.util.List;
+
+@Data
+public class TelegrafConfig implements MetricRuleConfig {
+
+ private String metricPrefix;
+ private String expSuffix;
+ private String expPrefix;
+ private String filter;
+ private String initExp;
+ private List<Rule> metricsRules;
+
+ @Data
+ @NoArgsConstructor
+ public static class Rule implements RuleConfig {
Review Comment:
This class is duplicate of `org.apache.skywalking.oap.meter.analyzer.prometheus.rule.MetricsRule`
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/config/TelegrafConfig.java:
##########
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.config;
+
+import lombok.Data;
+import lombok.NoArgsConstructor;
+import org.apache.skywalking.oap.meter.analyzer.MetricRuleConfig;
+
+import java.util.List;
+
+@Data
+public class TelegrafConfig implements MetricRuleConfig {
Review Comment:
This class is duplicate of `org.apache.skywalking.oap.meter.analyzer.prometheus.rule.Rule`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] mrproliu commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
mrproliu commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r1009228197
##########
oap-server/server-starter/src/main/resources/telegraf-rules/vm.yaml:
##########
@@ -0,0 +1,56 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+expSuffix: service(['host'], Layer.OS_LINUX)
+metricPrefix: meter_vm
+metricsRules:
+
+ # cpu
+ - name: cpu_total_percentage
+ exp: cpu_usage_user.tagEqual('cpu' , 'cpu-total')
+ - name: cpu_average_used_telegraf
+ exp: cpu_usage_user.tagEqual('cpu' , 'cpu-total')
+ - name: cpu_load1
+ exp: system_load1
+ - name: cpu_load5
+ exp: system_load5
+ - name: cpu_load15
+ exp: system_load15
+
+ # memory
+ - name: memory_total
+ exp: mem_total
+ - name: memory_available
+ exp: mem_available
+ - name: memory_used
+ exp: mem_used
+
+ # swap
+ - name: memory_swap_free
+ exp: swap_free
+ - name: memory_swap_total
+ exp: cpu_swap_total
+ - name: memory_swap_percentage
+ exp: 100 - ((swap_free * 100) / cpu_swap_total)
+
+ #node filesystem
+ - name: filesystem_percentage_telegraf
+ exp: disk_used_percent
Review Comment:
I have done a local test with the telegref, I think we need to use some aggregation function to aggregate the metrics:
1. The telegref will count the metrics every second and batch upload them.
2. The telegref report data with multiple labels. for example, the dist used percent meter will contain metrics with multiple devices.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] liuhaoyang commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
liuhaoyang commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r1012849613
##########
oap-server/server-starter/src/main/resources/telegraf-rules/vm.yaml:
##########
@@ -0,0 +1,56 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+expSuffix: service(['host'], Layer.OS_LINUX)
+metricPrefix: meter_vm
+metricsRules:
+
+ # cpu
+ - name: cpu_total_percentage
+ exp: cpu_usage_user.tagEqual('cpu' , 'cpu-total')
+ - name: cpu_average_used_telegraf
+ exp: cpu_usage_user.tagEqual('cpu' , 'cpu-total')
+ - name: cpu_load1
+ exp: system_load1
+ - name: cpu_load5
+ exp: system_load5
+ - name: cpu_load15
+ exp: system_load15
+
+ # memory
+ - name: memory_total
+ exp: mem_total
+ - name: memory_available
+ exp: mem_available
+ - name: memory_used
+ exp: mem_used
+
+ # swap
+ - name: memory_swap_free
+ exp: swap_free
+ - name: memory_swap_total
+ exp: cpu_swap_total
+ - name: memory_swap_percentage
+ exp: 100 - ((swap_free * 100) / cpu_swap_total)
+
+ #node filesystem
+ - name: filesystem_percentage_telegraf
+ exp: disk_used_percent
Review Comment:
It is feasible to change the collection interval of telegraf to `10s` or `30s`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r1013587002
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/handler/TelegrafServiceHandler.java:
##########
@@ -0,0 +1,129 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.handler;
+
+import com.google.common.collect.ImmutableMap;
+import com.linecorp.armeria.server.annotation.Post;
+import com.linecorp.armeria.server.annotation.RequestConverter;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.skywalking.apm.network.common.v3.Commands;
+import org.apache.skywalking.oap.meter.analyzer.MetricConvert;
+import org.apache.skywalking.oap.meter.analyzer.dsl.Sample;
+import org.apache.skywalking.oap.meter.analyzer.dsl.SampleFamily;
+import org.apache.skywalking.oap.meter.analyzer.dsl.SampleFamilyBuilder;
+import org.apache.skywalking.oap.meter.analyzer.prometheus.rule.Rule;
+import org.apache.skywalking.oap.server.core.analysis.meter.MeterSystem;
+import org.apache.skywalking.oap.server.library.module.ModuleManager;
+import org.apache.skywalking.oap.server.receiver.telegraf.provider.handler.pojo.TelegrafData;
+import org.apache.skywalking.oap.server.receiver.telegraf.provider.handler.pojo.TelegrafDatum;
+import org.apache.skywalking.oap.server.telemetry.TelemetryModule;
+import org.apache.skywalking.oap.server.telemetry.api.CounterMetrics;
+import org.apache.skywalking.oap.server.telemetry.api.HistogramMetrics;
+import org.apache.skywalking.oap.server.telemetry.api.MetricsCreator;
+import org.apache.skywalking.oap.server.telemetry.api.MetricsTag;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+@Slf4j
+public class TelegrafServiceHandler {
+
+ private final HistogramMetrics histogram;
+ private final CounterMetrics errorCounter;
+ private List<MetricConvert> metricConvert;
+
+ public TelegrafServiceHandler(ModuleManager moduleManager, MeterSystem meterSystem, List<Rule> rules) {
+
+ this.metricConvert = rules.stream().map(r -> new MetricConvert(r, meterSystem)).collect(Collectors.toList());
+
+ final MetricsCreator metricsCreator = moduleManager.find(TelemetryModule.NAME)
+ .provider()
+ .getService(MetricsCreator.class);
+
+ histogram = metricsCreator.createHistogramMetric(
+ "telegraf_in_latency", "The process latency of telegraf data",
+ new MetricsTag.Keys("protocol"), new MetricsTag.Values("http")
+ );
+
+ errorCounter = metricsCreator.createCounter(
+ "telegraf_error_count", "The error number of telegraf analysis",
+ new MetricsTag.Keys("protocol"), new MetricsTag.Values("http")
+ );
+ }
+
+ /**
+ * Convert the TelegrafData object to meter {@link Sample}
+ **/
+ public List<Sample> convertTelegraf(TelegrafDatum telegrafData) {
+
+ List<Sample> sampleList = new ArrayList<>();
+
+ Map<String, Object> fields = telegrafData.getFields();
+ String name = telegrafData.getName();
+ Map<String, String> tags = telegrafData.getTags();
+ ImmutableMap<String, String> immutableTags = ImmutableMap.copyOf(tags);
+ long timestamp = telegrafData.getTimestamp();
+
+ fields.forEach((key, value) -> {
+ if (value instanceof Number) {
+ Sample.SampleBuilder builder = Sample.builder();
+ Sample sample = builder.name(name + "_" + key)
+ .timestamp(timestamp * 1000L)
+ .value(((Number) value).doubleValue())
+ .labels(immutableTags).build();
+
+ sampleList.add(sample);
+ }
+ });
+ return sampleList;
+
+ }
+
+ public ImmutableMap<String, SampleFamily> convertSampleFamily(TelegrafData telegrafData) {
+ List<Sample> allSamples = new ArrayList<>();
+
+ List<TelegrafDatum> metrics = telegrafData.getMetrics();
+ for (TelegrafDatum m : metrics) {
+ List<Sample> samples = convertTelegraf(m);
+ allSamples.addAll(samples);
+ }
+
+ // Grouping all samples by their name, then build sampleFamily
Review Comment:
This policy seems not right. The samples should be grouped by bulk/sampling ID.
Because
1. Metrics in the same name with different timestamps should belong to different sample families.
2. Different metrics(name) collected in the same timestamp(one collecting) should belong to the same sample family. You could check the MAL doc, which mentioned metrics should be calculated through one formula, ref to https://skywalking.apache.org/docs/main/next/en/concepts-and-designs/mal/#binary-operators, `instance_trace_analysis_error_count / instance_trace_count`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#issuecomment-1253685688
Any update as has been reviewed for a week.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] soander commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
soander commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r1000967894
##########
docs/en/setup/backend/backend-vm-monitoring.md:
##########
@@ -3,38 +3,63 @@ SkyWalking leverages Prometheus node-exporter to collect metrics data from the V
[OpenTelemetry receiver](opentelemetry-receiver.md) and into the [Meter System](./../../concepts-and-designs/meter.md).
VM entity as a `Service` in OAP and on the `Layer: OS_LINUX`.
+SkyWalking also provides InfluxDB Telegraf to receive VMs' metrics data by [Telegraf receiver](./telegraf-receiver.md).
+The telegraf receiver plugin receiver, process and convert the metrics, then it send converted metrics to [Meter System](./../../concepts-and-designs/meter.md).
+VM entity as a `Service` in OAP and on the `Layer: OS_LINUX`.
+
## Data flow
+**For OpenTelemetry receiver:**
1. The Prometheus node-exporter collects metrics data from the VMs.
2. The OpenTelemetry Collector fetches metrics from node-exporter via Prometheus Receiver and pushes metrics to the SkyWalking OAP Server via the OpenCensus gRPC Exporter or OpenTelemetry gRPC exporter.
3. The SkyWalking OAP Server parses the expression with [MAL](../../concepts-and-designs/mal.md) to filter/calculate/aggregate and store the results.
+**For Telegraf receiver:**
+1. The InfluxDB Telegraf [input plugins](https://docs.influxdata.com/telegraf/v1.24/plugins/) collects various metrics data from the VMs.
+2. The InfluxDB Telegraf send `JSON` format metrics by `HTTP` messages to Telegraf Receiver, then pushes converted metrics to the SkyWalking OAP Server [Meter System](./../../concepts-and-designs/meter.md).
+3. The SkyWalking OAP Server parses the expression with [MAL](../../concepts-and-designs/mal.md) to filter/calculate/aggregate and store the results.
## Setup
-
+**For OpenTelemetry receiver:**
1. Setup [Prometheus node-exporter](https://prometheus.io/docs/guides/node-exporter/).
2. Setup [OpenTelemetry Collector ](https://opentelemetry.io/docs/collector/). This is an example for OpenTelemetry Collector configuration [otel-collector-config.yaml](../../../../test/e2e-v2/cases/vm/prometheus-node-exporter/otel-collector-config.yaml).
3. Config SkyWalking [OpenTelemetry receiver](opentelemetry-receiver.md).
+**For Telegraf receiver:**
+1. Setup InfluxDB Telegraf's `telegraf.conf file` according to [Telegraf office document](https://docs.influxdata.com/telegraf/v1.24/).
+2. Setup InfluxDB Telegraf's `telegraf.conf file` specific rules according to [Telegraf receiver document](telegraf-receiver.md).
+3. Config SkyWalking [Telegraf receiver](telegraf-receiver.md).
+
## Supported Metrics
+**For OpenTelemetry receiver:**
Review Comment:
> Metrics name for UI, or metrics name from telegraf?
Both metrics names are different. For example, telegraf metrics' `mem_total` shows `mem_total_info` in UI, but OpenTelemetry metrics' `meter_vm_memory_total` shows `memory_total` in UI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#issuecomment-1285073546
Besides e2e, you should add UTs(Unit Tests) for your format transfer, it could cover more cases easily and help the reviewers to understand the codes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#issuecomment-1285060792
4 files left for this PR? What happens?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] soander commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
soander commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r1000252495
##########
docs/en/setup/backend/backend-vm-monitoring.md:
##########
@@ -3,38 +3,63 @@ SkyWalking leverages Prometheus node-exporter to collect metrics data from the V
[OpenTelemetry receiver](opentelemetry-receiver.md) and into the [Meter System](./../../concepts-and-designs/meter.md).
VM entity as a `Service` in OAP and on the `Layer: OS_LINUX`.
+SkyWalking also provides InfluxDB Telegraf to receive VMs' metrics data by [Telegraf receiver](./telegraf-receiver.md).
+The telegraf receiver plugin receiver, process and convert the metrics, then it send converted metrics to [Meter System](./../../concepts-and-designs/meter.md).
+VM entity as a `Service` in OAP and on the `Layer: OS_LINUX`.
+
## Data flow
+**For OpenTelemetry receiver:**
1. The Prometheus node-exporter collects metrics data from the VMs.
2. The OpenTelemetry Collector fetches metrics from node-exporter via Prometheus Receiver and pushes metrics to the SkyWalking OAP Server via the OpenCensus gRPC Exporter or OpenTelemetry gRPC exporter.
3. The SkyWalking OAP Server parses the expression with [MAL](../../concepts-and-designs/mal.md) to filter/calculate/aggregate and store the results.
+**For Telegraf receiver:**
+1. The InfluxDB Telegraf [input plugins](https://docs.influxdata.com/telegraf/v1.24/plugins/) collects various metrics data from the VMs.
+2. The InfluxDB Telegraf send `JSON` format metrics by `HTTP` messages to Telegraf Receiver, then pushes converted metrics to the SkyWalking OAP Server [Meter System](./../../concepts-and-designs/meter.md).
+3. The SkyWalking OAP Server parses the expression with [MAL](../../concepts-and-designs/mal.md) to filter/calculate/aggregate and store the results.
## Setup
-
+**For OpenTelemetry receiver:**
1. Setup [Prometheus node-exporter](https://prometheus.io/docs/guides/node-exporter/).
2. Setup [OpenTelemetry Collector ](https://opentelemetry.io/docs/collector/). This is an example for OpenTelemetry Collector configuration [otel-collector-config.yaml](../../../../test/e2e-v2/cases/vm/prometheus-node-exporter/otel-collector-config.yaml).
3. Config SkyWalking [OpenTelemetry receiver](opentelemetry-receiver.md).
+**For Telegraf receiver:**
+1. Setup InfluxDB Telegraf's `telegraf.conf file` according to [Telegraf office document](https://docs.influxdata.com/telegraf/v1.24/).
+2. Setup InfluxDB Telegraf's `telegraf.conf file` specific rules according to [Telegraf receiver document](telegraf-receiver.md).
+3. Config SkyWalking [Telegraf receiver](telegraf-receiver.md).
+
## Supported Metrics
+**For OpenTelemetry receiver:**
Review Comment:
I just write the Data flow of OpenTelemetry receiver and Telegraf receiver. Actually, they share the metrics.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#issuecomment-1257519916
@soander Could you put a summary about which are changed from last comment?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r1009080120
##########
oap-server/server-starter/src/main/resources/telegraf-rules/vm.yaml:
##########
@@ -0,0 +1,56 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+expSuffix: service(['host'], Layer.OS_LINUX)
+metricPrefix: meter_vm
+metricsRules:
+
+ # cpu
+ - name: cpu_total_percentage
+ exp: cpu_usage_user.tagEqual('cpu' , 'cpu-total')
+ - name: cpu_average_used_telegraf
+ exp: cpu_usage_user.tagEqual('cpu' , 'cpu-total')
+ - name: cpu_load1
+ exp: system_load1
+ - name: cpu_load5
+ exp: system_load5
+ - name: cpu_load15
+ exp: system_load15
+
+ # memory
+ - name: memory_total
+ exp: mem_total
+ - name: memory_available
+ exp: mem_available
+ - name: memory_used
+ exp: mem_used
+
+ # swap
+ - name: memory_swap_free
+ exp: swap_free
+ - name: memory_swap_total
+ exp: cpu_swap_total
+ - name: memory_swap_percentage
+ exp: 100 - ((swap_free * 100) / cpu_swap_total)
Review Comment:
Percentage usually uses 0.01% precision. And UI would set `/ 100` to format the two decimal places.
Could you check what is your case?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] liuhaoyang commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
liuhaoyang commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r1012853663
##########
oap-server/server-starter/src/main/resources/telegraf-rules/vm.yaml:
##########
@@ -0,0 +1,56 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+expSuffix: service(['host'], Layer.OS_LINUX)
+metricPrefix: meter_vm
+metricsRules:
+
+ # cpu
+ - name: cpu_total_percentage
+ exp: cpu_usage_user.tagEqual('cpu' , 'cpu-total')
+ - name: cpu_average_used_telegraf
+ exp: cpu_usage_user.tagEqual('cpu' , 'cpu-total')
+ - name: cpu_load1
+ exp: system_load1
+ - name: cpu_load5
+ exp: system_load5
+ - name: cpu_load15
+ exp: system_load15
+
+ # memory
+ - name: memory_total
+ exp: mem_total
+ - name: memory_available
+ exp: mem_available
+ - name: memory_used
+ exp: mem_used
+
+ # swap
+ - name: memory_swap_free
+ exp: swap_free
+ - name: memory_swap_total
+ exp: cpu_swap_total
+ - name: memory_swap_percentage
+ exp: 100 - ((swap_free * 100) / cpu_swap_total)
+
+ #node filesystem
+ - name: filesystem_percentage_telegraf
+ exp: disk_used_percent
Review Comment:
The reason for this problem is the wrong config of telegraf, the collection interval is much shorter than the flusher interval.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] liuhaoyang commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
liuhaoyang commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r1011904922
##########
oap-server/server-starter/src/main/resources/telegraf-rules/vm.yaml:
##########
@@ -0,0 +1,56 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+expSuffix: service(['host'], Layer.OS_LINUX)
+metricPrefix: meter_vm
+metricsRules:
+
+ # cpu
+ - name: cpu_total_percentage
+ exp: cpu_usage_user.tagEqual('cpu' , 'cpu-total')
+ - name: cpu_average_used_telegraf
+ exp: cpu_usage_user.tagEqual('cpu' , 'cpu-total')
+ - name: cpu_load1
+ exp: system_load1
+ - name: cpu_load5
+ exp: system_load5
+ - name: cpu_load15
+ exp: system_load15
+
+ # memory
+ - name: memory_total
+ exp: mem_total
+ - name: memory_available
+ exp: mem_available
+ - name: memory_used
+ exp: mem_used
+
+ # swap
+ - name: memory_swap_free
+ exp: swap_free
+ - name: memory_swap_total
+ exp: cpu_swap_total
+ - name: memory_swap_percentage
+ exp: 100 - ((swap_free * 100) / cpu_swap_total)
+
+ #node filesystem
+ - name: filesystem_percentage_telegraf
+ exp: disk_used_percent
Review Comment:
for 1, the [interval](https://github.com/influxdata/telegraf/blob/master/docs/CONFIGURATION.md#intervals) is configurable in telegraf. In the telegraf.conf of the e2e test, in order to obtain data faster, configure the interval to `1s`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] soander commented on pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
soander commented on PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#issuecomment-1258878384
>
I have contacted my mentor, and I will check documents. Maybe I made some inconvenience because I am a newbie.
Anyway, thank you again and sorry for some mistakes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#issuecomment-1258885403
I recommend you `asking for your mentor`, because I think he may have more time than me to help you out with the env and setup issues.
This is not an issue from community side, we just have to wait for your tests ready.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] kezhenxu94 commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
kezhenxu94 commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r973546357
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/handler/TelegrafServiceHandler.java:
##########
@@ -0,0 +1,128 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.handler;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.google.common.collect.ImmutableMap;
+import com.linecorp.armeria.server.annotation.Post;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.skywalking.apm.network.common.v3.Commands;
+import org.apache.skywalking.oap.meter.analyzer.MetricConvert;
+import org.apache.skywalking.oap.meter.analyzer.dsl.Sample;
+import org.apache.skywalking.oap.meter.analyzer.dsl.SampleFamily;
+import org.apache.skywalking.oap.meter.analyzer.dsl.SampleFamilyBuilder;
+import org.apache.skywalking.oap.server.core.analysis.meter.MeterSystem;
+import org.apache.skywalking.oap.server.library.module.ModuleManager;
+import org.apache.skywalking.oap.server.receiver.telegraf.provider.config.TelegrafConfig;
+import org.apache.skywalking.oap.server.receiver.telegraf.provider.handler.pojo.TelegrafData;
+import org.apache.skywalking.oap.server.telemetry.TelemetryModule;
+import org.apache.skywalking.oap.server.telemetry.api.CounterMetrics;
+import org.apache.skywalking.oap.server.telemetry.api.HistogramMetrics;
+import org.apache.skywalking.oap.server.telemetry.api.MetricsCreator;
+import org.apache.skywalking.oap.server.telemetry.api.MetricsTag;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+@Slf4j
+public class TelegrafServiceHandler {
+
+ private final HistogramMetrics histogram;
+ private final CounterMetrics errorCounter;
+ private List<MetricConvert> metricConvert;
+
+ public TelegrafServiceHandler(ModuleManager moduleManager, MeterSystem meterSystem, List<TelegrafConfig> rules) {
+
+ this.metricConvert = rules.stream().map(r -> new MetricConvert(r, meterSystem)).collect(Collectors.toList());
+
+ final MetricsCreator metricsCreator = moduleManager.find(TelemetryModule.NAME)
+ .provider()
+ .getService(MetricsCreator.class);
+
+ histogram = metricsCreator.createHistogramMetric(
+ "telegraf_in_latency", "The process latency of telegraf data",
+ new MetricsTag.Keys("protocol"), new MetricsTag.Values("http")
+ );
+
+ errorCounter = metricsCreator.createCounter(
+ "telegraf_error_count", "The error number of telegraf analysis",
+ new MetricsTag.Keys("protocol"), new MetricsTag.Values("http")
+ );
+ }
+
+ /**
+ * Convert the TelegrafData object to meter {@link Sample}
+ **/
+ public List<Sample> convertTelegraf(TelegrafData telegrafData) {
+
+ List<Sample> sampleList = new ArrayList<>(Collections.emptyList());
+
+ Map<String, Object> fields = telegrafData.getFields();
+ String name = telegrafData.getName();
+ Map<String, String> tags = telegrafData.getTags();
+ ImmutableMap<String, String> immutableTags = ImmutableMap.copyOf(tags);
+ long timestamp = telegrafData.getTimestamp();
+
+ fields.forEach((key, value) -> {
+ Sample.SampleBuilder builder = Sample.builder();
+ if (value instanceof String) return;
+ Sample sample = builder.name(name + "_" + key)
+ .timestamp(timestamp * 1000L)
+ .value(((Number) value).doubleValue())
+ .labels(immutableTags).build();
+
+ sampleList.add(sample);
+ });
+ return sampleList;
+
+ }
+
+ @Post("/telegraf")
+ public Commands collectData(String jsonInfo) {
+ TelegrafData telegrafData;
Review Comment:
```suggestion
```
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/handler/TelegrafServiceHandler.java:
##########
@@ -0,0 +1,128 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.handler;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.google.common.collect.ImmutableMap;
+import com.linecorp.armeria.server.annotation.Post;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.skywalking.apm.network.common.v3.Commands;
+import org.apache.skywalking.oap.meter.analyzer.MetricConvert;
+import org.apache.skywalking.oap.meter.analyzer.dsl.Sample;
+import org.apache.skywalking.oap.meter.analyzer.dsl.SampleFamily;
+import org.apache.skywalking.oap.meter.analyzer.dsl.SampleFamilyBuilder;
+import org.apache.skywalking.oap.server.core.analysis.meter.MeterSystem;
+import org.apache.skywalking.oap.server.library.module.ModuleManager;
+import org.apache.skywalking.oap.server.receiver.telegraf.provider.config.TelegrafConfig;
+import org.apache.skywalking.oap.server.receiver.telegraf.provider.handler.pojo.TelegrafData;
+import org.apache.skywalking.oap.server.telemetry.TelemetryModule;
+import org.apache.skywalking.oap.server.telemetry.api.CounterMetrics;
+import org.apache.skywalking.oap.server.telemetry.api.HistogramMetrics;
+import org.apache.skywalking.oap.server.telemetry.api.MetricsCreator;
+import org.apache.skywalking.oap.server.telemetry.api.MetricsTag;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+@Slf4j
+public class TelegrafServiceHandler {
+
+ private final HistogramMetrics histogram;
+ private final CounterMetrics errorCounter;
+ private List<MetricConvert> metricConvert;
+
+ public TelegrafServiceHandler(ModuleManager moduleManager, MeterSystem meterSystem, List<TelegrafConfig> rules) {
+
+ this.metricConvert = rules.stream().map(r -> new MetricConvert(r, meterSystem)).collect(Collectors.toList());
+
+ final MetricsCreator metricsCreator = moduleManager.find(TelemetryModule.NAME)
+ .provider()
+ .getService(MetricsCreator.class);
+
+ histogram = metricsCreator.createHistogramMetric(
+ "telegraf_in_latency", "The process latency of telegraf data",
+ new MetricsTag.Keys("protocol"), new MetricsTag.Values("http")
+ );
+
+ errorCounter = metricsCreator.createCounter(
+ "telegraf_error_count", "The error number of telegraf analysis",
+ new MetricsTag.Keys("protocol"), new MetricsTag.Values("http")
+ );
+ }
+
+ /**
+ * Convert the TelegrafData object to meter {@link Sample}
+ **/
+ public List<Sample> convertTelegraf(TelegrafData telegrafData) {
+
+ List<Sample> sampleList = new ArrayList<>(Collections.emptyList());
+
+ Map<String, Object> fields = telegrafData.getFields();
+ String name = telegrafData.getName();
+ Map<String, String> tags = telegrafData.getTags();
+ ImmutableMap<String, String> immutableTags = ImmutableMap.copyOf(tags);
+ long timestamp = telegrafData.getTimestamp();
+
+ fields.forEach((key, value) -> {
+ Sample.SampleBuilder builder = Sample.builder();
+ if (value instanceof String) return;
+ Sample sample = builder.name(name + "_" + key)
+ .timestamp(timestamp * 1000L)
+ .value(((Number) value).doubleValue())
+ .labels(immutableTags).build();
+
+ sampleList.add(sample);
+ });
+ return sampleList;
+
+ }
+
+ @Post("/telegraf")
+ public Commands collectData(String jsonInfo) {
+ TelegrafData telegrafData;
+ try (HistogramMetrics.Timer ignored = histogram.createTimer()) {
+ List<Sample> allSamples = new ArrayList<>();
+ ObjectMapper mapper = new ObjectMapper();
+ JsonNode node = mapper.readTree(jsonInfo);
Review Comment:
Can you rename the original `TelegrafData` class to `TelegrafDatum`, and define a class `TelegrafData` to contain a field `List<TelegrafDatum> metrics`, so you don't need to `readTree` here, and just declare the method `public Commands collectData(TelegrafData telegrafData) {`
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/handler/pojo/TelegrafData.java:
##########
@@ -0,0 +1,42 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.handler.pojo;
+
+import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
+import com.fasterxml.jackson.annotation.JsonProperty;
+import lombok.AllArgsConstructor;
+import lombok.Data;
+import lombok.NoArgsConstructor;
+
+import java.util.Map;
+
+@Data
+@NoArgsConstructor
+@AllArgsConstructor
+@JsonIgnoreProperties(ignoreUnknown = true)
+public class TelegrafData {
+ @JsonProperty("fields")
+ private Map<String, Object> fields;
+ @JsonProperty("name")
+ private String name;
+ @JsonProperty("tags")
+ private Map<String, String> tags;
+ @JsonProperty("timestamp")
+ private long timestamp;
Review Comment:
If the json field names are the same as the Java field names, `@JsonProperty` is redundant
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/handler/TelegrafServiceHandler.java:
##########
@@ -0,0 +1,128 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.handler;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.google.common.collect.ImmutableMap;
+import com.linecorp.armeria.server.annotation.Post;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.skywalking.apm.network.common.v3.Commands;
+import org.apache.skywalking.oap.meter.analyzer.MetricConvert;
+import org.apache.skywalking.oap.meter.analyzer.dsl.Sample;
+import org.apache.skywalking.oap.meter.analyzer.dsl.SampleFamily;
+import org.apache.skywalking.oap.meter.analyzer.dsl.SampleFamilyBuilder;
+import org.apache.skywalking.oap.server.core.analysis.meter.MeterSystem;
+import org.apache.skywalking.oap.server.library.module.ModuleManager;
+import org.apache.skywalking.oap.server.receiver.telegraf.provider.config.TelegrafConfig;
+import org.apache.skywalking.oap.server.receiver.telegraf.provider.handler.pojo.TelegrafData;
+import org.apache.skywalking.oap.server.telemetry.TelemetryModule;
+import org.apache.skywalking.oap.server.telemetry.api.CounterMetrics;
+import org.apache.skywalking.oap.server.telemetry.api.HistogramMetrics;
+import org.apache.skywalking.oap.server.telemetry.api.MetricsCreator;
+import org.apache.skywalking.oap.server.telemetry.api.MetricsTag;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+@Slf4j
+public class TelegrafServiceHandler {
+
+ private final HistogramMetrics histogram;
+ private final CounterMetrics errorCounter;
+ private List<MetricConvert> metricConvert;
+
+ public TelegrafServiceHandler(ModuleManager moduleManager, MeterSystem meterSystem, List<TelegrafConfig> rules) {
+
+ this.metricConvert = rules.stream().map(r -> new MetricConvert(r, meterSystem)).collect(Collectors.toList());
+
+ final MetricsCreator metricsCreator = moduleManager.find(TelemetryModule.NAME)
+ .provider()
+ .getService(MetricsCreator.class);
+
+ histogram = metricsCreator.createHistogramMetric(
+ "telegraf_in_latency", "The process latency of telegraf data",
+ new MetricsTag.Keys("protocol"), new MetricsTag.Values("http")
+ );
+
+ errorCounter = metricsCreator.createCounter(
+ "telegraf_error_count", "The error number of telegraf analysis",
+ new MetricsTag.Keys("protocol"), new MetricsTag.Values("http")
+ );
+ }
+
+ /**
+ * Convert the TelegrafData object to meter {@link Sample}
+ **/
+ public List<Sample> convertTelegraf(TelegrafData telegrafData) {
+
+ List<Sample> sampleList = new ArrayList<>(Collections.emptyList());
Review Comment:
```suggestion
List<Sample> sampleList = new ArrayList<>();
```
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/handler/TelegrafServiceHandler.java:
##########
@@ -0,0 +1,128 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.handler;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.google.common.collect.ImmutableMap;
+import com.linecorp.armeria.server.annotation.Post;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.skywalking.apm.network.common.v3.Commands;
+import org.apache.skywalking.oap.meter.analyzer.MetricConvert;
+import org.apache.skywalking.oap.meter.analyzer.dsl.Sample;
+import org.apache.skywalking.oap.meter.analyzer.dsl.SampleFamily;
+import org.apache.skywalking.oap.meter.analyzer.dsl.SampleFamilyBuilder;
+import org.apache.skywalking.oap.server.core.analysis.meter.MeterSystem;
+import org.apache.skywalking.oap.server.library.module.ModuleManager;
+import org.apache.skywalking.oap.server.receiver.telegraf.provider.config.TelegrafConfig;
+import org.apache.skywalking.oap.server.receiver.telegraf.provider.handler.pojo.TelegrafData;
+import org.apache.skywalking.oap.server.telemetry.TelemetryModule;
+import org.apache.skywalking.oap.server.telemetry.api.CounterMetrics;
+import org.apache.skywalking.oap.server.telemetry.api.HistogramMetrics;
+import org.apache.skywalking.oap.server.telemetry.api.MetricsCreator;
+import org.apache.skywalking.oap.server.telemetry.api.MetricsTag;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+@Slf4j
+public class TelegrafServiceHandler {
+
+ private final HistogramMetrics histogram;
+ private final CounterMetrics errorCounter;
+ private List<MetricConvert> metricConvert;
+
+ public TelegrafServiceHandler(ModuleManager moduleManager, MeterSystem meterSystem, List<TelegrafConfig> rules) {
+
+ this.metricConvert = rules.stream().map(r -> new MetricConvert(r, meterSystem)).collect(Collectors.toList());
+
+ final MetricsCreator metricsCreator = moduleManager.find(TelemetryModule.NAME)
+ .provider()
+ .getService(MetricsCreator.class);
+
+ histogram = metricsCreator.createHistogramMetric(
+ "telegraf_in_latency", "The process latency of telegraf data",
+ new MetricsTag.Keys("protocol"), new MetricsTag.Values("http")
+ );
+
+ errorCounter = metricsCreator.createCounter(
+ "telegraf_error_count", "The error number of telegraf analysis",
+ new MetricsTag.Keys("protocol"), new MetricsTag.Values("http")
+ );
+ }
+
+ /**
+ * Convert the TelegrafData object to meter {@link Sample}
+ **/
+ public List<Sample> convertTelegraf(TelegrafData telegrafData) {
+
+ List<Sample> sampleList = new ArrayList<>(Collections.emptyList());
+
+ Map<String, Object> fields = telegrafData.getFields();
+ String name = telegrafData.getName();
+ Map<String, String> tags = telegrafData.getTags();
+ ImmutableMap<String, String> immutableTags = ImmutableMap.copyOf(tags);
+ long timestamp = telegrafData.getTimestamp();
+
+ fields.forEach((key, value) -> {
+ Sample.SampleBuilder builder = Sample.builder();
+ if (value instanceof String) return;
+ Sample sample = builder.name(name + "_" + key)
+ .timestamp(timestamp * 1000L)
+ .value(((Number) value).doubleValue())
+ .labels(immutableTags).build();
+
+ sampleList.add(sample);
+ });
+ return sampleList;
+
+ }
+
+ @Post("/telegraf")
+ public Commands collectData(String jsonInfo) {
+ TelegrafData telegrafData;
+ try (HistogramMetrics.Timer ignored = histogram.createTimer()) {
+ List<Sample> allSamples = new ArrayList<>();
+ ObjectMapper mapper = new ObjectMapper();
Review Comment:
Extract this to a field, the mapper should not be instantiated every time.
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/handler/TelegrafServiceHandler.java:
##########
@@ -0,0 +1,128 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.handler;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.google.common.collect.ImmutableMap;
+import com.linecorp.armeria.server.annotation.Post;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.skywalking.apm.network.common.v3.Commands;
+import org.apache.skywalking.oap.meter.analyzer.MetricConvert;
+import org.apache.skywalking.oap.meter.analyzer.dsl.Sample;
+import org.apache.skywalking.oap.meter.analyzer.dsl.SampleFamily;
+import org.apache.skywalking.oap.meter.analyzer.dsl.SampleFamilyBuilder;
+import org.apache.skywalking.oap.server.core.analysis.meter.MeterSystem;
+import org.apache.skywalking.oap.server.library.module.ModuleManager;
+import org.apache.skywalking.oap.server.receiver.telegraf.provider.config.TelegrafConfig;
+import org.apache.skywalking.oap.server.receiver.telegraf.provider.handler.pojo.TelegrafData;
+import org.apache.skywalking.oap.server.telemetry.TelemetryModule;
+import org.apache.skywalking.oap.server.telemetry.api.CounterMetrics;
+import org.apache.skywalking.oap.server.telemetry.api.HistogramMetrics;
+import org.apache.skywalking.oap.server.telemetry.api.MetricsCreator;
+import org.apache.skywalking.oap.server.telemetry.api.MetricsTag;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+@Slf4j
+public class TelegrafServiceHandler {
+
+ private final HistogramMetrics histogram;
+ private final CounterMetrics errorCounter;
+ private List<MetricConvert> metricConvert;
+
+ public TelegrafServiceHandler(ModuleManager moduleManager, MeterSystem meterSystem, List<TelegrafConfig> rules) {
+
+ this.metricConvert = rules.stream().map(r -> new MetricConvert(r, meterSystem)).collect(Collectors.toList());
+
+ final MetricsCreator metricsCreator = moduleManager.find(TelemetryModule.NAME)
+ .provider()
+ .getService(MetricsCreator.class);
+
+ histogram = metricsCreator.createHistogramMetric(
+ "telegraf_in_latency", "The process latency of telegraf data",
+ new MetricsTag.Keys("protocol"), new MetricsTag.Values("http")
+ );
+
+ errorCounter = metricsCreator.createCounter(
+ "telegraf_error_count", "The error number of telegraf analysis",
+ new MetricsTag.Keys("protocol"), new MetricsTag.Values("http")
+ );
+ }
+
+ /**
+ * Convert the TelegrafData object to meter {@link Sample}
+ **/
+ public List<Sample> convertTelegraf(TelegrafData telegrafData) {
+
+ List<Sample> sampleList = new ArrayList<>(Collections.emptyList());
+
+ Map<String, Object> fields = telegrafData.getFields();
+ String name = telegrafData.getName();
+ Map<String, String> tags = telegrafData.getTags();
+ ImmutableMap<String, String> immutableTags = ImmutableMap.copyOf(tags);
+ long timestamp = telegrafData.getTimestamp();
+
+ fields.forEach((key, value) -> {
+ Sample.SampleBuilder builder = Sample.builder();
+ if (value instanceof String) return;
+ Sample sample = builder.name(name + "_" + key)
+ .timestamp(timestamp * 1000L)
+ .value(((Number) value).doubleValue())
+ .labels(immutableTags).build();
+
+ sampleList.add(sample);
+ });
+ return sampleList;
+
+ }
+
+ @Post("/telegraf")
+ public Commands collectData(String jsonInfo) {
+ TelegrafData telegrafData;
+ try (HistogramMetrics.Timer ignored = histogram.createTimer()) {
+ List<Sample> allSamples = new ArrayList<>();
+ ObjectMapper mapper = new ObjectMapper();
+ JsonNode node = mapper.readTree(jsonInfo);
+
+ // Read each metrics json and convert it to Sample
+ JsonNode metrics = node.get("metrics");
+ for (JsonNode m : metrics) {
+ telegrafData = mapper.convertValue(m, TelegrafData.class);
Review Comment:
```suggestion
TelegrafData telegrafData = mapper.convertValue(m, TelegrafData.class);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wankai123 commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
wankai123 commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r973532485
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/handler/TelegrafServiceHandler.java:
##########
@@ -0,0 +1,128 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.handler;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.google.common.collect.ImmutableMap;
+import com.linecorp.armeria.server.annotation.Post;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.skywalking.apm.network.common.v3.Commands;
+import org.apache.skywalking.oap.meter.analyzer.MetricConvert;
+import org.apache.skywalking.oap.meter.analyzer.dsl.Sample;
+import org.apache.skywalking.oap.meter.analyzer.dsl.SampleFamily;
+import org.apache.skywalking.oap.meter.analyzer.dsl.SampleFamilyBuilder;
+import org.apache.skywalking.oap.server.core.analysis.meter.MeterSystem;
+import org.apache.skywalking.oap.server.library.module.ModuleManager;
+import org.apache.skywalking.oap.server.receiver.telegraf.provider.config.TelegrafConfig;
+import org.apache.skywalking.oap.server.receiver.telegraf.provider.handler.pojo.TelegrafData;
+import org.apache.skywalking.oap.server.telemetry.TelemetryModule;
+import org.apache.skywalking.oap.server.telemetry.api.CounterMetrics;
+import org.apache.skywalking.oap.server.telemetry.api.HistogramMetrics;
+import org.apache.skywalking.oap.server.telemetry.api.MetricsCreator;
+import org.apache.skywalking.oap.server.telemetry.api.MetricsTag;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+@Slf4j
+public class TelegrafServiceHandler {
+
+ private final HistogramMetrics histogram;
+ private final CounterMetrics errorCounter;
+ private List<MetricConvert> metricConvert;
+
+ public TelegrafServiceHandler(ModuleManager moduleManager, MeterSystem meterSystem, List<TelegrafConfig> rules) {
+
+ this.metricConvert = rules.stream().map(r -> new MetricConvert(r, meterSystem)).collect(Collectors.toList());
+
+ final MetricsCreator metricsCreator = moduleManager.find(TelemetryModule.NAME)
+ .provider()
+ .getService(MetricsCreator.class);
+
+ histogram = metricsCreator.createHistogramMetric(
+ "telegraf_in_latency", "The process latency of telegraf data",
+ new MetricsTag.Keys("protocol"), new MetricsTag.Values("http")
+ );
+
+ errorCounter = metricsCreator.createCounter(
+ "telegraf_error_count", "The error number of telegraf analysis",
+ new MetricsTag.Keys("protocol"), new MetricsTag.Values("http")
+ );
+ }
+
+ /**
+ * Convert the TelegrafData object to meter {@link Sample}
+ **/
+ public List<Sample> convertTelegraf(TelegrafData telegrafData) {
+
+ List<Sample> sampleList = new ArrayList<>(Collections.emptyList());
+
+ Map<String, Object> fields = telegrafData.getFields();
+ String name = telegrafData.getName();
+ Map<String, String> tags = telegrafData.getTags();
+ ImmutableMap<String, String> immutableTags = ImmutableMap.copyOf(tags);
+ long timestamp = telegrafData.getTimestamp();
+
+ fields.forEach((key, value) -> {
+ Sample.SampleBuilder builder = Sample.builder();
+ if (value instanceof String) return;
Review Comment:
Hi, could you provide some JSON data samples? I want to know more details about the convert logic. Such as why the value might be String.
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/handler/TelegrafServiceHandler.java:
##########
@@ -0,0 +1,128 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.handler;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.google.common.collect.ImmutableMap;
+import com.linecorp.armeria.server.annotation.Post;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.skywalking.apm.network.common.v3.Commands;
+import org.apache.skywalking.oap.meter.analyzer.MetricConvert;
+import org.apache.skywalking.oap.meter.analyzer.dsl.Sample;
+import org.apache.skywalking.oap.meter.analyzer.dsl.SampleFamily;
+import org.apache.skywalking.oap.meter.analyzer.dsl.SampleFamilyBuilder;
+import org.apache.skywalking.oap.server.core.analysis.meter.MeterSystem;
+import org.apache.skywalking.oap.server.library.module.ModuleManager;
+import org.apache.skywalking.oap.server.receiver.telegraf.provider.config.TelegrafConfig;
+import org.apache.skywalking.oap.server.receiver.telegraf.provider.handler.pojo.TelegrafData;
+import org.apache.skywalking.oap.server.telemetry.TelemetryModule;
+import org.apache.skywalking.oap.server.telemetry.api.CounterMetrics;
+import org.apache.skywalking.oap.server.telemetry.api.HistogramMetrics;
+import org.apache.skywalking.oap.server.telemetry.api.MetricsCreator;
+import org.apache.skywalking.oap.server.telemetry.api.MetricsTag;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+@Slf4j
+public class TelegrafServiceHandler {
+
+ private final HistogramMetrics histogram;
+ private final CounterMetrics errorCounter;
+ private List<MetricConvert> metricConvert;
+
+ public TelegrafServiceHandler(ModuleManager moduleManager, MeterSystem meterSystem, List<TelegrafConfig> rules) {
+
+ this.metricConvert = rules.stream().map(r -> new MetricConvert(r, meterSystem)).collect(Collectors.toList());
+
+ final MetricsCreator metricsCreator = moduleManager.find(TelemetryModule.NAME)
+ .provider()
+ .getService(MetricsCreator.class);
+
+ histogram = metricsCreator.createHistogramMetric(
+ "telegraf_in_latency", "The process latency of telegraf data",
+ new MetricsTag.Keys("protocol"), new MetricsTag.Values("http")
+ );
+
+ errorCounter = metricsCreator.createCounter(
+ "telegraf_error_count", "The error number of telegraf analysis",
+ new MetricsTag.Keys("protocol"), new MetricsTag.Values("http")
+ );
+ }
+
+ /**
+ * Convert the TelegrafData object to meter {@link Sample}
+ **/
+ public List<Sample> convertTelegraf(TelegrafData telegrafData) {
+
+ List<Sample> sampleList = new ArrayList<>(Collections.emptyList());
+
+ Map<String, Object> fields = telegrafData.getFields();
+ String name = telegrafData.getName();
+ Map<String, String> tags = telegrafData.getTags();
+ ImmutableMap<String, String> immutableTags = ImmutableMap.copyOf(tags);
+ long timestamp = telegrafData.getTimestamp();
+
+ fields.forEach((key, value) -> {
+ Sample.SampleBuilder builder = Sample.builder();
+ if (value instanceof String) return;
+ Sample sample = builder.name(name + "_" + key)
+ .timestamp(timestamp * 1000L)
Review Comment:
can you confirm the timestamp unit is `second` by default?
I found a doc: https://docs.influxdata.com/telegraf/v1.23/data_formats/output/json/
maybe the timestamp unit is configurable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] kezhenxu94 commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
kezhenxu94 commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r973545999
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/TelegrafReceiverProvider.java:
##########
@@ -0,0 +1,93 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider;
+
+import com.google.common.base.Splitter;
+import com.linecorp.armeria.common.HttpMethod;
+import org.apache.skywalking.oap.server.core.CoreModule;
+import org.apache.skywalking.oap.server.core.analysis.meter.MeterSystem;
+import org.apache.skywalking.oap.server.core.server.HTTPHandlerRegister;
+import org.apache.skywalking.oap.server.library.module.ModuleConfig;
+import org.apache.skywalking.oap.server.library.module.ModuleDefine;
+import org.apache.skywalking.oap.server.library.module.ModuleProvider;
+import org.apache.skywalking.oap.server.library.module.ModuleStartException;
+import org.apache.skywalking.oap.server.library.module.ServiceNotProvidedException;
+import org.apache.skywalking.oap.server.library.util.CollectionUtils;
+import org.apache.skywalking.oap.server.library.util.StringUtil;
+import org.apache.skywalking.oap.server.receiver.sharing.server.SharingServerModule;
+import org.apache.skywalking.oap.server.receiver.telegraf.module.TelegrafReceiverModule;
+import org.apache.skywalking.oap.server.receiver.telegraf.provider.config.TelegrafConfig;
+import org.apache.skywalking.oap.server.receiver.telegraf.provider.config.TelegrafConfigs;
+import org.apache.skywalking.oap.server.receiver.telegraf.provider.handler.TelegrafServiceHandler;
+
+import java.util.Collections;
+import java.util.List;
+
+public class TelegrafReceiverProvider extends ModuleProvider {
+ private List<TelegrafConfig> configs;
+ private TelegrafModuleConfig moduleConfig;
+
+ @Override
+ public String name() {
+ return "default";
+ }
+
+ @Override
+ public Class<? extends ModuleDefine> module() {
+ return TelegrafReceiverModule.class;
+ }
+
+ @Override
+ public ModuleConfig createConfigBeanIfAbsent() {
+ moduleConfig = new TelegrafModuleConfig();
+ return moduleConfig;
Review Comment:
This is not `create if absent` (`createConfigBeanIfAbsent`)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r973554475
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/handler/TelegrafServiceHandler.java:
##########
@@ -0,0 +1,128 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.handler;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.google.common.collect.ImmutableMap;
+import com.linecorp.armeria.server.annotation.Post;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.skywalking.apm.network.common.v3.Commands;
+import org.apache.skywalking.oap.meter.analyzer.MetricConvert;
+import org.apache.skywalking.oap.meter.analyzer.dsl.Sample;
+import org.apache.skywalking.oap.meter.analyzer.dsl.SampleFamily;
+import org.apache.skywalking.oap.meter.analyzer.dsl.SampleFamilyBuilder;
+import org.apache.skywalking.oap.server.core.analysis.meter.MeterSystem;
+import org.apache.skywalking.oap.server.library.module.ModuleManager;
+import org.apache.skywalking.oap.server.receiver.telegraf.provider.config.TelegrafConfig;
+import org.apache.skywalking.oap.server.receiver.telegraf.provider.handler.pojo.TelegrafData;
+import org.apache.skywalking.oap.server.telemetry.TelemetryModule;
+import org.apache.skywalking.oap.server.telemetry.api.CounterMetrics;
+import org.apache.skywalking.oap.server.telemetry.api.HistogramMetrics;
+import org.apache.skywalking.oap.server.telemetry.api.MetricsCreator;
+import org.apache.skywalking.oap.server.telemetry.api.MetricsTag;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+@Slf4j
+public class TelegrafServiceHandler {
+
+ private final HistogramMetrics histogram;
+ private final CounterMetrics errorCounter;
+ private List<MetricConvert> metricConvert;
+
+ public TelegrafServiceHandler(ModuleManager moduleManager, MeterSystem meterSystem, List<TelegrafConfig> rules) {
+
+ this.metricConvert = rules.stream().map(r -> new MetricConvert(r, meterSystem)).collect(Collectors.toList());
+
+ final MetricsCreator metricsCreator = moduleManager.find(TelemetryModule.NAME)
+ .provider()
+ .getService(MetricsCreator.class);
+
+ histogram = metricsCreator.createHistogramMetric(
+ "telegraf_in_latency", "The process latency of telegraf data",
+ new MetricsTag.Keys("protocol"), new MetricsTag.Values("http")
+ );
+
+ errorCounter = metricsCreator.createCounter(
+ "telegraf_error_count", "The error number of telegraf analysis",
+ new MetricsTag.Keys("protocol"), new MetricsTag.Values("http")
+ );
+ }
+
+ /**
+ * Convert the TelegrafData object to meter {@link Sample}
+ **/
+ public List<Sample> convertTelegraf(TelegrafData telegrafData) {
+
+ List<Sample> sampleList = new ArrayList<>(Collections.emptyList());
+
+ Map<String, Object> fields = telegrafData.getFields();
+ String name = telegrafData.getName();
+ Map<String, String> tags = telegrafData.getTags();
+ ImmutableMap<String, String> immutableTags = ImmutableMap.copyOf(tags);
+ long timestamp = telegrafData.getTimestamp();
+
+ fields.forEach((key, value) -> {
+ Sample.SampleBuilder builder = Sample.builder();
+ if (value instanceof String) return;
+ Sample sample = builder.name(name + "_" + key)
+ .timestamp(timestamp * 1000L)
Review Comment:
We should have a configuration to match telegraf configuration about time precision.
In the MAL, the timestamp(from Java spec) is ms, please refer to the Java spec
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] soander commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
soander commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r973545116
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/handler/TelegrafServiceHandler.java:
##########
@@ -0,0 +1,128 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.handler;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.google.common.collect.ImmutableMap;
+import com.linecorp.armeria.server.annotation.Post;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.skywalking.apm.network.common.v3.Commands;
+import org.apache.skywalking.oap.meter.analyzer.MetricConvert;
+import org.apache.skywalking.oap.meter.analyzer.dsl.Sample;
+import org.apache.skywalking.oap.meter.analyzer.dsl.SampleFamily;
+import org.apache.skywalking.oap.meter.analyzer.dsl.SampleFamilyBuilder;
+import org.apache.skywalking.oap.server.core.analysis.meter.MeterSystem;
+import org.apache.skywalking.oap.server.library.module.ModuleManager;
+import org.apache.skywalking.oap.server.receiver.telegraf.provider.config.TelegrafConfig;
+import org.apache.skywalking.oap.server.receiver.telegraf.provider.handler.pojo.TelegrafData;
+import org.apache.skywalking.oap.server.telemetry.TelemetryModule;
+import org.apache.skywalking.oap.server.telemetry.api.CounterMetrics;
+import org.apache.skywalking.oap.server.telemetry.api.HistogramMetrics;
+import org.apache.skywalking.oap.server.telemetry.api.MetricsCreator;
+import org.apache.skywalking.oap.server.telemetry.api.MetricsTag;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+@Slf4j
+public class TelegrafServiceHandler {
+
+ private final HistogramMetrics histogram;
+ private final CounterMetrics errorCounter;
+ private List<MetricConvert> metricConvert;
+
+ public TelegrafServiceHandler(ModuleManager moduleManager, MeterSystem meterSystem, List<TelegrafConfig> rules) {
+
+ this.metricConvert = rules.stream().map(r -> new MetricConvert(r, meterSystem)).collect(Collectors.toList());
+
+ final MetricsCreator metricsCreator = moduleManager.find(TelemetryModule.NAME)
+ .provider()
+ .getService(MetricsCreator.class);
+
+ histogram = metricsCreator.createHistogramMetric(
+ "telegraf_in_latency", "The process latency of telegraf data",
+ new MetricsTag.Keys("protocol"), new MetricsTag.Values("http")
+ );
+
+ errorCounter = metricsCreator.createCounter(
+ "telegraf_error_count", "The error number of telegraf analysis",
+ new MetricsTag.Keys("protocol"), new MetricsTag.Values("http")
+ );
+ }
+
+ /**
+ * Convert the TelegrafData object to meter {@link Sample}
+ **/
+ public List<Sample> convertTelegraf(TelegrafData telegrafData) {
+
+ List<Sample> sampleList = new ArrayList<>(Collections.emptyList());
+
+ Map<String, Object> fields = telegrafData.getFields();
+ String name = telegrafData.getName();
+ Map<String, String> tags = telegrafData.getTags();
+ ImmutableMap<String, String> immutableTags = ImmutableMap.copyOf(tags);
+ long timestamp = telegrafData.getTimestamp();
+
+ fields.forEach((key, value) -> {
+ Sample.SampleBuilder builder = Sample.builder();
+ if (value instanceof String) return;
+ Sample sample = builder.name(name + "_" + key)
+ .timestamp(timestamp * 1000L)
Review Comment:
> can you confirm the timestamp unit is `second` by default? I found a doc: https://docs.influxdata.com/telegraf/v1.23/data_formats/output/json/ maybe the timestamp unit is configurable.
Yeah, I can confirm that the default timestamp unit is `second`. Because I just set the `data_format = "json"`, and didn't configure the `json_timestamp_units`. According to output timestamp result, the default timestamp is `second`.
The following screenshot is my configuration.
<img width="538" alt="image" src="https://user-images.githubusercontent.com/79000735/190842615-9c08fc66-70be-405f-892c-8958de710c2b.png">
So what should I do to handle different timestamp unit, such as `"1ns", "1us", "1ms", "10ms"`? I will appreciate your guidance!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] soander commented on pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
soander commented on PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#issuecomment-1257540488
> @soander Could you put a summary about which are changed from last comment?
@wu-sheng Compared with the last comment:
**Code file**
-
- Changed code according to **suggested changes** like `createConfigBeanIfAbsent` method in the `TelegrafReceiverProvider` class, `ModuleStartException` in the `TelegrafConfigs` class
- Refactor code about **converting** JSON format telegraf data to Java POJO, created new class `TelegrafDatum` and rewrite class 'TelegrafData'
- Removed redundant `@JsonProperty` annotations in the class `TelegrafDatum`
**Configuration file**
-
- Add the telegraf module configuration in the `application.yml` to activate it
**Document file**
-
- Update the `changes.md`
- Added the `Design`, `Step`, `Attention` in PR description
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] soander commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
soander commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r1000329220
##########
docs/en/setup/backend/backend-vm-monitoring.md:
##########
@@ -3,38 +3,63 @@ SkyWalking leverages Prometheus node-exporter to collect metrics data from the V
[OpenTelemetry receiver](opentelemetry-receiver.md) and into the [Meter System](./../../concepts-and-designs/meter.md).
VM entity as a `Service` in OAP and on the `Layer: OS_LINUX`.
+SkyWalking also provides InfluxDB Telegraf to receive VMs' metrics data by [Telegraf receiver](./telegraf-receiver.md).
+The telegraf receiver plugin receiver, process and convert the metrics, then it send converted metrics to [Meter System](./../../concepts-and-designs/meter.md).
+VM entity as a `Service` in OAP and on the `Layer: OS_LINUX`.
+
## Data flow
+**For OpenTelemetry receiver:**
1. The Prometheus node-exporter collects metrics data from the VMs.
2. The OpenTelemetry Collector fetches metrics from node-exporter via Prometheus Receiver and pushes metrics to the SkyWalking OAP Server via the OpenCensus gRPC Exporter or OpenTelemetry gRPC exporter.
3. The SkyWalking OAP Server parses the expression with [MAL](../../concepts-and-designs/mal.md) to filter/calculate/aggregate and store the results.
+**For Telegraf receiver:**
+1. The InfluxDB Telegraf [input plugins](https://docs.influxdata.com/telegraf/v1.24/plugins/) collects various metrics data from the VMs.
+2. The InfluxDB Telegraf send `JSON` format metrics by `HTTP` messages to Telegraf Receiver, then pushes converted metrics to the SkyWalking OAP Server [Meter System](./../../concepts-and-designs/meter.md).
+3. The SkyWalking OAP Server parses the expression with [MAL](../../concepts-and-designs/mal.md) to filter/calculate/aggregate and store the results.
## Setup
-
+**For OpenTelemetry receiver:**
1. Setup [Prometheus node-exporter](https://prometheus.io/docs/guides/node-exporter/).
2. Setup [OpenTelemetry Collector ](https://opentelemetry.io/docs/collector/). This is an example for OpenTelemetry Collector configuration [otel-collector-config.yaml](../../../../test/e2e-v2/cases/vm/prometheus-node-exporter/otel-collector-config.yaml).
3. Config SkyWalking [OpenTelemetry receiver](opentelemetry-receiver.md).
+**For Telegraf receiver:**
+1. Setup InfluxDB Telegraf's `telegraf.conf file` according to [Telegraf office document](https://docs.influxdata.com/telegraf/v1.24/).
+2. Setup InfluxDB Telegraf's `telegraf.conf file` specific rules according to [Telegraf receiver document](telegraf-receiver.md).
+3. Config SkyWalking [Telegraf receiver](telegraf-receiver.md).
+
## Supported Metrics
+**For OpenTelemetry receiver:**
Review Comment:
> If they are same, keep dataflow doc is fine, but you don't have to list two sheets for the same metrics. Right?
But the metrics name and data source are different. If the telegraf receiver sheet is redundant I can delete it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r1000245837
##########
docs/en/setup/backend/backend-vm-monitoring.md:
##########
@@ -3,38 +3,63 @@ SkyWalking leverages Prometheus node-exporter to collect metrics data from the V
[OpenTelemetry receiver](opentelemetry-receiver.md) and into the [Meter System](./../../concepts-and-designs/meter.md).
VM entity as a `Service` in OAP and on the `Layer: OS_LINUX`.
+SkyWalking also provides InfluxDB Telegraf to receive VMs' metrics data by [Telegraf receiver](./telegraf-receiver.md).
+The telegraf receiver plugin receiver, process and convert the metrics, then it send converted metrics to [Meter System](./../../concepts-and-designs/meter.md).
+VM entity as a `Service` in OAP and on the `Layer: OS_LINUX`.
+
## Data flow
+**For OpenTelemetry receiver:**
1. The Prometheus node-exporter collects metrics data from the VMs.
2. The OpenTelemetry Collector fetches metrics from node-exporter via Prometheus Receiver and pushes metrics to the SkyWalking OAP Server via the OpenCensus gRPC Exporter or OpenTelemetry gRPC exporter.
3. The SkyWalking OAP Server parses the expression with [MAL](../../concepts-and-designs/mal.md) to filter/calculate/aggregate and store the results.
+**For Telegraf receiver:**
+1. The InfluxDB Telegraf [input plugins](https://docs.influxdata.com/telegraf/v1.24/plugins/) collects various metrics data from the VMs.
+2. The InfluxDB Telegraf send `JSON` format metrics by `HTTP` messages to Telegraf Receiver, then pushes converted metrics to the SkyWalking OAP Server [Meter System](./../../concepts-and-designs/meter.md).
+3. The SkyWalking OAP Server parses the expression with [MAL](../../concepts-and-designs/mal.md) to filter/calculate/aggregate and store the results.
## Setup
-
+**For OpenTelemetry receiver:**
1. Setup [Prometheus node-exporter](https://prometheus.io/docs/guides/node-exporter/).
2. Setup [OpenTelemetry Collector ](https://opentelemetry.io/docs/collector/). This is an example for OpenTelemetry Collector configuration [otel-collector-config.yaml](../../../../test/e2e-v2/cases/vm/prometheus-node-exporter/otel-collector-config.yaml).
3. Config SkyWalking [OpenTelemetry receiver](opentelemetry-receiver.md).
+**For Telegraf receiver:**
+1. Setup InfluxDB Telegraf's `telegraf.conf file` according to [Telegraf office document](https://docs.influxdata.com/telegraf/v1.24/).
+2. Setup InfluxDB Telegraf's `telegraf.conf file` specific rules according to [Telegraf receiver document](telegraf-receiver.md).
+3. Config SkyWalking [Telegraf receiver](telegraf-receiver.md).
+
## Supported Metrics
+**For OpenTelemetry receiver:**
Review Comment:
What does this mean? OpenTelemetry receiver and Telegraf receiver don't share the metrics?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] sonatype-lift[bot] commented on pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
sonatype-lift[bot] commented on PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#issuecomment-1280439757
:warning: **6 God Classes** were detected by Lift in this project. [Visit the Lift web console](https://lift.sonatype.com/results/github.com/apache/skywalking/01GFJCQRHYBTC5TGZP9AT2BFTS?tab=technical-debt&utm_source=github.com&utm_campaign=lift-comment&utm_content=apache\%20skywalking) for more details.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#issuecomment-1302919275
The one thing that needs to be fixed is this,
```
For the screenshot, the biggest issue is, SampleFamily should have a concept of the group for the bulk. We should not face one SampleFamily have metrics belonging to the same metric and labels, but with multiple timestamps.
Otherwise, the aggregation result would be strange and unpredictable.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#issuecomment-1296678495
@wankai123 @kezhenxu94 @mrproliu I received @liuhaoyang 's messages, this PR should be functional and working. Please take a deeper review.
This is his local testing. (Those two red rectangles are difference metrics from two metric sources.)
<img width="1512" alt="image" src="https://user-images.githubusercontent.com/5441976/198950670-56b0a5c1-beff-4f4a-9bf5-b6b6e2e9ecb9.png">
@soander @liuhaoyang It is good to see this is getting improved, we need more time to get this ready for the upstream user.
Considering this is a new protocol integration, we must be a little more cautiousγ
I know this may relate to Summer 2022 program somehow, if there is anything that needs my collaboration with the committee, please let me know.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] liuhaoyang merged pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
liuhaoyang merged PR #9620:
URL: https://github.com/apache/skywalking/pull/9620
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wankai123 commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
wankai123 commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r1013908075
##########
oap-server/server-starter/src/main/resources/telegraf-rules/vm.yaml:
##########
@@ -0,0 +1,72 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+expSuffix: service(['host'], Layer.OS_LINUX)
+metricPrefix: meter_vm
+metricsRules:
+
+ # cpu
+ - name: cpu_total_percentage
+ exp: cpu_usage_active.tagEqual('cpu', 'cpu-total')
+ - name: cpu_average_used
+ exp: cpu_usage_active.tagNotEqual('cpu', 'cpu-total').avg(['host', 'cpu'])
Review Comment:
Maybe this expression does not calculate the `The percentage usage of the CPU core in each mode` as the doc explains.
If this metric has a different meaning by using the telegraf, I think you should better notice it in the doc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] soander commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
soander commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r977221289
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/handler/TelegrafServiceHandler.java:
##########
@@ -0,0 +1,128 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.handler;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.google.common.collect.ImmutableMap;
+import com.linecorp.armeria.server.annotation.Post;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.skywalking.apm.network.common.v3.Commands;
+import org.apache.skywalking.oap.meter.analyzer.MetricConvert;
+import org.apache.skywalking.oap.meter.analyzer.dsl.Sample;
+import org.apache.skywalking.oap.meter.analyzer.dsl.SampleFamily;
+import org.apache.skywalking.oap.meter.analyzer.dsl.SampleFamilyBuilder;
+import org.apache.skywalking.oap.server.core.analysis.meter.MeterSystem;
+import org.apache.skywalking.oap.server.library.module.ModuleManager;
+import org.apache.skywalking.oap.server.receiver.telegraf.provider.config.TelegrafConfig;
+import org.apache.skywalking.oap.server.receiver.telegraf.provider.handler.pojo.TelegrafData;
+import org.apache.skywalking.oap.server.telemetry.TelemetryModule;
+import org.apache.skywalking.oap.server.telemetry.api.CounterMetrics;
+import org.apache.skywalking.oap.server.telemetry.api.HistogramMetrics;
+import org.apache.skywalking.oap.server.telemetry.api.MetricsCreator;
+import org.apache.skywalking.oap.server.telemetry.api.MetricsTag;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+@Slf4j
+public class TelegrafServiceHandler {
+
+ private final HistogramMetrics histogram;
+ private final CounterMetrics errorCounter;
+ private List<MetricConvert> metricConvert;
+
+ public TelegrafServiceHandler(ModuleManager moduleManager, MeterSystem meterSystem, List<TelegrafConfig> rules) {
+
+ this.metricConvert = rules.stream().map(r -> new MetricConvert(r, meterSystem)).collect(Collectors.toList());
+
+ final MetricsCreator metricsCreator = moduleManager.find(TelemetryModule.NAME)
+ .provider()
+ .getService(MetricsCreator.class);
+
+ histogram = metricsCreator.createHistogramMetric(
+ "telegraf_in_latency", "The process latency of telegraf data",
+ new MetricsTag.Keys("protocol"), new MetricsTag.Values("http")
+ );
+
+ errorCounter = metricsCreator.createCounter(
+ "telegraf_error_count", "The error number of telegraf analysis",
+ new MetricsTag.Keys("protocol"), new MetricsTag.Values("http")
+ );
+ }
+
+ /**
+ * Convert the TelegrafData object to meter {@link Sample}
+ **/
+ public List<Sample> convertTelegraf(TelegrafData telegrafData) {
+
+ List<Sample> sampleList = new ArrayList<>(Collections.emptyList());
+
+ Map<String, Object> fields = telegrafData.getFields();
+ String name = telegrafData.getName();
+ Map<String, String> tags = telegrafData.getTags();
+ ImmutableMap<String, String> immutableTags = ImmutableMap.copyOf(tags);
+ long timestamp = telegrafData.getTimestamp();
+
+ fields.forEach((key, value) -> {
+ Sample.SampleBuilder builder = Sample.builder();
+ if (value instanceof String) return;
+ Sample sample = builder.name(name + "_" + key)
+ .timestamp(timestamp * 1000L)
+ .value(((Number) value).doubleValue())
+ .labels(immutableTags).build();
+
+ sampleList.add(sample);
+ });
+ return sampleList;
+
+ }
+
+ @Post("/telegraf")
+ public Commands collectData(String jsonInfo) {
+ TelegrafData telegrafData;
+ try (HistogramMetrics.Timer ignored = histogram.createTimer()) {
+ List<Sample> allSamples = new ArrayList<>();
+ ObjectMapper mapper = new ObjectMapper();
+ JsonNode node = mapper.readTree(jsonInfo);
Review Comment:
> Can you rename the original `TelegrafData` class to `TelegrafDatum`, and define a class `TelegrafData` to contain a field `List<TelegrafDatum> metrics`, so you don't need to `readTree` here, and just declare the method `public Commands collectData(TelegrafData telegrafData) {`
I change code according to your suggestion. Everything works fine.
This is `TelegrafData` class.
This is `public Commands collectData(TelegrafData telegrafData)` method.
Is this modification good? If there are still problems, what else do I need to fix? Thank you for your guidance!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#issuecomment-1286663423
https://github.com/apache/skywalking/actions/runs/3288941909
You could find logs from this page, at the bottom of the page. Most likely, the package of this branch can't boot up.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] liuhaoyang commented on pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
liuhaoyang commented on PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#issuecomment-1286660477
@wu-sheng I see all the e2e tests return failure, what could be causing this ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] kezhenxu94 commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
kezhenxu94 commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r973546199
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/config/TelegrafConfigs.java:
##########
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.config;
+
+import lombok.extern.slf4j.Slf4j;
+import org.apache.skywalking.oap.server.library.module.ModuleStartException;
+import org.apache.skywalking.oap.server.library.util.CollectionUtils;
+import org.apache.skywalking.oap.server.library.util.ResourceUtils;
+import org.yaml.snakeyaml.Yaml;
+
+import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.FileReader;
+import java.io.IOException;
+import java.io.Reader;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Objects;
+import java.util.stream.Collectors;
+
+@Slf4j
+public class TelegrafConfigs {
+ public static List<TelegrafConfig> loadConfigs(String path, List<String> fileNames) throws ModuleStartException {
+ if (CollectionUtils.isEmpty(fileNames)) {
+ return Collections.emptyList();
+ }
+
+ File[] configs;
+ try {
+ configs = ResourceUtils.getPathFiles(path);
+ } catch (FileNotFoundException e) {
+ throw new ModuleStartException("Load telegraf configs failed", e);
+ }
+
+ return Arrays.stream(configs)
+ .filter(File::isFile)
+ .map(f -> {
+ String fileName = f.getName();
+ int dotIndex = fileName.lastIndexOf('.');
+ fileName = (dotIndex == -1) ? fileName : fileName.substring(0, dotIndex);
+ if (!fileNames.contains(fileName)) {
+ return null;
+ }
+ try (Reader r = new FileReader(f)) {
+ return new Yaml().loadAs(r, TelegrafConfig.class);
+ } catch (IOException e) {
+ log.warn("Reading file {} failed", f, e);
Review Comment:
Please throw module start exception, this is a fatal error
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r972851604
##########
oap-server/server-starter/src/main/resources/telegraf-rules/default.yaml:
##########
@@ -0,0 +1,50 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+expSuffix: service(['host'], Layer.OS_LINUX)
Review Comment:
Is this file working well as same as OTEL Linux monitoring? Could we activate them at the same time?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] kezhenxu94 commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
kezhenxu94 commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r973549453
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/config/TelegrafConfigs.java:
##########
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.config;
+
+import lombok.extern.slf4j.Slf4j;
+import org.apache.skywalking.oap.server.library.module.ModuleStartException;
+import org.apache.skywalking.oap.server.library.util.CollectionUtils;
+import org.apache.skywalking.oap.server.library.util.ResourceUtils;
+import org.yaml.snakeyaml.Yaml;
+
+import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.FileReader;
+import java.io.IOException;
+import java.io.Reader;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Objects;
+import java.util.stream.Collectors;
+
+@Slf4j
+public class TelegrafConfigs {
+ public static List<TelegrafConfig> loadConfigs(String path, List<String> fileNames) throws ModuleStartException {
+ if (CollectionUtils.isEmpty(fileNames)) {
+ return Collections.emptyList();
+ }
+
+ File[] configs;
+ try {
+ configs = ResourceUtils.getPathFiles(path);
+ } catch (FileNotFoundException e) {
+ throw new ModuleStartException("Load telegraf configs failed", e);
+ }
+
+ return Arrays.stream(configs)
+ .filter(File::isFile)
+ .map(f -> {
+ String fileName = f.getName();
+ int dotIndex = fileName.lastIndexOf('.');
+ fileName = (dotIndex == -1) ? fileName : fileName.substring(0, dotIndex);
+ if (!fileNames.contains(fileName)) {
+ return null;
+ }
+ try (Reader r = new FileReader(f)) {
+ return new Yaml().loadAs(r, TelegrafConfig.class);
+ } catch (IOException e) {
+ log.warn("Reading file {} failed", f, e);
Review Comment:
> > Please throw module start exception, this is a fatal error
>
> I set the `loadConfigs` method throws the `ModuleStartException` , should I change it? 
I mean you should throw a `ModuleStartException` here, logging it is not enough.
```suggestion
throw new ModuleStartException(e);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] sonatype-lift[bot] commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
sonatype-lift[bot] commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r972876306
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/config/TelegrafConfigs.java:
##########
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.config;
+
+import lombok.extern.slf4j.Slf4j;
+import org.apache.skywalking.oap.server.library.module.ModuleStartException;
+import org.apache.skywalking.oap.server.library.util.CollectionUtils;
+import org.apache.skywalking.oap.server.library.util.ResourceUtils;
+import org.yaml.snakeyaml.Yaml;
+
+import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.FileReader;
+import java.io.IOException;
+import java.io.Reader;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Objects;
+import java.util.stream.Collectors;
+
+@Slf4j
+public class TelegrafConfigs {
+ public static List<TelegrafConfig> loadConfigs(String path, List<String> fileNames) throws ModuleStartException {
+ if (CollectionUtils.isEmpty(fileNames)) {
+ return Collections.emptyList();
+ }
+
+ File[] configs;
+ try {
+ configs = ResourceUtils.getPathFiles(path);
+ } catch (FileNotFoundException e) {
+ throw new ModuleStartException("Load telegraf configs failed", e);
+ }
+
+ return Arrays.stream(configs)
+ .filter(File::isFile)
+ .map(f -> {
+ String fileName = f.getName();
+ int dotIndex = fileName.lastIndexOf('.');
+ fileName = (dotIndex == -1) ? fileName : fileName.substring(0, dotIndex);
+ if (!fileNames.contains(fileName)) {
+ return null;
+ }
+ try (Reader r = new FileReader(f)) {
Review Comment:
*[DefaultCharset](https://errorprone.info/bugpattern/DefaultCharset):* Implicit use of the platform default charset, which can result in differing behaviour between JVM executions or incorrect behavior if the encoding of the data source doesn't match expectations.
---
```suggestion
try (Reader r = Files.newBufferedReader(f.toPath(), Charset.defaultCharset())) {
```
---
<details><summary><b>βΉοΈ Learn about @sonatype-lift commands</b></summary>
You can reply with the following commands. For example, reply with ***@sonatype-lift ignoreall*** to leave out all findings.
| **Command** | **Usage** |
| ------------- | ------------- |
| `@sonatype-lift ignore` | Leave out the above finding from this PR |
| `@sonatype-lift ignoreall` | Leave out all the existing findings from this PR |
| `@sonatype-lift exclude <file\|issue\|path\|tool>` | Exclude specified `file\|issue\|path\|tool` from Lift findings by updating your config.toml file |
**Note:** When talking to LiftBot, you need to **refresh** the page to see its response.
<sub>[Click here](https://github.com/apps/sonatype-lift/installations/new) to add LiftBot to another repo.</sub></details>
---
Was this a good recommendation?
[ [π Not relevant](https://www.sonatype.com/lift-comment-rating?comment=333280424&lift_comment_rating=1) ] - [ [π Won't fix](https://www.sonatype.com/lift-comment-rating?comment=333280424&lift_comment_rating=2) ] - [ [π Not critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=333280424&lift_comment_rating=3) ] - [ [π Critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=333280424&lift_comment_rating=4) ] - [ [π Critical, fixing now](https://www.sonatype.com/lift-comment-rating?comment=333280424&lift_comment_rating=5) ]
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/config/TelegrafConfig.java:
##########
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.config;
+
+import lombok.Data;
+import lombok.NoArgsConstructor;
+import org.apache.skywalking.oap.meter.analyzer.MetricRuleConfig;
+
+import java.util.List;
+
+@Data
+public class TelegrafConfig implements MetricRuleConfig {
+
+ private String metricPrefix;
+ private String expSuffix;
+ private String expPrefix;
+ private String filter;
+ private String initExp;
+ private List<Rule> metricsRules;
+
+ @Data
+ @NoArgsConstructor
+ public static class Rule implements RuleConfig {
+ private String name;
Review Comment:
*[MissingOverride](https://errorprone.info/bugpattern/MissingOverride):* getName implements method in RuleConfig; expected @Override
---
```suggestion
private String @Override name;
```
---
<details><summary><b>βΉοΈ Learn about @sonatype-lift commands</b></summary>
You can reply with the following commands. For example, reply with ***@sonatype-lift ignoreall*** to leave out all findings.
| **Command** | **Usage** |
| ------------- | ------------- |
| `@sonatype-lift ignore` | Leave out the above finding from this PR |
| `@sonatype-lift ignoreall` | Leave out all the existing findings from this PR |
| `@sonatype-lift exclude <file\|issue\|path\|tool>` | Exclude specified `file\|issue\|path\|tool` from Lift findings by updating your config.toml file |
**Note:** When talking to LiftBot, you need to **refresh** the page to see its response.
<sub>[Click here](https://github.com/apps/sonatype-lift/installations/new) to add LiftBot to another repo.</sub></details>
---
Was this a good recommendation?
[ [π Not relevant](https://www.sonatype.com/lift-comment-rating?comment=333280309&lift_comment_rating=1) ] - [ [π Won't fix](https://www.sonatype.com/lift-comment-rating?comment=333280309&lift_comment_rating=2) ] - [ [π Not critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=333280309&lift_comment_rating=3) ] - [ [π Critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=333280309&lift_comment_rating=4) ] - [ [π Critical, fixing now](https://www.sonatype.com/lift-comment-rating?comment=333280309&lift_comment_rating=5) ]
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/config/TelegrafConfig.java:
##########
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.config;
+
+import lombok.Data;
+import lombok.NoArgsConstructor;
+import org.apache.skywalking.oap.meter.analyzer.MetricRuleConfig;
+
+import java.util.List;
+
+@Data
+public class TelegrafConfig implements MetricRuleConfig {
+
+ private String metricPrefix;
+ private String expSuffix;
+ private String expPrefix;
Review Comment:
*[MissingOverride](https://errorprone.info/bugpattern/MissingOverride):* getExpPrefix implements method in MetricRuleConfig; expected @Override
---
```suggestion
private String @Override expPrefix;
```
---
<details><summary><b>βΉοΈ Learn about @sonatype-lift commands</b></summary>
You can reply with the following commands. For example, reply with ***@sonatype-lift ignoreall*** to leave out all findings.
| **Command** | **Usage** |
| ------------- | ------------- |
| `@sonatype-lift ignore` | Leave out the above finding from this PR |
| `@sonatype-lift ignoreall` | Leave out all the existing findings from this PR |
| `@sonatype-lift exclude <file\|issue\|path\|tool>` | Exclude specified `file\|issue\|path\|tool` from Lift findings by updating your config.toml file |
**Note:** When talking to LiftBot, you need to **refresh** the page to see its response.
<sub>[Click here](https://github.com/apps/sonatype-lift/installations/new) to add LiftBot to another repo.</sub></details>
---
Was this a good recommendation?
[ [π Not relevant](https://www.sonatype.com/lift-comment-rating?comment=333280322&lift_comment_rating=1) ] - [ [π Won't fix](https://www.sonatype.com/lift-comment-rating?comment=333280322&lift_comment_rating=2) ] - [ [π Not critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=333280322&lift_comment_rating=3) ] - [ [π Critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=333280322&lift_comment_rating=4) ] - [ [π Critical, fixing now](https://www.sonatype.com/lift-comment-rating?comment=333280322&lift_comment_rating=5) ]
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/config/TelegrafConfig.java:
##########
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.config;
+
+import lombok.Data;
+import lombok.NoArgsConstructor;
+import org.apache.skywalking.oap.meter.analyzer.MetricRuleConfig;
+
+import java.util.List;
+
+@Data
+public class TelegrafConfig implements MetricRuleConfig {
+
+ private String metricPrefix;
+ private String expSuffix;
+ private String expPrefix;
+ private String filter;
+ private String initExp;
+ private List<Rule> metricsRules;
Review Comment:
*[MissingOverride](https://errorprone.info/bugpattern/MissingOverride):* getMetricsRules implements method in MetricRuleConfig; expected @Override
---
```suggestion
private List<Rule> @Override metricsRules;
```
---
<details><summary><b>βΉοΈ Learn about @sonatype-lift commands</b></summary>
You can reply with the following commands. For example, reply with ***@sonatype-lift ignoreall*** to leave out all findings.
| **Command** | **Usage** |
| ------------- | ------------- |
| `@sonatype-lift ignore` | Leave out the above finding from this PR |
| `@sonatype-lift ignoreall` | Leave out all the existing findings from this PR |
| `@sonatype-lift exclude <file\|issue\|path\|tool>` | Exclude specified `file\|issue\|path\|tool` from Lift findings by updating your config.toml file |
**Note:** When talking to LiftBot, you need to **refresh** the page to see its response.
<sub>[Click here](https://github.com/apps/sonatype-lift/installations/new) to add LiftBot to another repo.</sub></details>
---
Was this a good recommendation?
[ [π Not relevant](https://www.sonatype.com/lift-comment-rating?comment=333280334&lift_comment_rating=1) ] - [ [π Won't fix](https://www.sonatype.com/lift-comment-rating?comment=333280334&lift_comment_rating=2) ] - [ [π Not critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=333280334&lift_comment_rating=3) ] - [ [π Critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=333280334&lift_comment_rating=4) ] - [ [π Critical, fixing now](https://www.sonatype.com/lift-comment-rating?comment=333280334&lift_comment_rating=5) ]
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/config/TelegrafConfig.java:
##########
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.config;
+
+import lombok.Data;
+import lombok.NoArgsConstructor;
+import org.apache.skywalking.oap.meter.analyzer.MetricRuleConfig;
+
+import java.util.List;
+
+@Data
+public class TelegrafConfig implements MetricRuleConfig {
+
+ private String metricPrefix;
+ private String expSuffix;
Review Comment:
*[MissingOverride](https://errorprone.info/bugpattern/MissingOverride):* getExpSuffix implements method in MetricRuleConfig; expected @Override
---
```suggestion
private String @Override expSuffix;
```
---
<details><summary><b>βΉοΈ Learn about @sonatype-lift commands</b></summary>
You can reply with the following commands. For example, reply with ***@sonatype-lift ignoreall*** to leave out all findings.
| **Command** | **Usage** |
| ------------- | ------------- |
| `@sonatype-lift ignore` | Leave out the above finding from this PR |
| `@sonatype-lift ignoreall` | Leave out all the existing findings from this PR |
| `@sonatype-lift exclude <file\|issue\|path\|tool>` | Exclude specified `file\|issue\|path\|tool` from Lift findings by updating your config.toml file |
**Note:** When talking to LiftBot, you need to **refresh** the page to see its response.
<sub>[Click here](https://github.com/apps/sonatype-lift/installations/new) to add LiftBot to another repo.</sub></details>
---
Was this a good recommendation?
[ [π Not relevant](https://www.sonatype.com/lift-comment-rating?comment=333280349&lift_comment_rating=1) ] - [ [π Won't fix](https://www.sonatype.com/lift-comment-rating?comment=333280349&lift_comment_rating=2) ] - [ [π Not critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=333280349&lift_comment_rating=3) ] - [ [π Critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=333280349&lift_comment_rating=4) ] - [ [π Critical, fixing now](https://www.sonatype.com/lift-comment-rating?comment=333280349&lift_comment_rating=5) ]
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/config/TelegrafConfig.java:
##########
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.config;
+
+import lombok.Data;
+import lombok.NoArgsConstructor;
+import org.apache.skywalking.oap.meter.analyzer.MetricRuleConfig;
+
+import java.util.List;
+
+@Data
+public class TelegrafConfig implements MetricRuleConfig {
+
+ private String metricPrefix;
+ private String expSuffix;
+ private String expPrefix;
+ private String filter;
Review Comment:
*[MissingOverride](https://errorprone.info/bugpattern/MissingOverride):* getFilter implements method in MetricRuleConfig; expected @Override
---
```suggestion
private String @Override filter;
```
---
<details><summary><b>βΉοΈ Learn about @sonatype-lift commands</b></summary>
You can reply with the following commands. For example, reply with ***@sonatype-lift ignoreall*** to leave out all findings.
| **Command** | **Usage** |
| ------------- | ------------- |
| `@sonatype-lift ignore` | Leave out the above finding from this PR |
| `@sonatype-lift ignoreall` | Leave out all the existing findings from this PR |
| `@sonatype-lift exclude <file\|issue\|path\|tool>` | Exclude specified `file\|issue\|path\|tool` from Lift findings by updating your config.toml file |
**Note:** When talking to LiftBot, you need to **refresh** the page to see its response.
<sub>[Click here](https://github.com/apps/sonatype-lift/installations/new) to add LiftBot to another repo.</sub></details>
---
Was this a good recommendation?
[ [π Not relevant](https://www.sonatype.com/lift-comment-rating?comment=333280433&lift_comment_rating=1) ] - [ [π Won't fix](https://www.sonatype.com/lift-comment-rating?comment=333280433&lift_comment_rating=2) ] - [ [π Not critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=333280433&lift_comment_rating=3) ] - [ [π Critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=333280433&lift_comment_rating=4) ] - [ [π Critical, fixing now](https://www.sonatype.com/lift-comment-rating?comment=333280433&lift_comment_rating=5) ]
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/config/TelegrafConfig.java:
##########
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.config;
+
+import lombok.Data;
+import lombok.NoArgsConstructor;
+import org.apache.skywalking.oap.meter.analyzer.MetricRuleConfig;
+
+import java.util.List;
+
+@Data
+public class TelegrafConfig implements MetricRuleConfig {
+
+ private String metricPrefix;
+ private String expSuffix;
+ private String expPrefix;
+ private String filter;
+ private String initExp;
+ private List<Rule> metricsRules;
+
+ @Data
+ @NoArgsConstructor
+ public static class Rule implements RuleConfig {
+ private String name;
+ private String exp;
Review Comment:
*[MissingOverride](https://errorprone.info/bugpattern/MissingOverride):* getExp implements method in RuleConfig; expected @Override
---
```suggestion
private String @Override exp;
```
---
<details><summary><b>βΉοΈ Learn about @sonatype-lift commands</b></summary>
You can reply with the following commands. For example, reply with ***@sonatype-lift ignoreall*** to leave out all findings.
| **Command** | **Usage** |
| ------------- | ------------- |
| `@sonatype-lift ignore` | Leave out the above finding from this PR |
| `@sonatype-lift ignoreall` | Leave out all the existing findings from this PR |
| `@sonatype-lift exclude <file\|issue\|path\|tool>` | Exclude specified `file\|issue\|path\|tool` from Lift findings by updating your config.toml file |
**Note:** When talking to LiftBot, you need to **refresh** the page to see its response.
<sub>[Click here](https://github.com/apps/sonatype-lift/installations/new) to add LiftBot to another repo.</sub></details>
---
Was this a good recommendation?
[ [π Not relevant](https://www.sonatype.com/lift-comment-rating?comment=333280522&lift_comment_rating=1) ] - [ [π Won't fix](https://www.sonatype.com/lift-comment-rating?comment=333280522&lift_comment_rating=2) ] - [ [π Not critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=333280522&lift_comment_rating=3) ] - [ [π Critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=333280522&lift_comment_rating=4) ] - [ [π Critical, fixing now](https://www.sonatype.com/lift-comment-rating?comment=333280522&lift_comment_rating=5) ]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#issuecomment-1296285305
I still noticed there are several mentioned things not been fixed.
1. There is no unit test to cover telegraf format converting. `TelegrafServiceHandler` is a HTTP handler. What it received is an HTTP text, what it looks like, and how it could be converted to SampleFamily should be well tested, including legal text or illegal text cases.
2. You don't example what is the relationship between this VM monitoring and OpenTelemetry's Linux monitoring.
3. In [backend-vm-monitoring.md](https://github.com/apache/skywalking/pull/9620/files#diff-9b751b71ba658be11f59887b6462a97bd3675a181c3cd3fcf7d5c4e3d452c295), you mentioned, there are two ways to support VM monitoring. But they seem not the same, such as not using the same metric name. So, how the dashboard would work? In the document, there is not any document mentioning that.
I still can't follow the whole picture of this feature implementation. Could any of you explain it? We can't deliver this to the community with this confusion. Especially, `telegraf-rules/vm.yaml` is included as a part of the official release.
All default activated MAL scripts match with the dashboard settings, as well as the document description about the data flow.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] kezhenxu94 commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
kezhenxu94 commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r1008977366
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/config/TelegrafConfigs.java:
##########
@@ -0,0 +1,75 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.config;
+
+import lombok.extern.slf4j.Slf4j;
+import org.apache.skywalking.oap.server.library.module.ModuleStartException;
+import org.apache.skywalking.oap.server.library.util.CollectionUtils;
+import org.apache.skywalking.oap.server.library.util.ResourceUtils;
+import org.yaml.snakeyaml.Yaml;
+
+import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.FileReader;
+import java.io.IOException;
+import java.io.Reader;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Objects;
+import java.util.stream.Collectors;
+
+@Slf4j
+public class TelegrafConfigs {
+ public static List<TelegrafConfig> loadConfigs(String path, List<String> fileNames) throws ModuleStartException {
+ if (CollectionUtils.isEmpty(fileNames)) {
+ return Collections.emptyList();
+ }
+
+ File[] configs;
+ try {
+ configs = ResourceUtils.getPathFiles(path);
+ } catch (FileNotFoundException e) {
+ throw new ModuleStartException("Load telegraf configs failed", e);
+ }
+
+ return Arrays.stream(configs)
+ .filter(File::isFile)
+ .map(f -> {
+ String fileName = f.getName();
+ int dotIndex = fileName.lastIndexOf('.');
+ fileName = (dotIndex == -1) ? fileName : fileName.substring(0, dotIndex);
+ if (!fileNames.contains(fileName)) {
+ return null;
+ }
+ try (Reader r = new FileReader(f)) {
+ return new Yaml().loadAs(r, TelegrafConfig.class);
+ } catch (IOException e) {
+ log.warn("Reading file {} failed", f, e);
+ try {
+ throw new ModuleStartException(e.getMessage(), e);
+ } catch (ModuleStartException ex) {
+ return null;
+ }
Review Comment:
What the meaning here π€―
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#issuecomment-1296655539
The document seems should be polished after the code's part is ready.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] liuhaoyang commented on pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
liuhaoyang commented on PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#issuecomment-1302098904
There seems to be a problem with the e2e test case of vm-telegraf, @soander can you fix it?
Alse, I used the latest commit to test, and the dashboards can be displayed normally (I did not find the Filefd Allocated indicator collection plugin in telegraf, so this chart has no data display for the time being
Waiting for your review @wu-sheng @mrproliu @kezhenxu94 , thanks again
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r1013956364
##########
oap-server/server-starter/src/main/resources/telegraf-rules/vm.yaml:
##########
@@ -0,0 +1,72 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+expSuffix: service(['host'], Layer.OS_LINUX)
+metricPrefix: meter_vm
+metricsRules:
+
+ # cpu
+ - name: cpu_total_percentage
+ exp: cpu_usage_active.tagEqual('cpu', 'cpu-total')
+ - name: cpu_average_used
+ exp: cpu_usage_active.tagNotEqual('cpu', 'cpu-total').avg(['host', 'cpu'])
Review Comment:
@soander Please take a look. We may need some polish.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r1012564291
##########
docs/menu.yml:
##########
@@ -115,6 +115,8 @@ catalog:
path: "/en/guides/backend-oal-scripts"
- name: "OpenTelemetry Metrics"
path: "/en/setup/backend/opentelemetry-receiver"
+ - name: "Telegraf Metrics"
+ path: "/en/setup/backend/telegraf-receiver"
Review Comment:
Move this next to the Meter Analysis.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r1009076618
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/test/java/org/apache/skywalking/oap/server/receiver/telegraf/JSONConvertTest.java:
##########
@@ -0,0 +1,126 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf;
+
+import com.google.common.collect.ImmutableMap;
+import lombok.Builder;
+import lombok.EqualsAndHashCode;
+import lombok.ToString;
+import lombok.Getter;
+import lombok.AllArgsConstructor;
+import org.apache.skywalking.oap.server.receiver.telegraf.provider.handler.TelegrafServiceHandler;
+import org.apache.skywalking.oap.server.receiver.telegraf.provider.handler.pojo.TelegrafDatum;
+import org.junit.Before;
+import org.junit.Ignore;
+import org.junit.Test;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import static org.junit.Assert.assertEquals;
+
+public class JSONConvertTest {
Review Comment:
Could you check `ZabbixMetricsTest` and `ZabbixProtocolHandlerTest` about how to do UT? What we need to test is that, we have a JSON text(same format as telegraf reported, that could link to a doc), then, the UT calls some methods in your handler(or something here). Then we could verify the sample family with expectations.
In the current one, you seem using a mock class to test another mock class.
cc @kezhenxu94
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] liuhaoyang commented on pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
liuhaoyang commented on PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#issuecomment-1296439661
> I still noticed there are several mentioned things not been fixed.
>
> 1. There is no unit test to cover telegraf format converting. `TelegrafServiceHandler` is a HTTP handler. What it received is an HTTP text, what it looks like, and how it could be converted to SampleFamily should be well tested, including legal text or illegal text cases.
> 2. You don't explain what is the relationship between this VM monitoring and OpenTelemetry's Linux monitoring.
> 3. In [backend-vm-monitoring.md](https://github.com/apache/skywalking/pull/9620/files#diff-9b751b71ba658be11f59887b6462a97bd3675a181c3cd3fcf7d5c4e3d452c295), you mentioned, there are two ways to support VM monitoring. But they seem not the same, such as not using the same metric name. So, how the dashboard would work? In the document, there is not any document mentioning that.
>
> I still can't follow the whole picture of this feature implementation. Could any of you explain it? We can't deliver this to the community with this confusion. Especially, `telegraf-rules/vm.yaml` is included as a part of the official release.
>
> All default activated MAL scripts match with the dashboard settings, as well as the document description about the data flow.
This PR contains two features
1. Support telegraf-receiver to receive data in telegraf json format and write it to storage
2. Provide the default MAL configuration, and parse the CPU, memory and system metrics reported by telegraf into vm meter, so that these metrics can be displayed on the linux monitoring page. (No disk and network metrics are provided for the time being
For 1, users can use their own MAL to parse more telegraf metrics.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] soander commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
soander commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r1009038949
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/config/TelegrafConfigs.java:
##########
@@ -0,0 +1,75 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.config;
+
+import lombok.extern.slf4j.Slf4j;
+import org.apache.skywalking.oap.server.library.module.ModuleStartException;
+import org.apache.skywalking.oap.server.library.util.CollectionUtils;
+import org.apache.skywalking.oap.server.library.util.ResourceUtils;
+import org.yaml.snakeyaml.Yaml;
+
+import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.FileReader;
+import java.io.IOException;
+import java.io.Reader;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Objects;
+import java.util.stream.Collectors;
+
+@Slf4j
+public class TelegrafConfigs {
+ public static List<TelegrafConfig> loadConfigs(String path, List<String> fileNames) throws ModuleStartException {
+ if (CollectionUtils.isEmpty(fileNames)) {
+ return Collections.emptyList();
+ }
+
+ File[] configs;
+ try {
+ configs = ResourceUtils.getPathFiles(path);
+ } catch (FileNotFoundException e) {
+ throw new ModuleStartException("Load telegraf configs failed", e);
+ }
+
+ return Arrays.stream(configs)
+ .filter(File::isFile)
+ .map(f -> {
+ String fileName = f.getName();
+ int dotIndex = fileName.lastIndexOf('.');
+ fileName = (dotIndex == -1) ? fileName : fileName.substring(0, dotIndex);
+ if (!fileNames.contains(fileName)) {
+ return null;
+ }
+ try (Reader r = new FileReader(f)) {
+ return new Yaml().loadAs(r, TelegrafConfig.class);
+ } catch (IOException e) {
+ log.warn("Reading file {} failed", f, e);
+ try {
+ throw new ModuleStartException(e.getMessage(), e);
+ } catch (ModuleStartException ex) {
+ return null;
+ }
Review Comment:
> What the meaning here π€―
It's an error, I already fix it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] soander commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
soander commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r1009038864
##########
oap-server/server-receiver-plugin/skywalking-telegraf-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/telegraf/provider/config/TelegrafConfigs.java:
##########
@@ -0,0 +1,75 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.telegraf.provider.config;
+
+import lombok.extern.slf4j.Slf4j;
+import org.apache.skywalking.oap.server.library.module.ModuleStartException;
+import org.apache.skywalking.oap.server.library.util.CollectionUtils;
+import org.apache.skywalking.oap.server.library.util.ResourceUtils;
+import org.yaml.snakeyaml.Yaml;
+
+import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.FileReader;
+import java.io.IOException;
+import java.io.Reader;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.List;
+import java.util.Objects;
+import java.util.stream.Collectors;
+
+@Slf4j
+public class TelegrafConfigs {
+ public static List<TelegrafConfig> loadConfigs(String path, List<String> fileNames) throws ModuleStartException {
+ if (CollectionUtils.isEmpty(fileNames)) {
+ return Collections.emptyList();
+ }
+
+ File[] configs;
+ try {
+ configs = ResourceUtils.getPathFiles(path);
+ } catch (FileNotFoundException e) {
+ throw new ModuleStartException("Load telegraf configs failed", e);
+ }
+
+ return Arrays.stream(configs)
+ .filter(File::isFile)
+ .map(f -> {
+ String fileName = f.getName();
+ int dotIndex = fileName.lastIndexOf('.');
+ fileName = (dotIndex == -1) ? fileName : fileName.substring(0, dotIndex);
+ if (!fileNames.contains(fileName)) {
+ return null;
+ }
+ try (Reader r = new FileReader(f)) {
+ return new Yaml().loadAs(r, TelegrafConfig.class);
+ } catch (IOException e) {
+ log.warn("Reading file {} failed", f, e);
+ try {
+ throw new ModuleStartException(e.getMessage(), e);
+ } catch (ModuleStartException ex) {
+ return null;
+ }
Review Comment:
It's an error, I already fix it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] liuhaoyang commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
liuhaoyang commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r1009087136
##########
oap-server/server-starter/src/main/resources/ui-initialized-templates/os_linux/linux-service.json:
##########
@@ -156,10 +156,12 @@
"showYAxis": true
},
"metrics": [
- "meter_vm_filesystem_percentage"
+ "meter_vm_filesystem_percentage",
+ "meter_vm_filesystem_percentage_telegraf"
],
"metricTypes": [
- "readLabeledMetricsValues"
+ "readLabeledMetricsValues",
+ "readMetricsValues"
Review Comment:
I don't think these two metrics will appear at the same time in the same dashboard. The two indicators are only for compatibility with the data that the chart can query.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#issuecomment-1304387487
@liuhaoyang Please make sure squash merge log is short and clear.
You can see there are a lot of noices in the last commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#issuecomment-1303762437
One more thing, `telegraf` should be added as a label for our repo, https://github.com/apache/skywalking/blob/master/.asf.yaml
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] soander commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
soander commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r977014600
##########
oap-server/server-starter/src/main/resources/telegraf-rules/default.yaml:
##########
@@ -0,0 +1,50 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+expSuffix: service(['host'], Layer.OS_LINUX)
Review Comment:
> Is this file working well as same as OTEL Linux monitoring? Could we activate them at the same time?
I am not sure whether they can activate at the same time, I will test it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#issuecomment-1285071358
What is the current status of this PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] liuhaoyang commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
liuhaoyang commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r1000322616
##########
test/e2e-v2/cases/vm/telegraf/telegraf.conf:
##########
@@ -0,0 +1,9559 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
Review Comment:
`telegraf.conf` can be made shorter. You should delete unused configs instead of just commenting them out.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] soander commented on a diff in pull request #9620: Support the telegraf receiver plugin module
Posted by GitBox <gi...@apache.org>.
soander commented on code in PR #9620:
URL: https://github.com/apache/skywalking/pull/9620#discussion_r1000330613
##########
test/e2e-v2/cases/vm/telegraf/telegraf.conf:
##########
@@ -0,0 +1,9559 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
Review Comment:
> `telegraf.conf` can be made shorter. You should delete unused configs instead of just commenting them out.
Ok, get it. I will delete redundant configs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org