You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by wu...@apache.org on 2022/07/12 14:12:20 UTC
[skywalking] branch remove created (now df9ed7a35a)
This is an automated email from the ASF dual-hosted git repository.
wusheng pushed a change to branch remove
in repository https://gitbox.apache.org/repos/asf/skywalking.git
at df9ed7a35a Remove legacy OAL `percentile` functions, `p99`, `p95`, `p90`, `p75`, `p50` func(s)
This branch includes the following new commits:
new df9ed7a35a Remove legacy OAL `percentile` functions, `p99`, `p95`, `p90`, `p75`, `p50` func(s)
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
[skywalking] 01/01: Remove legacy OAL `percentile` functions, `p99`, `p95`, `p90`, `p75`, `p50` func(s)
Posted by wu...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wusheng pushed a commit to branch remove
in repository https://gitbox.apache.org/repos/asf/skywalking.git
commit df9ed7a35a26661d4cf718fcbdc9eb3a8df40249
Author: Wu Sheng <wu...@foxmail.com>
AuthorDate: Tue Jul 12 22:12:04 2022 +0800
Remove legacy OAL `percentile` functions, `p99`, `p95`, `p90`, `p75`, `p50` func(s)
---
docs/en/changes/changes.md | 3 +-
docs/en/concepts-and-designs/oal.md | 2 -
.../server/core/analysis/metrics/P50Metrics.java | 31 ----
.../server/core/analysis/metrics/P75Metrics.java | 31 ----
.../server/core/analysis/metrics/P90Metrics.java | 31 ----
.../server/core/analysis/metrics/P95Metrics.java | 31 ----
.../server/core/analysis/metrics/P99Metrics.java | 31 ----
.../core/analysis/metrics/PercentileMetrics.java | 6 +-
.../server/core/analysis/metrics/PxxMetrics.java | 119 ---------------
.../core/analysis/metrics/PxxMetricsTest.java | 170 ---------------------
10 files changed, 6 insertions(+), 449 deletions(-)
diff --git a/docs/en/changes/changes.md b/docs/en/changes/changes.md
index aab20cec47..451e4c7377 100644
--- a/docs/en/changes/changes.md
+++ b/docs/en/changes/changes.md
@@ -17,7 +17,8 @@
* Fix a NullPointerException in the endpoint analysis, which would cause missing MQ-related `LocalSpan` in the trace.
* Add `forEach`, `processRelation` function to MAL expression.
* Add `expPrefix`, `initExp` in MAL config.
-* Add component ID(7015) for Python Bottle plugin
+* Add component ID(7015) for Python Bottle plugin.
+* Remove legacy OAL `percentile` functions, `p99`, `p95`, `p90`, `p75`, `p50` func(s).
#### UI
diff --git a/docs/en/concepts-and-designs/oal.md b/docs/en/concepts-and-designs/oal.md
index b07c2d2144..981c07325b 100644
--- a/docs/en/concepts-and-designs/oal.md
+++ b/docs/en/concepts-and-designs/oal.md
@@ -84,8 +84,6 @@ Parameter (2) is the status of this request. The status(success/failure) reflect
**percentile** is the first multiple-value metric, which has been introduced since 7.0.0. As a metric with multiple values, it could be queried through the `getMultipleLinearIntValues` GraphQL query.
In this case, see `p99`, `p95`, `p90`, `p75`, and `p50` of all incoming requests. The parameter is precise to a latency at p99, such as in the above case, and 120ms and 124ms are considered to produce the same response time.
-Before 7.0.0, `p99`, `p95`, `p90`, `p75`, `p50` func(s) are used to calculate metrics separately. They are still supported in 7.x, but they are no longer recommended and are not included in the current official OAL script.
-> service_p99 = from(Service.latency).p99(10);
In this case, the p99 value of all incoming requests. The parameter is precise to a latency at p99, such as in the above case, and 120ms and 124ms are considered to produce the same response time.
diff --git a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/metrics/P50Metrics.java b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/metrics/P50Metrics.java
deleted file mode 100644
index bc20a15dd6..0000000000
--- a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/metrics/P50Metrics.java
+++ /dev/null
@@ -1,31 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.core.analysis.metrics;
-
-import org.apache.skywalking.oap.server.core.analysis.metrics.annotation.MetricsFunction;
-
-/**
- * P50
- */
-@MetricsFunction(functionName = "p50")
-public abstract class P50Metrics extends PxxMetrics {
- public P50Metrics() {
- super(50);
- }
-}
diff --git a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/metrics/P75Metrics.java b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/metrics/P75Metrics.java
deleted file mode 100644
index c6b68a28f8..0000000000
--- a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/metrics/P75Metrics.java
+++ /dev/null
@@ -1,31 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.core.analysis.metrics;
-
-import org.apache.skywalking.oap.server.core.analysis.metrics.annotation.MetricsFunction;
-
-/**
- * P75
- */
-@MetricsFunction(functionName = "p75")
-public abstract class P75Metrics extends PxxMetrics {
- public P75Metrics() {
- super(75);
- }
-}
diff --git a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/metrics/P90Metrics.java b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/metrics/P90Metrics.java
deleted file mode 100644
index 058fc24c29..0000000000
--- a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/metrics/P90Metrics.java
+++ /dev/null
@@ -1,31 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.core.analysis.metrics;
-
-import org.apache.skywalking.oap.server.core.analysis.metrics.annotation.MetricsFunction;
-
-/**
- * P90
- */
-@MetricsFunction(functionName = "p90")
-public abstract class P90Metrics extends PxxMetrics {
- public P90Metrics() {
- super(90);
- }
-}
diff --git a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/metrics/P95Metrics.java b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/metrics/P95Metrics.java
deleted file mode 100644
index e4c4ffecdb..0000000000
--- a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/metrics/P95Metrics.java
+++ /dev/null
@@ -1,31 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.core.analysis.metrics;
-
-import org.apache.skywalking.oap.server.core.analysis.metrics.annotation.MetricsFunction;
-
-/**
- * P95
- */
-@MetricsFunction(functionName = "p95")
-public abstract class P95Metrics extends PxxMetrics {
- public P95Metrics() {
- super(95);
- }
-}
diff --git a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/metrics/P99Metrics.java b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/metrics/P99Metrics.java
deleted file mode 100644
index a97d88e1da..0000000000
--- a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/metrics/P99Metrics.java
+++ /dev/null
@@ -1,31 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.core.analysis.metrics;
-
-import org.apache.skywalking.oap.server.core.analysis.metrics.annotation.MetricsFunction;
-
-/**
- * P99
- */
-@MetricsFunction(functionName = "p99")
-public abstract class P99Metrics extends PxxMetrics {
- public P99Metrics() {
- super(99);
- }
-}
diff --git a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/metrics/PercentileMetrics.java b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/metrics/PercentileMetrics.java
index 0e1b2a6691..1f3ca142bc 100644
--- a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/metrics/PercentileMetrics.java
+++ b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/metrics/PercentileMetrics.java
@@ -30,8 +30,10 @@ import org.apache.skywalking.oap.server.core.analysis.metrics.annotation.SourceF
import org.apache.skywalking.oap.server.core.storage.annotation.Column;
/**
- * Percentile is a better implementation than {@link PxxMetrics}. It is introduced since 7.0.0, it could calculate the
- * multiple P50/75/90/95/99 values once for all.
+ * Percentile is a better implementation than deprecated PxxMetrics in older releases.
+ * This could calculate the multiple P50/75/90/95/99 values once for all.
+ *
+ * @since 7.0.0
*/
@MetricsFunction(functionName = "percentile")
public abstract class PercentileMetrics extends Metrics implements MultiIntValuesHolder {
diff --git a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/metrics/PxxMetrics.java b/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/metrics/PxxMetrics.java
deleted file mode 100644
index 65d773e22e..0000000000
--- a/oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/metrics/PxxMetrics.java
+++ /dev/null
@@ -1,119 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.core.analysis.metrics;
-
-import java.util.Comparator;
-import java.util.List;
-import lombok.Getter;
-import lombok.Setter;
-import org.apache.skywalking.oap.server.core.analysis.metrics.annotation.Arg;
-import org.apache.skywalking.oap.server.core.analysis.metrics.annotation.Entrance;
-import org.apache.skywalking.oap.server.core.analysis.metrics.annotation.SourceFrom;
-import org.apache.skywalking.oap.server.core.query.sql.Function;
-import org.apache.skywalking.oap.server.core.storage.annotation.Column;
-
-/**
- * PxxMetrics is a parent metrics for p99/p95/p90/p75/p50 metrics. P(xx) metrics is also for P(xx) percentile.
- * <p>
- * A percentile (or a centile) is a measure used in statistics indicating the value below which a given percentage of
- * observations in a group of observations fall. For example, the 20th percentile is the value (or score) below which
- * 20% of the observations may be found.
- */
-public abstract class PxxMetrics extends Metrics implements IntValueHolder {
-
- protected static final String DETAIL_GROUP = "detail_group";
- protected static final String VALUE = "value";
- protected static final String PRECISION = "precision";
-
- @Getter
- @Setter
- @Column(columnName = VALUE, dataType = Column.ValueDataType.HISTOGRAM, function = Function.Avg)
- private int value;
- @Getter
- @Setter
- @Column(columnName = PRECISION, storageOnly = true)
- private int precision;
- @Getter
- @Setter
- @Column(columnName = DETAIL_GROUP, storageOnly = true)
- private DataTable detailGroup;
-
- private final int percentileRank;
- private boolean isCalculated;
-
- public PxxMetrics(int percentileRank) {
- this.percentileRank = percentileRank;
- detailGroup = new DataTable(30);
- }
-
- @Entrance
- public final void combine(@SourceFrom int value, @Arg int precision) {
- this.isCalculated = false;
- this.precision = precision;
-
- String index = String.valueOf(value / precision);
- Long element = detailGroup.get(index);
- if (element == null) {
- element = 1L;
- } else {
- element++;
- }
- detailGroup.put(index, element);
- }
-
- @Override
- public boolean combine(Metrics metrics) {
- this.isCalculated = false;
-
- PxxMetrics pxxMetrics = (PxxMetrics) metrics;
- this.detailGroup.append(pxxMetrics.detailGroup);
- return true;
- }
-
- @Override
- public final void calculate() {
-
- if (!isCalculated) {
- long total = detailGroup.sumOfValues();
- int roof = Math.round(total * percentileRank * 1.0f / 100);
-
- long count = 0;
- final List<String> sortedKeys = detailGroup.sortedKeys(Comparator.comparingInt(Integer::parseInt));
-
- for (String index : sortedKeys) {
- final Long value = detailGroup.get(index);
- count += value;
- if (count >= roof) {
- this.value = Integer.parseInt(index) * precision;
- return;
- }
- }
- }
- }
-
- @Override
- public boolean haveDefault() {
- return true;
- }
-
- @Override
- public boolean isDefaultValue() {
- return value == 0;
- }
-}
diff --git a/oap-server/server-core/src/test/java/org/apache/skywalking/oap/server/core/analysis/metrics/PxxMetricsTest.java b/oap-server/server-core/src/test/java/org/apache/skywalking/oap/server/core/analysis/metrics/PxxMetricsTest.java
deleted file mode 100644
index fc1b8e977a..0000000000
--- a/oap-server/server-core/src/test/java/org/apache/skywalking/oap/server/core/analysis/metrics/PxxMetricsTest.java
+++ /dev/null
@@ -1,170 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- *
- */
-
-package org.apache.skywalking.oap.server.core.analysis.metrics;
-
-import org.apache.skywalking.oap.server.core.remote.grpc.proto.RemoteData;
-import org.junit.Assert;
-import org.junit.Test;
-
-public class PxxMetricsTest {
- private int precision = 10; //ms
-
- @Test
- public void p99Test() {
- PxxMetricsMocker metricsMocker = new PxxMetricsMocker(99);
-
- metricsMocker.combine(110, precision);
- metricsMocker.combine(100, precision);
- metricsMocker.combine(100, precision);
- metricsMocker.combine(100, precision);
- metricsMocker.combine(50, precision);
- metricsMocker.combine(50, precision);
- metricsMocker.combine(50, precision);
- metricsMocker.combine(61, precision);
- metricsMocker.combine(100, precision);
- metricsMocker.combine(100, precision);
- metricsMocker.combine(100, precision);
-
- metricsMocker.calculate();
-
- Assert.assertEquals(110, metricsMocker.getValue());
- }
-
- @Test
- public void p75Test() {
- PxxMetricsMocker metricsMocker = new PxxMetricsMocker(75);
-
- metricsMocker.combine(110, precision);
- metricsMocker.combine(100, precision);
- metricsMocker.combine(100, precision);
- metricsMocker.combine(100, precision);
- metricsMocker.combine(50, precision);
- metricsMocker.combine(50, precision);
- metricsMocker.combine(50, precision);
- metricsMocker.combine(61, precision);
- metricsMocker.combine(61, precision);
- metricsMocker.combine(71, precision);
- metricsMocker.combine(100, precision);
-
- metricsMocker.calculate();
-
- // precision = 10, 71 ~= 70
- Assert.assertEquals(100, metricsMocker.getValue());
- }
-
- @Test
- public void p50Test() {
- PxxMetricsMocker metricsMocker = new PxxMetricsMocker(50);
-
- metricsMocker.combine(110, precision);
- metricsMocker.combine(100, precision);
- metricsMocker.combine(100, precision);
- metricsMocker.combine(100, precision);
- metricsMocker.combine(50, precision);
- metricsMocker.combine(50, precision);
- metricsMocker.combine(50, precision);
- metricsMocker.combine(61, precision);
- metricsMocker.combine(61, precision);
- metricsMocker.combine(71, precision);
- metricsMocker.combine(100, precision);
-
- metricsMocker.calculate();
-
- // precision = 10, 71 ~= 70
- Assert.assertEquals(70, metricsMocker.getValue());
- }
-
- public class PxxMetricsMocker extends PxxMetrics {
-
- public PxxMetricsMocker(int percentileRank) {
- super(percentileRank);
- }
-
- @Override
- protected String id0() {
- return null;
- }
-
- @Override
- public Metrics toHour() {
- return null;
- }
-
- @Override
- public Metrics toDay() {
- return null;
- }
-
- @Override
- public void deserialize(RemoteData remoteData) {
-
- }
-
- @Override
- public RemoteData.Builder serialize() {
- return null;
- }
-
- @Override
- public int remoteHashCode() {
- return 0;
- }
- }
-
- @Test
- public void testAccurate() {
- DataTable map = new DataTable();
- map.toObject("0,109|128,3|130,1|131,1|132,2|5,16|6,23|10,1|12,1|13,25|14,10|15,2|17,1|146,2|18,1|19,16|20,9|21,4|22,1|23,2|152,1|25,4|26,4|27,3|28,1|31,1|32,2|34,1|44,1|318,1|319,7|320,2|321,1|323,1|324,1|325,2|326,1|327,3|328,1|330,2|205,27|206,14|208,1|337,1|219,15|220,2|221,2|222,1|224,1|352,1|225,1|226,3|227,1|229,1|232,2|105,16|233,1|106,13|108,1|113,20|114,4|115,3|116,2|118,6|119,12|120,4|121,4|122,6|250,1|124,4|125,1|126,4|127,2");
-
- PxxMetricsMocker metrics50Mocker = new PxxMetricsMocker(50);
- metrics50Mocker.setDetailGroup(map);
- metrics50Mocker.setPrecision(10);
- metrics50Mocker.calculate();
- int p50 = metrics50Mocker.getValue();
-
- PxxMetricsMocker metrics75Mocker = new PxxMetricsMocker(75);
- metrics75Mocker.setDetailGroup(map);
- metrics75Mocker.setPrecision(10);
- metrics75Mocker.calculate();
- int p75 = metrics75Mocker.getValue();
-
- PxxMetricsMocker metrics90Mocker = new PxxMetricsMocker(90);
- metrics90Mocker.setDetailGroup(map);
- metrics90Mocker.setPrecision(10);
- metrics90Mocker.calculate();
- int p90 = metrics90Mocker.getValue();
-
- PxxMetricsMocker metrics95Mocker = new PxxMetricsMocker(95);
- metrics95Mocker.setDetailGroup(map);
- metrics95Mocker.setPrecision(10);
- metrics95Mocker.calculate();
- int p95 = metrics95Mocker.getValue();
-
- PxxMetricsMocker metrics99Mocker = new PxxMetricsMocker(99);
- metrics99Mocker.setDetailGroup(map);
- metrics99Mocker.setPrecision(10);
- metrics99Mocker.calculate();
- int p99 = metrics99Mocker.getValue();
-
- Assert.assertTrue(p50 < p75);
- Assert.assertTrue(p75 < p90);
- Assert.assertTrue(p90 < p95);
- Assert.assertTrue(p95 < p99);
- }
-}