You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by GitBox <gi...@apache.org> on 2022/04/29 09:27:40 UTC
[GitHub] [iotdb] SteveYurongSu commented on a diff in pull request #5682: [IoTDB-2991] Equal size bucket sampling UDFs: EQUAL_SIZE_BUCKET_OUTLIER_SAMPLE
SteveYurongSu commented on code in PR #5682:
URL: https://github.com/apache/iotdb/pull/5682#discussion_r861613385
##########
docs/UserGuide/Query-Data/Select-Expression.md:
##########
@@ -668,6 +668,80 @@ Result:
Total line number = 12
It costs 0.065s
```
+#### Equal Size Bucket Outlier Sample
+This function samples the input sequence with equal number of bucket outliers, that is, according to the downsampling ratio given by the user and the number of samples in the bucket, the input sequence is divided into several buckets according to a fixed number of points. Sampling by the given outlier sampling method within each bucket.
+
+| Function Name | Allowed Input Series Data Types | Required Attributes | Output Series Data Type | Series Data Type Description |
+|----------|--------------------------------|---------------------------------------|------------|--------------------------------------------------|
+| EQUAL_SIZE_BUCKET_OUTLIER_SAMPLE | INT32 / INT64 / FLOAT / DOUBLE | The value range of `proportion` is `(0, 1]`, the default is `0.1`</br> The value of `type` is `avg` or `stendis` or `cos` or `prenextdis`, the default is `avg` </br>The value of `number` should be greater than 0, the default is `3`| INT32 / INT64 / FLOAT / DOUBLE | Returns outlier samples in equal buckets that match the sampling ratio and the number of samples in the bucket |
+
+Parameter Description
+- `proportion`: sampling ratio
+- `number`: the number of samples in each bucket, default `3`
+- `type`: outlier sampling method, the value is
+ - `avg`: Take the average of the data points in the bucket, and find the `top number` farthest from the average according to the sampling ratio
+ - `stendis`: Take the vertical distance between each data point in the bucket and the first and last data points of the bucket to form a straight line, and according to the sampling ratio, find the `top number` with the largest distance
+ - `cos`: Set a data point in the bucket as b, the data point on the left of b as a, and the data point on the right of b as c, then take the cosine value of the angle between the ab and bc vectors. The larger the angle, the more likely it is an outlier. Find the `top number` with the smallest cos value
+ - `prenextdis`: Set a data point in the bucket as b, the data point to the left of b as a, and the data point to the right of b as c, then take the cosine value of the angle between the ab and bc vectors. The larger the angle, the more likely it is an outlier. Find the `top number` with the smallest cos value
Review Comment:
Is it the same with `cos`?
##########
docs/zh/UserGuide/Query-Data/Select-Expression.md:
##########
@@ -672,6 +672,82 @@ Total line number = 12
It costs 0.065s
```
+#### 等数量分桶离群值采样
+本函数对输入序列进行等数量分桶离群值采样,即根据用户给定的降采样比例和桶内采样个数将输入序列按固定点数等分为若干桶,在每个桶内通过给定的离群值采样方法进行采样。
+
+| 函数名 | 可接收的输入序列类型 | 必要的属性参数 | 输出序列类型 | 功能类型 |
+|----------|--------------------------------|---------------------------------------|------------|--------------------------------------------------|
+| EQUAL_SIZE_BUCKET_OUTLIER_SAMPLE | INT32 / INT64 / FLOAT / DOUBLE | `proportion`取值范围为`(0, 1]`,默认为`0.1`</br>`type`取值为`avg`或`stendis`或`cos`或`prenextdis`,默认为`avg`</br>`number`取值应大于0,默认`3`| INT32 / INT64 / FLOAT / DOUBLE | 返回符合采样比例和桶内采样个数的等分桶离群值采样 |
+
+参数说明
+- `proportion`: 采样比例
+ - `number`: 每个桶内的采样个数,默认`3`
+- `type`: 离群值采样方法,取值为
+ - `avg`: 取桶内数据点的平均值,并根据采样比例,找到距离均值最远的top number个
+ - `stendis`: 取桶内每一个数据点距离桶的首末数据点连成直线的垂直距离,并根据采样比例,找到距离最大的top number个
+ - `cos`: 设桶内一个数据点为b,b左边的数据点为a,b右边的数据点为c,则取ab与bc向量的夹角的余弦值,值越小,说明形成的角度越大,越可能是异常值。找到cos值最小的top number个
+ - `prenextdis`: 设桶内一个数据点为b,b左边的数据点为a,b右边的数据点为c,则取ab与bc向量的夹角的余弦值,值越小,说明形成的角度越大,越可能是异常值。找到cos值最小的top number个
Review Comment:
Is it the same with `cos`?
##########
server/src/main/java/org/apache/iotdb/db/query/udf/builtin/UDTFEqualSizeBucketOutlierSample.java:
##########
@@ -0,0 +1,1067 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.query.udf.builtin;
+
+import org.apache.iotdb.db.exception.metadata.MetadataException;
+import org.apache.iotdb.db.query.udf.api.access.Row;
+import org.apache.iotdb.db.query.udf.api.access.RowWindow;
+import org.apache.iotdb.db.query.udf.api.collector.PointCollector;
+import org.apache.iotdb.db.query.udf.api.customizer.config.UDTFConfigurations;
+import org.apache.iotdb.db.query.udf.api.customizer.parameter.UDFParameterValidator;
+import org.apache.iotdb.db.query.udf.api.customizer.parameter.UDFParameters;
+import org.apache.iotdb.db.query.udf.api.customizer.strategy.SlidingSizeWindowAccessStrategy;
+import org.apache.iotdb.db.query.udf.api.exception.UDFException;
+import org.apache.iotdb.db.query.udf.api.exception.UDFInputSeriesDataTypeNotValidException;
+import org.apache.iotdb.db.query.udf.api.exception.UDFParameterNotValidException;
+import org.apache.iotdb.tsfile.file.metadata.enums.TSDataType;
+import org.apache.iotdb.tsfile.utils.Pair;
+
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Comparator;
+import java.util.PriorityQueue;
+
+public class UDTFEqualSizeBucketOutlierSample extends UDTFEqualSizeBucketSample {
+
+ private String type;
+ private int number;
+ private OutlierSampler outlierSampler;
+
+ private interface OutlierSampler {
+
+ void outlierSampleInt(RowWindow rowWindow, PointCollector collector) throws IOException;
+
+ void outlierSampleLong(RowWindow rowWindow, PointCollector collector) throws IOException;
+
+ void outlierSampleFloat(RowWindow rowWindow, PointCollector collector) throws IOException;
+
+ void outlierSampleDouble(RowWindow rowWindow, PointCollector collector) throws IOException;
+ }
+
+ private class AvgOutlierSampler implements OutlierSampler {
+
+ @Override
+ public void outlierSampleInt(RowWindow rowWindow, PointCollector collector) throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ }
+
+ double avg = 0;
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+ for (int i = 0; i < windowSize; i++) {
+ avg += rowWindow.getRow(i).getInt(0);
+ }
+ avg /= windowSize;
+ for (int i = 0; i < windowSize; i++) {
+ double value = rowWindow.getRow(i).getInt(0) - avg;
Review Comment:
```suggestion
double value = Math.abs(rowWindow.getRow(i).getInt(0) - avg);
```
##########
server/src/main/java/org/apache/iotdb/db/query/udf/builtin/UDTFEqualSizeBucketOutlierSample.java:
##########
@@ -0,0 +1,1067 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.query.udf.builtin;
+
+import org.apache.iotdb.db.exception.metadata.MetadataException;
+import org.apache.iotdb.db.query.udf.api.access.Row;
+import org.apache.iotdb.db.query.udf.api.access.RowWindow;
+import org.apache.iotdb.db.query.udf.api.collector.PointCollector;
+import org.apache.iotdb.db.query.udf.api.customizer.config.UDTFConfigurations;
+import org.apache.iotdb.db.query.udf.api.customizer.parameter.UDFParameterValidator;
+import org.apache.iotdb.db.query.udf.api.customizer.parameter.UDFParameters;
+import org.apache.iotdb.db.query.udf.api.customizer.strategy.SlidingSizeWindowAccessStrategy;
+import org.apache.iotdb.db.query.udf.api.exception.UDFException;
+import org.apache.iotdb.db.query.udf.api.exception.UDFInputSeriesDataTypeNotValidException;
+import org.apache.iotdb.db.query.udf.api.exception.UDFParameterNotValidException;
+import org.apache.iotdb.tsfile.file.metadata.enums.TSDataType;
+import org.apache.iotdb.tsfile.utils.Pair;
+
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Comparator;
+import java.util.PriorityQueue;
+
+public class UDTFEqualSizeBucketOutlierSample extends UDTFEqualSizeBucketSample {
+
+ private String type;
+ private int number;
+ private OutlierSampler outlierSampler;
+
+ private interface OutlierSampler {
+
+ void outlierSampleInt(RowWindow rowWindow, PointCollector collector) throws IOException;
+
+ void outlierSampleLong(RowWindow rowWindow, PointCollector collector) throws IOException;
+
+ void outlierSampleFloat(RowWindow rowWindow, PointCollector collector) throws IOException;
+
+ void outlierSampleDouble(RowWindow rowWindow, PointCollector collector) throws IOException;
+ }
+
+ private class AvgOutlierSampler implements OutlierSampler {
+
+ @Override
+ public void outlierSampleInt(RowWindow rowWindow, PointCollector collector) throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ }
+
+ double avg = 0;
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+ for (int i = 0; i < windowSize; i++) {
+ avg += rowWindow.getRow(i).getInt(0);
+ }
+ avg /= windowSize;
+ for (int i = 0; i < windowSize; i++) {
+ double value = rowWindow.getRow(i).getInt(0) - avg;
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putInt(rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getInt(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleLong(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putLong(row.getTime(), row.getLong(0));
+ }
+ return;
+ }
+
+ double avg = 0;
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+ for (int i = 0; i < windowSize; i++) {
+ avg += rowWindow.getRow(i).getLong(0);
+ }
+ avg /= windowSize;
+ for (int i = 0; i < windowSize; i++) {
+ double value = rowWindow.getRow(i).getLong(0) - avg;
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putLong(rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getLong(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleFloat(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putFloat(row.getTime(), row.getFloat(0));
+ }
+ return;
+ }
+
+ double avg = 0;
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+ for (int i = 0; i < windowSize; i++) {
+ avg += rowWindow.getRow(i).getFloat(0);
+ }
+ avg /= windowSize;
+ for (int i = 0; i < windowSize; i++) {
+ double value = rowWindow.getRow(i).getFloat(0) - avg;
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putFloat(
+ rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getFloat(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleDouble(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putDouble(row.getTime(), row.getDouble(0));
+ }
+ return;
+ }
+
+ double avg = 0;
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+ for (int i = 0; i < windowSize; i++) {
+ avg += rowWindow.getRow(i).getDouble(0);
+ }
+ avg /= windowSize;
+ for (int i = 0; i < windowSize; i++) {
+ double value = rowWindow.getRow(i).getDouble(0) - avg;
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putDouble(
+ rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getDouble(0));
+ }
+ }
+ }
+
+ private class StendisOutlierSampler implements OutlierSampler {
+
+ @Override
+ public void outlierSampleInt(RowWindow rowWindow, PointCollector collector) throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ }
+
+ long row0x = rowWindow.getRow(0).getTime(),
+ row1x = rowWindow.getRow(windowSize - 1).getTime();
+ int row0y = rowWindow.getRow(0).getInt(0), row1y = rowWindow.getRow(windowSize - 1).getInt(0);
+
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+
+ double A = row0y - row1y;
+ double B = row1x - row0x;
+ double C = row0x * row1y - row1x * row0y;
+ double denominator = Math.sqrt(A * A + B * B);
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ double value = Math.abs(A * row.getTime() + B * row.getInt(0) + C) / denominator;
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putInt(rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getInt(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleLong(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putLong(row.getTime(), row.getLong(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putLong(row.getTime(), row.getLong(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putLong(row.getTime(), row.getLong(0));
+ }
+ return;
+ }
+
+ long row0x = rowWindow.getRow(0).getTime(),
+ row1x = rowWindow.getRow(windowSize - 1).getTime();
+ long row0y = rowWindow.getRow(0).getLong(0),
+ row1y = rowWindow.getRow(windowSize - 1).getLong(0);
+
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+
+ double A = row0y - row1y;
+ double B = row1x - row0x;
+ double C = row0x * row1y - row1x * row0y;
+ double denominator = Math.sqrt(A * A + B * B);
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ double value = Math.abs(A * row.getTime() + B * row.getLong(0) + C) / denominator;
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putLong(rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getLong(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleFloat(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putFloat(row.getTime(), row.getFloat(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putFloat(row.getTime(), row.getFloat(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putFloat(row.getTime(), row.getFloat(0));
+ }
+ return;
+ }
+
+ long row0x = rowWindow.getRow(0).getTime(),
+ row1x = rowWindow.getRow(windowSize - 1).getTime();
+ float row0y = rowWindow.getRow(0).getFloat(0),
+ row1y = rowWindow.getRow(windowSize - 1).getFloat(0);
+
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+
+ double A = row0y - row1y;
+ double B = row1x - row0x;
+ double C = row0x * row1y - row1x * row0y;
+ double denominator = Math.sqrt(A * A + B * B);
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ double value = Math.abs(A * row.getTime() + B * row.getFloat(0) + C) / denominator;
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putFloat(
+ rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getFloat(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleDouble(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putDouble(row.getTime(), row.getDouble(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putDouble(row.getTime(), row.getDouble(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putDouble(row.getTime(), row.getDouble(0));
+ }
+ return;
+ }
+
+ long row0x = rowWindow.getRow(0).getTime(),
+ row1x = rowWindow.getRow(windowSize - 1).getTime();
+ double row0y = rowWindow.getRow(0).getDouble(0),
+ row1y = rowWindow.getRow(windowSize - 1).getDouble(0);
+
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+
+ double A = row0y - row1y;
+ double B = row1x - row0x;
+ double C = row0x * row1y - row1x * row0y;
+ double denominator = Math.sqrt(A * A + B * B);
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ double value = Math.abs(A * row.getTime() + B * row.getDouble(0) + C) / denominator;
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putDouble(
+ rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getDouble(0));
+ }
+ }
+ }
+
+ private class CosOutlierSampler implements OutlierSampler {
+
+ @Override
+ public void outlierSampleInt(RowWindow rowWindow, PointCollector collector) throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ }
+
+ // o -> -o.right, max heap
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> -o.right));
+
+ long lastTime, currentTime, nextTime, x1, x2;
+ int lastValue, currentValue, nextValue, y1, y2;
+ double value;
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ lastTime = rowWindow.getRow(i - 1).getTime();
+ currentTime = rowWindow.getRow(i).getTime();
+ nextTime = rowWindow.getRow(i + 1).getTime();
+
+ lastValue = rowWindow.getRow(i - 1).getInt(0);
+ currentValue = rowWindow.getRow(i).getInt(0);
+ nextValue = rowWindow.getRow(i + 1).getInt(0);
+
+ x1 = currentTime - lastTime;
+ x2 = nextTime - currentTime;
+ y1 = currentValue - lastValue;
+ y2 = nextValue - currentValue;
+
+ value =
+ (x1 * x2 + y1 * y2)
+ / (Math.sqrt((double) x1 * x1 + y1 * y1) * Math.sqrt((double) x2 * x2 + y2 * y2));
+
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value < pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putInt(rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getInt(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleLong(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putLong(row.getTime(), row.getLong(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putLong(row.getTime(), row.getLong(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putLong(row.getTime(), row.getLong(0));
+ }
+ return;
+ }
+
+ // o -> -o.right, max heap
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> -o.right));
+
+ long lastTime, currentTime, nextTime, x1, x2;
+ long lastValue, currentValue, nextValue, y1, y2;
+ double value;
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ lastTime = rowWindow.getRow(i - 1).getTime();
+ currentTime = rowWindow.getRow(i).getTime();
+ nextTime = rowWindow.getRow(i + 1).getTime();
+
+ lastValue = rowWindow.getRow(i - 1).getLong(0);
+ currentValue = rowWindow.getRow(i).getLong(0);
+ nextValue = rowWindow.getRow(i + 1).getLong(0);
+
+ x1 = currentTime - lastTime;
+ x2 = nextTime - currentTime;
+ y1 = currentValue - lastValue;
+ y2 = nextValue - currentValue;
+
+ value =
+ (x1 * x2 + y1 * y2)
+ / (Math.sqrt((double) x1 * x1 + y1 * y1) * Math.sqrt((double) x2 * x2 + y2 * y2));
+
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value < pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putLong(rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getLong(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleFloat(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putFloat(row.getTime(), row.getFloat(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putFloat(row.getTime(), row.getFloat(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putFloat(row.getTime(), row.getFloat(0));
+ }
+ return;
+ }
+
+ // o -> -o.right, max heap
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> -o.right));
+
+ long lastTime, currentTime, nextTime, x1, x2;
+ float lastValue, currentValue, nextValue, y1, y2;
+ double value;
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ lastTime = rowWindow.getRow(i - 1).getTime();
+ currentTime = rowWindow.getRow(i).getTime();
+ nextTime = rowWindow.getRow(i + 1).getTime();
+
+ lastValue = rowWindow.getRow(i - 1).getFloat(0);
+ currentValue = rowWindow.getRow(i).getFloat(0);
+ nextValue = rowWindow.getRow(i + 1).getFloat(0);
+
+ x1 = currentTime - lastTime;
+ x2 = nextTime - currentTime;
+ y1 = currentValue - lastValue;
+ y2 = nextValue - currentValue;
+
+ value = (x1 * x2 + y1 * y2) / (Math.sqrt(x1 * x1 + y1 * y1) * Math.sqrt(x2 * x2 + y2 * y2));
+
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value < pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putFloat(
+ rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getFloat(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleDouble(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putDouble(row.getTime(), row.getDouble(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putDouble(row.getTime(), row.getDouble(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putDouble(row.getTime(), row.getDouble(0));
+ }
+ return;
+ }
+
+ // o -> -o.right, max heap
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> -o.right));
+
+ long lastTime, currentTime, nextTime, x1, x2;
+ double lastValue, currentValue, nextValue, y1, y2;
+ double value;
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ lastTime = rowWindow.getRow(i - 1).getTime();
+ currentTime = rowWindow.getRow(i).getTime();
+ nextTime = rowWindow.getRow(i + 1).getTime();
+
+ lastValue = rowWindow.getRow(i - 1).getDouble(0);
+ currentValue = rowWindow.getRow(i).getDouble(0);
+ nextValue = rowWindow.getRow(i + 1).getDouble(0);
+
+ x1 = currentTime - lastTime;
+ x2 = nextTime - currentTime;
+ y1 = currentValue - lastValue;
+ y2 = nextValue - currentValue;
+
+ value = (x1 * x2 + y1 * y2) / (Math.sqrt(x1 * x1 + y1 * y1) * Math.sqrt(x2 * x2 + y2 * y2));
+
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value < pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putDouble(
+ rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getDouble(0));
+ }
+ }
+ }
+
+ private class PrenextdisOutlierSampler implements OutlierSampler {
+
+ @Override
+ public void outlierSampleInt(RowWindow rowWindow, PointCollector collector) throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ }
+
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+
+ long lastTime, currentTime, nextTime, x1, x2;
+ int lastValue, currentValue, nextValue, y1, y2;
+ double value;
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ lastTime = rowWindow.getRow(i - 1).getTime();
+ currentTime = rowWindow.getRow(i).getTime();
+ nextTime = rowWindow.getRow(i + 1).getTime();
+
+ lastValue = rowWindow.getRow(i - 1).getInt(0);
+ currentValue = rowWindow.getRow(i).getInt(0);
+ nextValue = rowWindow.getRow(i + 1).getInt(0);
+
+ x1 = currentTime - lastTime;
+ x2 = nextTime - currentTime;
+ y1 = currentValue - lastValue;
+ y2 = nextValue - currentValue;
+
+ value =
+ (x1 > 0 ? x1 : -x1) + (y1 > 0 ? y1 : -y1) + (x2 > 0 ? x2 : -x2) + (y2 > 0 ? y2 : -y2);
+
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putInt(rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getInt(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleLong(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putLong(row.getTime(), row.getLong(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putLong(row.getTime(), row.getLong(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putLong(row.getTime(), row.getLong(0));
+ }
+ return;
+ }
+
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+
+ long lastTime, currentTime, nextTime, x1, x2;
+ long lastValue, currentValue, nextValue, y1, y2;
+ double value;
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ lastTime = rowWindow.getRow(i - 1).getTime();
+ currentTime = rowWindow.getRow(i).getTime();
+ nextTime = rowWindow.getRow(i + 1).getTime();
+
+ lastValue = rowWindow.getRow(i - 1).getLong(0);
+ currentValue = rowWindow.getRow(i).getLong(0);
+ nextValue = rowWindow.getRow(i + 1).getLong(0);
+
+ x1 = currentTime - lastTime;
+ x2 = nextTime - currentTime;
+ y1 = currentValue - lastValue;
+ y2 = nextValue - currentValue;
+
+ value =
+ (x1 > 0 ? x1 : -x1) + (y1 > 0 ? y1 : -y1) + (x2 > 0 ? x2 : -x2) + (y2 > 0 ? y2 : -y2);
+
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putLong(rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getLong(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleFloat(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putFloat(row.getTime(), row.getFloat(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putFloat(row.getTime(), row.getFloat(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putFloat(row.getTime(), row.getFloat(0));
+ }
+ return;
+ }
+
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+
+ long lastTime, currentTime, nextTime, x1, x2;
+ float lastValue, currentValue, nextValue, y1, y2;
+ double value;
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ lastTime = rowWindow.getRow(i - 1).getTime();
+ currentTime = rowWindow.getRow(i).getTime();
+ nextTime = rowWindow.getRow(i + 1).getTime();
+
+ lastValue = rowWindow.getRow(i - 1).getFloat(0);
+ currentValue = rowWindow.getRow(i).getFloat(0);
+ nextValue = rowWindow.getRow(i + 1).getFloat(0);
+
+ x1 = currentTime - lastTime;
+ x2 = nextTime - currentTime;
+ y1 = currentValue - lastValue;
+ y2 = nextValue - currentValue;
+
+ value =
+ (x1 > 0 ? x1 : -x1) + (y1 > 0 ? y1 : -y1) + (x2 > 0 ? x2 : -x2) + (y2 > 0 ? y2 : -y2);
+
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putFloat(
+ rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getFloat(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleDouble(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putDouble(row.getTime(), row.getDouble(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putDouble(row.getTime(), row.getDouble(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putDouble(row.getTime(), row.getDouble(0));
+ }
+ return;
+ }
+
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+
+ long lastTime, currentTime, nextTime, x1, x2;
+ double lastValue, currentValue, nextValue, y1, y2;
+ double value;
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ lastTime = rowWindow.getRow(i - 1).getTime();
+ currentTime = rowWindow.getRow(i).getTime();
+ nextTime = rowWindow.getRow(i + 1).getTime();
+
+ lastValue = rowWindow.getRow(i - 1).getDouble(0);
+ currentValue = rowWindow.getRow(i).getDouble(0);
+ nextValue = rowWindow.getRow(i + 1).getDouble(0);
+
+ x1 = currentTime - lastTime;
+ x2 = nextTime - currentTime;
+ y1 = currentValue - lastValue;
+ y2 = nextValue - currentValue;
+
+ value =
+ (x1 > 0 ? x1 : -x1) + (y1 > 0 ? y1 : -y1) + (x2 > 0 ? x2 : -x2) + (y2 > 0 ? y2 : -y2);
+
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putDouble(
+ rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getDouble(0));
+ }
Review Comment:
我们是不是可以进一步把这段逻辑抽出来?
##########
server/src/main/java/org/apache/iotdb/db/query/udf/builtin/UDTFEqualSizeBucketOutlierSample.java:
##########
@@ -0,0 +1,1067 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.query.udf.builtin;
+
+import org.apache.iotdb.db.exception.metadata.MetadataException;
+import org.apache.iotdb.db.query.udf.api.access.Row;
+import org.apache.iotdb.db.query.udf.api.access.RowWindow;
+import org.apache.iotdb.db.query.udf.api.collector.PointCollector;
+import org.apache.iotdb.db.query.udf.api.customizer.config.UDTFConfigurations;
+import org.apache.iotdb.db.query.udf.api.customizer.parameter.UDFParameterValidator;
+import org.apache.iotdb.db.query.udf.api.customizer.parameter.UDFParameters;
+import org.apache.iotdb.db.query.udf.api.customizer.strategy.SlidingSizeWindowAccessStrategy;
+import org.apache.iotdb.db.query.udf.api.exception.UDFException;
+import org.apache.iotdb.db.query.udf.api.exception.UDFInputSeriesDataTypeNotValidException;
+import org.apache.iotdb.db.query.udf.api.exception.UDFParameterNotValidException;
+import org.apache.iotdb.tsfile.file.metadata.enums.TSDataType;
+import org.apache.iotdb.tsfile.utils.Pair;
+
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Comparator;
+import java.util.PriorityQueue;
+
+public class UDTFEqualSizeBucketOutlierSample extends UDTFEqualSizeBucketSample {
+
+ private String type;
+ private int number;
+ private OutlierSampler outlierSampler;
+
+ private interface OutlierSampler {
+
+ void outlierSampleInt(RowWindow rowWindow, PointCollector collector) throws IOException;
+
+ void outlierSampleLong(RowWindow rowWindow, PointCollector collector) throws IOException;
+
+ void outlierSampleFloat(RowWindow rowWindow, PointCollector collector) throws IOException;
+
+ void outlierSampleDouble(RowWindow rowWindow, PointCollector collector) throws IOException;
+ }
+
+ private class AvgOutlierSampler implements OutlierSampler {
+
+ @Override
+ public void outlierSampleInt(RowWindow rowWindow, PointCollector collector) throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ }
+
+ double avg = 0;
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+ for (int i = 0; i < windowSize; i++) {
+ avg += rowWindow.getRow(i).getInt(0);
+ }
+ avg /= windowSize;
+ for (int i = 0; i < windowSize; i++) {
+ double value = rowWindow.getRow(i).getInt(0) - avg;
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putInt(rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getInt(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleLong(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putLong(row.getTime(), row.getLong(0));
+ }
+ return;
+ }
+
+ double avg = 0;
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+ for (int i = 0; i < windowSize; i++) {
+ avg += rowWindow.getRow(i).getLong(0);
+ }
+ avg /= windowSize;
+ for (int i = 0; i < windowSize; i++) {
+ double value = rowWindow.getRow(i).getLong(0) - avg;
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putLong(rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getLong(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleFloat(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putFloat(row.getTime(), row.getFloat(0));
+ }
+ return;
+ }
+
+ double avg = 0;
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+ for (int i = 0; i < windowSize; i++) {
+ avg += rowWindow.getRow(i).getFloat(0);
+ }
+ avg /= windowSize;
+ for (int i = 0; i < windowSize; i++) {
+ double value = rowWindow.getRow(i).getFloat(0) - avg;
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putFloat(
+ rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getFloat(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleDouble(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putDouble(row.getTime(), row.getDouble(0));
+ }
+ return;
+ }
+
+ double avg = 0;
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+ for (int i = 0; i < windowSize; i++) {
+ avg += rowWindow.getRow(i).getDouble(0);
+ }
+ avg /= windowSize;
+ for (int i = 0; i < windowSize; i++) {
+ double value = rowWindow.getRow(i).getDouble(0) - avg;
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putDouble(
+ rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getDouble(0));
+ }
+ }
+ }
+
+ private class StendisOutlierSampler implements OutlierSampler {
+
+ @Override
+ public void outlierSampleInt(RowWindow rowWindow, PointCollector collector) throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ }
Review Comment:
这一段是不是可以抽出来
##########
server/src/main/java/org/apache/iotdb/db/query/udf/builtin/UDTFEqualSizeBucketOutlierSample.java:
##########
@@ -0,0 +1,1067 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.query.udf.builtin;
+
+import org.apache.iotdb.db.exception.metadata.MetadataException;
+import org.apache.iotdb.db.query.udf.api.access.Row;
+import org.apache.iotdb.db.query.udf.api.access.RowWindow;
+import org.apache.iotdb.db.query.udf.api.collector.PointCollector;
+import org.apache.iotdb.db.query.udf.api.customizer.config.UDTFConfigurations;
+import org.apache.iotdb.db.query.udf.api.customizer.parameter.UDFParameterValidator;
+import org.apache.iotdb.db.query.udf.api.customizer.parameter.UDFParameters;
+import org.apache.iotdb.db.query.udf.api.customizer.strategy.SlidingSizeWindowAccessStrategy;
+import org.apache.iotdb.db.query.udf.api.exception.UDFException;
+import org.apache.iotdb.db.query.udf.api.exception.UDFInputSeriesDataTypeNotValidException;
+import org.apache.iotdb.db.query.udf.api.exception.UDFParameterNotValidException;
+import org.apache.iotdb.tsfile.file.metadata.enums.TSDataType;
+import org.apache.iotdb.tsfile.utils.Pair;
+
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Comparator;
+import java.util.PriorityQueue;
+
+public class UDTFEqualSizeBucketOutlierSample extends UDTFEqualSizeBucketSample {
+
+ private String type;
+ private int number;
+ private OutlierSampler outlierSampler;
+
+ private interface OutlierSampler {
+
+ void outlierSampleInt(RowWindow rowWindow, PointCollector collector) throws IOException;
+
+ void outlierSampleLong(RowWindow rowWindow, PointCollector collector) throws IOException;
+
+ void outlierSampleFloat(RowWindow rowWindow, PointCollector collector) throws IOException;
+
+ void outlierSampleDouble(RowWindow rowWindow, PointCollector collector) throws IOException;
+ }
+
+ private class AvgOutlierSampler implements OutlierSampler {
+
+ @Override
+ public void outlierSampleInt(RowWindow rowWindow, PointCollector collector) throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ }
+
+ double avg = 0;
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+ for (int i = 0; i < windowSize; i++) {
+ avg += rowWindow.getRow(i).getInt(0);
+ }
+ avg /= windowSize;
+ for (int i = 0; i < windowSize; i++) {
+ double value = rowWindow.getRow(i).getInt(0) - avg;
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putInt(rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getInt(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleLong(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putLong(row.getTime(), row.getLong(0));
+ }
+ return;
+ }
+
+ double avg = 0;
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+ for (int i = 0; i < windowSize; i++) {
+ avg += rowWindow.getRow(i).getLong(0);
+ }
+ avg /= windowSize;
+ for (int i = 0; i < windowSize; i++) {
+ double value = rowWindow.getRow(i).getLong(0) - avg;
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putLong(rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getLong(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleFloat(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putFloat(row.getTime(), row.getFloat(0));
+ }
+ return;
+ }
+
+ double avg = 0;
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+ for (int i = 0; i < windowSize; i++) {
+ avg += rowWindow.getRow(i).getFloat(0);
+ }
+ avg /= windowSize;
+ for (int i = 0; i < windowSize; i++) {
+ double value = rowWindow.getRow(i).getFloat(0) - avg;
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putFloat(
+ rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getFloat(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleDouble(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putDouble(row.getTime(), row.getDouble(0));
+ }
+ return;
+ }
+
+ double avg = 0;
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+ for (int i = 0; i < windowSize; i++) {
+ avg += rowWindow.getRow(i).getDouble(0);
+ }
+ avg /= windowSize;
+ for (int i = 0; i < windowSize; i++) {
+ double value = rowWindow.getRow(i).getDouble(0) - avg;
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putDouble(
+ rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getDouble(0));
+ }
+ }
+ }
+
+ private class StendisOutlierSampler implements OutlierSampler {
+
+ @Override
+ public void outlierSampleInt(RowWindow rowWindow, PointCollector collector) throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ }
+
+ long row0x = rowWindow.getRow(0).getTime(),
+ row1x = rowWindow.getRow(windowSize - 1).getTime();
+ int row0y = rowWindow.getRow(0).getInt(0), row1y = rowWindow.getRow(windowSize - 1).getInt(0);
+
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+
+ double A = row0y - row1y;
+ double B = row1x - row0x;
+ double C = row0x * row1y - row1x * row0y;
+ double denominator = Math.sqrt(A * A + B * B);
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ double value = Math.abs(A * row.getTime() + B * row.getInt(0) + C) / denominator;
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putInt(rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getInt(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleLong(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putLong(row.getTime(), row.getLong(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putLong(row.getTime(), row.getLong(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putLong(row.getTime(), row.getLong(0));
+ }
+ return;
+ }
+
+ long row0x = rowWindow.getRow(0).getTime(),
+ row1x = rowWindow.getRow(windowSize - 1).getTime();
+ long row0y = rowWindow.getRow(0).getLong(0),
+ row1y = rowWindow.getRow(windowSize - 1).getLong(0);
+
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+
+ double A = row0y - row1y;
+ double B = row1x - row0x;
+ double C = row0x * row1y - row1x * row0y;
+ double denominator = Math.sqrt(A * A + B * B);
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ double value = Math.abs(A * row.getTime() + B * row.getLong(0) + C) / denominator;
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putLong(rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getLong(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleFloat(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putFloat(row.getTime(), row.getFloat(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putFloat(row.getTime(), row.getFloat(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putFloat(row.getTime(), row.getFloat(0));
+ }
+ return;
+ }
+
+ long row0x = rowWindow.getRow(0).getTime(),
+ row1x = rowWindow.getRow(windowSize - 1).getTime();
+ float row0y = rowWindow.getRow(0).getFloat(0),
+ row1y = rowWindow.getRow(windowSize - 1).getFloat(0);
+
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+
+ double A = row0y - row1y;
+ double B = row1x - row0x;
+ double C = row0x * row1y - row1x * row0y;
+ double denominator = Math.sqrt(A * A + B * B);
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ double value = Math.abs(A * row.getTime() + B * row.getFloat(0) + C) / denominator;
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putFloat(
+ rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getFloat(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleDouble(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putDouble(row.getTime(), row.getDouble(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putDouble(row.getTime(), row.getDouble(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putDouble(row.getTime(), row.getDouble(0));
+ }
+ return;
+ }
+
+ long row0x = rowWindow.getRow(0).getTime(),
+ row1x = rowWindow.getRow(windowSize - 1).getTime();
+ double row0y = rowWindow.getRow(0).getDouble(0),
+ row1y = rowWindow.getRow(windowSize - 1).getDouble(0);
+
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+
+ double A = row0y - row1y;
+ double B = row1x - row0x;
+ double C = row0x * row1y - row1x * row0y;
+ double denominator = Math.sqrt(A * A + B * B);
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ double value = Math.abs(A * row.getTime() + B * row.getDouble(0) + C) / denominator;
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putDouble(
+ rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getDouble(0));
+ }
+ }
+ }
+
+ private class CosOutlierSampler implements OutlierSampler {
+
+ @Override
+ public void outlierSampleInt(RowWindow rowWindow, PointCollector collector) throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ }
+
+ // o -> -o.right, max heap
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> -o.right));
+
+ long lastTime, currentTime, nextTime, x1, x2;
+ int lastValue, currentValue, nextValue, y1, y2;
+ double value;
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ lastTime = rowWindow.getRow(i - 1).getTime();
+ currentTime = rowWindow.getRow(i).getTime();
+ nextTime = rowWindow.getRow(i + 1).getTime();
+
+ lastValue = rowWindow.getRow(i - 1).getInt(0);
+ currentValue = rowWindow.getRow(i).getInt(0);
+ nextValue = rowWindow.getRow(i + 1).getInt(0);
+
+ x1 = currentTime - lastTime;
+ x2 = nextTime - currentTime;
+ y1 = currentValue - lastValue;
+ y2 = nextValue - currentValue;
+
+ value =
+ (x1 * x2 + y1 * y2)
+ / (Math.sqrt((double) x1 * x1 + y1 * y1) * Math.sqrt((double) x2 * x2 + y2 * y2));
+
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value < pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putInt(rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getInt(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleLong(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putLong(row.getTime(), row.getLong(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putLong(row.getTime(), row.getLong(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putLong(row.getTime(), row.getLong(0));
+ }
+ return;
+ }
+
+ // o -> -o.right, max heap
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> -o.right));
+
+ long lastTime, currentTime, nextTime, x1, x2;
+ long lastValue, currentValue, nextValue, y1, y2;
+ double value;
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ lastTime = rowWindow.getRow(i - 1).getTime();
+ currentTime = rowWindow.getRow(i).getTime();
+ nextTime = rowWindow.getRow(i + 1).getTime();
+
+ lastValue = rowWindow.getRow(i - 1).getLong(0);
+ currentValue = rowWindow.getRow(i).getLong(0);
+ nextValue = rowWindow.getRow(i + 1).getLong(0);
+
+ x1 = currentTime - lastTime;
+ x2 = nextTime - currentTime;
+ y1 = currentValue - lastValue;
+ y2 = nextValue - currentValue;
+
+ value =
+ (x1 * x2 + y1 * y2)
+ / (Math.sqrt((double) x1 * x1 + y1 * y1) * Math.sqrt((double) x2 * x2 + y2 * y2));
+
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value < pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putLong(rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getLong(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleFloat(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putFloat(row.getTime(), row.getFloat(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putFloat(row.getTime(), row.getFloat(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putFloat(row.getTime(), row.getFloat(0));
+ }
+ return;
+ }
+
+ // o -> -o.right, max heap
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> -o.right));
+
+ long lastTime, currentTime, nextTime, x1, x2;
+ float lastValue, currentValue, nextValue, y1, y2;
+ double value;
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ lastTime = rowWindow.getRow(i - 1).getTime();
+ currentTime = rowWindow.getRow(i).getTime();
+ nextTime = rowWindow.getRow(i + 1).getTime();
+
+ lastValue = rowWindow.getRow(i - 1).getFloat(0);
+ currentValue = rowWindow.getRow(i).getFloat(0);
+ nextValue = rowWindow.getRow(i + 1).getFloat(0);
+
+ x1 = currentTime - lastTime;
+ x2 = nextTime - currentTime;
+ y1 = currentValue - lastValue;
+ y2 = nextValue - currentValue;
+
+ value = (x1 * x2 + y1 * y2) / (Math.sqrt(x1 * x1 + y1 * y1) * Math.sqrt(x2 * x2 + y2 * y2));
+
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value < pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putFloat(
+ rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getFloat(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleDouble(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putDouble(row.getTime(), row.getDouble(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putDouble(row.getTime(), row.getDouble(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putDouble(row.getTime(), row.getDouble(0));
+ }
+ return;
+ }
+
+ // o -> -o.right, max heap
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> -o.right));
+
+ long lastTime, currentTime, nextTime, x1, x2;
+ double lastValue, currentValue, nextValue, y1, y2;
+ double value;
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ lastTime = rowWindow.getRow(i - 1).getTime();
+ currentTime = rowWindow.getRow(i).getTime();
+ nextTime = rowWindow.getRow(i + 1).getTime();
+
+ lastValue = rowWindow.getRow(i - 1).getDouble(0);
+ currentValue = rowWindow.getRow(i).getDouble(0);
+ nextValue = rowWindow.getRow(i + 1).getDouble(0);
+
+ x1 = currentTime - lastTime;
+ x2 = nextTime - currentTime;
+ y1 = currentValue - lastValue;
+ y2 = nextValue - currentValue;
+
+ value = (x1 * x2 + y1 * y2) / (Math.sqrt(x1 * x1 + y1 * y1) * Math.sqrt(x2 * x2 + y2 * y2));
+
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value < pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putDouble(
+ rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getDouble(0));
+ }
+ }
+ }
+
+ private class PrenextdisOutlierSampler implements OutlierSampler {
+
+ @Override
+ public void outlierSampleInt(RowWindow rowWindow, PointCollector collector) throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ }
+
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+
+ long lastTime, currentTime, nextTime, x1, x2;
+ int lastValue, currentValue, nextValue, y1, y2;
+ double value;
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ lastTime = rowWindow.getRow(i - 1).getTime();
+ currentTime = rowWindow.getRow(i).getTime();
+ nextTime = rowWindow.getRow(i + 1).getTime();
+
+ lastValue = rowWindow.getRow(i - 1).getInt(0);
+ currentValue = rowWindow.getRow(i).getInt(0);
+ nextValue = rowWindow.getRow(i + 1).getInt(0);
+
+ x1 = currentTime - lastTime;
+ x2 = nextTime - currentTime;
+ y1 = currentValue - lastValue;
+ y2 = nextValue - currentValue;
+
+ value =
+ (x1 > 0 ? x1 : -x1) + (y1 > 0 ? y1 : -y1) + (x2 > 0 ? x2 : -x2) + (y2 > 0 ? y2 : -y2);
+
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putInt(rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getInt(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleLong(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putLong(row.getTime(), row.getLong(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putLong(row.getTime(), row.getLong(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putLong(row.getTime(), row.getLong(0));
+ }
+ return;
+ }
+
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+
+ long lastTime, currentTime, nextTime, x1, x2;
+ long lastValue, currentValue, nextValue, y1, y2;
+ double value;
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ lastTime = rowWindow.getRow(i - 1).getTime();
+ currentTime = rowWindow.getRow(i).getTime();
+ nextTime = rowWindow.getRow(i + 1).getTime();
+
+ lastValue = rowWindow.getRow(i - 1).getLong(0);
+ currentValue = rowWindow.getRow(i).getLong(0);
+ nextValue = rowWindow.getRow(i + 1).getLong(0);
+
+ x1 = currentTime - lastTime;
+ x2 = nextTime - currentTime;
+ y1 = currentValue - lastValue;
+ y2 = nextValue - currentValue;
+
+ value =
+ (x1 > 0 ? x1 : -x1) + (y1 > 0 ? y1 : -y1) + (x2 > 0 ? x2 : -x2) + (y2 > 0 ? y2 : -y2);
+
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putLong(rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getLong(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleFloat(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putFloat(row.getTime(), row.getFloat(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putFloat(row.getTime(), row.getFloat(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putFloat(row.getTime(), row.getFloat(0));
+ }
+ return;
+ }
+
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+
+ long lastTime, currentTime, nextTime, x1, x2;
+ float lastValue, currentValue, nextValue, y1, y2;
+ double value;
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ lastTime = rowWindow.getRow(i - 1).getTime();
+ currentTime = rowWindow.getRow(i).getTime();
+ nextTime = rowWindow.getRow(i + 1).getTime();
+
+ lastValue = rowWindow.getRow(i - 1).getFloat(0);
+ currentValue = rowWindow.getRow(i).getFloat(0);
+ nextValue = rowWindow.getRow(i + 1).getFloat(0);
+
+ x1 = currentTime - lastTime;
+ x2 = nextTime - currentTime;
+ y1 = currentValue - lastValue;
+ y2 = nextValue - currentValue;
+
+ value =
+ (x1 > 0 ? x1 : -x1) + (y1 > 0 ? y1 : -y1) + (x2 > 0 ? x2 : -x2) + (y2 > 0 ? y2 : -y2);
+
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putFloat(
+ rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getFloat(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleDouble(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putDouble(row.getTime(), row.getDouble(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putDouble(row.getTime(), row.getDouble(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putDouble(row.getTime(), row.getDouble(0));
+ }
+ return;
+ }
+
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+
+ long lastTime, currentTime, nextTime, x1, x2;
+ double lastValue, currentValue, nextValue, y1, y2;
+ double value;
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ lastTime = rowWindow.getRow(i - 1).getTime();
+ currentTime = rowWindow.getRow(i).getTime();
+ nextTime = rowWindow.getRow(i + 1).getTime();
+
+ lastValue = rowWindow.getRow(i - 1).getDouble(0);
+ currentValue = rowWindow.getRow(i).getDouble(0);
+ nextValue = rowWindow.getRow(i + 1).getDouble(0);
+
+ x1 = currentTime - lastTime;
+ x2 = nextTime - currentTime;
+ y1 = currentValue - lastValue;
+ y2 = nextValue - currentValue;
+
+ value =
+ (x1 > 0 ? x1 : -x1) + (y1 > 0 ? y1 : -y1) + (x2 > 0 ? x2 : -x2) + (y2 > 0 ? y2 : -y2);
Review Comment:
Use Math.abs instead
##########
server/src/main/java/org/apache/iotdb/db/query/udf/builtin/UDTFEqualSizeBucketOutlierSample.java:
##########
@@ -0,0 +1,1067 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.query.udf.builtin;
+
+import org.apache.iotdb.db.exception.metadata.MetadataException;
+import org.apache.iotdb.db.query.udf.api.access.Row;
+import org.apache.iotdb.db.query.udf.api.access.RowWindow;
+import org.apache.iotdb.db.query.udf.api.collector.PointCollector;
+import org.apache.iotdb.db.query.udf.api.customizer.config.UDTFConfigurations;
+import org.apache.iotdb.db.query.udf.api.customizer.parameter.UDFParameterValidator;
+import org.apache.iotdb.db.query.udf.api.customizer.parameter.UDFParameters;
+import org.apache.iotdb.db.query.udf.api.customizer.strategy.SlidingSizeWindowAccessStrategy;
+import org.apache.iotdb.db.query.udf.api.exception.UDFException;
+import org.apache.iotdb.db.query.udf.api.exception.UDFInputSeriesDataTypeNotValidException;
+import org.apache.iotdb.db.query.udf.api.exception.UDFParameterNotValidException;
+import org.apache.iotdb.tsfile.file.metadata.enums.TSDataType;
+import org.apache.iotdb.tsfile.utils.Pair;
+
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Comparator;
+import java.util.PriorityQueue;
+
+public class UDTFEqualSizeBucketOutlierSample extends UDTFEqualSizeBucketSample {
+
+ private String type;
+ private int number;
+ private OutlierSampler outlierSampler;
+
+ private interface OutlierSampler {
+
+ void outlierSampleInt(RowWindow rowWindow, PointCollector collector) throws IOException;
+
+ void outlierSampleLong(RowWindow rowWindow, PointCollector collector) throws IOException;
+
+ void outlierSampleFloat(RowWindow rowWindow, PointCollector collector) throws IOException;
+
+ void outlierSampleDouble(RowWindow rowWindow, PointCollector collector) throws IOException;
+ }
+
+ private class AvgOutlierSampler implements OutlierSampler {
+
+ @Override
+ public void outlierSampleInt(RowWindow rowWindow, PointCollector collector) throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ }
+
+ double avg = 0;
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+ for (int i = 0; i < windowSize; i++) {
+ avg += rowWindow.getRow(i).getInt(0);
+ }
+ avg /= windowSize;
+ for (int i = 0; i < windowSize; i++) {
+ double value = rowWindow.getRow(i).getInt(0) - avg;
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putInt(rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getInt(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleLong(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putLong(row.getTime(), row.getLong(0));
+ }
+ return;
+ }
+
+ double avg = 0;
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+ for (int i = 0; i < windowSize; i++) {
+ avg += rowWindow.getRow(i).getLong(0);
+ }
+ avg /= windowSize;
+ for (int i = 0; i < windowSize; i++) {
+ double value = rowWindow.getRow(i).getLong(0) - avg;
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putLong(rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getLong(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleFloat(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putFloat(row.getTime(), row.getFloat(0));
+ }
+ return;
+ }
+
+ double avg = 0;
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+ for (int i = 0; i < windowSize; i++) {
+ avg += rowWindow.getRow(i).getFloat(0);
+ }
+ avg /= windowSize;
+ for (int i = 0; i < windowSize; i++) {
+ double value = rowWindow.getRow(i).getFloat(0) - avg;
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putFloat(
+ rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getFloat(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleDouble(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putDouble(row.getTime(), row.getDouble(0));
+ }
+ return;
+ }
+
+ double avg = 0;
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+ for (int i = 0; i < windowSize; i++) {
+ avg += rowWindow.getRow(i).getDouble(0);
+ }
+ avg /= windowSize;
+ for (int i = 0; i < windowSize; i++) {
+ double value = rowWindow.getRow(i).getDouble(0) - avg;
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putDouble(
+ rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getDouble(0));
+ }
+ }
+ }
+
+ private class StendisOutlierSampler implements OutlierSampler {
+
+ @Override
+ public void outlierSampleInt(RowWindow rowWindow, PointCollector collector) throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ }
+
+ long row0x = rowWindow.getRow(0).getTime(),
+ row1x = rowWindow.getRow(windowSize - 1).getTime();
+ int row0y = rowWindow.getRow(0).getInt(0), row1y = rowWindow.getRow(windowSize - 1).getInt(0);
+
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+
+ double A = row0y - row1y;
+ double B = row1x - row0x;
+ double C = row0x * row1y - row1x * row0y;
+ double denominator = Math.sqrt(A * A + B * B);
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ double value = Math.abs(A * row.getTime() + B * row.getInt(0) + C) / denominator;
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putInt(rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getInt(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleLong(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putLong(row.getTime(), row.getLong(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putLong(row.getTime(), row.getLong(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putLong(row.getTime(), row.getLong(0));
+ }
+ return;
+ }
+
+ long row0x = rowWindow.getRow(0).getTime(),
+ row1x = rowWindow.getRow(windowSize - 1).getTime();
+ long row0y = rowWindow.getRow(0).getLong(0),
+ row1y = rowWindow.getRow(windowSize - 1).getLong(0);
+
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+
+ double A = row0y - row1y;
+ double B = row1x - row0x;
+ double C = row0x * row1y - row1x * row0y;
+ double denominator = Math.sqrt(A * A + B * B);
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ double value = Math.abs(A * row.getTime() + B * row.getLong(0) + C) / denominator;
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putLong(rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getLong(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleFloat(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putFloat(row.getTime(), row.getFloat(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putFloat(row.getTime(), row.getFloat(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putFloat(row.getTime(), row.getFloat(0));
+ }
+ return;
+ }
+
+ long row0x = rowWindow.getRow(0).getTime(),
+ row1x = rowWindow.getRow(windowSize - 1).getTime();
+ float row0y = rowWindow.getRow(0).getFloat(0),
+ row1y = rowWindow.getRow(windowSize - 1).getFloat(0);
+
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+
+ double A = row0y - row1y;
+ double B = row1x - row0x;
+ double C = row0x * row1y - row1x * row0y;
+ double denominator = Math.sqrt(A * A + B * B);
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ double value = Math.abs(A * row.getTime() + B * row.getFloat(0) + C) / denominator;
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putFloat(
+ rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getFloat(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleDouble(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putDouble(row.getTime(), row.getDouble(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putDouble(row.getTime(), row.getDouble(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putDouble(row.getTime(), row.getDouble(0));
+ }
+ return;
+ }
+
+ long row0x = rowWindow.getRow(0).getTime(),
+ row1x = rowWindow.getRow(windowSize - 1).getTime();
+ double row0y = rowWindow.getRow(0).getDouble(0),
+ row1y = rowWindow.getRow(windowSize - 1).getDouble(0);
+
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+
+ double A = row0y - row1y;
+ double B = row1x - row0x;
+ double C = row0x * row1y - row1x * row0y;
+ double denominator = Math.sqrt(A * A + B * B);
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ double value = Math.abs(A * row.getTime() + B * row.getDouble(0) + C) / denominator;
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putDouble(
+ rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getDouble(0));
+ }
+ }
+ }
+
+ private class CosOutlierSampler implements OutlierSampler {
+
+ @Override
+ public void outlierSampleInt(RowWindow rowWindow, PointCollector collector) throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ }
+
+ // o -> -o.right, max heap
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> -o.right));
+
+ long lastTime, currentTime, nextTime, x1, x2;
+ int lastValue, currentValue, nextValue, y1, y2;
+ double value;
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ lastTime = rowWindow.getRow(i - 1).getTime();
+ currentTime = rowWindow.getRow(i).getTime();
+ nextTime = rowWindow.getRow(i + 1).getTime();
+
+ lastValue = rowWindow.getRow(i - 1).getInt(0);
+ currentValue = rowWindow.getRow(i).getInt(0);
+ nextValue = rowWindow.getRow(i + 1).getInt(0);
+
+ x1 = currentTime - lastTime;
+ x2 = nextTime - currentTime;
+ y1 = currentValue - lastValue;
+ y2 = nextValue - currentValue;
+
+ value =
+ (x1 * x2 + y1 * y2)
+ / (Math.sqrt((double) x1 * x1 + y1 * y1) * Math.sqrt((double) x2 * x2 + y2 * y2));
+
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value < pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putInt(rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getInt(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleLong(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putLong(row.getTime(), row.getLong(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putLong(row.getTime(), row.getLong(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putLong(row.getTime(), row.getLong(0));
+ }
+ return;
+ }
+
+ // o -> -o.right, max heap
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> -o.right));
+
+ long lastTime, currentTime, nextTime, x1, x2;
+ long lastValue, currentValue, nextValue, y1, y2;
+ double value;
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ lastTime = rowWindow.getRow(i - 1).getTime();
+ currentTime = rowWindow.getRow(i).getTime();
+ nextTime = rowWindow.getRow(i + 1).getTime();
+
+ lastValue = rowWindow.getRow(i - 1).getLong(0);
+ currentValue = rowWindow.getRow(i).getLong(0);
+ nextValue = rowWindow.getRow(i + 1).getLong(0);
+
+ x1 = currentTime - lastTime;
+ x2 = nextTime - currentTime;
+ y1 = currentValue - lastValue;
+ y2 = nextValue - currentValue;
+
+ value =
+ (x1 * x2 + y1 * y2)
+ / (Math.sqrt((double) x1 * x1 + y1 * y1) * Math.sqrt((double) x2 * x2 + y2 * y2));
+
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value < pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putLong(rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getLong(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleFloat(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putFloat(row.getTime(), row.getFloat(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putFloat(row.getTime(), row.getFloat(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putFloat(row.getTime(), row.getFloat(0));
+ }
+ return;
+ }
+
+ // o -> -o.right, max heap
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> -o.right));
+
+ long lastTime, currentTime, nextTime, x1, x2;
+ float lastValue, currentValue, nextValue, y1, y2;
+ double value;
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ lastTime = rowWindow.getRow(i - 1).getTime();
+ currentTime = rowWindow.getRow(i).getTime();
+ nextTime = rowWindow.getRow(i + 1).getTime();
+
+ lastValue = rowWindow.getRow(i - 1).getFloat(0);
+ currentValue = rowWindow.getRow(i).getFloat(0);
+ nextValue = rowWindow.getRow(i + 1).getFloat(0);
+
+ x1 = currentTime - lastTime;
+ x2 = nextTime - currentTime;
+ y1 = currentValue - lastValue;
+ y2 = nextValue - currentValue;
+
+ value = (x1 * x2 + y1 * y2) / (Math.sqrt(x1 * x1 + y1 * y1) * Math.sqrt(x2 * x2 + y2 * y2));
+
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value < pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putFloat(
+ rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getFloat(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleDouble(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putDouble(row.getTime(), row.getDouble(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putDouble(row.getTime(), row.getDouble(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putDouble(row.getTime(), row.getDouble(0));
+ }
+ return;
+ }
+
+ // o -> -o.right, max heap
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> -o.right));
+
+ long lastTime, currentTime, nextTime, x1, x2;
+ double lastValue, currentValue, nextValue, y1, y2;
+ double value;
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ lastTime = rowWindow.getRow(i - 1).getTime();
+ currentTime = rowWindow.getRow(i).getTime();
+ nextTime = rowWindow.getRow(i + 1).getTime();
+
+ lastValue = rowWindow.getRow(i - 1).getDouble(0);
+ currentValue = rowWindow.getRow(i).getDouble(0);
+ nextValue = rowWindow.getRow(i + 1).getDouble(0);
+
+ x1 = currentTime - lastTime;
+ x2 = nextTime - currentTime;
+ y1 = currentValue - lastValue;
+ y2 = nextValue - currentValue;
+
+ value = (x1 * x2 + y1 * y2) / (Math.sqrt(x1 * x1 + y1 * y1) * Math.sqrt(x2 * x2 + y2 * y2));
+
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value < pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putDouble(
+ rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getDouble(0));
+ }
+ }
+ }
+
+ private class PrenextdisOutlierSampler implements OutlierSampler {
+
+ @Override
+ public void outlierSampleInt(RowWindow rowWindow, PointCollector collector) throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ }
+
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+
+ long lastTime, currentTime, nextTime, x1, x2;
+ int lastValue, currentValue, nextValue, y1, y2;
+ double value;
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ lastTime = rowWindow.getRow(i - 1).getTime();
+ currentTime = rowWindow.getRow(i).getTime();
+ nextTime = rowWindow.getRow(i + 1).getTime();
+
+ lastValue = rowWindow.getRow(i - 1).getInt(0);
+ currentValue = rowWindow.getRow(i).getInt(0);
+ nextValue = rowWindow.getRow(i + 1).getInt(0);
+
+ x1 = currentTime - lastTime;
+ x2 = nextTime - currentTime;
+ y1 = currentValue - lastValue;
+ y2 = nextValue - currentValue;
+
+ value =
+ (x1 > 0 ? x1 : -x1) + (y1 > 0 ? y1 : -y1) + (x2 > 0 ? x2 : -x2) + (y2 > 0 ? y2 : -y2);
+
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putInt(rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getInt(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleLong(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putLong(row.getTime(), row.getLong(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putLong(row.getTime(), row.getLong(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putLong(row.getTime(), row.getLong(0));
+ }
+ return;
+ }
+
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+
+ long lastTime, currentTime, nextTime, x1, x2;
+ long lastValue, currentValue, nextValue, y1, y2;
+ double value;
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ lastTime = rowWindow.getRow(i - 1).getTime();
+ currentTime = rowWindow.getRow(i).getTime();
+ nextTime = rowWindow.getRow(i + 1).getTime();
+
+ lastValue = rowWindow.getRow(i - 1).getLong(0);
+ currentValue = rowWindow.getRow(i).getLong(0);
+ nextValue = rowWindow.getRow(i + 1).getLong(0);
+
+ x1 = currentTime - lastTime;
+ x2 = nextTime - currentTime;
+ y1 = currentValue - lastValue;
+ y2 = nextValue - currentValue;
+
+ value =
+ (x1 > 0 ? x1 : -x1) + (y1 > 0 ? y1 : -y1) + (x2 > 0 ? x2 : -x2) + (y2 > 0 ? y2 : -y2);
+
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putLong(rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getLong(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleFloat(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putFloat(row.getTime(), row.getFloat(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putFloat(row.getTime(), row.getFloat(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putFloat(row.getTime(), row.getFloat(0));
+ }
+ return;
+ }
+
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+
+ long lastTime, currentTime, nextTime, x1, x2;
+ float lastValue, currentValue, nextValue, y1, y2;
+ double value;
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ lastTime = rowWindow.getRow(i - 1).getTime();
+ currentTime = rowWindow.getRow(i).getTime();
+ nextTime = rowWindow.getRow(i + 1).getTime();
+
+ lastValue = rowWindow.getRow(i - 1).getFloat(0);
+ currentValue = rowWindow.getRow(i).getFloat(0);
+ nextValue = rowWindow.getRow(i + 1).getFloat(0);
+
+ x1 = currentTime - lastTime;
+ x2 = nextTime - currentTime;
+ y1 = currentValue - lastValue;
+ y2 = nextValue - currentValue;
+
+ value =
+ (x1 > 0 ? x1 : -x1) + (y1 > 0 ? y1 : -y1) + (x2 > 0 ? x2 : -x2) + (y2 > 0 ? y2 : -y2);
+
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putFloat(
+ rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getFloat(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleDouble(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putDouble(row.getTime(), row.getDouble(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putDouble(row.getTime(), row.getDouble(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putDouble(row.getTime(), row.getDouble(0));
+ }
+ return;
+ }
+
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+
+ long lastTime, currentTime, nextTime, x1, x2;
+ double lastValue, currentValue, nextValue, y1, y2;
+ double value;
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ lastTime = rowWindow.getRow(i - 1).getTime();
+ currentTime = rowWindow.getRow(i).getTime();
+ nextTime = rowWindow.getRow(i + 1).getTime();
+
+ lastValue = rowWindow.getRow(i - 1).getDouble(0);
+ currentValue = rowWindow.getRow(i).getDouble(0);
+ nextValue = rowWindow.getRow(i + 1).getDouble(0);
+
+ x1 = currentTime - lastTime;
+ x2 = nextTime - currentTime;
+ y1 = currentValue - lastValue;
+ y2 = nextValue - currentValue;
+
Review Comment:
```suggestion
x1 = Math.abs(currentTime - lastTime);
x2 = Math.abs(nextTime - currentTime);
y1 = Math.abs(currentValue - lastValue);
y2 = Math.abs(nextValue - currentValue);
```
##########
server/src/main/java/org/apache/iotdb/db/query/udf/builtin/UDTFEqualSizeBucketOutlierSample.java:
##########
@@ -0,0 +1,1067 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.query.udf.builtin;
+
+import org.apache.iotdb.db.exception.metadata.MetadataException;
+import org.apache.iotdb.db.query.udf.api.access.Row;
+import org.apache.iotdb.db.query.udf.api.access.RowWindow;
+import org.apache.iotdb.db.query.udf.api.collector.PointCollector;
+import org.apache.iotdb.db.query.udf.api.customizer.config.UDTFConfigurations;
+import org.apache.iotdb.db.query.udf.api.customizer.parameter.UDFParameterValidator;
+import org.apache.iotdb.db.query.udf.api.customizer.parameter.UDFParameters;
+import org.apache.iotdb.db.query.udf.api.customizer.strategy.SlidingSizeWindowAccessStrategy;
+import org.apache.iotdb.db.query.udf.api.exception.UDFException;
+import org.apache.iotdb.db.query.udf.api.exception.UDFInputSeriesDataTypeNotValidException;
+import org.apache.iotdb.db.query.udf.api.exception.UDFParameterNotValidException;
+import org.apache.iotdb.tsfile.file.metadata.enums.TSDataType;
+import org.apache.iotdb.tsfile.utils.Pair;
+
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Comparator;
+import java.util.PriorityQueue;
+
+public class UDTFEqualSizeBucketOutlierSample extends UDTFEqualSizeBucketSample {
+
+ private String type;
+ private int number;
+ private OutlierSampler outlierSampler;
+
+ private interface OutlierSampler {
+
+ void outlierSampleInt(RowWindow rowWindow, PointCollector collector) throws IOException;
+
+ void outlierSampleLong(RowWindow rowWindow, PointCollector collector) throws IOException;
+
+ void outlierSampleFloat(RowWindow rowWindow, PointCollector collector) throws IOException;
+
+ void outlierSampleDouble(RowWindow rowWindow, PointCollector collector) throws IOException;
+ }
+
+ private class AvgOutlierSampler implements OutlierSampler {
+
+ @Override
+ public void outlierSampleInt(RowWindow rowWindow, PointCollector collector) throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ }
+
+ double avg = 0;
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+ for (int i = 0; i < windowSize; i++) {
+ avg += rowWindow.getRow(i).getInt(0);
+ }
+ avg /= windowSize;
+ for (int i = 0; i < windowSize; i++) {
+ double value = rowWindow.getRow(i).getInt(0) - avg;
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putInt(rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getInt(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleLong(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putLong(row.getTime(), row.getLong(0));
+ }
+ return;
+ }
+
+ double avg = 0;
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+ for (int i = 0; i < windowSize; i++) {
+ avg += rowWindow.getRow(i).getLong(0);
+ }
+ avg /= windowSize;
+ for (int i = 0; i < windowSize; i++) {
+ double value = rowWindow.getRow(i).getLong(0) - avg;
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putLong(rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getLong(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleFloat(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putFloat(row.getTime(), row.getFloat(0));
+ }
+ return;
+ }
+
+ double avg = 0;
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+ for (int i = 0; i < windowSize; i++) {
+ avg += rowWindow.getRow(i).getFloat(0);
+ }
+ avg /= windowSize;
+ for (int i = 0; i < windowSize; i++) {
+ double value = rowWindow.getRow(i).getFloat(0) - avg;
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putFloat(
+ rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getFloat(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleDouble(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putDouble(row.getTime(), row.getDouble(0));
+ }
+ return;
+ }
+
+ double avg = 0;
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+ for (int i = 0; i < windowSize; i++) {
+ avg += rowWindow.getRow(i).getDouble(0);
+ }
+ avg /= windowSize;
+ for (int i = 0; i < windowSize; i++) {
+ double value = rowWindow.getRow(i).getDouble(0) - avg;
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putDouble(
+ rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getDouble(0));
+ }
+ }
+ }
+
+ private class StendisOutlierSampler implements OutlierSampler {
+
+ @Override
+ public void outlierSampleInt(RowWindow rowWindow, PointCollector collector) throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ }
+
+ long row0x = rowWindow.getRow(0).getTime(),
+ row1x = rowWindow.getRow(windowSize - 1).getTime();
+ int row0y = rowWindow.getRow(0).getInt(0), row1y = rowWindow.getRow(windowSize - 1).getInt(0);
+
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+
+ double A = row0y - row1y;
+ double B = row1x - row0x;
+ double C = row0x * row1y - row1x * row0y;
+ double denominator = Math.sqrt(A * A + B * B);
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ double value = Math.abs(A * row.getTime() + B * row.getInt(0) + C) / denominator;
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putInt(rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getInt(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleLong(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putLong(row.getTime(), row.getLong(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putLong(row.getTime(), row.getLong(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putLong(row.getTime(), row.getLong(0));
+ }
+ return;
+ }
+
+ long row0x = rowWindow.getRow(0).getTime(),
+ row1x = rowWindow.getRow(windowSize - 1).getTime();
+ long row0y = rowWindow.getRow(0).getLong(0),
+ row1y = rowWindow.getRow(windowSize - 1).getLong(0);
+
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+
+ double A = row0y - row1y;
+ double B = row1x - row0x;
+ double C = row0x * row1y - row1x * row0y;
+ double denominator = Math.sqrt(A * A + B * B);
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ double value = Math.abs(A * row.getTime() + B * row.getLong(0) + C) / denominator;
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putLong(rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getLong(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleFloat(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putFloat(row.getTime(), row.getFloat(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putFloat(row.getTime(), row.getFloat(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putFloat(row.getTime(), row.getFloat(0));
+ }
+ return;
+ }
+
+ long row0x = rowWindow.getRow(0).getTime(),
+ row1x = rowWindow.getRow(windowSize - 1).getTime();
+ float row0y = rowWindow.getRow(0).getFloat(0),
+ row1y = rowWindow.getRow(windowSize - 1).getFloat(0);
+
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+
+ double A = row0y - row1y;
+ double B = row1x - row0x;
+ double C = row0x * row1y - row1x * row0y;
+ double denominator = Math.sqrt(A * A + B * B);
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ double value = Math.abs(A * row.getTime() + B * row.getFloat(0) + C) / denominator;
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putFloat(
+ rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getFloat(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleDouble(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putDouble(row.getTime(), row.getDouble(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putDouble(row.getTime(), row.getDouble(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putDouble(row.getTime(), row.getDouble(0));
+ }
+ return;
+ }
+
+ long row0x = rowWindow.getRow(0).getTime(),
+ row1x = rowWindow.getRow(windowSize - 1).getTime();
+ double row0y = rowWindow.getRow(0).getDouble(0),
+ row1y = rowWindow.getRow(windowSize - 1).getDouble(0);
+
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+
+ double A = row0y - row1y;
+ double B = row1x - row0x;
+ double C = row0x * row1y - row1x * row0y;
+ double denominator = Math.sqrt(A * A + B * B);
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ double value = Math.abs(A * row.getTime() + B * row.getDouble(0) + C) / denominator;
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putDouble(
+ rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getDouble(0));
+ }
+ }
+ }
+
+ private class CosOutlierSampler implements OutlierSampler {
+
+ @Override
+ public void outlierSampleInt(RowWindow rowWindow, PointCollector collector) throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ }
+
+ // o -> -o.right, max heap
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> -o.right));
+
+ long lastTime, currentTime, nextTime, x1, x2;
+ int lastValue, currentValue, nextValue, y1, y2;
+ double value;
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ lastTime = rowWindow.getRow(i - 1).getTime();
+ currentTime = rowWindow.getRow(i).getTime();
+ nextTime = rowWindow.getRow(i + 1).getTime();
+
+ lastValue = rowWindow.getRow(i - 1).getInt(0);
+ currentValue = rowWindow.getRow(i).getInt(0);
+ nextValue = rowWindow.getRow(i + 1).getInt(0);
+
+ x1 = currentTime - lastTime;
+ x2 = nextTime - currentTime;
+ y1 = currentValue - lastValue;
+ y2 = nextValue - currentValue;
+
+ value =
+ (x1 * x2 + y1 * y2)
+ / (Math.sqrt((double) x1 * x1 + y1 * y1) * Math.sqrt((double) x2 * x2 + y2 * y2));
+
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value < pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putInt(rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getInt(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleLong(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putLong(row.getTime(), row.getLong(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putLong(row.getTime(), row.getLong(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putLong(row.getTime(), row.getLong(0));
+ }
+ return;
+ }
+
+ // o -> -o.right, max heap
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> -o.right));
+
+ long lastTime, currentTime, nextTime, x1, x2;
+ long lastValue, currentValue, nextValue, y1, y2;
+ double value;
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ lastTime = rowWindow.getRow(i - 1).getTime();
+ currentTime = rowWindow.getRow(i).getTime();
+ nextTime = rowWindow.getRow(i + 1).getTime();
+
+ lastValue = rowWindow.getRow(i - 1).getLong(0);
+ currentValue = rowWindow.getRow(i).getLong(0);
+ nextValue = rowWindow.getRow(i + 1).getLong(0);
+
+ x1 = currentTime - lastTime;
+ x2 = nextTime - currentTime;
+ y1 = currentValue - lastValue;
+ y2 = nextValue - currentValue;
+
+ value =
+ (x1 * x2 + y1 * y2)
+ / (Math.sqrt((double) x1 * x1 + y1 * y1) * Math.sqrt((double) x2 * x2 + y2 * y2));
+
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value < pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putLong(rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getLong(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleFloat(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putFloat(row.getTime(), row.getFloat(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putFloat(row.getTime(), row.getFloat(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putFloat(row.getTime(), row.getFloat(0));
+ }
+ return;
+ }
+
+ // o -> -o.right, max heap
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> -o.right));
+
+ long lastTime, currentTime, nextTime, x1, x2;
+ float lastValue, currentValue, nextValue, y1, y2;
+ double value;
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ lastTime = rowWindow.getRow(i - 1).getTime();
+ currentTime = rowWindow.getRow(i).getTime();
+ nextTime = rowWindow.getRow(i + 1).getTime();
+
+ lastValue = rowWindow.getRow(i - 1).getFloat(0);
+ currentValue = rowWindow.getRow(i).getFloat(0);
+ nextValue = rowWindow.getRow(i + 1).getFloat(0);
+
+ x1 = currentTime - lastTime;
+ x2 = nextTime - currentTime;
+ y1 = currentValue - lastValue;
+ y2 = nextValue - currentValue;
+
+ value = (x1 * x2 + y1 * y2) / (Math.sqrt(x1 * x1 + y1 * y1) * Math.sqrt(x2 * x2 + y2 * y2));
+
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value < pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putFloat(
+ rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getFloat(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleDouble(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putDouble(row.getTime(), row.getDouble(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putDouble(row.getTime(), row.getDouble(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putDouble(row.getTime(), row.getDouble(0));
+ }
+ return;
+ }
+
+ // o -> -o.right, max heap
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> -o.right));
+
+ long lastTime, currentTime, nextTime, x1, x2;
+ double lastValue, currentValue, nextValue, y1, y2;
+ double value;
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ lastTime = rowWindow.getRow(i - 1).getTime();
+ currentTime = rowWindow.getRow(i).getTime();
+ nextTime = rowWindow.getRow(i + 1).getTime();
+
+ lastValue = rowWindow.getRow(i - 1).getDouble(0);
+ currentValue = rowWindow.getRow(i).getDouble(0);
+ nextValue = rowWindow.getRow(i + 1).getDouble(0);
+
+ x1 = currentTime - lastTime;
+ x2 = nextTime - currentTime;
+ y1 = currentValue - lastValue;
+ y2 = nextValue - currentValue;
+
+ value = (x1 * x2 + y1 * y2) / (Math.sqrt(x1 * x1 + y1 * y1) * Math.sqrt(x2 * x2 + y2 * y2));
+
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value < pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putDouble(
+ rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getDouble(0));
+ }
+ }
+ }
+
+ private class PrenextdisOutlierSampler implements OutlierSampler {
+
+ @Override
+ public void outlierSampleInt(RowWindow rowWindow, PointCollector collector) throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putInt(row.getTime(), row.getInt(0));
+ }
+ return;
+ }
+
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+
+ long lastTime, currentTime, nextTime, x1, x2;
+ int lastValue, currentValue, nextValue, y1, y2;
+ double value;
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ lastTime = rowWindow.getRow(i - 1).getTime();
+ currentTime = rowWindow.getRow(i).getTime();
+ nextTime = rowWindow.getRow(i + 1).getTime();
+
+ lastValue = rowWindow.getRow(i - 1).getInt(0);
+ currentValue = rowWindow.getRow(i).getInt(0);
+ nextValue = rowWindow.getRow(i + 1).getInt(0);
+
+ x1 = currentTime - lastTime;
+ x2 = nextTime - currentTime;
+ y1 = currentValue - lastValue;
+ y2 = nextValue - currentValue;
+
+ value =
+ (x1 > 0 ? x1 : -x1) + (y1 > 0 ? y1 : -y1) + (x2 > 0 ? x2 : -x2) + (y2 > 0 ? y2 : -y2);
+
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putInt(rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getInt(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleLong(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putLong(row.getTime(), row.getLong(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putLong(row.getTime(), row.getLong(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putLong(row.getTime(), row.getLong(0));
+ }
+ return;
+ }
+
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+
+ long lastTime, currentTime, nextTime, x1, x2;
+ long lastValue, currentValue, nextValue, y1, y2;
+ double value;
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ lastTime = rowWindow.getRow(i - 1).getTime();
+ currentTime = rowWindow.getRow(i).getTime();
+ nextTime = rowWindow.getRow(i + 1).getTime();
+
+ lastValue = rowWindow.getRow(i - 1).getLong(0);
+ currentValue = rowWindow.getRow(i).getLong(0);
+ nextValue = rowWindow.getRow(i + 1).getLong(0);
+
+ x1 = currentTime - lastTime;
+ x2 = nextTime - currentTime;
+ y1 = currentValue - lastValue;
+ y2 = nextValue - currentValue;
+
+ value =
+ (x1 > 0 ? x1 : -x1) + (y1 > 0 ? y1 : -y1) + (x2 > 0 ? x2 : -x2) + (y2 > 0 ? y2 : -y2);
+
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putLong(rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getLong(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleFloat(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putFloat(row.getTime(), row.getFloat(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putFloat(row.getTime(), row.getFloat(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putFloat(row.getTime(), row.getFloat(0));
+ }
+ return;
+ }
+
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+
+ long lastTime, currentTime, nextTime, x1, x2;
+ float lastValue, currentValue, nextValue, y1, y2;
+ double value;
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ lastTime = rowWindow.getRow(i - 1).getTime();
+ currentTime = rowWindow.getRow(i).getTime();
+ nextTime = rowWindow.getRow(i + 1).getTime();
+
+ lastValue = rowWindow.getRow(i - 1).getFloat(0);
+ currentValue = rowWindow.getRow(i).getFloat(0);
+ nextValue = rowWindow.getRow(i + 1).getFloat(0);
+
+ x1 = currentTime - lastTime;
+ x2 = nextTime - currentTime;
+ y1 = currentValue - lastValue;
+ y2 = nextValue - currentValue;
+
+ value =
+ (x1 > 0 ? x1 : -x1) + (y1 > 0 ? y1 : -y1) + (x2 > 0 ? x2 : -x2) + (y2 > 0 ? y2 : -y2);
+
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
+ }
+
+ int[] arr = new int[number];
+ for (int i = 0; i < number; i++) {
+ arr[i] = pq.peek().left;
+ pq.poll();
+ }
+ Arrays.sort(arr);
+ for (int i = 0; i < number; i++) {
+ collector.putFloat(
+ rowWindow.getRow(arr[i]).getTime(), rowWindow.getRow(arr[i]).getFloat(0));
+ }
+ }
+
+ @Override
+ public void outlierSampleDouble(RowWindow rowWindow, PointCollector collector)
+ throws IOException {
+ int windowSize = rowWindow.windowSize();
+ if (windowSize <= number) {
+ for (int i = 0; i < windowSize; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putDouble(row.getTime(), row.getDouble(0));
+ }
+ return;
+ } else if (windowSize == number + 1) {
+ for (int i = 0; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putDouble(row.getTime(), row.getDouble(0));
+ }
+ return;
+ } else if (windowSize == number + 2) {
+ for (int i = 1; i < windowSize - 1; i++) {
+ Row row = rowWindow.getRow(i);
+ collector.putDouble(row.getTime(), row.getDouble(0));
+ }
+ return;
+ }
+
+ PriorityQueue<Pair<Integer, Double>> pq =
+ new PriorityQueue<>(number, Comparator.comparing(o -> o.right));
+
+ long lastTime, currentTime, nextTime, x1, x2;
+ double lastValue, currentValue, nextValue, y1, y2;
+ double value;
+
+ for (int i = 1; i < windowSize - 1; i++) {
+ lastTime = rowWindow.getRow(i - 1).getTime();
+ currentTime = rowWindow.getRow(i).getTime();
+ nextTime = rowWindow.getRow(i + 1).getTime();
+
+ lastValue = rowWindow.getRow(i - 1).getDouble(0);
+ currentValue = rowWindow.getRow(i).getDouble(0);
+ nextValue = rowWindow.getRow(i + 1).getDouble(0);
+
+ x1 = currentTime - lastTime;
+ x2 = nextTime - currentTime;
+ y1 = currentValue - lastValue;
+ y2 = nextValue - currentValue;
+
+ value =
+ (x1 > 0 ? x1 : -x1) + (y1 > 0 ? y1 : -y1) + (x2 > 0 ? x2 : -x2) + (y2 > 0 ? y2 : -y2);
+
+ if (pq.size() < number) {
+ pq.add(new Pair<>(i, value));
+ } else if (value > pq.peek().right) {
+ pq.poll();
+ pq.add(new Pair<>(i, value));
+ }
Review Comment:
我们是不是可以进一步把这段逻辑抽出来?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org