You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2021/10/07 21:40:28 UTC
[GitHub] [pinot] yupeng9 opened a new pull request #7542: Add IN function
yupeng9 opened a new pull request #7542:
URL: https://github.com/apache/pinot/pull/7542
Add IN function to allow queries like
```
select group_city, group_city in('Sheffield', 'Waltham', 'Ahmedabad') as isin from meetupRsvp order by group_city limit 10
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] richardstartin commented on a change in pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#discussion_r725168560
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/InTransformFunction.java
##########
@@ -81,77 +95,112 @@ public TransformResultMetadata getResultMetadata() {
switch (storedType) {
case INT:
int[] intValues = _transformFunction.transformToIntValuesSV(projectionBlock);
- int[][] inIntValues = new int[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inIntValues[i] = _valueTransformFunctions[i].transformToIntValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inIntValues.length; j++) {
- _results[i] = inIntValues[j][i] == intValues[i] ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
+ Set<Integer> inIntValues = new HashSet<>();
+ for (String inValue : _stringValueSet) {
+ inIntValues.add(Integer.parseInt(inValue));
+ }
+ for (int i = 0; i < length; i++) {
+ _results[i] = inIntValues.contains(intValues[i]) ? 1 : 0;
+ }
+ } else {
+ int[][] inIntValues = new int[_valueTransformFunctions.length][];
+ for (int i = 0; i < _valueTransformFunctions.length; i++) {
+ inIntValues[i] = _valueTransformFunctions[i].transformToIntValuesSV(projectionBlock);
+ }
+ for (int i = 0; i < length; i++) {
+ for (int j = 0; j < inIntValues.length; j++) {
+ _results[i] = inIntValues[j][i] == intValues[i] ? 1 : _results[i];
+ }
}
}
break;
case LONG:
long[] longValues = _transformFunction.transformToLongValuesSV(projectionBlock);
- long[][] inLongValues = new long[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inLongValues[i] = _valueTransformFunctions[i].transformToLongValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inLongValues.length; j++) {
- _results[i] = inLongValues[j][i] == longValues[i] ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
+ Set<Long> inLongValues = new HashSet<>();
+ for (String inValue : _stringValueSet) {
+ inLongValues.add(Long.parseLong(inValue));
+ }
+ for (int i = 0; i < length; i++) {
+ _results[i] = inLongValues.contains(longValues[i]) ? 1 : 0;
+ }
+ } else {
+ long[][] inLongValues = new long[_valueTransformFunctions.length][];
+ for (int i = 0; i < _valueTransformFunctions.length; i++) {
+ inLongValues[i] = _valueTransformFunctions[i].transformToLongValuesSV(projectionBlock);
+ }
+ for (int i = 0; i < length; i++) {
+ for (int j = 0; j < inLongValues.length; j++) {
+ _results[i] = inLongValues[j][i] == longValues[i] ? 1 : _results[i];
+ }
}
}
break;
case FLOAT:
float[] floatValues = _transformFunction.transformToFloatValuesSV(projectionBlock);
- float[][] inFloatValues = new float[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inFloatValues[i] = _valueTransformFunctions[i].transformToFloatValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inFloatValues.length; j++) {
- _results[i] = Float.compare(inFloatValues[j][i], floatValues[i]) == 0 ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
Review comment:
This test fails
```
@Test
public void testFloatSet() {
Set<Float> set = new HashSet<>();
set.add(1F);
assertTrue(set.contains(0.01F + 0.9F));
}
```
Which is to say that floating point equality is pernicious, making set membership an equally relation.
Not only is it difficult to implement, I wonder if anyone would ever write the query with floats?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#issuecomment-938200237
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#discussion_r725170326
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/InTransformFunction.java
##########
@@ -0,0 +1,161 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import com.google.common.base.Preconditions;
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.common.function.TransformFunctionType;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.core.plan.DocIdSetPlanNode;
+import org.apache.pinot.segment.spi.datasource.DataSource;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.utils.ByteArray;
+
+
+/**
+ * The IN transform function takes more than 1 arguments:
+ * <ul>
+ * <li>Expression: a single-value expression</li>
+ * <li>values: a set of literal strings</li>
+ * </ul>
+ * <p>For each docId, the function returns {@code 1} if the set of values contains the value of the expression,
+ * {code 0} if not.
+ * <p>E.g. {@code SELECT col IN ('a','b','c') FROM myTable)}
+ */
+public class InTransformFunction extends BaseTransformFunction {
+ private TransformFunction _transformFunction;
+ private TransformFunction[] _valueTransformFunctions;
+ private int[] _results;
+
+ @Override
+ public String getName() {
+ return TransformFunctionType.IN.getName();
+ }
+
+ @Override
+ public void init(List<TransformFunction> arguments, Map<String, DataSource> dataSourceMap) {
+ Preconditions.checkArgument(arguments.size() >= 2,
+ "at least 2 arguments are required for IN transform function: expression, values");
+ Preconditions.checkArgument(arguments.get(0).getResultMetadata().isSingleValue(),
+ "First argument for IN transform function must be a single-value expression");
+ _transformFunction = arguments.get(0);
+ _valueTransformFunctions = new TransformFunction[arguments.size() - 1];
Review comment:
We don't support non-literal within the `IN` predicate within the filter as of now, so IMO it is okay to just keep the same behavior as they are essentially the same (WHERE should be able to table boolean as the input)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] richardstartin commented on a change in pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#discussion_r725168560
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/InTransformFunction.java
##########
@@ -81,77 +95,112 @@ public TransformResultMetadata getResultMetadata() {
switch (storedType) {
case INT:
int[] intValues = _transformFunction.transformToIntValuesSV(projectionBlock);
- int[][] inIntValues = new int[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inIntValues[i] = _valueTransformFunctions[i].transformToIntValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inIntValues.length; j++) {
- _results[i] = inIntValues[j][i] == intValues[i] ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
+ Set<Integer> inIntValues = new HashSet<>();
+ for (String inValue : _stringValueSet) {
+ inIntValues.add(Integer.parseInt(inValue));
+ }
+ for (int i = 0; i < length; i++) {
+ _results[i] = inIntValues.contains(intValues[i]) ? 1 : 0;
+ }
+ } else {
+ int[][] inIntValues = new int[_valueTransformFunctions.length][];
+ for (int i = 0; i < _valueTransformFunctions.length; i++) {
+ inIntValues[i] = _valueTransformFunctions[i].transformToIntValuesSV(projectionBlock);
+ }
+ for (int i = 0; i < length; i++) {
+ for (int j = 0; j < inIntValues.length; j++) {
+ _results[i] = inIntValues[j][i] == intValues[i] ? 1 : _results[i];
+ }
}
}
break;
case LONG:
long[] longValues = _transformFunction.transformToLongValuesSV(projectionBlock);
- long[][] inLongValues = new long[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inLongValues[i] = _valueTransformFunctions[i].transformToLongValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inLongValues.length; j++) {
- _results[i] = inLongValues[j][i] == longValues[i] ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
+ Set<Long> inLongValues = new HashSet<>();
+ for (String inValue : _stringValueSet) {
+ inLongValues.add(Long.parseLong(inValue));
+ }
+ for (int i = 0; i < length; i++) {
+ _results[i] = inLongValues.contains(longValues[i]) ? 1 : 0;
+ }
+ } else {
+ long[][] inLongValues = new long[_valueTransformFunctions.length][];
+ for (int i = 0; i < _valueTransformFunctions.length; i++) {
+ inLongValues[i] = _valueTransformFunctions[i].transformToLongValuesSV(projectionBlock);
+ }
+ for (int i = 0; i < length; i++) {
+ for (int j = 0; j < inLongValues.length; j++) {
+ _results[i] = inLongValues[j][i] == longValues[i] ? 1 : _results[i];
+ }
}
}
break;
case FLOAT:
float[] floatValues = _transformFunction.transformToFloatValuesSV(projectionBlock);
- float[][] inFloatValues = new float[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inFloatValues[i] = _valueTransformFunctions[i].transformToFloatValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inFloatValues.length; j++) {
- _results[i] = Float.compare(inFloatValues[j][i], floatValues[i]) == 0 ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
Review comment:
This test fails
```
@Test
public void testFloatSet() {
Set<Float> set = new HashSet<>();
set.add(1F);
assertTrue(set.contains(0.01F + 0.9F));
}
```
Not only is it difficult to implement, I wonder if anyone would ever write the query with floats?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#discussion_r725168802
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/InTransformFunction.java
##########
@@ -0,0 +1,161 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import com.google.common.base.Preconditions;
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.common.function.TransformFunctionType;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.core.plan.DocIdSetPlanNode;
+import org.apache.pinot.segment.spi.datasource.DataSource;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.utils.ByteArray;
+
+
+/**
+ * The IN transform function takes more than 1 arguments:
+ * <ul>
+ * <li>Expression: a single-value expression</li>
+ * <li>values: a set of literal strings</li>
+ * </ul>
+ * <p>For each docId, the function returns {@code 1} if the set of values contains the value of the expression,
+ * {code 0} if not.
+ * <p>E.g. {@code SELECT col IN ('a','b','c') FROM myTable)}
+ */
+public class InTransformFunction extends BaseTransformFunction {
+ private TransformFunction _transformFunction;
+ private TransformFunction[] _valueTransformFunctions;
+ private int[] _results;
+
+ @Override
+ public String getName() {
+ return TransformFunctionType.IN.getName();
+ }
+
+ @Override
+ public void init(List<TransformFunction> arguments, Map<String, DataSource> dataSourceMap) {
+ Preconditions.checkArgument(arguments.size() >= 2,
+ "at least 2 arguments are required for IN transform function: expression, values");
+ Preconditions.checkArgument(arguments.get(0).getResultMetadata().isSingleValue(),
+ "First argument for IN transform function must be a single-value expression");
Review comment:
IMO, this `IN` should have the same behavior as the `IN` within the filter. E.g. I have a MV [1, 2, 3], if any value in the `IN` clause is 1 or 2 or 3, it is a match.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] yupeng9 commented on a change in pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
yupeng9 commented on a change in pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#discussion_r725178903
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/InTransformFunction.java
##########
@@ -81,77 +95,112 @@ public TransformResultMetadata getResultMetadata() {
switch (storedType) {
case INT:
int[] intValues = _transformFunction.transformToIntValuesSV(projectionBlock);
- int[][] inIntValues = new int[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inIntValues[i] = _valueTransformFunctions[i].transformToIntValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inIntValues.length; j++) {
- _results[i] = inIntValues[j][i] == intValues[i] ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
+ Set<Integer> inIntValues = new HashSet<>();
+ for (String inValue : _stringValueSet) {
+ inIntValues.add(Integer.parseInt(inValue));
+ }
+ for (int i = 0; i < length; i++) {
+ _results[i] = inIntValues.contains(intValues[i]) ? 1 : 0;
+ }
+ } else {
+ int[][] inIntValues = new int[_valueTransformFunctions.length][];
+ for (int i = 0; i < _valueTransformFunctions.length; i++) {
+ inIntValues[i] = _valueTransformFunctions[i].transformToIntValuesSV(projectionBlock);
+ }
+ for (int i = 0; i < length; i++) {
+ for (int j = 0; j < inIntValues.length; j++) {
+ _results[i] = inIntValues[j][i] == intValues[i] ? 1 : _results[i];
+ }
}
}
break;
case LONG:
long[] longValues = _transformFunction.transformToLongValuesSV(projectionBlock);
- long[][] inLongValues = new long[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inLongValues[i] = _valueTransformFunctions[i].transformToLongValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inLongValues.length; j++) {
- _results[i] = inLongValues[j][i] == longValues[i] ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
+ Set<Long> inLongValues = new HashSet<>();
+ for (String inValue : _stringValueSet) {
+ inLongValues.add(Long.parseLong(inValue));
+ }
+ for (int i = 0; i < length; i++) {
+ _results[i] = inLongValues.contains(longValues[i]) ? 1 : 0;
+ }
+ } else {
+ long[][] inLongValues = new long[_valueTransformFunctions.length][];
+ for (int i = 0; i < _valueTransformFunctions.length; i++) {
+ inLongValues[i] = _valueTransformFunctions[i].transformToLongValuesSV(projectionBlock);
+ }
+ for (int i = 0; i < length; i++) {
+ for (int j = 0; j < inLongValues.length; j++) {
+ _results[i] = inLongValues[j][i] == longValues[i] ? 1 : _results[i];
+ }
}
}
break;
case FLOAT:
float[] floatValues = _transformFunction.transformToFloatValuesSV(projectionBlock);
- float[][] inFloatValues = new float[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inFloatValues[i] = _valueTransformFunctions[i].transformToFloatValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inFloatValues.length; j++) {
- _results[i] = Float.compare(inFloatValues[j][i], floatValues[i]) == 0 ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
Review comment:
hmm, any suggestion on supporting use case of `x IN (0.5, 0.7, 0.9)`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang merged pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang merged pull request #7542:
URL: https://github.com/apache/pinot/pull/7542
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#discussion_r726442106
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/InTransformFunction.java
##########
@@ -86,125 +139,239 @@ public TransformResultMetadata getResultMetadata() {
@Override
public int[] transformToIntValuesSV(ProjectionBlock projectionBlock) {
- if (_results == null) {
- _results = new int[DocIdSetPlanNode.MAX_DOC_PER_CALL];
+ if (_intValuesSV == null) {
+ _intValuesSV = new int[DocIdSetPlanNode.MAX_DOC_PER_CALL];
+ } else {
+ Arrays.fill(_intValuesSV, 0);
}
int length = projectionBlock.getNumDocs();
- FieldSpec.DataType storedType = _transformFunction.getResultMetadata().getDataType().getStoredType();
- switch (storedType) {
- case INT:
- int[] intValues = _transformFunction.transformToIntValuesSV(projectionBlock);
- if (!_stringValueSet.isEmpty()) {
- Set<Integer> inIntValues = new HashSet<>();
- for (String inValue : _stringValueSet) {
- inIntValues.add(Integer.parseInt(inValue));
- }
- for (int i = 0; i < length; i++) {
- _results[i] = inIntValues.contains(intValues[i]) ? 1 : 0;
- }
- } else {
- int[][] inIntValues = new int[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inIntValues[i] = _valueTransformFunctions[i].transformToIntValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inIntValues.length; j++) {
- _results[i] = inIntValues[j][i] == intValues[i] ? 1 : _results[i];
+ TransformResultMetadata mainFunctionMetadata = _mainFunction.getResultMetadata();
+ DataType storedType = mainFunctionMetadata.getDataType().getStoredType();
+ if (_valueSet != null) {
+ if (_mainFunction.getResultMetadata().isSingleValue()) {
+ switch (storedType) {
+ case INT:
+ IntOpenHashSet inIntValues = (IntOpenHashSet) _valueSet;
+ int[] intValues = _mainFunction.transformToIntValuesSV(projectionBlock);
+ for (int i = 0; i < length; i++) {
+ if (inIntValues.contains(intValues[i])) {
+ _intValuesSV[i] = 1;
+ }
}
- }
+ break;
+ case LONG:
+ LongOpenHashSet inLongValues = (LongOpenHashSet) _valueSet;
+ long[] longValues = _mainFunction.transformToLongValuesSV(projectionBlock);
+ for (int i = 0; i < length; i++) {
+ if (inLongValues.contains(longValues[i])) {
+ _intValuesSV[i] = 1;
+ }
+ }
+ break;
+ case FLOAT:
+ FloatOpenHashSet inFloatValues = (FloatOpenHashSet) _valueSet;
+ float[] floatValues = _mainFunction.transformToFloatValuesSV(projectionBlock);
+ for (int i = 0; i < length; i++) {
+ if (inFloatValues.contains(floatValues[i])) {
+ _intValuesSV[i] = 1;
+ }
+ }
+ break;
+ case DOUBLE:
+ DoubleOpenHashSet inDoubleValues = (DoubleOpenHashSet) _valueSet;
+ double[] doubleValues = _mainFunction.transformToDoubleValuesSV(projectionBlock);
+ for (int i = 0; i < length; i++) {
+ if (inDoubleValues.contains(doubleValues[i])) {
+ _intValuesSV[i] = 1;
+ }
+ }
+ break;
+ case STRING:
+ ObjectOpenHashSet<String> inStringValues = (ObjectOpenHashSet<String>) _valueSet;
+ String[] stringValues = _mainFunction.transformToStringValuesSV(projectionBlock);
+ for (int i = 0; i < length; i++) {
+ if (inStringValues.contains(stringValues[i])) {
+ _intValuesSV[i] = 1;
+ }
+ }
+ break;
+ case BYTES:
+ ObjectOpenHashSet<ByteArray> inBytesValues = (ObjectOpenHashSet<ByteArray>) _valueSet;
+ byte[][] bytesValues = _mainFunction.transformToBytesValuesSV(projectionBlock);
+ for (int i = 0; i < length; i++) {
+ if (inBytesValues.contains(new ByteArray(bytesValues[i]))) {
+ _intValuesSV[i] = 1;
+ }
+ }
+ break;
+ default:
+ throw new IllegalStateException();
}
- break;
- case LONG:
- long[] longValues = _transformFunction.transformToLongValuesSV(projectionBlock);
- if (!_stringValueSet.isEmpty()) {
- Set<Long> inLongValues = new HashSet<>();
- for (String inValue : _stringValueSet) {
- inLongValues.add(Long.parseLong(inValue));
- }
- for (int i = 0; i < length; i++) {
- _results[i] = inLongValues.contains(longValues[i]) ? 1 : 0;
- }
- } else {
- long[][] inLongValues = new long[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inLongValues[i] = _valueTransformFunctions[i].transformToLongValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inLongValues.length; j++) {
- _results[i] = inLongValues[j][i] == longValues[i] ? 1 : _results[i];
+ } else {
+ switch (storedType) {
+ case INT:
+ IntOpenHashSet inIntValues = (IntOpenHashSet) _valueSet;
+ int[][] intValues = _mainFunction.transformToIntValuesMV(projectionBlock);
+ for (int i = 0; i < length; i++) {
+ for (int intValue : intValues[i]) {
+ if (inIntValues.contains(intValue)) {
+ _intValuesSV[i] = 1;
+ break;
+ }
+ }
}
- }
+ break;
+ case LONG:
+ LongOpenHashSet inLongValues = (LongOpenHashSet) _valueSet;
+ long[][] longValues = _mainFunction.transformToLongValuesMV(projectionBlock);
+ for (int i = 0; i < length; i++) {
+ for (long longValue : longValues[i]) {
+ if (inLongValues.contains(longValue)) {
+ _intValuesSV[i] = 1;
+ break;
+ }
+ }
+ }
+ break;
+ case FLOAT:
+ FloatOpenHashSet inFloatValues = (FloatOpenHashSet) _valueSet;
+ float[][] floatValues = _mainFunction.transformToFloatValuesMV(projectionBlock);
+ for (int i = 0; i < length; i++) {
+ for (float floatValue : floatValues[i]) {
+ if (inFloatValues.contains(floatValue)) {
+ _intValuesSV[i] = 1;
+ break;
+ }
+ }
+ }
+ break;
+ case DOUBLE:
+ DoubleOpenHashSet inDoubleValues = (DoubleOpenHashSet) _valueSet;
+ double[][] doubleValues = _mainFunction.transformToDoubleValuesMV(projectionBlock);
+ for (int i = 0; i < length; i++) {
+ for (double doubleValue : doubleValues[i]) {
+ if (inDoubleValues.contains(doubleValue)) {
+ _intValuesSV[i] = 1;
+ break;
+ }
+ }
+ }
+ break;
+ case STRING:
+ ObjectOpenHashSet<String> inStringValues = (ObjectOpenHashSet<String>) _valueSet;
+ String[][] stringValues = _mainFunction.transformToStringValuesMV(projectionBlock);
+ for (int i = 0; i < length; i++) {
+ for (String stringValue : stringValues[i]) {
+ if (inStringValues.contains(stringValue)) {
+ _intValuesSV[i] = 1;
+ break;
+ }
+ }
+ }
+ break;
+ default:
+ throw new IllegalStateException();
}
- break;
- case FLOAT:
- float[] floatValues = _transformFunction.transformToFloatValuesSV(projectionBlock);
- if (!_stringValueSet.isEmpty()) {
- Set<Float> inFloatValues = new HashSet<>();
- for (String inValue : _stringValueSet) {
- inFloatValues.add(Float.parseFloat(inValue));
+ }
+ } else {
+ int numValues = _valueFunctions.length;
+ switch (storedType) {
+ case INT:
+ int[] intValues = _mainFunction.transformToIntValuesSV(projectionBlock);
+ int[][] inIntValues = new int[numValues][];
+ for (int i = 0; i < numValues; i++) {
+ inIntValues[i] = _valueFunctions[i].transformToIntValuesSV(projectionBlock);
}
for (int i = 0; i < length; i++) {
- _results[i] = inFloatValues.contains(floatValues[i]) ? 1 : 0;
+ for (int[] inIntValue : inIntValues) {
+ if (intValues[i] == inIntValue[i]) {
+ _intValuesSV[i] = 1;
+ break;
+ }
+ }
}
- } else {
- float[][] inFloatValues = new float[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inFloatValues[i] = _valueTransformFunctions[i].transformToFloatValuesSV(projectionBlock);
+ break;
+ case LONG:
+ long[] longValues = _mainFunction.transformToLongValuesSV(projectionBlock);
+ long[][] inLongValues = new long[numValues][];
+ for (int i = 0; i < numValues; i++) {
+ inLongValues[i] = _valueFunctions[i].transformToLongValuesSV(projectionBlock);
}
for (int i = 0; i < length; i++) {
- for (int j = 0; j < inFloatValues.length; j++) {
- _results[i] = Float.compare(inFloatValues[j][i], floatValues[i]) == 0 ? 1 : _results[i];
+ for (long[] inLongValue : inLongValues) {
+ if (longValues[i] == inLongValue[i]) {
+ _intValuesSV[i] = 1;
+ break;
+ }
}
}
- }
- break;
- case DOUBLE:
- double[] doubleValues = _transformFunction.transformToDoubleValuesSV(projectionBlock);
- if (!_stringValueSet.isEmpty()) {
- Set<Double> inDoubleValues = new HashSet<>();
- for (String inValue : _stringValueSet) {
- inDoubleValues.add(Double.parseDouble(inValue));
+ break;
+ case FLOAT:
+ float[] floatValues = _mainFunction.transformToFloatValuesSV(projectionBlock);
+ float[][] inFloatValues = new float[numValues][];
+ for (int i = 0; i < numValues; i++) {
+ inFloatValues[i] = _valueFunctions[i].transformToFloatValuesSV(projectionBlock);
}
for (int i = 0; i < length; i++) {
- _results[i] = inDoubleValues.contains(doubleValues[i]) ? 1 : 0;
+ for (float[] inFloatValue : inFloatValues) {
+ if (floatValues[i] == inFloatValue[i]) {
+ _intValuesSV[i] = 1;
+ break;
+ }
+ }
}
- } else {
- double[][] inDoubleValues = new double[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inDoubleValues[i] = _valueTransformFunctions[i].transformToDoubleValuesSV(projectionBlock);
+ break;
+ case DOUBLE:
+ double[] doubleValues = _mainFunction.transformToDoubleValuesSV(projectionBlock);
+ double[][] inDoubleValues = new double[numValues][];
+ for (int i = 0; i < numValues; i++) {
+ inDoubleValues[i] = _valueFunctions[i].transformToDoubleValuesSV(projectionBlock);
}
for (int i = 0; i < length; i++) {
- for (int j = 0; j < inDoubleValues.length; j++) {
- _results[i] = Double.compare(inDoubleValues[j][i], doubleValues[i]) == 0 ? 1 : _results[i];
+ for (double[] inDoubleValue : inDoubleValues) {
+ if (doubleValues[i] == inDoubleValue[i]) {
Review comment:
Good point. Changed it to the same way as `equals()` so that the behavior is aligned with the `HashSet`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] yupeng9 commented on a change in pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
yupeng9 commented on a change in pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#discussion_r725179622
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/InTransformFunction.java
##########
@@ -81,77 +95,112 @@ public TransformResultMetadata getResultMetadata() {
switch (storedType) {
case INT:
int[] intValues = _transformFunction.transformToIntValuesSV(projectionBlock);
- int[][] inIntValues = new int[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inIntValues[i] = _valueTransformFunctions[i].transformToIntValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inIntValues.length; j++) {
- _results[i] = inIntValues[j][i] == intValues[i] ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
+ Set<Integer> inIntValues = new HashSet<>();
+ for (String inValue : _stringValueSet) {
+ inIntValues.add(Integer.parseInt(inValue));
+ }
+ for (int i = 0; i < length; i++) {
+ _results[i] = inIntValues.contains(intValues[i]) ? 1 : 0;
+ }
+ } else {
+ int[][] inIntValues = new int[_valueTransformFunctions.length][];
+ for (int i = 0; i < _valueTransformFunctions.length; i++) {
+ inIntValues[i] = _valueTransformFunctions[i].transformToIntValuesSV(projectionBlock);
+ }
+ for (int i = 0; i < length; i++) {
+ for (int j = 0; j < inIntValues.length; j++) {
+ _results[i] = inIntValues[j][i] == intValues[i] ? 1 : _results[i];
+ }
}
}
break;
case LONG:
long[] longValues = _transformFunction.transformToLongValuesSV(projectionBlock);
- long[][] inLongValues = new long[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inLongValues[i] = _valueTransformFunctions[i].transformToLongValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inLongValues.length; j++) {
- _results[i] = inLongValues[j][i] == longValues[i] ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
+ Set<Long> inLongValues = new HashSet<>();
+ for (String inValue : _stringValueSet) {
+ inLongValues.add(Long.parseLong(inValue));
+ }
+ for (int i = 0; i < length; i++) {
+ _results[i] = inLongValues.contains(longValues[i]) ? 1 : 0;
+ }
+ } else {
+ long[][] inLongValues = new long[_valueTransformFunctions.length][];
+ for (int i = 0; i < _valueTransformFunctions.length; i++) {
+ inLongValues[i] = _valueTransformFunctions[i].transformToLongValuesSV(projectionBlock);
+ }
+ for (int i = 0; i < length; i++) {
+ for (int j = 0; j < inLongValues.length; j++) {
+ _results[i] = inLongValues[j][i] == longValues[i] ? 1 : _results[i];
+ }
}
}
break;
case FLOAT:
float[] floatValues = _transformFunction.transformToFloatValuesSV(projectionBlock);
- float[][] inFloatValues = new float[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inFloatValues[i] = _valueTransformFunctions[i].transformToFloatValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inFloatValues.length; j++) {
- _results[i] = Float.compare(inFloatValues[j][i], floatValues[i]) == 0 ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
+ Set<Float> inFloatValues = new HashSet<>();
+ for (String inValue : _stringValueSet) {
+ inFloatValues.add(Float.parseFloat(inValue));
+ }
+ for (int i = 0; i < length; i++) {
+ _results[i] = inFloatValues.contains(floatValues[i]) ? 1 : 0;
+ }
+ } else {
+ float[][] inFloatValues = new float[_valueTransformFunctions.length][];
+ for (int i = 0; i < _valueTransformFunctions.length; i++) {
+ inFloatValues[i] = _valueTransformFunctions[i].transformToFloatValuesSV(projectionBlock);
+ }
+ for (int i = 0; i < length; i++) {
+ for (int j = 0; j < inFloatValues.length; j++) {
+ _results[i] = Float.compare(inFloatValues[j][i], floatValues[i]) == 0 ? 1 : _results[i];
+ }
}
}
break;
case DOUBLE:
double[] doubleValues = _transformFunction.transformToDoubleValuesSV(projectionBlock);
- double[][] inDoubleValues = new double[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inDoubleValues[i] = _valueTransformFunctions[i].transformToDoubleValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inDoubleValues.length; j++) {
- _results[i] = Double.compare(inDoubleValues[j][i], doubleValues[i]) == 0 ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
+ Set<Double> inDoubleValues = new HashSet<>();
+ for (String inValue : _stringValueSet) {
+ inDoubleValues.add(Double.parseDouble(inValue));
+ }
+ for (int i = 0; i < length; i++) {
+ _results[i] = inDoubleValues.contains(doubleValues[i]) ? 1 : 0;
+ }
+ } else {
+ double[][] inDoubleValues = new double[_valueTransformFunctions.length][];
+ for (int i = 0; i < _valueTransformFunctions.length; i++) {
+ inDoubleValues[i] = _valueTransformFunctions[i].transformToDoubleValuesSV(projectionBlock);
+ }
+ for (int i = 0; i < length; i++) {
+ for (int j = 0; j < inDoubleValues.length; j++) {
+ _results[i] = Double.compare(inDoubleValues[j][i], doubleValues[i]) == 0 ? 1 : _results[i];
+ }
}
}
break;
case STRING:
String[] stringValues = _transformFunction.transformToStringValuesSV(projectionBlock);
- String[][] inStringValues = new String[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inStringValues[i] = _valueTransformFunctions[i].transformToStringValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inStringValues.length; j++) {
- _results[i] = inStringValues[j][i].equals(stringValues[i]) ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
+ for (int i = 0; i < length; i++) {
+ _results[i] = _stringValueSet.contains(stringValues[i]) ? 1 : 0;
}
- }
- break;
- case BYTES:
- byte[][] bytesValues = _transformFunction.transformToBytesValuesSV(projectionBlock);
- byte[][][] inBytesValues = new byte[_valueTransformFunctions.length][][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inBytesValues[i] = _valueTransformFunctions[i].transformToBytesValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inBytesValues.length; j++) {
- _results[i] =
- new ByteArray(inBytesValues[j][i]).compareTo(new ByteArray(bytesValues[i])) == 0 ? 1 : _results[i];
+ } else {
+ String[][] inStringValues = new String[_valueTransformFunctions.length][];
+ for (int i = 0; i < _valueTransformFunctions.length; i++) {
+ inStringValues[i] = _valueTransformFunctions[i].transformToStringValuesSV(projectionBlock);
+ }
+ for (int i = 0; i < length; i++) {
+ for (int j = 0; j < inStringValues.length; j++) {
+ _results[i] = inStringValues[j][i].equals(stringValues[i]) ? 1 : _results[i];
+ }
}
}
break;
+ case BYTES:
+ throw new UnsupportedOperationException();
Review comment:
Do we have an example of reading bytes from literals? in hexString?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] richardstartin commented on a change in pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#discussion_r725170595
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/InTransformFunction.java
##########
@@ -81,77 +95,112 @@ public TransformResultMetadata getResultMetadata() {
switch (storedType) {
case INT:
int[] intValues = _transformFunction.transformToIntValuesSV(projectionBlock);
- int[][] inIntValues = new int[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inIntValues[i] = _valueTransformFunctions[i].transformToIntValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inIntValues.length; j++) {
- _results[i] = inIntValues[j][i] == intValues[i] ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
+ Set<Integer> inIntValues = new HashSet<>();
+ for (String inValue : _stringValueSet) {
+ inIntValues.add(Integer.parseInt(inValue));
+ }
+ for (int i = 0; i < length; i++) {
+ _results[i] = inIntValues.contains(intValues[i]) ? 1 : 0;
+ }
+ } else {
+ int[][] inIntValues = new int[_valueTransformFunctions.length][];
+ for (int i = 0; i < _valueTransformFunctions.length; i++) {
+ inIntValues[i] = _valueTransformFunctions[i].transformToIntValuesSV(projectionBlock);
+ }
+ for (int i = 0; i < length; i++) {
+ for (int j = 0; j < inIntValues.length; j++) {
+ _results[i] = inIntValues[j][i] == intValues[i] ? 1 : _results[i];
+ }
}
}
break;
case LONG:
long[] longValues = _transformFunction.transformToLongValuesSV(projectionBlock);
- long[][] inLongValues = new long[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inLongValues[i] = _valueTransformFunctions[i].transformToLongValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inLongValues.length; j++) {
- _results[i] = inLongValues[j][i] == longValues[i] ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
+ Set<Long> inLongValues = new HashSet<>();
+ for (String inValue : _stringValueSet) {
+ inLongValues.add(Long.parseLong(inValue));
+ }
+ for (int i = 0; i < length; i++) {
+ _results[i] = inLongValues.contains(longValues[i]) ? 1 : 0;
+ }
+ } else {
+ long[][] inLongValues = new long[_valueTransformFunctions.length][];
+ for (int i = 0; i < _valueTransformFunctions.length; i++) {
+ inLongValues[i] = _valueTransformFunctions[i].transformToLongValuesSV(projectionBlock);
+ }
+ for (int i = 0; i < length; i++) {
+ for (int j = 0; j < inLongValues.length; j++) {
+ _results[i] = inLongValues[j][i] == longValues[i] ? 1 : _results[i];
+ }
}
}
break;
case FLOAT:
float[] floatValues = _transformFunction.transformToFloatValuesSV(projectionBlock);
- float[][] inFloatValues = new float[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inFloatValues[i] = _valueTransformFunctions[i].transformToFloatValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inFloatValues.length; j++) {
- _results[i] = Float.compare(inFloatValues[j][i], floatValues[i]) == 0 ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
+ Set<Float> inFloatValues = new HashSet<>();
+ for (String inValue : _stringValueSet) {
+ inFloatValues.add(Float.parseFloat(inValue));
+ }
+ for (int i = 0; i < length; i++) {
+ _results[i] = inFloatValues.contains(floatValues[i]) ? 1 : 0;
+ }
+ } else {
+ float[][] inFloatValues = new float[_valueTransformFunctions.length][];
+ for (int i = 0; i < _valueTransformFunctions.length; i++) {
+ inFloatValues[i] = _valueTransformFunctions[i].transformToFloatValuesSV(projectionBlock);
+ }
+ for (int i = 0; i < length; i++) {
+ for (int j = 0; j < inFloatValues.length; j++) {
+ _results[i] = Float.compare(inFloatValues[j][i], floatValues[i]) == 0 ? 1 : _results[i];
+ }
}
}
break;
case DOUBLE:
double[] doubleValues = _transformFunction.transformToDoubleValuesSV(projectionBlock);
- double[][] inDoubleValues = new double[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inDoubleValues[i] = _valueTransformFunctions[i].transformToDoubleValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inDoubleValues.length; j++) {
- _results[i] = Double.compare(inDoubleValues[j][i], doubleValues[i]) == 0 ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
+ Set<Double> inDoubleValues = new HashSet<>();
+ for (String inValue : _stringValueSet) {
+ inDoubleValues.add(Double.parseDouble(inValue));
+ }
+ for (int i = 0; i < length; i++) {
+ _results[i] = inDoubleValues.contains(doubleValues[i]) ? 1 : 0;
+ }
+ } else {
+ double[][] inDoubleValues = new double[_valueTransformFunctions.length][];
+ for (int i = 0; i < _valueTransformFunctions.length; i++) {
+ inDoubleValues[i] = _valueTransformFunctions[i].transformToDoubleValuesSV(projectionBlock);
+ }
+ for (int i = 0; i < length; i++) {
+ for (int j = 0; j < inDoubleValues.length; j++) {
+ _results[i] = Double.compare(inDoubleValues[j][i], doubleValues[i]) == 0 ? 1 : _results[i];
+ }
}
}
break;
case STRING:
String[] stringValues = _transformFunction.transformToStringValuesSV(projectionBlock);
- String[][] inStringValues = new String[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inStringValues[i] = _valueTransformFunctions[i].transformToStringValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inStringValues.length; j++) {
- _results[i] = inStringValues[j][i].equals(stringValues[i]) ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
+ for (int i = 0; i < length; i++) {
+ _results[i] = _stringValueSet.contains(stringValues[i]) ? 1 : 0;
}
- }
- break;
- case BYTES:
- byte[][] bytesValues = _transformFunction.transformToBytesValuesSV(projectionBlock);
- byte[][][] inBytesValues = new byte[_valueTransformFunctions.length][][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inBytesValues[i] = _valueTransformFunctions[i].transformToBytesValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inBytesValues.length; j++) {
- _results[i] =
- new ByteArray(inBytesValues[j][i]).compareTo(new ByteArray(bytesValues[i])) == 0 ? 1 : _results[i];
+ } else {
+ String[][] inStringValues = new String[_valueTransformFunctions.length][];
+ for (int i = 0; i < _valueTransformFunctions.length; i++) {
+ inStringValues[i] = _valueTransformFunctions[i].transformToStringValuesSV(projectionBlock);
+ }
+ for (int i = 0; i < length; i++) {
+ for (int j = 0; j < inStringValues.length; j++) {
+ _results[i] = inStringValues[j][i].equals(stringValues[i]) ? 1 : _results[i];
+ }
}
}
break;
+ case BYTES:
+ throw new UnsupportedOperationException();
Review comment:
I can imagine a use case for `BYTES` but not floating point.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#discussion_r724634537
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/InTransformFunction.java
##########
@@ -0,0 +1,161 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import com.google.common.base.Preconditions;
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.common.function.TransformFunctionType;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.core.plan.DocIdSetPlanNode;
+import org.apache.pinot.segment.spi.datasource.DataSource;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.utils.ByteArray;
+
+
+/**
+ * The IN transform function takes more than 1 arguments:
+ * <ul>
+ * <li>Expression: a single-value expression</li>
+ * <li>values: a set of literal strings</li>
+ * </ul>
+ * <p>For each docId, the function returns {@code 1} if the set of values contains the value of the expression,
+ * {code 0} if not.
+ * <p>E.g. {@code SELECT col IN ('a','b','c') FROM myTable)}
+ */
+public class InTransformFunction extends BaseTransformFunction {
+ private TransformFunction _transformFunction;
+ private TransformFunction[] _valueTransformFunctions;
+ private int[] _results;
+
+ @Override
+ public String getName() {
+ return TransformFunctionType.IN.getName();
+ }
+
+ @Override
+ public void init(List<TransformFunction> arguments, Map<String, DataSource> dataSourceMap) {
+ Preconditions.checkArgument(arguments.size() >= 2,
+ "at least 2 arguments are required for IN transform function: expression, values");
+ Preconditions.checkArgument(arguments.get(0).getResultMetadata().isSingleValue(),
+ "First argument for IN transform function must be a single-value expression");
+ _transformFunction = arguments.get(0);
+ _valueTransformFunctions = new TransformFunction[arguments.size() - 1];
Review comment:
No, but I doubt if any user will use it that way. Doing per-value check will be super expensive. I think the `in` will be quite similar to `valuein`, where all the values must be literals. You may refer to the `ValueInTransformFunction` class where we also optimize the dictionary-encoded columns.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#discussion_r725540682
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/InTransformFunction.java
##########
@@ -81,77 +95,112 @@ public TransformResultMetadata getResultMetadata() {
switch (storedType) {
case INT:
int[] intValues = _transformFunction.transformToIntValuesSV(projectionBlock);
- int[][] inIntValues = new int[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inIntValues[i] = _valueTransformFunctions[i].transformToIntValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inIntValues.length; j++) {
- _results[i] = inIntValues[j][i] == intValues[i] ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
+ Set<Integer> inIntValues = new HashSet<>();
+ for (String inValue : _stringValueSet) {
+ inIntValues.add(Integer.parseInt(inValue));
+ }
+ for (int i = 0; i < length; i++) {
+ _results[i] = inIntValues.contains(intValues[i]) ? 1 : 0;
+ }
+ } else {
+ int[][] inIntValues = new int[_valueTransformFunctions.length][];
+ for (int i = 0; i < _valueTransformFunctions.length; i++) {
+ inIntValues[i] = _valueTransformFunctions[i].transformToIntValuesSV(projectionBlock);
+ }
+ for (int i = 0; i < length; i++) {
+ for (int j = 0; j < inIntValues.length; j++) {
+ _results[i] = inIntValues[j][i] == intValues[i] ? 1 : _results[i];
+ }
}
}
break;
case LONG:
long[] longValues = _transformFunction.transformToLongValuesSV(projectionBlock);
- long[][] inLongValues = new long[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inLongValues[i] = _valueTransformFunctions[i].transformToLongValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inLongValues.length; j++) {
- _results[i] = inLongValues[j][i] == longValues[i] ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
+ Set<Long> inLongValues = new HashSet<>();
+ for (String inValue : _stringValueSet) {
+ inLongValues.add(Long.parseLong(inValue));
+ }
+ for (int i = 0; i < length; i++) {
+ _results[i] = inLongValues.contains(longValues[i]) ? 1 : 0;
+ }
+ } else {
+ long[][] inLongValues = new long[_valueTransformFunctions.length][];
+ for (int i = 0; i < _valueTransformFunctions.length; i++) {
+ inLongValues[i] = _valueTransformFunctions[i].transformToLongValuesSV(projectionBlock);
+ }
+ for (int i = 0; i < length; i++) {
+ for (int j = 0; j < inLongValues.length; j++) {
+ _results[i] = inLongValues[j][i] == longValues[i] ? 1 : _results[i];
+ }
}
}
break;
case FLOAT:
float[] floatValues = _transformFunction.transformToFloatValuesSV(projectionBlock);
- float[][] inFloatValues = new float[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inFloatValues[i] = _valueTransformFunctions[i].transformToFloatValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inFloatValues.length; j++) {
- _results[i] = Float.compare(inFloatValues[j][i], floatValues[i]) == 0 ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
Review comment:
@richardstartin Does the test fail because it should be `0.1F + 0.9F`? I tried it and it works.
Good point on the fuzzy comparison, but currently we don't have that and only use exact match for floating point values (same as the IN in where clause, and EQUALS, NOT_EQUALS etc.). I'd suggest keeping the current behavior and revisit if necessary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] yupeng9 commented on a change in pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
yupeng9 commented on a change in pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#discussion_r725664968
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/InTransformFunction.java
##########
@@ -86,125 +139,239 @@ public TransformResultMetadata getResultMetadata() {
@Override
public int[] transformToIntValuesSV(ProjectionBlock projectionBlock) {
- if (_results == null) {
- _results = new int[DocIdSetPlanNode.MAX_DOC_PER_CALL];
+ if (_intValuesSV == null) {
+ _intValuesSV = new int[DocIdSetPlanNode.MAX_DOC_PER_CALL];
+ } else {
+ Arrays.fill(_intValuesSV, 0);
}
int length = projectionBlock.getNumDocs();
- FieldSpec.DataType storedType = _transformFunction.getResultMetadata().getDataType().getStoredType();
- switch (storedType) {
- case INT:
- int[] intValues = _transformFunction.transformToIntValuesSV(projectionBlock);
- if (!_stringValueSet.isEmpty()) {
- Set<Integer> inIntValues = new HashSet<>();
- for (String inValue : _stringValueSet) {
- inIntValues.add(Integer.parseInt(inValue));
- }
- for (int i = 0; i < length; i++) {
- _results[i] = inIntValues.contains(intValues[i]) ? 1 : 0;
- }
- } else {
- int[][] inIntValues = new int[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inIntValues[i] = _valueTransformFunctions[i].transformToIntValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inIntValues.length; j++) {
- _results[i] = inIntValues[j][i] == intValues[i] ? 1 : _results[i];
+ TransformResultMetadata mainFunctionMetadata = _mainFunction.getResultMetadata();
+ DataType storedType = mainFunctionMetadata.getDataType().getStoredType();
+ if (_valueSet != null) {
+ if (_mainFunction.getResultMetadata().isSingleValue()) {
+ switch (storedType) {
+ case INT:
+ IntOpenHashSet inIntValues = (IntOpenHashSet) _valueSet;
+ int[] intValues = _mainFunction.transformToIntValuesSV(projectionBlock);
+ for (int i = 0; i < length; i++) {
+ if (inIntValues.contains(intValues[i])) {
+ _intValuesSV[i] = 1;
+ }
}
- }
+ break;
+ case LONG:
+ LongOpenHashSet inLongValues = (LongOpenHashSet) _valueSet;
+ long[] longValues = _mainFunction.transformToLongValuesSV(projectionBlock);
+ for (int i = 0; i < length; i++) {
+ if (inLongValues.contains(longValues[i])) {
+ _intValuesSV[i] = 1;
+ }
+ }
+ break;
+ case FLOAT:
+ FloatOpenHashSet inFloatValues = (FloatOpenHashSet) _valueSet;
+ float[] floatValues = _mainFunction.transformToFloatValuesSV(projectionBlock);
+ for (int i = 0; i < length; i++) {
+ if (inFloatValues.contains(floatValues[i])) {
+ _intValuesSV[i] = 1;
+ }
+ }
+ break;
+ case DOUBLE:
+ DoubleOpenHashSet inDoubleValues = (DoubleOpenHashSet) _valueSet;
+ double[] doubleValues = _mainFunction.transformToDoubleValuesSV(projectionBlock);
+ for (int i = 0; i < length; i++) {
+ if (inDoubleValues.contains(doubleValues[i])) {
+ _intValuesSV[i] = 1;
+ }
+ }
+ break;
+ case STRING:
+ ObjectOpenHashSet<String> inStringValues = (ObjectOpenHashSet<String>) _valueSet;
+ String[] stringValues = _mainFunction.transformToStringValuesSV(projectionBlock);
+ for (int i = 0; i < length; i++) {
+ if (inStringValues.contains(stringValues[i])) {
+ _intValuesSV[i] = 1;
+ }
+ }
+ break;
+ case BYTES:
+ ObjectOpenHashSet<ByteArray> inBytesValues = (ObjectOpenHashSet<ByteArray>) _valueSet;
+ byte[][] bytesValues = _mainFunction.transformToBytesValuesSV(projectionBlock);
+ for (int i = 0; i < length; i++) {
+ if (inBytesValues.contains(new ByteArray(bytesValues[i]))) {
+ _intValuesSV[i] = 1;
+ }
+ }
+ break;
+ default:
+ throw new IllegalStateException();
}
- break;
- case LONG:
- long[] longValues = _transformFunction.transformToLongValuesSV(projectionBlock);
- if (!_stringValueSet.isEmpty()) {
- Set<Long> inLongValues = new HashSet<>();
- for (String inValue : _stringValueSet) {
- inLongValues.add(Long.parseLong(inValue));
- }
- for (int i = 0; i < length; i++) {
- _results[i] = inLongValues.contains(longValues[i]) ? 1 : 0;
- }
- } else {
- long[][] inLongValues = new long[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inLongValues[i] = _valueTransformFunctions[i].transformToLongValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inLongValues.length; j++) {
- _results[i] = inLongValues[j][i] == longValues[i] ? 1 : _results[i];
+ } else {
+ switch (storedType) {
+ case INT:
+ IntOpenHashSet inIntValues = (IntOpenHashSet) _valueSet;
+ int[][] intValues = _mainFunction.transformToIntValuesMV(projectionBlock);
+ for (int i = 0; i < length; i++) {
+ for (int intValue : intValues[i]) {
+ if (inIntValues.contains(intValue)) {
+ _intValuesSV[i] = 1;
+ break;
+ }
+ }
}
- }
+ break;
+ case LONG:
+ LongOpenHashSet inLongValues = (LongOpenHashSet) _valueSet;
+ long[][] longValues = _mainFunction.transformToLongValuesMV(projectionBlock);
+ for (int i = 0; i < length; i++) {
+ for (long longValue : longValues[i]) {
+ if (inLongValues.contains(longValue)) {
+ _intValuesSV[i] = 1;
+ break;
+ }
+ }
+ }
+ break;
+ case FLOAT:
+ FloatOpenHashSet inFloatValues = (FloatOpenHashSet) _valueSet;
+ float[][] floatValues = _mainFunction.transformToFloatValuesMV(projectionBlock);
+ for (int i = 0; i < length; i++) {
+ for (float floatValue : floatValues[i]) {
+ if (inFloatValues.contains(floatValue)) {
+ _intValuesSV[i] = 1;
+ break;
+ }
+ }
+ }
+ break;
+ case DOUBLE:
+ DoubleOpenHashSet inDoubleValues = (DoubleOpenHashSet) _valueSet;
+ double[][] doubleValues = _mainFunction.transformToDoubleValuesMV(projectionBlock);
+ for (int i = 0; i < length; i++) {
+ for (double doubleValue : doubleValues[i]) {
+ if (inDoubleValues.contains(doubleValue)) {
+ _intValuesSV[i] = 1;
+ break;
+ }
+ }
+ }
+ break;
+ case STRING:
+ ObjectOpenHashSet<String> inStringValues = (ObjectOpenHashSet<String>) _valueSet;
+ String[][] stringValues = _mainFunction.transformToStringValuesMV(projectionBlock);
+ for (int i = 0; i < length; i++) {
+ for (String stringValue : stringValues[i]) {
+ if (inStringValues.contains(stringValue)) {
+ _intValuesSV[i] = 1;
+ break;
+ }
+ }
+ }
+ break;
+ default:
+ throw new IllegalStateException();
}
- break;
- case FLOAT:
- float[] floatValues = _transformFunction.transformToFloatValuesSV(projectionBlock);
- if (!_stringValueSet.isEmpty()) {
- Set<Float> inFloatValues = new HashSet<>();
- for (String inValue : _stringValueSet) {
- inFloatValues.add(Float.parseFloat(inValue));
+ }
+ } else {
+ int numValues = _valueFunctions.length;
+ switch (storedType) {
+ case INT:
+ int[] intValues = _mainFunction.transformToIntValuesSV(projectionBlock);
+ int[][] inIntValues = new int[numValues][];
+ for (int i = 0; i < numValues; i++) {
+ inIntValues[i] = _valueFunctions[i].transformToIntValuesSV(projectionBlock);
}
for (int i = 0; i < length; i++) {
- _results[i] = inFloatValues.contains(floatValues[i]) ? 1 : 0;
+ for (int[] inIntValue : inIntValues) {
+ if (intValues[i] == inIntValue[i]) {
+ _intValuesSV[i] = 1;
+ break;
+ }
+ }
}
- } else {
- float[][] inFloatValues = new float[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inFloatValues[i] = _valueTransformFunctions[i].transformToFloatValuesSV(projectionBlock);
+ break;
+ case LONG:
+ long[] longValues = _mainFunction.transformToLongValuesSV(projectionBlock);
+ long[][] inLongValues = new long[numValues][];
+ for (int i = 0; i < numValues; i++) {
+ inLongValues[i] = _valueFunctions[i].transformToLongValuesSV(projectionBlock);
}
for (int i = 0; i < length; i++) {
- for (int j = 0; j < inFloatValues.length; j++) {
- _results[i] = Float.compare(inFloatValues[j][i], floatValues[i]) == 0 ? 1 : _results[i];
+ for (long[] inLongValue : inLongValues) {
+ if (longValues[i] == inLongValue[i]) {
+ _intValuesSV[i] = 1;
+ break;
+ }
}
}
- }
- break;
- case DOUBLE:
- double[] doubleValues = _transformFunction.transformToDoubleValuesSV(projectionBlock);
- if (!_stringValueSet.isEmpty()) {
- Set<Double> inDoubleValues = new HashSet<>();
- for (String inValue : _stringValueSet) {
- inDoubleValues.add(Double.parseDouble(inValue));
+ break;
+ case FLOAT:
+ float[] floatValues = _mainFunction.transformToFloatValuesSV(projectionBlock);
+ float[][] inFloatValues = new float[numValues][];
+ for (int i = 0; i < numValues; i++) {
+ inFloatValues[i] = _valueFunctions[i].transformToFloatValuesSV(projectionBlock);
}
for (int i = 0; i < length; i++) {
- _results[i] = inDoubleValues.contains(doubleValues[i]) ? 1 : 0;
+ for (float[] inFloatValue : inFloatValues) {
+ if (floatValues[i] == inFloatValue[i]) {
+ _intValuesSV[i] = 1;
+ break;
+ }
+ }
}
- } else {
- double[][] inDoubleValues = new double[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inDoubleValues[i] = _valueTransformFunctions[i].transformToDoubleValuesSV(projectionBlock);
+ break;
+ case DOUBLE:
+ double[] doubleValues = _mainFunction.transformToDoubleValuesSV(projectionBlock);
+ double[][] inDoubleValues = new double[numValues][];
+ for (int i = 0; i < numValues; i++) {
+ inDoubleValues[i] = _valueFunctions[i].transformToDoubleValuesSV(projectionBlock);
}
for (int i = 0; i < length; i++) {
- for (int j = 0; j < inDoubleValues.length; j++) {
- _results[i] = Double.compare(inDoubleValues[j][i], doubleValues[i]) == 0 ? 1 : _results[i];
+ for (double[] inDoubleValue : inDoubleValues) {
+ if (doubleValues[i] == inDoubleValue[i]) {
Review comment:
shall we use Double.compare, which has some handling of edge cases http://hg.openjdk.java.net/jdk/jdk11/file/1ddf9a99e4ad/src/java.base/share/classes/java/lang/Double.java#l1016?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] yupeng9 commented on pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
yupeng9 commented on pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#issuecomment-938203773
Yes, it's standard SQL and calcite supports it.
It's usually used together with `CASE WHEN`. For example,
``
select COUNT(CASE WHEN status IN ('Open', 'Dispatched') THEN driveruuid END) as ActiveDriverCount`
``
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] yupeng9 commented on pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
yupeng9 commented on pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#issuecomment-939512723
LGTM. Thanks for the code improvement!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] yupeng9 commented on a change in pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
yupeng9 commented on a change in pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#discussion_r724611951
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/InTransformFunction.java
##########
@@ -0,0 +1,161 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import com.google.common.base.Preconditions;
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.common.function.TransformFunctionType;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.core.plan.DocIdSetPlanNode;
+import org.apache.pinot.segment.spi.datasource.DataSource;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.utils.ByteArray;
+
+
+/**
+ * The IN transform function takes more than 1 arguments:
+ * <ul>
+ * <li>Expression: a single-value expression</li>
+ * <li>values: a set of literal strings</li>
+ * </ul>
+ * <p>For each docId, the function returns {@code 1} if the set of values contains the value of the expression,
+ * {code 0} if not.
+ * <p>E.g. {@code SELECT col IN ('a','b','c') FROM myTable)}
+ */
+public class InTransformFunction extends BaseTransformFunction {
+ private TransformFunction _transformFunction;
+ private TransformFunction[] _valueTransformFunctions;
+ private int[] _results;
+
+ @Override
+ public String getName() {
+ return TransformFunctionType.IN.getName();
+ }
+
+ @Override
+ public void init(List<TransformFunction> arguments, Map<String, DataSource> dataSourceMap) {
+ Preconditions.checkArgument(arguments.size() >= 2,
+ "at least 2 arguments are required for IN transform function: expression, values");
+ Preconditions.checkArgument(arguments.get(0).getResultMetadata().isSingleValue(),
+ "First argument for IN transform function must be a single-value expression");
Review comment:
there's a `valueIn` function for this purpose
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] richardstartin commented on a change in pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#discussion_r725545318
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/InTransformFunction.java
##########
@@ -81,77 +95,112 @@ public TransformResultMetadata getResultMetadata() {
switch (storedType) {
case INT:
int[] intValues = _transformFunction.transformToIntValuesSV(projectionBlock);
- int[][] inIntValues = new int[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inIntValues[i] = _valueTransformFunctions[i].transformToIntValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inIntValues.length; j++) {
- _results[i] = inIntValues[j][i] == intValues[i] ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
+ Set<Integer> inIntValues = new HashSet<>();
+ for (String inValue : _stringValueSet) {
+ inIntValues.add(Integer.parseInt(inValue));
+ }
+ for (int i = 0; i < length; i++) {
+ _results[i] = inIntValues.contains(intValues[i]) ? 1 : 0;
+ }
+ } else {
+ int[][] inIntValues = new int[_valueTransformFunctions.length][];
+ for (int i = 0; i < _valueTransformFunctions.length; i++) {
+ inIntValues[i] = _valueTransformFunctions[i].transformToIntValuesSV(projectionBlock);
+ }
+ for (int i = 0; i < length; i++) {
+ for (int j = 0; j < inIntValues.length; j++) {
+ _results[i] = inIntValues[j][i] == intValues[i] ? 1 : _results[i];
+ }
}
}
break;
case LONG:
long[] longValues = _transformFunction.transformToLongValuesSV(projectionBlock);
- long[][] inLongValues = new long[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inLongValues[i] = _valueTransformFunctions[i].transformToLongValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inLongValues.length; j++) {
- _results[i] = inLongValues[j][i] == longValues[i] ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
+ Set<Long> inLongValues = new HashSet<>();
+ for (String inValue : _stringValueSet) {
+ inLongValues.add(Long.parseLong(inValue));
+ }
+ for (int i = 0; i < length; i++) {
+ _results[i] = inLongValues.contains(longValues[i]) ? 1 : 0;
+ }
+ } else {
+ long[][] inLongValues = new long[_valueTransformFunctions.length][];
+ for (int i = 0; i < _valueTransformFunctions.length; i++) {
+ inLongValues[i] = _valueTransformFunctions[i].transformToLongValuesSV(projectionBlock);
+ }
+ for (int i = 0; i < length; i++) {
+ for (int j = 0; j < inLongValues.length; j++) {
+ _results[i] = inLongValues[j][i] == longValues[i] ? 1 : _results[i];
+ }
}
}
break;
case FLOAT:
float[] floatValues = _transformFunction.transformToFloatValuesSV(projectionBlock);
- float[][] inFloatValues = new float[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inFloatValues[i] = _valueTransformFunctions[i].transformToFloatValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inFloatValues.length; j++) {
- _results[i] = Float.compare(inFloatValues[j][i], floatValues[i]) == 0 ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
Review comment:
Oh, what a silly typo when trying to demonstrate the problem with the concept. Try this one instead:
```java
@Test
public void testFloatSet() {
Set<Float> set = new HashSet<>();
set.add(0.8F);
assertTrue(set.contains(0.1F + 0.1F + 0.1F + 0.1F + 0.1F + 0.1F + 0.1F + 0.1F));
}
```
I think you understand the pitfalls of floating point equality and the point I am trying to make. Distinguishing between 0.8 and 0.8000001 is poor UX.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] richardstartin commented on a change in pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#discussion_r725191626
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/InTransformFunction.java
##########
@@ -81,77 +95,112 @@ public TransformResultMetadata getResultMetadata() {
switch (storedType) {
case INT:
int[] intValues = _transformFunction.transformToIntValuesSV(projectionBlock);
- int[][] inIntValues = new int[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inIntValues[i] = _valueTransformFunctions[i].transformToIntValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inIntValues.length; j++) {
- _results[i] = inIntValues[j][i] == intValues[i] ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
+ Set<Integer> inIntValues = new HashSet<>();
+ for (String inValue : _stringValueSet) {
+ inIntValues.add(Integer.parseInt(inValue));
+ }
+ for (int i = 0; i < length; i++) {
+ _results[i] = inIntValues.contains(intValues[i]) ? 1 : 0;
+ }
+ } else {
+ int[][] inIntValues = new int[_valueTransformFunctions.length][];
+ for (int i = 0; i < _valueTransformFunctions.length; i++) {
+ inIntValues[i] = _valueTransformFunctions[i].transformToIntValuesSV(projectionBlock);
+ }
+ for (int i = 0; i < length; i++) {
+ for (int j = 0; j < inIntValues.length; j++) {
+ _results[i] = inIntValues[j][i] == intValues[i] ? 1 : _results[i];
+ }
}
}
break;
case LONG:
long[] longValues = _transformFunction.transformToLongValuesSV(projectionBlock);
- long[][] inLongValues = new long[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inLongValues[i] = _valueTransformFunctions[i].transformToLongValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inLongValues.length; j++) {
- _results[i] = inLongValues[j][i] == longValues[i] ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
+ Set<Long> inLongValues = new HashSet<>();
+ for (String inValue : _stringValueSet) {
+ inLongValues.add(Long.parseLong(inValue));
+ }
+ for (int i = 0; i < length; i++) {
+ _results[i] = inLongValues.contains(longValues[i]) ? 1 : 0;
+ }
+ } else {
+ long[][] inLongValues = new long[_valueTransformFunctions.length][];
+ for (int i = 0; i < _valueTransformFunctions.length; i++) {
+ inLongValues[i] = _valueTransformFunctions[i].transformToLongValuesSV(projectionBlock);
+ }
+ for (int i = 0; i < length; i++) {
+ for (int j = 0; j < inLongValues.length; j++) {
+ _results[i] = inLongValues[j][i] == longValues[i] ? 1 : _results[i];
+ }
}
}
break;
case FLOAT:
float[] floatValues = _transformFunction.transformToFloatValuesSV(projectionBlock);
- float[][] inFloatValues = new float[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inFloatValues[i] = _valueTransformFunctions[i].transformToFloatValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inFloatValues.length; j++) {
- _results[i] = Float.compare(inFloatValues[j][i], floatValues[i]) == 0 ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
+ Set<Float> inFloatValues = new HashSet<>();
+ for (String inValue : _stringValueSet) {
+ inFloatValues.add(Float.parseFloat(inValue));
+ }
+ for (int i = 0; i < length; i++) {
+ _results[i] = inFloatValues.contains(floatValues[i]) ? 1 : 0;
+ }
+ } else {
+ float[][] inFloatValues = new float[_valueTransformFunctions.length][];
+ for (int i = 0; i < _valueTransformFunctions.length; i++) {
+ inFloatValues[i] = _valueTransformFunctions[i].transformToFloatValuesSV(projectionBlock);
+ }
+ for (int i = 0; i < length; i++) {
+ for (int j = 0; j < inFloatValues.length; j++) {
+ _results[i] = Float.compare(inFloatValues[j][i], floatValues[i]) == 0 ? 1 : _results[i];
+ }
}
}
break;
case DOUBLE:
double[] doubleValues = _transformFunction.transformToDoubleValuesSV(projectionBlock);
- double[][] inDoubleValues = new double[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inDoubleValues[i] = _valueTransformFunctions[i].transformToDoubleValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inDoubleValues.length; j++) {
- _results[i] = Double.compare(inDoubleValues[j][i], doubleValues[i]) == 0 ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
+ Set<Double> inDoubleValues = new HashSet<>();
+ for (String inValue : _stringValueSet) {
+ inDoubleValues.add(Double.parseDouble(inValue));
+ }
+ for (int i = 0; i < length; i++) {
+ _results[i] = inDoubleValues.contains(doubleValues[i]) ? 1 : 0;
+ }
+ } else {
+ double[][] inDoubleValues = new double[_valueTransformFunctions.length][];
+ for (int i = 0; i < _valueTransformFunctions.length; i++) {
+ inDoubleValues[i] = _valueTransformFunctions[i].transformToDoubleValuesSV(projectionBlock);
+ }
+ for (int i = 0; i < length; i++) {
+ for (int j = 0; j < inDoubleValues.length; j++) {
+ _results[i] = Double.compare(inDoubleValues[j][i], doubleValues[i]) == 0 ? 1 : _results[i];
+ }
}
}
break;
case STRING:
String[] stringValues = _transformFunction.transformToStringValuesSV(projectionBlock);
- String[][] inStringValues = new String[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inStringValues[i] = _valueTransformFunctions[i].transformToStringValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inStringValues.length; j++) {
- _results[i] = inStringValues[j][i].equals(stringValues[i]) ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
+ for (int i = 0; i < length; i++) {
+ _results[i] = _stringValueSet.contains(stringValues[i]) ? 1 : 0;
}
- }
- break;
- case BYTES:
- byte[][] bytesValues = _transformFunction.transformToBytesValuesSV(projectionBlock);
- byte[][][] inBytesValues = new byte[_valueTransformFunctions.length][][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inBytesValues[i] = _valueTransformFunctions[i].transformToBytesValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inBytesValues.length; j++) {
- _results[i] =
- new ByteArray(inBytesValues[j][i]).compareTo(new ByteArray(bytesValues[i])) == 0 ? 1 : _results[i];
+ } else {
+ String[][] inStringValues = new String[_valueTransformFunctions.length][];
+ for (int i = 0; i < _valueTransformFunctions.length; i++) {
+ inStringValues[i] = _valueTransformFunctions[i].transformToStringValuesSV(projectionBlock);
+ }
+ for (int i = 0; i < length; i++) {
+ for (int j = 0; j < inStringValues.length; j++) {
+ _results[i] = inStringValues[j][i].equals(stringValues[i]) ? 1 : _results[i];
+ }
}
}
break;
+ case BYTES:
+ throw new UnsupportedOperationException();
Review comment:
Sorry I'm new to pinot, it doesn't seem that there is a base64 function to specify binary literals. So I wouldn't suggest adding to implement this for bytes, I can just imagine wanting to do this for e.g. security use cases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] yupeng9 commented on a change in pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
yupeng9 commented on a change in pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#discussion_r725176140
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/InTransformFunction.java
##########
@@ -0,0 +1,161 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import com.google.common.base.Preconditions;
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.common.function.TransformFunctionType;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.core.plan.DocIdSetPlanNode;
+import org.apache.pinot.segment.spi.datasource.DataSource;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.utils.ByteArray;
+
+
+/**
+ * The IN transform function takes more than 1 arguments:
+ * <ul>
+ * <li>Expression: a single-value expression</li>
+ * <li>values: a set of literal strings</li>
+ * </ul>
+ * <p>For each docId, the function returns {@code 1} if the set of values contains the value of the expression,
+ * {code 0} if not.
+ * <p>E.g. {@code SELECT col IN ('a','b','c') FROM myTable)}
+ */
+public class InTransformFunction extends BaseTransformFunction {
+ private TransformFunction _transformFunction;
+ private TransformFunction[] _valueTransformFunctions;
+ private int[] _results;
+
+ @Override
+ public String getName() {
+ return TransformFunctionType.IN.getName();
+ }
+
+ @Override
+ public void init(List<TransformFunction> arguments, Map<String, DataSource> dataSourceMap) {
+ Preconditions.checkArgument(arguments.size() >= 2,
+ "at least 2 arguments are required for IN transform function: expression, values");
+ Preconditions.checkArgument(arguments.get(0).getResultMetadata().isSingleValue(),
+ "First argument for IN transform function must be a single-value expression");
Review comment:
hmm, can you write a SQL query to illustrate this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#issuecomment-938200237
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7542](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (d38c57a) into [master](https://codecov.io/gh/apache/pinot/commit/7be21c80b352a02c58807b3c95d535f5fae394cc?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (7be21c8) will **decrease** coverage by `6.74%`.
> The diff coverage is `47.98%`.
> :exclamation: Current head d38c57a differs from pull request most recent head 3c3ba00. Consider uploading reports for the commit 3c3ba00 to get more accurate results
[](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7542 +/- ##
============================================
- Coverage 72.54% 65.79% -6.75%
- Complexity 3405 3414 +9
============================================
Files 1523 1479 -44
Lines 75653 74061 -1592
Branches 11032 10906 -126
============================================
- Hits 54880 48728 -6152
- Misses 17078 21874 +4796
+ Partials 3695 3459 -236
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| integration2 | `?` | |
| unittests1 | `69.77% <50.79%> (-0.04%)` | :arrow_down: |
| unittests2 | `15.20% <5.03%> (-0.06%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [.../org/apache/pinot/core/common/MinionConstants.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9jb21tb24vTWluaW9uQ29uc3RhbnRzLmphdmE=) | `0.00% <ø> (ø)` | |
| [...ava/org/apache/pinot/minion/BaseMinionStarter.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtbWluaW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9taW5pb24vQmFzZU1pbmlvblN0YXJ0ZXIuamF2YQ==) | `0.00% <0.00%> (-77.68%)` | :arrow_down: |
| [...t/minion/executor/TaskExecutorFactoryRegistry.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtbWluaW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9taW5pb24vZXhlY3V0b3IvVGFza0V4ZWN1dG9yRmFjdG9yeVJlZ2lzdHJ5LmphdmE=) | `12.00% <0.00%> (-76.00%)` | :arrow_down: |
| [...rawindex/ConvertToRawIndexTaskExecutorFactory.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvY29udmVydHRvcmF3aW5kZXgvQ29udmVydFRvUmF3SW5kZXhUYXNrRXhlY3V0b3JGYWN0b3J5LmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...ks/mergerollup/MergeRollupTaskExecutorFactory.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvbWVyZ2Vyb2xsdXAvTWVyZ2VSb2xsdXBUYXNrRXhlY3V0b3JGYWN0b3J5LmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...on/tasks/mergerollup/MergeRollupTaskGenerator.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvbWVyZ2Vyb2xsdXAvTWVyZ2VSb2xsdXBUYXNrR2VuZXJhdG9yLmphdmE=) | `76.36% <ø> (-11.37%)` | :arrow_down: |
| [...n/minion/tasks/purge/PurgeTaskExecutorFactory.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvcHVyZ2UvUHVyZ2VUYXNrRXhlY3V0b3JGYWN0b3J5LmphdmE=) | `0.00% <0.00%> (-75.00%)` | :arrow_down: |
| [.../RealtimeToOfflineSegmentsTaskExecutorFactory.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvcmVhbHRpbWV0b29mZmxpbmVzZWdtZW50cy9SZWFsdGltZVRvT2ZmbGluZVNlZ21lbnRzVGFza0V4ZWN1dG9yRmFjdG9yeS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...h/SegmentGenerationAndPushTaskExecutorFactory.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3Mvc2VnbWVudGdlbmVyYXRpb25hbmRwdXNoL1NlZ21lbnRHZW5lcmF0aW9uQW5kUHVzaFRhc2tFeGVjdXRvckZhY3RvcnkuamF2YQ==) | `0.00% <0.00%> (-75.00%)` | :arrow_down: |
| [.../main/java/org/apache/pinot/minion/MinionConf.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtbWluaW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9taW5pb24vTWluaW9uQ29uZi5qYXZh) | `16.66% <16.66%> (ø)` | |
| ... and [375 more](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [19f66fd...3c3ba00](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#issuecomment-938200237
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7542](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (4942bc7) into [master](https://codecov.io/gh/apache/pinot/commit/7be21c80b352a02c58807b3c95d535f5fae394cc?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (7be21c8) will **decrease** coverage by `2.71%`.
> The diff coverage is `55.86%`.
[](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7542 +/- ##
============================================
- Coverage 72.54% 69.82% -2.72%
+ Complexity 3405 3334 -71
============================================
Files 1523 1130 -393
Lines 75653 53690 -21963
Branches 11032 8128 -2904
============================================
- Hits 54880 37488 -17392
+ Misses 17078 13548 -3530
+ Partials 3695 2654 -1041
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| integration2 | `?` | |
| unittests1 | `69.82% <55.86%> (+0.01%)` | :arrow_up: |
| unittests2 | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...erator/transform/function/InTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vSW5UcmFuc2Zvcm1GdW5jdGlvbi5qYXZh) | `55.45% <55.45%> (ø)` | |
| [...e/pinot/common/function/TransformFunctionType.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vVHJhbnNmb3JtRnVuY3Rpb25UeXBlLmphdmE=) | `100.00% <100.00%> (ø)` | |
| [...r/transform/function/TransformFunctionFactory.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vVHJhbnNmb3JtRnVuY3Rpb25GYWN0b3J5LmphdmE=) | `81.18% <100.00%> (-2.82%)` | :arrow_down: |
| [...a/org/apache/pinot/common/metrics/MinionMeter.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vbWV0cmljcy9NaW5pb25NZXRlci5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...g/apache/pinot/common/metrics/ControllerMeter.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vbWV0cmljcy9Db250cm9sbGVyTWV0ZXIuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [.../apache/pinot/common/metrics/BrokerQueryPhase.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vbWV0cmljcy9Ccm9rZXJRdWVyeVBoYXNlLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [.../apache/pinot/common/metrics/MinionQueryPhase.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vbWV0cmljcy9NaW5pb25RdWVyeVBoYXNlLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...he/pinot/common/messages/SegmentReloadMessage.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vbWVzc2FnZXMvU2VnbWVudFJlbG9hZE1lc3NhZ2UuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...pinot/core/data/manager/realtime/TimerService.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9kYXRhL21hbmFnZXIvcmVhbHRpbWUvVGltZXJTZXJ2aWNlLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...not/common/exception/HttpErrorStatusException.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZXhjZXB0aW9uL0h0dHBFcnJvclN0YXR1c0V4Y2VwdGlvbi5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| ... and [599 more](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [19f66fd...4942bc7](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#issuecomment-938200237
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7542](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (4942bc7) into [master](https://codecov.io/gh/apache/pinot/commit/7be21c80b352a02c58807b3c95d535f5fae394cc?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (7be21c8) will **increase** coverage by `0.05%`.
> The diff coverage is `55.86%`.
[](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7542 +/- ##
============================================
+ Coverage 72.54% 72.59% +0.05%
- Complexity 3405 3414 +9
============================================
Files 1523 1525 +2
Lines 75653 75912 +259
Branches 11032 11106 +74
============================================
+ Hits 54880 55110 +230
- Misses 17078 17105 +27
- Partials 3695 3697 +2
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `30.67% <0.93%> (+0.17%)` | :arrow_up: |
| integration2 | `29.08% <0.93%> (+0.08%)` | :arrow_up: |
| unittests1 | `69.82% <55.86%> (+0.01%)` | :arrow_up: |
| unittests2 | `15.26% <0.00%> (-0.01%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...erator/transform/function/InTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vSW5UcmFuc2Zvcm1GdW5jdGlvbi5qYXZh) | `55.45% <55.45%> (ø)` | |
| [...e/pinot/common/function/TransformFunctionType.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vVHJhbnNmb3JtRnVuY3Rpb25UeXBlLmphdmE=) | `100.00% <100.00%> (ø)` | |
| [...r/transform/function/TransformFunctionFactory.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vVHJhbnNmb3JtRnVuY3Rpb25GYWN0b3J5LmphdmE=) | `84.15% <100.00%> (+0.15%)` | :arrow_up: |
| [.../RealtimeToOfflineSegmentsTaskExecutorFactory.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvcmVhbHRpbWV0b29mZmxpbmVzZWdtZW50cy9SZWFsdGltZVRvT2ZmbGluZVNlZ21lbnRzVGFza0V4ZWN1dG9yRmFjdG9yeS5qYXZh) | `75.00% <0.00%> (-25.00%)` | :arrow_down: |
| [...rawindex/ConvertToRawIndexTaskExecutorFactory.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvY29udmVydHRvcmF3aW5kZXgvQ29udmVydFRvUmF3SW5kZXhUYXNrRXhlY3V0b3JGYWN0b3J5LmphdmE=) | `80.00% <0.00%> (-20.00%)` | :arrow_down: |
| [...ks/mergerollup/MergeRollupTaskExecutorFactory.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvbWVyZ2Vyb2xsdXAvTWVyZ2VSb2xsdXBUYXNrRXhlY3V0b3JGYWN0b3J5LmphdmE=) | `83.33% <0.00%> (-16.67%)` | :arrow_down: |
| [...n/minion/tasks/purge/PurgeTaskExecutorFactory.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvcHVyZ2UvUHVyZ2VUYXNrRXhlY3V0b3JGYWN0b3J5LmphdmE=) | `60.00% <0.00%> (-15.00%)` | :arrow_down: |
| [...h/SegmentGenerationAndPushTaskExecutorFactory.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3Mvc2VnbWVudGdlbmVyYXRpb25hbmRwdXNoL1NlZ21lbnRHZW5lcmF0aW9uQW5kUHVzaFRhc2tFeGVjdXRvckZhY3RvcnkuamF2YQ==) | `60.00% <0.00%> (-15.00%)` | :arrow_down: |
| [...r/helix/SegmentOnlineOfflineStateModelFactory.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zZXJ2ZXIvc3RhcnRlci9oZWxpeC9TZWdtZW50T25saW5lT2ZmbGluZVN0YXRlTW9kZWxGYWN0b3J5LmphdmE=) | `58.49% <0.00%> (-4.72%)` | :arrow_down: |
| [...n/src/main/java/org/apache/pinot/common/Utils.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vVXRpbHMuamF2YQ==) | `60.46% <0.00%> (-2.33%)` | :arrow_down: |
| ... and [37 more](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [19f66fd...4942bc7](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#issuecomment-938200237
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7542](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (4ef9627) into [master](https://codecov.io/gh/apache/pinot/commit/7be21c80b352a02c58807b3c95d535f5fae394cc?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (7be21c8) will **decrease** coverage by `6.63%`.
> The diff coverage is `66.66%`.
[](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7542 +/- ##
============================================
- Coverage 72.54% 65.90% -6.64%
- Complexity 3405 3412 +7
============================================
Files 1523 1478 -45
Lines 75653 73900 -1753
Branches 11032 10866 -166
============================================
- Hits 54880 48706 -6174
- Misses 17078 21734 +4656
+ Partials 3695 3460 -235
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| integration2 | `?` | |
| unittests1 | `69.86% <66.66%> (+0.05%)` | :arrow_up: |
| unittests2 | `15.28% <0.00%> (+0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...erator/transform/function/InTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vSW5UcmFuc2Zvcm1GdW5jdGlvbi5qYXZh) | `65.67% <65.67%> (ø)` | |
| [...e/pinot/common/function/TransformFunctionType.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vVHJhbnNmb3JtRnVuY3Rpb25UeXBlLmphdmE=) | `100.00% <100.00%> (ø)` | |
| [...r/transform/function/TransformFunctionFactory.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vVHJhbnNmb3JtRnVuY3Rpb25GYWN0b3J5LmphdmE=) | `81.18% <100.00%> (-2.82%)` | :arrow_down: |
| [...a/org/apache/pinot/common/metrics/MinionMeter.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vbWV0cmljcy9NaW5pb25NZXRlci5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...g/apache/pinot/common/metrics/ControllerMeter.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vbWV0cmljcy9Db250cm9sbGVyTWV0ZXIuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [.../apache/pinot/common/metrics/BrokerQueryPhase.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vbWV0cmljcy9Ccm9rZXJRdWVyeVBoYXNlLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [.../apache/pinot/common/metrics/MinionQueryPhase.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vbWV0cmljcy9NaW5pb25RdWVyeVBoYXNlLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...he/pinot/common/messages/SegmentReloadMessage.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vbWVzc2FnZXMvU2VnbWVudFJlbG9hZE1lc3NhZ2UuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...pinot/core/data/manager/realtime/TimerService.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9kYXRhL21hbmFnZXIvcmVhbHRpbWUvVGltZXJTZXJ2aWNlLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...pinot/minion/exception/TaskCancelledException.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtbWluaW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9taW5pb24vZXhjZXB0aW9uL1Rhc2tDYW5jZWxsZWRFeGNlcHRpb24uamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| ... and [356 more](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [19f66fd...4ef9627](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#issuecomment-938200237
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7542](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (c7fe9a0) into [master](https://codecov.io/gh/apache/pinot/commit/7be21c80b352a02c58807b3c95d535f5fae394cc?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (7be21c8) will **decrease** coverage by `6.71%`.
> The diff coverage is `53.84%`.
[](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7542 +/- ##
============================================
- Coverage 72.54% 65.83% -6.72%
- Complexity 3405 3410 +5
============================================
Files 1523 1479 -44
Lines 75653 73952 -1701
Branches 11032 10883 -149
============================================
- Hits 54880 48683 -6197
- Misses 17078 21804 +4726
+ Partials 3695 3465 -230
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| integration2 | `?` | |
| unittests1 | `69.82% <53.84%> (+0.01%)` | :arrow_up: |
| unittests2 | `15.23% <0.00%> (-0.03%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...erator/transform/function/InTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vSW5UcmFuc2Zvcm1GdW5jdGlvbi5qYXZh) | `52.94% <52.94%> (ø)` | |
| [...e/pinot/common/function/TransformFunctionType.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vVHJhbnNmb3JtRnVuY3Rpb25UeXBlLmphdmE=) | `100.00% <100.00%> (ø)` | |
| [...r/transform/function/TransformFunctionFactory.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vVHJhbnNmb3JtRnVuY3Rpb25GYWN0b3J5LmphdmE=) | `81.18% <100.00%> (-2.82%)` | :arrow_down: |
| [...a/org/apache/pinot/common/metrics/MinionMeter.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vbWV0cmljcy9NaW5pb25NZXRlci5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...g/apache/pinot/common/metrics/ControllerMeter.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vbWV0cmljcy9Db250cm9sbGVyTWV0ZXIuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [.../apache/pinot/common/metrics/BrokerQueryPhase.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vbWV0cmljcy9Ccm9rZXJRdWVyeVBoYXNlLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [.../apache/pinot/common/metrics/MinionQueryPhase.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vbWV0cmljcy9NaW5pb25RdWVyeVBoYXNlLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...he/pinot/common/messages/SegmentReloadMessage.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vbWVzc2FnZXMvU2VnbWVudFJlbG9hZE1lc3NhZ2UuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...pinot/core/data/manager/realtime/TimerService.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9kYXRhL21hbmFnZXIvcmVhbHRpbWUvVGltZXJTZXJ2aWNlLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...pinot/minion/exception/TaskCancelledException.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtbWluaW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9taW5pb24vZXhjZXB0aW9uL1Rhc2tDYW5jZWxsZWRFeGNlcHRpb24uamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| ... and [357 more](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [19f66fd...c7fe9a0](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] yupeng9 commented on a change in pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
yupeng9 commented on a change in pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#discussion_r726249550
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/InTransformFunction.java
##########
@@ -81,77 +95,112 @@ public TransformResultMetadata getResultMetadata() {
switch (storedType) {
case INT:
int[] intValues = _transformFunction.transformToIntValuesSV(projectionBlock);
- int[][] inIntValues = new int[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inIntValues[i] = _valueTransformFunctions[i].transformToIntValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inIntValues.length; j++) {
- _results[i] = inIntValues[j][i] == intValues[i] ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
+ Set<Integer> inIntValues = new HashSet<>();
+ for (String inValue : _stringValueSet) {
+ inIntValues.add(Integer.parseInt(inValue));
+ }
+ for (int i = 0; i < length; i++) {
+ _results[i] = inIntValues.contains(intValues[i]) ? 1 : 0;
+ }
+ } else {
+ int[][] inIntValues = new int[_valueTransformFunctions.length][];
+ for (int i = 0; i < _valueTransformFunctions.length; i++) {
+ inIntValues[i] = _valueTransformFunctions[i].transformToIntValuesSV(projectionBlock);
+ }
+ for (int i = 0; i < length; i++) {
+ for (int j = 0; j < inIntValues.length; j++) {
+ _results[i] = inIntValues[j][i] == intValues[i] ? 1 : _results[i];
+ }
}
}
break;
case LONG:
long[] longValues = _transformFunction.transformToLongValuesSV(projectionBlock);
- long[][] inLongValues = new long[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inLongValues[i] = _valueTransformFunctions[i].transformToLongValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inLongValues.length; j++) {
- _results[i] = inLongValues[j][i] == longValues[i] ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
+ Set<Long> inLongValues = new HashSet<>();
+ for (String inValue : _stringValueSet) {
+ inLongValues.add(Long.parseLong(inValue));
+ }
+ for (int i = 0; i < length; i++) {
+ _results[i] = inLongValues.contains(longValues[i]) ? 1 : 0;
+ }
+ } else {
+ long[][] inLongValues = new long[_valueTransformFunctions.length][];
+ for (int i = 0; i < _valueTransformFunctions.length; i++) {
+ inLongValues[i] = _valueTransformFunctions[i].transformToLongValuesSV(projectionBlock);
+ }
+ for (int i = 0; i < length; i++) {
+ for (int j = 0; j < inLongValues.length; j++) {
+ _results[i] = inLongValues[j][i] == longValues[i] ? 1 : _results[i];
+ }
}
}
break;
case FLOAT:
float[] floatValues = _transformFunction.transformToFloatValuesSV(projectionBlock);
- float[][] inFloatValues = new float[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inFloatValues[i] = _valueTransformFunctions[i].transformToFloatValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inFloatValues.length; j++) {
- _results[i] = Float.compare(inFloatValues[j][i], floatValues[i]) == 0 ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
Review comment:
Agreed. This is a general issue for Pinot on float/double comparison. A configurable epsilon of the difference would be good.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] kishoreg commented on pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
kishoreg commented on pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#issuecomment-938200566
can you add more description on what the in function is supposed to return? I had to look at the code to see that it's 1 or 0.
Also, what are the scenarios/usecases where we need this?
Is this standard SQL? what is the equivalent of this in MySQL, Presto etc.
I was wondering if IN would clash with IN in the filter clause. Should we come up with a better name?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter commented on pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#issuecomment-938200237
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7542](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (4ef9627) into [master](https://codecov.io/gh/apache/pinot/commit/7be21c80b352a02c58807b3c95d535f5fae394cc?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (7be21c8) will **decrease** coverage by `2.67%`.
> The diff coverage is `66.66%`.
[](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7542 +/- ##
============================================
- Coverage 72.54% 69.86% -2.68%
+ Complexity 3405 3332 -73
============================================
Files 1523 1130 -393
Lines 75653 53546 -22107
Branches 11032 8087 -2945
============================================
- Hits 54880 37412 -17468
+ Misses 17078 13489 -3589
+ Partials 3695 2645 -1050
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| integration2 | `?` | |
| unittests1 | `69.86% <66.66%> (+0.05%)` | :arrow_up: |
| unittests2 | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...erator/transform/function/InTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vSW5UcmFuc2Zvcm1GdW5jdGlvbi5qYXZh) | `65.67% <65.67%> (ø)` | |
| [...e/pinot/common/function/TransformFunctionType.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vVHJhbnNmb3JtRnVuY3Rpb25UeXBlLmphdmE=) | `100.00% <100.00%> (ø)` | |
| [...r/transform/function/TransformFunctionFactory.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vVHJhbnNmb3JtRnVuY3Rpb25GYWN0b3J5LmphdmE=) | `81.18% <100.00%> (-2.82%)` | :arrow_down: |
| [...a/org/apache/pinot/common/metrics/MinionMeter.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vbWV0cmljcy9NaW5pb25NZXRlci5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...g/apache/pinot/common/metrics/ControllerMeter.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vbWV0cmljcy9Db250cm9sbGVyTWV0ZXIuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [.../apache/pinot/common/metrics/BrokerQueryPhase.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vbWV0cmljcy9Ccm9rZXJRdWVyeVBoYXNlLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [.../apache/pinot/common/metrics/MinionQueryPhase.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vbWV0cmljcy9NaW5pb25RdWVyeVBoYXNlLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...he/pinot/common/messages/SegmentReloadMessage.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vbWVzc2FnZXMvU2VnbWVudFJlbG9hZE1lc3NhZ2UuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...pinot/core/data/manager/realtime/TimerService.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9kYXRhL21hbmFnZXIvcmVhbHRpbWUvVGltZXJTZXJ2aWNlLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...not/common/exception/HttpErrorStatusException.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZXhjZXB0aW9uL0h0dHBFcnJvclN0YXR1c0V4Y2VwdGlvbi5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| ... and [598 more](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [19f66fd...4ef9627](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] yupeng9 commented on a change in pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
yupeng9 commented on a change in pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#discussion_r725092409
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/InTransformFunction.java
##########
@@ -0,0 +1,161 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import com.google.common.base.Preconditions;
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.common.function.TransformFunctionType;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.core.plan.DocIdSetPlanNode;
+import org.apache.pinot.segment.spi.datasource.DataSource;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.utils.ByteArray;
+
+
+/**
+ * The IN transform function takes more than 1 arguments:
+ * <ul>
+ * <li>Expression: a single-value expression</li>
+ * <li>values: a set of literal strings</li>
+ * </ul>
+ * <p>For each docId, the function returns {@code 1} if the set of values contains the value of the expression,
+ * {code 0} if not.
+ * <p>E.g. {@code SELECT col IN ('a','b','c') FROM myTable)}
+ */
+public class InTransformFunction extends BaseTransformFunction {
+ private TransformFunction _transformFunction;
+ private TransformFunction[] _valueTransformFunctions;
+ private int[] _results;
+
+ @Override
+ public String getName() {
+ return TransformFunctionType.IN.getName();
+ }
+
+ @Override
+ public void init(List<TransformFunction> arguments, Map<String, DataSource> dataSourceMap) {
+ Preconditions.checkArgument(arguments.size() >= 2,
+ "at least 2 arguments are required for IN transform function: expression, values");
+ Preconditions.checkArgument(arguments.get(0).getResultMetadata().isSingleValue(),
+ "First argument for IN transform function must be a single-value expression");
+ _transformFunction = arguments.get(0);
+ _valueTransformFunctions = new TransformFunction[arguments.size() - 1];
Review comment:
Understood the perf issue. But let me see if it's possible to make it an optimization when all values are literals.
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/InTransformFunction.java
##########
@@ -0,0 +1,161 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import com.google.common.base.Preconditions;
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.common.function.TransformFunctionType;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.core.plan.DocIdSetPlanNode;
+import org.apache.pinot.segment.spi.datasource.DataSource;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.utils.ByteArray;
+
+
+/**
+ * The IN transform function takes more than 1 arguments:
+ * <ul>
+ * <li>Expression: a single-value expression</li>
+ * <li>values: a set of literal strings</li>
+ * </ul>
+ * <p>For each docId, the function returns {@code 1} if the set of values contains the value of the expression,
+ * {code 0} if not.
+ * <p>E.g. {@code SELECT col IN ('a','b','c') FROM myTable)}
+ */
+public class InTransformFunction extends BaseTransformFunction {
+ private TransformFunction _transformFunction;
+ private TransformFunction[] _valueTransformFunctions;
+ private int[] _results;
+
+ @Override
+ public String getName() {
+ return TransformFunctionType.IN.getName();
+ }
+
+ @Override
+ public void init(List<TransformFunction> arguments, Map<String, DataSource> dataSourceMap) {
+ Preconditions.checkArgument(arguments.size() >= 2,
+ "at least 2 arguments are required for IN transform function: expression, values");
+ Preconditions.checkArgument(arguments.get(0).getResultMetadata().isSingleValue(),
+ "First argument for IN transform function must be a single-value expression");
Review comment:
can you give an example of the MV case in this `IN` function?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter commented on pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#issuecomment-938200237
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7542](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (4ef9627) into [master](https://codecov.io/gh/apache/pinot/commit/7be21c80b352a02c58807b3c95d535f5fae394cc?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (7be21c8) will **decrease** coverage by `2.67%`.
> The diff coverage is `66.66%`.
[](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7542 +/- ##
============================================
- Coverage 72.54% 69.86% -2.68%
+ Complexity 3405 3332 -73
============================================
Files 1523 1130 -393
Lines 75653 53546 -22107
Branches 11032 8087 -2945
============================================
- Hits 54880 37412 -17468
+ Misses 17078 13489 -3589
+ Partials 3695 2645 -1050
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| integration2 | `?` | |
| unittests1 | `69.86% <66.66%> (+0.05%)` | :arrow_up: |
| unittests2 | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...erator/transform/function/InTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vSW5UcmFuc2Zvcm1GdW5jdGlvbi5qYXZh) | `65.67% <65.67%> (ø)` | |
| [...e/pinot/common/function/TransformFunctionType.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vVHJhbnNmb3JtRnVuY3Rpb25UeXBlLmphdmE=) | `100.00% <100.00%> (ø)` | |
| [...r/transform/function/TransformFunctionFactory.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vVHJhbnNmb3JtRnVuY3Rpb25GYWN0b3J5LmphdmE=) | `81.18% <100.00%> (-2.82%)` | :arrow_down: |
| [...a/org/apache/pinot/common/metrics/MinionMeter.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vbWV0cmljcy9NaW5pb25NZXRlci5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...g/apache/pinot/common/metrics/ControllerMeter.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vbWV0cmljcy9Db250cm9sbGVyTWV0ZXIuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [.../apache/pinot/common/metrics/BrokerQueryPhase.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vbWV0cmljcy9Ccm9rZXJRdWVyeVBoYXNlLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [.../apache/pinot/common/metrics/MinionQueryPhase.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vbWV0cmljcy9NaW5pb25RdWVyeVBoYXNlLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...he/pinot/common/messages/SegmentReloadMessage.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vbWVzc2FnZXMvU2VnbWVudFJlbG9hZE1lc3NhZ2UuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...pinot/core/data/manager/realtime/TimerService.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9kYXRhL21hbmFnZXIvcmVhbHRpbWUvVGltZXJTZXJ2aWNlLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...not/common/exception/HttpErrorStatusException.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZXhjZXB0aW9uL0h0dHBFcnJvclN0YXR1c0V4Y2VwdGlvbi5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| ... and [598 more](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [19f66fd...4ef9627](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] richardstartin commented on a change in pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#discussion_r725191626
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/InTransformFunction.java
##########
@@ -81,77 +95,112 @@ public TransformResultMetadata getResultMetadata() {
switch (storedType) {
case INT:
int[] intValues = _transformFunction.transformToIntValuesSV(projectionBlock);
- int[][] inIntValues = new int[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inIntValues[i] = _valueTransformFunctions[i].transformToIntValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inIntValues.length; j++) {
- _results[i] = inIntValues[j][i] == intValues[i] ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
+ Set<Integer> inIntValues = new HashSet<>();
+ for (String inValue : _stringValueSet) {
+ inIntValues.add(Integer.parseInt(inValue));
+ }
+ for (int i = 0; i < length; i++) {
+ _results[i] = inIntValues.contains(intValues[i]) ? 1 : 0;
+ }
+ } else {
+ int[][] inIntValues = new int[_valueTransformFunctions.length][];
+ for (int i = 0; i < _valueTransformFunctions.length; i++) {
+ inIntValues[i] = _valueTransformFunctions[i].transformToIntValuesSV(projectionBlock);
+ }
+ for (int i = 0; i < length; i++) {
+ for (int j = 0; j < inIntValues.length; j++) {
+ _results[i] = inIntValues[j][i] == intValues[i] ? 1 : _results[i];
+ }
}
}
break;
case LONG:
long[] longValues = _transformFunction.transformToLongValuesSV(projectionBlock);
- long[][] inLongValues = new long[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inLongValues[i] = _valueTransformFunctions[i].transformToLongValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inLongValues.length; j++) {
- _results[i] = inLongValues[j][i] == longValues[i] ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
+ Set<Long> inLongValues = new HashSet<>();
+ for (String inValue : _stringValueSet) {
+ inLongValues.add(Long.parseLong(inValue));
+ }
+ for (int i = 0; i < length; i++) {
+ _results[i] = inLongValues.contains(longValues[i]) ? 1 : 0;
+ }
+ } else {
+ long[][] inLongValues = new long[_valueTransformFunctions.length][];
+ for (int i = 0; i < _valueTransformFunctions.length; i++) {
+ inLongValues[i] = _valueTransformFunctions[i].transformToLongValuesSV(projectionBlock);
+ }
+ for (int i = 0; i < length; i++) {
+ for (int j = 0; j < inLongValues.length; j++) {
+ _results[i] = inLongValues[j][i] == longValues[i] ? 1 : _results[i];
+ }
}
}
break;
case FLOAT:
float[] floatValues = _transformFunction.transformToFloatValuesSV(projectionBlock);
- float[][] inFloatValues = new float[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inFloatValues[i] = _valueTransformFunctions[i].transformToFloatValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inFloatValues.length; j++) {
- _results[i] = Float.compare(inFloatValues[j][i], floatValues[i]) == 0 ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
+ Set<Float> inFloatValues = new HashSet<>();
+ for (String inValue : _stringValueSet) {
+ inFloatValues.add(Float.parseFloat(inValue));
+ }
+ for (int i = 0; i < length; i++) {
+ _results[i] = inFloatValues.contains(floatValues[i]) ? 1 : 0;
+ }
+ } else {
+ float[][] inFloatValues = new float[_valueTransformFunctions.length][];
+ for (int i = 0; i < _valueTransformFunctions.length; i++) {
+ inFloatValues[i] = _valueTransformFunctions[i].transformToFloatValuesSV(projectionBlock);
+ }
+ for (int i = 0; i < length; i++) {
+ for (int j = 0; j < inFloatValues.length; j++) {
+ _results[i] = Float.compare(inFloatValues[j][i], floatValues[i]) == 0 ? 1 : _results[i];
+ }
}
}
break;
case DOUBLE:
double[] doubleValues = _transformFunction.transformToDoubleValuesSV(projectionBlock);
- double[][] inDoubleValues = new double[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inDoubleValues[i] = _valueTransformFunctions[i].transformToDoubleValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inDoubleValues.length; j++) {
- _results[i] = Double.compare(inDoubleValues[j][i], doubleValues[i]) == 0 ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
+ Set<Double> inDoubleValues = new HashSet<>();
+ for (String inValue : _stringValueSet) {
+ inDoubleValues.add(Double.parseDouble(inValue));
+ }
+ for (int i = 0; i < length; i++) {
+ _results[i] = inDoubleValues.contains(doubleValues[i]) ? 1 : 0;
+ }
+ } else {
+ double[][] inDoubleValues = new double[_valueTransformFunctions.length][];
+ for (int i = 0; i < _valueTransformFunctions.length; i++) {
+ inDoubleValues[i] = _valueTransformFunctions[i].transformToDoubleValuesSV(projectionBlock);
+ }
+ for (int i = 0; i < length; i++) {
+ for (int j = 0; j < inDoubleValues.length; j++) {
+ _results[i] = Double.compare(inDoubleValues[j][i], doubleValues[i]) == 0 ? 1 : _results[i];
+ }
}
}
break;
case STRING:
String[] stringValues = _transformFunction.transformToStringValuesSV(projectionBlock);
- String[][] inStringValues = new String[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inStringValues[i] = _valueTransformFunctions[i].transformToStringValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inStringValues.length; j++) {
- _results[i] = inStringValues[j][i].equals(stringValues[i]) ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
+ for (int i = 0; i < length; i++) {
+ _results[i] = _stringValueSet.contains(stringValues[i]) ? 1 : 0;
}
- }
- break;
- case BYTES:
- byte[][] bytesValues = _transformFunction.transformToBytesValuesSV(projectionBlock);
- byte[][][] inBytesValues = new byte[_valueTransformFunctions.length][][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inBytesValues[i] = _valueTransformFunctions[i].transformToBytesValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inBytesValues.length; j++) {
- _results[i] =
- new ByteArray(inBytesValues[j][i]).compareTo(new ByteArray(bytesValues[i])) == 0 ? 1 : _results[i];
+ } else {
+ String[][] inStringValues = new String[_valueTransformFunctions.length][];
+ for (int i = 0; i < _valueTransformFunctions.length; i++) {
+ inStringValues[i] = _valueTransformFunctions[i].transformToStringValuesSV(projectionBlock);
+ }
+ for (int i = 0; i < length; i++) {
+ for (int j = 0; j < inStringValues.length; j++) {
+ _results[i] = inStringValues[j][i].equals(stringValues[i]) ? 1 : _results[i];
+ }
}
}
break;
+ case BYTES:
+ throw new UnsupportedOperationException();
Review comment:
Sorry I'm new to pinot, it doesn't seem that there is a base64 function to specify binary literals. So I wouldn't suggest adding one now to implement this for bytes, I can just imagine wanting to do this for e.g. security use cases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] yupeng9 commented on pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
yupeng9 commented on pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#issuecomment-938203773
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] richardstartin commented on a change in pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#discussion_r725190126
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/InTransformFunction.java
##########
@@ -81,77 +95,112 @@ public TransformResultMetadata getResultMetadata() {
switch (storedType) {
case INT:
int[] intValues = _transformFunction.transformToIntValuesSV(projectionBlock);
- int[][] inIntValues = new int[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inIntValues[i] = _valueTransformFunctions[i].transformToIntValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inIntValues.length; j++) {
- _results[i] = inIntValues[j][i] == intValues[i] ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
+ Set<Integer> inIntValues = new HashSet<>();
+ for (String inValue : _stringValueSet) {
+ inIntValues.add(Integer.parseInt(inValue));
+ }
+ for (int i = 0; i < length; i++) {
+ _results[i] = inIntValues.contains(intValues[i]) ? 1 : 0;
+ }
+ } else {
+ int[][] inIntValues = new int[_valueTransformFunctions.length][];
+ for (int i = 0; i < _valueTransformFunctions.length; i++) {
+ inIntValues[i] = _valueTransformFunctions[i].transformToIntValuesSV(projectionBlock);
+ }
+ for (int i = 0; i < length; i++) {
+ for (int j = 0; j < inIntValues.length; j++) {
+ _results[i] = inIntValues[j][i] == intValues[i] ? 1 : _results[i];
+ }
}
}
break;
case LONG:
long[] longValues = _transformFunction.transformToLongValuesSV(projectionBlock);
- long[][] inLongValues = new long[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inLongValues[i] = _valueTransformFunctions[i].transformToLongValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inLongValues.length; j++) {
- _results[i] = inLongValues[j][i] == longValues[i] ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
+ Set<Long> inLongValues = new HashSet<>();
+ for (String inValue : _stringValueSet) {
+ inLongValues.add(Long.parseLong(inValue));
+ }
+ for (int i = 0; i < length; i++) {
+ _results[i] = inLongValues.contains(longValues[i]) ? 1 : 0;
+ }
+ } else {
+ long[][] inLongValues = new long[_valueTransformFunctions.length][];
+ for (int i = 0; i < _valueTransformFunctions.length; i++) {
+ inLongValues[i] = _valueTransformFunctions[i].transformToLongValuesSV(projectionBlock);
+ }
+ for (int i = 0; i < length; i++) {
+ for (int j = 0; j < inLongValues.length; j++) {
+ _results[i] = inLongValues[j][i] == longValues[i] ? 1 : _results[i];
+ }
}
}
break;
case FLOAT:
float[] floatValues = _transformFunction.transformToFloatValuesSV(projectionBlock);
- float[][] inFloatValues = new float[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inFloatValues[i] = _valueTransformFunctions[i].transformToFloatValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inFloatValues.length; j++) {
- _results[i] = Float.compare(inFloatValues[j][i], floatValues[i]) == 0 ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
Review comment:
I suppose I'm asking why would someone want to do that?
If it's a valid use case, it could be implemented by sorting the values in the in clause and doing a check within an epsilon of say 1e-7.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] richardstartin commented on a change in pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#discussion_r725168560
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/InTransformFunction.java
##########
@@ -81,77 +95,112 @@ public TransformResultMetadata getResultMetadata() {
switch (storedType) {
case INT:
int[] intValues = _transformFunction.transformToIntValuesSV(projectionBlock);
- int[][] inIntValues = new int[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inIntValues[i] = _valueTransformFunctions[i].transformToIntValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inIntValues.length; j++) {
- _results[i] = inIntValues[j][i] == intValues[i] ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
+ Set<Integer> inIntValues = new HashSet<>();
+ for (String inValue : _stringValueSet) {
+ inIntValues.add(Integer.parseInt(inValue));
+ }
+ for (int i = 0; i < length; i++) {
+ _results[i] = inIntValues.contains(intValues[i]) ? 1 : 0;
+ }
+ } else {
+ int[][] inIntValues = new int[_valueTransformFunctions.length][];
+ for (int i = 0; i < _valueTransformFunctions.length; i++) {
+ inIntValues[i] = _valueTransformFunctions[i].transformToIntValuesSV(projectionBlock);
+ }
+ for (int i = 0; i < length; i++) {
+ for (int j = 0; j < inIntValues.length; j++) {
+ _results[i] = inIntValues[j][i] == intValues[i] ? 1 : _results[i];
+ }
}
}
break;
case LONG:
long[] longValues = _transformFunction.transformToLongValuesSV(projectionBlock);
- long[][] inLongValues = new long[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inLongValues[i] = _valueTransformFunctions[i].transformToLongValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inLongValues.length; j++) {
- _results[i] = inLongValues[j][i] == longValues[i] ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
+ Set<Long> inLongValues = new HashSet<>();
+ for (String inValue : _stringValueSet) {
+ inLongValues.add(Long.parseLong(inValue));
+ }
+ for (int i = 0; i < length; i++) {
+ _results[i] = inLongValues.contains(longValues[i]) ? 1 : 0;
+ }
+ } else {
+ long[][] inLongValues = new long[_valueTransformFunctions.length][];
+ for (int i = 0; i < _valueTransformFunctions.length; i++) {
+ inLongValues[i] = _valueTransformFunctions[i].transformToLongValuesSV(projectionBlock);
+ }
+ for (int i = 0; i < length; i++) {
+ for (int j = 0; j < inLongValues.length; j++) {
+ _results[i] = inLongValues[j][i] == longValues[i] ? 1 : _results[i];
+ }
}
}
break;
case FLOAT:
float[] floatValues = _transformFunction.transformToFloatValuesSV(projectionBlock);
- float[][] inFloatValues = new float[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inFloatValues[i] = _valueTransformFunctions[i].transformToFloatValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inFloatValues.length; j++) {
- _results[i] = Float.compare(inFloatValues[j][i], floatValues[i]) == 0 ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
Review comment:
This test fails
```
@Test
public void testFloatSet() {
Set<Float> set = new HashSet<>();
set.add(1F);
assertTrue(set.contains(0.01F + 0.9F));
}
```
Which is to say that floating point equality is pernicious, making set membership an equally pernicious relation.
Not only is it difficult to implement, I wonder if anyone would ever write the query with floats?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] yupeng9 edited a comment on pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
yupeng9 edited a comment on pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#issuecomment-939512723
LGTM. Thanks for the code improvement! @Jackie-Jiang
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#issuecomment-938200237
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7542](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (d38c57a) into [master](https://codecov.io/gh/apache/pinot/commit/7be21c80b352a02c58807b3c95d535f5fae394cc?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (7be21c8) will **decrease** coverage by `2.76%`.
> The diff coverage is `50.79%`.
> :exclamation: Current head d38c57a differs from pull request most recent head 3c3ba00. Consider uploading reports for the commit 3c3ba00 to get more accurate results
[](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7542 +/- ##
============================================
- Coverage 72.54% 69.77% -2.77%
+ Complexity 3405 3334 -71
============================================
Files 1523 1130 -393
Lines 75653 53690 -21963
Branches 11032 8128 -2904
============================================
- Hits 54880 37464 -17416
+ Misses 17078 13576 -3502
+ Partials 3695 2650 -1045
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| integration2 | `?` | |
| unittests1 | `69.77% <50.79%> (-0.04%)` | :arrow_down: |
| unittests2 | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [.../org/apache/pinot/core/common/MinionConstants.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9jb21tb24vTWluaW9uQ29uc3RhbnRzLmphdmE=) | `0.00% <ø> (ø)` | |
| [...erator/transform/function/InTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vSW5UcmFuc2Zvcm1GdW5jdGlvbi5qYXZh) | `41.70% <41.70%> (ø)` | |
| [...he/pinot/segment/local/utils/TableConfigUtils.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc2VnbWVudC1sb2NhbC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VnbWVudC9sb2NhbC91dGlscy9UYWJsZUNvbmZpZ1V0aWxzLmphdmE=) | `69.87% <96.55%> (+3.74%)` | :arrow_up: |
| [...e/pinot/common/function/TransformFunctionType.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vVHJhbnNmb3JtRnVuY3Rpb25UeXBlLmphdmE=) | `100.00% <100.00%> (ø)` | |
| [...r/transform/function/TransformFunctionFactory.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vVHJhbnNmb3JtRnVuY3Rpb25GYWN0b3J5LmphdmE=) | `81.18% <100.00%> (-2.82%)` | :arrow_down: |
| [...ry/optimizer/statement/JsonStatementOptimizer.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9xdWVyeS9vcHRpbWl6ZXIvc3RhdGVtZW50L0pzb25TdGF0ZW1lbnRPcHRpbWl6ZXIuamF2YQ==) | `77.63% <100.00%> (+1.16%)` | :arrow_up: |
| [...a/org/apache/pinot/common/metrics/MinionMeter.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vbWV0cmljcy9NaW5pb25NZXRlci5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...g/apache/pinot/common/metrics/ControllerMeter.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vbWV0cmljcy9Db250cm9sbGVyTWV0ZXIuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [.../apache/pinot/common/metrics/BrokerQueryPhase.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vbWV0cmljcy9Ccm9rZXJRdWVyeVBoYXNlLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [.../apache/pinot/common/metrics/MinionQueryPhase.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vbWV0cmljcy9NaW5pb25RdWVyeVBoYXNlLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| ... and [591 more](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [19f66fd...3c3ba00](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] yupeng9 commented on a change in pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
yupeng9 commented on a change in pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#discussion_r724612297
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/InTransformFunction.java
##########
@@ -0,0 +1,161 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import com.google.common.base.Preconditions;
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.common.function.TransformFunctionType;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.core.plan.DocIdSetPlanNode;
+import org.apache.pinot.segment.spi.datasource.DataSource;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.utils.ByteArray;
+
+
+/**
+ * The IN transform function takes more than 1 arguments:
+ * <ul>
+ * <li>Expression: a single-value expression</li>
+ * <li>values: a set of literal strings</li>
+ * </ul>
+ * <p>For each docId, the function returns {@code 1} if the set of values contains the value of the expression,
+ * {code 0} if not.
+ * <p>E.g. {@code SELECT col IN ('a','b','c') FROM myTable)}
+ */
+public class InTransformFunction extends BaseTransformFunction {
+ private TransformFunction _transformFunction;
+ private TransformFunction[] _valueTransformFunctions;
+ private int[] _results;
+
+ @Override
+ public String getName() {
+ return TransformFunctionType.IN.getName();
+ }
+
+ @Override
+ public void init(List<TransformFunction> arguments, Map<String, DataSource> dataSourceMap) {
+ Preconditions.checkArgument(arguments.size() >= 2,
+ "at least 2 arguments are required for IN transform function: expression, values");
+ Preconditions.checkArgument(arguments.get(0).getResultMetadata().isSingleValue(),
+ "First argument for IN transform function must be a single-value expression");
+ _transformFunction = arguments.get(0);
+ _valueTransformFunctions = new TransformFunction[arguments.size() - 1];
Review comment:
is this true? would that support func like `a IN (3+3, 5+5)` or `b IN (concat(c,d),concate(e,f))`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] richardstartin commented on a change in pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#discussion_r725169730
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/InTransformFunction.java
##########
@@ -81,77 +95,112 @@ public TransformResultMetadata getResultMetadata() {
switch (storedType) {
case INT:
int[] intValues = _transformFunction.transformToIntValuesSV(projectionBlock);
- int[][] inIntValues = new int[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inIntValues[i] = _valueTransformFunctions[i].transformToIntValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inIntValues.length; j++) {
- _results[i] = inIntValues[j][i] == intValues[i] ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
+ Set<Integer> inIntValues = new HashSet<>();
Review comment:
Could this use an `IntOpenHashSet` or `RoaringBitmap` instead?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#issuecomment-938200237
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7542](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (c7fe9a0) into [master](https://codecov.io/gh/apache/pinot/commit/7be21c80b352a02c58807b3c95d535f5fae394cc?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (7be21c8) will **decrease** coverage by `57.30%`.
> The diff coverage is `0.00%`.
[](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7542 +/- ##
=============================================
- Coverage 72.54% 15.23% -57.31%
+ Complexity 3405 80 -3325
=============================================
Files 1523 1479 -44
Lines 75653 73952 -1701
Branches 11032 10883 -149
=============================================
- Hits 54880 11268 -43612
- Misses 17078 61874 +44796
+ Partials 3695 810 -2885
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| integration2 | `?` | |
| unittests1 | `?` | |
| unittests2 | `15.23% <0.00%> (-0.03%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...e/pinot/common/function/TransformFunctionType.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vVHJhbnNmb3JtRnVuY3Rpb25UeXBlLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...erator/transform/function/InTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vSW5UcmFuc2Zvcm1GdW5jdGlvbi5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...r/transform/function/TransformFunctionFactory.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vVHJhbnNmb3JtRnVuY3Rpb25GYWN0b3J5LmphdmE=) | `0.00% <0.00%> (-84.00%)` | :arrow_down: |
| [...c/main/java/org/apache/pinot/common/tier/Tier.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdGllci9UaWVyLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...ain/java/org/apache/pinot/core/data/table/Key.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9kYXRhL3RhYmxlL0tleS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [.../java/org/apache/pinot/spi/utils/BooleanUtils.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc3BpL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcGkvdXRpbHMvQm9vbGVhblV0aWxzLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [.../java/org/apache/pinot/core/data/table/Record.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9kYXRhL3RhYmxlL1JlY29yZC5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [.../java/org/apache/pinot/core/util/GroupByUtils.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS91dGlsL0dyb3VwQnlVdGlscy5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...ava/org/apache/pinot/spi/data/MetricFieldSpec.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc3BpL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcGkvZGF0YS9NZXRyaWNGaWVsZFNwZWMuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...va/org/apache/pinot/spi/utils/BigDecimalUtils.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc3BpL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcGkvdXRpbHMvQmlnRGVjaW1hbFV0aWxzLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| ... and [1207 more](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [19f66fd...c7fe9a0](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] kishoreg commented on pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
kishoreg commented on pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#issuecomment-938200566
can you add more description on what the in function is supposed to return? I had to look at the code to see that it's 1 or 0.
Also, what are the scenarios/usecases where we need this?
Is this standard SQL? what is the equivalent of this in MySQL, Presto etc.
I was wondering if IN would clash with IN in the filter clause. Should we come up with a better name?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#discussion_r725540745
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/InTransformFunction.java
##########
@@ -81,77 +95,112 @@ public TransformResultMetadata getResultMetadata() {
switch (storedType) {
case INT:
int[] intValues = _transformFunction.transformToIntValuesSV(projectionBlock);
- int[][] inIntValues = new int[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inIntValues[i] = _valueTransformFunctions[i].transformToIntValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inIntValues.length; j++) {
- _results[i] = inIntValues[j][i] == intValues[i] ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
+ Set<Integer> inIntValues = new HashSet<>();
+ for (String inValue : _stringValueSet) {
+ inIntValues.add(Integer.parseInt(inValue));
+ }
+ for (int i = 0; i < length; i++) {
+ _results[i] = inIntValues.contains(intValues[i]) ? 1 : 0;
+ }
+ } else {
+ int[][] inIntValues = new int[_valueTransformFunctions.length][];
+ for (int i = 0; i < _valueTransformFunctions.length; i++) {
+ inIntValues[i] = _valueTransformFunctions[i].transformToIntValuesSV(projectionBlock);
+ }
+ for (int i = 0; i < length; i++) {
+ for (int j = 0; j < inIntValues.length; j++) {
+ _results[i] = inIntValues[j][i] == intValues[i] ? 1 : _results[i];
+ }
}
}
break;
case LONG:
long[] longValues = _transformFunction.transformToLongValuesSV(projectionBlock);
- long[][] inLongValues = new long[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inLongValues[i] = _valueTransformFunctions[i].transformToLongValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inLongValues.length; j++) {
- _results[i] = inLongValues[j][i] == longValues[i] ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
+ Set<Long> inLongValues = new HashSet<>();
+ for (String inValue : _stringValueSet) {
+ inLongValues.add(Long.parseLong(inValue));
+ }
+ for (int i = 0; i < length; i++) {
+ _results[i] = inLongValues.contains(longValues[i]) ? 1 : 0;
+ }
+ } else {
+ long[][] inLongValues = new long[_valueTransformFunctions.length][];
+ for (int i = 0; i < _valueTransformFunctions.length; i++) {
+ inLongValues[i] = _valueTransformFunctions[i].transformToLongValuesSV(projectionBlock);
+ }
+ for (int i = 0; i < length; i++) {
+ for (int j = 0; j < inLongValues.length; j++) {
+ _results[i] = inLongValues[j][i] == longValues[i] ? 1 : _results[i];
+ }
}
}
break;
case FLOAT:
float[] floatValues = _transformFunction.transformToFloatValuesSV(projectionBlock);
- float[][] inFloatValues = new float[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inFloatValues[i] = _valueTransformFunctions[i].transformToFloatValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inFloatValues.length; j++) {
- _results[i] = Float.compare(inFloatValues[j][i], floatValues[i]) == 0 ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
+ Set<Float> inFloatValues = new HashSet<>();
+ for (String inValue : _stringValueSet) {
+ inFloatValues.add(Float.parseFloat(inValue));
+ }
+ for (int i = 0; i < length; i++) {
+ _results[i] = inFloatValues.contains(floatValues[i]) ? 1 : 0;
+ }
+ } else {
+ float[][] inFloatValues = new float[_valueTransformFunctions.length][];
+ for (int i = 0; i < _valueTransformFunctions.length; i++) {
+ inFloatValues[i] = _valueTransformFunctions[i].transformToFloatValuesSV(projectionBlock);
+ }
+ for (int i = 0; i < length; i++) {
+ for (int j = 0; j < inFloatValues.length; j++) {
+ _results[i] = Float.compare(inFloatValues[j][i], floatValues[i]) == 0 ? 1 : _results[i];
+ }
}
}
break;
case DOUBLE:
double[] doubleValues = _transformFunction.transformToDoubleValuesSV(projectionBlock);
- double[][] inDoubleValues = new double[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inDoubleValues[i] = _valueTransformFunctions[i].transformToDoubleValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inDoubleValues.length; j++) {
- _results[i] = Double.compare(inDoubleValues[j][i], doubleValues[i]) == 0 ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
+ Set<Double> inDoubleValues = new HashSet<>();
+ for (String inValue : _stringValueSet) {
+ inDoubleValues.add(Double.parseDouble(inValue));
+ }
+ for (int i = 0; i < length; i++) {
+ _results[i] = inDoubleValues.contains(doubleValues[i]) ? 1 : 0;
+ }
+ } else {
+ double[][] inDoubleValues = new double[_valueTransformFunctions.length][];
+ for (int i = 0; i < _valueTransformFunctions.length; i++) {
+ inDoubleValues[i] = _valueTransformFunctions[i].transformToDoubleValuesSV(projectionBlock);
+ }
+ for (int i = 0; i < length; i++) {
+ for (int j = 0; j < inDoubleValues.length; j++) {
+ _results[i] = Double.compare(inDoubleValues[j][i], doubleValues[i]) == 0 ? 1 : _results[i];
+ }
}
}
break;
case STRING:
String[] stringValues = _transformFunction.transformToStringValuesSV(projectionBlock);
- String[][] inStringValues = new String[_valueTransformFunctions.length][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inStringValues[i] = _valueTransformFunctions[i].transformToStringValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inStringValues.length; j++) {
- _results[i] = inStringValues[j][i].equals(stringValues[i]) ? 1 : _results[i];
+ if (!_stringValueSet.isEmpty()) {
+ for (int i = 0; i < length; i++) {
+ _results[i] = _stringValueSet.contains(stringValues[i]) ? 1 : 0;
}
- }
- break;
- case BYTES:
- byte[][] bytesValues = _transformFunction.transformToBytesValuesSV(projectionBlock);
- byte[][][] inBytesValues = new byte[_valueTransformFunctions.length][][];
- for (int i = 0; i < _valueTransformFunctions.length; i++) {
- inBytesValues[i] = _valueTransformFunctions[i].transformToBytesValuesSV(projectionBlock);
- }
- for (int i = 0; i < length; i++) {
- for (int j = 0; j < inBytesValues.length; j++) {
- _results[i] =
- new ByteArray(inBytesValues[j][i]).compareTo(new ByteArray(bytesValues[i])) == 0 ? 1 : _results[i];
+ } else {
+ String[][] inStringValues = new String[_valueTransformFunctions.length][];
+ for (int i = 0; i < _valueTransformFunctions.length; i++) {
+ inStringValues[i] = _valueTransformFunctions[i].transformToStringValuesSV(projectionBlock);
+ }
+ for (int i = 0; i < length; i++) {
+ for (int j = 0; j < inStringValues.length; j++) {
+ _results[i] = inStringValues[j][i].equals(stringValues[i]) ? 1 : _results[i];
+ }
}
}
break;
+ case BYTES:
+ throw new UnsupportedOperationException();
Review comment:
We use hex encoding for binary literals. Added the support
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] yupeng9 commented on pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
yupeng9 commented on pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#issuecomment-938204553
Added some description to the PR header
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#discussion_r724633088
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/InTransformFunction.java
##########
@@ -0,0 +1,161 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import com.google.common.base.Preconditions;
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.common.function.TransformFunctionType;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.core.plan.DocIdSetPlanNode;
+import org.apache.pinot.segment.spi.datasource.DataSource;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.utils.ByteArray;
+
+
+/**
+ * The IN transform function takes more than 1 arguments:
+ * <ul>
+ * <li>Expression: a single-value expression</li>
+ * <li>values: a set of literal strings</li>
+ * </ul>
+ * <p>For each docId, the function returns {@code 1} if the set of values contains the value of the expression,
+ * {code 0} if not.
+ * <p>E.g. {@code SELECT col IN ('a','b','c') FROM myTable)}
+ */
+public class InTransformFunction extends BaseTransformFunction {
+ private TransformFunction _transformFunction;
+ private TransformFunction[] _valueTransformFunctions;
+ private int[] _results;
+
+ @Override
+ public String getName() {
+ return TransformFunctionType.IN.getName();
+ }
+
+ @Override
+ public void init(List<TransformFunction> arguments, Map<String, DataSource> dataSourceMap) {
+ Preconditions.checkArgument(arguments.size() >= 2,
+ "at least 2 arguments are required for IN transform function: expression, values");
+ Preconditions.checkArgument(arguments.get(0).getResultMetadata().isSingleValue(),
+ "First argument for IN transform function must be a single-value expression");
Review comment:
That is different. `valueIn` returns another MV column that only contains the filter values, `in` returns the boolean value. It should have the same behavior as the `in` filter clause.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#issuecomment-938200237
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7542](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (4942bc7) into [master](https://codecov.io/gh/apache/pinot/commit/7be21c80b352a02c58807b3c95d535f5fae394cc?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (7be21c8) will **decrease** coverage by `1.19%`.
> The diff coverage is `55.86%`.
[](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7542 +/- ##
============================================
- Coverage 72.54% 71.34% -1.20%
- Complexity 3405 3414 +9
============================================
Files 1523 1525 +2
Lines 75653 75912 +259
Branches 11032 11106 +74
============================================
- Hits 54880 54158 -722
- Misses 17078 18058 +980
- Partials 3695 3696 +1
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| integration2 | `29.08% <0.93%> (+0.08%)` | :arrow_up: |
| unittests1 | `69.82% <55.86%> (+0.01%)` | :arrow_up: |
| unittests2 | `15.26% <0.00%> (-0.01%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...erator/transform/function/InTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vSW5UcmFuc2Zvcm1GdW5jdGlvbi5qYXZh) | `55.45% <55.45%> (ø)` | |
| [...e/pinot/common/function/TransformFunctionType.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vVHJhbnNmb3JtRnVuY3Rpb25UeXBlLmphdmE=) | `100.00% <100.00%> (ø)` | |
| [...r/transform/function/TransformFunctionFactory.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vVHJhbnNmb3JtRnVuY3Rpb25GYWN0b3J5LmphdmE=) | `84.15% <100.00%> (+0.15%)` | :arrow_up: |
| [...pinot/minion/exception/TaskCancelledException.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtbWluaW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9taW5pb24vZXhjZXB0aW9uL1Rhc2tDYW5jZWxsZWRFeGNlcHRpb24uamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...nverttorawindex/ConvertToRawIndexTaskExecutor.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1taW5pb24tdGFza3MvcGlub3QtbWluaW9uLWJ1aWx0aW4tdGFza3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3BsdWdpbi9taW5pb24vdGFza3MvY29udmVydHRvcmF3aW5kZXgvQ29udmVydFRvUmF3SW5kZXhUYXNrRXhlY3V0b3IuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...plugin/segmentuploader/SegmentUploaderDefault.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtcGx1Z2lucy9waW5vdC1zZWdtZW50LXVwbG9hZGVyL3Bpbm90LXNlZ21lbnQtdXBsb2FkZXItZGVmYXVsdC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvcGx1Z2luL3NlZ21lbnR1cGxvYWRlci9TZWdtZW50VXBsb2FkZXJEZWZhdWx0LmphdmE=) | `0.00% <0.00%> (-87.10%)` | :arrow_down: |
| [...ore/startree/executor/StarTreeGroupByExecutor.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9zdGFydHJlZS9leGVjdXRvci9TdGFyVHJlZUdyb3VwQnlFeGVjdXRvci5qYXZh) | `0.00% <0.00%> (-86.67%)` | :arrow_down: |
| [.../transform/function/MapValueTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vTWFwVmFsdWVUcmFuc2Zvcm1GdW5jdGlvbi5qYXZh) | `0.00% <0.00%> (-85.30%)` | :arrow_down: |
| [...ot/common/messages/RoutingTableRebuildMessage.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vbWVzc2FnZXMvUm91dGluZ1RhYmxlUmVidWlsZE1lc3NhZ2UuamF2YQ==) | `0.00% <0.00%> (-81.82%)` | :arrow_down: |
| [...e/pinot/common/minion/MergeRollupTaskMetadata.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vbWluaW9uL01lcmdlUm9sbHVwVGFza01ldGFkYXRhLmphdmE=) | `0.00% <0.00%> (-78.27%)` | :arrow_down: |
| ... and [112 more](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [19f66fd...4942bc7](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#issuecomment-938200237
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7542](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (4942bc7) into [master](https://codecov.io/gh/apache/pinot/commit/7be21c80b352a02c58807b3c95d535f5fae394cc?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (7be21c8) will **decrease** coverage by `6.66%`.
> The diff coverage is `55.86%`.
[](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7542 +/- ##
============================================
- Coverage 72.54% 65.87% -6.67%
- Complexity 3405 3414 +9
============================================
Files 1523 1479 -44
Lines 75653 74061 -1592
Branches 11032 10906 -126
============================================
- Hits 54880 48790 -6090
- Misses 17078 21802 +4724
+ Partials 3695 3469 -226
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| integration2 | `?` | |
| unittests1 | `69.82% <55.86%> (+0.01%)` | :arrow_up: |
| unittests2 | `15.26% <0.00%> (-0.01%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...erator/transform/function/InTransformFunction.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vSW5UcmFuc2Zvcm1GdW5jdGlvbi5qYXZh) | `55.45% <55.45%> (ø)` | |
| [...e/pinot/common/function/TransformFunctionType.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vVHJhbnNmb3JtRnVuY3Rpb25UeXBlLmphdmE=) | `100.00% <100.00%> (ø)` | |
| [...r/transform/function/TransformFunctionFactory.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vVHJhbnNmb3JtRnVuY3Rpb25GYWN0b3J5LmphdmE=) | `81.18% <100.00%> (-2.82%)` | :arrow_down: |
| [...a/org/apache/pinot/common/metrics/MinionMeter.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vbWV0cmljcy9NaW5pb25NZXRlci5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...g/apache/pinot/common/metrics/ControllerMeter.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vbWV0cmljcy9Db250cm9sbGVyTWV0ZXIuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [.../apache/pinot/common/metrics/BrokerQueryPhase.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vbWV0cmljcy9Ccm9rZXJRdWVyeVBoYXNlLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [.../apache/pinot/common/metrics/MinionQueryPhase.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vbWV0cmljcy9NaW5pb25RdWVyeVBoYXNlLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...he/pinot/common/messages/SegmentReloadMessage.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vbWVzc2FnZXMvU2VnbWVudFJlbG9hZE1lc3NhZ2UuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...pinot/core/data/manager/realtime/TimerService.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9kYXRhL21hbmFnZXIvcmVhbHRpbWUvVGltZXJTZXJ2aWNlLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...pinot/minion/exception/TaskCancelledException.java](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtbWluaW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9taW5pb24vZXhjZXB0aW9uL1Rhc2tDYW5jZWxsZWRFeGNlcHRpb24uamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| ... and [359 more](https://codecov.io/gh/apache/pinot/pull/7542/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [19f66fd...4942bc7](https://codecov.io/gh/apache/pinot/pull/7542?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #7542: Add IN function
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #7542:
URL: https://github.com/apache/pinot/pull/7542#discussion_r724598767
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/InTransformFunction.java
##########
@@ -0,0 +1,161 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import com.google.common.base.Preconditions;
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.common.function.TransformFunctionType;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.core.plan.DocIdSetPlanNode;
+import org.apache.pinot.segment.spi.datasource.DataSource;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.utils.ByteArray;
+
+
+/**
+ * The IN transform function takes more than 1 arguments:
+ * <ul>
+ * <li>Expression: a single-value expression</li>
+ * <li>values: a set of literal strings</li>
+ * </ul>
+ * <p>For each docId, the function returns {@code 1} if the set of values contains the value of the expression,
+ * {code 0} if not.
+ * <p>E.g. {@code SELECT col IN ('a','b','c') FROM myTable)}
+ */
+public class InTransformFunction extends BaseTransformFunction {
+ private TransformFunction _transformFunction;
+ private TransformFunction[] _valueTransformFunctions;
+ private int[] _results;
+
+ @Override
+ public String getName() {
+ return TransformFunctionType.IN.getName();
+ }
+
+ @Override
+ public void init(List<TransformFunction> arguments, Map<String, DataSource> dataSourceMap) {
+ Preconditions.checkArgument(arguments.size() >= 2,
+ "at least 2 arguments are required for IN transform function: expression, values");
+ Preconditions.checkArgument(arguments.get(0).getResultMetadata().isSingleValue(),
+ "First argument for IN transform function must be a single-value expression");
Review comment:
We should consider supporting MV as well (any value matches the IN values are count as match)
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/InTransformFunction.java
##########
@@ -0,0 +1,161 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import com.google.common.base.Preconditions;
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.common.function.TransformFunctionType;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.core.plan.DocIdSetPlanNode;
+import org.apache.pinot.segment.spi.datasource.DataSource;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.utils.ByteArray;
+
+
+/**
+ * The IN transform function takes more than 1 arguments:
+ * <ul>
+ * <li>Expression: a single-value expression</li>
+ * <li>values: a set of literal strings</li>
+ * </ul>
+ * <p>For each docId, the function returns {@code 1} if the set of values contains the value of the expression,
+ * {code 0} if not.
+ * <p>E.g. {@code SELECT col IN ('a','b','c') FROM myTable)}
+ */
+public class InTransformFunction extends BaseTransformFunction {
+ private TransformFunction _transformFunction;
+ private TransformFunction[] _valueTransformFunctions;
+ private int[] _results;
+
+ @Override
+ public String getName() {
+ return TransformFunctionType.IN.getName();
+ }
+
+ @Override
+ public void init(List<TransformFunction> arguments, Map<String, DataSource> dataSourceMap) {
+ Preconditions.checkArgument(arguments.size() >= 2,
Review comment:
(nit) cache `arguments.size()` into a local variable
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/InTransformFunction.java
##########
@@ -0,0 +1,161 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import com.google.common.base.Preconditions;
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.common.function.TransformFunctionType;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.core.plan.DocIdSetPlanNode;
+import org.apache.pinot.segment.spi.datasource.DataSource;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.utils.ByteArray;
+
+
+/**
+ * The IN transform function takes more than 1 arguments:
+ * <ul>
+ * <li>Expression: a single-value expression</li>
+ * <li>values: a set of literal strings</li>
+ * </ul>
+ * <p>For each docId, the function returns {@code 1} if the set of values contains the value of the expression,
+ * {code 0} if not.
+ * <p>E.g. {@code SELECT col IN ('a','b','c') FROM myTable)}
+ */
+public class InTransformFunction extends BaseTransformFunction {
+ private TransformFunction _transformFunction;
+ private TransformFunction[] _valueTransformFunctions;
+ private int[] _results;
+
+ @Override
+ public String getName() {
+ return TransformFunctionType.IN.getName();
+ }
+
+ @Override
+ public void init(List<TransformFunction> arguments, Map<String, DataSource> dataSourceMap) {
+ Preconditions.checkArgument(arguments.size() >= 2,
+ "at least 2 arguments are required for IN transform function: expression, values");
+ Preconditions.checkArgument(arguments.get(0).getResultMetadata().isSingleValue(),
+ "First argument for IN transform function must be a single-value expression");
+ _transformFunction = arguments.get(0);
+ _valueTransformFunctions = new TransformFunction[arguments.size() - 1];
+ for (int i = 1; i < arguments.size(); i++) {
+ Preconditions.checkArgument(arguments.get(i).getResultMetadata().isSingleValue(),
+ "The values argument for IN transform function must be single value");
+ _valueTransformFunctions[i - 1] = arguments.get(i);
+ }
+ }
+
+ @Override
+ public TransformResultMetadata getResultMetadata() {
+ return BOOLEAN_SV_NO_DICTIONARY_METADATA;
+ }
+
+ @Override
+ public int[] transformToIntValuesSV(ProjectionBlock projectionBlock) {
+ if (_results == null) {
+ _results = new int[DocIdSetPlanNode.MAX_DOC_PER_CALL];
+ }
+
+ int length = projectionBlock.getNumDocs();
+ FieldSpec.DataType storedType = _transformFunction.getResultMetadata().getDataType().getStoredType();
+ switch (storedType) {
+ case INT:
+ int[] intValues = _transformFunction.transformToIntValuesSV(projectionBlock);
+ int[][] inIntValues = new int[_valueTransformFunctions.length][];
Review comment:
We should pre-process the IN values in the `init()` as mentioned above instead of doing per-block per-value match
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/InTransformFunction.java
##########
@@ -0,0 +1,161 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import com.google.common.base.Preconditions;
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.common.function.TransformFunctionType;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.core.plan.DocIdSetPlanNode;
+import org.apache.pinot.segment.spi.datasource.DataSource;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.utils.ByteArray;
+
+
+/**
+ * The IN transform function takes more than 1 arguments:
+ * <ul>
+ * <li>Expression: a single-value expression</li>
+ * <li>values: a set of literal strings</li>
+ * </ul>
+ * <p>For each docId, the function returns {@code 1} if the set of values contains the value of the expression,
+ * {code 0} if not.
+ * <p>E.g. {@code SELECT col IN ('a','b','c') FROM myTable)}
+ */
+public class InTransformFunction extends BaseTransformFunction {
+ private TransformFunction _transformFunction;
+ private TransformFunction[] _valueTransformFunctions;
+ private int[] _results;
Review comment:
You may directly use the `_intValuesSV` from the `BaseTransformFunction` without creating another member variable
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/TransformFunctionFactory.java
##########
@@ -94,6 +94,7 @@ private TransformFunctionFactory() {
put(canonicalize(TransformFunctionType.DATETRUNC.getName().toLowerCase()), DateTruncTransformFunction.class);
put(canonicalize(TransformFunctionType.ARRAYLENGTH.getName().toLowerCase()),
ArrayLengthTransformFunction.class);
+ put(canonicalize(TransformFunctionType.IN.getName().toLowerCase()), InTransformFunction.class);
Review comment:
(nit) Put it below the `LESS_THAN_OR_EQUAL` since they are of the same category and return BOOLEAN values?
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/InTransformFunction.java
##########
@@ -0,0 +1,161 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.operator.transform.function;
+
+import com.google.common.base.Preconditions;
+import java.util.List;
+import java.util.Map;
+import org.apache.pinot.common.function.TransformFunctionType;
+import org.apache.pinot.core.operator.blocks.ProjectionBlock;
+import org.apache.pinot.core.operator.transform.TransformResultMetadata;
+import org.apache.pinot.core.plan.DocIdSetPlanNode;
+import org.apache.pinot.segment.spi.datasource.DataSource;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.utils.ByteArray;
+
+
+/**
+ * The IN transform function takes more than 1 arguments:
+ * <ul>
+ * <li>Expression: a single-value expression</li>
+ * <li>values: a set of literal strings</li>
+ * </ul>
+ * <p>For each docId, the function returns {@code 1} if the set of values contains the value of the expression,
+ * {code 0} if not.
+ * <p>E.g. {@code SELECT col IN ('a','b','c') FROM myTable)}
+ */
+public class InTransformFunction extends BaseTransformFunction {
+ private TransformFunction _transformFunction;
+ private TransformFunction[] _valueTransformFunctions;
+ private int[] _results;
+
+ @Override
+ public String getName() {
+ return TransformFunctionType.IN.getName();
+ }
+
+ @Override
+ public void init(List<TransformFunction> arguments, Map<String, DataSource> dataSourceMap) {
+ Preconditions.checkArgument(arguments.size() >= 2,
+ "at least 2 arguments are required for IN transform function: expression, values");
+ Preconditions.checkArgument(arguments.get(0).getResultMetadata().isSingleValue(),
+ "First argument for IN transform function must be a single-value expression");
+ _transformFunction = arguments.get(0);
+ _valueTransformFunctions = new TransformFunction[arguments.size() - 1];
Review comment:
The values here should always be `LiteralTransformFunction`. We can construct a IN value set here based on the main function data type. The IN value can be read via `LiteralTransformFunction.getLiteral()`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org