You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2021/10/12 19:07:04 UTC
[GitHub] [pinot] amrishlal opened a new pull request #7568: [WORK IN PROGRESS] EXPLAIN PLAN code
amrishlal opened a new pull request #7568:
URL: https://github.com/apache/pinot/pull/7568
## Description
**DO NOT REVIEW**
## Upgrade Notes
Does this PR prevent a zero down-time upgrade? (Assume upgrade order: Controller, Broker, Server, Minion)
* [ ] Yes (Please label as **<code>backward-incompat</code>**, and complete the section below on Release Notes)
Does this PR fix a zero-downtime upgrade introduced earlier?
* [ ] Yes (Please label this as **<code>backward-incompat</code>**, and complete the section below on Release Notes)
Does this PR otherwise need attention when creating release notes? Things to consider:
- New configuration options
- Deprecation of configurations
- Signature changes to public methods/interfaces
- New plugins added or old plugins removed
* [ ] Yes (Please label this PR as **<code>release-notes</code>** and complete the section on Release Notes)
## Release Notes
<!-- If you have tagged this as either backward-incompat or release-notes,
you MUST add text here that you would like to see appear in release notes of the
next release. -->
<!-- If you have a series of commits adding or enabling a feature, then
add this section only in final commit that marks the feature completed.
Refer to earlier release notes to see examples of text.
-->
## Documentation
<!-- If you have introduced a new feature or configuration, please add it to the documentation as well.
See https://docs.pinot.apache.org/developers/developers-and-contributors/update-document
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7568](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (c1c1d01) into [master](https://codecov.io/gh/apache/pinot/commit/2107b2cd1880ec74934ec5412dbee8c874f05610?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (2107b2c) will **decrease** coverage by `16.33%`.
> The diff coverage is `0.82%`.
> :exclamation: Current head c1c1d01 differs from pull request most recent head ef9643f. Consider uploading reports for the commit ef9643f to get more accurate results
[](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7568 +/- ##
=============================================
- Coverage 30.79% 14.46% -16.34%
- Complexity 0 80 +80
=============================================
Files 1570 1532 -38
Lines 80024 79031 -993
Branches 11904 11822 -82
=============================================
- Hits 24644 11430 -13214
- Misses 53274 66764 +13490
+ Partials 2106 837 -1269
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| integration2 | `?` | |
| unittests2 | `14.46% <0.82%> (?)` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...rg/apache/pinot/common/function/JsonPathCache.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vSnNvblBhdGhDYWNoZS5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...he/pinot/common/function/scalar/JsonFunctions.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL0pzb25GdW5jdGlvbnMuamF2YQ==) | `0.00% <0.00%> (-30.51%)` | :arrow_down: |
| [...apache/pinot/common/metrics/ValidationMetrics.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vbWV0cmljcy9WYWxpZGF0aW9uTWV0cmljcy5qYXZh) | `0.00% <0.00%> (-80.96%)` | :arrow_down: |
| [...ache/pinot/common/tier/PinotServerTierStorage.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdGllci9QaW5vdFNlcnZlclRpZXJTdG9yYWdlLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...c/main/java/org/apache/pinot/common/tier/Tier.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdGllci9UaWVyLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...java/org/apache/pinot/common/tier/TierFactory.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdGllci9UaWVyRmFjdG9yeS5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...inot/common/tier/TimeBasedTierSegmentSelector.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdGllci9UaW1lQmFzZWRUaWVyU2VnbWVudFNlbGVjdG9yLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [.../pinot/common/utils/ClientSSLContextGenerator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvQ2xpZW50U1NMQ29udGV4dEdlbmVyYXRvci5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...java/org/apache/pinot/common/utils/DataSchema.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvRGF0YVNjaGVtYS5qYXZh) | `0.00% <0.00%> (-46.19%)` | :arrow_down: |
| [...e/pinot/common/utils/FileUploadDownloadClient.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvRmlsZVVwbG9hZERvd25sb2FkQ2xpZW50LmphdmE=) | `0.00% <0.00%> (-59.75%)` | :arrow_down: |
| ... and [907 more](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [2107b2c...ef9643f](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] siddharthteotia commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
siddharthteotia commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r745814132
##########
File path: pinot-core/src/test/java/org/apache/pinot/queries/ExplainPlanQueriesTest.java
##########
@@ -0,0 +1,510 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.queries;
+
+import java.io.File;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.List;
+import org.apache.commons.io.FileUtils;
+import org.apache.pinot.common.response.broker.BrokerResponseNative;
+import org.apache.pinot.common.response.broker.ResultTable;
+import org.apache.pinot.common.utils.DataSchema;
+import org.apache.pinot.segment.local.indexsegment.immutable.ImmutableSegmentLoader;
+import org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl;
+import org.apache.pinot.segment.local.segment.index.loader.IndexLoadingConfig;
+import org.apache.pinot.segment.local.segment.readers.GenericRowRecordReader;
+import org.apache.pinot.segment.spi.ImmutableSegment;
+import org.apache.pinot.segment.spi.IndexSegment;
+import org.apache.pinot.segment.spi.creator.SegmentGeneratorConfig;
+import org.apache.pinot.spi.config.table.IndexingConfig;
+import org.apache.pinot.spi.config.table.TableConfig;
+import org.apache.pinot.spi.config.table.TableType;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.data.Schema;
+import org.apache.pinot.spi.data.readers.GenericRow;
+import org.apache.pinot.spi.utils.ReadMode;
+import org.apache.pinot.spi.utils.builder.TableConfigBuilder;
+import org.testng.Assert;
+import org.testng.annotations.BeforeClass;
+import org.testng.annotations.Test;
+
+
+public class ExplainPlanQueriesTest extends BaseQueriesTest {
+ private static final File INDEX_DIR = new File(FileUtils.getTempDirectory(), "ExplainPlanQueriesTest");
+ private static final String RAW_TABLE_NAME = "testTable";
+ private static final String SEGMENT_NAME = "testSegment";
+ private static final int NUM_RECORDS = 10;
+
+ private final static String COL1_NO_INDEX = "noIndexCol1";
+ private final static String COL2_NO_INDEX = "noIndexCol2";
+ private final static String COL3_NO_INDEX = "noIndexCol3";
+ private final static String COL1_INVERTED_INDEX = "invertedIndexCol1";
+ private final static String COL2_INVERTED_INDEX = "invertedIndexCol2";
+ private final static String COL3_INVERTED_INDEX = "invertedIndexCol3";
+ private final static String COL1_RANGE_INDEX = "rangeIndexCol1";
+ private final static String COL2_RANGE_INDEX = "rangeIndexCol2";
+ private final static String COL3_RANGE_INDEX = "rangeIndexCol3";
+ private final static String COL1_SORTED_INDEX = "sortedIndexCol1";
+ private final static String COL1_JSON_INDEX = "jsonIndexCol1";
+ private final static String COL1_TEXT_INDEX = "textIndexCol1";
+
+ private static final Schema SCHEMA = new Schema.SchemaBuilder().setSchemaName(RAW_TABLE_NAME)
+ .addSingleValueDimension(COL1_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL2_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_INVERTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_INVERTED_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_INVERTED_INDEX, FieldSpec.DataType.STRING)
+ .addSingleValueDimension(COL1_RANGE_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_SORTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL1_JSON_INDEX, FieldSpec.DataType.JSON)
+ .addSingleValueDimension(COL1_TEXT_INDEX, FieldSpec.DataType.STRING).build();
+
+ private static final DataSchema DATA_SCHEMA = new DataSchema(new String[]{"Operator", "Operator_Id", "Parent_Id"},
+ new DataSchema.ColumnDataType[]{DataSchema.ColumnDataType.STRING, DataSchema.ColumnDataType.INT,
+ DataSchema.ColumnDataType.INT});
+
+ private static final TableConfig TABLE_CONFIG =
+ new TableConfigBuilder(TableType.OFFLINE).setTableName(RAW_TABLE_NAME).build();
+
+ private IndexSegment _indexSegment;
+ private List<IndexSegment> _indexSegments;
+
+ @Override
+ protected String getFilter() {
+ return "";
+ }
+
+ @Override
+ protected IndexSegment getIndexSegment() {
+ return _indexSegment;
+ }
+
+ @Override
+ protected List<IndexSegment> getIndexSegments() {
+ return _indexSegments;
+ }
+
+ GenericRow createRecord(double doubleValue, int intValue, String stringValue, String jsonValue) {
+ GenericRow record = new GenericRow();
+ record.putValue(COL1_NO_INDEX, intValue);
+ record.putValue(COL2_NO_INDEX, intValue);
+ record.putValue(COL3_NO_INDEX, intValue);
+
+ record.putValue(COL1_INVERTED_INDEX, doubleValue);
+ record.putValue(COL2_INVERTED_INDEX, intValue);
+ record.putValue(COL3_INVERTED_INDEX, intValue);
+
+ record.putValue(COL1_RANGE_INDEX, doubleValue);
+ record.putValue(COL2_RANGE_INDEX, intValue);
+ record.putValue(COL3_RANGE_INDEX, intValue);
+
+ record.putValue(COL1_JSON_INDEX, jsonValue);
+ record.putValue(COL1_TEXT_INDEX, stringValue);
+
+ return record;
+ }
+
+ @BeforeClass
+ public void setUp()
+ throws Exception {
+ FileUtils.deleteDirectory(INDEX_DIR);
+
+ List<GenericRow> records = new ArrayList<>(NUM_RECORDS);
+ records.add(createRecord(1.1, 2, "daffy", "{\"first\": \"daffy\", \"last\": \"duck\"}"));
+
+ IndexingConfig indexingConfig = TABLE_CONFIG.getIndexingConfig();
+
+ List<String> invertedIndexColumns = Arrays.asList(COL1_INVERTED_INDEX, COL2_INVERTED_INDEX, COL3_INVERTED_INDEX);
+ indexingConfig.setInvertedIndexColumns(invertedIndexColumns);
+
+ List<String> rangeIndexColumns = Arrays.asList(COL1_RANGE_INDEX, COL2_RANGE_INDEX, COL3_RANGE_INDEX);
+ indexingConfig.setRangeIndexColumns(rangeIndexColumns);
+
+ List<String> jsonIndexColumns = Arrays.asList(COL1_JSON_INDEX);
+ indexingConfig.setJsonIndexColumns(jsonIndexColumns);
+
+ SegmentGeneratorConfig segmentGeneratorConfig = new SegmentGeneratorConfig(TABLE_CONFIG, SCHEMA);
+ segmentGeneratorConfig.setTableName(RAW_TABLE_NAME);
+ segmentGeneratorConfig.setSegmentName(SEGMENT_NAME);
+ segmentGeneratorConfig.setOutDir(INDEX_DIR.getPath());
+
+ SegmentIndexCreationDriverImpl driver = new SegmentIndexCreationDriverImpl();
+ driver.init(segmentGeneratorConfig, new GenericRowRecordReader(records));
+ driver.build();
+
+ IndexLoadingConfig indexLoadingConfig = new IndexLoadingConfig();
+ indexLoadingConfig.setTableConfig(TABLE_CONFIG);
+ indexLoadingConfig.setInvertedIndexColumns(new HashSet<>(invertedIndexColumns));
+ indexLoadingConfig.setRangeIndexColumns(new HashSet<>(rangeIndexColumns));
+ indexLoadingConfig.setJsonIndexColumns(new HashSet<String>(jsonIndexColumns));
+ indexLoadingConfig.setReadMode(ReadMode.mmap);
+
+ ImmutableSegment immutableSegment =
+ ImmutableSegmentLoader.load(new File(INDEX_DIR, SEGMENT_NAME), indexLoadingConfig);
+ _indexSegment = immutableSegment;
+ _indexSegments = Arrays.asList(immutableSegment);
+ }
+
+ /** Checks the correctness of EXPLAIN PLAN output. */
+ private void check(String query, ResultTable expected) {
+ BrokerResponseNative response = getBrokerResponseForSqlQuery(query);
+ ResultTable actual = response.getResultTable();
+
+ System.out.println(resultTableToString("Expected", expected));
+ System.out.println(resultTableToString("Actual", actual));
+
+ Assert.assertEquals(actual.getDataSchema(), expected.getDataSchema());
+ Assert.assertEquals(actual.getRows().size(), expected.getRows().size());
+
+ List<Object[]> expectedRows = expected.getRows();
+ List<Object[]> actualRows = actual.getRows();
+ for (int i = 0; i < expectedRows.size(); i++) {
+ Object[] expectedRow = expectedRows.get(i);
+ Object[] actualRow = actualRows.get(i);
+ Assert.assertEquals(actualRow.length, expectedRow.length);
+
+ for (int j = 0; j < actualRow.length; j++) {
+ Assert.assertEquals(actualRow[j], expectedRow[j]);
+ }
+ }
+ }
+
+ /** This function is used to generated expected results while creating test cases; otherwise, not used. */
+ private static String resultTableToString(String tag, ResultTable table) {
+ System.out.println("\n" + tag + ":");
+ List<Object[]> rows = table.getRows();
+
+ StringBuffer buffer = new StringBuffer();
+ for (Object[] row : rows) {
+ buffer.append("result.add(new Object[]{\"" + row[0] + "\", " + row[1] + ", " + row[2] + "});\n");
+ }
+
+ return buffer.toString();
+ }
+
+ @Test
+ public void testSelect() {
+ String query1 = "EXPLAIN PLAN FOR SELECT * FROM testTable";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(
+ new Object[]{"SELECT(selectList:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, jsonIndexCol1, "
+ + "noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, "
+ + "jsonIndexCol1, noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 3, 2});
+ result1.add(
+ new Object[]{"PROJECT(sortedIndexCol1, noIndexCol3, rangeIndexCol1, rangeIndexCol2, jsonIndexCol1, "
+ + "invertedIndexCol1, noIndexCol2, invertedIndexCol2, noIndexCol1, invertedIndexCol3, rangeIndexCol3, "
+ + "textIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 = "EXPLAIN PLAN FOR SELECT 'mickey' FROM testTable";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:'mickey')", 2, 1});
+ result2.add(new Object[]{"PROJECT(ALL)", 3, 2});
+ result2.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 4, 3});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+
+ String query3 = "EXPLAIN PLAN FOR SELECT invertedIndexCol1, noIndexCol1 FROM testTable LIMIT 100";
+ List<Object[]> result3 = new ArrayList<>();
+ result3.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result3.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result3.add(new Object[]{"SELECT(selectList:invertedIndexCol1, noIndexCol1)", 2, 1});
+ result3.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result3.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result3.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query3, new ResultTable(DATA_SCHEMA, result3));
+
+ String query4 = "EXPLAIN PLAN FOR SELECT DISTINCT invertedIndexCol1, noIndexCol1 FROM testTable LIMIT 100";
+ List<Object[]> result4 = new ArrayList<>();
+ result4.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result4.add(new Object[]{"COMBINE_DISTINCT", 1, 0});
+ result4.add(new Object[]{"DISTINCT(keyColumns:invertedIndexCol1, noIndexCol1)", 2, 1});
+ result4.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result4.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result4.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query4, new ResultTable(DATA_SCHEMA, result4));
+ }
Review comment:
I don't see tests for SELECT queries with transform functions. Also, for queries with ORDER BY
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7568](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (57ceb4a) into [master](https://codecov.io/gh/apache/pinot/commit/2107b2cd1880ec74934ec5412dbee8c874f05610?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (2107b2c) will **increase** coverage by `0.08%`.
> The diff coverage is `22.43%`.
> :exclamation: Current head 57ceb4a differs from pull request most recent head 03cae5a. Consider uploading reports for the commit 03cae5a to get more accurate results
[](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7568 +/- ##
==========================================
+ Coverage 30.79% 30.88% +0.08%
==========================================
Files 1570 1569 -1
Lines 80024 80550 +526
Branches 11904 11988 +84
==========================================
+ Hits 24644 24874 +230
- Misses 53274 53551 +277
- Partials 2106 2125 +19
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `29.28% <21.46%> (+0.06%)` | :arrow_up: |
| integration2 | `27.81% <21.06%> (+0.10%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...ache/pinot/common/tier/PinotServerTierStorage.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdGllci9QaW5vdFNlcnZlclRpZXJTdG9yYWdlLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...c/main/java/org/apache/pinot/common/tier/Tier.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdGllci9UaWVyLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...java/org/apache/pinot/common/tier/TierFactory.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdGllci9UaWVyRmFjdG9yeS5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...inot/common/tier/TimeBasedTierSegmentSelector.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdGllci9UaW1lQmFzZWRUaWVyU2VnbWVudFNlbGVjdG9yLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [.../pinot/common/utils/ClientSSLContextGenerator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvQ2xpZW50U1NMQ29udGV4dEdlbmVyYXRvci5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...che/pinot/common/utils/config/TierConfigUtils.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvY29uZmlnL1RpZXJDb25maWdVdGlscy5qYXZh) | `4.16% <0.00%> (-1.39%)` | :arrow_down: |
| [...e/pinot/controller/helix/SegmentStatusChecker.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9oZWxpeC9TZWdtZW50U3RhdHVzQ2hlY2tlci5qYXZh) | `70.42% <0.00%> (-6.68%)` | :arrow_down: |
| [...troller/helix/core/minion/ClusterInfoAccessor.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9oZWxpeC9jb3JlL21pbmlvbi9DbHVzdGVySW5mb0FjY2Vzc29yLmphdmE=) | `67.85% <0.00%> (-5.22%)` | :arrow_down: |
| [...ntroller/helix/core/rebalance/TableRebalancer.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9oZWxpeC9jb3JlL3JlYmFsYW5jZS9UYWJsZVJlYmFsYW5jZXIuamF2YQ==) | `53.82% <0.00%> (ø)` | |
| [...ller/validation/OfflineSegmentIntervalChecker.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci92YWxpZGF0aW9uL09mZmxpbmVTZWdtZW50SW50ZXJ2YWxDaGVja2VyLmphdmE=) | `76.40% <0.00%> (-9.68%)` | :arrow_down: |
| ... and [171 more](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [2107b2c...03cae5a](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] amrishlal commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
amrishlal commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r749484722
##########
File path: pinot-common/src/main/java/org/apache/pinot/common/request/PinotQuery.java
##########
@@ -593,6 +604,28 @@ public void setQueryOptionsIsSet(boolean value) {
}
}
+ public boolean isExplain() {
+ return this.explain;
+ }
+
+ public void setExplain(boolean explain) {
+ this.explain = explain;
+ setExplainIsSet(true);
Review comment:
This is a thrift generated method to set a flag which indicates whether a specific field is set or not.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter commented on pull request #7568: [WORK IN PROGRESS] EXPLAIN PLAN code
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7568](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (c0d1298) into [master](https://codecov.io/gh/apache/pinot/commit/1dda74a6ecd96e2704147ad803135d5046a1e13d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (1dda74a) will **decrease** coverage by `57.46%`.
> The diff coverage is `2.10%`.
[](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7568 +/- ##
=============================================
- Coverage 72.63% 15.17% -57.47%
+ Complexity 3416 80 -3336
=============================================
Files 1522 1477 -45
Lines 75905 74415 -1490
Branches 11093 10952 -141
=============================================
- Hits 55133 11289 -43844
- Misses 17075 62313 +45238
+ Partials 3697 813 -2884
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| integration2 | `?` | |
| unittests1 | `?` | |
| unittests2 | `15.17% <2.10%> (-0.08%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...java/org/apache/pinot/common/utils/DataSchema.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvRGF0YVNjaGVtYS5qYXZh) | `0.00% <0.00%> (-75.00%)` | :arrow_down: |
| [...org/apache/pinot/sql/parsers/CalciteSqlParser.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcWwvcGFyc2Vycy9DYWxjaXRlU3FsUGFyc2VyLmphdmE=) | `0.00% <0.00%> (-91.66%)` | :arrow_down: |
| [...operator/AcquireReleaseColumnsSegmentOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9BY3F1aXJlUmVsZWFzZUNvbHVtbnNTZWdtZW50T3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...a/org/apache/pinot/core/operator/BaseOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CYXNlT3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (-35.72%)` | :arrow_down: |
| [...he/pinot/core/operator/BitmapDocIdSetOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CaXRtYXBEb2NJZFNldE9wZXJhdG9yLmphdmE=) | `0.00% <0.00%> (-75.00%)` | :arrow_down: |
| [...g/apache/pinot/core/operator/DocIdSetOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9Eb2NJZFNldE9wZXJhdG9yLmphdmE=) | `0.00% <0.00%> (-92.31%)` | :arrow_down: |
| [.../pinot/core/operator/InstanceResponseOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9JbnN0YW5jZVJlc3BvbnNlT3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (-83.88%)` | :arrow_down: |
| [...apache/pinot/core/operator/ProjectionOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9Qcm9qZWN0aW9uT3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (-73.34%)` | :arrow_down: |
| [...erator/combine/AggregationOnlyCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0FnZ3JlZ2F0aW9uT25seUNvbWJpbmVPcGVyYXRvci5qYXZh) | `0.00% <0.00%> (-75.00%)` | :arrow_down: |
| [...not/core/operator/combine/BaseCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0Jhc2VDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (-87.84%)` | :arrow_down: |
| ... and [1264 more](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [1dda74a...c0d1298](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7568: [WORK IN PROGRESS] EXPLAIN PLAN code
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7568](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (f228b9d) into [master](https://codecov.io/gh/apache/pinot/commit/1dda74a6ecd96e2704147ad803135d5046a1e13d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (1dda74a) will **decrease** coverage by `1.35%`.
> The diff coverage is `65.10%`.
[](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7568 +/- ##
============================================
- Coverage 72.63% 71.28% -1.36%
- Complexity 3416 3423 +7
============================================
Files 1522 1523 +1
Lines 75905 76261 +356
Branches 11093 11153 +60
============================================
- Hits 55133 54359 -774
- Misses 17075 18199 +1124
- Partials 3697 3703 +6
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| integration2 | `28.83% <12.76%> (-0.10%)` | :arrow_down: |
| unittests1 | `69.91% <64.36%> (+0.06%)` | :arrow_up: |
| unittests2 | `15.15% <2.08%> (-0.10%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...operator/AcquireReleaseColumnsSegmentOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9BY3F1aXJlUmVsZWFzZUNvbHVtbnNTZWdtZW50T3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...he/pinot/core/operator/BitmapDocIdSetOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CaXRtYXBEb2NJZFNldE9wZXJhdG9yLmphdmE=) | `71.42% <0.00%> (-3.58%)` | :arrow_down: |
| [.../core/operator/combine/GroupByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0dyb3VwQnlDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `78.68% <0.00%> (-1.32%)` | :arrow_down: |
| [...nMaxValueBasedSelectionOrderByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL01pbk1heFZhbHVlQmFzZWRTZWxlY3Rpb25PcmRlckJ5Q29tYmluZU9wZXJhdG9yLmphdmE=) | `73.80% <0.00%> (-1.40%)` | :arrow_down: |
| [...rator/combine/SelectionOrderByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL1NlbGVjdGlvbk9yZGVyQnlDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `50.00% <0.00%> (-1.73%)` | :arrow_down: |
| [...or/dociditerators/ExpressionScanDocIdIterator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9kb2NpZGl0ZXJhdG9ycy9FeHByZXNzaW9uU2NhbkRvY0lkSXRlcmF0b3IuamF2YQ==) | `52.48% <0.00%> (-13.95%)` | :arrow_down: |
| [...ore/operator/filter/BitmapBasedFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvQml0bWFwQmFzZWRGaWx0ZXJPcGVyYXRvci5qYXZh) | `78.26% <0.00%> (-14.05%)` | :arrow_down: |
| [...ot/core/operator/filter/H3IndexFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvSDNJbmRleEZpbHRlck9wZXJhdG9yLmphdmE=) | `82.10% <0.00%> (-4.57%)` | :arrow_down: |
| [...e/pinot/core/operator/filter/OrFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvT3JGaWx0ZXJPcGVyYXRvci5qYXZh) | `56.25% <0.00%> (-33.75%)` | :arrow_down: |
| [...operator/filter/RangeIndexBasedFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvUmFuZ2VJbmRleEJhc2VkRmlsdGVyT3BlcmF0b3IuamF2YQ==) | `36.53% <0.00%> (-3.89%)` | :arrow_down: |
| ... and [175 more](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [1dda74a...f228b9d](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter commented on pull request #7568: [WORK IN PROGRESS] EXPLAIN PLAN code
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7568](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (c0d1298) into [master](https://codecov.io/gh/apache/pinot/commit/1dda74a6ecd96e2704147ad803135d5046a1e13d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (1dda74a) will **decrease** coverage by `57.46%`.
> The diff coverage is `2.10%`.
[](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7568 +/- ##
=============================================
- Coverage 72.63% 15.17% -57.47%
+ Complexity 3416 80 -3336
=============================================
Files 1522 1477 -45
Lines 75905 74415 -1490
Branches 11093 10952 -141
=============================================
- Hits 55133 11289 -43844
- Misses 17075 62313 +45238
+ Partials 3697 813 -2884
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| integration2 | `?` | |
| unittests1 | `?` | |
| unittests2 | `15.17% <2.10%> (-0.08%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...java/org/apache/pinot/common/utils/DataSchema.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvRGF0YVNjaGVtYS5qYXZh) | `0.00% <0.00%> (-75.00%)` | :arrow_down: |
| [...org/apache/pinot/sql/parsers/CalciteSqlParser.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcWwvcGFyc2Vycy9DYWxjaXRlU3FsUGFyc2VyLmphdmE=) | `0.00% <0.00%> (-91.66%)` | :arrow_down: |
| [...operator/AcquireReleaseColumnsSegmentOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9BY3F1aXJlUmVsZWFzZUNvbHVtbnNTZWdtZW50T3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...a/org/apache/pinot/core/operator/BaseOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CYXNlT3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (-35.72%)` | :arrow_down: |
| [...he/pinot/core/operator/BitmapDocIdSetOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CaXRtYXBEb2NJZFNldE9wZXJhdG9yLmphdmE=) | `0.00% <0.00%> (-75.00%)` | :arrow_down: |
| [...g/apache/pinot/core/operator/DocIdSetOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9Eb2NJZFNldE9wZXJhdG9yLmphdmE=) | `0.00% <0.00%> (-92.31%)` | :arrow_down: |
| [.../pinot/core/operator/InstanceResponseOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9JbnN0YW5jZVJlc3BvbnNlT3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (-83.88%)` | :arrow_down: |
| [...apache/pinot/core/operator/ProjectionOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9Qcm9qZWN0aW9uT3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (-73.34%)` | :arrow_down: |
| [...erator/combine/AggregationOnlyCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0FnZ3JlZ2F0aW9uT25seUNvbWJpbmVPcGVyYXRvci5qYXZh) | `0.00% <0.00%> (-75.00%)` | :arrow_down: |
| [...not/core/operator/combine/BaseCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0Jhc2VDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (-87.84%)` | :arrow_down: |
| ... and [1264 more](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [1dda74a...c0d1298](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7568: [WORK IN PROGRESS] EXPLAIN PLAN code
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7568](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (0ab950d) into [master](https://codecov.io/gh/apache/pinot/commit/5eee297dbea7b3b73ae853f2479a89e9ce017363?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (5eee297) will **increase** coverage by `33.71%`.
> The diff coverage is `64.33%`.
[](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7568 +/- ##
=============================================
+ Coverage 32.16% 65.87% +33.71%
- Complexity 0 3417 +3417
=============================================
Files 1513 1477 -36
Lines 75557 74418 -1139
Branches 11055 10949 -106
=============================================
+ Hits 24305 49025 +24720
+ Misses 49156 21942 -27214
- Partials 2096 3451 +1355
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| integration2 | `?` | |
| unittests1 | `69.87% <65.90%> (?)` | |
| unittests2 | `15.16% <1.92%> (?)` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...ospatial/transform/function/StPolygonFunction.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9nZW9zcGF0aWFsL3RyYW5zZm9ybS9mdW5jdGlvbi9TdFBvbHlnb25GdW5jdGlvbi5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...operator/AcquireReleaseColumnsSegmentOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9BY3F1aXJlUmVsZWFzZUNvbHVtbnNTZWdtZW50T3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...he/pinot/core/operator/BitmapDocIdSetOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CaXRtYXBEb2NJZFNldE9wZXJhdG9yLmphdmE=) | `71.42% <0.00%> (-3.58%)` | :arrow_down: |
| [.../core/operator/combine/GroupByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0dyb3VwQnlDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `78.68% <0.00%> (+3.68%)` | :arrow_up: |
| [...nMaxValueBasedSelectionOrderByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL01pbk1heFZhbHVlQmFzZWRTZWxlY3Rpb25PcmRlckJ5Q29tYmluZU9wZXJhdG9yLmphdmE=) | `61.90% <0.00%> (-13.30%)` | :arrow_down: |
| [...rator/combine/SelectionOrderByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL1NlbGVjdGlvbk9yZGVyQnlDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `23.33% <0.00%> (-28.40%)` | :arrow_down: |
| [...or/dociditerators/ExpressionScanDocIdIterator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9kb2NpZGl0ZXJhdG9ycy9FeHByZXNzaW9uU2NhbkRvY0lkSXRlcmF0b3IuamF2YQ==) | `47.51% <0.00%> (-15.34%)` | :arrow_down: |
| [...ore/operator/filter/BitmapBasedFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvQml0bWFwQmFzZWRGaWx0ZXJPcGVyYXRvci5qYXZh) | `58.69% <0.00%> (-33.62%)` | :arrow_down: |
| [...ot/core/operator/filter/H3IndexFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvSDNJbmRleEZpbHRlck9wZXJhdG9yLmphdmE=) | `82.10% <0.00%> (+82.10%)` | :arrow_up: |
| [...e/pinot/core/operator/filter/OrFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvT3JGaWx0ZXJPcGVyYXRvci5qYXZh) | `56.25% <0.00%> (-33.75%)` | :arrow_down: |
| ... and [1296 more](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [0ecfc11...0ab950d](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] jackjlli commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
jackjlli commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r747901291
##########
File path: pinot-common/src/main/java/org/apache/pinot/common/request/PinotQuery.java
##########
@@ -593,6 +604,28 @@ public void setQueryOptionsIsSet(boolean value) {
}
}
+ public boolean isExplain() {
+ return this.explain;
+ }
+
+ public void setExplain(boolean explain) {
+ this.explain = explain;
+ setExplainIsSet(true);
Review comment:
What's the purpose of this method? If it's necessary, should it set the value from the `explain` parameter?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] amrishlal commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
amrishlal commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r759008062
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/query/request/context/utils/QueryContextUtils.java
##########
@@ -52,4 +58,79 @@ public static boolean isDistinctQuery(QueryContext query) {
return aggregationFunctions != null && aggregationFunctions.length == 1
&& aggregationFunctions[0] instanceof DistinctAggregationFunction;
}
+
+ /** Collect aggregation functions (except for the ones in filter). */
+ public static void generateTransforms(QueryContext queryContext, Set<String> postAggregations) {
Review comment:
Done.
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/filter/EmptyFilterOperator.java
##########
@@ -26,10 +29,9 @@
* Singleton class which extends {@link BaseFilterOperator} that is empty, i.e. does not match any document.
*/
public final class EmptyFilterOperator extends BaseFilterOperator {
- private EmptyFilterOperator() {
Review comment:
Fixed.
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/BaseOperator.java
##########
@@ -52,4 +52,14 @@ public final T nextBlock() {
// Make it protected because we should always call nextBlock()
protected abstract T getNextBlock();
+
+ @Override
+ public String toExplainString() {
+ return getExplainPlanName();
+ }
+
+ @Override
+ public ExecutionStatistics getExecutionStatistics() {
Review comment:
Done.
##########
File path: pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseBrokerRequestHandler.java
##########
@@ -519,6 +545,15 @@ private BrokerResponseNative handleSQLRequest(long requestId, String query, Json
return brokerResponse;
}
+ /** Set EXPLAIN PLAN query to route to only one server. */
+ private void setRoutingToOneServer(Map<ServerInstance, List<String>> routingTable) {
Review comment:
Done.
##########
File path: pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseBrokerRequestHandler.java
##########
@@ -491,6 +504,19 @@ private BrokerResponseNative handleSQLRequest(long requestId, String query, Json
// Execute the query
ServerStats serverStats = new ServerStats();
+ if (brokerRequest.getPinotQuery().isExplain()) {
Review comment:
Done
##########
File path: pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseBrokerRequestHandler.java
##########
@@ -106,6 +106,7 @@
private static final String IN_SUBQUERY = "inSubquery";
private static final Expression FALSE = RequestUtils.getLiteralExpression(false);
private static final Expression TRUE = RequestUtils.getLiteralExpression(true);
+ private static final BrokerResponseNative BROKER_ONLY_EXPLAIN_PLAN_OUTPUT = getBrokerResponseExplainPlanOutput();
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r755656841
##########
File path: pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseBrokerRequestHandler.java
##########
@@ -519,6 +543,18 @@ private BrokerResponseNative handleSQLRequest(long requestId, String query, Json
return brokerResponse;
}
+ /** Set EXPLAIN PLAN query to route to only one server. */
+ private void setRoutingToOneServer(Map<ServerInstance, List<String>> routingTable) {
+ Set<Map.Entry<ServerInstance, List<String>>> servers = routingTable.entrySet();
+ // only send request to 1 server
+ Map.Entry<ServerInstance, List<String>> server = servers.iterator().next();
+ routingTable.clear();
+ List<String> segments = new ArrayList<>();
+ // only generate the plan for 1 segment
+ segments.add(server.getValue().get(0));
+ routingTable.put(server.getKey(), segments);
Review comment:
(minor) No need to create an extra list
```suggestion
routingTable.put(entry.getKey(), entry.getValue());
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] jackjlli commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
jackjlli commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r730106820
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/filter/OrFilterOperator.java
##########
@@ -49,4 +50,18 @@ protected FilterBlock getNextBlock() {
public String getOperatorName() {
return OPERATOR_NAME;
}
+
+ @Override
+ public String getExplainPlanName() {
+ return EXPLAIN_NAME;
+ }
+
+ @Override
+ public List<Operator> getChildOperators() {
+ List<Operator> operators = new ArrayList<>();
Review comment:
Same here.
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/query/AggregationOperator.java
##########
@@ -75,11 +79,36 @@ public String getOperatorName() {
return OPERATOR_NAME;
}
+ @Override
+ public String getExplainPlanName() {
+ return EXPLAIN_NAME;
+ }
+
+ @Override
+ public List<Operator> getChildOperators() {
+ return Arrays.asList(_transformOperator);
+ }
+
@Override
public ExecutionStatistics getExecutionStatistics() {
long numEntriesScannedInFilter = _transformOperator.getExecutionStatistics().getNumEntriesScannedInFilter();
long numEntriesScannedPostFilter = (long) _numDocsScanned * _transformOperator.getNumColumnsProjected();
return new ExecutionStatistics(_numDocsScanned, numEntriesScannedInFilter, numEntriesScannedPostFilter,
_numTotalDocs);
}
+
+ @Override
+ public String toExplainString() {
+ StringBuilder stringBuilder = new StringBuilder(getExplainPlanName()).append("(aggregations:");
+ int count = 0;
+ for (AggregationFunction func : _aggregationFunctions) {
Review comment:
I found there are two formats to form the explain string; one is to use a for loop like this and the other one is to use a variable `i` like the one in `DistinctOperator`. Can we unify the behavior?
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/filter/AndFilterOperator.java
##########
@@ -47,4 +49,18 @@ protected FilterBlock getNextBlock() {
public String getOperatorName() {
return OPERATOR_NAME;
}
+
+ @Override
+ public List<Operator> getChildOperators() {
+ ArrayList<Operator> result = new ArrayList<>();
Review comment:
Can it be replaced by `List<Operator> result = new ArrayList<>(_filterOperators);` ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] amrishlal commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
amrishlal commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r747069070
##########
File path: pinot-core/src/test/java/org/apache/pinot/queries/ExplainPlanQueriesTest.java
##########
@@ -0,0 +1,510 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.queries;
+
+import java.io.File;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.List;
+import org.apache.commons.io.FileUtils;
+import org.apache.pinot.common.response.broker.BrokerResponseNative;
+import org.apache.pinot.common.response.broker.ResultTable;
+import org.apache.pinot.common.utils.DataSchema;
+import org.apache.pinot.segment.local.indexsegment.immutable.ImmutableSegmentLoader;
+import org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl;
+import org.apache.pinot.segment.local.segment.index.loader.IndexLoadingConfig;
+import org.apache.pinot.segment.local.segment.readers.GenericRowRecordReader;
+import org.apache.pinot.segment.spi.ImmutableSegment;
+import org.apache.pinot.segment.spi.IndexSegment;
+import org.apache.pinot.segment.spi.creator.SegmentGeneratorConfig;
+import org.apache.pinot.spi.config.table.IndexingConfig;
+import org.apache.pinot.spi.config.table.TableConfig;
+import org.apache.pinot.spi.config.table.TableType;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.data.Schema;
+import org.apache.pinot.spi.data.readers.GenericRow;
+import org.apache.pinot.spi.utils.ReadMode;
+import org.apache.pinot.spi.utils.builder.TableConfigBuilder;
+import org.testng.Assert;
+import org.testng.annotations.BeforeClass;
+import org.testng.annotations.Test;
+
+
+public class ExplainPlanQueriesTest extends BaseQueriesTest {
+ private static final File INDEX_DIR = new File(FileUtils.getTempDirectory(), "ExplainPlanQueriesTest");
+ private static final String RAW_TABLE_NAME = "testTable";
+ private static final String SEGMENT_NAME = "testSegment";
+ private static final int NUM_RECORDS = 10;
+
+ private final static String COL1_NO_INDEX = "noIndexCol1";
+ private final static String COL2_NO_INDEX = "noIndexCol2";
+ private final static String COL3_NO_INDEX = "noIndexCol3";
+ private final static String COL1_INVERTED_INDEX = "invertedIndexCol1";
+ private final static String COL2_INVERTED_INDEX = "invertedIndexCol2";
+ private final static String COL3_INVERTED_INDEX = "invertedIndexCol3";
+ private final static String COL1_RANGE_INDEX = "rangeIndexCol1";
+ private final static String COL2_RANGE_INDEX = "rangeIndexCol2";
+ private final static String COL3_RANGE_INDEX = "rangeIndexCol3";
+ private final static String COL1_SORTED_INDEX = "sortedIndexCol1";
+ private final static String COL1_JSON_INDEX = "jsonIndexCol1";
+ private final static String COL1_TEXT_INDEX = "textIndexCol1";
+
+ private static final Schema SCHEMA = new Schema.SchemaBuilder().setSchemaName(RAW_TABLE_NAME)
+ .addSingleValueDimension(COL1_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL2_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_INVERTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_INVERTED_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_INVERTED_INDEX, FieldSpec.DataType.STRING)
+ .addSingleValueDimension(COL1_RANGE_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_SORTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL1_JSON_INDEX, FieldSpec.DataType.JSON)
+ .addSingleValueDimension(COL1_TEXT_INDEX, FieldSpec.DataType.STRING).build();
+
+ private static final DataSchema DATA_SCHEMA = new DataSchema(new String[]{"Operator", "Operator_Id", "Parent_Id"},
+ new DataSchema.ColumnDataType[]{DataSchema.ColumnDataType.STRING, DataSchema.ColumnDataType.INT,
+ DataSchema.ColumnDataType.INT});
+
+ private static final TableConfig TABLE_CONFIG =
+ new TableConfigBuilder(TableType.OFFLINE).setTableName(RAW_TABLE_NAME).build();
+
+ private IndexSegment _indexSegment;
+ private List<IndexSegment> _indexSegments;
+
+ @Override
+ protected String getFilter() {
+ return "";
+ }
+
+ @Override
+ protected IndexSegment getIndexSegment() {
+ return _indexSegment;
+ }
+
+ @Override
+ protected List<IndexSegment> getIndexSegments() {
+ return _indexSegments;
+ }
+
+ GenericRow createRecord(double doubleValue, int intValue, String stringValue, String jsonValue) {
+ GenericRow record = new GenericRow();
+ record.putValue(COL1_NO_INDEX, intValue);
+ record.putValue(COL2_NO_INDEX, intValue);
+ record.putValue(COL3_NO_INDEX, intValue);
+
+ record.putValue(COL1_INVERTED_INDEX, doubleValue);
+ record.putValue(COL2_INVERTED_INDEX, intValue);
+ record.putValue(COL3_INVERTED_INDEX, intValue);
+
+ record.putValue(COL1_RANGE_INDEX, doubleValue);
+ record.putValue(COL2_RANGE_INDEX, intValue);
+ record.putValue(COL3_RANGE_INDEX, intValue);
+
+ record.putValue(COL1_JSON_INDEX, jsonValue);
+ record.putValue(COL1_TEXT_INDEX, stringValue);
+
+ return record;
+ }
+
+ @BeforeClass
+ public void setUp()
+ throws Exception {
+ FileUtils.deleteDirectory(INDEX_DIR);
+
+ List<GenericRow> records = new ArrayList<>(NUM_RECORDS);
+ records.add(createRecord(1.1, 2, "daffy", "{\"first\": \"daffy\", \"last\": \"duck\"}"));
+
+ IndexingConfig indexingConfig = TABLE_CONFIG.getIndexingConfig();
+
+ List<String> invertedIndexColumns = Arrays.asList(COL1_INVERTED_INDEX, COL2_INVERTED_INDEX, COL3_INVERTED_INDEX);
+ indexingConfig.setInvertedIndexColumns(invertedIndexColumns);
+
+ List<String> rangeIndexColumns = Arrays.asList(COL1_RANGE_INDEX, COL2_RANGE_INDEX, COL3_RANGE_INDEX);
+ indexingConfig.setRangeIndexColumns(rangeIndexColumns);
+
+ List<String> jsonIndexColumns = Arrays.asList(COL1_JSON_INDEX);
+ indexingConfig.setJsonIndexColumns(jsonIndexColumns);
+
+ SegmentGeneratorConfig segmentGeneratorConfig = new SegmentGeneratorConfig(TABLE_CONFIG, SCHEMA);
+ segmentGeneratorConfig.setTableName(RAW_TABLE_NAME);
+ segmentGeneratorConfig.setSegmentName(SEGMENT_NAME);
+ segmentGeneratorConfig.setOutDir(INDEX_DIR.getPath());
+
+ SegmentIndexCreationDriverImpl driver = new SegmentIndexCreationDriverImpl();
+ driver.init(segmentGeneratorConfig, new GenericRowRecordReader(records));
+ driver.build();
+
+ IndexLoadingConfig indexLoadingConfig = new IndexLoadingConfig();
+ indexLoadingConfig.setTableConfig(TABLE_CONFIG);
+ indexLoadingConfig.setInvertedIndexColumns(new HashSet<>(invertedIndexColumns));
+ indexLoadingConfig.setRangeIndexColumns(new HashSet<>(rangeIndexColumns));
+ indexLoadingConfig.setJsonIndexColumns(new HashSet<String>(jsonIndexColumns));
+ indexLoadingConfig.setReadMode(ReadMode.mmap);
+
+ ImmutableSegment immutableSegment =
+ ImmutableSegmentLoader.load(new File(INDEX_DIR, SEGMENT_NAME), indexLoadingConfig);
+ _indexSegment = immutableSegment;
+ _indexSegments = Arrays.asList(immutableSegment);
+ }
+
+ /** Checks the correctness of EXPLAIN PLAN output. */
+ private void check(String query, ResultTable expected) {
+ BrokerResponseNative response = getBrokerResponseForSqlQuery(query);
+ ResultTable actual = response.getResultTable();
+
+ System.out.println(resultTableToString("Expected", expected));
+ System.out.println(resultTableToString("Actual", actual));
+
+ Assert.assertEquals(actual.getDataSchema(), expected.getDataSchema());
+ Assert.assertEquals(actual.getRows().size(), expected.getRows().size());
+
+ List<Object[]> expectedRows = expected.getRows();
+ List<Object[]> actualRows = actual.getRows();
+ for (int i = 0; i < expectedRows.size(); i++) {
+ Object[] expectedRow = expectedRows.get(i);
+ Object[] actualRow = actualRows.get(i);
+ Assert.assertEquals(actualRow.length, expectedRow.length);
+
+ for (int j = 0; j < actualRow.length; j++) {
+ Assert.assertEquals(actualRow[j], expectedRow[j]);
+ }
+ }
+ }
+
+ /** This function is used to generated expected results while creating test cases; otherwise, not used. */
+ private static String resultTableToString(String tag, ResultTable table) {
+ System.out.println("\n" + tag + ":");
+ List<Object[]> rows = table.getRows();
+
+ StringBuffer buffer = new StringBuffer();
+ for (Object[] row : rows) {
+ buffer.append("result.add(new Object[]{\"" + row[0] + "\", " + row[1] + ", " + row[2] + "});\n");
+ }
+
+ return buffer.toString();
+ }
+
+ @Test
+ public void testSelect() {
+ String query1 = "EXPLAIN PLAN FOR SELECT * FROM testTable";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(
+ new Object[]{"SELECT(selectList:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, jsonIndexCol1, "
+ + "noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, "
+ + "jsonIndexCol1, noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 3, 2});
+ result1.add(
+ new Object[]{"PROJECT(sortedIndexCol1, noIndexCol3, rangeIndexCol1, rangeIndexCol2, jsonIndexCol1, "
+ + "invertedIndexCol1, noIndexCol2, invertedIndexCol2, noIndexCol1, invertedIndexCol3, rangeIndexCol3, "
+ + "textIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 = "EXPLAIN PLAN FOR SELECT 'mickey' FROM testTable";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:'mickey')", 2, 1});
+ result2.add(new Object[]{"PROJECT(ALL)", 3, 2});
+ result2.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 4, 3});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+
+ String query3 = "EXPLAIN PLAN FOR SELECT invertedIndexCol1, noIndexCol1 FROM testTable LIMIT 100";
+ List<Object[]> result3 = new ArrayList<>();
+ result3.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result3.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result3.add(new Object[]{"SELECT(selectList:invertedIndexCol1, noIndexCol1)", 2, 1});
+ result3.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result3.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result3.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query3, new ResultTable(DATA_SCHEMA, result3));
+
+ String query4 = "EXPLAIN PLAN FOR SELECT DISTINCT invertedIndexCol1, noIndexCol1 FROM testTable LIMIT 100";
+ List<Object[]> result4 = new ArrayList<>();
+ result4.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result4.add(new Object[]{"COMBINE_DISTINCT", 1, 0});
+ result4.add(new Object[]{"DISTINCT(keyColumns:invertedIndexCol1, noIndexCol1)", 2, 1});
+ result4.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result4.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result4.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query4, new ResultTable(DATA_SCHEMA, result4));
+ }
+
+ @Test
+ public void testSelectColumnsUsingFilterIndex() {
+ String query1 = "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1, sortedIndexCol1 FROM testTable WHERE "
+ + "invertedIndexCol1 = 1.5 AND sortedIndexCol1 = 1 LIMIT 100";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1, sortedIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, sortedIndexCol1)", 3, 2});
+ result1.add(new Object[]{"PROJECT(sortedIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 = "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1, sortedIndexCol1 FROM testTable WHERE "
+ + "(invertedIndexCol1 = 1.5 AND sortedIndexCol1 = 1) OR (invertedIndexCol1 BETWEEN 100 and 200 AND "
+ + "rangeIndexCol1 > 20)";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1, sortedIndexCol1)", 2, 1});
+ result2.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, sortedIndexCol1)", 3, 2});
+ result2.add(new Object[]{"PROJECT(sortedIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result2.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+
+ String query3 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1 FROM testTable WHERE invertedIndexCol1 = 1.5 AND "
+ + "invertedIndexCol2 IN (10, 20, 30) AND invertedIndexCol3 NOT IN ('foo', 'bar') LIMIT 100";
+ List<Object[]> result3 = new ArrayList<>();
+ result3.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result3.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result3.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1)", 2, 1});
+ result3.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result3.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result3.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query3, new ResultTable(DATA_SCHEMA, result3));
+
+ String query4 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, noIndexCol2, sortedIndexCol1 FROM testTable WHERE sortedIndexCol1 = 1.5"
+ + " OR sortedIndexCol1 != 5 OR sortedIndexCol1 IN (10, 20, 30) LIMIT 100";
+ List<Object[]> result4 = new ArrayList<>();
+ result4.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result4.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result4.add(new Object[]{"SELECT(selectList:noIndexCol1, noIndexCol2, sortedIndexCol1)", 2, 1});
+ result4.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2, sortedIndexCol1)", 3, 2});
+ result4.add(new Object[]{"PROJECT(sortedIndexCol1, noIndexCol2, noIndexCol1)", 4, 3});
+ result4.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query4, new ResultTable(DATA_SCHEMA, result4));
+
+ String query5 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, noIndexCol2 FROM testTable WHERE DIV(noIndexCol1, noIndexCol2) BETWEEN "
+ + "10 AND 20 AND invertedIndexCol1 * 5 < 1000";
+ List<Object[]> result5 = new ArrayList<>();
+ result5.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result5.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result5.add(new Object[]{"SELECT(selectList:noIndexCol1, noIndexCol2)", 2, 1});
+ result5.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2)", 3, 2});
+ result5.add(new Object[]{"PROJECT(noIndexCol2, noIndexCol1)", 4, 3});
+ result5.add(new Object[]{"FILTER_AND", 5, 4});
+ result5.add(
+ new Object[]{"FILTER_EXPRESSION(operator:RANGE,predicate:div(noIndexCol1,noIndexCol2) BETWEEN '10' AND '20')",
+ 6, 5});
+ result5
+ .add(new Object[]{"FILTER_EXPRESSION(operator:RANGE,predicate:times(invertedIndexCol1,'5') < '1000')", 7, 5});
+ check(query5, new ResultTable(DATA_SCHEMA, result5));
+ }
+
+ @Test
+ public void testSelectColumnUsingFilterRange() {
+ // select * query triggering range index
+ // checks using RANGE (>, >=, <, <=, BETWEEN ..) on a column with range index should use RANGE_INDEX_SCAN
+ String query1 =
+ "EXPLAIN PLAN FOR SELECT invertedIndexCol1, noIndexCol1, rangeIndexCol1 FROM testTable WHERE rangeIndexCol1 >"
+ + " 10 AND rangeIndexCol2 >= 20 AND rangeIndexCol3 BETWEEN 50 AND 60 LIMIT 100";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(new Object[]{"SELECT(selectList:invertedIndexCol1, noIndexCol1, rangeIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, rangeIndexCol1)", 3, 2});
+ result1.add(new Object[]{"PROJECT(rangeIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1 FROM testTable WHERE noIndexCol1 > 10 AND noIndexCol2"
+ + " BETWEEN 50 AND 60 LIMIT 100";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1)", 2, 1});
+ result2.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result2.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result2.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+ }
+
+ @Test
+ public void testSelectColumnUsingFilterJson() {
+ String query =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1 FROM testTable WHERE (invertedIndexCol1 IN (10, 20, "
+ + "30) AND sortedIndexCol1 != 100) OR (noIndexCol1 NOT IN (10, 20, 30) AND rangeIndexCol1 != 20 AND "
+ + "JSON_MATCH(jsonIndexCol1, 'key=1') AND TEXT_MATCH(textIndexCol1, 'foo'))";
+ List<Object[]> result = new ArrayList<>();
+ result.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1)", 2, 1});
+ result.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result.add(new Object[]{"FILTER_AND", 5, 4});
+ result.add(new Object[]{"FILTER_JSON_INDEX(indexLookUp:json_index,operator:JSON_MATCH,predicate:json_match"
+ + "(jsonIndexCol1,'key=1'))", 6, 5});
+ result.add(new Object[]{"FILTER_TEXT_INDEX(indexLookUp:text_index,operator:TEXT_MATCH,predicate:text_match"
+ + "(textIndexCol1,'foo'))", 7, 5});
+ check(query, new ResultTable(DATA_SCHEMA, result));
+ }
+
+ @Test
+ public void testSelectAggregateUsingFilterText() {
+ String query1 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, noIndexCol2, max(noIndexCol2), min(noIndexCol3) FROM testTable WHERE "
+ + "TEXT_MATCH(textIndexCol1, 'foo') GROUP BY noIndexCol1, noIndexCol2";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result1.add(new Object[]{"COMBINE_GROUPBY_ORDERBY", 1, 0});
+ result1.add(new Object[]{
+ "AGGREGATE_GROUPBY_ORDERBY(groupKeys:noIndexCol1, noIndexCol2, aggregations:max(noIndexCol2), min"
+ + "(noIndexCol3))", 2, 1});
+ result1.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2, noIndexCol3)", 3, 2});
+ result1.add(new Object[]{"PROJECT(noIndexCol3, noIndexCol2, noIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_TEXT_INDEX(indexLookUp:text_index,operator:TEXT_MATCH,predicate:text_match"
+ + "(textIndexCol1,'foo'))", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, max(noIndexCol2), min(noIndexCol3) FROM testTable WHERE TEXT_MATCH"
+ + "(textIndexCol1, 'foo') GROUP BY noIndexCol1, noIndexCol2 ORDER BY noIndexCol1, max(noIndexCol2)";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(sort:[noIndexCol1 ASC, max(noIndexCol2) ASC],limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_GROUPBY_ORDERBY", 1, 0});
+ result2.add(new Object[]{
+ "AGGREGATE_GROUPBY_ORDERBY(groupKeys:noIndexCol1,noIndexCol2,aggregations:max(noIndexCol2), min"
+ + "(noIndexCol3))", 2, 1});
+ result2.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, invertedIndexCol1)", 3, 2});
+ result2.add(new Object[]{"PROJECT(noIndexCol3, noIndexCol2, noIndexCol1)", 4, 3});
+ result2.add(new Object[]{"FILTER_TEXT_INDEX(indexLookUp:text_index,operator:TEXT_MATCH,predicate:text_match"
+ + "(textIndexCol1,'foo'))", 5, 4});
+ }
+
+ @Test
+ public void testSelectAggregate() {
+ String query1 = "EXPLAIN PLAN FOR SELECT count(*) FROM testTable";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result1.add(new Object[]{"COMBINE_AGGREGATE", 1, 0});
+ result1.add(new Object[]{"AGGREGATE_METADATA", 2, 1});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
Review comment:
Done.
##########
File path: pinot-core/src/test/java/org/apache/pinot/queries/ExplainPlanQueriesTest.java
##########
@@ -0,0 +1,510 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.queries;
+
+import java.io.File;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.List;
+import org.apache.commons.io.FileUtils;
+import org.apache.pinot.common.response.broker.BrokerResponseNative;
+import org.apache.pinot.common.response.broker.ResultTable;
+import org.apache.pinot.common.utils.DataSchema;
+import org.apache.pinot.segment.local.indexsegment.immutable.ImmutableSegmentLoader;
+import org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl;
+import org.apache.pinot.segment.local.segment.index.loader.IndexLoadingConfig;
+import org.apache.pinot.segment.local.segment.readers.GenericRowRecordReader;
+import org.apache.pinot.segment.spi.ImmutableSegment;
+import org.apache.pinot.segment.spi.IndexSegment;
+import org.apache.pinot.segment.spi.creator.SegmentGeneratorConfig;
+import org.apache.pinot.spi.config.table.IndexingConfig;
+import org.apache.pinot.spi.config.table.TableConfig;
+import org.apache.pinot.spi.config.table.TableType;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.data.Schema;
+import org.apache.pinot.spi.data.readers.GenericRow;
+import org.apache.pinot.spi.utils.ReadMode;
+import org.apache.pinot.spi.utils.builder.TableConfigBuilder;
+import org.testng.Assert;
+import org.testng.annotations.BeforeClass;
+import org.testng.annotations.Test;
+
+
+public class ExplainPlanQueriesTest extends BaseQueriesTest {
+ private static final File INDEX_DIR = new File(FileUtils.getTempDirectory(), "ExplainPlanQueriesTest");
+ private static final String RAW_TABLE_NAME = "testTable";
+ private static final String SEGMENT_NAME = "testSegment";
+ private static final int NUM_RECORDS = 10;
+
+ private final static String COL1_NO_INDEX = "noIndexCol1";
+ private final static String COL2_NO_INDEX = "noIndexCol2";
+ private final static String COL3_NO_INDEX = "noIndexCol3";
+ private final static String COL1_INVERTED_INDEX = "invertedIndexCol1";
+ private final static String COL2_INVERTED_INDEX = "invertedIndexCol2";
+ private final static String COL3_INVERTED_INDEX = "invertedIndexCol3";
+ private final static String COL1_RANGE_INDEX = "rangeIndexCol1";
+ private final static String COL2_RANGE_INDEX = "rangeIndexCol2";
+ private final static String COL3_RANGE_INDEX = "rangeIndexCol3";
+ private final static String COL1_SORTED_INDEX = "sortedIndexCol1";
+ private final static String COL1_JSON_INDEX = "jsonIndexCol1";
+ private final static String COL1_TEXT_INDEX = "textIndexCol1";
+
+ private static final Schema SCHEMA = new Schema.SchemaBuilder().setSchemaName(RAW_TABLE_NAME)
+ .addSingleValueDimension(COL1_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL2_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_INVERTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_INVERTED_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_INVERTED_INDEX, FieldSpec.DataType.STRING)
+ .addSingleValueDimension(COL1_RANGE_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_SORTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL1_JSON_INDEX, FieldSpec.DataType.JSON)
+ .addSingleValueDimension(COL1_TEXT_INDEX, FieldSpec.DataType.STRING).build();
+
+ private static final DataSchema DATA_SCHEMA = new DataSchema(new String[]{"Operator", "Operator_Id", "Parent_Id"},
+ new DataSchema.ColumnDataType[]{DataSchema.ColumnDataType.STRING, DataSchema.ColumnDataType.INT,
+ DataSchema.ColumnDataType.INT});
+
+ private static final TableConfig TABLE_CONFIG =
+ new TableConfigBuilder(TableType.OFFLINE).setTableName(RAW_TABLE_NAME).build();
+
+ private IndexSegment _indexSegment;
+ private List<IndexSegment> _indexSegments;
+
+ @Override
+ protected String getFilter() {
+ return "";
+ }
+
+ @Override
+ protected IndexSegment getIndexSegment() {
+ return _indexSegment;
+ }
+
+ @Override
+ protected List<IndexSegment> getIndexSegments() {
+ return _indexSegments;
+ }
+
+ GenericRow createRecord(double doubleValue, int intValue, String stringValue, String jsonValue) {
+ GenericRow record = new GenericRow();
+ record.putValue(COL1_NO_INDEX, intValue);
+ record.putValue(COL2_NO_INDEX, intValue);
+ record.putValue(COL3_NO_INDEX, intValue);
+
+ record.putValue(COL1_INVERTED_INDEX, doubleValue);
+ record.putValue(COL2_INVERTED_INDEX, intValue);
+ record.putValue(COL3_INVERTED_INDEX, intValue);
+
+ record.putValue(COL1_RANGE_INDEX, doubleValue);
+ record.putValue(COL2_RANGE_INDEX, intValue);
+ record.putValue(COL3_RANGE_INDEX, intValue);
+
+ record.putValue(COL1_JSON_INDEX, jsonValue);
+ record.putValue(COL1_TEXT_INDEX, stringValue);
+
+ return record;
+ }
+
+ @BeforeClass
+ public void setUp()
+ throws Exception {
+ FileUtils.deleteDirectory(INDEX_DIR);
+
+ List<GenericRow> records = new ArrayList<>(NUM_RECORDS);
+ records.add(createRecord(1.1, 2, "daffy", "{\"first\": \"daffy\", \"last\": \"duck\"}"));
+
+ IndexingConfig indexingConfig = TABLE_CONFIG.getIndexingConfig();
+
+ List<String> invertedIndexColumns = Arrays.asList(COL1_INVERTED_INDEX, COL2_INVERTED_INDEX, COL3_INVERTED_INDEX);
+ indexingConfig.setInvertedIndexColumns(invertedIndexColumns);
+
+ List<String> rangeIndexColumns = Arrays.asList(COL1_RANGE_INDEX, COL2_RANGE_INDEX, COL3_RANGE_INDEX);
+ indexingConfig.setRangeIndexColumns(rangeIndexColumns);
+
+ List<String> jsonIndexColumns = Arrays.asList(COL1_JSON_INDEX);
+ indexingConfig.setJsonIndexColumns(jsonIndexColumns);
+
+ SegmentGeneratorConfig segmentGeneratorConfig = new SegmentGeneratorConfig(TABLE_CONFIG, SCHEMA);
+ segmentGeneratorConfig.setTableName(RAW_TABLE_NAME);
+ segmentGeneratorConfig.setSegmentName(SEGMENT_NAME);
+ segmentGeneratorConfig.setOutDir(INDEX_DIR.getPath());
+
+ SegmentIndexCreationDriverImpl driver = new SegmentIndexCreationDriverImpl();
+ driver.init(segmentGeneratorConfig, new GenericRowRecordReader(records));
+ driver.build();
+
+ IndexLoadingConfig indexLoadingConfig = new IndexLoadingConfig();
+ indexLoadingConfig.setTableConfig(TABLE_CONFIG);
+ indexLoadingConfig.setInvertedIndexColumns(new HashSet<>(invertedIndexColumns));
+ indexLoadingConfig.setRangeIndexColumns(new HashSet<>(rangeIndexColumns));
+ indexLoadingConfig.setJsonIndexColumns(new HashSet<String>(jsonIndexColumns));
+ indexLoadingConfig.setReadMode(ReadMode.mmap);
+
+ ImmutableSegment immutableSegment =
+ ImmutableSegmentLoader.load(new File(INDEX_DIR, SEGMENT_NAME), indexLoadingConfig);
+ _indexSegment = immutableSegment;
+ _indexSegments = Arrays.asList(immutableSegment);
+ }
+
+ /** Checks the correctness of EXPLAIN PLAN output. */
+ private void check(String query, ResultTable expected) {
+ BrokerResponseNative response = getBrokerResponseForSqlQuery(query);
+ ResultTable actual = response.getResultTable();
+
+ System.out.println(resultTableToString("Expected", expected));
+ System.out.println(resultTableToString("Actual", actual));
+
+ Assert.assertEquals(actual.getDataSchema(), expected.getDataSchema());
+ Assert.assertEquals(actual.getRows().size(), expected.getRows().size());
+
+ List<Object[]> expectedRows = expected.getRows();
+ List<Object[]> actualRows = actual.getRows();
+ for (int i = 0; i < expectedRows.size(); i++) {
+ Object[] expectedRow = expectedRows.get(i);
+ Object[] actualRow = actualRows.get(i);
+ Assert.assertEquals(actualRow.length, expectedRow.length);
+
+ for (int j = 0; j < actualRow.length; j++) {
+ Assert.assertEquals(actualRow[j], expectedRow[j]);
+ }
+ }
+ }
+
+ /** This function is used to generated expected results while creating test cases; otherwise, not used. */
+ private static String resultTableToString(String tag, ResultTable table) {
+ System.out.println("\n" + tag + ":");
+ List<Object[]> rows = table.getRows();
+
+ StringBuffer buffer = new StringBuffer();
+ for (Object[] row : rows) {
+ buffer.append("result.add(new Object[]{\"" + row[0] + "\", " + row[1] + ", " + row[2] + "});\n");
+ }
+
+ return buffer.toString();
+ }
+
+ @Test
+ public void testSelect() {
+ String query1 = "EXPLAIN PLAN FOR SELECT * FROM testTable";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(
+ new Object[]{"SELECT(selectList:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, jsonIndexCol1, "
+ + "noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, "
+ + "jsonIndexCol1, noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 3, 2});
+ result1.add(
+ new Object[]{"PROJECT(sortedIndexCol1, noIndexCol3, rangeIndexCol1, rangeIndexCol2, jsonIndexCol1, "
+ + "invertedIndexCol1, noIndexCol2, invertedIndexCol2, noIndexCol1, invertedIndexCol3, rangeIndexCol3, "
+ + "textIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 = "EXPLAIN PLAN FOR SELECT 'mickey' FROM testTable";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:'mickey')", 2, 1});
+ result2.add(new Object[]{"PROJECT(ALL)", 3, 2});
+ result2.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 4, 3});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+
+ String query3 = "EXPLAIN PLAN FOR SELECT invertedIndexCol1, noIndexCol1 FROM testTable LIMIT 100";
+ List<Object[]> result3 = new ArrayList<>();
+ result3.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result3.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result3.add(new Object[]{"SELECT(selectList:invertedIndexCol1, noIndexCol1)", 2, 1});
+ result3.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result3.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result3.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query3, new ResultTable(DATA_SCHEMA, result3));
+
+ String query4 = "EXPLAIN PLAN FOR SELECT DISTINCT invertedIndexCol1, noIndexCol1 FROM testTable LIMIT 100";
+ List<Object[]> result4 = new ArrayList<>();
+ result4.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result4.add(new Object[]{"COMBINE_DISTINCT", 1, 0});
+ result4.add(new Object[]{"DISTINCT(keyColumns:invertedIndexCol1, noIndexCol1)", 2, 1});
+ result4.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result4.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result4.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query4, new ResultTable(DATA_SCHEMA, result4));
+ }
+
+ @Test
+ public void testSelectColumnsUsingFilterIndex() {
+ String query1 = "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1, sortedIndexCol1 FROM testTable WHERE "
+ + "invertedIndexCol1 = 1.5 AND sortedIndexCol1 = 1 LIMIT 100";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1, sortedIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, sortedIndexCol1)", 3, 2});
+ result1.add(new Object[]{"PROJECT(sortedIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 = "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1, sortedIndexCol1 FROM testTable WHERE "
+ + "(invertedIndexCol1 = 1.5 AND sortedIndexCol1 = 1) OR (invertedIndexCol1 BETWEEN 100 and 200 AND "
+ + "rangeIndexCol1 > 20)";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1, sortedIndexCol1)", 2, 1});
+ result2.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, sortedIndexCol1)", 3, 2});
+ result2.add(new Object[]{"PROJECT(sortedIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result2.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+
+ String query3 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1 FROM testTable WHERE invertedIndexCol1 = 1.5 AND "
+ + "invertedIndexCol2 IN (10, 20, 30) AND invertedIndexCol3 NOT IN ('foo', 'bar') LIMIT 100";
+ List<Object[]> result3 = new ArrayList<>();
+ result3.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result3.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result3.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1)", 2, 1});
+ result3.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result3.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result3.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query3, new ResultTable(DATA_SCHEMA, result3));
+
+ String query4 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, noIndexCol2, sortedIndexCol1 FROM testTable WHERE sortedIndexCol1 = 1.5"
+ + " OR sortedIndexCol1 != 5 OR sortedIndexCol1 IN (10, 20, 30) LIMIT 100";
+ List<Object[]> result4 = new ArrayList<>();
+ result4.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result4.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result4.add(new Object[]{"SELECT(selectList:noIndexCol1, noIndexCol2, sortedIndexCol1)", 2, 1});
+ result4.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2, sortedIndexCol1)", 3, 2});
+ result4.add(new Object[]{"PROJECT(sortedIndexCol1, noIndexCol2, noIndexCol1)", 4, 3});
+ result4.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query4, new ResultTable(DATA_SCHEMA, result4));
+
+ String query5 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, noIndexCol2 FROM testTable WHERE DIV(noIndexCol1, noIndexCol2) BETWEEN "
+ + "10 AND 20 AND invertedIndexCol1 * 5 < 1000";
+ List<Object[]> result5 = new ArrayList<>();
+ result5.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result5.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result5.add(new Object[]{"SELECT(selectList:noIndexCol1, noIndexCol2)", 2, 1});
+ result5.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2)", 3, 2});
+ result5.add(new Object[]{"PROJECT(noIndexCol2, noIndexCol1)", 4, 3});
+ result5.add(new Object[]{"FILTER_AND", 5, 4});
+ result5.add(
+ new Object[]{"FILTER_EXPRESSION(operator:RANGE,predicate:div(noIndexCol1,noIndexCol2) BETWEEN '10' AND '20')",
+ 6, 5});
+ result5
+ .add(new Object[]{"FILTER_EXPRESSION(operator:RANGE,predicate:times(invertedIndexCol1,'5') < '1000')", 7, 5});
+ check(query5, new ResultTable(DATA_SCHEMA, result5));
+ }
+
+ @Test
+ public void testSelectColumnUsingFilterRange() {
+ // select * query triggering range index
+ // checks using RANGE (>, >=, <, <=, BETWEEN ..) on a column with range index should use RANGE_INDEX_SCAN
+ String query1 =
+ "EXPLAIN PLAN FOR SELECT invertedIndexCol1, noIndexCol1, rangeIndexCol1 FROM testTable WHERE rangeIndexCol1 >"
+ + " 10 AND rangeIndexCol2 >= 20 AND rangeIndexCol3 BETWEEN 50 AND 60 LIMIT 100";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(new Object[]{"SELECT(selectList:invertedIndexCol1, noIndexCol1, rangeIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, rangeIndexCol1)", 3, 2});
+ result1.add(new Object[]{"PROJECT(rangeIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1 FROM testTable WHERE noIndexCol1 > 10 AND noIndexCol2"
+ + " BETWEEN 50 AND 60 LIMIT 100";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1)", 2, 1});
+ result2.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result2.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result2.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+ }
+
+ @Test
+ public void testSelectColumnUsingFilterJson() {
+ String query =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1 FROM testTable WHERE (invertedIndexCol1 IN (10, 20, "
+ + "30) AND sortedIndexCol1 != 100) OR (noIndexCol1 NOT IN (10, 20, 30) AND rangeIndexCol1 != 20 AND "
+ + "JSON_MATCH(jsonIndexCol1, 'key=1') AND TEXT_MATCH(textIndexCol1, 'foo'))";
+ List<Object[]> result = new ArrayList<>();
+ result.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1)", 2, 1});
+ result.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result.add(new Object[]{"FILTER_AND", 5, 4});
+ result.add(new Object[]{"FILTER_JSON_INDEX(indexLookUp:json_index,operator:JSON_MATCH,predicate:json_match"
+ + "(jsonIndexCol1,'key=1'))", 6, 5});
+ result.add(new Object[]{"FILTER_TEXT_INDEX(indexLookUp:text_index,operator:TEXT_MATCH,predicate:text_match"
+ + "(textIndexCol1,'foo'))", 7, 5});
+ check(query, new ResultTable(DATA_SCHEMA, result));
+ }
+
+ @Test
+ public void testSelectAggregateUsingFilterText() {
+ String query1 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, noIndexCol2, max(noIndexCol2), min(noIndexCol3) FROM testTable WHERE "
+ + "TEXT_MATCH(textIndexCol1, 'foo') GROUP BY noIndexCol1, noIndexCol2";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result1.add(new Object[]{"COMBINE_GROUPBY_ORDERBY", 1, 0});
+ result1.add(new Object[]{
+ "AGGREGATE_GROUPBY_ORDERBY(groupKeys:noIndexCol1, noIndexCol2, aggregations:max(noIndexCol2), min"
+ + "(noIndexCol3))", 2, 1});
+ result1.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2, noIndexCol3)", 3, 2});
+ result1.add(new Object[]{"PROJECT(noIndexCol3, noIndexCol2, noIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_TEXT_INDEX(indexLookUp:text_index,operator:TEXT_MATCH,predicate:text_match"
+ + "(textIndexCol1,'foo'))", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, max(noIndexCol2), min(noIndexCol3) FROM testTable WHERE TEXT_MATCH"
+ + "(textIndexCol1, 'foo') GROUP BY noIndexCol1, noIndexCol2 ORDER BY noIndexCol1, max(noIndexCol2)";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(sort:[noIndexCol1 ASC, max(noIndexCol2) ASC],limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_GROUPBY_ORDERBY", 1, 0});
+ result2.add(new Object[]{
+ "AGGREGATE_GROUPBY_ORDERBY(groupKeys:noIndexCol1,noIndexCol2,aggregations:max(noIndexCol2), min"
+ + "(noIndexCol3))", 2, 1});
+ result2.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, invertedIndexCol1)", 3, 2});
+ result2.add(new Object[]{"PROJECT(noIndexCol3, noIndexCol2, noIndexCol1)", 4, 3});
+ result2.add(new Object[]{"FILTER_TEXT_INDEX(indexLookUp:text_index,operator:TEXT_MATCH,predicate:text_match"
+ + "(textIndexCol1,'foo'))", 5, 4});
+ }
+
+ @Test
+ public void testSelectAggregate() {
+ String query1 = "EXPLAIN PLAN FOR SELECT count(*) FROM testTable";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result1.add(new Object[]{"COMBINE_AGGREGATE", 1, 0});
+ result1.add(new Object[]{"AGGREGATE_METADATA", 2, 1});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 =
+ "EXPLAIN PLAN FOR SELECT count(*), max(noIndexCol1), sum(noIndexCol2), avg(noIndexCol2) FROM testTable";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_AGGREGATE", 1, 0});
+ result2.add(
+ new Object[]{"AGGREGATE(aggregations:count(*), max(noIndexCol1), sum(noIndexCol2), avg(noIndexCol2))", 2, 1});
+ result2.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2)", 3, 2});
+ result2.add(new Object[]{"PROJECT(noIndexCol2, noIndexCol1)", 4, 3});
+ result2.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+ }
+
+ @Test
+ public void testSelectAggregateUsingFilter() {
+ String query = "EXPLAIN PLAN FOR SELECT sum(add(noIndexCol1, noIndexCol2)), MIN(ADD(DIV(noIndexCol1,noIndexCol2),"
+ + "noIndexCol3)) FROM testTable";
+ List<Object[]> result = new ArrayList<>();
Review comment:
Moved test case to testSelectAggregate.
##########
File path: pinot-core/src/test/java/org/apache/pinot/queries/ExplainPlanQueriesTest.java
##########
@@ -0,0 +1,510 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.queries;
+
+import java.io.File;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.List;
+import org.apache.commons.io.FileUtils;
+import org.apache.pinot.common.response.broker.BrokerResponseNative;
+import org.apache.pinot.common.response.broker.ResultTable;
+import org.apache.pinot.common.utils.DataSchema;
+import org.apache.pinot.segment.local.indexsegment.immutable.ImmutableSegmentLoader;
+import org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl;
+import org.apache.pinot.segment.local.segment.index.loader.IndexLoadingConfig;
+import org.apache.pinot.segment.local.segment.readers.GenericRowRecordReader;
+import org.apache.pinot.segment.spi.ImmutableSegment;
+import org.apache.pinot.segment.spi.IndexSegment;
+import org.apache.pinot.segment.spi.creator.SegmentGeneratorConfig;
+import org.apache.pinot.spi.config.table.IndexingConfig;
+import org.apache.pinot.spi.config.table.TableConfig;
+import org.apache.pinot.spi.config.table.TableType;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.data.Schema;
+import org.apache.pinot.spi.data.readers.GenericRow;
+import org.apache.pinot.spi.utils.ReadMode;
+import org.apache.pinot.spi.utils.builder.TableConfigBuilder;
+import org.testng.Assert;
+import org.testng.annotations.BeforeClass;
+import org.testng.annotations.Test;
+
+
+public class ExplainPlanQueriesTest extends BaseQueriesTest {
+ private static final File INDEX_DIR = new File(FileUtils.getTempDirectory(), "ExplainPlanQueriesTest");
+ private static final String RAW_TABLE_NAME = "testTable";
+ private static final String SEGMENT_NAME = "testSegment";
+ private static final int NUM_RECORDS = 10;
+
+ private final static String COL1_NO_INDEX = "noIndexCol1";
+ private final static String COL2_NO_INDEX = "noIndexCol2";
+ private final static String COL3_NO_INDEX = "noIndexCol3";
+ private final static String COL1_INVERTED_INDEX = "invertedIndexCol1";
+ private final static String COL2_INVERTED_INDEX = "invertedIndexCol2";
+ private final static String COL3_INVERTED_INDEX = "invertedIndexCol3";
+ private final static String COL1_RANGE_INDEX = "rangeIndexCol1";
+ private final static String COL2_RANGE_INDEX = "rangeIndexCol2";
+ private final static String COL3_RANGE_INDEX = "rangeIndexCol3";
+ private final static String COL1_SORTED_INDEX = "sortedIndexCol1";
+ private final static String COL1_JSON_INDEX = "jsonIndexCol1";
+ private final static String COL1_TEXT_INDEX = "textIndexCol1";
+
+ private static final Schema SCHEMA = new Schema.SchemaBuilder().setSchemaName(RAW_TABLE_NAME)
+ .addSingleValueDimension(COL1_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL2_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_INVERTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_INVERTED_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_INVERTED_INDEX, FieldSpec.DataType.STRING)
+ .addSingleValueDimension(COL1_RANGE_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_SORTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL1_JSON_INDEX, FieldSpec.DataType.JSON)
+ .addSingleValueDimension(COL1_TEXT_INDEX, FieldSpec.DataType.STRING).build();
+
+ private static final DataSchema DATA_SCHEMA = new DataSchema(new String[]{"Operator", "Operator_Id", "Parent_Id"},
+ new DataSchema.ColumnDataType[]{DataSchema.ColumnDataType.STRING, DataSchema.ColumnDataType.INT,
+ DataSchema.ColumnDataType.INT});
+
+ private static final TableConfig TABLE_CONFIG =
+ new TableConfigBuilder(TableType.OFFLINE).setTableName(RAW_TABLE_NAME).build();
+
+ private IndexSegment _indexSegment;
+ private List<IndexSegment> _indexSegments;
+
+ @Override
+ protected String getFilter() {
+ return "";
+ }
+
+ @Override
+ protected IndexSegment getIndexSegment() {
+ return _indexSegment;
+ }
+
+ @Override
+ protected List<IndexSegment> getIndexSegments() {
+ return _indexSegments;
+ }
+
+ GenericRow createRecord(double doubleValue, int intValue, String stringValue, String jsonValue) {
+ GenericRow record = new GenericRow();
+ record.putValue(COL1_NO_INDEX, intValue);
+ record.putValue(COL2_NO_INDEX, intValue);
+ record.putValue(COL3_NO_INDEX, intValue);
+
+ record.putValue(COL1_INVERTED_INDEX, doubleValue);
+ record.putValue(COL2_INVERTED_INDEX, intValue);
+ record.putValue(COL3_INVERTED_INDEX, intValue);
+
+ record.putValue(COL1_RANGE_INDEX, doubleValue);
+ record.putValue(COL2_RANGE_INDEX, intValue);
+ record.putValue(COL3_RANGE_INDEX, intValue);
+
+ record.putValue(COL1_JSON_INDEX, jsonValue);
+ record.putValue(COL1_TEXT_INDEX, stringValue);
+
+ return record;
+ }
+
+ @BeforeClass
+ public void setUp()
+ throws Exception {
+ FileUtils.deleteDirectory(INDEX_DIR);
+
+ List<GenericRow> records = new ArrayList<>(NUM_RECORDS);
+ records.add(createRecord(1.1, 2, "daffy", "{\"first\": \"daffy\", \"last\": \"duck\"}"));
+
+ IndexingConfig indexingConfig = TABLE_CONFIG.getIndexingConfig();
+
+ List<String> invertedIndexColumns = Arrays.asList(COL1_INVERTED_INDEX, COL2_INVERTED_INDEX, COL3_INVERTED_INDEX);
+ indexingConfig.setInvertedIndexColumns(invertedIndexColumns);
+
+ List<String> rangeIndexColumns = Arrays.asList(COL1_RANGE_INDEX, COL2_RANGE_INDEX, COL3_RANGE_INDEX);
+ indexingConfig.setRangeIndexColumns(rangeIndexColumns);
+
+ List<String> jsonIndexColumns = Arrays.asList(COL1_JSON_INDEX);
+ indexingConfig.setJsonIndexColumns(jsonIndexColumns);
+
+ SegmentGeneratorConfig segmentGeneratorConfig = new SegmentGeneratorConfig(TABLE_CONFIG, SCHEMA);
+ segmentGeneratorConfig.setTableName(RAW_TABLE_NAME);
+ segmentGeneratorConfig.setSegmentName(SEGMENT_NAME);
+ segmentGeneratorConfig.setOutDir(INDEX_DIR.getPath());
+
+ SegmentIndexCreationDriverImpl driver = new SegmentIndexCreationDriverImpl();
+ driver.init(segmentGeneratorConfig, new GenericRowRecordReader(records));
+ driver.build();
+
+ IndexLoadingConfig indexLoadingConfig = new IndexLoadingConfig();
+ indexLoadingConfig.setTableConfig(TABLE_CONFIG);
+ indexLoadingConfig.setInvertedIndexColumns(new HashSet<>(invertedIndexColumns));
+ indexLoadingConfig.setRangeIndexColumns(new HashSet<>(rangeIndexColumns));
+ indexLoadingConfig.setJsonIndexColumns(new HashSet<String>(jsonIndexColumns));
+ indexLoadingConfig.setReadMode(ReadMode.mmap);
+
+ ImmutableSegment immutableSegment =
+ ImmutableSegmentLoader.load(new File(INDEX_DIR, SEGMENT_NAME), indexLoadingConfig);
+ _indexSegment = immutableSegment;
+ _indexSegments = Arrays.asList(immutableSegment);
+ }
+
+ /** Checks the correctness of EXPLAIN PLAN output. */
+ private void check(String query, ResultTable expected) {
+ BrokerResponseNative response = getBrokerResponseForSqlQuery(query);
+ ResultTable actual = response.getResultTable();
+
+ System.out.println(resultTableToString("Expected", expected));
+ System.out.println(resultTableToString("Actual", actual));
+
+ Assert.assertEquals(actual.getDataSchema(), expected.getDataSchema());
+ Assert.assertEquals(actual.getRows().size(), expected.getRows().size());
+
+ List<Object[]> expectedRows = expected.getRows();
+ List<Object[]> actualRows = actual.getRows();
+ for (int i = 0; i < expectedRows.size(); i++) {
+ Object[] expectedRow = expectedRows.get(i);
+ Object[] actualRow = actualRows.get(i);
+ Assert.assertEquals(actualRow.length, expectedRow.length);
+
+ for (int j = 0; j < actualRow.length; j++) {
+ Assert.assertEquals(actualRow[j], expectedRow[j]);
+ }
+ }
+ }
+
+ /** This function is used to generated expected results while creating test cases; otherwise, not used. */
+ private static String resultTableToString(String tag, ResultTable table) {
+ System.out.println("\n" + tag + ":");
+ List<Object[]> rows = table.getRows();
+
+ StringBuffer buffer = new StringBuffer();
+ for (Object[] row : rows) {
+ buffer.append("result.add(new Object[]{\"" + row[0] + "\", " + row[1] + ", " + row[2] + "});\n");
+ }
+
+ return buffer.toString();
+ }
+
+ @Test
+ public void testSelect() {
+ String query1 = "EXPLAIN PLAN FOR SELECT * FROM testTable";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(
+ new Object[]{"SELECT(selectList:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, jsonIndexCol1, "
+ + "noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, "
+ + "jsonIndexCol1, noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 3, 2});
+ result1.add(
+ new Object[]{"PROJECT(sortedIndexCol1, noIndexCol3, rangeIndexCol1, rangeIndexCol2, jsonIndexCol1, "
+ + "invertedIndexCol1, noIndexCol2, invertedIndexCol2, noIndexCol1, invertedIndexCol3, rangeIndexCol3, "
+ + "textIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 = "EXPLAIN PLAN FOR SELECT 'mickey' FROM testTable";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:'mickey')", 2, 1});
+ result2.add(new Object[]{"PROJECT(ALL)", 3, 2});
+ result2.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 4, 3});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+
+ String query3 = "EXPLAIN PLAN FOR SELECT invertedIndexCol1, noIndexCol1 FROM testTable LIMIT 100";
+ List<Object[]> result3 = new ArrayList<>();
+ result3.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result3.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result3.add(new Object[]{"SELECT(selectList:invertedIndexCol1, noIndexCol1)", 2, 1});
+ result3.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result3.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result3.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query3, new ResultTable(DATA_SCHEMA, result3));
+
+ String query4 = "EXPLAIN PLAN FOR SELECT DISTINCT invertedIndexCol1, noIndexCol1 FROM testTable LIMIT 100";
+ List<Object[]> result4 = new ArrayList<>();
+ result4.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result4.add(new Object[]{"COMBINE_DISTINCT", 1, 0});
+ result4.add(new Object[]{"DISTINCT(keyColumns:invertedIndexCol1, noIndexCol1)", 2, 1});
+ result4.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result4.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result4.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query4, new ResultTable(DATA_SCHEMA, result4));
+ }
+
+ @Test
+ public void testSelectColumnsUsingFilterIndex() {
+ String query1 = "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1, sortedIndexCol1 FROM testTable WHERE "
+ + "invertedIndexCol1 = 1.5 AND sortedIndexCol1 = 1 LIMIT 100";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1, sortedIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, sortedIndexCol1)", 3, 2});
+ result1.add(new Object[]{"PROJECT(sortedIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 = "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1, sortedIndexCol1 FROM testTable WHERE "
+ + "(invertedIndexCol1 = 1.5 AND sortedIndexCol1 = 1) OR (invertedIndexCol1 BETWEEN 100 and 200 AND "
+ + "rangeIndexCol1 > 20)";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1, sortedIndexCol1)", 2, 1});
+ result2.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, sortedIndexCol1)", 3, 2});
+ result2.add(new Object[]{"PROJECT(sortedIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result2.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+
+ String query3 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1 FROM testTable WHERE invertedIndexCol1 = 1.5 AND "
+ + "invertedIndexCol2 IN (10, 20, 30) AND invertedIndexCol3 NOT IN ('foo', 'bar') LIMIT 100";
+ List<Object[]> result3 = new ArrayList<>();
+ result3.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result3.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result3.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1)", 2, 1});
+ result3.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result3.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result3.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query3, new ResultTable(DATA_SCHEMA, result3));
+
+ String query4 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, noIndexCol2, sortedIndexCol1 FROM testTable WHERE sortedIndexCol1 = 1.5"
+ + " OR sortedIndexCol1 != 5 OR sortedIndexCol1 IN (10, 20, 30) LIMIT 100";
+ List<Object[]> result4 = new ArrayList<>();
+ result4.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result4.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result4.add(new Object[]{"SELECT(selectList:noIndexCol1, noIndexCol2, sortedIndexCol1)", 2, 1});
+ result4.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2, sortedIndexCol1)", 3, 2});
+ result4.add(new Object[]{"PROJECT(sortedIndexCol1, noIndexCol2, noIndexCol1)", 4, 3});
+ result4.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query4, new ResultTable(DATA_SCHEMA, result4));
+
+ String query5 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, noIndexCol2 FROM testTable WHERE DIV(noIndexCol1, noIndexCol2) BETWEEN "
+ + "10 AND 20 AND invertedIndexCol1 * 5 < 1000";
+ List<Object[]> result5 = new ArrayList<>();
+ result5.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result5.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result5.add(new Object[]{"SELECT(selectList:noIndexCol1, noIndexCol2)", 2, 1});
+ result5.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2)", 3, 2});
+ result5.add(new Object[]{"PROJECT(noIndexCol2, noIndexCol1)", 4, 3});
+ result5.add(new Object[]{"FILTER_AND", 5, 4});
+ result5.add(
+ new Object[]{"FILTER_EXPRESSION(operator:RANGE,predicate:div(noIndexCol1,noIndexCol2) BETWEEN '10' AND '20')",
+ 6, 5});
+ result5
+ .add(new Object[]{"FILTER_EXPRESSION(operator:RANGE,predicate:times(invertedIndexCol1,'5') < '1000')", 7, 5});
+ check(query5, new ResultTable(DATA_SCHEMA, result5));
+ }
+
+ @Test
+ public void testSelectColumnUsingFilterRange() {
+ // select * query triggering range index
+ // checks using RANGE (>, >=, <, <=, BETWEEN ..) on a column with range index should use RANGE_INDEX_SCAN
+ String query1 =
+ "EXPLAIN PLAN FOR SELECT invertedIndexCol1, noIndexCol1, rangeIndexCol1 FROM testTable WHERE rangeIndexCol1 >"
+ + " 10 AND rangeIndexCol2 >= 20 AND rangeIndexCol3 BETWEEN 50 AND 60 LIMIT 100";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(new Object[]{"SELECT(selectList:invertedIndexCol1, noIndexCol1, rangeIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, rangeIndexCol1)", 3, 2});
+ result1.add(new Object[]{"PROJECT(rangeIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1 FROM testTable WHERE noIndexCol1 > 10 AND noIndexCol2"
+ + " BETWEEN 50 AND 60 LIMIT 100";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1)", 2, 1});
+ result2.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result2.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result2.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+ }
+
+ @Test
+ public void testSelectColumnUsingFilterJson() {
+ String query =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1 FROM testTable WHERE (invertedIndexCol1 IN (10, 20, "
+ + "30) AND sortedIndexCol1 != 100) OR (noIndexCol1 NOT IN (10, 20, 30) AND rangeIndexCol1 != 20 AND "
+ + "JSON_MATCH(jsonIndexCol1, 'key=1') AND TEXT_MATCH(textIndexCol1, 'foo'))";
+ List<Object[]> result = new ArrayList<>();
+ result.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1)", 2, 1});
+ result.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result.add(new Object[]{"FILTER_AND", 5, 4});
Review comment:
Fixed.
##########
File path: pinot-core/src/test/java/org/apache/pinot/queries/ExplainPlanQueriesTest.java
##########
@@ -0,0 +1,510 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.queries;
+
+import java.io.File;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.List;
+import org.apache.commons.io.FileUtils;
+import org.apache.pinot.common.response.broker.BrokerResponseNative;
+import org.apache.pinot.common.response.broker.ResultTable;
+import org.apache.pinot.common.utils.DataSchema;
+import org.apache.pinot.segment.local.indexsegment.immutable.ImmutableSegmentLoader;
+import org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl;
+import org.apache.pinot.segment.local.segment.index.loader.IndexLoadingConfig;
+import org.apache.pinot.segment.local.segment.readers.GenericRowRecordReader;
+import org.apache.pinot.segment.spi.ImmutableSegment;
+import org.apache.pinot.segment.spi.IndexSegment;
+import org.apache.pinot.segment.spi.creator.SegmentGeneratorConfig;
+import org.apache.pinot.spi.config.table.IndexingConfig;
+import org.apache.pinot.spi.config.table.TableConfig;
+import org.apache.pinot.spi.config.table.TableType;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.data.Schema;
+import org.apache.pinot.spi.data.readers.GenericRow;
+import org.apache.pinot.spi.utils.ReadMode;
+import org.apache.pinot.spi.utils.builder.TableConfigBuilder;
+import org.testng.Assert;
+import org.testng.annotations.BeforeClass;
+import org.testng.annotations.Test;
+
+
+public class ExplainPlanQueriesTest extends BaseQueriesTest {
+ private static final File INDEX_DIR = new File(FileUtils.getTempDirectory(), "ExplainPlanQueriesTest");
+ private static final String RAW_TABLE_NAME = "testTable";
+ private static final String SEGMENT_NAME = "testSegment";
+ private static final int NUM_RECORDS = 10;
+
+ private final static String COL1_NO_INDEX = "noIndexCol1";
+ private final static String COL2_NO_INDEX = "noIndexCol2";
+ private final static String COL3_NO_INDEX = "noIndexCol3";
+ private final static String COL1_INVERTED_INDEX = "invertedIndexCol1";
+ private final static String COL2_INVERTED_INDEX = "invertedIndexCol2";
+ private final static String COL3_INVERTED_INDEX = "invertedIndexCol3";
+ private final static String COL1_RANGE_INDEX = "rangeIndexCol1";
+ private final static String COL2_RANGE_INDEX = "rangeIndexCol2";
+ private final static String COL3_RANGE_INDEX = "rangeIndexCol3";
+ private final static String COL1_SORTED_INDEX = "sortedIndexCol1";
+ private final static String COL1_JSON_INDEX = "jsonIndexCol1";
+ private final static String COL1_TEXT_INDEX = "textIndexCol1";
+
+ private static final Schema SCHEMA = new Schema.SchemaBuilder().setSchemaName(RAW_TABLE_NAME)
+ .addSingleValueDimension(COL1_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL2_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_INVERTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_INVERTED_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_INVERTED_INDEX, FieldSpec.DataType.STRING)
+ .addSingleValueDimension(COL1_RANGE_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_SORTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL1_JSON_INDEX, FieldSpec.DataType.JSON)
+ .addSingleValueDimension(COL1_TEXT_INDEX, FieldSpec.DataType.STRING).build();
+
+ private static final DataSchema DATA_SCHEMA = new DataSchema(new String[]{"Operator", "Operator_Id", "Parent_Id"},
+ new DataSchema.ColumnDataType[]{DataSchema.ColumnDataType.STRING, DataSchema.ColumnDataType.INT,
+ DataSchema.ColumnDataType.INT});
+
+ private static final TableConfig TABLE_CONFIG =
+ new TableConfigBuilder(TableType.OFFLINE).setTableName(RAW_TABLE_NAME).build();
+
+ private IndexSegment _indexSegment;
+ private List<IndexSegment> _indexSegments;
+
+ @Override
+ protected String getFilter() {
+ return "";
+ }
+
+ @Override
+ protected IndexSegment getIndexSegment() {
+ return _indexSegment;
+ }
+
+ @Override
+ protected List<IndexSegment> getIndexSegments() {
+ return _indexSegments;
+ }
+
+ GenericRow createRecord(double doubleValue, int intValue, String stringValue, String jsonValue) {
+ GenericRow record = new GenericRow();
+ record.putValue(COL1_NO_INDEX, intValue);
+ record.putValue(COL2_NO_INDEX, intValue);
+ record.putValue(COL3_NO_INDEX, intValue);
+
+ record.putValue(COL1_INVERTED_INDEX, doubleValue);
+ record.putValue(COL2_INVERTED_INDEX, intValue);
+ record.putValue(COL3_INVERTED_INDEX, intValue);
+
+ record.putValue(COL1_RANGE_INDEX, doubleValue);
+ record.putValue(COL2_RANGE_INDEX, intValue);
+ record.putValue(COL3_RANGE_INDEX, intValue);
+
+ record.putValue(COL1_JSON_INDEX, jsonValue);
+ record.putValue(COL1_TEXT_INDEX, stringValue);
+
+ return record;
+ }
+
+ @BeforeClass
+ public void setUp()
+ throws Exception {
+ FileUtils.deleteDirectory(INDEX_DIR);
+
+ List<GenericRow> records = new ArrayList<>(NUM_RECORDS);
+ records.add(createRecord(1.1, 2, "daffy", "{\"first\": \"daffy\", \"last\": \"duck\"}"));
+
+ IndexingConfig indexingConfig = TABLE_CONFIG.getIndexingConfig();
+
+ List<String> invertedIndexColumns = Arrays.asList(COL1_INVERTED_INDEX, COL2_INVERTED_INDEX, COL3_INVERTED_INDEX);
+ indexingConfig.setInvertedIndexColumns(invertedIndexColumns);
+
+ List<String> rangeIndexColumns = Arrays.asList(COL1_RANGE_INDEX, COL2_RANGE_INDEX, COL3_RANGE_INDEX);
+ indexingConfig.setRangeIndexColumns(rangeIndexColumns);
+
+ List<String> jsonIndexColumns = Arrays.asList(COL1_JSON_INDEX);
+ indexingConfig.setJsonIndexColumns(jsonIndexColumns);
+
+ SegmentGeneratorConfig segmentGeneratorConfig = new SegmentGeneratorConfig(TABLE_CONFIG, SCHEMA);
+ segmentGeneratorConfig.setTableName(RAW_TABLE_NAME);
+ segmentGeneratorConfig.setSegmentName(SEGMENT_NAME);
+ segmentGeneratorConfig.setOutDir(INDEX_DIR.getPath());
+
+ SegmentIndexCreationDriverImpl driver = new SegmentIndexCreationDriverImpl();
+ driver.init(segmentGeneratorConfig, new GenericRowRecordReader(records));
+ driver.build();
+
+ IndexLoadingConfig indexLoadingConfig = new IndexLoadingConfig();
+ indexLoadingConfig.setTableConfig(TABLE_CONFIG);
+ indexLoadingConfig.setInvertedIndexColumns(new HashSet<>(invertedIndexColumns));
+ indexLoadingConfig.setRangeIndexColumns(new HashSet<>(rangeIndexColumns));
+ indexLoadingConfig.setJsonIndexColumns(new HashSet<String>(jsonIndexColumns));
+ indexLoadingConfig.setReadMode(ReadMode.mmap);
+
+ ImmutableSegment immutableSegment =
+ ImmutableSegmentLoader.load(new File(INDEX_DIR, SEGMENT_NAME), indexLoadingConfig);
+ _indexSegment = immutableSegment;
+ _indexSegments = Arrays.asList(immutableSegment);
+ }
+
+ /** Checks the correctness of EXPLAIN PLAN output. */
+ private void check(String query, ResultTable expected) {
+ BrokerResponseNative response = getBrokerResponseForSqlQuery(query);
+ ResultTable actual = response.getResultTable();
+
+ System.out.println(resultTableToString("Expected", expected));
+ System.out.println(resultTableToString("Actual", actual));
+
+ Assert.assertEquals(actual.getDataSchema(), expected.getDataSchema());
+ Assert.assertEquals(actual.getRows().size(), expected.getRows().size());
+
+ List<Object[]> expectedRows = expected.getRows();
+ List<Object[]> actualRows = actual.getRows();
+ for (int i = 0; i < expectedRows.size(); i++) {
+ Object[] expectedRow = expectedRows.get(i);
+ Object[] actualRow = actualRows.get(i);
+ Assert.assertEquals(actualRow.length, expectedRow.length);
+
+ for (int j = 0; j < actualRow.length; j++) {
+ Assert.assertEquals(actualRow[j], expectedRow[j]);
+ }
+ }
+ }
+
+ /** This function is used to generated expected results while creating test cases; otherwise, not used. */
+ private static String resultTableToString(String tag, ResultTable table) {
+ System.out.println("\n" + tag + ":");
+ List<Object[]> rows = table.getRows();
+
+ StringBuffer buffer = new StringBuffer();
+ for (Object[] row : rows) {
+ buffer.append("result.add(new Object[]{\"" + row[0] + "\", " + row[1] + ", " + row[2] + "});\n");
+ }
+
+ return buffer.toString();
+ }
+
+ @Test
+ public void testSelect() {
+ String query1 = "EXPLAIN PLAN FOR SELECT * FROM testTable";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(
+ new Object[]{"SELECT(selectList:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, jsonIndexCol1, "
+ + "noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, "
+ + "jsonIndexCol1, noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 3, 2});
+ result1.add(
+ new Object[]{"PROJECT(sortedIndexCol1, noIndexCol3, rangeIndexCol1, rangeIndexCol2, jsonIndexCol1, "
+ + "invertedIndexCol1, noIndexCol2, invertedIndexCol2, noIndexCol1, invertedIndexCol3, rangeIndexCol3, "
+ + "textIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 = "EXPLAIN PLAN FOR SELECT 'mickey' FROM testTable";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:'mickey')", 2, 1});
+ result2.add(new Object[]{"PROJECT(ALL)", 3, 2});
+ result2.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 4, 3});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+
+ String query3 = "EXPLAIN PLAN FOR SELECT invertedIndexCol1, noIndexCol1 FROM testTable LIMIT 100";
+ List<Object[]> result3 = new ArrayList<>();
+ result3.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result3.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result3.add(new Object[]{"SELECT(selectList:invertedIndexCol1, noIndexCol1)", 2, 1});
+ result3.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result3.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result3.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query3, new ResultTable(DATA_SCHEMA, result3));
+
+ String query4 = "EXPLAIN PLAN FOR SELECT DISTINCT invertedIndexCol1, noIndexCol1 FROM testTable LIMIT 100";
+ List<Object[]> result4 = new ArrayList<>();
+ result4.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result4.add(new Object[]{"COMBINE_DISTINCT", 1, 0});
+ result4.add(new Object[]{"DISTINCT(keyColumns:invertedIndexCol1, noIndexCol1)", 2, 1});
+ result4.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result4.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result4.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query4, new ResultTable(DATA_SCHEMA, result4));
+ }
+
+ @Test
+ public void testSelectColumnsUsingFilterIndex() {
+ String query1 = "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1, sortedIndexCol1 FROM testTable WHERE "
+ + "invertedIndexCol1 = 1.5 AND sortedIndexCol1 = 1 LIMIT 100";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1, sortedIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, sortedIndexCol1)", 3, 2});
+ result1.add(new Object[]{"PROJECT(sortedIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 = "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1, sortedIndexCol1 FROM testTable WHERE "
+ + "(invertedIndexCol1 = 1.5 AND sortedIndexCol1 = 1) OR (invertedIndexCol1 BETWEEN 100 and 200 AND "
+ + "rangeIndexCol1 > 20)";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1, sortedIndexCol1)", 2, 1});
+ result2.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, sortedIndexCol1)", 3, 2});
+ result2.add(new Object[]{"PROJECT(sortedIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result2.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+
+ String query3 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1 FROM testTable WHERE invertedIndexCol1 = 1.5 AND "
+ + "invertedIndexCol2 IN (10, 20, 30) AND invertedIndexCol3 NOT IN ('foo', 'bar') LIMIT 100";
+ List<Object[]> result3 = new ArrayList<>();
+ result3.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result3.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result3.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1)", 2, 1});
+ result3.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result3.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result3.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query3, new ResultTable(DATA_SCHEMA, result3));
+
+ String query4 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, noIndexCol2, sortedIndexCol1 FROM testTable WHERE sortedIndexCol1 = 1.5"
+ + " OR sortedIndexCol1 != 5 OR sortedIndexCol1 IN (10, 20, 30) LIMIT 100";
+ List<Object[]> result4 = new ArrayList<>();
+ result4.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result4.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result4.add(new Object[]{"SELECT(selectList:noIndexCol1, noIndexCol2, sortedIndexCol1)", 2, 1});
+ result4.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2, sortedIndexCol1)", 3, 2});
+ result4.add(new Object[]{"PROJECT(sortedIndexCol1, noIndexCol2, noIndexCol1)", 4, 3});
+ result4.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query4, new ResultTable(DATA_SCHEMA, result4));
+
+ String query5 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, noIndexCol2 FROM testTable WHERE DIV(noIndexCol1, noIndexCol2) BETWEEN "
+ + "10 AND 20 AND invertedIndexCol1 * 5 < 1000";
+ List<Object[]> result5 = new ArrayList<>();
+ result5.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result5.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result5.add(new Object[]{"SELECT(selectList:noIndexCol1, noIndexCol2)", 2, 1});
+ result5.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2)", 3, 2});
+ result5.add(new Object[]{"PROJECT(noIndexCol2, noIndexCol1)", 4, 3});
+ result5.add(new Object[]{"FILTER_AND", 5, 4});
+ result5.add(
+ new Object[]{"FILTER_EXPRESSION(operator:RANGE,predicate:div(noIndexCol1,noIndexCol2) BETWEEN '10' AND '20')",
+ 6, 5});
+ result5
+ .add(new Object[]{"FILTER_EXPRESSION(operator:RANGE,predicate:times(invertedIndexCol1,'5') < '1000')", 7, 5});
+ check(query5, new ResultTable(DATA_SCHEMA, result5));
+ }
+
+ @Test
+ public void testSelectColumnUsingFilterRange() {
+ // select * query triggering range index
+ // checks using RANGE (>, >=, <, <=, BETWEEN ..) on a column with range index should use RANGE_INDEX_SCAN
+ String query1 =
+ "EXPLAIN PLAN FOR SELECT invertedIndexCol1, noIndexCol1, rangeIndexCol1 FROM testTable WHERE rangeIndexCol1 >"
+ + " 10 AND rangeIndexCol2 >= 20 AND rangeIndexCol3 BETWEEN 50 AND 60 LIMIT 100";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(new Object[]{"SELECT(selectList:invertedIndexCol1, noIndexCol1, rangeIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, rangeIndexCol1)", 3, 2});
+ result1.add(new Object[]{"PROJECT(rangeIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_EMPTY", 5, 4});
Review comment:
Done. I have left one instance of MATCH_ENTIRE_SEGMENT when index are being used.
##########
File path: pinot-core/src/test/java/org/apache/pinot/queries/ExplainPlanQueriesTest.java
##########
@@ -0,0 +1,510 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.queries;
+
+import java.io.File;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.List;
+import org.apache.commons.io.FileUtils;
+import org.apache.pinot.common.response.broker.BrokerResponseNative;
+import org.apache.pinot.common.response.broker.ResultTable;
+import org.apache.pinot.common.utils.DataSchema;
+import org.apache.pinot.segment.local.indexsegment.immutable.ImmutableSegmentLoader;
+import org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl;
+import org.apache.pinot.segment.local.segment.index.loader.IndexLoadingConfig;
+import org.apache.pinot.segment.local.segment.readers.GenericRowRecordReader;
+import org.apache.pinot.segment.spi.ImmutableSegment;
+import org.apache.pinot.segment.spi.IndexSegment;
+import org.apache.pinot.segment.spi.creator.SegmentGeneratorConfig;
+import org.apache.pinot.spi.config.table.IndexingConfig;
+import org.apache.pinot.spi.config.table.TableConfig;
+import org.apache.pinot.spi.config.table.TableType;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.data.Schema;
+import org.apache.pinot.spi.data.readers.GenericRow;
+import org.apache.pinot.spi.utils.ReadMode;
+import org.apache.pinot.spi.utils.builder.TableConfigBuilder;
+import org.testng.Assert;
+import org.testng.annotations.BeforeClass;
+import org.testng.annotations.Test;
+
+
+public class ExplainPlanQueriesTest extends BaseQueriesTest {
+ private static final File INDEX_DIR = new File(FileUtils.getTempDirectory(), "ExplainPlanQueriesTest");
+ private static final String RAW_TABLE_NAME = "testTable";
+ private static final String SEGMENT_NAME = "testSegment";
+ private static final int NUM_RECORDS = 10;
+
+ private final static String COL1_NO_INDEX = "noIndexCol1";
+ private final static String COL2_NO_INDEX = "noIndexCol2";
+ private final static String COL3_NO_INDEX = "noIndexCol3";
+ private final static String COL1_INVERTED_INDEX = "invertedIndexCol1";
+ private final static String COL2_INVERTED_INDEX = "invertedIndexCol2";
+ private final static String COL3_INVERTED_INDEX = "invertedIndexCol3";
+ private final static String COL1_RANGE_INDEX = "rangeIndexCol1";
+ private final static String COL2_RANGE_INDEX = "rangeIndexCol2";
+ private final static String COL3_RANGE_INDEX = "rangeIndexCol3";
+ private final static String COL1_SORTED_INDEX = "sortedIndexCol1";
+ private final static String COL1_JSON_INDEX = "jsonIndexCol1";
+ private final static String COL1_TEXT_INDEX = "textIndexCol1";
+
+ private static final Schema SCHEMA = new Schema.SchemaBuilder().setSchemaName(RAW_TABLE_NAME)
+ .addSingleValueDimension(COL1_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL2_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_INVERTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_INVERTED_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_INVERTED_INDEX, FieldSpec.DataType.STRING)
+ .addSingleValueDimension(COL1_RANGE_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_SORTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL1_JSON_INDEX, FieldSpec.DataType.JSON)
+ .addSingleValueDimension(COL1_TEXT_INDEX, FieldSpec.DataType.STRING).build();
+
+ private static final DataSchema DATA_SCHEMA = new DataSchema(new String[]{"Operator", "Operator_Id", "Parent_Id"},
+ new DataSchema.ColumnDataType[]{DataSchema.ColumnDataType.STRING, DataSchema.ColumnDataType.INT,
+ DataSchema.ColumnDataType.INT});
+
+ private static final TableConfig TABLE_CONFIG =
+ new TableConfigBuilder(TableType.OFFLINE).setTableName(RAW_TABLE_NAME).build();
+
+ private IndexSegment _indexSegment;
+ private List<IndexSegment> _indexSegments;
+
+ @Override
+ protected String getFilter() {
+ return "";
+ }
+
+ @Override
+ protected IndexSegment getIndexSegment() {
+ return _indexSegment;
+ }
+
+ @Override
+ protected List<IndexSegment> getIndexSegments() {
+ return _indexSegments;
+ }
+
+ GenericRow createRecord(double doubleValue, int intValue, String stringValue, String jsonValue) {
+ GenericRow record = new GenericRow();
+ record.putValue(COL1_NO_INDEX, intValue);
+ record.putValue(COL2_NO_INDEX, intValue);
+ record.putValue(COL3_NO_INDEX, intValue);
+
+ record.putValue(COL1_INVERTED_INDEX, doubleValue);
+ record.putValue(COL2_INVERTED_INDEX, intValue);
+ record.putValue(COL3_INVERTED_INDEX, intValue);
+
+ record.putValue(COL1_RANGE_INDEX, doubleValue);
+ record.putValue(COL2_RANGE_INDEX, intValue);
+ record.putValue(COL3_RANGE_INDEX, intValue);
+
+ record.putValue(COL1_JSON_INDEX, jsonValue);
+ record.putValue(COL1_TEXT_INDEX, stringValue);
+
+ return record;
+ }
+
+ @BeforeClass
+ public void setUp()
+ throws Exception {
+ FileUtils.deleteDirectory(INDEX_DIR);
+
+ List<GenericRow> records = new ArrayList<>(NUM_RECORDS);
+ records.add(createRecord(1.1, 2, "daffy", "{\"first\": \"daffy\", \"last\": \"duck\"}"));
+
+ IndexingConfig indexingConfig = TABLE_CONFIG.getIndexingConfig();
+
+ List<String> invertedIndexColumns = Arrays.asList(COL1_INVERTED_INDEX, COL2_INVERTED_INDEX, COL3_INVERTED_INDEX);
+ indexingConfig.setInvertedIndexColumns(invertedIndexColumns);
+
+ List<String> rangeIndexColumns = Arrays.asList(COL1_RANGE_INDEX, COL2_RANGE_INDEX, COL3_RANGE_INDEX);
+ indexingConfig.setRangeIndexColumns(rangeIndexColumns);
+
+ List<String> jsonIndexColumns = Arrays.asList(COL1_JSON_INDEX);
+ indexingConfig.setJsonIndexColumns(jsonIndexColumns);
+
+ SegmentGeneratorConfig segmentGeneratorConfig = new SegmentGeneratorConfig(TABLE_CONFIG, SCHEMA);
+ segmentGeneratorConfig.setTableName(RAW_TABLE_NAME);
+ segmentGeneratorConfig.setSegmentName(SEGMENT_NAME);
+ segmentGeneratorConfig.setOutDir(INDEX_DIR.getPath());
+
+ SegmentIndexCreationDriverImpl driver = new SegmentIndexCreationDriverImpl();
+ driver.init(segmentGeneratorConfig, new GenericRowRecordReader(records));
+ driver.build();
+
+ IndexLoadingConfig indexLoadingConfig = new IndexLoadingConfig();
+ indexLoadingConfig.setTableConfig(TABLE_CONFIG);
+ indexLoadingConfig.setInvertedIndexColumns(new HashSet<>(invertedIndexColumns));
+ indexLoadingConfig.setRangeIndexColumns(new HashSet<>(rangeIndexColumns));
+ indexLoadingConfig.setJsonIndexColumns(new HashSet<String>(jsonIndexColumns));
+ indexLoadingConfig.setReadMode(ReadMode.mmap);
+
+ ImmutableSegment immutableSegment =
+ ImmutableSegmentLoader.load(new File(INDEX_DIR, SEGMENT_NAME), indexLoadingConfig);
+ _indexSegment = immutableSegment;
+ _indexSegments = Arrays.asList(immutableSegment);
+ }
+
+ /** Checks the correctness of EXPLAIN PLAN output. */
+ private void check(String query, ResultTable expected) {
+ BrokerResponseNative response = getBrokerResponseForSqlQuery(query);
+ ResultTable actual = response.getResultTable();
+
+ System.out.println(resultTableToString("Expected", expected));
+ System.out.println(resultTableToString("Actual", actual));
+
+ Assert.assertEquals(actual.getDataSchema(), expected.getDataSchema());
+ Assert.assertEquals(actual.getRows().size(), expected.getRows().size());
+
+ List<Object[]> expectedRows = expected.getRows();
+ List<Object[]> actualRows = actual.getRows();
+ for (int i = 0; i < expectedRows.size(); i++) {
+ Object[] expectedRow = expectedRows.get(i);
+ Object[] actualRow = actualRows.get(i);
+ Assert.assertEquals(actualRow.length, expectedRow.length);
+
+ for (int j = 0; j < actualRow.length; j++) {
+ Assert.assertEquals(actualRow[j], expectedRow[j]);
+ }
+ }
+ }
+
+ /** This function is used to generated expected results while creating test cases; otherwise, not used. */
+ private static String resultTableToString(String tag, ResultTable table) {
+ System.out.println("\n" + tag + ":");
+ List<Object[]> rows = table.getRows();
+
+ StringBuffer buffer = new StringBuffer();
+ for (Object[] row : rows) {
+ buffer.append("result.add(new Object[]{\"" + row[0] + "\", " + row[1] + ", " + row[2] + "});\n");
+ }
+
+ return buffer.toString();
+ }
+
+ @Test
+ public void testSelect() {
+ String query1 = "EXPLAIN PLAN FOR SELECT * FROM testTable";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(
+ new Object[]{"SELECT(selectList:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, jsonIndexCol1, "
+ + "noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, "
+ + "jsonIndexCol1, noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 3, 2});
+ result1.add(
+ new Object[]{"PROJECT(sortedIndexCol1, noIndexCol3, rangeIndexCol1, rangeIndexCol2, jsonIndexCol1, "
+ + "invertedIndexCol1, noIndexCol2, invertedIndexCol2, noIndexCol1, invertedIndexCol3, rangeIndexCol3, "
+ + "textIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 = "EXPLAIN PLAN FOR SELECT 'mickey' FROM testTable";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:'mickey')", 2, 1});
+ result2.add(new Object[]{"PROJECT(ALL)", 3, 2});
+ result2.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 4, 3});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+
+ String query3 = "EXPLAIN PLAN FOR SELECT invertedIndexCol1, noIndexCol1 FROM testTable LIMIT 100";
+ List<Object[]> result3 = new ArrayList<>();
+ result3.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result3.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result3.add(new Object[]{"SELECT(selectList:invertedIndexCol1, noIndexCol1)", 2, 1});
+ result3.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result3.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result3.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query3, new ResultTable(DATA_SCHEMA, result3));
+
+ String query4 = "EXPLAIN PLAN FOR SELECT DISTINCT invertedIndexCol1, noIndexCol1 FROM testTable LIMIT 100";
+ List<Object[]> result4 = new ArrayList<>();
+ result4.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result4.add(new Object[]{"COMBINE_DISTINCT", 1, 0});
+ result4.add(new Object[]{"DISTINCT(keyColumns:invertedIndexCol1, noIndexCol1)", 2, 1});
+ result4.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result4.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result4.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query4, new ResultTable(DATA_SCHEMA, result4));
+ }
+
+ @Test
Review comment:
Done.
##########
File path: pinot-core/src/test/java/org/apache/pinot/queries/ExplainPlanQueriesTest.java
##########
@@ -0,0 +1,510 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.queries;
+
+import java.io.File;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.List;
+import org.apache.commons.io.FileUtils;
+import org.apache.pinot.common.response.broker.BrokerResponseNative;
+import org.apache.pinot.common.response.broker.ResultTable;
+import org.apache.pinot.common.utils.DataSchema;
+import org.apache.pinot.segment.local.indexsegment.immutable.ImmutableSegmentLoader;
+import org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl;
+import org.apache.pinot.segment.local.segment.index.loader.IndexLoadingConfig;
+import org.apache.pinot.segment.local.segment.readers.GenericRowRecordReader;
+import org.apache.pinot.segment.spi.ImmutableSegment;
+import org.apache.pinot.segment.spi.IndexSegment;
+import org.apache.pinot.segment.spi.creator.SegmentGeneratorConfig;
+import org.apache.pinot.spi.config.table.IndexingConfig;
+import org.apache.pinot.spi.config.table.TableConfig;
+import org.apache.pinot.spi.config.table.TableType;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.data.Schema;
+import org.apache.pinot.spi.data.readers.GenericRow;
+import org.apache.pinot.spi.utils.ReadMode;
+import org.apache.pinot.spi.utils.builder.TableConfigBuilder;
+import org.testng.Assert;
+import org.testng.annotations.BeforeClass;
+import org.testng.annotations.Test;
+
+
+public class ExplainPlanQueriesTest extends BaseQueriesTest {
+ private static final File INDEX_DIR = new File(FileUtils.getTempDirectory(), "ExplainPlanQueriesTest");
+ private static final String RAW_TABLE_NAME = "testTable";
+ private static final String SEGMENT_NAME = "testSegment";
+ private static final int NUM_RECORDS = 10;
+
+ private final static String COL1_NO_INDEX = "noIndexCol1";
+ private final static String COL2_NO_INDEX = "noIndexCol2";
+ private final static String COL3_NO_INDEX = "noIndexCol3";
+ private final static String COL1_INVERTED_INDEX = "invertedIndexCol1";
+ private final static String COL2_INVERTED_INDEX = "invertedIndexCol2";
+ private final static String COL3_INVERTED_INDEX = "invertedIndexCol3";
+ private final static String COL1_RANGE_INDEX = "rangeIndexCol1";
+ private final static String COL2_RANGE_INDEX = "rangeIndexCol2";
+ private final static String COL3_RANGE_INDEX = "rangeIndexCol3";
+ private final static String COL1_SORTED_INDEX = "sortedIndexCol1";
+ private final static String COL1_JSON_INDEX = "jsonIndexCol1";
+ private final static String COL1_TEXT_INDEX = "textIndexCol1";
+
+ private static final Schema SCHEMA = new Schema.SchemaBuilder().setSchemaName(RAW_TABLE_NAME)
+ .addSingleValueDimension(COL1_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL2_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_INVERTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_INVERTED_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_INVERTED_INDEX, FieldSpec.DataType.STRING)
+ .addSingleValueDimension(COL1_RANGE_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_SORTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL1_JSON_INDEX, FieldSpec.DataType.JSON)
+ .addSingleValueDimension(COL1_TEXT_INDEX, FieldSpec.DataType.STRING).build();
+
+ private static final DataSchema DATA_SCHEMA = new DataSchema(new String[]{"Operator", "Operator_Id", "Parent_Id"},
+ new DataSchema.ColumnDataType[]{DataSchema.ColumnDataType.STRING, DataSchema.ColumnDataType.INT,
+ DataSchema.ColumnDataType.INT});
+
+ private static final TableConfig TABLE_CONFIG =
+ new TableConfigBuilder(TableType.OFFLINE).setTableName(RAW_TABLE_NAME).build();
+
+ private IndexSegment _indexSegment;
+ private List<IndexSegment> _indexSegments;
+
+ @Override
+ protected String getFilter() {
+ return "";
+ }
+
+ @Override
+ protected IndexSegment getIndexSegment() {
+ return _indexSegment;
+ }
+
+ @Override
+ protected List<IndexSegment> getIndexSegments() {
+ return _indexSegments;
+ }
+
+ GenericRow createRecord(double doubleValue, int intValue, String stringValue, String jsonValue) {
+ GenericRow record = new GenericRow();
+ record.putValue(COL1_NO_INDEX, intValue);
+ record.putValue(COL2_NO_INDEX, intValue);
+ record.putValue(COL3_NO_INDEX, intValue);
+
+ record.putValue(COL1_INVERTED_INDEX, doubleValue);
+ record.putValue(COL2_INVERTED_INDEX, intValue);
+ record.putValue(COL3_INVERTED_INDEX, intValue);
+
+ record.putValue(COL1_RANGE_INDEX, doubleValue);
+ record.putValue(COL2_RANGE_INDEX, intValue);
+ record.putValue(COL3_RANGE_INDEX, intValue);
+
+ record.putValue(COL1_JSON_INDEX, jsonValue);
+ record.putValue(COL1_TEXT_INDEX, stringValue);
+
+ return record;
+ }
+
+ @BeforeClass
+ public void setUp()
+ throws Exception {
+ FileUtils.deleteDirectory(INDEX_DIR);
+
+ List<GenericRow> records = new ArrayList<>(NUM_RECORDS);
+ records.add(createRecord(1.1, 2, "daffy", "{\"first\": \"daffy\", \"last\": \"duck\"}"));
+
+ IndexingConfig indexingConfig = TABLE_CONFIG.getIndexingConfig();
+
+ List<String> invertedIndexColumns = Arrays.asList(COL1_INVERTED_INDEX, COL2_INVERTED_INDEX, COL3_INVERTED_INDEX);
+ indexingConfig.setInvertedIndexColumns(invertedIndexColumns);
+
+ List<String> rangeIndexColumns = Arrays.asList(COL1_RANGE_INDEX, COL2_RANGE_INDEX, COL3_RANGE_INDEX);
+ indexingConfig.setRangeIndexColumns(rangeIndexColumns);
+
+ List<String> jsonIndexColumns = Arrays.asList(COL1_JSON_INDEX);
+ indexingConfig.setJsonIndexColumns(jsonIndexColumns);
+
+ SegmentGeneratorConfig segmentGeneratorConfig = new SegmentGeneratorConfig(TABLE_CONFIG, SCHEMA);
+ segmentGeneratorConfig.setTableName(RAW_TABLE_NAME);
+ segmentGeneratorConfig.setSegmentName(SEGMENT_NAME);
+ segmentGeneratorConfig.setOutDir(INDEX_DIR.getPath());
+
+ SegmentIndexCreationDriverImpl driver = new SegmentIndexCreationDriverImpl();
+ driver.init(segmentGeneratorConfig, new GenericRowRecordReader(records));
+ driver.build();
+
+ IndexLoadingConfig indexLoadingConfig = new IndexLoadingConfig();
+ indexLoadingConfig.setTableConfig(TABLE_CONFIG);
+ indexLoadingConfig.setInvertedIndexColumns(new HashSet<>(invertedIndexColumns));
+ indexLoadingConfig.setRangeIndexColumns(new HashSet<>(rangeIndexColumns));
+ indexLoadingConfig.setJsonIndexColumns(new HashSet<String>(jsonIndexColumns));
+ indexLoadingConfig.setReadMode(ReadMode.mmap);
+
+ ImmutableSegment immutableSegment =
+ ImmutableSegmentLoader.load(new File(INDEX_DIR, SEGMENT_NAME), indexLoadingConfig);
+ _indexSegment = immutableSegment;
+ _indexSegments = Arrays.asList(immutableSegment);
+ }
+
+ /** Checks the correctness of EXPLAIN PLAN output. */
+ private void check(String query, ResultTable expected) {
+ BrokerResponseNative response = getBrokerResponseForSqlQuery(query);
+ ResultTable actual = response.getResultTable();
+
+ System.out.println(resultTableToString("Expected", expected));
+ System.out.println(resultTableToString("Actual", actual));
+
+ Assert.assertEquals(actual.getDataSchema(), expected.getDataSchema());
+ Assert.assertEquals(actual.getRows().size(), expected.getRows().size());
+
+ List<Object[]> expectedRows = expected.getRows();
+ List<Object[]> actualRows = actual.getRows();
+ for (int i = 0; i < expectedRows.size(); i++) {
+ Object[] expectedRow = expectedRows.get(i);
+ Object[] actualRow = actualRows.get(i);
+ Assert.assertEquals(actualRow.length, expectedRow.length);
+
+ for (int j = 0; j < actualRow.length; j++) {
+ Assert.assertEquals(actualRow[j], expectedRow[j]);
+ }
+ }
+ }
+
+ /** This function is used to generated expected results while creating test cases; otherwise, not used. */
+ private static String resultTableToString(String tag, ResultTable table) {
+ System.out.println("\n" + tag + ":");
+ List<Object[]> rows = table.getRows();
+
+ StringBuffer buffer = new StringBuffer();
+ for (Object[] row : rows) {
+ buffer.append("result.add(new Object[]{\"" + row[0] + "\", " + row[1] + ", " + row[2] + "});\n");
+ }
+
+ return buffer.toString();
+ }
+
+ @Test
+ public void testSelect() {
+ String query1 = "EXPLAIN PLAN FOR SELECT * FROM testTable";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(
+ new Object[]{"SELECT(selectList:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, jsonIndexCol1, "
+ + "noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, "
+ + "jsonIndexCol1, noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 3, 2});
+ result1.add(
+ new Object[]{"PROJECT(sortedIndexCol1, noIndexCol3, rangeIndexCol1, rangeIndexCol2, jsonIndexCol1, "
+ + "invertedIndexCol1, noIndexCol2, invertedIndexCol2, noIndexCol1, invertedIndexCol3, rangeIndexCol3, "
+ + "textIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 = "EXPLAIN PLAN FOR SELECT 'mickey' FROM testTable";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:'mickey')", 2, 1});
+ result2.add(new Object[]{"PROJECT(ALL)", 3, 2});
+ result2.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 4, 3});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+
+ String query3 = "EXPLAIN PLAN FOR SELECT invertedIndexCol1, noIndexCol1 FROM testTable LIMIT 100";
+ List<Object[]> result3 = new ArrayList<>();
+ result3.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result3.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result3.add(new Object[]{"SELECT(selectList:invertedIndexCol1, noIndexCol1)", 2, 1});
+ result3.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result3.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result3.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query3, new ResultTable(DATA_SCHEMA, result3));
+
+ String query4 = "EXPLAIN PLAN FOR SELECT DISTINCT invertedIndexCol1, noIndexCol1 FROM testTable LIMIT 100";
+ List<Object[]> result4 = new ArrayList<>();
+ result4.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result4.add(new Object[]{"COMBINE_DISTINCT", 1, 0});
+ result4.add(new Object[]{"DISTINCT(keyColumns:invertedIndexCol1, noIndexCol1)", 2, 1});
+ result4.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result4.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result4.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query4, new ResultTable(DATA_SCHEMA, result4));
+ }
+
+ @Test
+ public void testSelectColumnsUsingFilterIndex() {
+ String query1 = "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1, sortedIndexCol1 FROM testTable WHERE "
+ + "invertedIndexCol1 = 1.5 AND sortedIndexCol1 = 1 LIMIT 100";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1, sortedIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, sortedIndexCol1)", 3, 2});
+ result1.add(new Object[]{"PROJECT(sortedIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 = "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1, sortedIndexCol1 FROM testTable WHERE "
+ + "(invertedIndexCol1 = 1.5 AND sortedIndexCol1 = 1) OR (invertedIndexCol1 BETWEEN 100 and 200 AND "
+ + "rangeIndexCol1 > 20)";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1, sortedIndexCol1)", 2, 1});
+ result2.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, sortedIndexCol1)", 3, 2});
+ result2.add(new Object[]{"PROJECT(sortedIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result2.add(new Object[]{"FILTER_EMPTY", 5, 4});
Review comment:
Fixed.
##########
File path: pinot-core/src/test/java/org/apache/pinot/queries/ExplainPlanQueriesTest.java
##########
@@ -0,0 +1,510 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.queries;
+
+import java.io.File;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.List;
+import org.apache.commons.io.FileUtils;
+import org.apache.pinot.common.response.broker.BrokerResponseNative;
+import org.apache.pinot.common.response.broker.ResultTable;
+import org.apache.pinot.common.utils.DataSchema;
+import org.apache.pinot.segment.local.indexsegment.immutable.ImmutableSegmentLoader;
+import org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl;
+import org.apache.pinot.segment.local.segment.index.loader.IndexLoadingConfig;
+import org.apache.pinot.segment.local.segment.readers.GenericRowRecordReader;
+import org.apache.pinot.segment.spi.ImmutableSegment;
+import org.apache.pinot.segment.spi.IndexSegment;
+import org.apache.pinot.segment.spi.creator.SegmentGeneratorConfig;
+import org.apache.pinot.spi.config.table.IndexingConfig;
+import org.apache.pinot.spi.config.table.TableConfig;
+import org.apache.pinot.spi.config.table.TableType;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.data.Schema;
+import org.apache.pinot.spi.data.readers.GenericRow;
+import org.apache.pinot.spi.utils.ReadMode;
+import org.apache.pinot.spi.utils.builder.TableConfigBuilder;
+import org.testng.Assert;
+import org.testng.annotations.BeforeClass;
+import org.testng.annotations.Test;
+
+
+public class ExplainPlanQueriesTest extends BaseQueriesTest {
+ private static final File INDEX_DIR = new File(FileUtils.getTempDirectory(), "ExplainPlanQueriesTest");
+ private static final String RAW_TABLE_NAME = "testTable";
+ private static final String SEGMENT_NAME = "testSegment";
+ private static final int NUM_RECORDS = 10;
+
+ private final static String COL1_NO_INDEX = "noIndexCol1";
+ private final static String COL2_NO_INDEX = "noIndexCol2";
+ private final static String COL3_NO_INDEX = "noIndexCol3";
+ private final static String COL1_INVERTED_INDEX = "invertedIndexCol1";
+ private final static String COL2_INVERTED_INDEX = "invertedIndexCol2";
+ private final static String COL3_INVERTED_INDEX = "invertedIndexCol3";
+ private final static String COL1_RANGE_INDEX = "rangeIndexCol1";
+ private final static String COL2_RANGE_INDEX = "rangeIndexCol2";
+ private final static String COL3_RANGE_INDEX = "rangeIndexCol3";
+ private final static String COL1_SORTED_INDEX = "sortedIndexCol1";
+ private final static String COL1_JSON_INDEX = "jsonIndexCol1";
+ private final static String COL1_TEXT_INDEX = "textIndexCol1";
+
+ private static final Schema SCHEMA = new Schema.SchemaBuilder().setSchemaName(RAW_TABLE_NAME)
+ .addSingleValueDimension(COL1_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL2_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_INVERTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_INVERTED_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_INVERTED_INDEX, FieldSpec.DataType.STRING)
+ .addSingleValueDimension(COL1_RANGE_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_SORTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL1_JSON_INDEX, FieldSpec.DataType.JSON)
+ .addSingleValueDimension(COL1_TEXT_INDEX, FieldSpec.DataType.STRING).build();
+
+ private static final DataSchema DATA_SCHEMA = new DataSchema(new String[]{"Operator", "Operator_Id", "Parent_Id"},
+ new DataSchema.ColumnDataType[]{DataSchema.ColumnDataType.STRING, DataSchema.ColumnDataType.INT,
+ DataSchema.ColumnDataType.INT});
+
+ private static final TableConfig TABLE_CONFIG =
+ new TableConfigBuilder(TableType.OFFLINE).setTableName(RAW_TABLE_NAME).build();
+
+ private IndexSegment _indexSegment;
+ private List<IndexSegment> _indexSegments;
+
+ @Override
+ protected String getFilter() {
+ return "";
+ }
+
+ @Override
+ protected IndexSegment getIndexSegment() {
+ return _indexSegment;
+ }
+
+ @Override
+ protected List<IndexSegment> getIndexSegments() {
+ return _indexSegments;
+ }
+
+ GenericRow createRecord(double doubleValue, int intValue, String stringValue, String jsonValue) {
+ GenericRow record = new GenericRow();
+ record.putValue(COL1_NO_INDEX, intValue);
+ record.putValue(COL2_NO_INDEX, intValue);
+ record.putValue(COL3_NO_INDEX, intValue);
+
+ record.putValue(COL1_INVERTED_INDEX, doubleValue);
+ record.putValue(COL2_INVERTED_INDEX, intValue);
+ record.putValue(COL3_INVERTED_INDEX, intValue);
+
+ record.putValue(COL1_RANGE_INDEX, doubleValue);
+ record.putValue(COL2_RANGE_INDEX, intValue);
+ record.putValue(COL3_RANGE_INDEX, intValue);
+
+ record.putValue(COL1_JSON_INDEX, jsonValue);
+ record.putValue(COL1_TEXT_INDEX, stringValue);
+
+ return record;
+ }
+
+ @BeforeClass
+ public void setUp()
+ throws Exception {
+ FileUtils.deleteDirectory(INDEX_DIR);
+
+ List<GenericRow> records = new ArrayList<>(NUM_RECORDS);
+ records.add(createRecord(1.1, 2, "daffy", "{\"first\": \"daffy\", \"last\": \"duck\"}"));
+
+ IndexingConfig indexingConfig = TABLE_CONFIG.getIndexingConfig();
+
+ List<String> invertedIndexColumns = Arrays.asList(COL1_INVERTED_INDEX, COL2_INVERTED_INDEX, COL3_INVERTED_INDEX);
+ indexingConfig.setInvertedIndexColumns(invertedIndexColumns);
+
+ List<String> rangeIndexColumns = Arrays.asList(COL1_RANGE_INDEX, COL2_RANGE_INDEX, COL3_RANGE_INDEX);
+ indexingConfig.setRangeIndexColumns(rangeIndexColumns);
+
+ List<String> jsonIndexColumns = Arrays.asList(COL1_JSON_INDEX);
+ indexingConfig.setJsonIndexColumns(jsonIndexColumns);
+
+ SegmentGeneratorConfig segmentGeneratorConfig = new SegmentGeneratorConfig(TABLE_CONFIG, SCHEMA);
+ segmentGeneratorConfig.setTableName(RAW_TABLE_NAME);
+ segmentGeneratorConfig.setSegmentName(SEGMENT_NAME);
+ segmentGeneratorConfig.setOutDir(INDEX_DIR.getPath());
+
+ SegmentIndexCreationDriverImpl driver = new SegmentIndexCreationDriverImpl();
+ driver.init(segmentGeneratorConfig, new GenericRowRecordReader(records));
+ driver.build();
+
+ IndexLoadingConfig indexLoadingConfig = new IndexLoadingConfig();
+ indexLoadingConfig.setTableConfig(TABLE_CONFIG);
+ indexLoadingConfig.setInvertedIndexColumns(new HashSet<>(invertedIndexColumns));
+ indexLoadingConfig.setRangeIndexColumns(new HashSet<>(rangeIndexColumns));
+ indexLoadingConfig.setJsonIndexColumns(new HashSet<String>(jsonIndexColumns));
+ indexLoadingConfig.setReadMode(ReadMode.mmap);
+
+ ImmutableSegment immutableSegment =
+ ImmutableSegmentLoader.load(new File(INDEX_DIR, SEGMENT_NAME), indexLoadingConfig);
+ _indexSegment = immutableSegment;
+ _indexSegments = Arrays.asList(immutableSegment);
+ }
+
+ /** Checks the correctness of EXPLAIN PLAN output. */
+ private void check(String query, ResultTable expected) {
+ BrokerResponseNative response = getBrokerResponseForSqlQuery(query);
+ ResultTable actual = response.getResultTable();
+
+ System.out.println(resultTableToString("Expected", expected));
+ System.out.println(resultTableToString("Actual", actual));
+
+ Assert.assertEquals(actual.getDataSchema(), expected.getDataSchema());
+ Assert.assertEquals(actual.getRows().size(), expected.getRows().size());
+
+ List<Object[]> expectedRows = expected.getRows();
+ List<Object[]> actualRows = actual.getRows();
+ for (int i = 0; i < expectedRows.size(); i++) {
+ Object[] expectedRow = expectedRows.get(i);
+ Object[] actualRow = actualRows.get(i);
+ Assert.assertEquals(actualRow.length, expectedRow.length);
+
+ for (int j = 0; j < actualRow.length; j++) {
+ Assert.assertEquals(actualRow[j], expectedRow[j]);
+ }
+ }
+ }
+
+ /** This function is used to generated expected results while creating test cases; otherwise, not used. */
+ private static String resultTableToString(String tag, ResultTable table) {
+ System.out.println("\n" + tag + ":");
+ List<Object[]> rows = table.getRows();
+
+ StringBuffer buffer = new StringBuffer();
+ for (Object[] row : rows) {
+ buffer.append("result.add(new Object[]{\"" + row[0] + "\", " + row[1] + ", " + row[2] + "});\n");
+ }
+
+ return buffer.toString();
+ }
+
+ @Test
+ public void testSelect() {
+ String query1 = "EXPLAIN PLAN FOR SELECT * FROM testTable";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(
+ new Object[]{"SELECT(selectList:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, jsonIndexCol1, "
+ + "noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, "
+ + "jsonIndexCol1, noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 3, 2});
+ result1.add(
+ new Object[]{"PROJECT(sortedIndexCol1, noIndexCol3, rangeIndexCol1, rangeIndexCol2, jsonIndexCol1, "
+ + "invertedIndexCol1, noIndexCol2, invertedIndexCol2, noIndexCol1, invertedIndexCol3, rangeIndexCol3, "
+ + "textIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 = "EXPLAIN PLAN FOR SELECT 'mickey' FROM testTable";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:'mickey')", 2, 1});
+ result2.add(new Object[]{"PROJECT(ALL)", 3, 2});
+ result2.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 4, 3});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+
+ String query3 = "EXPLAIN PLAN FOR SELECT invertedIndexCol1, noIndexCol1 FROM testTable LIMIT 100";
+ List<Object[]> result3 = new ArrayList<>();
+ result3.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result3.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result3.add(new Object[]{"SELECT(selectList:invertedIndexCol1, noIndexCol1)", 2, 1});
+ result3.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result3.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result3.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query3, new ResultTable(DATA_SCHEMA, result3));
+
+ String query4 = "EXPLAIN PLAN FOR SELECT DISTINCT invertedIndexCol1, noIndexCol1 FROM testTable LIMIT 100";
+ List<Object[]> result4 = new ArrayList<>();
+ result4.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result4.add(new Object[]{"COMBINE_DISTINCT", 1, 0});
+ result4.add(new Object[]{"DISTINCT(keyColumns:invertedIndexCol1, noIndexCol1)", 2, 1});
+ result4.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result4.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result4.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query4, new ResultTable(DATA_SCHEMA, result4));
+ }
+
+ @Test
+ public void testSelectColumnsUsingFilterIndex() {
+ String query1 = "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1, sortedIndexCol1 FROM testTable WHERE "
+ + "invertedIndexCol1 = 1.5 AND sortedIndexCol1 = 1 LIMIT 100";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1, sortedIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, sortedIndexCol1)", 3, 2});
+ result1.add(new Object[]{"PROJECT(sortedIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_EMPTY", 5, 4});
Review comment:
Fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7568](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (57ceb4a) into [master](https://codecov.io/gh/apache/pinot/commit/2107b2cd1880ec74934ec5412dbee8c874f05610?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (2107b2c) will **decrease** coverage by `1.50%`.
> The diff coverage is `21.46%`.
> :exclamation: Current head 57ceb4a differs from pull request most recent head 03cae5a. Consider uploading reports for the commit 03cae5a to get more accurate results
[](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7568 +/- ##
==========================================
- Coverage 30.79% 29.28% -1.51%
==========================================
Files 1570 1569 -1
Lines 80024 80550 +526
Branches 11904 11988 +84
==========================================
- Hits 24644 23590 -1054
- Misses 53274 54884 +1610
+ Partials 2106 2076 -30
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `29.28% <21.46%> (+0.06%)` | :arrow_up: |
| integration2 | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...ache/pinot/common/tier/PinotServerTierStorage.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdGllci9QaW5vdFNlcnZlclRpZXJTdG9yYWdlLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...c/main/java/org/apache/pinot/common/tier/Tier.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdGllci9UaWVyLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...java/org/apache/pinot/common/tier/TierFactory.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdGllci9UaWVyRmFjdG9yeS5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...inot/common/tier/TimeBasedTierSegmentSelector.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdGllci9UaW1lQmFzZWRUaWVyU2VnbWVudFNlbGVjdG9yLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [.../pinot/common/utils/ClientSSLContextGenerator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvQ2xpZW50U1NMQ29udGV4dEdlbmVyYXRvci5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...che/pinot/common/utils/config/TierConfigUtils.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvY29uZmlnL1RpZXJDb25maWdVdGlscy5qYXZh) | `4.16% <0.00%> (-1.39%)` | :arrow_down: |
| [...e/pinot/controller/helix/SegmentStatusChecker.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9oZWxpeC9TZWdtZW50U3RhdHVzQ2hlY2tlci5qYXZh) | `70.42% <0.00%> (-6.68%)` | :arrow_down: |
| [...troller/helix/core/minion/ClusterInfoAccessor.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9oZWxpeC9jb3JlL21pbmlvbi9DbHVzdGVySW5mb0FjY2Vzc29yLmphdmE=) | `67.85% <0.00%> (-5.22%)` | :arrow_down: |
| [...ntroller/helix/core/rebalance/TableRebalancer.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9oZWxpeC9jb3JlL3JlYmFsYW5jZS9UYWJsZVJlYmFsYW5jZXIuamF2YQ==) | `53.82% <0.00%> (ø)` | |
| [...ller/validation/OfflineSegmentIntervalChecker.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci92YWxpZGF0aW9uL09mZmxpbmVTZWdtZW50SW50ZXJ2YWxDaGVja2VyLmphdmE=) | `76.40% <0.00%> (-9.68%)` | :arrow_down: |
| ... and [273 more](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [2107b2c...03cae5a](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] amrishlal commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
amrishlal commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r747807507
##########
File path: pinot-core/src/test/java/org/apache/pinot/queries/ExplainPlanQueriesTest.java
##########
@@ -0,0 +1,510 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.queries;
+
+import java.io.File;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.List;
+import org.apache.commons.io.FileUtils;
+import org.apache.pinot.common.response.broker.BrokerResponseNative;
+import org.apache.pinot.common.response.broker.ResultTable;
+import org.apache.pinot.common.utils.DataSchema;
+import org.apache.pinot.segment.local.indexsegment.immutable.ImmutableSegmentLoader;
+import org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl;
+import org.apache.pinot.segment.local.segment.index.loader.IndexLoadingConfig;
+import org.apache.pinot.segment.local.segment.readers.GenericRowRecordReader;
+import org.apache.pinot.segment.spi.ImmutableSegment;
+import org.apache.pinot.segment.spi.IndexSegment;
+import org.apache.pinot.segment.spi.creator.SegmentGeneratorConfig;
+import org.apache.pinot.spi.config.table.IndexingConfig;
+import org.apache.pinot.spi.config.table.TableConfig;
+import org.apache.pinot.spi.config.table.TableType;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.data.Schema;
+import org.apache.pinot.spi.data.readers.GenericRow;
+import org.apache.pinot.spi.utils.ReadMode;
+import org.apache.pinot.spi.utils.builder.TableConfigBuilder;
+import org.testng.Assert;
+import org.testng.annotations.BeforeClass;
+import org.testng.annotations.Test;
+
+
+public class ExplainPlanQueriesTest extends BaseQueriesTest {
+ private static final File INDEX_DIR = new File(FileUtils.getTempDirectory(), "ExplainPlanQueriesTest");
+ private static final String RAW_TABLE_NAME = "testTable";
+ private static final String SEGMENT_NAME = "testSegment";
+ private static final int NUM_RECORDS = 10;
+
+ private final static String COL1_NO_INDEX = "noIndexCol1";
+ private final static String COL2_NO_INDEX = "noIndexCol2";
+ private final static String COL3_NO_INDEX = "noIndexCol3";
+ private final static String COL1_INVERTED_INDEX = "invertedIndexCol1";
+ private final static String COL2_INVERTED_INDEX = "invertedIndexCol2";
+ private final static String COL3_INVERTED_INDEX = "invertedIndexCol3";
+ private final static String COL1_RANGE_INDEX = "rangeIndexCol1";
+ private final static String COL2_RANGE_INDEX = "rangeIndexCol2";
+ private final static String COL3_RANGE_INDEX = "rangeIndexCol3";
+ private final static String COL1_SORTED_INDEX = "sortedIndexCol1";
+ private final static String COL1_JSON_INDEX = "jsonIndexCol1";
+ private final static String COL1_TEXT_INDEX = "textIndexCol1";
+
+ private static final Schema SCHEMA = new Schema.SchemaBuilder().setSchemaName(RAW_TABLE_NAME)
+ .addSingleValueDimension(COL1_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL2_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_INVERTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_INVERTED_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_INVERTED_INDEX, FieldSpec.DataType.STRING)
+ .addSingleValueDimension(COL1_RANGE_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_SORTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL1_JSON_INDEX, FieldSpec.DataType.JSON)
+ .addSingleValueDimension(COL1_TEXT_INDEX, FieldSpec.DataType.STRING).build();
+
+ private static final DataSchema DATA_SCHEMA = new DataSchema(new String[]{"Operator", "Operator_Id", "Parent_Id"},
+ new DataSchema.ColumnDataType[]{DataSchema.ColumnDataType.STRING, DataSchema.ColumnDataType.INT,
+ DataSchema.ColumnDataType.INT});
+
+ private static final TableConfig TABLE_CONFIG =
+ new TableConfigBuilder(TableType.OFFLINE).setTableName(RAW_TABLE_NAME).build();
+
+ private IndexSegment _indexSegment;
+ private List<IndexSegment> _indexSegments;
+
+ @Override
+ protected String getFilter() {
+ return "";
+ }
+
+ @Override
+ protected IndexSegment getIndexSegment() {
+ return _indexSegment;
+ }
+
+ @Override
+ protected List<IndexSegment> getIndexSegments() {
+ return _indexSegments;
+ }
+
+ GenericRow createRecord(double doubleValue, int intValue, String stringValue, String jsonValue) {
+ GenericRow record = new GenericRow();
+ record.putValue(COL1_NO_INDEX, intValue);
+ record.putValue(COL2_NO_INDEX, intValue);
+ record.putValue(COL3_NO_INDEX, intValue);
+
+ record.putValue(COL1_INVERTED_INDEX, doubleValue);
+ record.putValue(COL2_INVERTED_INDEX, intValue);
+ record.putValue(COL3_INVERTED_INDEX, intValue);
+
+ record.putValue(COL1_RANGE_INDEX, doubleValue);
+ record.putValue(COL2_RANGE_INDEX, intValue);
+ record.putValue(COL3_RANGE_INDEX, intValue);
+
+ record.putValue(COL1_JSON_INDEX, jsonValue);
+ record.putValue(COL1_TEXT_INDEX, stringValue);
+
+ return record;
+ }
+
+ @BeforeClass
+ public void setUp()
+ throws Exception {
+ FileUtils.deleteDirectory(INDEX_DIR);
+
+ List<GenericRow> records = new ArrayList<>(NUM_RECORDS);
+ records.add(createRecord(1.1, 2, "daffy", "{\"first\": \"daffy\", \"last\": \"duck\"}"));
+
+ IndexingConfig indexingConfig = TABLE_CONFIG.getIndexingConfig();
+
+ List<String> invertedIndexColumns = Arrays.asList(COL1_INVERTED_INDEX, COL2_INVERTED_INDEX, COL3_INVERTED_INDEX);
+ indexingConfig.setInvertedIndexColumns(invertedIndexColumns);
+
+ List<String> rangeIndexColumns = Arrays.asList(COL1_RANGE_INDEX, COL2_RANGE_INDEX, COL3_RANGE_INDEX);
+ indexingConfig.setRangeIndexColumns(rangeIndexColumns);
+
+ List<String> jsonIndexColumns = Arrays.asList(COL1_JSON_INDEX);
+ indexingConfig.setJsonIndexColumns(jsonIndexColumns);
+
+ SegmentGeneratorConfig segmentGeneratorConfig = new SegmentGeneratorConfig(TABLE_CONFIG, SCHEMA);
+ segmentGeneratorConfig.setTableName(RAW_TABLE_NAME);
+ segmentGeneratorConfig.setSegmentName(SEGMENT_NAME);
+ segmentGeneratorConfig.setOutDir(INDEX_DIR.getPath());
+
+ SegmentIndexCreationDriverImpl driver = new SegmentIndexCreationDriverImpl();
+ driver.init(segmentGeneratorConfig, new GenericRowRecordReader(records));
+ driver.build();
+
+ IndexLoadingConfig indexLoadingConfig = new IndexLoadingConfig();
+ indexLoadingConfig.setTableConfig(TABLE_CONFIG);
+ indexLoadingConfig.setInvertedIndexColumns(new HashSet<>(invertedIndexColumns));
+ indexLoadingConfig.setRangeIndexColumns(new HashSet<>(rangeIndexColumns));
+ indexLoadingConfig.setJsonIndexColumns(new HashSet<String>(jsonIndexColumns));
+ indexLoadingConfig.setReadMode(ReadMode.mmap);
+
+ ImmutableSegment immutableSegment =
+ ImmutableSegmentLoader.load(new File(INDEX_DIR, SEGMENT_NAME), indexLoadingConfig);
+ _indexSegment = immutableSegment;
+ _indexSegments = Arrays.asList(immutableSegment);
+ }
+
+ /** Checks the correctness of EXPLAIN PLAN output. */
+ private void check(String query, ResultTable expected) {
+ BrokerResponseNative response = getBrokerResponseForSqlQuery(query);
+ ResultTable actual = response.getResultTable();
+
+ System.out.println(resultTableToString("Expected", expected));
+ System.out.println(resultTableToString("Actual", actual));
+
+ Assert.assertEquals(actual.getDataSchema(), expected.getDataSchema());
+ Assert.assertEquals(actual.getRows().size(), expected.getRows().size());
+
+ List<Object[]> expectedRows = expected.getRows();
+ List<Object[]> actualRows = actual.getRows();
+ for (int i = 0; i < expectedRows.size(); i++) {
+ Object[] expectedRow = expectedRows.get(i);
+ Object[] actualRow = actualRows.get(i);
+ Assert.assertEquals(actualRow.length, expectedRow.length);
+
+ for (int j = 0; j < actualRow.length; j++) {
+ Assert.assertEquals(actualRow[j], expectedRow[j]);
+ }
+ }
+ }
+
+ /** This function is used to generated expected results while creating test cases; otherwise, not used. */
+ private static String resultTableToString(String tag, ResultTable table) {
+ System.out.println("\n" + tag + ":");
+ List<Object[]> rows = table.getRows();
+
+ StringBuffer buffer = new StringBuffer();
+ for (Object[] row : rows) {
+ buffer.append("result.add(new Object[]{\"" + row[0] + "\", " + row[1] + ", " + row[2] + "});\n");
+ }
+
+ return buffer.toString();
+ }
+
+ @Test
+ public void testSelect() {
+ String query1 = "EXPLAIN PLAN FOR SELECT * FROM testTable";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(
+ new Object[]{"SELECT(selectList:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, jsonIndexCol1, "
+ + "noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, "
+ + "jsonIndexCol1, noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 3, 2});
+ result1.add(
+ new Object[]{"PROJECT(sortedIndexCol1, noIndexCol3, rangeIndexCol1, rangeIndexCol2, jsonIndexCol1, "
+ + "invertedIndexCol1, noIndexCol2, invertedIndexCol2, noIndexCol1, invertedIndexCol3, rangeIndexCol3, "
+ + "textIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 = "EXPLAIN PLAN FOR SELECT 'mickey' FROM testTable";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:'mickey')", 2, 1});
+ result2.add(new Object[]{"PROJECT(ALL)", 3, 2});
+ result2.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 4, 3});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+
+ String query3 = "EXPLAIN PLAN FOR SELECT invertedIndexCol1, noIndexCol1 FROM testTable LIMIT 100";
+ List<Object[]> result3 = new ArrayList<>();
+ result3.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result3.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result3.add(new Object[]{"SELECT(selectList:invertedIndexCol1, noIndexCol1)", 2, 1});
+ result3.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result3.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result3.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query3, new ResultTable(DATA_SCHEMA, result3));
+
+ String query4 = "EXPLAIN PLAN FOR SELECT DISTINCT invertedIndexCol1, noIndexCol1 FROM testTable LIMIT 100";
+ List<Object[]> result4 = new ArrayList<>();
+ result4.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result4.add(new Object[]{"COMBINE_DISTINCT", 1, 0});
+ result4.add(new Object[]{"DISTINCT(keyColumns:invertedIndexCol1, noIndexCol1)", 2, 1});
+ result4.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result4.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result4.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query4, new ResultTable(DATA_SCHEMA, result4));
+ }
+
+ @Test
+ public void testSelectColumnsUsingFilterIndex() {
+ String query1 = "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1, sortedIndexCol1 FROM testTable WHERE "
+ + "invertedIndexCol1 = 1.5 AND sortedIndexCol1 = 1 LIMIT 100";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1, sortedIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, sortedIndexCol1)", 3, 2});
+ result1.add(new Object[]{"PROJECT(sortedIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 = "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1, sortedIndexCol1 FROM testTable WHERE "
+ + "(invertedIndexCol1 = 1.5 AND sortedIndexCol1 = 1) OR (invertedIndexCol1 BETWEEN 100 and 200 AND "
+ + "rangeIndexCol1 > 20)";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1, sortedIndexCol1)", 2, 1});
+ result2.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, sortedIndexCol1)", 3, 2});
+ result2.add(new Object[]{"PROJECT(sortedIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result2.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+
+ String query3 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1 FROM testTable WHERE invertedIndexCol1 = 1.5 AND "
+ + "invertedIndexCol2 IN (10, 20, 30) AND invertedIndexCol3 NOT IN ('foo', 'bar') LIMIT 100";
+ List<Object[]> result3 = new ArrayList<>();
+ result3.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result3.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result3.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1)", 2, 1});
+ result3.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result3.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result3.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query3, new ResultTable(DATA_SCHEMA, result3));
+
+ String query4 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, noIndexCol2, sortedIndexCol1 FROM testTable WHERE sortedIndexCol1 = 1.5"
+ + " OR sortedIndexCol1 != 5 OR sortedIndexCol1 IN (10, 20, 30) LIMIT 100";
+ List<Object[]> result4 = new ArrayList<>();
+ result4.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result4.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result4.add(new Object[]{"SELECT(selectList:noIndexCol1, noIndexCol2, sortedIndexCol1)", 2, 1});
+ result4.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2, sortedIndexCol1)", 3, 2});
+ result4.add(new Object[]{"PROJECT(sortedIndexCol1, noIndexCol2, noIndexCol1)", 4, 3});
+ result4.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query4, new ResultTable(DATA_SCHEMA, result4));
+
+ String query5 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, noIndexCol2 FROM testTable WHERE DIV(noIndexCol1, noIndexCol2) BETWEEN "
+ + "10 AND 20 AND invertedIndexCol1 * 5 < 1000";
+ List<Object[]> result5 = new ArrayList<>();
+ result5.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result5.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result5.add(new Object[]{"SELECT(selectList:noIndexCol1, noIndexCol2)", 2, 1});
+ result5.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2)", 3, 2});
+ result5.add(new Object[]{"PROJECT(noIndexCol2, noIndexCol1)", 4, 3});
+ result5.add(new Object[]{"FILTER_AND", 5, 4});
+ result5.add(
+ new Object[]{"FILTER_EXPRESSION(operator:RANGE,predicate:div(noIndexCol1,noIndexCol2) BETWEEN '10' AND '20')",
+ 6, 5});
+ result5
+ .add(new Object[]{"FILTER_EXPRESSION(operator:RANGE,predicate:times(invertedIndexCol1,'5') < '1000')", 7, 5});
+ check(query5, new ResultTable(DATA_SCHEMA, result5));
+ }
+
+ @Test
+ public void testSelectColumnUsingFilterRange() {
+ // select * query triggering range index
+ // checks using RANGE (>, >=, <, <=, BETWEEN ..) on a column with range index should use RANGE_INDEX_SCAN
+ String query1 =
+ "EXPLAIN PLAN FOR SELECT invertedIndexCol1, noIndexCol1, rangeIndexCol1 FROM testTable WHERE rangeIndexCol1 >"
+ + " 10 AND rangeIndexCol2 >= 20 AND rangeIndexCol3 BETWEEN 50 AND 60 LIMIT 100";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(new Object[]{"SELECT(selectList:invertedIndexCol1, noIndexCol1, rangeIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, rangeIndexCol1)", 3, 2});
+ result1.add(new Object[]{"PROJECT(rangeIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1 FROM testTable WHERE noIndexCol1 > 10 AND noIndexCol2"
+ + " BETWEEN 50 AND 60 LIMIT 100";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1)", 2, 1});
+ result2.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result2.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result2.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+ }
+
+ @Test
+ public void testSelectColumnUsingFilterJson() {
+ String query =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1 FROM testTable WHERE (invertedIndexCol1 IN (10, 20, "
+ + "30) AND sortedIndexCol1 != 100) OR (noIndexCol1 NOT IN (10, 20, 30) AND rangeIndexCol1 != 20 AND "
+ + "JSON_MATCH(jsonIndexCol1, 'key=1') AND TEXT_MATCH(textIndexCol1, 'foo'))";
+ List<Object[]> result = new ArrayList<>();
+ result.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1)", 2, 1});
+ result.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result.add(new Object[]{"FILTER_AND", 5, 4});
+ result.add(new Object[]{"FILTER_JSON_INDEX(indexLookUp:json_index,operator:JSON_MATCH,predicate:json_match"
+ + "(jsonIndexCol1,'key=1'))", 6, 5});
+ result.add(new Object[]{"FILTER_TEXT_INDEX(indexLookUp:text_index,operator:TEXT_MATCH,predicate:text_match"
+ + "(textIndexCol1,'foo'))", 7, 5});
+ check(query, new ResultTable(DATA_SCHEMA, result));
+ }
+
+ @Test
+ public void testSelectAggregateUsingFilterText() {
+ String query1 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, noIndexCol2, max(noIndexCol2), min(noIndexCol3) FROM testTable WHERE "
+ + "TEXT_MATCH(textIndexCol1, 'foo') GROUP BY noIndexCol1, noIndexCol2";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result1.add(new Object[]{"COMBINE_GROUPBY_ORDERBY", 1, 0});
+ result1.add(new Object[]{
+ "AGGREGATE_GROUPBY_ORDERBY(groupKeys:noIndexCol1, noIndexCol2, aggregations:max(noIndexCol2), min"
+ + "(noIndexCol3))", 2, 1});
+ result1.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2, noIndexCol3)", 3, 2});
+ result1.add(new Object[]{"PROJECT(noIndexCol3, noIndexCol2, noIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_TEXT_INDEX(indexLookUp:text_index,operator:TEXT_MATCH,predicate:text_match"
+ + "(textIndexCol1,'foo'))", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, max(noIndexCol2), min(noIndexCol3) FROM testTable WHERE TEXT_MATCH"
+ + "(textIndexCol1, 'foo') GROUP BY noIndexCol1, noIndexCol2 ORDER BY noIndexCol1, max(noIndexCol2)";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(sort:[noIndexCol1 ASC, max(noIndexCol2) ASC],limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_GROUPBY_ORDERBY", 1, 0});
+ result2.add(new Object[]{
+ "AGGREGATE_GROUPBY_ORDERBY(groupKeys:noIndexCol1,noIndexCol2,aggregations:max(noIndexCol2), min"
+ + "(noIndexCol3))", 2, 1});
+ result2.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, invertedIndexCol1)", 3, 2});
+ result2.add(new Object[]{"PROJECT(noIndexCol3, noIndexCol2, noIndexCol1)", 4, 3});
+ result2.add(new Object[]{"FILTER_TEXT_INDEX(indexLookUp:text_index,operator:TEXT_MATCH,predicate:text_match"
+ + "(textIndexCol1,'foo'))", 5, 4});
+ }
+
+ @Test
+ public void testSelectAggregate() {
+ String query1 = "EXPLAIN PLAN FOR SELECT count(*) FROM testTable";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result1.add(new Object[]{"COMBINE_AGGREGATE", 1, 0});
+ result1.add(new Object[]{"AGGREGATE_METADATA", 2, 1});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 =
+ "EXPLAIN PLAN FOR SELECT count(*), max(noIndexCol1), sum(noIndexCol2), avg(noIndexCol2) FROM testTable";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_AGGREGATE", 1, 0});
+ result2.add(
+ new Object[]{"AGGREGATE(aggregations:count(*), max(noIndexCol1), sum(noIndexCol2), avg(noIndexCol2))", 2, 1});
+ result2.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2)", 3, 2});
+ result2.add(new Object[]{"PROJECT(noIndexCol2, noIndexCol1)", 4, 3});
+ result2.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+ }
+
+ @Test
+ public void testSelectAggregateUsingFilter() {
+ String query = "EXPLAIN PLAN FOR SELECT sum(add(noIndexCol1, noIndexCol2)), MIN(ADD(DIV(noIndexCol1,noIndexCol2),"
+ + "noIndexCol3)) FROM testTable";
+ List<Object[]> result = new ArrayList<>();
+ result.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result.add(new Object[]{"COMBINE_AGGREGATE", 1, 0});
+ result.add(new Object[]{
+ "AGGREGATE(aggregations:sum(add(noIndexCol1,noIndexCol2)), min(add(div(noIndexCol1,noIndexCol2),"
Review comment:
TRANSFORM operator was getting ignored in EXPLAIN PLAN output. Added it back.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] amrishlal commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
amrishlal commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r746144675
##########
File path: pinot-core/src/test/java/org/apache/pinot/queries/ExplainPlanQueriesTest.java
##########
@@ -0,0 +1,510 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.queries;
+
+import java.io.File;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.List;
+import org.apache.commons.io.FileUtils;
+import org.apache.pinot.common.response.broker.BrokerResponseNative;
+import org.apache.pinot.common.response.broker.ResultTable;
+import org.apache.pinot.common.utils.DataSchema;
+import org.apache.pinot.segment.local.indexsegment.immutable.ImmutableSegmentLoader;
+import org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl;
+import org.apache.pinot.segment.local.segment.index.loader.IndexLoadingConfig;
+import org.apache.pinot.segment.local.segment.readers.GenericRowRecordReader;
+import org.apache.pinot.segment.spi.ImmutableSegment;
+import org.apache.pinot.segment.spi.IndexSegment;
+import org.apache.pinot.segment.spi.creator.SegmentGeneratorConfig;
+import org.apache.pinot.spi.config.table.IndexingConfig;
+import org.apache.pinot.spi.config.table.TableConfig;
+import org.apache.pinot.spi.config.table.TableType;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.data.Schema;
+import org.apache.pinot.spi.data.readers.GenericRow;
+import org.apache.pinot.spi.utils.ReadMode;
+import org.apache.pinot.spi.utils.builder.TableConfigBuilder;
+import org.testng.Assert;
+import org.testng.annotations.BeforeClass;
+import org.testng.annotations.Test;
+
+
+public class ExplainPlanQueriesTest extends BaseQueriesTest {
+ private static final File INDEX_DIR = new File(FileUtils.getTempDirectory(), "ExplainPlanQueriesTest");
+ private static final String RAW_TABLE_NAME = "testTable";
+ private static final String SEGMENT_NAME = "testSegment";
+ private static final int NUM_RECORDS = 10;
+
+ private final static String COL1_NO_INDEX = "noIndexCol1";
+ private final static String COL2_NO_INDEX = "noIndexCol2";
+ private final static String COL3_NO_INDEX = "noIndexCol3";
+ private final static String COL1_INVERTED_INDEX = "invertedIndexCol1";
+ private final static String COL2_INVERTED_INDEX = "invertedIndexCol2";
+ private final static String COL3_INVERTED_INDEX = "invertedIndexCol3";
+ private final static String COL1_RANGE_INDEX = "rangeIndexCol1";
+ private final static String COL2_RANGE_INDEX = "rangeIndexCol2";
+ private final static String COL3_RANGE_INDEX = "rangeIndexCol3";
+ private final static String COL1_SORTED_INDEX = "sortedIndexCol1";
+ private final static String COL1_JSON_INDEX = "jsonIndexCol1";
+ private final static String COL1_TEXT_INDEX = "textIndexCol1";
+
+ private static final Schema SCHEMA = new Schema.SchemaBuilder().setSchemaName(RAW_TABLE_NAME)
+ .addSingleValueDimension(COL1_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL2_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_INVERTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_INVERTED_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_INVERTED_INDEX, FieldSpec.DataType.STRING)
+ .addSingleValueDimension(COL1_RANGE_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_SORTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL1_JSON_INDEX, FieldSpec.DataType.JSON)
+ .addSingleValueDimension(COL1_TEXT_INDEX, FieldSpec.DataType.STRING).build();
+
+ private static final DataSchema DATA_SCHEMA = new DataSchema(new String[]{"Operator", "Operator_Id", "Parent_Id"},
+ new DataSchema.ColumnDataType[]{DataSchema.ColumnDataType.STRING, DataSchema.ColumnDataType.INT,
+ DataSchema.ColumnDataType.INT});
+
+ private static final TableConfig TABLE_CONFIG =
+ new TableConfigBuilder(TableType.OFFLINE).setTableName(RAW_TABLE_NAME).build();
+
+ private IndexSegment _indexSegment;
+ private List<IndexSegment> _indexSegments;
+
+ @Override
+ protected String getFilter() {
+ return "";
+ }
+
+ @Override
+ protected IndexSegment getIndexSegment() {
+ return _indexSegment;
+ }
+
+ @Override
+ protected List<IndexSegment> getIndexSegments() {
+ return _indexSegments;
+ }
+
+ GenericRow createRecord(double doubleValue, int intValue, String stringValue, String jsonValue) {
+ GenericRow record = new GenericRow();
+ record.putValue(COL1_NO_INDEX, intValue);
+ record.putValue(COL2_NO_INDEX, intValue);
+ record.putValue(COL3_NO_INDEX, intValue);
+
+ record.putValue(COL1_INVERTED_INDEX, doubleValue);
+ record.putValue(COL2_INVERTED_INDEX, intValue);
+ record.putValue(COL3_INVERTED_INDEX, intValue);
+
+ record.putValue(COL1_RANGE_INDEX, doubleValue);
+ record.putValue(COL2_RANGE_INDEX, intValue);
+ record.putValue(COL3_RANGE_INDEX, intValue);
+
+ record.putValue(COL1_JSON_INDEX, jsonValue);
+ record.putValue(COL1_TEXT_INDEX, stringValue);
+
+ return record;
+ }
+
+ @BeforeClass
+ public void setUp()
+ throws Exception {
+ FileUtils.deleteDirectory(INDEX_DIR);
+
+ List<GenericRow> records = new ArrayList<>(NUM_RECORDS);
+ records.add(createRecord(1.1, 2, "daffy", "{\"first\": \"daffy\", \"last\": \"duck\"}"));
+
+ IndexingConfig indexingConfig = TABLE_CONFIG.getIndexingConfig();
+
+ List<String> invertedIndexColumns = Arrays.asList(COL1_INVERTED_INDEX, COL2_INVERTED_INDEX, COL3_INVERTED_INDEX);
+ indexingConfig.setInvertedIndexColumns(invertedIndexColumns);
+
+ List<String> rangeIndexColumns = Arrays.asList(COL1_RANGE_INDEX, COL2_RANGE_INDEX, COL3_RANGE_INDEX);
+ indexingConfig.setRangeIndexColumns(rangeIndexColumns);
+
+ List<String> jsonIndexColumns = Arrays.asList(COL1_JSON_INDEX);
+ indexingConfig.setJsonIndexColumns(jsonIndexColumns);
+
+ SegmentGeneratorConfig segmentGeneratorConfig = new SegmentGeneratorConfig(TABLE_CONFIG, SCHEMA);
+ segmentGeneratorConfig.setTableName(RAW_TABLE_NAME);
+ segmentGeneratorConfig.setSegmentName(SEGMENT_NAME);
+ segmentGeneratorConfig.setOutDir(INDEX_DIR.getPath());
+
+ SegmentIndexCreationDriverImpl driver = new SegmentIndexCreationDriverImpl();
+ driver.init(segmentGeneratorConfig, new GenericRowRecordReader(records));
+ driver.build();
+
+ IndexLoadingConfig indexLoadingConfig = new IndexLoadingConfig();
+ indexLoadingConfig.setTableConfig(TABLE_CONFIG);
+ indexLoadingConfig.setInvertedIndexColumns(new HashSet<>(invertedIndexColumns));
+ indexLoadingConfig.setRangeIndexColumns(new HashSet<>(rangeIndexColumns));
+ indexLoadingConfig.setJsonIndexColumns(new HashSet<String>(jsonIndexColumns));
+ indexLoadingConfig.setReadMode(ReadMode.mmap);
+
+ ImmutableSegment immutableSegment =
+ ImmutableSegmentLoader.load(new File(INDEX_DIR, SEGMENT_NAME), indexLoadingConfig);
+ _indexSegment = immutableSegment;
+ _indexSegments = Arrays.asList(immutableSegment);
+ }
+
+ /** Checks the correctness of EXPLAIN PLAN output. */
+ private void check(String query, ResultTable expected) {
+ BrokerResponseNative response = getBrokerResponseForSqlQuery(query);
+ ResultTable actual = response.getResultTable();
+
+ System.out.println(resultTableToString("Expected", expected));
+ System.out.println(resultTableToString("Actual", actual));
+
+ Assert.assertEquals(actual.getDataSchema(), expected.getDataSchema());
+ Assert.assertEquals(actual.getRows().size(), expected.getRows().size());
+
+ List<Object[]> expectedRows = expected.getRows();
+ List<Object[]> actualRows = actual.getRows();
+ for (int i = 0; i < expectedRows.size(); i++) {
+ Object[] expectedRow = expectedRows.get(i);
+ Object[] actualRow = actualRows.get(i);
+ Assert.assertEquals(actualRow.length, expectedRow.length);
+
+ for (int j = 0; j < actualRow.length; j++) {
+ Assert.assertEquals(actualRow[j], expectedRow[j]);
+ }
+ }
+ }
+
+ /** This function is used to generated expected results while creating test cases; otherwise, not used. */
+ private static String resultTableToString(String tag, ResultTable table) {
+ System.out.println("\n" + tag + ":");
+ List<Object[]> rows = table.getRows();
+
+ StringBuffer buffer = new StringBuffer();
+ for (Object[] row : rows) {
+ buffer.append("result.add(new Object[]{\"" + row[0] + "\", " + row[1] + ", " + row[2] + "});\n");
+ }
+
+ return buffer.toString();
+ }
+
+ @Test
+ public void testSelect() {
+ String query1 = "EXPLAIN PLAN FOR SELECT * FROM testTable";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(
+ new Object[]{"SELECT(selectList:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, jsonIndexCol1, "
+ + "noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, "
+ + "jsonIndexCol1, noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 3, 2});
+ result1.add(
+ new Object[]{"PROJECT(sortedIndexCol1, noIndexCol3, rangeIndexCol1, rangeIndexCol2, jsonIndexCol1, "
+ + "invertedIndexCol1, noIndexCol2, invertedIndexCol2, noIndexCol1, invertedIndexCol3, rangeIndexCol3, "
+ + "textIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 = "EXPLAIN PLAN FOR SELECT 'mickey' FROM testTable";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:'mickey')", 2, 1});
+ result2.add(new Object[]{"PROJECT(ALL)", 3, 2});
Review comment:
It should probably be `PROJECT()` since the statement is projecting a literal value, but not any columns. I am not sure if there is a lot of value in listing all the columns of the PROJECT in the explain plan output specially since we are querying only one table? What do you think about just putting a count of columns being projected?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] amrishlal commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
amrishlal commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r746092849
##########
File path: pinot-common/src/main/java/org/apache/pinot/common/request/Selection.java
##########
@@ -124,15 +124,15 @@ public short getThriftFieldId() {
public static final java.util.Map<_Fields, org.apache.thrift.meta_data.FieldMetaData> metaDataMap;
static {
java.util.Map<_Fields, org.apache.thrift.meta_data.FieldMetaData> tmpMap = new java.util.EnumMap<_Fields, org.apache.thrift.meta_data.FieldMetaData>(_Fields.class);
- tmpMap.put(_Fields.SELECTION_COLUMNS, new org.apache.thrift.meta_data.FieldMetaData("selectionColumns", org.apache.thrift.TFieldRequirementType.OPTIONAL,
- new org.apache.thrift.meta_data.ListMetaData(org.apache.thrift.protocol.TType.LIST,
+ tmpMap.put(_Fields.SELECTION_COLUMNS, new org.apache.thrift.meta_data.FieldMetaData("selectionColumns", org.apache.thrift.TFieldRequirementType.OPTIONAL,
Review comment:
There are extra spaces in the current formatting. Running thrift seems to have fixed the spaces.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] amrishlal commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
amrishlal commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r746173133
##########
File path: pinot-core/src/test/java/org/apache/pinot/queries/ExplainPlanQueriesTest.java
##########
@@ -0,0 +1,510 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.queries;
+
+import java.io.File;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.List;
+import org.apache.commons.io.FileUtils;
+import org.apache.pinot.common.response.broker.BrokerResponseNative;
+import org.apache.pinot.common.response.broker.ResultTable;
+import org.apache.pinot.common.utils.DataSchema;
+import org.apache.pinot.segment.local.indexsegment.immutable.ImmutableSegmentLoader;
+import org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl;
+import org.apache.pinot.segment.local.segment.index.loader.IndexLoadingConfig;
+import org.apache.pinot.segment.local.segment.readers.GenericRowRecordReader;
+import org.apache.pinot.segment.spi.ImmutableSegment;
+import org.apache.pinot.segment.spi.IndexSegment;
+import org.apache.pinot.segment.spi.creator.SegmentGeneratorConfig;
+import org.apache.pinot.spi.config.table.IndexingConfig;
+import org.apache.pinot.spi.config.table.TableConfig;
+import org.apache.pinot.spi.config.table.TableType;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.data.Schema;
+import org.apache.pinot.spi.data.readers.GenericRow;
+import org.apache.pinot.spi.utils.ReadMode;
+import org.apache.pinot.spi.utils.builder.TableConfigBuilder;
+import org.testng.Assert;
+import org.testng.annotations.BeforeClass;
+import org.testng.annotations.Test;
+
+
+public class ExplainPlanQueriesTest extends BaseQueriesTest {
+ private static final File INDEX_DIR = new File(FileUtils.getTempDirectory(), "ExplainPlanQueriesTest");
+ private static final String RAW_TABLE_NAME = "testTable";
+ private static final String SEGMENT_NAME = "testSegment";
+ private static final int NUM_RECORDS = 10;
+
+ private final static String COL1_NO_INDEX = "noIndexCol1";
+ private final static String COL2_NO_INDEX = "noIndexCol2";
+ private final static String COL3_NO_INDEX = "noIndexCol3";
+ private final static String COL1_INVERTED_INDEX = "invertedIndexCol1";
+ private final static String COL2_INVERTED_INDEX = "invertedIndexCol2";
+ private final static String COL3_INVERTED_INDEX = "invertedIndexCol3";
+ private final static String COL1_RANGE_INDEX = "rangeIndexCol1";
+ private final static String COL2_RANGE_INDEX = "rangeIndexCol2";
+ private final static String COL3_RANGE_INDEX = "rangeIndexCol3";
+ private final static String COL1_SORTED_INDEX = "sortedIndexCol1";
+ private final static String COL1_JSON_INDEX = "jsonIndexCol1";
+ private final static String COL1_TEXT_INDEX = "textIndexCol1";
+
+ private static final Schema SCHEMA = new Schema.SchemaBuilder().setSchemaName(RAW_TABLE_NAME)
+ .addSingleValueDimension(COL1_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL2_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_INVERTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_INVERTED_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_INVERTED_INDEX, FieldSpec.DataType.STRING)
+ .addSingleValueDimension(COL1_RANGE_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_SORTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL1_JSON_INDEX, FieldSpec.DataType.JSON)
+ .addSingleValueDimension(COL1_TEXT_INDEX, FieldSpec.DataType.STRING).build();
+
+ private static final DataSchema DATA_SCHEMA = new DataSchema(new String[]{"Operator", "Operator_Id", "Parent_Id"},
+ new DataSchema.ColumnDataType[]{DataSchema.ColumnDataType.STRING, DataSchema.ColumnDataType.INT,
+ DataSchema.ColumnDataType.INT});
+
+ private static final TableConfig TABLE_CONFIG =
+ new TableConfigBuilder(TableType.OFFLINE).setTableName(RAW_TABLE_NAME).build();
+
+ private IndexSegment _indexSegment;
+ private List<IndexSegment> _indexSegments;
+
+ @Override
+ protected String getFilter() {
+ return "";
+ }
+
+ @Override
+ protected IndexSegment getIndexSegment() {
+ return _indexSegment;
+ }
+
+ @Override
+ protected List<IndexSegment> getIndexSegments() {
+ return _indexSegments;
+ }
+
+ GenericRow createRecord(double doubleValue, int intValue, String stringValue, String jsonValue) {
+ GenericRow record = new GenericRow();
+ record.putValue(COL1_NO_INDEX, intValue);
+ record.putValue(COL2_NO_INDEX, intValue);
+ record.putValue(COL3_NO_INDEX, intValue);
+
+ record.putValue(COL1_INVERTED_INDEX, doubleValue);
+ record.putValue(COL2_INVERTED_INDEX, intValue);
+ record.putValue(COL3_INVERTED_INDEX, intValue);
+
+ record.putValue(COL1_RANGE_INDEX, doubleValue);
+ record.putValue(COL2_RANGE_INDEX, intValue);
+ record.putValue(COL3_RANGE_INDEX, intValue);
+
+ record.putValue(COL1_JSON_INDEX, jsonValue);
+ record.putValue(COL1_TEXT_INDEX, stringValue);
+
+ return record;
+ }
+
+ @BeforeClass
+ public void setUp()
+ throws Exception {
+ FileUtils.deleteDirectory(INDEX_DIR);
+
+ List<GenericRow> records = new ArrayList<>(NUM_RECORDS);
+ records.add(createRecord(1.1, 2, "daffy", "{\"first\": \"daffy\", \"last\": \"duck\"}"));
+
+ IndexingConfig indexingConfig = TABLE_CONFIG.getIndexingConfig();
+
+ List<String> invertedIndexColumns = Arrays.asList(COL1_INVERTED_INDEX, COL2_INVERTED_INDEX, COL3_INVERTED_INDEX);
+ indexingConfig.setInvertedIndexColumns(invertedIndexColumns);
+
+ List<String> rangeIndexColumns = Arrays.asList(COL1_RANGE_INDEX, COL2_RANGE_INDEX, COL3_RANGE_INDEX);
+ indexingConfig.setRangeIndexColumns(rangeIndexColumns);
+
+ List<String> jsonIndexColumns = Arrays.asList(COL1_JSON_INDEX);
+ indexingConfig.setJsonIndexColumns(jsonIndexColumns);
+
+ SegmentGeneratorConfig segmentGeneratorConfig = new SegmentGeneratorConfig(TABLE_CONFIG, SCHEMA);
+ segmentGeneratorConfig.setTableName(RAW_TABLE_NAME);
+ segmentGeneratorConfig.setSegmentName(SEGMENT_NAME);
+ segmentGeneratorConfig.setOutDir(INDEX_DIR.getPath());
+
+ SegmentIndexCreationDriverImpl driver = new SegmentIndexCreationDriverImpl();
+ driver.init(segmentGeneratorConfig, new GenericRowRecordReader(records));
+ driver.build();
+
+ IndexLoadingConfig indexLoadingConfig = new IndexLoadingConfig();
+ indexLoadingConfig.setTableConfig(TABLE_CONFIG);
+ indexLoadingConfig.setInvertedIndexColumns(new HashSet<>(invertedIndexColumns));
+ indexLoadingConfig.setRangeIndexColumns(new HashSet<>(rangeIndexColumns));
+ indexLoadingConfig.setJsonIndexColumns(new HashSet<String>(jsonIndexColumns));
+ indexLoadingConfig.setReadMode(ReadMode.mmap);
+
+ ImmutableSegment immutableSegment =
+ ImmutableSegmentLoader.load(new File(INDEX_DIR, SEGMENT_NAME), indexLoadingConfig);
+ _indexSegment = immutableSegment;
+ _indexSegments = Arrays.asList(immutableSegment);
+ }
+
+ /** Checks the correctness of EXPLAIN PLAN output. */
+ private void check(String query, ResultTable expected) {
+ BrokerResponseNative response = getBrokerResponseForSqlQuery(query);
+ ResultTable actual = response.getResultTable();
+
+ System.out.println(resultTableToString("Expected", expected));
+ System.out.println(resultTableToString("Actual", actual));
+
+ Assert.assertEquals(actual.getDataSchema(), expected.getDataSchema());
+ Assert.assertEquals(actual.getRows().size(), expected.getRows().size());
+
+ List<Object[]> expectedRows = expected.getRows();
+ List<Object[]> actualRows = actual.getRows();
+ for (int i = 0; i < expectedRows.size(); i++) {
+ Object[] expectedRow = expectedRows.get(i);
+ Object[] actualRow = actualRows.get(i);
+ Assert.assertEquals(actualRow.length, expectedRow.length);
+
+ for (int j = 0; j < actualRow.length; j++) {
+ Assert.assertEquals(actualRow[j], expectedRow[j]);
+ }
+ }
+ }
+
+ /** This function is used to generated expected results while creating test cases; otherwise, not used. */
+ private static String resultTableToString(String tag, ResultTable table) {
+ System.out.println("\n" + tag + ":");
+ List<Object[]> rows = table.getRows();
+
+ StringBuffer buffer = new StringBuffer();
+ for (Object[] row : rows) {
+ buffer.append("result.add(new Object[]{\"" + row[0] + "\", " + row[1] + ", " + row[2] + "});\n");
+ }
+
+ return buffer.toString();
+ }
+
+ @Test
+ public void testSelect() {
+ String query1 = "EXPLAIN PLAN FOR SELECT * FROM testTable";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(
+ new Object[]{"SELECT(selectList:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, jsonIndexCol1, "
+ + "noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, "
+ + "jsonIndexCol1, noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 3, 2});
+ result1.add(
+ new Object[]{"PROJECT(sortedIndexCol1, noIndexCol3, rangeIndexCol1, rangeIndexCol2, jsonIndexCol1, "
+ + "invertedIndexCol1, noIndexCol2, invertedIndexCol2, noIndexCol1, invertedIndexCol3, rangeIndexCol3, "
+ + "textIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 = "EXPLAIN PLAN FOR SELECT 'mickey' FROM testTable";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:'mickey')", 2, 1});
+ result2.add(new Object[]{"PROJECT(ALL)", 3, 2});
+ result2.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 4, 3});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+
+ String query3 = "EXPLAIN PLAN FOR SELECT invertedIndexCol1, noIndexCol1 FROM testTable LIMIT 100";
+ List<Object[]> result3 = new ArrayList<>();
+ result3.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result3.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result3.add(new Object[]{"SELECT(selectList:invertedIndexCol1, noIndexCol1)", 2, 1});
+ result3.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result3.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result3.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query3, new ResultTable(DATA_SCHEMA, result3));
+
+ String query4 = "EXPLAIN PLAN FOR SELECT DISTINCT invertedIndexCol1, noIndexCol1 FROM testTable LIMIT 100";
+ List<Object[]> result4 = new ArrayList<>();
+ result4.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result4.add(new Object[]{"COMBINE_DISTINCT", 1, 0});
+ result4.add(new Object[]{"DISTINCT(keyColumns:invertedIndexCol1, noIndexCol1)", 2, 1});
+ result4.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result4.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result4.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query4, new ResultTable(DATA_SCHEMA, result4));
+ }
Review comment:
Added test cases for these.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7568](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (01e4713) into [master](https://codecov.io/gh/apache/pinot/commit/767aa8abfb5bf085ba0a7ae5ff4024679f27816e?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (767aa8a) will **decrease** coverage by `40.50%`.
> The diff coverage is `49.81%`.
> :exclamation: Current head 01e4713 differs from pull request most recent head 0a55f32. Consider uploading reports for the commit 0a55f32 to get more accurate results
[](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7568 +/- ##
=============================================
- Coverage 71.69% 31.18% -40.51%
=============================================
Files 1579 1571 -8
Lines 80790 80843 +53
Branches 12003 12039 +36
=============================================
- Hits 57922 25213 -32709
- Misses 18981 53460 +34479
+ Partials 3887 2170 -1717
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `29.53% <49.09%> (+0.08%)` | :arrow_up: |
| integration2 | `27.92% <47.45%> (-0.01%)` | :arrow_down: |
| unittests1 | `?` | |
| unittests2 | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...ker/routing/instanceselector/InstanceSelector.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcm91dGluZy9pbnN0YW5jZXNlbGVjdG9yL0luc3RhbmNlU2VsZWN0b3IuamF2YQ==) | `100.00% <ø> (ø)` | |
| [...ceselector/StrictReplicaGroupInstanceSelector.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcm91dGluZy9pbnN0YW5jZXNlbGVjdG9yL1N0cmljdFJlcGxpY2FHcm91cEluc3RhbmNlU2VsZWN0b3IuamF2YQ==) | `62.12% <ø> (-27.28%)` | :arrow_down: |
| [...roker/routing/segmentpruner/TimeSegmentPruner.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcm91dGluZy9zZWdtZW50cHJ1bmVyL1RpbWVTZWdtZW50UHJ1bmVyLmphdmE=) | `0.00% <0.00%> (-80.96%)` | :arrow_down: |
| [...va/org/apache/pinot/controller/ControllerConf.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9Db250cm9sbGVyQ29uZi5qYXZh) | `50.00% <ø> (-7.32%)` | :arrow_down: |
| [...in/java/org/apache/pinot/core/common/Operator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9jb21tb24vT3BlcmF0b3IuamF2YQ==) | `0.00% <ø> (ø)` | |
| [...operator/AcquireReleaseColumnsSegmentOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9BY3F1aXJlUmVsZWFzZUNvbHVtbnNTZWdtZW50T3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...he/pinot/core/operator/BitmapDocIdSetOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CaXRtYXBEb2NJZFNldE9wZXJhdG9yLmphdmE=) | `68.18% <0.00%> (-6.82%)` | :arrow_down: |
| [...erator/combine/AggregationOnlyCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0FnZ3JlZ2F0aW9uT25seUNvbWJpbmVPcGVyYXRvci5qYXZh) | `69.23% <0.00%> (-5.77%)` | :arrow_down: |
| [...core/operator/combine/DistinctCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0Rpc3RpbmN0Q29tYmluZU9wZXJhdG9yLmphdmE=) | `77.77% <0.00%> (-3.00%)` | :arrow_down: |
| [.../core/operator/combine/GroupByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0dyb3VwQnlDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `73.91% <0.00%> (-1.09%)` | :arrow_down: |
| ... and [1132 more](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [767aa8a...0a55f32](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] amrishlal commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
amrishlal commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r759890644
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/DocIdSetOperator.java
##########
@@ -34,6 +37,7 @@
*/
public class DocIdSetOperator extends BaseOperator<DocIdSetBlock> {
private static final String OPERATOR_NAME = "DocIdSetOperator";
+ private static final String EXPLAIN_NAME = null;
Review comment:
There isn't any big reason behind it, except that we didn't think this would be useful enough to show in the EXPLAIN PLAN output. @siddharthteotia ?
In either case, I would suggest that we not get into particular operators of EXPLAIN PLAN and output display format etc. in this PR. As the PR states EXPLAIN PLAN output is only for information purposes and we are not guaranteeing that the output will remain backward compatible. So we can change what is displayed in the EXPLAIN PLAN output anytime as needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] amrishlal commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
amrishlal commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r758809219
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/InstanceResponseOperator.java
##########
@@ -113,9 +116,34 @@ private IntermediateResultsBlock getCombinedResults() {
}
}
- @Override
- public String getOperatorName() {
- return OPERATOR_NAME;
+ /*
+ * Calculate totalThreadCpuTimeNs based on totalWallClockTimeNs, multipleThreadCpuTimeNs, and numServerThreads.
+ * System activities time such as OS paging, GC, context switching are not captured by totalThreadCpuTimeNs.
+ * For example, let's divide query processing into 4 phases:
+ * - phase 1: single thread preparing. Time used: T1
+ * - phase 2: N threads processing segments in parallel, each thread use time T2
+ * - phase 3: GC/OS paging. Time used: T3
+ * - phase 4: single thread merging intermediate results blocks. Time used: T4
+ *
+ * Then we have following equations:
+ * - singleThreadCpuTimeNs = T1 + T4
+ * - multipleThreadCpuTimeNs = T2 * N
+ * - totalWallClockTimeNs = T1 + T2 + T3 + T4 = singleThreadCpuTimeNs + T2 + T3
+ * - totalThreadCpuTimeNsWithoutSystemActivities = T1 + T2 * N + T4 = singleThreadCpuTimeNs + T2 * N
+ * - systemActivitiesTimeNs = T3 = (totalWallClockTimeNs - totalThreadCpuTimeNsWithoutSystemActivities) + T2 * (N - 1)
+ *
+ * Thus:
+ * totalThreadCpuTimeNsWithSystemActivities = totalThreadCpuTimeNsWithoutSystemActivities + systemActivitiesTimeNs
+ * = totalThreadCpuTimeNsWithoutSystemActivities + T3
+ * = totalThreadCpuTimeNsWithoutSystemActivities + (totalWallClockTimeNs -
+ * totalThreadCpuTimeNsWithoutSystemActivities) + T2 * (N - 1)
+ * = totalWallClockTimeNs + T2 * (N - 1)
+ * = totalWallClockTimeNs + (multipleThreadCpuTimeNs / N) * (N - 1)
+ */
+ public static long calTotalThreadCpuTimeNs(long totalWallClockTimeNs, long multipleThreadCpuTimeNs,
Review comment:
Seems like a merge conflict issue. Will fix.
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/filter/predicate/PredicateEvaluator.java
##########
@@ -28,6 +28,16 @@
* APIs for both dictionary based and raw value based predicate evaluator
*/
+ /**
+ * Set the predicate
+ */
+ void setPredicate(Predicate predicate);
Review comment:
I see what you mean, but It is used in line 107 of `PredicateEvaluator.java` to pass predicate into `PredicateEvaluator`. Let me know if you have a better way to doing this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7568](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (576ceaf) into [master](https://codecov.io/gh/apache/pinot/commit/2107b2cd1880ec74934ec5412dbee8c874f05610?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (2107b2c) will **decrease** coverage by `16.29%`.
> The diff coverage is `2.16%`.
[](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7568 +/- ##
=============================================
- Coverage 30.79% 14.50% -16.30%
- Complexity 0 80 +80
=============================================
Files 1570 1534 -36
Lines 80024 78852 -1172
Branches 11904 11799 -105
=============================================
- Hits 24644 11434 -13210
- Misses 53274 66580 +13306
+ Partials 2106 838 -1268
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| integration2 | `?` | |
| unittests2 | `14.50% <2.16%> (?)` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...java/org/apache/pinot/common/utils/DataSchema.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvRGF0YVNjaGVtYS5qYXZh) | `0.00% <0.00%> (-46.19%)` | :arrow_down: |
| [...org/apache/pinot/sql/parsers/CalciteSqlParser.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcWwvcGFyc2Vycy9DYWxjaXRlU3FsUGFyc2VyLmphdmE=) | `0.00% <0.00%> (-69.74%)` | :arrow_down: |
| [...in/java/org/apache/pinot/core/common/Operator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9jb21tb24vT3BlcmF0b3IuamF2YQ==) | `0.00% <ø> (ø)` | |
| [...operator/AcquireReleaseColumnsSegmentOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9BY3F1aXJlUmVsZWFzZUNvbHVtbnNTZWdtZW50T3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...a/org/apache/pinot/core/operator/BaseOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CYXNlT3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (-38.47%)` | :arrow_down: |
| [...he/pinot/core/operator/BitmapDocIdSetOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CaXRtYXBEb2NJZFNldE9wZXJhdG9yLmphdmE=) | `0.00% <0.00%> (-75.00%)` | :arrow_down: |
| [...g/apache/pinot/core/operator/DocIdSetOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9Eb2NJZFNldE9wZXJhdG9yLmphdmE=) | `0.00% <0.00%> (-92.31%)` | :arrow_down: |
| [.../pinot/core/operator/InstanceResponseOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9JbnN0YW5jZVJlc3BvbnNlT3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (-83.88%)` | :arrow_down: |
| [...apache/pinot/core/operator/ProjectionOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9Qcm9qZWN0aW9uT3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (-73.34%)` | :arrow_down: |
| [...erator/combine/AggregationOnlyCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0FnZ3JlZ2F0aW9uT25seUNvbWJpbmVPcGVyYXRvci5qYXZh) | `0.00% <0.00%> (-75.00%)` | :arrow_down: |
| ... and [845 more](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [2107b2c...576ceaf](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] amrishlal commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
amrishlal commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r741522618
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/query/AggregationOperator.java
##########
@@ -75,11 +79,36 @@ public String getOperatorName() {
return OPERATOR_NAME;
}
+ @Override
+ public String getExplainPlanName() {
+ return EXPLAIN_NAME;
+ }
+
+ @Override
+ public List<Operator> getChildOperators() {
+ return Arrays.asList(_transformOperator);
+ }
+
@Override
public ExecutionStatistics getExecutionStatistics() {
long numEntriesScannedInFilter = _transformOperator.getExecutionStatistics().getNumEntriesScannedInFilter();
long numEntriesScannedPostFilter = (long) _numDocsScanned * _transformOperator.getNumColumnsProjected();
return new ExecutionStatistics(_numDocsScanned, numEntriesScannedInFilter, numEntriesScannedPostFilter,
_numTotalDocs);
}
+
+ @Override
+ public String toExplainString() {
+ StringBuilder stringBuilder = new StringBuilder(getExplainPlanName()).append("(aggregations:");
+ int count = 0;
+ for (AggregationFunction func : _aggregationFunctions) {
Review comment:
I reworked this a bit. Please take a look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7568](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (7564824) into [master](https://codecov.io/gh/apache/pinot/commit/2107b2cd1880ec74934ec5412dbee8c874f05610?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (2107b2c) will **decrease** coverage by `16.28%`.
> The diff coverage is `2.16%`.
[](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7568 +/- ##
=============================================
- Coverage 30.79% 14.50% -16.29%
- Complexity 0 80 +80
=============================================
Files 1570 1534 -36
Lines 80024 78853 -1171
Branches 11904 11799 -105
=============================================
- Hits 24644 11440 -13204
- Misses 53274 66575 +13301
+ Partials 2106 838 -1268
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| integration2 | `?` | |
| unittests2 | `14.50% <2.16%> (?)` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...java/org/apache/pinot/common/utils/DataSchema.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvRGF0YVNjaGVtYS5qYXZh) | `0.00% <0.00%> (-46.19%)` | :arrow_down: |
| [...org/apache/pinot/sql/parsers/CalciteSqlParser.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcWwvcGFyc2Vycy9DYWxjaXRlU3FsUGFyc2VyLmphdmE=) | `0.00% <0.00%> (-69.74%)` | :arrow_down: |
| [...in/java/org/apache/pinot/core/common/Operator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9jb21tb24vT3BlcmF0b3IuamF2YQ==) | `0.00% <ø> (ø)` | |
| [...operator/AcquireReleaseColumnsSegmentOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9BY3F1aXJlUmVsZWFzZUNvbHVtbnNTZWdtZW50T3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...a/org/apache/pinot/core/operator/BaseOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CYXNlT3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (-38.47%)` | :arrow_down: |
| [...he/pinot/core/operator/BitmapDocIdSetOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CaXRtYXBEb2NJZFNldE9wZXJhdG9yLmphdmE=) | `0.00% <0.00%> (-75.00%)` | :arrow_down: |
| [...g/apache/pinot/core/operator/DocIdSetOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9Eb2NJZFNldE9wZXJhdG9yLmphdmE=) | `0.00% <0.00%> (-92.31%)` | :arrow_down: |
| [.../pinot/core/operator/InstanceResponseOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9JbnN0YW5jZVJlc3BvbnNlT3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (-83.88%)` | :arrow_down: |
| [...apache/pinot/core/operator/ProjectionOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9Qcm9qZWN0aW9uT3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (-73.34%)` | :arrow_down: |
| [...erator/combine/AggregationOnlyCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0FnZ3JlZ2F0aW9uT25seUNvbWJpbmVPcGVyYXRvci5qYXZh) | `0.00% <0.00%> (-75.00%)` | :arrow_down: |
| ... and [845 more](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [2107b2c...7564824](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] amrishlal commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
amrishlal commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r741522258
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/AcquireReleaseColumnsSegmentOperator.java
##########
@@ -77,6 +81,16 @@ public String getOperatorName() {
return OPERATOR_NAME;
}
+ @Override
+ public String getExplainPlanName() {
+ return EXPLAIN_NAME;
+ }
+
+ @Override
+ public List<Operator> getChildOperators() {
+ return Arrays.asList(_childOperator);
Review comment:
Fixed.
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/filter/AndFilterOperator.java
##########
@@ -47,4 +49,18 @@ protected FilterBlock getNextBlock() {
public String getOperatorName() {
return OPERATOR_NAME;
}
+
+ @Override
+ public List<Operator> getChildOperators() {
+ ArrayList<Operator> result = new ArrayList<>();
Review comment:
Fixed.
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/filter/OrFilterOperator.java
##########
@@ -49,4 +50,18 @@ protected FilterBlock getNextBlock() {
public String getOperatorName() {
return OPERATOR_NAME;
}
+
+ @Override
+ public String getExplainPlanName() {
+ return EXPLAIN_NAME;
+ }
+
+ @Override
+ public List<Operator> getChildOperators() {
+ List<Operator> operators = new ArrayList<>();
Review comment:
Fixed.
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/query/AggregationOperator.java
##########
@@ -75,11 +79,36 @@ public String getOperatorName() {
return OPERATOR_NAME;
}
+ @Override
+ public String getExplainPlanName() {
+ return EXPLAIN_NAME;
+ }
+
+ @Override
+ public List<Operator> getChildOperators() {
+ return Arrays.asList(_transformOperator);
+ }
+
@Override
public ExecutionStatistics getExecutionStatistics() {
long numEntriesScannedInFilter = _transformOperator.getExecutionStatistics().getNumEntriesScannedInFilter();
long numEntriesScannedPostFilter = (long) _numDocsScanned * _transformOperator.getNumColumnsProjected();
return new ExecutionStatistics(_numDocsScanned, numEntriesScannedInFilter, numEntriesScannedPostFilter,
_numTotalDocs);
}
+
+ @Override
+ public String toExplainString() {
+ StringBuilder stringBuilder = new StringBuilder(getExplainPlanName()).append("(aggregations:");
+ int count = 0;
+ for (AggregationFunction func : _aggregationFunctions) {
Review comment:
I reworked this a bit. Please take a look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] amrishlal commented on a change in pull request #7568: [WORK IN PROGRESS] EXPLAIN PLAN code
Posted by GitBox <gi...@apache.org>.
amrishlal commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r727439934
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/AcquireReleaseColumnsSegmentOperator.java
##########
@@ -77,6 +81,16 @@ public String getOperatorName() {
return OPERATOR_NAME;
}
+ @Override
+ public String getExplainPlanName() {
+ return EXPLAIN_NAME;
+ }
+
+ @Override
+ public List<Operator> getChildOperators() {
+ return Arrays.asList(_childOperator);
Review comment:
OK, but let's please hold on the reviews for now until I can get the test cases passing and add a decent description :-) I will open it for review soon.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7568](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (bdba462) into [master](https://codecov.io/gh/apache/pinot/commit/2107b2cd1880ec74934ec5412dbee8c874f05610?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (2107b2c) will **increase** coverage by `34.37%`.
> The diff coverage is `68.66%`.
[](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7568 +/- ##
=============================================
+ Coverage 30.79% 65.17% +34.37%
- Complexity 0 4066 +4066
=============================================
Files 1570 1532 -38
Lines 80024 79030 -994
Branches 11904 11824 -80
=============================================
+ Hits 24644 51505 +26861
+ Misses 53274 23867 -29407
- Partials 2106 3658 +1552
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| integration2 | `?` | |
| unittests1 | `68.67% <70.63%> (?)` | |
| unittests2 | `14.49% <2.08%> (?)` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...in/java/org/apache/pinot/core/common/Operator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9jb21tb24vT3BlcmF0b3IuamF2YQ==) | `0.00% <ø> (ø)` | |
| [...operator/AcquireReleaseColumnsSegmentOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9BY3F1aXJlUmVsZWFzZUNvbHVtbnNTZWdtZW50T3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...he/pinot/core/operator/BitmapDocIdSetOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CaXRtYXBEb2NJZFNldE9wZXJhdG9yLmphdmE=) | `68.18% <0.00%> (-6.82%)` | :arrow_down: |
| [.../core/operator/combine/GroupByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0dyb3VwQnlDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `73.91% <0.00%> (-5.46%)` | :arrow_down: |
| [...nMaxValueBasedSelectionOrderByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL01pbk1heFZhbHVlQmFzZWRTZWxlY3Rpb25PcmRlckJ5Q29tYmluZU9wZXJhdG9yLmphdmE=) | `60.60% <0.00%> (-14.80%)` | :arrow_down: |
| [...or/dociditerators/ExpressionScanDocIdIterator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9kb2NpZGl0ZXJhdG9ycy9FeHByZXNzaW9uU2NhbkRvY0lkSXRlcmF0b3IuamF2YQ==) | `47.18% <0.00%> (-15.68%)` | :arrow_down: |
| [...inot/core/operator/filter/EmptyFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvRW1wdHlGaWx0ZXJPcGVyYXRvci5qYXZh) | `62.50% <0.00%> (-17.50%)` | :arrow_down: |
| [...ot/core/operator/filter/H3IndexFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvSDNJbmRleEZpbHRlck9wZXJhdG9yLmphdmE=) | `81.25% <0.00%> (+81.25%)` | :arrow_up: |
| [...ore/operator/query/AggregationGroupByOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9xdWVyeS9BZ2dyZWdhdGlvbkdyb3VwQnlPcGVyYXRvci5qYXZh) | `47.05% <0.00%> (-47.68%)` | :arrow_down: |
| [...perator/query/DictionaryBasedDistinctOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9xdWVyeS9EaWN0aW9uYXJ5QmFzZWREaXN0aW5jdE9wZXJhdG9yLmphdmE=) | `75.00% <0.00%> (+25.00%)` | :arrow_up: |
| ... and [1342 more](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [2107b2c...bdba462](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] amrishlal commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
amrishlal commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r746094606
##########
File path: pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseBrokerRequestHandler.java
##########
@@ -225,7 +225,16 @@ private BrokerResponseNative handleSQLRequest(long requestId, String query, Json
if (isLiteralOnlyQuery(pinotQuery)) {
LOGGER.debug("Request {} contains only Literal, skipping server query: {}", requestId, query);
try {
- return processLiteralOnlyQuery(pinotQuery, compilationStartTimeNs, requestStatistics);
+ BrokerResponseNative responseForLiteralOnly =
+ processLiteralOnlyQuery(pinotQuery, compilationStartTimeNs, requestStatistics);
+ if (pinotQuery.isExplain()) {
+ // Generate explain results to show that this is a SELECT on a literal value.
+ List<Object[]> rows = new ArrayList<>();
+ rows.add(new Object[]{"SELECT(selectList:literal)", 0, -1});
Review comment:
I think we should avoid regenerating the original query within the output of EXPLAIN PLAN. Simply saying that the select list contained only literals should be enough since the user can look up and correlate the explain plan output with the actual query. In future though, it is quite possible that more details may need to be added, but for now it will be good idea to keep the output minimal.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] amrishlal commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
amrishlal commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r746092017
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/plan/FilterPlanNode.java
##########
@@ -217,9 +216,9 @@ private BaseFilterOperator constructPhysicalOperator(FilterContext filter) {
return FilterOperatorUtils.getLeafFilterOperator(predicateEvaluator, dataSource, _numDocs);
case JSON_MATCH:
JsonIndexReader jsonIndex = dataSource.getJsonIndex();
- Preconditions.checkState(jsonIndex != null, "Cannot apply JSON_MATCH on column: %s without json index",
- column);
- return new JsonMatchFilterOperator(jsonIndex, ((JsonMatchPredicate) predicate).getValue(), _numDocs);
+ Preconditions
+ .checkState(jsonIndex != null, "Cannot apply JSON_MATCH on column: %s without json index", column);
Review comment:
Fixed.
##########
File path: pinot-broker/src/test/java/org/apache/pinot/broker/requesthandler/LiteralOnlyBrokerRequestTest.java
##########
@@ -177,4 +177,55 @@ public void testBrokerRequestHandlerWithAsFunction()
Assert.assertEquals(brokerResponse.getResultTable().getRows().get(0)[1], 1577836800000L);
Assert.assertEquals(brokerResponse.getTotalDocs(), 0);
}
+
+ /** Tests for EXPLAIN PLAN for literal only queries. */
+ @Test
+ public void testExplainPlanLiteralOnly()
+ throws Exception {
+ SingleConnectionBrokerRequestHandler requestHandler =
+ new SingleConnectionBrokerRequestHandler(new PinotConfiguration(), null, ACCESS_CONTROL_FACTORY, null, null,
+ new BrokerMetrics("", PinotMetricUtils.getPinotMetricsRegistry(), true, Collections.emptySet()), null);
+
+ // Test 1: select constant
+ JsonNode request = new ObjectMapper().readTree("{\"sql\":\"EXPLAIN PLAN FOR SELECT 1.5, 'test'\"}");
Review comment:
Fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] amrishlal commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
amrishlal commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r747816929
##########
File path: pinot-core/src/test/java/org/apache/pinot/queries/ExplainPlanQueriesTest.java
##########
@@ -0,0 +1,510 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.queries;
+
+import java.io.File;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.List;
+import org.apache.commons.io.FileUtils;
+import org.apache.pinot.common.response.broker.BrokerResponseNative;
+import org.apache.pinot.common.response.broker.ResultTable;
+import org.apache.pinot.common.utils.DataSchema;
+import org.apache.pinot.segment.local.indexsegment.immutable.ImmutableSegmentLoader;
+import org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl;
+import org.apache.pinot.segment.local.segment.index.loader.IndexLoadingConfig;
+import org.apache.pinot.segment.local.segment.readers.GenericRowRecordReader;
+import org.apache.pinot.segment.spi.ImmutableSegment;
+import org.apache.pinot.segment.spi.IndexSegment;
+import org.apache.pinot.segment.spi.creator.SegmentGeneratorConfig;
+import org.apache.pinot.spi.config.table.IndexingConfig;
+import org.apache.pinot.spi.config.table.TableConfig;
+import org.apache.pinot.spi.config.table.TableType;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.data.Schema;
+import org.apache.pinot.spi.data.readers.GenericRow;
+import org.apache.pinot.spi.utils.ReadMode;
+import org.apache.pinot.spi.utils.builder.TableConfigBuilder;
+import org.testng.Assert;
+import org.testng.annotations.BeforeClass;
+import org.testng.annotations.Test;
+
+
+public class ExplainPlanQueriesTest extends BaseQueriesTest {
+ private static final File INDEX_DIR = new File(FileUtils.getTempDirectory(), "ExplainPlanQueriesTest");
+ private static final String RAW_TABLE_NAME = "testTable";
+ private static final String SEGMENT_NAME = "testSegment";
+ private static final int NUM_RECORDS = 10;
+
+ private final static String COL1_NO_INDEX = "noIndexCol1";
+ private final static String COL2_NO_INDEX = "noIndexCol2";
+ private final static String COL3_NO_INDEX = "noIndexCol3";
+ private final static String COL1_INVERTED_INDEX = "invertedIndexCol1";
+ private final static String COL2_INVERTED_INDEX = "invertedIndexCol2";
+ private final static String COL3_INVERTED_INDEX = "invertedIndexCol3";
+ private final static String COL1_RANGE_INDEX = "rangeIndexCol1";
+ private final static String COL2_RANGE_INDEX = "rangeIndexCol2";
+ private final static String COL3_RANGE_INDEX = "rangeIndexCol3";
+ private final static String COL1_SORTED_INDEX = "sortedIndexCol1";
+ private final static String COL1_JSON_INDEX = "jsonIndexCol1";
+ private final static String COL1_TEXT_INDEX = "textIndexCol1";
+
+ private static final Schema SCHEMA = new Schema.SchemaBuilder().setSchemaName(RAW_TABLE_NAME)
+ .addSingleValueDimension(COL1_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL2_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_INVERTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_INVERTED_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_INVERTED_INDEX, FieldSpec.DataType.STRING)
+ .addSingleValueDimension(COL1_RANGE_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_SORTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL1_JSON_INDEX, FieldSpec.DataType.JSON)
+ .addSingleValueDimension(COL1_TEXT_INDEX, FieldSpec.DataType.STRING).build();
+
+ private static final DataSchema DATA_SCHEMA = new DataSchema(new String[]{"Operator", "Operator_Id", "Parent_Id"},
+ new DataSchema.ColumnDataType[]{DataSchema.ColumnDataType.STRING, DataSchema.ColumnDataType.INT,
+ DataSchema.ColumnDataType.INT});
+
+ private static final TableConfig TABLE_CONFIG =
+ new TableConfigBuilder(TableType.OFFLINE).setTableName(RAW_TABLE_NAME).build();
+
+ private IndexSegment _indexSegment;
+ private List<IndexSegment> _indexSegments;
+
+ @Override
+ protected String getFilter() {
+ return "";
+ }
+
+ @Override
+ protected IndexSegment getIndexSegment() {
+ return _indexSegment;
+ }
+
+ @Override
+ protected List<IndexSegment> getIndexSegments() {
+ return _indexSegments;
+ }
+
+ GenericRow createRecord(double doubleValue, int intValue, String stringValue, String jsonValue) {
+ GenericRow record = new GenericRow();
+ record.putValue(COL1_NO_INDEX, intValue);
+ record.putValue(COL2_NO_INDEX, intValue);
+ record.putValue(COL3_NO_INDEX, intValue);
+
+ record.putValue(COL1_INVERTED_INDEX, doubleValue);
+ record.putValue(COL2_INVERTED_INDEX, intValue);
+ record.putValue(COL3_INVERTED_INDEX, intValue);
+
+ record.putValue(COL1_RANGE_INDEX, doubleValue);
+ record.putValue(COL2_RANGE_INDEX, intValue);
+ record.putValue(COL3_RANGE_INDEX, intValue);
+
+ record.putValue(COL1_JSON_INDEX, jsonValue);
+ record.putValue(COL1_TEXT_INDEX, stringValue);
+
+ return record;
+ }
+
+ @BeforeClass
+ public void setUp()
+ throws Exception {
+ FileUtils.deleteDirectory(INDEX_DIR);
+
+ List<GenericRow> records = new ArrayList<>(NUM_RECORDS);
+ records.add(createRecord(1.1, 2, "daffy", "{\"first\": \"daffy\", \"last\": \"duck\"}"));
+
+ IndexingConfig indexingConfig = TABLE_CONFIG.getIndexingConfig();
+
+ List<String> invertedIndexColumns = Arrays.asList(COL1_INVERTED_INDEX, COL2_INVERTED_INDEX, COL3_INVERTED_INDEX);
+ indexingConfig.setInvertedIndexColumns(invertedIndexColumns);
+
+ List<String> rangeIndexColumns = Arrays.asList(COL1_RANGE_INDEX, COL2_RANGE_INDEX, COL3_RANGE_INDEX);
+ indexingConfig.setRangeIndexColumns(rangeIndexColumns);
+
+ List<String> jsonIndexColumns = Arrays.asList(COL1_JSON_INDEX);
+ indexingConfig.setJsonIndexColumns(jsonIndexColumns);
+
+ SegmentGeneratorConfig segmentGeneratorConfig = new SegmentGeneratorConfig(TABLE_CONFIG, SCHEMA);
+ segmentGeneratorConfig.setTableName(RAW_TABLE_NAME);
+ segmentGeneratorConfig.setSegmentName(SEGMENT_NAME);
+ segmentGeneratorConfig.setOutDir(INDEX_DIR.getPath());
+
+ SegmentIndexCreationDriverImpl driver = new SegmentIndexCreationDriverImpl();
+ driver.init(segmentGeneratorConfig, new GenericRowRecordReader(records));
+ driver.build();
+
+ IndexLoadingConfig indexLoadingConfig = new IndexLoadingConfig();
+ indexLoadingConfig.setTableConfig(TABLE_CONFIG);
+ indexLoadingConfig.setInvertedIndexColumns(new HashSet<>(invertedIndexColumns));
+ indexLoadingConfig.setRangeIndexColumns(new HashSet<>(rangeIndexColumns));
+ indexLoadingConfig.setJsonIndexColumns(new HashSet<String>(jsonIndexColumns));
+ indexLoadingConfig.setReadMode(ReadMode.mmap);
+
+ ImmutableSegment immutableSegment =
+ ImmutableSegmentLoader.load(new File(INDEX_DIR, SEGMENT_NAME), indexLoadingConfig);
+ _indexSegment = immutableSegment;
+ _indexSegments = Arrays.asList(immutableSegment);
+ }
+
+ /** Checks the correctness of EXPLAIN PLAN output. */
+ private void check(String query, ResultTable expected) {
+ BrokerResponseNative response = getBrokerResponseForSqlQuery(query);
+ ResultTable actual = response.getResultTable();
+
+ System.out.println(resultTableToString("Expected", expected));
+ System.out.println(resultTableToString("Actual", actual));
+
+ Assert.assertEquals(actual.getDataSchema(), expected.getDataSchema());
+ Assert.assertEquals(actual.getRows().size(), expected.getRows().size());
+
+ List<Object[]> expectedRows = expected.getRows();
+ List<Object[]> actualRows = actual.getRows();
+ for (int i = 0; i < expectedRows.size(); i++) {
+ Object[] expectedRow = expectedRows.get(i);
+ Object[] actualRow = actualRows.get(i);
+ Assert.assertEquals(actualRow.length, expectedRow.length);
+
+ for (int j = 0; j < actualRow.length; j++) {
+ Assert.assertEquals(actualRow[j], expectedRow[j]);
+ }
+ }
+ }
+
+ /** This function is used to generated expected results while creating test cases; otherwise, not used. */
+ private static String resultTableToString(String tag, ResultTable table) {
+ System.out.println("\n" + tag + ":");
+ List<Object[]> rows = table.getRows();
+
+ StringBuffer buffer = new StringBuffer();
+ for (Object[] row : rows) {
+ buffer.append("result.add(new Object[]{\"" + row[0] + "\", " + row[1] + ", " + row[2] + "});\n");
+ }
+
+ return buffer.toString();
+ }
+
+ @Test
+ public void testSelect() {
+ String query1 = "EXPLAIN PLAN FOR SELECT * FROM testTable";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(
+ new Object[]{"SELECT(selectList:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, jsonIndexCol1, "
+ + "noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, "
+ + "jsonIndexCol1, noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 3, 2});
+ result1.add(
+ new Object[]{"PROJECT(sortedIndexCol1, noIndexCol3, rangeIndexCol1, rangeIndexCol2, jsonIndexCol1, "
+ + "invertedIndexCol1, noIndexCol2, invertedIndexCol2, noIndexCol1, invertedIndexCol3, rangeIndexCol3, "
+ + "textIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 = "EXPLAIN PLAN FOR SELECT 'mickey' FROM testTable";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:'mickey')", 2, 1});
+ result2.add(new Object[]{"PROJECT(ALL)", 3, 2});
+ result2.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 4, 3});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+
+ String query3 = "EXPLAIN PLAN FOR SELECT invertedIndexCol1, noIndexCol1 FROM testTable LIMIT 100";
+ List<Object[]> result3 = new ArrayList<>();
+ result3.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result3.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result3.add(new Object[]{"SELECT(selectList:invertedIndexCol1, noIndexCol1)", 2, 1});
+ result3.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result3.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result3.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query3, new ResultTable(DATA_SCHEMA, result3));
+
+ String query4 = "EXPLAIN PLAN FOR SELECT DISTINCT invertedIndexCol1, noIndexCol1 FROM testTable LIMIT 100";
+ List<Object[]> result4 = new ArrayList<>();
+ result4.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result4.add(new Object[]{"COMBINE_DISTINCT", 1, 0});
+ result4.add(new Object[]{"DISTINCT(keyColumns:invertedIndexCol1, noIndexCol1)", 2, 1});
+ result4.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result4.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result4.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query4, new ResultTable(DATA_SCHEMA, result4));
+ }
+
+ @Test
+ public void testSelectColumnsUsingFilterIndex() {
+ String query1 = "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1, sortedIndexCol1 FROM testTable WHERE "
+ + "invertedIndexCol1 = 1.5 AND sortedIndexCol1 = 1 LIMIT 100";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1, sortedIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, sortedIndexCol1)", 3, 2});
+ result1.add(new Object[]{"PROJECT(sortedIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 = "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1, sortedIndexCol1 FROM testTable WHERE "
+ + "(invertedIndexCol1 = 1.5 AND sortedIndexCol1 = 1) OR (invertedIndexCol1 BETWEEN 100 and 200 AND "
+ + "rangeIndexCol1 > 20)";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1, sortedIndexCol1)", 2, 1});
+ result2.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, sortedIndexCol1)", 3, 2});
+ result2.add(new Object[]{"PROJECT(sortedIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result2.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+
+ String query3 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1 FROM testTable WHERE invertedIndexCol1 = 1.5 AND "
+ + "invertedIndexCol2 IN (10, 20, 30) AND invertedIndexCol3 NOT IN ('foo', 'bar') LIMIT 100";
+ List<Object[]> result3 = new ArrayList<>();
+ result3.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result3.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result3.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1)", 2, 1});
+ result3.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result3.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result3.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query3, new ResultTable(DATA_SCHEMA, result3));
+
+ String query4 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, noIndexCol2, sortedIndexCol1 FROM testTable WHERE sortedIndexCol1 = 1.5"
+ + " OR sortedIndexCol1 != 5 OR sortedIndexCol1 IN (10, 20, 30) LIMIT 100";
+ List<Object[]> result4 = new ArrayList<>();
+ result4.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result4.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result4.add(new Object[]{"SELECT(selectList:noIndexCol1, noIndexCol2, sortedIndexCol1)", 2, 1});
+ result4.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2, sortedIndexCol1)", 3, 2});
+ result4.add(new Object[]{"PROJECT(sortedIndexCol1, noIndexCol2, noIndexCol1)", 4, 3});
+ result4.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query4, new ResultTable(DATA_SCHEMA, result4));
+
+ String query5 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, noIndexCol2 FROM testTable WHERE DIV(noIndexCol1, noIndexCol2) BETWEEN "
+ + "10 AND 20 AND invertedIndexCol1 * 5 < 1000";
+ List<Object[]> result5 = new ArrayList<>();
+ result5.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result5.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result5.add(new Object[]{"SELECT(selectList:noIndexCol1, noIndexCol2)", 2, 1});
+ result5.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2)", 3, 2});
+ result5.add(new Object[]{"PROJECT(noIndexCol2, noIndexCol1)", 4, 3});
+ result5.add(new Object[]{"FILTER_AND", 5, 4});
+ result5.add(
+ new Object[]{"FILTER_EXPRESSION(operator:RANGE,predicate:div(noIndexCol1,noIndexCol2) BETWEEN '10' AND '20')",
+ 6, 5});
+ result5
+ .add(new Object[]{"FILTER_EXPRESSION(operator:RANGE,predicate:times(invertedIndexCol1,'5') < '1000')", 7, 5});
+ check(query5, new ResultTable(DATA_SCHEMA, result5));
+ }
+
+ @Test
+ public void testSelectColumnUsingFilterRange() {
+ // select * query triggering range index
+ // checks using RANGE (>, >=, <, <=, BETWEEN ..) on a column with range index should use RANGE_INDEX_SCAN
+ String query1 =
+ "EXPLAIN PLAN FOR SELECT invertedIndexCol1, noIndexCol1, rangeIndexCol1 FROM testTable WHERE rangeIndexCol1 >"
+ + " 10 AND rangeIndexCol2 >= 20 AND rangeIndexCol3 BETWEEN 50 AND 60 LIMIT 100";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(new Object[]{"SELECT(selectList:invertedIndexCol1, noIndexCol1, rangeIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, rangeIndexCol1)", 3, 2});
+ result1.add(new Object[]{"PROJECT(rangeIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1 FROM testTable WHERE noIndexCol1 > 10 AND noIndexCol2"
+ + " BETWEEN 50 AND 60 LIMIT 100";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1)", 2, 1});
+ result2.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result2.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result2.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+ }
+
+ @Test
+ public void testSelectColumnUsingFilterJson() {
+ String query =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1 FROM testTable WHERE (invertedIndexCol1 IN (10, 20, "
+ + "30) AND sortedIndexCol1 != 100) OR (noIndexCol1 NOT IN (10, 20, 30) AND rangeIndexCol1 != 20 AND "
+ + "JSON_MATCH(jsonIndexCol1, 'key=1') AND TEXT_MATCH(textIndexCol1, 'foo'))";
+ List<Object[]> result = new ArrayList<>();
+ result.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1)", 2, 1});
+ result.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result.add(new Object[]{"FILTER_AND", 5, 4});
+ result.add(new Object[]{"FILTER_JSON_INDEX(indexLookUp:json_index,operator:JSON_MATCH,predicate:json_match"
+ + "(jsonIndexCol1,'key=1'))", 6, 5});
+ result.add(new Object[]{"FILTER_TEXT_INDEX(indexLookUp:text_index,operator:TEXT_MATCH,predicate:text_match"
+ + "(textIndexCol1,'foo'))", 7, 5});
+ check(query, new ResultTable(DATA_SCHEMA, result));
+ }
+
+ @Test
+ public void testSelectAggregateUsingFilterText() {
+ String query1 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, noIndexCol2, max(noIndexCol2), min(noIndexCol3) FROM testTable WHERE "
+ + "TEXT_MATCH(textIndexCol1, 'foo') GROUP BY noIndexCol1, noIndexCol2";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result1.add(new Object[]{"COMBINE_GROUPBY_ORDERBY", 1, 0});
+ result1.add(new Object[]{
+ "AGGREGATE_GROUPBY_ORDERBY(groupKeys:noIndexCol1, noIndexCol2, aggregations:max(noIndexCol2), min"
+ + "(noIndexCol3))", 2, 1});
+ result1.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2, noIndexCol3)", 3, 2});
+ result1.add(new Object[]{"PROJECT(noIndexCol3, noIndexCol2, noIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_TEXT_INDEX(indexLookUp:text_index,operator:TEXT_MATCH,predicate:text_match"
+ + "(textIndexCol1,'foo'))", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, max(noIndexCol2), min(noIndexCol3) FROM testTable WHERE TEXT_MATCH"
+ + "(textIndexCol1, 'foo') GROUP BY noIndexCol1, noIndexCol2 ORDER BY noIndexCol1, max(noIndexCol2)";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(sort:[noIndexCol1 ASC, max(noIndexCol2) ASC],limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_GROUPBY_ORDERBY", 1, 0});
+ result2.add(new Object[]{
+ "AGGREGATE_GROUPBY_ORDERBY(groupKeys:noIndexCol1,noIndexCol2,aggregations:max(noIndexCol2), min"
+ + "(noIndexCol3))", 2, 1});
+ result2.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, invertedIndexCol1)", 3, 2});
+ result2.add(new Object[]{"PROJECT(noIndexCol3, noIndexCol2, noIndexCol1)", 4, 3});
+ result2.add(new Object[]{"FILTER_TEXT_INDEX(indexLookUp:text_index,operator:TEXT_MATCH,predicate:text_match"
+ + "(textIndexCol1,'foo'))", 5, 4});
+ }
+
+ @Test
+ public void testSelectAggregate() {
+ String query1 = "EXPLAIN PLAN FOR SELECT count(*) FROM testTable";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result1.add(new Object[]{"COMBINE_AGGREGATE", 1, 0});
+ result1.add(new Object[]{"AGGREGATE_METADATA", 2, 1});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 =
+ "EXPLAIN PLAN FOR SELECT count(*), max(noIndexCol1), sum(noIndexCol2), avg(noIndexCol2) FROM testTable";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_AGGREGATE", 1, 0});
+ result2.add(
+ new Object[]{"AGGREGATE(aggregations:count(*), max(noIndexCol1), sum(noIndexCol2), avg(noIndexCol2))", 2, 1});
+ result2.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2)", 3, 2});
+ result2.add(new Object[]{"PROJECT(noIndexCol2, noIndexCol1)", 4, 3});
+ result2.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+ }
+
+ @Test
+ public void testSelectAggregateUsingFilter() {
+ String query = "EXPLAIN PLAN FOR SELECT sum(add(noIndexCol1, noIndexCol2)), MIN(ADD(DIV(noIndexCol1,noIndexCol2),"
+ + "noIndexCol3)) FROM testTable";
+ List<Object[]> result = new ArrayList<>();
+ result.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result.add(new Object[]{"COMBINE_AGGREGATE", 1, 0});
+ result.add(new Object[]{
+ "AGGREGATE(aggregations:sum(add(noIndexCol1,noIndexCol2)), min(add(div(noIndexCol1,noIndexCol2),"
+ + "noIndexCol3)))", 2, 1});
+ result.add(new Object[]{"PROJECT(noIndexCol3, noIndexCol2, noIndexCol1)", 3, 2});
+ result.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 4, 3});
+ check(query, new ResultTable(DATA_SCHEMA, result));
+ }
+
+ @Test
+ public void testSelectAggregateUsingFilterGroupBy() {
+ String query =
+ "EXPLAIN PLAN FOR SELECT noIndexCol2, sum(add(noIndexCol1, noIndexCol2)), min(noIndexCol3) FROM testTable "
+ + "WHERE noIndexCol1 < 100 GROUP BY noIndexCol2";
+ List<Object[]> result = new ArrayList<>();
+ result.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result.add(new Object[]{"COMBINE_GROUPBY_ORDERBY", 1, 0});
+ result.add(new Object[]{
+ "AGGREGATE_GROUPBY_ORDERBY(groupKeys:noIndexCol2, aggregations:sum(add(noIndexCol1,noIndexCol2)),"
+ + " min(noIndexCol3))", 2, 1});
+ result.add(new Object[]{"PROJECT(noIndexCol3, noIndexCol2, noIndexCol1)", 3, 2});
+ result.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 4, 3});
+ check(query, new ResultTable(DATA_SCHEMA, result));
+ }
+
+ @Test
+ public void testSelectAggregateUsingFilterIndex() {
+ String query =
+ "EXPLAIN PLAN FOR SELECT count(*), max(noIndexCol1), sum(noIndexCol2), avg(noIndexCol3) FROM testTable WHERE "
+ + "invertedIndexCol1 = 1 AND col1 = 20";
+ List<Object[]> result = new ArrayList<>();
+ result.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result.add(new Object[]{"COMBINE_AGGREGATE", 1, 0});
+ result.add(
+ new Object[]{"AGGREGATE(aggregations:count(*), max(noIndexCol1), sum(noIndexCol2), avg(noIndexCol3))", 2, 1});
+ result.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2, noIndexCol3)", 3, 2});
+ result.add(new Object[]{"PROJECT(noIndexCol3, noIndexCol2, noIndexCol1)", 4, 3});
+ result.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query, new ResultTable(DATA_SCHEMA, result));
+ }
+
+ @Test
+ public void testSelectAggregateUsingFilterIndexGroupBy() {
+ String query = "EXPLAIN PLAN FOR SELECT noIndexCol1, max(noIndexCol2), min(noIndexCol3) FROM testTable WHERE "
+ + "invertedIndexCol2 = 10 GROUP BY noIndexCol1";
+ List<Object[]> result = new ArrayList<>();
+ result.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result.add(new Object[]{"COMBINE_GROUPBY_ORDERBY", 1, 0});
+ result.add(new Object[]{
+ "AGGREGATE_GROUPBY_ORDERBY(groupKeys:noIndexCol1, aggregations:max(noIndexCol2), min(noIndexCol3)" + ")", 2,
+ 1});
+ result.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2, noIndexCol3)", 3, 2});
+ result.add(new Object[]{"PROJECT(noIndexCol3, noIndexCol2, noIndexCol1)", 4, 3});
+ result.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query, new ResultTable(DATA_SCHEMA, result));
+ }
+
+ @Test
+ public void testSelectAggregateUsingFilterIndexGroupByHaving() {
Review comment:
Transform in:
- filter: see `testSelectColumnsUsingFilterIndex`.
- select: see `testSelectAggregate`
- group by and order by: see `testSelectAggregateUsingFilterIndexGroupByOrderBy`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7568: [WORK IN PROGRESS] EXPLAIN PLAN code
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7568](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (de9bace) into [master](https://codecov.io/gh/apache/pinot/commit/1dda74a6ecd96e2704147ad803135d5046a1e13d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (1dda74a) will **decrease** coverage by `40.58%`.
> The diff coverage is `13.80%`.
[](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7568 +/- ##
=============================================
- Coverage 72.63% 32.04% -40.59%
=============================================
Files 1522 1514 -8
Lines 75905 75913 +8
Branches 11093 11115 +22
=============================================
- Hits 55133 24330 -30803
- Misses 17075 49479 +32404
+ Partials 3697 2104 -1593
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `30.41% <13.28%> (-0.27%)` | :arrow_down: |
| integration2 | `28.85% <12.76%> (-0.09%)` | :arrow_down: |
| unittests1 | `?` | |
| unittests2 | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...operator/AcquireReleaseColumnsSegmentOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9BY3F1aXJlUmVsZWFzZUNvbHVtbnNTZWdtZW50T3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...a/org/apache/pinot/core/operator/BaseOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CYXNlT3BlcmF0b3IuamF2YQ==) | `31.25% <0.00%> (-4.47%)` | :arrow_down: |
| [...he/pinot/core/operator/BitmapDocIdSetOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CaXRtYXBEb2NJZFNldE9wZXJhdG9yLmphdmE=) | `71.42% <0.00%> (-3.58%)` | :arrow_down: |
| [.../pinot/core/operator/InstanceResponseOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9JbnN0YW5jZVJlc3BvbnNlT3BlcmF0b3IuamF2YQ==) | `78.78% <0.00%> (-5.09%)` | :arrow_down: |
| [...apache/pinot/core/operator/ProjectionOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9Qcm9qZWN0aW9uT3BlcmF0b3IuamF2YQ==) | `39.28% <0.00%> (-34.05%)` | :arrow_down: |
| [...erator/combine/AggregationOnlyCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0FnZ3JlZ2F0aW9uT25seUNvbWJpbmVPcGVyYXRvci5qYXZh) | `69.23% <0.00%> (-5.77%)` | :arrow_down: |
| [...not/core/operator/combine/BaseCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0Jhc2VDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `82.66% <0.00%> (-5.18%)` | :arrow_down: |
| [...core/operator/combine/DistinctCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0Rpc3RpbmN0Q29tYmluZU9wZXJhdG9yLmphdmE=) | `66.66% <0.00%> (-14.11%)` | :arrow_down: |
| [.../core/operator/combine/GroupByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0dyb3VwQnlDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `77.04% <0.00%> (-2.96%)` | :arrow_down: |
| [...perator/combine/GroupByOrderByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0dyb3VwQnlPcmRlckJ5Q29tYmluZU9wZXJhdG9yLmphdmE=) | `59.30% <0.00%> (-21.88%)` | :arrow_down: |
| ... and [1075 more](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [1dda74a...de9bace](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7568: [WORK IN PROGRESS] EXPLAIN PLAN code
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7568](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (0ab950d) into [master](https://codecov.io/gh/apache/pinot/commit/0ecfc11535e24ed72404662e4c92c9cb869228fc?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (0ecfc11) will **increase** coverage by `42.30%`.
> The diff coverage is `65.10%`.
[](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7568 +/- ##
=============================================
+ Coverage 29.00% 71.31% +42.30%
- Complexity 0 3417 +3417
=============================================
Files 1513 1523 +10
Lines 75564 76269 +705
Branches 11051 11149 +98
=============================================
+ Hits 21921 54389 +32468
+ Misses 51655 18176 -33479
- Partials 1988 3704 +1716
```
| Flag | Coverage Δ | |
|---|---|---|
| integration2 | `28.88% <12.76%> (-0.13%)` | :arrow_down: |
| unittests1 | `69.87% <64.36%> (?)` | |
| unittests2 | `15.16% <2.08%> (?)` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...operator/AcquireReleaseColumnsSegmentOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9BY3F1aXJlUmVsZWFzZUNvbHVtbnNTZWdtZW50T3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...he/pinot/core/operator/BitmapDocIdSetOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CaXRtYXBEb2NJZFNldE9wZXJhdG9yLmphdmE=) | `71.42% <0.00%> (-3.58%)` | :arrow_down: |
| [.../core/operator/combine/GroupByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0dyb3VwQnlDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `78.68% <0.00%> (+3.68%)` | :arrow_up: |
| [...nMaxValueBasedSelectionOrderByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL01pbk1heFZhbHVlQmFzZWRTZWxlY3Rpb25PcmRlckJ5Q29tYmluZU9wZXJhdG9yLmphdmE=) | `73.80% <0.00%> (-0.60%)` | :arrow_down: |
| [...rator/combine/SelectionOrderByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL1NlbGVjdGlvbk9yZGVyQnlDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `50.00% <0.00%> (-1.73%)` | :arrow_down: |
| [...or/dociditerators/ExpressionScanDocIdIterator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9kb2NpZGl0ZXJhdG9ycy9FeHByZXNzaW9uU2NhbkRvY0lkSXRlcmF0b3IuamF2YQ==) | `52.48% <0.00%> (+3.19%)` | :arrow_up: |
| [...ore/operator/filter/BitmapBasedFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvQml0bWFwQmFzZWRGaWx0ZXJPcGVyYXRvci5qYXZh) | `78.26% <0.00%> (-14.05%)` | :arrow_down: |
| [...ot/core/operator/filter/H3IndexFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvSDNJbmRleEZpbHRlck9wZXJhdG9yLmphdmE=) | `82.10% <0.00%> (+82.10%)` | :arrow_up: |
| [...e/pinot/core/operator/filter/OrFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvT3JGaWx0ZXJPcGVyYXRvci5qYXZh) | `56.25% <0.00%> (-33.75%)` | :arrow_down: |
| [...operator/filter/RangeIndexBasedFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvUmFuZ2VJbmRleEJhc2VkRmlsdGVyT3BlcmF0b3IuamF2YQ==) | `36.53% <0.00%> (-3.89%)` | :arrow_down: |
| ... and [1089 more](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [0ecfc11...0ab950d](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7568](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (11aad26) into [master](https://codecov.io/gh/apache/pinot/commit/767aa8abfb5bf085ba0a7ae5ff4024679f27816e?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (767aa8a) will **decrease** coverage by `57.11%`.
> The diff coverage is `1.04%`.
[](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7568 +/- ##
=============================================
- Coverage 71.69% 14.58% -57.12%
+ Complexity 4086 80 -4006
=============================================
Files 1579 1535 -44
Lines 80790 79336 -1454
Branches 12003 11874 -129
=============================================
- Hits 57922 11570 -46352
- Misses 18981 66909 +47928
+ Partials 3887 857 -3030
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| integration2 | `?` | |
| unittests1 | `?` | |
| unittests2 | `14.58% <1.04%> (-0.03%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...java/org/apache/pinot/common/utils/DataSchema.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvRGF0YVNjaGVtYS5qYXZh) | `0.00% <0.00%> (-79.24%)` | :arrow_down: |
| [...org/apache/pinot/sql/parsers/CalciteSqlParser.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcWwvcGFyc2Vycy9DYWxjaXRlU3FsUGFyc2VyLmphdmE=) | `0.00% <0.00%> (-91.89%)` | :arrow_down: |
| [...in/java/org/apache/pinot/core/common/Operator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9jb21tb24vT3BlcmF0b3IuamF2YQ==) | `0.00% <ø> (ø)` | |
| [...operator/AcquireReleaseColumnsSegmentOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9BY3F1aXJlUmVsZWFzZUNvbHVtbnNTZWdtZW50T3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...a/org/apache/pinot/core/operator/BaseOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CYXNlT3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (-38.47%)` | :arrow_down: |
| [...he/pinot/core/operator/BitmapDocIdSetOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CaXRtYXBEb2NJZFNldE9wZXJhdG9yLmphdmE=) | `0.00% <0.00%> (-75.00%)` | :arrow_down: |
| [...g/apache/pinot/core/operator/DocIdSetOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9Eb2NJZFNldE9wZXJhdG9yLmphdmE=) | `0.00% <0.00%> (-92.31%)` | :arrow_down: |
| [.../pinot/core/operator/InstanceResponseOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9JbnN0YW5jZVJlc3BvbnNlT3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (-86.85%)` | :arrow_down: |
| [...apache/pinot/core/operator/ProjectionOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9Qcm9qZWN0aW9uT3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (-73.34%)` | :arrow_down: |
| [...erator/combine/AggregationOnlyCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0FnZ3JlZ2F0aW9uT25seUNvbWJpbmVPcGVyYXRvci5qYXZh) | `0.00% <0.00%> (-75.00%)` | :arrow_down: |
| ... and [1316 more](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [767aa8a...11aad26](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7568](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (88f44eb) into [master](https://codecov.io/gh/apache/pinot/commit/d1727372a5bfd48b6f3c1fc0c8a22d39c243c1bc?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (d172737) will **decrease** coverage by `1.23%`.
> The diff coverage is `73.42%`.
[](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7568 +/- ##
============================================
- Coverage 71.64% 70.40% -1.24%
- Complexity 4083 4087 +4
============================================
Files 1579 1580 +1
Lines 80819 81155 +336
Branches 12017 12077 +60
============================================
- Hits 57902 57139 -763
- Misses 19017 20105 +1088
- Partials 3900 3911 +11
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| integration2 | `27.95% <43.01%> (+0.07%)` | :arrow_up: |
| unittests1 | `68.72% <70.52%> (+0.08%)` | :arrow_up: |
| unittests2 | `14.63% <1.36%> (-0.01%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...in/java/org/apache/pinot/core/common/Operator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9jb21tb24vT3BlcmF0b3IuamF2YQ==) | `0.00% <ø> (ø)` | |
| [...operator/AcquireReleaseColumnsSegmentOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9BY3F1aXJlUmVsZWFzZUNvbHVtbnNTZWdtZW50T3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...he/pinot/core/operator/BitmapDocIdSetOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CaXRtYXBEb2NJZFNldE9wZXJhdG9yLmphdmE=) | `68.18% <0.00%> (-6.82%)` | :arrow_down: |
| [.../core/operator/combine/GroupByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0dyb3VwQnlDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `73.91% <0.00%> (-1.09%)` | :arrow_down: |
| [...nMaxValueBasedSelectionOrderByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL01pbk1heFZhbHVlQmFzZWRTZWxlY3Rpb25PcmRlckJ5Q29tYmluZU9wZXJhdG9yLmphdmE=) | `70.45% <0.00%> (-2.07%)` | :arrow_down: |
| [...or/dociditerators/ExpressionScanDocIdIterator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9kb2NpZGl0ZXJhdG9ycy9FeHByZXNzaW9uU2NhbkRvY0lkSXRlcmF0b3IuamF2YQ==) | `52.11% <0.00%> (-14.32%)` | :arrow_down: |
| [...inot/core/operator/filter/EmptyFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvRW1wdHlGaWx0ZXJPcGVyYXRvci5qYXZh) | `57.14% <0.00%> (-22.86%)` | :arrow_down: |
| [...ot/core/operator/filter/H3IndexFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvSDNJbmRleEZpbHRlck9wZXJhdG9yLmphdmE=) | `82.10% <0.00%> (-4.57%)` | :arrow_down: |
| [...ore/operator/query/AggregationGroupByOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9xdWVyeS9BZ2dyZWdhdGlvbkdyb3VwQnlPcGVyYXRvci5qYXZh) | `48.48% <0.00%> (-46.26%)` | :arrow_down: |
| [...perator/query/DictionaryBasedDistinctOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9xdWVyeS9EaWN0aW9uYXJ5QmFzZWREaXN0aW5jdE9wZXJhdG9yLmphdmE=) | `75.00% <0.00%> (-3.27%)` | :arrow_down: |
| ... and [160 more](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [d172737...88f44eb](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] amrishlal commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
amrishlal commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r741522258
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/AcquireReleaseColumnsSegmentOperator.java
##########
@@ -77,6 +81,16 @@ public String getOperatorName() {
return OPERATOR_NAME;
}
+ @Override
+ public String getExplainPlanName() {
+ return EXPLAIN_NAME;
+ }
+
+ @Override
+ public List<Operator> getChildOperators() {
+ return Arrays.asList(_childOperator);
Review comment:
Fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] amrishlal commented on a change in pull request #7568: [WORK IN PROGRESS] EXPLAIN PLAN code
Posted by GitBox <gi...@apache.org>.
amrishlal commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r727439934
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/AcquireReleaseColumnsSegmentOperator.java
##########
@@ -77,6 +81,16 @@ public String getOperatorName() {
return OPERATOR_NAME;
}
+ @Override
+ public String getExplainPlanName() {
+ return EXPLAIN_NAME;
+ }
+
+ @Override
+ public List<Operator> getChildOperators() {
+ return Arrays.asList(_childOperator);
Review comment:
OK, but let's please hold on the reviews for now until I can get the test cases passing along with a decent description :-) I will open it for review soon.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] mqliang commented on pull request #7568: [WORK IN PROGRESS] EXPLAIN PLAN code
Posted by GitBox <gi...@apache.org>.
mqliang commented on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941631706
Can you post a few examples of the EXPLAIN PLAN query result?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7568](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (6f50ec3) into [master](https://codecov.io/gh/apache/pinot/commit/2107b2cd1880ec74934ec5412dbee8c874f05610?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (2107b2c) will **decrease** coverage by `2.77%`.
> The diff coverage is `30.03%`.
> :exclamation: Current head 6f50ec3 differs from pull request most recent head b2ffa0b. Consider uploading reports for the commit b2ffa0b to get more accurate results
[](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7568 +/- ##
==========================================
- Coverage 30.79% 28.01% -2.78%
==========================================
Files 1570 1570
Lines 80024 80638 +614
Branches 11904 12004 +100
==========================================
- Hits 24644 22593 -2051
- Misses 53274 55996 +2722
+ Partials 2106 2049 -57
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| integration2 | `28.01% <30.03%> (+0.31%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...ache/pinot/common/tier/PinotServerTierStorage.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdGllci9QaW5vdFNlcnZlclRpZXJTdG9yYWdlLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...c/main/java/org/apache/pinot/common/tier/Tier.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdGllci9UaWVyLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...java/org/apache/pinot/common/tier/TierFactory.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdGllci9UaWVyRmFjdG9yeS5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...inot/common/tier/TimeBasedTierSegmentSelector.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdGllci9UaW1lQmFzZWRUaWVyU2VnbWVudFNlbGVjdG9yLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [.../pinot/common/utils/ClientSSLContextGenerator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvQ2xpZW50U1NMQ29udGV4dEdlbmVyYXRvci5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...che/pinot/common/utils/config/TierConfigUtils.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvY29uZmlnL1RpZXJDb25maWdVdGlscy5qYXZh) | `4.16% <0.00%> (-1.39%)` | :arrow_down: |
| [...ces/PinotSegmentUploadDownloadRestletResource.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9hcGkvcmVzb3VyY2VzL1Bpbm90U2VnbWVudFVwbG9hZERvd25sb2FkUmVzdGxldFJlc291cmNlLmphdmE=) | `49.77% <0.00%> (-4.98%)` | :arrow_down: |
| [...e/pinot/controller/helix/SegmentStatusChecker.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9oZWxpeC9TZWdtZW50U3RhdHVzQ2hlY2tlci5qYXZh) | `64.78% <0.00%> (-12.32%)` | :arrow_down: |
| [...ntroller/helix/core/PinotHelixResourceManager.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9oZWxpeC9jb3JlL1Bpbm90SGVsaXhSZXNvdXJjZU1hbmFnZXIuamF2YQ==) | `33.93% <0.00%> (-10.11%)` | :arrow_down: |
| [...troller/helix/core/minion/ClusterInfoAccessor.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9oZWxpeC9jb3JlL21pbmlvbi9DbHVzdGVySW5mb0FjY2Vzc29yLmphdmE=) | `50.00% <0.00%> (-23.08%)` | :arrow_down: |
| ... and [330 more](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [2107b2c...b2ffa0b](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] amrishlal commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
amrishlal commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r741522329
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/filter/AndFilterOperator.java
##########
@@ -47,4 +49,18 @@ protected FilterBlock getNextBlock() {
public String getOperatorName() {
return OPERATOR_NAME;
}
+
+ @Override
+ public List<Operator> getChildOperators() {
+ ArrayList<Operator> result = new ArrayList<>();
Review comment:
Fixed.
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/filter/OrFilterOperator.java
##########
@@ -49,4 +50,18 @@ protected FilterBlock getNextBlock() {
public String getOperatorName() {
return OPERATOR_NAME;
}
+
+ @Override
+ public String getExplainPlanName() {
+ return EXPLAIN_NAME;
+ }
+
+ @Override
+ public List<Operator> getChildOperators() {
+ List<Operator> operators = new ArrayList<>();
Review comment:
Fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] amrishlal commented on pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
amrishlal commented on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-944787130
> The link of the design doc in the description seems broken. Could you update it?
> Also, could you link the open source issue to this PR?
Fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] amrishlal commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
amrishlal commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r746092243
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/DocIdSetOperator.java
##########
@@ -34,6 +37,7 @@
*/
public class DocIdSetOperator extends BaseOperator<DocIdSetBlock> {
private static final String OPERATOR_NAME = "DocIdSetOperator";
+ private static final String EXPLAIN_NAME = null;
Review comment:
Yes, for an operator to be included in EXPLAIN PLAN output, it must define an EXPLAIN PLAN name.
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/BaseOperator.java
##########
@@ -52,4 +54,25 @@ public final T nextBlock() {
// Make it protected because we should always call nextBlock()
protected abstract T getNextBlock();
+
+ // Enforcing sub-class to implement the getOperatorName(), as they can just return a static final,
+ // as opposed to this super class calling getClass().getSimpleName().
+ public abstract String getOperatorName();
Review comment:
That method was being only used twice (`DocIdSetOperator` and `TransformOperator`), so adding the function at the interface level didn't make sense. The way it is set up now is that if an operator doesn't define a non-null value for getExplainPlanName(), it won't be included in EXPLAIN PLAN output.
```
/**
* @return EXPLAIN PLAN name if one is defined; otherwise, null. null EXPLAIN PLAN name means that this operator
* won't be included in EXPLAIN PLAN output.
*/
String getExplainPlanName();
```
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/BaseOperator.java
##########
@@ -52,4 +54,25 @@ public final T nextBlock() {
// Make it protected because we should always call nextBlock()
protected abstract T getNextBlock();
+
+ // Enforcing sub-class to implement the getOperatorName(), as they can just return a static final,
+ // as opposed to this super class calling getClass().getSimpleName().
+ public abstract String getOperatorName();
+
+ public abstract String getExplainPlanName();
+
+ @Override
+ public List<Operator> getChildOperators() {
+ return new ArrayList<>();
+ }
+
+ @Override
+ public String toExplainString() {
+ return getExplainPlanName();
+ }
+
+ @Override
Review comment:
By default, the explain plan description for an operator is its name, unless the operator overrides `toExplainString` function. If I remember correctly this is same as what was in the original code except that I simplified things a bit.
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/BaseOperator.java
##########
@@ -52,4 +54,25 @@ public final T nextBlock() {
// Make it protected because we should always call nextBlock()
protected abstract T getNextBlock();
+
+ // Enforcing sub-class to implement the getOperatorName(), as they can just return a static final,
+ // as opposed to this super class calling getClass().getSimpleName().
+ public abstract String getOperatorName();
+
+ public abstract String getExplainPlanName();
+
+ @Override
+ public List<Operator> getChildOperators() {
+ return new ArrayList<>();
Review comment:
Fixed. `getChildOperators()` is abstract now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7568](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (22e210a) into [master](https://codecov.io/gh/apache/pinot/commit/2107b2cd1880ec74934ec5412dbee8c874f05610?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (2107b2c) will **increase** coverage by `34.23%`.
> The diff coverage is `60.31%`.
[](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7568 +/- ##
=============================================
+ Coverage 30.79% 65.02% +34.23%
- Complexity 0 4039 +4039
=============================================
Files 1570 1530 -40
Lines 80024 78771 -1253
Branches 11904 11794 -110
=============================================
+ Hits 24644 51222 +26578
+ Misses 53274 23909 -29365
- Partials 2106 3640 +1534
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| integration2 | `?` | |
| unittests1 | `68.53% <61.77%> (?)` | |
| unittests2 | `14.51% <2.08%> (?)` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...in/java/org/apache/pinot/core/common/Operator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9jb21tb24vT3BlcmF0b3IuamF2YQ==) | `0.00% <ø> (ø)` | |
| [...operator/AcquireReleaseColumnsSegmentOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9BY3F1aXJlUmVsZWFzZUNvbHVtbnNTZWdtZW50T3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...he/pinot/core/operator/BitmapDocIdSetOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CaXRtYXBEb2NJZFNldE9wZXJhdG9yLmphdmE=) | `68.18% <0.00%> (-6.82%)` | :arrow_down: |
| [.../core/operator/combine/GroupByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0dyb3VwQnlDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `73.91% <0.00%> (-5.46%)` | :arrow_down: |
| [...nMaxValueBasedSelectionOrderByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL01pbk1heFZhbHVlQmFzZWRTZWxlY3Rpb25PcmRlckJ5Q29tYmluZU9wZXJhdG9yLmphdmE=) | `60.60% <0.00%> (-14.80%)` | :arrow_down: |
| [...rator/combine/SelectionOrderByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL1NlbGVjdGlvbk9yZGVyQnlDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `23.33% <0.00%> (-28.40%)` | :arrow_down: |
| [...or/dociditerators/ExpressionScanDocIdIterator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9kb2NpZGl0ZXJhdG9ycy9FeHByZXNzaW9uU2NhbkRvY0lkSXRlcmF0b3IuamF2YQ==) | `47.18% <0.00%> (-15.68%)` | :arrow_down: |
| [...ore/operator/filter/BitmapBasedFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvQml0bWFwQmFzZWRGaWx0ZXJPcGVyYXRvci5qYXZh) | `57.44% <0.00%> (-34.87%)` | :arrow_down: |
| [...ot/core/operator/filter/H3IndexFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvSDNJbmRleEZpbHRlck9wZXJhdG9yLmphdmE=) | `81.25% <0.00%> (+81.25%)` | :arrow_up: |
| [...e/pinot/core/operator/filter/OrFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvT3JGaWx0ZXJPcGVyYXRvci5qYXZh) | `75.00% <0.00%> (-15.00%)` | :arrow_down: |
| ... and [1339 more](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [2107b2c...22e210a](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7568](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (bdba462) into [master](https://codecov.io/gh/apache/pinot/commit/2107b2cd1880ec74934ec5412dbee8c874f05610?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (2107b2c) will **increase** coverage by `39.54%`.
> The diff coverage is `70.75%`.
[](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7568 +/- ##
=============================================
+ Coverage 30.79% 70.34% +39.54%
- Complexity 0 4066 +4066
=============================================
Files 1570 1578 +8
Lines 80024 80897 +873
Branches 11904 12026 +122
=============================================
+ Hits 24644 56903 +32259
+ Misses 53274 20101 -33173
- Partials 2106 3893 +1787
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| integration2 | `27.75% <13.57%> (+0.04%)` | :arrow_up: |
| unittests1 | `68.67% <70.63%> (?)` | |
| unittests2 | `14.49% <2.08%> (?)` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...in/java/org/apache/pinot/core/common/Operator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9jb21tb24vT3BlcmF0b3IuamF2YQ==) | `0.00% <ø> (ø)` | |
| [...operator/AcquireReleaseColumnsSegmentOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9BY3F1aXJlUmVsZWFzZUNvbHVtbnNTZWdtZW50T3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...he/pinot/core/operator/BitmapDocIdSetOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CaXRtYXBEb2NJZFNldE9wZXJhdG9yLmphdmE=) | `68.18% <0.00%> (-6.82%)` | :arrow_down: |
| [.../core/operator/combine/GroupByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0dyb3VwQnlDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `73.91% <0.00%> (-5.46%)` | :arrow_down: |
| [...nMaxValueBasedSelectionOrderByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL01pbk1heFZhbHVlQmFzZWRTZWxlY3Rpb25PcmRlckJ5Q29tYmluZU9wZXJhdG9yLmphdmE=) | `72.72% <0.00%> (-2.67%)` | :arrow_down: |
| [...or/dociditerators/ExpressionScanDocIdIterator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9kb2NpZGl0ZXJhdG9ycy9FeHByZXNzaW9uU2NhbkRvY0lkSXRlcmF0b3IuamF2YQ==) | `52.11% <0.00%> (-10.75%)` | :arrow_down: |
| [...inot/core/operator/filter/EmptyFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvRW1wdHlGaWx0ZXJPcGVyYXRvci5qYXZh) | `62.50% <0.00%> (-17.50%)` | :arrow_down: |
| [...ot/core/operator/filter/H3IndexFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvSDNJbmRleEZpbHRlck9wZXJhdG9yLmphdmE=) | `81.25% <0.00%> (+81.25%)` | :arrow_up: |
| [...ore/operator/query/AggregationGroupByOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9xdWVyeS9BZ2dyZWdhdGlvbkdyb3VwQnlPcGVyYXRvci5qYXZh) | `47.05% <0.00%> (-47.68%)` | :arrow_down: |
| [...perator/query/DictionaryBasedDistinctOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9xdWVyeS9EaWN0aW9uYXJ5QmFzZWREaXN0aW5jdE9wZXJhdG9yLmphdmE=) | `75.00% <0.00%> (+25.00%)` | :arrow_up: |
| ... and [1178 more](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [2107b2c...bdba462](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7568](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (bdba462) into [master](https://codecov.io/gh/apache/pinot/commit/2107b2cd1880ec74934ec5412dbee8c874f05610?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (2107b2c) will **increase** coverage by `40.79%`.
> The diff coverage is `70.75%`.
[](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7568 +/- ##
=============================================
+ Coverage 30.79% 71.58% +40.79%
- Complexity 0 4066 +4066
=============================================
Files 1570 1578 +8
Lines 80024 80897 +873
Branches 11904 12026 +122
=============================================
+ Hits 24644 57912 +33268
+ Misses 53274 19086 -34188
- Partials 2106 3899 +1793
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `29.11% <14.09%> (-0.12%)` | :arrow_down: |
| integration2 | `27.75% <13.57%> (+0.04%)` | :arrow_up: |
| unittests1 | `68.67% <70.63%> (?)` | |
| unittests2 | `14.49% <2.08%> (?)` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...in/java/org/apache/pinot/core/common/Operator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9jb21tb24vT3BlcmF0b3IuamF2YQ==) | `0.00% <ø> (ø)` | |
| [...operator/AcquireReleaseColumnsSegmentOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9BY3F1aXJlUmVsZWFzZUNvbHVtbnNTZWdtZW50T3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...he/pinot/core/operator/BitmapDocIdSetOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CaXRtYXBEb2NJZFNldE9wZXJhdG9yLmphdmE=) | `68.18% <0.00%> (-6.82%)` | :arrow_down: |
| [.../core/operator/combine/GroupByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0dyb3VwQnlDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `73.91% <0.00%> (-5.46%)` | :arrow_down: |
| [...nMaxValueBasedSelectionOrderByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL01pbk1heFZhbHVlQmFzZWRTZWxlY3Rpb25PcmRlckJ5Q29tYmluZU9wZXJhdG9yLmphdmE=) | `72.72% <0.00%> (-2.67%)` | :arrow_down: |
| [...or/dociditerators/ExpressionScanDocIdIterator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9kb2NpZGl0ZXJhdG9ycy9FeHByZXNzaW9uU2NhbkRvY0lkSXRlcmF0b3IuamF2YQ==) | `65.49% <0.00%> (+2.63%)` | :arrow_up: |
| [...inot/core/operator/filter/EmptyFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvRW1wdHlGaWx0ZXJPcGVyYXRvci5qYXZh) | `62.50% <0.00%> (-17.50%)` | :arrow_down: |
| [...ot/core/operator/filter/H3IndexFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvSDNJbmRleEZpbHRlck9wZXJhdG9yLmphdmE=) | `81.25% <0.00%> (+81.25%)` | :arrow_up: |
| [...ore/operator/query/AggregationGroupByOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9xdWVyeS9BZ2dyZWdhdGlvbkdyb3VwQnlPcGVyYXRvci5qYXZh) | `52.94% <0.00%> (-41.80%)` | :arrow_down: |
| [...perator/query/DictionaryBasedDistinctOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9xdWVyeS9EaWN0aW9uYXJ5QmFzZWREaXN0aW5jdE9wZXJhdG9yLmphdmE=) | `75.00% <0.00%> (+25.00%)` | :arrow_up: |
| ... and [1138 more](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [2107b2c...bdba462](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] amrishlal commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
amrishlal commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r758831328
##########
File path: pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseBrokerRequestHandler.java
##########
@@ -225,7 +225,13 @@ private BrokerResponseNative handleSQLRequest(long requestId, String query, Json
if (isLiteralOnlyQuery(pinotQuery)) {
LOGGER.debug("Request {} contains only Literal, skipping server query: {}", requestId, query);
try {
- return processLiteralOnlyQuery(pinotQuery, compilationStartTimeNs, requestStatistics);
+ BrokerResponseNative responseForLiteralOnly =
+ processLiteralOnlyQuery(pinotQuery, compilationStartTimeNs, requestStatistics);
Review comment:
Fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7568](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (e8f8685) into [master](https://codecov.io/gh/apache/pinot/commit/767aa8abfb5bf085ba0a7ae5ff4024679f27816e?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (767aa8a) will **decrease** coverage by `6.47%`.
> The diff coverage is `68.31%`.
[](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7568 +/- ##
============================================
- Coverage 71.69% 65.22% -6.48%
Complexity 4086 4086
============================================
Files 1579 1535 -44
Lines 80790 79339 -1451
Branches 12003 11874 -129
============================================
- Hits 57922 51748 -6174
- Misses 18981 23906 +4925
+ Partials 3887 3685 -202
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| integration2 | `?` | |
| unittests1 | `68.72% <70.91%> (+0.04%)` | :arrow_up: |
| unittests2 | `14.54% <1.81%> (-0.07%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...in/java/org/apache/pinot/core/common/Operator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9jb21tb24vT3BlcmF0b3IuamF2YQ==) | `0.00% <ø> (ø)` | |
| [...operator/AcquireReleaseColumnsSegmentOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9BY3F1aXJlUmVsZWFzZUNvbHVtbnNTZWdtZW50T3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...he/pinot/core/operator/BitmapDocIdSetOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CaXRtYXBEb2NJZFNldE9wZXJhdG9yLmphdmE=) | `68.18% <0.00%> (-6.82%)` | :arrow_down: |
| [.../core/operator/combine/GroupByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0dyb3VwQnlDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `73.91% <0.00%> (-1.09%)` | :arrow_down: |
| [...nMaxValueBasedSelectionOrderByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL01pbk1heFZhbHVlQmFzZWRTZWxlY3Rpb25PcmRlckJ5Q29tYmluZU9wZXJhdG9yLmphdmE=) | `60.60% <0.00%> (-11.92%)` | :arrow_down: |
| [...or/dociditerators/ExpressionScanDocIdIterator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9kb2NpZGl0ZXJhdG9ycy9FeHByZXNzaW9uU2NhbkRvY0lkSXRlcmF0b3IuamF2YQ==) | `47.18% <0.00%> (-19.25%)` | :arrow_down: |
| [...inot/core/operator/filter/EmptyFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvRW1wdHlGaWx0ZXJPcGVyYXRvci5qYXZh) | `62.50% <0.00%> (-17.50%)` | :arrow_down: |
| [...ot/core/operator/filter/H3IndexFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvSDNJbmRleEZpbHRlck9wZXJhdG9yLmphdmE=) | `81.25% <0.00%> (-5.42%)` | :arrow_down: |
| [...ore/operator/query/AggregationGroupByOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9xdWVyeS9BZ2dyZWdhdGlvbkdyb3VwQnlPcGVyYXRvci5qYXZh) | `47.05% <0.00%> (-47.68%)` | :arrow_down: |
| [...perator/query/DictionaryBasedDistinctOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9xdWVyeS9EaWN0aW9uYXJ5QmFzZWREaXN0aW5jdE9wZXJhdG9yLmphdmE=) | `75.00% <0.00%> (-3.27%)` | :arrow_down: |
| ... and [429 more](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [767aa8a...e8f8685](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7568](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (9957ddd) into [master](https://codecov.io/gh/apache/pinot/commit/767aa8abfb5bf085ba0a7ae5ff4024679f27816e?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (767aa8a) will **decrease** coverage by `1.21%`.
> The diff coverage is `73.24%`.
[](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7568 +/- ##
============================================
- Coverage 71.69% 70.47% -1.22%
- Complexity 4086 4090 +4
============================================
Files 1579 1580 +1
Lines 80790 81209 +419
Branches 12003 12077 +74
============================================
- Hits 57922 57233 -689
- Misses 18981 20067 +1086
- Partials 3887 3909 +22
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| integration2 | `28.02% <41.29%> (+0.09%)` | :arrow_up: |
| unittests1 | `68.75% <70.91%> (+0.08%)` | :arrow_up: |
| unittests2 | `14.51% <1.81%> (-0.09%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...in/java/org/apache/pinot/core/common/Operator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9jb21tb24vT3BlcmF0b3IuamF2YQ==) | `0.00% <ø> (ø)` | |
| [...operator/AcquireReleaseColumnsSegmentOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9BY3F1aXJlUmVsZWFzZUNvbHVtbnNTZWdtZW50T3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...he/pinot/core/operator/BitmapDocIdSetOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CaXRtYXBEb2NJZFNldE9wZXJhdG9yLmphdmE=) | `68.18% <0.00%> (-6.82%)` | :arrow_down: |
| [.../core/operator/combine/GroupByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0dyb3VwQnlDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `73.91% <0.00%> (-1.09%)` | :arrow_down: |
| [...nMaxValueBasedSelectionOrderByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL01pbk1heFZhbHVlQmFzZWRTZWxlY3Rpb25PcmRlckJ5Q29tYmluZU9wZXJhdG9yLmphdmE=) | `72.72% <0.00%> (+0.20%)` | :arrow_up: |
| [...or/dociditerators/ExpressionScanDocIdIterator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9kb2NpZGl0ZXJhdG9ycy9FeHByZXNzaW9uU2NhbkRvY0lkSXRlcmF0b3IuamF2YQ==) | `52.11% <0.00%> (-14.32%)` | :arrow_down: |
| [...inot/core/operator/filter/EmptyFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvRW1wdHlGaWx0ZXJPcGVyYXRvci5qYXZh) | `62.50% <0.00%> (-17.50%)` | :arrow_down: |
| [...ot/core/operator/filter/H3IndexFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvSDNJbmRleEZpbHRlck9wZXJhdG9yLmphdmE=) | `81.25% <0.00%> (-5.42%)` | :arrow_down: |
| [...ore/operator/query/AggregationGroupByOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9xdWVyeS9BZ2dyZWdhdGlvbkdyb3VwQnlPcGVyYXRvci5qYXZh) | `47.05% <0.00%> (-47.68%)` | :arrow_down: |
| [...perator/query/DictionaryBasedDistinctOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9xdWVyeS9EaWN0aW9uYXJ5QmFzZWREaXN0aW5jdE9wZXJhdG9yLmphdmE=) | `75.00% <0.00%> (-3.27%)` | :arrow_down: |
| ... and [179 more](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [767aa8a...9957ddd](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] siddharthteotia merged pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
siddharthteotia merged pull request #7568:
URL: https://github.com/apache/pinot/pull/7568
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] amrishlal edited a comment on pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
amrishlal edited a comment on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-982387933
> What do you think about merging `getExplainPlanName` into `toExplainString`?'
Yes, I don't see any issues with this. Will make the change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] siddharthteotia commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
siddharthteotia commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r760532746
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/DocIdSetOperator.java
##########
@@ -34,6 +37,7 @@
*/
public class DocIdSetOperator extends BaseOperator<DocIdSetBlock> {
private static final String OPERATOR_NAME = "DocIdSetOperator";
+ private static final String EXPLAIN_NAME = null;
Review comment:
I agree this should be part of the tree.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] richardstartin commented on a change in pull request #7568: [WORK IN PROGRESS] EXPLAIN PLAN code
Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r727460652
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/AcquireReleaseColumnsSegmentOperator.java
##########
@@ -77,6 +81,16 @@ public String getOperatorName() {
return OPERATOR_NAME;
}
+ @Override
+ public String getExplainPlanName() {
+ return EXPLAIN_NAME;
+ }
+
+ @Override
+ public List<Operator> getChildOperators() {
+ return Arrays.asList(_childOperator);
Review comment:
Of course, it wasn't a really a _review_ comment, just something that stood out at me glancing at the diff.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] richardstartin commented on a change in pull request #7568: [WORK IN PROGRESS] EXPLAIN PLAN code
Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r727438086
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/AcquireReleaseColumnsSegmentOperator.java
##########
@@ -77,6 +81,16 @@ public String getOperatorName() {
return OPERATOR_NAME;
}
+ @Override
+ public String getExplainPlanName() {
+ return EXPLAIN_NAME;
+ }
+
+ @Override
+ public List<Operator> getChildOperators() {
+ return Arrays.asList(_childOperator);
Review comment:
Can you replace all of these one element `Arrays.asList(arg)` with `Collections.singletonList(arg)` - just because it creates IDE warnings otherwise, and it's nice to keep SNR high with IDE warnings.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7568](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (a73b310) into [master](https://codecov.io/gh/apache/pinot/commit/2107b2cd1880ec74934ec5412dbee8c874f05610?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (2107b2c) will **decrease** coverage by `16.25%`.
> The diff coverage is `3.46%`.
> :exclamation: Current head a73b310 differs from pull request most recent head 978c92a. Consider uploading reports for the commit 978c92a to get more accurate results
[](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7568 +/- ##
=============================================
- Coverage 30.79% 14.53% -16.26%
- Complexity 0 80 +80
=============================================
Files 1570 1534 -36
Lines 80024 79126 -898
Branches 11904 11843 -61
=============================================
- Hits 24644 11502 -13142
- Misses 53274 66781 +13507
+ Partials 2106 843 -1263
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| integration2 | `?` | |
| unittests2 | `14.53% <3.46%> (?)` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...rg/apache/pinot/common/function/JsonPathCache.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vSnNvblBhdGhDYWNoZS5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...he/pinot/common/function/scalar/JsonFunctions.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZnVuY3Rpb24vc2NhbGFyL0pzb25GdW5jdGlvbnMuamF2YQ==) | `0.00% <0.00%> (-30.51%)` | :arrow_down: |
| [...apache/pinot/common/metrics/ValidationMetrics.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vbWV0cmljcy9WYWxpZGF0aW9uTWV0cmljcy5qYXZh) | `0.00% <0.00%> (-80.96%)` | :arrow_down: |
| [...ache/pinot/common/tier/PinotServerTierStorage.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdGllci9QaW5vdFNlcnZlclRpZXJTdG9yYWdlLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...c/main/java/org/apache/pinot/common/tier/Tier.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdGllci9UaWVyLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...java/org/apache/pinot/common/tier/TierFactory.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdGllci9UaWVyRmFjdG9yeS5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...inot/common/tier/TimeBasedTierSegmentSelector.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdGllci9UaW1lQmFzZWRUaWVyU2VnbWVudFNlbGVjdG9yLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [.../pinot/common/utils/ClientSSLContextGenerator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvQ2xpZW50U1NMQ29udGV4dEdlbmVyYXRvci5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...java/org/apache/pinot/common/utils/DataSchema.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvRGF0YVNjaGVtYS5qYXZh) | `0.00% <0.00%> (-46.19%)` | :arrow_down: |
| [...e/pinot/common/utils/FileUploadDownloadClient.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvRmlsZVVwbG9hZERvd25sb2FkQ2xpZW50LmphdmE=) | `0.00% <0.00%> (-59.75%)` | :arrow_down: |
| ... and [936 more](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [2107b2c...978c92a](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7568](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (c549639) into [master](https://codecov.io/gh/apache/pinot/commit/d42e415020d53afca55f04a3f89f988ffb9ca045?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (d42e415) will **decrease** coverage by `57.03%`.
> The diff coverage is `2.10%`.
[](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7568 +/- ##
=============================================
- Coverage 71.60% 14.57% -57.04%
+ Complexity 3942 80 -3862
=============================================
Files 1562 1517 -45
Lines 79465 77963 -1502
Branches 11766 11626 -140
=============================================
- Hits 56903 11365 -45538
- Misses 18724 65784 +47060
+ Partials 3838 814 -3024
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| integration2 | `?` | |
| unittests1 | `?` | |
| unittests2 | `14.57% <2.10%> (-0.07%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...java/org/apache/pinot/common/utils/DataSchema.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvRGF0YVNjaGVtYS5qYXZh) | `0.00% <0.00%> (-79.24%)` | :arrow_down: |
| [...org/apache/pinot/sql/parsers/CalciteSqlParser.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcWwvcGFyc2Vycy9DYWxjaXRlU3FsUGFyc2VyLmphdmE=) | `0.00% <0.00%> (-91.40%)` | :arrow_down: |
| [...operator/AcquireReleaseColumnsSegmentOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9BY3F1aXJlUmVsZWFzZUNvbHVtbnNTZWdtZW50T3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...a/org/apache/pinot/core/operator/BaseOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CYXNlT3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (-35.72%)` | :arrow_down: |
| [...he/pinot/core/operator/BitmapDocIdSetOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CaXRtYXBEb2NJZFNldE9wZXJhdG9yLmphdmE=) | `0.00% <0.00%> (-75.00%)` | :arrow_down: |
| [...g/apache/pinot/core/operator/DocIdSetOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9Eb2NJZFNldE9wZXJhdG9yLmphdmE=) | `0.00% <0.00%> (-92.31%)` | :arrow_down: |
| [.../pinot/core/operator/InstanceResponseOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9JbnN0YW5jZVJlc3BvbnNlT3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (-83.88%)` | :arrow_down: |
| [...apache/pinot/core/operator/ProjectionOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9Qcm9qZWN0aW9uT3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (-73.34%)` | :arrow_down: |
| [...erator/combine/AggregationOnlyCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0FnZ3JlZ2F0aW9uT25seUNvbWJpbmVPcGVyYXRvci5qYXZh) | `0.00% <0.00%> (-75.00%)` | :arrow_down: |
| [...not/core/operator/combine/BaseCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0Jhc2VDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (-87.84%)` | :arrow_down: |
| ... and [1291 more](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [d42e415...c549639](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] siddharthteotia commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
siddharthteotia commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r745782668
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/BaseOperator.java
##########
@@ -52,4 +54,25 @@ public final T nextBlock() {
// Make it protected because we should always call nextBlock()
protected abstract T getNextBlock();
+
+ // Enforcing sub-class to implement the getOperatorName(), as they can just return a static final,
+ // as opposed to this super class calling getClass().getSimpleName().
+ public abstract String getOperatorName();
+
+ public abstract String getExplainPlanName();
+
+ @Override
+ public List<Operator> getChildOperators() {
+ return new ArrayList<>();
Review comment:
Collections.emptyList() might be better ?
##########
File path: pinot-broker/src/test/java/org/apache/pinot/broker/requesthandler/LiteralOnlyBrokerRequestTest.java
##########
@@ -177,4 +177,55 @@ public void testBrokerRequestHandlerWithAsFunction()
Assert.assertEquals(brokerResponse.getResultTable().getRows().get(0)[1], 1577836800000L);
Assert.assertEquals(brokerResponse.getTotalDocs(), 0);
}
+
+ /** Tests for EXPLAIN PLAN for literal only queries. */
+ @Test
+ public void testExplainPlanLiteralOnly()
+ throws Exception {
+ SingleConnectionBrokerRequestHandler requestHandler =
+ new SingleConnectionBrokerRequestHandler(new PinotConfiguration(), null, ACCESS_CONTROL_FACTORY, null, null,
+ new BrokerMetrics("", PinotMetricUtils.getPinotMetricsRegistry(), true, Collections.emptySet()), null);
+
+ // Test 1: select constant
+ JsonNode request = new ObjectMapper().readTree("{\"sql\":\"EXPLAIN PLAN FOR SELECT 1.5, 'test'\"}");
+ RequestStatistics requestStats = new RequestStatistics();
+ BrokerResponseNative brokerResponse = requestHandler.handleRequest(request, null, requestStats);
+
+ checkExplainResultSchema(brokerResponse.getResultTable().getDataSchema(),
+ new String[]{"Operator", "Operator_Id", "Parent_Id"},
+ new DataSchema.ColumnDataType[]{DataSchema.ColumnDataType.STRING, DataSchema.ColumnDataType.INT,
+ DataSchema.ColumnDataType.INT});
+
+ Assert.assertEquals(brokerResponse.getResultTable().getRows().size(), 1);
+ Assert.assertEquals(brokerResponse.getResultTable().getRows().get(0),
+ new Object[]{"SELECT(selectList:literal)", 0, -1});
+ Assert.assertEquals(brokerResponse.getTotalDocs(), 0);
+
+ // Test 2: invoke compile time function -> literal only
+ long currentTsMin = System.currentTimeMillis();
+ request = new ObjectMapper().readTree(
+ "{\"sql\":\"EXPLAIN PLAN FOR SELECT 6+8 as addition, fromDateTime('2020-01-01 UTC', 'yyyy-MM-dd z') as "
+ + "firstDayOf2020\"}");
+ requestStats = new RequestStatistics();
+ brokerResponse = requestHandler.handleRequest(request, null, requestStats);
+
+ checkExplainResultSchema(brokerResponse.getResultTable().getDataSchema(),
+ new String[]{"Operator", "Operator_Id", "Parent_Id"},
+ new DataSchema.ColumnDataType[]{DataSchema.ColumnDataType.STRING, DataSchema.ColumnDataType.INT,
+ DataSchema.ColumnDataType.INT});
+
+ Assert.assertEquals(brokerResponse.getResultTable().getRows().size(), 1);
+ Assert.assertEquals(brokerResponse.getResultTable().getRows().get(0),
+ new Object[]{"SELECT(selectList:literal)", 0, -1});
Review comment:
This generic selectList:literal plan does not look correct especially for this query for couple of reasons
- We are selecting 2 columns 6+8 (14) and fromDateTime value. The plan only shows one column
- This is a combination of both compile time function invocation followed by literal only. The plan doesn't reflect that
##########
File path: pinot-common/src/main/java/org/apache/pinot/common/request/Selection.java
##########
@@ -124,15 +124,15 @@ public short getThriftFieldId() {
public static final java.util.Map<_Fields, org.apache.thrift.meta_data.FieldMetaData> metaDataMap;
static {
java.util.Map<_Fields, org.apache.thrift.meta_data.FieldMetaData> tmpMap = new java.util.EnumMap<_Fields, org.apache.thrift.meta_data.FieldMetaData>(_Fields.class);
- tmpMap.put(_Fields.SELECTION_COLUMNS, new org.apache.thrift.meta_data.FieldMetaData("selectionColumns", org.apache.thrift.TFieldRequirementType.OPTIONAL,
- new org.apache.thrift.meta_data.ListMetaData(org.apache.thrift.protocol.TType.LIST,
+ tmpMap.put(_Fields.SELECTION_COLUMNS, new org.apache.thrift.meta_data.FieldMetaData("selectionColumns", org.apache.thrift.TFieldRequirementType.OPTIONAL,
Review comment:
Was the current (before this PR) formatting incorrect for these files in` common/request` ?
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/DocIdSetOperator.java
##########
@@ -34,6 +37,7 @@
*/
public class DocIdSetOperator extends BaseOperator<DocIdSetBlock> {
private static final String OPERATOR_NAME = "DocIdSetOperator";
+ private static final String EXPLAIN_NAME = null;
Review comment:
Why null ? Is it because we want to skip this operator ?
##########
File path: pinot-broker/src/test/java/org/apache/pinot/broker/requesthandler/LiteralOnlyBrokerRequestTest.java
##########
@@ -177,4 +177,55 @@ public void testBrokerRequestHandlerWithAsFunction()
Assert.assertEquals(brokerResponse.getResultTable().getRows().get(0)[1], 1577836800000L);
Assert.assertEquals(brokerResponse.getTotalDocs(), 0);
}
+
+ /** Tests for EXPLAIN PLAN for literal only queries. */
+ @Test
+ public void testExplainPlanLiteralOnly()
+ throws Exception {
+ SingleConnectionBrokerRequestHandler requestHandler =
+ new SingleConnectionBrokerRequestHandler(new PinotConfiguration(), null, ACCESS_CONTROL_FACTORY, null, null,
+ new BrokerMetrics("", PinotMetricUtils.getPinotMetricsRegistry(), true, Collections.emptySet()), null);
+
+ // Test 1: select constant
+ JsonNode request = new ObjectMapper().readTree("{\"sql\":\"EXPLAIN PLAN FOR SELECT 1.5, 'test'\"}");
+ RequestStatistics requestStats = new RequestStatistics();
+ BrokerResponseNative brokerResponse = requestHandler.handleRequest(request, null, requestStats);
+
+ checkExplainResultSchema(brokerResponse.getResultTable().getDataSchema(),
+ new String[]{"Operator", "Operator_Id", "Parent_Id"},
+ new DataSchema.ColumnDataType[]{DataSchema.ColumnDataType.STRING, DataSchema.ColumnDataType.INT,
+ DataSchema.ColumnDataType.INT});
+
+ Assert.assertEquals(brokerResponse.getResultTable().getRows().size(), 1);
+ Assert.assertEquals(brokerResponse.getResultTable().getRows().get(0),
+ new Object[]{"SELECT(selectList:literal)", 0, -1});
Review comment:
It should show 1.5 imo
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/BaseOperator.java
##########
@@ -52,4 +54,25 @@ public final T nextBlock() {
// Make it protected because we should always call nextBlock()
protected abstract T getNextBlock();
+
+ // Enforcing sub-class to implement the getOperatorName(), as they can just return a static final,
+ // as opposed to this super class calling getClass().getSimpleName().
+ public abstract String getOperatorName();
Review comment:
I thought there was another method we discussed w.r.t whether a node should be skipped or not ?
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/ProjectionOperator.java
##########
@@ -68,6 +72,36 @@ public String getOperatorName() {
return OPERATOR_NAME;
}
+ @Override
+ public String getExplainPlanName() {
+ return EXPLAIN_NAME;
+ }
+
+ @Override
+ public List<Operator> getChildOperators() {
+ return Collections.singletonList(_docIdSetOperator);
+ }
+
+ @Override
+ public String toExplainString() {
+ StringBuilder stringBuilder = new StringBuilder(getExplainPlanName()).append('(');
+ if (_dataSourceMap.keySet().isEmpty()) {
+ // count aggregation function has empty input expressions
+ stringBuilder.append("ALL");
Review comment:
I don't follow this. `PROJECT ALL` makes sense when we are projecting / selecting all columns. However, this comment is implying that we will do `PROJECT ALL` for count aggregation function which doesn't sound right
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/BaseOperator.java
##########
@@ -52,4 +54,25 @@ public final T nextBlock() {
// Make it protected because we should always call nextBlock()
protected abstract T getNextBlock();
+
+ // Enforcing sub-class to implement the getOperatorName(), as they can just return a static final,
+ // as opposed to this super class calling getClass().getSimpleName().
+ public abstract String getOperatorName();
+
+ public abstract String getExplainPlanName();
+
+ @Override
+ public List<Operator> getChildOperators() {
+ return new ArrayList<>();
Review comment:
I also feel that not making this abstract has the potential for problems if someone who is writing a new operator misses out on implementing this interface (because compiler won't complain) but the operator has one or more child operators. So I would suggest to make this abstract and force every operator to implement this even if it has to return empty list
##########
File path: pinot-core/src/test/java/org/apache/pinot/queries/ExplainPlanQueriesTest.java
##########
@@ -0,0 +1,510 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.queries;
+
+import java.io.File;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.List;
+import org.apache.commons.io.FileUtils;
+import org.apache.pinot.common.response.broker.BrokerResponseNative;
+import org.apache.pinot.common.response.broker.ResultTable;
+import org.apache.pinot.common.utils.DataSchema;
+import org.apache.pinot.segment.local.indexsegment.immutable.ImmutableSegmentLoader;
+import org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl;
+import org.apache.pinot.segment.local.segment.index.loader.IndexLoadingConfig;
+import org.apache.pinot.segment.local.segment.readers.GenericRowRecordReader;
+import org.apache.pinot.segment.spi.ImmutableSegment;
+import org.apache.pinot.segment.spi.IndexSegment;
+import org.apache.pinot.segment.spi.creator.SegmentGeneratorConfig;
+import org.apache.pinot.spi.config.table.IndexingConfig;
+import org.apache.pinot.spi.config.table.TableConfig;
+import org.apache.pinot.spi.config.table.TableType;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.data.Schema;
+import org.apache.pinot.spi.data.readers.GenericRow;
+import org.apache.pinot.spi.utils.ReadMode;
+import org.apache.pinot.spi.utils.builder.TableConfigBuilder;
+import org.testng.Assert;
+import org.testng.annotations.BeforeClass;
+import org.testng.annotations.Test;
+
+
+public class ExplainPlanQueriesTest extends BaseQueriesTest {
+ private static final File INDEX_DIR = new File(FileUtils.getTempDirectory(), "ExplainPlanQueriesTest");
+ private static final String RAW_TABLE_NAME = "testTable";
+ private static final String SEGMENT_NAME = "testSegment";
+ private static final int NUM_RECORDS = 10;
+
+ private final static String COL1_NO_INDEX = "noIndexCol1";
+ private final static String COL2_NO_INDEX = "noIndexCol2";
+ private final static String COL3_NO_INDEX = "noIndexCol3";
+ private final static String COL1_INVERTED_INDEX = "invertedIndexCol1";
+ private final static String COL2_INVERTED_INDEX = "invertedIndexCol2";
+ private final static String COL3_INVERTED_INDEX = "invertedIndexCol3";
+ private final static String COL1_RANGE_INDEX = "rangeIndexCol1";
+ private final static String COL2_RANGE_INDEX = "rangeIndexCol2";
+ private final static String COL3_RANGE_INDEX = "rangeIndexCol3";
+ private final static String COL1_SORTED_INDEX = "sortedIndexCol1";
+ private final static String COL1_JSON_INDEX = "jsonIndexCol1";
+ private final static String COL1_TEXT_INDEX = "textIndexCol1";
+
+ private static final Schema SCHEMA = new Schema.SchemaBuilder().setSchemaName(RAW_TABLE_NAME)
+ .addSingleValueDimension(COL1_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL2_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_INVERTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_INVERTED_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_INVERTED_INDEX, FieldSpec.DataType.STRING)
+ .addSingleValueDimension(COL1_RANGE_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_SORTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL1_JSON_INDEX, FieldSpec.DataType.JSON)
+ .addSingleValueDimension(COL1_TEXT_INDEX, FieldSpec.DataType.STRING).build();
+
+ private static final DataSchema DATA_SCHEMA = new DataSchema(new String[]{"Operator", "Operator_Id", "Parent_Id"},
+ new DataSchema.ColumnDataType[]{DataSchema.ColumnDataType.STRING, DataSchema.ColumnDataType.INT,
+ DataSchema.ColumnDataType.INT});
+
+ private static final TableConfig TABLE_CONFIG =
+ new TableConfigBuilder(TableType.OFFLINE).setTableName(RAW_TABLE_NAME).build();
+
+ private IndexSegment _indexSegment;
+ private List<IndexSegment> _indexSegments;
+
+ @Override
+ protected String getFilter() {
+ return "";
+ }
+
+ @Override
+ protected IndexSegment getIndexSegment() {
+ return _indexSegment;
+ }
+
+ @Override
+ protected List<IndexSegment> getIndexSegments() {
+ return _indexSegments;
+ }
+
+ GenericRow createRecord(double doubleValue, int intValue, String stringValue, String jsonValue) {
+ GenericRow record = new GenericRow();
+ record.putValue(COL1_NO_INDEX, intValue);
+ record.putValue(COL2_NO_INDEX, intValue);
+ record.putValue(COL3_NO_INDEX, intValue);
+
+ record.putValue(COL1_INVERTED_INDEX, doubleValue);
+ record.putValue(COL2_INVERTED_INDEX, intValue);
+ record.putValue(COL3_INVERTED_INDEX, intValue);
+
+ record.putValue(COL1_RANGE_INDEX, doubleValue);
+ record.putValue(COL2_RANGE_INDEX, intValue);
+ record.putValue(COL3_RANGE_INDEX, intValue);
+
+ record.putValue(COL1_JSON_INDEX, jsonValue);
+ record.putValue(COL1_TEXT_INDEX, stringValue);
+
+ return record;
+ }
+
+ @BeforeClass
+ public void setUp()
+ throws Exception {
+ FileUtils.deleteDirectory(INDEX_DIR);
+
+ List<GenericRow> records = new ArrayList<>(NUM_RECORDS);
+ records.add(createRecord(1.1, 2, "daffy", "{\"first\": \"daffy\", \"last\": \"duck\"}"));
+
+ IndexingConfig indexingConfig = TABLE_CONFIG.getIndexingConfig();
+
+ List<String> invertedIndexColumns = Arrays.asList(COL1_INVERTED_INDEX, COL2_INVERTED_INDEX, COL3_INVERTED_INDEX);
+ indexingConfig.setInvertedIndexColumns(invertedIndexColumns);
+
+ List<String> rangeIndexColumns = Arrays.asList(COL1_RANGE_INDEX, COL2_RANGE_INDEX, COL3_RANGE_INDEX);
+ indexingConfig.setRangeIndexColumns(rangeIndexColumns);
+
+ List<String> jsonIndexColumns = Arrays.asList(COL1_JSON_INDEX);
+ indexingConfig.setJsonIndexColumns(jsonIndexColumns);
+
+ SegmentGeneratorConfig segmentGeneratorConfig = new SegmentGeneratorConfig(TABLE_CONFIG, SCHEMA);
+ segmentGeneratorConfig.setTableName(RAW_TABLE_NAME);
+ segmentGeneratorConfig.setSegmentName(SEGMENT_NAME);
+ segmentGeneratorConfig.setOutDir(INDEX_DIR.getPath());
+
+ SegmentIndexCreationDriverImpl driver = new SegmentIndexCreationDriverImpl();
+ driver.init(segmentGeneratorConfig, new GenericRowRecordReader(records));
+ driver.build();
+
+ IndexLoadingConfig indexLoadingConfig = new IndexLoadingConfig();
+ indexLoadingConfig.setTableConfig(TABLE_CONFIG);
+ indexLoadingConfig.setInvertedIndexColumns(new HashSet<>(invertedIndexColumns));
+ indexLoadingConfig.setRangeIndexColumns(new HashSet<>(rangeIndexColumns));
+ indexLoadingConfig.setJsonIndexColumns(new HashSet<String>(jsonIndexColumns));
+ indexLoadingConfig.setReadMode(ReadMode.mmap);
+
+ ImmutableSegment immutableSegment =
+ ImmutableSegmentLoader.load(new File(INDEX_DIR, SEGMENT_NAME), indexLoadingConfig);
+ _indexSegment = immutableSegment;
+ _indexSegments = Arrays.asList(immutableSegment);
+ }
+
+ /** Checks the correctness of EXPLAIN PLAN output. */
+ private void check(String query, ResultTable expected) {
+ BrokerResponseNative response = getBrokerResponseForSqlQuery(query);
+ ResultTable actual = response.getResultTable();
+
+ System.out.println(resultTableToString("Expected", expected));
+ System.out.println(resultTableToString("Actual", actual));
+
+ Assert.assertEquals(actual.getDataSchema(), expected.getDataSchema());
+ Assert.assertEquals(actual.getRows().size(), expected.getRows().size());
+
+ List<Object[]> expectedRows = expected.getRows();
+ List<Object[]> actualRows = actual.getRows();
+ for (int i = 0; i < expectedRows.size(); i++) {
+ Object[] expectedRow = expectedRows.get(i);
+ Object[] actualRow = actualRows.get(i);
+ Assert.assertEquals(actualRow.length, expectedRow.length);
+
+ for (int j = 0; j < actualRow.length; j++) {
+ Assert.assertEquals(actualRow[j], expectedRow[j]);
+ }
+ }
+ }
+
+ /** This function is used to generated expected results while creating test cases; otherwise, not used. */
+ private static String resultTableToString(String tag, ResultTable table) {
+ System.out.println("\n" + tag + ":");
+ List<Object[]> rows = table.getRows();
+
+ StringBuffer buffer = new StringBuffer();
+ for (Object[] row : rows) {
+ buffer.append("result.add(new Object[]{\"" + row[0] + "\", " + row[1] + ", " + row[2] + "});\n");
+ }
+
+ return buffer.toString();
+ }
+
+ @Test
+ public void testSelect() {
+ String query1 = "EXPLAIN PLAN FOR SELECT * FROM testTable";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(
+ new Object[]{"SELECT(selectList:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, jsonIndexCol1, "
+ + "noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, "
+ + "jsonIndexCol1, noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 3, 2});
+ result1.add(
+ new Object[]{"PROJECT(sortedIndexCol1, noIndexCol3, rangeIndexCol1, rangeIndexCol2, jsonIndexCol1, "
+ + "invertedIndexCol1, noIndexCol2, invertedIndexCol2, noIndexCol1, invertedIndexCol3, rangeIndexCol3, "
+ + "textIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 = "EXPLAIN PLAN FOR SELECT 'mickey' FROM testTable";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:'mickey')", 2, 1});
+ result2.add(new Object[]{"PROJECT(ALL)", 3, 2});
Review comment:
This is confusing. Why use `PROJECT (ALL)` when we are only selecting / projecting a single column. In fact, this is counter-intuitive. Previous query uses `SELECT *` so `PROJECT(ALL)` makes sense there but we don't use `PROJECT (ALL)`
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/BaseOperator.java
##########
@@ -52,4 +54,25 @@ public final T nextBlock() {
// Make it protected because we should always call nextBlock()
protected abstract T getNextBlock();
+
+ // Enforcing sub-class to implement the getOperatorName(), as they can just return a static final,
+ // as opposed to this super class calling getClass().getSimpleName().
+ public abstract String getOperatorName();
+
+ public abstract String getExplainPlanName();
+
+ @Override
+ public List<Operator> getChildOperators() {
+ return new ArrayList<>();
+ }
+
+ @Override
+ public String toExplainString() {
+ return getExplainPlanName();
+ }
+
+ @Override
Review comment:
Why this change ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] amrishlal commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
amrishlal commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r746093704
##########
File path: pinot-broker/src/test/java/org/apache/pinot/broker/requesthandler/LiteralOnlyBrokerRequestTest.java
##########
@@ -177,4 +177,55 @@ public void testBrokerRequestHandlerWithAsFunction()
Assert.assertEquals(brokerResponse.getResultTable().getRows().get(0)[1], 1577836800000L);
Assert.assertEquals(brokerResponse.getTotalDocs(), 0);
}
+
+ /** Tests for EXPLAIN PLAN for literal only queries. */
+ @Test
+ public void testExplainPlanLiteralOnly()
+ throws Exception {
+ SingleConnectionBrokerRequestHandler requestHandler =
+ new SingleConnectionBrokerRequestHandler(new PinotConfiguration(), null, ACCESS_CONTROL_FACTORY, null, null,
+ new BrokerMetrics("", PinotMetricUtils.getPinotMetricsRegistry(), true, Collections.emptySet()), null);
+
+ // Test 1: select constant
+ JsonNode request = new ObjectMapper().readTree("{\"sql\":\"EXPLAIN PLAN FOR SELECT 1.5, 'test'\"}");
+ RequestStatistics requestStats = new RequestStatistics();
+ BrokerResponseNative brokerResponse = requestHandler.handleRequest(request, null, requestStats);
+
+ checkExplainResultSchema(brokerResponse.getResultTable().getDataSchema(),
+ new String[]{"Operator", "Operator_Id", "Parent_Id"},
+ new DataSchema.ColumnDataType[]{DataSchema.ColumnDataType.STRING, DataSchema.ColumnDataType.INT,
+ DataSchema.ColumnDataType.INT});
+
+ Assert.assertEquals(brokerResponse.getResultTable().getRows().size(), 1);
+ Assert.assertEquals(brokerResponse.getResultTable().getRows().get(0),
+ new Object[]{"SELECT(selectList:literal)", 0, -1});
+ Assert.assertEquals(brokerResponse.getTotalDocs(), 0);
+
+ // Test 2: invoke compile time function -> literal only
+ long currentTsMin = System.currentTimeMillis();
+ request = new ObjectMapper().readTree(
+ "{\"sql\":\"EXPLAIN PLAN FOR SELECT 6+8 as addition, fromDateTime('2020-01-01 UTC', 'yyyy-MM-dd z') as "
+ + "firstDayOf2020\"}");
+ requestStats = new RequestStatistics();
+ brokerResponse = requestHandler.handleRequest(request, null, requestStats);
+
+ checkExplainResultSchema(brokerResponse.getResultTable().getDataSchema(),
+ new String[]{"Operator", "Operator_Id", "Parent_Id"},
+ new DataSchema.ColumnDataType[]{DataSchema.ColumnDataType.STRING, DataSchema.ColumnDataType.INT,
+ DataSchema.ColumnDataType.INT});
+
+ Assert.assertEquals(brokerResponse.getResultTable().getRows().size(), 1);
+ Assert.assertEquals(brokerResponse.getResultTable().getRows().get(0),
+ new Object[]{"SELECT(selectList:literal)", 0, -1});
Review comment:
`SELECT(selectList:literal)` will only show up if ALL the column in the select list are literals, so no point in displaying something like `SELECT(selectList:literal, literal, literal, ...)`. Changed it to plural though: `SELECT(selectList:literals)`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7568](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (22e210a) into [master](https://codecov.io/gh/apache/pinot/commit/2107b2cd1880ec74934ec5412dbee8c874f05610?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (2107b2c) will **decrease** coverage by `16.28%`.
> The diff coverage is `2.08%`.
[](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7568 +/- ##
=============================================
- Coverage 30.79% 14.51% -16.29%
- Complexity 0 80 +80
=============================================
Files 1570 1530 -40
Lines 80024 78771 -1253
Branches 11904 11794 -110
=============================================
- Hits 24644 11433 -13211
- Misses 53274 66505 +13231
+ Partials 2106 833 -1273
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| integration2 | `?` | |
| unittests2 | `14.51% <2.08%> (?)` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...java/org/apache/pinot/common/utils/DataSchema.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvRGF0YVNjaGVtYS5qYXZh) | `0.00% <0.00%> (-46.19%)` | :arrow_down: |
| [...org/apache/pinot/sql/parsers/CalciteSqlParser.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcWwvcGFyc2Vycy9DYWxjaXRlU3FsUGFyc2VyLmphdmE=) | `0.00% <0.00%> (-69.74%)` | :arrow_down: |
| [...in/java/org/apache/pinot/core/common/Operator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9jb21tb24vT3BlcmF0b3IuamF2YQ==) | `0.00% <ø> (ø)` | |
| [...operator/AcquireReleaseColumnsSegmentOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9BY3F1aXJlUmVsZWFzZUNvbHVtbnNTZWdtZW50T3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...a/org/apache/pinot/core/operator/BaseOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CYXNlT3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (-38.47%)` | :arrow_down: |
| [...he/pinot/core/operator/BitmapDocIdSetOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CaXRtYXBEb2NJZFNldE9wZXJhdG9yLmphdmE=) | `0.00% <0.00%> (-75.00%)` | :arrow_down: |
| [...g/apache/pinot/core/operator/DocIdSetOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9Eb2NJZFNldE9wZXJhdG9yLmphdmE=) | `0.00% <0.00%> (-92.31%)` | :arrow_down: |
| [.../pinot/core/operator/InstanceResponseOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9JbnN0YW5jZVJlc3BvbnNlT3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (-83.88%)` | :arrow_down: |
| [...apache/pinot/core/operator/ProjectionOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9Qcm9qZWN0aW9uT3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (-73.34%)` | :arrow_down: |
| [...erator/combine/AggregationOnlyCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0FnZ3JlZ2F0aW9uT25seUNvbWJpbmVPcGVyYXRvci5qYXZh) | `0.00% <0.00%> (-75.00%)` | :arrow_down: |
| ... and [867 more](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [2107b2c...22e210a](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r758918478
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/query/request/context/utils/QueryContextUtils.java
##########
@@ -52,4 +58,79 @@ public static boolean isDistinctQuery(QueryContext query) {
return aggregationFunctions != null && aggregationFunctions.length == 1
&& aggregationFunctions[0] instanceof DistinctAggregationFunction;
}
+
+ /** Collect aggregation functions (except for the ones in filter). */
+ public static void generateTransforms(QueryContext queryContext, Set<String> postAggregations) {
Review comment:
Rename it to `collectPostAggregations`? Same for other methods
##########
File path: pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseBrokerRequestHandler.java
##########
@@ -519,6 +545,15 @@ private BrokerResponseNative handleSQLRequest(long requestId, String query, Json
return brokerResponse;
}
+ /** Set EXPLAIN PLAN query to route to only one server. */
+ private void setRoutingToOneServer(Map<ServerInstance, List<String>> routingTable) {
Review comment:
(minor) Rename the method to reflect that only one segment is queried
##########
File path: pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseBrokerRequestHandler.java
##########
@@ -491,6 +504,19 @@ private BrokerResponseNative handleSQLRequest(long requestId, String query, Json
// Execute the query
ServerStats serverStats = new ServerStats();
+ if (brokerRequest.getPinotQuery().isExplain()) {
Review comment:
(minor)
```suggestion
if (pinotQuery.isExplain()) {
```
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/BaseOperator.java
##########
@@ -52,4 +52,14 @@ public final T nextBlock() {
// Make it protected because we should always call nextBlock()
protected abstract T getNextBlock();
+
+ @Override
+ public String toExplainString() {
+ return getExplainPlanName();
+ }
+
+ @Override
+ public ExecutionStatistics getExecutionStatistics() {
Review comment:
This method override can be removed because it is the same as the default implementation in the interface
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/filter/EmptyFilterOperator.java
##########
@@ -26,10 +29,9 @@
* Singleton class which extends {@link BaseFilterOperator} that is empty, i.e. does not match any document.
*/
public final class EmptyFilterOperator extends BaseFilterOperator {
- private EmptyFilterOperator() {
Review comment:
Don't remove this private constructor. We want to use this operator as a singleton
##########
File path: pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseBrokerRequestHandler.java
##########
@@ -106,6 +106,7 @@
private static final String IN_SUBQUERY = "inSubquery";
private static final Expression FALSE = RequestUtils.getLiteralExpression(false);
private static final Expression TRUE = RequestUtils.getLiteralExpression(true);
+ private static final BrokerResponseNative BROKER_ONLY_EXPLAIN_PLAN_OUTPUT = getBrokerResponseExplainPlanOutput();
Review comment:
Suggest moving this to `BrokerResponseNative` class which contains all the constant responses
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/common/Operator.java
##########
@@ -39,12 +40,20 @@
*/
T nextBlock();
+ /** @return Name of this operator */
+ String getOperatorName();
+
+ /** @return List of {@link Operator}s that this operator depends upon. */
+ List<Operator> getChildOperators();
+
/**
- * Returns the name of the operator.
- * NOTE: This method is called for tracing purpose. The sub-class should try to return a constant to avoid the
- * unnecessary overhead.
+ * @return EXPLAIN PLAN name if one is defined; otherwise, null. null EXPLAIN PLAN name means that this operator
+ * won't be included in EXPLAIN PLAN output.
*/
- String getOperatorName();
+ String getExplainPlanName();
Review comment:
^^
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/filter/predicate/PredicateEvaluator.java
##########
@@ -28,6 +28,16 @@
* APIs for both dictionary based and raw value based predicate evaluator
*/
+ /**
+ * Set the predicate
+ */
+ void setPredicate(Predicate predicate);
Review comment:
#7840
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] amrishlal commented on a change in pull request #7568: [WORK IN PROGRESS] EXPLAIN PLAN code
Posted by GitBox <gi...@apache.org>.
amrishlal commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r727439934
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/AcquireReleaseColumnsSegmentOperator.java
##########
@@ -77,6 +81,16 @@ public String getOperatorName() {
return OPERATOR_NAME;
}
+ @Override
+ public String getExplainPlanName() {
+ return EXPLAIN_NAME;
+ }
+
+ @Override
+ public List<Operator> getChildOperators() {
+ return Arrays.asList(_childOperator);
Review comment:
OK, but let's please hold on the reviews for now until I can get the test cases passing along with a decent description :-) I will open it for review soon.
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/AcquireReleaseColumnsSegmentOperator.java
##########
@@ -77,6 +81,16 @@ public String getOperatorName() {
return OPERATOR_NAME;
}
+ @Override
+ public String getExplainPlanName() {
+ return EXPLAIN_NAME;
+ }
+
+ @Override
+ public List<Operator> getChildOperators() {
+ return Arrays.asList(_childOperator);
Review comment:
OK, but let's please hold on the reviews for now until I can get the test cases passing and add a decent description :-) I will open it for review soon.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] mqliang commented on pull request #7568: [WORK IN PROGRESS] EXPLAIN PLAN code
Posted by GitBox <gi...@apache.org>.
mqliang commented on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941631706
Can you post a few examples of the EXPLAIN PLAN query result?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] amrishlal commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
amrishlal commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r749486130
##########
File path: pinot-common/src/main/java/org/apache/pinot/common/request/PinotQuery.java
##########
@@ -593,6 +604,28 @@ public void setQueryOptionsIsSet(boolean value) {
}
}
+ public boolean isExplain() {
+ return this.explain;
+ }
+
+ public void setExplain(boolean explain) {
+ this.explain = explain;
+ setExplainIsSet(true);
Review comment:
> Can we also add some tests under integration tests, like `OfflineClusterIntegrationTest`?
Added one integration test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7568](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (f140c7b) into [master](https://codecov.io/gh/apache/pinot/commit/2107b2cd1880ec74934ec5412dbee8c874f05610?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (2107b2c) will **decrease** coverage by `16.28%`.
> The diff coverage is `2.07%`.
[](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7568 +/- ##
=============================================
- Coverage 30.79% 14.51% -16.29%
- Complexity 0 80 +80
=============================================
Files 1570 1534 -36
Lines 80024 79126 -898
Branches 11904 11843 -61
=============================================
- Hits 24644 11484 -13160
- Misses 53274 66798 +13524
+ Partials 2106 844 -1262
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| integration2 | `?` | |
| unittests2 | `14.51% <2.07%> (?)` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...java/org/apache/pinot/common/utils/DataSchema.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvRGF0YVNjaGVtYS5qYXZh) | `0.00% <0.00%> (-46.19%)` | :arrow_down: |
| [...org/apache/pinot/sql/parsers/CalciteSqlParser.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcWwvcGFyc2Vycy9DYWxjaXRlU3FsUGFyc2VyLmphdmE=) | `0.00% <0.00%> (-69.74%)` | :arrow_down: |
| [...in/java/org/apache/pinot/core/common/Operator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9jb21tb24vT3BlcmF0b3IuamF2YQ==) | `0.00% <ø> (ø)` | |
| [...operator/AcquireReleaseColumnsSegmentOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9BY3F1aXJlUmVsZWFzZUNvbHVtbnNTZWdtZW50T3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...a/org/apache/pinot/core/operator/BaseOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CYXNlT3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (-38.47%)` | :arrow_down: |
| [...he/pinot/core/operator/BitmapDocIdSetOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CaXRtYXBEb2NJZFNldE9wZXJhdG9yLmphdmE=) | `0.00% <0.00%> (-75.00%)` | :arrow_down: |
| [...g/apache/pinot/core/operator/DocIdSetOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9Eb2NJZFNldE9wZXJhdG9yLmphdmE=) | `0.00% <0.00%> (-92.31%)` | :arrow_down: |
| [.../pinot/core/operator/InstanceResponseOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9JbnN0YW5jZVJlc3BvbnNlT3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (-83.88%)` | :arrow_down: |
| [...apache/pinot/core/operator/ProjectionOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9Qcm9qZWN0aW9uT3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (-73.34%)` | :arrow_down: |
| [...erator/combine/AggregationOnlyCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0FnZ3JlZ2F0aW9uT25seUNvbWJpbmVPcGVyYXRvci5qYXZh) | `0.00% <0.00%> (-75.00%)` | :arrow_down: |
| ... and [900 more](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [2107b2c...f140c7b](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7568](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (c549639) into [master](https://codecov.io/gh/apache/pinot/commit/d42e415020d53afca55f04a3f89f988ffb9ca045?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (d42e415) will **decrease** coverage by `0.08%`.
> The diff coverage is `64.47%`.
[](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7568 +/- ##
============================================
- Coverage 71.60% 71.52% -0.09%
+ Complexity 3942 3939 -3
============================================
Files 1562 1563 +1
Lines 79465 79817 +352
Branches 11766 11826 +60
============================================
+ Hits 56903 57086 +183
- Misses 18724 18878 +154
- Partials 3838 3853 +15
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `29.00% <13.42%> (-0.41%)` | :arrow_down: |
| integration2 | `27.70% <12.89%> (-0.04%)` | :arrow_down: |
| unittests1 | `68.54% <63.68%> (-0.04%)` | :arrow_down: |
| unittests2 | `14.57% <2.10%> (-0.07%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...operator/AcquireReleaseColumnsSegmentOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9BY3F1aXJlUmVsZWFzZUNvbHVtbnNTZWdtZW50T3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...he/pinot/core/operator/BitmapDocIdSetOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CaXRtYXBEb2NJZFNldE9wZXJhdG9yLmphdmE=) | `71.42% <0.00%> (-3.58%)` | :arrow_down: |
| [.../core/operator/combine/GroupByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0dyb3VwQnlDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `78.12% <0.00%> (-1.25%)` | :arrow_down: |
| [...nMaxValueBasedSelectionOrderByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL01pbk1heFZhbHVlQmFzZWRTZWxlY3Rpb25PcmRlckJ5Q29tYmluZU9wZXJhdG9yLmphdmE=) | `74.01% <0.00%> (-0.59%)` | :arrow_down: |
| [...rator/combine/SelectionOrderByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL1NlbGVjdGlvbk9yZGVyQnlDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `50.00% <0.00%> (-1.73%)` | :arrow_down: |
| [...or/dociditerators/ExpressionScanDocIdIterator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9kb2NpZGl0ZXJhdG9ycy9FeHByZXNzaW9uU2NhbkRvY0lkSXRlcmF0b3IuamF2YQ==) | `65.95% <0.00%> (-0.48%)` | :arrow_down: |
| [...ore/operator/filter/BitmapBasedFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvQml0bWFwQmFzZWRGaWx0ZXJPcGVyYXRvci5qYXZh) | `78.26% <0.00%> (-14.05%)` | :arrow_down: |
| [...ot/core/operator/filter/H3IndexFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvSDNJbmRleEZpbHRlck9wZXJhdG9yLmphdmE=) | `82.10% <0.00%> (-4.57%)` | :arrow_down: |
| [...e/pinot/core/operator/filter/OrFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvT3JGaWx0ZXJPcGVyYXRvci5qYXZh) | `56.25% <0.00%> (-33.75%)` | :arrow_down: |
| [...operator/filter/RangeIndexBasedFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvUmFuZ2VJbmRleEJhc2VkRmlsdGVyT3BlcmF0b3IuamF2YQ==) | `36.53% <0.00%> (-3.89%)` | :arrow_down: |
| ... and [79 more](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [d42e415...c549639](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7568](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (c549639) into [master](https://codecov.io/gh/apache/pinot/commit/d42e415020d53afca55f04a3f89f988ffb9ca045?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (d42e415) will **decrease** coverage by `6.49%`.
> The diff coverage is `62.10%`.
[](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7568 +/- ##
============================================
- Coverage 71.60% 65.11% -6.50%
+ Complexity 3942 3939 -3
============================================
Files 1562 1517 -45
Lines 79465 77963 -1502
Branches 11766 11626 -140
============================================
- Hits 56903 50765 -6138
- Misses 18724 23598 +4874
+ Partials 3838 3600 -238
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| integration2 | `?` | |
| unittests1 | `68.54% <63.68%> (-0.04%)` | :arrow_down: |
| unittests2 | `14.57% <2.10%> (-0.07%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...operator/AcquireReleaseColumnsSegmentOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9BY3F1aXJlUmVsZWFzZUNvbHVtbnNTZWdtZW50T3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...he/pinot/core/operator/BitmapDocIdSetOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CaXRtYXBEb2NJZFNldE9wZXJhdG9yLmphdmE=) | `71.42% <0.00%> (-3.58%)` | :arrow_down: |
| [.../core/operator/combine/GroupByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0dyb3VwQnlDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `78.12% <0.00%> (-1.25%)` | :arrow_down: |
| [...nMaxValueBasedSelectionOrderByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL01pbk1heFZhbHVlQmFzZWRTZWxlY3Rpb25PcmRlckJ5Q29tYmluZU9wZXJhdG9yLmphdmE=) | `62.20% <0.00%> (-12.40%)` | :arrow_down: |
| [...rator/combine/SelectionOrderByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL1NlbGVjdGlvbk9yZGVyQnlDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `23.33% <0.00%> (-28.40%)` | :arrow_down: |
| [...or/dociditerators/ExpressionScanDocIdIterator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9kb2NpZGl0ZXJhdG9ycy9FeHByZXNzaW9uU2NhbkRvY0lkSXRlcmF0b3IuamF2YQ==) | `47.51% <0.00%> (-18.92%)` | :arrow_down: |
| [...ore/operator/filter/BitmapBasedFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvQml0bWFwQmFzZWRGaWx0ZXJPcGVyYXRvci5qYXZh) | `58.69% <0.00%> (-33.62%)` | :arrow_down: |
| [...ot/core/operator/filter/H3IndexFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvSDNJbmRleEZpbHRlck9wZXJhdG9yLmphdmE=) | `82.10% <0.00%> (-4.57%)` | :arrow_down: |
| [...e/pinot/core/operator/filter/OrFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvT3JGaWx0ZXJPcGVyYXRvci5qYXZh) | `56.25% <0.00%> (-33.75%)` | :arrow_down: |
| [...operator/filter/RangeIndexBasedFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvUmFuZ2VJbmRleEJhc2VkRmlsdGVyT3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (-40.43%)` | :arrow_down: |
| ... and [415 more](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [d42e415...c549639](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] jackjlli commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
jackjlli commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r747901291
##########
File path: pinot-common/src/main/java/org/apache/pinot/common/request/PinotQuery.java
##########
@@ -593,6 +604,28 @@ public void setQueryOptionsIsSet(boolean value) {
}
}
+ public boolean isExplain() {
+ return this.explain;
+ }
+
+ public void setExplain(boolean explain) {
+ this.explain = explain;
+ setExplainIsSet(true);
Review comment:
What's the purpose of this method?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] siddharthteotia commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
siddharthteotia commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r754730082
##########
File path: pinot-core/src/test/java/org/apache/pinot/queries/ExplainPlanQueriesTest.java
##########
@@ -0,0 +1,510 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.queries;
+
+import java.io.File;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.List;
+import org.apache.commons.io.FileUtils;
+import org.apache.pinot.common.response.broker.BrokerResponseNative;
+import org.apache.pinot.common.response.broker.ResultTable;
+import org.apache.pinot.common.utils.DataSchema;
+import org.apache.pinot.segment.local.indexsegment.immutable.ImmutableSegmentLoader;
+import org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl;
+import org.apache.pinot.segment.local.segment.index.loader.IndexLoadingConfig;
+import org.apache.pinot.segment.local.segment.readers.GenericRowRecordReader;
+import org.apache.pinot.segment.spi.ImmutableSegment;
+import org.apache.pinot.segment.spi.IndexSegment;
+import org.apache.pinot.segment.spi.creator.SegmentGeneratorConfig;
+import org.apache.pinot.spi.config.table.IndexingConfig;
+import org.apache.pinot.spi.config.table.TableConfig;
+import org.apache.pinot.spi.config.table.TableType;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.data.Schema;
+import org.apache.pinot.spi.data.readers.GenericRow;
+import org.apache.pinot.spi.utils.ReadMode;
+import org.apache.pinot.spi.utils.builder.TableConfigBuilder;
+import org.testng.Assert;
+import org.testng.annotations.BeforeClass;
+import org.testng.annotations.Test;
+
+
+public class ExplainPlanQueriesTest extends BaseQueriesTest {
+ private static final File INDEX_DIR = new File(FileUtils.getTempDirectory(), "ExplainPlanQueriesTest");
+ private static final String RAW_TABLE_NAME = "testTable";
+ private static final String SEGMENT_NAME = "testSegment";
+ private static final int NUM_RECORDS = 10;
+
+ private final static String COL1_NO_INDEX = "noIndexCol1";
+ private final static String COL2_NO_INDEX = "noIndexCol2";
+ private final static String COL3_NO_INDEX = "noIndexCol3";
+ private final static String COL1_INVERTED_INDEX = "invertedIndexCol1";
+ private final static String COL2_INVERTED_INDEX = "invertedIndexCol2";
+ private final static String COL3_INVERTED_INDEX = "invertedIndexCol3";
+ private final static String COL1_RANGE_INDEX = "rangeIndexCol1";
+ private final static String COL2_RANGE_INDEX = "rangeIndexCol2";
+ private final static String COL3_RANGE_INDEX = "rangeIndexCol3";
+ private final static String COL1_SORTED_INDEX = "sortedIndexCol1";
+ private final static String COL1_JSON_INDEX = "jsonIndexCol1";
+ private final static String COL1_TEXT_INDEX = "textIndexCol1";
+
+ private static final Schema SCHEMA = new Schema.SchemaBuilder().setSchemaName(RAW_TABLE_NAME)
+ .addSingleValueDimension(COL1_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL2_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_INVERTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_INVERTED_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_INVERTED_INDEX, FieldSpec.DataType.STRING)
+ .addSingleValueDimension(COL1_RANGE_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_SORTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL1_JSON_INDEX, FieldSpec.DataType.JSON)
+ .addSingleValueDimension(COL1_TEXT_INDEX, FieldSpec.DataType.STRING).build();
+
+ private static final DataSchema DATA_SCHEMA = new DataSchema(new String[]{"Operator", "Operator_Id", "Parent_Id"},
+ new DataSchema.ColumnDataType[]{DataSchema.ColumnDataType.STRING, DataSchema.ColumnDataType.INT,
+ DataSchema.ColumnDataType.INT});
+
+ private static final TableConfig TABLE_CONFIG =
+ new TableConfigBuilder(TableType.OFFLINE).setTableName(RAW_TABLE_NAME).build();
+
+ private IndexSegment _indexSegment;
+ private List<IndexSegment> _indexSegments;
+
+ @Override
+ protected String getFilter() {
+ return "";
+ }
+
+ @Override
+ protected IndexSegment getIndexSegment() {
+ return _indexSegment;
+ }
+
+ @Override
+ protected List<IndexSegment> getIndexSegments() {
+ return _indexSegments;
+ }
+
+ GenericRow createRecord(double doubleValue, int intValue, String stringValue, String jsonValue) {
+ GenericRow record = new GenericRow();
+ record.putValue(COL1_NO_INDEX, intValue);
+ record.putValue(COL2_NO_INDEX, intValue);
+ record.putValue(COL3_NO_INDEX, intValue);
+
+ record.putValue(COL1_INVERTED_INDEX, doubleValue);
+ record.putValue(COL2_INVERTED_INDEX, intValue);
+ record.putValue(COL3_INVERTED_INDEX, intValue);
+
+ record.putValue(COL1_RANGE_INDEX, doubleValue);
+ record.putValue(COL2_RANGE_INDEX, intValue);
+ record.putValue(COL3_RANGE_INDEX, intValue);
+
+ record.putValue(COL1_JSON_INDEX, jsonValue);
+ record.putValue(COL1_TEXT_INDEX, stringValue);
+
+ return record;
+ }
+
+ @BeforeClass
+ public void setUp()
+ throws Exception {
+ FileUtils.deleteDirectory(INDEX_DIR);
+
+ List<GenericRow> records = new ArrayList<>(NUM_RECORDS);
+ records.add(createRecord(1.1, 2, "daffy", "{\"first\": \"daffy\", \"last\": \"duck\"}"));
+
+ IndexingConfig indexingConfig = TABLE_CONFIG.getIndexingConfig();
+
+ List<String> invertedIndexColumns = Arrays.asList(COL1_INVERTED_INDEX, COL2_INVERTED_INDEX, COL3_INVERTED_INDEX);
+ indexingConfig.setInvertedIndexColumns(invertedIndexColumns);
+
+ List<String> rangeIndexColumns = Arrays.asList(COL1_RANGE_INDEX, COL2_RANGE_INDEX, COL3_RANGE_INDEX);
+ indexingConfig.setRangeIndexColumns(rangeIndexColumns);
+
+ List<String> jsonIndexColumns = Arrays.asList(COL1_JSON_INDEX);
+ indexingConfig.setJsonIndexColumns(jsonIndexColumns);
+
+ SegmentGeneratorConfig segmentGeneratorConfig = new SegmentGeneratorConfig(TABLE_CONFIG, SCHEMA);
+ segmentGeneratorConfig.setTableName(RAW_TABLE_NAME);
+ segmentGeneratorConfig.setSegmentName(SEGMENT_NAME);
+ segmentGeneratorConfig.setOutDir(INDEX_DIR.getPath());
+
+ SegmentIndexCreationDriverImpl driver = new SegmentIndexCreationDriverImpl();
+ driver.init(segmentGeneratorConfig, new GenericRowRecordReader(records));
+ driver.build();
+
+ IndexLoadingConfig indexLoadingConfig = new IndexLoadingConfig();
+ indexLoadingConfig.setTableConfig(TABLE_CONFIG);
+ indexLoadingConfig.setInvertedIndexColumns(new HashSet<>(invertedIndexColumns));
+ indexLoadingConfig.setRangeIndexColumns(new HashSet<>(rangeIndexColumns));
+ indexLoadingConfig.setJsonIndexColumns(new HashSet<String>(jsonIndexColumns));
+ indexLoadingConfig.setReadMode(ReadMode.mmap);
+
+ ImmutableSegment immutableSegment =
+ ImmutableSegmentLoader.load(new File(INDEX_DIR, SEGMENT_NAME), indexLoadingConfig);
+ _indexSegment = immutableSegment;
+ _indexSegments = Arrays.asList(immutableSegment);
+ }
+
+ /** Checks the correctness of EXPLAIN PLAN output. */
+ private void check(String query, ResultTable expected) {
+ BrokerResponseNative response = getBrokerResponseForSqlQuery(query);
+ ResultTable actual = response.getResultTable();
+
+ System.out.println(resultTableToString("Expected", expected));
+ System.out.println(resultTableToString("Actual", actual));
+
+ Assert.assertEquals(actual.getDataSchema(), expected.getDataSchema());
+ Assert.assertEquals(actual.getRows().size(), expected.getRows().size());
+
+ List<Object[]> expectedRows = expected.getRows();
+ List<Object[]> actualRows = actual.getRows();
+ for (int i = 0; i < expectedRows.size(); i++) {
+ Object[] expectedRow = expectedRows.get(i);
+ Object[] actualRow = actualRows.get(i);
+ Assert.assertEquals(actualRow.length, expectedRow.length);
+
+ for (int j = 0; j < actualRow.length; j++) {
+ Assert.assertEquals(actualRow[j], expectedRow[j]);
+ }
+ }
+ }
+
+ /** This function is used to generated expected results while creating test cases; otherwise, not used. */
+ private static String resultTableToString(String tag, ResultTable table) {
+ System.out.println("\n" + tag + ":");
+ List<Object[]> rows = table.getRows();
+
+ StringBuffer buffer = new StringBuffer();
+ for (Object[] row : rows) {
+ buffer.append("result.add(new Object[]{\"" + row[0] + "\", " + row[1] + ", " + row[2] + "});\n");
+ }
+
+ return buffer.toString();
+ }
+
+ @Test
+ public void testSelect() {
+ String query1 = "EXPLAIN PLAN FOR SELECT * FROM testTable";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(
+ new Object[]{"SELECT(selectList:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, jsonIndexCol1, "
+ + "noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, "
+ + "jsonIndexCol1, noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 3, 2});
+ result1.add(
+ new Object[]{"PROJECT(sortedIndexCol1, noIndexCol3, rangeIndexCol1, rangeIndexCol2, jsonIndexCol1, "
+ + "invertedIndexCol1, noIndexCol2, invertedIndexCol2, noIndexCol1, invertedIndexCol3, rangeIndexCol3, "
+ + "textIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 = "EXPLAIN PLAN FOR SELECT 'mickey' FROM testTable";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:'mickey')", 2, 1});
+ result2.add(new Object[]{"PROJECT(ALL)", 3, 2});
+ result2.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 4, 3});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+
+ String query3 = "EXPLAIN PLAN FOR SELECT invertedIndexCol1, noIndexCol1 FROM testTable LIMIT 100";
+ List<Object[]> result3 = new ArrayList<>();
+ result3.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result3.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result3.add(new Object[]{"SELECT(selectList:invertedIndexCol1, noIndexCol1)", 2, 1});
+ result3.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result3.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result3.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query3, new ResultTable(DATA_SCHEMA, result3));
+
+ String query4 = "EXPLAIN PLAN FOR SELECT DISTINCT invertedIndexCol1, noIndexCol1 FROM testTable LIMIT 100";
+ List<Object[]> result4 = new ArrayList<>();
+ result4.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result4.add(new Object[]{"COMBINE_DISTINCT", 1, 0});
+ result4.add(new Object[]{"DISTINCT(keyColumns:invertedIndexCol1, noIndexCol1)", 2, 1});
+ result4.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result4.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result4.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query4, new ResultTable(DATA_SCHEMA, result4));
+ }
+
+ @Test
+ public void testSelectColumnsUsingFilterIndex() {
+ String query1 = "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1, sortedIndexCol1 FROM testTable WHERE "
+ + "invertedIndexCol1 = 1.5 AND sortedIndexCol1 = 1 LIMIT 100";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1, sortedIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, sortedIndexCol1)", 3, 2});
+ result1.add(new Object[]{"PROJECT(sortedIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 = "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1, sortedIndexCol1 FROM testTable WHERE "
+ + "(invertedIndexCol1 = 1.5 AND sortedIndexCol1 = 1) OR (invertedIndexCol1 BETWEEN 100 and 200 AND "
+ + "rangeIndexCol1 > 20)";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1, sortedIndexCol1)", 2, 1});
+ result2.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, sortedIndexCol1)", 3, 2});
+ result2.add(new Object[]{"PROJECT(sortedIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result2.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+
+ String query3 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1 FROM testTable WHERE invertedIndexCol1 = 1.5 AND "
+ + "invertedIndexCol2 IN (10, 20, 30) AND invertedIndexCol3 NOT IN ('foo', 'bar') LIMIT 100";
+ List<Object[]> result3 = new ArrayList<>();
+ result3.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result3.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result3.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1)", 2, 1});
+ result3.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result3.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result3.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query3, new ResultTable(DATA_SCHEMA, result3));
+
+ String query4 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, noIndexCol2, sortedIndexCol1 FROM testTable WHERE sortedIndexCol1 = 1.5"
+ + " OR sortedIndexCol1 != 5 OR sortedIndexCol1 IN (10, 20, 30) LIMIT 100";
+ List<Object[]> result4 = new ArrayList<>();
+ result4.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result4.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result4.add(new Object[]{"SELECT(selectList:noIndexCol1, noIndexCol2, sortedIndexCol1)", 2, 1});
+ result4.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2, sortedIndexCol1)", 3, 2});
+ result4.add(new Object[]{"PROJECT(sortedIndexCol1, noIndexCol2, noIndexCol1)", 4, 3});
+ result4.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query4, new ResultTable(DATA_SCHEMA, result4));
+
+ String query5 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, noIndexCol2 FROM testTable WHERE DIV(noIndexCol1, noIndexCol2) BETWEEN "
+ + "10 AND 20 AND invertedIndexCol1 * 5 < 1000";
+ List<Object[]> result5 = new ArrayList<>();
+ result5.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result5.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result5.add(new Object[]{"SELECT(selectList:noIndexCol1, noIndexCol2)", 2, 1});
+ result5.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2)", 3, 2});
+ result5.add(new Object[]{"PROJECT(noIndexCol2, noIndexCol1)", 4, 3});
+ result5.add(new Object[]{"FILTER_AND", 5, 4});
+ result5.add(
+ new Object[]{"FILTER_EXPRESSION(operator:RANGE,predicate:div(noIndexCol1,noIndexCol2) BETWEEN '10' AND '20')",
+ 6, 5});
+ result5
+ .add(new Object[]{"FILTER_EXPRESSION(operator:RANGE,predicate:times(invertedIndexCol1,'5') < '1000')", 7, 5});
+ check(query5, new ResultTable(DATA_SCHEMA, result5));
+ }
+
+ @Test
+ public void testSelectColumnUsingFilterRange() {
+ // select * query triggering range index
+ // checks using RANGE (>, >=, <, <=, BETWEEN ..) on a column with range index should use RANGE_INDEX_SCAN
+ String query1 =
+ "EXPLAIN PLAN FOR SELECT invertedIndexCol1, noIndexCol1, rangeIndexCol1 FROM testTable WHERE rangeIndexCol1 >"
+ + " 10 AND rangeIndexCol2 >= 20 AND rangeIndexCol3 BETWEEN 50 AND 60 LIMIT 100";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(new Object[]{"SELECT(selectList:invertedIndexCol1, noIndexCol1, rangeIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, rangeIndexCol1)", 3, 2});
+ result1.add(new Object[]{"PROJECT(rangeIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1 FROM testTable WHERE noIndexCol1 > 10 AND noIndexCol2"
+ + " BETWEEN 50 AND 60 LIMIT 100";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1)", 2, 1});
+ result2.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result2.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result2.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+ }
+
+ @Test
+ public void testSelectColumnUsingFilterJson() {
+ String query =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1 FROM testTable WHERE (invertedIndexCol1 IN (10, 20, "
+ + "30) AND sortedIndexCol1 != 100) OR (noIndexCol1 NOT IN (10, 20, 30) AND rangeIndexCol1 != 20 AND "
+ + "JSON_MATCH(jsonIndexCol1, 'key=1') AND TEXT_MATCH(textIndexCol1, 'foo'))";
+ List<Object[]> result = new ArrayList<>();
+ result.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1)", 2, 1});
+ result.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result.add(new Object[]{"FILTER_AND", 5, 4});
+ result.add(new Object[]{"FILTER_JSON_INDEX(indexLookUp:json_index,operator:JSON_MATCH,predicate:json_match"
+ + "(jsonIndexCol1,'key=1'))", 6, 5});
+ result.add(new Object[]{"FILTER_TEXT_INDEX(indexLookUp:text_index,operator:TEXT_MATCH,predicate:text_match"
+ + "(textIndexCol1,'foo'))", 7, 5});
+ check(query, new ResultTable(DATA_SCHEMA, result));
+ }
+
+ @Test
+ public void testSelectAggregateUsingFilterText() {
+ String query1 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, noIndexCol2, max(noIndexCol2), min(noIndexCol3) FROM testTable WHERE "
+ + "TEXT_MATCH(textIndexCol1, 'foo') GROUP BY noIndexCol1, noIndexCol2";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result1.add(new Object[]{"COMBINE_GROUPBY_ORDERBY", 1, 0});
+ result1.add(new Object[]{
+ "AGGREGATE_GROUPBY_ORDERBY(groupKeys:noIndexCol1, noIndexCol2, aggregations:max(noIndexCol2), min"
+ + "(noIndexCol3))", 2, 1});
+ result1.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2, noIndexCol3)", 3, 2});
+ result1.add(new Object[]{"PROJECT(noIndexCol3, noIndexCol2, noIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_TEXT_INDEX(indexLookUp:text_index,operator:TEXT_MATCH,predicate:text_match"
+ + "(textIndexCol1,'foo'))", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, max(noIndexCol2), min(noIndexCol3) FROM testTable WHERE TEXT_MATCH"
+ + "(textIndexCol1, 'foo') GROUP BY noIndexCol1, noIndexCol2 ORDER BY noIndexCol1, max(noIndexCol2)";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(sort:[noIndexCol1 ASC, max(noIndexCol2) ASC],limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_GROUPBY_ORDERBY", 1, 0});
+ result2.add(new Object[]{
+ "AGGREGATE_GROUPBY_ORDERBY(groupKeys:noIndexCol1,noIndexCol2,aggregations:max(noIndexCol2), min"
+ + "(noIndexCol3))", 2, 1});
+ result2.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, invertedIndexCol1)", 3, 2});
+ result2.add(new Object[]{"PROJECT(noIndexCol3, noIndexCol2, noIndexCol1)", 4, 3});
+ result2.add(new Object[]{"FILTER_TEXT_INDEX(indexLookUp:text_index,operator:TEXT_MATCH,predicate:text_match"
+ + "(textIndexCol1,'foo'))", 5, 4});
+ }
+
+ @Test
+ public void testSelectAggregate() {
+ String query1 = "EXPLAIN PLAN FOR SELECT count(*) FROM testTable";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result1.add(new Object[]{"COMBINE_AGGREGATE", 1, 0});
+ result1.add(new Object[]{"AGGREGATE_METADATA", 2, 1});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 =
+ "EXPLAIN PLAN FOR SELECT count(*), max(noIndexCol1), sum(noIndexCol2), avg(noIndexCol2) FROM testTable";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_AGGREGATE", 1, 0});
+ result2.add(
+ new Object[]{"AGGREGATE(aggregations:count(*), max(noIndexCol1), sum(noIndexCol2), avg(noIndexCol2))", 2, 1});
+ result2.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2)", 3, 2});
+ result2.add(new Object[]{"PROJECT(noIndexCol2, noIndexCol1)", 4, 3});
+ result2.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+ }
+
+ @Test
+ public void testSelectAggregateUsingFilter() {
+ String query = "EXPLAIN PLAN FOR SELECT sum(add(noIndexCol1, noIndexCol2)), MIN(ADD(DIV(noIndexCol1,noIndexCol2),"
+ + "noIndexCol3)) FROM testTable";
+ List<Object[]> result = new ArrayList<>();
+ result.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result.add(new Object[]{"COMBINE_AGGREGATE", 1, 0});
+ result.add(new Object[]{
+ "AGGREGATE(aggregations:sum(add(noIndexCol1,noIndexCol2)), min(add(div(noIndexCol1,noIndexCol2),"
+ + "noIndexCol3)))", 2, 1});
+ result.add(new Object[]{"PROJECT(noIndexCol3, noIndexCol2, noIndexCol1)", 3, 2});
+ result.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 4, 3});
+ check(query, new ResultTable(DATA_SCHEMA, result));
+ }
+
+ @Test
+ public void testSelectAggregateUsingFilterGroupBy() {
+ String query =
+ "EXPLAIN PLAN FOR SELECT noIndexCol2, sum(add(noIndexCol1, noIndexCol2)), min(noIndexCol3) FROM testTable "
+ + "WHERE noIndexCol1 < 100 GROUP BY noIndexCol2";
+ List<Object[]> result = new ArrayList<>();
+ result.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result.add(new Object[]{"COMBINE_GROUPBY_ORDERBY", 1, 0});
+ result.add(new Object[]{
+ "AGGREGATE_GROUPBY_ORDERBY(groupKeys:noIndexCol2, aggregations:sum(add(noIndexCol1,noIndexCol2)),"
+ + " min(noIndexCol3))", 2, 1});
+ result.add(new Object[]{"PROJECT(noIndexCol3, noIndexCol2, noIndexCol1)", 3, 2});
+ result.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 4, 3});
+ check(query, new ResultTable(DATA_SCHEMA, result));
+ }
+
+ @Test
+ public void testSelectAggregateUsingFilterIndex() {
+ String query =
+ "EXPLAIN PLAN FOR SELECT count(*), max(noIndexCol1), sum(noIndexCol2), avg(noIndexCol3) FROM testTable WHERE "
+ + "invertedIndexCol1 = 1 AND col1 = 20";
+ List<Object[]> result = new ArrayList<>();
+ result.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result.add(new Object[]{"COMBINE_AGGREGATE", 1, 0});
+ result.add(
+ new Object[]{"AGGREGATE(aggregations:count(*), max(noIndexCol1), sum(noIndexCol2), avg(noIndexCol3))", 2, 1});
+ result.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2, noIndexCol3)", 3, 2});
+ result.add(new Object[]{"PROJECT(noIndexCol3, noIndexCol2, noIndexCol1)", 4, 3});
+ result.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query, new ResultTable(DATA_SCHEMA, result));
+ }
+
+ @Test
+ public void testSelectAggregateUsingFilterIndexGroupBy() {
+ String query = "EXPLAIN PLAN FOR SELECT noIndexCol1, max(noIndexCol2), min(noIndexCol3) FROM testTable WHERE "
+ + "invertedIndexCol2 = 10 GROUP BY noIndexCol1";
+ List<Object[]> result = new ArrayList<>();
+ result.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result.add(new Object[]{"COMBINE_GROUPBY_ORDERBY", 1, 0});
+ result.add(new Object[]{
+ "AGGREGATE_GROUPBY_ORDERBY(groupKeys:noIndexCol1, aggregations:max(noIndexCol2), min(noIndexCol3)" + ")", 2,
+ 1});
+ result.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2, noIndexCol3)", 3, 2});
+ result.add(new Object[]{"PROJECT(noIndexCol3, noIndexCol2, noIndexCol1)", 4, 3});
+ result.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query, new ResultTable(DATA_SCHEMA, result));
+ }
+
+ @Test
+ public void testSelectAggregateUsingFilterIndexGroupByHaving() {
Review comment:
Thanks for adding remaining tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7568](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (8e824b3) into [master](https://codecov.io/gh/apache/pinot/commit/d1727372a5bfd48b6f3c1fc0c8a22d39c243c1bc?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (d172737) will **decrease** coverage by `42.24%`.
> The diff coverage is `43.56%`.
> :exclamation: Current head 8e824b3 differs from pull request most recent head 88f44eb. Consider uploading reports for the commit 88f44eb to get more accurate results
[](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7568 +/- ##
=============================================
- Coverage 71.64% 29.39% -42.25%
=============================================
Files 1579 1571 -8
Lines 80819 80807 -12
Branches 12017 12039 +22
=============================================
- Hits 57902 23756 -34146
- Misses 19017 54935 +35918
+ Partials 3900 2116 -1784
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `29.39% <43.56%> (-0.17%)` | :arrow_down: |
| integration2 | `?` | |
| unittests1 | `?` | |
| unittests2 | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...in/java/org/apache/pinot/core/common/Operator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9jb21tb24vT3BlcmF0b3IuamF2YQ==) | `0.00% <ø> (ø)` | |
| [...operator/AcquireReleaseColumnsSegmentOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9BY3F1aXJlUmVsZWFzZUNvbHVtbnNTZWdtZW50T3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...he/pinot/core/operator/BitmapDocIdSetOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CaXRtYXBEb2NJZFNldE9wZXJhdG9yLmphdmE=) | `68.18% <0.00%> (-6.82%)` | :arrow_down: |
| [...erator/combine/AggregationOnlyCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0FnZ3JlZ2F0aW9uT25seUNvbWJpbmVPcGVyYXRvci5qYXZh) | `69.23% <0.00%> (-5.77%)` | :arrow_down: |
| [...core/operator/combine/DistinctCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0Rpc3RpbmN0Q29tYmluZU9wZXJhdG9yLmphdmE=) | `77.77% <0.00%> (-3.00%)` | :arrow_down: |
| [.../core/operator/combine/GroupByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0dyb3VwQnlDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `71.01% <0.00%> (-3.99%)` | :arrow_down: |
| [...nMaxValueBasedSelectionOrderByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL01pbk1heFZhbHVlQmFzZWRTZWxlY3Rpb25PcmRlckJ5Q29tYmluZU9wZXJhdG9yLmphdmE=) | `70.45% <0.00%> (-2.07%)` | :arrow_down: |
| [...operator/combine/SelectionOnlyCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL1NlbGVjdGlvbk9ubHlDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `56.66% <0.00%> (-1.96%)` | :arrow_down: |
| [...rator/combine/SelectionOrderByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL1NlbGVjdGlvbk9yZGVyQnlDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `50.00% <0.00%> (-1.73%)` | :arrow_down: |
| [...or/dociditerators/ExpressionScanDocIdIterator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9kb2NpZGl0ZXJhdG9ycy9FeHByZXNzaW9uU2NhbkRvY0lkSXRlcmF0b3IuamF2YQ==) | `61.97% <0.00%> (-4.46%)` | :arrow_down: |
| ... and [1157 more](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [d172737...88f44eb](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7568](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (e8f8685) into [master](https://codecov.io/gh/apache/pinot/commit/767aa8abfb5bf085ba0a7ae5ff4024679f27816e?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (767aa8a) will **decrease** coverage by `57.15%`.
> The diff coverage is `1.81%`.
[](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7568 +/- ##
=============================================
- Coverage 71.69% 14.54% -57.16%
+ Complexity 4086 80 -4006
=============================================
Files 1579 1535 -44
Lines 80790 79339 -1451
Branches 12003 11874 -129
=============================================
- Hits 57922 11536 -46386
- Misses 18981 66950 +47969
+ Partials 3887 853 -3034
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| integration2 | `?` | |
| unittests1 | `?` | |
| unittests2 | `14.54% <1.81%> (-0.07%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...java/org/apache/pinot/common/utils/DataSchema.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvRGF0YVNjaGVtYS5qYXZh) | `0.00% <0.00%> (-79.24%)` | :arrow_down: |
| [...org/apache/pinot/sql/parsers/CalciteSqlParser.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcWwvcGFyc2Vycy9DYWxjaXRlU3FsUGFyc2VyLmphdmE=) | `0.00% <0.00%> (-91.89%)` | :arrow_down: |
| [...in/java/org/apache/pinot/core/common/Operator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9jb21tb24vT3BlcmF0b3IuamF2YQ==) | `0.00% <ø> (ø)` | |
| [...operator/AcquireReleaseColumnsSegmentOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9BY3F1aXJlUmVsZWFzZUNvbHVtbnNTZWdtZW50T3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...a/org/apache/pinot/core/operator/BaseOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CYXNlT3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (-38.47%)` | :arrow_down: |
| [...he/pinot/core/operator/BitmapDocIdSetOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CaXRtYXBEb2NJZFNldE9wZXJhdG9yLmphdmE=) | `0.00% <0.00%> (-75.00%)` | :arrow_down: |
| [...g/apache/pinot/core/operator/DocIdSetOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9Eb2NJZFNldE9wZXJhdG9yLmphdmE=) | `0.00% <0.00%> (-92.31%)` | :arrow_down: |
| [.../pinot/core/operator/InstanceResponseOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9JbnN0YW5jZVJlc3BvbnNlT3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (-86.85%)` | :arrow_down: |
| [...apache/pinot/core/operator/ProjectionOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9Qcm9qZWN0aW9uT3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (-73.34%)` | :arrow_down: |
| [...erator/combine/AggregationOnlyCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0FnZ3JlZ2F0aW9uT25seUNvbWJpbmVPcGVyYXRvci5qYXZh) | `0.00% <0.00%> (-75.00%)` | :arrow_down: |
| ... and [1316 more](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [767aa8a...e8f8685](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] siddharthteotia commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
siddharthteotia commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r745770403
##########
File path: pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseBrokerRequestHandler.java
##########
@@ -225,7 +225,16 @@ private BrokerResponseNative handleSQLRequest(long requestId, String query, Json
if (isLiteralOnlyQuery(pinotQuery)) {
LOGGER.debug("Request {} contains only Literal, skipping server query: {}", requestId, query);
try {
- return processLiteralOnlyQuery(pinotQuery, compilationStartTimeNs, requestStatistics);
+ BrokerResponseNative responseForLiteralOnly =
+ processLiteralOnlyQuery(pinotQuery, compilationStartTimeNs, requestStatistics);
+ if (pinotQuery.isExplain()) {
+ // Generate explain results to show that this is a SELECT on a literal value.
+ List<Object[]> rows = new ArrayList<>();
+ rows.add(new Object[]{"SELECT(selectList:literal)", 0, -1});
Review comment:
Should we not use the actual literal ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] amrishlal commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
amrishlal commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r746144491
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/ProjectionOperator.java
##########
@@ -68,6 +72,36 @@ public String getOperatorName() {
return OPERATOR_NAME;
}
+ @Override
+ public String getExplainPlanName() {
+ return EXPLAIN_NAME;
+ }
+
+ @Override
+ public List<Operator> getChildOperators() {
+ return Collections.singletonList(_docIdSetOperator);
+ }
+
+ @Override
+ public String toExplainString() {
+ StringBuilder stringBuilder = new StringBuilder(getExplainPlanName()).append('(');
+ if (_dataSourceMap.keySet().isEmpty()) {
+ // count aggregation function has empty input expressions
+ stringBuilder.append("ALL");
Review comment:
The comment appears to be incorrect. I don't see this block of code being invoked when `count(*)` is used. There is a test case for `count(*)` query.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] richardstartin commented on a change in pull request #7568: [WORK IN PROGRESS] EXPLAIN PLAN code
Posted by GitBox <gi...@apache.org>.
richardstartin commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r727438086
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/AcquireReleaseColumnsSegmentOperator.java
##########
@@ -77,6 +81,16 @@ public String getOperatorName() {
return OPERATOR_NAME;
}
+ @Override
+ public String getExplainPlanName() {
+ return EXPLAIN_NAME;
+ }
+
+ @Override
+ public List<Operator> getChildOperators() {
+ return Arrays.asList(_childOperator);
Review comment:
Can you replace all of these one element `Arrays.asList(arg)` with `Collections.singletonList(arg)` - just because it creates IDE warnings otherwise, and it's nice to keep SNR high with IDE warnings.
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/AcquireReleaseColumnsSegmentOperator.java
##########
@@ -77,6 +81,16 @@ public String getOperatorName() {
return OPERATOR_NAME;
}
+ @Override
+ public String getExplainPlanName() {
+ return EXPLAIN_NAME;
+ }
+
+ @Override
+ public List<Operator> getChildOperators() {
+ return Arrays.asList(_childOperator);
Review comment:
Of course, it wasn't a really a _review_ comment, just something that stood out at me glancing at the diff.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] amrishlal commented on pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
amrishlal commented on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-963714474
@mayankshriv @Jackie-Jiang @kishoreg Please review. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7568](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (c549639) into [master](https://codecov.io/gh/apache/pinot/commit/d42e415020d53afca55f04a3f89f988ffb9ca045?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (d42e415) will **decrease** coverage by `1.05%`.
> The diff coverage is `63.94%`.
[](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7568 +/- ##
============================================
- Coverage 71.60% 70.55% -1.06%
+ Complexity 3942 3939 -3
============================================
Files 1562 1563 +1
Lines 79465 79817 +352
Branches 11766 11826 +60
============================================
- Hits 56903 56314 -589
- Misses 18724 19660 +936
- Partials 3838 3843 +5
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `29.00% <13.42%> (-0.41%)` | :arrow_down: |
| integration2 | `?` | |
| unittests1 | `68.54% <63.68%> (-0.04%)` | :arrow_down: |
| unittests2 | `14.57% <2.10%> (-0.07%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...operator/AcquireReleaseColumnsSegmentOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9BY3F1aXJlUmVsZWFzZUNvbHVtbnNTZWdtZW50T3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...he/pinot/core/operator/BitmapDocIdSetOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CaXRtYXBEb2NJZFNldE9wZXJhdG9yLmphdmE=) | `71.42% <0.00%> (-3.58%)` | :arrow_down: |
| [.../core/operator/combine/GroupByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0dyb3VwQnlDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `78.12% <0.00%> (-1.25%)` | :arrow_down: |
| [...nMaxValueBasedSelectionOrderByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL01pbk1heFZhbHVlQmFzZWRTZWxlY3Rpb25PcmRlckJ5Q29tYmluZU9wZXJhdG9yLmphdmE=) | `72.44% <0.00%> (-2.17%)` | :arrow_down: |
| [...rator/combine/SelectionOrderByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL1NlbGVjdGlvbk9yZGVyQnlDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `50.00% <0.00%> (-1.73%)` | :arrow_down: |
| [...or/dociditerators/ExpressionScanDocIdIterator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9kb2NpZGl0ZXJhdG9ycy9FeHByZXNzaW9uU2NhbkRvY0lkSXRlcmF0b3IuamF2YQ==) | `65.95% <0.00%> (-0.48%)` | :arrow_down: |
| [...ore/operator/filter/BitmapBasedFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvQml0bWFwQmFzZWRGaWx0ZXJPcGVyYXRvci5qYXZh) | `78.26% <0.00%> (-14.05%)` | :arrow_down: |
| [...ot/core/operator/filter/H3IndexFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvSDNJbmRleEZpbHRlck9wZXJhdG9yLmphdmE=) | `82.10% <0.00%> (-4.57%)` | :arrow_down: |
| [...e/pinot/core/operator/filter/OrFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvT3JGaWx0ZXJPcGVyYXRvci5qYXZh) | `56.25% <0.00%> (-33.75%)` | :arrow_down: |
| [...operator/filter/RangeIndexBasedFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvUmFuZ2VJbmRleEJhc2VkRmlsdGVyT3BlcmF0b3IuamF2YQ==) | `36.53% <0.00%> (-3.89%)` | :arrow_down: |
| ... and [140 more](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [d42e415...c549639](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] siddharthteotia commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
siddharthteotia commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r745817088
##########
File path: pinot-core/src/test/java/org/apache/pinot/queries/ExplainPlanQueriesTest.java
##########
@@ -0,0 +1,510 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.queries;
+
+import java.io.File;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.List;
+import org.apache.commons.io.FileUtils;
+import org.apache.pinot.common.response.broker.BrokerResponseNative;
+import org.apache.pinot.common.response.broker.ResultTable;
+import org.apache.pinot.common.utils.DataSchema;
+import org.apache.pinot.segment.local.indexsegment.immutable.ImmutableSegmentLoader;
+import org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl;
+import org.apache.pinot.segment.local.segment.index.loader.IndexLoadingConfig;
+import org.apache.pinot.segment.local.segment.readers.GenericRowRecordReader;
+import org.apache.pinot.segment.spi.ImmutableSegment;
+import org.apache.pinot.segment.spi.IndexSegment;
+import org.apache.pinot.segment.spi.creator.SegmentGeneratorConfig;
+import org.apache.pinot.spi.config.table.IndexingConfig;
+import org.apache.pinot.spi.config.table.TableConfig;
+import org.apache.pinot.spi.config.table.TableType;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.data.Schema;
+import org.apache.pinot.spi.data.readers.GenericRow;
+import org.apache.pinot.spi.utils.ReadMode;
+import org.apache.pinot.spi.utils.builder.TableConfigBuilder;
+import org.testng.Assert;
+import org.testng.annotations.BeforeClass;
+import org.testng.annotations.Test;
+
+
+public class ExplainPlanQueriesTest extends BaseQueriesTest {
+ private static final File INDEX_DIR = new File(FileUtils.getTempDirectory(), "ExplainPlanQueriesTest");
+ private static final String RAW_TABLE_NAME = "testTable";
+ private static final String SEGMENT_NAME = "testSegment";
+ private static final int NUM_RECORDS = 10;
+
+ private final static String COL1_NO_INDEX = "noIndexCol1";
+ private final static String COL2_NO_INDEX = "noIndexCol2";
+ private final static String COL3_NO_INDEX = "noIndexCol3";
+ private final static String COL1_INVERTED_INDEX = "invertedIndexCol1";
+ private final static String COL2_INVERTED_INDEX = "invertedIndexCol2";
+ private final static String COL3_INVERTED_INDEX = "invertedIndexCol3";
+ private final static String COL1_RANGE_INDEX = "rangeIndexCol1";
+ private final static String COL2_RANGE_INDEX = "rangeIndexCol2";
+ private final static String COL3_RANGE_INDEX = "rangeIndexCol3";
+ private final static String COL1_SORTED_INDEX = "sortedIndexCol1";
+ private final static String COL1_JSON_INDEX = "jsonIndexCol1";
+ private final static String COL1_TEXT_INDEX = "textIndexCol1";
+
+ private static final Schema SCHEMA = new Schema.SchemaBuilder().setSchemaName(RAW_TABLE_NAME)
+ .addSingleValueDimension(COL1_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL2_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_INVERTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_INVERTED_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_INVERTED_INDEX, FieldSpec.DataType.STRING)
+ .addSingleValueDimension(COL1_RANGE_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_SORTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL1_JSON_INDEX, FieldSpec.DataType.JSON)
+ .addSingleValueDimension(COL1_TEXT_INDEX, FieldSpec.DataType.STRING).build();
+
+ private static final DataSchema DATA_SCHEMA = new DataSchema(new String[]{"Operator", "Operator_Id", "Parent_Id"},
+ new DataSchema.ColumnDataType[]{DataSchema.ColumnDataType.STRING, DataSchema.ColumnDataType.INT,
+ DataSchema.ColumnDataType.INT});
+
+ private static final TableConfig TABLE_CONFIG =
+ new TableConfigBuilder(TableType.OFFLINE).setTableName(RAW_TABLE_NAME).build();
+
+ private IndexSegment _indexSegment;
+ private List<IndexSegment> _indexSegments;
+
+ @Override
+ protected String getFilter() {
+ return "";
+ }
+
+ @Override
+ protected IndexSegment getIndexSegment() {
+ return _indexSegment;
+ }
+
+ @Override
+ protected List<IndexSegment> getIndexSegments() {
+ return _indexSegments;
+ }
+
+ GenericRow createRecord(double doubleValue, int intValue, String stringValue, String jsonValue) {
+ GenericRow record = new GenericRow();
+ record.putValue(COL1_NO_INDEX, intValue);
+ record.putValue(COL2_NO_INDEX, intValue);
+ record.putValue(COL3_NO_INDEX, intValue);
+
+ record.putValue(COL1_INVERTED_INDEX, doubleValue);
+ record.putValue(COL2_INVERTED_INDEX, intValue);
+ record.putValue(COL3_INVERTED_INDEX, intValue);
+
+ record.putValue(COL1_RANGE_INDEX, doubleValue);
+ record.putValue(COL2_RANGE_INDEX, intValue);
+ record.putValue(COL3_RANGE_INDEX, intValue);
+
+ record.putValue(COL1_JSON_INDEX, jsonValue);
+ record.putValue(COL1_TEXT_INDEX, stringValue);
+
+ return record;
+ }
+
+ @BeforeClass
+ public void setUp()
+ throws Exception {
+ FileUtils.deleteDirectory(INDEX_DIR);
+
+ List<GenericRow> records = new ArrayList<>(NUM_RECORDS);
+ records.add(createRecord(1.1, 2, "daffy", "{\"first\": \"daffy\", \"last\": \"duck\"}"));
+
+ IndexingConfig indexingConfig = TABLE_CONFIG.getIndexingConfig();
+
+ List<String> invertedIndexColumns = Arrays.asList(COL1_INVERTED_INDEX, COL2_INVERTED_INDEX, COL3_INVERTED_INDEX);
+ indexingConfig.setInvertedIndexColumns(invertedIndexColumns);
+
+ List<String> rangeIndexColumns = Arrays.asList(COL1_RANGE_INDEX, COL2_RANGE_INDEX, COL3_RANGE_INDEX);
+ indexingConfig.setRangeIndexColumns(rangeIndexColumns);
+
+ List<String> jsonIndexColumns = Arrays.asList(COL1_JSON_INDEX);
+ indexingConfig.setJsonIndexColumns(jsonIndexColumns);
+
+ SegmentGeneratorConfig segmentGeneratorConfig = new SegmentGeneratorConfig(TABLE_CONFIG, SCHEMA);
+ segmentGeneratorConfig.setTableName(RAW_TABLE_NAME);
+ segmentGeneratorConfig.setSegmentName(SEGMENT_NAME);
+ segmentGeneratorConfig.setOutDir(INDEX_DIR.getPath());
+
+ SegmentIndexCreationDriverImpl driver = new SegmentIndexCreationDriverImpl();
+ driver.init(segmentGeneratorConfig, new GenericRowRecordReader(records));
+ driver.build();
+
+ IndexLoadingConfig indexLoadingConfig = new IndexLoadingConfig();
+ indexLoadingConfig.setTableConfig(TABLE_CONFIG);
+ indexLoadingConfig.setInvertedIndexColumns(new HashSet<>(invertedIndexColumns));
+ indexLoadingConfig.setRangeIndexColumns(new HashSet<>(rangeIndexColumns));
+ indexLoadingConfig.setJsonIndexColumns(new HashSet<String>(jsonIndexColumns));
+ indexLoadingConfig.setReadMode(ReadMode.mmap);
+
+ ImmutableSegment immutableSegment =
+ ImmutableSegmentLoader.load(new File(INDEX_DIR, SEGMENT_NAME), indexLoadingConfig);
+ _indexSegment = immutableSegment;
+ _indexSegments = Arrays.asList(immutableSegment);
+ }
+
+ /** Checks the correctness of EXPLAIN PLAN output. */
+ private void check(String query, ResultTable expected) {
+ BrokerResponseNative response = getBrokerResponseForSqlQuery(query);
+ ResultTable actual = response.getResultTable();
+
+ System.out.println(resultTableToString("Expected", expected));
+ System.out.println(resultTableToString("Actual", actual));
+
+ Assert.assertEquals(actual.getDataSchema(), expected.getDataSchema());
+ Assert.assertEquals(actual.getRows().size(), expected.getRows().size());
+
+ List<Object[]> expectedRows = expected.getRows();
+ List<Object[]> actualRows = actual.getRows();
+ for (int i = 0; i < expectedRows.size(); i++) {
+ Object[] expectedRow = expectedRows.get(i);
+ Object[] actualRow = actualRows.get(i);
+ Assert.assertEquals(actualRow.length, expectedRow.length);
+
+ for (int j = 0; j < actualRow.length; j++) {
+ Assert.assertEquals(actualRow[j], expectedRow[j]);
+ }
+ }
+ }
+
+ /** This function is used to generated expected results while creating test cases; otherwise, not used. */
+ private static String resultTableToString(String tag, ResultTable table) {
+ System.out.println("\n" + tag + ":");
+ List<Object[]> rows = table.getRows();
+
+ StringBuffer buffer = new StringBuffer();
+ for (Object[] row : rows) {
+ buffer.append("result.add(new Object[]{\"" + row[0] + "\", " + row[1] + ", " + row[2] + "});\n");
+ }
+
+ return buffer.toString();
+ }
+
+ @Test
+ public void testSelect() {
+ String query1 = "EXPLAIN PLAN FOR SELECT * FROM testTable";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(
+ new Object[]{"SELECT(selectList:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, jsonIndexCol1, "
+ + "noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, "
+ + "jsonIndexCol1, noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 3, 2});
+ result1.add(
+ new Object[]{"PROJECT(sortedIndexCol1, noIndexCol3, rangeIndexCol1, rangeIndexCol2, jsonIndexCol1, "
+ + "invertedIndexCol1, noIndexCol2, invertedIndexCol2, noIndexCol1, invertedIndexCol3, rangeIndexCol3, "
+ + "textIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 = "EXPLAIN PLAN FOR SELECT 'mickey' FROM testTable";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:'mickey')", 2, 1});
+ result2.add(new Object[]{"PROJECT(ALL)", 3, 2});
+ result2.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 4, 3});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+
+ String query3 = "EXPLAIN PLAN FOR SELECT invertedIndexCol1, noIndexCol1 FROM testTable LIMIT 100";
+ List<Object[]> result3 = new ArrayList<>();
+ result3.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result3.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result3.add(new Object[]{"SELECT(selectList:invertedIndexCol1, noIndexCol1)", 2, 1});
+ result3.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result3.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result3.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query3, new ResultTable(DATA_SCHEMA, result3));
+
+ String query4 = "EXPLAIN PLAN FOR SELECT DISTINCT invertedIndexCol1, noIndexCol1 FROM testTable LIMIT 100";
+ List<Object[]> result4 = new ArrayList<>();
+ result4.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result4.add(new Object[]{"COMBINE_DISTINCT", 1, 0});
+ result4.add(new Object[]{"DISTINCT(keyColumns:invertedIndexCol1, noIndexCol1)", 2, 1});
+ result4.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result4.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result4.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query4, new ResultTable(DATA_SCHEMA, result4));
+ }
+
+ @Test
+ public void testSelectColumnsUsingFilterIndex() {
+ String query1 = "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1, sortedIndexCol1 FROM testTable WHERE "
+ + "invertedIndexCol1 = 1.5 AND sortedIndexCol1 = 1 LIMIT 100";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1, sortedIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, sortedIndexCol1)", 3, 2});
+ result1.add(new Object[]{"PROJECT(sortedIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 = "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1, sortedIndexCol1 FROM testTable WHERE "
+ + "(invertedIndexCol1 = 1.5 AND sortedIndexCol1 = 1) OR (invertedIndexCol1 BETWEEN 100 and 200 AND "
+ + "rangeIndexCol1 > 20)";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1, sortedIndexCol1)", 2, 1});
+ result2.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, sortedIndexCol1)", 3, 2});
+ result2.add(new Object[]{"PROJECT(sortedIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result2.add(new Object[]{"FILTER_EMPTY", 5, 4});
Review comment:
Same here. We should have seen the complex filter with root as OR and then two sub-trees in the EXPLAIN PLAN output.
##########
File path: pinot-core/src/test/java/org/apache/pinot/queries/ExplainPlanQueriesTest.java
##########
@@ -0,0 +1,510 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.queries;
+
+import java.io.File;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.List;
+import org.apache.commons.io.FileUtils;
+import org.apache.pinot.common.response.broker.BrokerResponseNative;
+import org.apache.pinot.common.response.broker.ResultTable;
+import org.apache.pinot.common.utils.DataSchema;
+import org.apache.pinot.segment.local.indexsegment.immutable.ImmutableSegmentLoader;
+import org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl;
+import org.apache.pinot.segment.local.segment.index.loader.IndexLoadingConfig;
+import org.apache.pinot.segment.local.segment.readers.GenericRowRecordReader;
+import org.apache.pinot.segment.spi.ImmutableSegment;
+import org.apache.pinot.segment.spi.IndexSegment;
+import org.apache.pinot.segment.spi.creator.SegmentGeneratorConfig;
+import org.apache.pinot.spi.config.table.IndexingConfig;
+import org.apache.pinot.spi.config.table.TableConfig;
+import org.apache.pinot.spi.config.table.TableType;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.data.Schema;
+import org.apache.pinot.spi.data.readers.GenericRow;
+import org.apache.pinot.spi.utils.ReadMode;
+import org.apache.pinot.spi.utils.builder.TableConfigBuilder;
+import org.testng.Assert;
+import org.testng.annotations.BeforeClass;
+import org.testng.annotations.Test;
+
+
+public class ExplainPlanQueriesTest extends BaseQueriesTest {
+ private static final File INDEX_DIR = new File(FileUtils.getTempDirectory(), "ExplainPlanQueriesTest");
+ private static final String RAW_TABLE_NAME = "testTable";
+ private static final String SEGMENT_NAME = "testSegment";
+ private static final int NUM_RECORDS = 10;
+
+ private final static String COL1_NO_INDEX = "noIndexCol1";
+ private final static String COL2_NO_INDEX = "noIndexCol2";
+ private final static String COL3_NO_INDEX = "noIndexCol3";
+ private final static String COL1_INVERTED_INDEX = "invertedIndexCol1";
+ private final static String COL2_INVERTED_INDEX = "invertedIndexCol2";
+ private final static String COL3_INVERTED_INDEX = "invertedIndexCol3";
+ private final static String COL1_RANGE_INDEX = "rangeIndexCol1";
+ private final static String COL2_RANGE_INDEX = "rangeIndexCol2";
+ private final static String COL3_RANGE_INDEX = "rangeIndexCol3";
+ private final static String COL1_SORTED_INDEX = "sortedIndexCol1";
+ private final static String COL1_JSON_INDEX = "jsonIndexCol1";
+ private final static String COL1_TEXT_INDEX = "textIndexCol1";
+
+ private static final Schema SCHEMA = new Schema.SchemaBuilder().setSchemaName(RAW_TABLE_NAME)
+ .addSingleValueDimension(COL1_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL2_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_INVERTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_INVERTED_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_INVERTED_INDEX, FieldSpec.DataType.STRING)
+ .addSingleValueDimension(COL1_RANGE_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_SORTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL1_JSON_INDEX, FieldSpec.DataType.JSON)
+ .addSingleValueDimension(COL1_TEXT_INDEX, FieldSpec.DataType.STRING).build();
+
+ private static final DataSchema DATA_SCHEMA = new DataSchema(new String[]{"Operator", "Operator_Id", "Parent_Id"},
+ new DataSchema.ColumnDataType[]{DataSchema.ColumnDataType.STRING, DataSchema.ColumnDataType.INT,
+ DataSchema.ColumnDataType.INT});
+
+ private static final TableConfig TABLE_CONFIG =
+ new TableConfigBuilder(TableType.OFFLINE).setTableName(RAW_TABLE_NAME).build();
+
+ private IndexSegment _indexSegment;
+ private List<IndexSegment> _indexSegments;
+
+ @Override
+ protected String getFilter() {
+ return "";
+ }
+
+ @Override
+ protected IndexSegment getIndexSegment() {
+ return _indexSegment;
+ }
+
+ @Override
+ protected List<IndexSegment> getIndexSegments() {
+ return _indexSegments;
+ }
+
+ GenericRow createRecord(double doubleValue, int intValue, String stringValue, String jsonValue) {
+ GenericRow record = new GenericRow();
+ record.putValue(COL1_NO_INDEX, intValue);
+ record.putValue(COL2_NO_INDEX, intValue);
+ record.putValue(COL3_NO_INDEX, intValue);
+
+ record.putValue(COL1_INVERTED_INDEX, doubleValue);
+ record.putValue(COL2_INVERTED_INDEX, intValue);
+ record.putValue(COL3_INVERTED_INDEX, intValue);
+
+ record.putValue(COL1_RANGE_INDEX, doubleValue);
+ record.putValue(COL2_RANGE_INDEX, intValue);
+ record.putValue(COL3_RANGE_INDEX, intValue);
+
+ record.putValue(COL1_JSON_INDEX, jsonValue);
+ record.putValue(COL1_TEXT_INDEX, stringValue);
+
+ return record;
+ }
+
+ @BeforeClass
+ public void setUp()
+ throws Exception {
+ FileUtils.deleteDirectory(INDEX_DIR);
+
+ List<GenericRow> records = new ArrayList<>(NUM_RECORDS);
+ records.add(createRecord(1.1, 2, "daffy", "{\"first\": \"daffy\", \"last\": \"duck\"}"));
+
+ IndexingConfig indexingConfig = TABLE_CONFIG.getIndexingConfig();
+
+ List<String> invertedIndexColumns = Arrays.asList(COL1_INVERTED_INDEX, COL2_INVERTED_INDEX, COL3_INVERTED_INDEX);
+ indexingConfig.setInvertedIndexColumns(invertedIndexColumns);
+
+ List<String> rangeIndexColumns = Arrays.asList(COL1_RANGE_INDEX, COL2_RANGE_INDEX, COL3_RANGE_INDEX);
+ indexingConfig.setRangeIndexColumns(rangeIndexColumns);
+
+ List<String> jsonIndexColumns = Arrays.asList(COL1_JSON_INDEX);
+ indexingConfig.setJsonIndexColumns(jsonIndexColumns);
+
+ SegmentGeneratorConfig segmentGeneratorConfig = new SegmentGeneratorConfig(TABLE_CONFIG, SCHEMA);
+ segmentGeneratorConfig.setTableName(RAW_TABLE_NAME);
+ segmentGeneratorConfig.setSegmentName(SEGMENT_NAME);
+ segmentGeneratorConfig.setOutDir(INDEX_DIR.getPath());
+
+ SegmentIndexCreationDriverImpl driver = new SegmentIndexCreationDriverImpl();
+ driver.init(segmentGeneratorConfig, new GenericRowRecordReader(records));
+ driver.build();
+
+ IndexLoadingConfig indexLoadingConfig = new IndexLoadingConfig();
+ indexLoadingConfig.setTableConfig(TABLE_CONFIG);
+ indexLoadingConfig.setInvertedIndexColumns(new HashSet<>(invertedIndexColumns));
+ indexLoadingConfig.setRangeIndexColumns(new HashSet<>(rangeIndexColumns));
+ indexLoadingConfig.setJsonIndexColumns(new HashSet<String>(jsonIndexColumns));
+ indexLoadingConfig.setReadMode(ReadMode.mmap);
+
+ ImmutableSegment immutableSegment =
+ ImmutableSegmentLoader.load(new File(INDEX_DIR, SEGMENT_NAME), indexLoadingConfig);
+ _indexSegment = immutableSegment;
+ _indexSegments = Arrays.asList(immutableSegment);
+ }
+
+ /** Checks the correctness of EXPLAIN PLAN output. */
+ private void check(String query, ResultTable expected) {
+ BrokerResponseNative response = getBrokerResponseForSqlQuery(query);
+ ResultTable actual = response.getResultTable();
+
+ System.out.println(resultTableToString("Expected", expected));
+ System.out.println(resultTableToString("Actual", actual));
+
+ Assert.assertEquals(actual.getDataSchema(), expected.getDataSchema());
+ Assert.assertEquals(actual.getRows().size(), expected.getRows().size());
+
+ List<Object[]> expectedRows = expected.getRows();
+ List<Object[]> actualRows = actual.getRows();
+ for (int i = 0; i < expectedRows.size(); i++) {
+ Object[] expectedRow = expectedRows.get(i);
+ Object[] actualRow = actualRows.get(i);
+ Assert.assertEquals(actualRow.length, expectedRow.length);
+
+ for (int j = 0; j < actualRow.length; j++) {
+ Assert.assertEquals(actualRow[j], expectedRow[j]);
+ }
+ }
+ }
+
+ /** This function is used to generated expected results while creating test cases; otherwise, not used. */
+ private static String resultTableToString(String tag, ResultTable table) {
+ System.out.println("\n" + tag + ":");
+ List<Object[]> rows = table.getRows();
+
+ StringBuffer buffer = new StringBuffer();
+ for (Object[] row : rows) {
+ buffer.append("result.add(new Object[]{\"" + row[0] + "\", " + row[1] + ", " + row[2] + "});\n");
+ }
+
+ return buffer.toString();
+ }
+
+ @Test
+ public void testSelect() {
+ String query1 = "EXPLAIN PLAN FOR SELECT * FROM testTable";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(
+ new Object[]{"SELECT(selectList:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, jsonIndexCol1, "
+ + "noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, "
+ + "jsonIndexCol1, noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 3, 2});
+ result1.add(
+ new Object[]{"PROJECT(sortedIndexCol1, noIndexCol3, rangeIndexCol1, rangeIndexCol2, jsonIndexCol1, "
+ + "invertedIndexCol1, noIndexCol2, invertedIndexCol2, noIndexCol1, invertedIndexCol3, rangeIndexCol3, "
+ + "textIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 = "EXPLAIN PLAN FOR SELECT 'mickey' FROM testTable";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:'mickey')", 2, 1});
+ result2.add(new Object[]{"PROJECT(ALL)", 3, 2});
+ result2.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 4, 3});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+
+ String query3 = "EXPLAIN PLAN FOR SELECT invertedIndexCol1, noIndexCol1 FROM testTable LIMIT 100";
+ List<Object[]> result3 = new ArrayList<>();
+ result3.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result3.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result3.add(new Object[]{"SELECT(selectList:invertedIndexCol1, noIndexCol1)", 2, 1});
+ result3.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result3.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result3.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query3, new ResultTable(DATA_SCHEMA, result3));
+
+ String query4 = "EXPLAIN PLAN FOR SELECT DISTINCT invertedIndexCol1, noIndexCol1 FROM testTable LIMIT 100";
+ List<Object[]> result4 = new ArrayList<>();
+ result4.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result4.add(new Object[]{"COMBINE_DISTINCT", 1, 0});
+ result4.add(new Object[]{"DISTINCT(keyColumns:invertedIndexCol1, noIndexCol1)", 2, 1});
+ result4.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result4.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result4.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query4, new ResultTable(DATA_SCHEMA, result4));
+ }
+
+ @Test
+ public void testSelectColumnsUsingFilterIndex() {
+ String query1 = "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1, sortedIndexCol1 FROM testTable WHERE "
+ + "invertedIndexCol1 = 1.5 AND sortedIndexCol1 = 1 LIMIT 100";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1, sortedIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, sortedIndexCol1)", 3, 2});
+ result1.add(new Object[]{"PROJECT(sortedIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 = "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1, sortedIndexCol1 FROM testTable WHERE "
+ + "(invertedIndexCol1 = 1.5 AND sortedIndexCol1 = 1) OR (invertedIndexCol1 BETWEEN 100 and 200 AND "
+ + "rangeIndexCol1 > 20)";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1, sortedIndexCol1)", 2, 1});
+ result2.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, sortedIndexCol1)", 3, 2});
+ result2.add(new Object[]{"PROJECT(sortedIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result2.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+
+ String query3 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1 FROM testTable WHERE invertedIndexCol1 = 1.5 AND "
+ + "invertedIndexCol2 IN (10, 20, 30) AND invertedIndexCol3 NOT IN ('foo', 'bar') LIMIT 100";
+ List<Object[]> result3 = new ArrayList<>();
+ result3.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result3.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result3.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1)", 2, 1});
+ result3.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result3.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result3.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query3, new ResultTable(DATA_SCHEMA, result3));
+
+ String query4 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, noIndexCol2, sortedIndexCol1 FROM testTable WHERE sortedIndexCol1 = 1.5"
+ + " OR sortedIndexCol1 != 5 OR sortedIndexCol1 IN (10, 20, 30) LIMIT 100";
+ List<Object[]> result4 = new ArrayList<>();
+ result4.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result4.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result4.add(new Object[]{"SELECT(selectList:noIndexCol1, noIndexCol2, sortedIndexCol1)", 2, 1});
+ result4.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2, sortedIndexCol1)", 3, 2});
+ result4.add(new Object[]{"PROJECT(sortedIndexCol1, noIndexCol2, noIndexCol1)", 4, 3});
+ result4.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query4, new ResultTable(DATA_SCHEMA, result4));
+
+ String query5 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, noIndexCol2 FROM testTable WHERE DIV(noIndexCol1, noIndexCol2) BETWEEN "
+ + "10 AND 20 AND invertedIndexCol1 * 5 < 1000";
+ List<Object[]> result5 = new ArrayList<>();
+ result5.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result5.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result5.add(new Object[]{"SELECT(selectList:noIndexCol1, noIndexCol2)", 2, 1});
+ result5.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2)", 3, 2});
+ result5.add(new Object[]{"PROJECT(noIndexCol2, noIndexCol1)", 4, 3});
+ result5.add(new Object[]{"FILTER_AND", 5, 4});
+ result5.add(
+ new Object[]{"FILTER_EXPRESSION(operator:RANGE,predicate:div(noIndexCol1,noIndexCol2) BETWEEN '10' AND '20')",
+ 6, 5});
+ result5
+ .add(new Object[]{"FILTER_EXPRESSION(operator:RANGE,predicate:times(invertedIndexCol1,'5') < '1000')", 7, 5});
+ check(query5, new ResultTable(DATA_SCHEMA, result5));
+ }
+
+ @Test
+ public void testSelectColumnUsingFilterRange() {
+ // select * query triggering range index
+ // checks using RANGE (>, >=, <, <=, BETWEEN ..) on a column with range index should use RANGE_INDEX_SCAN
+ String query1 =
+ "EXPLAIN PLAN FOR SELECT invertedIndexCol1, noIndexCol1, rangeIndexCol1 FROM testTable WHERE rangeIndexCol1 >"
+ + " 10 AND rangeIndexCol2 >= 20 AND rangeIndexCol3 BETWEEN 50 AND 60 LIMIT 100";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(new Object[]{"SELECT(selectList:invertedIndexCol1, noIndexCol1, rangeIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, rangeIndexCol1)", 3, 2});
+ result1.add(new Object[]{"PROJECT(rangeIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1 FROM testTable WHERE noIndexCol1 > 10 AND noIndexCol2"
+ + " BETWEEN 50 AND 60 LIMIT 100";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1)", 2, 1});
+ result2.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result2.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result2.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+ }
+
+ @Test
+ public void testSelectColumnUsingFilterJson() {
+ String query =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1 FROM testTable WHERE (invertedIndexCol1 IN (10, 20, "
+ + "30) AND sortedIndexCol1 != 100) OR (noIndexCol1 NOT IN (10, 20, 30) AND rangeIndexCol1 != 20 AND "
+ + "JSON_MATCH(jsonIndexCol1, 'key=1') AND TEXT_MATCH(textIndexCol1, 'foo'))";
+ List<Object[]> result = new ArrayList<>();
+ result.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1)", 2, 1});
+ result.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result.add(new Object[]{"FILTER_AND", 5, 4});
+ result.add(new Object[]{"FILTER_JSON_INDEX(indexLookUp:json_index,operator:JSON_MATCH,predicate:json_match"
+ + "(jsonIndexCol1,'key=1'))", 6, 5});
+ result.add(new Object[]{"FILTER_TEXT_INDEX(indexLookUp:text_index,operator:TEXT_MATCH,predicate:text_match"
+ + "(textIndexCol1,'foo'))", 7, 5});
+ check(query, new ResultTable(DATA_SCHEMA, result));
+ }
+
+ @Test
+ public void testSelectAggregateUsingFilterText() {
+ String query1 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, noIndexCol2, max(noIndexCol2), min(noIndexCol3) FROM testTable WHERE "
+ + "TEXT_MATCH(textIndexCol1, 'foo') GROUP BY noIndexCol1, noIndexCol2";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result1.add(new Object[]{"COMBINE_GROUPBY_ORDERBY", 1, 0});
+ result1.add(new Object[]{
+ "AGGREGATE_GROUPBY_ORDERBY(groupKeys:noIndexCol1, noIndexCol2, aggregations:max(noIndexCol2), min"
+ + "(noIndexCol3))", 2, 1});
+ result1.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2, noIndexCol3)", 3, 2});
+ result1.add(new Object[]{"PROJECT(noIndexCol3, noIndexCol2, noIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_TEXT_INDEX(indexLookUp:text_index,operator:TEXT_MATCH,predicate:text_match"
+ + "(textIndexCol1,'foo'))", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, max(noIndexCol2), min(noIndexCol3) FROM testTable WHERE TEXT_MATCH"
+ + "(textIndexCol1, 'foo') GROUP BY noIndexCol1, noIndexCol2 ORDER BY noIndexCol1, max(noIndexCol2)";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(sort:[noIndexCol1 ASC, max(noIndexCol2) ASC],limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_GROUPBY_ORDERBY", 1, 0});
+ result2.add(new Object[]{
+ "AGGREGATE_GROUPBY_ORDERBY(groupKeys:noIndexCol1,noIndexCol2,aggregations:max(noIndexCol2), min"
+ + "(noIndexCol3))", 2, 1});
+ result2.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, invertedIndexCol1)", 3, 2});
+ result2.add(new Object[]{"PROJECT(noIndexCol3, noIndexCol2, noIndexCol1)", 4, 3});
+ result2.add(new Object[]{"FILTER_TEXT_INDEX(indexLookUp:text_index,operator:TEXT_MATCH,predicate:text_match"
+ + "(textIndexCol1,'foo'))", 5, 4});
+ }
+
+ @Test
+ public void testSelectAggregate() {
+ String query1 = "EXPLAIN PLAN FOR SELECT count(*) FROM testTable";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result1.add(new Object[]{"COMBINE_AGGREGATE", 1, 0});
+ result1.add(new Object[]{"AGGREGATE_METADATA", 2, 1});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
Review comment:
Can you add tests for Dictionary based aggregation as well
##########
File path: pinot-core/src/test/java/org/apache/pinot/queries/ExplainPlanQueriesTest.java
##########
@@ -0,0 +1,510 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.queries;
+
+import java.io.File;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.List;
+import org.apache.commons.io.FileUtils;
+import org.apache.pinot.common.response.broker.BrokerResponseNative;
+import org.apache.pinot.common.response.broker.ResultTable;
+import org.apache.pinot.common.utils.DataSchema;
+import org.apache.pinot.segment.local.indexsegment.immutable.ImmutableSegmentLoader;
+import org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl;
+import org.apache.pinot.segment.local.segment.index.loader.IndexLoadingConfig;
+import org.apache.pinot.segment.local.segment.readers.GenericRowRecordReader;
+import org.apache.pinot.segment.spi.ImmutableSegment;
+import org.apache.pinot.segment.spi.IndexSegment;
+import org.apache.pinot.segment.spi.creator.SegmentGeneratorConfig;
+import org.apache.pinot.spi.config.table.IndexingConfig;
+import org.apache.pinot.spi.config.table.TableConfig;
+import org.apache.pinot.spi.config.table.TableType;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.data.Schema;
+import org.apache.pinot.spi.data.readers.GenericRow;
+import org.apache.pinot.spi.utils.ReadMode;
+import org.apache.pinot.spi.utils.builder.TableConfigBuilder;
+import org.testng.Assert;
+import org.testng.annotations.BeforeClass;
+import org.testng.annotations.Test;
+
+
+public class ExplainPlanQueriesTest extends BaseQueriesTest {
+ private static final File INDEX_DIR = new File(FileUtils.getTempDirectory(), "ExplainPlanQueriesTest");
+ private static final String RAW_TABLE_NAME = "testTable";
+ private static final String SEGMENT_NAME = "testSegment";
+ private static final int NUM_RECORDS = 10;
+
+ private final static String COL1_NO_INDEX = "noIndexCol1";
+ private final static String COL2_NO_INDEX = "noIndexCol2";
+ private final static String COL3_NO_INDEX = "noIndexCol3";
+ private final static String COL1_INVERTED_INDEX = "invertedIndexCol1";
+ private final static String COL2_INVERTED_INDEX = "invertedIndexCol2";
+ private final static String COL3_INVERTED_INDEX = "invertedIndexCol3";
+ private final static String COL1_RANGE_INDEX = "rangeIndexCol1";
+ private final static String COL2_RANGE_INDEX = "rangeIndexCol2";
+ private final static String COL3_RANGE_INDEX = "rangeIndexCol3";
+ private final static String COL1_SORTED_INDEX = "sortedIndexCol1";
+ private final static String COL1_JSON_INDEX = "jsonIndexCol1";
+ private final static String COL1_TEXT_INDEX = "textIndexCol1";
+
+ private static final Schema SCHEMA = new Schema.SchemaBuilder().setSchemaName(RAW_TABLE_NAME)
+ .addSingleValueDimension(COL1_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL2_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_INVERTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_INVERTED_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_INVERTED_INDEX, FieldSpec.DataType.STRING)
+ .addSingleValueDimension(COL1_RANGE_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_SORTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL1_JSON_INDEX, FieldSpec.DataType.JSON)
+ .addSingleValueDimension(COL1_TEXT_INDEX, FieldSpec.DataType.STRING).build();
+
+ private static final DataSchema DATA_SCHEMA = new DataSchema(new String[]{"Operator", "Operator_Id", "Parent_Id"},
+ new DataSchema.ColumnDataType[]{DataSchema.ColumnDataType.STRING, DataSchema.ColumnDataType.INT,
+ DataSchema.ColumnDataType.INT});
+
+ private static final TableConfig TABLE_CONFIG =
+ new TableConfigBuilder(TableType.OFFLINE).setTableName(RAW_TABLE_NAME).build();
+
+ private IndexSegment _indexSegment;
+ private List<IndexSegment> _indexSegments;
+
+ @Override
+ protected String getFilter() {
+ return "";
+ }
+
+ @Override
+ protected IndexSegment getIndexSegment() {
+ return _indexSegment;
+ }
+
+ @Override
+ protected List<IndexSegment> getIndexSegments() {
+ return _indexSegments;
+ }
+
+ GenericRow createRecord(double doubleValue, int intValue, String stringValue, String jsonValue) {
+ GenericRow record = new GenericRow();
+ record.putValue(COL1_NO_INDEX, intValue);
+ record.putValue(COL2_NO_INDEX, intValue);
+ record.putValue(COL3_NO_INDEX, intValue);
+
+ record.putValue(COL1_INVERTED_INDEX, doubleValue);
+ record.putValue(COL2_INVERTED_INDEX, intValue);
+ record.putValue(COL3_INVERTED_INDEX, intValue);
+
+ record.putValue(COL1_RANGE_INDEX, doubleValue);
+ record.putValue(COL2_RANGE_INDEX, intValue);
+ record.putValue(COL3_RANGE_INDEX, intValue);
+
+ record.putValue(COL1_JSON_INDEX, jsonValue);
+ record.putValue(COL1_TEXT_INDEX, stringValue);
+
+ return record;
+ }
+
+ @BeforeClass
+ public void setUp()
+ throws Exception {
+ FileUtils.deleteDirectory(INDEX_DIR);
+
+ List<GenericRow> records = new ArrayList<>(NUM_RECORDS);
+ records.add(createRecord(1.1, 2, "daffy", "{\"first\": \"daffy\", \"last\": \"duck\"}"));
+
+ IndexingConfig indexingConfig = TABLE_CONFIG.getIndexingConfig();
+
+ List<String> invertedIndexColumns = Arrays.asList(COL1_INVERTED_INDEX, COL2_INVERTED_INDEX, COL3_INVERTED_INDEX);
+ indexingConfig.setInvertedIndexColumns(invertedIndexColumns);
+
+ List<String> rangeIndexColumns = Arrays.asList(COL1_RANGE_INDEX, COL2_RANGE_INDEX, COL3_RANGE_INDEX);
+ indexingConfig.setRangeIndexColumns(rangeIndexColumns);
+
+ List<String> jsonIndexColumns = Arrays.asList(COL1_JSON_INDEX);
+ indexingConfig.setJsonIndexColumns(jsonIndexColumns);
+
+ SegmentGeneratorConfig segmentGeneratorConfig = new SegmentGeneratorConfig(TABLE_CONFIG, SCHEMA);
+ segmentGeneratorConfig.setTableName(RAW_TABLE_NAME);
+ segmentGeneratorConfig.setSegmentName(SEGMENT_NAME);
+ segmentGeneratorConfig.setOutDir(INDEX_DIR.getPath());
+
+ SegmentIndexCreationDriverImpl driver = new SegmentIndexCreationDriverImpl();
+ driver.init(segmentGeneratorConfig, new GenericRowRecordReader(records));
+ driver.build();
+
+ IndexLoadingConfig indexLoadingConfig = new IndexLoadingConfig();
+ indexLoadingConfig.setTableConfig(TABLE_CONFIG);
+ indexLoadingConfig.setInvertedIndexColumns(new HashSet<>(invertedIndexColumns));
+ indexLoadingConfig.setRangeIndexColumns(new HashSet<>(rangeIndexColumns));
+ indexLoadingConfig.setJsonIndexColumns(new HashSet<String>(jsonIndexColumns));
+ indexLoadingConfig.setReadMode(ReadMode.mmap);
+
+ ImmutableSegment immutableSegment =
+ ImmutableSegmentLoader.load(new File(INDEX_DIR, SEGMENT_NAME), indexLoadingConfig);
+ _indexSegment = immutableSegment;
+ _indexSegments = Arrays.asList(immutableSegment);
+ }
+
+ /** Checks the correctness of EXPLAIN PLAN output. */
+ private void check(String query, ResultTable expected) {
+ BrokerResponseNative response = getBrokerResponseForSqlQuery(query);
+ ResultTable actual = response.getResultTable();
+
+ System.out.println(resultTableToString("Expected", expected));
+ System.out.println(resultTableToString("Actual", actual));
+
+ Assert.assertEquals(actual.getDataSchema(), expected.getDataSchema());
+ Assert.assertEquals(actual.getRows().size(), expected.getRows().size());
+
+ List<Object[]> expectedRows = expected.getRows();
+ List<Object[]> actualRows = actual.getRows();
+ for (int i = 0; i < expectedRows.size(); i++) {
+ Object[] expectedRow = expectedRows.get(i);
+ Object[] actualRow = actualRows.get(i);
+ Assert.assertEquals(actualRow.length, expectedRow.length);
+
+ for (int j = 0; j < actualRow.length; j++) {
+ Assert.assertEquals(actualRow[j], expectedRow[j]);
+ }
+ }
+ }
+
+ /** This function is used to generated expected results while creating test cases; otherwise, not used. */
+ private static String resultTableToString(String tag, ResultTable table) {
+ System.out.println("\n" + tag + ":");
+ List<Object[]> rows = table.getRows();
+
+ StringBuffer buffer = new StringBuffer();
+ for (Object[] row : rows) {
+ buffer.append("result.add(new Object[]{\"" + row[0] + "\", " + row[1] + ", " + row[2] + "});\n");
+ }
+
+ return buffer.toString();
+ }
+
+ @Test
+ public void testSelect() {
+ String query1 = "EXPLAIN PLAN FOR SELECT * FROM testTable";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(
+ new Object[]{"SELECT(selectList:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, jsonIndexCol1, "
+ + "noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, "
+ + "jsonIndexCol1, noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 3, 2});
+ result1.add(
+ new Object[]{"PROJECT(sortedIndexCol1, noIndexCol3, rangeIndexCol1, rangeIndexCol2, jsonIndexCol1, "
+ + "invertedIndexCol1, noIndexCol2, invertedIndexCol2, noIndexCol1, invertedIndexCol3, rangeIndexCol3, "
+ + "textIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 = "EXPLAIN PLAN FOR SELECT 'mickey' FROM testTable";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:'mickey')", 2, 1});
+ result2.add(new Object[]{"PROJECT(ALL)", 3, 2});
+ result2.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 4, 3});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+
+ String query3 = "EXPLAIN PLAN FOR SELECT invertedIndexCol1, noIndexCol1 FROM testTable LIMIT 100";
+ List<Object[]> result3 = new ArrayList<>();
+ result3.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result3.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result3.add(new Object[]{"SELECT(selectList:invertedIndexCol1, noIndexCol1)", 2, 1});
+ result3.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result3.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result3.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query3, new ResultTable(DATA_SCHEMA, result3));
+
+ String query4 = "EXPLAIN PLAN FOR SELECT DISTINCT invertedIndexCol1, noIndexCol1 FROM testTable LIMIT 100";
+ List<Object[]> result4 = new ArrayList<>();
+ result4.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result4.add(new Object[]{"COMBINE_DISTINCT", 1, 0});
+ result4.add(new Object[]{"DISTINCT(keyColumns:invertedIndexCol1, noIndexCol1)", 2, 1});
+ result4.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result4.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result4.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query4, new ResultTable(DATA_SCHEMA, result4));
+ }
+
+ @Test
Review comment:
For people who will fix/enhance this test in future, I suggest adding a comment to indicate this test is for inverted index and sorted index filter only
##########
File path: pinot-core/src/test/java/org/apache/pinot/queries/ExplainPlanQueriesTest.java
##########
@@ -0,0 +1,510 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.queries;
+
+import java.io.File;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.List;
+import org.apache.commons.io.FileUtils;
+import org.apache.pinot.common.response.broker.BrokerResponseNative;
+import org.apache.pinot.common.response.broker.ResultTable;
+import org.apache.pinot.common.utils.DataSchema;
+import org.apache.pinot.segment.local.indexsegment.immutable.ImmutableSegmentLoader;
+import org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl;
+import org.apache.pinot.segment.local.segment.index.loader.IndexLoadingConfig;
+import org.apache.pinot.segment.local.segment.readers.GenericRowRecordReader;
+import org.apache.pinot.segment.spi.ImmutableSegment;
+import org.apache.pinot.segment.spi.IndexSegment;
+import org.apache.pinot.segment.spi.creator.SegmentGeneratorConfig;
+import org.apache.pinot.spi.config.table.IndexingConfig;
+import org.apache.pinot.spi.config.table.TableConfig;
+import org.apache.pinot.spi.config.table.TableType;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.data.Schema;
+import org.apache.pinot.spi.data.readers.GenericRow;
+import org.apache.pinot.spi.utils.ReadMode;
+import org.apache.pinot.spi.utils.builder.TableConfigBuilder;
+import org.testng.Assert;
+import org.testng.annotations.BeforeClass;
+import org.testng.annotations.Test;
+
+
+public class ExplainPlanQueriesTest extends BaseQueriesTest {
+ private static final File INDEX_DIR = new File(FileUtils.getTempDirectory(), "ExplainPlanQueriesTest");
+ private static final String RAW_TABLE_NAME = "testTable";
+ private static final String SEGMENT_NAME = "testSegment";
+ private static final int NUM_RECORDS = 10;
+
+ private final static String COL1_NO_INDEX = "noIndexCol1";
+ private final static String COL2_NO_INDEX = "noIndexCol2";
+ private final static String COL3_NO_INDEX = "noIndexCol3";
+ private final static String COL1_INVERTED_INDEX = "invertedIndexCol1";
+ private final static String COL2_INVERTED_INDEX = "invertedIndexCol2";
+ private final static String COL3_INVERTED_INDEX = "invertedIndexCol3";
+ private final static String COL1_RANGE_INDEX = "rangeIndexCol1";
+ private final static String COL2_RANGE_INDEX = "rangeIndexCol2";
+ private final static String COL3_RANGE_INDEX = "rangeIndexCol3";
+ private final static String COL1_SORTED_INDEX = "sortedIndexCol1";
+ private final static String COL1_JSON_INDEX = "jsonIndexCol1";
+ private final static String COL1_TEXT_INDEX = "textIndexCol1";
+
+ private static final Schema SCHEMA = new Schema.SchemaBuilder().setSchemaName(RAW_TABLE_NAME)
+ .addSingleValueDimension(COL1_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL2_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_INVERTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_INVERTED_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_INVERTED_INDEX, FieldSpec.DataType.STRING)
+ .addSingleValueDimension(COL1_RANGE_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_SORTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL1_JSON_INDEX, FieldSpec.DataType.JSON)
+ .addSingleValueDimension(COL1_TEXT_INDEX, FieldSpec.DataType.STRING).build();
+
+ private static final DataSchema DATA_SCHEMA = new DataSchema(new String[]{"Operator", "Operator_Id", "Parent_Id"},
+ new DataSchema.ColumnDataType[]{DataSchema.ColumnDataType.STRING, DataSchema.ColumnDataType.INT,
+ DataSchema.ColumnDataType.INT});
+
+ private static final TableConfig TABLE_CONFIG =
+ new TableConfigBuilder(TableType.OFFLINE).setTableName(RAW_TABLE_NAME).build();
+
+ private IndexSegment _indexSegment;
+ private List<IndexSegment> _indexSegments;
+
+ @Override
+ protected String getFilter() {
+ return "";
+ }
+
+ @Override
+ protected IndexSegment getIndexSegment() {
+ return _indexSegment;
+ }
+
+ @Override
+ protected List<IndexSegment> getIndexSegments() {
+ return _indexSegments;
+ }
+
+ GenericRow createRecord(double doubleValue, int intValue, String stringValue, String jsonValue) {
+ GenericRow record = new GenericRow();
+ record.putValue(COL1_NO_INDEX, intValue);
+ record.putValue(COL2_NO_INDEX, intValue);
+ record.putValue(COL3_NO_INDEX, intValue);
+
+ record.putValue(COL1_INVERTED_INDEX, doubleValue);
+ record.putValue(COL2_INVERTED_INDEX, intValue);
+ record.putValue(COL3_INVERTED_INDEX, intValue);
+
+ record.putValue(COL1_RANGE_INDEX, doubleValue);
+ record.putValue(COL2_RANGE_INDEX, intValue);
+ record.putValue(COL3_RANGE_INDEX, intValue);
+
+ record.putValue(COL1_JSON_INDEX, jsonValue);
+ record.putValue(COL1_TEXT_INDEX, stringValue);
+
+ return record;
+ }
+
+ @BeforeClass
+ public void setUp()
+ throws Exception {
+ FileUtils.deleteDirectory(INDEX_DIR);
+
+ List<GenericRow> records = new ArrayList<>(NUM_RECORDS);
+ records.add(createRecord(1.1, 2, "daffy", "{\"first\": \"daffy\", \"last\": \"duck\"}"));
+
+ IndexingConfig indexingConfig = TABLE_CONFIG.getIndexingConfig();
+
+ List<String> invertedIndexColumns = Arrays.asList(COL1_INVERTED_INDEX, COL2_INVERTED_INDEX, COL3_INVERTED_INDEX);
+ indexingConfig.setInvertedIndexColumns(invertedIndexColumns);
+
+ List<String> rangeIndexColumns = Arrays.asList(COL1_RANGE_INDEX, COL2_RANGE_INDEX, COL3_RANGE_INDEX);
+ indexingConfig.setRangeIndexColumns(rangeIndexColumns);
+
+ List<String> jsonIndexColumns = Arrays.asList(COL1_JSON_INDEX);
+ indexingConfig.setJsonIndexColumns(jsonIndexColumns);
+
+ SegmentGeneratorConfig segmentGeneratorConfig = new SegmentGeneratorConfig(TABLE_CONFIG, SCHEMA);
+ segmentGeneratorConfig.setTableName(RAW_TABLE_NAME);
+ segmentGeneratorConfig.setSegmentName(SEGMENT_NAME);
+ segmentGeneratorConfig.setOutDir(INDEX_DIR.getPath());
+
+ SegmentIndexCreationDriverImpl driver = new SegmentIndexCreationDriverImpl();
+ driver.init(segmentGeneratorConfig, new GenericRowRecordReader(records));
+ driver.build();
+
+ IndexLoadingConfig indexLoadingConfig = new IndexLoadingConfig();
+ indexLoadingConfig.setTableConfig(TABLE_CONFIG);
+ indexLoadingConfig.setInvertedIndexColumns(new HashSet<>(invertedIndexColumns));
+ indexLoadingConfig.setRangeIndexColumns(new HashSet<>(rangeIndexColumns));
+ indexLoadingConfig.setJsonIndexColumns(new HashSet<String>(jsonIndexColumns));
+ indexLoadingConfig.setReadMode(ReadMode.mmap);
+
+ ImmutableSegment immutableSegment =
+ ImmutableSegmentLoader.load(new File(INDEX_DIR, SEGMENT_NAME), indexLoadingConfig);
+ _indexSegment = immutableSegment;
+ _indexSegments = Arrays.asList(immutableSegment);
+ }
+
+ /** Checks the correctness of EXPLAIN PLAN output. */
+ private void check(String query, ResultTable expected) {
+ BrokerResponseNative response = getBrokerResponseForSqlQuery(query);
+ ResultTable actual = response.getResultTable();
+
+ System.out.println(resultTableToString("Expected", expected));
+ System.out.println(resultTableToString("Actual", actual));
+
+ Assert.assertEquals(actual.getDataSchema(), expected.getDataSchema());
+ Assert.assertEquals(actual.getRows().size(), expected.getRows().size());
+
+ List<Object[]> expectedRows = expected.getRows();
+ List<Object[]> actualRows = actual.getRows();
+ for (int i = 0; i < expectedRows.size(); i++) {
+ Object[] expectedRow = expectedRows.get(i);
+ Object[] actualRow = actualRows.get(i);
+ Assert.assertEquals(actualRow.length, expectedRow.length);
+
+ for (int j = 0; j < actualRow.length; j++) {
+ Assert.assertEquals(actualRow[j], expectedRow[j]);
+ }
+ }
+ }
+
+ /** This function is used to generated expected results while creating test cases; otherwise, not used. */
+ private static String resultTableToString(String tag, ResultTable table) {
+ System.out.println("\n" + tag + ":");
+ List<Object[]> rows = table.getRows();
+
+ StringBuffer buffer = new StringBuffer();
+ for (Object[] row : rows) {
+ buffer.append("result.add(new Object[]{\"" + row[0] + "\", " + row[1] + ", " + row[2] + "});\n");
+ }
+
+ return buffer.toString();
+ }
+
+ @Test
+ public void testSelect() {
+ String query1 = "EXPLAIN PLAN FOR SELECT * FROM testTable";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(
+ new Object[]{"SELECT(selectList:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, jsonIndexCol1, "
+ + "noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, "
+ + "jsonIndexCol1, noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 3, 2});
+ result1.add(
+ new Object[]{"PROJECT(sortedIndexCol1, noIndexCol3, rangeIndexCol1, rangeIndexCol2, jsonIndexCol1, "
+ + "invertedIndexCol1, noIndexCol2, invertedIndexCol2, noIndexCol1, invertedIndexCol3, rangeIndexCol3, "
+ + "textIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 = "EXPLAIN PLAN FOR SELECT 'mickey' FROM testTable";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:'mickey')", 2, 1});
+ result2.add(new Object[]{"PROJECT(ALL)", 3, 2});
+ result2.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 4, 3});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+
+ String query3 = "EXPLAIN PLAN FOR SELECT invertedIndexCol1, noIndexCol1 FROM testTable LIMIT 100";
+ List<Object[]> result3 = new ArrayList<>();
+ result3.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result3.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result3.add(new Object[]{"SELECT(selectList:invertedIndexCol1, noIndexCol1)", 2, 1});
+ result3.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result3.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result3.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query3, new ResultTable(DATA_SCHEMA, result3));
+
+ String query4 = "EXPLAIN PLAN FOR SELECT DISTINCT invertedIndexCol1, noIndexCol1 FROM testTable LIMIT 100";
+ List<Object[]> result4 = new ArrayList<>();
+ result4.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result4.add(new Object[]{"COMBINE_DISTINCT", 1, 0});
+ result4.add(new Object[]{"DISTINCT(keyColumns:invertedIndexCol1, noIndexCol1)", 2, 1});
+ result4.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result4.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result4.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query4, new ResultTable(DATA_SCHEMA, result4));
+ }
+
+ @Test
+ public void testSelectColumnsUsingFilterIndex() {
+ String query1 = "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1, sortedIndexCol1 FROM testTable WHERE "
+ + "invertedIndexCol1 = 1.5 AND sortedIndexCol1 = 1 LIMIT 100";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1, sortedIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, sortedIndexCol1)", 3, 2});
+ result1.add(new Object[]{"PROJECT(sortedIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 = "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1, sortedIndexCol1 FROM testTable WHERE "
+ + "(invertedIndexCol1 = 1.5 AND sortedIndexCol1 = 1) OR (invertedIndexCol1 BETWEEN 100 and 200 AND "
+ + "rangeIndexCol1 > 20)";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1, sortedIndexCol1)", 2, 1});
+ result2.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, sortedIndexCol1)", 3, 2});
+ result2.add(new Object[]{"PROJECT(sortedIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result2.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+
+ String query3 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1 FROM testTable WHERE invertedIndexCol1 = 1.5 AND "
+ + "invertedIndexCol2 IN (10, 20, 30) AND invertedIndexCol3 NOT IN ('foo', 'bar') LIMIT 100";
+ List<Object[]> result3 = new ArrayList<>();
+ result3.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result3.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result3.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1)", 2, 1});
+ result3.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result3.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result3.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query3, new ResultTable(DATA_SCHEMA, result3));
+
+ String query4 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, noIndexCol2, sortedIndexCol1 FROM testTable WHERE sortedIndexCol1 = 1.5"
+ + " OR sortedIndexCol1 != 5 OR sortedIndexCol1 IN (10, 20, 30) LIMIT 100";
+ List<Object[]> result4 = new ArrayList<>();
+ result4.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result4.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result4.add(new Object[]{"SELECT(selectList:noIndexCol1, noIndexCol2, sortedIndexCol1)", 2, 1});
+ result4.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2, sortedIndexCol1)", 3, 2});
+ result4.add(new Object[]{"PROJECT(sortedIndexCol1, noIndexCol2, noIndexCol1)", 4, 3});
+ result4.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query4, new ResultTable(DATA_SCHEMA, result4));
+
+ String query5 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, noIndexCol2 FROM testTable WHERE DIV(noIndexCol1, noIndexCol2) BETWEEN "
+ + "10 AND 20 AND invertedIndexCol1 * 5 < 1000";
+ List<Object[]> result5 = new ArrayList<>();
+ result5.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result5.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result5.add(new Object[]{"SELECT(selectList:noIndexCol1, noIndexCol2)", 2, 1});
+ result5.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2)", 3, 2});
+ result5.add(new Object[]{"PROJECT(noIndexCol2, noIndexCol1)", 4, 3});
+ result5.add(new Object[]{"FILTER_AND", 5, 4});
+ result5.add(
+ new Object[]{"FILTER_EXPRESSION(operator:RANGE,predicate:div(noIndexCol1,noIndexCol2) BETWEEN '10' AND '20')",
+ 6, 5});
+ result5
+ .add(new Object[]{"FILTER_EXPRESSION(operator:RANGE,predicate:times(invertedIndexCol1,'5') < '1000')", 7, 5});
+ check(query5, new ResultTable(DATA_SCHEMA, result5));
+ }
+
+ @Test
+ public void testSelectColumnUsingFilterRange() {
+ // select * query triggering range index
+ // checks using RANGE (>, >=, <, <=, BETWEEN ..) on a column with range index should use RANGE_INDEX_SCAN
+ String query1 =
+ "EXPLAIN PLAN FOR SELECT invertedIndexCol1, noIndexCol1, rangeIndexCol1 FROM testTable WHERE rangeIndexCol1 >"
+ + " 10 AND rangeIndexCol2 >= 20 AND rangeIndexCol3 BETWEEN 50 AND 60 LIMIT 100";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(new Object[]{"SELECT(selectList:invertedIndexCol1, noIndexCol1, rangeIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, rangeIndexCol1)", 3, 2});
+ result1.add(new Object[]{"PROJECT(rangeIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1 FROM testTable WHERE noIndexCol1 > 10 AND noIndexCol2"
+ + " BETWEEN 50 AND 60 LIMIT 100";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1)", 2, 1});
+ result2.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result2.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result2.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+ }
+
+ @Test
+ public void testSelectColumnUsingFilterJson() {
+ String query =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1 FROM testTable WHERE (invertedIndexCol1 IN (10, 20, "
+ + "30) AND sortedIndexCol1 != 100) OR (noIndexCol1 NOT IN (10, 20, 30) AND rangeIndexCol1 != 20 AND "
+ + "JSON_MATCH(jsonIndexCol1, 'key=1') AND TEXT_MATCH(textIndexCol1, 'foo'))";
+ List<Object[]> result = new ArrayList<>();
+ result.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1)", 2, 1});
+ result.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result.add(new Object[]{"FILTER_AND", 5, 4});
+ result.add(new Object[]{"FILTER_JSON_INDEX(indexLookUp:json_index,operator:JSON_MATCH,predicate:json_match"
+ + "(jsonIndexCol1,'key=1'))", 6, 5});
+ result.add(new Object[]{"FILTER_TEXT_INDEX(indexLookUp:text_index,operator:TEXT_MATCH,predicate:text_match"
+ + "(textIndexCol1,'foo'))", 7, 5});
+ check(query, new ResultTable(DATA_SCHEMA, result));
+ }
+
+ @Test
+ public void testSelectAggregateUsingFilterText() {
+ String query1 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, noIndexCol2, max(noIndexCol2), min(noIndexCol3) FROM testTable WHERE "
+ + "TEXT_MATCH(textIndexCol1, 'foo') GROUP BY noIndexCol1, noIndexCol2";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result1.add(new Object[]{"COMBINE_GROUPBY_ORDERBY", 1, 0});
+ result1.add(new Object[]{
+ "AGGREGATE_GROUPBY_ORDERBY(groupKeys:noIndexCol1, noIndexCol2, aggregations:max(noIndexCol2), min"
+ + "(noIndexCol3))", 2, 1});
+ result1.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2, noIndexCol3)", 3, 2});
+ result1.add(new Object[]{"PROJECT(noIndexCol3, noIndexCol2, noIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_TEXT_INDEX(indexLookUp:text_index,operator:TEXT_MATCH,predicate:text_match"
+ + "(textIndexCol1,'foo'))", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, max(noIndexCol2), min(noIndexCol3) FROM testTable WHERE TEXT_MATCH"
+ + "(textIndexCol1, 'foo') GROUP BY noIndexCol1, noIndexCol2 ORDER BY noIndexCol1, max(noIndexCol2)";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(sort:[noIndexCol1 ASC, max(noIndexCol2) ASC],limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_GROUPBY_ORDERBY", 1, 0});
+ result2.add(new Object[]{
+ "AGGREGATE_GROUPBY_ORDERBY(groupKeys:noIndexCol1,noIndexCol2,aggregations:max(noIndexCol2), min"
+ + "(noIndexCol3))", 2, 1});
+ result2.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, invertedIndexCol1)", 3, 2});
+ result2.add(new Object[]{"PROJECT(noIndexCol3, noIndexCol2, noIndexCol1)", 4, 3});
+ result2.add(new Object[]{"FILTER_TEXT_INDEX(indexLookUp:text_index,operator:TEXT_MATCH,predicate:text_match"
+ + "(textIndexCol1,'foo'))", 5, 4});
+ }
+
+ @Test
+ public void testSelectAggregate() {
+ String query1 = "EXPLAIN PLAN FOR SELECT count(*) FROM testTable";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result1.add(new Object[]{"COMBINE_AGGREGATE", 1, 0});
+ result1.add(new Object[]{"AGGREGATE_METADATA", 2, 1});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 =
+ "EXPLAIN PLAN FOR SELECT count(*), max(noIndexCol1), sum(noIndexCol2), avg(noIndexCol2) FROM testTable";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_AGGREGATE", 1, 0});
+ result2.add(
+ new Object[]{"AGGREGATE(aggregations:count(*), max(noIndexCol1), sum(noIndexCol2), avg(noIndexCol2))", 2, 1});
+ result2.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2)", 3, 2});
+ result2.add(new Object[]{"PROJECT(noIndexCol2, noIndexCol1)", 4, 3});
+ result2.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+ }
+
+ @Test
+ public void testSelectAggregateUsingFilter() {
+ String query = "EXPLAIN PLAN FOR SELECT sum(add(noIndexCol1, noIndexCol2)), MIN(ADD(DIV(noIndexCol1,noIndexCol2),"
+ + "noIndexCol3)) FROM testTable";
+ List<Object[]> result = new ArrayList<>();
+ result.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result.add(new Object[]{"COMBINE_AGGREGATE", 1, 0});
+ result.add(new Object[]{
+ "AGGREGATE(aggregations:sum(add(noIndexCol1,noIndexCol2)), min(add(div(noIndexCol1,noIndexCol2),"
+ + "noIndexCol3)))", 2, 1});
+ result.add(new Object[]{"PROJECT(noIndexCol3, noIndexCol2, noIndexCol1)", 3, 2});
+ result.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 4, 3});
+ check(query, new ResultTable(DATA_SCHEMA, result));
+ }
+
+ @Test
+ public void testSelectAggregateUsingFilterGroupBy() {
+ String query =
+ "EXPLAIN PLAN FOR SELECT noIndexCol2, sum(add(noIndexCol1, noIndexCol2)), min(noIndexCol3) FROM testTable "
+ + "WHERE noIndexCol1 < 100 GROUP BY noIndexCol2";
+ List<Object[]> result = new ArrayList<>();
+ result.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result.add(new Object[]{"COMBINE_GROUPBY_ORDERBY", 1, 0});
+ result.add(new Object[]{
+ "AGGREGATE_GROUPBY_ORDERBY(groupKeys:noIndexCol2, aggregations:sum(add(noIndexCol1,noIndexCol2)),"
+ + " min(noIndexCol3))", 2, 1});
+ result.add(new Object[]{"PROJECT(noIndexCol3, noIndexCol2, noIndexCol1)", 3, 2});
+ result.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 4, 3});
+ check(query, new ResultTable(DATA_SCHEMA, result));
+ }
+
+ @Test
+ public void testSelectAggregateUsingFilterIndex() {
+ String query =
+ "EXPLAIN PLAN FOR SELECT count(*), max(noIndexCol1), sum(noIndexCol2), avg(noIndexCol3) FROM testTable WHERE "
+ + "invertedIndexCol1 = 1 AND col1 = 20";
+ List<Object[]> result = new ArrayList<>();
+ result.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result.add(new Object[]{"COMBINE_AGGREGATE", 1, 0});
+ result.add(
+ new Object[]{"AGGREGATE(aggregations:count(*), max(noIndexCol1), sum(noIndexCol2), avg(noIndexCol3))", 2, 1});
+ result.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2, noIndexCol3)", 3, 2});
+ result.add(new Object[]{"PROJECT(noIndexCol3, noIndexCol2, noIndexCol1)", 4, 3});
+ result.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query, new ResultTable(DATA_SCHEMA, result));
+ }
+
+ @Test
+ public void testSelectAggregateUsingFilterIndexGroupBy() {
+ String query = "EXPLAIN PLAN FOR SELECT noIndexCol1, max(noIndexCol2), min(noIndexCol3) FROM testTable WHERE "
+ + "invertedIndexCol2 = 10 GROUP BY noIndexCol1";
+ List<Object[]> result = new ArrayList<>();
+ result.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result.add(new Object[]{"COMBINE_GROUPBY_ORDERBY", 1, 0});
+ result.add(new Object[]{
+ "AGGREGATE_GROUPBY_ORDERBY(groupKeys:noIndexCol1, aggregations:max(noIndexCol2), min(noIndexCol3)" + ")", 2,
+ 1});
+ result.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2, noIndexCol3)", 3, 2});
+ result.add(new Object[]{"PROJECT(noIndexCol3, noIndexCol2, noIndexCol1)", 4, 3});
+ result.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query, new ResultTable(DATA_SCHEMA, result));
+ }
+
+ @Test
+ public void testSelectAggregateUsingFilterIndexGroupByHaving() {
Review comment:
Please add following tests to improve coverage of this functionality
- Transform in filter. Example - `AND arrayLength(col) > 2 " + "AND invertedIndexCol1 * 5 < 1000`
- Transform in select list and GROUP BY and ORDER BY. The plan should show the actual transformation happening. Right now the only transform we show is the PASS_THROUGH transform on identifiers
- Transform in select list inside aggregations. You have this but transform part is skipped from the plan which is not correct
##########
File path: pinot-core/src/test/java/org/apache/pinot/queries/ExplainPlanQueriesTest.java
##########
@@ -0,0 +1,510 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.queries;
+
+import java.io.File;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.List;
+import org.apache.commons.io.FileUtils;
+import org.apache.pinot.common.response.broker.BrokerResponseNative;
+import org.apache.pinot.common.response.broker.ResultTable;
+import org.apache.pinot.common.utils.DataSchema;
+import org.apache.pinot.segment.local.indexsegment.immutable.ImmutableSegmentLoader;
+import org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl;
+import org.apache.pinot.segment.local.segment.index.loader.IndexLoadingConfig;
+import org.apache.pinot.segment.local.segment.readers.GenericRowRecordReader;
+import org.apache.pinot.segment.spi.ImmutableSegment;
+import org.apache.pinot.segment.spi.IndexSegment;
+import org.apache.pinot.segment.spi.creator.SegmentGeneratorConfig;
+import org.apache.pinot.spi.config.table.IndexingConfig;
+import org.apache.pinot.spi.config.table.TableConfig;
+import org.apache.pinot.spi.config.table.TableType;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.data.Schema;
+import org.apache.pinot.spi.data.readers.GenericRow;
+import org.apache.pinot.spi.utils.ReadMode;
+import org.apache.pinot.spi.utils.builder.TableConfigBuilder;
+import org.testng.Assert;
+import org.testng.annotations.BeforeClass;
+import org.testng.annotations.Test;
+
+
+public class ExplainPlanQueriesTest extends BaseQueriesTest {
+ private static final File INDEX_DIR = new File(FileUtils.getTempDirectory(), "ExplainPlanQueriesTest");
+ private static final String RAW_TABLE_NAME = "testTable";
+ private static final String SEGMENT_NAME = "testSegment";
+ private static final int NUM_RECORDS = 10;
+
+ private final static String COL1_NO_INDEX = "noIndexCol1";
+ private final static String COL2_NO_INDEX = "noIndexCol2";
+ private final static String COL3_NO_INDEX = "noIndexCol3";
+ private final static String COL1_INVERTED_INDEX = "invertedIndexCol1";
+ private final static String COL2_INVERTED_INDEX = "invertedIndexCol2";
+ private final static String COL3_INVERTED_INDEX = "invertedIndexCol3";
+ private final static String COL1_RANGE_INDEX = "rangeIndexCol1";
+ private final static String COL2_RANGE_INDEX = "rangeIndexCol2";
+ private final static String COL3_RANGE_INDEX = "rangeIndexCol3";
+ private final static String COL1_SORTED_INDEX = "sortedIndexCol1";
+ private final static String COL1_JSON_INDEX = "jsonIndexCol1";
+ private final static String COL1_TEXT_INDEX = "textIndexCol1";
+
+ private static final Schema SCHEMA = new Schema.SchemaBuilder().setSchemaName(RAW_TABLE_NAME)
+ .addSingleValueDimension(COL1_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL2_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_INVERTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_INVERTED_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_INVERTED_INDEX, FieldSpec.DataType.STRING)
+ .addSingleValueDimension(COL1_RANGE_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_SORTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL1_JSON_INDEX, FieldSpec.DataType.JSON)
+ .addSingleValueDimension(COL1_TEXT_INDEX, FieldSpec.DataType.STRING).build();
+
+ private static final DataSchema DATA_SCHEMA = new DataSchema(new String[]{"Operator", "Operator_Id", "Parent_Id"},
+ new DataSchema.ColumnDataType[]{DataSchema.ColumnDataType.STRING, DataSchema.ColumnDataType.INT,
+ DataSchema.ColumnDataType.INT});
+
+ private static final TableConfig TABLE_CONFIG =
+ new TableConfigBuilder(TableType.OFFLINE).setTableName(RAW_TABLE_NAME).build();
+
+ private IndexSegment _indexSegment;
+ private List<IndexSegment> _indexSegments;
+
+ @Override
+ protected String getFilter() {
+ return "";
+ }
+
+ @Override
+ protected IndexSegment getIndexSegment() {
+ return _indexSegment;
+ }
+
+ @Override
+ protected List<IndexSegment> getIndexSegments() {
+ return _indexSegments;
+ }
+
+ GenericRow createRecord(double doubleValue, int intValue, String stringValue, String jsonValue) {
+ GenericRow record = new GenericRow();
+ record.putValue(COL1_NO_INDEX, intValue);
+ record.putValue(COL2_NO_INDEX, intValue);
+ record.putValue(COL3_NO_INDEX, intValue);
+
+ record.putValue(COL1_INVERTED_INDEX, doubleValue);
+ record.putValue(COL2_INVERTED_INDEX, intValue);
+ record.putValue(COL3_INVERTED_INDEX, intValue);
+
+ record.putValue(COL1_RANGE_INDEX, doubleValue);
+ record.putValue(COL2_RANGE_INDEX, intValue);
+ record.putValue(COL3_RANGE_INDEX, intValue);
+
+ record.putValue(COL1_JSON_INDEX, jsonValue);
+ record.putValue(COL1_TEXT_INDEX, stringValue);
+
+ return record;
+ }
+
+ @BeforeClass
+ public void setUp()
+ throws Exception {
+ FileUtils.deleteDirectory(INDEX_DIR);
+
+ List<GenericRow> records = new ArrayList<>(NUM_RECORDS);
+ records.add(createRecord(1.1, 2, "daffy", "{\"first\": \"daffy\", \"last\": \"duck\"}"));
+
+ IndexingConfig indexingConfig = TABLE_CONFIG.getIndexingConfig();
+
+ List<String> invertedIndexColumns = Arrays.asList(COL1_INVERTED_INDEX, COL2_INVERTED_INDEX, COL3_INVERTED_INDEX);
+ indexingConfig.setInvertedIndexColumns(invertedIndexColumns);
+
+ List<String> rangeIndexColumns = Arrays.asList(COL1_RANGE_INDEX, COL2_RANGE_INDEX, COL3_RANGE_INDEX);
+ indexingConfig.setRangeIndexColumns(rangeIndexColumns);
+
+ List<String> jsonIndexColumns = Arrays.asList(COL1_JSON_INDEX);
+ indexingConfig.setJsonIndexColumns(jsonIndexColumns);
+
+ SegmentGeneratorConfig segmentGeneratorConfig = new SegmentGeneratorConfig(TABLE_CONFIG, SCHEMA);
+ segmentGeneratorConfig.setTableName(RAW_TABLE_NAME);
+ segmentGeneratorConfig.setSegmentName(SEGMENT_NAME);
+ segmentGeneratorConfig.setOutDir(INDEX_DIR.getPath());
+
+ SegmentIndexCreationDriverImpl driver = new SegmentIndexCreationDriverImpl();
+ driver.init(segmentGeneratorConfig, new GenericRowRecordReader(records));
+ driver.build();
+
+ IndexLoadingConfig indexLoadingConfig = new IndexLoadingConfig();
+ indexLoadingConfig.setTableConfig(TABLE_CONFIG);
+ indexLoadingConfig.setInvertedIndexColumns(new HashSet<>(invertedIndexColumns));
+ indexLoadingConfig.setRangeIndexColumns(new HashSet<>(rangeIndexColumns));
+ indexLoadingConfig.setJsonIndexColumns(new HashSet<String>(jsonIndexColumns));
+ indexLoadingConfig.setReadMode(ReadMode.mmap);
+
+ ImmutableSegment immutableSegment =
+ ImmutableSegmentLoader.load(new File(INDEX_DIR, SEGMENT_NAME), indexLoadingConfig);
+ _indexSegment = immutableSegment;
+ _indexSegments = Arrays.asList(immutableSegment);
+ }
+
+ /** Checks the correctness of EXPLAIN PLAN output. */
+ private void check(String query, ResultTable expected) {
+ BrokerResponseNative response = getBrokerResponseForSqlQuery(query);
+ ResultTable actual = response.getResultTable();
+
+ System.out.println(resultTableToString("Expected", expected));
+ System.out.println(resultTableToString("Actual", actual));
+
+ Assert.assertEquals(actual.getDataSchema(), expected.getDataSchema());
+ Assert.assertEquals(actual.getRows().size(), expected.getRows().size());
+
+ List<Object[]> expectedRows = expected.getRows();
+ List<Object[]> actualRows = actual.getRows();
+ for (int i = 0; i < expectedRows.size(); i++) {
+ Object[] expectedRow = expectedRows.get(i);
+ Object[] actualRow = actualRows.get(i);
+ Assert.assertEquals(actualRow.length, expectedRow.length);
+
+ for (int j = 0; j < actualRow.length; j++) {
+ Assert.assertEquals(actualRow[j], expectedRow[j]);
+ }
+ }
+ }
+
+ /** This function is used to generated expected results while creating test cases; otherwise, not used. */
+ private static String resultTableToString(String tag, ResultTable table) {
+ System.out.println("\n" + tag + ":");
+ List<Object[]> rows = table.getRows();
+
+ StringBuffer buffer = new StringBuffer();
+ for (Object[] row : rows) {
+ buffer.append("result.add(new Object[]{\"" + row[0] + "\", " + row[1] + ", " + row[2] + "});\n");
+ }
+
+ return buffer.toString();
+ }
+
+ @Test
+ public void testSelect() {
+ String query1 = "EXPLAIN PLAN FOR SELECT * FROM testTable";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(
+ new Object[]{"SELECT(selectList:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, jsonIndexCol1, "
+ + "noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, "
+ + "jsonIndexCol1, noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 3, 2});
+ result1.add(
+ new Object[]{"PROJECT(sortedIndexCol1, noIndexCol3, rangeIndexCol1, rangeIndexCol2, jsonIndexCol1, "
+ + "invertedIndexCol1, noIndexCol2, invertedIndexCol2, noIndexCol1, invertedIndexCol3, rangeIndexCol3, "
+ + "textIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 = "EXPLAIN PLAN FOR SELECT 'mickey' FROM testTable";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:'mickey')", 2, 1});
+ result2.add(new Object[]{"PROJECT(ALL)", 3, 2});
+ result2.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 4, 3});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+
+ String query3 = "EXPLAIN PLAN FOR SELECT invertedIndexCol1, noIndexCol1 FROM testTable LIMIT 100";
+ List<Object[]> result3 = new ArrayList<>();
+ result3.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result3.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result3.add(new Object[]{"SELECT(selectList:invertedIndexCol1, noIndexCol1)", 2, 1});
+ result3.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result3.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result3.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query3, new ResultTable(DATA_SCHEMA, result3));
+
+ String query4 = "EXPLAIN PLAN FOR SELECT DISTINCT invertedIndexCol1, noIndexCol1 FROM testTable LIMIT 100";
+ List<Object[]> result4 = new ArrayList<>();
+ result4.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result4.add(new Object[]{"COMBINE_DISTINCT", 1, 0});
+ result4.add(new Object[]{"DISTINCT(keyColumns:invertedIndexCol1, noIndexCol1)", 2, 1});
+ result4.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result4.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result4.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query4, new ResultTable(DATA_SCHEMA, result4));
+ }
+
+ @Test
+ public void testSelectColumnsUsingFilterIndex() {
+ String query1 = "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1, sortedIndexCol1 FROM testTable WHERE "
+ + "invertedIndexCol1 = 1.5 AND sortedIndexCol1 = 1 LIMIT 100";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1, sortedIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, sortedIndexCol1)", 3, 2});
+ result1.add(new Object[]{"PROJECT(sortedIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_EMPTY", 5, 4});
Review comment:
Can we generate data that will actually correspond to this test ?
For this query, I was expecting a root AND filter operator with 2 child leaf operators for inv index and sorted index lookup. But, we instead see FILTER_EMPTY which doesn't seem to achieve the purpose of this test
##########
File path: pinot-core/src/test/java/org/apache/pinot/queries/ExplainPlanQueriesTest.java
##########
@@ -0,0 +1,510 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.queries;
+
+import java.io.File;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.List;
+import org.apache.commons.io.FileUtils;
+import org.apache.pinot.common.response.broker.BrokerResponseNative;
+import org.apache.pinot.common.response.broker.ResultTable;
+import org.apache.pinot.common.utils.DataSchema;
+import org.apache.pinot.segment.local.indexsegment.immutable.ImmutableSegmentLoader;
+import org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl;
+import org.apache.pinot.segment.local.segment.index.loader.IndexLoadingConfig;
+import org.apache.pinot.segment.local.segment.readers.GenericRowRecordReader;
+import org.apache.pinot.segment.spi.ImmutableSegment;
+import org.apache.pinot.segment.spi.IndexSegment;
+import org.apache.pinot.segment.spi.creator.SegmentGeneratorConfig;
+import org.apache.pinot.spi.config.table.IndexingConfig;
+import org.apache.pinot.spi.config.table.TableConfig;
+import org.apache.pinot.spi.config.table.TableType;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.data.Schema;
+import org.apache.pinot.spi.data.readers.GenericRow;
+import org.apache.pinot.spi.utils.ReadMode;
+import org.apache.pinot.spi.utils.builder.TableConfigBuilder;
+import org.testng.Assert;
+import org.testng.annotations.BeforeClass;
+import org.testng.annotations.Test;
+
+
+public class ExplainPlanQueriesTest extends BaseQueriesTest {
+ private static final File INDEX_DIR = new File(FileUtils.getTempDirectory(), "ExplainPlanQueriesTest");
+ private static final String RAW_TABLE_NAME = "testTable";
+ private static final String SEGMENT_NAME = "testSegment";
+ private static final int NUM_RECORDS = 10;
+
+ private final static String COL1_NO_INDEX = "noIndexCol1";
+ private final static String COL2_NO_INDEX = "noIndexCol2";
+ private final static String COL3_NO_INDEX = "noIndexCol3";
+ private final static String COL1_INVERTED_INDEX = "invertedIndexCol1";
+ private final static String COL2_INVERTED_INDEX = "invertedIndexCol2";
+ private final static String COL3_INVERTED_INDEX = "invertedIndexCol3";
+ private final static String COL1_RANGE_INDEX = "rangeIndexCol1";
+ private final static String COL2_RANGE_INDEX = "rangeIndexCol2";
+ private final static String COL3_RANGE_INDEX = "rangeIndexCol3";
+ private final static String COL1_SORTED_INDEX = "sortedIndexCol1";
+ private final static String COL1_JSON_INDEX = "jsonIndexCol1";
+ private final static String COL1_TEXT_INDEX = "textIndexCol1";
+
+ private static final Schema SCHEMA = new Schema.SchemaBuilder().setSchemaName(RAW_TABLE_NAME)
+ .addSingleValueDimension(COL1_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL2_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_INVERTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_INVERTED_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_INVERTED_INDEX, FieldSpec.DataType.STRING)
+ .addSingleValueDimension(COL1_RANGE_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_SORTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL1_JSON_INDEX, FieldSpec.DataType.JSON)
+ .addSingleValueDimension(COL1_TEXT_INDEX, FieldSpec.DataType.STRING).build();
+
+ private static final DataSchema DATA_SCHEMA = new DataSchema(new String[]{"Operator", "Operator_Id", "Parent_Id"},
+ new DataSchema.ColumnDataType[]{DataSchema.ColumnDataType.STRING, DataSchema.ColumnDataType.INT,
+ DataSchema.ColumnDataType.INT});
+
+ private static final TableConfig TABLE_CONFIG =
+ new TableConfigBuilder(TableType.OFFLINE).setTableName(RAW_TABLE_NAME).build();
+
+ private IndexSegment _indexSegment;
+ private List<IndexSegment> _indexSegments;
+
+ @Override
+ protected String getFilter() {
+ return "";
+ }
+
+ @Override
+ protected IndexSegment getIndexSegment() {
+ return _indexSegment;
+ }
+
+ @Override
+ protected List<IndexSegment> getIndexSegments() {
+ return _indexSegments;
+ }
+
+ GenericRow createRecord(double doubleValue, int intValue, String stringValue, String jsonValue) {
+ GenericRow record = new GenericRow();
+ record.putValue(COL1_NO_INDEX, intValue);
+ record.putValue(COL2_NO_INDEX, intValue);
+ record.putValue(COL3_NO_INDEX, intValue);
+
+ record.putValue(COL1_INVERTED_INDEX, doubleValue);
+ record.putValue(COL2_INVERTED_INDEX, intValue);
+ record.putValue(COL3_INVERTED_INDEX, intValue);
+
+ record.putValue(COL1_RANGE_INDEX, doubleValue);
+ record.putValue(COL2_RANGE_INDEX, intValue);
+ record.putValue(COL3_RANGE_INDEX, intValue);
+
+ record.putValue(COL1_JSON_INDEX, jsonValue);
+ record.putValue(COL1_TEXT_INDEX, stringValue);
+
+ return record;
+ }
+
+ @BeforeClass
+ public void setUp()
+ throws Exception {
+ FileUtils.deleteDirectory(INDEX_DIR);
+
+ List<GenericRow> records = new ArrayList<>(NUM_RECORDS);
+ records.add(createRecord(1.1, 2, "daffy", "{\"first\": \"daffy\", \"last\": \"duck\"}"));
+
+ IndexingConfig indexingConfig = TABLE_CONFIG.getIndexingConfig();
+
+ List<String> invertedIndexColumns = Arrays.asList(COL1_INVERTED_INDEX, COL2_INVERTED_INDEX, COL3_INVERTED_INDEX);
+ indexingConfig.setInvertedIndexColumns(invertedIndexColumns);
+
+ List<String> rangeIndexColumns = Arrays.asList(COL1_RANGE_INDEX, COL2_RANGE_INDEX, COL3_RANGE_INDEX);
+ indexingConfig.setRangeIndexColumns(rangeIndexColumns);
+
+ List<String> jsonIndexColumns = Arrays.asList(COL1_JSON_INDEX);
+ indexingConfig.setJsonIndexColumns(jsonIndexColumns);
+
+ SegmentGeneratorConfig segmentGeneratorConfig = new SegmentGeneratorConfig(TABLE_CONFIG, SCHEMA);
+ segmentGeneratorConfig.setTableName(RAW_TABLE_NAME);
+ segmentGeneratorConfig.setSegmentName(SEGMENT_NAME);
+ segmentGeneratorConfig.setOutDir(INDEX_DIR.getPath());
+
+ SegmentIndexCreationDriverImpl driver = new SegmentIndexCreationDriverImpl();
+ driver.init(segmentGeneratorConfig, new GenericRowRecordReader(records));
+ driver.build();
+
+ IndexLoadingConfig indexLoadingConfig = new IndexLoadingConfig();
+ indexLoadingConfig.setTableConfig(TABLE_CONFIG);
+ indexLoadingConfig.setInvertedIndexColumns(new HashSet<>(invertedIndexColumns));
+ indexLoadingConfig.setRangeIndexColumns(new HashSet<>(rangeIndexColumns));
+ indexLoadingConfig.setJsonIndexColumns(new HashSet<String>(jsonIndexColumns));
+ indexLoadingConfig.setReadMode(ReadMode.mmap);
+
+ ImmutableSegment immutableSegment =
+ ImmutableSegmentLoader.load(new File(INDEX_DIR, SEGMENT_NAME), indexLoadingConfig);
+ _indexSegment = immutableSegment;
+ _indexSegments = Arrays.asList(immutableSegment);
+ }
+
+ /** Checks the correctness of EXPLAIN PLAN output. */
+ private void check(String query, ResultTable expected) {
+ BrokerResponseNative response = getBrokerResponseForSqlQuery(query);
+ ResultTable actual = response.getResultTable();
+
+ System.out.println(resultTableToString("Expected", expected));
+ System.out.println(resultTableToString("Actual", actual));
+
+ Assert.assertEquals(actual.getDataSchema(), expected.getDataSchema());
+ Assert.assertEquals(actual.getRows().size(), expected.getRows().size());
+
+ List<Object[]> expectedRows = expected.getRows();
+ List<Object[]> actualRows = actual.getRows();
+ for (int i = 0; i < expectedRows.size(); i++) {
+ Object[] expectedRow = expectedRows.get(i);
+ Object[] actualRow = actualRows.get(i);
+ Assert.assertEquals(actualRow.length, expectedRow.length);
+
+ for (int j = 0; j < actualRow.length; j++) {
+ Assert.assertEquals(actualRow[j], expectedRow[j]);
+ }
+ }
+ }
+
+ /** This function is used to generated expected results while creating test cases; otherwise, not used. */
+ private static String resultTableToString(String tag, ResultTable table) {
+ System.out.println("\n" + tag + ":");
+ List<Object[]> rows = table.getRows();
+
+ StringBuffer buffer = new StringBuffer();
+ for (Object[] row : rows) {
+ buffer.append("result.add(new Object[]{\"" + row[0] + "\", " + row[1] + ", " + row[2] + "});\n");
+ }
+
+ return buffer.toString();
+ }
+
+ @Test
+ public void testSelect() {
+ String query1 = "EXPLAIN PLAN FOR SELECT * FROM testTable";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(
+ new Object[]{"SELECT(selectList:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, jsonIndexCol1, "
+ + "noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, "
+ + "jsonIndexCol1, noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 3, 2});
+ result1.add(
+ new Object[]{"PROJECT(sortedIndexCol1, noIndexCol3, rangeIndexCol1, rangeIndexCol2, jsonIndexCol1, "
+ + "invertedIndexCol1, noIndexCol2, invertedIndexCol2, noIndexCol1, invertedIndexCol3, rangeIndexCol3, "
+ + "textIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 = "EXPLAIN PLAN FOR SELECT 'mickey' FROM testTable";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:'mickey')", 2, 1});
+ result2.add(new Object[]{"PROJECT(ALL)", 3, 2});
+ result2.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 4, 3});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+
+ String query3 = "EXPLAIN PLAN FOR SELECT invertedIndexCol1, noIndexCol1 FROM testTable LIMIT 100";
+ List<Object[]> result3 = new ArrayList<>();
+ result3.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result3.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result3.add(new Object[]{"SELECT(selectList:invertedIndexCol1, noIndexCol1)", 2, 1});
+ result3.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result3.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result3.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query3, new ResultTable(DATA_SCHEMA, result3));
+
+ String query4 = "EXPLAIN PLAN FOR SELECT DISTINCT invertedIndexCol1, noIndexCol1 FROM testTable LIMIT 100";
+ List<Object[]> result4 = new ArrayList<>();
+ result4.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result4.add(new Object[]{"COMBINE_DISTINCT", 1, 0});
+ result4.add(new Object[]{"DISTINCT(keyColumns:invertedIndexCol1, noIndexCol1)", 2, 1});
+ result4.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result4.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result4.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query4, new ResultTable(DATA_SCHEMA, result4));
+ }
+
+ @Test
+ public void testSelectColumnsUsingFilterIndex() {
+ String query1 = "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1, sortedIndexCol1 FROM testTable WHERE "
+ + "invertedIndexCol1 = 1.5 AND sortedIndexCol1 = 1 LIMIT 100";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1, sortedIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, sortedIndexCol1)", 3, 2});
+ result1.add(new Object[]{"PROJECT(sortedIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 = "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1, sortedIndexCol1 FROM testTable WHERE "
+ + "(invertedIndexCol1 = 1.5 AND sortedIndexCol1 = 1) OR (invertedIndexCol1 BETWEEN 100 and 200 AND "
+ + "rangeIndexCol1 > 20)";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1, sortedIndexCol1)", 2, 1});
+ result2.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, sortedIndexCol1)", 3, 2});
+ result2.add(new Object[]{"PROJECT(sortedIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result2.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+
+ String query3 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1 FROM testTable WHERE invertedIndexCol1 = 1.5 AND "
+ + "invertedIndexCol2 IN (10, 20, 30) AND invertedIndexCol3 NOT IN ('foo', 'bar') LIMIT 100";
+ List<Object[]> result3 = new ArrayList<>();
+ result3.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result3.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result3.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1)", 2, 1});
+ result3.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result3.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result3.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query3, new ResultTable(DATA_SCHEMA, result3));
+
+ String query4 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, noIndexCol2, sortedIndexCol1 FROM testTable WHERE sortedIndexCol1 = 1.5"
+ + " OR sortedIndexCol1 != 5 OR sortedIndexCol1 IN (10, 20, 30) LIMIT 100";
+ List<Object[]> result4 = new ArrayList<>();
+ result4.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result4.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result4.add(new Object[]{"SELECT(selectList:noIndexCol1, noIndexCol2, sortedIndexCol1)", 2, 1});
+ result4.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2, sortedIndexCol1)", 3, 2});
+ result4.add(new Object[]{"PROJECT(sortedIndexCol1, noIndexCol2, noIndexCol1)", 4, 3});
+ result4.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query4, new ResultTable(DATA_SCHEMA, result4));
+
+ String query5 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, noIndexCol2 FROM testTable WHERE DIV(noIndexCol1, noIndexCol2) BETWEEN "
+ + "10 AND 20 AND invertedIndexCol1 * 5 < 1000";
+ List<Object[]> result5 = new ArrayList<>();
+ result5.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result5.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result5.add(new Object[]{"SELECT(selectList:noIndexCol1, noIndexCol2)", 2, 1});
+ result5.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2)", 3, 2});
+ result5.add(new Object[]{"PROJECT(noIndexCol2, noIndexCol1)", 4, 3});
+ result5.add(new Object[]{"FILTER_AND", 5, 4});
+ result5.add(
+ new Object[]{"FILTER_EXPRESSION(operator:RANGE,predicate:div(noIndexCol1,noIndexCol2) BETWEEN '10' AND '20')",
+ 6, 5});
+ result5
+ .add(new Object[]{"FILTER_EXPRESSION(operator:RANGE,predicate:times(invertedIndexCol1,'5') < '1000')", 7, 5});
+ check(query5, new ResultTable(DATA_SCHEMA, result5));
+ }
+
+ @Test
+ public void testSelectColumnUsingFilterRange() {
+ // select * query triggering range index
+ // checks using RANGE (>, >=, <, <=, BETWEEN ..) on a column with range index should use RANGE_INDEX_SCAN
+ String query1 =
+ "EXPLAIN PLAN FOR SELECT invertedIndexCol1, noIndexCol1, rangeIndexCol1 FROM testTable WHERE rangeIndexCol1 >"
+ + " 10 AND rangeIndexCol2 >= 20 AND rangeIndexCol3 BETWEEN 50 AND 60 LIMIT 100";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(new Object[]{"SELECT(selectList:invertedIndexCol1, noIndexCol1, rangeIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, rangeIndexCol1)", 3, 2});
+ result1.add(new Object[]{"PROJECT(rangeIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1 FROM testTable WHERE noIndexCol1 > 10 AND noIndexCol2"
+ + " BETWEEN 50 AND 60 LIMIT 100";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1)", 2, 1});
+ result2.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result2.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result2.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+ }
+
+ @Test
+ public void testSelectColumnUsingFilterJson() {
+ String query =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1 FROM testTable WHERE (invertedIndexCol1 IN (10, 20, "
+ + "30) AND sortedIndexCol1 != 100) OR (noIndexCol1 NOT IN (10, 20, 30) AND rangeIndexCol1 != 20 AND "
+ + "JSON_MATCH(jsonIndexCol1, 'key=1') AND TEXT_MATCH(textIndexCol1, 'foo'))";
+ List<Object[]> result = new ArrayList<>();
+ result.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1)", 2, 1});
+ result.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result.add(new Object[]{"FILTER_AND", 5, 4});
+ result.add(new Object[]{"FILTER_JSON_INDEX(indexLookUp:json_index,operator:JSON_MATCH,predicate:json_match"
+ + "(jsonIndexCol1,'key=1'))", 6, 5});
+ result.add(new Object[]{"FILTER_TEXT_INDEX(indexLookUp:text_index,operator:TEXT_MATCH,predicate:text_match"
+ + "(textIndexCol1,'foo'))", 7, 5});
+ check(query, new ResultTable(DATA_SCHEMA, result));
+ }
+
+ @Test
+ public void testSelectAggregateUsingFilterText() {
+ String query1 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, noIndexCol2, max(noIndexCol2), min(noIndexCol3) FROM testTable WHERE "
+ + "TEXT_MATCH(textIndexCol1, 'foo') GROUP BY noIndexCol1, noIndexCol2";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result1.add(new Object[]{"COMBINE_GROUPBY_ORDERBY", 1, 0});
+ result1.add(new Object[]{
+ "AGGREGATE_GROUPBY_ORDERBY(groupKeys:noIndexCol1, noIndexCol2, aggregations:max(noIndexCol2), min"
+ + "(noIndexCol3))", 2, 1});
+ result1.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2, noIndexCol3)", 3, 2});
+ result1.add(new Object[]{"PROJECT(noIndexCol3, noIndexCol2, noIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_TEXT_INDEX(indexLookUp:text_index,operator:TEXT_MATCH,predicate:text_match"
+ + "(textIndexCol1,'foo'))", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, max(noIndexCol2), min(noIndexCol3) FROM testTable WHERE TEXT_MATCH"
+ + "(textIndexCol1, 'foo') GROUP BY noIndexCol1, noIndexCol2 ORDER BY noIndexCol1, max(noIndexCol2)";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(sort:[noIndexCol1 ASC, max(noIndexCol2) ASC],limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_GROUPBY_ORDERBY", 1, 0});
+ result2.add(new Object[]{
+ "AGGREGATE_GROUPBY_ORDERBY(groupKeys:noIndexCol1,noIndexCol2,aggregations:max(noIndexCol2), min"
+ + "(noIndexCol3))", 2, 1});
+ result2.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, invertedIndexCol1)", 3, 2});
+ result2.add(new Object[]{"PROJECT(noIndexCol3, noIndexCol2, noIndexCol1)", 4, 3});
+ result2.add(new Object[]{"FILTER_TEXT_INDEX(indexLookUp:text_index,operator:TEXT_MATCH,predicate:text_match"
+ + "(textIndexCol1,'foo'))", 5, 4});
+ }
+
+ @Test
+ public void testSelectAggregate() {
+ String query1 = "EXPLAIN PLAN FOR SELECT count(*) FROM testTable";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result1.add(new Object[]{"COMBINE_AGGREGATE", 1, 0});
+ result1.add(new Object[]{"AGGREGATE_METADATA", 2, 1});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 =
+ "EXPLAIN PLAN FOR SELECT count(*), max(noIndexCol1), sum(noIndexCol2), avg(noIndexCol2) FROM testTable";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_AGGREGATE", 1, 0});
+ result2.add(
+ new Object[]{"AGGREGATE(aggregations:count(*), max(noIndexCol1), sum(noIndexCol2), avg(noIndexCol2))", 2, 1});
+ result2.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2)", 3, 2});
+ result2.add(new Object[]{"PROJECT(noIndexCol2, noIndexCol1)", 4, 3});
+ result2.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+ }
+
+ @Test
+ public void testSelectAggregateUsingFilter() {
+ String query = "EXPLAIN PLAN FOR SELECT sum(add(noIndexCol1, noIndexCol2)), MIN(ADD(DIV(noIndexCol1,noIndexCol2),"
+ + "noIndexCol3)) FROM testTable";
+ List<Object[]> result = new ArrayList<>();
Review comment:
The query does not have a WHERE clause. Why are we calling this test `UsingFilter`
##########
File path: pinot-core/src/test/java/org/apache/pinot/queries/ExplainPlanQueriesTest.java
##########
@@ -0,0 +1,510 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.queries;
+
+import java.io.File;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.List;
+import org.apache.commons.io.FileUtils;
+import org.apache.pinot.common.response.broker.BrokerResponseNative;
+import org.apache.pinot.common.response.broker.ResultTable;
+import org.apache.pinot.common.utils.DataSchema;
+import org.apache.pinot.segment.local.indexsegment.immutable.ImmutableSegmentLoader;
+import org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl;
+import org.apache.pinot.segment.local.segment.index.loader.IndexLoadingConfig;
+import org.apache.pinot.segment.local.segment.readers.GenericRowRecordReader;
+import org.apache.pinot.segment.spi.ImmutableSegment;
+import org.apache.pinot.segment.spi.IndexSegment;
+import org.apache.pinot.segment.spi.creator.SegmentGeneratorConfig;
+import org.apache.pinot.spi.config.table.IndexingConfig;
+import org.apache.pinot.spi.config.table.TableConfig;
+import org.apache.pinot.spi.config.table.TableType;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.data.Schema;
+import org.apache.pinot.spi.data.readers.GenericRow;
+import org.apache.pinot.spi.utils.ReadMode;
+import org.apache.pinot.spi.utils.builder.TableConfigBuilder;
+import org.testng.Assert;
+import org.testng.annotations.BeforeClass;
+import org.testng.annotations.Test;
+
+
+public class ExplainPlanQueriesTest extends BaseQueriesTest {
+ private static final File INDEX_DIR = new File(FileUtils.getTempDirectory(), "ExplainPlanQueriesTest");
+ private static final String RAW_TABLE_NAME = "testTable";
+ private static final String SEGMENT_NAME = "testSegment";
+ private static final int NUM_RECORDS = 10;
+
+ private final static String COL1_NO_INDEX = "noIndexCol1";
+ private final static String COL2_NO_INDEX = "noIndexCol2";
+ private final static String COL3_NO_INDEX = "noIndexCol3";
+ private final static String COL1_INVERTED_INDEX = "invertedIndexCol1";
+ private final static String COL2_INVERTED_INDEX = "invertedIndexCol2";
+ private final static String COL3_INVERTED_INDEX = "invertedIndexCol3";
+ private final static String COL1_RANGE_INDEX = "rangeIndexCol1";
+ private final static String COL2_RANGE_INDEX = "rangeIndexCol2";
+ private final static String COL3_RANGE_INDEX = "rangeIndexCol3";
+ private final static String COL1_SORTED_INDEX = "sortedIndexCol1";
+ private final static String COL1_JSON_INDEX = "jsonIndexCol1";
+ private final static String COL1_TEXT_INDEX = "textIndexCol1";
+
+ private static final Schema SCHEMA = new Schema.SchemaBuilder().setSchemaName(RAW_TABLE_NAME)
+ .addSingleValueDimension(COL1_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL2_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_INVERTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_INVERTED_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_INVERTED_INDEX, FieldSpec.DataType.STRING)
+ .addSingleValueDimension(COL1_RANGE_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_SORTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL1_JSON_INDEX, FieldSpec.DataType.JSON)
+ .addSingleValueDimension(COL1_TEXT_INDEX, FieldSpec.DataType.STRING).build();
+
+ private static final DataSchema DATA_SCHEMA = new DataSchema(new String[]{"Operator", "Operator_Id", "Parent_Id"},
+ new DataSchema.ColumnDataType[]{DataSchema.ColumnDataType.STRING, DataSchema.ColumnDataType.INT,
+ DataSchema.ColumnDataType.INT});
+
+ private static final TableConfig TABLE_CONFIG =
+ new TableConfigBuilder(TableType.OFFLINE).setTableName(RAW_TABLE_NAME).build();
+
+ private IndexSegment _indexSegment;
+ private List<IndexSegment> _indexSegments;
+
+ @Override
+ protected String getFilter() {
+ return "";
+ }
+
+ @Override
+ protected IndexSegment getIndexSegment() {
+ return _indexSegment;
+ }
+
+ @Override
+ protected List<IndexSegment> getIndexSegments() {
+ return _indexSegments;
+ }
+
+ GenericRow createRecord(double doubleValue, int intValue, String stringValue, String jsonValue) {
+ GenericRow record = new GenericRow();
+ record.putValue(COL1_NO_INDEX, intValue);
+ record.putValue(COL2_NO_INDEX, intValue);
+ record.putValue(COL3_NO_INDEX, intValue);
+
+ record.putValue(COL1_INVERTED_INDEX, doubleValue);
+ record.putValue(COL2_INVERTED_INDEX, intValue);
+ record.putValue(COL3_INVERTED_INDEX, intValue);
+
+ record.putValue(COL1_RANGE_INDEX, doubleValue);
+ record.putValue(COL2_RANGE_INDEX, intValue);
+ record.putValue(COL3_RANGE_INDEX, intValue);
+
+ record.putValue(COL1_JSON_INDEX, jsonValue);
+ record.putValue(COL1_TEXT_INDEX, stringValue);
+
+ return record;
+ }
+
+ @BeforeClass
+ public void setUp()
+ throws Exception {
+ FileUtils.deleteDirectory(INDEX_DIR);
+
+ List<GenericRow> records = new ArrayList<>(NUM_RECORDS);
+ records.add(createRecord(1.1, 2, "daffy", "{\"first\": \"daffy\", \"last\": \"duck\"}"));
+
+ IndexingConfig indexingConfig = TABLE_CONFIG.getIndexingConfig();
+
+ List<String> invertedIndexColumns = Arrays.asList(COL1_INVERTED_INDEX, COL2_INVERTED_INDEX, COL3_INVERTED_INDEX);
+ indexingConfig.setInvertedIndexColumns(invertedIndexColumns);
+
+ List<String> rangeIndexColumns = Arrays.asList(COL1_RANGE_INDEX, COL2_RANGE_INDEX, COL3_RANGE_INDEX);
+ indexingConfig.setRangeIndexColumns(rangeIndexColumns);
+
+ List<String> jsonIndexColumns = Arrays.asList(COL1_JSON_INDEX);
+ indexingConfig.setJsonIndexColumns(jsonIndexColumns);
+
+ SegmentGeneratorConfig segmentGeneratorConfig = new SegmentGeneratorConfig(TABLE_CONFIG, SCHEMA);
+ segmentGeneratorConfig.setTableName(RAW_TABLE_NAME);
+ segmentGeneratorConfig.setSegmentName(SEGMENT_NAME);
+ segmentGeneratorConfig.setOutDir(INDEX_DIR.getPath());
+
+ SegmentIndexCreationDriverImpl driver = new SegmentIndexCreationDriverImpl();
+ driver.init(segmentGeneratorConfig, new GenericRowRecordReader(records));
+ driver.build();
+
+ IndexLoadingConfig indexLoadingConfig = new IndexLoadingConfig();
+ indexLoadingConfig.setTableConfig(TABLE_CONFIG);
+ indexLoadingConfig.setInvertedIndexColumns(new HashSet<>(invertedIndexColumns));
+ indexLoadingConfig.setRangeIndexColumns(new HashSet<>(rangeIndexColumns));
+ indexLoadingConfig.setJsonIndexColumns(new HashSet<String>(jsonIndexColumns));
+ indexLoadingConfig.setReadMode(ReadMode.mmap);
+
+ ImmutableSegment immutableSegment =
+ ImmutableSegmentLoader.load(new File(INDEX_DIR, SEGMENT_NAME), indexLoadingConfig);
+ _indexSegment = immutableSegment;
+ _indexSegments = Arrays.asList(immutableSegment);
+ }
+
+ /** Checks the correctness of EXPLAIN PLAN output. */
+ private void check(String query, ResultTable expected) {
+ BrokerResponseNative response = getBrokerResponseForSqlQuery(query);
+ ResultTable actual = response.getResultTable();
+
+ System.out.println(resultTableToString("Expected", expected));
+ System.out.println(resultTableToString("Actual", actual));
+
+ Assert.assertEquals(actual.getDataSchema(), expected.getDataSchema());
+ Assert.assertEquals(actual.getRows().size(), expected.getRows().size());
+
+ List<Object[]> expectedRows = expected.getRows();
+ List<Object[]> actualRows = actual.getRows();
+ for (int i = 0; i < expectedRows.size(); i++) {
+ Object[] expectedRow = expectedRows.get(i);
+ Object[] actualRow = actualRows.get(i);
+ Assert.assertEquals(actualRow.length, expectedRow.length);
+
+ for (int j = 0; j < actualRow.length; j++) {
+ Assert.assertEquals(actualRow[j], expectedRow[j]);
+ }
+ }
+ }
+
+ /** This function is used to generated expected results while creating test cases; otherwise, not used. */
+ private static String resultTableToString(String tag, ResultTable table) {
+ System.out.println("\n" + tag + ":");
+ List<Object[]> rows = table.getRows();
+
+ StringBuffer buffer = new StringBuffer();
+ for (Object[] row : rows) {
+ buffer.append("result.add(new Object[]{\"" + row[0] + "\", " + row[1] + ", " + row[2] + "});\n");
+ }
+
+ return buffer.toString();
+ }
+
+ @Test
+ public void testSelect() {
+ String query1 = "EXPLAIN PLAN FOR SELECT * FROM testTable";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(
+ new Object[]{"SELECT(selectList:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, jsonIndexCol1, "
+ + "noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, "
+ + "jsonIndexCol1, noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 3, 2});
+ result1.add(
+ new Object[]{"PROJECT(sortedIndexCol1, noIndexCol3, rangeIndexCol1, rangeIndexCol2, jsonIndexCol1, "
+ + "invertedIndexCol1, noIndexCol2, invertedIndexCol2, noIndexCol1, invertedIndexCol3, rangeIndexCol3, "
+ + "textIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 = "EXPLAIN PLAN FOR SELECT 'mickey' FROM testTable";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:'mickey')", 2, 1});
+ result2.add(new Object[]{"PROJECT(ALL)", 3, 2});
+ result2.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 4, 3});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+
+ String query3 = "EXPLAIN PLAN FOR SELECT invertedIndexCol1, noIndexCol1 FROM testTable LIMIT 100";
+ List<Object[]> result3 = new ArrayList<>();
+ result3.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result3.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result3.add(new Object[]{"SELECT(selectList:invertedIndexCol1, noIndexCol1)", 2, 1});
+ result3.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result3.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result3.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query3, new ResultTable(DATA_SCHEMA, result3));
+
+ String query4 = "EXPLAIN PLAN FOR SELECT DISTINCT invertedIndexCol1, noIndexCol1 FROM testTable LIMIT 100";
+ List<Object[]> result4 = new ArrayList<>();
+ result4.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result4.add(new Object[]{"COMBINE_DISTINCT", 1, 0});
+ result4.add(new Object[]{"DISTINCT(keyColumns:invertedIndexCol1, noIndexCol1)", 2, 1});
+ result4.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result4.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result4.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query4, new ResultTable(DATA_SCHEMA, result4));
+ }
+
+ @Test
+ public void testSelectColumnsUsingFilterIndex() {
+ String query1 = "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1, sortedIndexCol1 FROM testTable WHERE "
+ + "invertedIndexCol1 = 1.5 AND sortedIndexCol1 = 1 LIMIT 100";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1, sortedIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, sortedIndexCol1)", 3, 2});
+ result1.add(new Object[]{"PROJECT(sortedIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 = "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1, sortedIndexCol1 FROM testTable WHERE "
+ + "(invertedIndexCol1 = 1.5 AND sortedIndexCol1 = 1) OR (invertedIndexCol1 BETWEEN 100 and 200 AND "
+ + "rangeIndexCol1 > 20)";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1, sortedIndexCol1)", 2, 1});
+ result2.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, sortedIndexCol1)", 3, 2});
+ result2.add(new Object[]{"PROJECT(sortedIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result2.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+
+ String query3 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1 FROM testTable WHERE invertedIndexCol1 = 1.5 AND "
+ + "invertedIndexCol2 IN (10, 20, 30) AND invertedIndexCol3 NOT IN ('foo', 'bar') LIMIT 100";
+ List<Object[]> result3 = new ArrayList<>();
+ result3.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result3.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result3.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1)", 2, 1});
+ result3.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result3.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result3.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query3, new ResultTable(DATA_SCHEMA, result3));
+
+ String query4 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, noIndexCol2, sortedIndexCol1 FROM testTable WHERE sortedIndexCol1 = 1.5"
+ + " OR sortedIndexCol1 != 5 OR sortedIndexCol1 IN (10, 20, 30) LIMIT 100";
+ List<Object[]> result4 = new ArrayList<>();
+ result4.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result4.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result4.add(new Object[]{"SELECT(selectList:noIndexCol1, noIndexCol2, sortedIndexCol1)", 2, 1});
+ result4.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2, sortedIndexCol1)", 3, 2});
+ result4.add(new Object[]{"PROJECT(sortedIndexCol1, noIndexCol2, noIndexCol1)", 4, 3});
+ result4.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query4, new ResultTable(DATA_SCHEMA, result4));
+
+ String query5 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, noIndexCol2 FROM testTable WHERE DIV(noIndexCol1, noIndexCol2) BETWEEN "
+ + "10 AND 20 AND invertedIndexCol1 * 5 < 1000";
+ List<Object[]> result5 = new ArrayList<>();
+ result5.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result5.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result5.add(new Object[]{"SELECT(selectList:noIndexCol1, noIndexCol2)", 2, 1});
+ result5.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2)", 3, 2});
+ result5.add(new Object[]{"PROJECT(noIndexCol2, noIndexCol1)", 4, 3});
+ result5.add(new Object[]{"FILTER_AND", 5, 4});
+ result5.add(
+ new Object[]{"FILTER_EXPRESSION(operator:RANGE,predicate:div(noIndexCol1,noIndexCol2) BETWEEN '10' AND '20')",
+ 6, 5});
+ result5
+ .add(new Object[]{"FILTER_EXPRESSION(operator:RANGE,predicate:times(invertedIndexCol1,'5') < '1000')", 7, 5});
+ check(query5, new ResultTable(DATA_SCHEMA, result5));
+ }
+
+ @Test
+ public void testSelectColumnUsingFilterRange() {
+ // select * query triggering range index
+ // checks using RANGE (>, >=, <, <=, BETWEEN ..) on a column with range index should use RANGE_INDEX_SCAN
+ String query1 =
+ "EXPLAIN PLAN FOR SELECT invertedIndexCol1, noIndexCol1, rangeIndexCol1 FROM testTable WHERE rangeIndexCol1 >"
+ + " 10 AND rangeIndexCol2 >= 20 AND rangeIndexCol3 BETWEEN 50 AND 60 LIMIT 100";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(new Object[]{"SELECT(selectList:invertedIndexCol1, noIndexCol1, rangeIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, rangeIndexCol1)", 3, 2});
+ result1.add(new Object[]{"PROJECT(rangeIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_EMPTY", 5, 4});
Review comment:
Let's please add tests that don't use FILTER_EMPTY and MATCH_ALL
##########
File path: pinot-core/src/test/java/org/apache/pinot/queries/ExplainPlanQueriesTest.java
##########
@@ -0,0 +1,510 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.queries;
+
+import java.io.File;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.List;
+import org.apache.commons.io.FileUtils;
+import org.apache.pinot.common.response.broker.BrokerResponseNative;
+import org.apache.pinot.common.response.broker.ResultTable;
+import org.apache.pinot.common.utils.DataSchema;
+import org.apache.pinot.segment.local.indexsegment.immutable.ImmutableSegmentLoader;
+import org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl;
+import org.apache.pinot.segment.local.segment.index.loader.IndexLoadingConfig;
+import org.apache.pinot.segment.local.segment.readers.GenericRowRecordReader;
+import org.apache.pinot.segment.spi.ImmutableSegment;
+import org.apache.pinot.segment.spi.IndexSegment;
+import org.apache.pinot.segment.spi.creator.SegmentGeneratorConfig;
+import org.apache.pinot.spi.config.table.IndexingConfig;
+import org.apache.pinot.spi.config.table.TableConfig;
+import org.apache.pinot.spi.config.table.TableType;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.data.Schema;
+import org.apache.pinot.spi.data.readers.GenericRow;
+import org.apache.pinot.spi.utils.ReadMode;
+import org.apache.pinot.spi.utils.builder.TableConfigBuilder;
+import org.testng.Assert;
+import org.testng.annotations.BeforeClass;
+import org.testng.annotations.Test;
+
+
+public class ExplainPlanQueriesTest extends BaseQueriesTest {
+ private static final File INDEX_DIR = new File(FileUtils.getTempDirectory(), "ExplainPlanQueriesTest");
+ private static final String RAW_TABLE_NAME = "testTable";
+ private static final String SEGMENT_NAME = "testSegment";
+ private static final int NUM_RECORDS = 10;
+
+ private final static String COL1_NO_INDEX = "noIndexCol1";
+ private final static String COL2_NO_INDEX = "noIndexCol2";
+ private final static String COL3_NO_INDEX = "noIndexCol3";
+ private final static String COL1_INVERTED_INDEX = "invertedIndexCol1";
+ private final static String COL2_INVERTED_INDEX = "invertedIndexCol2";
+ private final static String COL3_INVERTED_INDEX = "invertedIndexCol3";
+ private final static String COL1_RANGE_INDEX = "rangeIndexCol1";
+ private final static String COL2_RANGE_INDEX = "rangeIndexCol2";
+ private final static String COL3_RANGE_INDEX = "rangeIndexCol3";
+ private final static String COL1_SORTED_INDEX = "sortedIndexCol1";
+ private final static String COL1_JSON_INDEX = "jsonIndexCol1";
+ private final static String COL1_TEXT_INDEX = "textIndexCol1";
+
+ private static final Schema SCHEMA = new Schema.SchemaBuilder().setSchemaName(RAW_TABLE_NAME)
+ .addSingleValueDimension(COL1_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL2_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_INVERTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_INVERTED_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_INVERTED_INDEX, FieldSpec.DataType.STRING)
+ .addSingleValueDimension(COL1_RANGE_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_SORTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL1_JSON_INDEX, FieldSpec.DataType.JSON)
+ .addSingleValueDimension(COL1_TEXT_INDEX, FieldSpec.DataType.STRING).build();
+
+ private static final DataSchema DATA_SCHEMA = new DataSchema(new String[]{"Operator", "Operator_Id", "Parent_Id"},
+ new DataSchema.ColumnDataType[]{DataSchema.ColumnDataType.STRING, DataSchema.ColumnDataType.INT,
+ DataSchema.ColumnDataType.INT});
+
+ private static final TableConfig TABLE_CONFIG =
+ new TableConfigBuilder(TableType.OFFLINE).setTableName(RAW_TABLE_NAME).build();
+
+ private IndexSegment _indexSegment;
+ private List<IndexSegment> _indexSegments;
+
+ @Override
+ protected String getFilter() {
+ return "";
+ }
+
+ @Override
+ protected IndexSegment getIndexSegment() {
+ return _indexSegment;
+ }
+
+ @Override
+ protected List<IndexSegment> getIndexSegments() {
+ return _indexSegments;
+ }
+
+ GenericRow createRecord(double doubleValue, int intValue, String stringValue, String jsonValue) {
+ GenericRow record = new GenericRow();
+ record.putValue(COL1_NO_INDEX, intValue);
+ record.putValue(COL2_NO_INDEX, intValue);
+ record.putValue(COL3_NO_INDEX, intValue);
+
+ record.putValue(COL1_INVERTED_INDEX, doubleValue);
+ record.putValue(COL2_INVERTED_INDEX, intValue);
+ record.putValue(COL3_INVERTED_INDEX, intValue);
+
+ record.putValue(COL1_RANGE_INDEX, doubleValue);
+ record.putValue(COL2_RANGE_INDEX, intValue);
+ record.putValue(COL3_RANGE_INDEX, intValue);
+
+ record.putValue(COL1_JSON_INDEX, jsonValue);
+ record.putValue(COL1_TEXT_INDEX, stringValue);
+
+ return record;
+ }
+
+ @BeforeClass
+ public void setUp()
+ throws Exception {
+ FileUtils.deleteDirectory(INDEX_DIR);
+
+ List<GenericRow> records = new ArrayList<>(NUM_RECORDS);
+ records.add(createRecord(1.1, 2, "daffy", "{\"first\": \"daffy\", \"last\": \"duck\"}"));
+
+ IndexingConfig indexingConfig = TABLE_CONFIG.getIndexingConfig();
+
+ List<String> invertedIndexColumns = Arrays.asList(COL1_INVERTED_INDEX, COL2_INVERTED_INDEX, COL3_INVERTED_INDEX);
+ indexingConfig.setInvertedIndexColumns(invertedIndexColumns);
+
+ List<String> rangeIndexColumns = Arrays.asList(COL1_RANGE_INDEX, COL2_RANGE_INDEX, COL3_RANGE_INDEX);
+ indexingConfig.setRangeIndexColumns(rangeIndexColumns);
+
+ List<String> jsonIndexColumns = Arrays.asList(COL1_JSON_INDEX);
+ indexingConfig.setJsonIndexColumns(jsonIndexColumns);
+
+ SegmentGeneratorConfig segmentGeneratorConfig = new SegmentGeneratorConfig(TABLE_CONFIG, SCHEMA);
+ segmentGeneratorConfig.setTableName(RAW_TABLE_NAME);
+ segmentGeneratorConfig.setSegmentName(SEGMENT_NAME);
+ segmentGeneratorConfig.setOutDir(INDEX_DIR.getPath());
+
+ SegmentIndexCreationDriverImpl driver = new SegmentIndexCreationDriverImpl();
+ driver.init(segmentGeneratorConfig, new GenericRowRecordReader(records));
+ driver.build();
+
+ IndexLoadingConfig indexLoadingConfig = new IndexLoadingConfig();
+ indexLoadingConfig.setTableConfig(TABLE_CONFIG);
+ indexLoadingConfig.setInvertedIndexColumns(new HashSet<>(invertedIndexColumns));
+ indexLoadingConfig.setRangeIndexColumns(new HashSet<>(rangeIndexColumns));
+ indexLoadingConfig.setJsonIndexColumns(new HashSet<String>(jsonIndexColumns));
+ indexLoadingConfig.setReadMode(ReadMode.mmap);
+
+ ImmutableSegment immutableSegment =
+ ImmutableSegmentLoader.load(new File(INDEX_DIR, SEGMENT_NAME), indexLoadingConfig);
+ _indexSegment = immutableSegment;
+ _indexSegments = Arrays.asList(immutableSegment);
+ }
+
+ /** Checks the correctness of EXPLAIN PLAN output. */
+ private void check(String query, ResultTable expected) {
+ BrokerResponseNative response = getBrokerResponseForSqlQuery(query);
+ ResultTable actual = response.getResultTable();
+
+ System.out.println(resultTableToString("Expected", expected));
+ System.out.println(resultTableToString("Actual", actual));
+
+ Assert.assertEquals(actual.getDataSchema(), expected.getDataSchema());
+ Assert.assertEquals(actual.getRows().size(), expected.getRows().size());
+
+ List<Object[]> expectedRows = expected.getRows();
+ List<Object[]> actualRows = actual.getRows();
+ for (int i = 0; i < expectedRows.size(); i++) {
+ Object[] expectedRow = expectedRows.get(i);
+ Object[] actualRow = actualRows.get(i);
+ Assert.assertEquals(actualRow.length, expectedRow.length);
+
+ for (int j = 0; j < actualRow.length; j++) {
+ Assert.assertEquals(actualRow[j], expectedRow[j]);
+ }
+ }
+ }
+
+ /** This function is used to generated expected results while creating test cases; otherwise, not used. */
+ private static String resultTableToString(String tag, ResultTable table) {
+ System.out.println("\n" + tag + ":");
+ List<Object[]> rows = table.getRows();
+
+ StringBuffer buffer = new StringBuffer();
+ for (Object[] row : rows) {
+ buffer.append("result.add(new Object[]{\"" + row[0] + "\", " + row[1] + ", " + row[2] + "});\n");
+ }
+
+ return buffer.toString();
+ }
+
+ @Test
+ public void testSelect() {
+ String query1 = "EXPLAIN PLAN FOR SELECT * FROM testTable";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(
+ new Object[]{"SELECT(selectList:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, jsonIndexCol1, "
+ + "noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, "
+ + "jsonIndexCol1, noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 3, 2});
+ result1.add(
+ new Object[]{"PROJECT(sortedIndexCol1, noIndexCol3, rangeIndexCol1, rangeIndexCol2, jsonIndexCol1, "
+ + "invertedIndexCol1, noIndexCol2, invertedIndexCol2, noIndexCol1, invertedIndexCol3, rangeIndexCol3, "
+ + "textIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 = "EXPLAIN PLAN FOR SELECT 'mickey' FROM testTable";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:'mickey')", 2, 1});
+ result2.add(new Object[]{"PROJECT(ALL)", 3, 2});
+ result2.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 4, 3});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+
+ String query3 = "EXPLAIN PLAN FOR SELECT invertedIndexCol1, noIndexCol1 FROM testTable LIMIT 100";
+ List<Object[]> result3 = new ArrayList<>();
+ result3.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result3.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result3.add(new Object[]{"SELECT(selectList:invertedIndexCol1, noIndexCol1)", 2, 1});
+ result3.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result3.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result3.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query3, new ResultTable(DATA_SCHEMA, result3));
+
+ String query4 = "EXPLAIN PLAN FOR SELECT DISTINCT invertedIndexCol1, noIndexCol1 FROM testTable LIMIT 100";
+ List<Object[]> result4 = new ArrayList<>();
+ result4.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result4.add(new Object[]{"COMBINE_DISTINCT", 1, 0});
+ result4.add(new Object[]{"DISTINCT(keyColumns:invertedIndexCol1, noIndexCol1)", 2, 1});
+ result4.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result4.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result4.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query4, new ResultTable(DATA_SCHEMA, result4));
+ }
+
+ @Test
+ public void testSelectColumnsUsingFilterIndex() {
+ String query1 = "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1, sortedIndexCol1 FROM testTable WHERE "
+ + "invertedIndexCol1 = 1.5 AND sortedIndexCol1 = 1 LIMIT 100";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1, sortedIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, sortedIndexCol1)", 3, 2});
+ result1.add(new Object[]{"PROJECT(sortedIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 = "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1, sortedIndexCol1 FROM testTable WHERE "
+ + "(invertedIndexCol1 = 1.5 AND sortedIndexCol1 = 1) OR (invertedIndexCol1 BETWEEN 100 and 200 AND "
+ + "rangeIndexCol1 > 20)";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1, sortedIndexCol1)", 2, 1});
+ result2.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, sortedIndexCol1)", 3, 2});
+ result2.add(new Object[]{"PROJECT(sortedIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result2.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+
+ String query3 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1 FROM testTable WHERE invertedIndexCol1 = 1.5 AND "
+ + "invertedIndexCol2 IN (10, 20, 30) AND invertedIndexCol3 NOT IN ('foo', 'bar') LIMIT 100";
+ List<Object[]> result3 = new ArrayList<>();
+ result3.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result3.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result3.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1)", 2, 1});
+ result3.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result3.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result3.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query3, new ResultTable(DATA_SCHEMA, result3));
+
+ String query4 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, noIndexCol2, sortedIndexCol1 FROM testTable WHERE sortedIndexCol1 = 1.5"
+ + " OR sortedIndexCol1 != 5 OR sortedIndexCol1 IN (10, 20, 30) LIMIT 100";
+ List<Object[]> result4 = new ArrayList<>();
+ result4.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result4.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result4.add(new Object[]{"SELECT(selectList:noIndexCol1, noIndexCol2, sortedIndexCol1)", 2, 1});
+ result4.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2, sortedIndexCol1)", 3, 2});
+ result4.add(new Object[]{"PROJECT(sortedIndexCol1, noIndexCol2, noIndexCol1)", 4, 3});
+ result4.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query4, new ResultTable(DATA_SCHEMA, result4));
+
+ String query5 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, noIndexCol2 FROM testTable WHERE DIV(noIndexCol1, noIndexCol2) BETWEEN "
+ + "10 AND 20 AND invertedIndexCol1 * 5 < 1000";
+ List<Object[]> result5 = new ArrayList<>();
+ result5.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result5.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result5.add(new Object[]{"SELECT(selectList:noIndexCol1, noIndexCol2)", 2, 1});
+ result5.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2)", 3, 2});
+ result5.add(new Object[]{"PROJECT(noIndexCol2, noIndexCol1)", 4, 3});
+ result5.add(new Object[]{"FILTER_AND", 5, 4});
+ result5.add(
+ new Object[]{"FILTER_EXPRESSION(operator:RANGE,predicate:div(noIndexCol1,noIndexCol2) BETWEEN '10' AND '20')",
+ 6, 5});
+ result5
+ .add(new Object[]{"FILTER_EXPRESSION(operator:RANGE,predicate:times(invertedIndexCol1,'5') < '1000')", 7, 5});
+ check(query5, new ResultTable(DATA_SCHEMA, result5));
+ }
+
+ @Test
+ public void testSelectColumnUsingFilterRange() {
+ // select * query triggering range index
+ // checks using RANGE (>, >=, <, <=, BETWEEN ..) on a column with range index should use RANGE_INDEX_SCAN
+ String query1 =
+ "EXPLAIN PLAN FOR SELECT invertedIndexCol1, noIndexCol1, rangeIndexCol1 FROM testTable WHERE rangeIndexCol1 >"
+ + " 10 AND rangeIndexCol2 >= 20 AND rangeIndexCol3 BETWEEN 50 AND 60 LIMIT 100";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(new Object[]{"SELECT(selectList:invertedIndexCol1, noIndexCol1, rangeIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, rangeIndexCol1)", 3, 2});
+ result1.add(new Object[]{"PROJECT(rangeIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1 FROM testTable WHERE noIndexCol1 > 10 AND noIndexCol2"
+ + " BETWEEN 50 AND 60 LIMIT 100";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1)", 2, 1});
+ result2.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result2.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result2.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+ }
+
+ @Test
+ public void testSelectColumnUsingFilterJson() {
+ String query =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1 FROM testTable WHERE (invertedIndexCol1 IN (10, 20, "
+ + "30) AND sortedIndexCol1 != 100) OR (noIndexCol1 NOT IN (10, 20, 30) AND rangeIndexCol1 != 20 AND "
+ + "JSON_MATCH(jsonIndexCol1, 'key=1') AND TEXT_MATCH(textIndexCol1, 'foo'))";
+ List<Object[]> result = new ArrayList<>();
+ result.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1)", 2, 1});
+ result.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result.add(new Object[]{"FILTER_AND", 5, 4});
+ result.add(new Object[]{"FILTER_JSON_INDEX(indexLookUp:json_index,operator:JSON_MATCH,predicate:json_match"
+ + "(jsonIndexCol1,'key=1'))", 6, 5});
+ result.add(new Object[]{"FILTER_TEXT_INDEX(indexLookUp:text_index,operator:TEXT_MATCH,predicate:text_match"
+ + "(textIndexCol1,'foo'))", 7, 5});
+ check(query, new ResultTable(DATA_SCHEMA, result));
+ }
+
+ @Test
+ public void testSelectAggregateUsingFilterText() {
+ String query1 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, noIndexCol2, max(noIndexCol2), min(noIndexCol3) FROM testTable WHERE "
+ + "TEXT_MATCH(textIndexCol1, 'foo') GROUP BY noIndexCol1, noIndexCol2";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result1.add(new Object[]{"COMBINE_GROUPBY_ORDERBY", 1, 0});
+ result1.add(new Object[]{
+ "AGGREGATE_GROUPBY_ORDERBY(groupKeys:noIndexCol1, noIndexCol2, aggregations:max(noIndexCol2), min"
+ + "(noIndexCol3))", 2, 1});
+ result1.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2, noIndexCol3)", 3, 2});
+ result1.add(new Object[]{"PROJECT(noIndexCol3, noIndexCol2, noIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_TEXT_INDEX(indexLookUp:text_index,operator:TEXT_MATCH,predicate:text_match"
+ + "(textIndexCol1,'foo'))", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, max(noIndexCol2), min(noIndexCol3) FROM testTable WHERE TEXT_MATCH"
+ + "(textIndexCol1, 'foo') GROUP BY noIndexCol1, noIndexCol2 ORDER BY noIndexCol1, max(noIndexCol2)";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(sort:[noIndexCol1 ASC, max(noIndexCol2) ASC],limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_GROUPBY_ORDERBY", 1, 0});
+ result2.add(new Object[]{
+ "AGGREGATE_GROUPBY_ORDERBY(groupKeys:noIndexCol1,noIndexCol2,aggregations:max(noIndexCol2), min"
+ + "(noIndexCol3))", 2, 1});
+ result2.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, invertedIndexCol1)", 3, 2});
+ result2.add(new Object[]{"PROJECT(noIndexCol3, noIndexCol2, noIndexCol1)", 4, 3});
+ result2.add(new Object[]{"FILTER_TEXT_INDEX(indexLookUp:text_index,operator:TEXT_MATCH,predicate:text_match"
+ + "(textIndexCol1,'foo'))", 5, 4});
+ }
+
+ @Test
+ public void testSelectAggregate() {
+ String query1 = "EXPLAIN PLAN FOR SELECT count(*) FROM testTable";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result1.add(new Object[]{"COMBINE_AGGREGATE", 1, 0});
+ result1.add(new Object[]{"AGGREGATE_METADATA", 2, 1});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 =
+ "EXPLAIN PLAN FOR SELECT count(*), max(noIndexCol1), sum(noIndexCol2), avg(noIndexCol2) FROM testTable";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_AGGREGATE", 1, 0});
+ result2.add(
+ new Object[]{"AGGREGATE(aggregations:count(*), max(noIndexCol1), sum(noIndexCol2), avg(noIndexCol2))", 2, 1});
+ result2.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2)", 3, 2});
+ result2.add(new Object[]{"PROJECT(noIndexCol2, noIndexCol1)", 4, 3});
+ result2.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+ }
+
+ @Test
+ public void testSelectAggregateUsingFilter() {
+ String query = "EXPLAIN PLAN FOR SELECT sum(add(noIndexCol1, noIndexCol2)), MIN(ADD(DIV(noIndexCol1,noIndexCol2),"
+ + "noIndexCol3)) FROM testTable";
+ List<Object[]> result = new ArrayList<>();
+ result.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result.add(new Object[]{"COMBINE_AGGREGATE", 1, 0});
+ result.add(new Object[]{
+ "AGGREGATE(aggregations:sum(add(noIndexCol1,noIndexCol2)), min(add(div(noIndexCol1,noIndexCol2),"
Review comment:
The plan is not correct. Since aggregation is happening on top of a transform, we should reflect that in the plan
##########
File path: pinot-core/src/test/java/org/apache/pinot/queries/ExplainPlanQueriesTest.java
##########
@@ -0,0 +1,510 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.queries;
+
+import java.io.File;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.List;
+import org.apache.commons.io.FileUtils;
+import org.apache.pinot.common.response.broker.BrokerResponseNative;
+import org.apache.pinot.common.response.broker.ResultTable;
+import org.apache.pinot.common.utils.DataSchema;
+import org.apache.pinot.segment.local.indexsegment.immutable.ImmutableSegmentLoader;
+import org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl;
+import org.apache.pinot.segment.local.segment.index.loader.IndexLoadingConfig;
+import org.apache.pinot.segment.local.segment.readers.GenericRowRecordReader;
+import org.apache.pinot.segment.spi.ImmutableSegment;
+import org.apache.pinot.segment.spi.IndexSegment;
+import org.apache.pinot.segment.spi.creator.SegmentGeneratorConfig;
+import org.apache.pinot.spi.config.table.IndexingConfig;
+import org.apache.pinot.spi.config.table.TableConfig;
+import org.apache.pinot.spi.config.table.TableType;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.data.Schema;
+import org.apache.pinot.spi.data.readers.GenericRow;
+import org.apache.pinot.spi.utils.ReadMode;
+import org.apache.pinot.spi.utils.builder.TableConfigBuilder;
+import org.testng.Assert;
+import org.testng.annotations.BeforeClass;
+import org.testng.annotations.Test;
+
+
+public class ExplainPlanQueriesTest extends BaseQueriesTest {
+ private static final File INDEX_DIR = new File(FileUtils.getTempDirectory(), "ExplainPlanQueriesTest");
+ private static final String RAW_TABLE_NAME = "testTable";
+ private static final String SEGMENT_NAME = "testSegment";
+ private static final int NUM_RECORDS = 10;
+
+ private final static String COL1_NO_INDEX = "noIndexCol1";
+ private final static String COL2_NO_INDEX = "noIndexCol2";
+ private final static String COL3_NO_INDEX = "noIndexCol3";
+ private final static String COL1_INVERTED_INDEX = "invertedIndexCol1";
+ private final static String COL2_INVERTED_INDEX = "invertedIndexCol2";
+ private final static String COL3_INVERTED_INDEX = "invertedIndexCol3";
+ private final static String COL1_RANGE_INDEX = "rangeIndexCol1";
+ private final static String COL2_RANGE_INDEX = "rangeIndexCol2";
+ private final static String COL3_RANGE_INDEX = "rangeIndexCol3";
+ private final static String COL1_SORTED_INDEX = "sortedIndexCol1";
+ private final static String COL1_JSON_INDEX = "jsonIndexCol1";
+ private final static String COL1_TEXT_INDEX = "textIndexCol1";
+
+ private static final Schema SCHEMA = new Schema.SchemaBuilder().setSchemaName(RAW_TABLE_NAME)
+ .addSingleValueDimension(COL1_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL2_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_INVERTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_INVERTED_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_INVERTED_INDEX, FieldSpec.DataType.STRING)
+ .addSingleValueDimension(COL1_RANGE_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_SORTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL1_JSON_INDEX, FieldSpec.DataType.JSON)
+ .addSingleValueDimension(COL1_TEXT_INDEX, FieldSpec.DataType.STRING).build();
+
+ private static final DataSchema DATA_SCHEMA = new DataSchema(new String[]{"Operator", "Operator_Id", "Parent_Id"},
+ new DataSchema.ColumnDataType[]{DataSchema.ColumnDataType.STRING, DataSchema.ColumnDataType.INT,
+ DataSchema.ColumnDataType.INT});
+
+ private static final TableConfig TABLE_CONFIG =
+ new TableConfigBuilder(TableType.OFFLINE).setTableName(RAW_TABLE_NAME).build();
+
+ private IndexSegment _indexSegment;
+ private List<IndexSegment> _indexSegments;
+
+ @Override
+ protected String getFilter() {
+ return "";
+ }
+
+ @Override
+ protected IndexSegment getIndexSegment() {
+ return _indexSegment;
+ }
+
+ @Override
+ protected List<IndexSegment> getIndexSegments() {
+ return _indexSegments;
+ }
+
+ GenericRow createRecord(double doubleValue, int intValue, String stringValue, String jsonValue) {
+ GenericRow record = new GenericRow();
+ record.putValue(COL1_NO_INDEX, intValue);
+ record.putValue(COL2_NO_INDEX, intValue);
+ record.putValue(COL3_NO_INDEX, intValue);
+
+ record.putValue(COL1_INVERTED_INDEX, doubleValue);
+ record.putValue(COL2_INVERTED_INDEX, intValue);
+ record.putValue(COL3_INVERTED_INDEX, intValue);
+
+ record.putValue(COL1_RANGE_INDEX, doubleValue);
+ record.putValue(COL2_RANGE_INDEX, intValue);
+ record.putValue(COL3_RANGE_INDEX, intValue);
+
+ record.putValue(COL1_JSON_INDEX, jsonValue);
+ record.putValue(COL1_TEXT_INDEX, stringValue);
+
+ return record;
+ }
+
+ @BeforeClass
+ public void setUp()
+ throws Exception {
+ FileUtils.deleteDirectory(INDEX_DIR);
+
+ List<GenericRow> records = new ArrayList<>(NUM_RECORDS);
+ records.add(createRecord(1.1, 2, "daffy", "{\"first\": \"daffy\", \"last\": \"duck\"}"));
+
+ IndexingConfig indexingConfig = TABLE_CONFIG.getIndexingConfig();
+
+ List<String> invertedIndexColumns = Arrays.asList(COL1_INVERTED_INDEX, COL2_INVERTED_INDEX, COL3_INVERTED_INDEX);
+ indexingConfig.setInvertedIndexColumns(invertedIndexColumns);
+
+ List<String> rangeIndexColumns = Arrays.asList(COL1_RANGE_INDEX, COL2_RANGE_INDEX, COL3_RANGE_INDEX);
+ indexingConfig.setRangeIndexColumns(rangeIndexColumns);
+
+ List<String> jsonIndexColumns = Arrays.asList(COL1_JSON_INDEX);
+ indexingConfig.setJsonIndexColumns(jsonIndexColumns);
+
+ SegmentGeneratorConfig segmentGeneratorConfig = new SegmentGeneratorConfig(TABLE_CONFIG, SCHEMA);
+ segmentGeneratorConfig.setTableName(RAW_TABLE_NAME);
+ segmentGeneratorConfig.setSegmentName(SEGMENT_NAME);
+ segmentGeneratorConfig.setOutDir(INDEX_DIR.getPath());
+
+ SegmentIndexCreationDriverImpl driver = new SegmentIndexCreationDriverImpl();
+ driver.init(segmentGeneratorConfig, new GenericRowRecordReader(records));
+ driver.build();
+
+ IndexLoadingConfig indexLoadingConfig = new IndexLoadingConfig();
+ indexLoadingConfig.setTableConfig(TABLE_CONFIG);
+ indexLoadingConfig.setInvertedIndexColumns(new HashSet<>(invertedIndexColumns));
+ indexLoadingConfig.setRangeIndexColumns(new HashSet<>(rangeIndexColumns));
+ indexLoadingConfig.setJsonIndexColumns(new HashSet<String>(jsonIndexColumns));
+ indexLoadingConfig.setReadMode(ReadMode.mmap);
+
+ ImmutableSegment immutableSegment =
+ ImmutableSegmentLoader.load(new File(INDEX_DIR, SEGMENT_NAME), indexLoadingConfig);
+ _indexSegment = immutableSegment;
+ _indexSegments = Arrays.asList(immutableSegment);
+ }
+
+ /** Checks the correctness of EXPLAIN PLAN output. */
+ private void check(String query, ResultTable expected) {
+ BrokerResponseNative response = getBrokerResponseForSqlQuery(query);
+ ResultTable actual = response.getResultTable();
+
+ System.out.println(resultTableToString("Expected", expected));
+ System.out.println(resultTableToString("Actual", actual));
+
+ Assert.assertEquals(actual.getDataSchema(), expected.getDataSchema());
+ Assert.assertEquals(actual.getRows().size(), expected.getRows().size());
+
+ List<Object[]> expectedRows = expected.getRows();
+ List<Object[]> actualRows = actual.getRows();
+ for (int i = 0; i < expectedRows.size(); i++) {
+ Object[] expectedRow = expectedRows.get(i);
+ Object[] actualRow = actualRows.get(i);
+ Assert.assertEquals(actualRow.length, expectedRow.length);
+
+ for (int j = 0; j < actualRow.length; j++) {
+ Assert.assertEquals(actualRow[j], expectedRow[j]);
+ }
+ }
+ }
+
+ /** This function is used to generated expected results while creating test cases; otherwise, not used. */
+ private static String resultTableToString(String tag, ResultTable table) {
+ System.out.println("\n" + tag + ":");
+ List<Object[]> rows = table.getRows();
+
+ StringBuffer buffer = new StringBuffer();
+ for (Object[] row : rows) {
+ buffer.append("result.add(new Object[]{\"" + row[0] + "\", " + row[1] + ", " + row[2] + "});\n");
+ }
+
+ return buffer.toString();
+ }
+
+ @Test
+ public void testSelect() {
+ String query1 = "EXPLAIN PLAN FOR SELECT * FROM testTable";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(
+ new Object[]{"SELECT(selectList:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, jsonIndexCol1, "
+ + "noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, "
+ + "jsonIndexCol1, noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 3, 2});
+ result1.add(
+ new Object[]{"PROJECT(sortedIndexCol1, noIndexCol3, rangeIndexCol1, rangeIndexCol2, jsonIndexCol1, "
+ + "invertedIndexCol1, noIndexCol2, invertedIndexCol2, noIndexCol1, invertedIndexCol3, rangeIndexCol3, "
+ + "textIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 = "EXPLAIN PLAN FOR SELECT 'mickey' FROM testTable";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:'mickey')", 2, 1});
+ result2.add(new Object[]{"PROJECT(ALL)", 3, 2});
+ result2.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 4, 3});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+
+ String query3 = "EXPLAIN PLAN FOR SELECT invertedIndexCol1, noIndexCol1 FROM testTable LIMIT 100";
+ List<Object[]> result3 = new ArrayList<>();
+ result3.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result3.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result3.add(new Object[]{"SELECT(selectList:invertedIndexCol1, noIndexCol1)", 2, 1});
+ result3.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result3.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result3.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query3, new ResultTable(DATA_SCHEMA, result3));
+
+ String query4 = "EXPLAIN PLAN FOR SELECT DISTINCT invertedIndexCol1, noIndexCol1 FROM testTable LIMIT 100";
+ List<Object[]> result4 = new ArrayList<>();
+ result4.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result4.add(new Object[]{"COMBINE_DISTINCT", 1, 0});
+ result4.add(new Object[]{"DISTINCT(keyColumns:invertedIndexCol1, noIndexCol1)", 2, 1});
+ result4.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result4.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result4.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query4, new ResultTable(DATA_SCHEMA, result4));
+ }
+
+ @Test
+ public void testSelectColumnsUsingFilterIndex() {
+ String query1 = "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1, sortedIndexCol1 FROM testTable WHERE "
+ + "invertedIndexCol1 = 1.5 AND sortedIndexCol1 = 1 LIMIT 100";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1, sortedIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, sortedIndexCol1)", 3, 2});
+ result1.add(new Object[]{"PROJECT(sortedIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 = "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1, sortedIndexCol1 FROM testTable WHERE "
+ + "(invertedIndexCol1 = 1.5 AND sortedIndexCol1 = 1) OR (invertedIndexCol1 BETWEEN 100 and 200 AND "
+ + "rangeIndexCol1 > 20)";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1, sortedIndexCol1)", 2, 1});
+ result2.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, sortedIndexCol1)", 3, 2});
+ result2.add(new Object[]{"PROJECT(sortedIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result2.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+
+ String query3 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1 FROM testTable WHERE invertedIndexCol1 = 1.5 AND "
+ + "invertedIndexCol2 IN (10, 20, 30) AND invertedIndexCol3 NOT IN ('foo', 'bar') LIMIT 100";
+ List<Object[]> result3 = new ArrayList<>();
+ result3.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result3.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result3.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1)", 2, 1});
+ result3.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result3.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result3.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query3, new ResultTable(DATA_SCHEMA, result3));
+
+ String query4 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, noIndexCol2, sortedIndexCol1 FROM testTable WHERE sortedIndexCol1 = 1.5"
+ + " OR sortedIndexCol1 != 5 OR sortedIndexCol1 IN (10, 20, 30) LIMIT 100";
+ List<Object[]> result4 = new ArrayList<>();
+ result4.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result4.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result4.add(new Object[]{"SELECT(selectList:noIndexCol1, noIndexCol2, sortedIndexCol1)", 2, 1});
+ result4.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2, sortedIndexCol1)", 3, 2});
+ result4.add(new Object[]{"PROJECT(sortedIndexCol1, noIndexCol2, noIndexCol1)", 4, 3});
+ result4.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query4, new ResultTable(DATA_SCHEMA, result4));
+
+ String query5 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, noIndexCol2 FROM testTable WHERE DIV(noIndexCol1, noIndexCol2) BETWEEN "
+ + "10 AND 20 AND invertedIndexCol1 * 5 < 1000";
+ List<Object[]> result5 = new ArrayList<>();
+ result5.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result5.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result5.add(new Object[]{"SELECT(selectList:noIndexCol1, noIndexCol2)", 2, 1});
+ result5.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:noIndexCol1, noIndexCol2)", 3, 2});
+ result5.add(new Object[]{"PROJECT(noIndexCol2, noIndexCol1)", 4, 3});
+ result5.add(new Object[]{"FILTER_AND", 5, 4});
+ result5.add(
+ new Object[]{"FILTER_EXPRESSION(operator:RANGE,predicate:div(noIndexCol1,noIndexCol2) BETWEEN '10' AND '20')",
+ 6, 5});
+ result5
+ .add(new Object[]{"FILTER_EXPRESSION(operator:RANGE,predicate:times(invertedIndexCol1,'5') < '1000')", 7, 5});
+ check(query5, new ResultTable(DATA_SCHEMA, result5));
+ }
+
+ @Test
+ public void testSelectColumnUsingFilterRange() {
+ // select * query triggering range index
+ // checks using RANGE (>, >=, <, <=, BETWEEN ..) on a column with range index should use RANGE_INDEX_SCAN
+ String query1 =
+ "EXPLAIN PLAN FOR SELECT invertedIndexCol1, noIndexCol1, rangeIndexCol1 FROM testTable WHERE rangeIndexCol1 >"
+ + " 10 AND rangeIndexCol2 >= 20 AND rangeIndexCol3 BETWEEN 50 AND 60 LIMIT 100";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(new Object[]{"SELECT(selectList:invertedIndexCol1, noIndexCol1, rangeIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1, rangeIndexCol1)", 3, 2});
+ result1.add(new Object[]{"PROJECT(rangeIndexCol1, invertedIndexCol1, noIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1 FROM testTable WHERE noIndexCol1 > 10 AND noIndexCol2"
+ + " BETWEEN 50 AND 60 LIMIT 100";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:100)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1)", 2, 1});
+ result2.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result2.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result2.add(new Object[]{"FILTER_EMPTY", 5, 4});
+ check(query2, new ResultTable(DATA_SCHEMA, result2));
+ }
+
+ @Test
+ public void testSelectColumnUsingFilterJson() {
+ String query =
+ "EXPLAIN PLAN FOR SELECT noIndexCol1, invertedIndexCol1 FROM testTable WHERE (invertedIndexCol1 IN (10, 20, "
+ + "30) AND sortedIndexCol1 != 100) OR (noIndexCol1 NOT IN (10, 20, 30) AND rangeIndexCol1 != 20 AND "
+ + "JSON_MATCH(jsonIndexCol1, 'key=1') AND TEXT_MATCH(textIndexCol1, 'foo'))";
+ List<Object[]> result = new ArrayList<>();
+ result.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result.add(new Object[]{"SELECT(selectList:noIndexCol1, invertedIndexCol1)", 2, 1});
+ result.add(new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, noIndexCol1)", 3, 2});
+ result.add(new Object[]{"PROJECT(invertedIndexCol1, noIndexCol1)", 4, 3});
+ result.add(new Object[]{"FILTER_AND", 5, 4});
Review comment:
The filter tree doesn't look correct. The leaf operators for IN, !=, NOT IN seem to be missing from the subtree
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] jackjlli commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
jackjlli commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r746061067
##########
File path: pinot-broker/src/test/java/org/apache/pinot/broker/requesthandler/LiteralOnlyBrokerRequestTest.java
##########
@@ -177,4 +177,55 @@ public void testBrokerRequestHandlerWithAsFunction()
Assert.assertEquals(brokerResponse.getResultTable().getRows().get(0)[1], 1577836800000L);
Assert.assertEquals(brokerResponse.getTotalDocs(), 0);
}
+
+ /** Tests for EXPLAIN PLAN for literal only queries. */
+ @Test
+ public void testExplainPlanLiteralOnly()
+ throws Exception {
+ SingleConnectionBrokerRequestHandler requestHandler =
+ new SingleConnectionBrokerRequestHandler(new PinotConfiguration(), null, ACCESS_CONTROL_FACTORY, null, null,
+ new BrokerMetrics("", PinotMetricUtils.getPinotMetricsRegistry(), true, Collections.emptySet()), null);
+
+ // Test 1: select constant
+ JsonNode request = new ObjectMapper().readTree("{\"sql\":\"EXPLAIN PLAN FOR SELECT 1.5, 'test'\"}");
Review comment:
`ObjectMapper` can be instantiated only once.
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/plan/FilterPlanNode.java
##########
@@ -217,9 +216,9 @@ private BaseFilterOperator constructPhysicalOperator(FilterContext filter) {
return FilterOperatorUtils.getLeafFilterOperator(predicateEvaluator, dataSource, _numDocs);
case JSON_MATCH:
JsonIndexReader jsonIndex = dataSource.getJsonIndex();
- Preconditions.checkState(jsonIndex != null, "Cannot apply JSON_MATCH on column: %s without json index",
- column);
- return new JsonMatchFilterOperator(jsonIndex, ((JsonMatchPredicate) predicate).getValue(), _numDocs);
+ Preconditions
+ .checkState(jsonIndex != null, "Cannot apply JSON_MATCH on column: %s without json index", column);
Review comment:
Can be replaced with `Preconditions.checkNotNull();`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] amrishlal commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
amrishlal commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r746093798
##########
File path: pinot-broker/src/test/java/org/apache/pinot/broker/requesthandler/LiteralOnlyBrokerRequestTest.java
##########
@@ -177,4 +177,55 @@ public void testBrokerRequestHandlerWithAsFunction()
Assert.assertEquals(brokerResponse.getResultTable().getRows().get(0)[1], 1577836800000L);
Assert.assertEquals(brokerResponse.getTotalDocs(), 0);
}
+
+ /** Tests for EXPLAIN PLAN for literal only queries. */
+ @Test
+ public void testExplainPlanLiteralOnly()
+ throws Exception {
+ SingleConnectionBrokerRequestHandler requestHandler =
+ new SingleConnectionBrokerRequestHandler(new PinotConfiguration(), null, ACCESS_CONTROL_FACTORY, null, null,
+ new BrokerMetrics("", PinotMetricUtils.getPinotMetricsRegistry(), true, Collections.emptySet()), null);
+
+ // Test 1: select constant
+ JsonNode request = new ObjectMapper().readTree("{\"sql\":\"EXPLAIN PLAN FOR SELECT 1.5, 'test'\"}");
+ RequestStatistics requestStats = new RequestStatistics();
+ BrokerResponseNative brokerResponse = requestHandler.handleRequest(request, null, requestStats);
+
+ checkExplainResultSchema(brokerResponse.getResultTable().getDataSchema(),
+ new String[]{"Operator", "Operator_Id", "Parent_Id"},
+ new DataSchema.ColumnDataType[]{DataSchema.ColumnDataType.STRING, DataSchema.ColumnDataType.INT,
+ DataSchema.ColumnDataType.INT});
+
+ Assert.assertEquals(brokerResponse.getResultTable().getRows().size(), 1);
+ Assert.assertEquals(brokerResponse.getResultTable().getRows().get(0),
+ new Object[]{"SELECT(selectList:literal)", 0, -1});
Review comment:
1.5 is also a literal, so same reason as last comment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] siddharthteotia commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
siddharthteotia commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r754729798
##########
File path: pinot-core/src/test/java/org/apache/pinot/queries/ExplainPlanQueriesTest.java
##########
@@ -0,0 +1,510 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.queries;
+
+import java.io.File;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.List;
+import org.apache.commons.io.FileUtils;
+import org.apache.pinot.common.response.broker.BrokerResponseNative;
+import org.apache.pinot.common.response.broker.ResultTable;
+import org.apache.pinot.common.utils.DataSchema;
+import org.apache.pinot.segment.local.indexsegment.immutable.ImmutableSegmentLoader;
+import org.apache.pinot.segment.local.segment.creator.impl.SegmentIndexCreationDriverImpl;
+import org.apache.pinot.segment.local.segment.index.loader.IndexLoadingConfig;
+import org.apache.pinot.segment.local.segment.readers.GenericRowRecordReader;
+import org.apache.pinot.segment.spi.ImmutableSegment;
+import org.apache.pinot.segment.spi.IndexSegment;
+import org.apache.pinot.segment.spi.creator.SegmentGeneratorConfig;
+import org.apache.pinot.spi.config.table.IndexingConfig;
+import org.apache.pinot.spi.config.table.TableConfig;
+import org.apache.pinot.spi.config.table.TableType;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.data.Schema;
+import org.apache.pinot.spi.data.readers.GenericRow;
+import org.apache.pinot.spi.utils.ReadMode;
+import org.apache.pinot.spi.utils.builder.TableConfigBuilder;
+import org.testng.Assert;
+import org.testng.annotations.BeforeClass;
+import org.testng.annotations.Test;
+
+
+public class ExplainPlanQueriesTest extends BaseQueriesTest {
+ private static final File INDEX_DIR = new File(FileUtils.getTempDirectory(), "ExplainPlanQueriesTest");
+ private static final String RAW_TABLE_NAME = "testTable";
+ private static final String SEGMENT_NAME = "testSegment";
+ private static final int NUM_RECORDS = 10;
+
+ private final static String COL1_NO_INDEX = "noIndexCol1";
+ private final static String COL2_NO_INDEX = "noIndexCol2";
+ private final static String COL3_NO_INDEX = "noIndexCol3";
+ private final static String COL1_INVERTED_INDEX = "invertedIndexCol1";
+ private final static String COL2_INVERTED_INDEX = "invertedIndexCol2";
+ private final static String COL3_INVERTED_INDEX = "invertedIndexCol3";
+ private final static String COL1_RANGE_INDEX = "rangeIndexCol1";
+ private final static String COL2_RANGE_INDEX = "rangeIndexCol2";
+ private final static String COL3_RANGE_INDEX = "rangeIndexCol3";
+ private final static String COL1_SORTED_INDEX = "sortedIndexCol1";
+ private final static String COL1_JSON_INDEX = "jsonIndexCol1";
+ private final static String COL1_TEXT_INDEX = "textIndexCol1";
+
+ private static final Schema SCHEMA = new Schema.SchemaBuilder().setSchemaName(RAW_TABLE_NAME)
+ .addSingleValueDimension(COL1_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL2_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_NO_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_INVERTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_INVERTED_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_INVERTED_INDEX, FieldSpec.DataType.STRING)
+ .addSingleValueDimension(COL1_RANGE_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL2_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL3_RANGE_INDEX, FieldSpec.DataType.INT)
+ .addSingleValueDimension(COL1_SORTED_INDEX, FieldSpec.DataType.DOUBLE)
+ .addSingleValueDimension(COL1_JSON_INDEX, FieldSpec.DataType.JSON)
+ .addSingleValueDimension(COL1_TEXT_INDEX, FieldSpec.DataType.STRING).build();
+
+ private static final DataSchema DATA_SCHEMA = new DataSchema(new String[]{"Operator", "Operator_Id", "Parent_Id"},
+ new DataSchema.ColumnDataType[]{DataSchema.ColumnDataType.STRING, DataSchema.ColumnDataType.INT,
+ DataSchema.ColumnDataType.INT});
+
+ private static final TableConfig TABLE_CONFIG =
+ new TableConfigBuilder(TableType.OFFLINE).setTableName(RAW_TABLE_NAME).build();
+
+ private IndexSegment _indexSegment;
+ private List<IndexSegment> _indexSegments;
+
+ @Override
+ protected String getFilter() {
+ return "";
+ }
+
+ @Override
+ protected IndexSegment getIndexSegment() {
+ return _indexSegment;
+ }
+
+ @Override
+ protected List<IndexSegment> getIndexSegments() {
+ return _indexSegments;
+ }
+
+ GenericRow createRecord(double doubleValue, int intValue, String stringValue, String jsonValue) {
+ GenericRow record = new GenericRow();
+ record.putValue(COL1_NO_INDEX, intValue);
+ record.putValue(COL2_NO_INDEX, intValue);
+ record.putValue(COL3_NO_INDEX, intValue);
+
+ record.putValue(COL1_INVERTED_INDEX, doubleValue);
+ record.putValue(COL2_INVERTED_INDEX, intValue);
+ record.putValue(COL3_INVERTED_INDEX, intValue);
+
+ record.putValue(COL1_RANGE_INDEX, doubleValue);
+ record.putValue(COL2_RANGE_INDEX, intValue);
+ record.putValue(COL3_RANGE_INDEX, intValue);
+
+ record.putValue(COL1_JSON_INDEX, jsonValue);
+ record.putValue(COL1_TEXT_INDEX, stringValue);
+
+ return record;
+ }
+
+ @BeforeClass
+ public void setUp()
+ throws Exception {
+ FileUtils.deleteDirectory(INDEX_DIR);
+
+ List<GenericRow> records = new ArrayList<>(NUM_RECORDS);
+ records.add(createRecord(1.1, 2, "daffy", "{\"first\": \"daffy\", \"last\": \"duck\"}"));
+
+ IndexingConfig indexingConfig = TABLE_CONFIG.getIndexingConfig();
+
+ List<String> invertedIndexColumns = Arrays.asList(COL1_INVERTED_INDEX, COL2_INVERTED_INDEX, COL3_INVERTED_INDEX);
+ indexingConfig.setInvertedIndexColumns(invertedIndexColumns);
+
+ List<String> rangeIndexColumns = Arrays.asList(COL1_RANGE_INDEX, COL2_RANGE_INDEX, COL3_RANGE_INDEX);
+ indexingConfig.setRangeIndexColumns(rangeIndexColumns);
+
+ List<String> jsonIndexColumns = Arrays.asList(COL1_JSON_INDEX);
+ indexingConfig.setJsonIndexColumns(jsonIndexColumns);
+
+ SegmentGeneratorConfig segmentGeneratorConfig = new SegmentGeneratorConfig(TABLE_CONFIG, SCHEMA);
+ segmentGeneratorConfig.setTableName(RAW_TABLE_NAME);
+ segmentGeneratorConfig.setSegmentName(SEGMENT_NAME);
+ segmentGeneratorConfig.setOutDir(INDEX_DIR.getPath());
+
+ SegmentIndexCreationDriverImpl driver = new SegmentIndexCreationDriverImpl();
+ driver.init(segmentGeneratorConfig, new GenericRowRecordReader(records));
+ driver.build();
+
+ IndexLoadingConfig indexLoadingConfig = new IndexLoadingConfig();
+ indexLoadingConfig.setTableConfig(TABLE_CONFIG);
+ indexLoadingConfig.setInvertedIndexColumns(new HashSet<>(invertedIndexColumns));
+ indexLoadingConfig.setRangeIndexColumns(new HashSet<>(rangeIndexColumns));
+ indexLoadingConfig.setJsonIndexColumns(new HashSet<String>(jsonIndexColumns));
+ indexLoadingConfig.setReadMode(ReadMode.mmap);
+
+ ImmutableSegment immutableSegment =
+ ImmutableSegmentLoader.load(new File(INDEX_DIR, SEGMENT_NAME), indexLoadingConfig);
+ _indexSegment = immutableSegment;
+ _indexSegments = Arrays.asList(immutableSegment);
+ }
+
+ /** Checks the correctness of EXPLAIN PLAN output. */
+ private void check(String query, ResultTable expected) {
+ BrokerResponseNative response = getBrokerResponseForSqlQuery(query);
+ ResultTable actual = response.getResultTable();
+
+ System.out.println(resultTableToString("Expected", expected));
+ System.out.println(resultTableToString("Actual", actual));
+
+ Assert.assertEquals(actual.getDataSchema(), expected.getDataSchema());
+ Assert.assertEquals(actual.getRows().size(), expected.getRows().size());
+
+ List<Object[]> expectedRows = expected.getRows();
+ List<Object[]> actualRows = actual.getRows();
+ for (int i = 0; i < expectedRows.size(); i++) {
+ Object[] expectedRow = expectedRows.get(i);
+ Object[] actualRow = actualRows.get(i);
+ Assert.assertEquals(actualRow.length, expectedRow.length);
+
+ for (int j = 0; j < actualRow.length; j++) {
+ Assert.assertEquals(actualRow[j], expectedRow[j]);
+ }
+ }
+ }
+
+ /** This function is used to generated expected results while creating test cases; otherwise, not used. */
+ private static String resultTableToString(String tag, ResultTable table) {
+ System.out.println("\n" + tag + ":");
+ List<Object[]> rows = table.getRows();
+
+ StringBuffer buffer = new StringBuffer();
+ for (Object[] row : rows) {
+ buffer.append("result.add(new Object[]{\"" + row[0] + "\", " + row[1] + ", " + row[2] + "});\n");
+ }
+
+ return buffer.toString();
+ }
+
+ @Test
+ public void testSelect() {
+ String query1 = "EXPLAIN PLAN FOR SELECT * FROM testTable";
+ List<Object[]> result1 = new ArrayList<>();
+ result1.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result1.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result1.add(
+ new Object[]{"SELECT(selectList:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, jsonIndexCol1, "
+ + "noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 2, 1});
+ result1.add(
+ new Object[]{"TRANSFORM_PASSTHROUGH(transformFuncs:invertedIndexCol1, invertedIndexCol2, invertedIndexCol3, "
+ + "jsonIndexCol1, noIndexCol1, noIndexCol2, noIndexCol3, rangeIndexCol1, rangeIndexCol2, rangeIndexCol3, "
+ + "sortedIndexCol1, textIndexCol1)", 3, 2});
+ result1.add(
+ new Object[]{"PROJECT(sortedIndexCol1, noIndexCol3, rangeIndexCol1, rangeIndexCol2, jsonIndexCol1, "
+ + "invertedIndexCol1, noIndexCol2, invertedIndexCol2, noIndexCol1, invertedIndexCol3, rangeIndexCol3, "
+ + "textIndexCol1)", 4, 3});
+ result1.add(new Object[]{"FILTER_MATCH_ENTIRE_SEGMENT(docs:1)", 5, 4});
+ check(query1, new ResultTable(DATA_SCHEMA, result1));
+
+ String query2 = "EXPLAIN PLAN FOR SELECT 'mickey' FROM testTable";
+ List<Object[]> result2 = new ArrayList<>();
+ result2.add(new Object[]{"BROKER_REDUCE(limit:10)", 0, -1});
+ result2.add(new Object[]{"COMBINE_SELECT", 1, 0});
+ result2.add(new Object[]{"SELECT(selectList:'mickey')", 2, 1});
+ result2.add(new Object[]{"PROJECT(ALL)", 3, 2});
Review comment:
I think it should have the name of columns for clarity.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] amrishlal commented on pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
amrishlal commented on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-982387933
> What do you think about merging `getExplainPlanName` into `toExplainString`?'
Yes, I don't see any issues with this. Will make the change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] amrishlal commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
amrishlal commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r758809059
##########
File path: pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseBrokerRequestHandler.java
##########
@@ -519,6 +543,18 @@ private BrokerResponseNative handleSQLRequest(long requestId, String query, Json
return brokerResponse;
}
+ /** Set EXPLAIN PLAN query to route to only one server. */
+ private void setRoutingToOneServer(Map<ServerInstance, List<String>> routingTable) {
+ Set<Map.Entry<ServerInstance, List<String>>> servers = routingTable.entrySet();
+ // only send request to 1 server
+ Map.Entry<ServerInstance, List<String>> server = servers.iterator().next();
+ routingTable.clear();
+ List<String> segments = new ArrayList<>();
+ // only generate the plan for 1 segment
+ segments.add(server.getValue().get(0));
+ routingTable.put(server.getKey(), segments);
Review comment:
Integration test (OfflineClusterIntegrationTest.testExplainPlanQuery) starts failing with this change due to explain plan output output having repeated AGGREGATE_GROUPBY_ORDERBY operators.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r759796029
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/filter/predicate/PredicateEvaluatorProvider.java
##########
@@ -37,47 +37,65 @@ private PredicateEvaluatorProvider() {
public static PredicateEvaluator getPredicateEvaluator(Predicate predicate, @Nullable Dictionary dictionary,
DataType dataType) {
+ PredicateEvaluator predicateEvaluator;
Review comment:
This file can be reverted
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/DocIdSetOperator.java
##########
@@ -34,6 +37,7 @@
*/
public class DocIdSetOperator extends BaseOperator<DocIdSetBlock> {
private static final String OPERATOR_NAME = "DocIdSetOperator";
+ private static final String EXPLAIN_NAME = null;
Review comment:
Why do we want to skip this operator? It should sit between filter operator and projection operator
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/common/Operator.java
##########
@@ -46,6 +47,12 @@
*/
String getOperatorName();
+ /** @return List of {@link Operator}s that this operator depends upon. */
+ List<Operator> getChildOperators();
+
+ /** @return Description of this operator for Explain Plan */
+ String toExplainString();
Review comment:
Shall we annotate it as `nullable` and add some javadoc explaining the expected behavior when it returns `null`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7568: [WORK IN PROGRESS] EXPLAIN PLAN code
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7568](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (f228b9d) into [master](https://codecov.io/gh/apache/pinot/commit/1dda74a6ecd96e2704147ad803135d5046a1e13d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (1dda74a) will **decrease** coverage by `6.73%`.
> The diff coverage is `62.76%`.
[](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7568 +/- ##
============================================
- Coverage 72.63% 65.89% -6.74%
- Complexity 3416 3423 +7
============================================
Files 1522 1477 -45
Lines 75905 74410 -1495
Branches 11093 10953 -140
============================================
- Hits 55133 49036 -6097
- Misses 17075 21920 +4845
+ Partials 3697 3454 -243
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| integration2 | `?` | |
| unittests1 | `69.91% <64.36%> (+0.06%)` | :arrow_up: |
| unittests2 | `15.15% <2.08%> (-0.10%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...operator/AcquireReleaseColumnsSegmentOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9BY3F1aXJlUmVsZWFzZUNvbHVtbnNTZWdtZW50T3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...he/pinot/core/operator/BitmapDocIdSetOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CaXRtYXBEb2NJZFNldE9wZXJhdG9yLmphdmE=) | `71.42% <0.00%> (-3.58%)` | :arrow_down: |
| [.../core/operator/combine/GroupByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0dyb3VwQnlDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `78.68% <0.00%> (-1.32%)` | :arrow_down: |
| [...nMaxValueBasedSelectionOrderByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL01pbk1heFZhbHVlQmFzZWRTZWxlY3Rpb25PcmRlckJ5Q29tYmluZU9wZXJhdG9yLmphdmE=) | `61.90% <0.00%> (-13.30%)` | :arrow_down: |
| [...rator/combine/SelectionOrderByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL1NlbGVjdGlvbk9yZGVyQnlDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `23.33% <0.00%> (-28.40%)` | :arrow_down: |
| [...or/dociditerators/ExpressionScanDocIdIterator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9kb2NpZGl0ZXJhdG9ycy9FeHByZXNzaW9uU2NhbkRvY0lkSXRlcmF0b3IuamF2YQ==) | `47.51% <0.00%> (-18.92%)` | :arrow_down: |
| [...ore/operator/filter/BitmapBasedFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvQml0bWFwQmFzZWRGaWx0ZXJPcGVyYXRvci5qYXZh) | `58.69% <0.00%> (-33.62%)` | :arrow_down: |
| [...ot/core/operator/filter/H3IndexFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvSDNJbmRleEZpbHRlck9wZXJhdG9yLmphdmE=) | `82.10% <0.00%> (-4.57%)` | :arrow_down: |
| [...e/pinot/core/operator/filter/OrFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvT3JGaWx0ZXJPcGVyYXRvci5qYXZh) | `56.25% <0.00%> (-33.75%)` | :arrow_down: |
| [...operator/filter/RangeIndexBasedFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvUmFuZ2VJbmRleEJhc2VkRmlsdGVyT3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (-40.43%)` | :arrow_down: |
| ... and [423 more](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [1dda74a...f228b9d](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] mayankshriv commented on pull request #7568: [WORK IN PROGRESS] EXPLAIN PLAN code
Posted by GitBox <gi...@apache.org>.
mayankshriv commented on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941454856
This seems like it should go through PEP? If there's a thread on that already, perhaps we can link it here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] mayankshriv commented on pull request #7568: [WORK IN PROGRESS] EXPLAIN PLAN code
Posted by GitBox <gi...@apache.org>.
mayankshriv commented on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941454856
This seems like it should go through PEP? If there's a thread on that already, perhaps we can link it here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7568: [WORK IN PROGRESS] EXPLAIN PLAN code
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7568: [WORK IN PROGRESS] EXPLAIN PLAN code
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7568](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (de9bace) into [master](https://codecov.io/gh/apache/pinot/commit/1dda74a6ecd96e2704147ad803135d5046a1e13d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (1dda74a) will **decrease** coverage by `43.78%`.
> The diff coverage is `12.76%`.
[](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7568 +/- ##
=============================================
- Coverage 72.63% 28.85% -43.79%
=============================================
Files 1522 1514 -8
Lines 75905 75913 +8
Branches 11093 11115 +22
=============================================
- Hits 55133 21903 -33230
- Misses 17075 52018 +34943
+ Partials 3697 1992 -1705
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| integration2 | `28.85% <12.76%> (-0.09%)` | :arrow_down: |
| unittests1 | `?` | |
| unittests2 | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...operator/AcquireReleaseColumnsSegmentOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9BY3F1aXJlUmVsZWFzZUNvbHVtbnNTZWdtZW50T3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...a/org/apache/pinot/core/operator/BaseOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CYXNlT3BlcmF0b3IuamF2YQ==) | `31.25% <0.00%> (-4.47%)` | :arrow_down: |
| [...he/pinot/core/operator/BitmapDocIdSetOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CaXRtYXBEb2NJZFNldE9wZXJhdG9yLmphdmE=) | `71.42% <0.00%> (-3.58%)` | :arrow_down: |
| [.../pinot/core/operator/InstanceResponseOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9JbnN0YW5jZVJlc3BvbnNlT3BlcmF0b3IuamF2YQ==) | `78.78% <0.00%> (-5.09%)` | :arrow_down: |
| [...apache/pinot/core/operator/ProjectionOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9Qcm9qZWN0aW9uT3BlcmF0b3IuamF2YQ==) | `39.28% <0.00%> (-34.05%)` | :arrow_down: |
| [...erator/combine/AggregationOnlyCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0FnZ3JlZ2F0aW9uT25seUNvbWJpbmVPcGVyYXRvci5qYXZh) | `69.23% <0.00%> (-5.77%)` | :arrow_down: |
| [...not/core/operator/combine/BaseCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0Jhc2VDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `73.33% <0.00%> (-14.51%)` | :arrow_down: |
| [...core/operator/combine/DistinctCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0Rpc3RpbmN0Q29tYmluZU9wZXJhdG9yLmphdmE=) | `66.66% <0.00%> (-14.11%)` | :arrow_down: |
| [.../core/operator/combine/GroupByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0dyb3VwQnlDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `77.04% <0.00%> (-2.96%)` | :arrow_down: |
| [...perator/combine/GroupByOrderByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0dyb3VwQnlPcmRlckJ5Q29tYmluZU9wZXJhdG9yLmphdmE=) | `59.30% <0.00%> (-21.88%)` | :arrow_down: |
| ... and [1135 more](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [1dda74a...de9bace](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7568: [WORK IN PROGRESS] EXPLAIN PLAN code
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7568](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (0ab950d) into [master](https://codecov.io/gh/apache/pinot/commit/5eee297dbea7b3b73ae853f2479a89e9ce017363?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (5eee297) will **decrease** coverage by `17.00%`.
> The diff coverage is `1.92%`.
[](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7568 +/- ##
=============================================
- Coverage 32.16% 15.16% -17.01%
- Complexity 0 80 +80
=============================================
Files 1513 1477 -36
Lines 75557 74418 -1139
Branches 11055 10949 -106
=============================================
- Hits 24305 11284 -13021
- Misses 49156 62323 +13167
+ Partials 2096 811 -1285
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| integration2 | `?` | |
| unittests2 | `15.16% <1.92%> (?)` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...java/org/apache/pinot/common/utils/DataSchema.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vdXRpbHMvRGF0YVNjaGVtYS5qYXZh) | `0.00% <0.00%> (-42.68%)` | :arrow_down: |
| [...org/apache/pinot/sql/parsers/CalciteSqlParser.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcWwvcGFyc2Vycy9DYWxjaXRlU3FsUGFyc2VyLmphdmE=) | `0.00% <0.00%> (-70.09%)` | :arrow_down: |
| [.../transform/function/ConstructFromTextFunction.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9nZW9zcGF0aWFsL3RyYW5zZm9ybS9mdW5jdGlvbi9Db25zdHJ1Y3RGcm9tVGV4dEZ1bmN0aW9uLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...l/transform/function/ConstructFromWKBFunction.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9nZW9zcGF0aWFsL3RyYW5zZm9ybS9mdW5jdGlvbi9Db25zdHJ1Y3RGcm9tV0tCRnVuY3Rpb24uamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...geospatial/transform/function/ScalarFunctions.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9nZW9zcGF0aWFsL3RyYW5zZm9ybS9mdW5jdGlvbi9TY2FsYXJGdW5jdGlvbnMuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...spatial/transform/function/StAsBinaryFunction.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9nZW9zcGF0aWFsL3RyYW5zZm9ybS9mdW5jdGlvbi9TdEFzQmluYXJ5RnVuY3Rpb24uamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...eospatial/transform/function/StAsTextFunction.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9nZW9zcGF0aWFsL3RyYW5zZm9ybS9mdW5jdGlvbi9TdEFzVGV4dEZ1bmN0aW9uLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...ial/transform/function/StGeogFromTextFunction.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9nZW9zcGF0aWFsL3RyYW5zZm9ybS9mdW5jdGlvbi9TdEdlb2dGcm9tVGV4dEZ1bmN0aW9uLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...tial/transform/function/StGeogFromWKBFunction.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9nZW9zcGF0aWFsL3RyYW5zZm9ybS9mdW5jdGlvbi9TdEdlb2dGcm9tV0tCRnVuY3Rpb24uamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...ial/transform/function/StGeomFromTextFunction.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9nZW9zcGF0aWFsL3RyYW5zZm9ybS9mdW5jdGlvbi9TdEdlb21Gcm9tVGV4dEZ1bmN0aW9uLmphdmE=) | `0.00% <0.00%> (ø)` | |
| ... and [845 more](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [0ecfc11...0ab950d](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] amrishlal commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
amrishlal commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r746092017
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/plan/FilterPlanNode.java
##########
@@ -217,9 +216,9 @@ private BaseFilterOperator constructPhysicalOperator(FilterContext filter) {
return FilterOperatorUtils.getLeafFilterOperator(predicateEvaluator, dataSource, _numDocs);
case JSON_MATCH:
JsonIndexReader jsonIndex = dataSource.getJsonIndex();
- Preconditions.checkState(jsonIndex != null, "Cannot apply JSON_MATCH on column: %s without json index",
- column);
- return new JsonMatchFilterOperator(jsonIndex, ((JsonMatchPredicate) predicate).getValue(), _numDocs);
+ Preconditions
+ .checkState(jsonIndex != null, "Cannot apply JSON_MATCH on column: %s without json index", column);
Review comment:
Leaving this out as it appears to be unrelated to this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7568](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (f140c7b) into [master](https://codecov.io/gh/apache/pinot/commit/2107b2cd1880ec74934ec5412dbee8c874f05610?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (2107b2c) will **increase** coverage by `39.60%`.
> The diff coverage is `73.50%`.
[](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7568 +/- ##
=============================================
+ Coverage 30.79% 70.39% +39.60%
- Complexity 0 4079 +4079
=============================================
Files 1570 1579 +9
Lines 80024 80996 +972
Branches 11904 12046 +142
=============================================
+ Hits 24644 57019 +32375
+ Misses 53274 20067 -33207
- Partials 2106 3910 +1804
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| integration2 | `27.88% <41.03%> (+0.18%)` | :arrow_up: |
| unittests1 | `68.66% <70.63%> (?)` | |
| unittests2 | `14.51% <2.07%> (?)` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...in/java/org/apache/pinot/core/common/Operator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9jb21tb24vT3BlcmF0b3IuamF2YQ==) | `0.00% <ø> (ø)` | |
| [...operator/AcquireReleaseColumnsSegmentOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9BY3F1aXJlUmVsZWFzZUNvbHVtbnNTZWdtZW50T3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...he/pinot/core/operator/BitmapDocIdSetOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CaXRtYXBEb2NJZFNldE9wZXJhdG9yLmphdmE=) | `68.18% <0.00%> (-6.82%)` | :arrow_down: |
| [.../core/operator/combine/GroupByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0dyb3VwQnlDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `73.91% <0.00%> (-5.46%)` | :arrow_down: |
| [...nMaxValueBasedSelectionOrderByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL01pbk1heFZhbHVlQmFzZWRTZWxlY3Rpb25PcmRlckJ5Q29tYmluZU9wZXJhdG9yLmphdmE=) | `71.96% <0.00%> (-3.43%)` | :arrow_down: |
| [...or/dociditerators/ExpressionScanDocIdIterator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9kb2NpZGl0ZXJhdG9ycy9FeHByZXNzaW9uU2NhbkRvY0lkSXRlcmF0b3IuamF2YQ==) | `52.11% <0.00%> (-10.75%)` | :arrow_down: |
| [...inot/core/operator/filter/EmptyFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvRW1wdHlGaWx0ZXJPcGVyYXRvci5qYXZh) | `62.50% <0.00%> (-17.50%)` | :arrow_down: |
| [...ot/core/operator/filter/H3IndexFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvSDNJbmRleEZpbHRlck9wZXJhdG9yLmphdmE=) | `81.25% <0.00%> (+81.25%)` | :arrow_up: |
| [...ore/operator/query/AggregationGroupByOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9xdWVyeS9BZ2dyZWdhdGlvbkdyb3VwQnlPcGVyYXRvci5qYXZh) | `47.05% <0.00%> (-47.68%)` | :arrow_down: |
| [...perator/query/DictionaryBasedDistinctOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9xdWVyeS9EaWN0aW9uYXJ5QmFzZWREaXN0aW5jdE9wZXJhdG9yLmphdmE=) | `75.00% <0.00%> (+25.00%)` | :arrow_up: |
| ... and [1191 more](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [2107b2c...f140c7b](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7568](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (4deac05) into [master](https://codecov.io/gh/apache/pinot/commit/2107b2cd1880ec74934ec5412dbee8c874f05610?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (2107b2c) will **decrease** coverage by `2.82%`.
> The diff coverage is `41.03%`.
[](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7568 +/- ##
==========================================
- Coverage 30.79% 27.96% -2.83%
==========================================
Files 1570 1570
Lines 80024 80638 +614
Branches 11904 12004 +100
==========================================
- Hits 24644 22554 -2090
- Misses 53274 56042 +2768
+ Partials 2106 2042 -64
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| integration2 | `27.96% <41.03%> (+0.26%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...in/java/org/apache/pinot/core/common/Operator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9jb21tb24vT3BlcmF0b3IuamF2YQ==) | `0.00% <ø> (ø)` | |
| [...operator/AcquireReleaseColumnsSegmentOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9BY3F1aXJlUmVsZWFzZUNvbHVtbnNTZWdtZW50T3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...he/pinot/core/operator/BitmapDocIdSetOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CaXRtYXBEb2NJZFNldE9wZXJhdG9yLmphdmE=) | `68.18% <0.00%> (-6.82%)` | :arrow_down: |
| [...erator/combine/AggregationOnlyCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0FnZ3JlZ2F0aW9uT25seUNvbWJpbmVPcGVyYXRvci5qYXZh) | `69.23% <0.00%> (-5.77%)` | :arrow_down: |
| [...core/operator/combine/DistinctCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0Rpc3RpbmN0Q29tYmluZU9wZXJhdG9yLmphdmE=) | `77.77% <0.00%> (+8.54%)` | :arrow_up: |
| [.../core/operator/combine/GroupByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0dyb3VwQnlDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `71.01% <0.00%> (-8.36%)` | :arrow_down: |
| [...nMaxValueBasedSelectionOrderByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL01pbk1heFZhbHVlQmFzZWRTZWxlY3Rpb25PcmRlckJ5Q29tYmluZU9wZXJhdG9yLmphdmE=) | `71.96% <0.00%> (-3.43%)` | :arrow_down: |
| [...operator/combine/SelectionOnlyCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL1NlbGVjdGlvbk9ubHlDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `56.66% <0.00%> (-1.96%)` | :arrow_down: |
| [...rator/combine/SelectionOrderByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL1NlbGVjdGlvbk9yZGVyQnlDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `50.00% <0.00%> (-1.73%)` | :arrow_down: |
| [...or/dociditerators/ExpressionScanDocIdIterator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9kb2NpZGl0ZXJhdG9ycy9FeHByZXNzaW9uU2NhbkRvY0lkSXRlcmF0b3IuamF2YQ==) | `48.59% <0.00%> (-14.27%)` | :arrow_down: |
| ... and [304 more](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [2107b2c...4deac05](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] amrishlal commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
amrishlal commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r749486130
##########
File path: pinot-common/src/main/java/org/apache/pinot/common/request/PinotQuery.java
##########
@@ -593,6 +604,28 @@ public void setQueryOptionsIsSet(boolean value) {
}
}
+ public boolean isExplain() {
+ return this.explain;
+ }
+
+ public void setExplain(boolean explain) {
+ this.explain = explain;
+ setExplainIsSet(true);
Review comment:
> Can we also add some tests under integration tests, like `OfflineClusterIntegrationTest`?
Added one integration test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] amrishlal commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
amrishlal commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r758809059
##########
File path: pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseBrokerRequestHandler.java
##########
@@ -519,6 +543,18 @@ private BrokerResponseNative handleSQLRequest(long requestId, String query, Json
return brokerResponse;
}
+ /** Set EXPLAIN PLAN query to route to only one server. */
+ private void setRoutingToOneServer(Map<ServerInstance, List<String>> routingTable) {
+ Set<Map.Entry<ServerInstance, List<String>>> servers = routingTable.entrySet();
+ // only send request to 1 server
+ Map.Entry<ServerInstance, List<String>> server = servers.iterator().next();
+ routingTable.clear();
+ List<String> segments = new ArrayList<>();
+ // only generate the plan for 1 segment
+ segments.add(server.getValue().get(0));
+ routingTable.put(server.getKey(), segments);
Review comment:
Done
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/common/Operator.java
##########
@@ -39,12 +40,20 @@
*/
T nextBlock();
+ /** @return Name of this operator */
+ String getOperatorName();
+
+ /** @return List of {@link Operator}s that this operator depends upon. */
+ List<Operator> getChildOperators();
+
/**
- * Returns the name of the operator.
- * NOTE: This method is called for tracing purpose. The sub-class should try to return a constant to avoid the
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7568](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (b485141) into [master](https://codecov.io/gh/apache/pinot/commit/d1727372a5bfd48b6f3c1fc0c8a22d39c243c1bc?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (d172737) will **decrease** coverage by `1.19%`.
> The diff coverage is `71.64%`.
[](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7568 +/- ##
============================================
- Coverage 71.64% 70.44% -1.20%
- Complexity 4083 4093 +10
============================================
Files 1579 1580 +1
Lines 80819 81136 +317
Branches 12017 12077 +60
============================================
- Hits 57902 57158 -744
- Misses 19017 20069 +1052
- Partials 3900 3909 +9
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| integration2 | `27.91% <38.50%> (+0.03%)` | :arrow_up: |
| unittests1 | `68.78% <70.25%> (+0.13%)` | :arrow_up: |
| unittests2 | `14.58% <1.49%> (-0.05%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...in/java/org/apache/pinot/core/common/Operator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9jb21tb24vT3BlcmF0b3IuamF2YQ==) | `0.00% <ø> (ø)` | |
| [...operator/AcquireReleaseColumnsSegmentOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9BY3F1aXJlUmVsZWFzZUNvbHVtbnNTZWdtZW50T3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...he/pinot/core/operator/BitmapDocIdSetOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CaXRtYXBEb2NJZFNldE9wZXJhdG9yLmphdmE=) | `68.18% <0.00%> (-6.82%)` | :arrow_down: |
| [.../core/operator/combine/GroupByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0dyb3VwQnlDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `73.91% <0.00%> (-1.09%)` | :arrow_down: |
| [...nMaxValueBasedSelectionOrderByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL01pbk1heFZhbHVlQmFzZWRTZWxlY3Rpb25PcmRlckJ5Q29tYmluZU9wZXJhdG9yLmphdmE=) | `71.96% <0.00%> (-0.55%)` | :arrow_down: |
| [...or/dociditerators/ExpressionScanDocIdIterator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9kb2NpZGl0ZXJhdG9ycy9FeHByZXNzaW9uU2NhbkRvY0lkSXRlcmF0b3IuamF2YQ==) | `52.11% <0.00%> (-14.32%)` | :arrow_down: |
| [...inot/core/operator/filter/EmptyFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvRW1wdHlGaWx0ZXJPcGVyYXRvci5qYXZh) | `57.14% <0.00%> (-22.86%)` | :arrow_down: |
| [...ot/core/operator/filter/H3IndexFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvSDNJbmRleEZpbHRlck9wZXJhdG9yLmphdmE=) | `82.10% <0.00%> (-4.57%)` | :arrow_down: |
| [...ore/operator/query/AggregationGroupByOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9xdWVyeS9BZ2dyZWdhdGlvbkdyb3VwQnlPcGVyYXRvci5qYXZh) | `48.48% <0.00%> (-46.26%)` | :arrow_down: |
| [...perator/query/DictionaryBasedDistinctOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9xdWVyeS9EaWN0aW9uYXJ5QmFzZWREaXN0aW5jdE9wZXJhdG9yLmphdmE=) | `75.00% <0.00%> (-3.27%)` | :arrow_down: |
| ... and [158 more](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [d172737...b485141](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] amrishlal commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
amrishlal commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r759008622
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/filter/predicate/PredicateEvaluator.java
##########
@@ -28,6 +28,16 @@
* APIs for both dictionary based and raw value based predicate evaluator
*/
+ /**
+ * Set the predicate
+ */
+ void setPredicate(Predicate predicate);
Review comment:
Please merge the PR and I will pick it up here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] amrishlal commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
amrishlal commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r759887289
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/common/Operator.java
##########
@@ -46,6 +47,12 @@
*/
String getOperatorName();
+ /** @return List of {@link Operator}s that this operator depends upon. */
+ List<Operator> getChildOperators();
+
+ /** @return Description of this operator for Explain Plan */
+ String toExplainString();
Review comment:
Done
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/filter/predicate/PredicateEvaluatorProvider.java
##########
@@ -37,47 +37,65 @@ private PredicateEvaluatorProvider() {
public static PredicateEvaluator getPredicateEvaluator(Predicate predicate, @Nullable Dictionary dictionary,
DataType dataType) {
+ PredicateEvaluator predicateEvaluator;
Review comment:
Fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r760502909
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/common/Operator.java
##########
@@ -46,6 +47,12 @@
*/
String getOperatorName();
+ /** @return List of {@link Operator}s that this operator depends upon. */
+ List<Operator> getChildOperators();
+
+ /** @return Description of this operator for Explain Plan */
+ String toExplainString();
Review comment:
Let's add something like `when it returns null, the operator won't be included in the explain plan tree`
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/DocIdSetOperator.java
##########
@@ -34,6 +37,7 @@
*/
public class DocIdSetOperator extends BaseOperator<DocIdSetBlock> {
private static final String OPERATOR_NAME = "DocIdSetOperator";
+ private static final String EXPLAIN_NAME = null;
Review comment:
Sure, I don't see this as a blocker, but I do feel this level is useful to understand the query execution layers, so suggest including it in the explain tree
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7568](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (7564824) into [master](https://codecov.io/gh/apache/pinot/commit/2107b2cd1880ec74934ec5412dbee8c874f05610?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (2107b2c) will **increase** coverage by `40.75%`.
> The diff coverage is `64.32%`.
[](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7568 +/- ##
=============================================
+ Coverage 30.79% 71.54% +40.75%
- Complexity 0 4034 +4034
=============================================
Files 1570 1580 +10
Lines 80024 80714 +690
Branches 11904 12001 +97
=============================================
+ Hits 24644 57748 +33104
+ Misses 53274 19063 -34211
- Partials 2106 3903 +1797
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `29.22% <13.78%> (-0.01%)` | :arrow_down: |
| integration2 | `27.63% <13.24%> (-0.08%)` | :arrow_down: |
| unittests1 | `68.60% <63.50%> (?)` | |
| unittests2 | `14.50% <2.16%> (?)` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...in/java/org/apache/pinot/core/common/Operator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9jb21tb24vT3BlcmF0b3IuamF2YQ==) | `0.00% <ø> (ø)` | |
| [...operator/AcquireReleaseColumnsSegmentOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9BY3F1aXJlUmVsZWFzZUNvbHVtbnNTZWdtZW50T3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...he/pinot/core/operator/BitmapDocIdSetOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CaXRtYXBEb2NJZFNldE9wZXJhdG9yLmphdmE=) | `71.42% <0.00%> (-3.58%)` | :arrow_down: |
| [.../core/operator/combine/GroupByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0dyb3VwQnlDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `78.12% <0.00%> (-1.25%)` | :arrow_down: |
| [...nMaxValueBasedSelectionOrderByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL01pbk1heFZhbHVlQmFzZWRTZWxlY3Rpb25PcmRlckJ5Q29tYmluZU9wZXJhdG9yLmphdmE=) | `74.01% <0.00%> (-1.39%)` | :arrow_down: |
| [...rator/combine/SelectionOrderByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL1NlbGVjdGlvbk9yZGVyQnlDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `50.00% <0.00%> (-1.73%)` | :arrow_down: |
| [...or/dociditerators/ExpressionScanDocIdIterator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9kb2NpZGl0ZXJhdG9ycy9FeHByZXNzaW9uU2NhbkRvY0lkSXRlcmF0b3IuamF2YQ==) | `65.95% <0.00%> (+3.10%)` | :arrow_up: |
| [...ore/operator/filter/BitmapBasedFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvQml0bWFwQmFzZWRGaWx0ZXJPcGVyYXRvci5qYXZh) | `78.26% <0.00%> (-14.05%)` | :arrow_down: |
| [...ot/core/operator/filter/H3IndexFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvSDNJbmRleEZpbHRlck9wZXJhdG9yLmphdmE=) | `82.10% <0.00%> (+82.10%)` | :arrow_up: |
| [...e/pinot/core/operator/filter/OrFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvT3JGaWx0ZXJPcGVyYXRvci5qYXZh) | `75.00% <0.00%> (-15.00%)` | :arrow_down: |
| ... and [1122 more](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [2107b2c...7564824](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7568](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (7564824) into [master](https://codecov.io/gh/apache/pinot/commit/2107b2cd1880ec74934ec5412dbee8c874f05610?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (2107b2c) will **increase** coverage by `39.50%`.
> The diff coverage is `64.32%`.
[](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7568 +/- ##
=============================================
+ Coverage 30.79% 70.30% +39.50%
- Complexity 0 4034 +4034
=============================================
Files 1570 1580 +10
Lines 80024 80714 +690
Branches 11904 12001 +97
=============================================
+ Hits 24644 56742 +32098
+ Misses 53274 20083 -33191
- Partials 2106 3889 +1783
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| integration2 | `27.63% <13.24%> (-0.08%)` | :arrow_down: |
| unittests1 | `68.60% <63.50%> (?)` | |
| unittests2 | `14.50% <2.16%> (?)` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...in/java/org/apache/pinot/core/common/Operator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9jb21tb24vT3BlcmF0b3IuamF2YQ==) | `0.00% <ø> (ø)` | |
| [...operator/AcquireReleaseColumnsSegmentOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9BY3F1aXJlUmVsZWFzZUNvbHVtbnNTZWdtZW50T3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...he/pinot/core/operator/BitmapDocIdSetOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CaXRtYXBEb2NJZFNldE9wZXJhdG9yLmphdmE=) | `71.42% <0.00%> (-3.58%)` | :arrow_down: |
| [.../core/operator/combine/GroupByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0dyb3VwQnlDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `78.12% <0.00%> (-1.25%)` | :arrow_down: |
| [...nMaxValueBasedSelectionOrderByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL01pbk1heFZhbHVlQmFzZWRTZWxlY3Rpb25PcmRlckJ5Q29tYmluZU9wZXJhdG9yLmphdmE=) | `74.01% <0.00%> (-1.39%)` | :arrow_down: |
| [...rator/combine/SelectionOrderByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL1NlbGVjdGlvbk9yZGVyQnlDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `50.00% <0.00%> (-1.73%)` | :arrow_down: |
| [...or/dociditerators/ExpressionScanDocIdIterator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9kb2NpZGl0ZXJhdG9ycy9FeHByZXNzaW9uU2NhbkRvY0lkSXRlcmF0b3IuamF2YQ==) | `52.48% <0.00%> (-10.38%)` | :arrow_down: |
| [...ore/operator/filter/BitmapBasedFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvQml0bWFwQmFzZWRGaWx0ZXJPcGVyYXRvci5qYXZh) | `78.26% <0.00%> (-14.05%)` | :arrow_down: |
| [...ot/core/operator/filter/H3IndexFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvSDNJbmRleEZpbHRlck9wZXJhdG9yLmphdmE=) | `82.10% <0.00%> (+82.10%)` | :arrow_up: |
| [...e/pinot/core/operator/filter/OrFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvT3JGaWx0ZXJPcGVyYXRvci5qYXZh) | `75.00% <0.00%> (-15.00%)` | :arrow_down: |
| ... and [1170 more](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [2107b2c...7564824](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter edited a comment on pull request #7568: [WORK IN PROGRESS] EXPLAIN PLAN code
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#issuecomment-941686660
# [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#7568](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (f228b9d) into [master](https://codecov.io/gh/apache/pinot/commit/1dda74a6ecd96e2704147ad803135d5046a1e13d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (1dda74a) will **decrease** coverage by `0.04%`.
> The diff coverage is `65.10%`.
[](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #7568 +/- ##
============================================
- Coverage 72.63% 72.59% -0.05%
- Complexity 3416 3423 +7
============================================
Files 1522 1523 +1
Lines 75905 76261 +356
Branches 11093 11153 +60
============================================
+ Hits 55133 55361 +228
- Misses 17075 17191 +116
- Partials 3697 3709 +12
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `30.48% <13.28%> (-0.19%)` | :arrow_down: |
| integration2 | `28.83% <12.76%> (-0.10%)` | :arrow_down: |
| unittests1 | `69.91% <64.36%> (+0.06%)` | :arrow_up: |
| unittests2 | `15.15% <2.08%> (-0.10%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...operator/AcquireReleaseColumnsSegmentOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9BY3F1aXJlUmVsZWFzZUNvbHVtbnNTZWdtZW50T3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...he/pinot/core/operator/BitmapDocIdSetOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CaXRtYXBEb2NJZFNldE9wZXJhdG9yLmphdmE=) | `71.42% <0.00%> (-3.58%)` | :arrow_down: |
| [.../core/operator/combine/GroupByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL0dyb3VwQnlDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `78.68% <0.00%> (-1.32%)` | :arrow_down: |
| [...nMaxValueBasedSelectionOrderByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL01pbk1heFZhbHVlQmFzZWRTZWxlY3Rpb25PcmRlckJ5Q29tYmluZU9wZXJhdG9yLmphdmE=) | `73.80% <0.00%> (-1.40%)` | :arrow_down: |
| [...rator/combine/SelectionOrderByCombineOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9jb21iaW5lL1NlbGVjdGlvbk9yZGVyQnlDb21iaW5lT3BlcmF0b3IuamF2YQ==) | `50.00% <0.00%> (-1.73%)` | :arrow_down: |
| [...or/dociditerators/ExpressionScanDocIdIterator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9kb2NpZGl0ZXJhdG9ycy9FeHByZXNzaW9uU2NhbkRvY0lkSXRlcmF0b3IuamF2YQ==) | `65.95% <0.00%> (-0.48%)` | :arrow_down: |
| [...ore/operator/filter/BitmapBasedFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvQml0bWFwQmFzZWRGaWx0ZXJPcGVyYXRvci5qYXZh) | `78.26% <0.00%> (-14.05%)` | :arrow_down: |
| [...ot/core/operator/filter/H3IndexFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvSDNJbmRleEZpbHRlck9wZXJhdG9yLmphdmE=) | `82.10% <0.00%> (-4.57%)` | :arrow_down: |
| [...e/pinot/core/operator/filter/OrFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvT3JGaWx0ZXJPcGVyYXRvci5qYXZh) | `56.25% <0.00%> (-33.75%)` | :arrow_down: |
| [...operator/filter/RangeIndexBasedFilterOperator.java](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9maWx0ZXIvUmFuZ2VJbmRleEJhc2VkRmlsdGVyT3BlcmF0b3IuamF2YQ==) | `36.53% <0.00%> (-3.89%)` | :arrow_down: |
| ... and [88 more](https://codecov.io/gh/apache/pinot/pull/7568/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [1dda74a...f228b9d](https://codecov.io/gh/apache/pinot/pull/7568?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on a change in pull request #7568: EXPLAIN PLAN
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #7568:
URL: https://github.com/apache/pinot/pull/7568#discussion_r755656841
##########
File path: pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseBrokerRequestHandler.java
##########
@@ -519,6 +543,18 @@ private BrokerResponseNative handleSQLRequest(long requestId, String query, Json
return brokerResponse;
}
+ /** Set EXPLAIN PLAN query to route to only one server. */
+ private void setRoutingToOneServer(Map<ServerInstance, List<String>> routingTable) {
+ Set<Map.Entry<ServerInstance, List<String>>> servers = routingTable.entrySet();
+ // only send request to 1 server
+ Map.Entry<ServerInstance, List<String>> server = servers.iterator().next();
+ routingTable.clear();
+ List<String> segments = new ArrayList<>();
+ // only generate the plan for 1 segment
+ segments.add(server.getValue().get(0));
+ routingTable.put(server.getKey(), segments);
Review comment:
No need to create an extra list
```suggestion
routingTable.put(entry.getKey(), entry.getValue());
```
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/InstanceResponseOperator.java
##########
@@ -113,9 +116,34 @@ private IntermediateResultsBlock getCombinedResults() {
}
}
- @Override
- public String getOperatorName() {
- return OPERATOR_NAME;
+ /*
+ * Calculate totalThreadCpuTimeNs based on totalWallClockTimeNs, multipleThreadCpuTimeNs, and numServerThreads.
+ * System activities time such as OS paging, GC, context switching are not captured by totalThreadCpuTimeNs.
+ * For example, let's divide query processing into 4 phases:
+ * - phase 1: single thread preparing. Time used: T1
+ * - phase 2: N threads processing segments in parallel, each thread use time T2
+ * - phase 3: GC/OS paging. Time used: T3
+ * - phase 4: single thread merging intermediate results blocks. Time used: T4
+ *
+ * Then we have following equations:
+ * - singleThreadCpuTimeNs = T1 + T4
+ * - multipleThreadCpuTimeNs = T2 * N
+ * - totalWallClockTimeNs = T1 + T2 + T3 + T4 = singleThreadCpuTimeNs + T2 + T3
+ * - totalThreadCpuTimeNsWithoutSystemActivities = T1 + T2 * N + T4 = singleThreadCpuTimeNs + T2 * N
+ * - systemActivitiesTimeNs = T3 = (totalWallClockTimeNs - totalThreadCpuTimeNsWithoutSystemActivities) + T2 * (N - 1)
+ *
+ * Thus:
+ * totalThreadCpuTimeNsWithSystemActivities = totalThreadCpuTimeNsWithoutSystemActivities + systemActivitiesTimeNs
+ * = totalThreadCpuTimeNsWithoutSystemActivities + T3
+ * = totalThreadCpuTimeNsWithoutSystemActivities + (totalWallClockTimeNs -
+ * totalThreadCpuTimeNsWithoutSystemActivities) + T2 * (N - 1)
+ * = totalWallClockTimeNs + T2 * (N - 1)
+ * = totalWallClockTimeNs + (multipleThreadCpuTimeNs / N) * (N - 1)
+ */
+ public static long calTotalThreadCpuTimeNs(long totalWallClockTimeNs, long multipleThreadCpuTimeNs,
Review comment:
Is this relevant to this PR?
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/common/Operator.java
##########
@@ -39,12 +40,20 @@
*/
T nextBlock();
+ /** @return Name of this operator */
+ String getOperatorName();
+
+ /** @return List of {@link Operator}s that this operator depends upon. */
+ List<Operator> getChildOperators();
+
/**
- * Returns the name of the operator.
- * NOTE: This method is called for tracing purpose. The sub-class should try to return a constant to avoid the
- * unnecessary overhead.
+ * @return EXPLAIN PLAN name if one is defined; otherwise, null. null EXPLAIN PLAN name means that this operator
+ * won't be included in EXPLAIN PLAN output.
*/
- String getOperatorName();
+ String getExplainPlanName();
Review comment:
I think we should be able to merge this with `toExplainString()`, and annotate it with nullable. Don't see the value of keeping 2 separate methods
##########
File path: pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/BaseBrokerRequestHandler.java
##########
@@ -225,7 +225,13 @@ private BrokerResponseNative handleSQLRequest(long requestId, String query, Json
if (isLiteralOnlyQuery(pinotQuery)) {
LOGGER.debug("Request {} contains only Literal, skipping server query: {}", requestId, query);
try {
- return processLiteralOnlyQuery(pinotQuery, compilationStartTimeNs, requestStatistics);
+ BrokerResponseNative responseForLiteralOnly =
+ processLiteralOnlyQuery(pinotQuery, compilationStartTimeNs, requestStatistics);
Review comment:
For explain query, we should not process the query here. We can add a BrokerResponseNative constant which represents the queries that can be solved on the broker side.
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/filter/predicate/PredicateEvaluator.java
##########
@@ -28,6 +28,16 @@
* APIs for both dictionary based and raw value based predicate evaluator
*/
+ /**
+ * Set the predicate
+ */
+ void setPredicate(Predicate predicate);
Review comment:
This should not be an interface method
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/common/Operator.java
##########
@@ -39,12 +40,20 @@
*/
T nextBlock();
+ /** @return Name of this operator */
+ String getOperatorName();
+
+ /** @return List of {@link Operator}s that this operator depends upon. */
+ List<Operator> getChildOperators();
+
/**
- * Returns the name of the operator.
- * NOTE: This method is called for tracing purpose. The sub-class should try to return a constant to avoid the
Review comment:
Can we keep the note?
Also suggest keeping the same convention for the javadoc. I know it can be personal preference, but we should try to keep consistency within the same file (especially interface)
##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/BaseOperator.java
##########
@@ -52,4 +54,25 @@ public final T nextBlock() {
// Make it protected because we should always call nextBlock()
protected abstract T getNextBlock();
+
+ // Enforcing sub-class to implement the getOperatorName(), as they can just return a static final,
+ // as opposed to this super class calling getClass().getSimpleName().
+ public abstract String getOperatorName();
Review comment:
These methods are already included in the interface, no need to redefine them within the base class
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org