You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2022/06/07 02:38:38 UTC
[GitHub] [pinot] saurabhd336 opened a new pull request, #8848: Fix multi value column index creation
saurabhd336 opened a new pull request, #8848:
URL: https://github.com/apache/pinot/pull/8848
Fixes https://github.com/apache/pinot/issues/8847
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang merged pull request #8848: Fix multi value column index creation
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang merged PR #8848:
URL: https://github.com/apache/pinot/pull/8848
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] saurabhd336 commented on a diff in pull request #8848: Fix multi value column index creation
Posted by GitBox <gi...@apache.org>.
saurabhd336 commented on code in PR #8848:
URL: https://github.com/apache/pinot/pull/8848#discussion_r890792792
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentColumnarIndexCreator.java:
##########
@@ -386,30 +386,78 @@ public void indexRow(GenericRow row)
// bloom filter
BloomFilterCreator bloomFilterCreator = _bloomFilterCreatorMap.get(columnName);
if (bloomFilterCreator != null) {
- bloomFilterCreator.add(columnValueToIndex.toString());
+ if (fieldSpec.isSingleValueField()) {
+ bloomFilterCreator.add(columnValueToIndex.toString());
+ } else {
+ Object[] values = (Object[]) columnValueToIndex;
+ for (Object value: values) {
+ bloomFilterCreator.add(value.toString());
+ }
+ }
}
// range index
CombinedInvertedIndexCreator combinedInvertedIndexCreator = _rangeIndexFilterCreatorMap.get(columnName);
if (combinedInvertedIndexCreator != null) {
if (dictionaryCreator != null) {
Review Comment:
I was following the segment loading part, and there seems to be handling for both single and multi value for non dictionary encoded columns.. Should be adding this constraint in the offline path?
https://github.com/apache/pinot/blob/master/pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/loader/invertedindex/RangeIndexHandler.java#L170
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #8848: Fix multi value column index creation
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on code in PR #8848:
URL: https://github.com/apache/pinot/pull/8848#discussion_r890769840
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentColumnarIndexCreator.java:
##########
@@ -386,30 +386,78 @@ public void indexRow(GenericRow row)
// bloom filter
BloomFilterCreator bloomFilterCreator = _bloomFilterCreatorMap.get(columnName);
if (bloomFilterCreator != null) {
- bloomFilterCreator.add(columnValueToIndex.toString());
+ if (fieldSpec.isSingleValueField()) {
+ bloomFilterCreator.add(columnValueToIndex.toString());
+ } else {
+ Object[] values = (Object[]) columnValueToIndex;
+ for (Object value: values) {
+ bloomFilterCreator.add(value.toString());
+ }
+ }
}
// range index
CombinedInvertedIndexCreator combinedInvertedIndexCreator = _rangeIndexFilterCreatorMap.get(columnName);
if (combinedInvertedIndexCreator != null) {
if (dictionaryCreator != null) {
Review Comment:
Suggest for checking SV then dictionary. MV is always dictionary encoded:
```
if (fieldSpec.isSingleValueField()) {
if (dictionaryCreator != null) {
...
} else {
switch ...
}
} else {
assert dictionaryCreator != null;
...
}
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentColumnarIndexCreator.java:
##########
@@ -386,30 +386,78 @@ public void indexRow(GenericRow row)
// bloom filter
BloomFilterCreator bloomFilterCreator = _bloomFilterCreatorMap.get(columnName);
if (bloomFilterCreator != null) {
- bloomFilterCreator.add(columnValueToIndex.toString());
+ if (fieldSpec.isSingleValueField()) {
+ bloomFilterCreator.add(columnValueToIndex.toString());
+ } else {
+ Object[] values = (Object[]) columnValueToIndex;
+ for (Object value: values) {
Review Comment:
(format)
```suggestion
for (Object value : values) {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] KKcorps commented on pull request #8848: Fix multi value column index creation
Posted by GitBox <gi...@apache.org>.
KKcorps commented on PR #8848:
URL: https://github.com/apache/pinot/pull/8848#issuecomment-1148180383
Any tests to check for this scenario
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] saurabhd336 commented on a diff in pull request #8848: Fix multi value column index creation
Posted by GitBox <gi...@apache.org>.
saurabhd336 commented on code in PR #8848:
URL: https://github.com/apache/pinot/pull/8848#discussion_r890969742
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentColumnarIndexCreator.java:
##########
@@ -386,30 +386,78 @@ public void indexRow(GenericRow row)
// bloom filter
BloomFilterCreator bloomFilterCreator = _bloomFilterCreatorMap.get(columnName);
if (bloomFilterCreator != null) {
- bloomFilterCreator.add(columnValueToIndex.toString());
+ if (fieldSpec.isSingleValueField()) {
+ bloomFilterCreator.add(columnValueToIndex.toString());
+ } else {
+ Object[] values = (Object[]) columnValueToIndex;
+ for (Object value: values) {
Review Comment:
Ack
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] saurabhd336 commented on pull request #8848: Fix multi value column index creation
Posted by GitBox <gi...@apache.org>.
saurabhd336 commented on PR #8848:
URL: https://github.com/apache/pinot/pull/8848#issuecomment-1148186028
> Any tests to check for this scenario
Yes I'm adding tests to this PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter commented on pull request #8848: Fix multi value column index creation
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #8848:
URL: https://github.com/apache/pinot/pull/8848#issuecomment-1148151797
# [Codecov](https://codecov.io/gh/apache/pinot/pull/8848?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#8848](https://codecov.io/gh/apache/pinot/pull/8848?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (c3789bb) into [master](https://codecov.io/gh/apache/pinot/commit/d1d2ddbc116f6dddbefdf1e69dd409434eaae0e4?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (d1d2ddb) will **decrease** coverage by `8.77%`.
> The diff coverage is `20.45%`.
> :exclamation: Current head c3789bb differs from pull request most recent head 1002e30. Consider uploading reports for the commit 1002e30 to get more accurate results
```diff
@@ Coverage Diff @@
## master #8848 +/- ##
============================================
- Coverage 69.80% 61.03% -8.78%
+ Complexity 4659 4399 -260
============================================
Files 1741 1729 -12
Lines 91476 91155 -321
Branches 13677 13648 -29
============================================
- Hits 63854 55634 -8220
- Misses 23199 31377 +8178
+ Partials 4423 4144 -279
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `26.89% <0.00%> (-0.19%)` | :arrow_down: |
| integration2 | `?` | |
| unittests1 | `66.19% <20.45%> (-0.04%)` | :arrow_down: |
| unittests2 | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/8848?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...ment/creator/impl/SegmentColumnarIndexCreator.java](https://codecov.io/gh/apache/pinot/pull/8848/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3Qtc2VnbWVudC1sb2NhbC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3Qvc2VnbWVudC9sb2NhbC9zZWdtZW50L2NyZWF0b3IvaW1wbC9TZWdtZW50Q29sdW1uYXJJbmRleENyZWF0b3IuamF2YQ==) | `78.80% <20.45%> (-5.56%)` | :arrow_down: |
| [...apache/pinot/common/helix/ExtraInstanceConfig.java](https://codecov.io/gh/apache/pinot/pull/8848/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vaGVsaXgvRXh0cmFJbnN0YW5jZUNvbmZpZy5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...e/pinot/controller/api/resources/InstanceInfo.java](https://codecov.io/gh/apache/pinot/pull/8848/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9hcGkvcmVzb3VyY2VzL0luc3RhbmNlSW5mby5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...t/core/plan/StreamingInstanceResponsePlanNode.java](https://codecov.io/gh/apache/pinot/pull/8848/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9wbGFuL1N0cmVhbWluZ0luc3RhbmNlUmVzcG9uc2VQbGFuTm9kZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...pinot/controller/recommender/io/ConfigManager.java](https://codecov.io/gh/apache/pinot/pull/8848/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9yZWNvbW1lbmRlci9pby9Db25maWdNYW5hZ2VyLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...ore/operator/streaming/StreamingResponseUtils.java](https://codecov.io/gh/apache/pinot/pull/8848/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9zdHJlYW1pbmcvU3RyZWFtaW5nUmVzcG9uc2VVdGlscy5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [.../org/apache/pinot/client/AggregationResultSet.java](https://codecov.io/gh/apache/pinot/pull/8848/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0FnZ3JlZ2F0aW9uUmVzdWx0U2V0LmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...troller/recommender/io/metadata/FieldMetadata.java](https://codecov.io/gh/apache/pinot/pull/8848/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9yZWNvbW1lbmRlci9pby9tZXRhZGF0YS9GaWVsZE1ldGFkYXRhLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...roller/recommender/rules/impl/BloomFilterRule.java](https://codecov.io/gh/apache/pinot/pull/8848/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9yZWNvbW1lbmRlci9ydWxlcy9pbXBsL0Jsb29tRmlsdGVyUnVsZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...oller/api/resources/PinotControllerAppConfigs.java](https://codecov.io/gh/apache/pinot/pull/8848/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9hcGkvcmVzb3VyY2VzL1Bpbm90Q29udHJvbGxlckFwcENvbmZpZ3MuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| ... and [329 more](https://codecov.io/gh/apache/pinot/pull/8848/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/pinot/pull/8848?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/pinot/pull/8848?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [d1d2ddb...1002e30](https://codecov.io/gh/apache/pinot/pull/8848?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] npawar commented on a diff in pull request #8848: Fix multi value column index creation
Posted by GitBox <gi...@apache.org>.
npawar commented on code in PR #8848:
URL: https://github.com/apache/pinot/pull/8848#discussion_r891482818
##########
pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/OfflineSegmentCreationMultiValueIndexingTest.java:
##########
@@ -0,0 +1,109 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.pinot.integration.tests;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import java.io.File;
+import java.util.List;
+import org.apache.pinot.spi.config.table.TableConfig;
+import org.apache.pinot.spi.config.table.TableType;
+import org.apache.pinot.spi.data.FieldSpec;
+import org.apache.pinot.spi.data.Schema;
+import org.apache.pinot.spi.utils.builder.TableConfigBuilder;
+import org.apache.pinot.util.TestUtils;
+import org.testng.Assert;
+import org.testng.annotations.BeforeClass;
+import org.testng.annotations.Test;
+
+public class OfflineSegmentCreationMultiValueIndexingTest extends BaseClusterIntegrationTestSet {
+
+ @Override
+ public String getTableName() {
+ return "OFFLINE_MULTIVALUE_TABLE";
+ }
+
+ @Override
+ protected String getAvroTarFileName() {
+ return "offlineMultiValueData.tar.gz";
+ }
+
+ @Override
+ protected long getCountStarResult() {
+ // Three distinct records are expected with pk values of 100000, 100001, 100002
+ return 5;
+ }
+
+ @BeforeClass
+ public void setUp()
+ throws Exception {
+ TestUtils.ensureDirectoriesExistAndEmpty(_tempDir, _segmentDir, _tarDir);
+
+ // Start the Pinot cluster
+ startZk();
+ startController();
+ startBrokers(1);
+ startServers(1);
+
+ // Add schema and table
+ Schema schema = new Schema.SchemaBuilder()
+ .setSchemaName("OFFLINE_MULTIVALUE_SCHEMA")
+ .addSingleValueDimension("id", FieldSpec.DataType.INT)
+ .addSingleValueDimension("name", FieldSpec.DataType.STRING)
+ .addMultiValueDimension("interests", FieldSpec.DataType.STRING)
+ .addMultiValueDimension("scores", FieldSpec.DataType.DOUBLE)
+ .build();
+ addSchema(schema);
+ TableConfig tableConfig = new TableConfigBuilder(TableType.OFFLINE)
+ .setTableName(getTableName())
+ .setSchemaName("OFFLINE_MULTIVALUE_SCHEMA")
+ .setBloomFilterColumns(List.of("interests"))
+ .setRangeIndexColumns(List.of("scores"))
+ .build();
+ sendPostRequest(_controllerRequestURLBuilder.forTableCreate(), tableConfig.toString(),
+ BasicAuthTestUtils.AUTH_HEADER);
+
+ // Ingest data
+ List<File> jsonFiles = unpackAvroData(_tempDir);
+ ClusterIntegrationTestUtils.buildSegmentFromJson(jsonFiles.get(0), tableConfig, schema, 0, _segmentDir, _tarDir);
+ uploadSegments(getTableName(), TableType.OFFLINE, _tarDir);
+ waitForAllDocsLoaded(600_000L);
+ }
+
+ @Test
+ public void testMvBloomFilterColumnCount()
+ throws Exception {
+ JsonNode queryResponse = postQuery("SELECT COUNT(*) FROM OFFLINE_MULTIVALUE_TABLE WHERE interests = 'i1'");
Review Comment:
mind adding some more queries like IN, valueIn, != ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #8848: Fix multi value column index creation
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on code in PR #8848:
URL: https://github.com/apache/pinot/pull/8848#discussion_r891555170
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentColumnarIndexCreator.java:
##########
@@ -386,30 +386,78 @@ public void indexRow(GenericRow row)
// bloom filter
BloomFilterCreator bloomFilterCreator = _bloomFilterCreatorMap.get(columnName);
if (bloomFilterCreator != null) {
- bloomFilterCreator.add(columnValueToIndex.toString());
+ if (fieldSpec.isSingleValueField()) {
+ bloomFilterCreator.add(columnValueToIndex.toString());
+ } else {
+ Object[] values = (Object[]) columnValueToIndex;
+ for (Object value: values) {
+ bloomFilterCreator.add(value.toString());
+ }
+ }
}
// range index
CombinedInvertedIndexCreator combinedInvertedIndexCreator = _rangeIndexFilterCreatorMap.get(columnName);
if (combinedInvertedIndexCreator != null) {
if (dictionaryCreator != null) {
Review Comment:
Good point. Let's keep them consistent
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org