You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "vrajat (via GitHub)" <gi...@apache.org> on 2025/04/11 12:51:28 UTC
[PR] Extract Table Processing Code in ServerExecutorV1Impl to TableExecutionInfo class [pinot]
vrajat opened a new pull request, #15523:
URL: https://github.com/apache/pinot/pull/15523
Instructions:
1. The PR has to be tagged with at least one of the following labels (*):
1. `feature`
2. `bugfix`
3. `performance`
4. `ui`
5. `backward-incompat`
6. `release-notes` (**)
2. Remove these instructions before publishing the PR.
(*) Other labels to consider:
- `testing`
- `dependencies`
- `docker`
- `kubernetes`
- `observability`
- `security`
- `code-style`
- `extension-point`
- `refactor`
- `cleanup`
(**) Use `release-notes` label for scenarios like:
- New configuration options
- Deprecation of configurations
- Signature changes to public methods/interfaces
- New plugins added or old plugins removed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Move Query Execution Planning to TableExecutionInfo in SSE [pinot]
Posted by "vrajat (via GitHub)" <gi...@apache.org>.
vrajat commented on code in PR #15523:
URL: https://github.com/apache/pinot/pull/15523#discussion_r2057507548
##########
pinot-core/src/main/java/org/apache/pinot/core/query/executor/ServerQueryExecutorV1Impl.java:
##########
@@ -199,128 +191,37 @@ private InstanceResponseBlock executeInternal(ServerQueryRequest queryRequest, E
return instanceResponse;
}
- TableDataManager tableDataManager = _instanceDataManager.getTableDataManager(tableNameWithType);
- if (tableDataManager == null) {
- String errorMessage = "Failed to find table: " + tableNameWithType + " on server: "
- + _instanceDataManager.getInstanceId();
+ TableExecutionInfo executionInfo;
+ try {
+ executionInfo =
+ SingleTableExecutionInfo.create(_instanceDataManager, tableNameWithType, queryRequest.getSegmentsToQuery(),
+ queryRequest.getOptionalSegments(), queryContext);
+ } catch (NoSuchElementException ne) {
Review Comment:
I've changed it to throw `TableNotFoundException`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Move Query Execution Planning to TableExecutionInfo in SSE [pinot]
Posted by "vrajat (via GitHub)" <gi...@apache.org>.
vrajat commented on code in PR #15523:
URL: https://github.com/apache/pinot/pull/15523#discussion_r2056539639
##########
pinot-core/src/main/java/org/apache/pinot/core/query/executor/TableExecutionInfo.java:
##########
@@ -0,0 +1,197 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.query.executor;
+
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ExecutorService;
+import javax.annotation.Nullable;
+import org.apache.pinot.core.query.pruner.SegmentPrunerService;
+import org.apache.pinot.core.query.pruner.SegmentPrunerStatistics;
+import org.apache.pinot.core.query.request.context.QueryContext;
+import org.apache.pinot.core.query.request.context.TimerContext;
+import org.apache.pinot.segment.local.data.manager.SegmentDataManager;
+import org.apache.pinot.segment.spi.IndexSegment;
+import org.apache.pinot.segment.spi.SegmentContext;
+
+
+public interface TableExecutionInfo {
+ /**
+ * Check if consuming segments are being queried.
+ * @return true if consuming segments are being queried, false otherwise
+ */
+ boolean hasRealtime();
+
+ /**
+ * Get the index segments for a table referenced in the query.
+ * @return A list of index segments for the table
+ */
+ List<IndexSegment> getIndexSegments();
+
+ /**
+ * Get a copy of the index segments for a table referenced in the query as some functions may mutate the list.
+ * @return A copy of the list of index segments for the table
+ */
+ List<IndexSegment> getCopyOfIndexSegments();
Review Comment:
For now, I'll remove it and make a copy in `selectedSegmentsInfo`. The usage is misleading as the logical table implementation is not in this PR. The implementation for `LogicalTable` is:
```
@Override
public SelectedSegmentsInfo getSelectedSegmentsInfo(QueryContext queryContext, TimerContext timerContext,
ExecutorService executorService, SegmentPrunerService segmentPrunerService) {
SelectedSegmentsInfo aggregatedSelectedSegmentsInfo = new SelectedSegmentsInfo();
for (SingleTableExecutionInfo tableExecutionInfo : _tableExecutionInfos) {
SelectedSegmentsInfo selectedSegmentsInfo =
tableExecutionInfo.getSelectedSegmentsInfo(queryContext, timerContext, executorService, segmentPrunerService);
aggregatedSelectedSegmentsInfo.aggregate(selectedSegmentsInfo);
}
LOGGER.debug("Matched {} segments after pruning", aggregatedSelectedSegmentsInfo._numSelectedSegments);
return aggregatedSelectedSegmentsInfo;
}
```
I think making a copy in `SingleTableExecutionInfo` should be fine. I'll recheck when creating a PR for logical tables.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Move Query Execution Planning to TableExecutionInfo in SSE [pinot]
Posted by "yashmayya (via GitHub)" <gi...@apache.org>.
yashmayya commented on code in PR #15523:
URL: https://github.com/apache/pinot/pull/15523#discussion_r2057524805
##########
pinot-core/src/main/java/org/apache/pinot/core/query/executor/TableExecutionInfo.java:
##########
@@ -0,0 +1,211 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.query.executor;
+
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ExecutorService;
+import javax.annotation.Nullable;
+import org.apache.pinot.core.query.pruner.SegmentPrunerService;
+import org.apache.pinot.core.query.pruner.SegmentPrunerStatistics;
+import org.apache.pinot.core.query.request.context.QueryContext;
+import org.apache.pinot.core.query.request.context.TimerContext;
+import org.apache.pinot.segment.local.data.manager.SegmentDataManager;
+import org.apache.pinot.segment.spi.IndexSegment;
+import org.apache.pinot.segment.spi.SegmentContext;
+
+
+public interface TableExecutionInfo {
+ /**
+ * Check if consuming segments are being queried.
+ * @return true if consuming segments are being queried, false otherwise
+ */
+ boolean hasRealtime();
+
+ /**
+ * Get the index segments for a table referenced in the query.
+ * @return A list of index segments for the table
+ */
+ List<IndexSegment> getIndexSegments();
+
+ /**
+ * Get the SegmentContext for all the index segments.
+ * @return A map of index segments to their SegmentContext
+ */
+ @Nullable
+ Map<IndexSegment, SegmentContext> getProvidedSegmentContexts();
+
+ /**
+ * List of segment names to query.
+ * @return A list of segment names to query
+ */
+ List<String> getSegmentsToQuery();
+
+ /**
+ * List of optional segments to query.
+ * @return A list of optional segments to query
+ */
+ List<String> getOptionalSegments();
+
+ /**
+ * Get the list of segment data managers for the segments that are being queried.
+ * @return A list of segment data managers for the segments that are being queried
+ */
+ List<SegmentDataManager> getSegmentDataManagers();
+
+ /**
+ * Release the lock acquired on the segment data managers.
+ */
+ void releaseSegmentDataManagers();
+
+ /**
+ * Get the list of SegmentContexts for the index segments selected in getSelectedSegmentsInfo.
+ * @param selectedSegments A list of index segments selected in getSelectedSegmentsInfo
+ * @param queryOptions A map of query options
+ * @return A list of SegmentContexts for the index segments selected in getSelectedSegmentsInfo
+ */
+ List<SegmentContext> getSegmentContexts(List<IndexSegment> selectedSegments, Map<String, String> queryOptions);
+
+ /**
+ * Get the list of segments that are not acquired.
+ * @return A list of segments that are not acquired
+ */
+ List<String> getNotAcquiredSegments();
+
+ /**
+ * Get the list of segments that are missing.
+ * @return A list of segments that are missing
+ */
+ List<String> getMissingSegments();
+
+ /**
+ * Get the number of segments acquired.
+ * @return The number of segments acquired
+ */
+ int getNumSegmentsAcquired();
+
+ /**
+ * Get the selected segments and segment contexts for a table referenced in a query. The information is gathered
+ * in a SelectSegmentsInfo object.
+ * @return A SelectSegmentsInfo object containing the selected segments and segment contexts
+ */
+ SelectedSegmentsInfo getSelectedSegmentsInfo(QueryContext queryContext, TimerContext timerContext,
+ ExecutorService executorService, SegmentPrunerService segmentPrunerService);
+
+ /**
+ * Generate metadata about the consuming segments queried.
+ * @return A ConsumingSegmentsInfo object containing the metadata about the consuming segments queried
+ */
+ ConsumingSegmentsInfo getConsumingSegmentsInfo();
+
+ /**
+ * If consuming segments are being queried, this class contains the information about the consuming segments such as
+ * the number of segments queried, the min index time, the min ingestion time and the max end time.
+ */
+ class ConsumingSegmentsInfo {
+ private final int _numConsumingSegmentsQueried;
+ private final long _minIndexTimeMs;
+ private final long _minIngestionTimeMs;
+ private final long _maxEndTimeMs;
+
+ public ConsumingSegmentsInfo(int numConsumingSegmentsQueried, long minIndexTimeMs, long minIngestionTimeMs,
+ long maxEndTimeMs) {
+ _numConsumingSegmentsQueried = numConsumingSegmentsQueried;
+ _minIndexTimeMs = minIndexTimeMs;
+ _minIngestionTimeMs = minIngestionTimeMs;
+ _maxEndTimeMs = maxEndTimeMs;
+ }
+
+ long getMinConsumingFreshnessTimeMs() {
+ long minConsumingFreshnessTimeMs = 0;
+ if (getMinIngestionTimeMs() != Long.MAX_VALUE) {
+ minConsumingFreshnessTimeMs = getMinIngestionTimeMs();
+ } else if (getMinIndexTimeMs() != Long.MAX_VALUE) {
+ minConsumingFreshnessTimeMs = getMinIndexTimeMs();
+ } else if (getMaxEndTimeMs() != Long.MIN_VALUE) {
+ minConsumingFreshnessTimeMs = getMaxEndTimeMs();
+ }
+ return minConsumingFreshnessTimeMs;
+ }
+
+ public int getNumConsumingSegmentsQueried() {
+ return _numConsumingSegmentsQueried;
+ }
+
+ public long getMinIndexTimeMs() {
+ return _minIndexTimeMs;
+ }
+
+ public long getMinIngestionTimeMs() {
+ return _minIngestionTimeMs;
+ }
+
+ public long getMaxEndTimeMs() {
+ return _maxEndTimeMs;
+ }
+ }
+
+ /**
+ * This class contains the information about the selected segments such as the number of segments queried, the
+ * number of segments selected and the number of total documents.
+ */
+ class SelectedSegmentsInfo {
+ private List<IndexSegment> _indexSegments;
+ private long _numTotalDocs;
+ private SegmentPrunerStatistics _prunerStats;
+ private int _numTotalSegments;
+ private int _numSelectedSegments;
+ private List<SegmentContext> _selectedSegmentContexts;
Review Comment:
nit: can be `final`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Move Query Execution Planning to TableExecutionInfo in SSE [pinot]
Posted by "vrajat (via GitHub)" <gi...@apache.org>.
vrajat commented on code in PR #15523:
URL: https://github.com/apache/pinot/pull/15523#discussion_r2057492572
##########
pinot-core/src/main/java/org/apache/pinot/core/query/executor/ServerQueryExecutorV1Impl.java:
##########
@@ -199,128 +190,36 @@ private InstanceResponseBlock executeInternal(ServerQueryRequest queryRequest, E
return instanceResponse;
}
- TableDataManager tableDataManager = _instanceDataManager.getTableDataManager(tableNameWithType);
- if (tableDataManager == null) {
- String errorMessage = "Failed to find table: " + tableNameWithType + " on server: "
- + _instanceDataManager.getInstanceId();
+ TableExecutionInfo executionInfo =
+ SingleTableExecutionInfo.create(_instanceDataManager, tableNameWithType, queryRequest.getSegmentsToQuery(),
+ queryRequest.getOptionalSegments(), queryContext);
+
+ if (executionInfo == null) {
Review Comment:
The point of this refactor is to be able to process a list of `TableDataManagers`. Now I remember why I changed it to a try/catch block. In logical tables, any one of them may not be found and I used an exception to return the table name as part of the message. I'll add back the try/catch block and use a better exception. cc @yashmayya
This is the code for `LogicalTableExecutionInfo.create` and it works well with exceptions.
```
List<TableSegmentsContext> tableSegmentsContexts = queryRequest.getTableSegmentsContexts();
List<SingleTableExecutionInfo> tableExecutionInfos = new ArrayList<>(tableSegmentsContexts.size());
for (TableSegmentsContext tableSegmentsInfo : tableSegmentsContexts) {
SingleTableExecutionInfo singleTableExecutionInfo =
SingleTableExecutionInfo.create(instanceDataManager, tableSegmentsInfo.getTableName(),
tableSegmentsInfo.getSegments(), tableSegmentsInfo.getOptionalSegments(), queryContext);
tableExecutionInfos.add(singleTableExecutionInfo);
}
return new LogicalTableExecutionInfo(tableExecutionInfos);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Move Query Execution Planning to TableExecutionInfo in SSE [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on code in PR #15523:
URL: https://github.com/apache/pinot/pull/15523#discussion_r2056686085
##########
pinot-core/src/main/java/org/apache/pinot/core/query/executor/ServerQueryExecutorV1Impl.java:
##########
@@ -199,128 +190,36 @@ private InstanceResponseBlock executeInternal(ServerQueryRequest queryRequest, E
return instanceResponse;
}
- TableDataManager tableDataManager = _instanceDataManager.getTableDataManager(tableNameWithType);
- if (tableDataManager == null) {
- String errorMessage = "Failed to find table: " + tableNameWithType + " on server: "
- + _instanceDataManager.getInstanceId();
+ TableExecutionInfo executionInfo =
+ SingleTableExecutionInfo.create(_instanceDataManager, tableNameWithType, queryRequest.getSegmentsToQuery(),
+ queryRequest.getOptionalSegments(), queryContext);
+
+ if (executionInfo == null) {
Review Comment:
Suggest not using `null` to indicate table not found. We can first get the `TableDataManager` here, then pass it into the `SingleTableExecutionInfo.create()`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Move Query Execution Planning to TableExecutionInfo in SSE [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on PR #15523:
URL: https://github.com/apache/pinot/pull/15523#issuecomment-2819693260
@gortiz @yashmayya Can you help review this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Move Query Execution Planning to TableExecutionInfo in SSE [pinot]
Posted by "vrajat (via GitHub)" <gi...@apache.org>.
vrajat commented on code in PR #15523:
URL: https://github.com/apache/pinot/pull/15523#discussion_r2057525988
##########
pinot-core/src/main/java/org/apache/pinot/core/query/executor/TableExecutionInfo.java:
##########
@@ -0,0 +1,211 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.query.executor;
+
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ExecutorService;
+import javax.annotation.Nullable;
+import org.apache.pinot.core.query.pruner.SegmentPrunerService;
+import org.apache.pinot.core.query.pruner.SegmentPrunerStatistics;
+import org.apache.pinot.core.query.request.context.QueryContext;
+import org.apache.pinot.core.query.request.context.TimerContext;
+import org.apache.pinot.segment.local.data.manager.SegmentDataManager;
+import org.apache.pinot.segment.spi.IndexSegment;
+import org.apache.pinot.segment.spi.SegmentContext;
+
+
+public interface TableExecutionInfo {
+ /**
+ * Check if consuming segments are being queried.
+ * @return true if consuming segments are being queried, false otherwise
+ */
+ boolean hasRealtime();
+
+ /**
+ * Get the index segments for a table referenced in the query.
+ * @return A list of index segments for the table
+ */
+ List<IndexSegment> getIndexSegments();
+
+ /**
+ * Get the SegmentContext for all the index segments.
+ * @return A map of index segments to their SegmentContext
+ */
+ @Nullable
+ Map<IndexSegment, SegmentContext> getProvidedSegmentContexts();
+
+ /**
+ * List of segment names to query.
+ * @return A list of segment names to query
+ */
+ List<String> getSegmentsToQuery();
+
+ /**
+ * List of optional segments to query.
+ * @return A list of optional segments to query
+ */
+ List<String> getOptionalSegments();
+
+ /**
+ * Get the list of segment data managers for the segments that are being queried.
+ * @return A list of segment data managers for the segments that are being queried
+ */
+ List<SegmentDataManager> getSegmentDataManagers();
+
+ /**
+ * Release the lock acquired on the segment data managers.
+ */
+ void releaseSegmentDataManagers();
+
+ /**
+ * Get the list of SegmentContexts for the index segments selected in getSelectedSegmentsInfo.
+ * @param selectedSegments A list of index segments selected in getSelectedSegmentsInfo
+ * @param queryOptions A map of query options
+ * @return A list of SegmentContexts for the index segments selected in getSelectedSegmentsInfo
+ */
+ List<SegmentContext> getSegmentContexts(List<IndexSegment> selectedSegments, Map<String, String> queryOptions);
+
+ /**
+ * Get the list of segments that are not acquired.
+ * @return A list of segments that are not acquired
+ */
+ List<String> getNotAcquiredSegments();
+
+ /**
+ * Get the list of segments that are missing.
+ * @return A list of segments that are missing
+ */
+ List<String> getMissingSegments();
+
+ /**
+ * Get the number of segments acquired.
+ * @return The number of segments acquired
+ */
+ int getNumSegmentsAcquired();
+
+ /**
+ * Get the selected segments and segment contexts for a table referenced in a query. The information is gathered
+ * in a SelectSegmentsInfo object.
+ * @return A SelectSegmentsInfo object containing the selected segments and segment contexts
+ */
+ SelectedSegmentsInfo getSelectedSegmentsInfo(QueryContext queryContext, TimerContext timerContext,
+ ExecutorService executorService, SegmentPrunerService segmentPrunerService);
+
+ /**
+ * Generate metadata about the consuming segments queried.
+ * @return A ConsumingSegmentsInfo object containing the metadata about the consuming segments queried
+ */
+ ConsumingSegmentsInfo getConsumingSegmentsInfo();
+
+ /**
+ * If consuming segments are being queried, this class contains the information about the consuming segments such as
+ * the number of segments queried, the min index time, the min ingestion time and the max end time.
+ */
+ class ConsumingSegmentsInfo {
+ private final int _numConsumingSegmentsQueried;
+ private final long _minIndexTimeMs;
+ private final long _minIngestionTimeMs;
+ private final long _maxEndTimeMs;
+
+ public ConsumingSegmentsInfo(int numConsumingSegmentsQueried, long minIndexTimeMs, long minIngestionTimeMs,
+ long maxEndTimeMs) {
+ _numConsumingSegmentsQueried = numConsumingSegmentsQueried;
+ _minIndexTimeMs = minIndexTimeMs;
+ _minIngestionTimeMs = minIngestionTimeMs;
+ _maxEndTimeMs = maxEndTimeMs;
+ }
+
+ long getMinConsumingFreshnessTimeMs() {
+ long minConsumingFreshnessTimeMs = 0;
+ if (getMinIngestionTimeMs() != Long.MAX_VALUE) {
+ minConsumingFreshnessTimeMs = getMinIngestionTimeMs();
+ } else if (getMinIndexTimeMs() != Long.MAX_VALUE) {
+ minConsumingFreshnessTimeMs = getMinIndexTimeMs();
+ } else if (getMaxEndTimeMs() != Long.MIN_VALUE) {
+ minConsumingFreshnessTimeMs = getMaxEndTimeMs();
+ }
+ return minConsumingFreshnessTimeMs;
+ }
+
+ public int getNumConsumingSegmentsQueried() {
+ return _numConsumingSegmentsQueried;
+ }
+
+ public long getMinIndexTimeMs() {
+ return _minIndexTimeMs;
+ }
+
+ public long getMinIngestionTimeMs() {
+ return _minIngestionTimeMs;
+ }
+
+ public long getMaxEndTimeMs() {
+ return _maxEndTimeMs;
+ }
+ }
+
+ /**
+ * This class contains the information about the selected segments such as the number of segments queried, the
+ * number of segments selected and the number of total documents.
+ */
+ class SelectedSegmentsInfo {
+ private List<IndexSegment> _indexSegments;
+ private long _numTotalDocs;
+ private SegmentPrunerStatistics _prunerStats;
+ private int _numTotalSegments;
+ private int _numSelectedSegments;
+ private List<SegmentContext> _selectedSegmentContexts;
Review Comment:
No. For logical tables, I will add this function to aggregate information across many tables.
```
public void aggregate(SelectedSegmentsInfo other) {
_indexSegments.addAll(other._indexSegments);
_numTotalDocs += other._numTotalDocs;
_numTotalSegments += other._numTotalSegments;
_numSelectedSegments += other._numSelectedSegments;
_selectedSegmentContexts.addAll(other._selectedSegmentContexts);
_prunerStats.setInvalidSegments(_prunerStats.getInvalidSegments() + other._prunerStats.getInvalidSegments());
_prunerStats.setValuePruned(_prunerStats.getValuePruned() + other._prunerStats.getValuePruned());
_prunerStats.setLimitPruned(_prunerStats.getLimitPruned() + other._prunerStats.getLimitPruned());
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Move Query Execution Planning to TableExecutionInfo in SSE [pinot]
Posted by "vrajat (via GitHub)" <gi...@apache.org>.
vrajat commented on code in PR #15523:
URL: https://github.com/apache/pinot/pull/15523#discussion_r2056543692
##########
pinot-core/src/main/java/org/apache/pinot/core/query/executor/TableExecutionInfo.java:
##########
@@ -0,0 +1,197 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.query.executor;
+
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ExecutorService;
+import javax.annotation.Nullable;
+import org.apache.pinot.core.query.pruner.SegmentPrunerService;
+import org.apache.pinot.core.query.pruner.SegmentPrunerStatistics;
+import org.apache.pinot.core.query.request.context.QueryContext;
+import org.apache.pinot.core.query.request.context.TimerContext;
+import org.apache.pinot.segment.local.data.manager.SegmentDataManager;
+import org.apache.pinot.segment.spi.IndexSegment;
+import org.apache.pinot.segment.spi.SegmentContext;
+
+
+public interface TableExecutionInfo {
+ /**
+ * Check if consuming segments are being queried.
+ * @return true if consuming segments are being queried, false otherwise
+ */
+ boolean hasRealtime();
+
+ /**
+ * Get the index segments for a table referenced in the query.
+ * @return A list of index segments for the table
+ */
+ List<IndexSegment> getIndexSegments();
+
+ /**
+ * Get a copy of the index segments for a table referenced in the query as some functions may mutate the list.
+ * @return A copy of the list of index segments for the table
+ */
+ List<IndexSegment> getCopyOfIndexSegments();
+
+ /**
+ * Get the SegmentContext for all the index segments.
+ * @return A map of index segments to their SegmentContext
+ */
+ @Nullable
+ Map<IndexSegment, SegmentContext> getProvidedSegmentContexts();
+
+ /**
+ * List of segment names to query.
+ * @return A list of segment names to query
+ */
+ List<String> getSegmentsToQuery();
+
+ /**
+ * List of optional segments to query.
+ * @return A list of optional segments to query
+ */
+ List<String> getOptionalSegments();
+
+ /**
+ * Get the list of segment data managers for the segments that are being queried.
+ * @return A list of segment data managers for the segments that are being queried
+ */
+ List<SegmentDataManager> getSegmentDataManagers();
+
+ /**
+ * Release the lock acquired on the segment data managers.
+ */
+ void releaseSegmentDataManagers();
+
+ /**
+ * Get the list of SegmentContexts for the index segments selected in getSelectedSegmentsInfo.
+ * @param selectedSegments A list of index segments selected in getSelectedSegmentsInfo
+ * @param queryOptions A map of query options
+ * @return A list of SegmentContexts for the index segments selected in getSelectedSegmentsInfo
+ */
+ List<SegmentContext> getSegmentContexts(List<IndexSegment> selectedSegments, Map<String, String> queryOptions);
+
+ /**
+ * Get the list of segments that are not acquired.
+ * @return A list of segments that are not acquired
+ */
+ List<String> getNotAcquiredSegments();
+
+ /**
+ * Get the list of segments that are missing.
+ * @return A list of segments that are missing
+ */
+ List<String> getMissingSegments();
+
+ /**
+ * Get the number of segments acquired.
+ * @return The number of segments acquired
+ */
+ int getNumSegmentsAcquired();
+
+ /**
+ * Get the selected segments and segment contexts for a table referenced in a query. The information is gathered
+ * in a SelectSegmentsInfo object.
+ * @param queryContext
+ * @param timerContext
+ * @param executorService
+ * @param segmentPrunerService
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Move Query Execution Planning to TableExecutionInfo in SSE [pinot]
Posted by "yashmayya (via GitHub)" <gi...@apache.org>.
yashmayya commented on code in PR #15523:
URL: https://github.com/apache/pinot/pull/15523#discussion_r2055953278
##########
pinot-core/src/main/java/org/apache/pinot/core/query/executor/ServerQueryExecutorV1Impl.java:
##########
@@ -199,128 +191,37 @@ private InstanceResponseBlock executeInternal(ServerQueryRequest queryRequest, E
return instanceResponse;
}
- TableDataManager tableDataManager = _instanceDataManager.getTableDataManager(tableNameWithType);
- if (tableDataManager == null) {
- String errorMessage = "Failed to find table: " + tableNameWithType + " on server: "
- + _instanceDataManager.getInstanceId();
+ TableExecutionInfo executionInfo;
+ try {
+ executionInfo =
Review Comment:
When a logical table implementation of `TableExecutionInfo` is introduced, how do we plug that in here? Basically how do we determine that the table in this request is a "single" table or a logical table? Will it be encoded in the request itself?
##########
pinot-core/src/main/java/org/apache/pinot/core/query/executor/ServerQueryExecutorV1Impl.java:
##########
@@ -199,128 +191,37 @@ private InstanceResponseBlock executeInternal(ServerQueryRequest queryRequest, E
return instanceResponse;
}
- TableDataManager tableDataManager = _instanceDataManager.getTableDataManager(tableNameWithType);
- if (tableDataManager == null) {
- String errorMessage = "Failed to find table: " + tableNameWithType + " on server: "
- + _instanceDataManager.getInstanceId();
+ TableExecutionInfo executionInfo;
+ try {
+ executionInfo =
+ SingleTableExecutionInfo.create(_instanceDataManager, tableNameWithType, queryRequest.getSegmentsToQuery(),
+ queryRequest.getOptionalSegments(), queryContext);
+ } catch (NoSuchElementException ne) {
Review Comment:
nit: Rather than throwing and catching an unchecked / runtime exception, I think it'd be nicer if we could return `null` from `SingleTableExecutionInfo.create` (and annotate it as nullable) in such scenarios; alternatively, we could throw a checked exception.
##########
pinot-core/src/main/java/org/apache/pinot/core/query/executor/TableExecutionInfo.java:
##########
@@ -0,0 +1,197 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.query.executor;
+
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ExecutorService;
+import javax.annotation.Nullable;
+import org.apache.pinot.core.query.pruner.SegmentPrunerService;
+import org.apache.pinot.core.query.pruner.SegmentPrunerStatistics;
+import org.apache.pinot.core.query.request.context.QueryContext;
+import org.apache.pinot.core.query.request.context.TimerContext;
+import org.apache.pinot.segment.local.data.manager.SegmentDataManager;
+import org.apache.pinot.segment.spi.IndexSegment;
+import org.apache.pinot.segment.spi.SegmentContext;
+
+
+public interface TableExecutionInfo {
+ /**
+ * Check if consuming segments are being queried.
+ * @return true if consuming segments are being queried, false otherwise
+ */
+ boolean hasRealtime();
+
+ /**
+ * Get the index segments for a table referenced in the query.
+ * @return A list of index segments for the table
+ */
+ List<IndexSegment> getIndexSegments();
+
+ /**
+ * Get a copy of the index segments for a table referenced in the query as some functions may mutate the list.
+ * @return A copy of the list of index segments for the table
+ */
+ List<IndexSegment> getCopyOfIndexSegments();
+
+ /**
+ * Get the SegmentContext for all the index segments.
+ * @return A map of index segments to their SegmentContext
+ */
+ @Nullable
+ Map<IndexSegment, SegmentContext> getProvidedSegmentContexts();
Review Comment:
nit: Why does it need to include the word `Provided` in the name?
##########
pinot-core/src/main/java/org/apache/pinot/core/query/executor/TableExecutionInfo.java:
##########
@@ -0,0 +1,197 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.query.executor;
+
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ExecutorService;
+import javax.annotation.Nullable;
+import org.apache.pinot.core.query.pruner.SegmentPrunerService;
+import org.apache.pinot.core.query.pruner.SegmentPrunerStatistics;
+import org.apache.pinot.core.query.request.context.QueryContext;
+import org.apache.pinot.core.query.request.context.TimerContext;
+import org.apache.pinot.segment.local.data.manager.SegmentDataManager;
+import org.apache.pinot.segment.spi.IndexSegment;
+import org.apache.pinot.segment.spi.SegmentContext;
+
+
+public interface TableExecutionInfo {
+ /**
+ * Check if consuming segments are being queried.
+ * @return true if consuming segments are being queried, false otherwise
+ */
+ boolean hasRealtime();
+
+ /**
+ * Get the index segments for a table referenced in the query.
+ * @return A list of index segments for the table
+ */
+ List<IndexSegment> getIndexSegments();
+
+ /**
+ * Get a copy of the index segments for a table referenced in the query as some functions may mutate the list.
+ * @return A copy of the list of index segments for the table
+ */
+ List<IndexSegment> getCopyOfIndexSegments();
Review Comment:
Does this really need to be an interface method? External callers can simply make a copy of the list returned by `getIndexSegments` if they wish to mutate it right? Mutating a returned list is bad practice and shouldn't be done anyway so I don't think it makes sense to introduce this method into the interface to specifically guard against that. Also, the only current usage of this method I see is internal to the implementation (`SingleTableExecutionInfo`) anyway.
##########
pinot-core/src/main/java/org/apache/pinot/core/query/executor/TableExecutionInfo.java:
##########
@@ -0,0 +1,197 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.query.executor;
+
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ExecutorService;
+import javax.annotation.Nullable;
+import org.apache.pinot.core.query.pruner.SegmentPrunerService;
+import org.apache.pinot.core.query.pruner.SegmentPrunerStatistics;
+import org.apache.pinot.core.query.request.context.QueryContext;
+import org.apache.pinot.core.query.request.context.TimerContext;
+import org.apache.pinot.segment.local.data.manager.SegmentDataManager;
+import org.apache.pinot.segment.spi.IndexSegment;
+import org.apache.pinot.segment.spi.SegmentContext;
+
+
+public interface TableExecutionInfo {
+ /**
+ * Check if consuming segments are being queried.
+ * @return true if consuming segments are being queried, false otherwise
+ */
+ boolean hasRealtime();
+
+ /**
+ * Get the index segments for a table referenced in the query.
+ * @return A list of index segments for the table
+ */
+ List<IndexSegment> getIndexSegments();
+
+ /**
+ * Get a copy of the index segments for a table referenced in the query as some functions may mutate the list.
+ * @return A copy of the list of index segments for the table
+ */
+ List<IndexSegment> getCopyOfIndexSegments();
+
+ /**
+ * Get the SegmentContext for all the index segments.
+ * @return A map of index segments to their SegmentContext
+ */
+ @Nullable
+ Map<IndexSegment, SegmentContext> getProvidedSegmentContexts();
+
+ /**
+ * List of segment names to query.
+ * @return A list of segment names to query
+ */
+ List<String> getSegmentsToQuery();
+
+ /**
+ * List of optional segments to query.
+ * @return A list of optional segments to query
+ */
+ List<String> getOptionalSegments();
+
+ /**
+ * Get the list of segment data managers for the segments that are being queried.
+ * @return A list of segment data managers for the segments that are being queried
+ */
+ List<SegmentDataManager> getSegmentDataManagers();
+
+ /**
+ * Release the lock acquired on the segment data managers.
+ */
+ void releaseSegmentDataManagers();
+
+ /**
+ * Get the list of SegmentContexts for the index segments selected in getSelectedSegmentsInfo.
+ * @param selectedSegments A list of index segments selected in getSelectedSegmentsInfo
+ * @param queryOptions A map of query options
+ * @return A list of SegmentContexts for the index segments selected in getSelectedSegmentsInfo
+ */
+ List<SegmentContext> getSegmentContexts(List<IndexSegment> selectedSegments, Map<String, String> queryOptions);
+
+ /**
+ * Get the list of segments that are not acquired.
+ * @return A list of segments that are not acquired
+ */
+ List<String> getNotAcquiredSegments();
+
+ /**
+ * Get the list of segments that are missing.
+ * @return A list of segments that are missing
+ */
+ List<String> getMissingSegments();
+
+ /**
+ * Get the number of segments acquired.
+ * @return The number of segments acquired
+ */
+ int getNumSegmentsAcquired();
+
+ /**
+ * Get the selected segments and segment contexts for a table referenced in a query. The information is gathered
+ * in a SelectSegmentsInfo object.
+ * @param queryContext
+ * @param timerContext
+ * @param executorService
+ * @param segmentPrunerService
Review Comment:
Empty param descriptions, can just remove these IMO since they're somewhat self explanatory.
##########
pinot-core/src/main/java/org/apache/pinot/core/query/executor/TableExecutionInfo.java:
##########
@@ -0,0 +1,197 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.query.executor;
+
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ExecutorService;
+import javax.annotation.Nullable;
+import org.apache.pinot.core.query.pruner.SegmentPrunerService;
+import org.apache.pinot.core.query.pruner.SegmentPrunerStatistics;
+import org.apache.pinot.core.query.request.context.QueryContext;
+import org.apache.pinot.core.query.request.context.TimerContext;
+import org.apache.pinot.segment.local.data.manager.SegmentDataManager;
+import org.apache.pinot.segment.spi.IndexSegment;
+import org.apache.pinot.segment.spi.SegmentContext;
+
+
+public interface TableExecutionInfo {
+ /**
+ * Check if consuming segments are being queried.
+ * @return true if consuming segments are being queried, false otherwise
+ */
+ boolean hasRealtime();
+
+ /**
+ * Get the index segments for a table referenced in the query.
+ * @return A list of index segments for the table
+ */
+ List<IndexSegment> getIndexSegments();
+
+ /**
+ * Get a copy of the index segments for a table referenced in the query as some functions may mutate the list.
+ * @return A copy of the list of index segments for the table
+ */
+ List<IndexSegment> getCopyOfIndexSegments();
+
+ /**
+ * Get the SegmentContext for all the index segments.
+ * @return A map of index segments to their SegmentContext
+ */
+ @Nullable
+ Map<IndexSegment, SegmentContext> getProvidedSegmentContexts();
+
+ /**
+ * List of segment names to query.
+ * @return A list of segment names to query
+ */
+ List<String> getSegmentsToQuery();
+
+ /**
+ * List of optional segments to query.
+ * @return A list of optional segments to query
+ */
+ List<String> getOptionalSegments();
+
+ /**
+ * Get the list of segment data managers for the segments that are being queried.
+ * @return A list of segment data managers for the segments that are being queried
+ */
+ List<SegmentDataManager> getSegmentDataManagers();
+
+ /**
+ * Release the lock acquired on the segment data managers.
+ */
+ void releaseSegmentDataManagers();
+
+ /**
+ * Get the list of SegmentContexts for the index segments selected in getSelectedSegmentsInfo.
+ * @param selectedSegments A list of index segments selected in getSelectedSegmentsInfo
+ * @param queryOptions A map of query options
+ * @return A list of SegmentContexts for the index segments selected in getSelectedSegmentsInfo
+ */
+ List<SegmentContext> getSegmentContexts(List<IndexSegment> selectedSegments, Map<String, String> queryOptions);
+
+ /**
+ * Get the list of segments that are not acquired.
+ * @return A list of segments that are not acquired
+ */
+ List<String> getNotAcquiredSegments();
+
+ /**
+ * Get the list of segments that are missing.
+ * @return A list of segments that are missing
+ */
+ List<String> getMissingSegments();
+
+ /**
+ * Get the number of segments acquired.
+ * @return The number of segments acquired
+ */
+ int getNumSegmentsAcquired();
+
+ /**
+ * Get the selected segments and segment contexts for a table referenced in a query. The information is gathered
+ * in a SelectSegmentsInfo object.
+ * @param queryContext
+ * @param timerContext
+ * @param executorService
+ * @param segmentPrunerService
+ * @return A SelectSegmentsInfo object containing the selected segments and segment contexts
+ */
+ SelectedSegmentsInfo getSelectedSegmentsInfo(QueryContext queryContext, TimerContext timerContext,
+ ExecutorService executorService, SegmentPrunerService segmentPrunerService);
+
+ /**
+ * Generate metadata about the consuming segments queried.
+ * @return A ConsumingSegmentsInfo object containing the metadata about the consuming segments queried
+ */
+ ConsumingSegmentsInfo getConsumingSegmentsInfo();
+
+ /**
+ * If consuming segments are being queried, this class contains the information about the consuming segments such as
+ * the number of segments queried, the min index time, the min ingestion time and the max end time.
+ */
+ class ConsumingSegmentsInfo {
+ private final int _numConsumingSegmentsQueried;
+ private final long _minIndexTimeMs;
+ private final long _minIngestionTimeMs;
+ private final long _maxEndTimeMs;
+
+ public ConsumingSegmentsInfo(int numConsumingSegmentsQueried, long minIndexTimeMs, long minIngestionTimeMs,
+ long maxEndTimeMs) {
+ _numConsumingSegmentsQueried = numConsumingSegmentsQueried;
+ _minIndexTimeMs = minIndexTimeMs;
+ _minIngestionTimeMs = minIngestionTimeMs;
+ _maxEndTimeMs = maxEndTimeMs;
+ }
+
+ long getMinConsumingFreshnessTimeMs() {
+ long minConsumingFreshnessTimeMs = 0;
+ if (getMinIngestionTimeMs() != Long.MAX_VALUE) {
+ minConsumingFreshnessTimeMs = getMinIngestionTimeMs();
+ } else if (getMinIndexTimeMs() != Long.MAX_VALUE) {
+ minConsumingFreshnessTimeMs = getMinIndexTimeMs();
+ } else if (getMaxEndTimeMs() != Long.MIN_VALUE) {
+ minConsumingFreshnessTimeMs = getMaxEndTimeMs();
+ }
+ return minConsumingFreshnessTimeMs;
+ }
+
+ public int getNumConsumingSegmentsQueried() {
+ return _numConsumingSegmentsQueried;
+ }
+
+ public long getMinIndexTimeMs() {
+ return _minIndexTimeMs;
+ }
+
+ public long getMinIngestionTimeMs() {
+ return _minIngestionTimeMs;
+ }
+
+ public long getMaxEndTimeMs() {
+ return _maxEndTimeMs;
+ }
+ }
+
+ /**
+ * This class contains the information about the selected segments such as the number of segments queried, the
+ * number of segments selected and the number of total documents.
+ */
+ class SelectedSegmentsInfo {
+ public List<IndexSegment> _indexSegments;
+ public long _numTotalDocs;
+ public SegmentPrunerStatistics _prunerStats;
+ public int _numTotalSegments;
+ public int _numSelectedSegments;
+ public List<SegmentContext> _selectedSegmentContexts;
Review Comment:
Are these intentionally publicly accessible and mutable? I don't see where we're mutating these? Why not make them private members accessible through getter methods like in `ConsumingSegmentsInfo`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Move Query Execution Planning to TableExecutionInfo in SSE [pinot]
Posted by "vrajat (via GitHub)" <gi...@apache.org>.
vrajat commented on code in PR #15523:
URL: https://github.com/apache/pinot/pull/15523#discussion_r2056542955
##########
pinot-core/src/main/java/org/apache/pinot/core/query/executor/TableExecutionInfo.java:
##########
@@ -0,0 +1,197 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.query.executor;
+
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ExecutorService;
+import javax.annotation.Nullable;
+import org.apache.pinot.core.query.pruner.SegmentPrunerService;
+import org.apache.pinot.core.query.pruner.SegmentPrunerStatistics;
+import org.apache.pinot.core.query.request.context.QueryContext;
+import org.apache.pinot.core.query.request.context.TimerContext;
+import org.apache.pinot.segment.local.data.manager.SegmentDataManager;
+import org.apache.pinot.segment.spi.IndexSegment;
+import org.apache.pinot.segment.spi.SegmentContext;
+
+
+public interface TableExecutionInfo {
+ /**
+ * Check if consuming segments are being queried.
+ * @return true if consuming segments are being queried, false otherwise
+ */
+ boolean hasRealtime();
+
+ /**
+ * Get the index segments for a table referenced in the query.
+ * @return A list of index segments for the table
+ */
+ List<IndexSegment> getIndexSegments();
+
+ /**
+ * Get a copy of the index segments for a table referenced in the query as some functions may mutate the list.
+ * @return A copy of the list of index segments for the table
+ */
+ List<IndexSegment> getCopyOfIndexSegments();
+
+ /**
+ * Get the SegmentContext for all the index segments.
+ * @return A map of index segments to their SegmentContext
+ */
+ @Nullable
+ Map<IndexSegment, SegmentContext> getProvidedSegmentContexts();
Review Comment:
I copied that name from this function signature:
```
private InstanceResponseBlock executeInternal(TableDataManager tableDataManager, List<IndexSegment> indexSegments,
@Nullable Map<IndexSegment, SegmentContext> providedSegmentContexts, QueryContext queryContext,
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Extract Table Processing Code in ServerExecutorV1Impl to TableExecutionInfo class [pinot]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #15523:
URL: https://github.com/apache/pinot/pull/15523#issuecomment-2796933506
## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/15523?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
Attention: Patch coverage is `40.27778%` with `86 lines` in your changes missing coverage. Please review.
> Project coverage is 55.83%. Comparing base [(`9145e92`)](https://app.codecov.io/gh/apache/pinot/commit/9145e92f655ed481d9ceac6e92ba1d0671b1d1c4?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to head [(`1d9ca7d`)](https://app.codecov.io/gh/apache/pinot/commit/1d9ca7d77dde026bf678b6c05ff0b8969db94ca9?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
> Report is 5 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/pinot/pull/15523?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...core/query/executor/ServerQueryExecutorV1Impl.java](https://app.codecov.io/gh/apache/pinot/pull/15523?src=pr&el=tree&filepath=pinot-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fpinot%2Fcore%2Fquery%2Fexecutor%2FServerQueryExecutorV1Impl.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9xdWVyeS9leGVjdXRvci9TZXJ2ZXJRdWVyeUV4ZWN1dG9yVjFJbXBsLmphdmE=) | 37.50% | [42 Missing and 3 partials :warning: ](https://app.codecov.io/gh/apache/pinot/pull/15523?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [.../core/query/executor/SingleTableExecutionInfo.java](https://app.codecov.io/gh/apache/pinot/pull/15523?src=pr&el=tree&filepath=pinot-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fpinot%2Fcore%2Fquery%2Fexecutor%2FSingleTableExecutionInfo.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9xdWVyeS9leGVjdXRvci9TaW5nbGVUYWJsZUV4ZWN1dGlvbkluZm8uamF2YQ==) | 46.29% | [27 Missing and 2 partials :warning: ](https://app.codecov.io/gh/apache/pinot/pull/15523?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [.../pinot/core/query/executor/TableExecutionInfo.java](https://app.codecov.io/gh/apache/pinot/pull/15523?src=pr&el=tree&filepath=pinot-core%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fpinot%2Fcore%2Fquery%2Fexecutor%2FTableExecutionInfo.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9xdWVyeS9leGVjdXRvci9UYWJsZUV4ZWN1dGlvbkluZm8uamF2YQ==) | 33.33% | [12 Missing :warning: ](https://app.codecov.io/gh/apache/pinot/pull/15523?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
> :exclamation: There is a different number of reports uploaded between BASE (9145e92) and HEAD (1d9ca7d). Click for more details.
>
> <details><summary>HEAD has 62 uploads less than BASE</summary>
>
>| Flag | BASE (9145e92) | HEAD (1d9ca7d) |
>|------|------|------|
>|unittests|6|2|
>|unittests1|3|2|
>|java-21|10|2|
>|skip-bytebuffers-false|8|1|
>|temurin|15|2|
>|skip-bytebuffers-true|4|1|
>|unittests2|3|0|
>|java-11|5|0|
>|integration|9|0|
>|integration2|3|0|
>|integration1|3|0|
>|custom-integration1|3|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #15523 +/- ##
============================================
- Coverage 62.95% 55.83% -7.12%
+ Complexity 1381 681 -700
============================================
Files 2848 2225 -623
Lines 161547 120127 -41420
Branches 24752 19186 -5566
============================================
- Hits 101700 67074 -34626
+ Misses 52167 47528 -4639
+ Partials 7680 5525 -2155
```
| [Flag](https://app.codecov.io/gh/apache/pinot/pull/15523/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [custom-integration1](https://app.codecov.io/gh/apache/pinot/pull/15523/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration](https://app.codecov.io/gh/apache/pinot/pull/15523/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration1](https://app.codecov.io/gh/apache/pinot/pull/15523/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration2](https://app.codecov.io/gh/apache/pinot/pull/15523/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [java-11](https://app.codecov.io/gh/apache/pinot/pull/15523/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [java-21](https://app.codecov.io/gh/apache/pinot/pull/15523/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `55.83% <40.27%> (-7.11%)` | :arrow_down: |
| [skip-bytebuffers-false](https://app.codecov.io/gh/apache/pinot/pull/15523/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `55.82% <40.27%> (-7.13%)` | :arrow_down: |
| [skip-bytebuffers-true](https://app.codecov.io/gh/apache/pinot/pull/15523/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `55.79% <40.27%> (-7.10%)` | :arrow_down: |
| [temurin](https://app.codecov.io/gh/apache/pinot/pull/15523/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `55.83% <40.27%> (-7.12%)` | :arrow_down: |
| [unittests](https://app.codecov.io/gh/apache/pinot/pull/15523/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `55.83% <40.27%> (-7.12%)` | :arrow_down: |
| [unittests1](https://app.codecov.io/gh/apache/pinot/pull/15523/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `55.83% <40.27%> (-0.09%)` | :arrow_down: |
| [unittests2](https://app.codecov.io/gh/apache/pinot/pull/15523/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/pinot/pull/15523?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Move Query Execution Planning to TableExecutionInfo in SSE [pinot]
Posted by "vrajat (via GitHub)" <gi...@apache.org>.
vrajat commented on code in PR #15523:
URL: https://github.com/apache/pinot/pull/15523#discussion_r2056548535
##########
pinot-core/src/main/java/org/apache/pinot/core/query/executor/TableExecutionInfo.java:
##########
@@ -0,0 +1,197 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.core.query.executor;
+
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ExecutorService;
+import javax.annotation.Nullable;
+import org.apache.pinot.core.query.pruner.SegmentPrunerService;
+import org.apache.pinot.core.query.pruner.SegmentPrunerStatistics;
+import org.apache.pinot.core.query.request.context.QueryContext;
+import org.apache.pinot.core.query.request.context.TimerContext;
+import org.apache.pinot.segment.local.data.manager.SegmentDataManager;
+import org.apache.pinot.segment.spi.IndexSegment;
+import org.apache.pinot.segment.spi.SegmentContext;
+
+
+public interface TableExecutionInfo {
+ /**
+ * Check if consuming segments are being queried.
+ * @return true if consuming segments are being queried, false otherwise
+ */
+ boolean hasRealtime();
+
+ /**
+ * Get the index segments for a table referenced in the query.
+ * @return A list of index segments for the table
+ */
+ List<IndexSegment> getIndexSegments();
+
+ /**
+ * Get a copy of the index segments for a table referenced in the query as some functions may mutate the list.
+ * @return A copy of the list of index segments for the table
+ */
+ List<IndexSegment> getCopyOfIndexSegments();
+
+ /**
+ * Get the SegmentContext for all the index segments.
+ * @return A map of index segments to their SegmentContext
+ */
+ @Nullable
+ Map<IndexSegment, SegmentContext> getProvidedSegmentContexts();
+
+ /**
+ * List of segment names to query.
+ * @return A list of segment names to query
+ */
+ List<String> getSegmentsToQuery();
+
+ /**
+ * List of optional segments to query.
+ * @return A list of optional segments to query
+ */
+ List<String> getOptionalSegments();
+
+ /**
+ * Get the list of segment data managers for the segments that are being queried.
+ * @return A list of segment data managers for the segments that are being queried
+ */
+ List<SegmentDataManager> getSegmentDataManagers();
+
+ /**
+ * Release the lock acquired on the segment data managers.
+ */
+ void releaseSegmentDataManagers();
+
+ /**
+ * Get the list of SegmentContexts for the index segments selected in getSelectedSegmentsInfo.
+ * @param selectedSegments A list of index segments selected in getSelectedSegmentsInfo
+ * @param queryOptions A map of query options
+ * @return A list of SegmentContexts for the index segments selected in getSelectedSegmentsInfo
+ */
+ List<SegmentContext> getSegmentContexts(List<IndexSegment> selectedSegments, Map<String, String> queryOptions);
+
+ /**
+ * Get the list of segments that are not acquired.
+ * @return A list of segments that are not acquired
+ */
+ List<String> getNotAcquiredSegments();
+
+ /**
+ * Get the list of segments that are missing.
+ * @return A list of segments that are missing
+ */
+ List<String> getMissingSegments();
+
+ /**
+ * Get the number of segments acquired.
+ * @return The number of segments acquired
+ */
+ int getNumSegmentsAcquired();
+
+ /**
+ * Get the selected segments and segment contexts for a table referenced in a query. The information is gathered
+ * in a SelectSegmentsInfo object.
+ * @param queryContext
+ * @param timerContext
+ * @param executorService
+ * @param segmentPrunerService
+ * @return A SelectSegmentsInfo object containing the selected segments and segment contexts
+ */
+ SelectedSegmentsInfo getSelectedSegmentsInfo(QueryContext queryContext, TimerContext timerContext,
+ ExecutorService executorService, SegmentPrunerService segmentPrunerService);
+
+ /**
+ * Generate metadata about the consuming segments queried.
+ * @return A ConsumingSegmentsInfo object containing the metadata about the consuming segments queried
+ */
+ ConsumingSegmentsInfo getConsumingSegmentsInfo();
+
+ /**
+ * If consuming segments are being queried, this class contains the information about the consuming segments such as
+ * the number of segments queried, the min index time, the min ingestion time and the max end time.
+ */
+ class ConsumingSegmentsInfo {
+ private final int _numConsumingSegmentsQueried;
+ private final long _minIndexTimeMs;
+ private final long _minIngestionTimeMs;
+ private final long _maxEndTimeMs;
+
+ public ConsumingSegmentsInfo(int numConsumingSegmentsQueried, long minIndexTimeMs, long minIngestionTimeMs,
+ long maxEndTimeMs) {
+ _numConsumingSegmentsQueried = numConsumingSegmentsQueried;
+ _minIndexTimeMs = minIndexTimeMs;
+ _minIngestionTimeMs = minIngestionTimeMs;
+ _maxEndTimeMs = maxEndTimeMs;
+ }
+
+ long getMinConsumingFreshnessTimeMs() {
+ long minConsumingFreshnessTimeMs = 0;
+ if (getMinIngestionTimeMs() != Long.MAX_VALUE) {
+ minConsumingFreshnessTimeMs = getMinIngestionTimeMs();
+ } else if (getMinIndexTimeMs() != Long.MAX_VALUE) {
+ minConsumingFreshnessTimeMs = getMinIndexTimeMs();
+ } else if (getMaxEndTimeMs() != Long.MIN_VALUE) {
+ minConsumingFreshnessTimeMs = getMaxEndTimeMs();
+ }
+ return minConsumingFreshnessTimeMs;
+ }
+
+ public int getNumConsumingSegmentsQueried() {
+ return _numConsumingSegmentsQueried;
+ }
+
+ public long getMinIndexTimeMs() {
+ return _minIndexTimeMs;
+ }
+
+ public long getMinIngestionTimeMs() {
+ return _minIngestionTimeMs;
+ }
+
+ public long getMaxEndTimeMs() {
+ return _maxEndTimeMs;
+ }
+ }
+
+ /**
+ * This class contains the information about the selected segments such as the number of segments queried, the
+ * number of segments selected and the number of total documents.
+ */
+ class SelectedSegmentsInfo {
+ public List<IndexSegment> _indexSegments;
+ public long _numTotalDocs;
+ public SegmentPrunerStatistics _prunerStats;
+ public int _numTotalSegments;
+ public int _numSelectedSegments;
+ public List<SegmentContext> _selectedSegmentContexts;
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Move Query Execution Planning to TableExecutionInfo in SSE [pinot]
Posted by "vrajat (via GitHub)" <gi...@apache.org>.
vrajat commented on code in PR #15523:
URL: https://github.com/apache/pinot/pull/15523#discussion_r2056520779
##########
pinot-core/src/main/java/org/apache/pinot/core/query/executor/ServerQueryExecutorV1Impl.java:
##########
@@ -199,128 +191,37 @@ private InstanceResponseBlock executeInternal(ServerQueryRequest queryRequest, E
return instanceResponse;
}
- TableDataManager tableDataManager = _instanceDataManager.getTableDataManager(tableNameWithType);
- if (tableDataManager == null) {
- String errorMessage = "Failed to find table: " + tableNameWithType + " on server: "
- + _instanceDataManager.getInstanceId();
+ TableExecutionInfo executionInfo;
+ try {
+ executionInfo =
Review Comment:
This is the whole block. `ServerQueryRequest` will have either `segments` (and others) OR `TableRouteInfo` list. If `TableRoutInfo` list is available, then logical table objects are created.
```
TableExecutionInfo executionInfo;
try {
if (queryRequest.getTableRouteInfos() != null && !queryRequest.getTableRouteInfos().isEmpty()) {
executionInfo = LogicalTableExecutionInfo.create(_instanceDataManager, queryRequest, queryContext);
} else {
executionInfo =
SingleTableExecutionInfo.create(_instanceDataManager, tableNameWithType, queryRequest.getSegmentsToQuery(),
queryRequest.getOptionalSegments(), queryContext);
}
} catch (NoSuchElementException ne) {
String errorMessage =
"Failed to find table: " + ne.getMessage() + " on server: " + _instanceDataManager.getInstanceId();
InstanceResponseBlock instanceResponse = new InstanceResponseBlock();
instanceResponse.addException(QueryErrorCode.SERVER_TABLE_MISSING, errorMessage);
LOGGER.error("{} while processing requestId: {}", errorMessage, requestId);
return instanceResponse;
}
```
The POC is in https://github.com/apache/pinot/pull/15151
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Move Query Execution Planning to TableExecutionInfo in SSE [pinot]
Posted by "yashmayya (via GitHub)" <gi...@apache.org>.
yashmayya merged PR #15523:
URL: https://github.com/apache/pinot/pull/15523
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Move Query Execution Planning to TableExecutionInfo in SSE [pinot]
Posted by "vrajat (via GitHub)" <gi...@apache.org>.
vrajat commented on PR #15523:
URL: https://github.com/apache/pinot/pull/15523#issuecomment-2813314137
Test failure is unrelated and seems like a flaky test. It passed locally.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org