You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "xiangfu0 (via GitHub)" <gi...@apache.org> on 2023/05/12 18:46:02 UTC
[GitHub] [pinot] xiangfu0 commented on a diff in pull request #10760: [multistage] follow up leaf scheduler
xiangfu0 commented on code in PR #10760:
URL: https://github.com/apache/pinot/pull/10760#discussion_r1192695138
##########
pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/operator/MailboxSendOperator.java:
##########
@@ -128,21 +129,21 @@ protected TransferableBlock getNextBlock() {
TransferableBlock transferableBlock;
try {
transferableBlock = _sourceOperator.nextBlock();
- while (!transferableBlock.isNoOpBlock()) {
- if (transferableBlock.isEndOfStreamBlock()) {
- if (transferableBlock.isSuccessfulEndOfStreamBlock()) {
- //Stats need to be populated here because the block is being sent to the mailbox
- // and the receiving opChain will not be able to access the stats from the previous opChain
- TransferableBlock eosBlockWithStats = TransferableBlockUtils.getEndOfStreamTransferableBlock(
- OperatorUtils.getMetadataFromOperatorStats(_opChainStats.getOperatorStatsMap()));
- _exchange.send(eosBlockWithStats);
- } else {
- _exchange.send(transferableBlock);
- }
- return transferableBlock;
+ if (transferableBlock.isNoOpBlock()) {
+ return transferableBlock;
+ } else if (transferableBlock.isEndOfStreamBlock()) {
+ if (transferableBlock.isSuccessfulEndOfStreamBlock()) {
Review Comment:
seems like only `isEndOfStreamBlock` is special handling, other cases you always `return transferableBlock;`
##########
pinot-query-runtime/src/main/java/org/apache/pinot/query/runtime/executor/OpChainSchedulerService.java:
##########
@@ -48,20 +48,16 @@ public class OpChainSchedulerService extends AbstractExecutionThreadService {
* Default cancel signal retention, this should be set to several times larger than
* {@link org.apache.pinot.query.service.QueryConfig#DEFAULT_SCHEDULER_RELEASE_TIMEOUT_MS}.
*/
- private static final long DEFAULT_SCHEDULER_CANCELLATION_SIGNAL_RETENTION_MS = 60_000L;
+ private static final long SCHEDULER_CANCELLATION_SIGNAL_RETENTION_MS = 60_000L;
private final OpChainScheduler _scheduler;
private final ExecutorService _workerPool;
- private final Cache<Long, Long> _cancelledRequests;
+ private final Cache<Long, Long> _cancelledRequests = CacheBuilder.newBuilder()
Review Comment:
Shall it still be tunable?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org