You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "jadami10 (via GitHub)" <gi...@apache.org> on 2023/04/04 13:53:16 UTC
[GitHub] [pinot] jadami10 opened a new issue, #10539: Segment Pruning does not respect query timeout
jadami10 opened a new issue, #10539:
URL: https://github.com/apache/pinot/issues/10539
We're seeing cases internally where segment pruning on the server side will run for minutes when the timeout is explicitly set to 30s. I imagine we would want to support timing out here, but I could use some design input.
1. should we still emit the `SEGMENT_PRUNING` time metric on timeout?
2. Does it make sense to dig into every implementer of the `SegmentPruner` and add timeouts or use something like the guava TimeLimiter to do it once (I need to dig into how this thing works and if it will actually interrupt a while loop or IO)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang closed issue #10539: Segment Pruning does not respect query timeout
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang closed issue #10539: Segment Pruning does not respect query timeout
URL: https://github.com/apache/pinot/issues/10539
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on issue #10539: Segment Pruning does not respect query timeout
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on issue #10539:
URL: https://github.com/apache/pinot/issues/10539#issuecomment-1496429768
Certain segment pruner requires loading data (e.g. bloom filter, dictionary) which might take time. IMO we should move the segment pruning step into the query plan phase so that we can utilize multiple threads to process them. Within the plan node, we get the early termination for free
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] jadami10 commented on issue #10539: Segment Pruning does not respect query timeout
Posted by "jadami10 (via GitHub)" <gi...@apache.org>.
jadami10 commented on issue #10539:
URL: https://github.com/apache/pinot/issues/10539#issuecomment-1528144765
looks like it merged! I'll help test it next week. cc @priyen-stripe
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] ankitsultana commented on issue #10539: Segment Pruning does not respect query timeout
Posted by "ankitsultana (via GitHub)" <gi...@apache.org>.
ankitsultana commented on issue #10539:
URL: https://github.com/apache/pinot/issues/10539#issuecomment-1496415897
The issue regarding timeout based pre-emption is general. For supporting interrupts we'll have to add interrupt checks everywhere so it is not limited to `SegmentPruner`.
I haven't checked Guava `TimeLimiter` either but I don't think it would be able to stop an arbitrary function unless the function itself supports interruption (by checking Thread interrupt status). For context even Future methods like `Future.cancel` only set the interrupt flag and don't actually "cancel" any arbitrary function.
The thing that is interesting though is why the segment pruner is taking this long. Can you share more context about the issue: how many segments you have, what's the query you are trying, etc.
I presume it is the `ColumnValueSegmentPruner` which is taking a long time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] klsince commented on issue #10539: Segment Pruning does not respect query timeout
Posted by "klsince (via GitHub)" <gi...@apache.org>.
klsince commented on issue #10539:
URL: https://github.com/apache/pinot/issues/10539#issuecomment-1527875065
https://github.com/apache/pinot/pull/10660 trying to tackle this issue: 1) to parallelize the BF based pruner; 2) and honor the timeout set in query option. pls help review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] priyen commented on issue #10539: Segment Pruning does not respect query timeout
Posted by "priyen (via GitHub)" <gi...@apache.org>.
priyen commented on issue #10539:
URL: https://github.com/apache/pinot/issues/10539#issuecomment-1496428631
@ankitsultana , I work with johan so I can also chime in here, this is on a instance using hard-drive's as the backing store, and one instance can also have thousands of segments. The single threaded nature of the current pruning + large segments in the thousands means pruning takes a while and sometimes longer then the query timeout.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] jadami10 commented on issue #10539: Segment Pruning does not respect query timeout
Posted by "jadami10 (via GitHub)" <gi...@apache.org>.
jadami10 commented on issue #10539:
URL: https://github.com/apache/pinot/issues/10539#issuecomment-1496827571
@Jackie-Jiang, for "the query plan phase", are you specifically referring to `InstancePlanMakerImplV2`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org