You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "jadami10 (via GitHub)" <gi...@apache.org> on 2024/11/01 23:06:19 UTC
[I] Time Pruner should support queries where we're operating on the time column itself [pinot]
jadami10 opened a new issue, #14362:
URL: https://github.com/apache/pinot/issues/14362
Today, we only apply time pruning when the left hand side of an operand is strictly the time colum, https://github.com/apache/pinot/blob/master/pinot-broker/src/main/java/org/apache/pinot/broker/routing/segmentpruner/TimeSegmentPruner.java#L370-L373
But we should still be able to apply time pruning when an operand is applied to the time column. The main case we've seen is folks will run a query with `where date_trunc('day', <ts_column>) < <static_time>`. This causes the time pruner to not be used.
The most naive way here would be to rebuild the interval tree in the broker with the min/max time columns for each segment also truncated. Though this would be problematic for tables with tens of thousands of segments.
I'm opening this as an issue for now to brainstorm if there's a universal solution or if there's easy operands that can be supported.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Time Pruner should support queries where we're operating on the time column itself [pinot]
Posted by "jadami10 (via GitHub)" <gi...@apache.org>.
jadami10 commented on issue #14362:
URL: https://github.com/apache/pinot/issues/14362#issuecomment-2455966396
absolutely! feel free to tag me in to review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Time Pruner should support queries where we're operating on the time column itself [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on issue #14362:
URL: https://github.com/apache/pinot/issues/14362#issuecomment-2452733962
We do have `TimePredicateFilterOptimizer` introduced in #6957. We should be able to enhance that to also support `DATE_TRUNC`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Time Pruner should support queries where we're operating on the time column itself [pinot]
Posted by "jadami10 (via GitHub)" <gi...@apache.org>.
jadami10 commented on issue #14362:
URL: https://github.com/apache/pinot/issues/14362#issuecomment-2453481978
great link, ty jackie. I won't have time to work on this immediately (our user also just modified their query instead), but I'll try to come back to it in free time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Time Pruner should support queries where we're operating on the time column itself [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang closed issue #14362: Time Pruner should support queries where we're operating on the time column itself
URL: https://github.com/apache/pinot/issues/14362
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Time Pruner should support queries where we're operating on the time column itself [pinot]
Posted by "ashishjayamohan (via GitHub)" <gi...@apache.org>.
ashishjayamohan commented on issue #14362:
URL: https://github.com/apache/pinot/issues/14362#issuecomment-2455829840
@jadami10
Mind if I take a stab at this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org