You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "anuragrai16 (via GitHub)" <gi...@apache.org> on 2025/10/31 05:34:09 UTC
[I] [MSE Lite] Support for sub-plan based execution for sub queries [pinot]
anuragrai16 opened a new issue, #17118:
URL: https://github.com/apache/pinot/issues/17118
When queries such as,
`SELECT *FROM main_table WHERE uuid IN ( SELECT uuid FROM lookup_table WHERE filter = 'true' );`
is executed in Pinot,
- The V1 engine Pinot V1 uses IN_SUBQUERY where the internal select is first executed to get the list of UUIDs
And then passed as a static list to run the outer filter
- The MSE Lite engine on the other hand will by default evaluate the `IN ...` subquery as a semi-join executed in the broker, which may or may not be performant (based on the size of the results)
We should look at some intelligent way of deciding between pushing the subquery down as a filter or use a SEMI_JOIN based on the result size. Or, we can expose a query hint that allows users to specify whether to use sub-plan based execution or not.
CC @ankitsultana
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] [multistage] Support for sub-plan based execution for sub queries [pinot]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #17118: [multistage] Support for sub-plan based execution for sub queries
URL: https://github.com/apache/pinot/issues/17118
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] [MSE Lite] Support for sub-plan based execution for sub queries [pinot]
Posted by "ankitsultana (via GitHub)" <gi...@apache.org>.
ankitsultana commented on issue #17118:
URL: https://github.com/apache/pinot/issues/17118#issuecomment-3472846825
Just a heads up that we want Sub-Plan execution support for both MSE Lite and regular MSE. Before we implement this let's go through a design review and PoC too. It's important to do this at the right layer so it fits in neatly with the rest of the optimization phases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] [MSE Lite] Support for sub-plan based execution for sub queries [pinot]
Posted by "rohityadav1993 (via GitHub)" <gi...@apache.org>.
rohityadav1993 commented on issue #17118:
URL: https://github.com/apache/pinot/issues/17118#issuecomment-3471418058
> We should look at some intelligent way of deciding between pushing the subquery down as a filter
Should we consider MSE Lite mode to always evaluate as filter?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] [multistage] Support for sub-plan based execution for sub queries [pinot]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #17118:
URL: https://github.com/apache/pinot/issues/17118#issuecomment-4210582854
This issue has been automatically marked as stale because it has not had any activity in the last 150 days. It will be closed in 30 days if no further activity occurs. If this issue is still relevant, please leave a comment to keep it open. Thank you for your contributions!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] [MSE Lite] Support for sub-plan based execution for sub queries [pinot]
Posted by "anuragrai16 (via GitHub)" <gi...@apache.org>.
anuragrai16 commented on issue #17118:
URL: https://github.com/apache/pinot/issues/17118#issuecomment-3471468485
> > We should look at some intelligent way of deciding between pushing the subquery down as a filter
>
> Should we consider MSE Lite mode to always evaluate as filter?
The decision between pushing it down as a filter v/s doing a semi-join depends on the result of the sub-query. If it is smaller - pushing it down as a filter makes sense. If it is quite large, a semi-join would be preferable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] [multistage] Support for sub-plan based execution for sub queries [pinot]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #17118:
URL: https://github.com/apache/pinot/issues/17118#issuecomment-4503799003
This issue has been closed because it has been inactive for 180 days. If you believe this issue is still relevant, please feel free to reopen it or create a new issue. Thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org