You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "timsaucer (via GitHub)" <gi...@apache.org> on 2025/09/15 23:11:12 UTC
[I] Expose physical plan optimizer [datafusion-python]
timsaucer opened a new issue, #1238:
URL: https://github.com/apache/datafusion-python/issues/1238
**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
I want to add custom physical plan optimizer rules, written in rust, to `datafusion-python`. Now that we have the ability to share execution plans across the FFI boundary we should be able to expose this interface.
**Describe the solution you'd like**
- Implement FFI upstream `datafusion`
- Expose `with_physical_optimizer_rules` and/or `with_physical_optimizer_rule`
**Describe alternatives you've considered**
None
**Additional context**
This will support things like https://github.com/datafusion-contrib/datafusion-distributed in `datafusion-python`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Expose physical plan optimizer [datafusion-python]
Posted by "renato2099 (via GitHub)" <gi...@apache.org>.
renato2099 commented on issue #1238:
URL: https://github.com/apache/datafusion-python/issues/1238#issuecomment-3740931642
couldn't [substrate.io](https://substrait.io/) be considered here as well ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Expose physical plan optimizer [datafusion-python]
Posted by "milenkovicm (via GitHub)" <gi...@apache.org>.
milenkovicm commented on issue #1238:
URL: https://github.com/apache/datafusion-python/issues/1238#issuecomment-3429370064
I guess this is the same issue like ballista support
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Expose physical plan optimizer [datafusion-python]
Posted by "timsaucer (via GitHub)" <gi...@apache.org>.
timsaucer commented on issue #1238:
URL: https://github.com/apache/datafusion-python/issues/1238#issuecomment-3304211141
Upon deeper evaluation, this seems non-trivial since the optimizer won't understand what kinds of plans its getting. I suspect we need to go a route of serializing/deserializing the plans.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Expose physical plan optimizer [datafusion-python]
Posted by "timsaucer (via GitHub)" <gi...@apache.org>.
timsaucer closed issue #1238: Expose physical plan optimizer
URL: https://github.com/apache/datafusion-python/issues/1238
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org