You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "irenjj (via GitHub)" <gi...@apache.org> on 2025/02/28 14:32:00 UTC
[PR] chore: forbide `with_default_features` override existing information [datafusion]
irenjj opened a new pull request, #14935:
URL: https://github.com/apache/datafusion/pull/14935
## Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes #123` indicates that this PR will close issue #123.
-->
- Closes #14899
## Rationale for this change
<!--
Why are you proposing this change? If this is already explained clearly in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand your changes and offer better suggestions for fixes.
-->
## What changes are included in this PR?
<!--
There is no need to duplicate the description in the issue here but it is sometimes worth providing a summary of the individual changes in this PR.
-->
## Are these changes tested?
<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
2. Serve as another way to document the expected behavior of the code
If tests are not included in your PR, please explain why (for example, are they covered by existing tests)?
-->
## Are there any user-facing changes?
<!--
If there are user-facing changes then we may require documentation to be updated before approving the PR.
-->
<!--
If there are any breaking changes to public APIs, please add the `api change` label.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] chore: forbide `with_default_features` override existing information [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #14935:
URL: https://github.com/apache/datafusion/pull/14935#discussion_r1977530272
##########
datafusion/core/src/execution/session_state.rs:
##########
@@ -1081,14 +1081,40 @@ impl SessionStateBuilder {
/// Create default builder with defaults for table_factories, file formats, expr_planners and builtin
/// scalar, aggregate and windows functions.
Review Comment:
```suggestion
/// Adds defaults for table_factories, file formats, expr_planners and builtin
/// scalar, aggregate and windows functions.
///
/// Note overwrites any previously registered items with the same name.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] chore: forbide `with_default_features` override existing information [datafusion]
Posted by "milenkovicm (via GitHub)" <gi...@apache.org>.
milenkovicm commented on PR #14935:
URL: https://github.com/apache/datafusion/pull/14935#issuecomment-2694440302
> Thank you @irenjj -- I think this makes a lot of sense to me
>
> > I just wonder if we should provide with_default_features or Default::default implementation for this case as well? wdyt @alamb ?
>
> @milenkovicm I think the usecase of not automatically registering all the default features is to allow creating a SessionState _without_ the defaults (e.g. if you don't want to defaults).
>
> Since there is no method like `SessionStateBuilder::clear` I am not sure how else one would accomplish this usecase 🤔
than makes sense, what I suggest is to keep all (new) methods as they are just provide new method which will create new with defaults. but I don't have strong opinion about it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] chore: forbide `with_default_features` override existing information [datafusion]
Posted by "irenjj (via GitHub)" <gi...@apache.org>.
irenjj commented on code in PR #14935:
URL: https://github.com/apache/datafusion/pull/14935#discussion_r1976239466
##########
datafusion/core/src/execution/session_state.rs:
##########
@@ -1081,6 +1081,8 @@ impl SessionStateBuilder {
/// Create default builder with defaults for table_factories, file formats, expr_planners and builtin
/// scalar, aggregate and windows functions.
+ /// For each setter method call, default values will only be created and set if the corresponding
+ /// field is currently None, otherwise the existing value will be preserved.
pub fn with_default_features(self) -> Self {
Review Comment:
Thanks for the suggestion. I think the second approach is more suitable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] chore: Update `SessionStateBuilder::with_default_features` does not replace existing features [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb merged PR #14935:
URL: https://github.com/apache/datafusion/pull/14935
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] chore: forbide `with_default_features` override existing information [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #14935:
URL: https://github.com/apache/datafusion/pull/14935#discussion_r1975987954
##########
datafusion/core/src/execution/session_state.rs:
##########
@@ -1144,7 +1146,9 @@ impl SessionStateBuilder {
mut self,
expr_planners: Vec<Arc<dyn ExprPlanner>>,
) -> Self {
- self.expr_planners = Some(expr_planners);
+ if self.expr_planners.is_none() {
Review Comment:
By putting these checks here I think the API will be quite confusing to use -- it means that calling `with_expr_planners` and the other methods below will only set the fields if there is no existing field set and silently ignore the argument if there is already planners specified
##########
datafusion/core/src/execution/session_state.rs:
##########
@@ -1081,6 +1081,8 @@ impl SessionStateBuilder {
/// Create default builder with defaults for table_factories, file formats, expr_planners and builtin
/// scalar, aggregate and windows functions.
+ /// For each setter method call, default values will only be created and set if the corresponding
+ /// field is currently None, otherwise the existing value will be preserved.
pub fn with_default_features(self) -> Self {
Review Comment:
Thanks @irenjj
I thought the idea was that `with_default_features` would be *additive* to the existing set of features
That is, instead of replacing any existing factors/formats/etc this methods would *append* the default features
So instead of
```rust
self.with_table_factories(SessionStateDefaults::default_table_factories())
```
Something more like
```rust
let mut table_factories = self.with_table_factories.take().unwrap_or_default();
table_factories.extend(SessionStateDefaults::default_table_factories())
self.table_factories = Some(table_factories)
```
🤔
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] chore: forbide `with_default_features` override existing information [datafusion]
Posted by "milenkovicm (via GitHub)" <gi...@apache.org>.
milenkovicm commented on PR #14935:
URL: https://github.com/apache/datafusion/pull/14935#issuecomment-2692244166
I just wonder if we should provide `with_default_features` or `Default::default` implementation for this case as well? wdyt @alamb ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] chore: Update `SessionStateBuilder::with_default_features` does not replace existing features [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #14935:
URL: https://github.com/apache/datafusion/pull/14935#issuecomment-2694451094
I also made a small PR to try and clarify the docs:
- https://github.com/apache/datafusion/pull/14980
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] chore: Update `SessionStateBuilder::with_default_features` does not replace existing features [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #14935:
URL: https://github.com/apache/datafusion/pull/14935#issuecomment-2694597493
> > Thank you @irenjj -- I think this makes a lot of sense to me
> > > I just wonder if we should provide with_default_features or Default::default implementation for this case as well? wdyt @alamb ?
> >
> >
> > @milenkovicm I think the usecase of not automatically registering all the default features is to allow creating a SessionState _without_ the defaults (e.g. if you don't want to defaults).
> > Since there is no method like `SessionStateBuilder::clear` I am not sure how else one would accomplish this usecase 🤔
>
> than makes sense, what I suggest is to keep all (new) methods as they are just provide new method which will create new with defaults. but I don't have strong opinion about it
- Filed https://github.com/apache/datafusion/issues/14981 to track
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] chore: Update `SessionStateBuilder::with_default_features` does not replace existing features [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on PR #14935:
URL: https://github.com/apache/datafusion/pull/14935#issuecomment-2697244168
Thanks again @irenjj and @milenkovicm
It seems as if @shruti2522 already has a PR to implement the new_with_default_features
- https://github.com/apache/datafusion/pull/14998
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org