You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "alamb (via GitHub)" <gi...@apache.org> on 2024/12/19 11:58:43 UTC
[I] Consolidate parquet.sql into [datafusion]
alamb opened a new issue, #13842:
URL: https://github.com/apache/datafusion/issues/13842
### Is your feature request related to a problem or challenge?
- Part of https://github.com/apache/datafusion/issues/11172
The large number of small examples in DataFusion leads to multiple problems:
- Unnecessarily large amounts of temporary disk space (each example binary takes 100+ MB)
- Larger `cargo t` time (each binary must be linked / created)
- Makes it harder to find the relevant examples
Thus I think the fewer examples we have the
1. faster our dev experience will be
2. easier it will be to find the relevant examples
### Describe the solution you'd like
- Consolidate the contents of [simplify_udaf_expression](https://github.com/apache/datafusion/blob/main/datafusion-examples/examples/simplify_udaf_expression.rs)
- Into [advanced_udaf.rs](https://github.com/apache/datafusion/blob/main/datafusion-examples/examples/advanced_udaf.rs)
Then update the readme page:
- https://github.com/apache/datafusion/blob/main/datafusion-examples/README.md
### Describe alternatives you've considered
_No response_
### Additional context
I think this is a good first issue as the task is straightforward and doesn't require deep understanding of the DataFusion codebase
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Consolidate Example: simplify_udaf_expression.rs into advanced_udaf.rs [datafusion]
Posted by "xudong963 (via GitHub)" <gi...@apache.org>.
xudong963 closed issue #13842: Consolidate Example: simplify_udaf_expression.rs into advanced_udaf.rs
URL: https://github.com/apache/datafusion/issues/13842
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Consolidate Example: simplify_udaf_expression.rs into advanced_udaf.rs [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #13842:
URL: https://github.com/apache/datafusion/issues/13842#issuecomment-2557716904
Thank you @takaebato 🙏
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Consolidate Example: simplify_udaf_expression.rs into advanced_udaf.rs [datafusion]
Posted by "takaebato (via GitHub)" <gi...@apache.org>.
takaebato commented on issue #13842:
URL: https://github.com/apache/datafusion/issues/13842#issuecomment-2562520582
@alamb
Thank you so much for the explanation!!
(I realized that if we want to use the actual implementation conditionally, we can return `Ok(Expr::AggregateFunction(aggregate_function))`.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Consolidate Example: simplify_udaf_expression.rs into advanced_udaf.rs [datafusion]
Posted by "takaebato (via GitHub)" <gi...@apache.org>.
takaebato commented on issue #13842:
URL: https://github.com/apache/datafusion/issues/13842#issuecomment-2557131459
take
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Consolidate Example: simplify_udaf_expression.rs into advanced_udaf.rs [datafusion]
Posted by "takaebato (via GitHub)" <gi...@apache.org>.
takaebato commented on issue #13842:
URL: https://github.com/apache/datafusion/issues/13842#issuecomment-2557129464
I'll work on this!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Consolidate Example: simplify_udaf_expression.rs into advanced_udaf.rs [datafusion]
Posted by "takaebato (via GitHub)" <gi...@apache.org>.
takaebato commented on issue #13842:
URL: https://github.com/apache/datafusion/issues/13842#issuecomment-2561286915
@alamb
Sorry for the delay! I have a quick question.
I’m not too sure about this, but should the return type of `AggregateFunctionSimplification` perhaps be `Result<ExprSimplifyResult>`?
Currently, it looks like there’s no way to skip simplification, as it always returns a simplified `Expr` (or Err).
The simplify method can return an `Option`, but the decision to simplify seems to rely on information passed to `AggregateFunctionSimplification`, so would it be better if this method can choose whether or not to simplify?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] Consolidate Example: simplify_udaf_expression.rs into advanced_udaf.rs [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #13842:
URL: https://github.com/apache/datafusion/issues/13842#issuecomment-2561469759
> @alamb
>
> Sorry for the delay🙏 But I have a quick question! I’m not too sure about this, but should the return type of `AggregateFunctionSimplification` perhaps be `Result<ExprSimplifyResult>`(or a similar enum that can indicate whether or not simplification occurred)?
>
> Currently, it doesn’t seem possible to skip simplification, as it always returns a simplified `Expr` (or Err). The `simplify` method can return an `Option`, but the decision to simplify seems to rely on information passed to `AggregateFunctionSimplification`, so might it be better if this method can choose whether or not to simplify..?
The rationale for returning ExprSimplifyResult is so that the argument doesn't have to be cloned -- aka to avoid coping the ownership of argument is passed to the function and there needs to be some way to get the original structuer back
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org