You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/11/23 01:40:25 UTC
[GitHub] [arrow-datafusion] jackwener opened a new pull request, #4333: API-break: Support `SubqueryAlias` and remove Projection-Alias
jackwener opened a new pull request, #4333:
URL: https://github.com/apache/arrow-datafusion/pull/4333
# Which issue does this PR close?
close #3927 #2212 #4291
# Rationale for this change
Remove `alias in projection`, and replace it by `SubqueryAlias`.
# What changes are included in this PR?
<!--
There is no need to duplicate the description in the issue here but it is sometimes worth providing a summary of the individual changes in this PR.
-->
# Are these changes tested?
<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
2. Serve as another way to document the expected behavior of the code
If tests are not included in your PR, please explain why (for example, are they covered by existing tests)?
-->
# Are there any user-facing changes?
`Projection` Struct change
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on pull request #4333: API-break: Support `SubqueryAlias` and remove `Alias in Projection`
Posted by GitBox <gi...@apache.org>.
alamb commented on PR #4333:
URL: https://github.com/apache/arrow-datafusion/pull/4333#issuecomment-1328231639
I merged this PR to the latest master branch locally to ensure it has no logical conflicts. Thanks again @jackwener
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] jackwener commented on pull request #4333: API-break: Support `SubqueryAlias` and remove Projection-Alias
Posted by GitBox <gi...@apache.org>.
jackwener commented on PR #4333:
URL: https://github.com/apache/arrow-datafusion/pull/4333#issuecomment-1324446360
#4293 is part job of this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] liukun4515 commented on pull request #4333: API-break: Support `SubqueryAlias` and remove `Alias in Projection`
Posted by GitBox <gi...@apache.org>.
liukun4515 commented on PR #4333:
URL: https://github.com/apache/arrow-datafusion/pull/4333#issuecomment-1325097709
> PTAL, @Dandandan @alamb @andygrove @liukun4515 @mingmwang
Thanks @jackwener I will review it later
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] jackwener commented on pull request #4333: API-break: Support `SubqueryAlias` and remove `Alias in Projection`
Posted by GitBox <gi...@apache.org>.
jackwener commented on PR #4333:
URL: https://github.com/apache/arrow-datafusion/pull/4333#issuecomment-1328068834
> The only thing I am concerned about is the regression in supporting limit pushdown through subquery. Otherwise I think this PR could be merged.
regression is resolved in #4384
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] jackwener commented on pull request #4333: API-break: Support `SubqueryAlias` and remove Projection-Alias
Posted by GitBox <gi...@apache.org>.
jackwener commented on PR #4333:
URL: https://github.com/apache/arrow-datafusion/pull/4333#issuecomment-1324446946
PTAL, @Dandandan @alamb @andygrove @liukun4515 @mingmwang
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] jackwener commented on a diff in pull request #4333: API-break: Support `SubqueryAlias` and remove `Alias in Projection`
Posted by GitBox <gi...@apache.org>.
jackwener commented on code in PR #4333:
URL: https://github.com/apache/arrow-datafusion/pull/4333#discussion_r1032798149
##########
datafusion/core/tests/sql/joins.rs:
##########
@@ -1635,16 +1635,16 @@ async fn reduce_left_join_3() -> Result<()> {
let expected = vec![
"Explain [plan_type:Utf8, plan:Utf8]",
" Projection: t3.t1_id, t3.t1_name, t3.t1_int, t2.t2_id, t2.t2_name, t2.t2_int [t1_id:UInt32;N, t1_name:Utf8;N, t1_int:UInt32;N, t2_id:UInt32;N, t2_name:Utf8;N, t2_int:UInt32;N]",
- " Left Join: t3.t1_int = t2.t2_int [t1_id:UInt32;N, t1_name:Utf8;N, t1_int:UInt32;N, t2_id:UInt32;N, t2_name:Utf8;N, t2_int:UInt32;N]",
- " Projection: t1.t1_id, t1.t1_name, t1.t1_int, alias=t3 [t1_id:UInt32;N, t1_name:Utf8;N, t1_int:UInt32;N]",
- " Projection: t1.t1_id, t1.t1_name, t1.t1_int [t1_id:UInt32;N, t1_name:Utf8;N, t1_int:UInt32;N]",
+ " Left Join: t3.t1_int = t2.t2_int [t1_id:UInt32;N, t1_name:Utf8;N, t1_int:UInt32;N, t2_id:UInt32;N, t2_name:Utf8;N, t2_int:UInt32;N, t2_id:UInt32;N, t2_name:Utf8;N, t2_int:UInt32;N]",
+ " Filter: t3.t1_id < UInt32(100) [t1_id:UInt32;N, t1_name:Utf8;N, t1_int:UInt32;N, t2_id:UInt32;N, t2_name:Utf8;N, t2_int:UInt32;N]",
+ " SubqueryAlias: t3 [t1_id:UInt32;N, t1_name:Utf8;N, t1_int:UInt32;N, t2_id:UInt32;N, t2_name:Utf8;N, t2_int:UInt32;N]",
Review Comment:
#4381 push_down_filter need to support `SubqueryAlias`
wait for #4365
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on pull request #4333: API-break: Support `SubqueryAlias` and remove `Alias in Projection`
Posted by GitBox <gi...@apache.org>.
alamb commented on PR #4333:
URL: https://github.com/apache/arrow-datafusion/pull/4333#issuecomment-1325450716
I will review this but it may take a few days -- there are a bunch of other PRs in the queue before this one
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] jackwener commented on a diff in pull request #4333: API-break: Support `SubqueryAlias` and remove `Alias in Projection`
Posted by GitBox <gi...@apache.org>.
jackwener commented on code in PR #4333:
URL: https://github.com/apache/arrow-datafusion/pull/4333#discussion_r1032793752
##########
datafusion/core/tests/sql/window.rs:
##########
@@ -335,18 +335,19 @@ async fn window_expr_eliminate() -> Result<()> {
" Sort: d.b ASC NULLS LAST [b:Utf8, max_a:Int64;N]",
" Projection: d.b, MAX(d.a) AS max_a [b:Utf8, max_a:Int64;N]",
" Aggregate: groupBy=[[d.b]], aggr=[[MAX(d.a)]] [b:Utf8, MAX(d.a):Int64;N]",
- " Projection: _data2.a, _data2.b, alias=d [a:Int64, b:Utf8]",
- " Projection: s.a, s.b, alias=_data2 [a:Int64, b:Utf8]",
- " Projection: a, b, alias=s [a:Int64, b:Utf8]",
- " Union [a:Int64, b:Utf8]",
- " Projection: Int64(1) AS a, Utf8(\"aa\") AS b [a:Int64, b:Utf8]",
- " EmptyRelation []",
- " Projection: Int64(3) AS a, Utf8(\"aa\") AS b [a:Int64, b:Utf8]",
- " EmptyRelation []",
- " Projection: Int64(5) AS a, Utf8(\"bb\") AS b [a:Int64, b:Utf8]",
- " EmptyRelation []",
- " Projection: Int64(7) AS a, Utf8(\"bb\") AS b [a:Int64, b:Utf8]",
- " EmptyRelation []",
+ " SubqueryAlias: d [a:Int64, b:Utf8]",
+ " SubqueryAlias: _data2 [a:Int64, b:Utf8]",
Review Comment:
Future ticket #4383
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] jackwener commented on a diff in pull request #4333: API-break: Support `SubqueryAlias` and remove `Alias in Projection`
Posted by GitBox <gi...@apache.org>.
jackwener commented on code in PR #4333:
URL: https://github.com/apache/arrow-datafusion/pull/4333#discussion_r1032798149
##########
datafusion/core/tests/sql/joins.rs:
##########
@@ -1635,16 +1635,16 @@ async fn reduce_left_join_3() -> Result<()> {
let expected = vec![
"Explain [plan_type:Utf8, plan:Utf8]",
" Projection: t3.t1_id, t3.t1_name, t3.t1_int, t2.t2_id, t2.t2_name, t2.t2_int [t1_id:UInt32;N, t1_name:Utf8;N, t1_int:UInt32;N, t2_id:UInt32;N, t2_name:Utf8;N, t2_int:UInt32;N]",
- " Left Join: t3.t1_int = t2.t2_int [t1_id:UInt32;N, t1_name:Utf8;N, t1_int:UInt32;N, t2_id:UInt32;N, t2_name:Utf8;N, t2_int:UInt32;N]",
- " Projection: t1.t1_id, t1.t1_name, t1.t1_int, alias=t3 [t1_id:UInt32;N, t1_name:Utf8;N, t1_int:UInt32;N]",
- " Projection: t1.t1_id, t1.t1_name, t1.t1_int [t1_id:UInt32;N, t1_name:Utf8;N, t1_int:UInt32;N]",
+ " Left Join: t3.t1_int = t2.t2_int [t1_id:UInt32;N, t1_name:Utf8;N, t1_int:UInt32;N, t2_id:UInt32;N, t2_name:Utf8;N, t2_int:UInt32;N, t2_id:UInt32;N, t2_name:Utf8;N, t2_int:UInt32;N]",
+ " Filter: t3.t1_id < UInt32(100) [t1_id:UInt32;N, t1_name:Utf8;N, t1_int:UInt32;N, t2_id:UInt32;N, t2_name:Utf8;N, t2_int:UInt32;N]",
+ " SubqueryAlias: t3 [t1_id:UInt32;N, t1_name:Utf8;N, t1_int:UInt32;N, t2_id:UInt32;N, t2_name:Utf8;N, t2_int:UInt32;N]",
Review Comment:
`Filter: t3.t1_id < UInt32(100)` don't be pushdown.
#4381 push_down_filter need to support `SubqueryAlias`
wait for #4365
has been resolved in #4384
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] jackwener commented on pull request #4333: API-break: Support `SubqueryAlias` and remove Projection-Alias
Posted by GitBox <gi...@apache.org>.
jackwener commented on PR #4333:
URL: https://github.com/apache/arrow-datafusion/pull/4333#issuecomment-1324445152
Github `file diff` rendering is weird. We should open `Hide whitespace`.
This is link with opening it. https://github.com/apache/arrow-datafusion/pull/4333/files?&w=1.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] jackwener commented on a diff in pull request #4333: API-break: Support `SubqueryAlias` and remove `Alias in Projection`
Posted by GitBox <gi...@apache.org>.
jackwener commented on code in PR #4333:
URL: https://github.com/apache/arrow-datafusion/pull/4333#discussion_r1032798149
##########
datafusion/core/tests/sql/joins.rs:
##########
@@ -1635,16 +1635,16 @@ async fn reduce_left_join_3() -> Result<()> {
let expected = vec![
"Explain [plan_type:Utf8, plan:Utf8]",
" Projection: t3.t1_id, t3.t1_name, t3.t1_int, t2.t2_id, t2.t2_name, t2.t2_int [t1_id:UInt32;N, t1_name:Utf8;N, t1_int:UInt32;N, t2_id:UInt32;N, t2_name:Utf8;N, t2_int:UInt32;N]",
- " Left Join: t3.t1_int = t2.t2_int [t1_id:UInt32;N, t1_name:Utf8;N, t1_int:UInt32;N, t2_id:UInt32;N, t2_name:Utf8;N, t2_int:UInt32;N]",
- " Projection: t1.t1_id, t1.t1_name, t1.t1_int, alias=t3 [t1_id:UInt32;N, t1_name:Utf8;N, t1_int:UInt32;N]",
- " Projection: t1.t1_id, t1.t1_name, t1.t1_int [t1_id:UInt32;N, t1_name:Utf8;N, t1_int:UInt32;N]",
+ " Left Join: t3.t1_int = t2.t2_int [t1_id:UInt32;N, t1_name:Utf8;N, t1_int:UInt32;N, t2_id:UInt32;N, t2_name:Utf8;N, t2_int:UInt32;N, t2_id:UInt32;N, t2_name:Utf8;N, t2_int:UInt32;N]",
+ " Filter: t3.t1_id < UInt32(100) [t1_id:UInt32;N, t1_name:Utf8;N, t1_int:UInt32;N, t2_id:UInt32;N, t2_name:Utf8;N, t2_int:UInt32;N]",
+ " SubqueryAlias: t3 [t1_id:UInt32;N, t1_name:Utf8;N, t1_int:UInt32;N, t2_id:UInt32;N, t2_name:Utf8;N, t2_int:UInt32;N]",
Review Comment:
#4381 push_down_filter need to support `SubqueryAlias`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] jackwener commented on a diff in pull request #4333: API-break: Support `SubqueryAlias` and remove `Alias in Projection`
Posted by GitBox <gi...@apache.org>.
jackwener commented on code in PR #4333:
URL: https://github.com/apache/arrow-datafusion/pull/4333#discussion_r1032788324
##########
datafusion/core/src/datasource/view.rs:
##########
@@ -474,7 +474,8 @@ mod tests {
let formatted = arrow::util::pretty::pretty_format_batches(&plan)
.unwrap()
.to_string();
- assert!(formatted.contains("ParquetExec: limit=Some(10)"));
+ // TODO: limit_push_down support SubqueryAlias
Review Comment:
Add a new issue to trace #4381.
Yes, I'll follow up on this soon, the reason I didn't do this in this PR is because I didn't want to mix too many features into one big PR (Not only pushdown limit, other rules also need to support it, I want to support them altogether, and add ut for it, it will be easy to review.).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb merged pull request #4333: API-break: Support `SubqueryAlias` and remove `Alias in Projection`
Posted by GitBox <gi...@apache.org>.
alamb merged PR #4333:
URL: https://github.com/apache/arrow-datafusion/pull/4333
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] jackwener commented on a diff in pull request #4333: API-break: Support `SubqueryAlias` and remove `Alias in Projection`
Posted by GitBox <gi...@apache.org>.
jackwener commented on code in PR #4333:
URL: https://github.com/apache/arrow-datafusion/pull/4333#discussion_r1032798149
##########
datafusion/core/tests/sql/joins.rs:
##########
@@ -1635,16 +1635,16 @@ async fn reduce_left_join_3() -> Result<()> {
let expected = vec![
"Explain [plan_type:Utf8, plan:Utf8]",
" Projection: t3.t1_id, t3.t1_name, t3.t1_int, t2.t2_id, t2.t2_name, t2.t2_int [t1_id:UInt32;N, t1_name:Utf8;N, t1_int:UInt32;N, t2_id:UInt32;N, t2_name:Utf8;N, t2_int:UInt32;N]",
- " Left Join: t3.t1_int = t2.t2_int [t1_id:UInt32;N, t1_name:Utf8;N, t1_int:UInt32;N, t2_id:UInt32;N, t2_name:Utf8;N, t2_int:UInt32;N]",
- " Projection: t1.t1_id, t1.t1_name, t1.t1_int, alias=t3 [t1_id:UInt32;N, t1_name:Utf8;N, t1_int:UInt32;N]",
- " Projection: t1.t1_id, t1.t1_name, t1.t1_int [t1_id:UInt32;N, t1_name:Utf8;N, t1_int:UInt32;N]",
+ " Left Join: t3.t1_int = t2.t2_int [t1_id:UInt32;N, t1_name:Utf8;N, t1_int:UInt32;N, t2_id:UInt32;N, t2_name:Utf8;N, t2_int:UInt32;N, t2_id:UInt32;N, t2_name:Utf8;N, t2_int:UInt32;N]",
+ " Filter: t3.t1_id < UInt32(100) [t1_id:UInt32;N, t1_name:Utf8;N, t1_int:UInt32;N, t2_id:UInt32;N, t2_name:Utf8;N, t2_int:UInt32;N]",
+ " SubqueryAlias: t3 [t1_id:UInt32;N, t1_name:Utf8;N, t1_int:UInt32;N, t2_id:UInt32;N, t2_name:Utf8;N, t2_int:UInt32;N]",
Review Comment:
#4381 push_down_filter need to support `SubqueryAlias`
wait for #4365
has been resolved in #4384
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] jackwener commented on a diff in pull request #4333: API-break: Support `SubqueryAlias` and remove `Alias in Projection`
Posted by GitBox <gi...@apache.org>.
jackwener commented on code in PR #4333:
URL: https://github.com/apache/arrow-datafusion/pull/4333#discussion_r1032788324
##########
datafusion/core/src/datasource/view.rs:
##########
@@ -474,7 +474,8 @@ mod tests {
let formatted = arrow::util::pretty::pretty_format_batches(&plan)
.unwrap()
.to_string();
- assert!(formatted.contains("ParquetExec: limit=Some(10)"));
+ // TODO: limit_push_down support SubqueryAlias
Review Comment:
Yes, I'll follow up on this soon, the reason I didn't do this in this PR is because I didn't want to mix too many features into one big PR (Not only pushdown limit, other rules also need to support it, I want to support them altogether, and add ut for it, it will be easy to review.).
Add a new issue to trace #4381
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] jackwener commented on pull request #4333: API-break: Support `SubqueryAlias` and remove `Alias in Projection`
Posted by GitBox <gi...@apache.org>.
jackwener commented on PR #4333:
URL: https://github.com/apache/arrow-datafusion/pull/4333#issuecomment-1327107010
Followup we need to remove alias in projection in `proto`, and [ballista](https://github.com/apache/arrow-ballista) also may need to do some change for it? cc @andygrove
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] ursabot commented on pull request #4333: API-break: Support `SubqueryAlias` and remove `Alias in Projection`
Posted by GitBox <gi...@apache.org>.
ursabot commented on PR #4333:
URL: https://github.com/apache/arrow-datafusion/pull/4333#issuecomment-1328233448
Benchmark runs are scheduled for baseline = da54fa584c7c4e8069ca6aed51e7c1f0dffc807c and contender = ad3df7dd49f37d642422d53c8e1a49e7828d7d40. ad3df7dd49f37d642422d53c8e1a49e7828d7d40 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Skipped :warning: Benchmarking of arrow-datafusion-commits is not supported on ec2-t3-xlarge-us-east-2] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/20a37d697d4942cc8e4c248879e45c7b...c604893879ce459f840e0edba7485883/)
[Skipped :warning: Benchmarking of arrow-datafusion-commits is not supported on test-mac-arm] [test-mac-arm](https://conbench.ursa.dev/compare/runs/51d487d68c1b4a81988afc2dff09a709...bc57c51faf3e409dbe823a94b0db18b5/)
[Skipped :warning: Benchmarking of arrow-datafusion-commits is not supported on ursa-i9-9960x] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/d500c8eaccfa4c1b87be4c4d44fb3f12...fa00c8d48bdc4d9eb327959e2da9dee6/)
[Skipped :warning: Benchmarking of arrow-datafusion-commits is not supported on ursa-thinkcentre-m75q] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/3a31df66345b4aef97678b1b06efb5cc...fbbf198d4de940c397cdeb2947cb7c45/)
Buildkite builds:
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on a diff in pull request #4333: API-break: Support `SubqueryAlias` and remove `Alias in Projection`
Posted by GitBox <gi...@apache.org>.
alamb commented on code in PR #4333:
URL: https://github.com/apache/arrow-datafusion/pull/4333#discussion_r1032786474
##########
datafusion/core/src/datasource/view.rs:
##########
@@ -474,7 +474,8 @@ mod tests {
let formatted = arrow::util::pretty::pretty_format_batches(&plan)
.unwrap()
.to_string();
- assert!(formatted.contains("ParquetExec: limit=Some(10)"));
+ // TODO: limit_push_down support SubqueryAlias
Review Comment:
We should perhaps track this with a ticket -- it seems like it is a regression not to push limits into the subquery
##########
datafusion/expr/src/logical_plan/builder.rs:
##########
@@ -980,19 +977,26 @@ pub fn project_with_alias(
exprlist_to_fields(&projected_expr, &plan)?,
plan.schema().metadata().clone(),
)?;
- let schema = match alias {
- Some(ref alias) => input_schema.replace_qualifier(alias.as_str()),
- None => input_schema,
- };
- Ok(LogicalPlan::Projection(
- Projection::try_new_with_schema_alias(
- projected_expr,
- Arc::new(plan.clone()),
- DFSchemaRef::new(schema),
- alias,
- )?,
- ))
+ let projection = LogicalPlan::Projection(Projection::try_new_with_schema(
+ projected_expr,
+ Arc::new(plan.clone()),
+ DFSchemaRef::new(input_schema),
+ )?);
+ match alias {
+ Some(alias) => Ok(with_alias(projection, alias)),
+ None => Ok(projection),
+ }
+}
+
+pub fn with_alias(plan: LogicalPlan, alias: String) -> LogicalPlan {
Review Comment:
Could you please add some docstrings to this function (perhaps as a follow on PR)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org