You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/08/24 14:27:16 UTC

[GitHub] [arrow-datafusion] andygrove opened a new issue, #3248: Remove redundant projection when querying a view

andygrove opened a new issue, #3248:
URL: https://github.com/apache/arrow-datafusion/issues/3248

   **Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
   PR https://github.com/apache/arrow-datafusion/pull/3242 fixes a correctness issue where a projection was being ignored. However, the physical plan now contains a redundant projection and it would be good to remove it.
   
   Example:
   
   ```
   DataFusion CLI v11.0.0
   ❯ create view v as select 1 as a, 2 as b, 3 as c;
   0 rows in set. Query took 0.002 seconds.
   
   ❯ explain select * from (select b from v);
   +---------------+---------------------------------------+
   | plan_type     | plan                                  |
   +---------------+---------------------------------------+
   | logical_plan  | Projection: #v.b                      |
   |               |   TableScan: v projection=[b]         |
   | physical_plan | ProjectionExec: expr=[b@0 as b]       |
   |               |   ProjectionExec: expr=[b@0 as b]     |
   |               |     ProjectionExec: expr=[2 as b]     |
   |               |       EmptyExec: produce_one_row=true |
   |               |                                       |
   +---------------+---------------------------------------+
   2 rows in set. Query took 0.009 seconds.
   ```
   The physical plan returned from `ViewTable::scan` is:
   
   ```
   ProjectionExec: expr=[b@0 as b]
     ProjectionExec: expr=[2 as b]
       EmptyExec: produce_one_row=true
   ```
   
   The physical plan shown in the `EXPLAIN` query above has an additional redundant projection added.
   
   **Describe the solution you'd like**
   Remove the redundant projection.
   
   **Describe alternatives you've considered**
   None
   
   **Additional context**
   None


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow-datafusion] alamb commented on issue #3248: Remove redundant projection when querying a view

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #3248:
URL: https://github.com/apache/arrow-datafusion/issues/3248#issuecomment-1483335495

   This appears to be complete now:
   
   ```
   git alamb@MacBook-Pro-8:~/Software/influxdb_iox/flightsql/src$ datafusion-cli
   DataFusion CLI v20.0.0
   ❯ create view v as select 1 as a, 2 as b, 3 as c;
   0 rows in set. Query took 0.022 seconds.
   ❯ explain select * from (select b from v);
   +---------------+-----------------------------------+
   | plan_type     | plan                              |
   +---------------+-----------------------------------+
   | logical_plan  | SubqueryAlias: v                  |
   |               |   Projection: Int64(2) AS b       |
   |               |     EmptyRelation                 |
   | physical_plan | ProjectionExec: expr=[2 as b]     |
   |               |   EmptyExec: produce_one_row=true |
   |               |                                   |
   +---------------+-----------------------------------+
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow-datafusion] comphead commented on issue #3248: Remove redundant projection when querying a view

Posted by GitBox <gi...@apache.org>.
comphead commented on issue #3248:
URL: https://github.com/apache/arrow-datafusion/issues/3248#issuecomment-1235749611

   The problem is 
   ```
   ❯ create view v as select 1 as a, 2 as b, 3 as c;
   0 rows in set. Query took 0.001 seconds.
   ❯ explain select * from (select b from v);
   +---------------+---------------------------------------+
   | plan_type     | plan                                  |
   +---------------+---------------------------------------+
   | logical_plan  | Projection: #v.b                      |
   |               |   TableScan: v projection=[b]         |
   | physical_plan | ProjectionExec: expr=[b@0 as b]       |
   |               |   ProjectionExec: expr=[b@0 as b]     |
   |               |     ProjectionExec: expr=[2 as b]     |
   |               |       EmptyExec: produce_one_row=true |
   |               |                                       |
   +---------------+---------------------------------------+
   ```
   The same for table 
   ```
   ❯ create table  v1 as select 1 as a, 2 as b, 3 as c;
   +---+---+---+
   | a | b | c |
   +---+---+---+
   | 1 | 2 | 3 |
   +---+---+---+
   1 row in set. Query took 0.003 seconds.
   ❯ explain select * from (select b from v1);
   +---------------+-------------------------------------------------+
   | plan_type     | plan                                            |
   +---------------+-------------------------------------------------+
   | logical_plan  | Projection: #v1.b                               |
   |               |   TableScan: v1 projection=[b]                  |
   | physical_plan | ProjectionExec: expr=[b@0 as b]                 |
   |               |   MemoryExec: partitions=1, partition_sizes=[1] |
   |               |                                                 |
   +---------------+-------------------------------------------------+
   ```
   @andygrove afaik those physical plans should be the same? 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow-datafusion] comphead commented on issue #3248: Remove redundant projection when querying a view

Posted by GitBox <gi...@apache.org>.
comphead commented on issue #3248:
URL: https://github.com/apache/arrow-datafusion/issues/3248#issuecomment-1230616080

   @andygrove I'll take it


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow-datafusion] HaoYang670 commented on issue #3248: Remove redundant projection when querying a view

Posted by GitBox <gi...@apache.org>.
HaoYang670 commented on issue #3248:
URL: https://github.com/apache/arrow-datafusion/issues/3248#issuecomment-1236238939

   Hi @andygrove  @alamb  @comphead, I guess both this issue and #3249 can be solved by unfolding `view` as a subquery:
   ```
   ❯ create view v as select 1 as a, 2 as b, 3 as c;
   0 rows in set. Query took 0.009 seconds.
   ❯ explain select * from (select b from v);
   +---------------+---------------------------------------+
   | plan_type     | plan                                  |
   +---------------+---------------------------------------+
   | logical_plan  | Projection: #v.b                      |
   |               |   TableScan: v projection=[b]         |
   | physical_plan | ProjectionExec: expr=[b@0 as b]       |
   |               |   ProjectionExec: expr=[b@0 as b]     |
   |               |     ProjectionExec: expr=[2 as b]     |
   |               |       EmptyExec: produce_one_row=true |
   |               |                                       |
   +---------------+---------------------------------------+
   2 rows in set. Query took 0.006 seconds.
   ❯ explain select * from (select b from (select 1 as a, 2 as b, 3 as c));
   +---------------+-------------------------------------+
   | plan_type     | plan                                |
   +---------------+-------------------------------------+
   | logical_plan  | Projection: #b                      |
   |               |   Projection: Int64(2) AS b         |
   |               |     EmptyRelation                   |
   | physical_plan | ProjectionExec: expr=[b@0 as b]     |
   |               |   ProjectionExec: expr=[2 as b]     |
   |               |     EmptyExec: produce_one_row=true |
   |               |                                     |
   +---------------+-------------------------------------+
   2 rows in set. Query took 0.005 seconds.
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow-datafusion] alamb closed issue #3248: Remove redundant projection when querying a view

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb closed issue #3248: Remove redundant projection when querying a view
URL: https://github.com/apache/arrow-datafusion/issues/3248


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow-datafusion] HaoYang670 commented on issue #3248: Remove redundant projection when querying a view

Posted by GitBox <gi...@apache.org>.
HaoYang670 commented on issue #3248:
URL: https://github.com/apache/arrow-datafusion/issues/3248#issuecomment-1237604498

   > Can we close this issue in favor of https://github.com/apache/arrow-datafusion/issues/3249 ?
   
   We should file a PR to close both this issue and 3249.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [arrow-datafusion] Dandandan commented on issue #3248: Remove redundant projection when querying a view

Posted by GitBox <gi...@apache.org>.
Dandandan commented on issue #3248:
URL: https://github.com/apache/arrow-datafusion/issues/3248#issuecomment-1237439664

   Can we close this issue in favor of https://github.com/apache/arrow-datafusion/issues/3249 ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org