[GitHub] [arrow-datafusion] mustafasrepo commented on a diff in pull request #7099: [BUG FIX]: Projection columns_map remove name search

["mustafasrepo (via GitHub)" <gi...@apache.org>]

mustafasrepo commented on code in PR #7099:
URL: https://github.com/apache/arrow-datafusion/pull/7099#discussion_r1274587454


##########
datafusion/core/tests/sqllogictests/test_files/groupby.slt:
##########
@@ -2568,6 +2568,52 @@ TUR 100 75 175
 GRC 80 30 110
 FRA 200 50 250
 
+query TT
+EXPLAIN SELECT s.zip_code, s.country, s.sn, s.ts, s.currency, LAST_VALUE(e.amount ORDER BY e.sn) AS last_rate
+FROM sales_global AS s
+JOIN sales_global AS e
+  ON s.currency = e.currency AND
+  s.ts >= e.ts
+GROUP BY s.sn, s.zip_code, s.country, s.ts, s.currency
+ORDER BY s.sn
+----
+logical_plan
+Sort: s.sn ASC NULLS LAST
+--Projection: s.zip_code, s.country, s.sn, s.ts, s.currency, LAST_VALUE(e.amount) ORDER BY [e.sn ASC NULLS LAST] AS last_rate
+----Aggregate: groupBy=[[s.sn, s.zip_code, s.country, s.ts, s.currency]], aggr=[[LAST_VALUE(e.amount) ORDER BY [e.sn ASC NULLS LAST]]]
+------Projection: s.zip_code, s.country, s.sn, s.ts, s.currency, e.sn, e.amount
+--------Inner Join: s.currency = e.currency Filter: s.ts >= e.ts
+----------SubqueryAlias: s
+------------TableScan: sales_global projection=[zip_code, country, sn, ts, currency]
+----------SubqueryAlias: e
+------------TableScan: sales_global projection=[sn, ts, currency, amount]
+physical_plan
+SortExec: expr=[sn@2 ASC NULLS LAST]

Review Comment:
   Previously this `SortExec` was optimized away. Because Projections thinks that `sn@5` and `sn@2` are equivalent, and one satisfies the other. Because `columns_map` relied on name search, previously, projection could produced wrong equivalence when schema contained columns with same name.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org