[GitHub] [spark] peter-toth commented on issue #25644: [SPARK-28940][SQL] Subquery reuse across all subquery levels

[GitBox <gi...@apache.org>]

peter-toth commented on issue #25644: [SPARK-28940][SQL] Subquery reuse across all subquery levels
URL: https://github.com/apache/spark/pull/25644#issuecomment-538660898
 
 
   After https://github.com/apache/spark/pull/26015 some of the new UTs showed and error and I realized that the new feature of that PR (exchange reuse across all subquery levels) conflicts with my implementation of subquery reuse across all subquery levels.
   This was because the `subqueryCache` that I added to the `QueryExecution` contained a cached subquery that was later removed by the new version of `ReuseExchange` rule. The subquery that was cached had a parent `Exchange` node that was replaced to a `ReusedExchangeExec` node.
   In the last commit I changed my subquery reuse implementation to something similar to exchange reuse implementation.
   I believe this new approach is a bit less efficient but the advantage is that there is no more need for `subqueryCache` in `QueryExecution` and overall the required change is smaller than before.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org