[GitHub] [spark] cloud-fan commented on pull request #42376: [SPARK-44700][SQL] Rule OptimizeCsvJsonExprs should not be applied to expression like from_json(regexp_replace)

["cloud-fan (via GitHub)" <gi...@apache.org>]

cloud-fan commented on PR #42376:
URL: https://github.com/apache/spark/pull/42376#issuecomment-1669240688

   cc @wangyum do you have any ideas? It seems any optimization that changes the expression shape may break common subexpression elimination (CSE). It's hard to come up with a good cost model to fix it. I think a better idea is to make CSE a plan-level optimization, so that we can find all common subexpressions before optimizing expressions. But it's hard to do.
   
   @monkeyboy123 is it possible to rewrite your query and use subquery alias or CTE to hold the expression result, to avoid repeated execution? or you can disable this optimization by setting `spark.sql.optimizer.excludedRules` to include this rule.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org