[GitHub] [spark] yeshengm opened a new pull request #25039: [WIP][SPARK-28237][SQL] Add an Idempotent batch strategy to catch potential bugs in corresponding rules

[GitBox <gi...@apache.org>]

yeshengm opened a new pull request #25039: [WIP][SPARK-28237][SQL] Add an Idempotent batch strategy to catch potential bugs in corresponding rules
URL: https://github.com/apache/spark/pull/25039
 
 
   ## What changes were proposed in this pull request?
   
   The current RuleExecutor system contains two kinds of strategies: Once and FixedPoint. The Once strategy is supposed to run once. However, for particular rules (e.g. PullOutNondeterministic/ResolveUDFs), they are designed to be idempotent, but Spark currently lacks corresponding mechanism to prevent such kind of non-idempotent behavior from happening.
   
   This PR added infrastructure to make these checks possible.
   
   ## How was this patch tested?
   New UTs.
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org