[GitHub] [spark] toderesa97 commented on issue #21764: [SPARK-24802][SQL] Add a new config for Optimization Rule Exclusion

[GitBox <gi...@apache.org>]

toderesa97 commented on issue #21764:
URL: https://github.com/apache/spark/pull/21764#issuecomment-616719975


   Hello,
   
    It is not very clear for me, how to exclude some rules. I have been digging into the [testing file for OptimizerRuleExclusionSuite.scala](https://github.com/apache/spark/blob/master/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/OptimizerRuleExclusionSuite.scala) and not able to exclude some rules. As the jira report for issue [SPARK-24802](https://issues.apache.org/jira/browse/SPARK-24802) says:
   
   > Add a new config spark.sql.optimizer.excludedRules, with the value being a list of rule names separated by comma
   
   I tried this:
   
   ```
   import org.apache.spark.sql.{SparkSession}
   import org.apache.spark.sql.catalyst.optimizer._
   
   
   object Main {
   
   
     def getExcludeRules: Seq[String] = {
       Seq(
         PushPredicateThroughJoin.ruleName
       )
     }
   
     def main(args: Array[String]): Unit = {
       val spark: SparkSession = SparkSession
       .builder
       .config("spark.sql.optimizer.excludedRules", getExcludeRules)
       .master("local[*]")
       .getOrCreate
       val sc = spark.sparkContext
       
       // whatever
   
     }
   }
   
   ```
   
   but it is not working since there is not such a method ``config`` that receives a second parameter a ``Seq[String]``.
   
   Thank you for any help you can provide.
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org