[GitHub] [spark] bozhang2820 commented on a change in pull request #31849: [SPARK-34757][CORE][DEPLOY] Ignore cache for SNAPSHOT dependencies in spark-submit

[GitBox <gi...@apache.org>]

bozhang2820 commented on a change in pull request #31849:
URL: https://github.com/apache/spark/pull/31849#discussion_r596582901



##########
File path: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala
##########
@@ -1153,6 +1153,8 @@ private[spark] object SparkSubmitUtils extends Logging {
     // We need a chain resolver if we want to check multiple repositories
     val cr = new ChainResolver
     cr.setName("spark-list")
+    cr.setChangingMatcher(PatternMatcher.REGEXP)
+    cr.setChangingPattern(".*-SNAPSHOT")

Review comment:
       The major reason for this change is to reduce frictions for developers, especially new ones, to contribute to Spark external modules. 
   
   I think the behavior proposed here matches the standard ones in Ivy. See "use a naming convention like a special suffix" in https://ant.apache.org/ivy/history/latest-milestone/bestpractices.html, and the implementation in ivy's `IBiblioResolver`: https://github.com/apache/ant-ivy/blob/master/src/java/org/apache/ivy/plugins/resolver/IBiblioResolver.java#L88-L92
   




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org