[GitHub] [spark] viirya commented on a change in pull request #28704: [SPARK-31777][ML] Add user-specified fold column to CrossValidator

[GitBox <gi...@apache.org>]

viirya commented on a change in pull request #28704:
URL: https://github.com/apache/spark/pull/28704#discussion_r434017713



##########
File path: mllib/src/main/scala/org/apache/spark/ml/tuning/CrossValidator.scala
##########
@@ -56,6 +56,19 @@ private[ml] trait CrossValidatorParams extends ValidatorParams {
   def getNumFolds: Int = $(numFolds)
 
   setDefault(numFolds -> 3)
+
+  /**
+   * Param for the column name of user specified fold number. Once this is specified,
+   * `CrossValidator` won't do random k-fold split. Note that this column should be
+   * integer type with range [0, numFolds) and Spark won't do sanity-check for this

Review comment:
       For doing sanity check, seems we need a UDF doing the check/throwing error? Could be bad for performance? I think mod numFolds sounds a better one. 




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org