[jira] [Created] (SPARK-16832) CrossValidator and TrainValidationSplit are not random without seed

["Max Moroz (JIRA)" <ji...@apache.org>]

Max Moroz created SPARK-16832:
---------------------------------

             Summary: CrossValidator and TrainValidationSplit are not random without seed
                 Key: SPARK-16832
                 URL: https://issues.apache.org/jira/browse/SPARK-16832
             Project: Spark
          Issue Type: Bug
          Components: ML, PySpark
    Affects Versions: 2.0.0
            Reporter: Max Moroz
            Priority: Minor


Repeatedly running CrossValidator or TrainValidationSplit without an explicit seed parameter does not change results. It is supposed to be seeded with a random seed, but it seems to be instead seeded with some constant. (If seed is explicitly provided, the two classes behave as expected.)

{code}
dataset = spark.createDataFrame(
  [(Vectors.dense([0.0]), 0.0),
   (Vectors.dense([0.4]), 1.0),
   (Vectors.dense([0.5]), 0.0),
   (Vectors.dense([0.6]), 1.0),
   (Vectors.dense([1.0]), 1.0)] * 1000,
  ["features", "label"]).cache()

paramGrid = pyspark.ml.tuning.ParamGridBuilder().build()
tvs = pyspark.ml.tuning.TrainValidationSplit(estimator=pyspark.ml.regression.LinearRegression(), 
                           estimatorParamMaps=paramGrid,
                           evaluator=pyspark.ml.evaluation.RegressionEvaluator(),
                           trainRatio=0.8)
model = tvs.fit(train)
print(model.validationMetrics)

for folds in (3, 5, 10):
  cv = pyspark.ml.tuning.CrossValidator(estimator=pyspark.ml.regression.LinearRegression(), 
                                      estimatorParamMaps=paramGrid, 
                                      evaluator=pyspark.ml.evaluation.RegressionEvaluator(),
                                      numFolds=folds
                                     )
  cvModel = cv.fit(dataset)
  print(folds, cvModel.avgMetrics)
{code}

This code produces identical results upon repeated calls.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org