[jira] [Created] (SPARK-19957) Inconsist KMeans initialization mode behavior between ML and MLlib

["yuhao yang (JIRA)" <ji...@apache.org>]

yuhao yang created SPARK-19957:
----------------------------------

             Summary: Inconsist KMeans initialization mode behavior between ML and MLlib
                 Key: SPARK-19957
                 URL: https://issues.apache.org/jira/browse/SPARK-19957
             Project: Spark
          Issue Type: Bug
          Components: ML
    Affects Versions: 2.1.0
            Reporter: yuhao yang
            Priority: Minor


when users set the initialization mode to "random", KMeans in ML and MLlib has inconsistent behavior for multiple runs:

MLlib will basically use new Random for each run.
ML Kmeans however will use the default random seed, which is {code}this.getClass.getName.hashCode.toLong{code}, and keep using the same number among multiple fitting.

I would expect the "random" initialization mode to be literally random. There're different solutions with different scope of impact. Adjusting the hasSeed trait may have a broader impact. We can always just set random default seed in KMeans. 

Appreciate your feedback.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org