You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (JIRA)" <ji...@apache.org> on 2019/05/21 05:35:32 UTC
[jira] [Updated] (SPARK-3218) K-Means clusterer can fail on
degenerate data
[ https://issues.apache.org/jira/browse/SPARK-3218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon updated SPARK-3218:
--------------------------------
Labels: bulk-closed clustering (was: clustering)
> K-Means clusterer can fail on degenerate data
> ---------------------------------------------
>
> Key: SPARK-3218
> URL: https://issues.apache.org/jira/browse/SPARK-3218
> Project: Spark
> Issue Type: Bug
> Components: MLlib
> Affects Versions: 1.0.2
> Reporter: Derrick Burns
> Assignee: Derrick Burns
> Priority: Minor
> Labels: bulk-closed, clustering
>
> The KMeans parallel implementation selects points to be cluster centers with probability weighted by their distance to cluster centers. However, if there are fewer than k DISTINCT points in the data set, this approach will fail.
> Further, the recent checkin to work around this problem results in selection of the same point repeatedly as a cluster center.
> The fix is to allow fewer than k cluster centers to be selected. This requires several changes to the code, as the number of cluster centers is woven into the implementation.
> I have a version of the code that addresses this problem, AND generalizes the distance metric. However, I see that there are literally hundreds of outstanding pull requests. If someone will commit to working with me to sponsor the pull request, I will create it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org