You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Manoj Kumar (JIRA)" <ji...@apache.org> on 2015/03/10 18:33:38 UTC
[jira] [Comment Edited] (SPARK-5987) Model import/export for
GaussianMixtureModel
[ https://issues.apache.org/jira/browse/SPARK-5987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14355274#comment-14355274 ]
Manoj Kumar edited comment on SPARK-5987 at 3/10/15 5:32 PM:
-------------------------------------------------------------
[~josephkb] I am stuck at this point. Would be great if you are able to help. I'm not sure if it is a bug or not. Basically I am not able to convert a distributed Array of MultivariateGaussian into a DataFrame, I am getting a MatchError.
{code}
import org.apache.spark.mllib.clustering.GaussianMixture
import org.apache.spark.mllib.linalg.Vectors
import org.apache.spark.mllib.stat.distribution.MultivariateGaussian
// Load and parse the data
val data = sc.textFile("data/mllib/gmm_data.txt")
val parsedData = data.map(s => Vectors.dense(s.trim.split(' ').map(_.toDouble))).cache()
// Cluster the data into two classes using GaussianMixture
val gmm = new GaussianMixture().setK(2).run(parsedData)
case class Data(weights: Array[Double], gaussians: Array[MultivariateGaussian])
val sqlContext = new SQLContext(sc)
import sqlContext.implicits._
val data = Data(gmm.weights, gmm.gaussians)
sc.parallelize(Seq(data), 1).toDF()
{code}
This raises a MatchError.
was (Author: mechcoder):
[~josephkb] I am stuck at this point. Would be great if you are able to help. I'm not sure if it is a bug or not. Basically I am not able to convert a distributed Array of MultivariateGaussian into a DataFrame, I am getting a MatchError.
{code}
import org.apache.spark.mllib.clustering.GaussianMixture
import org.apache.spark.mllib.linalg.Vectors
import org.apache.spark.mllib.stat.distribution.MultivariateGaussian
// Load and parse the data
val data = sc.textFile("data/mllib/gmm_data.txt")
val parsedData = data.map(s => Vectors.dense(s.trim.split(' ').map(_.toDouble))).cache()
// Cluster the data into two classes using GaussianMixture
val gmm = new GaussianMixture().setK(2).run(parsedData)
case class Data(weights: Array[Double], gaussians: Array[MultivariateGaussian])
val sqlContext = new SQLContext(sc)
import sqlContext.implicits._
val data = Data(gmm.weights, gmm.gaussians)
sc.parallelize(Seq(data), 1).toDF()
{code}
> Model import/export for GaussianMixtureModel
> --------------------------------------------
>
> Key: SPARK-5987
> URL: https://issues.apache.org/jira/browse/SPARK-5987
> Project: Spark
> Issue Type: Sub-task
> Components: MLlib
> Affects Versions: 1.3.0
> Reporter: Joseph K. Bradley
> Assignee: Manoj Kumar
>
> Support save/load for GaussianMixtureModel
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org