You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by jamborta <ja...@gmail.com> on 2016/04/07 00:59:22 UTC
Re: LabeledPoint with features in matrix form (word2vec matrix)
you probably better off defining your own data structure. labelled point can
store a label, vector. but in your case is more like a label, vector,
vector. i'd probably use tuples with breeze sparse arrays:
RDD[(label:Int, vector1:SparseArray[Double], vector2:SparseArray[Double])]
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/LabeledPoint-with-features-in-matrix-form-word2vec-matrix-tp26629p26690.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
Re: LabeledPoint with features in matrix form (word2vec matrix)
Posted by jamborta <ja...@gmail.com>.
depends, if you'd like to multiply matrices for each row in the data, then
you could use a breeze matrix, and do that locally on the nodes in a map or
similar.
if you'd like to multiply them across the rows, eg. a row in your data is a
row in the matrix, then you could use a distributed matrix like
indexedRowMatrix
(http://spark.apache.org/docs/latest/mllib-data-types.html#indexedrowmatrix).
--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/LabeledPoint-with-features-in-matrix-form-word2vec-matrix-tp26629p26696.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org