You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Soheil Pourbafrani <so...@gmail.com> on 2018/11/15 18:03:55 UTC

Using cosinSimilarity method for getting pairwise documents similarity

Hi,

I got the TF-IDF vector for the documents and store it in an RDD and
convert into RowMatrix type:

val mat = new RowMatrix(tweets_tfidf)

Every element of RDD is a sparse Vector related to a document.
The problem is the *cosinSimilarity *compute the similarity between
columns. Is there any way to make it compute the similarity between rows?
Or I should transpose the matrix?