You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Ilya Ganelin (JIRA)" <ji...@apache.org> on 2014/12/01 16:49:13 UTC
[jira] [Comment Edited] (SPARK-4101) [MLLIB] Improve API in
Word2Vec model
[ https://issues.apache.org/jira/browse/SPARK-4101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14229916#comment-14229916 ]
Ilya Ganelin edited comment on SPARK-4101 at 12/1/14 3:48 PM:
--------------------------------------------------------------
Hi Peter - did you have an algorithm in mind for doing the word analogy? I saw a brief mention here:
https://code.google.com/p/word2vec/
"
It was recently shown that the word vectors capture many linguistic regularities, for example vector operations vector('Paris') - vector('France') + vector('Italy') results in a vector that is very close to vector('Rome'), and vector('king') - vector('man') + vector('woman') is close to vector('queen') [3, 1]. You can try out a simple demo by running demo-analogy.sh.
"
Is that what you had in mind or were you thinking of another approach?
was (Author: ilganeli):
Hu Peter - did you have an algorithm in mind for doing the word analogy? I saw a brief mention here:
https://code.google.com/p/word2vec/
"
It was recently shown that the word vectors capture many linguistic regularities, for example vector operations vector('Paris') - vector('France') + vector('Italy') results in a vector that is very close to vector('Rome'), and vector('king') - vector('man') + vector('woman') is close to vector('queen') [3, 1]. You can try out a simple demo by running demo-analogy.sh.
"
Is that what you had in mind or were you thinking of another approach?
> [MLLIB] Improve API in Word2Vec model
> -------------------------------------
>
> Key: SPARK-4101
> URL: https://issues.apache.org/jira/browse/SPARK-4101
> Project: Spark
> Issue Type: Improvement
> Components: MLlib
> Affects Versions: 1.1.0
> Reporter: Peter Rudenko
> Priority: Minor
>
> 1) Would be nice to be able to retrieve underlying model map, to be able to work with it after (make an RDD, persist/load, online train, etc.). (Done by [SPARK-4582|https://issues.apache.org/jira/browse/SPARK-4582] )
> 2) Be able to extend Word2VecModel to add custom functionality (like add analogyWords(w1: String, w2: String, target: String, num: Int) method, which returns n words that relates to target as w1 to w2).
> 3) Make cosineSimilarity method public to be able to reuse it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org