You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Jean-Philippe Quemener (JIRA)" <ji...@apache.org> on 2014/11/19 17:03:33 UTC
[jira] [Created] (SPARK-4494) IDFModel.transform() add support for
single vectors
Jean-Philippe Quemener created SPARK-4494:
---------------------------------------------
Summary: IDFModel.transform() add support for single vectors
Key: SPARK-4494
URL: https://issues.apache.org/jira/browse/SPARK-4494
Project: Spark
Issue Type: New Feature
Components: MLlib
Reporter: Jean-Philippe Quemener
For now when using the tfidf implementation in mllib you have no other possibility to map your data back onto i.e. labels or ids than use a hackish way with ziping: {quote} 1. Persist input RDD. 2. Transform it to just vectors and apply IDFModel 3. zip with original RDD 4. transform label and new vector to LabeledPoint{quote}
Source:[http://stackoverflow.com/questions/26897908/spark-mllib-tfidf-implementation-for-logisticregression]
I think as in production alot of users want to map their data back to some identifier, it would be a good imporvement to allow using single vectors on IDFModel.transform()
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org