You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Igor (JIRA)" <ji...@apache.org> on 2015/12/08 22:59:11 UTC
[jira] [Comment Edited] (SPARK-6523) Error when get attribute of
StandardScalerModel, When use python api
[ https://issues.apache.org/jira/browse/SPARK-6523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15047543#comment-15047543 ]
Igor edited comment on SPARK-6523 at 12/8/15 9:58 PM:
------------------------------------------------------
SPARK-6643 did not address the issue. There is still no way to get means or variances out of StandardScalerModel in PySpark.
was (Author: igorsf):
SPARK-6643 did not address the issue. There is still no way to means or variances out of StandardScalerModel in PySpark.
> Error when get attribute of StandardScalerModel, When use python api
> --------------------------------------------------------------------
>
> Key: SPARK-6523
> URL: https://issues.apache.org/jira/browse/SPARK-6523
> Project: Spark
> Issue Type: Bug
> Components: Examples, MLlib, PySpark
> Affects Versions: 1.3.0
> Reporter: lee.xiaobo.2006
>
> test code
> =======================================
> from pyspark.mllib.util import MLUtils
> from pyspark.mllib.linalg import Vectors
> from pyspark.mllib.feature import StandardScaler
> conf = SparkConf().setAppName('Test')
> sc = SparkContext(conf=conf)
> data = MLUtils.loadLibSVMFile(sc, "data/mllib/sample_libsvm_data.txt")
> label = data.map(lambda x: x.label)
> features = data.map(lambda x: x.features)
> scaler1 = StandardScaler().fit(features)
> print scaler1.std # error
> sc.stop()
> -----------------------------------------------
> error:
> Traceback (most recent call last):
> File "/data1/s/apps/spark-app/app/test_ssm.py", line 22, in <module>
> print scaler1.std
> AttributeError: 'StandardScalerModel' object has no attribute 'std'
> 15/03/25 12:17:28 INFO Utils: path = /data1/s/apps/spark-1.4.0-SNAPSHOT/data/spark-eb1ed7c0-a5ce-4748-a817-3cb0687ee282/blockmgr-5398b477-127d-4259-a71b-608a324e1cd3, already present as root for deletion.
> =================================
> Another question, how to serialize or save the scaler model ?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org