You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Weichen Xu (Jira)" <ji...@apache.org> on 2020/02/13 16:03:00 UTC
[jira] [Resolved] (SPARK-30762) Add dtype="float32" support to
vector_to_array UDF
[ https://issues.apache.org/jira/browse/SPARK-30762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Weichen Xu resolved SPARK-30762.
--------------------------------
Target Version/s: 3.0.0, 3.1.0
Resolution: Done
Resolved by https://github.com/apache/spark/pull/27522
> Add dtype="float32" support to vector_to_array UDF
> --------------------------------------------------
>
> Key: SPARK-30762
> URL: https://issues.apache.org/jira/browse/SPARK-30762
> Project: Spark
> Issue Type: Story
> Components: MLlib, PySpark
> Affects Versions: 3.0.0
> Reporter: Liang Zhang
> Assignee: Liang Zhang
> Priority: Major
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> Previous PR: [https://github.com/apache/spark/blob/master/python/pyspark/ml/functions.py]
> In the previous PR, we introduced a UDF to convert a column of MLlib Vecters to a column of lists in python (Seq in scala). Currently, all the floating numbers in a vector is converted to Double in scala. In this issue, we will add a parameter in the python function {{vector_to_array(col)}} that allows converting to Float (32bits) in scala, which would be mapped to a numpy array of dtype=float32.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org