You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Xinrong Meng (Jira)" <ji...@apache.org> on 2022/10/31 20:53:00 UTC

[jira] [Commented] (SPARK-31776) Literal lit() supports lists and numpy arrays

    [ https://issues.apache.org/jira/browse/SPARK-31776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17626844#comment-17626844 ] 

Xinrong Meng commented on SPARK-31776:
--------------------------------------

`lit` supports Python list and NumPy arrays in https://issues.apache.org/jira/browse/SPARK-39405 in Spark 3.4.0.

> Literal lit() supports lists and numpy arrays
> ---------------------------------------------
>
>                 Key: SPARK-31776
>                 URL: https://issues.apache.org/jira/browse/SPARK-31776
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 3.1.0
>            Reporter: Xiangrui Meng
>            Priority: Major
>
> In ML workload, it is common to replace null feature vectors with some default value. However, lit() does not support Python list and numpy arrays at input. Users cannot simply use fillna() to get the job done.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org