[jira] [Commented] (SPARK-31776) Literal lit() supports lists and numpy arrays

["Zirui Xu (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-31776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17177281#comment-17177281 ] 

Zirui Xu commented on SPARK-31776:
----------------------------------

Hi [~mengxr], is this ticket suggesting adding support for Python list and numpy arrays in both lit() and fillna()? I understand that fillna() does not support Array type so adding Python list support to fillna() might involve adding Array support from the SQL side first.

> Literal lit() supports lists and numpy arrays
> ---------------------------------------------
>
>                 Key: SPARK-31776
>                 URL: https://issues.apache.org/jira/browse/SPARK-31776
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 3.1.0
>            Reporter: Xiangrui Meng
>            Priority: Major
>
> In ML workload, it is common to replace null feature vectors with some default value. However, lit() does not support Python list and numpy arrays at input. Users cannot simply use fillna() to get the job done.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org