[jira] [Assigned] (SPARK-40308) str_to_map should accept non-foldable delimiter arguments

["Apache Spark (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-40308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Apache Spark reassigned SPARK-40308:
------------------------------------

    Assignee:     (was: Apache Spark)

> str_to_map should accept non-foldable delimiter arguments
> ---------------------------------------------------------
>
>                 Key: SPARK-40308
>                 URL: https://issues.apache.org/jira/browse/SPARK-40308
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 3.4.0
>            Reporter: Bruce Robbins
>            Priority: Minor
>
> Currently, str_to_map requires the delimiter arguments to be foldable expressions. For example, the following doesn't work in Spark SQL:
> {noformat}
> drop table if exists maptbl;
> create table maptbl as select ',' as del1, ':' as del2, 'a:1,b:2,c:3' as str;
> insert into table maptbl select '%' as del1, '-' as del2, 'a-1%b-2%c-3' as str;
> select str, str_to_map(str, del1, del2) from maptbl;
> {noformat}
> You get the following error:
> {noformat}
> str_to_map's delimiters must be foldable.; line 1 pos 12;
> {noformat}
> However, the above example SQL statements do work in Hive 2.3.9. There, you get:
> {noformat}
> +--------------+----------------------------+
> |     str      |            _c1             |
> +--------------+----------------------------+
> | a:1,b:2,c:3  | {"a":"1","b":"2","c":"3"}  |
> | a-1%b-2%c-3  | {"a":"1","b":"2","c":"3"}  |
> +--------------+----------------------------+
> 2 rows selected (0.13 seconds)
> {noformat}
> It's unlikely that an input table would have the needed delimiters in columns. The use-case is more likely to be something like this, where the delimiters are determined based on some other value:
> {noformat}
> select
>   str,
>   str_to_map(str, ',', if(region = 0, ':', '#')) as m
> from
>   maptbl2;
> {noformat}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org