[jira] [Comment Edited] (SPARK-28889) Allow UDTs to define custom casting behavior

["Zachary S Ennenga (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-28889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16919822#comment-16919822 ] 

Zachary S Ennenga edited comment on SPARK-28889 at 8/30/19 6:43 PM:
--------------------------------------------------------------------

While I understand if the spark team is not particularly interested in solving this problem themselves at this time, I'm more concerned with understanding if this is in line with the eventual solution to UDTs and datasets. If it is, I'm about halfway through the PR as is, and I'm happy to complete it. If it's not, I'm curious what the plan is, and if it's represented in Jira, I'd love to know what tickets so I can follow along.


was (Author: zennenga):
While I understand if the spark team is not particularly interested in solving this problem themselves at this time, I'm more concerned with understanding if this is in line with the eventual solution to UDTs and datasets. If it is, I'm about halfway through the PR as is, and I'm happy to complete it.

> Allow UDTs to define custom casting behavior
> --------------------------------------------
>
>                 Key: SPARK-28889
>                 URL: https://issues.apache.org/jira/browse/SPARK-28889
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>    Affects Versions: 2.4.3
>            Reporter: Zachary S Ennenga
>            Priority: Minor
>
> Looking at `org.apache.spark.sql.catalyst.expressions.Cast`, UDTs do not support any sort of casting except for identity casts, IE:
> {code:java}
> case (udt1: UserDefinedType[_], udt2: UserDefinedType[_]) if udt1.userClass == udt2.userClass =>
>  true
> {code}
> I propose we add an additional piece of functionality here to allow UDTs to define their own canCast and cast functions to allow users to define their own cast mechanisms.
> An example of how this might look:
> {code:java}
> case (fromType, toType: UserDefinedType[_]) =>
>  toType.canCast(fromType) // Returns boolean
> {code}
> {code:java}
> case (fromType, toType: UserDefinedType[_]) =>
>  toType.cast(fromType) // Returns Casting function
> {code}
> The UDT base class would contain a default implementation that replicates current behavior (IE no casting).



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org