[jira] [Created] (SPARK-37560) equality comparison for map type field

["omar (Jira)" <ji...@apache.org>]

omar created SPARK-37560:
----------------------------

             Summary: equality comparison for map type field
                 Key: SPARK-37560
                 URL: https://issues.apache.org/jira/browse/SPARK-37560
             Project: Spark
          Issue Type: Improvement
          Components: Spark Core
    Affects Versions: 2.4.0
            Reporter: omar


hi

where I try to execute this kind of qury 

 
{code:java}
SELECT * FROM myTable where myMapField is null {code}
I get this king of error 

 
{code:java}
org.apache.spark.sql.AnalysisException: Cannot have map type columns in DataFrame which calls set operations(intersect, except, etc.), but the type of column is map ... {code}
According to this link [https://github.com/apache/spark/blob/branch-2.4/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala#L406] 

 

would  it be possible to assign a default comparaison function for Map type  ?

i'm working on spark 2.4, and it seams that this issue happens with version 3.2 too (same comment in 3.2 branch)

 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org