[jira] [Created] (SEDONA-221) Outer join throws NPE for null geometries

["Martin Andersson (Jira)" <ji...@apache.org>]

Martin Andersson created SEDONA-221:
---------------------------------------

             Summary: Outer join throws NPE for null geometries
                 Key: SEDONA-221
                 URL: https://issues.apache.org/jira/browse/SEDONA-221
             Project: Apache Sedona
          Issue Type: Bug
            Reporter: Martin Andersson


The following query throws a NullPointerException.
{code}
select /*+ BROADCAST(t2) */ * from t1 left join t2 on st_intersects(t1.geom, t2.geom)
{code}
{code}
java.lang.NullPointerException
	at org.locationtech.jts.io.WKBReader.read(WKBReader.java:159)
	at org.apache.sedona.sql.utils.GeometrySerializer$.deserialize(GeometrySerializer.scala:50)
	at org.apache.spark.sql.sedona_sql.strategy.join.TraitJoinQueryBase.$anonfun$toSpatialRDD$1(TraitJoinQueryBase.scala:45)
	at scala.collection.Iterator$$anon$10.next(Iterator.scala:461)
	at scala.collection.Iterator$$anon$11.nextCur(Iterator.scala:486)
	at scala.collection.Iterator$$anon$11.hasNext(Iterator.scala:492)
	at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460)
	at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.processNext(Unknown Source)
{code}

The failure happens when the streaming side is mapped to a SpatialRDD. The NPE doesn't happen for inner join with null geometries. I suspect Spark is pushing a not null predicate since rows with null geometries would be excluded in an inner join anyway.

Looking at the code I suspect there are more errors in the new broadcast join types. InternalRow is encoded in the user data field in the geometry. That doesn't work if the geometry is null. For a left join the InternalRow on the left side has to be emitted even if the geometry is null. Instead of using a SpatialRDD it might be better to map the RDD[InternalRow] to a RDD[Pair[Geometry, InternalRow]] where Geometry might be null.




--
This message was sent by Atlassian Jira
(v8.20.10#820010)