[I] GeoPandas: leading-null GeoSeries input loses Z dimensions [sedona]

["jiayuasu (via GitHub)" <gi...@apache.org>]

jiayuasu opened a new issue, #3369:
URL: https://github.com/apache/sedona/issues/3369

   ### Expected behavior
   
   A leading null should not change the dimensions of the other geometries in a GeoSeries. Empty XYZ geometries and points whose Z is NaN should retain their dimensional metadata.
   
   ### Actual behavior
   
   The constructor path added in #3349 for #3333 converts local geometries through WKB and ST_GeomFromWKB. Adding a leading null causes these geometries to become 2D:
   
   | Input geometry | GeoSeries([geometry, None]) | GeoSeries([None, geometry]) |
   |---|---|---|
   | POINT Z EMPTY | 3D | 2D |
   | LINESTRING Z EMPTY | 3D | 2D |
   | POLYGON Z EMPTY | 3D | 2D |
   | POINT Z (1 2 NaN) | 3D | 2D |
   | POINT Z (1 2 3) | 3D | 3D |
   | POINT EMPTY | 2D | 2D |
   
   ### Steps to reproduce
   
   With a SedonaContext initialized using the current-master JVM jar:
   
   ```python
   import shapely
   from sedona.spark.geopandas import GeoSeries
   
   g = shapely.from_wkt("POINT Z EMPTY")
   ordinary = GeoSeries([g, None]).to_geopandas().iloc[0]
   leading_null = GeoSeries([None, g]).to_geopandas().iloc[1]
   
   print(ordinary.wkt, shapely.get_coordinate_dimension(ordinary))
   # POINT Z EMPTY 3
   print(leading_null.wkt, shapely.get_coordinate_dimension(leading_null))
   # POINT EMPTY 2
   ```
   
   ### Sedona version
   
   2.0.0-SNAPSHOT, master commit 15e415af1df7620a25c2dda43532ef1211c9e9f4.
   
   The Spark 4.1 JVM jar was freshly built from this commit with `mvn -B -pl spark-shaded -am -Dspark=4.1 -Dscala=2.13 -DskipTests package`. Both Python serializer modes reproduced the same results: the pure-Python fallback and a native extension freshly compiled from the same source. The six inputs above were compared through both constructor paths in each mode.
   
   ### Environment
   
   - API: Python / GeoPandas compatibility API
   - Spark: 4.1.1; Scala: 2.13.17
   - Java: 17.0.13; Python: 3.11.13
   - Shapely: 2.1.2; GEOS: 3.13.1
   - GeoPandas: 1.1.1; pandas: 2.2.3
   - Local Spark, macOS arm64
   
   ### Root cause and suggested fix
   
   Direct JVM checks show that the WKB parser retains the 3D coordinate sequence. [GeometrySerializer](https://github.com/apache/sedona/blob/15e415af1df7620a25c2dda43532ef1211c9e9f4/common/src/main/java/org/apache/sedona/common/geometrySerde/GeometrySerializer.java#L477-L495) then chooses XY when the sequence has no coordinates or all Z values are NaN. JVM result rows are already 2D before Python decoding.
   
   Replace the [leading-null constructor's WKB conversion](https://github.com/apache/sedona/blob/15e415af1df7620a25c2dda43532ef1211c9e9f4/python/sedona/spark/geopandas/geoseries.py#L740-L754) with construction using an explicit Spark GeometryType schema and Sedona's Python geometry serializer. Preserve the pandas index, row order, name, CRS, embedded SRIDs, and normalization of all pandas missing-value variants. Add dimensional regression cases alongside the existing constructor tests and verify Spark 3.5 and 4.1.
   
   Changing the JVM serializer to treat every three-dimensional JTS sequence as XYZ would also promote ordinary XY data, because JTS can represent XY with a NaN Z slot.
   
   The reproduction measures returned geometry dimensions directly. It does not rely on ST_NDims, ST_CoordDim, or ST_Zmflag, whose empty-geometry results were intentionally changed in #3360.
   
   ### Existing issues
   
   - [x] I searched the existing issues and did not find a duplicate.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] GeoPandas: leading-null GeoSeries input loses Z dimensions [sedona]

["jiayuasu (via GitHub)" <gi...@apache.org>]

jiayuasu closed issue #3369: GeoPandas: leading-null GeoSeries input loses Z dimensions
URL: https://github.com/apache/sedona/issues/3369


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org