[I] GeoPandas: to_parquet does not write the CRS assigned on a GeoDataFrame built from a Spark DataFrame [sedona]

["jiayuasu (via GitHub)" <gi...@apache.org>]

jiayuasu opened a new issue, #3375:
URL: https://github.com/apache/sedona/issues/3375

   ### Expected behavior
   
   A `GeoDataFrame` built from a Spark DataFrame with an explicit CRS should write that CRS into the GeoParquet `geo` metadata, the same way GeoPandas does:
   
   `GeoDataFrame(spark_df, geometry="geometry", crs="EPSG:4326").to_parquet(path)` followed by `geopandas.read_parquet(path).crs` should give EPSG:4326.
   
   ### Actual behavior
   
   The frame reports the CRS, but the written files carry `"crs": null`, so `geopandas.read_parquet(path).crs` is `None`.
   
   `_to_file` in `python/sedona/spark/geopandas/io.py` writes `df._internal.spark_frame` with the `geoparquet` format and no `geoparquet.crs` option, so the writer derives the CRS from the geometry SRIDs. The CRS assigned on the frame is kept in the geometry column metadata and does not reach the SRIDs of the frame that gets written. `_to_file` also pops a `crs` keyword argument without using it.
   
   Readers that require a CRS reject the output. For example, reprojecting the file after reading it with GeoPandas fails with `Must pass either crs or epsg.`
   
   ### Steps to reproduce
   
   ```python
   import geopandas
   from sedona.spark import SedonaContext
   from sedona.spark.geopandas import GeoDataFrame
   
   sedona = SedonaContext.create(SedonaContext.builder().master("local[*]").getOrCreate())
   spark_df = sedona.sql(
       "SELECT id, ST_Point(CAST(id AS DOUBLE), 1.0) AS geometry FROM range(4)"
   )
   gdf = GeoDataFrame(spark_df, geometry="geometry", crs="EPSG:4326")
   
   gdf.to_parquet("/tmp/points.parquet")
   print(geopandas.read_parquet("/tmp/points.parquet").crs)  # None, expected EPSG:4326
   ```
   
   ### Sedona version
   
   master (2.0.0-SNAPSHOT, 45617c138d)
   
   ### Apache Spark version
   
   4.1.3
   
   ### API type
   
   Python
   
   ### Scala version
   
   2.13
   
   ### JRE version
   
   17
   
   ### Python version
   
   3.11
   
   ### Environment
   
   Standalone (local[*])
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] GeoPandas: to_parquet does not write the CRS assigned on a GeoDataFrame built from a Spark DataFrame [sedona]

["jiayuasu (via GitHub)" <gi...@apache.org>]

jiayuasu closed issue #3375: GeoPandas: to_parquet does not write the CRS assigned on a GeoDataFrame built from a Spark DataFrame
URL: https://github.com/apache/sedona/issues/3375


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org