You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@sedona.apache.org by ji...@apache.org on 2026/09/23 05:13:25 UTC
(sedona) branch master updated: [GH-3383] Preserve GeoParquet XYZ layouts and document M conversion (#3390)
This is an automated email from the ASF dual-hosted git repository.
jiayuasu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/sedona.git
The following commit(s) were added to refs/heads/master by this push:
new 986558a6dc2 [GH-3383] Preserve GeoParquet XYZ layouts and document M conversion (#3390)
986558a6dc2 is described below
commit 986558a6dc254c242380938a9edcbd1fa031effd
Author: Jia Yu <ji...@apache.org>
AuthorDate: Tue Sep 22 22:13:18 2026 -0700
[GH-3383] Preserve GeoParquet XYZ layouts and document M conversion (#3390)
---
docs/tutorial/files/geoparquet-sedona-spark.md | 6 +-
docs/tutorial/files/geoparquet-sedona-spark.zh.md | 6 +
.../geoparquet/GeoParquetWriteSupport.scala | 48 +--
.../sedona/sql/GeoParquetDimensionsSuite.scala | 322 +++++++++++++++++++++
.../geoparquet/GeoParquetWriteSupportTest.scala | 45 +++
5 files changed, 406 insertions(+), 21 deletions(-)
diff --git a/docs/tutorial/files/geoparquet-sedona-spark.md b/docs/tutorial/files/geoparquet-sedona-spark.md
index eeae899c038..503f07512d6 100644
--- a/docs/tutorial/files/geoparquet-sedona-spark.md
+++ b/docs/tutorial/files/geoparquet-sedona-spark.md
@@ -67,6 +67,10 @@ somewhere/
Sedona writes many Parquet files in parallel because that's much faster than writing a single file.
+GeoParquet writes preserve the declared XY or XYZ coordinate layout of empty `Point`, `LineString`, and `Polygon` geometries. XYZ geometries also retain their Z dimension when Z ordinates are `NaN`.
+
+[GeoParquet 1.1](https://geoparquet.org/releases/v1.1.0/#wkb) does not support M coordinates. Sedona discards M when writing GeoParquet: XYM becomes XY, and XYZM becomes XYZ. If you need the M values, store them separately before writing.
+
## Read GeoParquet files into Sedona DataFrame with Spark
Let's read the GeoParquet files into a Sedona DataFrame:
@@ -88,7 +92,7 @@ Here are the results:
+---+---------------------+
```
-Since Sedona 2.0.0, reads preserve declared Z/M dimensions even for empty points, lines, and polygons or NaN ordinates. If collecting mixed layouts raises a heterogeneous-layout error, see [ST_Collect](../../api/sql/Geometry-Editors/ST_Collect.md).
+Since Sedona 2.0.0, reads preserve declared Z/M dimensions even for empty points, lines, and polygons or NaN ordinates. This includes measured WKB inputs, although M coordinates are outside the GeoParquet 1.1 specification. If collecting mixed layouts raises a heterogeneous-layout error, see [ST_Collect](../../api/sql/Geometry-Editors/ST_Collect.md).
Here's how Sedona executes this query under the hood:
diff --git a/docs/tutorial/files/geoparquet-sedona-spark.zh.md b/docs/tutorial/files/geoparquet-sedona-spark.zh.md
index 4d3c1ccf40e..394ae53418f 100644
--- a/docs/tutorial/files/geoparquet-sedona-spark.zh.md
+++ b/docs/tutorial/files/geoparquet-sedona-spark.zh.md
@@ -67,6 +67,10 @@ somewhere/
Sedona 会并行写出多个 Parquet 文件，比写单一文件快得多。
+写出 GeoParquet 时，Sedona 会保留空 `Point`、`LineString` 和 `Polygon` 几何对象声明的 XY 或 XYZ 坐标布局。即使 Z 坐标值为 `NaN`，XYZ 几何对象也会保留 Z 维度。
+
+[GeoParquet 1.1](https://geoparquet.org/releases/v1.1.0/#wkb) 不支持 M 坐标。Sedona 在写出 GeoParquet 时会丢弃 M：XYM 转换为 XY，XYZM 转换为 XYZ。如果需要保留 M 值，请在写出前将其单独存储。
+
## 使用 Spark 将 GeoParquet 读入 Sedona DataFrame
将 GeoParquet 文件读入 Sedona DataFrame：
@@ -88,6 +92,8 @@ df.show(truncate=False)
+---+---------------------+
```
+自 Sedona 2.0.0 起，读取时会保留声明的 Z/M 维度，即使点、线或多边形为空，或坐标值为 `NaN`。这也适用于包含 M 坐标的 WKB 输入，但此类输入不符合 GeoParquet 1.1 规范。如果收集不同坐标布局的几何对象时出现布局不一致的错误，请参阅 [ST_Collect](../../api/sql/Geometry-Editors/ST_Collect.md)。
+
Sedona 在底层执行该查询的过程大致如下：
1. 从 GeoParquet 文件尾部读取 schema，因此无需 schema 推断。
diff --git a/spark/common/src/main/scala/org/apache/spark/sql/execution/datasources/geoparquet/GeoParquetWriteSupport.scala b/spark/common/src/main/scala/org/apache/spark/sql/execution/datasources/geoparquet/GeoParquetWriteSupport.scala
index c63437d22f8..517b81b19ee 100644
--- a/spark/common/src/main/scala/org/apache/spark/sql/execution/datasources/geoparquet/GeoParquetWriteSupport.scala
+++ b/spark/common/src/main/scala/org/apache/spark/sql/execution/datasources/geoparquet/GeoParquetWriteSupport.scala
@@ -24,7 +24,6 @@ import org.apache.parquet.hadoop.api.WriteSupport.FinalizedWriteContext
import org.apache.parquet.hadoop.api.WriteSupport.WriteContext
import org.apache.parquet.io.api.Binary
import org.apache.parquet.io.api.RecordConsumer
-import org.apache.sedona.common.utils.GeomUtils
import org.apache.spark.SPARK_VERSION_SHORT
import org.apache.spark.internal.Logging
import org.apache.spark.sql.catalyst.InternalRow
@@ -38,6 +37,8 @@ import org.apache.spark.sql.types._
import org.json4s.JValue
import org.json4s.jackson.JsonMethods.parse
import org.locationtech.jts.geom.Geometry
+import org.locationtech.jts.io.ByteOrderValues
+import org.datasyslab.jts.io.WKBWriter
import java.nio.ByteBuffer
import java.nio.ByteOrder
@@ -245,12 +246,9 @@ class GeoParquetWriteSupport extends WriteSupport[InternalRow] with Logging {
val columns = geometryColumnInfoMap.map { case (ordinal, columnInfo) =>
val columnName = schema.fields(ordinal).name
val geometryTypes = columnInfo.seenGeometryTypes.toSeq
- // Omit bbox from column metadata when no geometries were observed (e.g. an empty
- // Spark partition produces a zero-row file). Per the GeoParquet 1.1 spec, bbox is
- // optional and represents the extent of the geometries in the file; emitting
- // [0, 0, 0, 0] for an empty file falsely advertises data at Null Island and breaks
- // bbox-based file pruning in downstream readers.
- val bbox = if (geometryTypes.nonEmpty) {
+ // Empty geometries still contribute geometry_types but have no spatial extent.
+ // Omit bbox when the file contains only empty/null geometries or no rows.
+ val bbox = if (!columnInfo.bbox.isEmpty) {
Some(
Seq(
columnInfo.bbox.minX,
@@ -483,13 +481,23 @@ class GeoParquetWriteSupport extends WriteSupport[InternalRow] with Logging {
geometryColumnInfoMap.getOrElseUpdate(ordinal, new GeometryColumnInfo())
case None => null
}
+ val byteOrder = if (ByteOrder.nativeOrder() == ByteOrder.BIG_ENDIAN) {
+ ByteOrderValues.BIG_ENDIAN
+ } else ByteOrderValues.LITTLE_ENDIAN
+ // Preserve declared Z even for empty geometries, but keep GeoParquet output limited
+ // to XY/XYZ: the configured XYZ ordinates continue to exclude M.
+ val wkbWriter = new WKBWriter(3, byteOrder, false)
+ wkbWriter.setPreserveCoordinateDimensions(true)
(row: SpecializedGetters, ordinal: Int) => {
val serializedGeometry = row.getBinary(ordinal)
val geom = GeometryUDT.deserialize(serializedGeometry)
- val wkbWriter = GeomUtils.createWKBWriter(GeomUtils.getDimension(geom))
- recordConsumer.addBinary(Binary.fromReusedByteArray(wkbWriter.write(geom)))
+ val wkb = wkbWriter.write(geom)
+ recordConsumer.addBinary(Binary.fromReusedByteArray(wkb))
if (geometryColumnInfo != null) {
- geometryColumnInfo.update(geom)
+ // The writer resolves each geometry's actual layout, including collection members.
+ // Derive metadata from its EWKB type header rather than the first coordinate.
+ val typeCode = ByteBuffer.wrap(wkb, 1, 4).order(ByteOrder.nativeOrder()).getInt
+ geometryColumnInfo.update(geom, (typeCode & 0x80000000) != 0)
}
}
@@ -754,13 +762,9 @@ object GeoParquetWriteSupport {
else Some(_srid)
}
- def update(geom: Geometry): Unit = {
+ def update(geom: Geometry, hasZ: Boolean): Unit = {
bbox.update(geom)
- // In case of 3D geometries, a " Z" suffix gets added (e.g. ["Point Z"]).
- val hasZ = {
- val coordinate = geom.getCoordinate
- if (coordinate != null) !coordinate.getZ.isNaN else false
- }
+ // Match the dimensionality encoded in WKB, including typed empty geometries.
val geometryType = if (!hasZ) geom.getGeometryType else geom.getGeometryType + " Z"
seenGeometryTypes.add(geometryType)
@@ -781,12 +785,16 @@ object GeoParquetWriteSupport {
var minY: Double = Double.PositiveInfinity,
var maxX: Double = Double.NegativeInfinity,
var maxY: Double = Double.NegativeInfinity) {
+ def isEmpty: Boolean = minX > maxX || minY > maxY
+
def update(geom: Geometry): Unit = {
val env = geom.getEnvelopeInternal
- minX = math.min(minX, env.getMinX)
- minY = math.min(minY, env.getMinY)
- maxX = math.max(maxX, env.getMaxX)
- maxY = math.max(maxY, env.getMaxY)
+ if (!env.isNull) {
+ minX = math.min(minX, env.getMinX)
+ minY = math.min(minY, env.getMinY)
+ maxX = math.max(maxX, env.getMaxX)
+ maxY = math.max(maxY, env.getMaxY)
+ }
}
}
diff --git a/spark/common/src/test/scala/org/apache/sedona/sql/GeoParquetDimensionsSuite.scala b/spark/common/src/test/scala/org/apache/sedona/sql/GeoParquetDimensionsSuite.scala
new file mode 100644
index 00000000000..917d7079a8b
--- /dev/null
+++ b/spark/common/src/test/scala/org/apache/sedona/sql/GeoParquetDimensionsSuite.scala
@@ -0,0 +1,322 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.sedona.sql
+
+import java.io.File
+import java.nio.{ByteBuffer, ByteOrder}
+import java.nio.file.Files
+import java.util.Collections
+import scala.collection.JavaConverters._
+
+import org.apache.commons.io.FileUtils
+import org.apache.hadoop.conf.Configuration
+import org.apache.hadoop.fs.Path
+import org.apache.parquet.example.data.simple.SimpleGroupFactory
+import org.apache.parquet.hadoop.ParquetFileReader
+import org.apache.parquet.hadoop.util.HadoopInputFile
+import org.apache.parquet.column.Encoding
+import org.apache.parquet.hadoop.example.ExampleParquetWriter
+import org.apache.parquet.io.api.Binary
+import org.apache.parquet.schema.MessageTypeParser
+import org.apache.sedona.common.utils.GeometryEquality
+import org.apache.spark.sql.{DataFrame, Row}
+import org.apache.spark.sql.types.{BinaryType, IntegerType, StructField, StructType}
+import org.json4s.{DefaultFormats, JNothing, JValue}
+import org.json4s.jackson.parseJson
+import org.apache.spark.sql.sedona_sql.UDT.GeometryUDT
+import org.datasyslab.jts.io.WKBReader
+
+/** File-level tests use hand-built ISO WKB, independently of the GeoParquet writer. */
+class GeoParquetDimensionsSuite extends TestBaseScala {
+ private case class Sample(name: String, wkb: Array[Byte])
+
+ private def isoWkb(
+ kind: Int,
+ z: Boolean,
+ coordinates: Seq[Double] = Seq.empty,
+ children: Seq[Array[Byte]] = Seq.empty,
+ order: ByteOrder = ByteOrder.LITTLE_ENDIAN,
+ m: Boolean = false): Array[Byte] = {
+ val dimension = 2 + (if (z) 1 else 0) + (if (m) 1 else 0)
+ val payload =
+ if (kind == 1) coordinates.size * 8
+ else if (kind == 2) 4 + coordinates.size * 8
+ else if (kind >= 4 && kind <= 7) 4 + children.map(_.length).sum
+ else 4
+ val buffer = ByteBuffer.allocate(5 + payload).order(order)
+ buffer.put(if (order == ByteOrder.LITTLE_ENDIAN) 1.toByte else 0.toByte)
+ buffer.putInt(kind + (if (z) 1000 else 0) + (if (m) 2000 else 0))
+ if (kind == 1) coordinates.foreach(buffer.putDouble)
+ else if (kind == 2) {
+ require(coordinates.size % dimension == 0)
+ buffer.putInt(coordinates.size / dimension)
+ coordinates.foreach(buffer.putDouble)
+ } else if (kind >= 4 && kind <= 7) {
+ buffer.putInt(children.size)
+ children.foreach(buffer.put)
+ } else buffer.putInt(0)
+ buffer.array()
+ }
+
+ private def samples: Seq[Sample] = {
+ val primitives = for {
+ order <- Seq(ByteOrder.LITTLE_ENDIAN, ByteOrder.BIG_ENDIAN)
+ z <- Seq(false, true)
+ kind <- Seq(1, 2, 3)
+ } yield Sample(
+ s"empty-$kind-z=$z-$order",
+ isoWkb(
+ kind,
+ z,
+ if (kind == 1) Seq.fill(if (z) 3 else 2)(Double.NaN) else Seq.empty,
+ order = order))
+ val lines = Seq(ByteOrder.LITTLE_ENDIAN, ByteOrder.BIG_ENDIAN).map { order =>
+ Sample(
+ s"later-finite-z-$order",
+ isoWkb(2, true, Seq(1, 2, Double.NaN, 3, 4, 5), order = order))
+ }
+ primitives ++ lines ++ Seq(
+ Sample("null", null),
+ Sample("nonempty-xyz", isoWkb(1, true, Seq(10, 20, 30))),
+ Sample("nan-z", isoWkb(1, true, Seq(10, 20, Double.NaN))),
+ Sample("empty-collection", isoWkb(7, false)),
+ Sample(
+ "empty-multipoint",
+ isoWkb(4, true, children = Seq(isoWkb(1, true, Seq.fill(3)(Double.NaN))))),
+ Sample("empty-multilinestring", isoWkb(5, true, children = Seq(isoWkb(2, true)))),
+ Sample("empty-multipolygon", isoWkb(6, true, children = Seq(isoWkb(3, true)))),
+ Sample(
+ "mixed-empty-children",
+ isoWkb(7, true, children = Seq(isoWkb(2, false), isoWkb(3, true)))),
+ Sample(
+ "nested-empty-children",
+ isoWkb(
+ 7,
+ true,
+ children = Seq(isoWkb(7, true, children = Seq(isoWkb(2, true))), isoWkb(3, false)))))
+ }
+
+ private def withDirectory(f: File => Unit): Unit = {
+ val directory = Files.createTempDirectory("geoparquet-dimensions-").toFile
+ try f(directory)
+ finally FileUtils.deleteDirectory(directory)
+ }
+
+ private def writeFixture(file: File, dictionary: Boolean, values: Seq[Sample]): Unit = {
+ val schema = MessageTypeParser.parseMessageType(
+ "message fixture { required int32 id; optional binary geometry; }")
+ val factory = new SimpleGroupFactory(schema)
+ val metadata = Collections.singletonMap(
+ "geo",
+ """{"version":"1.1.0","primary_column":"geometry","columns":{"geometry":{"encoding":"WKB","geometry_types":[]}}}""")
+ val writer = ExampleParquetWriter
+ .builder(new Path(file.toURI))
+ .withConf(new Configuration())
+ .withType(schema)
+ .withExtraMetaData(metadata)
+ .withDictionaryEncoding(dictionary)
+ .build()
+ try {
+ // Repeated values exercise the dictionary decoder, while IDs retain stable ordering.
+ for (repeat <- 0 until 4; (sample, index) <- values.zipWithIndex) {
+ val row = factory.newGroup().append("id", repeat * values.size + index)
+ if (sample.wkb != null) row.append("geometry", Binary.fromConstantByteArray(sample.wkb))
+ writer.write(row)
+ }
+ } finally writer.close()
+ val reader =
+ ParquetFileReader.open(HadoopInputFile.fromPath(new Path(file.toURI), new Configuration()))
+ try {
+ val columns = reader.getFooter.getBlocks.asScala
+ .flatMap(_.getColumns.asScala)
+ .filter(_.getPath.toDotString == "geometry")
+ assert(columns.nonEmpty)
+ val encodedWithDictionary = columns.exists { column =>
+ column.getEncodings.contains(Encoding.RLE_DICTIONARY) ||
+ column.getEncodings.contains(Encoding.PLAIN_DICTIONARY)
+ }
+ assert(encodedWithDictionary == dictionary)
+ } finally reader.close()
+ }
+
+ private def assertFile(
+ path: String,
+ values: Seq[Sample],
+ repetitions: Int = 1,
+ expectedSrid: Int = 0): Unit = {
+ val df = sparkSession.read.format("geoparquet").load(path)
+ assertGeometries(df, values, "file read", repetitions, expectedSrid)
+ }
+
+ private def assertGeometries(
+ df: DataFrame,
+ values: Seq[Sample],
+ context: String,
+ repetitions: Int = 1,
+ expectedSrid: Int = 0): Unit = {
+ val projected = df.select("id", "geometry")
+ val rows = projected.queryExecution.toRdd.map(_.copy()).collect().sortBy(_.getInt(0))
+ assert(rows.length == values.size * repetitions)
+ rows.zipWithIndex.foreach { case (row, index) =>
+ val sample = values(index % values.size)
+ withClue(s"${sample.name}, $context: ") {
+ if (sample.wkb == null) assert(row.isNullAt(1))
+ else {
+ val expected = WKBReader.forDeclaredDimensions().read(sample.wkb)
+ val actual = GeometryUDT.deserialize(row.getBinary(1))
+ assert(actual.getSRID == expectedSrid, "column metadata SRID was not preserved")
+ assert(
+ GeometryEquality.equalsIdentical(actual, expected),
+ s"declared layout or ordinates changed: $actual")
+ }
+ }
+ }
+ }
+
+ private def hasZ(wkb: Array[Byte]): Boolean = {
+ val order = if (wkb(0) == 0) ByteOrder.BIG_ENDIAN else ByteOrder.LITTLE_ENDIAN
+ val kind = ByteBuffer.wrap(wkb).order(order).getInt(1)
+ (kind & 0x80000000) != 0 || ((kind & 0x0fffffff) / 1000) % 2 == 1
+ }
+
+ private def columnMetadata(file: File): JValue = {
+ val reader =
+ ParquetFileReader.open(HadoopInputFile.fromPath(new Path(file.toURI), new Configuration()))
+ try
+ parseJson(reader.getFooter.getFileMetaData.getKeyValueMetaData.get("geo")) \
+ "columns" \ "geometry"
+ finally reader.close()
+ }
+
+ private def assertWrittenFile(path: String, expected: Seq[Sample], allEmpty: Boolean): Unit = {
+ val rows = sparkSession.read
+ .parquet(path)
+ .select("id", "geometry")
+ .collect()
+ .sortBy(_.getInt(0))
+ assert(rows.length == expected.size)
+ rows.zip(expected).foreach { case (row, sample) =>
+ withClue(s"written ${sample.name}: ") {
+ if (sample.wkb == null) assert(row.isNullAt(1))
+ else {
+ val bytes = row.getAs[Array[Byte]](1)
+ assert(hasZ(bytes) == hasZ(sample.wkb), "top-level WKB Z header changed")
+ assert(
+ GeometryEquality.equalsIdentical(
+ WKBReader.forDeclaredDimensions().read(bytes),
+ WKBReader.forDeclaredDimensions().read(sample.wkb)),
+ "WKB coordinate layout or ordinates changed")
+ }
+ }
+ }
+ val files = new File(path).listFiles().filter(_.getName.endsWith(".parquet"))
+ assert(files.length == 1)
+ val metadata = columnMetadata(files.head)
+ implicit val formats: org.json4s.Formats = DefaultFormats
+ val expectedTypes = expected
+ .filter(_.wkb != null)
+ .map { sample =>
+ WKBReader.forDeclaredDimensions().read(sample.wkb).getGeometryType +
+ (if (hasZ(sample.wkb)) " Z" else "")
+ }
+ .toSet
+ assert((metadata \ "geometry_types").extract[Seq[String]].toSet == expectedTypes)
+ if (allEmpty) assert((metadata \ "bbox") == JNothing)
+ else assert((metadata \ "bbox").extract[Seq[Double]] == Seq(1, 2, 10, 20))
+ }
+
+ private def writeAndRoundTrip(directory: File, allEmpty: Boolean): Unit = {
+ val projectedMeasures = Seq(
+ Sample("xym", isoWkb(1, false, Seq(10, 20, 99), m = true)) ->
+ Sample("xym", isoWkb(1, false, Seq(10, 20))),
+ Sample("xyzm", isoWkb(1, true, Seq(10, 20, 30, 99), m = true)) ->
+ Sample("xyzm", isoWkb(1, true, Seq(10, 20, 30))))
+ val emptyMeasures = for {
+ z <- Seq(false, true)
+ kind <- Seq(1, 2, 3)
+ } yield {
+ val name = s"empty-measured-$kind-z=$z"
+ Sample(
+ name,
+ isoWkb(
+ kind,
+ z,
+ if (kind == 1) Seq.fill(if (z) 4 else 3)(Double.NaN) else Seq.empty,
+ m = true)) ->
+ Sample(
+ name,
+ isoWkb(kind, z, if (kind == 1) Seq.fill(if (z) 3 else 2)(Double.NaN) else Seq.empty))
+ }
+ val measuredLine = Sample(
+ "later-finite-z-measured",
+ isoWkb(2, true, Seq(1, 2, Double.NaN, 90, 3, 4, 5, 99), m = true)) ->
+ Sample("later-finite-z-measured", isoWkb(2, true, Seq(1, 2, Double.NaN, 3, 4, 5)))
+ val trailingXY = Sample("xy-after-measured", isoWkb(1, false, Seq(10, 20)))
+ val candidates = samples.map(s => s -> s) ++ projectedMeasures ++ emptyMeasures ++
+ Seq(measuredLine, trailingXY -> trailingXY)
+ val selected = if (allEmpty) candidates.filter { case (sample, _) =>
+ sample.wkb == null || WKBReader.forDeclaredDimensions().read(sample.wkb).isEmpty
+ }
+ else candidates
+ val input = selected.map(_._1)
+ val expected = selected.map(_._2)
+ val rows = input.zipWithIndex.map { case (sample, index) => Row(index, sample.wkb) }
+ val schema = StructType(
+ Seq(
+ StructField("id", IntegerType, nullable = false),
+ StructField("wkb", BinaryType, nullable = true)))
+ val path = new File(directory, "written").getPath
+ val df = sparkSession
+ .createDataFrame(rows.asJava, schema)
+ .selectExpr("id", "ST_GeomFromWKB(wkb) AS geometry")
+ assertGeometries(df, input, "before write")
+ df.coalesce(1).write.format("geoparquet").save(path)
+ assertWrittenFile(path, expected, allEmpty)
+ assertFile(path, expected)
+ }
+
+ describe("GeoParquet declared coordinate dimensions") {
+ for (dictionary <- Seq(false, true)) {
+ it(s"reads ISO WKB from a file (dictionary=$dictionary)") {
+ withDirectory { directory =>
+ val fixture = new File(directory, "fixture.parquet")
+ // Omitted CRS defaults to EPSG:4326, overriding embedded EWKB SRIDs.
+ val embeddedSrid = ByteBuffer
+ .allocate(33)
+ .order(ByteOrder.LITTLE_ENDIAN)
+ .put(1.toByte)
+ .putInt(0xa0000001)
+ .putInt(3857)
+ .putDouble(10)
+ .putDouble(20)
+ .putDouble(30)
+ .array()
+ val values = samples :+ Sample("embedded-srid", embeddedSrid)
+ writeFixture(fixture, dictionary, values)
+ assertFile(fixture.getPath, values, repetitions = 4, expectedSrid = 4326)
+ }
+ }
+ }
+ for (allEmpty <- Seq(false, true)) {
+ it(s"preserves XYZ, projects M, and writes matching metadata (allEmpty=$allEmpty)") {
+ withDirectory(directory => writeAndRoundTrip(directory, allEmpty))
+ }
+ }
+ }
+}
diff --git a/spark/common/src/test/scala/org/apache/spark/sql/execution/datasources/geoparquet/GeoParquetWriteSupportTest.scala b/spark/common/src/test/scala/org/apache/spark/sql/execution/datasources/geoparquet/GeoParquetWriteSupportTest.scala
new file mode 100644
index 00000000000..b8c59dfed74
--- /dev/null
+++ b/spark/common/src/test/scala/org/apache/spark/sql/execution/datasources/geoparquet/GeoParquetWriteSupportTest.scala
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.spark.sql.execution.datasources.geoparquet
+
+import org.apache.spark.sql.execution.datasources.geoparquet.GeoParquetWriteSupport.GeometryColumnBoundingBox
+import org.locationtech.jts.geom.{Coordinate, GeometryFactory}
+import org.scalatest.funsuite.AnyFunSuite
+
+class GeoParquetWriteSupportTest extends AnyFunSuite {
+ test("empty geometries do not contribute a column bounding box") {
+ val factory = new GeometryFactory()
+ val bbox = new GeometryColumnBoundingBox()
+ bbox.update(factory.createPoint())
+ bbox.update(factory.createLineString())
+ bbox.update(factory.createPolygon())
+ assert(bbox.minX == Double.PositiveInfinity)
+ assert(bbox.minY == Double.PositiveInfinity)
+ assert(bbox.maxX == Double.NegativeInfinity)
+ assert(bbox.maxY == Double.NegativeInfinity)
+ }
+
+ test("empty geometries do not extend a nonempty column bounding box") {
+ val factory = new GeometryFactory()
+ val bbox = new GeometryColumnBoundingBox()
+ bbox.update(factory.createPoint(new Coordinate(10, 20)))
+ bbox.update(factory.createPolygon())
+ assert(Seq(bbox.minX, bbox.minY, bbox.maxX, bbox.maxY) == Seq(10, 20, 10, 20))
+ }
+}