You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "huan233usc (via GitHub)" <gi...@apache.org> on 2026/07/11 04:10:14 UTC

[PR] Parquet: Compute geometry bounding box metrics [iceberg]

huan233usc opened a new pull request, #17161:
URL: https://github.com/apache/iceberg/pull/17161

   Computes a bounding box for `geometry` columns in Parquet so they can carry
   `lower_bounds`/`upper_bounds` and participate in data skipping. This is the first
   slice of the geo bounds work (Phase 2), scoped to the clean, unambiguous case.
   
   ## Problem
   
   The Parquet footer's lexicographic min/max over WKB bytes is not meaningful for
   spatial values, so `ParquetMetrics` diverts geometry/geography columns to
   counts-only metrics. As a result geometry columns get no bounds and cannot be
   data-skipped.
   
   ## Approach
   
   Compute the box **at write time**, since the footer cannot provide it. This reuses
   the existing writer-side value-scanning metrics channel — the same path `float`
   and `double` already use to track NaN counts the footer can't give
   (`ParquetValueWriter.metrics()` → `ParquetWriter.metrics()`):
   
   - A new `GeometryWriter` writes byte-identical WKB **and** folds each value's XY
     coordinates into a running box, emitting a `FieldMetrics<GeospatialBound>` whose
     lower/upper corners are the box. These serialize into the existing
     `lower_bounds`/`upper_bounds` maps through the geometry `Conversions` case. No
     changes to `ParquetMetrics`, `ParquetWriter`, or `Conversions`.
   - The value-scanned metrics take precedence over the counts-only footer branch, and
     the optional-field writer reconciles the null count, so the builder tracks only
     non-null values.
   
   `WKBBoundingBox` (new, in `api/geospatial` next to `GeospatialBound`/`BoundingBox`)
   is a pure-Java WKB coordinate scanner — **no JTS dependency**. It walks all OGC
   geometry types (point, linestring, polygon, multi*, collection), handles both byte
   orders (per-geometry, so mixed-endian collections work), reads past Z/M (XY-only
   box), skips NaN coordinates (so `POINT EMPTY` contributes nothing, per spec), and
   validates the buffer defensively (truncation, bad byte order, unknown type, and
   oversized counts all raise `IllegalArgumentException` rather than reading out of
   bounds).
   
   ## Scope
   
   **GEOMETRY only, 2D (XY), planar** — the case the spec calls straightforward and
   unambiguous. Deliberately left as follow-ups:
   
   - **GEOGRAPHY** bounds (longitude periodicity, latitude extrema via edges, numeric
     correction, out-of-range coordinate policy) — needs its own numeric design.
   - Higher dimensions (XYZM), the `content_stats` `geo_lower`/`geo_upper` struct write
     bridge, ORC/Avro geo bounds, and CRS validation.
   
   ## Tests
   
   - `TestWKBBoundingBox` — the scanner across all geometry types, Z/M skip, both/mixed
     endianness, nested collections, empties, NaN skip, ±Inf, and malformed inputs.
   - `TestGeometryFieldMetrics` — the builder's bounds, counts, and empty/no-value cases.
   - `TestMetrics.testMetricsForGeospatialTypes` (new, exercised via `TestParquetMetrics`)
     — end-to-end: two geometry points produce the expected bbox in `lower_bounds`/
     `upper_bounds`; geography stays bounds-less; ORC is skipped (geo is Parquet-only).
   - `TestParquetDataWriter.testGeospatialRoundTrip` still passes unchanged, confirming
     the WKB bytes written are byte-identical.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Parquet: Compute geometry bounding box metrics [iceberg]

Posted by "szehon-ho (via GitHub)" <gi...@apache.org>.
szehon-ho commented on code in PR #17161:
URL: https://github.com/apache/iceberg/pull/17161#discussion_r3599021823


##########
api/src/main/java/org/apache/iceberg/geospatial/WKBBoundingBox.java:
##########
@@ -0,0 +1,217 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iceberg.geospatial;
+
+import java.nio.ByteBuffer;
+import java.nio.ByteOrder;
+import org.apache.iceberg.relocated.com.google.common.base.Preconditions;
+
+/**
+ * Computes a two-dimensional (XY) bounding box from geometry values encoded as Well-Known Binary
+ * (WKB).
+ *
+ * <p>The bounding box is the minimum axis-aligned rectangle that contains every coordinate of the
+ * geometries. Only the X and Y dimensions are considered; any Z or M values present in the WKB are
+ * read past and ignored. Coordinates whose X or Y is {@code NaN} are skipped, matching the spec
+ * rule that null or NaN values in a coordinate dimension do not contribute to bounds (an empty
+ * geometry such as {@code POINT EMPTY} therefore contributes nothing).
+ *
+ * <p>Parsing follows the OGC Simple Feature Access WKB layout and validates the input defensively:
+ * a malformed or truncated buffer results in an {@link IllegalArgumentException} rather than an
+ * out-of-bounds read.
+ */
+public class WKBBoundingBox {
+
+  // OGC WKB base geometry type codes (after stripping the dimension offset).
+  private static final int TYPE_POINT = 1;
+  private static final int TYPE_LINE_STRING = 2;
+  private static final int TYPE_POLYGON = 3;
+  private static final int TYPE_MULTI_POINT = 4;
+  private static final int TYPE_MULTI_LINE_STRING = 5;
+  private static final int TYPE_MULTI_POLYGON = 6;
+  private static final int TYPE_GEOMETRY_COLLECTION = 7;
+
+  // Bounds the recursion depth for nested collections to reject pathological or malicious input.
+  private static final int MAX_DEPTH = 100;
+
+  private WKBBoundingBox() {}
+
+  /**
+   * Parses one WKB geometry and folds all of its XY coordinates into the given accumulator.
+   *
+   * <p>The buffer is read via a duplicate, so the caller's position and limit are left unchanged.
+   *
+   * @param wkb a buffer containing a single WKB geometry
+   * @param accumulator the accumulator to update with the geometry's coordinates
+   * @throws IllegalArgumentException if the buffer is not a well-formed WKB geometry
+   */
+  public static void accumulate(ByteBuffer wkb, XYAccumulator accumulator) {
+    Preconditions.checkArgument(wkb != null, "Invalid WKB buffer: null");
+    Preconditions.checkArgument(accumulator != null, "Invalid accumulator: null");
+    ByteBuffer buffer = wkb.duplicate();
+    parseGeometry(buffer, accumulator, 0);
+  }
+
+  private static void parseGeometry(ByteBuffer buffer, XYAccumulator accumulator, int depth) {
+    Preconditions.checkArgument(depth <= MAX_DEPTH, "Invalid WKB: nesting too deep");
+    checkRemaining(buffer, 5);
+
+    byte order = buffer.get();
+    if (order == 0) {
+      buffer.order(ByteOrder.BIG_ENDIAN);
+    } else if (order == 1) {
+      buffer.order(ByteOrder.LITTLE_ENDIAN);
+    } else {
+      throw new IllegalArgumentException("Invalid WKB byte order: " + order);
+    }
+
+    long typeCode = buffer.getInt() & 0xFFFFFFFFL;
+    int geometryType = (int) (typeCode % 1000);
+    int dimensionGroup = (int) (typeCode / 1000);
+    int numDimensions = numDimensions(dimensionGroup, typeCode);
+
+    switch (geometryType) {
+      case TYPE_POINT:
+        readCoordinate(buffer, accumulator, numDimensions);
+        break;
+      case TYPE_LINE_STRING:
+        readCoordinateSequence(buffer, accumulator, numDimensions);
+        break;
+      case TYPE_POLYGON:
+        int numRings = readCount(buffer);
+        for (int i = 0; i < numRings; i += 1) {
+          readCoordinateSequence(buffer, accumulator, numDimensions);
+        }
+        break;
+      case TYPE_MULTI_POINT:
+      case TYPE_MULTI_LINE_STRING:
+      case TYPE_MULTI_POLYGON:
+      case TYPE_GEOMETRY_COLLECTION:
+        int numElements = readCount(buffer);
+        for (int i = 0; i < numElements; i += 1) {
+          // Each child carries its own byte-order and type header.
+          parseGeometry(buffer, accumulator, depth + 1);
+        }
+        break;
+      default:
+        throw new IllegalArgumentException("Invalid or unsupported WKB geometry type: " + typeCode);
+    }
+  }
+
+  private static int numDimensions(int dimensionGroup, long typeCode) {
+    switch (dimensionGroup) {
+      case 0: // XY
+        return 2;
+      case 1: // XYZ
+      case 2: // XYM
+        return 3;
+      case 3: // XYZM
+        return 4;
+      default:
+        throw new IllegalArgumentException("Invalid or unsupported WKB geometry type: " + typeCode);
+    }
+  }
+
+  private static void readCoordinateSequence(
+      ByteBuffer buffer, XYAccumulator accumulator, int numDimensions) {
+    int numPoints = readCount(buffer);
+    // Validate the full extent up front so a bogus point count cannot drive a long read loop, but
+    // never pre-allocate from the count itself.
+    checkRemaining(buffer, (long) numPoints * numDimensions * Double.BYTES);
+    for (int i = 0; i < numPoints; i += 1) {
+      readCoordinate(buffer, accumulator, numDimensions);
+    }
+  }
+
+  private static void readCoordinate(
+      ByteBuffer buffer, XYAccumulator accumulator, int numDimensions) {
+    checkRemaining(buffer, (long) numDimensions * Double.BYTES);
+    double xCoord = buffer.getDouble();
+    double yCoord = buffer.getDouble();
+    // Skip any Z and/or M values; only X and Y contribute to the box.
+    for (int i = 2; i < numDimensions; i += 1) {
+      buffer.getDouble();
+    }
+    accumulator.addXY(xCoord, yCoord);
+  }
+
+  private static int readCount(ByteBuffer buffer) {
+    checkRemaining(buffer, Integer.BYTES);
+    long count = buffer.getInt() & 0xFFFFFFFFL;
+    Preconditions.checkArgument(count <= Integer.MAX_VALUE, "Invalid WKB element count: %s", count);
+    return (int) count;
+  }
+
+  private static void checkRemaining(ByteBuffer buffer, long bytes) {
+    Preconditions.checkArgument(
+        buffer.remaining() >= bytes, "Invalid WKB: unexpected end of buffer");
+  }
+
+  /**
+   * A mutable accumulator of the minimum and maximum X and Y coordinates seen so far.
+   *
+   * <p>Coordinates whose X or Y is {@code NaN} are ignored. An accumulator that has seen no non-NaN
+   * coordinate reports {@link #hasBounds()} as {@code false} and returns {@code null} bounds.
+   */
+  public static class XYAccumulator {
+    private double minX = Double.POSITIVE_INFINITY;
+    private double minY = Double.POSITIVE_INFINITY;
+    private double maxX = Double.NEGATIVE_INFINITY;
+    private double maxY = Double.NEGATIVE_INFINITY;
+    private boolean hasBounds = false;
+
+    /**
+     * Folds a single coordinate into the box, ignoring it if either dimension is {@code NaN}.
+     *
+     * @param xCoord the X coordinate
+     * @param yCoord the Y coordinate
+     */
+    public void addXY(double xCoord, double yCoord) {
+      if (Double.isNaN(xCoord) || Double.isNaN(yCoord)) {
+        return;

Review Comment:
   This skips the whole coordinate when either axis is NaN, but the spec skips per dimension. Consider accumulating X and Y independently so `POINT (1 NaN)` still contributes X=1 when other rows supply a valid Y.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Parquet: Compute geometry bounding box metrics [iceberg]

Posted by "szehon-ho (via GitHub)" <gi...@apache.org>.
szehon-ho commented on code in PR #17161:
URL: https://github.com/apache/iceberg/pull/17161#discussion_r3899700011


##########
parquet/src/main/java/org/apache/iceberg/parquet/ParquetValueWriters.java:
##########
@@ -362,6 +368,30 @@ public Stream<FieldMetrics<?>> metrics() {
     }
   }
 
+  private static class GeometryWriter extends PrimitiveWriter<ByteBuffer> {
+    private final GeometryFieldMetrics.Builder metricsBuilder;
+
+    private GeometryWriter(ColumnDescriptor desc, org.apache.iceberg.types.Type geometryType) {
+      super(desc);
+      this.metricsBuilder =
+          new GeometryFieldMetrics.Builder(
+              desc.getPrimitiveType().getId().intValue(), geometryType);
+    }
+
+    @Override
+    public void write(int repetitionLevel, ByteBuffer buffer) {
+      // Accumulate the bounding box before writing, so it reads the buffer's coordinates while the
+      // position is intact (the scanner reads a duplicate and leaves this buffer untouched).
+      metricsBuilder.addValue(buffer);

Review Comment:
   Please preserve the existing average WKB size metric while adding bounds. #17333 already made `GeospatialWriter` populate `avg_value_size_in_bytes`, but this writer replaces it with a builder that never records or returns the size, so generic Parquet geometry writes lose that metric. Could `GeometryFieldMetrics.Builder` accumulate the WKB size and carry the average into the returned `FieldMetrics`, and keep the existing `TestMetrics` assertion?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Parquet: Compute geometry bounding box metrics [iceberg]

Posted by "szehon-ho (via GitHub)" <gi...@apache.org>.
szehon-ho commented on code in PR #17161:
URL: https://github.com/apache/iceberg/pull/17161#discussion_r3993642179


##########
parquet/src/main/java/org/apache/iceberg/data/parquet/BaseParquetWriter.java:
##########
@@ -268,8 +268,11 @@ public Optional<ParquetValueWriter<?>> visit(
     @Override
     public Optional<ParquetValueWriter<?>> visit(
         LogicalTypeAnnotation.GeometryLogicalTypeAnnotation geometryType) {
-      // geometry values are pure WKB stored in a BINARY column
-      return Optional.of(ParquetValueWriters.geospatial(desc));
+      // geometry values are pure WKB stored in a BINARY column; the writer also scans the
+      // coordinates to produce a bounding box (the Parquet footer cannot). The concrete CRS is
+      // immaterial here: ParquetMetrics rewrites the bound's type from the table schema before
+      // serialization, and the serialization is keyed on the type id, which ignores the CRS.
+      return Optional.of(ParquetValueWriters.geometry(desc, Types.GeometryType.crs84()));

Review Comment:
   Could we avoid scanning WKB when the configured metrics mode does not retain bounds? `none` and `counts` still walk every coordinate and discard the result, which may add substantial CPU overhead for complex geometries.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Parquet: Compute geometry bounding box metrics [iceberg]

Posted by "szehon-ho (via GitHub)" <gi...@apache.org>.
szehon-ho commented on PR #17161:
URL: https://github.com/apache/iceberg/pull/17161#issuecomment-5641672088

   Merged, thanks @huan233usc !


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Parquet: Compute geometry bounding box metrics [iceberg]

Posted by "huan233usc (via GitHub)" <gi...@apache.org>.
huan233usc commented on code in PR #17161:
URL: https://github.com/apache/iceberg/pull/17161#discussion_r3993843788


##########
parquet/src/main/java/org/apache/iceberg/data/parquet/BaseParquetWriter.java:
##########
@@ -268,8 +268,11 @@ public Optional<ParquetValueWriter<?>> visit(
     @Override
     public Optional<ParquetValueWriter<?>> visit(
         LogicalTypeAnnotation.GeometryLogicalTypeAnnotation geometryType) {
-      // geometry values are pure WKB stored in a BINARY column
-      return Optional.of(ParquetValueWriters.geospatial(desc));
+      // geometry values are pure WKB stored in a BINARY column; the writer also scans the
+      // coordinates to produce a bounding box (the Parquet footer cannot). The concrete CRS is
+      // immaterial here: ParquetMetrics rewrites the bound's type from the table schema before
+      // serialization, and the serialization is keyed on the type id, which ignores the CRS.
+      return Optional.of(ParquetValueWriters.geometry(desc, Types.GeometryType.crs84()));

Review Comment:
   Thanks, agreed that the WKB walk should be avoided when bounds are not retained. Doing that requires threading the selected `MetricsConfig` into the value-writer configuration; the generic writer factory currently receives only the Iceberg schema and Parquet type. I’ll keep this PR focused and handle that plumbing in a follow-up (also listed in the PR scope).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Parquet: Compute geometry bounding box metrics [iceberg]

Posted by "huan233usc (via GitHub)" <gi...@apache.org>.
huan233usc commented on code in PR #17161:
URL: https://github.com/apache/iceberg/pull/17161#discussion_r3993499545


##########
api/src/main/java/org/apache/iceberg/geospatial/WKBBoundingBox.java:
##########
@@ -0,0 +1,217 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iceberg.geospatial;
+
+import java.nio.ByteBuffer;
+import java.nio.ByteOrder;
+import org.apache.iceberg.relocated.com.google.common.base.Preconditions;
+
+/**
+ * Computes a two-dimensional (XY) bounding box from geometry values encoded as Well-Known Binary
+ * (WKB).
+ *
+ * <p>The bounding box is the minimum axis-aligned rectangle that contains every coordinate of the
+ * geometries. Only the X and Y dimensions are considered; any Z or M values present in the WKB are
+ * read past and ignored. Coordinates whose X or Y is {@code NaN} are skipped, matching the spec
+ * rule that null or NaN values in a coordinate dimension do not contribute to bounds (an empty
+ * geometry such as {@code POINT EMPTY} therefore contributes nothing).
+ *
+ * <p>Parsing follows the OGC Simple Feature Access WKB layout and validates the input defensively:
+ * a malformed or truncated buffer results in an {@link IllegalArgumentException} rather than an
+ * out-of-bounds read.
+ */
+public class WKBBoundingBox {
+
+  // OGC WKB base geometry type codes (after stripping the dimension offset).
+  private static final int TYPE_POINT = 1;
+  private static final int TYPE_LINE_STRING = 2;
+  private static final int TYPE_POLYGON = 3;
+  private static final int TYPE_MULTI_POINT = 4;
+  private static final int TYPE_MULTI_LINE_STRING = 5;
+  private static final int TYPE_MULTI_POLYGON = 6;
+  private static final int TYPE_GEOMETRY_COLLECTION = 7;
+
+  // Bounds the recursion depth for nested collections to reject pathological or malicious input.
+  private static final int MAX_DEPTH = 100;
+
+  private WKBBoundingBox() {}
+
+  /**
+   * Parses one WKB geometry and folds all of its XY coordinates into the given accumulator.
+   *
+   * <p>The buffer is read via a duplicate, so the caller's position and limit are left unchanged.
+   *
+   * @param wkb a buffer containing a single WKB geometry
+   * @param accumulator the accumulator to update with the geometry's coordinates
+   * @throws IllegalArgumentException if the buffer is not a well-formed WKB geometry
+   */
+  public static void accumulate(ByteBuffer wkb, XYAccumulator accumulator) {
+    Preconditions.checkArgument(wkb != null, "Invalid WKB buffer: null");
+    Preconditions.checkArgument(accumulator != null, "Invalid accumulator: null");
+    ByteBuffer buffer = wkb.duplicate();
+    parseGeometry(buffer, accumulator, 0);
+  }
+
+  private static void parseGeometry(ByteBuffer buffer, XYAccumulator accumulator, int depth) {
+    Preconditions.checkArgument(depth <= MAX_DEPTH, "Invalid WKB: nesting too deep");
+    checkRemaining(buffer, 5);
+
+    byte order = buffer.get();
+    if (order == 0) {
+      buffer.order(ByteOrder.BIG_ENDIAN);
+    } else if (order == 1) {
+      buffer.order(ByteOrder.LITTLE_ENDIAN);
+    } else {
+      throw new IllegalArgumentException("Invalid WKB byte order: " + order);
+    }
+
+    long typeCode = buffer.getInt() & 0xFFFFFFFFL;
+    int geometryType = (int) (typeCode % 1000);
+    int dimensionGroup = (int) (typeCode / 1000);
+    int numDimensions = numDimensions(dimensionGroup, typeCode);
+
+    switch (geometryType) {
+      case TYPE_POINT:
+        readCoordinate(buffer, accumulator, numDimensions);
+        break;
+      case TYPE_LINE_STRING:
+        readCoordinateSequence(buffer, accumulator, numDimensions);
+        break;
+      case TYPE_POLYGON:
+        int numRings = readCount(buffer);
+        for (int i = 0; i < numRings; i += 1) {
+          readCoordinateSequence(buffer, accumulator, numDimensions);
+        }
+        break;
+      case TYPE_MULTI_POINT:
+      case TYPE_MULTI_LINE_STRING:
+      case TYPE_MULTI_POLYGON:
+      case TYPE_GEOMETRY_COLLECTION:
+        int numElements = readCount(buffer);
+        for (int i = 0; i < numElements; i += 1) {
+          // Each child carries its own byte-order and type header.
+          parseGeometry(buffer, accumulator, depth + 1);
+        }
+        break;
+      default:
+        throw new IllegalArgumentException("Invalid or unsupported WKB geometry type: " + typeCode);
+    }
+  }
+
+  private static int numDimensions(int dimensionGroup, long typeCode) {
+    switch (dimensionGroup) {
+      case 0: // XY
+        return 2;
+      case 1: // XYZ
+      case 2: // XYM
+        return 3;
+      case 3: // XYZM
+        return 4;
+      default:
+        throw new IllegalArgumentException("Invalid or unsupported WKB geometry type: " + typeCode);
+    }
+  }
+
+  private static void readCoordinateSequence(
+      ByteBuffer buffer, XYAccumulator accumulator, int numDimensions) {
+    int numPoints = readCount(buffer);
+    // Validate the full extent up front so a bogus point count cannot drive a long read loop, but
+    // never pre-allocate from the count itself.
+    checkRemaining(buffer, (long) numPoints * numDimensions * Double.BYTES);
+    for (int i = 0; i < numPoints; i += 1) {
+      readCoordinate(buffer, accumulator, numDimensions);
+    }
+  }
+
+  private static void readCoordinate(
+      ByteBuffer buffer, XYAccumulator accumulator, int numDimensions) {
+    checkRemaining(buffer, (long) numDimensions * Double.BYTES);
+    double xCoord = buffer.getDouble();
+    double yCoord = buffer.getDouble();
+    // Skip any Z and/or M values; only X and Y contribute to the box.
+    for (int i = 2; i < numDimensions; i += 1) {
+      buffer.getDouble();
+    }
+    accumulator.addXY(xCoord, yCoord);
+  }
+
+  private static int readCount(ByteBuffer buffer) {
+    checkRemaining(buffer, Integer.BYTES);
+    long count = buffer.getInt() & 0xFFFFFFFFL;
+    Preconditions.checkArgument(count <= Integer.MAX_VALUE, "Invalid WKB element count: %s", count);
+    return (int) count;
+  }
+
+  private static void checkRemaining(ByteBuffer buffer, long bytes) {
+    Preconditions.checkArgument(
+        buffer.remaining() >= bytes, "Invalid WKB: unexpected end of buffer");
+  }
+
+  /**
+   * A mutable accumulator of the minimum and maximum X and Y coordinates seen so far.
+   *
+   * <p>Coordinates whose X or Y is {@code NaN} are ignored. An accumulator that has seen no non-NaN
+   * coordinate reports {@link #hasBounds()} as {@code false} and returns {@code null} bounds.
+   */
+  public static class XYAccumulator {
+    private double minX = Double.POSITIVE_INFINITY;
+    private double minY = Double.POSITIVE_INFINITY;
+    private double maxX = Double.NEGATIVE_INFINITY;
+    private double maxY = Double.NEGATIVE_INFINITY;
+    private boolean hasBounds = false;
+
+    /**
+     * Folds a single coordinate into the box, ignoring it if either dimension is {@code NaN}.
+     *
+     * @param xCoord the X coordinate
+     * @param yCoord the Y coordinate
+     */
+    public void addXY(double xCoord, double yCoord) {
+      if (Double.isNaN(xCoord) || Double.isNaN(yCoord)) {
+        return;

Review Comment:
   Thanks. This now goes through `GeometryBoundsBuilder` (#17509), which accumulates X and Y independently and skips NaN per dimension. `POINT (1 NaN)` plus `POINT (NaN 2)` produces `(1, 2)–(1, 2)`, covered by `TestGeometryBoundsBuilder.boundsAcrossValuesWithMissingCoordinates`.



##########
parquet/src/main/java/org/apache/iceberg/parquet/ParquetValueWriters.java:
##########
@@ -362,6 +368,30 @@ public Stream<FieldMetrics<?>> metrics() {
     }
   }
 
+  private static class GeometryWriter extends PrimitiveWriter<ByteBuffer> {
+    private final GeometryFieldMetrics.Builder metricsBuilder;
+
+    private GeometryWriter(ColumnDescriptor desc, org.apache.iceberg.types.Type geometryType) {
+      super(desc);
+      this.metricsBuilder =
+          new GeometryFieldMetrics.Builder(
+              desc.getPrimitiveType().getId().intValue(), geometryType);
+    }
+
+    @Override
+    public void write(int repetitionLevel, ByteBuffer buffer) {
+      // Accumulate the bounding box before writing, so it reads the buffer's coordinates while the
+      // position is intact (the scanner reads a duplicate and leaves this buffer untouched).
+      metricsBuilder.addValue(buffer);

Review Comment:
   Thanks. `GeometryFieldMetrics.Builder` now accumulates WKB size over non-null values and carries `avg_value_size_in_bytes` on the returned `FieldMetrics`. The existing `TestMetrics` assertion is kept, and `TestParquetValueWriters.geometryValueSizeMetricsExcludeNulls` checks the same average after adding bounds.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Parquet: Compute geometry bounding box metrics [iceberg]

Posted by "szehon-ho (via GitHub)" <gi...@apache.org>.
szehon-ho merged PR #17161:
URL: https://github.com/apache/iceberg/pull/17161


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org