You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@sedona.apache.org by "edigonzales (via GitHub)" <gi...@apache.org> on 2025/04/06 15:18:07 UTC

[D] Creating an Iceberg table with a geometry column [sedona]

GitHub user edigonzales created a discussion: Creating an Iceberg table with a geometry column

Hi 

I'm trying to create an Iceberg table with a geometry column just link in [this](https://www.linkedin.com/posts/apache-sedona_you-can-create-an-apache-iceberg-table-with-activity-7307734602620522496-RpdA?utm_source=share&utm_medium=member_desktop&rcm=ACoAABamNI8BDFvLltP8Xxdvant9OdGaCtpB50U) example:

```

import org.apache.sedona.sql.utils.SedonaSQLRegistrator
SedonaSQLRegistrator.registerAll(spark)

val stmt = """
    CREATE TABLE local.myschema.geotable (id string, geom geometry)
    USING iceberg
    TBLPROPERTIES('format-version'='3');
"""

spark.sql(stmt).show()
``` 

But I get this error:

```
25/04/06 16:59:18 WARN UDTRegistration: Cannot register UDT for org.geotools.coverage.grid.GridCoverage2D, which is already registered.
25/04/06 16:59:18 WARN SimpleFunctionRegistry: The function rs_union_aggr replaced a previously registered function.
25/04/06 16:59:18 WARN UDTRegistration: Cannot register UDT for org.locationtech.jts.geom.Geometry, which is already registered.
25/04/06 16:59:18 WARN UDTRegistration: Cannot register UDT for org.apache.sedona.common.geometryObjects.Geography, which is already registered.
25/04/06 16:59:18 WARN UDTRegistration: Cannot register UDT for org.locationtech.jts.index.SpatialIndex, which is already registered.
25/04/06 16:59:18 WARN SimpleFunctionRegistry: The function st_envelope_aggr replaced a previously registered function.
25/04/06 16:59:18 WARN SimpleFunctionRegistry: The function st_intersection_aggr replaced a previously registered function.
25/04/06 16:59:18 WARN SimpleFunctionRegistry: The function st_union_aggr replaced a previously registered function.
java.lang.UnsupportedOperationException: User-defined types are not supported
  at org.apache.iceberg.spark.SparkTypeVisitor.visit(SparkTypeVisitor.java:52)
  at org.apache.iceberg.spark.SparkTypeVisitor.visit(SparkTypeVisitor.java:37)
  at org.apache.iceberg.spark.SparkSchemaUtil.convert(SparkSchemaUtil.java:126)
  at org.apache.iceberg.spark.SparkCatalog.createTable(SparkCatalog.java:238)
  at org.apache.spark.sql.connector.catalog.TableCatalog.createTable(TableCatalog.java:223)
```

I start the spark-shell with this command:

```
spark-shell \
  --conf spark.serializer=org.apache.spark.serializer.KryoSerializer \
  --conf spark.kryo.registrator=org.apache.sedona.core.serde.SedonaKryoRegistrator \
  --conf spark.sql.extensions=org.apache.sedona.sql.SedonaSqlExtensions \
  --conf spark.sql.catalog.local=org.apache.iceberg.spark.SparkCatalog \
  --conf spark.sql.catalog.local.type=hadoop \
  --conf spark.sql.catalog.local.warehouse=file:///Users/stefan/tmp/lakehouse \
  --packages org.apache.sedona:sedona-spark-shaded-3.5_2.12:1.7.1,org.datasyslab:geotools-wrapper:1.7.1-28.5,org.apache.iceberg:iceberg-spark-runtime-3.5_2.12:1.8.1
```

Any ideas what's wrong?

Stefan

GitHub link: https://github.com/apache/sedona/discussions/1911

----
This is an automatically sent email for issues@sedona.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@sedona.apache.org

Re: [D] Creating an Iceberg table with a geometry column [sedona]

Posted by "adaros92 (via GitHub)" <gi...@apache.org>.
GitHub user adaros92 added a comment to the discussion: Creating an Iceberg table with a geometry column

Thank you!

GitHub link: https://github.com/apache/sedona/discussions/1911#discussioncomment-15224748

----
This is an automatically sent email for issues@sedona.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@sedona.apache.org

Re: [D] Creating an Iceberg table with a geometry column [sedona]

Posted by "adaros92 (via GitHub)" <gi...@apache.org>.
GitHub user adaros92 added a comment to the discussion: Creating an Iceberg table with a geometry column

Any updates? :) 

GitHub link: https://github.com/apache/sedona/discussions/1911#discussioncomment-13688540

----
This is an automatically sent email for issues@sedona.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@sedona.apache.org

Re: [D] Creating an Iceberg table with a geometry column [sedona]

Posted by "jiayuasu (via GitHub)" <gi...@apache.org>.
GitHub user jiayuasu added a comment to the discussion: Creating an Iceberg table with a geometry column

@edigonzales This is an upcoming feature of Sedona 1.8.0. Please stay tuned about the release!

GitHub link: https://github.com/apache/sedona/discussions/1911#discussioncomment-12896066

----
This is an automatically sent email for issues@sedona.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@sedona.apache.org

Re: [D] Creating an Iceberg table with a geometry column [sedona]

Posted by "jiayuasu (via GitHub)" <gi...@apache.org>.
GitHub user jiayuasu added a comment to the discussion: Creating an Iceberg table with a geometry column

Not yet. We are still waiting for the Apache Iceberg community to merge a number of our PRs (forecasting Feb 2026 now).

This is one of the PRs: https://github.com/apache/iceberg/pull/14101  (it was created in Sept this year)

It would be great if you guys can comment on the PR and let the iceberg community know this is highly anticipated and please review the PR.

GitHub link: https://github.com/apache/sedona/discussions/1911#discussioncomment-15224644

----
This is an automatically sent email for issues@sedona.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@sedona.apache.org

Re: [D] Creating an Iceberg table with a geometry column [sedona]

Posted by "adaros92 (via GitHub)" <gi...@apache.org>.
GitHub user adaros92 added a comment to the discussion: Creating an Iceberg table with a geometry column

Is this supported in v1.8?

GitHub link: https://github.com/apache/sedona/discussions/1911#discussioncomment-15212637

----
This is an automatically sent email for issues@sedona.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@sedona.apache.org

Re: [D] Creating an Iceberg table with a geometry column [sedona]

Posted by "jiayuasu (via GitHub)" <gi...@apache.org>.
GitHub user jiayuasu added a comment to the discussion: Creating an Iceberg table with a geometry column

@adaros92 @edigonzales 

GitHub link: https://github.com/apache/sedona/discussions/1911#discussioncomment-15224645

----
This is an automatically sent email for issues@sedona.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@sedona.apache.org