You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@sedona.apache.org by ma...@apache.org on 2021/07/21 16:27:19 UTC
[incubator-sedona] branch Sedona-17_Shape&WKBSerDe updated (426ec68
-> dd2536e)
This is an automated email from the ASF dual-hosted git repository.
malka pushed a change to branch Sedona-17_Shape&WKBSerDe
in repository https://gitbox.apache.org/repos/asf/incubator-sedona.git.
discard 426ec68 Added fromId method to SerializerType Added one Int byte as a serializerType for python bindings to know which serde type is used
discard ff7a3df Added TestBaseWKBSerdeJava for testing using the WKB serde Moved WBBGeometrySerializerTest from the scala to java to eliminate spark session collisions
discard 0c8c9cb Added beforeEach and afterEach to close the spark session
discard 8cafa78 Fixed two tests: 1. ST_Intersection - intersects but left contains right -> geometry the same but order of vertices are different. 2. Passed ST_Intersection - not intersects -> first geom are polygon which looks like a line, cause to found non-noded intersection between LINESTRING exception
discard 7597353 Add WKBGeometrySerializerTest to test the sql wkb serializer Fixed failed ST_GeomFromWKT test (failed for the shape serde and worked for the wkb serde) by using the wkb serde with sedona.serializer.type, wkb
discard 82a9bea Fixed tests to use the new serializer
discard ec3667a Added SerializerType enum which contains the supported serializer types Added package object to create a global sedonaSerializer object for the Spark SQL module Create SerializerType in SedonaConf with shape as a default choice, this will bring the ability to choose other type of serializer like wkb
discard 80263ed Added SedonaSerializer trait Update WKB and Shape serializer to implement the trait
discard 1c11de6 Removed @ignore from tests CI's bug is from Github Actions: https://github.com/actions/virtual-environments/issues/3185
discard 4906746 Added @Ignore to tests to see if they are the reason that the tests are failed in Github Actions while success in my env
discard e3c0bd8 use after class instead of after for tear down spark context to fix creation of multiple spark contexts
discard 7b9e4ab Refactored SedonaKryoRegistratorHelper to eliminate useless if else block Added tests for SedonaKryoRegistrator and SedonaWKBKryoRegistrator
discard 2754a24 Moved GeometrySerde to serde folder Created abstract GeometrySerde class for specific serde implementations Created ShapeGeometrySerde and WKBGeometrySerde that implements GeometrySerde Added tests for WKB Serde
discard 8abbdbf Added WKBGeometrySerde to handle special geometry types that the Shape serde not handled well
discard ab2c4f7 Refactor actual registration to SedonaKryoRegistratorHelper Create SedonaWKBKryoRegistrator for WKB support
discard 94b388f Moved SpatialIndexSerde to serde dir->SpatialIndex Removed unused GeometrySerde object from SpatialIndexSerde (eliminate the requirement to handle WKB and Shape GeometrySerde)
add d8c2aae [SEDONA-50] Removing logging configuration as it causes errors on databricks. (#530)
add 8fd688f [SEDONA-30] Add raster data support in Sedona SQL (#523)
add c363cd8 [SEDONA-30] Changes to Geotiff Loader (#531)
add 81e92f1 Fix the failed CI in Example CI (#533)
add e6513be [SEDONA-53]fix SpatialKnnQuery NullPointerException (#532)
add 89c247b Moved SpatialIndexSerde to serde dir->SpatialIndex Removed unused GeometrySerde object from SpatialIndexSerde (eliminate the requirement to handle WKB and Shape GeometrySerde)
add 79fc93a Refactor actual registration to SedonaKryoRegistratorHelper Create SedonaWKBKryoRegistrator for WKB support
add bc4d118 Added WKBGeometrySerde to handle special geometry types that the Shape serde not handled well
add 0fe90ab Moved GeometrySerde to serde folder Created abstract GeometrySerde class for specific serde implementations Created ShapeGeometrySerde and WKBGeometrySerde that implements GeometrySerde Added tests for WKB Serde
add d3ac0a3 Refactored SedonaKryoRegistratorHelper to eliminate useless if else block Added tests for SedonaKryoRegistrator and SedonaWKBKryoRegistrator
add 04c585e use after class instead of after for tear down spark context to fix creation of multiple spark contexts
add 4641554 Added @Ignore to tests to see if they are the reason that the tests are failed in Github Actions while success in my env
add 12c88b1 Removed @ignore from tests CI's bug is from Github Actions: https://github.com/actions/virtual-environments/issues/3185
add 42a4a45 Added SedonaSerializer trait Update WKB and Shape serializer to implement the trait
add 169e5f2 Added SerializerType enum which contains the supported serializer types Added package object to create a global sedonaSerializer object for the Spark SQL module Create SerializerType in SedonaConf with shape as a default choice, this will bring the ability to choose other type of serializer like wkb
add 348a68b Fixed tests to use the new serializer
add 2097463 Add WKBGeometrySerializerTest to test the sql wkb serializer Fixed failed ST_GeomFromWKT test (failed for the shape serde and worked for the wkb serde) by using the wkb serde with sedona.serializer.type, wkb
add aec629e Fixed two tests: 1. ST_Intersection - intersects but left contains right -> geometry the same but order of vertices are different. 2. Passed ST_Intersection - not intersects -> first geom are polygon which looks like a line, cause to found non-noded intersection between LINESTRING exception
add c19c1f4 Added beforeEach and afterEach to close the spark session
add b03e5cd Added TestBaseWKBSerdeJava for testing using the WKB serde Moved WBBGeometrySerializerTest from the scala to java to eliminate spark session collisions
add dd2536e Added fromId method to SerializerType Added one Int byte as a serializerType for python bindings to know which serde type is used
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (426ec68)
\
N -- N -- N refs/heads/Sedona-17_Shape&WKBSerDe (dd2536e)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/workflows/example.yml | 4 +-
.gitignore | 2 +-
binder/ApacheSedonaRaster_1.ipynb | 871 +++++++++++++++++++++
binder/data/raster/T21HUB_4704_4736_8224_8256.tif | Bin 0 -> 6619 bytes
.../data/raster/vya_T21HUB_992_1024_4352_4384.tif | Bin 0 -> 7689 bytes
.../sedona/core/knnJudgement/KnnJudgement.java | 10 +-
.../sedona/core/spatialOperator/PointKnnTest.java | 19 +
core/src/test/resources/raster/test1.tiff | Bin 0 -> 174803 bytes
core/src/test/resources/raster/test2.tiff | Bin 0 -> 174803 bytes
core/src/test/resources/small/onepoint.csv | 1 +
docs/api/sql/Constructor.md | 2 +-
docs/api/sql/Raster-loader.md | 144 ++++
docs/api/sql/Raster-operators.md | 309 ++++++++
docs/tutorial/raster.md | 19 +
mkdocs.yml | 15 +-
pom.xml | 13 +
python-adapter/.gitignore | 3 +-
python-adapter/pom.xml | 13 +
python/sedona/core/jvm/config.py | 31 +-
spark-version-converter.py | 3 +-
...org.apache.spark.sql.sources.DataSourceRegister | 1 +
.../scala/org/apache/sedona/sql/UDF/Catalog.scala | 31 +-
.../sql/sedona_sql/expressions/Functions.scala | 3 +-
.../sedona_sql/expressions/raster/Functions.scala | 865 ++++++++++++++++++++
.../sql/sedona_sql/expressions/raster/IO.scala | 256 ++++++
.../sql/sedona_sql/io/GeotiffFileFormat.scala | 104 +++
.../spark/sql/sedona_sql/io/GeotiffSchema.scala | 195 +++++
.../spark/sql/sedona_sql/io/HadoopUtils.scala | 107 +++
.../spark/sql/sedona_sql/io/ImageOptions.scala | 12 +
.../org/apache/sedona/sql/TestBaseScala.scala | 1 +
.../scala/org/apache/sedona/sql/rasterIOTest.scala | 79 ++
.../org/apache/sedona/sql/rasteralgebraTest.scala | 208 +++++
32 files changed, 3292 insertions(+), 29 deletions(-)
create mode 100644 binder/ApacheSedonaRaster_1.ipynb
create mode 100644 binder/data/raster/T21HUB_4704_4736_8224_8256.tif
create mode 100644 binder/data/raster/vya_T21HUB_992_1024_4352_4384.tif
create mode 100644 core/src/test/resources/raster/test1.tiff
create mode 100644 core/src/test/resources/raster/test2.tiff
create mode 100644 core/src/test/resources/small/onepoint.csv
create mode 100644 docs/api/sql/Raster-loader.md
create mode 100644 docs/api/sql/Raster-operators.md
create mode 100644 docs/tutorial/raster.md
create mode 100644 sql/src/main/resources/META-INF/services/org.apache.spark.sql.sources.DataSourceRegister
create mode 100644 sql/src/main/scala/org/apache/spark/sql/sedona_sql/expressions/raster/Functions.scala
create mode 100644 sql/src/main/scala/org/apache/spark/sql/sedona_sql/expressions/raster/IO.scala
create mode 100644 sql/src/main/scala/org/apache/spark/sql/sedona_sql/io/GeotiffFileFormat.scala
create mode 100644 sql/src/main/scala/org/apache/spark/sql/sedona_sql/io/GeotiffSchema.scala
create mode 100644 sql/src/main/scala/org/apache/spark/sql/sedona_sql/io/HadoopUtils.scala
create mode 100644 sql/src/main/scala/org/apache/spark/sql/sedona_sql/io/ImageOptions.scala
create mode 100644 sql/src/test/scala/org/apache/sedona/sql/rasterIOTest.scala
create mode 100644 sql/src/test/scala/org/apache/sedona/sql/rasteralgebraTest.scala