You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@sedona.apache.org by "Robinlovelace (via GitHub)" <gi...@apache.org> on 2026/01/02 23:52:31 UTC

[D] Observations from R and Python benchmarks: performance bottlenecks and optimization ideas for sedona-db [sedona]

GitHub user Robinlovelace created a discussion: Observations from R and Python benchmarks: performance bottlenecks and optimization ideas for sedona-db

I have tested sedona-db R and Python interfaces vs established tools like geopandas and sf (R). You can find the full reproducible
  setup and current results here: Robinlovelace/geobench (https://github.com/Robinlovelace/geobench).

The benchmarks show impressive performance for Sedona. Hoping the results are of interest, and I think some of the observations below could point the way to speed-ups in the interfaces, I'm quite new to the project so sharing these observations as a discussion rather than bombarding the project with issues, I've already opened one as you'll see in link below!

  1. Python: The Shapely Deserialization Bottleneck
  In my Python benchmarks, I noticed a significant drop in performance when collecting query results using .to_pandas().

   * Observation: .to_pandas() triggers a conversion from internal Arrow/WKB binary geometries into Python shapely objects. For a 100k point dataset, this conversion becomes the
     dominant cost.
   * Workaround: By bypassing .to_pandas() and staying in the Arrow/Polars ecosystem, I achieved a ~5.5x speedup.
   * Implementation: See my bench_sedona_polars.py (https://github.com/Robinlovelace/geobench/blob/main/scripts/bench_sedona_polars.py) script.
   * Question: Would the team be open to adding a native .to_polars() method (or similar) to keep geometries in efficient binary format?

  2. R: Direct File Ingestion (GDAL/OGR)
  I found that the R interface currently lacks an equivalent to the Python sd.read_pyogrio(). 

   * Current Path: To benchmark loading from disk, I have to read via sf::st_read() and then pipe to sd_to_view(). This materializes heavy R sf objects in memory before they are
     converted back to Arrow for Sedona.
   * Opportunity: Implementing an sd_read_gdal() or sd_read_pyogrio() equivalent in the R interface (similar to the path in my R sedonadb script
     (https://github.com/Robinlovelace/geobench/blob/main/scripts/bench_sedona_r.R)) would allow direct file-to-engine ingestion via Arrow C Streams, drastically reducing startup
     overhead.

  3. Roadmap: Complex Linestring Operations
  I am curious about the roadmap for parallelized linestring operations in the native engine. Specifically:
   * Does the engine currently support (or plan to support) high-performance line merges and line splits (ST_LineMerge, ST_Subdivide)?
   * Given the scaling I see in spatial joins, I'm wondering if Sedona could be positioned to become the go-to tool for large-scale topological network cleaning.

  4. Optimizer Bug (Issue #477)
  I also encountered a specific PhysicalOptimizer schema mismatch error when joining Pandas-sourced tables of different sizes. I have documented this in detail here: sedona-db Issue
  #477 (https://github.com/apache/sedona-db/issues/477).

  I’d love to hear the community’s thoughts on these observations and how I can best contribute to testing these high-performance paths!


GitHub link: https://github.com/apache/sedona/discussions/2576

----
This is an automatically sent email for issues@sedona.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@sedona.apache.org


Re: [D] Observations from R and Python benchmarks: performance bottlenecks and optimization ideas for sedona-db [sedona]

Posted by "Robinlovelace (via GitHub)" <gi...@apache.org>.
GitHub user Robinlovelace added a comment to the discussion: Observations from R and Python benchmarks: performance bottlenecks and optimization ideas for sedona-db

Many thanks for the responses! This is definitely in the spirit of "let's kick the tires and give it a spin" so thanks for seeing it that way. Great point about not returning `sf`: I think R variants returning as arrow (what would that look like in R, an arrow object with a binary geo list column?) vs sf is a good plan and will open an issue on that. I don't have much time to dig deeper on this. However, I have got to the stage where I'm like "cool it works" :tada:  so yes I plan to write about it and will put in our geocompx blog with input from anyone who's interested in contributing. I haven't done a blog in a while, and want to make it as useful as possible for this project and open source software in general so will be great to get your/apache-db team input on that. This will be fun!

GitHub link: https://github.com/apache/sedona/discussions/2576#discussioncomment-15413622

----
This is an automatically sent email for issues@sedona.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@sedona.apache.org


Re: [D] Observations from R and Python benchmarks: performance bottlenecks and optimization ideas for sedona-db [sedona]

Posted by "jiayuasu (via GitHub)" <gi...@apache.org>.
GitHub user jiayuasu added a comment to the discussion: Observations from R and Python benchmarks: performance bottlenecks and optimization ideas for sedona-db

@paleolimbot @Kontinuation @zhangfengcdt @james-willis @Imbruced 

GitHub link: https://github.com/apache/sedona/discussions/2576#discussioncomment-15395240

----
This is an automatically sent email for issues@sedona.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@sedona.apache.org


Re: [D] Observations from R and Python benchmarks: performance bottlenecks and optimization ideas for sedona-db [sedona]

Posted by "Robinlovelace (via GitHub)" <gi...@apache.org>.
GitHub user Robinlovelace edited a discussion: Observations from R and Python benchmarks: performance bottlenecks and optimization ideas for sedona-db

I have tested sedona-db R and Python interfaces vs established tools like geopandas and sf (R). You can find the full reproducible
  setup and current results here: [Robinlovelace/geobench](https://github.com/Robinlovelace/geobench).

The benchmarks show impressive performance for Sedona. Hoping the results are of interest, and I think some of the observations below could point the way to speed-ups in the interfaces, I'm quite new to the project so sharing these observations as a discussion rather than bombarding the project with issues, I've already opened one as you'll see in link below!

  1. Python: The Shapely Deserialization Bottleneck
  In my Python benchmarks, I noticed a significant drop in performance when collecting query results using .to_pandas().

   * Observation: .to_pandas() triggers a conversion from internal Arrow/WKB binary geometries into Python shapely objects. For a 100k point dataset, this conversion becomes the
     dominant cost.
   * Workaround: By bypassing .to_pandas() and staying in the Arrow/Polars ecosystem, I achieved a ~5.5x speedup.
   * Implementation: See [bench_sedona_polars.py](https://github.com/Robinlovelace/geobench/blob/main/scripts/bench_sedona_polars.py) script.
   * Question: Would the team be open to adding a native .to_polars() method (or similar) to keep geometries in efficient binary format?

  2. R: Direct File Ingestion (GDAL/OGR)
  I found that the R interface currently lacks an equivalent to the Python sd.read_pyogrio(). 

   * Current Path: To benchmark loading from disk, I have to read via sf::st_read() and then pipe to sd_to_view(). This materializes heavy R sf objects in memory before they are
     converted back to Arrow for Sedona.
   * Opportunity: Implementing an sd_read_gdal() or sd_read_pyogrio() equivalent in the R interface (similar to the path in my [R sedonadb](https://github.com/Robinlovelace/geobench/blob/main/scripts/bench_sedona_r.R)) would allow direct file-to-engine ingestion via Arrow C Streams, drastically reducing startup
     overhead.

  3. Roadmap: Complex Linestring Operations
  I am curious about the roadmap for parallelized linestring operations in the native engine. Specifically:
   * Does the engine currently support (or plan to support) high-performance line merges and line splits (ST_LineMerge, ST_Subdivide)?
   * Given the scaling I see in spatial joins, I'm wondering if Sedona could be positioned to become the go-to tool for large-scale topological network cleaning.

  4. Optimizer Bug (Issue #477)
  I also encountered a specific PhysicalOptimizer schema mismatch error when joining Pandas-sourced tables of different sizes. I have documented this in detail here: sedona-db Issue
  #477 (https://github.com/apache/sedona-db/issues/477).

  I’d love to hear the community’s thoughts on these observations and how I can best contribute to testing these high-performance paths!


GitHub link: https://github.com/apache/sedona/discussions/2576

----
This is an automatically sent email for issues@sedona.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@sedona.apache.org


Re: [D] Observations from R and Python benchmarks: performance bottlenecks and optimization ideas for sedona-db [sedona]

Posted by "Robinlovelace (via GitHub)" <gi...@apache.org>.
GitHub user Robinlovelace edited a discussion: Observations from R and Python benchmarks: performance bottlenecks and optimization ideas for sedona-db

I have tested sedona-db R and Python interfaces vs established tools like geopandas and sf (R). You can find the full reproducible
  setup and current results here: [Robinlovelace/geobench](https://github.com/Robinlovelace/geobench).

The benchmarks show impressive performance for Sedona. Hoping the results are of interest, and I think some of the observations below could point the way to speed-ups in the interfaces, I'm quite new to the project so sharing these observations as a discussion rather than bombarding the project with issues, I've already opened one as you'll see in link below!

  1. Python: The Shapely Deserialization Bottleneck
  In my Python benchmarks, I noticed a significant drop in performance when collecting query results using .to_pandas().

   * Observation: .to_pandas() triggers a conversion from internal Arrow/WKB binary geometries into Python shapely objects. For a 100k point dataset, this conversion becomes the
     dominant cost.
   * Workaround: By bypassing .to_pandas() and staying in the Arrow/Polars ecosystem, I achieved a ~5.5x speedup.
   * Implementation: See [bench_sedona_polars.py](https://github.com/Robinlovelace/geobench/blob/main/scripts/bench_sedona_polars.py) script.
   * Question: Would the team be open to adding a native .to_polars() method (or similar) to keep geometries in efficient binary format?

  2. R: Direct File Ingestion (GDAL/OGR)
  I found that the R interface currently lacks an equivalent to the Python sd.read_pyogrio(). 

   * Current Path: To benchmark loading from disk, I have to read via sf::st_read() and then pipe to sd_to_view(). This materializes heavy R sf objects in memory before they are
     converted back to Arrow for Sedona.
   * Opportunity: Implementing an sd_read_gdal() or sd_read_pyogrio() equivalent in the R interface (similar to the path in my [R sedonadb](https://github.com/Robinlovelace/geobench/blob/main/scripts/bench_sedona_r.R)) would allow direct file-to-engine ingestion via Arrow C Streams, drastically reducing startup
     overhead.

  3. Roadmap: Complex Linestring Operations
  I am curious about the roadmap for parallelized linestring operations in the native engine. Specifically:
   * Does the engine currently support (or plan to support) high-performance line merges and line splits (ST_LineMerge, ST_Subdivide)?
   * Given the scaling I see in spatial joins, I'm wondering if Sedona could be positioned to become the go-to tool for large-scale topological network cleaning.

  4. I opened a specific issue: https://github.com/apache/sedona-db/issues/477

  I’d love to hear the community’s thoughts on these observations and how I can best contribute to testing these high-performance paths!


GitHub link: https://github.com/apache/sedona/discussions/2576

----
This is an automatically sent email for issues@sedona.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@sedona.apache.org


Re: [D] Observations from R and Python benchmarks: performance bottlenecks and optimization ideas for sedona-db [sedona]

Posted by "Robinlovelace (via GitHub)" <gi...@apache.org>.
GitHub user Robinlovelace edited a discussion: Observations from R and Python benchmarks: performance bottlenecks and optimization ideas for sedona-db

I have tested sedona-db R and Python interfaces vs established tools like geopandas and sf (R). You can find the full reproducible
  setup and current results here: [Robinlovelace/geobench](https://github.com/Robinlovelace/geobench).

The benchmarks show impressive performance for Sedona. Hoping the results are of interest, and I think some of the observations below could point the way to speed-ups in the interfaces, I'm quite new to the project so sharing these observations as a discussion rather than bombarding the project with issues, I've already opened one as you'll see in link below!

  1. Python: The Shapely Deserialization Bottleneck
  In my Python benchmarks, I noticed a significant drop in performance when collecting query results using .to_pandas().

   * Observation: .to_pandas() triggers a conversion from internal Arrow/WKB binary geometries into Python shapely objects. For a 100k point dataset, this conversion becomes the
     dominant cost.
   * Workaround: By bypassing .to_pandas() and staying in the Arrow/Polars ecosystem, I achieved a ~5.5x speedup.
   * Implementation: See [bench_sedona_polars.py](https://github.com/Robinlovelace/geobench/blob/main/scripts/bench_sedona_polars.py) script.
   * Question: Would the team be open to adding a native .to_polars() method (or similar) to keep geometries in efficient binary format?

  2. R: Direct File Ingestion (GDAL/OGR)
  I found that the R interface currently lacks an equivalent to the Python sd.read_pyogrio(). 

   * Current Path: To benchmark loading from disk, I have to read via sf::st_read() and then pipe to sd_to_view(). This materializes heavy R sf objects in memory before they are
     converted back to Arrow for Sedona.
   * Opportunity: Implementing an sd_read_gdal() or sd_read_pyogrio() equivalent in the R interface (similar to the path in my [R sedonadb](https://github.com/Robinlovelace/geobench/blob/main/scripts/bench_sedona_r.R)) would allow direct file-to-engine ingestion via Arrow C Streams, drastically reducing startup
     overhead.

  3. Roadmap: Complex Linestring Operations
  I am curious about the roadmap for parallelized linestring operations in the native engine. Specifically:
   * Does the engine currently support (or plan to support) high-performance line merges and line splits (ST_LineMerge, ST_Subdivide)?
   * Given the scaling I see in spatial joins, I'm wondering if Sedona could be positioned to become the go-to tool for large-scale topological network cleaning.

  4. Optimizer Bug (Issue #477)
  I also encountered a specific PhysicalOptimizer schema mismatch error when joining Pandas-sourced tables of different sizes: https://github.com/apache/sedona-db/issues/477

  I’d love to hear the community’s thoughts on these observations and how I can best contribute to testing these high-performance paths!


GitHub link: https://github.com/apache/sedona/discussions/2576

----
This is an automatically sent email for issues@sedona.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@sedona.apache.org


Re: [D] Observations from R and Python benchmarks: performance bottlenecks and optimization ideas for sedona-db [sedona]

Posted by "paleolimbot (via GitHub)" <gi...@apache.org>.
GitHub user paleolimbot added a comment to the discussion: Observations from R and Python benchmarks: performance bottlenecks and optimization ideas for sedona-db

Awesome! Feel free to ping me for a review on the blog post or anything else!

GitHub link: https://github.com/apache/sedona/discussions/2576#discussioncomment-15416228

----
This is an automatically sent email for issues@sedona.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@sedona.apache.org


Re: [D] Observations from R and Python benchmarks: performance bottlenecks and optimization ideas for sedona-db [sedona]

Posted by "paleolimbot (via GitHub)" <gi...@apache.org>.
GitHub user paleolimbot added a comment to the discussion: Observations from R and Python benchmarks: performance bottlenecks and optimization ideas for sedona-db

A few updates here!

> Would the team be open to adding a native .to_polars() method (or similar) to keep geometries in efficient binary format?

Becauase a SedonaDB DataFrame implements the Arrow PyCapsule protocol, I think you can just do `pl.from_arrow(df)` (e.g., no intermediary `to_arrow_table()`. I'm not sure if polars can stream (DuckDB can) but if it does it would be able to avoid loading the entire table into memory at once.

```python
import duckdb
import polars as pl
import sedona.db

sd = sedona.db.connect()

url = "https://github.com/geoarrow/geoarrow-data/releases/download/v0.2.0/ns-water_water-point.parquet"
df = sd.read_parquet(url)

# For polars
pl.from_arrow(df)

# For duckdb you can just treat 'df' like a table as of DuckDB 1.5.0
duckdb.sql("SELECT * FROM df LIMIT 5")
```

We should at least document this...I'm not personally keen on `.to_polars()` and `.to_duckdb()` but they are also not hard to implement.

> R: Direct File Ingestion (GDAL/OGR)

It's not quite as good as Python's `sd.read_pyogrio()`, but I did implement `sd_read_sf()` which takes care of many use cases.

```r
# install.packages(
#   "sedonadb",
#   repos = c("https://apache.r-universe.dev", "https://cloud.r-project.org")
# )

library(sedonadb)

tf <- tempfile(fileext = ".fgb")
curl::curl_download(
  "https://github.com/geoarrow/geoarrow-data/releases/download/v0.2.0/ns-water_elevation.fgb",
  tf
)

system.time(sedonadb::sd_read_sf(tf))
#>    user  system elapsed 
#>   1.126   0.336   1.658

system.time(sf::read_sf(tf))
#>    user  system elapsed 
#>  47.515   3.244  51.532
```

> Roadmap: Complex Linestring Operations

We don't have `ST_Subdivide()` yet but we do have `ST_LineMerge()` as of the forthcoming release!

```python
# pip install "apache-sedona[db]" --force-reinstall --pre --extra-index-url=https://pypi.fury.io/sedona-nightlies/
import sedona.db

sd = sedona.db.connect()
sd.sql("SELECT ST_LineMerge(ST_GeomFromWKT('MULTILINESTRING ((0 0, 1 0), (1 0, 1 1))')) AS g").show()
#> ┌─────────────────────────┐
#> │            g            │
#> │         geometry        │
#> ╞═════════════════════════╡
#> │ LINESTRING(0 0,1 0,1 1) │
#> └─────────────────────────┘
```

> I opened a specific issue:

I think we got this one solved!


GitHub link: https://github.com/apache/sedona/discussions/2576#discussioncomment-15958979

----
This is an automatically sent email for issues@sedona.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@sedona.apache.org


Re: [D] Observations from R and Python benchmarks: performance bottlenecks and optimization ideas for sedona-db [sedona]

Posted by "Robinlovelace (via GitHub)" <gi...@apache.org>.
GitHub user Robinlovelace added a comment to the discussion: Observations from R and Python benchmarks: performance bottlenecks and optimization ideas for sedona-db

Awesome, thanks for the updates and keep up the great work!

GitHub link: https://github.com/apache/sedona/discussions/2576#discussioncomment-15975167

----
This is an automatically sent email for issues@sedona.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@sedona.apache.org


Re: [D] Observations from R and Python benchmarks: performance bottlenecks and optimization ideas for sedona-db [sedona]

Posted by "Imbruced (via GitHub)" <gi...@apache.org>.
GitHub user Imbruced added a comment to the discussion: Observations from R and Python benchmarks: performance bottlenecks and optimization ideas for sedona-db

1. Did you call to_pandas only, or did you perform some operations on top of the result? The to_pandas method hardly relies on the 
GeoPandas from the arrow method. I assume that constructing shapely objects from wkb takes most of the time in this method. 
```python
GeoDataFrame.from_arrow
```

@Kontinuation, do you think there is a way to combine the C serde you wrote for Sedona and shapely conversions while ago? Do you think this even makes sense to do? 

I am not super familiar with Polars, but you mean to convert it to Polars or GeoPolars? I see that you did similar code to this.

```python
table = df.to_arrow_table()
polars_df = polars.from_arrow(table)
```

One thing that affected your benchmark is that, for SedondDB to pandas, you created shapely objects from the wkb in arrow, whereas for polars you just took the raw binary and did nothing with it. I am wondering if you could load it to geopolars maybe and do some operations later on it and with geopandas and compare the times?  

@Robinlovelace, by any chance, do you have some benchmarks on the SedonaDB UDFs? 




GitHub link: https://github.com/apache/sedona/discussions/2576#discussioncomment-15400494

----
This is an automatically sent email for issues@sedona.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@sedona.apache.org


Re: [D] Observations from R and Python benchmarks: performance bottlenecks and optimization ideas for sedona-db [sedona]

Posted by "paleolimbot (via GitHub)" <gi...@apache.org>.
GitHub user paleolimbot added a comment to the discussion: Observations from R and Python benchmarks: performance bottlenecks and optimization ideas for sedona-db

Very cool! Thank you for putting together these benchmarks! I think you're right about these bullets: 

- We can possibly make SedonaDB->geopandas faster
- We really do need to be able to read GDAL/OGR via R. I'll open a ticket for this and see if I can squeeze it in...it's more complicated than Python since we don't have pyogrio to help.
- We'd love to add ST_LineMerge and ST_Subdivide. These are "just" GEOS functions and have already been merged to georust/geos ( https://github.com/georust/geos/blob/47afbad2483e489911ddb456417808340e9342c3/src/geometry.rs#L2789-L2801 ). I'll open tickets for these.
- I'll fix the schema mismatch issue this week 🙂

Reading GeoPackages and converting outside the Arrow universe are always going to be slower than GeoParquet + staying inside the Arrow universe, and part of SedonaDB is strengthening those ecosystems to the point that those operations don't have to happen (i.e., we also want to make SedonaDB->geopandas/sf and reading .gpkg files unnecessary most of the time by making sure we support the next step).

If I'm reading these correctly, these benchmarks are of a `.gpkg` read(s), followed by some operation, with a collect back into various existing frameworks. I think the reason `sedonadb-sf` appears so fast is that you're using `sd_collect()`, which doesn't actually produce `sf` objects but something closer to a zero-copy ALTREP wrapper around the array (a `geoarrow_vctr`, to be precise). If you changed `sd_collect()` to `st_as_sf()` I think you'd see something more similar to `sedonadb-geopandas`.

I'm not sure why sedonadb-polars isn't identical to sedonadb-sf for the spatial_join benchmark (I would have expected those results to be identical). I think that geopandas caches the spatial index and I'm not sure you have a totally "fresh" GeoDataFrame for each iteration of your benchmark (or also, this might be a case where Python/R string handling shines over Arrow since there are a lot of repeated strings in the output). 16 Polygons x 100k points is pretty small and I'm pleased with the fact that SedonaDB doesn't add so much overhead that the performance on that sort of microbenchmark is reasonable.


> how I can best contribute to testing these high-performance paths!

Continuing to kick the tires and write about it is fantastic! Knowing that there's interest in SedonaDB for R is helpful (it's a bit of a side project for the other SedonaDB work I do and it's motivating if I know that anybody actually plans on using it 🙂 ). I'm not sure I will ever get to writing a GeocompX variant but in theory that's what we're trying to provide with SedonaDB and it's a great blueprint for stuff that SedonaDB should be able to do at some point.

GitHub link: https://github.com/apache/sedona/discussions/2576#discussioncomment-15402640

----
This is an automatically sent email for issues@sedona.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@sedona.apache.org


Re: [D] Observations from R and Python benchmarks: performance bottlenecks and optimization ideas for sedona-db [sedona]

Posted by "e-kotov (via GitHub)" <gi...@apache.org>.
GitHub user e-kotov added a comment to the discussion: Observations from R and Python benchmarks: performance bottlenecks and optimization ideas for sedona-db

While sedonaddb-r does not have a data-reader, I implemented a duckdb-based reader that passes the data over to SedonaDB via Arrow in https://github.com/e-kotov/sx


A script for benchmark:

```r
install.packages(
  'sedonadb',
  repos = c('https://apache.r-universe.dev', 'https://cloud.r-project.org')
)
pak::pak("e-kotov/sx")

library(sx)
library(sf)
library(dplyr)
library(microbenchmark)
library(spData)
library(sedonadb)

# --- Data Preparation ---
message("Preparing data...")
data(nz, package = "spData")
nz <- st_make_valid(nz)

# Check/Transform CRS (EPSG:2193)
if (st_crs(nz)$epsg != 2193) {
  nz <- st_transform(nz, 2193)
}

# Create temp files using our package helper
regions_path <- sx:::sx_create_temp_spatial_file(nz, ext = "gpkg")

n_points <- 100000
message("Generating ", n_points, " random points in NZ bbox...")
set.seed(42)
bbox_poly <- st_as_sfc(st_bbox(nz))
points_sfc <- st_sample(bbox_poly, size = n_points)
points_sf <- st_sf(geometry = points_sfc)
if (st_crs(points_sf)$epsg != 2193) {
  st_crs(points_sf) <- 2193
}

points_path <- sx:::sx_create_temp_spatial_file(points_sf, ext = "gpkg")

message("Starting SX Materialization Benchmarks...")

# Helper to log results
log_result <- function(operation, benchmark) {
  mean_sec <- mean(benchmark$time) / 1e9
  ops_per_sec <- 1 / mean_sec
  cat(sprintf("sx-package,R,%s,%.2f\n", operation, ops_per_sec))
}

# 1. Lazy Load (Disk -> Lazy View)
# sx_read stays lazy by default
message("\n--- Loading Data (Lazy) ---")
points <- sx_read(points_path, verbosity = "quiet")
regions <- sx_read(regions_path, view_name = "regions", verbosity = "quiet")

# 2. Buffer (1km)
message("\n--- Benchmark: Buffer (1km) ---")

# Scenario 1: Materialize to Sedona Table (Stay in DB)
message("Scenario 1: Materialize to SedonaDB Table")
bench_buffer_table <- microbenchmark(
  buffer_table = {
    x <- sx_buffer(points, dist = 1000, verbosity = "quiet") |>
      sedonadb::sd_compute() # force materialization
  },
  times = 5
)
print(bench_buffer_table)
log_result("buffer_table", bench_buffer_table)

# Scenario 2: Materialize to R (Collect to sf)
message("Scenario 2: Materialize to R (sf)")
bench_buffer_collect <- microbenchmark(
  buffer_collect = {
    x <- sx_buffer("points", dist = 1000, verbosity = "quiet") |>
      sx_collect(verbosity = "quiet")
  },
  times = 5
)
print(bench_buffer_collect)
log_result("buffer_collect", bench_buffer_collect)


# 3. Spatial Join (Intersects)
message("\n--- Benchmark: Spatial Join (Intersects) ---")

# Scenario 1: Materialize to Sedona Table
message("Scenario 1: Materialize to SedonaDB Table")
bench_sjoin_table <- microbenchmark(
  sjoin_table = {
    sx_join(
      points,
      regions,
      join = "intersects",
      left = TRUE,
      verbosity = "quiet"
    ) |>
      select(geom, Name) |>
      sedonadb::sd_compute() # force materialization
  },
  times = 5
)
print(bench_sjoin_table)
log_result("sjoin_table", bench_sjoin_table)

# Scenario 2: Materialize to R (Collect to sf)
message("Scenario 2: Materialize to R (sf)")
bench_sjoin_collect <- microbenchmark(
  sjoin_collect = {
    sx_join(
      points,
      regions,
      join = "intersects",
      left = TRUE,
      verbosity = "quiet"
    ) |>
      select(geom, Name) |>
      sx_collect(verbosity = "quiet")
  },
  times = 5
)
print(bench_sjoin_collect)
log_result("sjoin_collect", bench_sjoin_collect)

message("\nSX Materialization Benchmarks Complete.")

```

GitHub link: https://github.com/apache/sedona/discussions/2576#discussioncomment-15599883

----
This is an automatically sent email for issues@sedona.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@sedona.apache.org


Re: [D] Observations from R and Python benchmarks: performance bottlenecks and optimization ideas for sedona-db [sedona]

Posted by "Robinlovelace (via GitHub)" <gi...@apache.org>.
GitHub user Robinlovelace added a comment to the discussion: Observations from R and Python benchmarks: performance bottlenecks and optimization ideas for sedona-db

https://github.com/Robinlovelace/geobench/issues/2

GitHub link: https://github.com/apache/sedona/discussions/2576#discussioncomment-15413632

----
This is an automatically sent email for issues@sedona.apache.org.
To unsubscribe, please send an email to: issues-unsubscribe@sedona.apache.org