You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@sedona.apache.org by "petern48 (via GitHub)" <gi...@apache.org> on 2025/11/08 23:54:07 UTC
[PR] feat(c/sedona-geos): Implement ST_Reverse using `geos` [sedona-db]
petern48 opened a new pull request, #288:
URL: https://github.com/apache/sedona-db/pull/288
Building a step-by-step walkthrough of how I go about implementing a new `geos` function
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(c/sedona-geos): Implement ST_Reverse using `geos` [sedona-db]
Posted by "petern48 (via GitHub)" <gi...@apache.org>.
petern48 commented on PR #288:
URL: https://github.com/apache/sedona-db/pull/288#issuecomment-3507201280
Now, this file isn't actually known to the project yet. In fact, my IDE (Cursor/VS Code) and its Rust extension is warning me with white swiggly lines to tell me exactly that.
<img width="451" height="150" alt="image" src="https://github.com/user-attachments/assets/cb70fda9-21e5-40c3-9bf2-64ca5b3d67e0" />
To fix this, I add this new file to the corresponding `lib.rs` inside of the `c/sedona-geos/src` directory. Click the next commit to see the change. After which, the white lines are gone.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(c/sedona-geos): Implement ST_Reverse using `geos` [sedona-db]
Posted by "petern48 (via GitHub)" <gi...@apache.org>.
petern48 commented on code in PR #288:
URL: https://github.com/apache/sedona-db/pull/288#discussion_r2507384212
##########
c/sedona-geos/src/st_reverse.rs:
##########
@@ -0,0 +1,124 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+use std::sync::Arc;
+
+use arrow_array::builder::BinaryBuilder;
+use datafusion_common::{error::Result, DataFusionError};
+use datafusion_expr::ColumnarValue;
+use geos::Geom;
+use sedona_expr::scalar_udf::{ScalarKernelRef, SedonaScalarKernel};
+use sedona_geometry::wkb_factory::WKB_MIN_PROBABLE_BYTES;
+use sedona_schema::{
+ datatypes::{SedonaType, WKB_GEOMETRY},
+ matchers::ArgMatcher,
+};
+
+use crate::executor::GeosExecutor;
+
+/// ST_Reverse() implementation using the geos crate
+pub fn st_reverse_impl() -> ScalarKernelRef {
+ Arc::new(STReverse {})
+}
+
+#[derive(Debug)]
+struct STReverse {}
+
+impl SedonaScalarKernel for STReverse {
+ fn return_type(&self, args: &[SedonaType]) -> Result<Option<SedonaType>> {
+ let matcher = ArgMatcher::new(vec![ArgMatcher::is_geometry()], WKB_GEOMETRY);
+
+ matcher.match_args(args)
+ }
+
+ fn invoke_batch(
+ &self,
+ arg_types: &[SedonaType],
+ args: &[ColumnarValue],
+ ) -> Result<ColumnarValue> {
+ let executor = GeosExecutor::new(arg_types, args);
+ let mut builder = BinaryBuilder::with_capacity(
+ executor.num_iterations(),
+ WKB_MIN_PROBABLE_BYTES * executor.num_iterations(),
+ );
+ executor.execute_wkb_void(|maybe_wkb| {
+ match maybe_wkb {
+ Some(wkb) => {
+ invoke_scalar(&wkb, &mut builder)?;
+ builder.append_value([]);
+ }
+ _ => builder.append_null(),
+ }
+
+ Ok(())
+ })?;
+
+ executor.finish(Arc::new(builder.finish()))
+ }
+}
+
+fn invoke_scalar(geos_geom: &geos::Geometry, writer: &mut impl std::io::Write) -> Result<()> {
+ let geometry = geos_geom
+ .get_centroid()
+ .map_err(|e| DataFusionError::Execution(format!("Failed to calculate centroid: {e}")))?;
+
Review Comment:
Alright, **here's the main change of the entire PR**. `invoke_scalar` here is called above to perform the actual operation on each of the inputs rows. It still uses `get_centroid`. I need to figure out how to perform the reverse operation. Because of VS Code's great Rust extension, I can `Ctrl + click` the `get_centroid()` function and my IDE will automatically open the library file this function is located in, which is [here](https://github.com/georust/geos/blob/47afbad2483e489911ddb456417808340e9342c3/src/geometry.rs#L905). If your IDE doesn't do that you can use that link above to go to the file. Now I `Ctrl + f` and search for a reverse function. I find it [here](https://github.com/georust/geos/blob/47afbad2483e489911ddb456417808340e9342c3/src/geometry.rs#L2829). Hooray! there's an easy method that exists that will do all the work for me. In the next commit, I change the `.get_centroid()` call to `.reverse()`.
Notice: STCentroid's function wasn't called `.centroid()`, it was called `.get_centroid()`. Keep that in mind when you're searching for the function: it might not be named exactly what you'd guess. It will likely at least be in `snake_case` rather than `CamelCase`.
**NOTE**: If you don't find a convenient function (which is entirely possible because `geos` Rust might not have been implemented yet), you should probably STOP and try a different function. Unless you're particularly interested in that function, maybe try asking for help.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(c/sedona-geos): Implement ST_Reverse using `geos` [sedona-db]
Posted by "petern48 (via GitHub)" <gi...@apache.org>.
petern48 commented on PR #288:
URL: https://github.com/apache/sedona-db/pull/288#issuecomment-3507272685
The last step is to write Python integration tests. Here I'm trying to be more comprehensive than the rust tests, covering cases that I think are edge cases. Don't worry too much if you don't know what good tests are. Just write some, push it to your PR, and us reviewers can help you here during code review. Adjust the tests until they pass. Look at surrounding code and potentially similar functions for inspiration of how to write your tests.
To run the tests, I followed the [python instructions](https://github.com/apache/sedona-db/blob/94ce7db5287b39218b803eae057a4eb92a50c397/docs/contributors-guide.md#python) in the contributor guide and run this pytest command to test my exact function. It's based on the file name and test function name.
```
pytest tests/functions/test_functions.py::test_st_reverse
```
If I make changes to rust code, I run this command to update the python environment with the new changes. THIS IS IMPORTANT. If you don't run this command after making change to your rust implementation, the changes won't reflect in python.
```
maturin develop
```
Both of these commands assume you're in the `python/sedonadb/` directory.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(c/sedona-geos): Implement ST_Reverse using `geos` [sedona-db]
Posted by "petern48 (via GitHub)" <gi...@apache.org>.
petern48 commented on PR #288:
URL: https://github.com/apache/sedona-db/pull/288#issuecomment-3507287349
Once I'm done with my code, I run `cargo clippy` locally to see if there are any minor issues with my code (e.g unused imports), CI will fail and remind you otherwise. I didn't see any, so it looks like I'm good to go.
At this point, I just wait for someone to review my code. Please know, it is totally normal to get a review that suggests some changes. It's typical if you're new to the project, and that's okay! It's how you grow and learn from the reviewer. Hope this helps some people learn enough to contribute their first PR, and grow into contributing more.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(c/sedona-geos): Implement ST_Reverse using `geos` [sedona-db]
Posted by "petern48 (via GitHub)" <gi...@apache.org>.
petern48 commented on PR #288:
URL: https://github.com/apache/sedona-db/pull/288#issuecomment-3507179845
Alright, so first, I pick a function from the corresponding `geos` issue [here](https://github.com/apache/sedona-db/issues/224). In this case, I picked `ST_Reverse`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(c/sedona-geos): Implement ST_Reverse using `geos` [sedona-db]
Posted by "petern48 (via GitHub)" <gi...@apache.org>.
petern48 commented on PR #288:
URL: https://github.com/apache/sedona-db/pull/288#issuecomment-3507193414
The first thing I'm going to do is check the corresponding PostGIS docs for ST_Reverse [here](https://postgis.net/docs/ST_Reverse.html), since this is the behavior I want to follow. I found this by simply searching "ST_Reverse PostGIS" on Google.
Looking at the top, we see the function signature.
<img width="300" height="35" alt="image" src="https://github.com/user-attachments/assets/ec77266e-04c3-46ba-a4ed-ca5377c09cba" />
`geometry ST_Reverse(geometry g1);`
The first "geometry" indicates the return type, the second "geometry" indicates the arguments to the function. So in whole, this function takes a single geometry as input and outputs a geometry. Now the best way to go about implementing this whole thing is to find a similar function to model it as. Luckily, I noticed ST_Centroid has the same argument and return types (`geometry ST_Centroid(geometry g1);`), and it's already implemented in SedonaDB here [st_centroid.rs](https://github.com/apache/sedona-db/blob/94ce7db5287b39218b803eae057a4eb92a50c397/c/sedona-geos/src/st_centroid.rs). Now I'll create my new `st_reverse.rs` file in the location `c/sedona-geos/src/` along with the other geos functions. AND I'm going to **COPY-PASTE** the contents of [st_centroid.rs](https://github.com/apache/sedona-db/blob/94ce7db5287b39218b803eae057a4eb92a50c397/c/sedona-geos/src/st_centroid.rs) into it because I know most of the code will be the same. Click on the following commit to see it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(c/sedona-geos): Implement ST_Reverse using `geos` [sedona-db]
Posted by "petern48 (via GitHub)" <gi...@apache.org>.
petern48 commented on code in PR #288:
URL: https://github.com/apache/sedona-db/pull/288#discussion_r2507377613
##########
c/sedona-geos/src/lib.rs:
##########
@@ -31,6 +31,7 @@ mod st_isvalid;
mod st_isvalidreason;
mod st_length;
mod st_perimeter;
+mod st_reverse;
mod st_simplifypreservetopology;
Review Comment:
Also note, the convention is to put these in alphabetical order. Hence why it's not just at the bottom.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(c/sedona-geos): Implement ST_Reverse using `geos` [sedona-db]
Posted by "petern48 (via GitHub)" <gi...@apache.org>.
petern48 merged PR #288:
URL: https://github.com/apache/sedona-db/pull/288
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(c/sedona-geos): Implement ST_Reverse using `geos` [sedona-db]
Posted by "petern48 (via GitHub)" <gi...@apache.org>.
petern48 commented on code in PR #288:
URL: https://github.com/apache/sedona-db/pull/288#discussion_r2507393504
##########
c/sedona-geos/src/st_reverse.rs:
##########
@@ -104,19 +104,29 @@ mod tests {
let result = tester
.invoke_scalar("POLYGON ((0 0, 0 1, 1 1, 1 0, 0 0))")
.unwrap();
- tester.assert_scalar_result_equals(result, "POINT (0.5 0.5)");
+ tester.assert_scalar_result_equals(result, "POLYGON ((0 0, 1 0, 1 1, 0 1, 0 0))");
let result = tester.invoke_scalar(ScalarValue::Null).unwrap();
assert!(result.is_null());
let input_wkt = vec![
Some("POLYGON ((2 2, 2 3, 3 3, 3 2, 2 2))"),
Some("POINT EMPTY"),
+ Some("POINT (1 2)"),
+ Some("LINESTRING (1 2, 1 10)"),
+ Some("GEOMETRYCOLLECTION (MULTIPOINT (3 4, 1 2, 7 8, 5 6), LINESTRING (1 10, 1 2))"),
None,
];
let expected = create_array(
- &[Some("POINT (2.5 2.5)"), Some("POINT EMPTY"), None],
+ &[
+ Some("POLYGON ((2 2, 3 2, 3 3, 2 3, 2 2))"),
Review Comment:
In the last commit, I implemented the actual rust tests for ST_Reverse, since previously the tests were left over from when we copy-pasted from ST_Centroid.
Here, the top variable represents the inputs and the bottom represents the expected output. e.g Reading the tests, this means inputting the polygon at line 113 (the first geom of the input array) is expected to return the polygon at 123. This makes sense since the expected result is the same as the input except with it's xy, coords are reversed.
I'm testing this using `cargo test`. I keep going until they pass, and I feel like I've written a few decent cases.
The rest of this is **optional**, but a useful tool to check:
Oftentimes, I'm running queries against a PostGIS docker container to check what the expected results are for this function (e.g testing edge cases). I'm spinning up a PostGIS docker container using
```
docker compose up -d
```
And then enter the PostGIS shell (where I can run SQL queries) with this. There's a pretty high chance it's the exact same command for you.
```
docker exec -it sedona-db-postgis-1 psql -U postgres
```
Now I try some SQL queries (e.g this is the one for that first test case I was talking about above.
```sql
-- next line is my Spatial SQL query
SELECT ST_AsText(ST_Reverse(ST_GeomFromText('POLYGON ((2 2, 2 3, 3 3, 3 2, 2 2))')));
-- Below is the output
st_astext
--------------------------------
POLYGON((2 2,3 2,3 3,2 3,2 2))
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(c/sedona-geos): Implement ST_Reverse using `geos` [sedona-db]
Posted by "petern48 (via GitHub)" <gi...@apache.org>.
petern48 commented on code in PR #288:
URL: https://github.com/apache/sedona-db/pull/288#discussion_r2507393504
##########
c/sedona-geos/src/st_reverse.rs:
##########
@@ -104,19 +104,29 @@ mod tests {
let result = tester
.invoke_scalar("POLYGON ((0 0, 0 1, 1 1, 1 0, 0 0))")
.unwrap();
- tester.assert_scalar_result_equals(result, "POINT (0.5 0.5)");
+ tester.assert_scalar_result_equals(result, "POLYGON ((0 0, 1 0, 1 1, 0 1, 0 0))");
let result = tester.invoke_scalar(ScalarValue::Null).unwrap();
assert!(result.is_null());
let input_wkt = vec![
Some("POLYGON ((2 2, 2 3, 3 3, 3 2, 2 2))"),
Some("POINT EMPTY"),
+ Some("POINT (1 2)"),
+ Some("LINESTRING (1 2, 1 10)"),
+ Some("GEOMETRYCOLLECTION (MULTIPOINT (3 4, 1 2, 7 8, 5 6), LINESTRING (1 10, 1 2))"),
None,
];
let expected = create_array(
- &[Some("POINT (2.5 2.5)"), Some("POINT EMPTY"), None],
+ &[
+ Some("POLYGON ((2 2, 3 2, 3 3, 2 3, 2 2))"),
Review Comment:
In the last commit, I implemented the actual tests for ST_Reverse, since previously the tests were left over from when we copy-pasted from ST_Centroid.
Here, the top variable represents the inputs and the bottom represents the expected output. e.g Reading the tests, this means inputting the polygon at line 113 (the first geom of the input array) is expected to return the polygon at 123. This makes sense since the expected result is the same as the input except with it's xy, coords are reversed.
I'm testing this using `cargo test`. I keep going until they pass, and I feel like I've written a few decent cases.
Optional but useful:
Oftentimes, I'm running queries against a PostGIS docker container to check what the expected results are for this function (e.g testing edge cases). I'm spinning up a PostGIS docker container using
```
docker compose up -d
```
And then enter the PostGIS shell (where I can run SQL queries) with this. Extremely high chance it's the exact same command for you.
```
docker exec -it sedona-db-postgis-1 psql -U postgres
```
```sql
-- next line is my Spatial SQL query
SELECT ST_AsText(ST_Reverse(ST_GeomFromText('POLYGON ((2 2, 2 3, 3 3, 3 2, 2 2))')));
-- Below is the output
st_astext
--------------------------------
POLYGON((2 2,3 2,3 3,2 3,2 2))
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(c/sedona-geos): Implement ST_Reverse using `geos` [sedona-db]
Posted by "petern48 (via GitHub)" <gi...@apache.org>.
petern48 commented on PR #288:
URL: https://github.com/apache/sedona-db/pull/288#issuecomment-3507261421
Now our function exists, and the tests pass, but if you actually try it out, you'll see that the function doesn't exist because it's not registered yet.
A convenient way we can try it out is to run SedonaDB's CLI:
```bash
# cd back to the root directory, since I'm still in `c/sedona-geos/src`
cd ../../..
# cd into the cli directory
cd sedona-cli
# Run the CLI
cargo run
```
Now I run the same query I did in PostGIS:
```sql
Sedona CLI v0.2.0
> SELECT ST_AsText(ST_Reverse(ST_GeomFromText('POLYGON ((2 2, 2 3, 3 3, 3 2, 2 2))')));
-- The output: This is a sign our function isn't registered.
Error during planning: Invalid function 'st_reverse'.
Did you mean 'list_reverse'?
```
Registering is simple, just import and add the function to the [register.rs](https://github.com/apache/sedona-db/blob/94ce7db5287b39218b803eae057a4eb92a50c397/c/sedona-geos/src/register.rs#L4) file in `c/sedona-geos/src/`. See the next commit. Afterwards, running the query in the CLI works, and the warnings are gone. This is also needed for the function to work in Python.
Small note: I've been getting small warnings about my code not being used this whole time up until now (e.g see below). Registering the function fixes this too.
<img width="352" height="66" alt="image" src="https://github.com/user-attachments/assets/193dfd34-ce6c-4457-b110-62eabf7b21c2" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(c/sedona-geos): Implement ST_Reverse using `geos` [sedona-db]
Posted by "petern48 (via GitHub)" <gi...@apache.org>.
petern48 commented on PR #288:
URL: https://github.com/apache/sedona-db/pull/288#issuecomment-3507215451
In this next commit, I simply change most all "st_centroid" to "st_reverse". Though I haven't yet touched the `invoke_scalar` function, since that's where the core logic is.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org