You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@sedona.apache.org by "zhangfengcdt (via GitHub)" <gi...@apache.org> on 2025/07/10 20:27:06 UTC
[PR] [GH-2083] Implement intersection Method for GeoPandas.SpatialIndex [sedona]
zhangfengcdt opened a new pull request, #2088:
URL: https://github.com/apache/sedona/pull/2088
## Did you read the Contributor Guide?
- Yes, I have read the [Contributor Rules](https://sedona.apache.org/latest/community/rule/) and [Contributor Development Guide](https://sedona.apache.org/latest/community/develop/)
## Is this PR related to a ticket?
- Yes, and the PR name follows the format `[GH-XXX] my subject`. Closes #<issue_number>
## What changes were proposed in this PR?
This PR implements intersection Method for GeoPandas.SpatialIndex.
## How was this patch tested?
TestSpatialIndex
## Did this PR include necessary documentation updates?
- Yes, I have updated the documentation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [GH-2083] Implement intersection Method for GeoPandas.SpatialIndex [sedona]
Posted by "zhangfengcdt (via GitHub)" <gi...@apache.org>.
zhangfengcdt commented on code in PR #2088:
URL: https://github.com/apache/sedona/pull/2088#discussion_r2201066692
##########
python/sedona/geopandas/sindex.py:
##########
@@ -196,7 +196,38 @@ def intersection(self, bounds):
list
List of indices of matching geometries.
"""
- raise NotImplementedError("This method is not implemented yet.")
+ log_advice(
+ "`intersection` returns local list of indices of matching geometries onto driver's memory. "
+ "It should only be used if the resulting collection is expected to be small."
+ )
+
+ if self.is_empty:
+ return []
+
+ # Create a polygon from the bounds
+ from shapely.geometry import box
+
+ bbox = box(*bounds)
+
+ if self._is_spark:
+ # For Spark-based spatial index
+ from sedona.spark.core.spatialOperator import RangeQuery
+
+ # Execute the spatial range query with the bounding box
+ result_rdd = RangeQuery.SpatialRangeQuery(
+ self._indexed_rdd, bbox, True, True
+ )
+
+ results = result_rdd.collect()
Review Comment:
The query(), intersection(), and nearest() all returns list in geopandas. This is different than the operators on GeoSeries / GeoDataframe for which it could return a new GeoSeries / GeoDataframe and chain another operators.
The sindex is basically an efficient way to "search" the spatial data instead of transform the data. So mostly of the cases, it should return to driver for the end users.
I might be misunderstanding the use of sindex though.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [GH-2083] Implement intersection Method for GeoPandas.SpatialIndex [sedona]
Posted by "jiayuasu (via GitHub)" <gi...@apache.org>.
jiayuasu commented on code in PR #2088:
URL: https://github.com/apache/sedona/pull/2088#discussion_r2201033614
##########
python/sedona/geopandas/sindex.py:
##########
@@ -196,7 +196,38 @@ def intersection(self, bounds):
list
List of indices of matching geometries.
"""
- raise NotImplementedError("This method is not implemented yet.")
+ log_advice(
+ "`intersection` returns local list of indices of matching geometries onto driver's memory. "
+ "It should only be used if the resulting collection is expected to be small."
+ )
+
+ if self.is_empty:
+ return []
+
+ # Create a polygon from the bounds
+ from shapely.geometry import box
+
+ bbox = box(*bounds)
+
+ if self._is_spark:
+ # For Spark-based spatial index
+ from sedona.spark.core.spatialOperator import RangeQuery
+
+ # Execute the spatial range query with the bounding box
+ result_rdd = RangeQuery.SpatialRangeQuery(
+ self._indexed_rdd, bbox, True, True
+ )
+
+ results = result_rdd.collect()
Review Comment:
Why do we use `collect` here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [GH-2083] Implement intersection Method for GeoPandas.SpatialIndex [sedona]
Posted by "jiayuasu (via GitHub)" <gi...@apache.org>.
jiayuasu commented on code in PR #2088:
URL: https://github.com/apache/sedona/pull/2088#discussion_r2201089257
##########
python/sedona/geopandas/sindex.py:
##########
@@ -196,7 +196,38 @@ def intersection(self, bounds):
list
List of indices of matching geometries.
"""
- raise NotImplementedError("This method is not implemented yet.")
+ log_advice(
+ "`intersection` returns local list of indices of matching geometries onto driver's memory. "
+ "It should only be used if the resulting collection is expected to be small."
+ )
+
+ if self.is_empty:
+ return []
+
+ # Create a polygon from the bounds
+ from shapely.geometry import box
+
+ bbox = box(*bounds)
+
+ if self._is_spark:
+ # For Spark-based spatial index
+ from sedona.spark.core.spatialOperator import RangeQuery
+
+ # Execute the spatial range query with the bounding box
+ result_rdd = RangeQuery.SpatialRangeQuery(
+ self._indexed_rdd, bbox, True, True
+ )
+
+ results = result_rdd.collect()
Review Comment:
make sense!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [GH-2083] Implement intersection Method for GeoPandas.SpatialIndex [sedona]
Posted by "zhangfengcdt (via GitHub)" <gi...@apache.org>.
zhangfengcdt commented on code in PR #2088:
URL: https://github.com/apache/sedona/pull/2088#discussion_r2201070096
##########
python/sedona/geopandas/sindex.py:
##########
@@ -196,7 +196,38 @@ def intersection(self, bounds):
list
List of indices of matching geometries.
"""
- raise NotImplementedError("This method is not implemented yet.")
+ log_advice(
+ "`intersection` returns local list of indices of matching geometries onto driver's memory. "
+ "It should only be used if the resulting collection is expected to be small."
+ )
+
+ if self.is_empty:
+ return []
+
+ # Create a polygon from the bounds
+ from shapely.geometry import box
+
+ bbox = box(*bounds)
+
+ if self._is_spark:
+ # For Spark-based spatial index
+ from sedona.spark.core.spatialOperator import RangeQuery
+
+ # Execute the spatial range query with the bounding box
+ result_rdd = RangeQuery.SpatialRangeQuery(
+ self._indexed_rdd, bbox, True, True
+ )
+
+ results = result_rdd.collect()
Review Comment:
I do have a warning in each of these method similar to the to_genpandas() method in GeoSeries / GeoDataframe when users want to collect the results back to geopandas df.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [GH-2083] Implement intersection Method for GeoPandas.SpatialIndex [sedona]
Posted by "jiayuasu (via GitHub)" <gi...@apache.org>.
jiayuasu commented on code in PR #2088:
URL: https://github.com/apache/sedona/pull/2088#discussion_r2201033614
##########
python/sedona/geopandas/sindex.py:
##########
@@ -196,7 +196,38 @@ def intersection(self, bounds):
list
List of indices of matching geometries.
"""
- raise NotImplementedError("This method is not implemented yet.")
+ log_advice(
+ "`intersection` returns local list of indices of matching geometries onto driver's memory. "
+ "It should only be used if the resulting collection is expected to be small."
+ )
+
+ if self.is_empty:
+ return []
+
+ # Create a polygon from the bounds
+ from shapely.geometry import box
+
+ bbox = box(*bounds)
+
+ if self._is_spark:
+ # For Spark-based spatial index
+ from sedona.spark.core.spatialOperator import RangeQuery
+
+ # Execute the spatial range query with the bounding box
+ result_rdd = RangeQuery.SpatialRangeQuery(
+ self._indexed_rdd, bbox, True, True
+ )
+
+ results = result_rdd.collect()
Review Comment:
Why do we use `collect` here? This will crash the driver memory
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [GH-2083] Implement intersection Method for GeoPandas.SpatialIndex [sedona]
Posted by "jiayuasu (via GitHub)" <gi...@apache.org>.
jiayuasu merged PR #2088:
URL: https://github.com/apache/sedona/pull/2088
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@sedona.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org