You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sdap.apache.org by GitBox <gi...@apache.org> on 2022/03/17 17:03:10 UTC
[GitHub] [incubator-sdap-nexus] kevinmarlis opened a new pull request #156: SDAP-318 Updated timeSeriesSpark algorithm to find tiles by bounding box instead of polygon
kevinmarlis opened a new pull request #156:
URL: https://github.com/apache/incubator-sdap-nexus/pull/156
Apache ticket: [SDAP-318](https://issues.apache.org/jira/browse/SDAP-318)
This updates the timeSeriesSpark algorithm to use the bounding box (instead of polygon) to perform spatial searching on Solr. This is in line with how the other algorithms do spatial searching.
**The issue**: Polygon search runs into an inherent issue with WKT strings where bounding boxes with > 180 difference between maxLon and minLon. This results in no tiles being returned from Solr, and an empty list being returned for the algorithm results.
**The fix**: Update algorithm to use [get_tiles_bounded_by_box](https://github.com/apache/incubator-sdap-nexus/blob/6583534206a1d9dbfb1eb1bada50af6f3e09396a/data-access/nexustiles/nexustiles.py#L260) instead of [get_tiles_bounded_by_polygon](https://github.com/apache/incubator-sdap-nexus/blob/6583534206a1d9dbfb1eb1bada50af6f3e09396a/data-access/nexustiles/nexustiles.py#L269). In `spark_driver` the list of nexus tiles has been updated to use the polygon itself instead of the polygon's WKT string representation. This allows for the use of the polygon's bounds for `get_tiles_bounded_by_box`, and is consistent with how other algorithms pass bounds.
The algorithm's test has also been updated to reflect this change:
```
(nexus) marlis@RAYL-C01360 incubator-sdap-nexus % pytest analysis/tests/algorithms_spark/test_timeseriesspark.py
========================================== test session starts ==========================================
platform darwin -- Python 3.7.10, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
rootdir: /Users/marlis/Developer/ECCO/SDAP/incubator-sdap-nexus/analysis
collected 5 items
analysis/tests/algorithms_spark/test_timeseriesspark.py ..... [100%]
===================================== 5 passed, 9 warnings in 6.42s =====================================
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@sdap.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-sdap-nexus] kevinmarlis commented on pull request #156: SDAP-318 Updated timeSeriesSpark algorithm to find tiles by bounding box instead of polygon
Posted by GitBox <gi...@apache.org>.
kevinmarlis commented on pull request #156:
URL: https://github.com/apache/incubator-sdap-nexus/pull/156#issuecomment-1071770069
I temporarily deployed a test web app image to AQACF with the code changes. I tested on a dataset known to fail the time series algorithm when using a large bounding box. I used bounds of -180, -90, 180, 90 and received results:
```
{
"meta": [
{
"shortName": "nasa-merra-2-0.5-hourly-global-CDH",
"bounds": {
"east": 180.0,
"west": -180.0,
"north": 90.0,
"south": -90.0
},
"time": {
"start": 1530532800,
"stop": 1531569600,
"iso_start": "2018-07-02T12:00:00+0000",
"iso_stop": "2018-07-14T12:00:00+0000"
}
}
],
"data": [
[
{
"min": 0.00021394588111434132,
"max": 0.11129041761159897,
"mean": 0.00703836853349493,
"cnt": 440,
"std": 0.01253395141553278,
"time": 1530534600,
"iso_time": "2018-07-02T12:30:00+0000",
"ds": 0
}
], ...
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@sdap.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-sdap-nexus] kevinmarlis commented on pull request #156: SDAP-318 Updated timeSeriesSpark algorithm to find tiles by bounding box instead of polygon
Posted by GitBox <gi...@apache.org>.
kevinmarlis commented on pull request #156:
URL: https://github.com/apache/incubator-sdap-nexus/pull/156#issuecomment-1071770069
I temporarily deployed a test web app image to AQACF with the code changes. I tested on a dataset known to fail the time series algorithm when using a large bounding box. I used bounds of -180, -90, 180, 90 and received results:
```
{
"meta": [
{
"shortName": "nasa-merra-2-0.5-hourly-global-CDH",
"bounds": {
"east": 180.0,
"west": -180.0,
"north": 90.0,
"south": -90.0
},
"time": {
"start": 1530532800,
"stop": 1531569600,
"iso_start": "2018-07-02T12:00:00+0000",
"iso_stop": "2018-07-14T12:00:00+0000"
}
}
],
"data": [
[
{
"min": 0.00021394588111434132,
"max": 0.11129041761159897,
"mean": 0.00703836853349493,
"cnt": 440,
"std": 0.01253395141553278,
"time": 1530534600,
"iso_time": "2018-07-02T12:30:00+0000",
"ds": 0
}
], ...
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@sdap.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org