You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2021/11/15 09:47:06 UTC
[GitHub] [pinot] TheWinds opened a new issue #7766: H3 index bug
TheWinds opened a new issue #7766:
URL: https://github.com/apache/pinot/issues/7766
When using the H3 index, in some cases, the correct filtering of distances is not possible.
## Test case
### Data
#### geodata.csv
```csv
name,lng,lat
a,-103.34056886616187,20.63611463218247
b,-103.31104310932645,20.606231326034603
c,-103.33524736347661,20.595626162929634
```
#### schema
```json
{
"metricFieldSpecs": [],
"dimensionFieldSpecs": [
{
"dataType": "STRING",
"name": "name"
},
{
"dataType": "FLOAT",
"name": "lng"
},
{
"dataType": "FLOAT",
"name": "lat"
},
{
"dataType": "BYTES",
"name": "location_st_point",
"transformFunction": "toSphericalGeography(stPoint(lng,lat))"
}
],
"schemaName": "geodata"
}
```
#### table
```json
{
"tableName": "geodata",
"tableType": "OFFLINE",
"segmentsConfig": {
"retentionTimeUnit": "DAYS",
"retentionTimeValue": "1",
"segmentPushType": "APPEND",
"segmentAssignmentStrategy": "BalanceNumSegmentAssignmentStrategy",
"schemaName": "geodata",
"replication": "1"
},
"tenants": {
},
"fieldConfigList": [
{
"name": "location_st_point",
"encodingType": "RAW",
"indexType": "H3",
"properties": {
"resolutions": "7"
}
}
],
"tableIndexConfig": {
"loadMode": "MMAP",
"noDictionaryColumns": [
"location_st_point"
]
},
"metadata": {
"customConfigs": {
}
}
}
```
### Query
```sql
select name,
lat,
lng,
ST_DISTANCE(
location_st_point,
ST_Point(-103.34417375507813, 20.64061268636347, 1)
) as distance
from geodata
where ST_DISTANCE(
location_st_point,
ST_Point(-103.34417375507813, 20.64061268636347, 1)
) < 5000;
```
### Query Response
```csv
name,lat,lng,distance
a,20.636114,-103.34057,625.2019871997669
b,20.606232,-103.31104,5148.1557916677875
c,20.595627,-103.33525,5087.813601469894
```
## Analysis
**This appears to be a problem with KRing calculations***
### Code
https://github.com/apache/pinot/blob/e3d238ac1d8633331d9507713266e41e6b40f870/pinot-core/src/main/java/org/apache/pinot/core/operator/filter/H3IndexFilterOperator.java#L186-L198
### Visualize
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on issue #7766: H3 index bug
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on issue #7766:
URL: https://github.com/apache/pinot/issues/7766#issuecomment-969202151
Thanks for reporting this. What is the `edgeLength` for your H3 index? Based on the visualization, I think `(int) Math.floor((distance / _edgeLength - 2) / 1.7321)` should be `0` (distance should be less than `(2 + 1.7321) * _edgeLength`).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] TheWinds commented on issue #7766: H3 index bug
Posted by GitBox <gi...@apache.org>.
TheWinds commented on issue #7766:
URL: https://github.com/apache/pinot/issues/7766#issuecomment-969935558
> Thanks for reporting this. What is the `edgeLength` for your H3 index? Based on the visualization, I think `(int) Math.floor((distance / _edgeLength - 2) / 1.7321)` should be `0` (distance should be less than `(2 + 1.7321) * _edgeLength`).
The _edgeLength of `resolution 7` is `1220.629759`
```js
Math.floor(( 5000 / 1220.629759 - 2 ) / 1.7321) == 1
```
You can click the link to see the visualize result https://codepen.io/jsthewinds/pen/GRvwXoe
and the center point is dragable , drag it or modify config params to obverse it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on issue #7766: H3 index bug
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on issue #7766:
URL: https://github.com/apache/pinot/issues/7766#issuecomment-970795854
Adding @yupeng9
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on issue #7766: H3 index bug
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on issue #7766:
URL: https://github.com/apache/pinot/issues/7766#issuecomment-970794971
I somehow feel the `_edgeLength` is not accurate. Based on the visualization, the diameter of the circle should be less than `7 * _edgeLength`, but `10000 > 8 * 1220.629759`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] TheWinds edited a comment on issue #7766: H3 index bug
Posted by GitBox <gi...@apache.org>.
TheWinds edited a comment on issue #7766:
URL: https://github.com/apache/pinot/issues/7766#issuecomment-969935558
> Thanks for reporting this. What is the `edgeLength` for your H3 index? Based on the visualization, I think `(int) Math.floor((distance / _edgeLength - 2) / 1.7321)` should be `0` (distance should be less than `(2 + 1.7321) * _edgeLength`).
The _edgeLength of `resolution 7` is `1220.629759`
```js
Math.floor(( 5000 / 1220.629759 - 2 ) / 1.7321) == 1
```
You can click this link to see the visualize result https://codepen.io/jsthewinds/pen/GRvwXoe
and the center point is dragable , drag it or modify config params to obverse it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org