You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2020/12/02 06:30:32 UTC

[GitHub] [incubator-pinot] yupeng9 opened a new pull request #6306: some geo function improvements

yupeng9 opened a new pull request #6306:
URL: https://github.com/apache/incubator-pinot/pull/6306

## Description
- Add a third parameter to `ST_Point` to create the point in geography, this simplifies the geographic point creation. Instead of `ST_GeogFromText(ST_AsText(ST_Point(group_lat, group_lon))`, we can now use `ST_Point(group_lat, group_lon, 1)`.
- Support `ST_Point` creation from literal such as `ST_Point(44.183189,-115.761905)`
- Swapped latitude, longitude for `ST_Distance` calculation for geography.
, etc.
## Upgrade Notes

Does this PR otherwise need attention when creating release notes? Things to consider:
- The signature of `ST_Point` is changed to take an optional third parameter. 0 (default) means geometry, 1 means geography
- In `ST_Distance(ST_Point( y1,x1), ST_Point(y2, x2))` is changed to `ST_Distance(ST_Point(x1, y1), ST_Point(x1, y2))` for geography

## Release Notes
The sequence of latitude, longitude is changed for the points in ST_Distance

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [incubator-pinot] codecov-io commented on pull request #6306: some geo function improvements

Posted by GitBox <gi...@apache.org>.

codecov-io commented on pull request #6306:
URL: https://github.com/apache/incubator-pinot/pull/6306#issuecomment-737522439


   # [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6306?src=pr&el=h1) Report
   > Merging [#6306](https://codecov.io/gh/apache/incubator-pinot/pull/6306?src=pr&el=desc) (1d73550) into [master](https://codecov.io/gh/apache/incubator-pinot/commit/1beaab59b73f26c4e35f3b9bc856b03806cddf5a?el=desc) (1beaab5) will **decrease** coverage by `20.89%`.
   > The diff coverage is `51.88%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-pinot/pull/6306/graphs/tree.svg?width=650&height=150&src=pr&token=4ibza2ugkz)](https://codecov.io/gh/apache/incubator-pinot/pull/6306?src=pr&el=tree)
   
   ```diff
   @@             Coverage Diff             @@
   ##           master    #6306       +/-   ##
   ===========================================
   - Coverage   66.44%   45.55%   -20.90%     
   ===========================================
     Files        1075     1281      +206     
     Lines       54773    61789     +7016     
     Branches     8168     8979      +811     
   ===========================================
   - Hits        36396    28150     -8246     
   - Misses      15700    31332    +15632     
   + Partials     2677     2307      -370     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | integration | `45.55% <51.88%> (?)` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-pinot/pull/6306?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...ot/broker/broker/AllowAllAccessControlFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6306/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL0FsbG93QWxsQWNjZXNzQ29udHJvbEZhY3RvcnkuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [.../helix/BrokerUserDefinedMessageHandlerFactory.java](https://codecov.io/gh/apache/incubator-pinot/pull/6306/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL2hlbGl4L0Jyb2tlclVzZXJEZWZpbmVkTWVzc2FnZUhhbmRsZXJGYWN0b3J5LmphdmE=) | `52.83% <0.00%> (-13.84%)` | :arrow_down: |
   | [...org/apache/pinot/broker/queryquota/HitCounter.java](https://codecov.io/gh/apache/incubator-pinot/pull/6306/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcXVlcnlxdW90YS9IaXRDb3VudGVyLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
   | [...che/pinot/broker/queryquota/MaxHitRateTracker.java](https://codecov.io/gh/apache/incubator-pinot/pull/6306/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcXVlcnlxdW90YS9NYXhIaXRSYXRlVHJhY2tlci5qYXZh) | `0.00% <0.00%> (ø)` | |
   | [...ache/pinot/broker/queryquota/QueryQuotaEntity.java](https://codecov.io/gh/apache/incubator-pinot/pull/6306/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcXVlcnlxdW90YS9RdWVyeVF1b3RhRW50aXR5LmphdmE=) | `0.00% <0.00%> (-50.00%)` | :arrow_down: |
   | [...ker/routing/instanceselector/InstanceSelector.java](https://codecov.io/gh/apache/incubator-pinot/pull/6306/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcm91dGluZy9pbnN0YW5jZXNlbGVjdG9yL0luc3RhbmNlU2VsZWN0b3IuamF2YQ==) | `100.00% <ø> (ø)` | |
   | [...ceselector/StrictReplicaGroupInstanceSelector.java](https://codecov.io/gh/apache/incubator-pinot/pull/6306/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvcm91dGluZy9pbnN0YW5jZXNlbGVjdG9yL1N0cmljdFJlcGxpY2FHcm91cEluc3RhbmNlU2VsZWN0b3IuamF2YQ==) | `0.00% <0.00%> (ø)` | |
   | [.../main/java/org/apache/pinot/client/Connection.java](https://codecov.io/gh/apache/incubator-pinot/pull/6306/diff?src=pr&el=tree#diff-cGlub3QtY2xpZW50cy9waW5vdC1qYXZhLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY2xpZW50L0Nvbm5lY3Rpb24uamF2YQ==) | `22.22% <0.00%> (-26.62%)` | :arrow_down: |
   | [...not/common/assignment/InstancePartitionsUtils.java](https://codecov.io/gh/apache/incubator-pinot/pull/6306/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vYXNzaWdubWVudC9JbnN0YW5jZVBhcnRpdGlvbnNVdGlscy5qYXZh) | `64.28% <ø> (-8.89%)` | :arrow_down: |
   | [.../apache/pinot/common/exception/QueryException.java](https://codecov.io/gh/apache/incubator-pinot/pull/6306/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vZXhjZXB0aW9uL1F1ZXJ5RXhjZXB0aW9uLmphdmE=) | `90.27% <ø> (+5.55%)` | :arrow_up: |
   | ... and [1278 more](https://codecov.io/gh/apache/incubator-pinot/pull/6306/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6306?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6306?src=pr&el=footer). Last update [0565f86...292e538](https://codecov.io/gh/apache/incubator-pinot/pull/6306?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [incubator-pinot] yupeng9 commented on pull request #6306: some geo function improvements

Posted by GitBox <gi...@apache.org>.

yupeng9 commented on pull request #6306:
URL: https://github.com/apache/incubator-pinot/pull/6306#issuecomment-737030984


   @fx19880617 Thanks for taking a look. Comments addressed.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [incubator-pinot] yupeng9 commented on a change in pull request #6306: some geo function improvements

Posted by GitBox <gi...@apache.org>.

yupeng9 commented on a change in pull request #6306:
URL: https://github.com/apache/incubator-pinot/pull/6306#discussion_r533931936



##########
File path: pinot-tools/src/main/resources/examples/stream/meetupRsvp/meetupRsvp_realtime_table_config.json
##########
@@ -7,7 +7,9 @@
     "segmentPushType": "APPEND",
     "segmentAssignmentStrategy": "BalanceNumSegmentAssignmentStrategy",
     "schemaName": "meetupRsvp",
-    "replication": "1"
+    "replication": "1",

Review comment:
       adding retentionTimeUnit is helpful. I was annoyed by the NPE thrown by the RetentionManager when debugging the quickStart.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #6306: some geo function improvements

Posted by GitBox <gi...@apache.org>.

Jackie-Jiang commented on a change in pull request #6306:
URL: https://github.com/apache/incubator-pinot/pull/6306#discussion_r534478456



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/geospatial/transform/function/ScalarFunctions.java
##########
@@ -43,6 +44,23 @@
         .serialize(GeometryUtils.GEOMETRY_FACTORY.createPoint(new Coordinate(x, y)));
   }
 
+  /**
+   * Creates a point.
+   *
+   * @param x x
+   * @param y y
+   * @param isGeography if it's geography
+   * @return the created point
+   */
+  @ScalarFunction
+  public static byte[] stPoint(double x, double y, double isGeography) {

Review comment:
       The parser parses the value based on the method signature. If you change the third argument to `int`, it should be able to parse `1` to `int`




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #6306: some geo function improvements

Posted by GitBox <gi...@apache.org>.

Jackie-Jiang commented on a change in pull request #6306:
URL: https://github.com/apache/incubator-pinot/pull/6306#discussion_r534396822



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/geospatial/transform/function/ScalarFunctions.java
##########
@@ -43,6 +44,23 @@
         .serialize(GeometryUtils.GEOMETRY_FACTORY.createPoint(new Coordinate(x, y)));
   }
 
+  /**
+   * Creates a point.
+   *
+   * @param x x
+   * @param y y
+   * @param isGeography if it's geography
+   * @return the created point
+   */
+  @ScalarFunction
+  public static byte[] stPoint(double x, double y, double isGeography) {

Review comment:
       Does `boolean` work as argument? If not, `int` might be a better option?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [incubator-pinot] fx19880617 commented on a change in pull request #6306: some geo function improvements

Posted by GitBox <gi...@apache.org>.

fx19880617 commented on a change in pull request #6306:
URL: https://github.com/apache/incubator-pinot/pull/6306#discussion_r533928005



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/LiteralTransformFunction.java
##########
@@ -145,7 +147,11 @@ public Dictionary getDictionary() {
 
   @Override
   public byte[][] transformToBytesValuesSV(ProjectionBlock projectionBlock) {
-    throw new UnsupportedOperationException();
+    if(_bytesResult==null) {

Review comment:
       missing space 
   ```
   if (_bytesResult == null) {
   ```

##########
File path: pinot-tools/src/main/resources/examples/stream/meetupRsvp/meetupRsvp_realtime_table_config.json
##########
@@ -7,7 +7,9 @@
     "segmentPushType": "APPEND",
     "segmentAssignmentStrategy": "BalanceNumSegmentAssignmentStrategy",
     "schemaName": "meetupRsvp",
-    "replication": "1"
+    "replication": "1",

Review comment:
       no need to change this file?

##########
File path: pinot-core/src/main/java/org/apache/pinot/core/geospatial/transform/function/StDistanceFunction.java
##########
@@ -126,7 +127,8 @@ private static double sphericalDistance(Geometry leftGeometry, Geometry rightGeo
     Point leftPoint = (Point) leftGeometry;
     Point rightPoint = (Point) rightGeometry;
 
-    return greatCircleDistance(leftPoint.getY(), leftPoint.getX(), rightPoint.getY(), rightPoint.getX());
+//    return greatCircleDistance(leftPoint.getY(), leftPoint.getX(), rightPoint.getY(), rightPoint.getX());

Review comment:
       remove the comment?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [incubator-pinot] yupeng9 commented on a change in pull request #6306: some geo function improvements

Posted by GitBox <gi...@apache.org>.

yupeng9 commented on a change in pull request #6306:
URL: https://github.com/apache/incubator-pinot/pull/6306#discussion_r534439056



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/geospatial/transform/function/ScalarFunctions.java
##########
@@ -43,6 +44,23 @@
         .serialize(GeometryUtils.GEOMETRY_FACTORY.createPoint(new Coordinate(x, y)));
   }
 
+  /**
+   * Creates a point.
+   *
+   * @param x x
+   * @param y y
+   * @param isGeography if it's geography
+   * @return the created point
+   */
+  @ScalarFunction
+  public static byte[] stPoint(double x, double y, double isGeography) {

Review comment:
       Unfortunately it does not.... The PQL parser will invoke this scalar function and pass in the constant. Since boolean is not available in PQL/SQL, we need to use a number. Also, somehow the parser parses `1` into a double type, any idea on that? 




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [incubator-pinot] Jackie-Jiang merged pull request #6306: some geo function improvements

Posted by GitBox <gi...@apache.org>.

Jackie-Jiang merged pull request #6306:
URL: https://github.com/apache/incubator-pinot/pull/6306


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org