You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2020/01/24 08:32:16 UTC
[GitHub] [incubator-pinot] fx19880617 opened a new pull request #5014:
fixing converting PinotQuery to BrokerRequest for group by clause with
selections
fx19880617 opened a new pull request #5014: fixing converting PinotQuery to BrokerRequest for group by clause with selections
URL: https://github.com/apache/incubator-pinot/pull/5014
For Aggregation Group by query, we don't set selection field in BrokerRequest.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] fx19880617 commented on a change in pull request
#5014: fixing converting PinotQuery to BrokerRequest for group by clause
with selections
Posted by GitBox <gi...@apache.org>.
fx19880617 commented on a change in pull request #5014: fixing converting PinotQuery to BrokerRequest for group by clause with selections
URL: https://github.com/apache/incubator-pinot/pull/5014#discussion_r370819849
##########
File path: pinot-common/src/main/java/org/apache/pinot/sql/parsers/CalciteSqlParser.java
##########
@@ -92,6 +94,34 @@ public static PinotQuery compileToPinotQuery(String sql)
return pinotQuery;
}
+ static void validate(PinotQuery pinotQuery)
+ throws SqlCompilationException {
+ // Sanity check group by query: All identifiers in selection list should be also included in group by list.
Review comment:
GroupBy clause could be non-aggregate expressions, like dateConverter(), ADD(...), SUB(...)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] codecov-io edited a comment on issue #5014:
fixing converting PinotQuery to BrokerRequest for group by clause with
selections
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #5014: fixing converting PinotQuery to BrokerRequest for group by clause with selections
URL: https://github.com/apache/incubator-pinot/pull/5014#issuecomment-578049370
# [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/5014?src=pr&el=h1) Report
> Merging [#5014](https://codecov.io/gh/apache/incubator-pinot/pull/5014?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-pinot/commit/995ad8331ccde0cc4e5d85dde7f326a61e2ba1e6?src=pr&el=desc) will **decrease** coverage by `5.22%`.
> The diff coverage is `58.62%`.
[](https://codecov.io/gh/apache/incubator-pinot/pull/5014?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #5014 +/- ##
===========================================
- Coverage 57.03% 51.8% -5.23%
Complexity 12 12
===========================================
Files 1179 1179
Lines 62763 62818 +55
Branches 9213 9228 +15
===========================================
- Hits 35796 32545 -3251
- Misses 24348 27765 +3417
+ Partials 2619 2508 -111
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-pinot/pull/5014?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
|---|---|---|---|
| [...ot/parsers/utils/BrokerRequestComparisonUtils.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9wYXJzZXJzL3V0aWxzL0Jyb2tlclJlcXVlc3RDb21wYXJpc29uVXRpbHMuamF2YQ==) | `21.31% <0%> (-2.62%)` | `0 <0> (ø)` | |
| [...pql/parsers/PinotQuery2BrokerRequestConverter.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9wcWwvcGFyc2Vycy9QaW5vdFF1ZXJ5MkJyb2tlclJlcXVlc3RDb252ZXJ0ZXIuamF2YQ==) | `87.32% <33.33%> (-0.47%)` | `0 <0> (ø)` | |
| [...org/apache/pinot/sql/parsers/CalciteSqlParser.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcWwvcGFyc2Vycy9DYWxjaXRlU3FsUGFyc2VyLmphdmE=) | `90.22% <94.28%> (+1.02%)` | `0 <0> (ø)` | :arrow_down: |
| [...starter/helix/DefaultHelixStarterServerConfig.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3Qtc2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zZXJ2ZXIvc3RhcnRlci9oZWxpeC9EZWZhdWx0SGVsaXhTdGFydGVyU2VydmVyQ29uZmlnLmphdmE=) | `0% <0%> (-96.16%)` | `0% <0%> (ø)` | |
| [...converter/stats/RealtimeSegmentStatsContainer.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9yZWFsdGltZS9jb252ZXJ0ZXIvc3RhdHMvUmVhbHRpbWVTZWdtZW50U3RhdHNDb250YWluZXIuamF2YQ==) | `0% <0%> (-94.45%)` | `0% <0%> (ø)` | |
| [...rg/apache/pinot/tools/utils/KafkaStarterUtils.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3QtdG9vbHMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3Rvb2xzL3V0aWxzL0thZmthU3RhcnRlclV0aWxzLmphdmE=) | `0% <0%> (-93.19%)` | `0% <0%> (ø)` | |
| [...ot/tools/query/comparison/SegmentInfoProvider.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3QtdG9vbHMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3Rvb2xzL3F1ZXJ5L2NvbXBhcmlzb24vU2VnbWVudEluZm9Qcm92aWRlci5qYXZh) | `0% <0%> (-88.68%)` | `0% <0%> (ø)` | |
| [.../transform/function/MapValueTransformFunction.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vTWFwVmFsdWVUcmFuc2Zvcm1GdW5jdGlvbi5qYXZh) | `0% <0%> (-85.3%)` | `0% <0%> (ø)` | |
| [...rg/apache/pinot/server/starter/ServerInstance.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3Qtc2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zZXJ2ZXIvc3RhcnRlci9TZXJ2ZXJJbnN0YW5jZS5qYXZh) | `1.49% <0%> (-83.59%)` | `0% <0%> (ø)` | |
| [...pache/pinot/server/starter/helix/SegmentLocks.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3Qtc2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zZXJ2ZXIvc3RhcnRlci9oZWxpeC9TZWdtZW50TG9ja3MuamF2YQ==) | `0% <0%> (-83.34%)` | `0% <0%> (ø)` | |
| ... and [194 more](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/5014?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/5014?src=pr&el=footer). Last update [995ad83...dd08847](https://codecov.io/gh/apache/incubator-pinot/pull/5014?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] fx19880617 merged pull request #5014: fixing
converting PinotQuery to BrokerRequest for group by clause with selections
Posted by GitBox <gi...@apache.org>.
fx19880617 merged pull request #5014: fixing converting PinotQuery to BrokerRequest for group by clause with selections
URL: https://github.com/apache/incubator-pinot/pull/5014
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] siddharthteotia commented on a change in pull
request #5014: fixing converting PinotQuery to BrokerRequest for group by
clause with selections
Posted by GitBox <gi...@apache.org>.
siddharthteotia commented on a change in pull request #5014: fixing converting PinotQuery to BrokerRequest for group by clause with selections
URL: https://github.com/apache/incubator-pinot/pull/5014#discussion_r370772039
##########
File path: pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
##########
@@ -832,4 +849,17 @@ public void testSqlDistinctQueryCompilation() {
Assert.assertEquals(aggregationInfo.getAggregationParams().get(FunctionCallAstNode.COLUMN_KEY_IN_AGGREGATION_INFO),
"add(div(col1,col2),mul(col3,col4)):sub(col3,col4):col5:col6");
}
+
+ @Test
+ public void testQueryValidationFailure() {
+ String sql;
+ try {
+ sql = "select group_city,group_country, sum(rsvp_count), count(*) from meetupRsvp group by group_country, sum(rsvp_count), count(*) limit 50";
Review comment:
The error seen on executing the above examples as per the w3schoold website is:
Error 1: could not prepare statement (1 aggregate functions are not allowed in the GROUP BY clause).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] chenboat commented on a change in pull request
#5014: fixing converting PinotQuery to BrokerRequest for group by clause
with selections
Posted by GitBox <gi...@apache.org>.
chenboat commented on a change in pull request #5014: fixing converting PinotQuery to BrokerRequest for group by clause with selections
URL: https://github.com/apache/incubator-pinot/pull/5014#discussion_r370767580
##########
File path: pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
##########
@@ -832,4 +849,17 @@ public void testSqlDistinctQueryCompilation() {
Assert.assertEquals(aggregationInfo.getAggregationParams().get(FunctionCallAstNode.COLUMN_KEY_IN_AGGREGATION_INFO),
"add(div(col1,col2),mul(col3,col4)):sub(col3,col4):col5:col6");
}
+
+ @Test
+ public void testQueryValidationFailure() {
Review comment:
Add one more test to cover cases like "select A, count(*) from table group by A,B"? It is supposed to another validation error.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] chenboat commented on a change in pull request
#5014: fixing converting PinotQuery to BrokerRequest for group by clause
with selections
Posted by GitBox <gi...@apache.org>.
chenboat commented on a change in pull request #5014: fixing converting PinotQuery to BrokerRequest for group by clause with selections
URL: https://github.com/apache/incubator-pinot/pull/5014#discussion_r370768082
##########
File path: pinot-common/src/main/java/org/apache/pinot/sql/parsers/CalciteSqlParser.java
##########
@@ -92,6 +94,34 @@ public static PinotQuery compileToPinotQuery(String sql)
return pinotQuery;
}
+ static void validate(PinotQuery pinotQuery)
+ throws SqlCompilationException {
+ // Sanity check group by query: All identifiers in selection list should be also included in group by list.
+ if (pinotQuery.getGroupByList() != null) {
Review comment:
NIT: use if(pinotQuery.getGroupByList() == null) return; to reduce the nest levels.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] fx19880617 commented on a change in pull request
#5014: fixing converting PinotQuery to BrokerRequest for group by clause
with selections
Posted by GitBox <gi...@apache.org>.
fx19880617 commented on a change in pull request #5014: fixing converting PinotQuery to BrokerRequest for group by clause with selections
URL: https://github.com/apache/incubator-pinot/pull/5014#discussion_r370833433
##########
File path: pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
##########
@@ -832,4 +849,17 @@ public void testSqlDistinctQueryCompilation() {
Assert.assertEquals(aggregationInfo.getAggregationParams().get(FunctionCallAstNode.COLUMN_KEY_IN_AGGREGATION_INFO),
"add(div(col1,col2),mul(col3,col4)):sub(col3,col4):col5:col6");
}
+
+ @Test
+ public void testQueryValidationFailure() {
+ String sql;
+ try {
+ sql = "select group_city,group_country, sum(rsvp_count), count(*) from meetupRsvp group by group_country, sum(rsvp_count), count(*) limit 50";
Review comment:
Also added a validation to prevent aggregate expression in group by clause.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] siddharthteotia commented on issue #5014: fixing
converting PinotQuery to BrokerRequest for group by clause with selections
Posted by GitBox <gi...@apache.org>.
siddharthteotia commented on issue #5014: fixing converting PinotQuery to BrokerRequest for group by clause with selections
URL: https://github.com/apache/incubator-pinot/pull/5014#issuecomment-578318577
LGTM. Thanks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] siddharthteotia commented on a change in pull
request #5014: fixing converting PinotQuery to BrokerRequest for group by
clause with selections
Posted by GitBox <gi...@apache.org>.
siddharthteotia commented on a change in pull request #5014: fixing converting PinotQuery to BrokerRequest for group by clause with selections
URL: https://github.com/apache/incubator-pinot/pull/5014#discussion_r370769811
##########
File path: pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
##########
@@ -832,4 +849,17 @@ public void testSqlDistinctQueryCompilation() {
Assert.assertEquals(aggregationInfo.getAggregationParams().get(FunctionCallAstNode.COLUMN_KEY_IN_AGGREGATION_INFO),
"add(div(col1,col2),mul(col3,col4)):sub(col3,col4):col5:col6");
}
+
+ @Test
+ public void testQueryValidationFailure() {
+ String sql;
+ try {
+ sql = "select group_city,group_country, sum(rsvp_count), count(*) from meetupRsvp group by group_country, sum(rsvp_count), count(*) limit 50";
Review comment:
I agree that we should be checking for identifiers. However, I think a query like this:
SELECT State, Count(*) from Customers GROUP BY State, Count(*) should not be supported.
In other words, we should not have the aggregations as one of the GROUP BY columns.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] fx19880617 commented on a change in pull request
#5014: fixing converting PinotQuery to BrokerRequest for group by clause
with selections
Posted by GitBox <gi...@apache.org>.
fx19880617 commented on a change in pull request #5014: fixing converting PinotQuery to BrokerRequest for group by clause with selections
URL: https://github.com/apache/incubator-pinot/pull/5014#discussion_r370818498
##########
File path: pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
##########
@@ -832,4 +849,17 @@ public void testSqlDistinctQueryCompilation() {
Assert.assertEquals(aggregationInfo.getAggregationParams().get(FunctionCallAstNode.COLUMN_KEY_IN_AGGREGATION_INFO),
"add(div(col1,col2),mul(col3,col4)):sub(col3,col4):col5:col6");
}
+
+ @Test
+ public void testQueryValidationFailure() {
Review comment:
I thought this is a valid query ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] siddharthteotia commented on a change in pull
request #5014: fixing converting PinotQuery to BrokerRequest for group by
clause with selections
Posted by GitBox <gi...@apache.org>.
siddharthteotia commented on a change in pull request #5014: fixing converting PinotQuery to BrokerRequest for group by clause with selections
URL: https://github.com/apache/incubator-pinot/pull/5014#discussion_r370770882
##########
File path: pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
##########
@@ -832,4 +849,17 @@ public void testSqlDistinctQueryCompilation() {
Assert.assertEquals(aggregationInfo.getAggregationParams().get(FunctionCallAstNode.COLUMN_KEY_IN_AGGREGATION_INFO),
"add(div(col1,col2),mul(col3,col4)):sub(col3,col4):col5:col6");
}
+
+ @Test
+ public void testQueryValidationFailure() {
+ String sql;
+ try {
+ sql = "select group_city,group_country, sum(rsvp_count), count(*) from meetupRsvp group by group_country, sum(rsvp_count), count(*) limit 50";
Review comment:
So we should add tests for this case too
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] siddharthteotia commented on a change in pull
request #5014: fixing converting PinotQuery to BrokerRequest for group by
clause with selections
Posted by GitBox <gi...@apache.org>.
siddharthteotia commented on a change in pull request #5014: fixing converting PinotQuery to BrokerRequest for group by clause with selections
URL: https://github.com/apache/incubator-pinot/pull/5014#discussion_r370769811
##########
File path: pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
##########
@@ -832,4 +849,17 @@ public void testSqlDistinctQueryCompilation() {
Assert.assertEquals(aggregationInfo.getAggregationParams().get(FunctionCallAstNode.COLUMN_KEY_IN_AGGREGATION_INFO),
"add(div(col1,col2),mul(col3,col4)):sub(col3,col4):col5:col6");
}
+
+ @Test
+ public void testQueryValidationFailure() {
+ String sql;
+ try {
+ sql = "select group_city,group_country, sum(rsvp_count), count(*) from meetupRsvp group by group_country, sum(rsvp_count), count(*) limit 50";
Review comment:
I agree that we should be checking for identifiers. However, I don't think a query like this:
SELECT State, Count(*) from Customers GROUP BY State, Count(*) should not be supported.
In other words, we should not have the aggregations as one of the GROUP BY columns.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] siddharthteotia commented on a change in pull
request #5014: fixing converting PinotQuery to BrokerRequest for group by
clause with selections
Posted by GitBox <gi...@apache.org>.
siddharthteotia commented on a change in pull request #5014: fixing converting PinotQuery to BrokerRequest for group by clause with selections
URL: https://github.com/apache/incubator-pinot/pull/5014#discussion_r370772039
##########
File path: pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
##########
@@ -832,4 +849,17 @@ public void testSqlDistinctQueryCompilation() {
Assert.assertEquals(aggregationInfo.getAggregationParams().get(FunctionCallAstNode.COLUMN_KEY_IN_AGGREGATION_INFO),
"add(div(col1,col2),mul(col3,col4)):sub(col3,col4):col5:col6");
}
+
+ @Test
+ public void testQueryValidationFailure() {
+ String sql;
+ try {
+ sql = "select group_city,group_country, sum(rsvp_count), count(*) from meetupRsvp group by group_country, sum(rsvp_count), count(*) limit 50";
Review comment:
The error seen on executing the above examples as per the w3schools website is:
Error 1: could not prepare statement (1 aggregate functions are not allowed in the GROUP BY clause).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] codecov-io edited a comment on issue #5014:
fixing converting PinotQuery to BrokerRequest for group by clause with
selections
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #5014: fixing converting PinotQuery to BrokerRequest for group by clause with selections
URL: https://github.com/apache/incubator-pinot/pull/5014#issuecomment-578049370
# [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/5014?src=pr&el=h1) Report
> Merging [#5014](https://codecov.io/gh/apache/incubator-pinot/pull/5014?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-pinot/commit/995ad8331ccde0cc4e5d85dde7f326a61e2ba1e6?src=pr&el=desc) will **decrease** coverage by `5.22%`.
> The diff coverage is `58.62%`.
[](https://codecov.io/gh/apache/incubator-pinot/pull/5014?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #5014 +/- ##
===========================================
- Coverage 57.03% 51.8% -5.23%
Complexity 12 12
===========================================
Files 1179 1179
Lines 62763 62818 +55
Branches 9213 9228 +15
===========================================
- Hits 35796 32545 -3251
- Misses 24348 27765 +3417
+ Partials 2619 2508 -111
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-pinot/pull/5014?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
|---|---|---|---|
| [...ot/parsers/utils/BrokerRequestComparisonUtils.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9wYXJzZXJzL3V0aWxzL0Jyb2tlclJlcXVlc3RDb21wYXJpc29uVXRpbHMuamF2YQ==) | `21.31% <0%> (-2.62%)` | `0 <0> (ø)` | |
| [...pql/parsers/PinotQuery2BrokerRequestConverter.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9wcWwvcGFyc2Vycy9QaW5vdFF1ZXJ5MkJyb2tlclJlcXVlc3RDb252ZXJ0ZXIuamF2YQ==) | `87.32% <33.33%> (-0.47%)` | `0 <0> (ø)` | |
| [...org/apache/pinot/sql/parsers/CalciteSqlParser.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcWwvcGFyc2Vycy9DYWxjaXRlU3FsUGFyc2VyLmphdmE=) | `90.22% <94.28%> (+1.02%)` | `0 <0> (ø)` | :arrow_down: |
| [...starter/helix/DefaultHelixStarterServerConfig.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3Qtc2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zZXJ2ZXIvc3RhcnRlci9oZWxpeC9EZWZhdWx0SGVsaXhTdGFydGVyU2VydmVyQ29uZmlnLmphdmE=) | `0% <0%> (-96.16%)` | `0% <0%> (ø)` | |
| [...converter/stats/RealtimeSegmentStatsContainer.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9yZWFsdGltZS9jb252ZXJ0ZXIvc3RhdHMvUmVhbHRpbWVTZWdtZW50U3RhdHNDb250YWluZXIuamF2YQ==) | `0% <0%> (-94.45%)` | `0% <0%> (ø)` | |
| [...rg/apache/pinot/tools/utils/KafkaStarterUtils.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3QtdG9vbHMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3Rvb2xzL3V0aWxzL0thZmthU3RhcnRlclV0aWxzLmphdmE=) | `0% <0%> (-93.19%)` | `0% <0%> (ø)` | |
| [...ot/tools/query/comparison/SegmentInfoProvider.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3QtdG9vbHMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3Rvb2xzL3F1ZXJ5L2NvbXBhcmlzb24vU2VnbWVudEluZm9Qcm92aWRlci5qYXZh) | `0% <0%> (-88.68%)` | `0% <0%> (ø)` | |
| [.../transform/function/MapValueTransformFunction.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vTWFwVmFsdWVUcmFuc2Zvcm1GdW5jdGlvbi5qYXZh) | `0% <0%> (-85.3%)` | `0% <0%> (ø)` | |
| [...rg/apache/pinot/server/starter/ServerInstance.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3Qtc2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zZXJ2ZXIvc3RhcnRlci9TZXJ2ZXJJbnN0YW5jZS5qYXZh) | `1.49% <0%> (-83.59%)` | `0% <0%> (ø)` | |
| [...pache/pinot/server/starter/helix/SegmentLocks.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3Qtc2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zZXJ2ZXIvc3RhcnRlci9oZWxpeC9TZWdtZW50TG9ja3MuamF2YQ==) | `0% <0%> (-83.34%)` | `0% <0%> (ø)` | |
| ... and [194 more](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/5014?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/5014?src=pr&el=footer). Last update [995ad83...dd08847](https://codecov.io/gh/apache/incubator-pinot/pull/5014?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] fx19880617 commented on a change in pull request
#5014: fixing converting PinotQuery to BrokerRequest for group by clause
with selections
Posted by GitBox <gi...@apache.org>.
fx19880617 commented on a change in pull request #5014: fixing converting PinotQuery to BrokerRequest for group by clause with selections
URL: https://github.com/apache/incubator-pinot/pull/5014#discussion_r370823449
##########
File path: pinot-common/src/main/java/org/apache/pinot/sql/parsers/CalciteSqlParser.java
##########
@@ -92,6 +94,34 @@ public static PinotQuery compileToPinotQuery(String sql)
return pinotQuery;
}
+ static void validate(PinotQuery pinotQuery)
+ throws SqlCompilationException {
+ // Sanity check group by query: All identifiers in selection list should be also included in group by list.
+ if (pinotQuery.getGroupByList() != null) {
Review comment:
done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] siddharthteotia commented on a change in pull
request #5014: fixing converting PinotQuery to BrokerRequest for group by
clause with selections
Posted by GitBox <gi...@apache.org>.
siddharthteotia commented on a change in pull request #5014: fixing converting PinotQuery to BrokerRequest for group by clause with selections
URL: https://github.com/apache/incubator-pinot/pull/5014#discussion_r370770190
##########
File path: pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
##########
@@ -832,4 +849,17 @@ public void testSqlDistinctQueryCompilation() {
Assert.assertEquals(aggregationInfo.getAggregationParams().get(FunctionCallAstNode.COLUMN_KEY_IN_AGGREGATION_INFO),
"add(div(col1,col2),mul(col3,col4)):sub(col3,col4):col5:col6");
}
+
+ @Test
+ public void testQueryValidationFailure() {
+ String sql;
+ try {
+ sql = "select group_city,group_country, sum(rsvp_count), count(*) from meetupRsvp group by group_country, sum(rsvp_count), count(*) limit 50";
Review comment:
Try the following queries here https://www.w3schools.com/sql/trysql.asp?filename=trysql_select_no_distinct
(1) SELECT City, SUM(PostalCode) FROM Customers GROUP BY City, SUM(PostalCode)
(2) SELECT City, SUM(PostalCode) FROM Customers GROUP BY City
(2) is valid whereas (1) fails
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] chenboat commented on a change in pull request
#5014: fixing converting PinotQuery to BrokerRequest for group by clause
with selections
Posted by GitBox <gi...@apache.org>.
chenboat commented on a change in pull request #5014: fixing converting PinotQuery to BrokerRequest for group by clause with selections
URL: https://github.com/apache/incubator-pinot/pull/5014#discussion_r370767580
##########
File path: pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
##########
@@ -832,4 +849,17 @@ public void testSqlDistinctQueryCompilation() {
Assert.assertEquals(aggregationInfo.getAggregationParams().get(FunctionCallAstNode.COLUMN_KEY_IN_AGGREGATION_INFO),
"add(div(col1,col2),mul(col3,col4)):sub(col3,col4):col5:col6");
}
+
+ @Test
+ public void testQueryValidationFailure() {
Review comment:
Add one more test to cover cases like "select A, count(*) from table group by A,B"?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] fx19880617 commented on a change in pull request
#5014: fixing converting PinotQuery to BrokerRequest for group by clause
with selections
Posted by GitBox <gi...@apache.org>.
fx19880617 commented on a change in pull request #5014: fixing converting PinotQuery to BrokerRequest for group by clause with selections
URL: https://github.com/apache/incubator-pinot/pull/5014#discussion_r370823409
##########
File path: pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
##########
@@ -832,4 +849,17 @@ public void testSqlDistinctQueryCompilation() {
Assert.assertEquals(aggregationInfo.getAggregationParams().get(FunctionCallAstNode.COLUMN_KEY_IN_AGGREGATION_INFO),
"add(div(col1,col2),mul(col3,col4)):sub(col3,col4):col5:col6");
}
+
+ @Test
+ public void testQueryValidationFailure() {
Review comment:
added this to the test
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] siddharthteotia commented on a change in pull
request #5014: fixing converting PinotQuery to BrokerRequest for group by
clause with selections
Posted by GitBox <gi...@apache.org>.
siddharthteotia commented on a change in pull request #5014: fixing converting PinotQuery to BrokerRequest for group by clause with selections
URL: https://github.com/apache/incubator-pinot/pull/5014#discussion_r370769811
##########
File path: pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
##########
@@ -832,4 +849,17 @@ public void testSqlDistinctQueryCompilation() {
Assert.assertEquals(aggregationInfo.getAggregationParams().get(FunctionCallAstNode.COLUMN_KEY_IN_AGGREGATION_INFO),
"add(div(col1,col2),mul(col3,col4)):sub(col3,col4):col5:col6");
}
+
+ @Test
+ public void testQueryValidationFailure() {
+ String sql;
+ try {
+ sql = "select group_city,group_country, sum(rsvp_count), count(*) from meetupRsvp group by group_country, sum(rsvp_count), count(*) limit 50";
Review comment:
I agree that we should be checking for identifiers. However, I don't think a query like this:
SELECT State, Count(*) from Customers GROUP BY State, Count(*) should be supported.
In other words, we should not have the aggregations as one of the GROUP BY columns.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull
request #5014: fixing converting PinotQuery to BrokerRequest for group by
clause with selections
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on a change in pull request #5014: fixing converting PinotQuery to BrokerRequest for group by clause with selections
URL: https://github.com/apache/incubator-pinot/pull/5014#discussion_r370768769
##########
File path: pinot-common/src/main/java/org/apache/pinot/sql/parsers/CalciteSqlParser.java
##########
@@ -92,6 +94,34 @@ public static PinotQuery compileToPinotQuery(String sql)
return pinotQuery;
}
+ static void validate(PinotQuery pinotQuery)
+ throws SqlCompilationException {
+ // Sanity check group by query: All identifiers in selection list should be also included in group by list.
Review comment:
Should we check identifiers or expressions?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] siddharthteotia commented on a change in pull
request #5014: fixing converting PinotQuery to BrokerRequest for group by
clause with selections
Posted by GitBox <gi...@apache.org>.
siddharthteotia commented on a change in pull request #5014: fixing converting PinotQuery to BrokerRequest for group by clause with selections
URL: https://github.com/apache/incubator-pinot/pull/5014#discussion_r370770689
##########
File path: pinot-common/src/main/java/org/apache/pinot/sql/parsers/CalciteSqlParser.java
##########
@@ -92,6 +94,34 @@ public static PinotQuery compileToPinotQuery(String sql)
return pinotQuery;
}
+ static void validate(PinotQuery pinotQuery)
+ throws SqlCompilationException {
+ // Sanity check group by query: All identifiers in selection list should be also included in group by list.
Review comment:
I don't think we should support function expressions part of GROUP BY clause. See example here https://github.com/apache/incubator-pinot/pull/5014#discussion_r370769811
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] codecov-io commented on issue #5014: fixing
converting PinotQuery to BrokerRequest for group by clause with selections
Posted by GitBox <gi...@apache.org>.
codecov-io commented on issue #5014: fixing converting PinotQuery to BrokerRequest for group by clause with selections
URL: https://github.com/apache/incubator-pinot/pull/5014#issuecomment-578049370
# [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/5014?src=pr&el=h1) Report
> Merging [#5014](https://codecov.io/gh/apache/incubator-pinot/pull/5014?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-pinot/commit/995ad8331ccde0cc4e5d85dde7f326a61e2ba1e6?src=pr&el=desc) will **decrease** coverage by `<.01%`.
> The diff coverage is `60.65%`.
[](https://codecov.io/gh/apache/incubator-pinot/pull/5014?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #5014 +/- ##
============================================
- Coverage 57.03% 57.02% -0.01%
Complexity 12 12
============================================
Files 1179 1179
Lines 62763 62821 +58
Branches 9213 9231 +18
============================================
+ Hits 35796 35824 +28
- Misses 24348 24373 +25
- Partials 2619 2624 +5
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-pinot/pull/5014?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
|---|---|---|---|
| [...ot/parsers/utils/BrokerRequestComparisonUtils.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9wYXJzZXJzL3V0aWxzL0Jyb2tlclJlcXVlc3RDb21wYXJpc29uVXRpbHMuamF2YQ==) | `21.31% <0%> (-2.62%)` | `0 <0> (ø)` | |
| [...pql/parsers/PinotQuery2BrokerRequestConverter.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9wcWwvcGFyc2Vycy9QaW5vdFF1ZXJ5MkJyb2tlclJlcXVlc3RDb252ZXJ0ZXIuamF2YQ==) | `87.32% <33.33%> (-0.47%)` | `0 <0> (ø)` | |
| [...org/apache/pinot/sql/parsers/CalciteSqlParser.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcWwvcGFyc2Vycy9DYWxjaXRlU3FsUGFyc2VyLmphdmE=) | `90.39% <94.73%> (+1.18%)` | `0 <0> (ø)` | :arrow_down: |
| [...er/validation/BrokerResourceValidationManager.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci92YWxpZGF0aW9uL0Jyb2tlclJlc291cmNlVmFsaWRhdGlvbk1hbmFnZXIuamF2YQ==) | `25% <0%> (-56.25%)` | `0% <0%> (ø)` | |
| [...e/operator/dociditerators/MVScanDocIdIterator.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9kb2NpZGl0ZXJhdG9ycy9NVlNjYW5Eb2NJZEl0ZXJhdG9yLmphdmE=) | `46.03% <0%> (-17.47%)` | `0% <0%> (ø)` | |
| [...e/impl/dictionary/LongOnHeapMutableDictionary.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9yZWFsdGltZS9pbXBsL2RpY3Rpb25hcnkvTG9uZ09uSGVhcE11dGFibGVEaWN0aW9uYXJ5LmphdmE=) | `53.65% <0%> (-9.76%)` | `0% <0%> (ø)` | |
| [...pache/pinot/core/util/SortedRangeIntersection.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS91dGlsL1NvcnRlZFJhbmdlSW50ZXJzZWN0aW9uLmphdmE=) | `83.82% <0%> (-7.36%)` | `0% <0%> (ø)` | |
| [...not/broker/broker/helix/ClusterChangeMediator.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3QtYnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9icm9rZXIvYnJva2VyL2hlbGl4L0NsdXN0ZXJDaGFuZ2VNZWRpYXRvci5qYXZh) | `74.72% <0%> (-5.5%)` | `0% <0%> (ø)` | |
| [...impl/dictionary/FloatOffHeapMutableDictionary.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9yZWFsdGltZS9pbXBsL2RpY3Rpb25hcnkvRmxvYXRPZmZIZWFwTXV0YWJsZURpY3Rpb25hcnkuamF2YQ==) | `51.61% <0%> (-5.38%)` | `0% <0%> (ø)` | |
| [...lix/core/realtime/PinotRealtimeSegmentManager.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9oZWxpeC9jb3JlL3JlYWx0aW1lL1Bpbm90UmVhbHRpbWVTZWdtZW50TWFuYWdlci5qYXZh) | `80% <0%> (-2.06%)` | `0% <0%> (ø)` | |
| ... and [17 more](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/5014?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/5014?src=pr&el=footer). Last update [995ad83...766f01b](https://codecov.io/gh/apache/incubator-pinot/pull/5014?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] codecov-io edited a comment on issue #5014:
fixing converting PinotQuery to BrokerRequest for group by clause with
selections
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #5014: fixing converting PinotQuery to BrokerRequest for group by clause with selections
URL: https://github.com/apache/incubator-pinot/pull/5014#issuecomment-578049370
# [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/5014?src=pr&el=h1) Report
> Merging [#5014](https://codecov.io/gh/apache/incubator-pinot/pull/5014?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-pinot/commit/995ad8331ccde0cc4e5d85dde7f326a61e2ba1e6?src=pr&el=desc) will **decrease** coverage by `21.07%`.
> The diff coverage is `8.62%`.
[](https://codecov.io/gh/apache/incubator-pinot/pull/5014?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #5014 +/- ##
=============================================
- Coverage 57.03% 35.95% -21.08%
=============================================
Files 1179 1179
Lines 62763 62818 +55
Branches 9213 9228 +15
=============================================
- Hits 35796 22586 -13210
- Misses 24348 38264 +13916
+ Partials 2619 1968 -651
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-pinot/pull/5014?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
|---|---|---|---|
| [...ot/parsers/utils/BrokerRequestComparisonUtils.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9wYXJzZXJzL3V0aWxzL0Jyb2tlclJlcXVlc3RDb21wYXJpc29uVXRpbHMuamF2YQ==) | `0% <0%> (-23.93%)` | `0 <0> (ø)` | |
| [...org/apache/pinot/sql/parsers/CalciteSqlParser.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcWwvcGFyc2Vycy9DYWxjaXRlU3FsUGFyc2VyLmphdmE=) | `28.16% <11.42%> (-61.05%)` | `0 <0> (ø)` | |
| [...pql/parsers/PinotQuery2BrokerRequestConverter.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9wcWwvcGFyc2Vycy9QaW5vdFF1ZXJ5MkJyb2tlclJlcXVlc3RDb252ZXJ0ZXIuamF2YQ==) | `52.11% <33.33%> (-35.69%)` | `0 <0> (ø)` | |
| [.../org/apache/pinot/common/http/MultiGetRequest.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vaHR0cC9NdWx0aUdldFJlcXVlc3QuamF2YQ==) | `0% <0%> (-100%)` | `0% <0%> (ø)` | |
| [...che/pinot/common/restlet/resources/TablesList.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vcmVzdGxldC9yZXNvdXJjZXMvVGFibGVzTGlzdC5qYXZh) | `0% <0%> (-100%)` | `0% <0%> (ø)` | |
| [...che/pinot/pql/parsers/pql2/ast/OptionsAstNode.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9wcWwvcGFyc2Vycy9wcWwyL2FzdC9PcHRpb25zQXN0Tm9kZS5qYXZh) | `0% <0%> (-100%)` | `0% <0%> (ø)` | |
| [...ot/core/query/scheduler/TableBasedGroupMapper.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9xdWVyeS9zY2hlZHVsZXIvVGFibGVCYXNlZEdyb3VwTWFwcGVyLmphdmE=) | `0% <0%> (-100%)` | `0% <0%> (ø)` | |
| [...t/core/segment/index/readers/OnHeapDictionary.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9zZWdtZW50L2luZGV4L3JlYWRlcnMvT25IZWFwRGljdGlvbmFyeS5qYXZh) | `0% <0%> (-100%)` | `0% <0%> (ø)` | |
| [...t/core/query/scheduler/OutOfCapacityException.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9xdWVyeS9zY2hlZHVsZXIvT3V0T2ZDYXBhY2l0eUV4Y2VwdGlvbi5qYXZh) | `0% <0%> (-100%)` | `0% <0%> (ø)` | |
| [...ot/core/query/scheduler/SchedulerQueryContext.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9xdWVyeS9zY2hlZHVsZXIvU2NoZWR1bGVyUXVlcnlDb250ZXh0LmphdmE=) | `0% <0%> (-100%)` | `0% <0%> (ø)` | |
| ... and [585 more](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/5014?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/5014?src=pr&el=footer). Last update [995ad83...dd08847](https://codecov.io/gh/apache/incubator-pinot/pull/5014?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] fx19880617 commented on a change in pull request
#5014: fixing converting PinotQuery to BrokerRequest for group by clause
with selections
Posted by GitBox <gi...@apache.org>.
fx19880617 commented on a change in pull request #5014: fixing converting PinotQuery to BrokerRequest for group by clause with selections
URL: https://github.com/apache/incubator-pinot/pull/5014#discussion_r370819544
##########
File path: pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
##########
@@ -832,4 +849,17 @@ public void testSqlDistinctQueryCompilation() {
Assert.assertEquals(aggregationInfo.getAggregationParams().get(FunctionCallAstNode.COLUMN_KEY_IN_AGGREGATION_INFO),
"add(div(col1,col2),mul(col3,col4)):sub(col3,col4):col5:col6");
}
+
+ @Test
+ public void testQueryValidationFailure() {
+ String sql;
+ try {
+ sql = "select group_city,group_country, sum(rsvp_count), count(*) from meetupRsvp group by group_country, sum(rsvp_count), count(*) limit 50";
Review comment:
GROUP BY clause shouldn't contain aggregate expression like sum(PostalCode), however, you can still do dateConverter(secondsSinceEpoch) etc, which is non-aggregate Transform function. Or expressions like ADD(a,b).
My validate assumes that group by clause is always non-aggregate expression, so that we could extract all the identifier from them.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] siddharthteotia commented on a change in pull
request #5014: fixing converting PinotQuery to BrokerRequest for group by
clause with selections
Posted by GitBox <gi...@apache.org>.
siddharthteotia commented on a change in pull request #5014: fixing converting PinotQuery to BrokerRequest for group by clause with selections
URL: https://github.com/apache/incubator-pinot/pull/5014#discussion_r370770882
##########
File path: pinot-common/src/test/java/org/apache/pinot/sql/parsers/CalciteSqlCompilerTest.java
##########
@@ -832,4 +849,17 @@ public void testSqlDistinctQueryCompilation() {
Assert.assertEquals(aggregationInfo.getAggregationParams().get(FunctionCallAstNode.COLUMN_KEY_IN_AGGREGATION_INFO),
"add(div(col1,col2),mul(col3,col4)):sub(col3,col4):col5:col6");
}
+
+ @Test
+ public void testQueryValidationFailure() {
+ String sql;
+ try {
+ sql = "select group_city,group_country, sum(rsvp_count), count(*) from meetupRsvp group by group_country, sum(rsvp_count), count(*) limit 50";
Review comment:
So we should also add tests for this case too
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] codecov-io edited a comment on issue #5014:
fixing converting PinotQuery to BrokerRequest for group by clause with
selections
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #5014: fixing converting PinotQuery to BrokerRequest for group by clause with selections
URL: https://github.com/apache/incubator-pinot/pull/5014#issuecomment-578049370
# [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/5014?src=pr&el=h1) Report
> Merging [#5014](https://codecov.io/gh/apache/incubator-pinot/pull/5014?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-pinot/commit/995ad8331ccde0cc4e5d85dde7f326a61e2ba1e6?src=pr&el=desc) will **decrease** coverage by `5.36%`.
> The diff coverage is `46.34%`.
[](https://codecov.io/gh/apache/incubator-pinot/pull/5014?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #5014 +/- ##
============================================
- Coverage 57.03% 51.66% -5.37%
Complexity 12 12
============================================
Files 1179 1179
Lines 62763 62801 +38
Branches 9213 9224 +11
============================================
- Hits 35796 32446 -3350
- Misses 24348 27863 +3515
+ Partials 2619 2492 -127
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-pinot/pull/5014?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
|---|---|---|---|
| [...ot/parsers/utils/BrokerRequestComparisonUtils.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9wYXJzZXJzL3V0aWxzL0Jyb2tlclJlcXVlc3RDb21wYXJpc29uVXRpbHMuamF2YQ==) | `21.31% <0%> (-2.62%)` | `0 <0> (ø)` | |
| [...org/apache/pinot/sql/parsers/CalciteSqlParser.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcWwvcGFyc2Vycy9DYWxjaXRlU3FsUGFyc2VyLmphdmE=) | `90.44% <100%> (+1.23%)` | `0 <0> (ø)` | :arrow_down: |
| [...pql/parsers/PinotQuery2BrokerRequestConverter.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9wcWwvcGFyc2Vycy9QaW5vdFF1ZXJ5MkJyb2tlclJlcXVlc3RDb252ZXJ0ZXIuamF2YQ==) | `87.32% <33.33%> (-0.47%)` | `0 <0> (ø)` | |
| [...starter/helix/DefaultHelixStarterServerConfig.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3Qtc2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zZXJ2ZXIvc3RhcnRlci9oZWxpeC9EZWZhdWx0SGVsaXhTdGFydGVyU2VydmVyQ29uZmlnLmphdmE=) | `0% <0%> (-96.16%)` | `0% <0%> (ø)` | |
| [...converter/stats/RealtimeSegmentStatsContainer.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9yZWFsdGltZS9jb252ZXJ0ZXIvc3RhdHMvUmVhbHRpbWVTZWdtZW50U3RhdHNDb250YWluZXIuamF2YQ==) | `0% <0%> (-94.45%)` | `0% <0%> (ø)` | |
| [...rg/apache/pinot/tools/utils/KafkaStarterUtils.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3QtdG9vbHMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3Rvb2xzL3V0aWxzL0thZmthU3RhcnRlclV0aWxzLmphdmE=) | `0% <0%> (-93.19%)` | `0% <0%> (ø)` | |
| [...ot/tools/query/comparison/SegmentInfoProvider.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3QtdG9vbHMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL3Bpbm90L3Rvb2xzL3F1ZXJ5L2NvbXBhcmlzb24vU2VnbWVudEluZm9Qcm92aWRlci5qYXZh) | `0% <0%> (-88.68%)` | `0% <0%> (ø)` | |
| [.../transform/function/MapValueTransformFunction.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci90cmFuc2Zvcm0vZnVuY3Rpb24vTWFwVmFsdWVUcmFuc2Zvcm1GdW5jdGlvbi5qYXZh) | `0% <0%> (-85.3%)` | `0% <0%> (ø)` | |
| [...rg/apache/pinot/server/starter/ServerInstance.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3Qtc2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zZXJ2ZXIvc3RhcnRlci9TZXJ2ZXJJbnN0YW5jZS5qYXZh) | `1.49% <0%> (-83.59%)` | `0% <0%> (ø)` | |
| [...pache/pinot/server/starter/helix/SegmentLocks.java](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree#diff-cGlub3Qtc2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zZXJ2ZXIvc3RhcnRlci9oZWxpeC9TZWdtZW50TG9ja3MuamF2YQ==) | `0% <0%> (-83.34%)` | `0% <0%> (ø)` | |
| ... and [189 more](https://codecov.io/gh/apache/incubator-pinot/pull/5014/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/5014?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/5014?src=pr&el=footer). Last update [995ad83...766f01b](https://codecov.io/gh/apache/incubator-pinot/pull/5014?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org