You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "yashmayya (via GitHub)" <gi...@apache.org> on 2024/05/27 07:04:01 UTC
[PR] Return clear error message when no common broker found for multi-stage query with tables from different tenants [pinot]
yashmayya opened a new pull request, #13235:
URL: https://github.com/apache/pinot/pull/13235
- Fixes https://github.com/apache/pinot/issues/11990
- If there are two tables - `a` (with broker tenant config `BrokerTenant1`) and `b` (with broker tenant config `BrokerTenant2`) and we run a query like `SELECT * FROM a JOIN b ON a.col1 = b.col1;` (through the controller's `POST /sql` API), this results in the ambiguous error message: `ProcessingException(errorCode:450, message:InternalError)`. Even the controller logs contain only the highly misleading error log: `Instance ProcessingException(errorCode:410, message:BrokerResourceMissingError) not found`.
- This is because the case where no common broker instances (i.e., one with tenant tags for all tables involved in the query) are found [here](https://github.com/apache/pinot/blob/518fd180981ffa614042b8a5236650a2af41bfe5/pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotQueryResource.java#L239) isn't handled appropriately. The `instanceId` returned [here](https://github.com/apache/pinot/blob/518fd180981ffa614042b8a5236650a2af41bfe5/pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotQueryResource.java#L245) will be `ProcessingException(errorCode:410, message:BrokerResourceMissingError)` (from [here](https://github.com/apache/pinot/blob/518fd180981ffa614042b8a5236650a2af41bfe5/pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotQueryResource.java#L322)).
- The controller then attempts to find a broker with instance ID here `ProcessingException(errorCode:410, message:BrokerResourceMissingError)` which obviously does not exist and results in the internal error being returned here - https://github.com/apache/pinot/blob/518fd180981ffa614042b8a5236650a2af41bfe5/pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotQueryResource.java#L354-L358
- This patch fixes the issue by returning a clear error message when no common broker is found for a multi-stage query with tables from different tenants. For the above example query `SELECT * FROM a JOIN b ON a.col1 = b.col1;`, the returned error is now:
```
ProcessingException(errorCode:410, message:BrokerResourceMissingError:
java.lang.Exception: Unable to find a common broker instance for table tenants. Tables: [a, b], Tenants: [BrokerTenant2, BrokerTenant1]
at org.apache.pinot.controller.api.resources.PinotQueryResource.getMultiStageQueryResponse(PinotQueryResource.java:244)
at org.apache.pinot.controller.api.resources.PinotQueryResource.executeSqlQuery(PinotQueryResource.java:179)
at org.apache.pinot.controller.api.resources.PinotQueryResource.handlePostSql(PinotQueryResource.java:127)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method))
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Return clear error message when no common broker found for multi-stage query with tables from different tenants [pinot]
Posted by "yashmayya (via GitHub)" <gi...@apache.org>.
yashmayya commented on code in PR #13235:
URL: https://github.com/apache/pinot/pull/13235#discussion_r1615575781
##########
pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotQueryResource.java:
##########
@@ -237,6 +237,14 @@ private String getMultiStageQueryResponse(String query, String queryOptions, Htt
String.format("Unable to dispatch multistage query for tables: [%s]", tableNames))).toString();
}
instanceIds = findCommonBrokerInstances(brokerTenantsUnion);
+ if (instanceIds.isEmpty()) {
+ // No common broker found for table tenants
+ LOGGER.error("Unable to find a common broker instance for table tenants. Tables: {}, Tenants: {}",
+ tableNames, brokerTenantsUnion);
+ return QueryException.getException(QueryException.BROKER_RESOURCE_MISSING_ERROR,
Review Comment:
I'm not quite clear on the differences between these two error codes and which one would be more appropriate here - https://github.com/apache/pinot/blob/518fd180981ffa614042b8a5236650a2af41bfe5/pinot-common/src/main/java/org/apache/pinot/common/exception/QueryException.java#L72-L73
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Return clear error message when no common broker found for multi-stage query with tables from different tenants [pinot]
Posted by "vrajat (via GitHub)" <gi...@apache.org>.
vrajat commented on PR #13235:
URL: https://github.com/apache/pinot/pull/13235#issuecomment-2133359392
is it possible to add a test for this error condition ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Return clear error message when no common broker found for multi-stage query with tables from different tenants [pinot]
Posted by "yashmayya (via GitHub)" <gi...@apache.org>.
yashmayya commented on code in PR #13235:
URL: https://github.com/apache/pinot/pull/13235#discussion_r1615579138
##########
pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotQueryResource.java:
##########
@@ -237,6 +237,14 @@ private String getMultiStageQueryResponse(String query, String queryOptions, Htt
String.format("Unable to dispatch multistage query for tables: [%s]", tableNames))).toString();
}
instanceIds = findCommonBrokerInstances(brokerTenantsUnion);
+ if (instanceIds.isEmpty()) {
+ // No common broker found for table tenants
+ LOGGER.error("Unable to find a common broker instance for table tenants. Tables: {}, Tenants: {}",
+ tableNames, brokerTenantsUnion);
+ return QueryException.getException(QueryException.BROKER_RESOURCE_MISSING_ERROR,
Review Comment:
Ah, looks like `410` is for when no appropriate broker exists in the cluster and `420` is for when the appropriate broker isn't online based on this - https://github.com/apache/pinot/blob/518fd180981ffa614042b8a5236650a2af41bfe5/pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotQueryResource.java#L321-L328
So, I guess `410` / `BROKER_RESOURCE_MISSING_ERROR` is indeed the right one to use here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Return clear error message when no common broker found for multi-stage query with tables from different tenants [pinot]
Posted by "yashmayya (via GitHub)" <gi...@apache.org>.
yashmayya commented on PR #13235:
URL: https://github.com/apache/pinot/pull/13235#issuecomment-2133383108
> is it possible to add a test for this error condition ?
There doesn't appear to be a unit test suite for this part of the codebase. I thought it'd be overkill to add an entire new test suite just for this minor issue or even an integration test with a multiple broker setup. So I settled for simply manually testing this fix. If there's a straightforward way to add a test for this that I might be missing though, I'm all ears.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Return clear error message when no common broker found for multi-stage query with tables from different tenants [pinot]
Posted by "gortiz (via GitHub)" <gi...@apache.org>.
gortiz merged PR #13235:
URL: https://github.com/apache/pinot/pull/13235
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Return clear error message when no common broker found for multi-stage query with tables from different tenants [pinot]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #13235:
URL: https://github.com/apache/pinot/pull/13235#issuecomment-2132844317
## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/13235?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
Attention: Patch coverage is `0%` with `4 lines` in your changes are missing coverage. Please review.
> Project coverage is 0.00%. Comparing base [(`59551e4`)](https://app.codecov.io/gh/apache/pinot/commit/59551e45224f1535c4863fd577622b37366ccc97?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to head [(`d668570`)](https://app.codecov.io/gh/apache/pinot/commit/d6685704678acf833893ad2fd2327fecf860923b?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
> Report is 499 commits behind head on master.
| [Files](https://app.codecov.io/gh/apache/pinot/pull/13235?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...t/controller/api/resources/PinotQueryResource.java](https://app.codecov.io/gh/apache/pinot/pull/13235?src=pr&el=tree&filepath=pinot-controller%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fpinot%2Fcontroller%2Fapi%2Fresources%2FPinotQueryResource.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9hcGkvcmVzb3VyY2VzL1Bpbm90UXVlcnlSZXNvdXJjZS5qYXZh) | 0.00% | [4 Missing :warning: ](https://app.codecov.io/gh/apache/pinot/pull/13235?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #13235 +/- ##
=============================================
- Coverage 61.75% 0.00% -61.76%
=============================================
Files 2436 2459 +23
Lines 133233 135351 +2118
Branches 20636 20984 +348
=============================================
- Hits 82274 0 -82274
- Misses 44911 135351 +90440
+ Partials 6048 0 -6048
```
| [Flag](https://app.codecov.io/gh/apache/pinot/pull/13235/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [custom-integration1](https://app.codecov.io/gh/apache/pinot/pull/13235/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration](https://app.codecov.io/gh/apache/pinot/pull/13235/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `0.00% <0.00%> (-0.01%)` | :arrow_down: |
| [integration1](https://app.codecov.io/gh/apache/pinot/pull/13235/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration2](https://app.codecov.io/gh/apache/pinot/pull/13235/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `0.00% <0.00%> (ø)` | |
| [java-11](https://app.codecov.io/gh/apache/pinot/pull/13235/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [java-21](https://app.codecov.io/gh/apache/pinot/pull/13235/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `0.00% <0.00%> (-61.63%)` | :arrow_down: |
| [skip-bytebuffers-false](https://app.codecov.io/gh/apache/pinot/pull/13235/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `0.00% <0.00%> (-61.75%)` | :arrow_down: |
| [skip-bytebuffers-true](https://app.codecov.io/gh/apache/pinot/pull/13235/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `0.00% <0.00%> (-27.73%)` | :arrow_down: |
| [temurin](https://app.codecov.io/gh/apache/pinot/pull/13235/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `0.00% <0.00%> (-61.76%)` | :arrow_down: |
| [unittests](https://app.codecov.io/gh/apache/pinot/pull/13235/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [unittests1](https://app.codecov.io/gh/apache/pinot/pull/13235/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [unittests2](https://app.codecov.io/gh/apache/pinot/pull/13235/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/pinot/pull/13235?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org