You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "yashmayya (via GitHub)" <gi...@apache.org> on 2024/05/27 07:04:01 UTC

[PR] Return clear error message when no common broker found for multi-stage query with tables from different tenants [pinot]

yashmayya opened a new pull request, #13235:
URL: https://github.com/apache/pinot/pull/13235

   - Fixes https://github.com/apache/pinot/issues/11990
   - If there are two tables - `a` (with broker tenant config `BrokerTenant1`) and `b` (with broker tenant config `BrokerTenant2`) and we run a query like `SELECT * FROM a JOIN b ON a.col1 = b.col1;` (through the controller's `POST /sql` API), this results in the ambiguous error message: `ProcessingException(errorCode:450, message:InternalError)`. Even the controller logs contain only the highly misleading error log: `Instance ProcessingException(errorCode:410, message:BrokerResourceMissingError) not found`.
   - This is because the case where no common broker instances (i.e., one with tenant tags for all tables involved in the query) are found [here](https://github.com/apache/pinot/blob/518fd180981ffa614042b8a5236650a2af41bfe5/pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotQueryResource.java#L239) isn't handled appropriately. The `instanceId` returned [here](https://github.com/apache/pinot/blob/518fd180981ffa614042b8a5236650a2af41bfe5/pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotQueryResource.java#L245) will be `ProcessingException(errorCode:410, message:BrokerResourceMissingError)` (from [here](https://github.com/apache/pinot/blob/518fd180981ffa614042b8a5236650a2af41bfe5/pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotQueryResource.java#L322)).
   - The controller then attempts to find a broker with instance ID here `ProcessingException(errorCode:410, message:BrokerResourceMissingError)` which obviously does not exist and results in the internal error being returned here - https://github.com/apache/pinot/blob/518fd180981ffa614042b8a5236650a2af41bfe5/pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotQueryResource.java#L354-L358
   - This patch fixes the issue by returning a clear error message when no common broker is found for a multi-stage query with tables from different tenants. For the above example query `SELECT * FROM a JOIN b ON a.col1 = b.col1;`, the returned error is now:
   ```
   ProcessingException(errorCode:410, message:BrokerResourceMissingError:
   java.lang.Exception: Unable to find a common broker instance for table tenants. Tables: [a, b], Tenants: [BrokerTenant2, BrokerTenant1]
   	at org.apache.pinot.controller.api.resources.PinotQueryResource.getMultiStageQueryResponse(PinotQueryResource.java:244)
   	at org.apache.pinot.controller.api.resources.PinotQueryResource.executeSqlQuery(PinotQueryResource.java:179)
   	at org.apache.pinot.controller.api.resources.PinotQueryResource.handlePostSql(PinotQueryResource.java:127)
   	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method))
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Return clear error message when no common broker found for multi-stage query with tables from different tenants [pinot]

Posted by "yashmayya (via GitHub)" <gi...@apache.org>.
yashmayya commented on code in PR #13235:
URL: https://github.com/apache/pinot/pull/13235#discussion_r1615575781


##########
pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotQueryResource.java:
##########
@@ -237,6 +237,14 @@ private String getMultiStageQueryResponse(String query, String queryOptions, Htt
             String.format("Unable to dispatch multistage query for tables: [%s]", tableNames))).toString();
       }
       instanceIds = findCommonBrokerInstances(brokerTenantsUnion);
+      if (instanceIds.isEmpty()) {
+        // No common broker found for table tenants
+        LOGGER.error("Unable to find a common broker instance for table tenants. Tables: {}, Tenants: {}",
+            tableNames, brokerTenantsUnion);
+        return QueryException.getException(QueryException.BROKER_RESOURCE_MISSING_ERROR,

Review Comment:
   I'm not quite clear on the differences between these two error codes and which one would be more appropriate here - https://github.com/apache/pinot/blob/518fd180981ffa614042b8a5236650a2af41bfe5/pinot-common/src/main/java/org/apache/pinot/common/exception/QueryException.java#L72-L73



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Return clear error message when no common broker found for multi-stage query with tables from different tenants [pinot]

Posted by "vrajat (via GitHub)" <gi...@apache.org>.
vrajat commented on PR #13235:
URL: https://github.com/apache/pinot/pull/13235#issuecomment-2133359392

   is it possible to add a test for this error condition ? 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Return clear error message when no common broker found for multi-stage query with tables from different tenants [pinot]

Posted by "yashmayya (via GitHub)" <gi...@apache.org>.
yashmayya commented on code in PR #13235:
URL: https://github.com/apache/pinot/pull/13235#discussion_r1615579138


##########
pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotQueryResource.java:
##########
@@ -237,6 +237,14 @@ private String getMultiStageQueryResponse(String query, String queryOptions, Htt
             String.format("Unable to dispatch multistage query for tables: [%s]", tableNames))).toString();
       }
       instanceIds = findCommonBrokerInstances(brokerTenantsUnion);
+      if (instanceIds.isEmpty()) {
+        // No common broker found for table tenants
+        LOGGER.error("Unable to find a common broker instance for table tenants. Tables: {}, Tenants: {}",
+            tableNames, brokerTenantsUnion);
+        return QueryException.getException(QueryException.BROKER_RESOURCE_MISSING_ERROR,

Review Comment:
   Ah, looks like `410` is for when no appropriate broker exists in the cluster and `420` is for when the appropriate broker isn't online based on this - https://github.com/apache/pinot/blob/518fd180981ffa614042b8a5236650a2af41bfe5/pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotQueryResource.java#L321-L328
   
   So, I guess `410` / `BROKER_RESOURCE_MISSING_ERROR` is indeed the right one to use here?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Return clear error message when no common broker found for multi-stage query with tables from different tenants [pinot]

Posted by "yashmayya (via GitHub)" <gi...@apache.org>.
yashmayya commented on PR #13235:
URL: https://github.com/apache/pinot/pull/13235#issuecomment-2133383108

   > is it possible to add a test for this error condition ?
   
   There doesn't appear to be a unit test suite for this part of the codebase. I thought it'd be overkill to add an entire new test suite just for this minor issue or even an integration test with a multiple broker setup. So I settled for simply manually testing this fix. If there's a straightforward way to add a test for this that I might be missing though, I'm all ears. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Return clear error message when no common broker found for multi-stage query with tables from different tenants [pinot]

Posted by "gortiz (via GitHub)" <gi...@apache.org>.
gortiz merged PR #13235:
URL: https://github.com/apache/pinot/pull/13235


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [PR] Return clear error message when no common broker found for multi-stage query with tables from different tenants [pinot]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #13235:
URL: https://github.com/apache/pinot/pull/13235#issuecomment-2132844317

   ## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/13235?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   Attention: Patch coverage is `0%` with `4 lines` in your changes are missing coverage. Please review.
   > Project coverage is 0.00%. Comparing base [(`59551e4`)](https://app.codecov.io/gh/apache/pinot/commit/59551e45224f1535c4863fd577622b37366ccc97?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to head [(`d668570`)](https://app.codecov.io/gh/apache/pinot/commit/d6685704678acf833893ad2fd2327fecf860923b?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
   > Report is 499 commits behind head on master.
   
   | [Files](https://app.codecov.io/gh/apache/pinot/pull/13235?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
   |---|---|---|
   | [...t/controller/api/resources/PinotQueryResource.java](https://app.codecov.io/gh/apache/pinot/pull/13235?src=pr&el=tree&filepath=pinot-controller%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fpinot%2Fcontroller%2Fapi%2Fresources%2FPinotQueryResource.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-cGlub3QtY29udHJvbGxlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29udHJvbGxlci9hcGkvcmVzb3VyY2VzL1Bpbm90UXVlcnlSZXNvdXJjZS5qYXZh) | 0.00% | [4 Missing :warning: ](https://app.codecov.io/gh/apache/pinot/pull/13235?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
   
   <details><summary>Additional details and impacted files</summary>
   
   
   ```diff
   @@              Coverage Diff              @@
   ##             master   #13235       +/-   ##
   =============================================
   - Coverage     61.75%    0.00%   -61.76%     
   =============================================
     Files          2436     2459       +23     
     Lines        133233   135351     +2118     
     Branches      20636    20984      +348     
   =============================================
   - Hits          82274        0    -82274     
   - Misses        44911   135351    +90440     
   + Partials       6048        0     -6048     
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/pinot/pull/13235/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [custom-integration1](https://app.codecov.io/gh/apache/pinot/pull/13235/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [integration](https://app.codecov.io/gh/apache/pinot/pull/13235/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `0.00% <0.00%> (-0.01%)` | :arrow_down: |
   | [integration1](https://app.codecov.io/gh/apache/pinot/pull/13235/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [integration2](https://app.codecov.io/gh/apache/pinot/pull/13235/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `0.00% <0.00%> (ø)` | |
   | [java-11](https://app.codecov.io/gh/apache/pinot/pull/13235/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [java-21](https://app.codecov.io/gh/apache/pinot/pull/13235/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `0.00% <0.00%> (-61.63%)` | :arrow_down: |
   | [skip-bytebuffers-false](https://app.codecov.io/gh/apache/pinot/pull/13235/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `0.00% <0.00%> (-61.75%)` | :arrow_down: |
   | [skip-bytebuffers-true](https://app.codecov.io/gh/apache/pinot/pull/13235/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `0.00% <0.00%> (-27.73%)` | :arrow_down: |
   | [temurin](https://app.codecov.io/gh/apache/pinot/pull/13235/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `0.00% <0.00%> (-61.76%)` | :arrow_down: |
   | [unittests](https://app.codecov.io/gh/apache/pinot/pull/13235/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [unittests1](https://app.codecov.io/gh/apache/pinot/pull/13235/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [unittests2](https://app.codecov.io/gh/apache/pinot/pull/13235/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   
   </details>
   
   [:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/pinot/pull/13235?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).   
   :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org