You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "yashmayya (via GitHub)" <gi...@apache.org> on 2025/08/07 08:11:36 UTC
[PR] Add controller REST API to compile / validate MSE query [pinot]
yashmayya opened a new pull request, #16536:
URL: https://github.com/apache/pinot/pull/16536
- Sample requests and responses:
```
POST /validateMultiStageQuery {"sql": "SELECT invalidColumn FROM invalidTable"}
{
"errorMessage": "From line 1, column 27 to line 1, column 38: Object 'invalidTable' not found",
"errorCode": "TABLE_DOES_NOT_EXIST",
"compiledSuccessfully": false
}
```
```
POST /validateMultiStageQuery {"sql": "SELECT * FROM myTable"}
{
"errorMessage": null,
"errorCode": null,
"compiledSuccessfully": true
}
```
```
POST /validateMultiStageQuery {"sql": "SELECT CAST('abc' AS INT)"}
{
"errorMessage": "Error optimizing query: Caught exception while invoking method: cast with arguments: [abc, INT]: For input string: \"abc\"",
"errorCode": "QUERY_PLANNING",
"compiledSuccessfully": false
}
```
- This API can be useful for testing and during upgrades to ensure backward compatibility.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Add controller REST API to compile / validate MSE query [pinot]
Posted by "yashmayya (via GitHub)" <gi...@apache.org>.
yashmayya commented on code in PR #16536:
URL: https://github.com/apache/pinot/pull/16536#discussion_r2260444452
##########
pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotQueryResource.java:
##########
@@ -151,6 +151,59 @@ public StreamingOutput handleTimeSeriesQueryRange(@QueryParam("language") String
return executeTimeSeriesQueryCatching(httpHeaders, language, query, start, end, step);
}
+ @POST
+ @Path("validateMultiStageQuery")
+ public MultiStageQueryValidationResponse validateMultiStageQuery(String requestJsonStr,
+ @Context HttpHeaders httpHeaders) {
+ JsonNode requestJson;
+ try {
+ requestJson = JsonUtils.stringToJsonNode(requestJsonStr);
+ } catch (Exception e) {
+ LOGGER.warn("Caught exception while parsing request {}", e.getMessage());
+ return new MultiStageQueryValidationResponse(false, "Failed to parse request JSON: " + e.getMessage(), null);
+ }
+ if (!requestJson.has("sql")) {
+ return new MultiStageQueryValidationResponse(false, "JSON Payload is missing the query string field 'sql'", null);
+ }
+ String sqlQuery = requestJson.get("sql").asText();
+ Map<String, String> queryOptionsMap = RequestUtils.parseQuery(sqlQuery).getOptions();
+ String database = DatabaseUtils.extractDatabaseFromQueryRequest(queryOptionsMap, httpHeaders);
+ try (QueryEnvironment.CompiledQuery compiledQuery = new QueryEnvironment(database,
+ _pinotHelixResourceManager.getTableCache(), null).compile(sqlQuery)) {
+ return new MultiStageQueryValidationResponse(true, null, null);
+ } catch (QueryException e) {
+ LOGGER.info("Caught exception while compiling multi-stage query: {}", e.getMessage());
+ return new MultiStageQueryValidationResponse(false, e.getMessage(), e.getErrorCode());
+ }
+ }
+
+ private static class MultiStageQueryValidationResponse {
Review Comment:
I'm only using the class for JSON serde, but fair point.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Add controller REST API to compile / validate MSE query [pinot]
Posted by "gortiz (via GitHub)" <gi...@apache.org>.
gortiz commented on code in PR #16536:
URL: https://github.com/apache/pinot/pull/16536#discussion_r2260436730
##########
pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotQueryResource.java:
##########
@@ -151,6 +151,59 @@ public StreamingOutput handleTimeSeriesQueryRange(@QueryParam("language") String
return executeTimeSeriesQueryCatching(httpHeaders, language, query, start, end, step);
}
+ @POST
+ @Path("validateMultiStageQuery")
+ public MultiStageQueryValidationResponse validateMultiStageQuery(String requestJsonStr,
+ @Context HttpHeaders httpHeaders) {
+ JsonNode requestJson;
+ try {
+ requestJson = JsonUtils.stringToJsonNode(requestJsonStr);
+ } catch (Exception e) {
+ LOGGER.warn("Caught exception while parsing request {}", e.getMessage());
+ return new MultiStageQueryValidationResponse(false, "Failed to parse request JSON: " + e.getMessage(), null);
+ }
+ if (!requestJson.has("sql")) {
+ return new MultiStageQueryValidationResponse(false, "JSON Payload is missing the query string field 'sql'", null);
+ }
+ String sqlQuery = requestJson.get("sql").asText();
+ Map<String, String> queryOptionsMap = RequestUtils.parseQuery(sqlQuery).getOptions();
+ String database = DatabaseUtils.extractDatabaseFromQueryRequest(queryOptionsMap, httpHeaders);
+ try (QueryEnvironment.CompiledQuery compiledQuery = new QueryEnvironment(database,
+ _pinotHelixResourceManager.getTableCache(), null).compile(sqlQuery)) {
+ return new MultiStageQueryValidationResponse(true, null, null);
+ } catch (QueryException e) {
+ LOGGER.info("Caught exception while compiling multi-stage query: {}", e.getMessage());
+ return new MultiStageQueryValidationResponse(false, e.getMessage(), e.getErrorCode());
+ }
+ }
+
+ private static class MultiStageQueryValidationResponse {
Review Comment:
A public method should not have a private class as a return type.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Add controller REST API to compile / validate MSE query [pinot]
Posted by "yashmayya (via GitHub)" <gi...@apache.org>.
yashmayya merged PR #16536:
URL: https://github.com/apache/pinot/pull/16536
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Add controller REST API to compile / validate MSE query [pinot]
Posted by "gortiz (via GitHub)" <gi...@apache.org>.
gortiz commented on code in PR #16536:
URL: https://github.com/apache/pinot/pull/16536#discussion_r2260439562
##########
pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/MultiStageEngineIntegrationTest.java:
##########
@@ -1684,6 +1682,31 @@ public void testPolymorphicScalarArrayFunctions() throws Exception {
assertEquals(jsonNode.get("resultTable").get("rows").get(0).get(0).asInt(), 2);
}
+ @Test
+ public void testValidateQueryApi()
+ throws Exception {
+ JsonNode result = JsonUtils.stringToJsonNode(
+ sendPostRequest(getControllerBaseApiUrl() + "/validateMultiStageQuery",
+ "{\"sql\": \"SELECT invalidColumn FROM invalidTable\"}", null));
+ assertFalse(result.get("compiledSuccessfully").asBoolean());
+ assertEquals(result.get("errorCode").asText(), QueryErrorCode.TABLE_DOES_NOT_EXIST.name());
+ assertFalse(result.get("errorMessage").isNull());
+
+ result = JsonUtils.stringToJsonNode(
+ sendPostRequest(getControllerBaseApiUrl() + "/validateMultiStageQuery",
+ "{\"sql\": \"SELECT * FROM mytable\"}", null));
+ assertTrue(result.get("compiledSuccessfully").asBoolean());
+ assertTrue(result.get("errorCode").isNull());
+ assertTrue(result.get("errorMessage").isNull());
+
+ result = JsonUtils.stringToJsonNode(
+ sendPostRequest(getControllerBaseApiUrl() + "/validateMultiStageQuery",
+ "{\"sql\": \"SELECT CAST('abc' AS INT)\"}", null));
+ assertFalse(result.get("compiledSuccessfully").asBoolean());
+ assertEquals(result.get("errorCode").asText(), QueryErrorCode.QUERY_PLANNING.name());
+ assertFalse(result.get("errorMessage").isNull());
+ }
+
Review Comment:
Can we have a single test for each case?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Add controller REST API to compile / validate MSE query [pinot]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #16536:
URL: https://github.com/apache/pinot/pull/16536#issuecomment-3163187007
## [Codecov](https://app.codecov.io/gh/apache/pinot/pull/16536?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 56.33%. Comparing base ([`77d6de9`](https://app.codecov.io/gh/apache/pinot/commit/77d6de9d42ce7455996296f61b4258a4d02415c0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`7584e62`](https://app.codecov.io/gh/apache/pinot/commit/7584e6226d3755373dc1d91616606b6e34d91b68?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 11 commits behind head on master.
> :exclamation: There is a different number of reports uploaded between BASE (77d6de9) and HEAD (7584e62). Click for more details.
>
> <details><summary>HEAD has 36 uploads less than BASE</summary>
>
>| Flag | BASE (77d6de9) | HEAD (7584e62) |
>|------|------|------|
>|java-21|5|1|
>|unittests1|2|1|
>|unittests|4|1|
>|temurin|10|1|
>|java-11|5|0|
>|unittests2|2|0|
>|integration|6|0|
>|integration1|2|0|
>|custom-integration1|2|0|
>|integration2|2|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #16536 +/- ##
============================================
- Coverage 63.21% 56.33% -6.88%
+ Complexity 1362 699 -663
============================================
Files 3012 2356 -656
Lines 174298 129598 -44700
Branches 26686 20603 -6083
============================================
- Hits 110179 73010 -37169
+ Misses 55696 50568 -5128
+ Partials 8423 6020 -2403
```
| [Flag](https://app.codecov.io/gh/apache/pinot/pull/16536/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [custom-integration1](https://app.codecov.io/gh/apache/pinot/pull/16536/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration](https://app.codecov.io/gh/apache/pinot/pull/16536/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration1](https://app.codecov.io/gh/apache/pinot/pull/16536/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [integration2](https://app.codecov.io/gh/apache/pinot/pull/16536/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [java-11](https://app.codecov.io/gh/apache/pinot/pull/16536/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
| [java-21](https://app.codecov.io/gh/apache/pinot/pull/16536/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `56.33% <ø> (-6.86%)` | :arrow_down: |
| [temurin](https://app.codecov.io/gh/apache/pinot/pull/16536/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `56.33% <ø> (-6.88%)` | :arrow_down: |
| [unittests](https://app.codecov.io/gh/apache/pinot/pull/16536/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `56.33% <ø> (-6.88%)` | :arrow_down: |
| [unittests1](https://app.codecov.io/gh/apache/pinot/pull/16536/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `56.33% <ø> (-0.04%)` | :arrow_down: |
| [unittests2](https://app.codecov.io/gh/apache/pinot/pull/16536/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/pinot/pull/16536?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org