[GitHub] [incubator-pinot] mayankshriv commented on a change in pull request #6361: Detect invalid column names from query in Pinot server

[GitBox <gi...@apache.org>]

mayankshriv commented on a change in pull request #6361:
URL: https://github.com/apache/incubator-pinot/pull/6361#discussion_r545966725



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/query/executor/ServerQueryExecutorV1Impl.java
##########
@@ -259,6 +260,18 @@ private DataTable processQuery(List<IndexSegment> indexSegments, QueryContext qu
       ExecutorService executorService, @Nullable StreamObserver<Server.ServerResponse> responseObserver, long endTimeMs,
       boolean enableStreaming)
       throws Exception {
+
+    // Validate whether column names in the query are valid
+    Set<String> columnNamesFromSchema = _instanceDataManager.getColumnNamesByTable(queryContext.getTableName());
+    Set<String> columnNamesFromQuery = queryContext.getColumns();
+    if (!columnNamesFromSchema.isEmpty() && !columnNamesFromSchema.containsAll(columnNamesFromQuery)) {

Review comment:
       Is the empty check warranted?
   
   Also, does this work with aliasing? For example, `select foo as bar from table`?

##########
File path: pinot-common/src/main/java/org/apache/pinot/common/metrics/BrokerMeter.java
##########
@@ -87,6 +87,7 @@
 
   GROUP_BY_SIZE("queries", false),
   TOTAL_SERVER_RESPONSE_SIZE("queries", false),
+  QUERY_COLUMN_NAME_MISMATCH("queries", false),

Review comment:
       It is more of a case of column name in query missing from schema, than a column name mismatch, right?

##########
File path: pinot-core/src/main/java/org/apache/pinot/core/query/executor/ServerQueryExecutorV1Impl.java
##########
@@ -259,6 +260,18 @@ private DataTable processQuery(List<IndexSegment> indexSegments, QueryContext qu
       ExecutorService executorService, @Nullable StreamObserver<Server.ServerResponse> responseObserver, long endTimeMs,
       boolean enableStreaming)
       throws Exception {
+
+    // Validate whether column names in the query are valid
+    Set<String> columnNamesFromSchema = _instanceDataManager.getColumnNamesByTable(queryContext.getTableName());
+    Set<String> columnNamesFromQuery = queryContext.getColumns();
+    if (!columnNamesFromSchema.isEmpty() && !columnNamesFromSchema.containsAll(columnNamesFromQuery)) {
+      columnNamesFromQuery.removeAll(columnNamesFromSchema);

Review comment:
       Using `containsAll()` and `removeAll` seems inefficient.

##########
File path: pinot-core/src/main/java/org/apache/pinot/core/query/executor/ServerQueryExecutorV1Impl.java
##########
@@ -259,6 +260,18 @@ private DataTable processQuery(List<IndexSegment> indexSegments, QueryContext qu
       ExecutorService executorService, @Nullable StreamObserver<Server.ServerResponse> responseObserver, long endTimeMs,
       boolean enableStreaming)
       throws Exception {
+
+    // Validate whether column names in the query are valid

Review comment:
       I am a bit hesitant to add additional check to penalize all queries for catching an infrequent error that.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org