[GitHub] [incubator-pinot] mayankshriv commented on a change in pull request #6361: Detect invalid column names from query in Pinot server

[GitBox <gi...@apache.org>]

mayankshriv commented on a change in pull request #6361:
URL: https://github.com/apache/incubator-pinot/pull/6361#discussion_r568035246



##########
File path: pinot-server/src/main/java/org/apache/pinot/server/starter/helix/HelixInstanceDataManager.java
##########
@@ -326,6 +339,11 @@ public SegmentMetadata getSegmentMetadata(String tableNameWithType, String segme
     }
   }
 
+  @Override
+  public Set<String> getColumnNamesByTable(String tableNameWithType) {
+    return _tableColumnNamesMap.computeIfAbsent(tableNameWithType, k -> Collections.emptySet());

Review comment:
       Can this case memory issue if there are too many tables on an instance? We can check how the overhead of computing the map (if we can do once per query, not segment) compares to the overhead of caching (especially for large MT cluster).
   
   Also, what happens when a table is deleted, is this entry cleared?

##########
File path: pinot-integration-tests/src/test/java/org/apache/pinot/integration/tests/BaseClusterIntegrationTestSet.java
##########
@@ -623,8 +623,10 @@ public void testReload(boolean includeOfflineTable)
         JsonNode testQueryResponse = postQuery(testQuery);
         // Should not throw exception during reload
         assertEquals(testQueryResponse.get("exceptions").size(), 0);
-        // Total docs should not change during reload
-        assertEquals(testQueryResponse.get("totalDocs").asLong(), numTotalDocs);
+        // Total docs should eventually match
+        if (testQueryResponse.get("totalDocs").asLong() != numTotalDocs) {

Review comment:
       Why does the change in this PR need this change?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org