You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@pinot.apache.org by "tibrewalpratik17 (via GitHub)" <gi...@apache.org> on 2023/03/01 17:28:03 UTC

[GitHub] [pinot] tibrewalpratik17 opened a new pull request, #10363: Add additional metadata from Zookeeper in all segments metadata response

tibrewalpratik17 opened a new pull request, #10363:
URL: https://github.com/apache/pinot/pull/10363

   `/segments/{tableName}/metadata` returns the metadata for all table segments from server. This misses out on Zookeeper metadata like `downloadUrl`, `segmentPartitionMetadata` which can be obtained only by hitting this API `/segments/{tableName}/{segmentName}/metadata` for each segment separately.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #10363: Add controller API to fetch all segment zookeeper metadata for a table

Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.

Jackie-Jiang commented on code in PR #10363:
URL: https://github.com/apache/pinot/pull/10363#discussion_r1125223147


##########
pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotSegmentRestletResource.java:
##########
@@ -901,6 +902,37 @@ public String getServerMetadata(
     return segmentsMetadata;
   }
 
+  @GET
+  @Path("segments/{tableName}/zkmetadata")
+  @Produces(MediaType.APPLICATION_JSON)
+  @ApiOperation(value = "Get the zookeeper metadata for all table segments", notes = "Get the zookeeper metadata for "
+      + "all table segments")
+  public Map<String, Map<String, Object>> getZookeeperMetadata(
+      @ApiParam(value = "Name of the table", required = true) @PathParam("tableName") String tableName,
+      @ApiParam(value = "OFFLINE|REALTIME") @QueryParam("type") String tableTypeStr)
+      throws JsonProcessingException {
+    LOGGER.info("Received a request to fetch zookeeper metadata for all segments for table {}", tableName);
+    TableType tableType = Constants.validateTableType(tableTypeStr);
+
+    List<String> tableNamesWithType =
+        ResourceUtils.getExistingTableNamesWithType(_pinotHelixResourceManager, tableName, tableType, LOGGER);
+    String tableNameWithType;
+    if (tableNamesWithType.size() == 1) {
+      tableNameWithType = tableNamesWithType.get(0);
+    } else {
+      tableNameWithType = TableNameBuilder.OFFLINE.tableNameWithType(tableName);
+    }
+    Map<String, Map<String, Object>> segmentToMetadataMap = new HashMap<>();
+    List<SegmentZKMetadata> segmentZKMetadataList =
+        _pinotHelixResourceManager.getSegmentsZKMetadata(tableNameWithType);
+
+    for (SegmentZKMetadata segmentZKMetadata : segmentZKMetadataList) {
+      Map<String, Object> segmentZKMetadataObject = new HashMap<>(segmentZKMetadata.toMap());

Review Comment:
   No need to make a shallow copy of the map. The return type is always `Map<String, String>`



##########
pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotSegmentRestletResource.java:
##########
@@ -901,6 +902,37 @@ public String getServerMetadata(
     return segmentsMetadata;
   }
 
+  @GET
+  @Path("segments/{tableName}/zkmetadata")
+  @Produces(MediaType.APPLICATION_JSON)
+  @ApiOperation(value = "Get the zookeeper metadata for all table segments", notes = "Get the zookeeper metadata for "
+      + "all table segments")
+  public Map<String, Map<String, Object>> getZookeeperMetadata(
+      @ApiParam(value = "Name of the table", required = true) @PathParam("tableName") String tableName,
+      @ApiParam(value = "OFFLINE|REALTIME") @QueryParam("type") String tableTypeStr)
+      throws JsonProcessingException {
+    LOGGER.info("Received a request to fetch zookeeper metadata for all segments for table {}", tableName);
+    TableType tableType = Constants.validateTableType(tableTypeStr);
+
+    List<String> tableNamesWithType =
+        ResourceUtils.getExistingTableNamesWithType(_pinotHelixResourceManager, tableName, tableType, LOGGER);
+    String tableNameWithType;
+    if (tableNamesWithType.size() == 1) {
+      tableNameWithType = tableNamesWithType.get(0);
+    } else {
+      tableNameWithType = TableNameBuilder.OFFLINE.tableNameWithType(tableName);
+    }

Review Comment:
   This is equivalent to
   ```suggestion
       String tableNameWithType =
           ResourceUtils.getExistingTableNamesWithType(_pinotHelixResourceManager, tableName, tableType, LOGGER).get(0);
   ```
   
   Because when getting the existing table, we always put OFFLINE in front of REALTIME if both exist



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org

[GitHub] [pinot] Jackie-Jiang merged pull request #10363: Add controller API to fetch all segment zookeeper metadata for a table

Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.

Jackie-Jiang merged PR #10363:
URL: https://github.com/apache/pinot/pull/10363


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org