You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2020/06/24 20:44:53 UTC

[GitHub] [incubator-pinot] npawar commented on a change in pull request #5573: Truncate API added

npawar commented on a change in pull request #5573:
URL: https://github.com/apache/incubator-pinot/pull/5573#discussion_r445160636



##########
File path: pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotTableRestletResource.java
##########
@@ -341,6 +344,54 @@ public SuccessResponse updateTableConfig(
 
     return new SuccessResponse("Table config updated for " + tableName);
   }
+  
+  /**
+   * Truncate table, delete all contents of table without removing table configuration and schema. 
+   * 
+   * @param tableName , name of the table to truncate
+   * @param tableTypeStr, type of the table i.e 'offline|realtime'
+   * @return response of truncate operation.
+   * @throws Exception
+   */
+  @POST
+  @Path("/tables/{tableName}/truncate")
+  @Produces(MediaType.APPLICATION_JSON)
+  @ApiOperation(value = "Truncate table")
+  public SuccessResponse truncateTable(
+      @ApiParam(value = "Name of the table to update", required = true) @PathParam("tableName") String tableName,
+      @ApiParam(value = "realtime|offline") @QueryParam("type") String tableTypeStr) throws Exception {

Review comment:
       We still have to define what truncate means for realtime. Typically, when you create a realtime table, it immediately starts consumption. So on truncate, after dropping all segments, we should again restart consumption based on configured offset. 
   That will not happen with these changes. Consumption will be restarted only on the next run of the RealtimeSegmentValidationManager. Which I don't think is an acceptable flow.

##########
File path: pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotTableRestletResource.java
##########
@@ -341,6 +344,54 @@ public SuccessResponse updateTableConfig(
 
     return new SuccessResponse("Table config updated for " + tableName);
   }
+  
+  /**
+   * Truncate table, delete all contents of table without removing table configuration and schema. 
+   * 
+   * @param tableName , name of the table to truncate
+   * @param tableTypeStr, type of the table i.e 'offline|realtime'
+   * @return response of truncate operation.
+   * @throws Exception
+   */
+  @POST
+  @Path("/tables/{tableName}/truncate")
+  @Produces(MediaType.APPLICATION_JSON)
+  @ApiOperation(value = "Truncate table")
+  public SuccessResponse truncateTable(
+      @ApiParam(value = "Name of the table to update", required = true) @PathParam("tableName") String tableName,
+      @ApiParam(value = "realtime|offline") @QueryParam("type") String tableTypeStr) throws Exception {
+
+    TableType tableType = Constants.validateTableType(tableTypeStr);
+    if (tableType == null) {
+      throw new ControllerApplicationException(LOGGER, "Table type must not be null", Response.Status.BAD_REQUEST);
+    }
+
+    // Disable table on tableType
+    if (tableType != TableType.REALTIME && _pinotHelixResourceManager.hasOfflineTable(tableName)) {
+      String tableNameWithType = TableNameBuilder.OFFLINE.tableNameWithType(tableName);
+      // disable offline table
+      _pinotHelixResourceManager.toggleTableState(tableNameWithType, StateType.DISABLE);
+    }
+    if (tableType != TableType.OFFLINE && _pinotHelixResourceManager.hasOfflineTable(tableName)) {
+      String tableNameWithType = TableNameBuilder.REALTIME.tableNameWithType(tableName);
+      // disable real-time table
+      _pinotHelixResourceManager.toggleTableState(tableNameWithType, StateType.DISABLE);
+
+    }
+
+    // Get all segment names for table and delete all segments
+    String tableNameWithType = getExistingTableNamesWithType(tableName, tableType).get(0);
+    List<String> segmentNames = _pinotHelixResourceManager.getSegmentsFor(tableNameWithType);
+    PinotResourceManagerResponse response = _pinotHelixResourceManager.deleteSegments(tableNameWithType, segmentNames);
+
+    if (!response.isSuccessful()) {
+      throw new ControllerApplicationException(LOGGER,
+          "Failed to delete segments from table: " + tableNameWithType + ", error message: " + response.getMessage(),
+          Response.Status.INTERNAL_SERVER_ERROR);
+    }
+
+    return new SuccessResponse("Table " + tableName + " successfully truncated");

Review comment:
       shouldn't we re-enable the table?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org