You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2022/08/29 03:18:31 UTC
[GitHub] [pinot] npawar commented on a diff in pull request #9289: Add capabilities to ingest from another stream without disabling the realtime table
npawar commented on code in PR #9289:
URL: https://github.com/apache/pinot/pull/9289#discussion_r956857703
##########
pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotRealtimeTableResource.java:
##########
@@ -77,14 +77,29 @@ public Response pauseConsumption(
@POST
@Path("/tables/{tableName}/resumeConsumption")
@Produces(MediaType.APPLICATION_JSON)
- @ApiOperation(value = "Resume consumption of a realtime table",
- notes = "Resume the consumption for a realtime table")
+ @ApiOperation(value = "Resume consumption of a realtime table", notes =
+ "Resume the consumption for a realtime table. ConsumeFrom parameter indicates from which offsets "
+ + "consumption should resume. If consumeFrom parameter is not provided or 'best' option is chosen, "
+ + "consumption continues based on the offsets in segment ZK metadata, and in case the offsets are already "
+ + "gone, the first available offsets are picked to minimize the data loss.")
public Response resumeConsumption(
- @ApiParam(value = "Name of the table", required = true) @PathParam("tableName") String tableName) {
+ @ApiParam(value = "Name of the table", required = true) @PathParam("tableName") String tableName,
+ @ApiParam(value = "earliest | latest | best") @QueryParam("consumeFrom") String consumeFrom) {
String tableNameWithType = TableNameBuilder.REALTIME.tableNameWithType(tableName);
validate(tableNameWithType);
+ String offsetCriteria;
+ if ("earliest".equalsIgnoreCase(consumeFrom)) {
Review Comment:
why not just keep this as smallest/largest, since user is already familiar with those for table consumption?
##########
pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotRealtimeTableResource.java:
##########
@@ -77,14 +77,29 @@ public Response pauseConsumption(
@POST
@Path("/tables/{tableName}/resumeConsumption")
@Produces(MediaType.APPLICATION_JSON)
- @ApiOperation(value = "Resume consumption of a realtime table",
- notes = "Resume the consumption for a realtime table")
+ @ApiOperation(value = "Resume consumption of a realtime table", notes =
+ "Resume the consumption for a realtime table. ConsumeFrom parameter indicates from which offsets "
+ + "consumption should resume. If consumeFrom parameter is not provided or 'best' option is chosen, "
+ + "consumption continues based on the offsets in segment ZK metadata, and in case the offsets are already "
+ + "gone, the first available offsets are picked to minimize the data loss.")
public Response resumeConsumption(
- @ApiParam(value = "Name of the table", required = true) @PathParam("tableName") String tableName) {
+ @ApiParam(value = "Name of the table", required = true) @PathParam("tableName") String tableName,
+ @ApiParam(value = "earliest | latest | best") @QueryParam("consumeFrom") String consumeFrom) {
String tableNameWithType = TableNameBuilder.REALTIME.tableNameWithType(tableName);
validate(tableNameWithType);
+ String offsetCriteria;
+ if ("earliest".equalsIgnoreCase(consumeFrom)) {
+ offsetCriteria = "smallest";
+ } else if ("latest".equalsIgnoreCase(consumeFrom)) {
+ offsetCriteria = "largest";
+ } else if (consumeFrom == null || "best".equalsIgnoreCase(consumeFrom)) {
+ offsetCriteria = null;
+ } else {
Review Comment:
should we have the timestamp and period offset criteria also, now that we support those too?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org