You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/05/30 13:22:49 UTC
[GitHub] [pulsar] leizhiyuan opened a new pull request, #15843: fix: allow delete topic async
leizhiyuan opened a new pull request, #15843:
URL: https://github.com/apache/pulsar/pull/15843
<!--
### Contribution Checklist
- PR title format should be *[type][component] summary*. For details, see *[Guideline - Pulsar PR Naming Convention](https://docs.google.com/document/d/1d8Pw6ZbWk-_pCKdOmdvx9rnhPiyuxwq60_TrD68d7BA/edit#heading=h.trs9rsex3xom)*.
- Fill out the template below to describe the changes contributed by the pull request. That will give reviewers the context they need to do the review.
- Each pull request should address only one issue, not mix up code from multiple issues.
- Each commit in the pull request has a meaningful commit message
- Once all items of the checklist are addressed, remove the above text and this checklist, leaving only the filled out template below.
**(The sections below can be removed for hotfixes of typos)**
-->
*(If this PR fixes a github issue, please add `Fixes #<xyz>`.)*
Fixes #15841
*(or if this PR is one task of a github issue, please add `Master Issue: #<xyz>` to link to the master issue.)*
Master Issue: #<xyz>
### Motivation
*Explain here the context, and why you're making that change. What is the problem you're trying to solve.*
### Modifications
*Describe the modifications you've done.*
### Verifying this change
- [ ] Make sure that the change passes the CI checks.
*(Please pick either of the following options)*
This change is a trivial rework / code cleanup without any test coverage.
*(or)*
This change is already covered by existing tests, such as *(please describe tests)*.
*(or)*
This change added tests and can be verified as follows:
*(example:)*
- *Added integration tests for end-to-end deployment with large payloads (10MB)*
- *Extended integration test for recovery after broker failure*
### Does this pull request potentially affect one of the following parts:
*If `yes` was chosen, please highlight the changes*
- Dependencies (does it add or upgrade a dependency): (yes / no)
- The public API: (yes / no)
- The schema: (yes / no / don't know)
- The default values of configurations: (yes / no)
- The wire protocol: (yes / no)
- The rest endpoints: (yes / no)
- The admin cli options: (yes / no)
- Anything that affects deployment: (yes / no / don't know)
### Documentation
Check the box below or label this PR directly.
Need to update docs?
- [ ] `doc-required`
(Your PR needs to update docs and you will update later)
- [ ] `doc-not-needed`
(Please explain why)
- [ ] `doc`
(Your PR contains doc changes)
- [ ] `doc-complete`
(Docs have been already added)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #15843: fix: allow delete topic async
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #15843:
URL: https://github.com/apache/pulsar/pull/15843#issuecomment-1141166153
@leizhiyuan:Thanks for providing doc info!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Technoboy- commented on a diff in pull request #15843: [improve][broker] Make delete topic async
Posted by GitBox <gi...@apache.org>.
Technoboy- commented on code in PR #15843:
URL: https://github.com/apache/pulsar/pull/15843#discussion_r885336858
##########
pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java:
##########
@@ -1003,32 +1007,37 @@ private void internalUnloadTransactionCoordinatorAsync(AsyncResponse asyncRespon
});
}
- protected void internalDeleteTopic(boolean authoritative, boolean force) {
+ protected CompletableFuture<Void> internalDeleteTopic(boolean authoritative,
+ boolean force) {
if (force) {
- internalDeleteTopicForcefully(authoritative);
+ return internalDeleteTopicForcefully(authoritative);
} else {
- internalDeleteTopic(authoritative);
+ return internalDeleteTopic(authoritative);
}
}
- protected void internalDeleteTopic(boolean authoritative) {
- validateNamespaceOperation(topicName.getNamespaceObject(), NamespaceOperation.DELETE_TOPIC);
- validateTopicOwnership(topicName, authoritative);
+ protected CompletableFuture<Void> internalDeleteTopic(boolean authoritative) {
+ CompletableFuture<Void> ret;
+ ret = validateTopicOwnershipAsync(topicName, authoritative).thenCompose(
+ __ -> validateNamespaceOperationAsync(topicName.getNamespaceObject(),
+ NamespaceOperation.DELETE_TOPIC))
+ .thenCompose(__ -> pulsar().getBrokerService().deleteTopic(topicName.toString(), false))
+ .exceptionally(ex -> {
+ Throwable realCause = FutureUtil.unwrapCompletionException(ex);
+ if (realCause instanceof TopicBusyException) {
+ throw new RestException(Status.PRECONDITION_FAILED,
+ "Topic has active producers/subscriptions");
+ } else if (isManagedLedgerNotFoundException(realCause)) {
+ log.info("[{}] Topic was already not existing {}", clientAppId(), topicName, realCause);
+ throw new RestException(Status.NOT_FOUND,
+ getTopicNotFoundErrorMessage(topicName.toString()));
+ } else {
+ log.error("[{}] Failed to delete topic {}", clientAppId(), topicName, realCause);
Review Comment:
How about moving this log to the rest controller?
See below comment.
##########
pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/v1/PersistentTopics.java:
##########
@@ -357,12 +357,19 @@ public void unloadTopic(@Suspended final AsyncResponse asyncResponse, @PathParam
@ApiResponse(code = 403, message = "Don't have admin permission"),
@ApiResponse(code = 404, message = "Topic does not exist"),
@ApiResponse(code = 412, message = "Topic has active producers/subscriptions")})
- public void deleteTopic(@PathParam("property") String property, @PathParam("cluster") String cluster,
+ public void deleteTopic(
+ @Suspended final AsyncResponse asyncResponse,
+ @PathParam("property") String property, @PathParam("cluster") String cluster,
@PathParam("namespace") String namespace, @PathParam("topic") @Encoded String encodedTopic,
@QueryParam("force") @DefaultValue("false") boolean force,
@QueryParam("authoritative") @DefaultValue("false") boolean authoritative) {
validateTopicName(property, cluster, namespace, encodedTopic);
- internalDeleteTopic(authoritative, force);
+ internalDeleteTopic(authoritative, force).thenAccept(
+ __ -> asyncResponse.resume(Response.noContent().build()))
+ .exceptionally(ex -> {
+ resumeAsyncResponseExceptionally(asyncResponse, ex.getCause());
Review Comment:
Add error log here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #15843: fix: allow delete topic async
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #15843:
URL: https://github.com/apache/pulsar/pull/15843#issuecomment-1141162981
@leizhiyuan:Thanks for your contribution. For this PR, do we need to update docs?
(The [PR template contains info about doc](https://github.com/apache/pulsar/blob/master/.github/PULL_REQUEST_TEMPLATE.md#documentation), which helps others know more about the changes. Can you provide doc-related info in this and future PR descriptions? Thanks)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Technoboy- commented on a diff in pull request #15843: [improve][broker] Make delete topic async
Posted by GitBox <gi...@apache.org>.
Technoboy- commented on code in PR #15843:
URL: https://github.com/apache/pulsar/pull/15843#discussion_r885146165
##########
pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java:
##########
@@ -322,20 +322,23 @@ protected void internalGrantPermissionsOnTopic(final AsyncResponse asyncResponse
});
}
- protected void internalDeleteTopicForcefully(boolean authoritative) {
- validateTopicOwnership(topicName, authoritative);
- validateNamespaceOperation(topicName.getNamespaceObject(), NamespaceOperation.DELETE_TOPIC);
+ protected void internalDeleteTopicForcefully(AsyncResponse asyncResponse, boolean authoritative) {
Review Comment:
`AsyncResponse` is not allowed to pass here. Please ref https://github.com/apache/pulsar/issues/14365
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] leizhiyuan commented on a diff in pull request #15843: fix: allow delete topic async
Posted by GitBox <gi...@apache.org>.
leizhiyuan commented on code in PR #15843:
URL: https://github.com/apache/pulsar/pull/15843#discussion_r885117089
##########
pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java:
##########
@@ -1003,32 +1006,36 @@ private void internalUnloadTransactionCoordinatorAsync(AsyncResponse asyncRespon
});
}
- protected void internalDeleteTopic(boolean authoritative, boolean force) {
+ protected void internalDeleteTopic(AsyncResponse asyncResponse, boolean authoritative, boolean force) {
if (force) {
- internalDeleteTopicForcefully(authoritative);
+ internalDeleteTopicForcefully(asyncResponse, authoritative);
} else {
- internalDeleteTopic(authoritative);
+ internalDeleteTopic(asyncResponse, authoritative);
}
}
- protected void internalDeleteTopic(boolean authoritative) {
- validateNamespaceOperation(topicName.getNamespaceObject(), NamespaceOperation.DELETE_TOPIC);
- validateTopicOwnership(topicName, authoritative);
+ protected void internalDeleteTopic(AsyncResponse asyncResponse, boolean authoritative) {
- try {
- pulsar().getBrokerService().deleteTopic(topicName.toString(), false).get();
- log.info("[{}] Successfully removed topic {}", clientAppId(), topicName);
- } catch (Exception e) {
- Throwable t = e.getCause();
- log.error("[{}] Failed to delete topic {}", clientAppId(), topicName, t);
- if (t instanceof TopicBusyException) {
- throw new RestException(Status.PRECONDITION_FAILED, "Topic has active producers/subscriptions");
- } else if (isManagedLedgerNotFoundException(e)) {
- throw new RestException(Status.NOT_FOUND, getTopicNotFoundErrorMessage(topicName.toString()));
- } else {
- throw new RestException(t);
- }
- }
+ validateTopicOwnershipAsync(topicName, authoritative).thenCompose(
+ __ -> validateNamespaceOperationAsync(topicName.getNamespaceObject(),
+ NamespaceOperation.DELETE_TOPIC))
+ .thenAccept(__ -> pulsar().getBrokerService().deleteTopic(topicName.toString(), false))
Review Comment:
thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on pull request #15843: [improve][broker] Make delete topic async
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #15843:
URL: https://github.com/apache/pulsar/pull/15843#issuecomment-1172996966
The pr had no activity for 30 days, mark with Stale label.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] leizhiyuan commented on a diff in pull request #15843: [improve][broker] Make delete topic async
Posted by GitBox <gi...@apache.org>.
leizhiyuan commented on code in PR #15843:
URL: https://github.com/apache/pulsar/pull/15843#discussion_r885407281
##########
pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/v2/PersistentTopics.java:
##########
@@ -1011,7 +1012,12 @@ public void deleteTopic(
@ApiParam(value = "Is authentication required to perform this operation")
@QueryParam("authoritative") @DefaultValue("false") boolean authoritative) {
validateTopicName(tenant, namespace, encodedTopic);
- internalDeleteTopic(authoritative, force);
+ internalDeleteTopic(authoritative, force).thenAccept(
+ __ -> asyncResponse.resume(Response.noContent().build()))
+ .exceptionally(ex -> {
+ resumeAsyncResponseExceptionally(asyncResponse, ex.getCause());
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] AnonHxy commented on a diff in pull request #15843: fix: allow delete topic async
Posted by GitBox <gi...@apache.org>.
AnonHxy commented on code in PR #15843:
URL: https://github.com/apache/pulsar/pull/15843#discussion_r884996988
##########
pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java:
##########
@@ -1003,32 +1006,36 @@ private void internalUnloadTransactionCoordinatorAsync(AsyncResponse asyncRespon
});
}
- protected void internalDeleteTopic(boolean authoritative, boolean force) {
+ protected void internalDeleteTopic(AsyncResponse asyncResponse, boolean authoritative, boolean force) {
if (force) {
- internalDeleteTopicForcefully(authoritative);
+ internalDeleteTopicForcefully(asyncResponse, authoritative);
} else {
- internalDeleteTopic(authoritative);
+ internalDeleteTopic(asyncResponse, authoritative);
}
}
- protected void internalDeleteTopic(boolean authoritative) {
- validateNamespaceOperation(topicName.getNamespaceObject(), NamespaceOperation.DELETE_TOPIC);
- validateTopicOwnership(topicName, authoritative);
+ protected void internalDeleteTopic(AsyncResponse asyncResponse, boolean authoritative) {
- try {
- pulsar().getBrokerService().deleteTopic(topicName.toString(), false).get();
- log.info("[{}] Successfully removed topic {}", clientAppId(), topicName);
- } catch (Exception e) {
- Throwable t = e.getCause();
- log.error("[{}] Failed to delete topic {}", clientAppId(), topicName, t);
- if (t instanceof TopicBusyException) {
- throw new RestException(Status.PRECONDITION_FAILED, "Topic has active producers/subscriptions");
- } else if (isManagedLedgerNotFoundException(e)) {
- throw new RestException(Status.NOT_FOUND, getTopicNotFoundErrorMessage(topicName.toString()));
- } else {
- throw new RestException(t);
- }
- }
+ validateTopicOwnershipAsync(topicName, authoritative).thenCompose(
+ __ -> validateNamespaceOperationAsync(topicName.getNamespaceObject(),
+ NamespaceOperation.DELETE_TOPIC))
+ .thenAccept(__ -> pulsar().getBrokerService().deleteTopic(topicName.toString(), false))
Review Comment:
`deleteTopic` return `CompletableFuture<Void>`, so we need `thenCompose` here:
https://github.com/apache/pulsar/blob/b367b9153919b83fad829669f1f96281a2a8d8d5/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/BrokerService.java#L1018-L1024
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] gaozhangmin commented on pull request #15843: [improve][broker] Make delete topic async
Posted by GitBox <gi...@apache.org>.
gaozhangmin commented on PR #15843:
URL: https://github.com/apache/pulsar/pull/15843#issuecomment-1221929994
duplicate with #16232
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] gaozhangmin closed pull request #15843: [improve][broker] Make delete topic async
Posted by GitBox <gi...@apache.org>.
gaozhangmin closed pull request #15843: [improve][broker] Make delete topic async
URL: https://github.com/apache/pulsar/pull/15843
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Technoboy- commented on a diff in pull request #15843: [improve][broker] Make delete topic async
Posted by GitBox <gi...@apache.org>.
Technoboy- commented on code in PR #15843:
URL: https://github.com/apache/pulsar/pull/15843#discussion_r885337995
##########
pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/v1/PersistentTopics.java:
##########
@@ -357,12 +357,19 @@ public void unloadTopic(@Suspended final AsyncResponse asyncResponse, @PathParam
@ApiResponse(code = 403, message = "Don't have admin permission"),
@ApiResponse(code = 404, message = "Topic does not exist"),
@ApiResponse(code = 412, message = "Topic has active producers/subscriptions")})
- public void deleteTopic(@PathParam("property") String property, @PathParam("cluster") String cluster,
+ public void deleteTopic(
+ @Suspended final AsyncResponse asyncResponse,
+ @PathParam("property") String property, @PathParam("cluster") String cluster,
@PathParam("namespace") String namespace, @PathParam("topic") @Encoded String encodedTopic,
@QueryParam("force") @DefaultValue("false") boolean force,
@QueryParam("authoritative") @DefaultValue("false") boolean authoritative) {
validateTopicName(property, cluster, namespace, encodedTopic);
- internalDeleteTopic(authoritative, force);
+ internalDeleteTopic(authoritative, force).thenAccept(
+ __ -> asyncResponse.resume(Response.noContent().build()))
+ .exceptionally(ex -> {
+ resumeAsyncResponseExceptionally(asyncResponse, ex.getCause());
Review Comment:
if (!isRedirectException(ex)) {
log.error("[{}] Failed to delete topic {}", clientAppId(), topicName, realCause);
}
##########
pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/v2/PersistentTopics.java:
##########
@@ -1011,7 +1012,12 @@ public void deleteTopic(
@ApiParam(value = "Is authentication required to perform this operation")
@QueryParam("authoritative") @DefaultValue("false") boolean authoritative) {
validateTopicName(tenant, namespace, encodedTopic);
- internalDeleteTopic(authoritative, force);
+ internalDeleteTopic(authoritative, force).thenAccept(
+ __ -> asyncResponse.resume(Response.noContent().build()))
+ .exceptionally(ex -> {
+ resumeAsyncResponseExceptionally(asyncResponse, ex.getCause());
Review Comment:
add log
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org