You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/05/21 19:29:13 UTC

[GitHub] [pulsar] anvinjain opened a new issue #10673: update-partitioned-topic fails for newly created geo-replicated topic

anvinjain opened a new issue #10673:
URL: https://github.com/apache/pulsar/issues/10673


   **Describe the bug**
   If a partitioned topic is created from one cluster, C1 where namespace cluster-list=(C1,C2), then update-partitioned-topic fails from remote cluster(C2) unless produce/consume has been done at least once from the remote cluster(C2).
   
   **To Reproduce**
   Steps to reproduce the behavior:
   *How to reproduce*
   
   1. Have a namespace with replication set in two regions (clusters), R1 and R2: geo-1/ns-1
   1. Create partitioned-topic from R1: `./pulsar-admin topics create-partitioned-topic -p 4 persistent://geo-1/ns-1/t4`
   1. Verify the same in global zookeeper.
   1. List partitioned topics from R2: `./pulsar-admin topics list-partitioned-topics geo-1/ns-1`
   Output is as expected: `"persistent://geo-1/ns-1/t4"`
   1. **[Bug 1]: Update partitioned topic from R2: `./pulsar-admin topics update-partitioned-topic -p 6 persistent://geo-1/ns-1/t4` 500 Error returned.** Redacted broker logs (more in the comment below):
   ```
   18:56:12.405 [AsyncHttpClient-60-1] WARN  org.apache.pulsar.broker.admin.impl.PersistentTopicsBase - [AUTHN/viesti-super-user-1] Failed to get list of subscriptions of persistent://geo-1/ns-1/t4-partition-0
   java.util.concurrent.CompletionException: org.apache.pulsar.client.admin.PulsarAdminException$ServerSideErrorException: HTTP 500 Internal Server Errorjava.io.ByteArrayInputStream@6fb6769a
   	at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292) ~[?:1.8.0_282]
   	at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308) ~[?:1.8.0_282]
   ...
   18:56:12.407 [pulsar-web-38-5] ERROR org.apache.pulsar.broker.admin.impl.PersistentTopicsBase - [AUTHN/viesti-super-user-1] Failed to update partitioned topic persistent://geo-1/ns-1/t4
   java.util.concurrent.ExecutionException: org.apache.pulsar.client.admin.PulsarAdminException$ServerSideErrorException: HTTP 500 Internal Server Errorjava.io.ByteArrayInputStream@6fb6769a
   	at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357) ~[?:1.8.0_282]
   ...
   18:56:12.411 [pulsar-web-38-5] INFO  org.eclipse.jetty.server.RequestLog - 10.39.68.16 - - [21/May/2021:18:56:12 +0000] "POST /admin/v2/persistent/geo-1/ns-1/t4/partitions?updateLocalTopicOnly=true&authoritative=false HTTP/1.1" 500 12052 "-" "Pulsar-Java-v2.7.1" 200
   ```
   6. Update partitioned topic from R1 on the other hand succeeds and reflects in global zookeeper metadata too.
   6. **[Bug 2]: List topics in R2: `./pulsar-admin topics list geo-1/ns-1` Does not list partitions of t4** (they are listed in R1 where we originally created t4)
   7. Produce in R1, able to consume from R2. Produce in R2, able to consume from R1.
   8. After producing or consuming from R2, even update-partitioned-topic succeeds from R2
   
   **Expected behavior**
   Expected step 5 (update partitioned topic from R2) to succeed the first time itself.
   
   **Screenshots**
   NA. Detailed error logs in the comment below
   
   **Desktop (please complete the following information):**
    - OS: Debian 9
   
   **Additional context**
   Pulsar version: 2.7.1. Two deployments, one in R1 and another in R2 with a global zookeeper store in R1.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar] anvinjain edited a comment on issue #10673: update-partitioned-topic fails for newly created geo-replicated topic

Posted by GitBox <gi...@apache.org>.
anvinjain edited a comment on issue #10673:
URL: https://github.com/apache/pulsar/issues/10673#issuecomment-858631833


   @315157973 Here you are doing equivalent of list-partitioned-topics right? 
   `Awaitility.await().untilAsserted(() -&gt; assertEquals(admin2.topics().getPartitionedTopicList(namespace).get(0)                , persistentTopicName));`
   
   Notice this section in bug description
   > [Bug 2]: List topics in R2: ./pulsar-admin topics list geo-1/ns-1 Does not list partitions of t4 (they are listed in R1 where we originally created t4)
   
   list of topics (instead of list-partitioned-topics) from the other cluster, admin2 in your case, does not have the list of partitions as per the original bug report, unless you have produced and consumed.
   
   Also to reproduce 
   > [Bug 1]: Update partitioned topic from R2: ./pulsar-admin topics update-partitioned-topic -p 6 persistent://geo-1/ns-1/t4 500 Error returned.
   
   Please try updating number of partitions from the other cluster before doing produce and consume on this topic.
   
   I hope this clarified repro steps for both of the issues and how you can change the unit test to reflect these
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar] 315157973 edited a comment on issue #10673: update-partitioned-topic fails for newly created geo-replicated topic

Posted by GitBox <gi...@apache.org>.
315157973 edited a comment on issue #10673:
URL: https://github.com/apache/pulsar/issues/10673#issuecomment-858310114


   https://github.com/315157973/pulsar/blob/7ad58d0493395143739c0872063e32c015a6bca4/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/ReplicatorTest.java#L803
   
   Hello, I wrote a unit test based on your description, but it cannot be reproduced. Can you help me see which step is wrong? @anvinjain


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar] 315157973 commented on issue #10673: update-partitioned-topic fails for newly created geo-replicated topic

Posted by GitBox <gi...@apache.org>.
315157973 commented on issue #10673:
URL: https://github.com/apache/pulsar/issues/10673#issuecomment-859734534


   Thanks, I have found the reason, and I will fix it.  But you should not update the topic in cluster 2, if the replication is one-way


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar] anvinjain commented on issue #10673: update-partitioned-topic fails for newly created geo-replicated topic

Posted by GitBox <gi...@apache.org>.
anvinjain commented on issue #10673:
URL: https://github.com/apache/pulsar/issues/10673#issuecomment-859759521


   Can you elaborate on replication being "one-way". From what I understand, pulsar has active-active topics, i.e. topic should be present in all the clusters configured in the topic's namespace.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar] anvinjain commented on issue #10673: update-partitioned-topic fails for newly created geo-replicated topic

Posted by GitBox <gi...@apache.org>.
anvinjain commented on issue #10673:
URL: https://github.com/apache/pulsar/issues/10673#issuecomment-858631833


   @315157973 Here you are doing equivalent of list-partitioned-topics right? 
   `Awaitility.await().untilAsserted(() -&gt; assertEquals(admin2.topics().getPartitionedTopicList(namespace).get(0)                , persistentTopicName));`
   
   Notice this section in bug description
   > [Bug 2]: List topics in R2: ./pulsar-admin topics list geo-1/ns-1 Does not list partitions of t4 (they are listed in R1 where we originally created t4)
   list of topics (instead of partitioned topics) from the other cluster, admin2 in your case, does not have the list of partitions as per the original bug report, unless you have produced and consumed.
   
   Also to reproduce 
   > [Bug 1]: Update partitioned topic from R2: ./pulsar-admin topics update-partitioned-topic -p 6 persistent://geo-1/ns-1/t4 500 Error returned.
   Please try updating number of partitions from the other cluster before doing produce and consume on this topic.
   
   I hope this clarified repro steps for both of the issues and how you can change the unit test to reflect these
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar] anvinjain edited a comment on issue #10673: update-partitioned-topic fails for newly created geo-replicated topic

Posted by GitBox <gi...@apache.org>.
anvinjain edited a comment on issue #10673:
URL: https://github.com/apache/pulsar/issues/10673#issuecomment-859759521






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar] 315157973 commented on issue #10673: update-partitioned-topic fails for newly created geo-replicated topic

Posted by GitBox <gi...@apache.org>.
315157973 commented on issue #10673:
URL: https://github.com/apache/pulsar/issues/10673#issuecomment-854465666


   I will solve it


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar] anvinjain commented on issue #10673: update-partitioned-topic fails for newly created geo-replicated topic

Posted by GitBox <gi...@apache.org>.
anvinjain commented on issue #10673:
URL: https://github.com/apache/pulsar/issues/10673#issuecomment-846200321


   **Full stacktrace in broker logs**
   ```
   18:56:12.405 [AsyncHttpClient-60-1] WARN  org.apache.pulsar.broker.admin.impl.PersistentTopicsBase - [AUTHN/viesti-super-user-1] Failed to get list of subscriptions of persistent://geo-1/ns-1/t4-partition-0
   java.util.concurrent.CompletionException: org.apache.pulsar.client.admin.PulsarAdminException$ServerSideErrorException: HTTP 500 Internal Server Errorjava.io.ByteArrayInputStream@6fb6769a
   	at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292) ~[?:1.8.0_282]
   	at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308) ~[?:1.8.0_282]
   	at java.util.concurrent.CompletableFuture.uniAccept(CompletableFuture.java:661) ~[?:1.8.0_282]
   	at java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:646) ~[?:1.8.0_282]
   	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) ~[?:1.8.0_282]
   	at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990) ~[?:1.8.0_282]
   	at org.apache.pulsar.client.admin.internal.TopicsImpl$10.failed(TopicsImpl.java:663) ~[org.apache.pulsar-pulsar-client-admin-original-2.7.1.jar:2.7.1]
   	at org.glassfish.jersey.client.JerseyInvocation$1.failed(JerseyInvocation.java:839) ~[org.glassfish.jersey.core-jersey-client-2.31.jar:?]
   	at org.glassfish.jersey.client.JerseyInvocation$1.completed(JerseyInvocation.java:820) ~[org.glassfish.jersey.core-jersey-client-2.31.jar:?]
   	at org.glassfish.jersey.client.ClientRuntime.processResponse(ClientRuntime.java:229) ~[org.glassfish.jersey.core-jersey-client-2.31.jar:?]
   	at org.glassfish.jersey.client.ClientRuntime.access$200(ClientRuntime.java:62) ~[org.glassfish.jersey.core-jersey-client-2.31.jar:?]
   	at org.glassfish.jersey.client.ClientRuntime$2.lambda$response$0(ClientRuntime.java:173) ~[org.glassfish.jersey.core-jersey-client-2.31.jar:?]
   	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) ~[org.glassfish.jersey.core-jersey-common-2.31.jar:?]
   	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) ~[org.glassfish.jersey.core-jersey-common-2.31.jar:?]
   	at org.glassfish.jersey.internal.Errors.process(Errors.java:292) ~[org.glassfish.jersey.core-jersey-common-2.31.jar:?]
   	at org.glassfish.jersey.internal.Errors.process(Errors.java:274) ~[org.glassfish.jersey.core-jersey-common-2.31.jar:?]
   	at org.glassfish.jersey.internal.Errors.process(Errors.java:244) ~[org.glassfish.jersey.core-jersey-common-2.31.jar:?]
   	at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:288) ~[org.glassfish.jersey.core-jersey-common-2.31.jar:?]
   	at org.glassfish.jersey.client.ClientRuntime$2.response(ClientRuntime.java:173) ~[org.glassfish.jersey.core-jersey-client-2.31.jar:?]
   	at org.apache.pulsar.client.admin.internal.http.AsyncHttpConnector.lambda$apply$1(AsyncHttpConnector.java:212) ~[org.apache.pulsar-pulsar-client-admin-original-2.7.1.jar:2.7.1]
   	at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) ~[?:1.8.0_282]
   	at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) ~[?:1.8.0_282]
   	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) ~[?:1.8.0_282]
   	at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) ~[?:1.8.0_282]
   	at org.apache.pulsar.client.admin.internal.http.AsyncHttpConnector.lambda$retryOperation$3(AsyncHttpConnector.java:253) ~[org.apache.pulsar-pulsar-client-admin-original-2.7.1.jar:2.7.1]
   	at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) [?:1.8.0_282]
   	at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) [?:1.8.0_282]
   	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) [?:1.8.0_282]
   	at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) [?:1.8.0_282]
   	at org.asynchttpclient.netty.NettyResponseFuture.loadContent(NettyResponseFuture.java:222) [org.asynchttpclient-async-http-client-2.12.1.jar:?]
   	at org.asynchttpclient.netty.NettyResponseFuture.done(NettyResponseFuture.java:257) [org.asynchttpclient-async-http-client-2.12.1.jar:?]
   	at org.asynchttpclient.netty.handler.AsyncHttpClientHandler.finishUpdate(AsyncHttpClientHandler.java:241) [org.asynchttpclient-async-http-client-2.12.1.jar:?]
   	at org.asynchttpclient.netty.handler.HttpHandler.handleChunk(HttpHandler.java:114) [org.asynchttpclient-async-http-client-2.12.1.jar:?]
   	at org.asynchttpclient.netty.handler.HttpHandler.handleRead(HttpHandler.java:143) [org.asynchttpclient-async-http-client-2.12.1.jar:?]
   	at org.asynchttpclient.netty.handler.AsyncHttpClientHandler.channelRead(AsyncHttpClientHandler.java:78) [org.asynchttpclient-async-http-client-2.12.1.jar:?]
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) [io.netty-netty-codec-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436) [io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) [io.netty-netty-codec-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) [io.netty-netty-codec-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251) [io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) [io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650) [io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576) [io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) [io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [io.netty-netty-common-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [io.netty-netty-common-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [io.netty-netty-common-4.1.51.Final.jar:4.1.51.Final]
   	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_282]
   Caused by: org.apache.pulsar.client.admin.PulsarAdminException$ServerSideErrorException: HTTP 500 Internal Server Errorjava.io.ByteArrayInputStream@6fb6769a
   	at org.apache.pulsar.client.admin.internal.BaseResource.getApiException(BaseResource.java:208) ~[org.apache.pulsar-pulsar-client-admin-original-2.7.1.jar:2.7.1]
   	... 56 more
   Caused by: javax.ws.rs.InternalServerErrorException: HTTP 500 Internal Server Error
   	at org.glassfish.jersey.client.JerseyInvocation.convertToException(JerseyInvocation.java:914) ~[org.glassfish.jersey.core-jersey-client-2.31.jar:?]
   	at org.glassfish.jersey.client.JerseyInvocation.access$500(JerseyInvocation.java:77) ~[org.glassfish.jersey.core-jersey-client-2.31.jar:?]
   	... 54 more
   18:56:12.407 [pulsar-web-38-5] ERROR org.apache.pulsar.broker.admin.impl.PersistentTopicsBase - [AUTHN/viesti-super-user-1] Failed to update partitioned topic persistent://geo-1/ns-1/t4
   java.util.concurrent.ExecutionException: org.apache.pulsar.client.admin.PulsarAdminException$ServerSideErrorException: HTTP 500 Internal Server Errorjava.io.ByteArrayInputStream@6fb6769a
   	at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357) ~[?:1.8.0_282]
   	at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908) ~[?:1.8.0_282]
   	at org.apache.pulsar.broker.admin.impl.PersistentTopicsBase.internalUpdatePartitionedTopic(PersistentTopicsBase.java:510) ~[org.apache.pulsar-pulsar-broker-2.7.1.jar:2.7.1]
   	at org.apache.pulsar.broker.admin.v2.PersistentTopics.updatePartitionedTopic(PersistentTopics.java:661) ~[org.apache.pulsar-pulsar-broker-2.7.1.jar:2.7.1]
   	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_282]
   	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_282]
   	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_282]
   	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_282]
   	at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) ~[org.glassfish.jersey.core-jersey-server-2.31.jar:?]
   	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) ~[org.glassfish.jersey.core-jersey-server-2.31.jar:?]
   	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) ~[org.glassfish.jersey.core-jersey-server-2.31.jar:?]
   	at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$VoidOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:159) ~[org.glassfish.jersey.core-jersey-server-2.31.jar:?]
   	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) ~[org.glassfish.jersey.core-jersey-server-2.31.jar:?]
   	at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:469) ~[org.glassfish.jersey.core-jersey-server-2.31.jar:?]
   	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:391) ~[org.glassfish.jersey.core-jersey-server-2.31.jar:?]
   	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:80) ~[org.glassfish.jersey.core-jersey-server-2.31.jar:?]
   	at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:253) ~[org.glassfish.jersey.core-jersey-server-2.31.jar:?]
   	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) ~[org.glassfish.jersey.core-jersey-common-2.31.jar:?]
   	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) ~[org.glassfish.jersey.core-jersey-common-2.31.jar:?]
   	at org.glassfish.jersey.internal.Errors.process(Errors.java:292) ~[org.glassfish.jersey.core-jersey-common-2.31.jar:?]
   	at org.glassfish.jersey.internal.Errors.process(Errors.java:274) ~[org.glassfish.jersey.core-jersey-common-2.31.jar:?]
   	at org.glassfish.jersey.internal.Errors.process(Errors.java:244) ~[org.glassfish.jersey.core-jersey-common-2.31.jar:?]
   	at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) ~[org.glassfish.jersey.core-jersey-common-2.31.jar:?]
   	at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:232) ~[org.glassfish.jersey.core-jersey-server-2.31.jar:?]
   	at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) ~[org.glassfish.jersey.core-jersey-server-2.31.jar:?]
   	at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) ~[org.glassfish.jersey.containers-jersey-container-servlet-core-2.31.jar:?]
   	at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) ~[org.glassfish.jersey.containers-jersey-container-servlet-core-2.31.jar:?]
   	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) ~[org.glassfish.jersey.containers-jersey-container-servlet-core-2.31.jar:?]
   	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) ~[org.glassfish.jersey.containers-jersey-container-servlet-core-2.31.jar:?]
   	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) ~[org.glassfish.jersey.containers-jersey-container-servlet-core-2.31.jar:?]
   	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:791) ~[org.eclipse.jetty-jetty-servlet-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1626) ~[org.eclipse.jetty-jetty-servlet-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.apache.pulsar.broker.web.ResponseHandlerFilter.doFilter(ResponseHandlerFilter.java:66) ~[org.apache.pulsar-pulsar-broker-2.7.1.jar:2.7.1]
   	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) ~[org.eclipse.jetty-jetty-servlet-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) ~[org.eclipse.jetty-jetty-servlet-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.apache.pulsar.broker.web.AuthenticationFilter.doFilter(AuthenticationFilter.java:82) ~[org.apache.pulsar-pulsar-broker-common-2.7.1.jar:2.7.1]
   	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) ~[org.eclipse.jetty-jetty-servlet-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) ~[org.eclipse.jetty-jetty-servlet-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) ~[org.eclipse.jetty-jetty-servlet-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) ~[org.eclipse.jetty-jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) ~[org.eclipse.jetty-jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) ~[org.eclipse.jetty-jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1435) ~[org.eclipse.jetty-jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) ~[org.eclipse.jetty-jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) ~[org.eclipse.jetty-jetty-servlet-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) ~[org.eclipse.jetty-jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) ~[org.eclipse.jetty-jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1350) ~[org.eclipse.jetty-jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) ~[org.eclipse.jetty-jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:234) ~[org.eclipse.jetty-jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) ~[org.eclipse.jetty-jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:179) ~[org.eclipse.jetty-jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) ~[org.eclipse.jetty-jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.server.Server.handle(Server.java:516) ~[org.eclipse.jetty-jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:388) ~[org.eclipse.jetty-jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:633) [org.eclipse.jetty-jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:380) [org.eclipse.jetty-jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) [org.eclipse.jetty-jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) [org.eclipse.jetty-jetty-io-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) [org.eclipse.jetty-jetty-io-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) [org.eclipse.jetty-jetty-io-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) [org.eclipse.jetty-jetty-util-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) [org.eclipse.jetty-jetty-util-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) [org.eclipse.jetty-jetty-util-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) [org.eclipse.jetty-jetty-util-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) [org.eclipse.jetty-jetty-util-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_282]
   	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_282]
   	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [io.netty-netty-common-4.1.51.Final.jar:4.1.51.Final]
   	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_282]
   Caused by: org.apache.pulsar.client.admin.PulsarAdminException$ServerSideErrorException: HTTP 500 Internal Server Errorjava.io.ByteArrayInputStream@6fb6769a
   	at org.apache.pulsar.client.admin.internal.BaseResource.getApiException(BaseResource.java:208) ~[org.apache.pulsar-pulsar-client-admin-original-2.7.1.jar:2.7.1]
   	at org.apache.pulsar.client.admin.internal.TopicsImpl$10.failed(TopicsImpl.java:663) ~[org.apache.pulsar-pulsar-client-admin-original-2.7.1.jar:2.7.1]
   	at org.glassfish.jersey.client.JerseyInvocation$1.failed(JerseyInvocation.java:839) ~[org.glassfish.jersey.core-jersey-client-2.31.jar:?]
   	at org.glassfish.jersey.client.JerseyInvocation$1.completed(JerseyInvocation.java:820) ~[org.glassfish.jersey.core-jersey-client-2.31.jar:?]
   	at org.glassfish.jersey.client.ClientRuntime.processResponse(ClientRuntime.java:229) ~[org.glassfish.jersey.core-jersey-client-2.31.jar:?]
   	at org.glassfish.jersey.client.ClientRuntime.access$200(ClientRuntime.java:62) ~[org.glassfish.jersey.core-jersey-client-2.31.jar:?]
   	at org.glassfish.jersey.client.ClientRuntime$2.lambda$response$0(ClientRuntime.java:173) ~[org.glassfish.jersey.core-jersey-client-2.31.jar:?]
   	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) ~[org.glassfish.jersey.core-jersey-common-2.31.jar:?]
   	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) ~[org.glassfish.jersey.core-jersey-common-2.31.jar:?]
   	at org.glassfish.jersey.internal.Errors.process(Errors.java:292) ~[org.glassfish.jersey.core-jersey-common-2.31.jar:?]
   	at org.glassfish.jersey.internal.Errors.process(Errors.java:274) ~[org.glassfish.jersey.core-jersey-common-2.31.jar:?]
   	at org.glassfish.jersey.internal.Errors.process(Errors.java:244) ~[org.glassfish.jersey.core-jersey-common-2.31.jar:?]
   	at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:288) ~[org.glassfish.jersey.core-jersey-common-2.31.jar:?]
   	at org.glassfish.jersey.client.ClientRuntime$2.response(ClientRuntime.java:173) ~[org.glassfish.jersey.core-jersey-client-2.31.jar:?]
   	at org.apache.pulsar.client.admin.internal.http.AsyncHttpConnector.lambda$apply$1(AsyncHttpConnector.java:212) ~[org.apache.pulsar-pulsar-client-admin-original-2.7.1.jar:2.7.1]
   	at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) ~[?:1.8.0_282]
   	at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) ~[?:1.8.0_282]
   	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) ~[?:1.8.0_282]
   	at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) ~[?:1.8.0_282]
   	at org.apache.pulsar.client.admin.internal.http.AsyncHttpConnector.lambda$retryOperation$3(AsyncHttpConnector.java:253) ~[org.apache.pulsar-pulsar-client-admin-original-2.7.1.jar:2.7.1]
   	at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) ~[?:1.8.0_282]
   	at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) ~[?:1.8.0_282]
   	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) ~[?:1.8.0_282]
   	at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) ~[?:1.8.0_282]
   	at org.asynchttpclient.netty.NettyResponseFuture.loadContent(NettyResponseFuture.java:222) ~[org.asynchttpclient-async-http-client-2.12.1.jar:?]
   	at org.asynchttpclient.netty.NettyResponseFuture.done(NettyResponseFuture.java:257) ~[org.asynchttpclient-async-http-client-2.12.1.jar:?]
   	at org.asynchttpclient.netty.handler.AsyncHttpClientHandler.finishUpdate(AsyncHttpClientHandler.java:241) ~[org.asynchttpclient-async-http-client-2.12.1.jar:?]
   	at org.asynchttpclient.netty.handler.HttpHandler.handleChunk(HttpHandler.java:114) ~[org.asynchttpclient-async-http-client-2.12.1.jar:?]
   	at org.asynchttpclient.netty.handler.HttpHandler.handleRead(HttpHandler.java:143) ~[org.asynchttpclient-async-http-client-2.12.1.jar:?]
   	at org.asynchttpclient.netty.handler.AsyncHttpClientHandler.channelRead(AsyncHttpClientHandler.java:78) ~[org.asynchttpclient-async-http-client-2.12.1.jar:?]
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[io.netty-netty-codec-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436) ~[io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) ~[io.netty-netty-codec-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) ~[io.netty-netty-codec-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251) ~[io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650) ~[io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576) ~[io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[io.netty-netty-common-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[io.netty-netty-common-4.1.51.Final.jar:4.1.51.Final]
   	... 2 more
   Caused by: javax.ws.rs.InternalServerErrorException: HTTP 500 Internal Server Error
   	at org.glassfish.jersey.client.JerseyInvocation.convertToException(JerseyInvocation.java:914) ~[org.glassfish.jersey.core-jersey-client-2.31.jar:?]
   	at org.glassfish.jersey.client.JerseyInvocation.access$500(JerseyInvocation.java:77) ~[org.glassfish.jersey.core-jersey-client-2.31.jar:?]
   	at org.glassfish.jersey.client.JerseyInvocation$1.completed(JerseyInvocation.java:820) ~[org.glassfish.jersey.core-jersey-client-2.31.jar:?]
   	at org.glassfish.jersey.client.ClientRuntime.processResponse(ClientRuntime.java:229) ~[org.glassfish.jersey.core-jersey-client-2.31.jar:?]
   	at org.glassfish.jersey.client.ClientRuntime.access$200(ClientRuntime.java:62) ~[org.glassfish.jersey.core-jersey-client-2.31.jar:?]
   	at org.glassfish.jersey.client.ClientRuntime$2.lambda$response$0(ClientRuntime.java:173) ~[org.glassfish.jersey.core-jersey-client-2.31.jar:?]
   	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) ~[org.glassfish.jersey.core-jersey-common-2.31.jar:?]
   	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) ~[org.glassfish.jersey.core-jersey-common-2.31.jar:?]
   	at org.glassfish.jersey.internal.Errors.process(Errors.java:292) ~[org.glassfish.jersey.core-jersey-common-2.31.jar:?]
   	at org.glassfish.jersey.internal.Errors.process(Errors.java:274) ~[org.glassfish.jersey.core-jersey-common-2.31.jar:?]
   	at org.glassfish.jersey.internal.Errors.process(Errors.java:244) ~[org.glassfish.jersey.core-jersey-common-2.31.jar:?]
   	at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:288) ~[org.glassfish.jersey.core-jersey-common-2.31.jar:?]
   	at org.glassfish.jersey.client.ClientRuntime$2.response(ClientRuntime.java:173) ~[org.glassfish.jersey.core-jersey-client-2.31.jar:?]
   	at org.apache.pulsar.client.admin.internal.http.AsyncHttpConnector.lambda$apply$1(AsyncHttpConnector.java:212) ~[org.apache.pulsar-pulsar-client-admin-original-2.7.1.jar:2.7.1]
   	at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) ~[?:1.8.0_282]
   	at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) ~[?:1.8.0_282]
   	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) ~[?:1.8.0_282]
   	at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) ~[?:1.8.0_282]
   	at org.apache.pulsar.client.admin.internal.http.AsyncHttpConnector.lambda$retryOperation$3(AsyncHttpConnector.java:253) ~[org.apache.pulsar-pulsar-client-admin-original-2.7.1.jar:2.7.1]
   	at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) ~[?:1.8.0_282]
   	at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) ~[?:1.8.0_282]
   	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) ~[?:1.8.0_282]
   	at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) ~[?:1.8.0_282]
   	at org.asynchttpclient.netty.NettyResponseFuture.loadContent(NettyResponseFuture.java:222) ~[org.asynchttpclient-async-http-client-2.12.1.jar:?]
   	at org.asynchttpclient.netty.NettyResponseFuture.done(NettyResponseFuture.java:257) ~[org.asynchttpclient-async-http-client-2.12.1.jar:?]
   	at org.asynchttpclient.netty.handler.AsyncHttpClientHandler.finishUpdate(AsyncHttpClientHandler.java:241) ~[org.asynchttpclient-async-http-client-2.12.1.jar:?]
   	at org.asynchttpclient.netty.handler.HttpHandler.handleChunk(HttpHandler.java:114) ~[org.asynchttpclient-async-http-client-2.12.1.jar:?]
   	at org.asynchttpclient.netty.handler.HttpHandler.handleRead(HttpHandler.java:143) ~[org.asynchttpclient-async-http-client-2.12.1.jar:?]
   	at org.asynchttpclient.netty.handler.AsyncHttpClientHandler.channelRead(AsyncHttpClientHandler.java:78) ~[org.asynchttpclient-async-http-client-2.12.1.jar:?]
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[io.netty-netty-codec-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436) ~[io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) ~[io.netty-netty-codec-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) ~[io.netty-netty-codec-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251) ~[io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) ~[io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650) ~[io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576) ~[io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[io.netty-netty-transport-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[io.netty-netty-common-4.1.51.Final.jar:4.1.51.Final]
   	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[io.netty-netty-common-4.1.51.Final.jar:4.1.51.Final]
   	... 2 more
   18:56:12.411 [pulsar-web-38-5] INFO  org.eclipse.jetty.server.RequestLog - 10.39.68.16 - - [21/May/2021:18:56:12 +0000] "POST /admin/v2/persistent/geo-1/ns-1/t4/partitions?updateLocalTopicOnly=true&authoritative=false HTTP/1.1" 500 12052 "-" "Pulsar-Java-v2.7.1" 200
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar] 315157973 commented on issue #10673: update-partitioned-topic fails for newly created geo-replicated topic

Posted by GitBox <gi...@apache.org>.
315157973 commented on issue #10673:
URL: https://github.com/apache/pulsar/issues/10673#issuecomment-858310114


   https://github.com/315157973/pulsar/blob/42c4c7fea3ee69f49274bb77287414f1658ccf56/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/ReplicatorTest.java#L800-831
   
   Hello, I wrote a unit test based on your description, but it cannot be reproduced. Can you help me see which step is wrong? @anvinjain


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar] 315157973 edited a comment on issue #10673: update-partitioned-topic fails for newly created geo-replicated topic

Posted by GitBox <gi...@apache.org>.
315157973 edited a comment on issue #10673:
URL: https://github.com/apache/pulsar/issues/10673#issuecomment-858310114


   https://github.com/315157973/pulsar/blob/42c4c7fea3ee69f49274bb77287414f1658ccf56/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/ReplicatorTest.java#L800
   
   Hello, I wrote a unit test based on your description, but it cannot be reproduced. Can you help me see which step is wrong? @anvinjain


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar] 315157973 edited a comment on issue #10673: update-partitioned-topic fails for newly created geo-replicated topic

Posted by GitBox <gi...@apache.org>.
315157973 edited a comment on issue #10673:
URL: https://github.com/apache/pulsar/issues/10673#issuecomment-858310114


   https://github.com/315157973/pulsar/blob/42c4c7fea3ee69f49274bb77287414f1658ccf56/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/ReplicatorTest.java#L800-L831
   
   Hello, I wrote a unit test based on your description, but it cannot be reproduced. Can you help me see which step is wrong? @anvinjain


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar] anvinjain edited a comment on issue #10673: update-partitioned-topic fails for newly created geo-replicated topic

Posted by GitBox <gi...@apache.org>.
anvinjain edited a comment on issue #10673:
URL: https://github.com/apache/pulsar/issues/10673#issuecomment-858631833


   @315157973 Here you are doing equivalent of list-partitioned-topics right? 
   `Awaitility.await().untilAsserted(() -&gt; assertEquals(admin2.topics().getPartitionedTopicList(namespace).get(0)                , persistentTopicName));`
   
   Notice this section in bug description
   > [Bug 2]: List topics in R2: ./pulsar-admin topics list geo-1/ns-1 Does not list partitions of t4 (they are listed in R1 where we originally created t4)
   
   list of topics (instead of partitioned topics) from the other cluster, admin2 in your case, does not have the list of partitions as per the original bug report, unless you have produced and consumed.
   
   Also to reproduce 
   > [Bug 1]: Update partitioned topic from R2: ./pulsar-admin topics update-partitioned-topic -p 6 persistent://geo-1/ns-1/t4 500 Error returned.
   
   Please try updating number of partitions from the other cluster before doing produce and consume on this topic.
   
   I hope this clarified repro steps for both of the issues and how you can change the unit test to reflect these
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org