You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2020/07/20 09:20:46 UTC
[GitHub] [pulsar] Mefl opened a new issue #7604: [Rest admin api] 'Get message by its messageId' throw exception when the broker asked is not the good one
Mefl opened a new issue #7604:
URL: https://github.com/apache/pulsar/issues/7604
**Describe the bug**
When using the rest api `http://pulsar.apache.org/admin-rest-api/#operation/getLastMessageId` you may have exception if the broker asked is not the good one. By executing the same request several times, it will succeeded with the right answer few times or it will fail with this exception.
For exemple the request I used :
`curl --location --request GET 'https://host:2000/admin/v2/persistent/mytenant/myns/mytopic/ledger/757808/entry/0' `
```
--- An unexpected error occurred in the server ---
Message: org.apache.pulsar.broker.service.BrokerServiceException$ServiceUnitNotReadyException: Namespace bundle for topic (persistent://mytenant/myns/mytopic) not served by this instance. Please redo the lookup. Request is denied: namespace=mytenant/myns
Stacktrace:
java.util.concurrent.ExecutionException: org.apache.pulsar.broker.service.BrokerServiceException$ServiceUnitNotReadyException: Namespace bundle for topic (persistent://mytenant/myns/mytopic) not served by this instance. Please redo the lookup. Request is denied: namespace=mytenant/myns
at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1915)
at org.apache.pulsar.broker.admin.impl.PersistentTopicsBase.getTopicReference(PersistentTopicsBase.java:2353)
at org.apache.pulsar.broker.admin.impl.PersistentTopicsBase.internalGetMessageById(PersistentTopicsBase.java:1836)
at org.apache.pulsar.broker.admin.v2.PersistentTopics.getMessageById(PersistentTopics.java:956)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
....
Caused by: org.apache.pulsar.broker.service.BrokerServiceException$ServiceUnitNotReadyException: Namespace bundle for topic (persistent://mytenant/myns/mytopic) not served by this instance. Please redo the lookup. Request is denied: namespace=mytenant/myns
at org.apache.pulsar.broker.service.BrokerService.checkTopicNsOwnership(BrokerService.java:1372)
at org.apache.pulsar.broker.service.BrokerService.loadOrCreatePersistentTopic(BrokerService.java:955)
at org.apache.pulsar.broker.service.BrokerService.lambda$getTopic$13(BrokerService.java:732)
at org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap$Section.put(ConcurrentOpenHashMap.java:277)
at org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap.computeIfAbsent(ConcurrentOpenHashMap.java:130)
at org.apache.pulsar.broker.service.BrokerService.getTopic(BrokerService.java:731)
at org.apache.pulsar.broker.service.BrokerService.getTopicIfExists(BrokerService.java:711)
at org.apache.pulsar.broker.admin.impl.PersistentTopicsBase.getTopicReference(PersistentTopicsBase.java:2352)
... 73 more
```
**Expected behavior**
The request should always succeed and the request should be forwarded to the right broker.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Mefl commented on issue #7604: [Rest admin api] 'Get message by its messageId' throw exception when the broker asked is not the good one
Posted by GitBox <gi...@apache.org>.
Mefl commented on issue #7604:
URL: https://github.com/apache/pulsar/issues/7604#issuecomment-671429114
sorry for being late to answer ...
yes, I reproduce the same error on both getLastMessageId and getMessageById command
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Mefl commented on issue #7604: [Rest admin api] 'Get message by its messageId' throw exception when the broker asked is not the good one
Posted by GitBox <gi...@apache.org>.
Mefl commented on issue #7604:
URL: https://github.com/apache/pulsar/issues/7604#issuecomment-671785335
@KannarFr can you answer on the precise version of the cluster, thk
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] rudy2steiner commented on issue #7604: [Rest admin api] 'Get message by its messageId' throw exception when the broker asked is not the good one
Posted by GitBox <gi...@apache.org>.
rudy2steiner commented on issue #7604:
URL: https://github.com/apache/pulsar/issues/7604#issuecomment-671692957
> sorry for being late to answer ...
>
> yes, I reproduce the same error on both getLastMessageId and getMessageById command
which version you are using? base on the master currently, getLastMessageId can't happen the exception you metioned
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] rudy2steiner commented on issue #7604: [Rest admin api] 'Get message by its messageId' throw exception when the broker asked is not the good one
Posted by GitBox <gi...@apache.org>.
rudy2steiner commented on issue #7604:
URL: https://github.com/apache/pulsar/issues/7604#issuecomment-667686149
Do you meet the same exception:
```
Namespace bundle for topic (persistent://mytenant/myns/mytopic) not served by this instance. Please redo the lookup. Request is denied: namespace=mytenant/myns
```
on both getLastMessageId and getMessageById command? @Mefl
As I know, getLastMessageId command on v2 has checked topic namespace ownership and redirect to the right broker. but getMessageById missing the check logic may cause the exception.
@jiazhai I'm new to pulsar, it's ok to guide me if i have misunderstood on the issue.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] sijie closed issue #7604: [Rest admin api] 'Get message by its messageId' throw exception when the broker asked is not the good one
Posted by GitBox <gi...@apache.org>.
sijie closed issue #7604:
URL: https://github.com/apache/pulsar/issues/7604
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] jiazhai commented on issue #7604: [Rest admin api] 'Get message by its messageId' throw exception when the broker asked is not the good one
Posted by GitBox <gi...@apache.org>.
jiazhai commented on issue #7604:
URL: https://github.com/apache/pulsar/issues/7604#issuecomment-661551144
Looks like we should do redirect automatically. Marked as `help wanted`, since it is easy to reproduce, and there is other examples to follow in admin/rest commands.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] rudy2steiner commented on issue #7604: [Rest admin api] 'Get message by its messageId' throw exception when the broker asked is not the good one
Posted by GitBox <gi...@apache.org>.
rudy2steiner commented on issue #7604:
URL: https://github.com/apache/pulsar/issues/7604#issuecomment-661806530
I'd like to help, please assign to me!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] rudy2steiner commented on issue #7604: [Rest admin api] 'Get message by its messageId' throw exception when the broker asked is not the good one
Posted by GitBox <gi...@apache.org>.
rudy2steiner commented on issue #7604:
URL: https://github.com/apache/pulsar/issues/7604#issuecomment-667692343
we should call PersistentTopicsBase#validateReadOperationOnTopic method instead of verifyReadOperation(authoritative), is right ? @jiazhai
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Mefl commented on issue #7604: [Rest admin api] 'Get message by its messageId' throw exception when the broker asked is not the good one
Posted by GitBox <gi...@apache.org>.
Mefl commented on issue #7604:
URL: https://github.com/apache/pulsar/issues/7604#issuecomment-671790251
The exception I received with lastMessageId :
the url : `https://c1-pulsar:2000/admin/v2/persistent/orga_/pulsar_/action-b2a/lastMessageId`
the result :
`
--- An unexpected error occurred in the server ---
Message: org.apache.pulsar.broker.service.BrokerServiceException$ServiceUnitNotReadyException: Namespace bundle for topic (persistent://orga_/pulsar_/action-b2a) not served by this instance. Please redo the lookup. Request is denied: namespace=orga_/pulsar_
Stacktrace:
java.util.concurrent.ExecutionException: org.apache.pulsar.broker.service.BrokerServiceException$ServiceUnitNotReadyException: Namespace bundle for topic (persistent://orga_/pulsar_/action-b2a) not served by this instance. Please redo the lookup. Request is denied: namespace=orga_/pulsar_
at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1915)
at org.apache.pulsar.broker.admin.impl.PersistentTopicsBase.getTopicReference(PersistentTopicsBase.java:2353)
at org.apache.pulsar.broker.admin.impl.PersistentTopicsBase.internalGetLastMessageId(PersistentTopicsBase.java:2639)
at org.apache.pulsar.broker.admin.v2.PersistentTopics.getLastMessageId(PersistentTopics.java:1176)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76)
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148)
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191)
at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$VoidOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:183)
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103)
at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493)
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415)
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104)
at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268)
at org.glassfish.jersey.internal.Errors.process(Errors.java:316)
at org.glassfish.jersey.internal.Errors.process(Errors.java:298)
at org.glassfish.jersey.internal.Errors.process(Errors.java:268)
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289)
at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256)
at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703)
at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416)
at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370)
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389)
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342)
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1631)
at org.apache.pulsar.broker.web.ResponseHandlerFilter.doFilter(ResponseHandlerFilter.java:53)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1618)
at org.apache.pulsar.broker.web.AuthenticationFilter.doFilter(AuthenticationFilter.java:84)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1618)
at org.apache.pulsar.broker.intercept.BrokerInterceptor$BrokerInterceptorDisabled.onWebServiceRequest(BrokerInterceptor.java:73)
at org.apache.pulsar.broker.web.EventListenerFilter.doFilter(EventListenerFilter.java:46)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1618)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:549)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1369)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:489)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1284)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:221)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:173)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:501)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:543)
at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:398)
at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.pulsar.broker.service.BrokerServiceException$ServiceUnitNotReadyException: Namespace bundle for topic (persistent://orga_/pulsar_/action-b2a) not served by this instance. Please redo the lookup. Request is denied: namespace=orga_/pulsar_
at org.apache.pulsar.broker.service.BrokerService.checkTopicNsOwnership(BrokerService.java:1372)
at org.apache.pulsar.broker.service.BrokerService.loadOrCreatePersistentTopic(BrokerService.java:955)
at org.apache.pulsar.broker.service.BrokerService.lambda$getTopic$13(BrokerService.java:732)
at org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap$Section.put(ConcurrentOpenHashMap.java:277)
at org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap.computeIfAbsent(ConcurrentOpenHashMap.java:130)
at org.apache.pulsar.broker.service.BrokerService.getTopic(BrokerService.java:731)
at org.apache.pulsar.broker.service.BrokerService.getTopicIfExists(BrokerService.java:711)
at org.apache.pulsar.broker.admin.impl.PersistentTopicsBase.getTopicReference(PersistentTopicsBase.java:2352)
... 73 more
`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org