You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/11/12 10:32:46 UTC

[GitHub] [pulsar] Jason918 opened a new issue #12774: Flaky-test: [org.apache.pulsar.broker.service.ReplicatorTest].[testDoNotReplicateSystemTopic]

Jason918 opened a new issue #12774:
URL: https://github.com/apache/pulsar/issues/12774


   <!--- 
   
   Instructions for reporting a flaky test using this issue template:
   
   1. Replace [test class] in title and body with the test class name
   2. Replace [test method] in title and body with the test method that failed. Multiple methods are flaky, remove the content that refers to the test method.
   3. Replace "url here" with a url to an example failure. In the Github Actions workflow run logs, you can right click on the line number to copy a link to the line. Example of such url is https://github.com/apache/pulsar/pull/8892/checks?check_run_id=1531075794#step:9:377 . The logs are available for a limited amount of time (usually for a few weeks).
   4. Replace "relevant parts of the exception stacktrace here" with the a few lines of the stack trace that shows at least the exception message and the line of test code where the stacktrace occurred.
   5. Replace "full exception stacktrace here" with the full exception stacktrace from logs. This section will be hidden by default.
   6. Remove all unused fields / content to unclutter the reported issue. Remove this comment too.
   
   -->
   [org.apache.pulsar.broker.service.ReplicatorTest] is flaky. The [testDoNotReplicateSystemTopic] test method fails sporadically.
   
   [example failure](https://github.com/apache/pulsar/runs/4187735359?check_suite_focus=true#step:9:7731)
   
   ```
   org.apache.pulsar.client.admin.PulsarAdminException$NotFoundException: Topic not found
   
   ```
   
   <details>
   <summary>Full exception stacktrace</summary>
   <code><pre>
   org.apache.pulsar.client.admin.PulsarAdminException$NotFoundException: Topic not found
   	at org.apache.pulsar.client.admin.internal.BaseResource.getApiException(BaseResource.java:234)
   	at org.apache.pulsar.client.admin.internal.BaseResource$2.failed(BaseResource.java:157)
   	at org.glassfish.jersey.client.JerseyInvocation$1.failed(JerseyInvocation.java:882)
   	at org.glassfish.jersey.client.JerseyInvocation$1.completed(JerseyInvocation.java:863)
   	at org.glassfish.jersey.client.ClientRuntime.processResponse(ClientRuntime.java:229)
   	at org.glassfish.jersey.client.ClientRuntime.access$200(ClientRuntime.java:62)
   	at org.glassfish.jersey.client.ClientRuntime$2.lambda$response$0(ClientRuntime.java:173)
   	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
   	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)
   	at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
   	at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
   	at org.glassfish.jersey.internal.Errors.process(Errors.java:244)
   	at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:288)
   	at org.glassfish.jersey.client.ClientRuntime$2.response(ClientRuntime.java:173)
   	at org.apache.pulsar.client.admin.internal.http.AsyncHttpConnector.lambda$apply$1(AsyncHttpConnector.java:228)
   	at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859)
   	at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837)
   	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
   	at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2073)
   	at org.apache.pulsar.client.admin.internal.http.AsyncHttpConnector.lambda$retryOperation$4(AsyncHttpConnector.java:270)
   	at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859)
   	at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837)
   	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
   	at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2073)
   	at org.asynchttpclient.netty.NettyResponseFuture.loadContent(NettyResponseFuture.java:222)
   	at org.asynchttpclient.netty.NettyResponseFuture.done(NettyResponseFuture.java:257)
   	at org.asynchttpclient.netty.handler.AsyncHttpClientHandler.finishUpdate(AsyncHttpClientHandler.java:241)
   	at org.asynchttpclient.netty.handler.HttpHandler.handleChunk(HttpHandler.java:114)
   	at org.asynchttpclient.netty.handler.HttpHandler.handleRead(HttpHandler.java:143)
   	at org.asynchttpclient.netty.handler.AsyncHttpClientHandler.channelRead(AsyncHttpClientHandler.java:78)
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
   	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
   	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
   	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
   	at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
   	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324)
   	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296)
   	at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
   	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
   	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
   	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
   	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
   	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:719)
   	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655)
   	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581)
   	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
   	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
   	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
   	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
   	at java.base/java.lang.Thread.run(Thread.java:829)
   </pre></code>
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar] Jason918 edited a comment on issue #12774: Flaky-test: [org.apache.pulsar.broker.service.ReplicatorTest].[testDoNotReplicateSystemTopic]

Posted by GitBox <gi...@apache.org>.
Jason918 edited a comment on issue #12774:
URL: https://github.com/apache/pulsar/issues/12774#issuecomment-966995746


   Test locally with current master (c3da1452a444c9599cb85562a3faa82ddfdecec8), it still fails.
   I am looking into this.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar] codelipenghui closed issue #12774: Flaky-test: [org.apache.pulsar.broker.service.ReplicatorTest].[testDoNotReplicateSystemTopic]

Posted by GitBox <gi...@apache.org>.
codelipenghui closed issue #12774:
URL: https://github.com/apache/pulsar/issues/12774


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar] merlimat closed issue #12774: Flaky-test: ReplicatorTest.testDoNotReplicateSystemTopic

Posted by GitBox <gi...@apache.org>.
merlimat closed issue #12774:
URL: https://github.com/apache/pulsar/issues/12774


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar] Jason918 commented on issue #12774: Flaky-test: [org.apache.pulsar.broker.service.ReplicatorTest].[testDoNotReplicateSystemTopic]

Posted by GitBox <gi...@apache.org>.
Jason918 commented on issue #12774:
URL: https://github.com/apache/pulsar/issues/12774#issuecomment-966995746


   Test locally with current mast (c3da1452a444c9599cb85562a3faa82ddfdecec8), it still fails.
   I am looking into this.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar] Jason918 commented on issue #12774: Flaky-test: ReplicatorTest.testDoNotReplicateSystemTopic

Posted by GitBox <gi...@apache.org>.
Jason918 commented on issue #12774:
URL: https://github.com/apache/pulsar/issues/12774#issuecomment-1029927197


   Here is the related exceptions
   ```
   2022-02-04T17:01:04,284+0800 [main-EventThread] ERROR PulsarLedgerIdGenerator - Error generating ledger id: org.apache.pulsar.metadata.api.MetadataStoreException: org.apache.pulsar.metadata.api.MetadataStoreException$BadVersionException: org.apache.zookeeper.KeeperException$BadVersionException: KeeperErrorCode = BadVersion for /ledgers/idgen
   2022-02-04T17:01:04,285+0800 [BookKeeperClientWorker-OrderedExecutor-1-0] ERROR ManagedLedgerFactoryImpl - [pulsar/ns-2a526cfb-b9f1-451f-868f-6324b5ff8d1e/persistent/testDoesNotReplicateSystemTopic] Failed to initialize managed ledger: Error while using MetaStore
   2022-02-04T17:01:04,285+0800 [BookKeeperClientWorker-OrderedExecutor-1-0] INFO  ManagedLedgerImpl - [pulsar/ns-2a526cfb-b9f1-451f-868f-6324b5ff8d1e/persistent/testDoesNotReplicateSystemTopic] Closing managed ledger
   2022-02-04T17:01:04,285+0800 [BookKeeperClientWorker-OrderedExecutor-1-0] DEBUG EntryCacheManager - Removed cache for pulsar/ns-2a526cfb-b9f1-451f-868f-6324b5ff8d1e/persistent/testDoesNotReplicateSystemTopic - Size: 0.0 -- Current Size: 0.0
   2022-02-04T17:01:04,285+0800 [BookKeeperClientWorker-OrderedExecutor-1-0] WARN  BrokerService - Failed to create topic persistent://pulsar/ns-2a526cfb-b9f1-451f-868f-6324b5ff8d1e/testDoesNotReplicateSystemTopic
   org.apache.bookkeeper.mledger.ManagedLedgerException: Error while using MetaStore
   2022-02-04T17:01:04,285+0800 [pulsar-web-288-14] ERROR PersistentTopicsBase - [null] Failed to create non-partitioned topic persistent://pulsar/ns-2a526cfb-b9f1-451f-868f-6324b5ff8d1e/testDoesNotReplicateSystemTopic
   java.util.concurrent.CompletionException: org.apache.pulsar.broker.service.BrokerServiceException$PersistenceException: org.apache.bookkeeper.mledger.ManagedLedgerException: Error while using MetaStore
           at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:331) ~[?:?]
           at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:346) ~[?:?]
           at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:632) ~[?:?]
           at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) ~[?:?]
           at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088) ~[?:?]
           at org.apache.pulsar.broker.service.BrokerService$2.openLedgerFailed(BrokerService.java:1433) ~[classes/:?]
           at org.apache.bookkeeper.mledger.impl.ManagedLedgerFactoryImpl.lambda$asyncOpen$8(ManagedLedgerFactoryImpl.java:426) ~[classes/:?]
           at java.util.concurrent.CompletableFuture.uniExceptionally(CompletableFuture.java:986) ~[?:?]
           at java.util.concurrent.CompletableFuture$UniExceptionally.tryFire(CompletableFuture.java:970) ~[?:?]
           at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) ~[?:?]
           at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088) ~[?:?]
           at org.apache.bookkeeper.mledger.impl.ManagedLedgerFactoryImpl$2.initializeFailed(ManagedLedgerFactoryImpl.java:421) ~[classes/:?]
           at org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl.lambda$initializeBookKeeper$1(ManagedLedgerImpl.java:474) ~[classes/:?]
           at org.apache.bookkeeper.mledger.util.SafeRun$1.safeRun(SafeRun.java:32) ~[classes/:?]
           at org.apache.bookkeeper.common.util.SafeRunnable.run(SafeRunnable.java:36) ~[bookkeeper-common-4.14.4.jar:4.14.4]
           at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
           at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
           at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-common-4.1.73.Final.jar:4.1.73.Final]
           at java.lang.Thread.run(Thread.java:829) [?:?]
   Caused by: org.apache.pulsar.broker.service.BrokerServiceException$PersistenceException: org.apache.bookkeeper.mledger.ManagedLedgerException: Error while using MetaStore
           ... 14 more
   Caused by: org.apache.bookkeeper.mledger.ManagedLedgerException: Error while using MetaStore
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar] Jason918 commented on issue #12774: Flaky-test: [org.apache.pulsar.broker.service.ReplicatorTest].[testDoNotReplicateSystemTopic]

Posted by GitBox <gi...@apache.org>.
Jason918 commented on issue #12774:
URL: https://github.com/apache/pulsar/issues/12774#issuecomment-1016155097


   This test is updated since last fix.
   Reproduced this locally, I will look into this.
   
   ```
   2022-01-19T15:20:57,724+0800 [main-EventThread] ERROR PulsarLedgerIdGenerator - Error generating ledger id: org.apache.pulsar.metadata.api.MetadataStoreException: org.apache.pulsar.metadata.api.MetadataStoreException$BadVersionException: org.apache.zookeeper.KeeperException$BadVersionException: KeeperErrorCode = BadVersion for /ledgers/idgen
   2022-01-19T15:20:57,725+0800 [BookKeeperClientWorker-OrderedExecutor-7-0] ERROR ManagedLedgerFactoryImpl - [pulsar/ns-32310951-1aef-4c4a-b2e8-89cd72900689/persistent/testDoesNotReplicateSystemTopic] Failed to initialize managed ledger: Error while using MetaStore
   2022-01-19T15:20:57,725+0800 [BookKeeperClientWorker-OrderedExecutor-7-0] INFO  ManagedLedgerImpl - [pulsar/ns-32310951-1aef-4c4a-b2e8-89cd72900689/persistent/testDoesNotReplicateSystemTopic] Closing managed ledger
   2022-01-19T15:20:57,725+0800 [BookKeeperClientWorker-OrderedExecutor-7-0] DEBUG EntryCacheManager - Removed cache for pulsar/ns-32310951-1aef-4c4a-b2e8-89cd72900689/persistent/testDoesNotReplicateSystemTopic - Size: 0.0 -- Current Size: 0.0
   2022-01-19T15:20:57,725+0800 [BookKeeperClientWorker-OrderedExecutor-7-0] WARN  BrokerService - Failed to create topic persistent://pulsar/ns-32310951-1aef-4c4a-b2e8-89cd72900689/testDoesNotReplicateSystemTopic
   org.apache.bookkeeper.mledger.ManagedLedgerException: Error while using MetaStore
   2022-01-19T15:20:57,725+0800 [pulsar-web-265-14] ERROR PersistentTopicsBase - [null] Failed to create non-partitioned topic persistent://pulsar/ns-32310951-1aef-4c4a-b2e8-89cd72900689/testDoesNotReplicateSystemTopic
   java.util.concurrent.CompletionException: org.apache.pulsar.broker.service.BrokerServiceException$PersistenceException: org.apache.bookkeeper.mledger.ManagedLedgerException: Error while using MetaStore
   	at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:331) ~[?:?]
   	at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:346) ~[?:?]
   	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:632) ~[?:?]
   	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) ~[?:?]
   	at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088) ~[?:?]
   	at org.apache.pulsar.broker.service.BrokerService$2.openLedgerFailed(BrokerService.java:1429) ~[classes/:?]
   	at org.apache.bookkeeper.mledger.impl.ManagedLedgerFactoryImpl.lambda$asyncOpen$8(ManagedLedgerFactoryImpl.java:426) ~[classes/:?]
   	at java.util.concurrent.CompletableFuture.uniExceptionally(CompletableFuture.java:986) ~[?:?]
   	at java.util.concurrent.CompletableFuture$UniExceptionally.tryFire(CompletableFuture.java:970) ~[?:?]
   	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) ~[?:?]
   	at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088) ~[?:?]
   	at org.apache.bookkeeper.mledger.impl.ManagedLedgerFactoryImpl$2.initializeFailed(ManagedLedgerFactoryImpl.java:421) ~[classes/:?]
   	at org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl.lambda$initializeBookKeeper$1(ManagedLedgerImpl.java:473) ~[classes/:?]
   	at org.apache.bookkeeper.mledger.util.SafeRun$1.safeRun(SafeRun.java:32) ~[classes/:?]
   	at org.apache.bookkeeper.common.util.SafeRunnable.run(SafeRunnable.java:36) ~[bookkeeper-common-4.14.4.jar:4.14.4]
   	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
   	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
   	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-common-4.1.72.Final.jar:4.1.72.Final]
   	at java.lang.Thread.run(Thread.java:829) [?:?]
   Caused by: org.apache.pulsar.broker.service.BrokerServiceException$PersistenceException: org.apache.bookkeeper.mledger.ManagedLedgerException: Error while using MetaStore
   	... 14 more
   Caused by: org.apache.bookkeeper.mledger.ManagedLedgerException: Error while using MetaStore
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar] lhotari commented on issue #12774: Flaky-test: [org.apache.pulsar.broker.service.ReplicatorTest].[testDoNotReplicateSystemTopic]

Posted by GitBox <gi...@apache.org>.
lhotari commented on issue #12774:
URL: https://github.com/apache/pulsar/issues/12774#issuecomment-1016130498


   recent failure: https://github.com/apache/pulsar/runs/4863165088?check_suite_focus=true#step:9:2076
   ```
   Error:  testDoNotReplicateSystemTopic(org.apache.pulsar.broker.service.ReplicatorTest)  Time elapsed: 13.38 s  <<< FAILURE!
   org.apache.pulsar.client.admin.PulsarAdminException$ServerSideErrorException: HTTP 500 Internal Server Error
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar] Jason918 commented on issue #12774: Flaky-test: ReplicatorTest.testDoNotReplicateSystemTopic

Posted by GitBox <gi...@apache.org>.
Jason918 commented on issue #12774:
URL: https://github.com/apache/pulsar/issues/12774#issuecomment-1029928444


   After some digging, the root cause is that there is a race condition in `org.apache.pulsar.metadata.bookkeeper.PulsarLedgerIdGenerator#generateShortLedgerId` when creating the path `shortIdGenPath`.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [pulsar] lhotari commented on issue #12774: Flaky-test: ReplicatorTest.testDoNotReplicateSystemTopic

Posted by GitBox <gi...@apache.org>.
lhotari commented on issue #12774:
URL: https://github.com/apache/pulsar/issues/12774#issuecomment-1029276647


           
   Flaky-test: org.apache.pulsar.broker.service.ReplicatorTest.testDoNotReplicateSystemTopic
   Number of failures: 9
   
   org.apache.pulsar.broker.service.ReplicatorTest is flaky. The testDoNotReplicateSystemTopic test method fails sporadically.
   
   ```
   org.apache.pulsar.client.admin.PulsarAdminException$ServerSideErrorException: HTTP 500 Internal Server Error
   	at org.apache.pulsar.client.admin.internal.BaseResource.getApiException(BaseResource.java:223)
   	at org.apache.pulsar.client.admin.internal.BaseResource$1.failed(BaseResource.java:134)
   	at org.glassfish.jersey.client.JerseyInvocation$1.failed(JerseyInvocation.java:882)
   	at org.glassfish.jersey.client.JerseyInvocation$1.completed(JerseyInvocation.java:863)
   	at org.glassfish.jersey.client.ClientRuntime.processResponse(ClientRuntime.java:229)
   	at org.glassfish.jersey.client.ClientRuntime.access$200(ClientRuntime.java:62)
   	at org.glassfish.jersey.client.ClientRuntime$2.lambda$response$0(ClientRuntime.java:173)
   	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
   	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)
   	at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
   	at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
   	at org.glassfish.jersey.internal.Errors.process(Errors.java:244)
   	at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:288)
   	at org.glassfish.jersey.client.ClientRuntime$2.response(ClientRuntime.java:173)
   ```
   
   Usage tip: To enable automatic navigation to failure message, open the following links with CTRL/CMD-click.  
   [example failure 2022-02-02T13:12:33.6332790Z](https://github.com/apache/pulsar/runs/5036755113?check_suite_focus=true?check_suite_focus=true#step:9:244)  
   [example failure 2022-01-31T16:06:43.1488867Z](https://github.com/apache/pulsar/runs/5008094073?check_suite_focus=true?check_suite_focus=true#step:9:3675)  
   [example failure 2022-01-29T04:35:28.2343010Z](https://github.com/apache/pulsar/runs/4989272866?check_suite_focus=true?check_suite_focus=true#step:9:244)  
   [example failure 2022-01-28T19:52:19.7972963Z](https://github.com/apache/pulsar/runs/4985338436?check_suite_focus=true?check_suite_focus=true#step:9:5740)  
   [example failure 2022-01-28T19:32:03.9951364Z](https://github.com/apache/pulsar/runs/4985338436?check_suite_focus=true?check_suite_focus=true#step:9:250)  
   [example failure 2022-01-28T12:47:56.0423613Z](https://github.com/apache/pulsar/runs/4980584137?check_suite_focus=true?check_suite_focus=true#step:9:242)  
   [example failure 2022-01-28T12:40:39.4185713Z](https://github.com/apache/pulsar/runs/4980324517?check_suite_focus=true?check_suite_focus=true#step:9:3104)  
   [example failure 2022-01-28T09:15:16.9877679Z](https://github.com/apache/pulsar/runs/4978229090?check_suite_focus=true?check_suite_focus=true#step:9:240)  
   
   
   <details>
   <summary>Full exception stacktrace</summary>
   <code><pre>
   org.apache.pulsar.client.admin.PulsarAdminException$ServerSideErrorException: HTTP 500 Internal Server Error
   	at org.apache.pulsar.client.admin.internal.BaseResource.getApiException(BaseResource.java:223)
   	at org.apache.pulsar.client.admin.internal.BaseResource$1.failed(BaseResource.java:134)
   	at org.glassfish.jersey.client.JerseyInvocation$1.failed(JerseyInvocation.java:882)
   	at org.glassfish.jersey.client.JerseyInvocation$1.completed(JerseyInvocation.java:863)
   	at org.glassfish.jersey.client.ClientRuntime.processResponse(ClientRuntime.java:229)
   	at org.glassfish.jersey.client.ClientRuntime.access$200(ClientRuntime.java:62)
   	at org.glassfish.jersey.client.ClientRuntime$2.lambda$response$0(ClientRuntime.java:173)
   	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
   	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)
   	at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
   	at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
   	at org.glassfish.jersey.internal.Errors.process(Errors.java:244)
   	at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:288)
   	at org.glassfish.jersey.client.ClientRuntime$2.response(ClientRuntime.java:173)
   	at org.apache.pulsar.client.admin.internal.http.AsyncHttpConnector.lambda$apply$1(AsyncHttpConnector.java:228)
   	at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859)
   	at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837)
   	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
   	at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2073)
   	at org.apache.pulsar.client.admin.internal.http.AsyncHttpConnector.lambda$retryOperation$4(AsyncHttpConnector.java:270)
   	at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859)
   	at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837)
   	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
   	at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2073)
   	at org.asynchttpclient.netty.NettyResponseFuture.loadContent(NettyResponseFuture.java:222)
   	at org.asynchttpclient.netty.NettyResponseFuture.done(NettyResponseFuture.java:257)
   	at org.asynchttpclient.netty.handler.AsyncHttpClientHandler.finishUpdate(AsyncHttpClientHandler.java:241)
   	at org.asynchttpclient.netty.handler.HttpHandler.handleChunk(HttpHandler.java:114)
   	at org.asynchttpclient.netty.handler.HttpHandler.handleRead(HttpHandler.java:143)
   	at org.asynchttpclient.netty.handler.AsyncHttpClientHandler.channelRead(AsyncHttpClientHandler.java:78)
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
   	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
   	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
   	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
   	at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
   	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:327)
   	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:299)
   	at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
   	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
   	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
   	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
   	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
   	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:722)
   	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:658)
   	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:584)
   	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496)
   	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
   	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
   	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
   	at java.base/java.lang.Thread.run(Thread.java:829)
   Caused by: javax.ws.rs.InternalServerErrorException: HTTP 500 Internal Server Error
   	at org.glassfish.jersey.client.JerseyInvocation.convertToException(JerseyInvocation.java:960)
   	at org.glassfish.jersey.client.JerseyInvocation.access$700(JerseyInvocation.java:82)
   	... 54 more
   
   </pre></code>
   </details>
   
   [example failure 2022-01-27T10:12:11.6608308Z](https://github.com/apache/pulsar/runs/4964192400?check_suite_focus=true?check_suite_focus=true#step:9:238)  
   
   
   <details>
   <summary>Full exception stacktrace</summary>
   <code><pre>
   org.apache.pulsar.client.admin.PulsarAdminException$ServerSideErrorException: HTTP 500 Internal Server Error
   	at org.apache.pulsar.client.admin.internal.BaseResource.getApiException(BaseResource.java:223)
   	at org.apache.pulsar.client.admin.internal.BaseResource$1.failed(BaseResource.java:134)
   	at org.glassfish.jersey.client.JerseyInvocation$1.failed(JerseyInvocation.java:882)
   	at org.glassfish.jersey.client.JerseyInvocation$1.completed(JerseyInvocation.java:863)
   	at org.glassfish.jersey.client.ClientRuntime.processResponse(ClientRuntime.java:229)
   	at org.glassfish.jersey.client.ClientRuntime.access$200(ClientRuntime.java:62)
   	at org.glassfish.jersey.client.ClientRuntime$2.lambda$response$0(ClientRuntime.java:173)
   	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
   	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)
   	at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
   	at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
   	at org.glassfish.jersey.internal.Errors.process(Errors.java:244)
   	at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:288)
   	at org.glassfish.jersey.client.ClientRuntime$2.response(ClientRuntime.java:173)
   	at org.apache.pulsar.client.admin.internal.http.AsyncHttpConnector.lambda$apply$1(AsyncHttpConnector.java:228)
   	at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859)
   	at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837)
   	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
   	at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2073)
   	at org.apache.pulsar.client.admin.internal.http.AsyncHttpConnector.lambda$retryOperation$4(AsyncHttpConnector.java:270)
   	at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859)
   	at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837)
   	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
   	at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2073)
   	at org.asynchttpclient.netty.NettyResponseFuture.loadContent(NettyResponseFuture.java:222)
   	at org.asynchttpclient.netty.NettyResponseFuture.done(NettyResponseFuture.java:257)
   	at org.asynchttpclient.netty.handler.AsyncHttpClientHandler.finishUpdate(AsyncHttpClientHandler.java:241)
   	at org.asynchttpclient.netty.handler.HttpHandler.handleChunk(HttpHandler.java:114)
   	at org.asynchttpclient.netty.handler.HttpHandler.handleRead(HttpHandler.java:143)
   	at org.asynchttpclient.netty.handler.AsyncHttpClientHandler.channelRead(AsyncHttpClientHandler.java:78)
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
   	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
   	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
   	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
   	at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
   	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324)
   	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296)
   	at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
   	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
   	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
   	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
   	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
   	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
   	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:722)
   	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:658)
   	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:584)
   	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496)
   	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
   	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
   	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
   	at java.base/java.lang.Thread.run(Thread.java:829)
   Caused by: javax.ws.rs.InternalServerErrorException: HTTP 500 Internal Server Error
   	at org.glassfish.jersey.client.JerseyInvocation.convertToException(JerseyInvocation.java:960)
   	at org.glassfish.jersey.client.JerseyInvocation.access$700(JerseyInvocation.java:82)
   	... 54 more
   
   </pre></code>
   </details>
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org