You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/02/12 05:35:20 UTC

[GitHub] [pulsar] michaeljmarshall edited a comment on issue #9450: Flaky-test: org.apache.pulsar.broker.service.ReplicatorTest.testResetCursorNotFail

michaeljmarshall edited a comment on issue #9450:
URL: https://github.com/apache/pulsar/issues/9450#issuecomment-777984844


   I also noticed that there are also failures in this test class for `activeBrokerParse`. This test runs before the `testResetCursorNotFail` test. It seems like an additional test failure, but I haven't been able to figure that out yet. I'll look into both. Seems like some of the failures @lhotari references above have a failure from `activeBrokerParse`.
   
   ```
   ------- Starting test [TestClass name=class org.apache.pulsar.broker.service.ReplicatorTest].activeBrokerParse([])-------
   04:21:18.761 [pulsar-web-77-6] INFO  org.eclipse.jetty.server.RequestLog - 127.0.0.1 - - [12/Feb/2021:04:21:18 +0000] "DELETE /admin/namespaces/pulsar/global/ns/backlogQuota?backlogQuotaType=destination_storage HTTP/1.1" 204 0 "-" "Pulsar-Java-v2.8.0-SNAPSHOT" 13
   04:21:18.805 [pulsar-ordered-OrderedExecutor-1-0-EventThread] INFO  org.apache.pulsar.zookeeper.ZooKeeperChildrenCache - [State:CONNECTED Timeout:20000 sessionid:0x10000142f810001 local:/127.0.0.1:56928 remoteserver:localhost/127.0.0.1:33175 lastZxid:32 xid:31 sent:37 recv:42 queuedpkts:0 pendingresp:0 queuedevents:0] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/admin/clusters
   04:21:18.815 [pulsar-web-157-1] INFO  org.apache.pulsar.broker.admin.impl.ClustersBase - [null] Created cluster activeCLuster2
   04:21:18.816 [pulsar-ordered-OrderedExecutor-1-0-EventThread] INFO  org.apache.pulsar.zookeeper.ZooKeeperChildrenCache - reloadCache called in zookeeperChildrenCache for path /admin/clusters
   04:21:18.817 [pulsar-web-157-1] INFO  org.eclipse.jetty.server.RequestLog - 127.0.0.1 - - [12/Feb/2021:04:21:18 +0000] "PUT /admin/v2/clusters/activeCLuster2 HTTP/1.1" 204 0 "-" "Pulsar-Java-v2.8.0-SNAPSHOT" 30
   04:21:19.035 [pulsar-web-157-6] INFO  org.eclipse.jetty.server.RequestLog - 127.0.0.1 - - [12/Feb/2021:04:21:19 +0000] "GET /admin/v2/clusters/activeCLuster2 HTTP/1.1" 200 69 "-" "Pulsar-Java-v2.8.0-SNAPSHOT" 2
   04:21:19.077 [pulsar-web-77-5] INFO  org.eclipse.jetty.server.RequestLog - 127.0.0.1 - - [12/Feb/2021:04:21:19 +0000] "GET /admin/v2/brokers/activeCLuster2 HTTP/1.1" 307 0 "-" "Pulsar-Java-v2.8.0-SNAPSHOT" 23
   04:21:19.089 [pulsar-web-77-7] INFO  org.eclipse.jetty.server.RequestLog - 127.0.0.1 - - [12/Feb/2021:04:21:19 +0000] "GET /admin/v2/brokers/activeCLuster2 HTTP/1.1" 307 0 "-" "Pulsar-Java-v2.8.0-SNAPSHOT" 5
   04:21:19.095 [pulsar-web-77-8] INFO  org.eclipse.jetty.server.RequestLog - 127.0.0.1 - - [12/Feb/2021:04:21:19 +0000] "GET /admin/v2/brokers/activeCLuster2 HTTP/1.1" 307 0 "-" "Pulsar-Java-v2.8.0-SNAPSHOT" 2
   04:21:19.104 [pulsar-web-77-1] INFO  org.eclipse.jetty.server.RequestLog - 127.0.0.1 - - [12/Feb/2021:04:21:19 +0000] "GET /admin/v2/brokers/activeCLuster2 HTTP/1.1" 307 0 "-" "Pulsar-Java-v2.8.0-SNAPSHOT" 3
   04:21:19.112 [pulsar-web-77-6] INFO  org.eclipse.jetty.server.RequestLog - 127.0.0.1 - - [12/Feb/2021:04:21:19 +0000] "GET /admin/v2/brokers/activeCLuster2 HTTP/1.1" 307 0 "-" "Pulsar-Java-v2.8.0-SNAPSHOT" 3
   04:21:19.121 [pulsar-web-77-5] INFO  org.eclipse.jetty.server.RequestLog - 127.0.0.1 - - [12/Feb/2021:04:21:19 +0000] "GET /admin/v2/brokers/activeCLuster2 HTTP/1.1" 307 0 "-" "Pulsar-Java-v2.8.0-SNAPSHOT" 3
   ~~~~~~~~~ SKIPPED -- [TestClass name=class org.apache.pulsar.broker.service.ReplicatorTest].activeBrokerParse([])-------04:21:19.154 [pulsar-web-77-7] INFO  org.apache.pulsar.broker.admin.impl.NamespacesBase - [null] Successfully updated the on namespace /admin/policies/pulsar/ns
   04:21:19.154 [pulsar-web-77-7] INFO  org.apache.pulsar.broker.admin.impl.NamespacesBase - [null] Successfully removed backlog namespace=pulsar/ns, quota=destination_storage
   04:21:19.155 [pulsar-web-77-7] INFO  org.eclipse.jetty.server.RequestLog - 127.0.0.1 - - [12/Feb/2021:04:21:19 +0000] "DELETE /admin/v2/namespaces/pulsar/ns/backlogQuota?backlogQuotaType=destination_storage HTTP/1.1" 204 0 "-" "Pulsar-Java-v2.8.0-SNAPSHOT" 14
   04:21:19.163 [pulsar-ordered-OrderedExecutor-1-0-EventThread] INFO  org.apache.pulsar.zookeeper.ZooKeeperDataCache - [State:CONNECTED Timeout:20000 sessionid:0x10000142f810001 local:/127.0.0.1:56928 remoteserver:localhost/127.0.0.1:33175 lastZxid:34 xid:33 sent:39 recv:45 queuedpkts:0 pendingresp:0 queuedevents:0] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/admin/policies/pulsar/ns1
   04:21:19.164 [pulsar-ordered-OrderedExecutor-3-0-EventThread] INFO  org.apache.pulsar.zookeeper.ZooKeeperDataCache - [State:CONNECTED Timeout:20000 sessionid:0x10000142f810003 local:/127.0.0.1:56950 remoteserver:localhost/127.0.0.1:33175 lastZxid:32 xid:22 sent:26 recv:31 queuedpkts:0 pendingresp:0 queuedevents:0] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/admin/policies/pulsar/ns1
   04:21:19.165 [pulsar-web-77-8] INFO  org.apache.pulsar.broker.admin.impl.NamespacesBase - [null] Successfully updated the on namespace /admin/policies/pulsar/ns1
   04:21:19.165 [pulsar-web-77-8] INFO  org.apache.pulsar.broker.admin.impl.NamespacesBase - [null] Successfully removed backlog namespace=pulsar/ns1, quota=destination_storage
   04:21:19.166 [pulsar-web-77-8] INFO  org.eclipse.jetty.server.RequestLog - 127.0.0.1 - - [12/Feb/2021:04:21:19 +0000] "DELETE /admin/v2/namespaces/pulsar/ns1/backlogQuota?backlogQuotaType=destination_storage HTTP/1.1" 204 0 "-" "Pulsar-Java-v2.8.0-SNAPSHOT" 8
   04:21:19.170 [ForkJoinPool.commonPool-worker-0] INFO  org.apache.pulsar.broker.service.BrokerService - /admin/policies/pulsar/ns1 updating with Policies{auth_policies=org.apache.pulsar.common.policies.data.AuthPolicies@745f, replication_clusters=[r2, r1], bundles=BundlesData{numBundles=1, boundaries=[0x00000000, 0xffffffff]}, backlog_quota_map={}, persistence=null, deduplicationEnabled=null, autoTopicCreationOverride=null, autoSubscriptionCreationOverride=null, clusterDispatchRate={}, topicDispatchRate={}, subscriptionDispatchRate={}, replicatorDispatchRate={}, clusterSubscribeRate={}, publishMaxMessageRate={}, latency_stats_sample_rate={}, message_ttl_in_seconds=null, subscription_expiration_time_minutes=0, retention_policies=null, deleted=false, encryption_required=false, delayed_delivery_policies=null, inactive_topic_policies=null, subscription_auth_mode=None, max_producers_per_topic=null, max_consumers_per_topic=null, max_consumers_per_subscription=null, max_unacked_messages_p
 er_consumer=-1, max_unacked_messages_per_subscription=-1, compaction_threshold=0, offload_threshold=-1, offload_deletion_lag_ms=null, schema_auto_update_compatibility_strategy=Full, schema_validation_enforced=false, schema_compatibility_Strategy=UNDEFINED, is_allow_auto_update_Schema=true, offload_policies=null}
   04:21:19.170 [ForkJoinPool.commonPool-worker-0] INFO  org.apache.pulsar.broker.service.AbstractTopic - Disabling publish throttling for persistent://pulsar/ns1/producer_request_hold-1613103670515
   04:21:19.171 [ForkJoinPool.commonPool-worker-0] INFO  org.apache.pulsar.broker.service.persistent.PersistentTopic - [persistent://pulsar/ns1/producer_request_hold-1613103670515] Policies updated successfully
   04:21:19.171 [ForkJoinPool.commonPool-worker-0] WARN  org.apache.pulsar.broker.service.BrokerService - Topic level policies are not enabled. Please refer to systemTopicEnabled and topicLevelPoliciesEnabled on broker.conf
   04:21:19.171 [ForkJoinPool.commonPool-worker-0] INFO  org.apache.pulsar.broker.service.AbstractTopic - Disabling publish throttling for persistent://pulsar/ns1/producer_exception-1613103652346
   04:21:19.171 [ForkJoinPool.commonPool-worker-0] INFO  org.apache.pulsar.broker.service.persistent.PersistentTopic - [persistent://pulsar/ns1/producer_exception-1613103652346] Policies updated successfully
   04:21:19.172 [ForkJoinPool.commonPool-worker-0] WARN  org.apache.pulsar.broker.service.BrokerService - Topic level policies are not enabled. Please refer to systemTopicEnabled and topicLevelPoliciesEnabled on broker.conf
   04:21:19.176 [ForkJoinPool.commonPool-worker-0] INFO  org.apache.pulsar.broker.service.BrokerService - /admin/policies/pulsar/ns1 updating with Policies{auth_policies=org.apache.pulsar.common.policies.data.AuthPolicies@745f, replication_clusters=[r2, r1], bundles=BundlesData{numBundles=1, boundaries=[0x00000000, 0xffffffff]}, backlog_quota_map={}, persistence=null, deduplicationEnabled=null, autoTopicCreationOverride=null, autoSubscriptionCreationOverride=null, clusterDispatchRate={}, topicDispatchRate={}, subscriptionDispatchRate={}, replicatorDispatchRate={}, clusterSubscribeRate={}, publishMaxMessageRate={}, latency_stats_sample_rate={}, message_ttl_in_seconds=null, subscription_expiration_time_minutes=0, retention_policies=null, deleted=false, encryption_required=false, delayed_delivery_policies=null, inactive_topic_policies=null, subscription_auth_mode=None, max_producers_per_topic=null, max_consumers_per_topic=null, max_consumers_per_subscription=null, max_unacked_messages_p
 er_consumer=-1, max_unacked_messages_per_subscription=-1, compaction_threshold=0, offload_threshold=-1, offload_deletion_lag_ms=null, schema_auto_update_compatibility_strategy=Full, schema_validation_enforced=false, schema_compatibility_Strategy=UNDEFINED, is_allow_auto_update_Schema=true, offload_policies=null}
   04:21:19.177 [ForkJoinPool.commonPool-worker-0] INFO  org.apache.pulsar.broker.service.AbstractTopic - Disabling publish throttling for persistent://pulsar/ns1/producer_request_hold-1613103670515
   04:21:19.180 [pulsar-web-77-1] INFO  org.apache.pulsar.broker.admin.impl.NamespacesBase - [null] Successfully updated the on namespace /admin/policies/pulsar/global/ns
   04:21:19.181 [pulsar-web-77-1] INFO  org.apache.pulsar.broker.admin.impl.NamespacesBase - [null] Successfully removed backlog namespace=pulsar/global/ns, quota=destination_storage
   04:21:19.181 [pulsar-web-77-1] INFO  org.eclipse.jetty.server.RequestLog - 127.0.0.1 - - [12/Feb/2021:04:21:19 +0000] "DELETE /admin/namespaces/pulsar/global/ns/backlogQuota?backlogQuotaType=destination_storage HTTP/1.1" 204 0 "-" "Pulsar-Java-v2.8.0-SNAPSHOT" 11
   ------- Starting test [TestClass name=class org.apache.pulsar.broker.service.ReplicatorTest].activeBrokerParse([])-------
   04:21:19.188 [ForkJoinPool.commonPool-worker-0] INFO  org.apache.pulsar.broker.service.Producer - [null] is not allowed to produce on topic [persistent://pulsar/ns1/producer_request_hold-1613103670515] anymore
   04:21:19.189 [ForkJoinPool.commonPool-worker-0] INFO  org.apache.pulsar.broker.service.Producer - Disconnecting producer: Producer{topic=PersistentTopic{topic=persistent://pulsar/ns1/producer_request_hold-1613103670515}, client=/127.0.0.1:47894, producerName=pulsar.repl.r2, producerId=1}
   04:21:19.189 [ForkJoinPool.commonPool-worker-0] INFO  org.apache.pulsar.broker.service.persistent.PersistentTopic - [persistent://pulsar/ns1/producer_request_hold-1613103670515] Policies updated successfully
   04:21:19.190 [ForkJoinPool.commonPool-worker-0] WARN  org.apache.pulsar.broker.service.BrokerService - Topic level policies are not enabled. Please refer to systemTopicEnabled and topicLevelPoliciesEnabled on broker.conf
   04:21:19.190 [pulsar-io-139-2] INFO  org.apache.pulsar.client.impl.ClientCnx - [localhost/127.0.0.1:41689] Broker notification of Closed producer: 1
   04:21:19.190 [pulsar-io-139-2] INFO  org.apache.pulsar.client.impl.ConnectionHandler - [persistent://pulsar/ns1/producer_request_hold-1613103670515] [pulsar.repl.r2] Closed connection [id: 0x51209843, L:/127.0.0.1:47894 - R:localhost/127.0.0.1:41689] -- Will try again in 0.1 s
   04:21:19.190 [ForkJoinPool.commonPool-worker-0] INFO  org.apache.pulsar.broker.service.AbstractTopic - Disabling publish throttling for persistent://pulsar/ns1/producer_exception-1613103652346
   04:21:19.209 [ForkJoinPool.commonPool-worker-0] INFO  org.apache.pulsar.broker.service.Producer - [null] is not allowed to produce on topic [persistent://pulsar/ns1/producer_exception-1613103652346] anymore
   04:21:19.210 [ForkJoinPool.commonPool-worker-0] INFO  org.apache.pulsar.broker.service.Producer - Disconnecting producer: Producer{topic=PersistentTopic{topic=persistent://pulsar/ns1/producer_exception-1613103652346}, client=/127.0.0.1:47960, producerName=pulsar.repl.r2, producerId=0}
   04:21:19.210 [ForkJoinPool.commonPool-worker-0] INFO  org.apache.pulsar.broker.service.persistent.PersistentTopic - [persistent://pulsar/ns1/producer_exception-1613103652346] Policies updated successfully
   04:21:19.210 [pulsar-io-139-1] INFO  org.apache.pulsar.client.impl.ClientCnx - [localhost/127.0.0.1:41689] Broker notification of Closed producer: 0
   04:21:19.210 [ForkJoinPool.commonPool-worker-0] WARN  org.apache.pulsar.broker.service.BrokerService - Topic level policies are not enabled. Please refer to systemTopicEnabled and topicLevelPoliciesEnabled on broker.conf
   04:21:19.210 [pulsar-io-139-1] INFO  org.apache.pulsar.client.impl.ConnectionHandler - [persistent://pulsar/ns1/producer_exception-1613103652346] [pulsar.repl.r2] Closed connection [id: 0xb46cf921, L:/127.0.0.1:47960 - R:localhost/127.0.0.1:41689] -- Will try again in 0.1 s
   04:21:19.226 [pulsar-web-157-8] WARN  org.apache.pulsar.broker.admin.impl.ClustersBase - [null] Failed to create already existing cluster activeCLuster2
   04:21:19.226 [pulsar-web-157-8] ERROR org.apache.pulsar.broker.admin.impl.ClustersBase - [null] Failed to create cluster activeCLuster2
   org.apache.pulsar.broker.web.RestException: Cluster already exists
   	at org.apache.pulsar.broker.admin.impl.ClustersBase.createCluster(ClustersBase.java:171) ~[classes/:?]
   	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_275]
   	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_275]
   	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_275]
   	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_275]
   	at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) ~[jersey-server-2.31.jar:?]
   	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) ~[jersey-server-2.31.jar:?]
   	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) ~[jersey-server-2.31.jar:?]
   	at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$VoidOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:159) ~[jersey-server-2.31.jar:?]
   	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) ~[jersey-server-2.31.jar:?]
   	at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:469) ~[jersey-server-2.31.jar:?]
   	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:391) ~[jersey-server-2.31.jar:?]
   	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:80) ~[jersey-server-2.31.jar:?]
   	at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:253) ~[jersey-server-2.31.jar:?]
   	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) ~[jersey-common-2.31.jar:?]
   	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) ~[jersey-common-2.31.jar:?]
   	at org.glassfish.jersey.internal.Errors.process(Errors.java:292) ~[jersey-common-2.31.jar:?]
   	at org.glassfish.jersey.internal.Errors.process(Errors.java:274) ~[jersey-common-2.31.jar:?]
   	at org.glassfish.jersey.internal.Errors.process(Errors.java:244) ~[jersey-common-2.31.jar:?]
   	at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) ~[jersey-common-2.31.jar:?]
   	at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:232) ~[jersey-server-2.31.jar:?]
   	at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:680) ~[jersey-server-2.31.jar:?]
   	at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394) ~[jersey-container-servlet-core-2.31.jar:?]
   	at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346) ~[jersey-container-servlet-core-2.31.jar:?]
   	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:366) ~[jersey-container-servlet-core-2.31.jar:?]
   	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:319) ~[jersey-container-servlet-core-2.31.jar:?]
   	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205) ~[jersey-container-servlet-core-2.31.jar:?]
   	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:791) ~[jetty-servlet-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1626) ~[jetty-servlet-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.apache.pulsar.broker.web.ResponseHandlerFilter.doFilter(ResponseHandlerFilter.java:58) ~[classes/:?]
   	at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) ~[jetty-servlet-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) ~[jetty-servlet-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) ~[jetty-servlet-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1435) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) ~[jetty-servlet-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1350) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:234) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:179) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.server.Server.handle(Server.java:516) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:388) ~[jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:633) [jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:380) [jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) [jetty-server-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) [jetty-io-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) [jetty-io-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) [jetty-io-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) [jetty-util-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) [jetty-util-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) [jetty-util-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) [jetty-util-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) [jetty-util-9.4.35.v20201120.jar:9.4.35.v20201120]
   	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_275]
   	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_275]
   	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-common-4.1.51.Final.jar:4.1.51.Final]
   	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_275]
   04:21:19.230 [pulsar-web-157-8] INFO  org.eclipse.jetty.server.RequestLog - 127.0.0.1 - - [12/Feb/2021:04:21:19 +0000] "PUT /admin/v2/clusters/activeCLuster2 HTTP/1.1" 409 35 "-" "Pulsar-Java-v2.8.0-SNAPSHOT" 22
   04:21:19.237 [AsyncHttpClient-248-1] WARN  org.apache.pulsar.client.admin.internal.BaseResource - [http://localhost:44131/admin/v2/clusters/activeCLuster2] Failed to perform http put request: javax.ws.rs.ClientErrorException: HTTP 409 Conflict
   !!!!!!!!! FAILURE-- [TestClass name=class org.apache.pulsar.broker.service.ReplicatorTest].activeBrokerParse([])-------
   04:21:19.287 [pulsar-web-77-6] INFO  org.apache.pulsar.broker.admin.impl.NamespacesBase - [null] Successfully updated the on namespace /admin/policies/pulsar/ns
   04:21:19.288 [pulsar-web-77-6] INFO  org.apache.pulsar.broker.admin.impl.NamespacesBase - [null] Successfully removed backlog namespace=pulsar/ns, quota=destination_storage
   04:21:19.290 [pulsar-web-77-6] INFO  org.eclipse.jetty.server.RequestLog - 127.0.0.1 - - [12/Feb/2021:04:21:19 +0000] "DELETE /admin/v2/namespaces/pulsar/ns/backlogQuota?backlogQuotaType=destination_storage HTTP/1.1" 204 0 "-" "Pulsar-Java-v2.8.0-SNAPSHOT" 10
   04:21:19.291 [pulsar-timer-266-1] INFO  org.apache.pulsar.client.impl.ConnectionHandler - [persistent://pulsar/ns1/producer_request_hold-1613103670515] [pulsar.repl.r2] Reconnecting after timeout
   04:21:19.292 [pulsar-io-139-3] INFO  org.apache.pulsar.client.impl.ConnectionPool - [[id: 0x72857fce, L:/127.0.0.1:47990 - R:localhost/127.0.0.1:41689]] Connected to server
   04:21:19.293 [pulsar-io-59-4] INFO  org.apache.pulsar.broker.service.ServerCnx - New connection from /127.0.0.1:47990
   04:21:19.301 [pulsar-ordered-OrderedExecutor-3-0-EventThread] INFO  org.apache.pulsar.zookeeper.ZooKeeperDataCache - [State:CONNECTED Timeout:20000 sessionid:0x10000142f810003 local:/127.0.0.1:56950 remoteserver:localhost/127.0.0.1:33175 lastZxid:36 xid:23 sent:27 recv:33 queuedpkts:0 pendingresp:0 queuedevents:0] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/admin/policies/pulsar/ns1
   04:21:19.302 [pulsar-ordered-OrderedExecutor-1-0-EventThread] INFO  org.apache.pulsar.zookeeper.ZooKeeperDataCache - [State:CONNECTED Timeout:20000 sessionid:0x10000142f810001 local:/127.0.0.1:56928 remoteserver:localhost/127.0.0.1:33175 lastZxid:36 xid:34 sent:40 recv:47 queuedpkts:0 pendingresp:0 queuedevents:0] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/admin/policies/pulsar/ns1
   04:21:19.307 [pulsar-web-77-5] INFO  org.apache.pulsar.broker.admin.impl.NamespacesBase - [null] Successfully updated the on namespace /admin/policies/pulsar/ns1
   04:21:19.307 [pulsar-web-77-5] INFO  org.apache.pulsar.broker.admin.impl.NamespacesBase - [null] Successfully removed backlog namespace=pulsar/ns1, quota=destination_storage
   04:21:19.310 [pulsar-web-77-5] INFO  org.eclipse.jetty.server.RequestLog - 127.0.0.1 - - [12/Feb/2021:04:21:19 +0000] "DELETE /admin/v2/namespaces/pulsar/ns1/backlogQuota?backlogQuotaType=destination_storage HTTP/1.1" 204 0 "-" "Pulsar-Java-v2.8.0-SNAPSHOT" 17
   04:21:19.311 [pulsar-timer-266-1] INFO  org.apache.pulsar.client.impl.ConnectionHandler - [persistent://pulsar/ns1/producer_exception-1613103652346] [pulsar.repl.r2] Reconnecting after timeout
   04:21:19.315 [pulsar-io-59-4] WARN  org.apache.pulsar.broker.service.ServerCnx - Role null is not authorized to perform operation LOOKUP on topic persistent://pulsar/ns1/producer_request_hold-1613103670515
   04:21:19.315 [pulsar-io-59-4] WARN  org.apache.pulsar.broker.service.ServerCnx - [/127.0.0.1:47990] Proxy Client is not authorized to Lookup with role null on topic persistent://pulsar/ns1/producer_request_hold-1613103670515
   04:21:19.317 [pulsar-io-139-3] WARN  org.apache.pulsar.client.impl.BinaryProtoLookupService - [persistent://pulsar/ns1/producer_request_hold-1613103670515] failed to send lookup request : Proxy Client is not authorized to Lookup
   04:21:19.314 [ForkJoinPool.commonPool-worker-0] INFO  org.apache.pulsar.broker.service.BrokerService - /admin/policies/pulsar/ns1 updating with Policies{auth_policies=org.apache.pulsar.common.policies.data.AuthPolicies@745f, replication_clusters=[r2, r1], bundles=BundlesData{numBundles=1, boundaries=[0x00000000, 0xffffffff]}, backlog_quota_map={}, persistence=null, deduplicationEnabled=null, autoTopicCreationOverride=null, autoSubscriptionCreationOverride=null, clusterDispatchRate={}, topicDispatchRate={}, subscriptionDispatchRate={}, replicatorDispatchRate={}, clusterSubscribeRate={}, publishMaxMessageRate={}, latency_stats_sample_rate={}, message_ttl_in_seconds=null, subscription_expiration_time_minutes=0, retention_policies=null, deleted=false, encryption_required=false, delayed_delivery_policies=null, inactive_topic_policies=null, subscription_auth_mode=None, max_producers_per_topic=null, max_consumers_per_topic=null, max_consumers_per_subscription=null, max_unacked_messages_p
 er_consumer=-1, max_unacked_messages_per_subscription=-1, compaction_threshold=0, offload_threshold=-1, offload_deletion_lag_ms=null, schema_auto_update_compatibility_strategy=Full, schema_validation_enforced=false, schema_compatibility_Strategy=UNDEFINED, is_allow_auto_update_Schema=true, offload_policies=null}
   04:21:19.317 [ForkJoinPool.commonPool-worker-0] INFO  org.apache.pulsar.broker.service.AbstractTopic - Disabling publish throttling for persistent://pulsar/ns1/producer_request_hold-1613103670515
   04:21:19.319 [pulsar-io-139-3] WARN  org.apache.pulsar.client.impl.ConnectionHandler - [persistent://pulsar/ns1/producer_request_hold-1613103670515] [pulsar.repl.r2] Error connecting to broker: org.apache.pulsar.client.api.PulsarClientException$AuthorizationException: Proxy Client is not authorized to Lookup
   04:21:19.319 [pulsar-io-139-3] WARN  org.apache.pulsar.client.impl.ConnectionHandler - [persistent://pulsar/ns1/producer_request_hold-1613103670515] [pulsar.repl.r2] Could not get connection to broker: org.apache.pulsar.client.api.PulsarClientException$AuthorizationException: Proxy Client is not authorized to Lookup -- Will try again in 0.1 s
   04:21:19.319 [ForkJoinPool.commonPool-worker-0] INFO  org.apache.pulsar.broker.service.persistent.PersistentTopic - [persistent://pulsar/ns1/producer_request_hold-1613103670515] Policies updated successfully
   04:21:19.319 [ForkJoinPool.commonPool-worker-0] WARN  org.apache.pulsar.broker.service.BrokerService - Topic level policies are not enabled. Please refer to systemTopicEnabled and topicLevelPoliciesEnabled on broker.conf
   04:21:19.319 [ForkJoinPool.commonPool-worker-0] INFO  org.apache.pulsar.broker.service.AbstractTopic - Disabling publish throttling for persistent://pulsar/ns1/producer_exception-1613103652346
   04:21:19.320 [ForkJoinPool.commonPool-worker-0] INFO  org.apache.pulsar.broker.service.persistent.PersistentTopic - [persistent://pulsar/ns1/producer_exception-1613103652346] Policies updated successfully
   04:21:19.320 [ForkJoinPool.commonPool-worker-0] WARN  org.apache.pulsar.broker.service.BrokerService - Topic level policies are not enabled. Please refer to systemTopicEnabled and topicLevelPoliciesEnabled on broker.conf
   04:21:19.320 [ForkJoinPool.commonPool-worker-0] WARN  org.apache.pulsar.broker.service.ServerCnx - Role null is not authorized to perform operation LOOKUP on topic persistent://pulsar/ns1/producer_exception-1613103652346
   04:21:19.320 [ForkJoinPool.commonPool-worker-0] WARN  org.apache.pulsar.broker.service.ServerCnx - [/127.0.0.1:47960] Proxy Client is not authorized to Lookup with role null on topic persistent://pulsar/ns1/producer_exception-1613103652346
   04:21:19.320 [ForkJoinPool.commonPool-worker-0] INFO  org.apache.pulsar.broker.service.BrokerService - /admin/policies/pulsar/ns1 updating with Policies{auth_policies=org.apache.pulsar.common.policies.data.AuthPolicies@745f, replication_clusters=[r2, r1], bundles=BundlesData{numBundles=1, boundaries=[0x00000000, 0xffffffff]}, backlog_quota_map={}, persistence=null, deduplicationEnabled=null, autoTopicCreationOverride=null, autoSubscriptionCreationOverride=null, clusterDispatchRate={}, topicDispatchRate={}, subscriptionDispatchRate={}, replicatorDispatchRate={}, clusterSubscribeRate={}, publishMaxMessageRate={}, latency_stats_sample_rate={}, message_ttl_in_seconds=null, subscription_expiration_time_minutes=0, retention_policies=null, deleted=false, encryption_required=false, delayed_delivery_policies=null, inactive_topic_policies=null, subscription_auth_mode=None, max_producers_per_topic=null, max_consumers_per_topic=null, max_consumers_per_subscription=null, max_unacked_messages_p
 er_consumer=-1, max_unacked_messages_per_subscription=-1, compaction_threshold=0, offload_threshold=-1, offload_deletion_lag_ms=null, schema_auto_update_compatibility_strategy=Full, schema_validation_enforced=false, schema_compatibility_Strategy=UNDEFINED, is_allow_auto_update_Schema=true, offload_policies=null}
   04:21:19.321 [ForkJoinPool.commonPool-worker-0] INFO  org.apache.pulsar.broker.service.AbstractTopic - Disabling publish throttling for persistent://pulsar/ns1/producer_request_hold-1613103670515
   04:21:19.321 [pulsar-io-139-1] WARN  org.apache.pulsar.client.impl.BinaryProtoLookupService - [persistent://pulsar/ns1/producer_exception-1613103652346] failed to send lookup request : Proxy Client is not authorized to Lookup
   04:21:19.321 [pulsar-io-139-1] WARN  org.apache.pulsar.client.impl.ConnectionHandler - [persistent://pulsar/ns1/producer_exception-1613103652346] [pulsar.repl.r2] Error connecting to broker: org.apache.pulsar.client.api.PulsarClientException$AuthorizationException: Proxy Client is not authorized to Lookup
   04:21:19.321 [pulsar-io-139-1] WARN  org.apache.pulsar.client.impl.ConnectionHandler - [persistent://pulsar/ns1/producer_exception-1613103652346] [pulsar.repl.r2] Could not get connection to broker: org.apache.pulsar.client.api.PulsarClientException$AuthorizationException: Proxy Client is not authorized to Lookup -- Will try again in 0.1 s
   04:21:19.321 [ForkJoinPool.commonPool-worker-0] INFO  org.apache.pulsar.broker.service.persistent.PersistentTopic - [persistent://pulsar/ns1/producer_request_hold-1613103670515] Policies updated successfully
   04:21:19.321 [ForkJoinPool.commonPool-worker-0] WARN  org.apache.pulsar.broker.service.BrokerService - Topic level policies are not enabled. Please refer to systemTopicEnabled and topicLevelPoliciesEnabled on broker.conf
   04:21:19.321 [ForkJoinPool.commonPool-worker-0] INFO  org.apache.pulsar.broker.service.AbstractTopic - Disabling publish throttling for persistent://pulsar/ns1/producer_exception-1613103652346
   04:21:19.321 [ForkJoinPool.commonPool-worker-0] INFO  org.apache.pulsar.broker.service.persistent.PersistentTopic - [persistent://pulsar/ns1/producer_exception-1613103652346] Policies updated successfully
   04:21:19.321 [ForkJoinPool.commonPool-worker-0] WARN  org.apache.pulsar.broker.service.BrokerService - Topic level policies are not enabled. Please refer to systemTopicEnabled and topicLevelPoliciesEnabled on broker.conf
   04:21:19.326 [pulsar-web-77-7] INFO  org.apache.pulsar.broker.admin.impl.NamespacesBase - [null] Successfully updated the on namespace /admin/policies/pulsar/global/ns
   04:21:19.326 [pulsar-web-77-7] INFO  org.apache.pulsar.broker.admin.impl.NamespacesBase - [null] Successfully removed backlog namespace=pulsar/global/ns, quota=destination_storage
   04:21:19.327 [pulsar-web-77-7] INFO  org.eclipse.jetty.server.RequestLog - 127.0.0.1 - - [12/Feb/2021:04:21:19 +0000] "DELETE /admin/namespaces/pulsar/global/ns/backlogQuota?backlogQuotaType=destination_storage HTTP/1.1" 204 0 "-" "Pulsar-Java-v2.8.0-SNAPSHOT" 9
   ```


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org