You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2020/11/17 12:18:13 UTC
[GitHub] [pulsar] jiangmincong opened a new issue #8593: Java client connection lookup timed out, 7 lookup request timedout after ms 30000
jiangmincong opened a new issue #8593:
URL: https://github.com/apache/pulsar/issues/8593
#### Expected behavior
2020-11-06 14:13:16,906 | WARN | [pulsar-client-io-1-1] | Failed subscription for createPartitionedConsumer: persistent://tenant/namespace/topic_test 30, e:{} | org.apache.pulsar.client.impl.MultiTopicsConsumerImpl (MultiTopicsConsumerImpl.java:775)
java.util.concurrent.CompletionException: org.apache.pulsar.client.api.PulsarClientException: java.util.concurrent.CompletionException: org.apache.pulsar.client.api.PulsarClientException$TimeoutException: 7 lookup request timedout after ms 30000
at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292)
at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308)
at java.util.concurrent.CompletableFuture.biRelay(CompletableFuture.java:1284)
at java.util.concurrent.CompletableFuture$BiRelay.tryFire(CompletableFuture.java:1270)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977)
at org.apache.pulsar.client.impl.ConsumerImpl.connectionFailed(ConsumerImpl.java:710)
at org.apache.pulsar.client.impl.ConnectionHandler.handleConnectionError(ConnectionHandler.java:78)
at java.util.concurrent.CompletableFuture.uniExceptionally(CompletableFuture.java:870)
at java.util.concurrent.CompletableFuture$UniExceptionally.tryFire(CompletableFuture.java:852)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977)
at org.apache.pulsar.client.impl.BinaryProtoLookupService.lambda$null$3(BinaryProtoLookupService.java:147)
at java.util.concurrent.CompletableFuture.uniExceptionally(CompletableFuture.java:870)
at java.util.concurrent.CompletableFuture$UniExceptionally.tryFire(CompletableFuture.java:852)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977)
at org.apache.pulsar.client.impl.ClientCnx.lambda$addPendingLookupRequests$12(ClientCnx.java:537)
at io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98)
at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:150)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:510)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:518)
at io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1050)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
#### Actual behavior
Tell us what happens instead
#### Steps to reproduce
How can we reproduce the issue
#### System configuration
**Pulsar 2.6.1
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] jiangmincong commented on issue #8593: Java client connection lookup timed out, 7 lookup request timedout after ms 30000
Posted by GitBox <gi...@apache.org>.
jiangmincong commented on issue #8593:
URL: https://github.com/apache/pulsar/issues/8593#issuecomment-732617172
Hi， I looked at the monitoring and it seemed that the zookeeper status was not very busy during the abnormal time.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] jiangmincong commented on issue #8593: Java client connection lookup timed out, 7 lookup request timedout after ms 30000
Posted by GitBox <gi...@apache.org>.
jiangmincong commented on issue #8593:
URL: https://github.com/apache/pulsar/issues/8593#issuecomment-729567298
Our test found that this situation is random. Lookup timeouts are rare when there are not many consumers, and there is a high probability when there are many consumers. The situation we tested is 30 topic partitions, continuous When multiple programs are started, the probability of the later startup program reporting lookup timeout is significantly higher than that of the previous startup, causing the consumer program to hang
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] jiazhai commented on issue #8593: Java client connection lookup timed out, 7 lookup request timedout after ms 30000
Posted by GitBox <gi...@apache.org>.
jiazhai commented on issue #8593:
URL: https://github.com/apache/pulsar/issues/8593#issuecomment-732520610
@jiangmincong This seems more related to the issue https://github.com/apache/pulsar/issues/8533. And @reswqa is helping on this issue.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] jiangmincong commented on issue #8593: Java client connection lookup timed out, 7 lookup request timedout after ms 30000
Posted by GitBox <gi...@apache.org>.
jiangmincong commented on issue #8593:
URL: https://github.com/apache/pulsar/issues/8593#issuecomment-809947984
This exception occurs when there are two different versions (2.5.0/2.6.1) of client consumption or production data at the same time. After replacing the topic, this exception does not appear. I will close this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] jiazhai commented on issue #8593: Java client connection lookup timed out, 7 lookup request timedout after ms 30000
Posted by GitBox <gi...@apache.org>.
jiazhai commented on issue #8593:
URL: https://github.com/apache/pulsar/issues/8593#issuecomment-732543374
@jiangmincong What is the zookeeper side stats when the lookup hang happens? Is Zookeeper in a busy status?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] jiangmincong commented on issue #8593: Java client connection lookup timed out, 7 lookup request timedout after ms 30000
Posted by GitBox <gi...@apache.org>.
jiangmincong commented on issue #8593:
URL: https://github.com/apache/pulsar/issues/8593#issuecomment-732624080
From the JVM monitoring, there seems to be no exception
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] jiangmincong closed issue #8593: Java client connection lookup timed out, 7 lookup request timedout after ms 30000
Posted by GitBox <gi...@apache.org>.
jiangmincong closed issue #8593:
URL: https://github.com/apache/pulsar/issues/8593
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org