You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "hero6-coder (via GitHub)" <gi...@apache.org> on 2024/08/18 04:23:11 UTC
Re: [I] python client does not retry when get topic partition metadata to create producer fails [pulsar-client-python]
hero6-coder commented on issue #113:
URL: https://github.com/apache/pulsar-client-python/issues/113#issuecomment-2295105479
I have same issue with Java pulsar client & pulsar 3.2.
- Code:
` PulsarClient client = PulsarClient.builder()
.serviceUrl("pulsar://127.0.0.1:6650")
.authentication(AuthenticationFactory.token(token))
.build();`
- Log on client side:
10:57:51.287 [pulsar-client-io-1-3] INFO org.apache.pulsar.client.impl.ConnectionPool -- [[id: 0x65a00d98, L:/127.0.0.1:64047 - R:/127.0.0.1:6650]] Connected to server
10:57:52.111 [pulsar-client-io-1-3] INFO org.apache.pulsar.client.impl.ProducerStatsRecorderImpl -- Starting Pulsar producer perf with config: {"topicName":"test.topic","producerName":null,"sendTimeoutMs":30000,"blockIfQueueFull":false,"maxPendingMessages":0,"maxPendingMessagesAcrossPartitions":0,"messageRoutingMode":"RoundRobinPartition","hashingScheme":"JavaStringHash","cryptoFailureAction":"FAIL","batchingMaxPublishDelayMicros":1000,"batchingPartitionSwitchFrequencyByPublishDelay":10,"batchingMaxMessages":1000,"batchingMaxBytes":131072,"batchingEnabled":true,"chunkingEnabled":false,"chunkMaxMessageSize":-1,"encryptionKeys":[],"compressionType":"NONE","initialSequenceId":null,"autoUpdatePartitions":true,"autoUpdatePartitionsIntervalSeconds":60,"multiSchema":true,"accessMode":"Shared","lazyStartPartitionedProducers":false,"properties":{},"initialSubscriptionName":null}
10:57:52.129 [pulsar-client-io-1-3] INFO org.apache.pulsar.client.impl.ProducerStatsRecorderImpl -- Pulsar client config: {"serviceUrl":"pulsar://127.0.0.1:6650","authPluginClassName":null,"authParams":null,"authParamMap":null,"operationTimeoutMs":30000,"lookupTimeoutMs":30000,"statsIntervalSeconds":60,"numIoThreads":8,"numListenerThreads":8,"connectionsPerBroker":1,"connectionMaxIdleSeconds":180,"useTcpNoDelay":true,"useTls":false,"tlsKeyFilePath":null,"tlsCertificateFilePath":null,"tlsTrustCertsFilePath":null,"tlsAllowInsecureConnection":false,"tlsHostnameVerificationEnable":false,"concurrentLookupRequest":5000,"maxLookupRequest":50000,"maxLookupRedirects":20,"maxNumberOfRejectedRequestPerConnection":50,"keepAliveIntervalSeconds":30,"connectionTimeoutMs":10000,"requestTimeoutMs":60000,"readTimeoutMs":60000,"autoCertRefreshSeconds":300,"initialBackoffIntervalNanos":100000000,"maxBackoffIntervalNanos":60000000000,"enableBusyWait":false,"listenerName":null,"useKeyStoreTls":false,"s
slProvider":null,"tlsKeyStoreType":"JKS","tlsKeyStorePath":null,"tlsKeyStorePassword":null,"tlsTrustStoreType":"JKS","tlsTrustStorePath":null,"tlsTrustStorePassword":null,"tlsCiphers":[],"tlsProtocols":[],"memoryLimitBytes":67108864,"proxyServiceUrl":null,"proxyProtocol":null,"enableTransaction":false,"dnsLookupBindAddress":null,"dnsLookupBindPort":0,"dnsServerAddresses":[],"socks5ProxyAddress":null,"socks5ProxyUsername":null,"socks5ProxyPassword":null,"description":null}
10:57:52.152 [pulsar-client-io-1-3] INFO org.apache.pulsar.client.impl.ProducerImpl -- [test.topic] [null] Creating producer on cnx [id: 0x65a00d98, L:/127.0.0.1:64047 - R:/127.0.0.1:6650]
10:57:52.210 [pulsar-client-io-1-3] WARN org.apache.pulsar.client.impl.ClientCnx -- [id: 0x65a00d98, L:/127.0.0.1:64047 - R:/127.0.0.1:6650] Received error from server: Topic creation encountered an exception by initialize topic policies service. topic_name=persistent://public/default/test.topic error_message=Failed to authenticate
10:57:52.214 [pulsar-client-io-1-3] ERROR org.apache.pulsar.client.impl.ProducerImpl -- [test.topic] [null] Failed to create producer: {"errorMsg":"Topic creation encountered an exception by initialize topic policies service. topic_name=persistent://public/default/test.topic error_message=Failed to authenticate","reqId":3148985659011010964, "remote":"/127.0.0.1:6650", "local":"/127.0.0.1:64047"}
10:57:52.214 [pulsar-client-io-1-3] WARN org.apache.pulsar.client.impl.ConnectionHandler -- [test.topic] [null] Error connecting to broker: org.apache.pulsar.client.api.PulsarClientException$LookupException: {"errorMsg":"Topic creation encountered an exception by initialize topic policies service. topic_name=persistent://public/default/test.topic error_message=Failed to authenticate","reqId":3148985659011010964, "remote":"/127.0.0.1:6650", "local":"/127.0.0.1:64047"}
10:57:52.214 [pulsar-client-io-1-3] WARN org.apache.pulsar.client.impl.ConnectionHandler -- [test.topic] [null] Could not get connection to broker: org.apache.pulsar.client.api.PulsarClientException$LookupException: {"errorMsg":"Topic creation encountered an exception by initialize topic policies service. topic_name=persistent://public/default/test.topic error_message=Failed to authenticate","reqId":3148985659011010964, "remote":"/127.0.0.1:6650", "local":"/127.0.0.1:64047"} -- Will try again in 0.1 s
10:57:52.316 [pulsar-timer-27-1] INFO org.apache.pulsar.client.impl.ConnectionHandler -- [test.topic] [null] Reconnecting after connection was closed
- Log on server side:
2024-08-18T03:22:22,122+0000 [pulsar-io-3-13] DEBUG org.apache.pulsar.broker.service.ServerCnx - [/172.24.0.1:37336] connect state change to : [Connected]
2024-08-18T03:22:22,122+0000 [pulsar-io-3-13] INFO org.apache.pulsar.broker.service.ServerCnx - [/172.24.0.1:37336] connected with role=OWNER using authMethod=token, clientVersion=Pulsar-Java-v3.2.3, clientProtocolVersion=21, proxyVersion=null
2024-08-18T03:22:22,140+0000 [pulsar-io-3-13] DEBUG org.apache.pulsar.common.protocol.PulsarDecoder - [/172.24.0.1:37336] Received cmd PARTITIONED_METADATA
2024-08-18T03:22:22,140+0000 [pulsar-io-3-13] DEBUG org.apache.pulsar.broker.service.ServerCnx - [persistent://public/default/test.topic] Received PartitionMetadataLookup from /172.24.0.1:37336 for 1495031580381226501
2024-08-18T03:22:22,143+0000 [main-SendThread(zookeeper:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x1000073bd71000a, packet:: clientPath:null serverPath:null finished:false header:: 961,22 replyHeader:: 961,987,0 request:: org.apache.zookeeper.MultiOperationRecord@8adefa96 response:: org.apache.zookeeper.MultiResponse@dd603777
2024-08-18T03:22:22,148+0000 [main-SendThread(zookeeper:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x1000073bd71000a, packet:: clientPath:null serverPath:null finished:false header:: 962,22 replyHeader:: 962,987,0 request:: org.apache.zookeeper.MultiOperationRecord@8074b73c response:: org.apache.zookeeper.MultiResponse@ffffff9b
2024-08-18T03:22:22,149+0000 [main-SendThread(zookeeper:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x1000073bd71000a, packet:: clientPath:/managed-ledgers/public/default/persistent/test.topic serverPath:/managed-ledgers/public/default/persistent/test.topic finished:false header:: 963,3 replyHeader:: 963,987,-101 request:: '/managed-ledgers/public/default/persistent/test.topic,F response::
2024-08-18T03:22:22,150+0000 [pulsar-2-8] DEBUG org.apache.pulsar.broker.service.BrokerService - No autoTopicCreateOverride policy found for persistent://public/default/test.topic
2024-08-18T03:22:22,150+0000 [pulsar-2-8] DEBUG org.apache.pulsar.broker.admin.impl.PersistentTopicsBase - Total number of partitions for topic persistent://public/default/test.topic is 0
2024-08-18T03:22:22,288+0000 [pulsar-io-3-13] DEBUG org.apache.pulsar.common.protocol.PulsarDecoder - [/172.24.0.1:37336] Received cmd LOOKUP
2024-08-18T03:22:22,288+0000 [pulsar-io-3-13] DEBUG org.apache.pulsar.broker.service.ServerCnx - [persistent://public/default/test.topic] Received Lookup from /172.24.0.1:37336 for 1495031580381226502 requesting listener (none)
2024-08-18T03:22:22,292+0000 [main-SendThread(zookeeper:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply session id: 0x1000073bd71000a, packet:: clientPath:null serverPath:null finished:false header:: 964,22 replyHeader:: 964,987,0 request:: org.apache.zookeeper.MultiOperationRecord@609b619f response:: org.apache.zookeeper.MultiResponse@ac99062a
2024-08-18T03:22:22,294+0000 [metadata-store-9-1] INFO org.apache.pulsar.broker.loadbalance.extensions.manager.RedirectManager - No need to redirect, current load manager class name: org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerImpl
2024-08-18T03:22:22,294+0000 [metadata-store-9-1] DEBUG org.apache.pulsar.broker.namespace.NamespaceService - findBrokerServiceUrl: public/default/0xc0000000_0xd0000000 - options: LookupOptions(authoritative=false, readOnly=false, loadTopicsInBundle=true, requestHttps=false, advertisedListenerName=null)
2024-08-18T03:22:22,294+0000 [metadata-store-9-1] DEBUG org.apache.pulsar.broker.namespace.NamespaceService - Namespace bundle public/default/0xc0000000_0xd0000000 already owned by Optional[NamespaceEphemeralData(nativeUrl=pulsar://127.0.0.1:6650, nativeUrlTls=null, httpUrl=http://broker:8080, httpUrlTls=null, disabled=false, advertisedListeners={external=AdvertisedListener(brokerServiceUrl=pulsar://127.0.0.1:6650, brokerServiceUrlTls=null, brokerHttpUrl=null, brokerHttpsUrl=null)})]
2024-08-18T03:22:22,294+0000 [metadata-store-9-1] DEBUG org.apache.pulsar.broker.lookup.TopicLookupBase - [persistent://public/default/test.topic] Lookup result Optional[LookupResult [type=BrokerUrl, lookupData=LookupData{brokerUrl=pulsar://127.0.0.1:6650, brokerUrlTls=null, httpUrl=http://broker:8080, httpUrlTls=null}]]
2024-08-18T03:22:22,307+0000 [pulsar-io-3-13] DEBUG org.apache.pulsar.common.protocol.PulsarDecoder - [/172.24.0.1:37336] Received cmd PRODUCER
2024-08-18T03:22:22,307+0000 [pulsar-io-3-13] DEBUG org.apache.pulsar.broker.service.ServerCnx - [/172.24.0.1:37336] Client is authorized to Produce with role OWNER
2024-08-18T03:22:22,307+0000 [pulsar-io-3-13] DEBUG org.apache.pulsar.broker.service.ServerCnx - [/172.24.0.1:37336][persistent://public/default/test.topic] Creating producer. producerId=0, producerName=dev-es-pulsar-cluster-19-9, schema is present
2024-08-18T03:22:22,307+0000 [pulsar-io-3-13] DEBUG org.apache.pulsar.broker.service.BrokerService - No autoTopicCreateOverride policy found for persistent://public/default/test.topic
2024-08-18T03:22:22,307+0000 [pulsar-io-3-13] INFO org.apache.pulsar.broker.systopic.NamespaceEventsSystemTopicFactory - Create topic policies system topic client persistent://public/default/__change_events
2024-08-18T03:22:22,343+0000 [pulsar-io-3-13] DEBUG org.apache.pulsar.client.impl.ConnectionPool - Connection for 127.0.0.1/<unresolved>:6650 not found in cache
2024-08-18T03:22:22,345+0000 [pulsar-io-3-14] INFO org.apache.pulsar.client.impl.ConnectionPool - [[id: 0x78f03fc2, L:/127.0.0.1:37338 - R:/127.0.0.1:6650]] Connected to server
2024-08-18T03:22:22,345+0000 [pulsar-io-3-14] DEBUG org.apache.pulsar.common.protocol.PulsarHandler - [[id: 0x78f03fc2, L:/127.0.0.1:37338 - R:/127.0.0.1:6650]] Scheduling keep-alive task every 30 s
2024-08-18T03:22:22,346+0000 [pulsar-io-3-14] DEBUG org.apache.pulsar.client.impl.ClientCnx - [id: 0x78f03fc2, L:/127.0.0.1:37338 - R:/127.0.0.1:6650] Connected to broker
2024-08-18T03:22:22,346+0000 [pulsar-io-3-15] DEBUG org.apache.pulsar.common.protocol.PulsarHandler - [[id: 0x9b9ead57, L:/127.0.0.1:6650 - R:/127.0.0.1:37338]] Scheduling keep-alive task every 30 s
2024-08-18T03:22:22,347+0000 [pulsar-io-3-15] DEBUG org.apache.pulsar.broker.service.ServerCnx - New connection from /127.0.0.1:37338
2024-08-18T03:22:22,348+0000 [pulsar-io-3-14] DEBUG org.apache.pulsar.client.impl.ClientCnx - Complete: true
2024-08-18T03:22:22,349+0000 [pulsar-io-3-15] DEBUG org.apache.pulsar.common.protocol.PulsarDecoder - [/127.0.0.1:37338] Received cmd CONNECT
2024-08-18T03:22:22,349+0000 [pulsar-io-3-15] DEBUG org.apache.pulsar.broker.service.ServerCnx - Received CONNECT from /127.0.0.1:37338, auth enabled: true: has original principal = false, original principal = null
2024-08-18T03:22:22,350+0000 [pulsar-io-3-15] INFO org.apache.pulsar.broker.service.ServerCnx - [/127.0.0.1:37338] Failed to authenticate: operation=connect, principal=null, reason=No anonymous role, and no authentication provider configured
2024-08-18T03:22:22,352+0000 [pulsar-io-3-15] INFO org.apache.pulsar.broker.service.ServerCnx - Closed connection from /127.0.0.1:37338
2024-08-18T03:22:22,352+0000 [pulsar-io-3-14] DEBUG org.apache.pulsar.common.protocol.PulsarDecoder - [/127.0.0.1:6650] Received cmd ERROR
2024-08-18T03:22:22,353+0000 [pulsar-io-3-14] WARN org.apache.pulsar.client.impl.ClientCnx - [id: 0x78f03fc2, L:/127.0.0.1:37338 - R:/127.0.0.1:6650] Received error from server: Failed to authenticate
2024-08-18T03:22:22,353+0000 [pulsar-io-3-14] WARN org.apache.pulsar.client.impl.ConnectionPool - [[id: 0x78f03fc2, L:/127.0.0.1:37338 - R:/127.0.0.1:6650]] Connection handshake failed: org.apache.pulsar.client.api.PulsarClientException$AuthenticationException: Failed to authenticate
2024-08-18T03:22:22,353+0000 [pulsar-io-3-14] WARN org.apache.pulsar.client.impl.PulsarClientImpl - [persistent://public/default/__change_events] Failed to get partitioned topic metadata
java.util.concurrent.CompletionException: org.apache.pulsar.client.api.PulsarClientException$AuthenticationException: Failed to authenticate
at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:332) ~[?:?]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org