You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/12/06 04:49:58 UTC
[GitHub] [pulsar] wuxinle0412 opened a new issue #13149: pulsar-2.8.1中ledger长时间处于closing状态
wuxinle0412 opened a new issue #13149:
URL: https://github.com/apache/pulsar/issues/13149
1.集群版本与信息
pulsar服务端版本:2.8.1
broker: 3台机器，堆10G，直接内存20GB
bookie: 3台机器，堆10G，直接内存20GB
2.总体现象
使用了kop，在pulsar中创建了8个partition，设置的retention为6小时，message ttl为6小时，通过flume收集日志到到这个kop的集群，下游是flink程序进行消费，并行度为8，从pulsar-manager查看，大量ledger处于closing状态，导致无法触发TTL机制。
3.broker端日志
4.疑问
什么情况下，会导致ledger长时间处于closing状态。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] wuxinle0412 commented on issue #13149: pulsar-2.8.1中ledger长时间处于closing状态
Posted by GitBox <gi...@apache.org>.
wuxinle0412 commented on issue #13149:
URL: https://github.com/apache/pulsar/issues/13149#issuecomment-986444196
日志片段:
13:07:47.019 [pulsar-web-40-16] INFO org.eclipse.jetty.server.RequestLog - 10.1.2.84 - - [06/Dec/2021:13:07:47 +0800] "GET /admin/v2/brokers/prod-pulsar-cluster HTTP/1.1" 200 55 "-" "Pulsar-Java-v2.5.2" 1
13:07:47.019 [pulsar-web-40-15] INFO org.eclipse.jetty.server.RequestLog - 10.1.2.84 - - [06/Dec/2021:13:07:47 +0800] "GET /admin/v2/broker-stats/topics HTTP/1.1" 200 32349 "-" "Pulsar-Java-v2.5.2" 0
13:07:48.623 [pulsar-web-40-11] INFO org.eclipse.jetty.server.RequestLog - 10.1.18.17 - - [06/Dec/2021:13:07:48 +0800] "GET /admin/v2/persistent/gaodeng/ucampus/ucontent/partitions HTTP/1.1" 200 16 "-" "Pulsar-Java-v2.8.1" 0
13:07:49.426 [pulsar-web-40-3] INFO org.eclipse.jetty.server.RequestLog - 10.1.18.17 - - [06/Dec/2021:13:07:49 +0800] "GET /admin/v2/persistent/gaodeng/ucampus/ucontent/partitions HTTP/1.1" 200 16 "-" "Pulsar-Java-v2.8.1" 0
13:08:09.168 [pulsar-load-manager-1-1] INFO org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerImpl - Writing local data to metadata store because maximum change 15.651884216109032% exceeded threshold 10%;time since last report written is 85.0 seconds
13:08:17.714 [pulsar-web-40-9] INFO org.eclipse.jetty.server.RequestLog - 10.1.2.84 - - [06/Dec/2021:13:08:17 +0800] "GET /admin/v2/clusters HTTP/1.1" 200 23 "-" "Pulsar-Java-v2.5.2" 0
13:08:17.717 [pulsar-web-40-5] INFO org.eclipse.jetty.server.RequestLog - 10.1.2.84 - - [06/Dec/2021:13:08:17 +0800] "GET /admin/v2/clusters/prod-pulsar-cluster/failureDomains HTTP/1.1" 200 2 "-" "Pulsar-Java-v2.5.2" 2
13:08:17.717 [pulsar-web-40-8] INFO org.eclipse.jetty.server.RequestLog - 10.1.2.84 - - [06/Dec/2021:13:08:17 +0800] "GET /admin/v2/brokers/prod-pulsar-cluster HTTP/1.1" 200 55 "-" "Pulsar-Java-v2.5.2" 0
13:08:17.717 [pulsar-web-40-12] INFO org.eclipse.jetty.server.RequestLog - 10.1.2.84 - - [06/Dec/2021:13:08:17 +0800] "GET /admin/v2/clusters/prod-pulsar-cluster HTTP/1.1" 200 296 "-" "Pulsar-Java-v2.5.2" 0
13:08:17.718 [pulsar-io-4-4] INFO org.apache.pulsar.client.impl.ProducerStatsRecorderImpl - Starting Pulsar producer perf with config: {
"topicName" : "persistent://pulsar/prod-pulsar-cluster/10.1.18.17:8080/healthcheck",
"producerName" : null,
"sendTimeoutMs" : 30000,
"blockIfQueueFull" : false,
"maxPendingMessages" : 1000,
"maxPendingMessagesAcrossPartitions" : 50000,
"messageRoutingMode" : "RoundRobinPartition",
"hashingScheme" : "JavaStringHash",
"cryptoFailureAction" : "FAIL",
"batchingMaxPublishDelayMicros" : 1000,
"batchingPartitionSwitchFrequencyByPublishDelay" : 10,
"batchingMaxMessages" : 1000,
"batchingMaxBytes" : 131072,
"batchingEnabled" : true,
"chunkingEnabled" : false,
"compressionType" : "NONE",
"initialSequenceId" : null,
"autoUpdatePartitions" : true,
"autoUpdatePartitionsIntervalSeconds" : 60,
"multiSchema" : true,
"accessMode" : "Shared",
"properties" : { }
}
13:08:17.718 [pulsar-io-4-4] INFO org.apache.pulsar.client.impl.ProducerStatsRecorderImpl - Pulsar client config: {
"serviceUrl" : "pulsar://10.1.18.17:6650",
"authPluginClassName" : "org.apache.pulsar.client.impl.auth.AuthenticationDisabled",
"authParams" : "*****",
"authParamMap" : null,
"operationTimeoutMs" : 30000,
"statsIntervalSeconds" : 60,
"numIoThreads" : 1,
"numListenerThreads" : 1,
"connectionsPerBroker" : 1,
"useTcpNoDelay" : true,
"useTls" : false,
"tlsTrustCertsFilePath" : null,
"tlsAllowInsecureConnection" : false,
"tlsHostnameVerificationEnable" : false,
"concurrentLookupRequest" : 5000,
"maxLookupRequest" : 50000,
"maxLookupRedirects" : 20,
"maxNumberOfRejectedRequestPerConnection" : 50,
"keepAliveIntervalSeconds" : 30,
"connectionTimeoutMs" : 10000,
"requestTimeoutMs" : 60000,
"initialBackoffIntervalNanos" : 100000000,
"maxBackoffIntervalNanos" : 60000000000,
"enableBusyWait" : false,
"listenerName" : null,
"useKeyStoreTls" : false,
"sslProvider" : null,
"tlsTrustStoreType" : "JKS",
"tlsTrustStorePath" : null,
"tlsTrustStorePassword" : null,
"tlsCiphers" : [ ],
"tlsProtocols" : [ ],
"memoryLimitBytes" : 0,
"proxyServiceUrl" : null,
"proxyProtocol" : null,
"enableTransaction" : false
}
13:08:17.719 [pulsar-io-4-4] INFO org.apache.pulsar.client.impl.ConsumerStatsRecorderImpl - Starting Pulsar consumer status recorder with config: {
"topicNames" : [ "persistent://pulsar/prod-pulsar-cluster/10.1.18.17:8080/healthcheck" ],
"topicsPattern" : null,
"subscriptionName" : "reader-36059c0eb7",
"subscriptionType" : "Exclusive",
"subscriptionMode" : "NonDurable",
"receiverQueueSize" : 1000,
"acknowledgementsGroupTimeMicros" : 100000,
"negativeAckRedeliveryDelayMicros" : 60000000,
"maxTotalReceiverQueueSizeAcrossPartitions" : 50000,
"consumerName" : null,
"ackTimeoutMillis" : 0,
"tickDurationMillis" : 1000,
"priorityLevel" : 0,
"maxPendingChunkedMessage" : 10,
"autoAckOldestChunkedMessageOnQueueFull" : false,
"expireTimeOfIncompleteChunkedMessageMillis" : 60000,
"cryptoFailureAction" : "FAIL",
"properties" : { },
"readCompacted" : false,
"subscriptionInitialPosition" : "Latest",
"patternAutoDiscoveryPeriod" : 60,
"regexSubscriptionMode" : "PersistentOnly",
"deadLetterPolicy" : null,
"retryEnable" : false,
"autoUpdatePartitions" : true,
"autoUpdatePartitionsIntervalSeconds" : 60,
"replicateSubscriptionState" : false,
"resetIncludeHead" : false,
"keySharedPolicy" : null,
"batchIndexAckEnabled" : false,
"ackReceiptEnabled" : false,
"poolMessages" : false,
"maxPendingChuckedMessage" : 10
}
13:08:17.719 [pulsar-io-4-4] INFO org.apache.pulsar.client.impl.ConsumerStatsRecorderImpl - Pulsar client config: {
"serviceUrl" : "pulsar://10.1.18.17:6650",
"authPluginClassName" : "org.apache.pulsar.client.impl.auth.AuthenticationDisabled",
"authParams" : "*****",
"authParamMap" : null,
"operationTimeoutMs" : 30000,
"statsIntervalSeconds" : 60,
"numIoThreads" : 1,
"numListenerThreads" : 1,
"connectionsPerBroker" : 1,
"useTcpNoDelay" : true,
"useTls" : false,
"tlsTrustCertsFilePath" : null,
"tlsAllowInsecureConnection" : false,
"tlsHostnameVerificationEnable" : false,
"concurrentLookupRequest" : 5000,
"maxLookupRequest" : 50000,
"maxLookupRedirects" : 20,
"maxNumberOfRejectedRequestPerConnection" : 50,
"keepAliveIntervalSeconds" : 30,
"connectionTimeoutMs" : 10000,
"requestTimeoutMs" : 60000,
"initialBackoffIntervalNanos" : 100000000,
"maxBackoffIntervalNanos" : 60000000000,
"enableBusyWait" : false,
"listenerName" : null,
"useKeyStoreTls" : false,
"sslProvider" : null,
"tlsTrustStoreType" : "JKS",
"tlsTrustStorePath" : null,
"tlsTrustStorePassword" : null,
"tlsCiphers" : [ ],
"tlsProtocols" : [ ],
"memoryLimitBytes" : 0,
"proxyServiceUrl" : null,
"proxyProtocol" : null,
"enableTransaction" : false
}
13:08:17.719 [pulsar-io-4-4] INFO org.apache.pulsar.client.impl.ProducerImpl - [persistent://pulsar/prod-pulsar-cluster/10.1.18.17:8080/healthcheck] [null] Creating producer on cnx [id: 0xd793bcad, L:/10.1.18.17:31106 - R:10.1.18.17/10.1.18.17:6650]
13:08:17.719 [pulsar-io-4-4] INFO org.apache.pulsar.client.impl.ConsumerImpl - [persistent://pulsar/prod-pulsar-cluster/10.1.18.17:8080/healthcheck][reader-36059c0eb7] Subscribing to topic on cnx [id: 0xd793bcad, L:/10.1.18.17:31106 - R:10.1.18.17/10.1.18.17:6650], consumerId 11173
13:08:17.719 [pulsar-io-4-5] INFO org.apache.pulsar.broker.service.ServerCnx - [/10.1.18.17:31106][persistent://pulsar/prod-pulsar-cluster/10.1.18.17:8080/healthcheck] Creating producer. producerId=11169
13:08:17.721 [pulsar-io-4-5] INFO org.apache.pulsar.broker.service.ServerCnx - [/10.1.18.17:31106] Subscribing on topic persistent://pulsar/prod-pulsar-cluster/10.1.18.17:8080/healthcheck / reader-36059c0eb7
13:08:17.724 [BookKeeperClientWorker-OrderedExecutor-1-0] INFO org.apache.pulsar.broker.service.ServerCnx - [/10.1.18.17:31106] Created new producer: Producer{topic=PersistentTopic{topic=persistent://pulsar/prod-pulsar-cluster/10.1.18.17:8080/healthcheck}, client=/10.1.18.17:31106, producerName=prod-pulsar-cluster-14-12585, producerId=11169}
13:08:17.724 [pulsar-io-4-4] INFO org.apache.pulsar.client.impl.ProducerImpl - [persistent://pulsar/prod-pulsar-cluster/10.1.18.17:8080/healthcheck] [prod-pulsar-cluster-14-12585] Created producer on cnx [id: 0xd793bcad, L:/10.1.18.17:31106 - R:10.1.18.17/10.1.18.17:6650]
13:08:17.725 [ForkJoinPool.commonPool-worker-7] INFO org.apache.pulsar.broker.service.persistent.PersistentTopic - [persistent://pulsar/prod-pulsar-cluster/10.1.18.17:8080/healthcheck][reader-36059c0eb7] Creating non-durable subscription at msg id 9223372036854775807:9223372036854775807:-1:-1
13:08:17.725 [ForkJoinPool.commonPool-worker-7] INFO org.apache.bookkeeper.mledger.impl.NonDurableCursorImpl - [pulsar/prod-pulsar-cluster/10.1.18.17:8080/persistent/healthcheck] Created non-durable cursor read-position=94070:312 mark-delete-position=94070:311
13:08:17.725 [ForkJoinPool.commonPool-worker-7] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [pulsar/prod-pulsar-cluster/10.1.18.17:8080/persistent/healthcheck] Opened new cursor: NonDurableCursorImpl{ledger=pulsar/prod-pulsar-cluster/10.1.18.17:8080/persistent/healthcheck, ackPos=94070:311, readPos=94070:312}
13:08:17.725 [ForkJoinPool.commonPool-worker-7] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [pulsar/prod-pulsar-cluster/10.1.18.17:8080/persistent/healthcheck-reader-36059c0eb7] Rewind from 94070:312 to 94070:312
13:08:17.725 [ForkJoinPool.commonPool-worker-7] INFO org.apache.pulsar.broker.service.persistent.PersistentTopic - [persistent://pulsar/prod-pulsar-cluster/10.1.18.17:8080/healthcheck] There are no replicated subscriptions on the topic
13:08:17.725 [ForkJoinPool.commonPool-worker-7] INFO org.apache.pulsar.broker.service.persistent.PersistentTopic - [persistent://pulsar/prod-pulsar-cluster/10.1.18.17:8080/healthcheck][reader-36059c0eb7] Created new subscription for 11173
13:08:17.725 [ForkJoinPool.commonPool-worker-7] INFO org.apache.pulsar.broker.service.ServerCnx - [/10.1.18.17:31106] Created subscription on topic persistent://pulsar/prod-pulsar-cluster/10.1.18.17:8080/healthcheck / reader-36059c0eb7
13:08:17.725 [pulsar-io-4-4] INFO org.apache.pulsar.client.impl.ConsumerImpl - [persistent://pulsar/prod-pulsar-cluster/10.1.18.17:8080/healthcheck][reader-36059c0eb7] Subscribed to topic on 10.1.18.17/10.1.18.17:6650 -- consumer: 11173
13:08:17.737 [pulsar-io-4-5] INFO org.apache.pulsar.broker.service.ServerCnx - [PersistentTopic{topic=persistent://pulsar/prod-pulsar-cluster/10.1.18.17:8080/healthcheck}][prod-pulsar-cluster-14-12585] Closing producer on cnx /10.1.18.17:31106. producerId=11169
13:08:17.737 [pulsar-io-4-5] INFO org.apache.pulsar.broker.service.ServerCnx - [PersistentTopic{topic=persistent://pulsar/prod-pulsar-cluster/10.1.18.17:8080/healthcheck}][prod-pulsar-cluster-14-12585] Closed producer on cnx /10.1.18.17:31106. producerId=11169
13:08:17.737 [pulsar-client-internal-44-1] INFO org.eclipse.jetty.server.RequestLog - 10.1.2.84 - - [06/Dec/2021:13:08:17 +0800] "GET /admin/v2/brokers/health HTTP/1.1" 200 2 "-" "Pulsar-Java-v2.5.2" 19
13:08:17.737 [pulsar-io-4-4] INFO org.apache.pulsar.client.impl.ProducerImpl - [persistent://pulsar/prod-pulsar-cluster/10.1.18.17:8080/healthcheck] [prod-pulsar-cluster-14-12585] Closed Producer
13:08:17.737 [pulsar-io-4-5] INFO org.apache.pulsar.broker.service.ServerCnx - [/10.1.18.17:31106] Closing consumer: consumerId=11173
13:08:17.738 [pulsar-io-4-5] INFO org.apache.pulsar.broker.service.AbstractDispatcherSingleActiveConsumer - Removing consumer Consumer{subscription=PersistentSubscription{topic=persistent://pulsar/prod-pulsar-cluster/10.1.18.17:8080/healthcheck, name=reader-36059c0eb7}, consumerId=11173, consumerName=04b03, address=/10.1.18.17:31106}
13:08:17.739 [pulsar-io-4-5] INFO org.apache.pulsar.broker.service.persistent.PersistentSubscription - [persistent://pulsar/prod-pulsar-cluster/10.1.18.17:8080/healthcheck][reader-36059c0eb7] Successfully closed subscription [NonDurableCursorImpl{ledger=pulsar/prod-pulsar-cluster/10.1.18.17:8080/persistent/healthcheck, ackPos=94070:311, readPos=94070:313}]
13:08:17.739 [pulsar-io-4-5] INFO org.apache.pulsar.broker.service.persistent.PersistentSubscription - [persistent://pulsar/prod-pulsar-cluster/10.1.18.17:8080/healthcheck][reader-36059c0eb7] Successfully closed dispatcher for reader
13:08:17.739 [pulsar-io-4-5] INFO org.apache.pulsar.broker.service.ServerCnx - [/10.1.18.17:31106] Closed consumer, consumerId=11173
13:08:17.739 [pulsar-io-4-4] INFO org.apache.pulsar.client.impl.ConsumerImpl - [persistent://pulsar/prod-pulsar-cluster/10.1.18.17:8080/healthcheck] [reader-36059c0eb7] Closed consumer
13:08:17.739 [pulsar-web-40-13] INFO org.eclipse.jetty.server.RequestLog - 10.1.2.84 - - [06/Dec/2021:13:08:17 +0800] "GET /admin/v2/clusters/prod-pulsar-cluster/failureDomains HTTP/1.1" 200 2 "-" "Pulsar-Java-v2.5.2" 0
13:08:17.739 [pulsar-web-40-6] INFO org.eclipse.jetty.server.RequestLog - 10.1.2.84 - - [06/Dec/2021:13:08:17 +0800] "GET /admin/v2/brokers/prod-pulsar-cluster HTTP/1.1" 200 55 "-" "Pulsar-Java-v2.5.2" 0
13:08:17.740 [pulsar-web-40-1] INFO org.eclipse.jetty.server.RequestLog - 10.1.2.84 - - [06/Dec/2021:13:08:17 +0800] "GET /admin/v2/broker-stats/topics HTTP/1.1" 200 32357 "-" "Pulsar-Java-v2.5.2" 0
13:08:26.925 [pulsar-web-40-15] INFO org.eclipse.jetty.server.RequestLog - 10.1.2.84 - - [06/Dec/2021:13:08:26 +0800] "GET /admin/v2/clusters HTTP/1.1" 200 23 "-" "Pulsar-Java-v2.5.2" 0
13:08:26.926 [pulsar-web-40-11] INFO org.eclipse.jetty.server.RequestLog - 10.1.2.84 - - [06/Dec/2021:13:08:26 +0800] "GET /admin/v2/clusters/prod-pulsar-cluster HTTP/1.1" 200 296 "-" "Pulsar-Java-v2.5.2" 1
13:08:26.926 [pulsar-web-40-7] INFO org.eclipse.jetty.server.RequestLog - 10.1.2.84 - - [06/Dec/2021:13:08:26 +0800] "GET /admin/v2/clusters HTTP/1.1" 200 23 "-" "Pulsar-Java-v2.5.2" 0
13:08:26.927 [pulsar-web-40-2] INFO org.eclipse.jetty.server.RequestLog - 10.1.2.84 - - [06/Dec/2021:13:08:26 +0800] "GET /admin/v2/clusters/prod-pulsar-cluster HTTP/1.1" 200 296 "-" "Pulsar-Java-v2.5.2" 0
13:08:30.971 [BookKeeperClientWorker-OrderedExecutor-5-0] INFO org.apache.bookkeeper.mledger.impl.OpAddEntry - [gaodeng/ucampus/persistent/ucontent-partition-4] Closing ledger 95486 for being full
13:08:30.972 [main-EventThread] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [gaodeng/ucampus/persistent/ucontent-partition-4] Creating a new ledger
13:08:30.972 [main-EventThread] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [gaodeng/ucampus/persistent/ucontent-partition-4] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[103, 97, 111, 100, 101, 110, 103, 47, 117, 99, 97, 109, 112, 117, 115, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 117, 99, 111, 110, 116, 101, 110, 116, 45, 112, 97, 114, 116, 105, 116, 105, 111, 110, 45, 52], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds
13:08:30.972 [main-EventThread] WARN org.apache.bookkeeper.client.BookieWatcherImpl - New ensemble: [10.1.0.166:3181, 10.1.21.64:3181] is not adhering to Placement Policy. quarantinedBookies: []
13:08:30.975 [main-EventThread] INFO org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [10.1.0.166:3181, 10.1.21.64:3181] for ledger: 95622
13:08:30.975 [main-EventThread] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [gaodeng/ucampus/persistent/ucontent-partition-4] Created new ledger 95622
13:08:48.442 [pulsar-web-40-4] INFO org.eclipse.jetty.server.RequestLog - 10.1.2.84 - - [06/Dec/2021:13:08:48 +0800] "GET /admin/v2/clusters HTTP/1.1" 200 23 "-" "Pulsar-Java-v2.5.2" 0
13:08:48.443 [pulsar-web-40-9] INFO org.eclipse.jetty.server.RequestLog - 10.1.2.84 - - [06/Dec/2021:13:08:48 +0800] "GET /admin/v2/clusters/prod-pulsar-cluster/failureDomains HTTP/1.1" 200 2 "-" "Pulsar-Java-v2.5.2" 0
13:08:48.443 [pulsar-web-40-5] INFO org.eclipse.jetty.server.RequestLog - 10.1.2.84 - - [06/Dec/2021:13:08:48 +0800] "GET /admin/v2/brokers/prod-pulsar-cluster HTTP/1.1" 200 55 "-" "Pulsar-Java-v2.5.2" 0
13:08:48.444 [pulsar-web-40-8] INFO org.eclipse.jetty.server.RequestLog - 10.1.2.84 - - [06/Dec/2021:13:08:48 +0800] "GET /admin/v2/clusters/prod-pulsar-cluster HTTP/1.1" 200 296 "-" "Pulsar-Java-v2.5.2" 1
13:08:48.444 [pulsar-io-4-4] INFO org.apache.pulsar.client.impl.ProducerStatsRecorderImpl - Starting Pulsar producer perf with config: {
"topicName" : "persistent://pulsar/prod-pulsar-cluster/10.1.18.17:8080/healthcheck",
"producerName" : null,
"sendTimeoutMs" : 30000,
"blockIfQueueFull" : false,
"maxPendingMessages" : 1000,
"maxPendingMessagesAcrossPartitions" : 50000,
"messageRoutingMode" : "RoundRobinPartition",
"hashingScheme" : "JavaStringHash",
"cryptoFailureAction" : "FAIL",
"batchingMaxPublishDelayMicros" : 1000,
"batchingPartitionSwitchFrequencyByPublishDelay" : 10,
"batchingMaxMessages" : 1000,
"batchingMaxBytes" : 131072,
"batchingEnabled" : true,
"chunkingEnabled" : false,
"compressionType" : "NONE",
"initialSequenceId" : null,
"autoUpdatePartitions" : true,
"autoUpdatePartitionsIntervalSeconds" : 60,
"multiSchema" : true,
"accessMode" : "Shared",
"properties" : { }
}
13:08:48.445 [pulsar-io-4-4] INFO org.apache.pulsar.client.impl.ProducerStatsRecorderImpl - Pulsar client config: {
"serviceUrl" : "pulsar://10.1.18.17:6650",
"authPluginClassName" : "org.apache.pulsar.client.impl.auth.AuthenticationDisabled",
"authParams" : "*****",
"authParamMap" : null,
"operationTimeoutMs" : 30000,
"statsIntervalSeconds" : 60,
"numIoThreads" : 1,
"numListenerThreads" : 1,
"connectionsPerBroker" : 1,
"useTcpNoDelay" : true,
"useTls" : false,
"tlsTrustCertsFilePath" : null,
"tlsAllowInsecureConnection" : false,
"tlsHostnameVerificationEnable" : false,
"concurrentLookupRequest" : 5000,
"maxLookupRequest" : 50000,
"maxLookupRedirects" : 20,
"maxNumberOfRejectedRequestPerConnection" : 50,
"keepAliveIntervalSeconds" : 30,
"connectionTimeoutMs" : 10000,
"requestTimeoutMs" : 60000,
"initialBackoffIntervalNanos" : 100000000,
"maxBackoffIntervalNanos" : 60000000000,
"enableBusyWait" : false,
"listenerName" : null,
"useKeyStoreTls" : false,
"sslProvider" : null,
"tlsTrustStoreType" : "JKS",
"tlsTrustStorePath" : null,
"tlsTrustStorePassword" : null,
"tlsCiphers" : [ ],
"tlsProtocols" : [ ],
"memoryLimitBytes" : 0,
"proxyServiceUrl" : null,
"proxyProtocol" : null,
"enableTransaction" : false
}
13:08:48.445 [pulsar-io-4-4] INFO org.apache.pulsar.client.impl.ConsumerStatsRecorderImpl - Starting Pulsar consumer status recorder with config: {
"topicNames" : [ "persistent://pulsar/prod-pulsar-cluster/10.1.18.17:8080/healthcheck" ],
"topicsPattern" : null,
"subscriptionName" : "reader-5fa28ac47c",
"subscriptionType" : "Exclusive",
"subscriptionMode" : "NonDurable",
"receiverQueueSize" : 1000,
"acknowledgementsGroupTimeMicros" : 100000,
"negativeAckRedeliveryDelayMicros" : 60000000,
"maxTotalReceiverQueueSizeAcrossPartitions" : 50000,
"consumerName" : null,
"ackTimeoutMillis" : 0,
"tickDurationMillis" : 1000,
"priorityLevel" : 0,
"maxPendingChunkedMessage" : 10,
"autoAckOldestChunkedMessageOnQueueFull" : false,
"expireTimeOfIncompleteChunkedMessageMillis" : 60000,
"cryptoFailureAction" : "FAIL",
"properties" : { },
"readCompacted" : false,
"subscriptionInitialPosition" : "Latest",
"patternAutoDiscoveryPeriod" : 60,
"regexSubscriptionMode" : "PersistentOnly",
"deadLetterPolicy" : null,
"retryEnable" : false,
"autoUpdatePartitions" : true,
"autoUpdatePartitionsIntervalSeconds" : 60,
"replicateSubscriptionState" : false,
"resetIncludeHead" : false,
"keySharedPolicy" : null,
"batchIndexAckEnabled" : false,
"ackReceiptEnabled" : false,
"poolMessages" : false,
"maxPendingChuckedMessage" : 10
}
13:08:48.445 [pulsar-io-4-4] INFO org.apache.pulsar.client.impl.ConsumerStatsRecorderImpl - Pulsar client config: {
"serviceUrl" : "pulsar://10.1.18.17:6650",
"authPluginClassName" : "org.apache.pulsar.client.impl.auth.AuthenticationDisabled",
"authParams" : "*****",
"authParamMap" : null,
"operationTimeoutMs" : 30000,
"statsIntervalSeconds" : 60,
"numIoThreads" : 1,
"numListenerThreads" : 1,
"connectionsPerBroker" : 1,
"useTcpNoDelay" : true,
"useTls" : false,
"tlsTrustCertsFilePath" : null,
"tlsAllowInsecureConnection" : false,
"tlsHostnameVerificationEnable" : false,
"concurrentLookupRequest" : 5000,
"maxLookupRequest" : 50000,
"maxLookupRedirects" : 20,
"maxNumberOfRejectedRequestPerConnection" : 50,
"keepAliveIntervalSeconds" : 30,
"connectionTimeoutMs" : 10000,
"requestTimeoutMs" : 60000,
"initialBackoffIntervalNanos" : 100000000,
"maxBackoffIntervalNanos" : 60000000000,
"enableBusyWait" : false,
"listenerName" : null,
"useKeyStoreTls" : false,
"sslProvider" : null,
"tlsTrustStoreType" : "JKS",
"tlsTrustStorePath" : null,
"tlsTrustStorePassword" : null,
"tlsCiphers" : [ ],
"tlsProtocols" : [ ],
"memoryLimitBytes" : 0,
"proxyServiceUrl" : null,
"proxyProtocol" : null,
"enableTransaction" : false
}
13:08:48.445 [pulsar-io-4-4] INFO org.apache.pulsar.client.impl.ProducerImpl - [persistent://pulsar/prod-pulsar-cluster/10.1.18.17:8080/healthcheck] [null] Creating producer on cnx [id: 0xd793bcad, L:/10.1.18.17:31106 - R:10.1.18.17/10.1.18.17:6650]
13:08:48.445 [pulsar-io-4-4] INFO org.apache.pulsar.client.impl.ConsumerImpl - [persistent://pulsar/prod-pulsar-cluster/10.1.18.17:8080/healthcheck][reader-5fa28ac47c] Subscribing to topic on cnx [id: 0xd793bcad, L:/10.1.18.17:31106 - R:10.1.18.17/10.1.18.17:6650], consumerId 11174
13:08:48.445 [pulsar-io-4-5] INFO org.apache.pulsar.broker.service.ServerCnx - [/10.1.18.17:31106][persistent://pulsar/prod-pulsar-cluster/10.1.18.17:8080/healthcheck] Creating producer. producerId=11170
13:08:48.447 [pulsar-io-4-5] INFO org.apache.pulsar.broker.service.ServerCnx - [/10.1.18.17:31106] Subscribing on topic persistent://pulsar/prod-pulsar-cluster/10.1.18.17:8080/healthcheck / reader-5fa28ac47c
13:08:48.448 [BookKeeperClientWorker-OrderedExecutor-7-0] INFO org.apache.pulsar.broker.service.ServerCnx - [/10.1.18.17:31106] Created new producer: Producer{topic=PersistentTopic{topic=persistent://pulsar/prod-pulsar-cluster/10.1.18.17:8080/healthcheck}, client=/10.1.18.17:31106, producerName=prod-pulsar-cluster-14-12586, producerId=11170}
13:08:48.448 [pulsar-io-4-4] INFO org.apache.pulsar.client.impl.ProducerImpl - [persistent://pulsar/prod-pulsar-cluster/10.1.18.17:8080/healthcheck] [prod-pulsar-cluster-14-12586] Created producer on cnx [id: 0xd793bcad, L:/10.1.18.17:31106 - R:10.1.18.17/10.1.18.17:6650]
13:08:48.449 [ForkJoinPool.commonPool-worker-4] INFO org.apache.pulsar.broker.service.persistent.PersistentTopic - [persistent://pulsar/prod-pulsar-cluster/10.1.18.17:8080/healthcheck][reader-5fa28ac47c] Creating non-durable subscription at msg id 9223372036854775807:9223372036854775807:-1:-1
13:08:48.449 [ForkJoinPool.commonPool-worker-4] INFO org.apache.bookkeeper.mledger.impl.NonDurableCursorImpl - [pulsar/prod-pulsar-cluster/10.1.18.17:8080/persistent/healthcheck] Created non-durable cursor read-position=94070:313 mark-delete-position=94070:312
13:08:48.449 [ForkJoinPool.commonPool-worker-4] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [pulsar/prod-pulsar-cluster/10.1.18.17:8080/persistent/healthcheck] Opened new cursor: NonDurableCursorImpl{ledger=pulsar/prod-pulsar-cluster/10.1.18.17:8080/persistent/healthcheck, ackPos=94070:312, readPos=94070:313}
13:08:48.449 [ForkJoinPool.commonPool-worker-4] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [pulsar/prod-pulsar-cluster/10.1.18.17:8080/persistent/healthcheck-reader-5fa28ac47c] Rewind from 94070:313 to 94070:313
13:08:48.449 [ForkJoinPool.commonPool-worker-4] INFO org.apache.pulsar.broker.service.persistent.PersistentTopic - [persistent://pulsar/prod-pulsar-cluster/10.1.18.17:8080/healthcheck] There are no replicated subscriptions on the topic
13:08:48.449 [ForkJoinPool.commonPool-worker-4] INFO org.apache.pulsar.broker.service.persistent.PersistentTopic - [persistent://pulsar/prod-pulsar-cluster/10.1.18.17:8080/healthcheck][reader-5fa28ac47c] Created new subscription for 11174
13:08:48.449 [ForkJoinPool.commonPool-worker-4] INFO org.apache.pulsar.broker.service.ServerCnx - [/10.1.18.17:31106] Created subscription on topic persistent://pulsar/prod-pulsar-cluster/10.1.18.17:8080/healthcheck / reader-5fa28ac47c
13:08:48.449 [pulsar-io-4-4] INFO org.apache.pulsar.client.impl.ConsumerImpl - [persistent://pulsar/prod-pulsar-cluster/10.1.18.17:8080/healthcheck][reader-5fa28ac47c] Subscribed to topic on 10.1.18.17/10.1.18.17:6650 -- consumer: 11174
13:08:48.461 [pulsar-io-4-5] INFO org.apache.pulsar.broker.service.ServerCnx - [PersistentTopic{topic=persistent://pulsar/prod-pulsar-cluster/10.1.18.17:8080/healthcheck}][prod-pulsar-cluster-14-12586] Closing producer on cnx /10.1.18.17:31106. producerId=11170
13:08:48.461 [pulsar-io-4-5] INFO org.apache.pulsar.broker.service.ServerCnx - [PersistentTopic{topic=persistent://pulsar/prod-pulsar-cluster/10.1.18.17:8080/healthcheck}][prod-pulsar-cluster-14-12586] Closed producer on cnx /10.1.18.17:31106. producerId=11170
13:08:48.461 [pulsar-io-4-5] INFO org.apache.pulsar.broker.service.ServerCnx - [/10.1.18.17:31106] Closing consumer: consumerId=11174
13:08:48.461 [pulsar-client-internal-44-1] INFO org.eclipse.jetty.server.RequestLog - 10.1.2.84 - - [06/Dec/2021:13:08:48 +0800] "GET /admin/v2/brokers/health HTTP/1.1" 200 2 "-" "Pulsar-Java-v2.5.2" 17
13:08:48.461 [pulsar-io-4-4] INFO org.apache.pulsar.client.impl.ProducerImpl - [persistent://pulsar/prod-pulsar-cluster/10.1.18.17:8080/healthcheck] [prod-pulsar-cluster-14-12586] Closed Producer
13:08:48.462 [pulsar-io-4-5] INFO org.apache.pulsar.broker.service.AbstractDispatcherSingleActiveConsumer - Removing consumer Consumer{subscription=PersistentSubscription{topic=persistent://pulsar/prod-pulsar-cluster/10.1.18.17:8080/healthcheck, name=reader-5fa28ac47c}, consumerId=11174, consumerName=76836, address=/10.1.18.17:31106}
13:08:48.463 [pulsar-io-4-5] INFO org.apache.pulsar.broker.service.persistent.PersistentSubscription - [persistent://pulsar/prod-pulsar-cluster/10.1.18.17:8080/healthcheck][reader-5fa28ac47c] Successfully closed subscription [NonDurableCursorImpl{ledger=pulsar/prod-pulsar-cluster/10.1.18.17:8080/persistent/healthcheck, ackPos=94070:312, readPos=94070:314}]
13:08:48.463 [pulsar-io-4-5] INFO org.apache.pulsar.broker.service.persistent.PersistentSubscription - [persistent://pulsar/prod-pulsar-cluster/10.1.18.17:8080/healthcheck][reader-5fa28ac47c] Successfully closed dispatcher for reader
13:08:48.463 [pulsar-io-4-5] INFO org.apache.pulsar.broker.service.ServerCnx - [/10.1.18.17:31106] Closed consumer, consumerId=11174
13:08:48.463 [pulsar-io-4-4] INFO org.apache.pulsar.client.impl.ConsumerImpl - [persistent://pulsar/prod-pulsar-cluster/10.1.18.17:8080/healthcheck] [reader-5fa28ac47c] Closed consumer
13:08:48.463 [pulsar-web-40-14] INFO org.eclipse.jetty.server.RequestLog - 10.1.2.84 - - [06/Dec/2021:13:08:48 +0800] "GET /admin/v2/clusters/prod-pulsar-cluster/failureDomains HTTP/1.1" 200 2 "-" "Pulsar-Java-v2.5.2" 0
13:08:48.463 [pulsar-web-40-13] INFO org.eclipse.jetty.server.RequestLog - 10.1.2.84 - - [06/Dec/2021:13:08:48 +0800] "GET /admin/v2/brokers/prod-pulsar-cluster HTTP/1.1" 200 55 "-" "Pulsar-Java-v2.5.2" 0
13:08:48.464 [pulsar-web-40-6] INFO org.eclipse.jetty.server.RequestLog - 10.1.2.84 - - [06/Dec/2021:13:08:48 +0800] "GET /admin/v2/broker-stats/topics HTTP/1.1" 200 32357 "-" "Pulsar-Java-v2.5.2" 0
13:09:19.156 [pulsar-web-40-16] INFO org.eclipse.jetty.server.RequestLog - 10.1.2.84 - - [06/Dec/2021:13:09:19 +0800] "GET /admin/v2/clusters HTTP/1.1" 200 23 "-" "Pulsar-Java-v2.5.2" 0
13:09:19.158 [pulsar-web-40-3] INFO org.eclipse.jetty.server.RequestLog - 10.1.2.84 - - [06/Dec/2021:13:09:19 +0800] "GET /admin/v2/clusters/prod-pulsar-cluster/failureDomains HTTP/1.1" 200 2 "-" "Pulsar-Java-v2.5.2" 1
13:09:19.159 [pulsar-web-40-7] INFO org.eclipse.jetty.server.RequestLog - 10.1.2.84 - - [06/Dec/2021:13:09:19 +0800] "GET /admin/v2/brokers/prod-pulsar-cluster HTTP/1.1" 200 55 "-" "Pulsar-Java-v2.5.2" 1
13:09:19.159 [pulsar-web-40-2] INFO org.eclipse.jetty.server.RequestLog - 10.1.2.84 - - [06/Dec/2021:13:09:19 +0800] "GET /admin/v2/clusters/prod-pulsar-cluster HTTP/1.1" 200 296 "-" "Pulsar-Java-v2.5.2" 0
13:09:19.160 [pulsar-io-4-4] INFO org.apache.pulsar.client.impl.ProducerStatsRecorderImpl - Starting Pulsar producer perf with config: {
"topicName" : "persistent://pulsar/prod-pulsar-cluster/10.1.18.17:8080/healthcheck",
"producerName" : null,
"sendTimeoutMs" : 30000,
"blockIfQueueFull" : false,
"maxPendingMessages" : 1000,
"maxPendingMessagesAcrossPartitions" : 50000,
"messageRoutingMode" : "RoundRobinPartition",
"hashingScheme" : "JavaStringHash",
"cryptoFailureAction" : "FAIL",
"batchingMaxPublishDelayMicros" : 1000,
"batchingPartitionSwitchFrequencyByPublishDelay" : 10,
"batchingMaxMessages" : 1000,
"batchingMaxBytes" : 131072,
"batchingEnabled" : true,
"chunkingEnabled" : false,
"compressionType" : "NONE",
"initialSequenceId" : null,
"autoUpdatePartitions" : true,
"autoUpdatePartitionsIntervalSeconds" : 60,
"multiSchema" : true,
"accessMode" : "Shared",
"properties" : { }
}
13:09:19.160 [pulsar-io-4-4] INFO org.apache.pulsar.client.impl.ProducerStatsRecorderImpl - Pulsar client config: {
"serviceUrl" : "pulsar://10.1.18.17:6650",
"authPluginClassName" : "org.apache.pulsar.client.impl.auth.AuthenticationDisabled",
"authParams" : "*****",
"authParamMap" : null,
"operationTimeoutMs" : 30000,
"statsIntervalSeconds" : 60,
"numIoThreads" : 1,
"numListenerThreads" : 1,
"connectionsPerBroker" : 1,
"useTcpNoDelay" : true,
"useTls" : false,
"tlsTrustCertsFilePath" : null,
"tlsAllowInsecureConnection" : false,
"tlsHostnameVerificationEnable" : false,
"concurrentLookupRequest" : 5000,
"maxLookupRequest" : 50000,
"maxLookupRedirects" : 20,
"maxNumberOfRejectedRequestPerConnection" : 50,
"keepAliveIntervalSeconds" : 30,
"connectionTimeoutMs" : 10000,
"requestTimeoutMs" : 60000,
"initialBackoffIntervalNanos" : 100000000,
"maxBackoffIntervalNanos" : 60000000000,
"enableBusyWait" : false,
"listenerName" : null,
"useKeyStoreTls" : false,
"sslProvider" : null,
"tlsTrustStoreType" : "JKS",
"tlsTrustStorePath" : null,
"tlsTrustStorePassword" : null,
"tlsCiphers" : [ ],
"tlsProtocols" : [ ],
"memoryLimitBytes" : 0,
"proxyServiceUrl" : null,
"proxyProtocol" : null,
"enableTransaction" : false
}
13:09:19.160 [pulsar-io-4-4] INFO org.apache.pulsar.client.impl.ConsumerStatsRecorderImpl - Starting Pulsar consumer status recorder with config: {
"topicNames" : [ "persistent://pulsar/prod-pulsar-cluster/10.1.18.17:8080/healthcheck" ],
"topicsPattern" : null,
"subscriptionName" : "reader-632a65adbf",
"subscriptionType" : "Exclusive",
"subscriptionMode" : "NonDurable",
"receiverQueueSize" : 1000,
"acknowledgementsGroupTimeMicros" : 100000,
"negativeAckRedeliveryDelayMicros" : 60000000,
"maxTotalReceiverQueueSizeAcrossPartitions" : 50000,
"consumerName" : null,
"ackTimeoutMillis" : 0,
"tickDurationMillis" : 1000,
"priorityLevel" : 0,
"maxPendingChunkedMessage" : 10,
"autoAckOldestChunkedMessageOnQueueFull" : false,
"expireTimeOfIncompleteChunkedMessageMillis" : 60000,
"cryptoFailureAction" : "FAIL",
"properties" : { },
"readCompacted" : false,
"subscriptionInitialPosition" : "Latest",
"patternAutoDiscoveryPeriod" : 60,
"regexSubscriptionMode" : "PersistentOnly",
"deadLetterPolicy" : null,
"retryEnable" : false,
"autoUpdatePartitions" : true,
"autoUpdatePartitionsIntervalSeconds" : 60,
"replicateSubscriptionState" : false,
"resetIncludeHead" : false,
"keySharedPolicy" : null,
"batchIndexAckEnabled" : false,
"ackReceiptEnabled" : false,
"poolMessages" : false,
"maxPendingChuckedMessage" : 10
}
13:09:19.161 [pulsar-io-4-4] INFO org.apache.pulsar.client.impl.ConsumerStatsRecorderImpl - Pulsar client config: {
"serviceUrl" : "pulsar://10.1.18.17:6650",
"authPluginClassName" : "org.apache.pulsar.client.impl.auth.AuthenticationDisabled",
"authParams" : "*****",
"authParamMap" : null,
"operationTimeoutMs" : 30000,
"statsIntervalSeconds" : 60,
"numIoThreads" : 1,
"numListenerThreads" : 1,
"connectionsPerBroker" : 1,
"useTcpNoDelay" : true,
"useTls" : false,
"tlsTrustCertsFilePath" : null,
"tlsAllowInsecureConnection" : false,
"tlsHostnameVerificationEnable" : false,
"concurrentLookupRequest" : 5000,
"maxLookupRequest" : 50000,
"maxLookupRedirects" : 20,
"maxNumberOfRejectedRequestPerConnection" : 50,
"keepAliveIntervalSeconds" : 30,
"connectionTimeoutMs" : 10000,
"requestTimeoutMs" : 60000,
"initialBackoffIntervalNanos" : 100000000,
"maxBackoffIntervalNanos" : 60000000000,
"enableBusyWait" : false,
"listenerName" : null,
"useKeyStoreTls" : false,
"sslProvider" : null,
"tlsTrustStoreType" : "JKS",
"tlsTrustStorePath" : null,
"tlsTrustStorePassword" : null,
"tlsCiphers" : [ ],
"tlsProtocols" : [ ],
"memoryLimitBytes" : 0,
"proxyServiceUrl" : null,
"proxyProtocol" : null,
"enableTransaction" : false
}
13:09:19.161 [pulsar-io-4-4] INFO org.apache.pulsar.client.impl.ProducerImpl - [persistent://pulsar/prod-pulsar-cluster/10.1.18.17:8080/healthcheck] [null] Creating producer on cnx [id: 0xd793bcad, L:/10.1.18.17:31106 - R:10.1.18.17/10.1.18.17:6650]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] xuesongxs commented on issue #13149: pulsar-2.8.1中ledger长时间处于closing状态
Posted by GitBox <gi...@apache.org>.
xuesongxs commented on issue #13149:
URL: https://github.com/apache/pulsar/issues/13149#issuecomment-987704176
实际ledger根本就没有状态，pulsar-manager上看到的opening或closing只是pulsar-manager设置上去的，本质是按照ledgers列表的创建时间排序的，最新创建的ledger是opening，其它统一都设置为closing。如果你设置了消息保留策略，或者消费者还没有把backlog消费掉，这个列表里就是会显示多个ledger，如果使用默认的消息保留策略，或消费追得上生产，这里只会显示一个ledger，并且是opening的状态。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] wuxinle0412 closed issue #13149: pulsar-2.8.1中ledger长时间处于closing状态
Posted by GitBox <gi...@apache.org>.
wuxinle0412 closed issue #13149:
URL: https://github.com/apache/pulsar/issues/13149
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] wuxinle0412 commented on issue #13149: pulsar-2.8.1中ledger长时间处于closing状态
Posted by GitBox <gi...@apache.org>.
wuxinle0412 commented on issue #13149:
URL: https://github.com/apache/pulsar/issues/13149#issuecomment-987859031
感谢解答 @xuesongxs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org