You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2020/08/17 07:08:06 UTC
[GitHub] [pulsar] kuangye098 opened a new issue #7827: In failover mode, messages are not consumed normally.
kuangye098 opened a new issue #7827:
URL: https://github.com/apache/pulsar/issues/7827
**Describe the bug**
Consumers are stuck and do not consume in the failover mode. I wonder if it is caused by the same consumer name?
**To Reproduce**
Steps to reproduce the behavior:
1. Start Pulsar stand-alone mode.
2. The running version is 2.5.2.
3. Start the consumer and prepare to consume the message，consumers are also deployed locally.
**Expected behavior**
When a consumer with only one failover mode is started, data should be consumed normally.
Subscription information：
```shell
[jzyadm@uat_yw_zt_app_223 apache-pulsar-2.5.2]$ bin/pulsar-admin persistent stats persistent://public/default/CANCEL_ORDER_REQUEST
{
"msgRateIn" : 0.0,
"msgThroughputIn" : 0.0,
"msgRateOut" : 0.0,
"msgThroughputOut" : 0.0,
"bytesInCounter" : 29768,
"msgInCounter" : 105,
"bytesOutCounter" : 0,
"msgOutCounter" : 0,
"averageMsgSize" : 0.0,
"storageSize" : 15268,
"backlogSize" : 3403,
"publishers" : [ {
"msgRateIn" : 0.0,
"msgThroughputIn" : 0.0,
"averageMsgSize" : 0.0,
"producerId" : 18,
"metadata" : { },
"producerName" : "standalone-0-680",
"connectedSince" : "2020-08-14T00:20:30.285+08:00",
"clientVersion" : "2.5.2",
"address" : "/172.18.252.223:39544"
} ],
"subscriptions" : {
"subscription-com.x.x.service.OrderService-cancelOrderConsume" : {
"msgRateOut" : 0.0,
"msgThroughputOut" : 0.0,
"bytesOutCounter" : 0,
"msgOutCounter" : 0,
"msgRateRedeliver" : 0.0,
"msgBacklog" : 10,
"msgBacklogNoDelayed" : 10,
"blockedSubscriptionOnUnackedMsgs" : false,
"msgDelayed" : 0,
"unackedMessages" : 0,
"type" : "Failover",
"activeConsumerName" : "consumer-com.x.x.service.OrderService-cancelOrderConsume",
"msgRateExpired" : 0.0,
"lastExpireTimestamp" : 0,
"lastConsumedFlowTimestamp" : 1597634317683,
"lastConsumedTimestamp" : 0,
"lastAckedTimestamp" : 0,
"consumers" : [ {
"msgRateOut" : 0.0,
"msgThroughputOut" : 0.0,
"bytesOutCounter" : 0,
"msgOutCounter" : 0,
"msgRateRedeliver" : 0.0,
"consumerName" : "consumer-com.x.x.service.OrderService-cancelOrderConsume",
"availablePermits" : 0,
"unackedMessages" : 0,
"blockedConsumerOnUnackedMsgs" : false,
"lastAckedTimestamp" : 0,
"lastConsumedTimestamp" : 0,
"metadata" : { },
"connectedSince" : "2020-08-14T07:33:20.635+08:00",
"clientVersion" : "2.5.2",
"address" : "/172.18.252.223:19560"
}, {
"msgRateOut" : 0.0,
"msgThroughputOut" : 0.0,
"bytesOutCounter" : 0,
"msgOutCounter" : 0,
"msgRateRedeliver" : 0.0,
"consumerName" : "consumer-com.x.x.service.OrderService-cancelOrderConsume",
"availablePermits" : 0,
"unackedMessages" : 0,
"blockedConsumerOnUnackedMsgs" : false,
"lastAckedTimestamp" : 0,
"lastConsumedTimestamp" : 0,
"metadata" : { },
"connectedSince" : "2020-08-14T08:21:53.669+08:00",
"clientVersion" : "2.5.2",
"address" : "/172.18.252.223:26408"
}, {
"msgRateOut" : 0.0,
"msgThroughputOut" : 0.0,
"bytesOutCounter" : 0,
"msgOutCounter" : 0,
"msgRateRedeliver" : 0.0,
"consumerName" : "consumer-com.x.x.service.OrderService-cancelOrderConsume",
"availablePermits" : 1000,
"unackedMessages" : 0,
"blockedConsumerOnUnackedMsgs" : false,
"lastAckedTimestamp" : 0,
"lastConsumedTimestamp" : 0,
"metadata" : { },
"connectedSince" : "2020-08-17T11:18:37.68+08:00",
"clientVersion" : "2.5.2",
"address" : "/172.18.252.223:54738"
} ],
"isReplicated" : false
}
},
"replication" : { },
"deduplicationStatus" : "Disabled"
}
```
Local network connection information：
```shell
tcp 0 0 0.0.0.0:6650 0.0.0.0:* LISTEN 391/java
tcp 0 0 127.0.0.1:48690 127.0.0.1:6650 ESTABLISHED 391/java
tcp 0 0 172.18.252.223:54738 172.18.252.223:6650 ESTABLISHED 19089/java
tcp 0 0 172.18.252.223:6650 172.18.252.248:46862 ESTABLISHED 391/java
tcp 0 0 127.0.0.1:6650 127.0.0.1:48690 ESTABLISHED 391/java
tcp 0 0 172.18.252.223:6650 172.18.252.248:61114 ESTABLISHED 391/java
tcp 0 0 172.18.252.223:6650 172.18.252.248:29418 ESTABLISHED 391/java
tcp 0 0 172.18.252.223:6650 172.18.252.223:39544 ESTABLISHED 391/java
tcp 0 0 172.18.252.223:54734 172.18.252.223:6650 ESTABLISHED 19089/java
tcp 0 0 172.18.252.223:6650 172.18.252.223:62964 ESTABLISHED 391/java
tcp 0 0 172.18.252.223:6650 172.18.252.223:54738 ESTABLISHED 391/java
tcp 0 0 172.18.252.223:6650 172.18.252.223:54734 ESTABLISHED 391/java
tcp 0 0 172.18.252.223:6650 172.18.252.248:46866 ESTABLISHED 391/java
tcp 0 0 172.18.252.223:6650 172.18.252.248:29388 ESTABLISHED 391/java
tcp 0 0 172.18.252.223:62964 172.18.252.223:6650 ESTABLISHED 7701/java
tcp 0 0 172.18.252.223:39544 172.18.252.223:6650 ESTABLISHED 7701/java
tcp 0 0 172.18.252.223:6650 172.18.252.248:61110 ESTABLISHED 391/java
tcp 0 0 172.18.252.223:16650 172.18.252.224:8848 TIME_WAIT -
```
In fact, I only activated one consumer, but I don't know why there are 3 consumer information in the consumer status information, and the activated consumer has no normal consumption information.
**Desktop (please complete the following information):**
- OS: CentOS Linux release 7.7.1908 (Core)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] BewareMyPower commented on issue #7827: In failover mode, messages are not consumed normally.
Posted by GitBox <gi...@apache.org>.
BewareMyPower commented on issue #7827:
URL: https://github.com/apache/pulsar/issues/7827#issuecomment-674860397
From the stats and network connection, I guessed that the broker didn't remove first 2 consumers of this subscription from stats. Then the broker treat a no longer existing consumer as the active consumer, and the actually running consumer (port: 54738, process id: 19089) was treated as a failover consumer, so it cannot receive messages.
I can't reproduce the behavior with a newly started pulsar standalone.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] BewareMyPower edited a comment on issue #7827: In failover mode, messages are not consumed normally.
Posted by GitBox <gi...@apache.org>.
BewareMyPower edited a comment on issue #7827:
URL: https://github.com/apache/pulsar/issues/7827#issuecomment-674860397
From the stats and network connection, I guessed that the broker didn't remove first 2 consumers of this subscription from stats. Then the broker treat a no longer existing consumer as the active consumer, and the actually running consumer (port: 54738, process id: 19089) was treated as a failover consumer, so it cannot receive messages.
I can't reproduce the behavior with a newly started pulsar standalone. Could you provide your code? Is there some error log?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org