You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/03/29 09:27:58 UTC
[GitHub] [pulsar] zcola opened a new issue #14927: After using ns-isolation-policy, topics unload is invalid, only namespace unload is valid
zcola opened a new issue #14927:
URL: https://github.com/apache/pulsar/issues/14927
**Describe the bug**
After using ns-isolation-policy, topics unload is invalid, only namespace unload is valid
**To Reproduce**
Steps to reproduce the behavior:
```
1. ./bin/pulsar-admin ns-isolation-policy set --auto-failover-policy-type min_available \ --auto-failover-policy-params min_limit=1,usage_threshold=80 \ --namespaces zcola_20220329/test --primary 7.32.133.237.* \ pulsar-19990 policy-zcola_qos-test
2. ./bin/pulsar-admin topics partitioned-lookup persistent://zcola_20220329/test/test
3. ./bin/pulsar-admin topics unload zcola_20220329/test/test **not work**
./bin/pulsar-admin topics partitioned-lookup persistent://zcola_20220329/test/test
"persistent://zcola_20220329/test/test-partition-0 pulsar://7.32.133.235:6650"
"persistent://zcola_20220329/test/test-partition-1 pulsar://7.32.133.235:6650"
"persistent://zcola_20220329/test/test-partition-2 pulsar://7.32.133.235:6650"
5. ./bin/pulsar-admin namespaces unload zcola_20220329/test **it work**
./bin/pulsar-admin topics partitioned-lookup persistent://zcola_20220329/test/test
"persistent://zcola_20220329/test/test-partition-0 pulsar://7.32.133.237:6650"
"persistent://zcola_20220329/test/test-partition-1 pulsar://7.32.133.237:6650"
"persistent://zcola_20220329/test/test-partition-2 pulsar://7.32.133.237:6650"
```
**Expected behavior**
A clear and concise description of what you expected to happen.
**Screenshots**
If applicable, add screenshots to help explain your problem.
**Desktop (please complete the following information):**
- pulsar 2.9.1
**Additional context**
Ralated #14728
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] poorbarcode commented on issue #14927: After using ns-isolation-policy, topics unload is invalid, only namespace unload is valid
Posted by GitBox <gi...@apache.org>.
poorbarcode commented on issue #14927:
URL: https://github.com/apache/pulsar/issues/14927#issuecomment-1082913675
It's been try in two version: tag-2.9.1 and master
Could not reproduce.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Technoboy- commented on issue #14927: After using ns-isolation-policy, topics unload is invalid, only namespace unload is valid
Posted by GitBox <gi...@apache.org>.
Technoboy- commented on issue #14927:
URL: https://github.com/apache/pulsar/issues/14927#issuecomment-1081748860
I will try to reproduce.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] poorbarcode edited a comment on issue #14927: After using ns-isolation-policy, topics unload is invalid, only namespace unload is valid
Posted by GitBox <gi...@apache.org>.
poorbarcode edited a comment on issue #14927:
URL: https://github.com/apache/pulsar/issues/14927#issuecomment-1082913675
It's been try in two version: tag-2.9.1 and master
Could not reproduce.
Here's how I did it:
### 1. start pulsar
```
bin/pulsar standalone
```
### 2. auto create a topic: my-topic, and ensure topic exists
```
GET http://127.0.0.1:8080/admin/v2/persistent/public/default/my-topic/stats
response:
{
"offloadedStorageSize": 0,
"subscriptions": {
"my-subscription": {
"lastConsumedFlowTimestamp": 1648636226130,
"nonContiguousDeletedMessagesRangesSerializedSize": 0,
"msgRateExpired": 0.0,
"lastMarkDeleteAdvancedTimestamp": 0,
"msgOutCounter": 0,
"backlogSize": 0,
"type": "Exclusive",
"durable": true,
"bytesOutCounter": 0,
"totalMsgExpired": 0,
"nonContiguousDeletedMessagesRanges": 0,
"consumersAfterMarkDeletePosition": {
},
"consumers": [
{
"metadata": {
},
"address": "\/127.0.0.1:61469",
"msgRateRedeliver": 0.0,
"avgMessagesPerEntry": 0,
"msgOutCounter": 0,
"clientVersion": "2.10.0-SNAPSHOT",
"msgThroughputOut": 0.0,
"availablePermits": 1000,
"bytesOutCounter": 0,
"connectedSince": "2022-03-30T18:30:26.1286191+08:00",
"blockedConsumerOnUnackedMsgs": false,
"lastAckedTimestamp": 0,
"unackedMessages": 0,
"chunkedMessageRate": 0.0,
"msgRateOut": 0.0,
"consumerName": "682bf",
"lastConsumedTimestamp": 0
}
],
"blockedSubscriptionOnUnackedMsgs": false,
"msgDelayed": 0,
"unackedMessages": 0,
"chunkedMessageRate": 0,
"msgBacklog": 0,
"lastConsumedTimestamp": 0,
"activeConsumerName": "682bf",
"lastExpireTimestamp": 0,
"msgRateRedeliver": 0.0,
"isDurable": true,
"msgThroughputOut": 0.0,
"earliestMsgPublishTimeInBacklog": 0,
"subscriptionProperties": {
},
"isReplicated": false,
"allowOutOfOrderDelivery": false,
"lastAckedTimestamp": 0,
"msgBacklogNoDelayed": 0,
"replicated": false,
"msgRateOut": 0.0
}
},
"nonContiguousDeletedMessagesRangesSerializedSize": 0,
"lastOffloadFailureTimeStamp": 0,
"msgOutCounter": 0,
"backlogSize": 0,
"bytesInCounter": 0,
"lastOffloadLedgerId": 0,
"bytesOutCounter": 0,
"msgInCounter": 0,
"nonContiguousDeletedMessagesRanges": 0,
"publishers": [
{
"msgRateIn": 0.0,
"supportsPartialProducer": false,
"metadata": {
},
"address": "\/127.0.0.1:61443",
"connectedSince": "2022-03-30T18:30:26.1256204+08:00",
"producerName": "standalone-0-0",
"averageMsgSize": 0.0,
"producerId": 0,
"accessMode": "Shared",
"clientVersion": "2.10.0-SNAPSHOT",
"msgThroughputIn": 0.0,
"chunkedMessageRate": 0.0
}
],
"compaction": {
"lastCompactionDurationTimeInMills": 0,
"lastCompactionRemovedEventCount": 0,
"lastCompactionSucceedTimestamp": 0,
"lastCompactionFailedTimestamp": 0
},
"averageMsgSize": 0.0,
"msgRateIn": 0.0,
"replication": {
},
"earliestMsgPublishTimeInBacklogs": 0,
"msgChunkPublished": false,
"msgThroughputIn": 0.0,
"msgThroughputOut": 0.0,
"publishRateLimitedTimes": 0,
"waitingPublishers": 0,
"storageSize": 0,
"deduplicationStatus": "Disabled",
"lastOffloadSuccessTimeStamp": 0,
"msgRateOut": 0.0
}
```
### 3. add a policy at namespace: default
```
POST http://127.0.0.1:8080/admin/v2/clusters/standalone/namespaceIsolationPolicies/policy_test
POST data:
{
"namespaces": ["public/default"],
"primary": ["localhost:8080"],
"secondary": [],
"auto_failover_policy": {
"policy_type": "min_available",
"parameters": {
"min_limit": 1,
"usage_threshold": 80
}
}
}
```
### 4. ensure policy add success
```
GET http://127.0.0.1:8080/admin/v2/clusters/standalone/namespaceIsolationPolicies/brokers/localhost:8080
response:
{
"policyName": "policy_test",
"namespaceRegex": [
"public\/default"
],
"brokerName": "localhost:8080",
"primary": true
}
```
### 5. do unload topic: my-topic
```
PUT http://127.0.0.1:8080/admin/v2/persistent/public/default/my-topic/unload
```
## 6. ensure topic unload success
```
GET http://127.0.0.1:8080/admin/v2/persistent/public/default/my-topic/stats
response:
{
"offloadedStorageSize": 0,
"subscriptions": {
"my-subscription": {
"lastConsumedFlowTimestamp": 0,
"nonContiguousDeletedMessagesRangesSerializedSize": 0,
"msgRateExpired": 0.0,
"lastMarkDeleteAdvancedTimestamp": 0,
"msgOutCounter": 0,
"backlogSize": 0,
"type": "None",
"durable": true,
"bytesOutCounter": 0,
"totalMsgExpired": 0,
"nonContiguousDeletedMessagesRanges": 0,
"consumersAfterMarkDeletePosition": {
},
"consumers": [
],
"blockedSubscriptionOnUnackedMsgs": false,
"msgDelayed": 0,
"unackedMessages": 0,
"chunkedMessageRate": 0,
"msgBacklog": 0,
"lastConsumedTimestamp": 0,
"lastExpireTimestamp": 0,
"msgRateRedeliver": 0.0,
"isDurable": true,
"msgThroughputOut": 0.0,
"earliestMsgPublishTimeInBacklog": 0,
"subscriptionProperties": {
},
"isReplicated": false,
"allowOutOfOrderDelivery": false,
"lastAckedTimestamp": 0,
"msgBacklogNoDelayed": 0,
"replicated": false,
"msgRateOut": 0.0
}
},
"nonContiguousDeletedMessagesRangesSerializedSize": 0,
"lastOffloadFailureTimeStamp": 0,
"msgOutCounter": 0,
"backlogSize": 0,
"bytesInCounter": 0,
"lastOffloadLedgerId": 0,
"bytesOutCounter": 0,
"msgInCounter": 0,
"nonContiguousDeletedMessagesRanges": 0,
"publishers": [
],
"compaction": {
"lastCompactionDurationTimeInMills": 0,
"lastCompactionRemovedEventCount": 0,
"lastCompactionSucceedTimestamp": 0,
"lastCompactionFailedTimestamp": 0
},
"averageMsgSize": 0.0,
"msgRateIn": 0.0,
"replication": {
},
"earliestMsgPublishTimeInBacklogs": 0,
"msgChunkPublished": false,
"msgThroughputIn": 0.0,
"msgThroughputOut": 0.0,
"publishRateLimitedTimes": 0,
"waitingPublishers": 0,
"storageSize": 0,
"deduplicationStatus": "Disabled",
"lastOffloadSuccessTimeStamp": 0,
"msgRateOut": 0.0
}
```
### 7. a few secods later, producer and consumer will auto registry again
```
GET http://127.0.0.1:8080/admin/v2/persistent/public/default/my-topic/stats
response:
{
"offloadedStorageSize": 0,
"subscriptions": {
"my-subscription": {
"lastConsumedFlowTimestamp": 1648636819910,
"nonContiguousDeletedMessagesRangesSerializedSize": 0,
"msgRateExpired": 0.0,
"lastMarkDeleteAdvancedTimestamp": 0,
"msgOutCounter": 0,
"backlogSize": 0,
"type": "Exclusive",
"durable": true,
"bytesOutCounter": 0,
"totalMsgExpired": 0,
"nonContiguousDeletedMessagesRanges": 0,
"consumersAfterMarkDeletePosition": {
},
"consumers": [
{
"metadata": {
},
"address": "\/127.0.0.1:61469",
"msgRateRedeliver": 0.0,
"avgMessagesPerEntry": 0,
"msgOutCounter": 0,
"clientVersion": "2.10.0-SNAPSHOT",
"msgThroughputOut": 0.0,
"availablePermits": 1000,
"bytesOutCounter": 0,
"connectedSince": "2022-03-30T18:40:19.9067263+08:00",
"blockedConsumerOnUnackedMsgs": false,
"lastAckedTimestamp": 0,
"unackedMessages": 0,
"chunkedMessageRate": 0.0,
"msgRateOut": 0.0,
"consumerName": "682bf",
"lastConsumedTimestamp": 0
}
],
"blockedSubscriptionOnUnackedMsgs": false,
"msgDelayed": 0,
"unackedMessages": 0,
"chunkedMessageRate": 0,
"msgBacklog": 0,
"lastConsumedTimestamp": 0,
"activeConsumerName": "682bf",
"lastExpireTimestamp": 0,
"msgRateRedeliver": 0.0,
"isDurable": true,
"msgThroughputOut": 0.0,
"earliestMsgPublishTimeInBacklog": 0,
"subscriptionProperties": {
},
"isReplicated": false,
"allowOutOfOrderDelivery": false,
"lastAckedTimestamp": 0,
"msgBacklogNoDelayed": 0,
"replicated": false,
"msgRateOut": 0.0
}
},
"nonContiguousDeletedMessagesRangesSerializedSize": 0,
"lastOffloadFailureTimeStamp": 0,
"msgOutCounter": 0,
"backlogSize": 0,
"bytesInCounter": 0,
"lastOffloadLedgerId": 0,
"bytesOutCounter": 0,
"msgInCounter": 0,
"nonContiguousDeletedMessagesRanges": 0,
"publishers": [
{
"msgRateIn": 0.0,
"supportsPartialProducer": false,
"metadata": {
},
"address": "\/127.0.0.1:61443",
"connectedSince": "2022-03-30T18:40:19.9057264+08:00",
"producerName": "standalone-0-0",
"averageMsgSize": 0.0,
"producerId": 0,
"accessMode": "Shared",
"clientVersion": "2.10.0-SNAPSHOT",
"msgThroughputIn": 0.0,
"chunkedMessageRate": 0.0
}
],
"compaction": {
"lastCompactionDurationTimeInMills": 0,
"lastCompactionRemovedEventCount": 0,
"lastCompactionSucceedTimestamp": 0,
"lastCompactionFailedTimestamp": 0
},
"averageMsgSize": 0.0,
"msgRateIn": 0.0,
"replication": {
},
"earliestMsgPublishTimeInBacklogs": 0,
"msgChunkPublished": false,
"msgThroughputIn": 0.0,
"msgThroughputOut": 0.0,
"publishRateLimitedTimes": 0,
"waitingPublishers": 0,
"storageSize": 0,
"deduplicationStatus": "Disabled",
"lastOffloadSuccessTimeStamp": 0,
"msgRateOut": 0.0
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Technoboy- commented on issue #14927: After using ns-isolation-policy, topics unload is invalid, only namespace unload is valid
Posted by GitBox <gi...@apache.org>.
Technoboy- commented on issue #14927:
URL: https://github.com/apache/pulsar/issues/14927#issuecomment-1083058790
Hi @zcola ：
It's better to share the broker log when unload topic doesn't work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] poorbarcode edited a comment on issue #14927: After using ns-isolation-policy, topics unload is invalid, only namespace unload is valid
Posted by GitBox <gi...@apache.org>.
poorbarcode edited a comment on issue #14927:
URL: https://github.com/apache/pulsar/issues/14927#issuecomment-1082913675
Hi @codelipenghui @Technoboy- Could not reproduce. It's been try in two version: tag-2.9.1 and master
Here's how I did it:
### 1. start pulsar
```
bin/pulsar standalone
```
### 2. auto create a topic: my-topic, and ensure topic exists
```
GET http://127.0.0.1:8080/admin/v2/persistent/public/default/my-topic/stats
response:
{
"offloadedStorageSize": 0,
"subscriptions": {
"my-subscription": {
"lastConsumedFlowTimestamp": 1648636226130,
"nonContiguousDeletedMessagesRangesSerializedSize": 0,
"msgRateExpired": 0.0,
"lastMarkDeleteAdvancedTimestamp": 0,
"msgOutCounter": 0,
"backlogSize": 0,
"type": "Exclusive",
"durable": true,
"bytesOutCounter": 0,
"totalMsgExpired": 0,
"nonContiguousDeletedMessagesRanges": 0,
"consumersAfterMarkDeletePosition": {
},
"consumers": [
{
"metadata": {
},
"address": "\/127.0.0.1:61469",
"msgRateRedeliver": 0.0,
"avgMessagesPerEntry": 0,
"msgOutCounter": 0,
"clientVersion": "2.10.0-SNAPSHOT",
"msgThroughputOut": 0.0,
"availablePermits": 1000,
"bytesOutCounter": 0,
"connectedSince": "2022-03-30T18:30:26.1286191+08:00",
"blockedConsumerOnUnackedMsgs": false,
"lastAckedTimestamp": 0,
"unackedMessages": 0,
"chunkedMessageRate": 0.0,
"msgRateOut": 0.0,
"consumerName": "682bf",
"lastConsumedTimestamp": 0
}
],
"blockedSubscriptionOnUnackedMsgs": false,
"msgDelayed": 0,
"unackedMessages": 0,
"chunkedMessageRate": 0,
"msgBacklog": 0,
"lastConsumedTimestamp": 0,
"activeConsumerName": "682bf",
"lastExpireTimestamp": 0,
"msgRateRedeliver": 0.0,
"isDurable": true,
"msgThroughputOut": 0.0,
"earliestMsgPublishTimeInBacklog": 0,
"subscriptionProperties": {
},
"isReplicated": false,
"allowOutOfOrderDelivery": false,
"lastAckedTimestamp": 0,
"msgBacklogNoDelayed": 0,
"replicated": false,
"msgRateOut": 0.0
}
},
"nonContiguousDeletedMessagesRangesSerializedSize": 0,
"lastOffloadFailureTimeStamp": 0,
"msgOutCounter": 0,
"backlogSize": 0,
"bytesInCounter": 0,
"lastOffloadLedgerId": 0,
"bytesOutCounter": 0,
"msgInCounter": 0,
"nonContiguousDeletedMessagesRanges": 0,
"publishers": [
{
"msgRateIn": 0.0,
"supportsPartialProducer": false,
"metadata": {
},
"address": "\/127.0.0.1:61443",
"connectedSince": "2022-03-30T18:30:26.1256204+08:00",
"producerName": "standalone-0-0",
"averageMsgSize": 0.0,
"producerId": 0,
"accessMode": "Shared",
"clientVersion": "2.10.0-SNAPSHOT",
"msgThroughputIn": 0.0,
"chunkedMessageRate": 0.0
}
],
"compaction": {
"lastCompactionDurationTimeInMills": 0,
"lastCompactionRemovedEventCount": 0,
"lastCompactionSucceedTimestamp": 0,
"lastCompactionFailedTimestamp": 0
},
"averageMsgSize": 0.0,
"msgRateIn": 0.0,
"replication": {
},
"earliestMsgPublishTimeInBacklogs": 0,
"msgChunkPublished": false,
"msgThroughputIn": 0.0,
"msgThroughputOut": 0.0,
"publishRateLimitedTimes": 0,
"waitingPublishers": 0,
"storageSize": 0,
"deduplicationStatus": "Disabled",
"lastOffloadSuccessTimeStamp": 0,
"msgRateOut": 0.0
}
```
### 3. add a policy at namespace: default
```
POST http://127.0.0.1:8080/admin/v2/clusters/standalone/namespaceIsolationPolicies/policy_test
POST data:
{
"namespaces": ["public/default"],
"primary": ["localhost:8080"],
"secondary": [],
"auto_failover_policy": {
"policy_type": "min_available",
"parameters": {
"min_limit": 1,
"usage_threshold": 80
}
}
}
```
### 4. ensure policy add success
```
GET http://127.0.0.1:8080/admin/v2/clusters/standalone/namespaceIsolationPolicies/brokers/localhost:8080
response:
{
"policyName": "policy_test",
"namespaceRegex": [
"public\/default"
],
"brokerName": "localhost:8080",
"primary": true
}
```
### 5. do unload topic: my-topic
```
PUT http://127.0.0.1:8080/admin/v2/persistent/public/default/my-topic/unload
```
## 6. ensure topic unload success
```
GET http://127.0.0.1:8080/admin/v2/persistent/public/default/my-topic/stats
response:
{
"offloadedStorageSize": 0,
"subscriptions": {
"my-subscription": {
"lastConsumedFlowTimestamp": 0,
"nonContiguousDeletedMessagesRangesSerializedSize": 0,
"msgRateExpired": 0.0,
"lastMarkDeleteAdvancedTimestamp": 0,
"msgOutCounter": 0,
"backlogSize": 0,
"type": "None",
"durable": true,
"bytesOutCounter": 0,
"totalMsgExpired": 0,
"nonContiguousDeletedMessagesRanges": 0,
"consumersAfterMarkDeletePosition": {
},
"consumers": [
],
"blockedSubscriptionOnUnackedMsgs": false,
"msgDelayed": 0,
"unackedMessages": 0,
"chunkedMessageRate": 0,
"msgBacklog": 0,
"lastConsumedTimestamp": 0,
"lastExpireTimestamp": 0,
"msgRateRedeliver": 0.0,
"isDurable": true,
"msgThroughputOut": 0.0,
"earliestMsgPublishTimeInBacklog": 0,
"subscriptionProperties": {
},
"isReplicated": false,
"allowOutOfOrderDelivery": false,
"lastAckedTimestamp": 0,
"msgBacklogNoDelayed": 0,
"replicated": false,
"msgRateOut": 0.0
}
},
"nonContiguousDeletedMessagesRangesSerializedSize": 0,
"lastOffloadFailureTimeStamp": 0,
"msgOutCounter": 0,
"backlogSize": 0,
"bytesInCounter": 0,
"lastOffloadLedgerId": 0,
"bytesOutCounter": 0,
"msgInCounter": 0,
"nonContiguousDeletedMessagesRanges": 0,
"publishers": [
],
"compaction": {
"lastCompactionDurationTimeInMills": 0,
"lastCompactionRemovedEventCount": 0,
"lastCompactionSucceedTimestamp": 0,
"lastCompactionFailedTimestamp": 0
},
"averageMsgSize": 0.0,
"msgRateIn": 0.0,
"replication": {
},
"earliestMsgPublishTimeInBacklogs": 0,
"msgChunkPublished": false,
"msgThroughputIn": 0.0,
"msgThroughputOut": 0.0,
"publishRateLimitedTimes": 0,
"waitingPublishers": 0,
"storageSize": 0,
"deduplicationStatus": "Disabled",
"lastOffloadSuccessTimeStamp": 0,
"msgRateOut": 0.0
}
```
### 7. a few secods later, producer and consumer will auto registry again
```
GET http://127.0.0.1:8080/admin/v2/persistent/public/default/my-topic/stats
response:
{
"offloadedStorageSize": 0,
"subscriptions": {
"my-subscription": {
"lastConsumedFlowTimestamp": 1648636819910,
"nonContiguousDeletedMessagesRangesSerializedSize": 0,
"msgRateExpired": 0.0,
"lastMarkDeleteAdvancedTimestamp": 0,
"msgOutCounter": 0,
"backlogSize": 0,
"type": "Exclusive",
"durable": true,
"bytesOutCounter": 0,
"totalMsgExpired": 0,
"nonContiguousDeletedMessagesRanges": 0,
"consumersAfterMarkDeletePosition": {
},
"consumers": [
{
"metadata": {
},
"address": "\/127.0.0.1:61469",
"msgRateRedeliver": 0.0,
"avgMessagesPerEntry": 0,
"msgOutCounter": 0,
"clientVersion": "2.10.0-SNAPSHOT",
"msgThroughputOut": 0.0,
"availablePermits": 1000,
"bytesOutCounter": 0,
"connectedSince": "2022-03-30T18:40:19.9067263+08:00",
"blockedConsumerOnUnackedMsgs": false,
"lastAckedTimestamp": 0,
"unackedMessages": 0,
"chunkedMessageRate": 0.0,
"msgRateOut": 0.0,
"consumerName": "682bf",
"lastConsumedTimestamp": 0
}
],
"blockedSubscriptionOnUnackedMsgs": false,
"msgDelayed": 0,
"unackedMessages": 0,
"chunkedMessageRate": 0,
"msgBacklog": 0,
"lastConsumedTimestamp": 0,
"activeConsumerName": "682bf",
"lastExpireTimestamp": 0,
"msgRateRedeliver": 0.0,
"isDurable": true,
"msgThroughputOut": 0.0,
"earliestMsgPublishTimeInBacklog": 0,
"subscriptionProperties": {
},
"isReplicated": false,
"allowOutOfOrderDelivery": false,
"lastAckedTimestamp": 0,
"msgBacklogNoDelayed": 0,
"replicated": false,
"msgRateOut": 0.0
}
},
"nonContiguousDeletedMessagesRangesSerializedSize": 0,
"lastOffloadFailureTimeStamp": 0,
"msgOutCounter": 0,
"backlogSize": 0,
"bytesInCounter": 0,
"lastOffloadLedgerId": 0,
"bytesOutCounter": 0,
"msgInCounter": 0,
"nonContiguousDeletedMessagesRanges": 0,
"publishers": [
{
"msgRateIn": 0.0,
"supportsPartialProducer": false,
"metadata": {
},
"address": "\/127.0.0.1:61443",
"connectedSince": "2022-03-30T18:40:19.9057264+08:00",
"producerName": "standalone-0-0",
"averageMsgSize": 0.0,
"producerId": 0,
"accessMode": "Shared",
"clientVersion": "2.10.0-SNAPSHOT",
"msgThroughputIn": 0.0,
"chunkedMessageRate": 0.0
}
],
"compaction": {
"lastCompactionDurationTimeInMills": 0,
"lastCompactionRemovedEventCount": 0,
"lastCompactionSucceedTimestamp": 0,
"lastCompactionFailedTimestamp": 0
},
"averageMsgSize": 0.0,
"msgRateIn": 0.0,
"replication": {
},
"earliestMsgPublishTimeInBacklogs": 0,
"msgChunkPublished": false,
"msgThroughputIn": 0.0,
"msgThroughputOut": 0.0,
"publishRateLimitedTimes": 0,
"waitingPublishers": 0,
"storageSize": 0,
"deduplicationStatus": "Disabled",
"lastOffloadSuccessTimeStamp": 0,
"msgRateOut": 0.0
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] poorbarcode edited a comment on issue #14927: After using ns-isolation-policy, topics unload is invalid, only namespace unload is valid
Posted by GitBox <gi...@apache.org>.
poorbarcode edited a comment on issue #14927:
URL: https://github.com/apache/pulsar/issues/14927#issuecomment-1082913675
Hi @codelipenghui @Technoboy- Could not reproduce. It's been try in two version: tag-2.9.1 and master
Here's how I did it:
### 1. start pulsar
```
bin/pulsar standalone
```
### 2. auto create a topic: my-topic, and ensure topic exists, consumer & producer connected
```
GET http://127.0.0.1:8080/admin/v2/persistent/public/default/my-topic/stats
response:
{
"offloadedStorageSize": 0,
"subscriptions": {
"my-subscription": {
"lastConsumedFlowTimestamp": 1648636226130,
"nonContiguousDeletedMessagesRangesSerializedSize": 0,
"msgRateExpired": 0.0,
"lastMarkDeleteAdvancedTimestamp": 0,
"msgOutCounter": 0,
"backlogSize": 0,
"type": "Exclusive",
"durable": true,
"bytesOutCounter": 0,
"totalMsgExpired": 0,
"nonContiguousDeletedMessagesRanges": 0,
"consumersAfterMarkDeletePosition": {
},
"consumers": [
{
"metadata": {
},
"address": "\/127.0.0.1:61469",
"msgRateRedeliver": 0.0,
"avgMessagesPerEntry": 0,
"msgOutCounter": 0,
"clientVersion": "2.10.0-SNAPSHOT",
"msgThroughputOut": 0.0,
"availablePermits": 1000,
"bytesOutCounter": 0,
"connectedSince": "2022-03-30T18:30:26.1286191+08:00",
"blockedConsumerOnUnackedMsgs": false,
"lastAckedTimestamp": 0,
"unackedMessages": 0,
"chunkedMessageRate": 0.0,
"msgRateOut": 0.0,
"consumerName": "682bf",
"lastConsumedTimestamp": 0
}
],
"blockedSubscriptionOnUnackedMsgs": false,
"msgDelayed": 0,
"unackedMessages": 0,
"chunkedMessageRate": 0,
"msgBacklog": 0,
"lastConsumedTimestamp": 0,
"activeConsumerName": "682bf",
"lastExpireTimestamp": 0,
"msgRateRedeliver": 0.0,
"isDurable": true,
"msgThroughputOut": 0.0,
"earliestMsgPublishTimeInBacklog": 0,
"subscriptionProperties": {
},
"isReplicated": false,
"allowOutOfOrderDelivery": false,
"lastAckedTimestamp": 0,
"msgBacklogNoDelayed": 0,
"replicated": false,
"msgRateOut": 0.0
}
},
"nonContiguousDeletedMessagesRangesSerializedSize": 0,
"lastOffloadFailureTimeStamp": 0,
"msgOutCounter": 0,
"backlogSize": 0,
"bytesInCounter": 0,
"lastOffloadLedgerId": 0,
"bytesOutCounter": 0,
"msgInCounter": 0,
"nonContiguousDeletedMessagesRanges": 0,
"publishers": [
{
"msgRateIn": 0.0,
"supportsPartialProducer": false,
"metadata": {
},
"address": "\/127.0.0.1:61443",
"connectedSince": "2022-03-30T18:30:26.1256204+08:00",
"producerName": "standalone-0-0",
"averageMsgSize": 0.0,
"producerId": 0,
"accessMode": "Shared",
"clientVersion": "2.10.0-SNAPSHOT",
"msgThroughputIn": 0.0,
"chunkedMessageRate": 0.0
}
],
"compaction": {
"lastCompactionDurationTimeInMills": 0,
"lastCompactionRemovedEventCount": 0,
"lastCompactionSucceedTimestamp": 0,
"lastCompactionFailedTimestamp": 0
},
"averageMsgSize": 0.0,
"msgRateIn": 0.0,
"replication": {
},
"earliestMsgPublishTimeInBacklogs": 0,
"msgChunkPublished": false,
"msgThroughputIn": 0.0,
"msgThroughputOut": 0.0,
"publishRateLimitedTimes": 0,
"waitingPublishers": 0,
"storageSize": 0,
"deduplicationStatus": "Disabled",
"lastOffloadSuccessTimeStamp": 0,
"msgRateOut": 0.0
}
```
### 3. add a policy at namespace: default
```
POST http://127.0.0.1:8080/admin/v2/clusters/standalone/namespaceIsolationPolicies/policy_test
POST data:
{
"namespaces": ["public/default"],
"primary": ["localhost:8080"],
"secondary": [],
"auto_failover_policy": {
"policy_type": "min_available",
"parameters": {
"min_limit": 1,
"usage_threshold": 80
}
}
}
```
### 4. ensure policy add success
```
GET http://127.0.0.1:8080/admin/v2/clusters/standalone/namespaceIsolationPolicies/brokers/localhost:8080
response:
{
"policyName": "policy_test",
"namespaceRegex": [
"public\/default"
],
"brokerName": "localhost:8080",
"primary": true
}
```
### 5. do unload topic: my-topic
```
PUT http://127.0.0.1:8080/admin/v2/persistent/public/default/my-topic/unload
```
## 6. ensure topic unload success
```
GET http://127.0.0.1:8080/admin/v2/persistent/public/default/my-topic/stats
response:
{
"offloadedStorageSize": 0,
"subscriptions": {
"my-subscription": {
"lastConsumedFlowTimestamp": 0,
"nonContiguousDeletedMessagesRangesSerializedSize": 0,
"msgRateExpired": 0.0,
"lastMarkDeleteAdvancedTimestamp": 0,
"msgOutCounter": 0,
"backlogSize": 0,
"type": "None",
"durable": true,
"bytesOutCounter": 0,
"totalMsgExpired": 0,
"nonContiguousDeletedMessagesRanges": 0,
"consumersAfterMarkDeletePosition": {
},
"consumers": [
],
"blockedSubscriptionOnUnackedMsgs": false,
"msgDelayed": 0,
"unackedMessages": 0,
"chunkedMessageRate": 0,
"msgBacklog": 0,
"lastConsumedTimestamp": 0,
"lastExpireTimestamp": 0,
"msgRateRedeliver": 0.0,
"isDurable": true,
"msgThroughputOut": 0.0,
"earliestMsgPublishTimeInBacklog": 0,
"subscriptionProperties": {
},
"isReplicated": false,
"allowOutOfOrderDelivery": false,
"lastAckedTimestamp": 0,
"msgBacklogNoDelayed": 0,
"replicated": false,
"msgRateOut": 0.0
}
},
"nonContiguousDeletedMessagesRangesSerializedSize": 0,
"lastOffloadFailureTimeStamp": 0,
"msgOutCounter": 0,
"backlogSize": 0,
"bytesInCounter": 0,
"lastOffloadLedgerId": 0,
"bytesOutCounter": 0,
"msgInCounter": 0,
"nonContiguousDeletedMessagesRanges": 0,
"publishers": [
],
"compaction": {
"lastCompactionDurationTimeInMills": 0,
"lastCompactionRemovedEventCount": 0,
"lastCompactionSucceedTimestamp": 0,
"lastCompactionFailedTimestamp": 0
},
"averageMsgSize": 0.0,
"msgRateIn": 0.0,
"replication": {
},
"earliestMsgPublishTimeInBacklogs": 0,
"msgChunkPublished": false,
"msgThroughputIn": 0.0,
"msgThroughputOut": 0.0,
"publishRateLimitedTimes": 0,
"waitingPublishers": 0,
"storageSize": 0,
"deduplicationStatus": "Disabled",
"lastOffloadSuccessTimeStamp": 0,
"msgRateOut": 0.0
}
```
### 7. a few secods later, producer and consumer will auto registry again
```
GET http://127.0.0.1:8080/admin/v2/persistent/public/default/my-topic/stats
response:
{
"offloadedStorageSize": 0,
"subscriptions": {
"my-subscription": {
"lastConsumedFlowTimestamp": 1648636819910,
"nonContiguousDeletedMessagesRangesSerializedSize": 0,
"msgRateExpired": 0.0,
"lastMarkDeleteAdvancedTimestamp": 0,
"msgOutCounter": 0,
"backlogSize": 0,
"type": "Exclusive",
"durable": true,
"bytesOutCounter": 0,
"totalMsgExpired": 0,
"nonContiguousDeletedMessagesRanges": 0,
"consumersAfterMarkDeletePosition": {
},
"consumers": [
{
"metadata": {
},
"address": "\/127.0.0.1:61469",
"msgRateRedeliver": 0.0,
"avgMessagesPerEntry": 0,
"msgOutCounter": 0,
"clientVersion": "2.10.0-SNAPSHOT",
"msgThroughputOut": 0.0,
"availablePermits": 1000,
"bytesOutCounter": 0,
"connectedSince": "2022-03-30T18:40:19.9067263+08:00",
"blockedConsumerOnUnackedMsgs": false,
"lastAckedTimestamp": 0,
"unackedMessages": 0,
"chunkedMessageRate": 0.0,
"msgRateOut": 0.0,
"consumerName": "682bf",
"lastConsumedTimestamp": 0
}
],
"blockedSubscriptionOnUnackedMsgs": false,
"msgDelayed": 0,
"unackedMessages": 0,
"chunkedMessageRate": 0,
"msgBacklog": 0,
"lastConsumedTimestamp": 0,
"activeConsumerName": "682bf",
"lastExpireTimestamp": 0,
"msgRateRedeliver": 0.0,
"isDurable": true,
"msgThroughputOut": 0.0,
"earliestMsgPublishTimeInBacklog": 0,
"subscriptionProperties": {
},
"isReplicated": false,
"allowOutOfOrderDelivery": false,
"lastAckedTimestamp": 0,
"msgBacklogNoDelayed": 0,
"replicated": false,
"msgRateOut": 0.0
}
},
"nonContiguousDeletedMessagesRangesSerializedSize": 0,
"lastOffloadFailureTimeStamp": 0,
"msgOutCounter": 0,
"backlogSize": 0,
"bytesInCounter": 0,
"lastOffloadLedgerId": 0,
"bytesOutCounter": 0,
"msgInCounter": 0,
"nonContiguousDeletedMessagesRanges": 0,
"publishers": [
{
"msgRateIn": 0.0,
"supportsPartialProducer": false,
"metadata": {
},
"address": "\/127.0.0.1:61443",
"connectedSince": "2022-03-30T18:40:19.9057264+08:00",
"producerName": "standalone-0-0",
"averageMsgSize": 0.0,
"producerId": 0,
"accessMode": "Shared",
"clientVersion": "2.10.0-SNAPSHOT",
"msgThroughputIn": 0.0,
"chunkedMessageRate": 0.0
}
],
"compaction": {
"lastCompactionDurationTimeInMills": 0,
"lastCompactionRemovedEventCount": 0,
"lastCompactionSucceedTimestamp": 0,
"lastCompactionFailedTimestamp": 0
},
"averageMsgSize": 0.0,
"msgRateIn": 0.0,
"replication": {
},
"earliestMsgPublishTimeInBacklogs": 0,
"msgChunkPublished": false,
"msgThroughputIn": 0.0,
"msgThroughputOut": 0.0,
"publishRateLimitedTimes": 0,
"waitingPublishers": 0,
"storageSize": 0,
"deduplicationStatus": "Disabled",
"lastOffloadSuccessTimeStamp": 0,
"msgRateOut": 0.0
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org