You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2023/01/18 04:14:22 UTC
[GitHub] [pulsar] Debashish-Mallick opened a new issue, #19267: backlog quota is not reflecting on topic level
Debashish-Mallick opened a new issue, #19267:
URL: https://github.com/apache/pulsar/issues/19267
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
### Version
Pulsar version : 2.8.2
I am using below code to set backlog quota on topic level.
BacklogQuota quota= BacklogQuota.builder().limitSize(sizeLimit).retentionPolicy(BacklogQuota.RetentionPolicy.producer_exception).build();
pulsarAdmin.topics().setBacklogQuota("testtopic",quota);
In broker log Its showing Its successfully set but If I am trying to get the backlog quota value ,Its not showing .This behavior is not consistent , Once It shows for 1 topic , going forward for other topics its not getting set .
### Minimal reproduce step
Crete multiple topics they try to set topic level backlog quota for each newly created topics .
### What did you expect to see?
In broker log Its saying successfully set whereas in get-backlog-quotas shows empty. So the backlog quota is not going to honor for topic.
**broker log:**
2023-01-17T14:22:10,927 [pulsar-io-4-6] INFO org.apache.pulsar.broker.admin.impl.PersistentTopicsBase - [itom-di-message-bus-svc] Successfully updated backlog quota map: namespace=public/default, topic=demo7, map={"destination_storage":{"limit":349525,"limitSize":349525,"limitTime":-1,"policy":"producer_exception"}}
2023-01-17T14:22:10,928 [pulsar-io-4-7] INFO org.apache.pulsar.broker.service.ServerCnx - [SystemTopic{topic=persistent://public/default/__change_events}][itomdipulsar-2-831] Closing producer on cnx /172.16.2.92:39646. producerId=306
2023-01-17T14:22:10,928 [pulsar-io-4-7] INFO org.apache.pulsar.broker.service.ServerCnx - [SystemTopic{topic=persistent://public/default/__change_events}][itomdipulsar-2-831] Closed producer on cnx /172.16.2.92:39646. producerId=306
2023-01-17T14:22:10,928 [pulsar-io-4-6] INFO org.apache.pulsar.client.impl.ProducerImpl - [persistent://public/default/__change_events] [itomdipulsar-2-831] Closed Producer
2023-01-17T14:22:11,281 [pulsar-2-4] INFO org.apache.pulsar.broker.systopic.NamespaceEventsSystemTopicFactory - Create topic policies system topic client persistent://public/default/__change_events
2023-01-17T14:22:11,308 [pulsar-io-4-6] INFO org.apache.pulsar.client.impl.ProducerImpl - [persistent://public/default/__change_events] [null] Creating producer on cnx [id: 0xc16d3c33, L:/172.16.2.92:39646 - R:172.16.2.92/172.16.2.92:6651]
2023-01-17T14:22:11,309 [pulsar-io-4-7] INFO org.apache.pulsar.broker.service.ServerCnx - [/172.16.2.92:39646][persistent://public/default/__change_events] Creating producer. producerId=307
2023-01-17T14:22:11,319 [BookKeeperClientWorker-OrderedExecutor-1-0] INFO org.apache.pulsar.broker.service.ServerCnx - [/172.16.2.92:39646] Created new producer: Producer{topic=SystemTopic{topic=persistent://public/default/__change_events}, client=/172.16.2.92:39646, producerName=itomdipulsar-2-832, producerId=307}
2023-01-17T14:22:11,320 [pulsar-io-4-6] INFO org.apache.pulsar.client.impl.ProducerImpl - [persistent://public/default/__change_events] [itomdipulsar-2-832] Created producer on cnx [id: 0xc16d3c33, L:/172.16.2.92:39646 - R:172.16.2.92/172.16.2.92:6651]
2023-01-17T14:22:11,322 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/__change_events] Creating a new ledger
2023-01-17T14:22:11,322 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/__change_events] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[112, 117, 98, 108, 105, 99, 47, 100, 101, 102, 97, 117, 108, 116, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 95, 95, 99, 104, 97, 110, 103, 101, 95, 101, 118, 101, 110, 116, 115], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds
2023-01-17T14:22:11,338 [main-EventThread] INFO org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [itomdipulsar-bookkeeper-1.itomdipulsar-bookkeeper.core-helm.svc.cluster.local.:3181, itomdipulsar-bookkeeper-2.itomdipulsar-bookkeeper.core-helm.svc.cluster.local.:3181] for ledger: 250
2023-01-17T14:22:11,339 [main-EventThread] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/__change_events] Created new ledger 250
2023-01-17T14:22:11,349 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.zookeeper.ZooKeeperCache - [State:CONNECTED Timeout:30000 sessionid:0x300294417740015 local:/172.16.2.92:49624 remoteserver:itomdipulsar-zookeeper/172.16.2.93:2281 lastZxid:4294970959 xid:13128 sent:13128 recv:13244 queuedpkts:1 pendingresp:0 queuedevents:0] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/managed-ledgers/public/default/persistent/__change_events
2023-01-17T14:22:11,350 [bookkeeper-ml-scheduler-OrderedScheduler-4-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/__change_events, name=__compaction, ackPos=195:18, readPos=195:19} to 200:-1 since ledger consumed completely
2023-01-17T14:22:11,351 [bookkeeper-ml-scheduler-OrderedScheduler-4-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/__change_events, name=pulsar.dedup, ackPos=195:18, readPos=195:19} to 200:-1 since ledger consumed completely
2023-01-17T14:22:11,351 [bookkeeper-ml-scheduler-OrderedScheduler-4-0] WARN org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Cursor: NonDurableCursorImpl{ledger=public/default/persistent/__change_events, ackPos=134:2, readPos=200:1} does not exist in the managed-ledger.
### What did you see instead?
get-backlog-quotas shouldn't be empty and backlog quota is should honor for topics.
### Anything else?
Attached system topic stats.
[system_topic_stats.zip](https://github.com/apache/pulsar/files/10442050/system_topic_stats.zip)
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on issue #19267: backlog quota is not reflecting on topic level
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #19267:
URL: https://github.com/apache/pulsar/issues/19267#issuecomment-1435815923
The issue had no activity for 30 days, mark with Stale label.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] backlog quota is not reflecting on topic level [pulsar]
Posted by "merlimat (via GitHub)" <gi...@apache.org>.
merlimat commented on issue #19267:
URL: https://github.com/apache/pulsar/issues/19267#issuecomment-6386665595
Thanks @Debashish-Mallick for the report. This was reported on Pulsar 2.8, which is no longer supported; the supported releases are 4.0.x, 4.2.x and 5.0.x. Much of the code involved has changed since, and the problem hasn't been confirmed on a current release, so I'm closing it as part of cleaning up old reports. If you still see it on a supported release, please open a new issue with the version and steps to reproduce, and link this one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] sindhushreem commented on issue #19267: backlog quota is not reflecting on topic level
Posted by "sindhushreem (via GitHub)" <gi...@apache.org>.
sindhushreem commented on issue #19267:
URL: https://github.com/apache/pulsar/issues/19267#issuecomment-1521521948
@sijie can some one have help us in this issue ,its quite often seen in 2.8.2
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Debashish-Mallick commented on issue #19267: backlog quota is not reflecting on topic level
Posted by "Debashish-Mallick (via GitHub)" <gi...@apache.org>.
Debashish-Mallick commented on issue #19267:
URL: https://github.com/apache/pulsar/issues/19267#issuecomment-1442042018
hi Team,
Could anyone help me to get further an update on this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Debashish-Mallick commented on issue #19267: backlog quota is not reflecting on topic level
Posted by GitBox <gi...@apache.org>.
Debashish-Mallick commented on issue #19267:
URL: https://github.com/apache/pulsar/issues/19267#issuecomment-1396913622
> Could you provide the dump file of the issued broker?
Reproduced again and sharing the logs and dumps.
[backlog_quota_ingetion_rate_issue_1.zip](https://github.com/apache/pulsar/files/10456703/backlog_q
[backlog_quota_ingetion_rate_issue_2.zip](https://github.com/apache/pulsar/files/10456708/backlog_quota_ingetion_rate_issue_2.zip)
uota_ingetion_rate_issue_1.zip)
[backlog_quota_ingetion_rate_issue_3.zip](https://github.com/apache/pulsar/files/10456722/backlog_quota_ingetion_rate_issue_3.zip)
[backlog_quota_ingetion_rate_issue_4.zip](https://github.com/apache/pulsar/files/10456749/backlog_quota_ingetion_rate_issue_4.zip)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] backlog quota is not reflecting on topic level [pulsar]
Posted by "merlimat (via GitHub)" <gi...@apache.org>.
merlimat closed issue #19267: backlog quota is not reflecting on topic level
URL: https://github.com/apache/pulsar/issues/19267
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on issue #19267: backlog quota is not reflecting on topic level
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #19267:
URL: https://github.com/apache/pulsar/issues/19267#issuecomment-1484356521
The issue had no activity for 30 days, mark with Stale label.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on issue #19267: backlog quota is not reflecting on topic level
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #19267:
URL: https://github.com/apache/pulsar/issues/19267#issuecomment-1563711437
The issue had no activity for 30 days, mark with Stale label.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Technoboy- commented on issue #19267: backlog quota is not reflecting on topic level
Posted by GitBox <gi...@apache.org>.
Technoboy- commented on issue #19267:
URL: https://github.com/apache/pulsar/issues/19267#issuecomment-1396378853
Could you provide the dump file of the issued broker?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org