[GitHub] [pulsar] dhineshthiruppathi edited a discussion: Messages are not deleted after ledger rollover when retention in size limits are achieved

[GitBox <gi...@apache.org>]

GitHub user dhineshthiruppathi edited a discussion: Messages are not deleted after ledger rollover when retention in size limits are achieved

I have set Message retention limits for 5 mins of time and 5MB of size as shown below on a namespace,

```
bin/pulsar-admin namespaces get-retention sales/supplier
{
  "retentionTimeInMinutes" : 5,
  "retentionSizeInMB" : 5
}
```

Also at broker broker level we have the defaults for below parameters,

```
managedLedgerMaxEntriesPerLedger=50000
managedLedgerMinLedgerRolloverTimeMinutes=10
managedLedgerMaxLedgerRolloverTimeMinutes=240
```

We use below commands to produce and consume messages respectively,

```
1. bin/pulsar-perf produce persistent://sales/supplier/sample-topic -m 80000 -s 1024 -r 1000 --> goes to ledgerID-1
2. bin/pulsar-perf consume persistent://sales/supplier/sample-topic
3. bin/pulsar-perf produce persistent://sales/supplier/sample-topic -m 50 -s 1024 -r 1000 --> roll over to ledgerID-2
```

As per our observations, the ledgerID-1 was greater than 5 MB and messages were marked for deletion on all messages in this ledger after first publish. However it wasn't deleted though message retention size limit(5MB) is reached. As soon as we published another set of messages, ledgers got rolled over and created ledgerID-2 and waited till the retention time limit(5 Minutes) is achieved for ledgerID-1 to be deleted. Here we see no effect of retention using size limit. 

Kindly help us understanding.

Regards, Dhines

GitHub link: https://github.com/apache/pulsar/discussions/19076

----
This is an automatically sent email for commits@pulsar.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@pulsar.apache.org