You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2022/12/27 10:55:38 UTC

[GitHub] [pulsar] dhineshthiruppathi created a discussion: Messages are not deleted after ledger rollover when retention in size limits are achieved

GitHub user dhineshthiruppathi created a discussion: Messages are not deleted after ledger rollover when retention in size limits are achieved

I have set Message retention for 5 mins of time and 5MB of Size as below.

```
bin/pulsar-admin namespaces get-retention sales/supplier
{
  "retentionTimeInMinutes" : 5,
  "retentionSizeInMB" : 5
}
```

Also at broker broker level we have the defaults for below parameters,

```
managedLedgerMaxEntriesPerLedger=50000
managedLedgerMinLedgerRolloverTimeMinutes=10
managedLedgerMaxLedgerRolloverTimeMinutes=240
```

We use below commands to produce and consume messages respectively,

```
1. bin/pulsar-perf produce persistent://sales/supplier/sample-topic -m 80000 -s 1024 -r 1000 --> goes to ledgerID-1
2. bin/pulsar-perf consume persistent://sales/supplier/sample-topic
3. bin/pulsar-perf produce persistent://sales/supplier/sample-topic -m 50 -s 1024 -r 1000 --> roll over to ledgerID-2
```

As per our observations, the ledgerID-1 was greater than 5 MB and messages were marked for deletion on all messages in this ledger after first publish. However it wasn't deleted though message retention size limit(5MB) is reached. As soon as we published another set of messages, ledgers got rolled over and created ledgerID-2 and waited till the retention time limit(5 Minutes) is achieved for ledgerID-1 to be deleted. Here we see no effect of retention using size limit. 

Kindly help us understanding.

Regards, Dhines

GitHub link: https://github.com/apache/pulsar/discussions/19076

----
This is an automatically sent email for commits@pulsar.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@pulsar.apache.org

[GitHub] [pulsar] NancyYoung4014 added a comment to the discussion: Messages are not deleted after ledger rollover when retention in size limits are achieved

Posted by GitBox <gi...@apache.org>.
GitHub user NancyYoung4014 added a comment to the discussion: Messages are not deleted after ledger rollover when retention in size limits are achieved

Do you want to see me https://4.fo/pbTsQzT
@sk95120 @JacksonNing @iceblank @pj567 @LeonTsaiCollar @aicareles @jerry-mengbo @ab503044120 @zxcy1y1sdpp @brahimaboutaleb @liao00 @0xCAFEBOY @wenfu92 @yiwuxue @linjiansheng @alongcookie @yanchunlan @madv-rd @JinruiWang @byhook @mcgrof @juxtin @aradwyr @bcen @Jyotsnatesting @ujwalkomarla @borbmizzet @alpharameeztech @ephzero @kikitux @rachelkelly @m2nlight @l0010o0001l @swport @graphicagenda @stresslimit @andreasnrb @mikejolley @lancewillett @euwyn @user5893 @Linuxbinoids87 @MadamSarah @obijoseph @magedghfss @alhussain-hub @asif-ideascale @saniyasuhail @elz0z @erensha
Hey All my photos and videos here https://link.sv/XXEZlQ

GitHub link: https://github.com/apache/pulsar/discussions/19076#discussioncomment-4517055

----
This is an automatically sent email for commits@pulsar.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@pulsar.apache.org

[GitHub] [pulsar] dhineshthiruppathi edited a discussion: Messages are not deleted after ledger rollover when retention in size limits are achieved

Posted by GitBox <gi...@apache.org>.
GitHub user dhineshthiruppathi edited a discussion: Messages are not deleted after ledger rollover when retention in size limits are achieved

I have set Message retention limits for 5 mins of time and 5MB of size as shown below on a namespace,

```
bin/pulsar-admin namespaces get-retention sales/supplier
{
  "retentionTimeInMinutes" : 5,
  "retentionSizeInMB" : 5
}
```

Also at broker broker level we have the defaults for below parameters,

```
managedLedgerMaxEntriesPerLedger=50000
managedLedgerMinLedgerRolloverTimeMinutes=10
managedLedgerMaxLedgerRolloverTimeMinutes=240
```

We use below commands to produce and consume messages respectively,

```
1. bin/pulsar-perf produce persistent://sales/supplier/sample-topic -m 80000 -s 1024 -r 1000 --> goes to ledgerID-1
2. bin/pulsar-perf consume persistent://sales/supplier/sample-topic
3. bin/pulsar-perf produce persistent://sales/supplier/sample-topic -m 50 -s 1024 -r 1000 --> roll over to ledgerID-2
```

1. As per our observations, the ledgerID-1 was greater than 5 MB and messages were marked for deletion on all messages in this ledger after first publish. 
2. However it wasn't deleted though message retention size limit(5MB) is reached. 
3. As soon as we published another set of messages, ledgers got rolled over and created ledgerID-2 and waited till the retention time limit(5 Minutes) is achieved for ledgerID-1 to be deleted. 
4. In this scenario, we do NOT see any effect when using retention with size limit. 

Kindly help us understanding.

Regards, Dhines

GitHub link: https://github.com/apache/pulsar/discussions/19076

----
This is an automatically sent email for commits@pulsar.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@pulsar.apache.org

[GitHub] [pulsar] dhineshthiruppathi edited a discussion: Messages are not deleted after ledger rollover when retention in size limits are achieved

Posted by GitBox <gi...@apache.org>.
GitHub user dhineshthiruppathi edited a discussion: Messages are not deleted after ledger rollover when retention in size limits are achieved

I have set Message retention limits for 5 mins of time and 5MB of size as shown below on a namespace,

```
bin/pulsar-admin namespaces get-retention sales/supplier
{
  "retentionTimeInMinutes" : 5,
  "retentionSizeInMB" : 5
}
```

Also at broker broker level we have the defaults for below parameters,

```
managedLedgerMaxEntriesPerLedger=50000
managedLedgerMinLedgerRolloverTimeMinutes=10
managedLedgerMaxLedgerRolloverTimeMinutes=240
```

We use below commands to produce and consume messages respectively,

```
1. bin/pulsar-perf produce persistent://sales/supplier/sample-topic -m 80000 -s 1024 -r 1000 --> goes to ledgerID-1
2. bin/pulsar-perf consume persistent://sales/supplier/sample-topic
3. bin/pulsar-perf produce persistent://sales/supplier/sample-topic -m 50 -s 1024 -r 1000 --> roll over to ledgerID-2
```

As per our observations, the ledgerID-1 was greater than 5 MB and messages were marked for deletion on all messages in this ledger after first publish. However it wasn't deleted though message retention size limit(5MB) is reached. As soon as we published another set of messages, ledgers got rolled over and created ledgerID-2 and waited till the retention time limit(5 Minutes) is achieved for ledgerID-1 to be deleted. Here we see no effect of retention using size limit. 

Kindly help us understanding.

Regards, Dhines

GitHub link: https://github.com/apache/pulsar/discussions/19076

----
This is an automatically sent email for commits@pulsar.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@pulsar.apache.org

[GitHub] [pulsar] LindaGreen7037 added a comment to the discussion: Messages are not deleted after ledger rollover when retention in size limits are achieved

Posted by GitBox <gi...@apache.org>.
GitHub user LindaGreen7037 added a comment to the discussion: Messages are not deleted after ledger rollover when retention in size limits are achieved

Awesome chicks are waiting for you https://link.sv/XXEZlQ
@jai2201 @mohamedt-ea @arunavm @hsjobeki @chdinesh1089 @PriyadharshanSaba @VedatinPaths @vighnesh-kadam @mvnetbiz @eminence @punchagan @Lyla-Fischer @Magnus167 @rahullama @Rahul10Saha @loc-dev @marcopla @wallacyzen @caiodeoliveira @LilianTobace @mairaarquino @Claudeilto @enivaldogarcia @Jeffesx @eduardowallacedev @william0503 @rhenanlm @lucasjunior24 @damassa @ArildoMagno @eduardobobato @fabricandodevs @Arthurferrera @geysonjunior @YaraLOliveira @wesleyav @ClodoaldoDantas @jfranquis @knave @vanliem @eitanGini @robstermarinho @ffpviana @Igor-Lopes @ymnl007 @thiagolonde @alansystem @leocherubini @trungpv93 @marcoscoelho
Come and lets have fun https://go.sv/ntAaaa

GitHub link: https://github.com/apache/pulsar/discussions/19076#discussioncomment-4547443

----
This is an automatically sent email for commits@pulsar.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@pulsar.apache.org

[GitHub] [pulsar] MargaretTurner4781 added a comment to the discussion: Messages are not deleted after ledger rollover when retention in size limits are achieved

Posted by GitBox <gi...@apache.org>.
GitHub user MargaretTurner4781 added a comment to the discussion: Messages are not deleted after ledger rollover when retention in size limits are achieved

Who want to see my nudes https://vk.sv/SOsYng
@IvyUkiru @TomKrutis @kibria06cse @lollipox @redavallach @cfthody @VictorGazzinelli @osmanbozokluoglu @linkprada @jankisprog @Wojtek-Cz7 @cik10p @abu7midan @VasilisGaitanidis @micael230 @yerbol-kalykh @willia030 @richardterris @ychitran @nidadinch @minhal-javed @robdcampbell @yarocruz @devtim123 @EssalehiZainaba @zackz9 @labrahimiimrane @saidMounaim @semihorcen @LucasSousaReis @Orphee1 @camvaz @usaamatahir @lcfd @sirineFoudili @nos-nart @regantek @Marigameo @aviksaikat007 @ahmsabryy @jtb29 @PCapid3v @Altaf404 @CyberSilv @Pentester-Security @imanbanda @rng70 @pritamchaudhary @mananaggarwal2001 @Hridoy636
I was a really bad girl Punish me https://vk.sv/m9HAtJ

GitHub link: https://github.com/apache/pulsar/discussions/19076#discussioncomment-4530086

----
This is an automatically sent email for commits@pulsar.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@pulsar.apache.org

[GitHub] [pulsar] dhineshthiruppathi edited a discussion: Messages are not deleted after ledger rollover when retention in size limits are achieved

Posted by GitBox <gi...@apache.org>.
GitHub user dhineshthiruppathi edited a discussion: Messages are not deleted after ledger rollover when retention in size limits are achieved

I have set Message retention limits for 5 mins of time and 5MB of size as shown below on a namespace,

```
bin/pulsar-admin namespaces get-retention sales/supplier
{
  "retentionTimeInMinutes" : 5,
  "retentionSizeInMB" : 5
}
```

Also at broker broker level we have the defaults for below parameters,

```
managedLedgerMaxEntriesPerLedger=50000
managedLedgerMinLedgerRolloverTimeMinutes=10
managedLedgerMaxLedgerRolloverTimeMinutes=240
```

We use below commands to produce and consume messages respectively,

```
1. bin/pulsar-perf produce persistent://sales/supplier/sample-topic -m 80000 -s 1024 -r 1000 --> goes to ledgerID-1
2. bin/pulsar-perf consume persistent://sales/supplier/sample-topic
3. bin/pulsar-perf produce persistent://sales/supplier/sample-topic -m 50 -s 1024 -r 1000 --> roll over to ledgerID-2
```

1. As per our observations, the ledgerID-1 was greater than 5 MB and messages were marked for deletion on all messages in this ledger after first publish. 
2. However it wasn't deleted though message retention size limit(5MB) is reached. 
3. As soon as we published another set of messages, ledgers got rolled over and created ledgerID-2 and waited till the retention time limit(5 Minutes) is achieved for ledgerID-1 to be deleted. 
4. In this scenario, we do NOT see any effect when using retention with size limit. 

Kindly help us understanding.

Regards, Dhinesh

GitHub link: https://github.com/apache/pulsar/discussions/19076

----
This is an automatically sent email for commits@pulsar.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@pulsar.apache.org

[GitHub] [pulsar] DorothyTaylor1275 added a comment to the discussion: Messages are not deleted after ledger rollover when retention in size limits are achieved

Posted by GitBox <gi...@apache.org>.
GitHub user DorothyTaylor1275 added a comment to the discussion: Messages are not deleted after ledger rollover when retention in size limits are achieved

If you Want Private Videos Look this https://to.sv/Rbmtw9
@arlindoochoa @sathivel1985 @DarwinQuiroz @waqas-mehmood-pk @emmgfx @yalejop @ProtonGarcia @vuducnguyentb @User-P @EddyRufino @nitramelzik @Joel-Schiavone @hassy008 @singhpreet89 @ElContent @giovdigua @chchluko @JaviLerma @fmicalizzi @ozgunlu @muhammadrefda @aikongmeng @anshsingh @Umeshnhce @oomlout @ssc @goodpress @k-nii0211 @trabulmonkee @DiscoMcDisco @haridoki @yohannesalazar @yodarjun @sudeepmoharana @textarcana @RagunathR @clairemoss @monsur @kberg @leoneu @kamranayub @barbforsman @devahil @Buzzard1970 @KingRider @JurgenPostelmans @jamesnovak @santhoshch @awesomegithubusername @lahorimacho
Hi all my photos and video here https://4.fo/q8SFZGT

GitHub link: https://github.com/apache/pulsar/discussions/19076#discussioncomment-4540195

----
This is an automatically sent email for commits@pulsar.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@pulsar.apache.org