[GitHub] [pulsar] trexinc commented on issue #6935: The retention is not working due to non closing ledger

[GitBox <gi...@apache.org>]

trexinc commented on issue #6935:
URL: https://github.com/apache/pulsar/issues/6935#issuecomment-633184166


   Simple instruction to replicate issue (replicates in 2.5.2 as well):
   Create namespace internal/test with the following retention and TTL:
   
   > bin/pulsar-admin namespaces get-retention internal/test
   > {
   >   "retentionTimeInMinutes" : 60,
   >   "retentionSizeInMB" : 5120
   > }
   > 
   > bin/pulsar-admin namespaces get-message-ttl internal/test
   > 600
   
   Then run in parallel for about five minutes the following (you can add -m 60000 to the producer):
   > bin/pulsar-perf produce -r 40000 internal/test/test1
   > bin/pulsar-client consume -s test internal/test/test1
   
   The topic will then remain like this forever:
   
   > bin/pulsar-admin topics stats-internal internal/test/test1
   > {
   >   "entriesAddedCounter" : 64431,
   >   "numberOfEntries" : 64431,
   >   "totalSize" : 6548489946,
   >   "currentLedgerEntries" : 64431,
   >   "currentLedgerSize" : 6548489946,
   >   "lastLedgerCreatedTimestamp" : "2020-05-23T04:42:56.047Z",
   >   "waitingCursorsCount" : 0,
   >   "pendingAddEntriesCount" : 0,
   >   "lastConfirmedEntry" : "182338:64430",
   >   "state" : "LedgerOpened",
   >   "ledgers" : [ {
   >     "ledgerId" : 182338,
   >     "entries" : 0,
   >     "size" : 0,
   >     "offloaded" : false
   >   } ],
   >   "cursors" : {
   >     "sub" : {
   >       "markDeletePosition" : "182338:64430",
   >       "readPosition" : "182338:64431",
   >       "waitingReadOp" : false,
   >       "pendingReadOps" : 0,
   >       "messagesConsumedCounter" : 64431,
   >       "cursorLedger" : 182339,
   >       "cursorLedgerLastEntry" : 108,
   >       "individuallyDeletedMessages" : "[]",
   >       "lastLedgerSwitchTimestamp" : "2020-05-23T04:42:56.049Z",
   >       "state" : "Open",
   >       "numberOfEntriesSinceFirstNotAckedMessage" : 1,
   >       "totalNonContiguousDeletedMessagesRange" : 0,
   >       "properties" : { }
   >     }
   >   }
   > }
   > 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org