You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/09/22 08:57:01 UTC
[GitHub] [pulsar] aloyszhang opened a new issue #12131: broker not clean ledger from metastore successfully
aloyszhang opened a new issue #12131:
URL: https://github.com/apache/pulsar/issues/12131
**Describe the bug**
In our cluster, some ledger disks are used up(data4) but others are not.
```shell
Filesystem Size Used Avail Use% Mounted on
/dev/sdb1 1.8T 14G 1.7T 1% /data1
/dev/sdc1 1.8T 13G 1.7T 1% /data2
/dev/sdd1 1.8T 13G 1.7T 1% /data3
/dev/sde1 1.8T 1.7T 13G 100% /data4
/dev/sdf1 1.8T 316G 1.4T 19% /data5
/dev/sdg1 1.8T 329G 1.4T 19% /data6
/dev/sdh1 1.8T 486G 1.3T 28% /data7
/dev/sdi1 1.8T 316G 1.4T 19% /data8
/dev/sdj1 1.8T 319G 1.4T 19% /data9
/dev/sdk1 1.8T 323G 1.4T 19% /data10
/dev/sdl1 1.8T 383G 1.4T 22% /data11
/dev/sdm1 1.8T 1.1G 1.7T 1% /data12
```
After list the ledger file, the oldest file are created two monthes ago.
```shell
-rw-r--r-- 1 1073645320 Jun 15 19:02 367.log
-rw-r--r-- 1 1073666704 Jun 15 19:03 368.log
-rw-r--r-- 1 1073662419 Jun 15 19:15 36e.log
```
Then read the first file by `bin/bookkeeper shell readlog 367.log` and only on ledger 1110144 in this file. This ledger can be also found on the metastore.
And, I write a utils to read data from zk and get the meta info for this ledger
```shell
meta key pulsar/managed-ledger and meta value infosec/InterfaceGroup/persistent/gj630_msgcloud_delay_1-partition-18
```
This means the ledger belongs to topic ` infosec/InterfaceGroup/gj630_msgcloud_delay_1-partition-18`. And get the internal stats for this topic
```shell
{
"entriesAddedCounter" : 0,
"numberOfEntries" : 1099,
"totalSize" : 3108981,
"currentLedgerEntries" : 0,
"currentLedgerSize" : 0,
"lastLedgerCreatedTimestamp" : "2021-09-22T09:00:52.171+08:00",
"waitingCursorsCount" : 0,
"pendingAddEntriesCount" : 0,
"lastConfirmedEntry" : "1444577:1098",
"state" : "LedgerOpened",
"ledgers" : [ {
"ledgerId" : 1444577,
"entries" : 1099,
"size" : 3108981,
"offloaded" : false
}, {
"ledgerId" : 3525781,
"entries" : 0,
"size" : 0,
"offloaded" : false
} ],
```
Additionally, the TTL for the topic is 14400 seconds(1 day)
Retention size is -1 and retention time is 240 minutes(6 hours)
This topic only contains two ledger 1444577 and 3525781, but 1110144 are not found.
**Expected behavior**
Ledgers in metastore should be removed if retention policy meet.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on issue #12131: broker not clean ledger from metastore successfully
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #12131:
URL: https://github.com/apache/pulsar/issues/12131#issuecomment-1054902982
The issue had no activity for 30 days, mark with Stale label.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] aloyszhang commented on issue #12131: broker not clean ledger from metastore successfully
Posted by GitBox <gi...@apache.org>.
aloyszhang commented on issue #12131:
URL: https://github.com/apache/pulsar/issues/12131#issuecomment-924730508
@codelipenghui @hangc0276 PTAL, thx.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org