[jira] [Commented] (BOOKKEEPER-695) Some entry logs are not removed from the bookie storage

["Matteo Merli (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BOOKKEEPER-695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13797294#comment-13797294 ] 

Matteo Merli commented on BOOKKEEPER-695:
-----------------------------------------

I'm not sure where it would be best to handle this case. My understanding is that we should ignore the truncated entries and compact/gc the entry log as needed.

Perhaps it would be easier to handle it in GarbageCollectorThead.extractMetaFromEntryLog, so that it will only affect the GC scanning. If there are truncated/corrupted entries, most likely they will be replayed by the journal, but in any case we cannot do anything to bring them back to life, so we can just skip them.

Something like:
{code:java}
try {
    entryLogger.scanEntryLog(entryLogId, scanner);
} catch (IOException e) {
    /// print warning but continue
}
{code}

> Some entry logs are not removed from the bookie storage
> -------------------------------------------------------
>
>                 Key: BOOKKEEPER-695
>                 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-695
>             Project: Bookkeeper
>          Issue Type: Bug
>            Reporter: Matteo Merli
>            Priority: Minor
>
> Some entry logs appear to be truncated (possible at a momemnt when the bookie
> was shut down) and the compaction is never getting rid of them:
> {code}
> 00:00:06.448 [GarbageCollectorThread] INFO o.a.b.bookie.GarbageCollectorThread - Extracting entry log meta from entryLogId: 1497
> 00:00:07.140 [GarbageCollectorThread] WARN  o.a.b.bookie.GarbageCollectorThread - Premature exception when processing 1497 recovery will take care of the problem
> java.io.IOException: Short read for ledger entry from entryLog 1497@718702792(31356!=32840)
>         at org.apache.bookkeeper.bookie.EntryLogger.scanEntryLog(EntryLogger.java:514) ~[bookkeeper-server-4.2.1.15.jar:4.2.2-SNAPSHOT]
>         at org.apache.bookkeeper.bookie.GarbageCollectorThread.extractMetaFromEntryLog(GarbageCollectorThread.java:572) [bookkeeper-server-4.2.1.15.jar:4.2.2-SNAPSHOT]
>         at org.apache.bookkeeper.bookie.GarbageCollectorThread.extractMetaFromEntryLogs(GarbageCollectorThread.java:549) [bookkeeper-server-4.2.1.15.jar:4.2.2-SNAPSHOT]
>         at org.apache.bookkeeper.bookie.GarbageCollectorThread.run(GarbageCollectorThread.java:268) [bookkeeper-server-4.2.1.15.jar:4.2.2-SNAPSHOT]
> {code}
> These entry logs are not being removed: 
> {code}
> $ ll -h *.log
> -rw-r--r-- 1 yahoo users 686M Jul 17 10:45 5d9.log
> -rw-r--r-- 1 yahoo users 634M Jul 27 19:31 b2d.log
> -rw-r--r-- 1 yahoo users 1.5G Jul 28 15:22 b35.log
> -rw-r--r-- 1 yahoo users 2.0G Aug  4 04:57 dbd.log
> -rw-r--r-- 1 yahoo users 2.0G Aug  4 04:58 dbe.log
> .....
> {code}



--
This message was sent by Atlassian JIRA
(v6.1#6144)