You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bookkeeper.apache.org by "Sijie Guo (Created) (JIRA)" <ji...@apache.org> on 2012/03/26 04:54:25 UTC
[jira] [Created] (BOOKKEEPER-193) Ledger is garbage collected by
mistake.
Ledger is garbage collected by mistake.
---------------------------------------
Key: BOOKKEEPER-193
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-193
Project: Bookkeeper
Issue Type: Bug
Components: bookkeeper-server
Reporter: Sijie Guo
Fix For: 4.1.0
currently, we encountered such case: ledger is garbage collected by mistake, and following requests would fail due to NoLedgerException.
{code}
2012-03-23 19:10:47,403 - INFO [GarbageCollectorThread:GarbageCollectorThread@234] - Garbage collecting deleted ledger index files.
2012-03-23 19:10:48,702 - INFO [GarbageCollectorThread:LedgerCache@544] - Deleting ledgerId: 89408
2012-03-23 19:10:48,703 - INFO [GarbageCollectorThread:LedgerCache@577] - Deleted ledger : 89408
2012-03-23 19:11:10,013 - ERROR [NIOServerFactory-3181:BookieServer@361] - Error writing 1@89408
org.apache.bookkeeper.bookie.Bookie$NoLedgerException: Ledger 89408 not found
at org.apache.bookkeeper.bookie.LedgerCache.getFileInfo(LedgerCache.java:228)
at org.apache.bookkeeper.bookie.LedgerCache.updatePage(LedgerCache.java:260)
at org.apache.bookkeeper.bookie.LedgerCache.putEntryOffset(LedgerCache.java:158)
at org.apache.bookkeeper.bookie.LedgerDescriptor.addEntry(LedgerDescriptor.java:135)
at org.apache.bookkeeper.bookie.Bookie.addEntryInternal(Bookie.java:1059)
at org.apache.bookkeeper.bookie.Bookie.addEntry(Bookie.java:1099)
at org.apache.bookkeeper.proto.BookieServer.processPacket(BookieServer.java:357)
at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.readRequest(NIOServerFactory.java:315)
at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.doIO(NIOServerFactory.java:213)
at org.apache.bookkeeper.proto.NIOServerFactory.run(NIOServerFactory.java:124)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-193) Ledger is garbage collected by
mistake.
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13241055#comment-13241055 ]
jiraposter@reviews.apache.org commented on BOOKKEEPER-193:
----------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4481/#review6511
-----------------------------------------------------------
bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/FlatLedgerManager.java
<https://reviews.apache.org/r/4481/#comment14165>
Casting indicates that something is wrong in your abstraction. Perhaps activeLedgers.snapshot() should return a SortedMap, and doGC() should take a Map. It doesn't do anything that requires the ConcurrentMap interface.
bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/HierarchicalLedgerManager.java
<https://reviews.apache.org/r/4481/#comment14166>
why not just have the implementation of SnapshotMap use a ConcurrentSkipListMap?
- Ivan
On 2012-03-29 05:27:14, Sijie Guo wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4481/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-03-29 05:27:14)
bq.
bq.
bq. Review request for bookkeeper.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. create a snapshot of bookie active ledgers, and then fetch the list of zookeeper metadata, then it is safe to do garbage collection.
bq.
bq.
bq. This addresses bug BOOKKEEPER-193.
bq. https://issues.apache.org/jira/browse/BOOKKEEPER-193
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/SnapshotMap.java PRE-CREATION
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/FlatLedgerManager.java 100cdad
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/HierarchicalLedgerManager.java 169c906
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/GcLedgersTest.java PRE-CREATION
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/LedgerManagerTestCase.java PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/4481/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Sijie
bq.
bq.
> Ledger is garbage collected by mistake.
> ---------------------------------------
>
> Key: BOOKKEEPER-193
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-193
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Priority: Blocker
> Fix For: 4.1.0
>
> Attachments: BK-193.patch, BK-193.patch_v2
>
>
> currently, we encountered such case: ledger is garbage collected by mistake, and following requests would fail due to NoLedgerException.
> {code}
> 2012-03-23 19:10:47,403 - INFO [GarbageCollectorThread:GarbageCollectorThread@234] - Garbage collecting deleted ledger index files.
> 2012-03-23 19:10:48,702 - INFO [GarbageCollectorThread:LedgerCache@544] - Deleting ledgerId: 89408
> 2012-03-23 19:10:48,703 - INFO [GarbageCollectorThread:LedgerCache@577] - Deleted ledger : 89408
> 2012-03-23 19:11:10,013 - ERROR [NIOServerFactory-3181:BookieServer@361] - Error writing 1@89408
> org.apache.bookkeeper.bookie.Bookie$NoLedgerException: Ledger 89408 not found
> at org.apache.bookkeeper.bookie.LedgerCache.getFileInfo(LedgerCache.java:228)
> at org.apache.bookkeeper.bookie.LedgerCache.updatePage(LedgerCache.java:260)
> at org.apache.bookkeeper.bookie.LedgerCache.putEntryOffset(LedgerCache.java:158)
> at org.apache.bookkeeper.bookie.LedgerDescriptor.addEntry(LedgerDescriptor.java:135)
> at org.apache.bookkeeper.bookie.Bookie.addEntryInternal(Bookie.java:1059)
> at org.apache.bookkeeper.bookie.Bookie.addEntry(Bookie.java:1099)
> at org.apache.bookkeeper.proto.BookieServer.processPacket(BookieServer.java:357)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.readRequest(NIOServerFactory.java:315)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.doIO(NIOServerFactory.java:213)
> at org.apache.bookkeeper.proto.NIOServerFactory.run(NIOServerFactory.java:124)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-193) Ledger is garbage collected by
mistake.
Posted by "Ivan Kelly (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13242205#comment-13242205 ]
Ivan Kelly commented on BOOKKEEPER-193:
---------------------------------------
Actually, in the implementation there is one point where the snapshot
could be modified by another thread. But this is an implementation
detail. If we built the snapshot each time we called snapshot() it
wouldn't need to be concurrent.
> Ledger is garbage collected by mistake.
> ---------------------------------------
>
> Key: BOOKKEEPER-193
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-193
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Priority: Blocker
> Fix For: 4.1.0
>
> Attachments: BK-193.patch, BK-193.patch_v2, BOOKKEEPER-193.diff
>
>
> currently, we encountered such case: ledger is garbage collected by mistake, and following requests would fail due to NoLedgerException.
> {code}
> 2012-03-23 19:10:47,403 - INFO [GarbageCollectorThread:GarbageCollectorThread@234] - Garbage collecting deleted ledger index files.
> 2012-03-23 19:10:48,702 - INFO [GarbageCollectorThread:LedgerCache@544] - Deleting ledgerId: 89408
> 2012-03-23 19:10:48,703 - INFO [GarbageCollectorThread:LedgerCache@577] - Deleted ledger : 89408
> 2012-03-23 19:11:10,013 - ERROR [NIOServerFactory-3181:BookieServer@361] - Error writing 1@89408
> org.apache.bookkeeper.bookie.Bookie$NoLedgerException: Ledger 89408 not found
> at org.apache.bookkeeper.bookie.LedgerCache.getFileInfo(LedgerCache.java:228)
> at org.apache.bookkeeper.bookie.LedgerCache.updatePage(LedgerCache.java:260)
> at org.apache.bookkeeper.bookie.LedgerCache.putEntryOffset(LedgerCache.java:158)
> at org.apache.bookkeeper.bookie.LedgerDescriptor.addEntry(LedgerDescriptor.java:135)
> at org.apache.bookkeeper.bookie.Bookie.addEntryInternal(Bookie.java:1059)
> at org.apache.bookkeeper.bookie.Bookie.addEntry(Bookie.java:1099)
> at org.apache.bookkeeper.proto.BookieServer.processPacket(BookieServer.java:357)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.readRequest(NIOServerFactory.java:315)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.doIO(NIOServerFactory.java:213)
> at org.apache.bookkeeper.proto.NIOServerFactory.run(NIOServerFactory.java:124)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-193) Ledger is garbage collected by
mistake.
Posted by "Flavio Junqueira (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13242234#comment-13242234 ]
Flavio Junqueira commented on BOOKKEEPER-193:
---------------------------------------------
I actually got confused with the structures. I just realized that the snapshot itself is a ConcurrentSkipListMap.
One of my concerns in the discussion about ConcurrentHashMap is that articles like this:
http://www.informit.com/guides/content.aspx?g=java&seqNum=246
say that the performance of ConcurrentHashMaps is not great with a large number of items. updates and updatesToMerge, however, are not expected to hold a pretty large number of items, even if the bookie has a large number of active ledgers. Also, as Ivan pointed out offline, this is not on the critical path, so it is not much of a concern.
The patch is good for me, +1.
> Ledger is garbage collected by mistake.
> ---------------------------------------
>
> Key: BOOKKEEPER-193
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-193
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Priority: Blocker
> Fix For: 4.1.0
>
> Attachments: BK-193.patch, BK-193.patch_v2, BOOKKEEPER-193.diff
>
>
> currently, we encountered such case: ledger is garbage collected by mistake, and following requests would fail due to NoLedgerException.
> {code}
> 2012-03-23 19:10:47,403 - INFO [GarbageCollectorThread:GarbageCollectorThread@234] - Garbage collecting deleted ledger index files.
> 2012-03-23 19:10:48,702 - INFO [GarbageCollectorThread:LedgerCache@544] - Deleting ledgerId: 89408
> 2012-03-23 19:10:48,703 - INFO [GarbageCollectorThread:LedgerCache@577] - Deleted ledger : 89408
> 2012-03-23 19:11:10,013 - ERROR [NIOServerFactory-3181:BookieServer@361] - Error writing 1@89408
> org.apache.bookkeeper.bookie.Bookie$NoLedgerException: Ledger 89408 not found
> at org.apache.bookkeeper.bookie.LedgerCache.getFileInfo(LedgerCache.java:228)
> at org.apache.bookkeeper.bookie.LedgerCache.updatePage(LedgerCache.java:260)
> at org.apache.bookkeeper.bookie.LedgerCache.putEntryOffset(LedgerCache.java:158)
> at org.apache.bookkeeper.bookie.LedgerDescriptor.addEntry(LedgerDescriptor.java:135)
> at org.apache.bookkeeper.bookie.Bookie.addEntryInternal(Bookie.java:1059)
> at org.apache.bookkeeper.bookie.Bookie.addEntry(Bookie.java:1099)
> at org.apache.bookkeeper.proto.BookieServer.processPacket(BookieServer.java:357)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.readRequest(NIOServerFactory.java:315)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.doIO(NIOServerFactory.java:213)
> at org.apache.bookkeeper.proto.NIOServerFactory.run(NIOServerFactory.java:124)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-193) Ledger is garbage collected by
mistake.
Posted by "Flavio Junqueira (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Flavio Junqueira updated BOOKKEEPER-193:
----------------------------------------
Priority: Blocker (was: Major)
> Ledger is garbage collected by mistake.
> ---------------------------------------
>
> Key: BOOKKEEPER-193
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-193
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Reporter: Sijie Guo
> Priority: Blocker
> Fix For: 4.1.0
>
>
> currently, we encountered such case: ledger is garbage collected by mistake, and following requests would fail due to NoLedgerException.
> {code}
> 2012-03-23 19:10:47,403 - INFO [GarbageCollectorThread:GarbageCollectorThread@234] - Garbage collecting deleted ledger index files.
> 2012-03-23 19:10:48,702 - INFO [GarbageCollectorThread:LedgerCache@544] - Deleting ledgerId: 89408
> 2012-03-23 19:10:48,703 - INFO [GarbageCollectorThread:LedgerCache@577] - Deleted ledger : 89408
> 2012-03-23 19:11:10,013 - ERROR [NIOServerFactory-3181:BookieServer@361] - Error writing 1@89408
> org.apache.bookkeeper.bookie.Bookie$NoLedgerException: Ledger 89408 not found
> at org.apache.bookkeeper.bookie.LedgerCache.getFileInfo(LedgerCache.java:228)
> at org.apache.bookkeeper.bookie.LedgerCache.updatePage(LedgerCache.java:260)
> at org.apache.bookkeeper.bookie.LedgerCache.putEntryOffset(LedgerCache.java:158)
> at org.apache.bookkeeper.bookie.LedgerDescriptor.addEntry(LedgerDescriptor.java:135)
> at org.apache.bookkeeper.bookie.Bookie.addEntryInternal(Bookie.java:1059)
> at org.apache.bookkeeper.bookie.Bookie.addEntry(Bookie.java:1099)
> at org.apache.bookkeeper.proto.BookieServer.processPacket(BookieServer.java:357)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.readRequest(NIOServerFactory.java:315)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.doIO(NIOServerFactory.java:213)
> at org.apache.bookkeeper.proto.NIOServerFactory.run(NIOServerFactory.java:124)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-193) Ledger is garbage collected by
mistake.
Posted by "Ivan Kelly (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ivan Kelly updated BOOKKEEPER-193:
----------------------------------
Attachment: BOOKKEEPER-193.diff
Suggested changes to remove casts
> Ledger is garbage collected by mistake.
> ---------------------------------------
>
> Key: BOOKKEEPER-193
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-193
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Priority: Blocker
> Fix For: 4.1.0
>
> Attachments: BK-193.patch, BK-193.patch_v2, BOOKKEEPER-193.diff
>
>
> currently, we encountered such case: ledger is garbage collected by mistake, and following requests would fail due to NoLedgerException.
> {code}
> 2012-03-23 19:10:47,403 - INFO [GarbageCollectorThread:GarbageCollectorThread@234] - Garbage collecting deleted ledger index files.
> 2012-03-23 19:10:48,702 - INFO [GarbageCollectorThread:LedgerCache@544] - Deleting ledgerId: 89408
> 2012-03-23 19:10:48,703 - INFO [GarbageCollectorThread:LedgerCache@577] - Deleted ledger : 89408
> 2012-03-23 19:11:10,013 - ERROR [NIOServerFactory-3181:BookieServer@361] - Error writing 1@89408
> org.apache.bookkeeper.bookie.Bookie$NoLedgerException: Ledger 89408 not found
> at org.apache.bookkeeper.bookie.LedgerCache.getFileInfo(LedgerCache.java:228)
> at org.apache.bookkeeper.bookie.LedgerCache.updatePage(LedgerCache.java:260)
> at org.apache.bookkeeper.bookie.LedgerCache.putEntryOffset(LedgerCache.java:158)
> at org.apache.bookkeeper.bookie.LedgerDescriptor.addEntry(LedgerDescriptor.java:135)
> at org.apache.bookkeeper.bookie.Bookie.addEntryInternal(Bookie.java:1059)
> at org.apache.bookkeeper.bookie.Bookie.addEntry(Bookie.java:1099)
> at org.apache.bookkeeper.proto.BookieServer.processPacket(BookieServer.java:357)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.readRequest(NIOServerFactory.java:315)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.doIO(NIOServerFactory.java:213)
> at org.apache.bookkeeper.proto.NIOServerFactory.run(NIOServerFactory.java:124)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-193) Ledger is garbage collected by
mistake.
Posted by "Sijie Guo (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13238075#comment-13238075 ]
Sijie Guo commented on BOOKKEEPER-193:
--------------------------------------
this issue is a bug of the logic of garbage collection. currently the garbage collection is executed, by first fetching a list of all ledgers, fetching the active ledgers from bookie, then garbage collecting those active ledgers not in zookeeper list. there is a time period between fetching list from zookeeper and fetching list from bookie, if a ledger created in this time period, it would be garbage collected by mistake.
for FlatLedgerManager, this issue could be fixed easily. Since the ledgers are created in sequence, we can get the max ledger id when fetching list of all ledgers from zookeeper, during garbage collection, those ledgers are larger than max ledger id would not be garbage collected until next garbage collection is executed.
for HierarchicalLedgerManager, it is different, because the id generation and the ledger creation is two different operations running in asynchronous. one possible solution is fetching a copy of active ledgers from bookie first (the requests came in after fetching should not put in the list of active ledgers used for gc), then fetching the list of all ledgers from zookeeper, which can ensure we get the right list of all ledgers from zookeeper.
> Ledger is garbage collected by mistake.
> ---------------------------------------
>
> Key: BOOKKEEPER-193
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-193
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Reporter: Sijie Guo
> Fix For: 4.1.0
>
>
> currently, we encountered such case: ledger is garbage collected by mistake, and following requests would fail due to NoLedgerException.
> {code}
> 2012-03-23 19:10:47,403 - INFO [GarbageCollectorThread:GarbageCollectorThread@234] - Garbage collecting deleted ledger index files.
> 2012-03-23 19:10:48,702 - INFO [GarbageCollectorThread:LedgerCache@544] - Deleting ledgerId: 89408
> 2012-03-23 19:10:48,703 - INFO [GarbageCollectorThread:LedgerCache@577] - Deleted ledger : 89408
> 2012-03-23 19:11:10,013 - ERROR [NIOServerFactory-3181:BookieServer@361] - Error writing 1@89408
> org.apache.bookkeeper.bookie.Bookie$NoLedgerException: Ledger 89408 not found
> at org.apache.bookkeeper.bookie.LedgerCache.getFileInfo(LedgerCache.java:228)
> at org.apache.bookkeeper.bookie.LedgerCache.updatePage(LedgerCache.java:260)
> at org.apache.bookkeeper.bookie.LedgerCache.putEntryOffset(LedgerCache.java:158)
> at org.apache.bookkeeper.bookie.LedgerDescriptor.addEntry(LedgerDescriptor.java:135)
> at org.apache.bookkeeper.bookie.Bookie.addEntryInternal(Bookie.java:1059)
> at org.apache.bookkeeper.bookie.Bookie.addEntry(Bookie.java:1099)
> at org.apache.bookkeeper.proto.BookieServer.processPacket(BookieServer.java:357)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.readRequest(NIOServerFactory.java:315)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.doIO(NIOServerFactory.java:213)
> at org.apache.bookkeeper.proto.NIOServerFactory.run(NIOServerFactory.java:124)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-193) Ledger is garbage collected by
mistake.
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13240313#comment-13240313 ]
jiraposter@reviews.apache.org commented on BOOKKEEPER-193:
----------------------------------------------------------
bq. On 2012-03-28 08:10:21, fpj wrote:
bq. > bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/GcLedgersTest.java, line 124
bq. > <https://reviews.apache.org/r/4481/diff/1/?file=95778#file95778line124>
bq. >
bq. > Can we perhaps use another latch here instead of time? Relying on time doesn't always work and in many cases it will induce unnecessary waiting time.
agreed. it could use another latch. will fix it in new patch.
bq. On 2012-03-28 08:10:21, fpj wrote:
bq. > bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/SnapshotMap.java, line 37
bq. > <https://reviews.apache.org/r/4481/diff/1/?file=95777#file95777line37>
bq. >
bq. > Possibly not a big deal, but I wonder if it is really necessary to use this lock. In the way I read the code, I think it would work without it, but perhaps I'm missing something important.
the lock is used to avoid modifying two variables updates & updatesToMerge map, which is used to avoid inserting to updatesToMerge map during snapshot.
suppose two thread, one is doing insertion, the other one is done snapshot.
1) insertion thread: get the reference of updates map, tried to call #put.
2) snapshot thread: swap updates map to updatesToMerge map.
3) snapshot thread: iterates over the updatesToMerge map to merge them to snapshot map. (it may take times)
4) insertion thread: execute put. since updates map object has been assigned to updatesToMerge, so the put actually is applied in updatesToMerge, if the insertion position is large than the current position of iteration, it is OK. otherwise, the insertion will be lost.
the readwrite lock only blocks when modifying the updates & updatesToMerge references, I think it would not be expensive.
- Sijie
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4481/#review6481
-----------------------------------------------------------
On 2012-03-26 14:45:30, Sijie Guo wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4481/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-03-26 14:45:30)
bq.
bq.
bq. Review request for bookkeeper.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. create a snapshot of bookie active ledgers, and then fetch the list of zookeeper metadata, then it is safe to do garbage collection.
bq.
bq.
bq. This addresses bug BOOKKEEPER-193.
bq. https://issues.apache.org/jira/browse/BOOKKEEPER-193
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/LedgerManagerTestCase.java PRE-CREATION
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/GcLedgersTest.java PRE-CREATION
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/SnapshotMap.java PRE-CREATION
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/HierarchicalLedgerManager.java 9d9bf22
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/FlatLedgerManager.java 100cdad
bq.
bq. Diff: https://reviews.apache.org/r/4481/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Sijie
bq.
bq.
> Ledger is garbage collected by mistake.
> ---------------------------------------
>
> Key: BOOKKEEPER-193
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-193
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Priority: Blocker
> Fix For: 4.1.0
>
> Attachments: BK-193.patch
>
>
> currently, we encountered such case: ledger is garbage collected by mistake, and following requests would fail due to NoLedgerException.
> {code}
> 2012-03-23 19:10:47,403 - INFO [GarbageCollectorThread:GarbageCollectorThread@234] - Garbage collecting deleted ledger index files.
> 2012-03-23 19:10:48,702 - INFO [GarbageCollectorThread:LedgerCache@544] - Deleting ledgerId: 89408
> 2012-03-23 19:10:48,703 - INFO [GarbageCollectorThread:LedgerCache@577] - Deleted ledger : 89408
> 2012-03-23 19:11:10,013 - ERROR [NIOServerFactory-3181:BookieServer@361] - Error writing 1@89408
> org.apache.bookkeeper.bookie.Bookie$NoLedgerException: Ledger 89408 not found
> at org.apache.bookkeeper.bookie.LedgerCache.getFileInfo(LedgerCache.java:228)
> at org.apache.bookkeeper.bookie.LedgerCache.updatePage(LedgerCache.java:260)
> at org.apache.bookkeeper.bookie.LedgerCache.putEntryOffset(LedgerCache.java:158)
> at org.apache.bookkeeper.bookie.LedgerDescriptor.addEntry(LedgerDescriptor.java:135)
> at org.apache.bookkeeper.bookie.Bookie.addEntryInternal(Bookie.java:1059)
> at org.apache.bookkeeper.bookie.Bookie.addEntry(Bookie.java:1099)
> at org.apache.bookkeeper.proto.BookieServer.processPacket(BookieServer.java:357)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.readRequest(NIOServerFactory.java:315)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.doIO(NIOServerFactory.java:213)
> at org.apache.bookkeeper.proto.NIOServerFactory.run(NIOServerFactory.java:124)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-193) Ledger is garbage collected by
mistake.
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13238443#comment-13238443 ]
jiraposter@reviews.apache.org commented on BOOKKEEPER-193:
----------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4481/
-----------------------------------------------------------
Review request for bookkeeper.
Summary
-------
create a snapshot of bookie active ledgers, and then fetch the list of zookeeper metadata, then it is safe to do garbage collection.
This addresses bug BOOKKEEPER-193.
https://issues.apache.org/jira/browse/BOOKKEEPER-193
Diffs
-----
bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/LedgerManagerTestCase.java PRE-CREATION
bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/GcLedgersTest.java PRE-CREATION
bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/SnapshotMap.java PRE-CREATION
bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/HierarchicalLedgerManager.java 9d9bf22
bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/FlatLedgerManager.java 100cdad
Diff: https://reviews.apache.org/r/4481/diff
Testing
-------
Thanks,
Sijie
> Ledger is garbage collected by mistake.
> ---------------------------------------
>
> Key: BOOKKEEPER-193
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-193
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Reporter: Sijie Guo
> Priority: Blocker
> Fix For: 4.1.0
>
>
> currently, we encountered such case: ledger is garbage collected by mistake, and following requests would fail due to NoLedgerException.
> {code}
> 2012-03-23 19:10:47,403 - INFO [GarbageCollectorThread:GarbageCollectorThread@234] - Garbage collecting deleted ledger index files.
> 2012-03-23 19:10:48,702 - INFO [GarbageCollectorThread:LedgerCache@544] - Deleting ledgerId: 89408
> 2012-03-23 19:10:48,703 - INFO [GarbageCollectorThread:LedgerCache@577] - Deleted ledger : 89408
> 2012-03-23 19:11:10,013 - ERROR [NIOServerFactory-3181:BookieServer@361] - Error writing 1@89408
> org.apache.bookkeeper.bookie.Bookie$NoLedgerException: Ledger 89408 not found
> at org.apache.bookkeeper.bookie.LedgerCache.getFileInfo(LedgerCache.java:228)
> at org.apache.bookkeeper.bookie.LedgerCache.updatePage(LedgerCache.java:260)
> at org.apache.bookkeeper.bookie.LedgerCache.putEntryOffset(LedgerCache.java:158)
> at org.apache.bookkeeper.bookie.LedgerDescriptor.addEntry(LedgerDescriptor.java:135)
> at org.apache.bookkeeper.bookie.Bookie.addEntryInternal(Bookie.java:1059)
> at org.apache.bookkeeper.bookie.Bookie.addEntry(Bookie.java:1099)
> at org.apache.bookkeeper.proto.BookieServer.processPacket(BookieServer.java:357)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.readRequest(NIOServerFactory.java:315)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.doIO(NIOServerFactory.java:213)
> at org.apache.bookkeeper.proto.NIOServerFactory.run(NIOServerFactory.java:124)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-193) Ledger is garbage collected by
mistake.
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13240284#comment-13240284 ]
jiraposter@reviews.apache.org commented on BOOKKEEPER-193:
----------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4481/#review6481
-----------------------------------------------------------
It looks very good, Sijie. I have just a few points.
bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/SnapshotMap.java
<https://reviews.apache.org/r/4481/#comment14130>
I really like the idea of using a snapshot map!!
bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/SnapshotMap.java
<https://reviews.apache.org/r/4481/#comment14131>
Possibly not a big deal, but I wonder if it is really necessary to use this lock. In the way I read the code, I think it would work without it, but perhaps I'm missing something important.
bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/GcLedgersTest.java
<https://reviews.apache.org/r/4481/#comment14132>
Can we perhaps use another latch here instead of time? Relying on time doesn't always work and in many cases it will induce unnecessary waiting time.
- fpj
On 2012-03-26 14:45:30, Sijie Guo wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4481/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-03-26 14:45:30)
bq.
bq.
bq. Review request for bookkeeper.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. create a snapshot of bookie active ledgers, and then fetch the list of zookeeper metadata, then it is safe to do garbage collection.
bq.
bq.
bq. This addresses bug BOOKKEEPER-193.
bq. https://issues.apache.org/jira/browse/BOOKKEEPER-193
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/LedgerManagerTestCase.java PRE-CREATION
bq. bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/GcLedgersTest.java PRE-CREATION
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/SnapshotMap.java PRE-CREATION
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/HierarchicalLedgerManager.java 9d9bf22
bq. bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/FlatLedgerManager.java 100cdad
bq.
bq. Diff: https://reviews.apache.org/r/4481/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Sijie
bq.
bq.
> Ledger is garbage collected by mistake.
> ---------------------------------------
>
> Key: BOOKKEEPER-193
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-193
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Priority: Blocker
> Fix For: 4.1.0
>
> Attachments: BK-193.patch
>
>
> currently, we encountered such case: ledger is garbage collected by mistake, and following requests would fail due to NoLedgerException.
> {code}
> 2012-03-23 19:10:47,403 - INFO [GarbageCollectorThread:GarbageCollectorThread@234] - Garbage collecting deleted ledger index files.
> 2012-03-23 19:10:48,702 - INFO [GarbageCollectorThread:LedgerCache@544] - Deleting ledgerId: 89408
> 2012-03-23 19:10:48,703 - INFO [GarbageCollectorThread:LedgerCache@577] - Deleted ledger : 89408
> 2012-03-23 19:11:10,013 - ERROR [NIOServerFactory-3181:BookieServer@361] - Error writing 1@89408
> org.apache.bookkeeper.bookie.Bookie$NoLedgerException: Ledger 89408 not found
> at org.apache.bookkeeper.bookie.LedgerCache.getFileInfo(LedgerCache.java:228)
> at org.apache.bookkeeper.bookie.LedgerCache.updatePage(LedgerCache.java:260)
> at org.apache.bookkeeper.bookie.LedgerCache.putEntryOffset(LedgerCache.java:158)
> at org.apache.bookkeeper.bookie.LedgerDescriptor.addEntry(LedgerDescriptor.java:135)
> at org.apache.bookkeeper.bookie.Bookie.addEntryInternal(Bookie.java:1059)
> at org.apache.bookkeeper.bookie.Bookie.addEntry(Bookie.java:1099)
> at org.apache.bookkeeper.proto.BookieServer.processPacket(BookieServer.java:357)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.readRequest(NIOServerFactory.java:315)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.doIO(NIOServerFactory.java:213)
> at org.apache.bookkeeper.proto.NIOServerFactory.run(NIOServerFactory.java:124)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-193) Ledger is garbage collected by
mistake.
Posted by "Sijie Guo (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13242216#comment-13242216 ]
Sijie Guo commented on BOOKKEEPER-193:
--------------------------------------
for #snapshot(), it means returning a snapshot object for processing. currently only gcThread would call snapshot, so it is safe to define the return value as NavigableMap.
but the variable snapshot in SnapshotMap, it would be access in the contains/remove operation in SnapshotMap. Currently contains/remove seems just be accessed in GcThread, but I am not sure the operations especially contains would be accessed in other thread. so I think it would better to keep add/remove/contains as ThreadSafe.
> Ledger is garbage collected by mistake.
> ---------------------------------------
>
> Key: BOOKKEEPER-193
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-193
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Priority: Blocker
> Fix For: 4.1.0
>
> Attachments: BK-193.patch, BK-193.patch_v2, BOOKKEEPER-193.diff
>
>
> currently, we encountered such case: ledger is garbage collected by mistake, and following requests would fail due to NoLedgerException.
> {code}
> 2012-03-23 19:10:47,403 - INFO [GarbageCollectorThread:GarbageCollectorThread@234] - Garbage collecting deleted ledger index files.
> 2012-03-23 19:10:48,702 - INFO [GarbageCollectorThread:LedgerCache@544] - Deleting ledgerId: 89408
> 2012-03-23 19:10:48,703 - INFO [GarbageCollectorThread:LedgerCache@577] - Deleted ledger : 89408
> 2012-03-23 19:11:10,013 - ERROR [NIOServerFactory-3181:BookieServer@361] - Error writing 1@89408
> org.apache.bookkeeper.bookie.Bookie$NoLedgerException: Ledger 89408 not found
> at org.apache.bookkeeper.bookie.LedgerCache.getFileInfo(LedgerCache.java:228)
> at org.apache.bookkeeper.bookie.LedgerCache.updatePage(LedgerCache.java:260)
> at org.apache.bookkeeper.bookie.LedgerCache.putEntryOffset(LedgerCache.java:158)
> at org.apache.bookkeeper.bookie.LedgerDescriptor.addEntry(LedgerDescriptor.java:135)
> at org.apache.bookkeeper.bookie.Bookie.addEntryInternal(Bookie.java:1059)
> at org.apache.bookkeeper.bookie.Bookie.addEntry(Bookie.java:1099)
> at org.apache.bookkeeper.proto.BookieServer.processPacket(BookieServer.java:357)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.readRequest(NIOServerFactory.java:315)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.doIO(NIOServerFactory.java:213)
> at org.apache.bookkeeper.proto.NIOServerFactory.run(NIOServerFactory.java:124)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-193) Ledger is garbage collected by
mistake.
Posted by "Sijie Guo (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sijie Guo updated BOOKKEEPER-193:
---------------------------------
Attachment: BK-193.patch
attach a patch first creating a snapshot of bookie server active ledgers, then fetching list of zookeeper metadata, and do garbage collection finally.
this patch includes a test case reproduce the issue described in this jira.
> Ledger is garbage collected by mistake.
> ---------------------------------------
>
> Key: BOOKKEEPER-193
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-193
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Reporter: Sijie Guo
> Priority: Blocker
> Fix For: 4.1.0
>
> Attachments: BK-193.patch
>
>
> currently, we encountered such case: ledger is garbage collected by mistake, and following requests would fail due to NoLedgerException.
> {code}
> 2012-03-23 19:10:47,403 - INFO [GarbageCollectorThread:GarbageCollectorThread@234] - Garbage collecting deleted ledger index files.
> 2012-03-23 19:10:48,702 - INFO [GarbageCollectorThread:LedgerCache@544] - Deleting ledgerId: 89408
> 2012-03-23 19:10:48,703 - INFO [GarbageCollectorThread:LedgerCache@577] - Deleted ledger : 89408
> 2012-03-23 19:11:10,013 - ERROR [NIOServerFactory-3181:BookieServer@361] - Error writing 1@89408
> org.apache.bookkeeper.bookie.Bookie$NoLedgerException: Ledger 89408 not found
> at org.apache.bookkeeper.bookie.LedgerCache.getFileInfo(LedgerCache.java:228)
> at org.apache.bookkeeper.bookie.LedgerCache.updatePage(LedgerCache.java:260)
> at org.apache.bookkeeper.bookie.LedgerCache.putEntryOffset(LedgerCache.java:158)
> at org.apache.bookkeeper.bookie.LedgerDescriptor.addEntry(LedgerDescriptor.java:135)
> at org.apache.bookkeeper.bookie.Bookie.addEntryInternal(Bookie.java:1059)
> at org.apache.bookkeeper.bookie.Bookie.addEntry(Bookie.java:1099)
> at org.apache.bookkeeper.proto.BookieServer.processPacket(BookieServer.java:357)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.readRequest(NIOServerFactory.java:315)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.doIO(NIOServerFactory.java:213)
> at org.apache.bookkeeper.proto.NIOServerFactory.run(NIOServerFactory.java:124)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-193) Ledger is garbage collected by
mistake.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13243066#comment-13243066 ]
Hudson commented on BOOKKEEPER-193:
-----------------------------------
Integrated in bookkeeper-trunk #436 (See [https://builds.apache.org/job/bookkeeper-trunk/436/])
BOOKKEEPER-193: Ledger is garbage collected by mistake. (sijie, ivank via sijie) (Revision 1307725)
Result = SUCCESS
sijie :
Files :
* /zookeeper/bookkeeper/trunk/CHANGES.txt
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/AbstractZkLedgerManager.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/FlatLedgerManager.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/HierarchicalLedgerManager.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/SnapshotMap.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/GcLedgersTest.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/LedgerManagerTestCase.java
> Ledger is garbage collected by mistake.
> ---------------------------------------
>
> Key: BOOKKEEPER-193
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-193
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Priority: Blocker
> Fix For: 4.1.0
>
> Attachments: BK-193.patch, BK-193.patch_v2, BOOKKEEPER-193.diff
>
>
> currently, we encountered such case: ledger is garbage collected by mistake, and following requests would fail due to NoLedgerException.
> {code}
> 2012-03-23 19:10:47,403 - INFO [GarbageCollectorThread:GarbageCollectorThread@234] - Garbage collecting deleted ledger index files.
> 2012-03-23 19:10:48,702 - INFO [GarbageCollectorThread:LedgerCache@544] - Deleting ledgerId: 89408
> 2012-03-23 19:10:48,703 - INFO [GarbageCollectorThread:LedgerCache@577] - Deleted ledger : 89408
> 2012-03-23 19:11:10,013 - ERROR [NIOServerFactory-3181:BookieServer@361] - Error writing 1@89408
> org.apache.bookkeeper.bookie.Bookie$NoLedgerException: Ledger 89408 not found
> at org.apache.bookkeeper.bookie.LedgerCache.getFileInfo(LedgerCache.java:228)
> at org.apache.bookkeeper.bookie.LedgerCache.updatePage(LedgerCache.java:260)
> at org.apache.bookkeeper.bookie.LedgerCache.putEntryOffset(LedgerCache.java:158)
> at org.apache.bookkeeper.bookie.LedgerDescriptor.addEntry(LedgerDescriptor.java:135)
> at org.apache.bookkeeper.bookie.Bookie.addEntryInternal(Bookie.java:1059)
> at org.apache.bookkeeper.bookie.Bookie.addEntry(Bookie.java:1099)
> at org.apache.bookkeeper.proto.BookieServer.processPacket(BookieServer.java:357)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.readRequest(NIOServerFactory.java:315)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.doIO(NIOServerFactory.java:213)
> at org.apache.bookkeeper.proto.NIOServerFactory.run(NIOServerFactory.java:124)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-193) Ledger is garbage collected by
mistake.
Posted by "Flavio Junqueira (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13242170#comment-13242170 ]
Flavio Junqueira commented on BOOKKEEPER-193:
---------------------------------------------
I'm not sure how big of deal this is and I believe is more of a question than anything else, but an instance implementing NavigableMap is not necessarily thread safe, is it? In this patch it is because we return a ConcurrentHashMap from snapshot(), but I wonder if it compromises the abstraction here.
> Ledger is garbage collected by mistake.
> ---------------------------------------
>
> Key: BOOKKEEPER-193
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-193
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Priority: Blocker
> Fix For: 4.1.0
>
> Attachments: BK-193.patch, BK-193.patch_v2, BOOKKEEPER-193.diff
>
>
> currently, we encountered such case: ledger is garbage collected by mistake, and following requests would fail due to NoLedgerException.
> {code}
> 2012-03-23 19:10:47,403 - INFO [GarbageCollectorThread:GarbageCollectorThread@234] - Garbage collecting deleted ledger index files.
> 2012-03-23 19:10:48,702 - INFO [GarbageCollectorThread:LedgerCache@544] - Deleting ledgerId: 89408
> 2012-03-23 19:10:48,703 - INFO [GarbageCollectorThread:LedgerCache@577] - Deleted ledger : 89408
> 2012-03-23 19:11:10,013 - ERROR [NIOServerFactory-3181:BookieServer@361] - Error writing 1@89408
> org.apache.bookkeeper.bookie.Bookie$NoLedgerException: Ledger 89408 not found
> at org.apache.bookkeeper.bookie.LedgerCache.getFileInfo(LedgerCache.java:228)
> at org.apache.bookkeeper.bookie.LedgerCache.updatePage(LedgerCache.java:260)
> at org.apache.bookkeeper.bookie.LedgerCache.putEntryOffset(LedgerCache.java:158)
> at org.apache.bookkeeper.bookie.LedgerDescriptor.addEntry(LedgerDescriptor.java:135)
> at org.apache.bookkeeper.bookie.Bookie.addEntryInternal(Bookie.java:1059)
> at org.apache.bookkeeper.bookie.Bookie.addEntry(Bookie.java:1099)
> at org.apache.bookkeeper.proto.BookieServer.processPacket(BookieServer.java:357)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.readRequest(NIOServerFactory.java:315)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.doIO(NIOServerFactory.java:213)
> at org.apache.bookkeeper.proto.NIOServerFactory.run(NIOServerFactory.java:124)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-193) Ledger is garbage collected by
mistake.
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13240988#comment-13240988 ]
jiraposter@reviews.apache.org commented on BOOKKEEPER-193:
----------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4481/
-----------------------------------------------------------
(Updated 2012-03-29 05:27:14.643196)
Review request for bookkeeper.
Changes
-------
improve patch to use latch instead of time.
Summary
-------
create a snapshot of bookie active ledgers, and then fetch the list of zookeeper metadata, then it is safe to do garbage collection.
This addresses bug BOOKKEEPER-193.
https://issues.apache.org/jira/browse/BOOKKEEPER-193
Diffs (updated)
-----
bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/SnapshotMap.java PRE-CREATION
bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/FlatLedgerManager.java 100cdad
bookkeeper-server/src/main/java/org/apache/bookkeeper/meta/HierarchicalLedgerManager.java 169c906
bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/GcLedgersTest.java PRE-CREATION
bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/LedgerManagerTestCase.java PRE-CREATION
Diff: https://reviews.apache.org/r/4481/diff
Testing
-------
Thanks,
Sijie
> Ledger is garbage collected by mistake.
> ---------------------------------------
>
> Key: BOOKKEEPER-193
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-193
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Priority: Blocker
> Fix For: 4.1.0
>
> Attachments: BK-193.patch
>
>
> currently, we encountered such case: ledger is garbage collected by mistake, and following requests would fail due to NoLedgerException.
> {code}
> 2012-03-23 19:10:47,403 - INFO [GarbageCollectorThread:GarbageCollectorThread@234] - Garbage collecting deleted ledger index files.
> 2012-03-23 19:10:48,702 - INFO [GarbageCollectorThread:LedgerCache@544] - Deleting ledgerId: 89408
> 2012-03-23 19:10:48,703 - INFO [GarbageCollectorThread:LedgerCache@577] - Deleted ledger : 89408
> 2012-03-23 19:11:10,013 - ERROR [NIOServerFactory-3181:BookieServer@361] - Error writing 1@89408
> org.apache.bookkeeper.bookie.Bookie$NoLedgerException: Ledger 89408 not found
> at org.apache.bookkeeper.bookie.LedgerCache.getFileInfo(LedgerCache.java:228)
> at org.apache.bookkeeper.bookie.LedgerCache.updatePage(LedgerCache.java:260)
> at org.apache.bookkeeper.bookie.LedgerCache.putEntryOffset(LedgerCache.java:158)
> at org.apache.bookkeeper.bookie.LedgerDescriptor.addEntry(LedgerDescriptor.java:135)
> at org.apache.bookkeeper.bookie.Bookie.addEntryInternal(Bookie.java:1059)
> at org.apache.bookkeeper.bookie.Bookie.addEntry(Bookie.java:1099)
> at org.apache.bookkeeper.proto.BookieServer.processPacket(BookieServer.java:357)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.readRequest(NIOServerFactory.java:315)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.doIO(NIOServerFactory.java:213)
> at org.apache.bookkeeper.proto.NIOServerFactory.run(NIOServerFactory.java:124)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-193) Ledger is garbage collected by
mistake.
Posted by "Ivan Kelly (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13242174#comment-13242174 ]
Ivan Kelly commented on BOOKKEEPER-193:
---------------------------------------
NavigableMap is the interface of the object which we return which is
ConcurrentHashMap and therefore _is_ threadsafe.
> Ledger is garbage collected by mistake.
> ---------------------------------------
>
> Key: BOOKKEEPER-193
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-193
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Priority: Blocker
> Fix For: 4.1.0
>
> Attachments: BK-193.patch, BK-193.patch_v2, BOOKKEEPER-193.diff
>
>
> currently, we encountered such case: ledger is garbage collected by mistake, and following requests would fail due to NoLedgerException.
> {code}
> 2012-03-23 19:10:47,403 - INFO [GarbageCollectorThread:GarbageCollectorThread@234] - Garbage collecting deleted ledger index files.
> 2012-03-23 19:10:48,702 - INFO [GarbageCollectorThread:LedgerCache@544] - Deleting ledgerId: 89408
> 2012-03-23 19:10:48,703 - INFO [GarbageCollectorThread:LedgerCache@577] - Deleted ledger : 89408
> 2012-03-23 19:11:10,013 - ERROR [NIOServerFactory-3181:BookieServer@361] - Error writing 1@89408
> org.apache.bookkeeper.bookie.Bookie$NoLedgerException: Ledger 89408 not found
> at org.apache.bookkeeper.bookie.LedgerCache.getFileInfo(LedgerCache.java:228)
> at org.apache.bookkeeper.bookie.LedgerCache.updatePage(LedgerCache.java:260)
> at org.apache.bookkeeper.bookie.LedgerCache.putEntryOffset(LedgerCache.java:158)
> at org.apache.bookkeeper.bookie.LedgerDescriptor.addEntry(LedgerDescriptor.java:135)
> at org.apache.bookkeeper.bookie.Bookie.addEntryInternal(Bookie.java:1059)
> at org.apache.bookkeeper.bookie.Bookie.addEntry(Bookie.java:1099)
> at org.apache.bookkeeper.proto.BookieServer.processPacket(BookieServer.java:357)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.readRequest(NIOServerFactory.java:315)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.doIO(NIOServerFactory.java:213)
> at org.apache.bookkeeper.proto.NIOServerFactory.run(NIOServerFactory.java:124)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-193) Ledger is garbage collected by
mistake.
Posted by "Flavio Junqueira (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13242179#comment-13242179 ]
Flavio Junqueira commented on BOOKKEEPER-193:
---------------------------------------------
I'm not referring to the implementation, which I agreed before that it is ok, but the abstraction instead. If someone later comes and implements SnapshotMap', then there is nothing referring to the fact that the object snapshot returns is thread safe.
> Ledger is garbage collected by mistake.
> ---------------------------------------
>
> Key: BOOKKEEPER-193
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-193
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Priority: Blocker
> Fix For: 4.1.0
>
> Attachments: BK-193.patch, BK-193.patch_v2, BOOKKEEPER-193.diff
>
>
> currently, we encountered such case: ledger is garbage collected by mistake, and following requests would fail due to NoLedgerException.
> {code}
> 2012-03-23 19:10:47,403 - INFO [GarbageCollectorThread:GarbageCollectorThread@234] - Garbage collecting deleted ledger index files.
> 2012-03-23 19:10:48,702 - INFO [GarbageCollectorThread:LedgerCache@544] - Deleting ledgerId: 89408
> 2012-03-23 19:10:48,703 - INFO [GarbageCollectorThread:LedgerCache@577] - Deleted ledger : 89408
> 2012-03-23 19:11:10,013 - ERROR [NIOServerFactory-3181:BookieServer@361] - Error writing 1@89408
> org.apache.bookkeeper.bookie.Bookie$NoLedgerException: Ledger 89408 not found
> at org.apache.bookkeeper.bookie.LedgerCache.getFileInfo(LedgerCache.java:228)
> at org.apache.bookkeeper.bookie.LedgerCache.updatePage(LedgerCache.java:260)
> at org.apache.bookkeeper.bookie.LedgerCache.putEntryOffset(LedgerCache.java:158)
> at org.apache.bookkeeper.bookie.LedgerDescriptor.addEntry(LedgerDescriptor.java:135)
> at org.apache.bookkeeper.bookie.Bookie.addEntryInternal(Bookie.java:1059)
> at org.apache.bookkeeper.bookie.Bookie.addEntry(Bookie.java:1099)
> at org.apache.bookkeeper.proto.BookieServer.processPacket(BookieServer.java:357)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.readRequest(NIOServerFactory.java:315)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.doIO(NIOServerFactory.java:213)
> at org.apache.bookkeeper.proto.NIOServerFactory.run(NIOServerFactory.java:124)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-193) Ledger is garbage collected by
mistake.
Posted by "Sijie Guo (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sijie Guo updated BOOKKEEPER-193:
---------------------------------
Attachment: BK-193.patch_v2
attach a new patch to use latch instead of time.
> Ledger is garbage collected by mistake.
> ---------------------------------------
>
> Key: BOOKKEEPER-193
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-193
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Priority: Blocker
> Fix For: 4.1.0
>
> Attachments: BK-193.patch, BK-193.patch_v2
>
>
> currently, we encountered such case: ledger is garbage collected by mistake, and following requests would fail due to NoLedgerException.
> {code}
> 2012-03-23 19:10:47,403 - INFO [GarbageCollectorThread:GarbageCollectorThread@234] - Garbage collecting deleted ledger index files.
> 2012-03-23 19:10:48,702 - INFO [GarbageCollectorThread:LedgerCache@544] - Deleting ledgerId: 89408
> 2012-03-23 19:10:48,703 - INFO [GarbageCollectorThread:LedgerCache@577] - Deleted ledger : 89408
> 2012-03-23 19:11:10,013 - ERROR [NIOServerFactory-3181:BookieServer@361] - Error writing 1@89408
> org.apache.bookkeeper.bookie.Bookie$NoLedgerException: Ledger 89408 not found
> at org.apache.bookkeeper.bookie.LedgerCache.getFileInfo(LedgerCache.java:228)
> at org.apache.bookkeeper.bookie.LedgerCache.updatePage(LedgerCache.java:260)
> at org.apache.bookkeeper.bookie.LedgerCache.putEntryOffset(LedgerCache.java:158)
> at org.apache.bookkeeper.bookie.LedgerDescriptor.addEntry(LedgerDescriptor.java:135)
> at org.apache.bookkeeper.bookie.Bookie.addEntryInternal(Bookie.java:1059)
> at org.apache.bookkeeper.bookie.Bookie.addEntry(Bookie.java:1099)
> at org.apache.bookkeeper.proto.BookieServer.processPacket(BookieServer.java:357)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.readRequest(NIOServerFactory.java:315)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.doIO(NIOServerFactory.java:213)
> at org.apache.bookkeeper.proto.NIOServerFactory.run(NIOServerFactory.java:124)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-193) Ledger is garbage collected by
mistake.
Posted by "Sijie Guo (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13241247#comment-13241247 ]
Sijie Guo commented on BOOKKEEPER-193:
--------------------------------------
Thanks, Ivan. I am OK with such changes, +1 for Ivan's patch.
> Ledger is garbage collected by mistake.
> ---------------------------------------
>
> Key: BOOKKEEPER-193
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-193
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Priority: Blocker
> Fix For: 4.1.0
>
> Attachments: BK-193.patch, BK-193.patch_v2, BOOKKEEPER-193.diff
>
>
> currently, we encountered such case: ledger is garbage collected by mistake, and following requests would fail due to NoLedgerException.
> {code}
> 2012-03-23 19:10:47,403 - INFO [GarbageCollectorThread:GarbageCollectorThread@234] - Garbage collecting deleted ledger index files.
> 2012-03-23 19:10:48,702 - INFO [GarbageCollectorThread:LedgerCache@544] - Deleting ledgerId: 89408
> 2012-03-23 19:10:48,703 - INFO [GarbageCollectorThread:LedgerCache@577] - Deleted ledger : 89408
> 2012-03-23 19:11:10,013 - ERROR [NIOServerFactory-3181:BookieServer@361] - Error writing 1@89408
> org.apache.bookkeeper.bookie.Bookie$NoLedgerException: Ledger 89408 not found
> at org.apache.bookkeeper.bookie.LedgerCache.getFileInfo(LedgerCache.java:228)
> at org.apache.bookkeeper.bookie.LedgerCache.updatePage(LedgerCache.java:260)
> at org.apache.bookkeeper.bookie.LedgerCache.putEntryOffset(LedgerCache.java:158)
> at org.apache.bookkeeper.bookie.LedgerDescriptor.addEntry(LedgerDescriptor.java:135)
> at org.apache.bookkeeper.bookie.Bookie.addEntryInternal(Bookie.java:1059)
> at org.apache.bookkeeper.bookie.Bookie.addEntry(Bookie.java:1099)
> at org.apache.bookkeeper.proto.BookieServer.processPacket(BookieServer.java:357)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.readRequest(NIOServerFactory.java:315)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.doIO(NIOServerFactory.java:213)
> at org.apache.bookkeeper.proto.NIOServerFactory.run(NIOServerFactory.java:124)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-193) Ledger is garbage collected by
mistake.
Posted by "Flavio Junqueira (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13242201#comment-13242201 ]
Flavio Junqueira commented on BOOKKEEPER-193:
---------------------------------------------
Ok, I'm almost convinced. :-)
If it is only ever used by the gcThread, then the reason why we are creating a ConcurrentHashMap is just for additional safety? If we used a map that is not thread safe we would still be fine?
> Ledger is garbage collected by mistake.
> ---------------------------------------
>
> Key: BOOKKEEPER-193
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-193
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Priority: Blocker
> Fix For: 4.1.0
>
> Attachments: BK-193.patch, BK-193.patch_v2, BOOKKEEPER-193.diff
>
>
> currently, we encountered such case: ledger is garbage collected by mistake, and following requests would fail due to NoLedgerException.
> {code}
> 2012-03-23 19:10:47,403 - INFO [GarbageCollectorThread:GarbageCollectorThread@234] - Garbage collecting deleted ledger index files.
> 2012-03-23 19:10:48,702 - INFO [GarbageCollectorThread:LedgerCache@544] - Deleting ledgerId: 89408
> 2012-03-23 19:10:48,703 - INFO [GarbageCollectorThread:LedgerCache@577] - Deleted ledger : 89408
> 2012-03-23 19:11:10,013 - ERROR [NIOServerFactory-3181:BookieServer@361] - Error writing 1@89408
> org.apache.bookkeeper.bookie.Bookie$NoLedgerException: Ledger 89408 not found
> at org.apache.bookkeeper.bookie.LedgerCache.getFileInfo(LedgerCache.java:228)
> at org.apache.bookkeeper.bookie.LedgerCache.updatePage(LedgerCache.java:260)
> at org.apache.bookkeeper.bookie.LedgerCache.putEntryOffset(LedgerCache.java:158)
> at org.apache.bookkeeper.bookie.LedgerDescriptor.addEntry(LedgerDescriptor.java:135)
> at org.apache.bookkeeper.bookie.Bookie.addEntryInternal(Bookie.java:1059)
> at org.apache.bookkeeper.bookie.Bookie.addEntry(Bookie.java:1099)
> at org.apache.bookkeeper.proto.BookieServer.processPacket(BookieServer.java:357)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.readRequest(NIOServerFactory.java:315)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.doIO(NIOServerFactory.java:213)
> at org.apache.bookkeeper.proto.NIOServerFactory.run(NIOServerFactory.java:124)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-193) Ledger is garbage collected by
mistake.
Posted by "Ivan Kelly (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13242198#comment-13242198 ]
Ivan Kelly commented on BOOKKEEPER-193:
---------------------------------------
The object returned by snapshot doesn't necessarily need to be thread
safe. It's only ever used by the gcThread.
> Ledger is garbage collected by mistake.
> ---------------------------------------
>
> Key: BOOKKEEPER-193
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-193
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Priority: Blocker
> Fix For: 4.1.0
>
> Attachments: BK-193.patch, BK-193.patch_v2, BOOKKEEPER-193.diff
>
>
> currently, we encountered such case: ledger is garbage collected by mistake, and following requests would fail due to NoLedgerException.
> {code}
> 2012-03-23 19:10:47,403 - INFO [GarbageCollectorThread:GarbageCollectorThread@234] - Garbage collecting deleted ledger index files.
> 2012-03-23 19:10:48,702 - INFO [GarbageCollectorThread:LedgerCache@544] - Deleting ledgerId: 89408
> 2012-03-23 19:10:48,703 - INFO [GarbageCollectorThread:LedgerCache@577] - Deleted ledger : 89408
> 2012-03-23 19:11:10,013 - ERROR [NIOServerFactory-3181:BookieServer@361] - Error writing 1@89408
> org.apache.bookkeeper.bookie.Bookie$NoLedgerException: Ledger 89408 not found
> at org.apache.bookkeeper.bookie.LedgerCache.getFileInfo(LedgerCache.java:228)
> at org.apache.bookkeeper.bookie.LedgerCache.updatePage(LedgerCache.java:260)
> at org.apache.bookkeeper.bookie.LedgerCache.putEntryOffset(LedgerCache.java:158)
> at org.apache.bookkeeper.bookie.LedgerDescriptor.addEntry(LedgerDescriptor.java:135)
> at org.apache.bookkeeper.bookie.Bookie.addEntryInternal(Bookie.java:1059)
> at org.apache.bookkeeper.bookie.Bookie.addEntry(Bookie.java:1099)
> at org.apache.bookkeeper.proto.BookieServer.processPacket(BookieServer.java:357)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.readRequest(NIOServerFactory.java:315)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.doIO(NIOServerFactory.java:213)
> at org.apache.bookkeeper.proto.NIOServerFactory.run(NIOServerFactory.java:124)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-193) Ledger is garbage collected by
mistake.
Posted by "Flavio Junqueira (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13241053#comment-13241053 ]
Flavio Junqueira commented on BOOKKEEPER-193:
---------------------------------------------
+1, looks good to me! Since this is a blocker, I'd like to give an opportunity to others to have a look at it. If no one else says anything by the end of today, I'll commit it.
> Ledger is garbage collected by mistake.
> ---------------------------------------
>
> Key: BOOKKEEPER-193
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-193
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Priority: Blocker
> Fix For: 4.1.0
>
> Attachments: BK-193.patch, BK-193.patch_v2
>
>
> currently, we encountered such case: ledger is garbage collected by mistake, and following requests would fail due to NoLedgerException.
> {code}
> 2012-03-23 19:10:47,403 - INFO [GarbageCollectorThread:GarbageCollectorThread@234] - Garbage collecting deleted ledger index files.
> 2012-03-23 19:10:48,702 - INFO [GarbageCollectorThread:LedgerCache@544] - Deleting ledgerId: 89408
> 2012-03-23 19:10:48,703 - INFO [GarbageCollectorThread:LedgerCache@577] - Deleted ledger : 89408
> 2012-03-23 19:11:10,013 - ERROR [NIOServerFactory-3181:BookieServer@361] - Error writing 1@89408
> org.apache.bookkeeper.bookie.Bookie$NoLedgerException: Ledger 89408 not found
> at org.apache.bookkeeper.bookie.LedgerCache.getFileInfo(LedgerCache.java:228)
> at org.apache.bookkeeper.bookie.LedgerCache.updatePage(LedgerCache.java:260)
> at org.apache.bookkeeper.bookie.LedgerCache.putEntryOffset(LedgerCache.java:158)
> at org.apache.bookkeeper.bookie.LedgerDescriptor.addEntry(LedgerDescriptor.java:135)
> at org.apache.bookkeeper.bookie.Bookie.addEntryInternal(Bookie.java:1059)
> at org.apache.bookkeeper.bookie.Bookie.addEntry(Bookie.java:1099)
> at org.apache.bookkeeper.proto.BookieServer.processPacket(BookieServer.java:357)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.readRequest(NIOServerFactory.java:315)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.doIO(NIOServerFactory.java:213)
> at org.apache.bookkeeper.proto.NIOServerFactory.run(NIOServerFactory.java:124)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-193) Ledger is garbage collected by
mistake.
Posted by "Sijie Guo (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13243065#comment-13243065 ]
Sijie Guo commented on BOOKKEEPER-193:
--------------------------------------
committed as 1307725. thanks for Ivan's improvement, thanks Flavio for reviewing.
> Ledger is garbage collected by mistake.
> ---------------------------------------
>
> Key: BOOKKEEPER-193
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-193
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Priority: Blocker
> Fix For: 4.1.0
>
> Attachments: BK-193.patch, BK-193.patch_v2, BOOKKEEPER-193.diff
>
>
> currently, we encountered such case: ledger is garbage collected by mistake, and following requests would fail due to NoLedgerException.
> {code}
> 2012-03-23 19:10:47,403 - INFO [GarbageCollectorThread:GarbageCollectorThread@234] - Garbage collecting deleted ledger index files.
> 2012-03-23 19:10:48,702 - INFO [GarbageCollectorThread:LedgerCache@544] - Deleting ledgerId: 89408
> 2012-03-23 19:10:48,703 - INFO [GarbageCollectorThread:LedgerCache@577] - Deleted ledger : 89408
> 2012-03-23 19:11:10,013 - ERROR [NIOServerFactory-3181:BookieServer@361] - Error writing 1@89408
> org.apache.bookkeeper.bookie.Bookie$NoLedgerException: Ledger 89408 not found
> at org.apache.bookkeeper.bookie.LedgerCache.getFileInfo(LedgerCache.java:228)
> at org.apache.bookkeeper.bookie.LedgerCache.updatePage(LedgerCache.java:260)
> at org.apache.bookkeeper.bookie.LedgerCache.putEntryOffset(LedgerCache.java:158)
> at org.apache.bookkeeper.bookie.LedgerDescriptor.addEntry(LedgerDescriptor.java:135)
> at org.apache.bookkeeper.bookie.Bookie.addEntryInternal(Bookie.java:1059)
> at org.apache.bookkeeper.bookie.Bookie.addEntry(Bookie.java:1099)
> at org.apache.bookkeeper.proto.BookieServer.processPacket(BookieServer.java:357)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.readRequest(NIOServerFactory.java:315)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.doIO(NIOServerFactory.java:213)
> at org.apache.bookkeeper.proto.NIOServerFactory.run(NIOServerFactory.java:124)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira