You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bookkeeper.apache.org by "Sijie Guo (Created) (JIRA)" <ji...@apache.org> on 2012/03/21 08:47:56 UTC
[jira] [Created] (BOOKKEEPER-190) Add entries would fail when
number of open ledgers reaches more than openFileLimit.
Add entries would fail when number of open ledgers reaches more than openFileLimit.
-----------------------------------------------------------------------------------
Key: BOOKKEEPER-190
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-190
Project: Bookkeeper
Issue Type: Bug
Components: bookkeeper-server
Reporter: Sijie Guo
Assignee: Sijie Guo
Fix For: 4.1.0
when the number of open ledgers reaches more than openFileLimit, a file info will be closed and removed from opened ledgers list. And after BOOKKEEPER-137, the ledger index file creation delayed until necessary.
suppose ledger l is removed from opened ledger list, and its index file haven't been created.
new add entries operations of other ledgers came into bookie server, a new page need to be grab for them. so bookie server may need to flush the dirty pages of ledger l(when page cache is full). and the flush would fail due to NoLedgerException (no index file found).
actually the ledger l isn't lost, it could be recovered if restarting bookie server, but the bookie server would not work well on adding entries.
a proposal solution is that we need to force index creation when the ledger is evicted from open ledgers list.
{code}
2012-03-21 14:00:42,989 - DEBUG - [NIOServerFactory-5000:LedgerCache@235] - New ledger index file created for ledgerId: 4
2012-03-21 14:00:42,990 - INFO - [NIOServerFactory-5000:LedgerCache@241] - Ledger 2 is evicted from file info cache.
2012-03-21 14:00:42,990 - DEBUG - [New I/O client worker #1-1:PerChannelBookieClient$2@255] - Successfully wrote request for adding entry: 0 ledger-id: 4 bookie: /10.82.129.173:5000 entry length: 70
2012-03-21 14:00:42,990 - ERROR - [NIOServerFactory-5000:BookieServer@361] - Error writing 0@4
org.apache.bookkeeper.bookie.Bookie$NoLedgerException: Ledger 2 not found
at org.apache.bookkeeper.bookie.LedgerCache.getFileInfo(LedgerCache.java:228)
at org.apache.bookkeeper.bookie.LedgerCache.flushLedger(LedgerCache.java:359)
at org.apache.bookkeeper.bookie.LedgerCache.flushLedger(LedgerCache.java:292)
at org.apache.bookkeeper.bookie.LedgerCache.grabCleanPage(LedgerCache.java:447)
at org.apache.bookkeeper.bookie.LedgerCache.putEntryOffset(LedgerCache.java:157)
at org.apache.bookkeeper.bookie.LedgerDescriptor.addEntry(LedgerDescriptor.java:130)
at org.apache.bookkeeper.bookie.Bookie.addEntryInternal(Bookie.java:1059)
at org.apache.bookkeeper.bookie.Bookie.addEntry(Bookie.java:1099)
at org.apache.bookkeeper.proto.BookieServer.processPacket(BookieServer.java:357)
at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.readRequest(NIOServerFactory.java:315)
at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.doIO(NIOServerFactory.java:213)
at org.apache.bookkeeper.proto.NIOServerFactory.run(NIOServerFactory.java:124)
2012-03-21 14:00:42,991 - DEBUG - [pool-3-thread-1:PerChannelBookieClient@576] - Got response for add request from bookie: /10.82.129.173:5000 for ledger: 4 entry: 0 rc: 101
2012-03-21 14:00:42,991 - ERROR - [pool-3-thread-1:PerChannelBookieClient@594] - Add for ledger: 4, entry: 0 failed on bookie: /10.82.129.173:5000 with code: 101
2012-03-21 14:00:42,991 - WARN - [pool-3-thread-1:PendingAddOp@142] - Write did not succeed: 4, 0
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-190) Add entries would fail when
number of open ledgers reaches more than openFileLimit.
Posted by "Flavio Junqueira (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13235463#comment-13235463 ]
Flavio Junqueira commented on BOOKKEEPER-190:
---------------------------------------------
+1, looks great, Sijie. Thanks for changing the test.
> Add entries would fail when number of open ledgers reaches more than openFileLimit.
> -----------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-190
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-190
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.1.0
>
> Attachments: BOOKKEEPER-190.diff, BOOKKEEPER-190.diff_v2
>
>
> when the number of open ledgers reaches more than openFileLimit, a file info will be closed and removed from opened ledgers list. And after BOOKKEEPER-137, the ledger index file creation delayed until necessary.
> suppose ledger l is removed from opened ledger list, and its index file haven't been created.
> new add entries operations of other ledgers came into bookie server, a new page need to be grab for them. so bookie server may need to flush the dirty pages of ledger l(when page cache is full). and the flush would fail due to NoLedgerException (no index file found).
> actually the ledger l isn't lost, it could be recovered if restarting bookie server, but the bookie server would not work well on adding entries.
> a proposal solution is that we need to force index creation when the ledger is evicted from open ledgers list.
> {code}
> 2012-03-21 14:00:42,989 - DEBUG - [NIOServerFactory-5000:LedgerCache@235] - New ledger index file created for ledgerId: 4
> 2012-03-21 14:00:42,990 - INFO - [NIOServerFactory-5000:LedgerCache@241] - Ledger 2 is evicted from file info cache.
> 2012-03-21 14:00:42,990 - DEBUG - [New I/O client worker #1-1:PerChannelBookieClient$2@255] - Successfully wrote request for adding entry: 0 ledger-id: 4 bookie: /10.82.129.173:5000 entry length: 70
> 2012-03-21 14:00:42,990 - ERROR - [NIOServerFactory-5000:BookieServer@361] - Error writing 0@4
> org.apache.bookkeeper.bookie.Bookie$NoLedgerException: Ledger 2 not found
> at org.apache.bookkeeper.bookie.LedgerCache.getFileInfo(LedgerCache.java:228)
> at org.apache.bookkeeper.bookie.LedgerCache.flushLedger(LedgerCache.java:359)
> at org.apache.bookkeeper.bookie.LedgerCache.flushLedger(LedgerCache.java:292)
> at org.apache.bookkeeper.bookie.LedgerCache.grabCleanPage(LedgerCache.java:447)
> at org.apache.bookkeeper.bookie.LedgerCache.putEntryOffset(LedgerCache.java:157)
> at org.apache.bookkeeper.bookie.LedgerDescriptor.addEntry(LedgerDescriptor.java:130)
> at org.apache.bookkeeper.bookie.Bookie.addEntryInternal(Bookie.java:1059)
> at org.apache.bookkeeper.bookie.Bookie.addEntry(Bookie.java:1099)
> at org.apache.bookkeeper.proto.BookieServer.processPacket(BookieServer.java:357)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.readRequest(NIOServerFactory.java:315)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.doIO(NIOServerFactory.java:213)
> at org.apache.bookkeeper.proto.NIOServerFactory.run(NIOServerFactory.java:124)
> 2012-03-21 14:00:42,991 - DEBUG - [pool-3-thread-1:PerChannelBookieClient@576] - Got response for add request from bookie: /10.82.129.173:5000 for ledger: 4 entry: 0 rc: 101
> 2012-03-21 14:00:42,991 - ERROR - [pool-3-thread-1:PerChannelBookieClient@594] - Add for ledger: 4, entry: 0 failed on bookie: /10.82.129.173:5000 with code: 101
> 2012-03-21 14:00:42,991 - WARN - [pool-3-thread-1:PendingAddOp@142] - Write did not succeed: 4, 0
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-190) Add entries would fail when
number of open ledgers reaches more than openFileLimit.
Posted by "Sijie Guo (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sijie Guo updated BOOKKEEPER-190:
---------------------------------
Attachment: BOOKKEEPER-190.diff
attach a patch to fix this issue.
the patch is quite simple, just force index creation when the ledger is evicted from open ledgers list.
also it fixed the reference counting issue on FileInfo, to release useCount after use it. otherwise, the index file is not closed actually, will cause 'Too many opened files' problem.
> Add entries would fail when number of open ledgers reaches more than openFileLimit.
> -----------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-190
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-190
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.1.0
>
> Attachments: BOOKKEEPER-190.diff
>
>
> when the number of open ledgers reaches more than openFileLimit, a file info will be closed and removed from opened ledgers list. And after BOOKKEEPER-137, the ledger index file creation delayed until necessary.
> suppose ledger l is removed from opened ledger list, and its index file haven't been created.
> new add entries operations of other ledgers came into bookie server, a new page need to be grab for them. so bookie server may need to flush the dirty pages of ledger l(when page cache is full). and the flush would fail due to NoLedgerException (no index file found).
> actually the ledger l isn't lost, it could be recovered if restarting bookie server, but the bookie server would not work well on adding entries.
> a proposal solution is that we need to force index creation when the ledger is evicted from open ledgers list.
> {code}
> 2012-03-21 14:00:42,989 - DEBUG - [NIOServerFactory-5000:LedgerCache@235] - New ledger index file created for ledgerId: 4
> 2012-03-21 14:00:42,990 - INFO - [NIOServerFactory-5000:LedgerCache@241] - Ledger 2 is evicted from file info cache.
> 2012-03-21 14:00:42,990 - DEBUG - [New I/O client worker #1-1:PerChannelBookieClient$2@255] - Successfully wrote request for adding entry: 0 ledger-id: 4 bookie: /10.82.129.173:5000 entry length: 70
> 2012-03-21 14:00:42,990 - ERROR - [NIOServerFactory-5000:BookieServer@361] - Error writing 0@4
> org.apache.bookkeeper.bookie.Bookie$NoLedgerException: Ledger 2 not found
> at org.apache.bookkeeper.bookie.LedgerCache.getFileInfo(LedgerCache.java:228)
> at org.apache.bookkeeper.bookie.LedgerCache.flushLedger(LedgerCache.java:359)
> at org.apache.bookkeeper.bookie.LedgerCache.flushLedger(LedgerCache.java:292)
> at org.apache.bookkeeper.bookie.LedgerCache.grabCleanPage(LedgerCache.java:447)
> at org.apache.bookkeeper.bookie.LedgerCache.putEntryOffset(LedgerCache.java:157)
> at org.apache.bookkeeper.bookie.LedgerDescriptor.addEntry(LedgerDescriptor.java:130)
> at org.apache.bookkeeper.bookie.Bookie.addEntryInternal(Bookie.java:1059)
> at org.apache.bookkeeper.bookie.Bookie.addEntry(Bookie.java:1099)
> at org.apache.bookkeeper.proto.BookieServer.processPacket(BookieServer.java:357)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.readRequest(NIOServerFactory.java:315)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.doIO(NIOServerFactory.java:213)
> at org.apache.bookkeeper.proto.NIOServerFactory.run(NIOServerFactory.java:124)
> 2012-03-21 14:00:42,991 - DEBUG - [pool-3-thread-1:PerChannelBookieClient@576] - Got response for add request from bookie: /10.82.129.173:5000 for ledger: 4 entry: 0 rc: 101
> 2012-03-21 14:00:42,991 - ERROR - [pool-3-thread-1:PerChannelBookieClient@594] - Add for ledger: 4, entry: 0 failed on bookie: /10.82.129.173:5000 with code: 101
> 2012-03-21 14:00:42,991 - WARN - [pool-3-thread-1:PendingAddOp@142] - Write did not succeed: 4, 0
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-190) Add entries would fail when
number of open ledgers reaches more than openFileLimit.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13241282#comment-13241282 ]
Hudson commented on BOOKKEEPER-190:
-----------------------------------
Integrated in bookkeeper-trunk #433 (See [https://builds.apache.org/job/bookkeeper-trunk/433/])
BOOKKEEPER-190: Add entries would fail when number of open ledgers reaches more than openFileLimit. (sijie via ivank) (Revision 1306839)
Result = ABORTED
ivank :
Files :
* /zookeeper/bookkeeper/trunk/CHANGES.txt
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/FileInfo.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerCacheImpl.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/conf/ServerConfiguration.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/BookieJournalTest.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/LedgerCacheTest.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/UpgradeTest.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/client/LedgerCacheTest.java
> Add entries would fail when number of open ledgers reaches more than openFileLimit.
> -----------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-190
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-190
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Ivan Kelly
> Fix For: 4.1.0
>
> Attachments: BOOKKEEPER-190.diff, BOOKKEEPER-190.diff_v2, BOOKKEEPER-190.diff_v3
>
>
> when the number of open ledgers reaches more than openFileLimit, a file info will be closed and removed from opened ledgers list. And after BOOKKEEPER-137, the ledger index file creation delayed until necessary.
> suppose ledger l is removed from opened ledger list, and its index file haven't been created.
> new add entries operations of other ledgers came into bookie server, a new page need to be grab for them. so bookie server may need to flush the dirty pages of ledger l(when page cache is full). and the flush would fail due to NoLedgerException (no index file found).
> actually the ledger l isn't lost, it could be recovered if restarting bookie server, but the bookie server would not work well on adding entries.
> a proposal solution is that we need to force index creation when the ledger is evicted from open ledgers list.
> {code}
> 2012-03-21 14:00:42,989 - DEBUG - [NIOServerFactory-5000:LedgerCache@235] - New ledger index file created for ledgerId: 4
> 2012-03-21 14:00:42,990 - INFO - [NIOServerFactory-5000:LedgerCache@241] - Ledger 2 is evicted from file info cache.
> 2012-03-21 14:00:42,990 - DEBUG - [New I/O client worker #1-1:PerChannelBookieClient$2@255] - Successfully wrote request for adding entry: 0 ledger-id: 4 bookie: /10.82.129.173:5000 entry length: 70
> 2012-03-21 14:00:42,990 - ERROR - [NIOServerFactory-5000:BookieServer@361] - Error writing 0@4
> org.apache.bookkeeper.bookie.Bookie$NoLedgerException: Ledger 2 not found
> at org.apache.bookkeeper.bookie.LedgerCache.getFileInfo(LedgerCache.java:228)
> at org.apache.bookkeeper.bookie.LedgerCache.flushLedger(LedgerCache.java:359)
> at org.apache.bookkeeper.bookie.LedgerCache.flushLedger(LedgerCache.java:292)
> at org.apache.bookkeeper.bookie.LedgerCache.grabCleanPage(LedgerCache.java:447)
> at org.apache.bookkeeper.bookie.LedgerCache.putEntryOffset(LedgerCache.java:157)
> at org.apache.bookkeeper.bookie.LedgerDescriptor.addEntry(LedgerDescriptor.java:130)
> at org.apache.bookkeeper.bookie.Bookie.addEntryInternal(Bookie.java:1059)
> at org.apache.bookkeeper.bookie.Bookie.addEntry(Bookie.java:1099)
> at org.apache.bookkeeper.proto.BookieServer.processPacket(BookieServer.java:357)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.readRequest(NIOServerFactory.java:315)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.doIO(NIOServerFactory.java:213)
> at org.apache.bookkeeper.proto.NIOServerFactory.run(NIOServerFactory.java:124)
> 2012-03-21 14:00:42,991 - DEBUG - [pool-3-thread-1:PerChannelBookieClient@576] - Got response for add request from bookie: /10.82.129.173:5000 for ledger: 4 entry: 0 rc: 101
> 2012-03-21 14:00:42,991 - ERROR - [pool-3-thread-1:PerChannelBookieClient@594] - Add for ledger: 4, entry: 0 failed on bookie: /10.82.129.173:5000 with code: 101
> 2012-03-21 14:00:42,991 - WARN - [pool-3-thread-1:PendingAddOp@142] - Write did not succeed: 4, 0
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-190) Add entries would fail when
number of open ledgers reaches more than openFileLimit.
Posted by "Sijie Guo (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sijie Guo updated BOOKKEEPER-190:
---------------------------------
Attachment: BOOKKEEPER-190.diff_v2
@Flavio.
I agreed. since the test is related to cache eviction, so I moved the test case to LedgerCacheTest, and changed it as what you suggested. a new patch is attached.
> Add entries would fail when number of open ledgers reaches more than openFileLimit.
> -----------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-190
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-190
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.1.0
>
> Attachments: BOOKKEEPER-190.diff, BOOKKEEPER-190.diff_v2
>
>
> when the number of open ledgers reaches more than openFileLimit, a file info will be closed and removed from opened ledgers list. And after BOOKKEEPER-137, the ledger index file creation delayed until necessary.
> suppose ledger l is removed from opened ledger list, and its index file haven't been created.
> new add entries operations of other ledgers came into bookie server, a new page need to be grab for them. so bookie server may need to flush the dirty pages of ledger l(when page cache is full). and the flush would fail due to NoLedgerException (no index file found).
> actually the ledger l isn't lost, it could be recovered if restarting bookie server, but the bookie server would not work well on adding entries.
> a proposal solution is that we need to force index creation when the ledger is evicted from open ledgers list.
> {code}
> 2012-03-21 14:00:42,989 - DEBUG - [NIOServerFactory-5000:LedgerCache@235] - New ledger index file created for ledgerId: 4
> 2012-03-21 14:00:42,990 - INFO - [NIOServerFactory-5000:LedgerCache@241] - Ledger 2 is evicted from file info cache.
> 2012-03-21 14:00:42,990 - DEBUG - [New I/O client worker #1-1:PerChannelBookieClient$2@255] - Successfully wrote request for adding entry: 0 ledger-id: 4 bookie: /10.82.129.173:5000 entry length: 70
> 2012-03-21 14:00:42,990 - ERROR - [NIOServerFactory-5000:BookieServer@361] - Error writing 0@4
> org.apache.bookkeeper.bookie.Bookie$NoLedgerException: Ledger 2 not found
> at org.apache.bookkeeper.bookie.LedgerCache.getFileInfo(LedgerCache.java:228)
> at org.apache.bookkeeper.bookie.LedgerCache.flushLedger(LedgerCache.java:359)
> at org.apache.bookkeeper.bookie.LedgerCache.flushLedger(LedgerCache.java:292)
> at org.apache.bookkeeper.bookie.LedgerCache.grabCleanPage(LedgerCache.java:447)
> at org.apache.bookkeeper.bookie.LedgerCache.putEntryOffset(LedgerCache.java:157)
> at org.apache.bookkeeper.bookie.LedgerDescriptor.addEntry(LedgerDescriptor.java:130)
> at org.apache.bookkeeper.bookie.Bookie.addEntryInternal(Bookie.java:1059)
> at org.apache.bookkeeper.bookie.Bookie.addEntry(Bookie.java:1099)
> at org.apache.bookkeeper.proto.BookieServer.processPacket(BookieServer.java:357)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.readRequest(NIOServerFactory.java:315)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.doIO(NIOServerFactory.java:213)
> at org.apache.bookkeeper.proto.NIOServerFactory.run(NIOServerFactory.java:124)
> 2012-03-21 14:00:42,991 - DEBUG - [pool-3-thread-1:PerChannelBookieClient@576] - Got response for add request from bookie: /10.82.129.173:5000 for ledger: 4 entry: 0 rc: 101
> 2012-03-21 14:00:42,991 - ERROR - [pool-3-thread-1:PerChannelBookieClient@594] - Add for ledger: 4, entry: 0 failed on bookie: /10.82.129.173:5000 with code: 101
> 2012-03-21 14:00:42,991 - WARN - [pool-3-thread-1:PendingAddOp@142] - Write did not succeed: 4, 0
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-190) Add entries would fail when
number of open ledgers reaches more than openFileLimit.
Posted by "Flavio Junqueira (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13236940#comment-13236940 ]
Flavio Junqueira commented on BOOKKEEPER-190:
---------------------------------------------
Hi Sijie, We introduced LedgerCacheTest (and LedgerCacheTest#testAddEntryException) in BOOKKEEPER-22.
> Add entries would fail when number of open ledgers reaches more than openFileLimit.
> -----------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-190
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-190
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.1.0
>
> Attachments: BOOKKEEPER-190.diff, BOOKKEEPER-190.diff_v2
>
>
> when the number of open ledgers reaches more than openFileLimit, a file info will be closed and removed from opened ledgers list. And after BOOKKEEPER-137, the ledger index file creation delayed until necessary.
> suppose ledger l is removed from opened ledger list, and its index file haven't been created.
> new add entries operations of other ledgers came into bookie server, a new page need to be grab for them. so bookie server may need to flush the dirty pages of ledger l(when page cache is full). and the flush would fail due to NoLedgerException (no index file found).
> actually the ledger l isn't lost, it could be recovered if restarting bookie server, but the bookie server would not work well on adding entries.
> a proposal solution is that we need to force index creation when the ledger is evicted from open ledgers list.
> {code}
> 2012-03-21 14:00:42,989 - DEBUG - [NIOServerFactory-5000:LedgerCache@235] - New ledger index file created for ledgerId: 4
> 2012-03-21 14:00:42,990 - INFO - [NIOServerFactory-5000:LedgerCache@241] - Ledger 2 is evicted from file info cache.
> 2012-03-21 14:00:42,990 - DEBUG - [New I/O client worker #1-1:PerChannelBookieClient$2@255] - Successfully wrote request for adding entry: 0 ledger-id: 4 bookie: /10.82.129.173:5000 entry length: 70
> 2012-03-21 14:00:42,990 - ERROR - [NIOServerFactory-5000:BookieServer@361] - Error writing 0@4
> org.apache.bookkeeper.bookie.Bookie$NoLedgerException: Ledger 2 not found
> at org.apache.bookkeeper.bookie.LedgerCache.getFileInfo(LedgerCache.java:228)
> at org.apache.bookkeeper.bookie.LedgerCache.flushLedger(LedgerCache.java:359)
> at org.apache.bookkeeper.bookie.LedgerCache.flushLedger(LedgerCache.java:292)
> at org.apache.bookkeeper.bookie.LedgerCache.grabCleanPage(LedgerCache.java:447)
> at org.apache.bookkeeper.bookie.LedgerCache.putEntryOffset(LedgerCache.java:157)
> at org.apache.bookkeeper.bookie.LedgerDescriptor.addEntry(LedgerDescriptor.java:130)
> at org.apache.bookkeeper.bookie.Bookie.addEntryInternal(Bookie.java:1059)
> at org.apache.bookkeeper.bookie.Bookie.addEntry(Bookie.java:1099)
> at org.apache.bookkeeper.proto.BookieServer.processPacket(BookieServer.java:357)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.readRequest(NIOServerFactory.java:315)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.doIO(NIOServerFactory.java:213)
> at org.apache.bookkeeper.proto.NIOServerFactory.run(NIOServerFactory.java:124)
> 2012-03-21 14:00:42,991 - DEBUG - [pool-3-thread-1:PerChannelBookieClient@576] - Got response for add request from bookie: /10.82.129.173:5000 for ledger: 4 entry: 0 rc: 101
> 2012-03-21 14:00:42,991 - ERROR - [pool-3-thread-1:PerChannelBookieClient@594] - Add for ledger: 4, entry: 0 failed on bookie: /10.82.129.173:5000 with code: 101
> 2012-03-21 14:00:42,991 - WARN - [pool-3-thread-1:PendingAddOp@142] - Write did not succeed: 4, 0
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-190) Add entries would fail when
number of open ledgers reaches more than openFileLimit.
Posted by "Ivan Kelly (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13236674#comment-13236674 ]
Ivan Kelly commented on BOOKKEEPER-190:
---------------------------------------
I have a few small comments on the patch.
# In evictFileInfoIfNecessary() you should use {}, so that the string isn't constructed is info level is turned off. It's not an issue here really, but it's a good habit to get into.
# Why construct a Bookie at all? The LedgerCache can be used as a standalone component now, so we should test it as such.
> Add entries would fail when number of open ledgers reaches more than openFileLimit.
> -----------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-190
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-190
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.1.0
>
> Attachments: BOOKKEEPER-190.diff, BOOKKEEPER-190.diff_v2
>
>
> when the number of open ledgers reaches more than openFileLimit, a file info will be closed and removed from opened ledgers list. And after BOOKKEEPER-137, the ledger index file creation delayed until necessary.
> suppose ledger l is removed from opened ledger list, and its index file haven't been created.
> new add entries operations of other ledgers came into bookie server, a new page need to be grab for them. so bookie server may need to flush the dirty pages of ledger l(when page cache is full). and the flush would fail due to NoLedgerException (no index file found).
> actually the ledger l isn't lost, it could be recovered if restarting bookie server, but the bookie server would not work well on adding entries.
> a proposal solution is that we need to force index creation when the ledger is evicted from open ledgers list.
> {code}
> 2012-03-21 14:00:42,989 - DEBUG - [NIOServerFactory-5000:LedgerCache@235] - New ledger index file created for ledgerId: 4
> 2012-03-21 14:00:42,990 - INFO - [NIOServerFactory-5000:LedgerCache@241] - Ledger 2 is evicted from file info cache.
> 2012-03-21 14:00:42,990 - DEBUG - [New I/O client worker #1-1:PerChannelBookieClient$2@255] - Successfully wrote request for adding entry: 0 ledger-id: 4 bookie: /10.82.129.173:5000 entry length: 70
> 2012-03-21 14:00:42,990 - ERROR - [NIOServerFactory-5000:BookieServer@361] - Error writing 0@4
> org.apache.bookkeeper.bookie.Bookie$NoLedgerException: Ledger 2 not found
> at org.apache.bookkeeper.bookie.LedgerCache.getFileInfo(LedgerCache.java:228)
> at org.apache.bookkeeper.bookie.LedgerCache.flushLedger(LedgerCache.java:359)
> at org.apache.bookkeeper.bookie.LedgerCache.flushLedger(LedgerCache.java:292)
> at org.apache.bookkeeper.bookie.LedgerCache.grabCleanPage(LedgerCache.java:447)
> at org.apache.bookkeeper.bookie.LedgerCache.putEntryOffset(LedgerCache.java:157)
> at org.apache.bookkeeper.bookie.LedgerDescriptor.addEntry(LedgerDescriptor.java:130)
> at org.apache.bookkeeper.bookie.Bookie.addEntryInternal(Bookie.java:1059)
> at org.apache.bookkeeper.bookie.Bookie.addEntry(Bookie.java:1099)
> at org.apache.bookkeeper.proto.BookieServer.processPacket(BookieServer.java:357)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.readRequest(NIOServerFactory.java:315)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.doIO(NIOServerFactory.java:213)
> at org.apache.bookkeeper.proto.NIOServerFactory.run(NIOServerFactory.java:124)
> 2012-03-21 14:00:42,991 - DEBUG - [pool-3-thread-1:PerChannelBookieClient@576] - Got response for add request from bookie: /10.82.129.173:5000 for ledger: 4 entry: 0 rc: 101
> 2012-03-21 14:00:42,991 - ERROR - [pool-3-thread-1:PerChannelBookieClient@594] - Add for ledger: 4, entry: 0 failed on bookie: /10.82.129.173:5000 with code: 101
> 2012-03-21 14:00:42,991 - WARN - [pool-3-thread-1:PendingAddOp@142] - Write did not succeed: 4, 0
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (BOOKKEEPER-190) Add entries would
fail when number of open ledgers reaches more than openFileLimit.
Posted by "Ivan Kelly (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13241168#comment-13241168 ]
Ivan Kelly edited comment on BOOKKEEPER-190 at 3/29/12 1:22 PM:
----------------------------------------------------------------
Committed as r1306839. Thanks Sijie.
was (Author: ikelly):
Committed as r1306798. Thanks Sijie.
> Add entries would fail when number of open ledgers reaches more than openFileLimit.
> -----------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-190
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-190
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Ivan Kelly
> Fix For: 4.1.0
>
> Attachments: BOOKKEEPER-190.diff, BOOKKEEPER-190.diff_v2, BOOKKEEPER-190.diff_v3
>
>
> when the number of open ledgers reaches more than openFileLimit, a file info will be closed and removed from opened ledgers list. And after BOOKKEEPER-137, the ledger index file creation delayed until necessary.
> suppose ledger l is removed from opened ledger list, and its index file haven't been created.
> new add entries operations of other ledgers came into bookie server, a new page need to be grab for them. so bookie server may need to flush the dirty pages of ledger l(when page cache is full). and the flush would fail due to NoLedgerException (no index file found).
> actually the ledger l isn't lost, it could be recovered if restarting bookie server, but the bookie server would not work well on adding entries.
> a proposal solution is that we need to force index creation when the ledger is evicted from open ledgers list.
> {code}
> 2012-03-21 14:00:42,989 - DEBUG - [NIOServerFactory-5000:LedgerCache@235] - New ledger index file created for ledgerId: 4
> 2012-03-21 14:00:42,990 - INFO - [NIOServerFactory-5000:LedgerCache@241] - Ledger 2 is evicted from file info cache.
> 2012-03-21 14:00:42,990 - DEBUG - [New I/O client worker #1-1:PerChannelBookieClient$2@255] - Successfully wrote request for adding entry: 0 ledger-id: 4 bookie: /10.82.129.173:5000 entry length: 70
> 2012-03-21 14:00:42,990 - ERROR - [NIOServerFactory-5000:BookieServer@361] - Error writing 0@4
> org.apache.bookkeeper.bookie.Bookie$NoLedgerException: Ledger 2 not found
> at org.apache.bookkeeper.bookie.LedgerCache.getFileInfo(LedgerCache.java:228)
> at org.apache.bookkeeper.bookie.LedgerCache.flushLedger(LedgerCache.java:359)
> at org.apache.bookkeeper.bookie.LedgerCache.flushLedger(LedgerCache.java:292)
> at org.apache.bookkeeper.bookie.LedgerCache.grabCleanPage(LedgerCache.java:447)
> at org.apache.bookkeeper.bookie.LedgerCache.putEntryOffset(LedgerCache.java:157)
> at org.apache.bookkeeper.bookie.LedgerDescriptor.addEntry(LedgerDescriptor.java:130)
> at org.apache.bookkeeper.bookie.Bookie.addEntryInternal(Bookie.java:1059)
> at org.apache.bookkeeper.bookie.Bookie.addEntry(Bookie.java:1099)
> at org.apache.bookkeeper.proto.BookieServer.processPacket(BookieServer.java:357)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.readRequest(NIOServerFactory.java:315)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.doIO(NIOServerFactory.java:213)
> at org.apache.bookkeeper.proto.NIOServerFactory.run(NIOServerFactory.java:124)
> 2012-03-21 14:00:42,991 - DEBUG - [pool-3-thread-1:PerChannelBookieClient@576] - Got response for add request from bookie: /10.82.129.173:5000 for ledger: 4 entry: 0 rc: 101
> 2012-03-21 14:00:42,991 - ERROR - [pool-3-thread-1:PerChannelBookieClient@594] - Add for ledger: 4, entry: 0 failed on bookie: /10.82.129.173:5000 with code: 101
> 2012-03-21 14:00:42,991 - WARN - [pool-3-thread-1:PendingAddOp@142] - Write did not succeed: 4, 0
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-190) Add entries would fail when
number of open ledgers reaches more than openFileLimit.
Posted by "Sijie Guo (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sijie Guo updated BOOKKEEPER-190:
---------------------------------
Attachment: BOOKKEEPER-190.diff_v3
thanks, Flavio.
attach a new patch.
in this patch, I modified the LedgerCacheTest according to Ivan's suggestions, which make the test focus on testing ledger cache itself.
BTW, it remove/modify 'System.out' code in LedgerCache, and also fix a simple bug LedgerCache#grabCleanPage (we don't remove a ledger from cleanLedger list if we found that there is no clean page in it. then clean ledger list would not be empty, so no flush will be trigger).
> Add entries would fail when number of open ledgers reaches more than openFileLimit.
> -----------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-190
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-190
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.1.0
>
> Attachments: BOOKKEEPER-190.diff, BOOKKEEPER-190.diff_v2, BOOKKEEPER-190.diff_v3
>
>
> when the number of open ledgers reaches more than openFileLimit, a file info will be closed and removed from opened ledgers list. And after BOOKKEEPER-137, the ledger index file creation delayed until necessary.
> suppose ledger l is removed from opened ledger list, and its index file haven't been created.
> new add entries operations of other ledgers came into bookie server, a new page need to be grab for them. so bookie server may need to flush the dirty pages of ledger l(when page cache is full). and the flush would fail due to NoLedgerException (no index file found).
> actually the ledger l isn't lost, it could be recovered if restarting bookie server, but the bookie server would not work well on adding entries.
> a proposal solution is that we need to force index creation when the ledger is evicted from open ledgers list.
> {code}
> 2012-03-21 14:00:42,989 - DEBUG - [NIOServerFactory-5000:LedgerCache@235] - New ledger index file created for ledgerId: 4
> 2012-03-21 14:00:42,990 - INFO - [NIOServerFactory-5000:LedgerCache@241] - Ledger 2 is evicted from file info cache.
> 2012-03-21 14:00:42,990 - DEBUG - [New I/O client worker #1-1:PerChannelBookieClient$2@255] - Successfully wrote request for adding entry: 0 ledger-id: 4 bookie: /10.82.129.173:5000 entry length: 70
> 2012-03-21 14:00:42,990 - ERROR - [NIOServerFactory-5000:BookieServer@361] - Error writing 0@4
> org.apache.bookkeeper.bookie.Bookie$NoLedgerException: Ledger 2 not found
> at org.apache.bookkeeper.bookie.LedgerCache.getFileInfo(LedgerCache.java:228)
> at org.apache.bookkeeper.bookie.LedgerCache.flushLedger(LedgerCache.java:359)
> at org.apache.bookkeeper.bookie.LedgerCache.flushLedger(LedgerCache.java:292)
> at org.apache.bookkeeper.bookie.LedgerCache.grabCleanPage(LedgerCache.java:447)
> at org.apache.bookkeeper.bookie.LedgerCache.putEntryOffset(LedgerCache.java:157)
> at org.apache.bookkeeper.bookie.LedgerDescriptor.addEntry(LedgerDescriptor.java:130)
> at org.apache.bookkeeper.bookie.Bookie.addEntryInternal(Bookie.java:1059)
> at org.apache.bookkeeper.bookie.Bookie.addEntry(Bookie.java:1099)
> at org.apache.bookkeeper.proto.BookieServer.processPacket(BookieServer.java:357)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.readRequest(NIOServerFactory.java:315)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.doIO(NIOServerFactory.java:213)
> at org.apache.bookkeeper.proto.NIOServerFactory.run(NIOServerFactory.java:124)
> 2012-03-21 14:00:42,991 - DEBUG - [pool-3-thread-1:PerChannelBookieClient@576] - Got response for add request from bookie: /10.82.129.173:5000 for ledger: 4 entry: 0 rc: 101
> 2012-03-21 14:00:42,991 - ERROR - [pool-3-thread-1:PerChannelBookieClient@594] - Add for ledger: 4, entry: 0 failed on bookie: /10.82.129.173:5000 with code: 101
> 2012-03-21 14:00:42,991 - WARN - [pool-3-thread-1:PendingAddOp@142] - Write did not succeed: 4, 0
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-190) Add entries would fail when
number of open ledgers reaches more than openFileLimit.
Posted by "Flavio Junqueira (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13234497#comment-13234497 ]
Flavio Junqueira commented on BOOKKEEPER-190:
---------------------------------------------
hi sijie, it looks good, but let me propose a slightly different approach for the test. I think that it should be more focused and should be really just exercising the cache here instead of emulating a full run. We could start a single bookie, create ledgers through the bookie interface, and verify that we don't get an error when evicting the ledger. How does it sound to you?
> Add entries would fail when number of open ledgers reaches more than openFileLimit.
> -----------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-190
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-190
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.1.0
>
> Attachments: BOOKKEEPER-190.diff
>
>
> when the number of open ledgers reaches more than openFileLimit, a file info will be closed and removed from opened ledgers list. And after BOOKKEEPER-137, the ledger index file creation delayed until necessary.
> suppose ledger l is removed from opened ledger list, and its index file haven't been created.
> new add entries operations of other ledgers came into bookie server, a new page need to be grab for them. so bookie server may need to flush the dirty pages of ledger l(when page cache is full). and the flush would fail due to NoLedgerException (no index file found).
> actually the ledger l isn't lost, it could be recovered if restarting bookie server, but the bookie server would not work well on adding entries.
> a proposal solution is that we need to force index creation when the ledger is evicted from open ledgers list.
> {code}
> 2012-03-21 14:00:42,989 - DEBUG - [NIOServerFactory-5000:LedgerCache@235] - New ledger index file created for ledgerId: 4
> 2012-03-21 14:00:42,990 - INFO - [NIOServerFactory-5000:LedgerCache@241] - Ledger 2 is evicted from file info cache.
> 2012-03-21 14:00:42,990 - DEBUG - [New I/O client worker #1-1:PerChannelBookieClient$2@255] - Successfully wrote request for adding entry: 0 ledger-id: 4 bookie: /10.82.129.173:5000 entry length: 70
> 2012-03-21 14:00:42,990 - ERROR - [NIOServerFactory-5000:BookieServer@361] - Error writing 0@4
> org.apache.bookkeeper.bookie.Bookie$NoLedgerException: Ledger 2 not found
> at org.apache.bookkeeper.bookie.LedgerCache.getFileInfo(LedgerCache.java:228)
> at org.apache.bookkeeper.bookie.LedgerCache.flushLedger(LedgerCache.java:359)
> at org.apache.bookkeeper.bookie.LedgerCache.flushLedger(LedgerCache.java:292)
> at org.apache.bookkeeper.bookie.LedgerCache.grabCleanPage(LedgerCache.java:447)
> at org.apache.bookkeeper.bookie.LedgerCache.putEntryOffset(LedgerCache.java:157)
> at org.apache.bookkeeper.bookie.LedgerDescriptor.addEntry(LedgerDescriptor.java:130)
> at org.apache.bookkeeper.bookie.Bookie.addEntryInternal(Bookie.java:1059)
> at org.apache.bookkeeper.bookie.Bookie.addEntry(Bookie.java:1099)
> at org.apache.bookkeeper.proto.BookieServer.processPacket(BookieServer.java:357)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.readRequest(NIOServerFactory.java:315)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.doIO(NIOServerFactory.java:213)
> at org.apache.bookkeeper.proto.NIOServerFactory.run(NIOServerFactory.java:124)
> 2012-03-21 14:00:42,991 - DEBUG - [pool-3-thread-1:PerChannelBookieClient@576] - Got response for add request from bookie: /10.82.129.173:5000 for ledger: 4 entry: 0 rc: 101
> 2012-03-21 14:00:42,991 - ERROR - [pool-3-thread-1:PerChannelBookieClient@594] - Add for ledger: 4, entry: 0 failed on bookie: /10.82.129.173:5000 with code: 101
> 2012-03-21 14:00:42,991 - WARN - [pool-3-thread-1:PendingAddOp@142] - Write did not succeed: 4, 0
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-190) Add entries would fail when
number of open ledgers reaches more than openFileLimit.
Posted by "Sijie Guo (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13236699#comment-13236699 ]
Sijie Guo commented on BOOKKEEPER-190:
--------------------------------------
> 1. In evictFileInfoIfNecessary() you should use {}, so that the string isn't constructed is info level is turned off
OK. I will did the change.
> Why construct a Bookie at all? The LedgerCache can be used as a standalone component now, so we should test it as such.
yeah. seems that testLedgerEviction could use LedgerCache#putEntryOffset to test.
but I am not so clear that LedgerCacheTest#testAddEntryException tries to test what kind of case. seems that it tried to populate the ledger cache to force eviction, which is similar with testLedgerEviction. could any one explain it?
> Add entries would fail when number of open ledgers reaches more than openFileLimit.
> -----------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-190
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-190
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.1.0
>
> Attachments: BOOKKEEPER-190.diff, BOOKKEEPER-190.diff_v2
>
>
> when the number of open ledgers reaches more than openFileLimit, a file info will be closed and removed from opened ledgers list. And after BOOKKEEPER-137, the ledger index file creation delayed until necessary.
> suppose ledger l is removed from opened ledger list, and its index file haven't been created.
> new add entries operations of other ledgers came into bookie server, a new page need to be grab for them. so bookie server may need to flush the dirty pages of ledger l(when page cache is full). and the flush would fail due to NoLedgerException (no index file found).
> actually the ledger l isn't lost, it could be recovered if restarting bookie server, but the bookie server would not work well on adding entries.
> a proposal solution is that we need to force index creation when the ledger is evicted from open ledgers list.
> {code}
> 2012-03-21 14:00:42,989 - DEBUG - [NIOServerFactory-5000:LedgerCache@235] - New ledger index file created for ledgerId: 4
> 2012-03-21 14:00:42,990 - INFO - [NIOServerFactory-5000:LedgerCache@241] - Ledger 2 is evicted from file info cache.
> 2012-03-21 14:00:42,990 - DEBUG - [New I/O client worker #1-1:PerChannelBookieClient$2@255] - Successfully wrote request for adding entry: 0 ledger-id: 4 bookie: /10.82.129.173:5000 entry length: 70
> 2012-03-21 14:00:42,990 - ERROR - [NIOServerFactory-5000:BookieServer@361] - Error writing 0@4
> org.apache.bookkeeper.bookie.Bookie$NoLedgerException: Ledger 2 not found
> at org.apache.bookkeeper.bookie.LedgerCache.getFileInfo(LedgerCache.java:228)
> at org.apache.bookkeeper.bookie.LedgerCache.flushLedger(LedgerCache.java:359)
> at org.apache.bookkeeper.bookie.LedgerCache.flushLedger(LedgerCache.java:292)
> at org.apache.bookkeeper.bookie.LedgerCache.grabCleanPage(LedgerCache.java:447)
> at org.apache.bookkeeper.bookie.LedgerCache.putEntryOffset(LedgerCache.java:157)
> at org.apache.bookkeeper.bookie.LedgerDescriptor.addEntry(LedgerDescriptor.java:130)
> at org.apache.bookkeeper.bookie.Bookie.addEntryInternal(Bookie.java:1059)
> at org.apache.bookkeeper.bookie.Bookie.addEntry(Bookie.java:1099)
> at org.apache.bookkeeper.proto.BookieServer.processPacket(BookieServer.java:357)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.readRequest(NIOServerFactory.java:315)
> at org.apache.bookkeeper.proto.NIOServerFactory$Cnxn.doIO(NIOServerFactory.java:213)
> at org.apache.bookkeeper.proto.NIOServerFactory.run(NIOServerFactory.java:124)
> 2012-03-21 14:00:42,991 - DEBUG - [pool-3-thread-1:PerChannelBookieClient@576] - Got response for add request from bookie: /10.82.129.173:5000 for ledger: 4 entry: 0 rc: 101
> 2012-03-21 14:00:42,991 - ERROR - [pool-3-thread-1:PerChannelBookieClient@594] - Add for ledger: 4, entry: 0 failed on bookie: /10.82.129.173:5000 with code: 101
> 2012-03-21 14:00:42,991 - WARN - [pool-3-thread-1:PendingAddOp@142] - Write did not succeed: 4, 0
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira