You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bookkeeper.apache.org by "suja s (JIRA)" <ji...@apache.org> on 2012/06/08 12:36:22 UTC
[jira] [Created] (BOOKKEEPER-290) FileHandler leak at Bookkeeper
server side - addEntry and readEntries for ledgerHandle
suja s created BOOKKEEPER-290:
---------------------------------
Summary: FileHandler leak at Bookkeeper server side - addEntry and readEntries for ledgerHandle
Key: BOOKKEEPER-290
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-290
Project: Bookkeeper
Issue Type: Bug
Affects Versions: 4.1.0
Reporter: suja s
LSOF incrementing when the following 2 APIs are used
LedgerHandle.java
=================
public void addEntry(byte[] data) throws InterruptedException, BKException {
public Enumeration<LedgerEntry> readEntries(long firstEntry, long lastEntry)
throws InterruptedException, BKException {
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-290) FileHandler leak at Bookkeeper
server side - addEntry and readEntries for ledgerHandle
Posted by "suja s (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291714#comment-13291714 ]
suja s commented on BOOKKEEPER-290:
-----------------------------------
@Sijie
I have tested setting openFileLimit and when the number of ledgers opened in bookie server reaches the threshold old ledger files are closed.
> FileHandler leak at Bookkeeper server side - addEntry and readEntries for ledgerHandle
> --------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-290
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-290
> Project: Bookkeeper
> Issue Type: Bug
> Affects Versions: 4.0.0
> Reporter: suja s
>
> LSOF incrementing when the following 2 APIs are used
> LedgerHandle.java
> =================
> public void addEntry(byte[] data) throws InterruptedException, BKException {
> public Enumeration<LedgerEntry> readEntries(long firstEntry, long lastEntry)
> throws InterruptedException, BKException {
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-290) FileHandler leak at Bookkeeper
server side - addEntry and readEntries for ledgerHandle
Posted by "suja s (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
suja s updated BOOKKEEPER-290:
------------------------------
Affects Version/s: (was: 4.1.0)
4.0.0
> FileHandler leak at Bookkeeper server side - addEntry and readEntries for ledgerHandle
> --------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-290
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-290
> Project: Bookkeeper
> Issue Type: Bug
> Affects Versions: 4.0.0
> Reporter: suja s
>
> LSOF incrementing when the following 2 APIs are used
> LedgerHandle.java
> =================
> public void addEntry(byte[] data) throws InterruptedException, BKException {
> public Enumeration<LedgerEntry> readEntries(long firstEntry, long lastEntry)
> throws InterruptedException, BKException {
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (BOOKKEEPER-290) FileHandler leak at Bookkeeper
server side - addEntry and readEntries for ledgerHandle
Posted by "suja s (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
suja s resolved BOOKKEEPER-290.
-------------------------------
Resolution: Invalid
Invalidating as the number of fds doesn't increment after reaching openFileLimit
> FileHandler leak at Bookkeeper server side - addEntry and readEntries for ledgerHandle
> --------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-290
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-290
> Project: Bookkeeper
> Issue Type: Bug
> Affects Versions: 4.0.0
> Reporter: suja s
>
> LSOF incrementing when the following 2 APIs are used
> LedgerHandle.java
> =================
> public void addEntry(byte[] data) throws InterruptedException, BKException {
> public Enumeration<LedgerEntry> readEntries(long firstEntry, long lastEntry)
> throws InterruptedException, BKException {
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (BOOKKEEPER-290) FileHandler leak at
Bookkeeper server side - addEntry and readEntries for ledgerHandle
Posted by "Sijie Guo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291763#comment-13291763 ]
Sijie Guo edited comment on BOOKKEEPER-290 at 6/8/12 1:21 PM:
--------------------------------------------------------------
@suja, the number of fds doesn't increment after reaching openFileLimit, right? so this jira should be not a bug.
was (Author: hustlmsp):
@suja, the number of fds doesn't increment after reaching openFileLimit, right? so this jira should be a bug.
> FileHandler leak at Bookkeeper server side - addEntry and readEntries for ledgerHandle
> --------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-290
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-290
> Project: Bookkeeper
> Issue Type: Bug
> Affects Versions: 4.0.0
> Reporter: suja s
>
> LSOF incrementing when the following 2 APIs are used
> LedgerHandle.java
> =================
> public void addEntry(byte[] data) throws InterruptedException, BKException {
> public Enumeration<LedgerEntry> readEntries(long firstEntry, long lastEntry)
> throws InterruptedException, BKException {
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-290) FileHandler leak at Bookkeeper
server side - addEntry and readEntries for ledgerHandle
Posted by "Sijie Guo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291693#comment-13291693 ]
Sijie Guo commented on BOOKKEEPER-290:
--------------------------------------
@suja
yeah, this is an expected behavior because for each ledger there is a ledger index file for it in bookie server.
you could set a limitation on open ledger files in bookie server by setting openFileLimit. if number of ledgers opened in bookie server reaches *openFileLimit* threshold, old ledger file will be evicted to close.
BTW, How many ledgers you write for you test? and what is your openFileLimit setting?
> FileHandler leak at Bookkeeper server side - addEntry and readEntries for ledgerHandle
> --------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-290
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-290
> Project: Bookkeeper
> Issue Type: Bug
> Affects Versions: 4.0.0
> Reporter: suja s
>
> LSOF incrementing when the following 2 APIs are used
> LedgerHandle.java
> =================
> public void addEntry(byte[] data) throws InterruptedException, BKException {
> public Enumeration<LedgerEntry> readEntries(long firstEntry, long lastEntry)
> throws InterruptedException, BKException {
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-290) FileHandler leak at Bookkeeper
server side - addEntry and readEntries for ledgerHandle
Posted by "suja s (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291687#comment-13291687 ]
suja s commented on BOOKKEEPER-290:
-----------------------------------
Hi,
The following is the scenario.
1. Create a new ledger.
2. Add entry using the ledger handle.
3. Close the handle.
For every new ledger being created one handle is incremented.
> FileHandler leak at Bookkeeper server side - addEntry and readEntries for ledgerHandle
> --------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-290
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-290
> Project: Bookkeeper
> Issue Type: Bug
> Affects Versions: 4.0.0
> Reporter: suja s
>
> LSOF incrementing when the following 2 APIs are used
> LedgerHandle.java
> =================
> public void addEntry(byte[] data) throws InterruptedException, BKException {
> public Enumeration<LedgerEntry> readEntries(long firstEntry, long lastEntry)
> throws InterruptedException, BKException {
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-290) FileHandler leak at Bookkeeper
server side - addEntry and readEntries for ledgerHandle
Posted by "Sijie Guo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291763#comment-13291763 ]
Sijie Guo commented on BOOKKEEPER-290:
--------------------------------------
@suja, the number of fds doesn't increment after reaching openFileLimit, right? so this jira should be a bug.
> FileHandler leak at Bookkeeper server side - addEntry and readEntries for ledgerHandle
> --------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-290
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-290
> Project: Bookkeeper
> Issue Type: Bug
> Affects Versions: 4.0.0
> Reporter: suja s
>
> LSOF incrementing when the following 2 APIs are used
> LedgerHandle.java
> =================
> public void addEntry(byte[] data) throws InterruptedException, BKException {
> public Enumeration<LedgerEntry> readEntries(long firstEntry, long lastEntry)
> throws InterruptedException, BKException {
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-290) FileHandler leak at Bookkeeper
server side - addEntry and readEntries for ledgerHandle
Posted by "Sijie Guo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291676#comment-13291676 ]
Sijie Guo commented on BOOKKEEPER-290:
--------------------------------------
@suja
could you describe more about your issue?
are you writing/reading same ledger? or lots of different ledgers? If you are writing/reading different ledgers, each ledger would have a ledger index file in bookie server. there would be lots of files opened for writing/reading.
> FileHandler leak at Bookkeeper server side - addEntry and readEntries for ledgerHandle
> --------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-290
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-290
> Project: Bookkeeper
> Issue Type: Bug
> Affects Versions: 4.1.0
> Reporter: suja s
>
> LSOF incrementing when the following 2 APIs are used
> LedgerHandle.java
> =================
> public void addEntry(byte[] data) throws InterruptedException, BKException {
> public Enumeration<LedgerEntry> readEntries(long firstEntry, long lastEntry)
> throws InterruptedException, BKException {
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-290) FileHandler leak at Bookkeeper
server side - addEntry and readEntries for ledgerHandle
Posted by "Flavio Junqueira (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291688#comment-13291688 ]
Flavio Junqueira commented on BOOKKEEPER-290:
---------------------------------------------
This is still not enough detail. I don't understand where the handle leak is.
> FileHandler leak at Bookkeeper server side - addEntry and readEntries for ledgerHandle
> --------------------------------------------------------------------------------------
>
> Key: BOOKKEEPER-290
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-290
> Project: Bookkeeper
> Issue Type: Bug
> Affects Versions: 4.0.0
> Reporter: suja s
>
> LSOF incrementing when the following 2 APIs are used
> LedgerHandle.java
> =================
> public void addEntry(byte[] data) throws InterruptedException, BKException {
> public Enumeration<LedgerEntry> readEntries(long firstEntry, long lastEntry)
> throws InterruptedException, BKException {
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira