You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zookeeper.apache.org by "Flavio Paiva Junqueira (JIRA)" <ji...@apache.org> on 2009/07/24 03:04:14 UTC
[jira] Created: (ZOOKEEPER-486) Improve bookie performance for
large number of ledgers
Improve bookie performance for large number of ledgers
------------------------------------------------------
Key: ZOOKEEPER-486
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-486
Project: Zookeeper
Issue Type: Improvement
Components: contrib-bookkeeper
Reporter: Flavio Paiva Junqueira
If we write simultaneously to a large number of ledgers on a bookie, then performance drops significantly due to more seeks on the ledger device. In such cases, we should be clustering ledgers into files to reduce the number of seeks, and performing sequential writes on each file. Clustering ledgers will impact read performance, so we would to have a knob to control such a feature.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-486) Improve bookie performance for
large number of ledgers
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12790266#action_12790266 ]
Hadoop QA commented on ZOOKEEPER-486:
-------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12427942/ZOOKEEPER-486.patch
against trunk revision 889848.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 6 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/87/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/87/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/87/console
This message is automatically generated.
> Improve bookie performance for large number of ledgers
> ------------------------------------------------------
>
> Key: ZOOKEEPER-486
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-486
> Project: Zookeeper
> Issue Type: Improvement
> Components: contrib-bookkeeper
> Reporter: Flavio Paiva Junqueira
> Assignee: Benjamin Reed
> Attachments: ZOOKEEPER-486.patch, ZOOKEEPER-486.patch
>
>
> If we write simultaneously to a large number of ledgers on a bookie, then performance drops significantly due to more seeks on the ledger device. In such cases, we should be clustering ledgers into files to reduce the number of seeks, and performing sequential writes on each file. Clustering ledgers will impact read performance, so we would to have a knob to control such a feature.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (ZOOKEEPER-486) Improve bookie performance for
large number of ledgers
Posted by "Flavio Paiva Junqueira (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Flavio Paiva Junqueira resolved ZOOKEEPER-486.
----------------------------------------------
Resolution: Fixed
Fix Version/s: 3.3.0
This issue has been fixed as part of the patch of ZOOKEEPER-507.
> Improve bookie performance for large number of ledgers
> ------------------------------------------------------
>
> Key: ZOOKEEPER-486
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-486
> Project: Zookeeper
> Issue Type: Improvement
> Components: contrib-bookkeeper
> Reporter: Flavio Paiva Junqueira
> Assignee: Benjamin Reed
> Fix For: 3.3.0
>
> Attachments: ZOOKEEPER-486.patch, ZOOKEEPER-486.patch, ZOOKEEPER-486.patch, ZOOKEEPER-486.patch, ZOOKEEPER-486.patch
>
>
> If we write simultaneously to a large number of ledgers on a bookie, then performance drops significantly due to more seeks on the ledger device. In such cases, we should be clustering ledgers into files to reduce the number of seeks, and performing sequential writes on each file. Clustering ledgers will impact read performance, so we would to have a knob to control such a feature.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (ZOOKEEPER-486) Improve bookie performance for
large number of ledgers
Posted by "Flavio Paiva Junqueira (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Flavio Paiva Junqueira reassigned ZOOKEEPER-486:
------------------------------------------------
Assignee: Benjamin Reed
> Improve bookie performance for large number of ledgers
> ------------------------------------------------------
>
> Key: ZOOKEEPER-486
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-486
> Project: Zookeeper
> Issue Type: Improvement
> Components: contrib-bookkeeper
> Reporter: Flavio Paiva Junqueira
> Assignee: Benjamin Reed
>
> If we write simultaneously to a large number of ledgers on a bookie, then performance drops significantly due to more seeks on the ledger device. In such cases, we should be clustering ledgers into files to reduce the number of seeks, and performing sequential writes on each file. Clustering ledgers will impact read performance, so we would to have a knob to control such a feature.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-486) Improve bookie performance for
large number of ledgers
Posted by "Flavio Paiva Junqueira (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Flavio Paiva Junqueira updated ZOOKEEPER-486:
---------------------------------------------
Attachment: ZOOKEEPER-486.patch
Separated the bookie part of patch ZOOKEEPER-507. There is one test failing because a bookie is apparently failing to handle a missing ledger.
> Improve bookie performance for large number of ledgers
> ------------------------------------------------------
>
> Key: ZOOKEEPER-486
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-486
> Project: Zookeeper
> Issue Type: Improvement
> Components: contrib-bookkeeper
> Reporter: Flavio Paiva Junqueira
> Assignee: Benjamin Reed
> Attachments: ZOOKEEPER-486.patch
>
>
> If we write simultaneously to a large number of ledgers on a bookie, then performance drops significantly due to more seeks on the ledger device. In such cases, we should be clustering ledgers into files to reduce the number of seeks, and performing sequential writes on each file. Clustering ledgers will impact read performance, so we would to have a knob to control such a feature.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-486) Improve bookie performance for
large number of ledgers
Posted by "Utkarsh Srivastava (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12768919#action_12768919 ]
Utkarsh Srivastava commented on ZOOKEEPER-486:
----------------------------------------------
Changes to handle this are included in the patch for ZOOKEEPER-507.
> Improve bookie performance for large number of ledgers
> ------------------------------------------------------
>
> Key: ZOOKEEPER-486
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-486
> Project: Zookeeper
> Issue Type: Improvement
> Components: contrib-bookkeeper
> Reporter: Flavio Paiva Junqueira
>
> If we write simultaneously to a large number of ledgers on a bookie, then performance drops significantly due to more seeks on the ledger device. In such cases, we should be clustering ledgers into files to reduce the number of seeks, and performing sequential writes on each file. Clustering ledgers will impact read performance, so we would to have a knob to control such a feature.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-486) Improve bookie performance for
large number of ledgers
Posted by "Benjamin Reed (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Benjamin Reed updated ZOOKEEPER-486:
------------------------------------
Status: Patch Available (was: Open)
> Improve bookie performance for large number of ledgers
> ------------------------------------------------------
>
> Key: ZOOKEEPER-486
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-486
> Project: Zookeeper
> Issue Type: Improvement
> Components: contrib-bookkeeper
> Reporter: Flavio Paiva Junqueira
> Assignee: Benjamin Reed
> Attachments: ZOOKEEPER-486.patch, ZOOKEEPER-486.patch
>
>
> If we write simultaneously to a large number of ledgers on a bookie, then performance drops significantly due to more seeks on the ledger device. In such cases, we should be clustering ledgers into files to reduce the number of seeks, and performing sequential writes on each file. Clustering ledgers will impact read performance, so we would to have a knob to control such a feature.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-486) Improve bookie performance for
large number of ledgers
Posted by "Benjamin Reed (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Benjamin Reed updated ZOOKEEPER-486:
------------------------------------
Attachment: ZOOKEEPER-486.patch
added comments
> Improve bookie performance for large number of ledgers
> ------------------------------------------------------
>
> Key: ZOOKEEPER-486
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-486
> Project: Zookeeper
> Issue Type: Improvement
> Components: contrib-bookkeeper
> Reporter: Flavio Paiva Junqueira
> Assignee: Benjamin Reed
> Attachments: ZOOKEEPER-486.patch, ZOOKEEPER-486.patch, ZOOKEEPER-486.patch
>
>
> If we write simultaneously to a large number of ledgers on a bookie, then performance drops significantly due to more seeks on the ledger device. In such cases, we should be clustering ledgers into files to reduce the number of seeks, and performing sequential writes on each file. Clustering ledgers will impact read performance, so we would to have a knob to control such a feature.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-486) Improve bookie performance for
large number of ledgers
Posted by "Benjamin Reed (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Benjamin Reed updated ZOOKEEPER-486:
------------------------------------
Attachment: ZOOKEEPER-486.patch
the patch was rooted in bookkeeper rather than the top.
> Improve bookie performance for large number of ledgers
> ------------------------------------------------------
>
> Key: ZOOKEEPER-486
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-486
> Project: Zookeeper
> Issue Type: Improvement
> Components: contrib-bookkeeper
> Reporter: Flavio Paiva Junqueira
> Assignee: Benjamin Reed
> Attachments: ZOOKEEPER-486.patch, ZOOKEEPER-486.patch, ZOOKEEPER-486.patch, ZOOKEEPER-486.patch, ZOOKEEPER-486.patch
>
>
> If we write simultaneously to a large number of ledgers on a bookie, then performance drops significantly due to more seeks on the ledger device. In such cases, we should be clustering ledgers into files to reduce the number of seeks, and performing sequential writes on each file. Clustering ledgers will impact read performance, so we would to have a knob to control such a feature.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-486) Improve bookie performance for
large number of ledgers
Posted by "Benjamin Reed (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Benjamin Reed updated ZOOKEEPER-486:
------------------------------------
Attachment: ZOOKEEPER-486.patch
fixed the check for no lodger in bookie.java
> Improve bookie performance for large number of ledgers
> ------------------------------------------------------
>
> Key: ZOOKEEPER-486
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-486
> Project: Zookeeper
> Issue Type: Improvement
> Components: contrib-bookkeeper
> Reporter: Flavio Paiva Junqueira
> Assignee: Benjamin Reed
> Attachments: ZOOKEEPER-486.patch, ZOOKEEPER-486.patch
>
>
> If we write simultaneously to a large number of ledgers on a bookie, then performance drops significantly due to more seeks on the ledger device. In such cases, we should be clustering ledgers into files to reduce the number of seeks, and performing sequential writes on each file. Clustering ledgers will impact read performance, so we would to have a knob to control such a feature.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-486) Improve bookie performance for
large number of ledgers
Posted by "Benjamin Reed (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Benjamin Reed updated ZOOKEEPER-486:
------------------------------------
Attachment: ZOOKEEPER-486.patch
added the missing files.
> Improve bookie performance for large number of ledgers
> ------------------------------------------------------
>
> Key: ZOOKEEPER-486
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-486
> Project: Zookeeper
> Issue Type: Improvement
> Components: contrib-bookkeeper
> Reporter: Flavio Paiva Junqueira
> Assignee: Benjamin Reed
> Attachments: ZOOKEEPER-486.patch, ZOOKEEPER-486.patch, ZOOKEEPER-486.patch, ZOOKEEPER-486.patch
>
>
> If we write simultaneously to a large number of ledgers on a bookie, then performance drops significantly due to more seeks on the ledger device. In such cases, we should be clustering ledgers into files to reduce the number of seeks, and performing sequential writes on each file. Clustering ledgers will impact read performance, so we would to have a knob to control such a feature.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-486) Improve bookie performance for
large number of ledgers
Posted by "Flavio Paiva Junqueira (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12784717#action_12784717 ]
Flavio Paiva Junqueira commented on ZOOKEEPER-486:
--------------------------------------------------
Here is the log output for the failing test:
{noformat}
2009-12-02 09:46:22,952 - INFO - [main:BookieClientTest@198] - Starting no ledger test
2009-12-02 09:46:22,956 - INFO - [Thread-4:BookieClientTest$1@75] - Capacity 16, 0
2009-12-02 09:46:22,957 - INFO - [Thread-4:BookieClientTest$1@83] - Received 0
2009-12-02 09:46:23,001 - ERROR - [SyncThread:Bookie$LastLogMark@295] - Problems writing to /var/folders/iq/iqmyL+XiGjqE2AjRkKTe-E+++TI/-Tmp-/bookie2550661167481866393test/lastMark
java.io.FileNotFoundException: /var/folders/iq/iqmyL+XiGjqE2AjRkKTe-E+++TI/-Tmp-/bookie2550661167481866393test/lastMark (No such file or directory)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.<init>(FileOutputStream.java:179)
at java.io.FileOutputStream.<init>(FileOutputStream.java:131)
at org.apache.bookkeeper.bookie.Bookie$LastLogMark.rollLog(Bookie.java:290)
at org.apache.bookkeeper.bookie.Bookie$SyncThread.run(Bookie.java:116)
2009-12-02 09:46:23,002 - FATAL - [Thread-3:Bookie@375] - Bookie thread exiting
java.lang.InterruptedException
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:1899)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1934)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at org.apache.bookkeeper.bookie.Bookie.run(Bookie.java:341)
{noformat}
and the assert statement that fails:
{noformat}
Testcase: testNoLedger took 0.132 sec
FAILED
expected:<1> but was:<2>
junit.framework.AssertionFailedError: expected:<1> but was:<2>
at org.apache.bookkeeper.test.BookieClientTest.testNoLedger(BookieClientTest.java:204)
{noformat}
It seems to me that the problem is that the bookie is not processing the FileNotFoundException properly. It should send a ENOLEDGER value back.
> Improve bookie performance for large number of ledgers
> ------------------------------------------------------
>
> Key: ZOOKEEPER-486
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-486
> Project: Zookeeper
> Issue Type: Improvement
> Components: contrib-bookkeeper
> Reporter: Flavio Paiva Junqueira
> Assignee: Benjamin Reed
> Attachments: ZOOKEEPER-486.patch
>
>
> If we write simultaneously to a large number of ledgers on a bookie, then performance drops significantly due to more seeks on the ledger device. In such cases, we should be clustering ledgers into files to reduce the number of seeks, and performing sequential writes on each file. Clustering ledgers will impact read performance, so we would to have a knob to control such a feature.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-486) Improve bookie performance for
large number of ledgers
Posted by "Flavio Paiva Junqueira (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Flavio Paiva Junqueira updated ZOOKEEPER-486:
---------------------------------------------
Status: Open (was: Patch Available)
Ben, this patch is broken. It is missing a few classes, like LedgerCache and LedgerEntryPage.
> Improve bookie performance for large number of ledgers
> ------------------------------------------------------
>
> Key: ZOOKEEPER-486
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-486
> Project: Zookeeper
> Issue Type: Improvement
> Components: contrib-bookkeeper
> Reporter: Flavio Paiva Junqueira
> Assignee: Benjamin Reed
> Attachments: ZOOKEEPER-486.patch, ZOOKEEPER-486.patch, ZOOKEEPER-486.patch
>
>
> If we write simultaneously to a large number of ledgers on a bookie, then performance drops significantly due to more seeks on the ledger device. In such cases, we should be clustering ledgers into files to reduce the number of seeks, and performing sequential writes on each file. Clustering ledgers will impact read performance, so we would to have a knob to control such a feature.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.