You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bookkeeper.apache.org by "Sijie Guo (Created) (JIRA)" <ji...@apache.org> on 2012/03/13 08:21:39 UTC
[jira] [Created] (BOOKKEEPER-184) CompactionTest failing on Jenkins
CompactionTest failing on Jenkins
---------------------------------
Key: BOOKKEEPER-184
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-184
Project: Bookkeeper
Issue Type: Bug
Components: bookkeeper-server
Reporter: Sijie Guo
Assignee: Sijie Guo
Fix For: 4.1.0
the compaction test hanging on Jenkins.
Ivan did some investigation on this issue, and found that it wasn't hanging, it was just taking a really long time. He suggested that it is because it does a lot of I/O, theres a couple of ways we could reduce this.
1. Only create 1 bookie
2. Not inherit from BaseTestCase
3. Reduce the number of entries
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-184) CompactionTest failing on Jenkins
Posted by "Sijie Guo (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sijie Guo updated BOOKKEEPER-184:
---------------------------------
Attachment: BK-184.diff
attach a patch according to Ivan's suggestion.
1) move bookkeeper related code into BookKeeperClusterTestCase, which don't need to run multiple time for different digestType. BaseTestCase inherits BookKeeperClusterTestCase, which is used to test digestType related cases.
2) reduce num of bookies to 1 in CompactionTest
3) reduce number of entries to 100 in CompactionTest
> CompactionTest failing on Jenkins
> ---------------------------------
>
> Key: BOOKKEEPER-184
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-184
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.1.0
>
> Attachments: BK-184.diff
>
>
> the compaction test hanging on Jenkins.
> Ivan did some investigation on this issue, and found that it wasn't hanging, it was just taking a really long time. He suggested that it is because it does a lot of I/O, theres a couple of ways we could reduce this.
> 1. Only create 1 bookie
> 2. Not inherit from BaseTestCase
> 3. Reduce the number of entries
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-184) CompactionTest failing on
Jenkins
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13228267#comment-13228267 ]
jiraposter@reviews.apache.org commented on BOOKKEEPER-184:
----------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4304/
-----------------------------------------------------------
Review request for bookkeeper.
Summary
-------
he compaction test hanging on Jenkins.
Ivan did some investigation on this issue, and found that it wasn't hanging, it was just taking a really long time. He suggested that it is because it does a lot of I/O, theres a couple of ways we could reduce this.
1. Only create 1 bookie
2. Not inherit from BaseTestCase
3. Reduce the number of entries
This addresses bug BOOKKEEPER-184.
https://issues.apache.org/jira/browse/BOOKKEEPER-184
Diffs
-----
bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/CompactionTest.java 34dcc74
bookkeeper-server/src/test/java/org/apache/bookkeeper/client/TestReadTimeout.java 8da000e
bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/HierarchicalLedgerDeleteTest.java 6fd1c5c
bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/LedgerLayoutTest.java 8235ce5
bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/TestLedgerManager.java 099eb7c
bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BaseTestCase.java 0fe5aba
bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BookKeeperClusterTestCase.java PRE-CREATION
bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BookieJournalRollingTest.java 17f5da2
bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BookieZKExpireTest.java ee5d24e
bookkeeper-server/src/test/java/org/apache/bookkeeper/test/IndexCorruptionTest.java 5dcb964
bookkeeper-server/src/test/java/org/apache/bookkeeper/test/LedgerDeleteTest.java 5364fdd
Diff: https://reviews.apache.org/r/4304/diff
Testing
-------
Thanks,
Sijie
> CompactionTest failing on Jenkins
> ---------------------------------
>
> Key: BOOKKEEPER-184
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-184
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.1.0
>
>
> the compaction test hanging on Jenkins.
> Ivan did some investigation on this issue, and found that it wasn't hanging, it was just taking a really long time. He suggested that it is because it does a lot of I/O, theres a couple of ways we could reduce this.
> 1. Only create 1 bookie
> 2. Not inherit from BaseTestCase
> 3. Reduce the number of entries
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-184) CompactionTest failing on
Jenkins
Posted by "Sijie Guo (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13228342#comment-13228342 ]
Sijie Guo commented on BOOKKEEPER-184:
--------------------------------------
the disabling of compaction did work before this change, which use #restartBookies()
{code}
// before restartBookies
2012-03-13 19:43:05,553 - INFO - [main:GarbageCollectorThread@169] - Minor Compaction : enabled=true, threshold=0.10000000149011612, interval=2000
2012-03-13 19:43:05,553 - INFO - [main:GarbageCollectorThread@171] - Major Compaction : enabled=true, threshold=0.5, interval=4000
// after restartBookies
2012-03-13 19:43:35,123 - INFO - [main:GarbageCollectorThread@169] - Minor Compaction : enabled=false, threshold=0.0, interval=2000
2012-03-13 19:43:35,123 - INFO - [main:GarbageCollectorThread@171] - Major Compaction : enabled=false, threshold=0.0, interval=4000
{code}
because ServerConfiguration is a CompositeConfiguration. A CompositeConfiguration object contains a list of other configuration objects. When properties are accessed from a composite configuration the object takes the passed in property key and iterates over the list of the contained configurations. So when we changed baseConf, the changes would be took placed thru CompositeConfiguration.
This change use #restartBookies(baseConf), which just make code more clearly. it is same as #restartBookies()
> CompactionTest failing on Jenkins
> ---------------------------------
>
> Key: BOOKKEEPER-184
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-184
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.1.0
>
> Attachments: BK-184.diff
>
>
> the compaction test hanging on Jenkins.
> Ivan did some investigation on this issue, and found that it wasn't hanging, it was just taking a really long time. He suggested that it is because it does a lot of I/O, theres a couple of ways we could reduce this.
> 1. Only create 1 bookie
> 2. Not inherit from BaseTestCase
> 3. Reduce the number of entries
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-184) CompactionTest failing on
Jenkins
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13230036#comment-13230036 ]
Hudson commented on BOOKKEEPER-184:
-----------------------------------
Integrated in bookkeeper-trunk #410 (See [https://builds.apache.org/job/bookkeeper-trunk/410/])
BOOKKEEPER-184: CompactionTest failing on Jenkins (sijie via ivank) (Revision 1300862)
Result = ABORTED
ivank :
Files :
* /zookeeper/bookkeeper/trunk/CHANGES.txt
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/CompactionTest.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/client/TestReadTimeout.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/HierarchicalLedgerDeleteTest.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/LedgerLayoutTest.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/meta/TestLedgerManager.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BaseTestCase.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BookKeeperClusterTestCase.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BookieJournalRollingTest.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/test/BookieZKExpireTest.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/test/IndexCorruptionTest.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/test/LedgerDeleteTest.java
> CompactionTest failing on Jenkins
> ---------------------------------
>
> Key: BOOKKEEPER-184
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-184
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Ivan Kelly
> Fix For: 4.1.0
>
> Attachments: BK-184.diff
>
>
> the compaction test hanging on Jenkins.
> Ivan did some investigation on this issue, and found that it wasn't hanging, it was just taking a really long time. He suggested that it is because it does a lot of I/O, theres a couple of ways we could reduce this.
> 1. Only create 1 bookie
> 2. Not inherit from BaseTestCase
> 3. Reduce the number of entries
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-184) CompactionTest failing on
Jenkins
Posted by "Ivan Kelly (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13230087#comment-13230087 ]
Ivan Kelly commented on BOOKKEEPER-184:
---------------------------------------
Regarding the test working before the change, I had read the logs wrong. It was working fine :), just slow
> CompactionTest failing on Jenkins
> ---------------------------------
>
> Key: BOOKKEEPER-184
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-184
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Ivan Kelly
> Fix For: 4.1.0
>
> Attachments: BK-184.diff
>
>
> the compaction test hanging on Jenkins.
> Ivan did some investigation on this issue, and found that it wasn't hanging, it was just taking a really long time. He suggested that it is because it does a lot of I/O, theres a couple of ways we could reduce this.
> 1. Only create 1 bookie
> 2. Not inherit from BaseTestCase
> 3. Reduce the number of entries
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-184) CompactionTest failing on
Jenkins
Posted by "Ivan Kelly (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13228324#comment-13228324 ]
Ivan Kelly commented on BOOKKEEPER-184:
---------------------------------------
Did you figure out why the test cases were not failing? Before this change, the disabling of compaction in #testDisableCompaction had no affect. However, the test still passed happily, which indicates that the test wasn't correct.
> CompactionTest failing on Jenkins
> ---------------------------------
>
> Key: BOOKKEEPER-184
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-184
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-server
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.1.0
>
> Attachments: BK-184.diff
>
>
> the compaction test hanging on Jenkins.
> Ivan did some investigation on this issue, and found that it wasn't hanging, it was just taking a really long time. He suggested that it is because it does a lot of I/O, theres a couple of ways we could reduce this.
> 1. Only create 1 bookie
> 2. Not inherit from BaseTestCase
> 3. Reduce the number of entries
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira