You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zookeeper.apache.org by "Mahadev konar (JIRA)" <ji...@apache.org> on 2010/03/09 02:43:27 UTC
[jira] Created: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
AsyncTestHammer test fails on hudson.
-------------------------------------
Key: ZOOKEEPER-690
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
Project: Zookeeper
Issue Type: Bug
Reporter: Mahadev konar
Assignee: Mahadev konar
Priority: Blocker
Fix For: 3.4.0
the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Alan Cabrera (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12858980#action_12858980 ]
Alan Cabrera commented on ZOOKEEPER-690:
----------------------------------------
I reran the tests a minute or two later, without having done anything, and the test got hung. I attached the thread dump to this issue.
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Patrick Hunt
> Priority: Critical
> Fix For: 3.3.1
>
> Attachments: TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Flavio Paiva Junqueira (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Flavio Paiva Junqueira updated ZOOKEEPER-690:
---------------------------------------------
Fix Version/s: (was: 3.3.1)
Priority: Major (was: Blocker)
Removing the blocker flag and making it major for 3.4.0.
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Henry Robinson
> Fix For: 3.4.0
>
> Attachments: jstack-201004201053.txt, jstack-201004291409.txt, jstack-201004291527.txt, jstack-AsyncHammerTest-201004301209.txt, nohup-201004201053.txt, nohup-201004291409.txt, nohup-201004291527.txt, nohup-AsyncHammerTest-201004301209.txt, nohup-QuorumPeerMainTest-201004301209.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Henry Robinson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12862351#action_12862351 ]
Henry Robinson commented on ZOOKEEPER-690:
------------------------------------------
Alan - can you try this patch to see if it fixes things?
Thanks,
Henry
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Henry Robinson
> Priority: Blocker
> Fix For: 3.3.1, 3.4.0
>
> Attachments: jstack-201004201053.txt, nohup-201004201053.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log, ZOOKEEPER-690.patch
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Alan Cabrera (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12863922#action_12863922 ]
Alan Cabrera commented on ZOOKEEPER-690:
----------------------------------------
Correct. The child seem to have completed. The parent was the one that seemed to have locked up.
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Henry Robinson
> Priority: Blocker
> Fix For: 3.3.1, 3.4.0
>
> Attachments: jstack-201004201053.txt, jstack-201004291409.txt, jstack-201004291527.txt, jstack-AsyncHammerTest-201004301209.txt, nohup-201004201053.txt, nohup-201004291409.txt, nohup-201004291527.txt, nohup-AsyncHammerTest-201004301209.txt, nohup-QuorumPeerMainTest-201004301209.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Alan Cabrera (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alan Cabrera updated ZOOKEEPER-690:
-----------------------------------
Attachment: nohup-AsyncHammerTest-201004301209.txt
jstack-AsyncHammerTest-201004301209.txt
nohup-QuorumPeerMainTest-201004301209.txt
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Henry Robinson
> Priority: Blocker
> Fix For: 3.3.1, 3.4.0
>
> Attachments: jstack-201004201053.txt, jstack-201004291409.txt, jstack-201004291527.txt, jstack-AsyncHammerTest-201004301209.txt, nohup-201004201053.txt, nohup-201004291409.txt, nohup-201004291527.txt, nohup-AsyncHammerTest-201004301209.txt, nohup-QuorumPeerMainTest-201004301209.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Henry Robinson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12862400#action_12862400 ]
Henry Robinson commented on ZOOKEEPER-690:
------------------------------------------
Note that both testHammer and testObserversHammer fail in the most recent log.
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Henry Robinson
> Priority: Blocker
> Fix For: 3.3.1, 3.4.0
>
> Attachments: jstack-201004201053.txt, nohup-201004201053.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log, ZOOKEEPER-690.patch
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Alan Cabrera (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alan Cabrera updated ZOOKEEPER-690:
-----------------------------------
Attachment: (was: TEST-org.apache.zookeeper.test.AsyncHammerTest.txt)
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Henry Robinson
> Priority: Blocker
> Fix For: 3.3.1, 3.4.0
>
> Attachments: jstack-201004201053.txt, nohup-201004201053.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log, ZOOKEEPER-690.patch
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Benjamin Reed (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12863917#action_12863917 ]
Benjamin Reed commented on ZOOKEEPER-690:
-----------------------------------------
alan it looks like you took the jstack of the parent process rather than the child.
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Henry Robinson
> Priority: Blocker
> Fix For: 3.3.1, 3.4.0
>
> Attachments: jstack-201004201053.txt, jstack-201004291409.txt, jstack-201004291527.txt, jstack-AsyncHammerTest-201004301209.txt, nohup-201004201053.txt, nohup-201004291409.txt, nohup-201004291527.txt, nohup-AsyncHammerTest-201004301209.txt, nohup-QuorumPeerMainTest-201004301209.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Alan Cabrera (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12858997#action_12858997 ]
Alan Cabrera commented on ZOOKEEPER-690:
----------------------------------------
One interesting thing I noticed is that after I kill the process it turns into a zombie process.
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Patrick Hunt
> Priority: Critical
> Fix For: 3.3.1
>
> Attachments: jstack-201004201053.txt, nohup-201004201053.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Alan Cabrera (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alan Cabrera updated ZOOKEEPER-690:
-----------------------------------
Attachment: TEST-org.apache.zookeeper.test.AsyncHammerTest.txt
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Henry Robinson
> Priority: Blocker
> Fix For: 3.3.1, 3.4.0
>
> Attachments: jstack-201004201053.txt, nohup-201004201053.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log, ZOOKEEPER-690.patch
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12864014#action_12864014 ]
Patrick Hunt commented on ZOOKEEPER-690:
----------------------------------------
I think Flavio is saying it works for him.
Alan, any idea what's different btw the versions that are running fine, and the one where it fails? Java VM version? CPU core count, networking setup, is there something that stands out?
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Henry Robinson
> Fix For: 3.4.0
>
> Attachments: jstack-201004201053.txt, jstack-201004291409.txt, jstack-201004291527.txt, jstack-AsyncHammerTest-201004301209.txt, nohup-201004201053.txt, nohup-201004291409.txt, nohup-201004291527.txt, nohup-AsyncHammerTest-201004301209.txt, nohup-QuorumPeerMainTest-201004301209.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated ZOOKEEPER-690:
------------------------------------
Fix Version/s: (was: 3.4.0)
3.3.0
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Mahadev konar
> Priority: Blocker
> Fix For: 3.3.0
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Alan Cabrera (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12864009#action_12864009 ]
Alan Cabrera commented on ZOOKEEPER-690:
----------------------------------------
Are you saying that the AsyncHammerTest also fails on your machines w/ out the patch? I ask this because {{trunk}}, with no patches, builds and tests fine on all my other Macs except the one at work.
My JAVA_HOME is properly set.
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Henry Robinson
> Priority: Blocker
> Fix For: 3.3.1, 3.4.0
>
> Attachments: jstack-201004201053.txt, jstack-201004291409.txt, jstack-201004291527.txt, jstack-AsyncHammerTest-201004301209.txt, nohup-201004201053.txt, nohup-201004291409.txt, nohup-201004291527.txt, nohup-AsyncHammerTest-201004301209.txt, nohup-QuorumPeerMainTest-201004301209.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Flavio Paiva Junqueira (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12861909#action_12861909 ]
Flavio Paiva Junqueira commented on ZOOKEEPER-690:
--------------------------------------------------
FLE relies upon a correct implementation of the voting view. My understanding is that if an observer is being elected leader, then the following predicate is evaluating to false:
{noformat}
(!self.getVotingView().containsKey(response.sid))
{noformat}
This is line 201 of FastLeaderElection.java. If this predicate is true, meaning that an observer is not in the voting view of a server, then the server will send a response right away and won't consider the vote of the observer. Does it make sense?
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Henry Robinson
> Priority: Blocker
> Fix For: 3.3.1, 3.4.0
>
> Attachments: jstack-201004201053.txt, nohup-201004201053.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Henry Robinson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12862482#action_12862482 ]
Henry Robinson commented on ZOOKEEPER-690:
------------------------------------------
Ben -
Agreed. I see this as the same as setMyid(...) - it sets an immutable value and should only be called once. I'd prefer if these parameters were 'final' in QuorumPeer and set in the constructor, but that's not the way that runFromConfig (the only place outside of tests that these methods are called) is written. Then we could get rid of setLearnerType, for sure.
The real error here, I think, is duplicating the learnertype between QuorumPeer and QuorumServer. If we are going to have the list of QuorumServers, then getLearnerType should lookup the learner type in the peer map. Same for the serverid, perhaps, and we should just save a reference to the QuorumServer that represents our Quorumpeer.
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Henry Robinson
> Priority: Blocker
> Fix For: 3.3.1, 3.4.0
>
> Attachments: jstack-201004201053.txt, jstack-201004291409.txt, jstack-201004291527.txt, nohup-201004201053.txt, nohup-201004291409.txt, nohup-201004291527.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Benjamin Reed (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12862414#action_12862414 ]
Benjamin Reed commented on ZOOKEEPER-690:
-----------------------------------------
i think the key fix is here:
{quote}
public void setLearnerType(LearnerType p) {
learnerType = p;
if (quorumPeers.containsValue(this.myid)) {
this.quorumPeers.get(myid).type = p;
} else {
LOG.error("Setting LearnerType to " + p + " but " + myid
+ " not in QuorumPeers. ");
}
...
}
{quote}
right?
the problem i see is that we are only updating the quorumPeers for the one peer. the other peers are going to be thinking it is a participant.
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Henry Robinson
> Priority: Blocker
> Fix For: 3.3.1, 3.4.0
>
> Attachments: jstack-201004201053.txt, jstack-201004291409.txt, nohup-201004201053.txt, nohup-201004291409.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log, ZOOKEEPER-690.patch
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12862594#action_12862594 ]
Hadoop QA commented on ZOOKEEPER-690:
-------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12443246/ZOOKEEPER-690.patch
against trunk revision 939172.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 12 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/77/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/77/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/77/console
This message is automatically generated.
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Henry Robinson
> Priority: Blocker
> Fix For: 3.3.1, 3.4.0
>
> Attachments: jstack-201004201053.txt, jstack-201004291409.txt, jstack-201004291527.txt, nohup-201004201053.txt, nohup-201004291409.txt, nohup-201004291527.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Flavio Paiva Junqueira (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12863996#action_12863996 ]
Flavio Paiva Junqueira commented on ZOOKEEPER-690:
--------------------------------------------------
+1, the patch looks good.
I believe I have the same setup as Alan: Mac OS X 10.5.8 and the same java version. It runs fine for me. I have run from Eclipse and from the command line multiple times, and I observed no problem. I have also asked a colleague with the same setup to try to reproduce, and also runs for my colleague.
Alan, could you make sure that JAVA_HOME is set to the correct path before you run ant from the command line? If the variable is not set, then ant might not pick the correct java version to run, even if it is marked in the java preferences utility.
We should keep investigating this problem, but given that it runs in two computers with similar configurations, I recommend we remove the blocker flag for this issue.
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Henry Robinson
> Priority: Blocker
> Fix For: 3.3.1, 3.4.0
>
> Attachments: jstack-201004201053.txt, jstack-201004291409.txt, jstack-201004291527.txt, jstack-AsyncHammerTest-201004301209.txt, nohup-201004201053.txt, nohup-201004291409.txt, nohup-201004291527.txt, nohup-AsyncHammerTest-201004301209.txt, nohup-QuorumPeerMainTest-201004301209.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12858984#action_12858984 ]
Patrick Hunt commented on ZOOKEEPER-690:
----------------------------------------
Can you attach the log, that would be very useful for me to review prior to digging into the thread dump.
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Patrick Hunt
> Priority: Critical
> Fix For: 3.3.1
>
> Attachments: TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Henry Robinson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Henry Robinson updated ZOOKEEPER-690:
-------------------------------------
Status: Patch Available (was: Open)
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Henry Robinson
> Priority: Blocker
> Fix For: 3.3.1, 3.4.0
>
> Attachments: jstack-201004201053.txt, nohup-201004201053.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log, ZOOKEEPER-690.patch
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Henry Robinson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Henry Robinson updated ZOOKEEPER-690:
-------------------------------------
Attachment: ZOOKEEPER-690.patch
Fixing that bug unmasked another ugliness with the way that tests were started and stopped. This patch makes the process cleaner. Let's see if it helps, Alan?
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Henry Robinson
> Priority: Blocker
> Fix For: 3.3.1, 3.4.0
>
> Attachments: jstack-201004201053.txt, jstack-201004291409.txt, jstack-201004291527.txt, nohup-201004201053.txt, nohup-201004291409.txt, nohup-201004291527.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Henry Robinson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Henry Robinson updated ZOOKEEPER-690:
-------------------------------------
Attachment: ZOOKEEPER-690.patch
Alan - would you mind trying this new patch? Thanks for your patience. I suspect that something might still be a bit flaky with these tests (not the code, but the tests), but I hope this will fix this particular problem.
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Henry Robinson
> Priority: Blocker
> Fix For: 3.3.1, 3.4.0
>
> Attachments: jstack-201004201053.txt, jstack-201004291409.txt, nohup-201004201053.txt, nohup-201004291409.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Henry Robinson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12861865#action_12861865 ]
Henry Robinson commented on ZOOKEEPER-690:
------------------------------------------
Progress update - possibly to do with a bug in FLE allowing an Observer to be elected. We're looking into this now.
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Henry Robinson
> Priority: Blocker
> Fix For: 3.3.1, 3.4.0
>
> Attachments: jstack-201004201053.txt, nohup-201004201053.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Hunt updated ZOOKEEPER-690:
-----------------------------------
Fix Version/s: 3.4.0
Priority: Blocker (was: Critical)
Moving this as a blocker for 3.3.1 - I think we need to fix this as part of the release.
Henry, any progress? It would be great to get 3.3.1 to a vote soon (to get out a fix for the stat issue)
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Patrick Hunt
> Priority: Blocker
> Fix For: 3.3.1, 3.4.0
>
> Attachments: jstack-201004201053.txt, nohup-201004201053.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Henry Robinson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12858665#action_12858665 ]
Henry Robinson commented on ZOOKEEPER-690:
------------------------------------------
Alan - that would be great. If you can take a jstack dump of the process when it hangs we can do some forensics.
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Patrick Hunt
> Priority: Critical
> Fix For: 3.3.1
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Alan Cabrera (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12858644#action_12858644 ]
Alan Cabrera commented on ZOOKEEPER-690:
----------------------------------------
This consistently hangs on my machine. I'll try to get some time to see why this is so.
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Patrick Hunt
> Priority: Critical
> Fix For: 3.3.1
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Hunt reassigned ZOOKEEPER-690:
--------------------------------------
Assignee: Henry Robinson (was: Patrick Hunt)
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Henry Robinson
> Priority: Blocker
> Fix For: 3.3.1, 3.4.0
>
> Attachments: jstack-201004201053.txt, nohup-201004201053.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Henry Robinson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12862424#action_12862424 ]
Henry Robinson commented on ZOOKEEPER-690:
------------------------------------------
This map is, I think, shared between the quorumpeers for the purposes of the test (and in general there aren't two quorumpeers sharing this datastructure when running normally).
But! The error here is that I'm dumb (and that Java's type-checking leaves a little to be desired). I've written quorumPeers.containsValue up there, but actually it should be quorumPeers.containsKey. New patch on the way, let's see if that fixes it.
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Henry Robinson
> Priority: Blocker
> Fix For: 3.3.1, 3.4.0
>
> Attachments: jstack-201004201053.txt, jstack-201004291409.txt, nohup-201004201053.txt, nohup-201004291409.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log, ZOOKEEPER-690.patch
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12858767#action_12858767 ]
Patrick Hunt commented on ZOOKEEPER-690:
----------------------------------------
The following snippets are from TEST-org.apache.zookeeper.test.AsyncHammerTest.txt
Looks like there's a problem with the address already being in use. Maybe you can
try editing src/java/test/org/apache/zookeeper/PortAssignment.java and see if that helps?
2010-04-19 14:31:55,010 - INFO [main:QuorumBase@88] - Ports are: 127.0.0.1:11222,127.0.0.1:11223,127.0.0.1:11224,127.0.0.1:11225,127.0.0.1:11226
2010-04-19 14:31:55,015 - INFO [main:QuorumBase@145] - creating QuorumPeer 1 port 11222
2010-04-19 14:31:55,062 - INFO [main:NIOServerCnxn$Factory@144] - binding to port 0.0.0.0/0.0.0.0:11222
2010-04-19 14:31:55,063 - INFO [main:AsyncHammerTest@68] - Test clients shutting down
2010-04-19 14:31:55,064 - INFO [main:QuorumBase@254] - TearDown started
Testcase: testHammer took 0.443 sec
Caused an ERROR
Address already in use
java.net.BindException: Address already in use
at sun.nio.ch.Net.bind(Native Method)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:119)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:59)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:52)
at org.apache.zookeeper.server.NIOServerCnxn$Factory.<init>(NIOServerCnxn.java:145)
at org.apache.zookeeper.server.NIOServerCnxn$Factory.<init>(NIOServerCnxn.java:126)
at org.apache.zookeeper.server.quorum.QuorumPeer.<init>(QuorumPeer.java:450)
at org.apache.zookeeper.test.QuorumBase.startServers(QuorumBase.java:146)
at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:96)
at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:60)
at org.apache.zookeeper.test.AsyncHammerTest.setUp(AsyncHammerTest.java:54)
Caused an ERROR
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Patrick Hunt
> Priority: Critical
> Fix For: 3.3.1
>
> Attachments: TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Alan Cabrera (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alan Cabrera updated ZOOKEEPER-690:
-----------------------------------
Attachment: jstack-201004201053.txt
nohup-201004201053.txt
Stack dump and log.
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Patrick Hunt
> Priority: Critical
> Fix For: 3.3.1
>
> Attachments: jstack-201004201053.txt, nohup-201004201053.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Alan Cabrera (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12858986#action_12858986 ]
Alan Cabrera commented on ZOOKEEPER-690:
----------------------------------------
There's nothing in the log:
{code}
[acabrera-md:zookeeper 507]$ cat build/test/logs/TEST-org.apache.zookeeper.test.AsyncHammerTest.txt
Testsuite: org.apache.zookeeper.test.AsyncHammerTest
{code}
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Patrick Hunt
> Priority: Critical
> Fix For: 3.3.1
>
> Attachments: TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Alan Cabrera (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alan Cabrera updated ZOOKEEPER-690:
-----------------------------------
Attachment: nohup-201004291409.txt
jstack-201004291409.txt
Disregard the last comment.
I have now cleaned and rebuilt and now it still locks up.
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Henry Robinson
> Priority: Blocker
> Fix For: 3.3.1, 3.4.0
>
> Attachments: jstack-201004201053.txt, jstack-201004291409.txt, nohup-201004201053.txt, nohup-201004291409.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log, ZOOKEEPER-690.patch
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Alan Cabrera (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12863912#action_12863912 ]
Alan Cabrera commented on ZOOKEEPER-690:
----------------------------------------
The first run is the run where Ant locks up. The second run is the subsequent run after the first run zombies.
Rebooting clears everything for the first test run to lock up and become zombied.
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Henry Robinson
> Priority: Blocker
> Fix For: 3.3.1, 3.4.0
>
> Attachments: jstack-201004201053.txt, jstack-201004291409.txt, jstack-201004291527.txt, jstack-AsyncHammerTest-201004301209.txt, nohup-201004201053.txt, nohup-201004291409.txt, nohup-201004291527.txt, nohup-AsyncHammerTest-201004301209.txt, nohup-QuorumPeerMainTest-201004301209.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Henry Robinson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12863902#action_12863902 ]
Henry Robinson commented on ZOOKEEPER-690:
------------------------------------------
Hi Alan -
Looking at this attachment: nohup-AsyncHammerTest-201004301209.txt - the tests appear to be run twice. The first testObserversHammer completes successfully, the second fails. Were you running the tests until you experienced the failure?
Henry
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Henry Robinson
> Priority: Blocker
> Fix For: 3.3.1, 3.4.0
>
> Attachments: jstack-201004201053.txt, jstack-201004291409.txt, jstack-201004291527.txt, jstack-AsyncHammerTest-201004301209.txt, nohup-201004201053.txt, nohup-201004291409.txt, nohup-201004291527.txt, nohup-AsyncHammerTest-201004301209.txt, nohup-QuorumPeerMainTest-201004301209.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12858988#action_12858988 ]
Mahadev konar commented on ZOOKEEPER-690:
-----------------------------------------
alan,
junit doesnt persist the logs into the testfiles in case the test hangs. So, you will have to print out the logs on the console so that you know where it is hanging -
try
ant test -Dtestcase=AsyncHammerTest -Dtest.output=yes
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Patrick Hunt
> Priority: Critical
> Fix For: 3.3.1
>
> Attachments: TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12858991#action_12858991 ]
Patrick Hunt commented on ZOOKEEPER-690:
----------------------------------------
Right, also > test.out 2>&1
will help to capture the output.
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Patrick Hunt
> Priority: Critical
> Fix For: 3.3.1
>
> Attachments: TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar reassigned ZOOKEEPER-690:
---------------------------------------
Assignee: Patrick Hunt (was: Mahadev konar)
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Patrick Hunt
> Priority: Blocker
> Fix For: 3.3.0
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Hunt updated ZOOKEEPER-690:
-----------------------------------
Fix Version/s: (was: 3.3.0)
3.3.1
not a blocker for 3.3.0, tracking in 3.3.1. Have not seen these since, ie not reproduceable but let's keep an eye open.
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Patrick Hunt
> Priority: Critical
> Fix For: 3.3.1
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Benjamin Reed (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12862480#action_12862480 ]
Benjamin Reed commented on ZOOKEEPER-690:
-----------------------------------------
henry, i think this may show that we can't really have a setLearnerType() method. In the real distributed setting, each peer will have its own list, so we should really think of the peers list as immutable.
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Henry Robinson
> Priority: Blocker
> Fix For: 3.3.1, 3.4.0
>
> Attachments: jstack-201004201053.txt, jstack-201004291409.txt, jstack-201004291527.txt, nohup-201004201053.txt, nohup-201004291409.txt, nohup-201004291527.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Alan Cabrera (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alan Cabrera updated ZOOKEEPER-690:
-----------------------------------
Attachment: zoo.log
Ok, this run of the test has been running for about 10 minutes
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Patrick Hunt
> Priority: Critical
> Fix For: 3.3.1
>
> Attachments: TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Henry Robinson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12863915#action_12863915 ]
Henry Robinson commented on ZOOKEEPER-690:
------------------------------------------
Weird - it looks like the test is shutting down correctly:
[junit] 2010-04-30 11:41:52,896 - INFO [main:ClientBase@222] - connecting to 127.0.0.1 11233
[junit] 2010-04-30 11:41:52,896 - INFO [main:QuorumBase@277] - 127.0.0.1:11233 is no longer accepting client connections
[junit] 2010-04-30 11:41:52,896 - INFO [main:ClientBase@222] - connecting to 127.0.0.1 11234
[junit] 2010-04-30 11:41:52,897 - INFO [main:QuorumBase@277] - 127.0.0.1:11234 is no longer accepting client connections
[junit] 2010-04-30 11:41:52,897 - INFO [main:ClientBase@222] - connecting to 127.0.0.1 11235
[junit] 2010-04-30 11:41:52,897 - INFO [main:QuorumBase@277] - 127.0.0.1:11235 is no longer accepting client connections
[junit] 2010-04-30 11:41:52,897 - INFO [main:ClientBase@222] - connecting to 127.0.0.1 11236
[junit] 2010-04-30 11:41:52,898 - INFO [main:QuorumBase@277] - 127.0.0.1:11236 is no longer accepting client connections
[junit] 2010-04-30 11:41:52,898 - INFO [main:ClientBase@222] - connecting to 127.0.0.1 11237
[junit] 2010-04-30 11:41:52,898 - INFO [main:QuorumBase@277] - 127.0.0.1:11237 is no longer accepting client connections
[junit] 2010-04-30 11:41:52,901 - INFO [main:JUnit4ZKTestRunner$LoggedInvokeMethod@56] - FINISHED TEST METHOD testObserversHammer
[junit] 2010-04-30 11:41:52,901 - INFO [main:ZKTestCase$1@59] - SUCCEEDED testObserversHammer
[junit] 2010-04-30 11:41:52,901 - INFO [main:ZKTestCase$1@54] - FINISHED testObserversHammer
and then it goes into trying the C tests which fail for an unrelated reason - does it lock up at this point or does it actually fail out to the CLI? If it locks up, is the jstack output you attached from that run?
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Henry Robinson
> Priority: Blocker
> Fix For: 3.3.1, 3.4.0
>
> Attachments: jstack-201004201053.txt, jstack-201004291409.txt, jstack-201004291527.txt, jstack-AsyncHammerTest-201004301209.txt, nohup-201004201053.txt, nohup-201004291409.txt, nohup-201004291527.txt, nohup-AsyncHammerTest-201004301209.txt, nohup-QuorumPeerMainTest-201004301209.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Hunt updated ZOOKEEPER-690:
-----------------------------------
Priority: Critical (was: Blocker)
downgrading from blocker, haven't seen this again after many runs. we'll keep an eye but shouldn't block 3.3.0 (and may have been fixed)
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Patrick Hunt
> Priority: Critical
> Fix For: 3.3.0
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12858684#action_12858684 ]
Patrick Hunt commented on ZOOKEEPER-690:
----------------------------------------
Details such as os and java version would be really helpful. If you could upload the log file from the failed test that would be super useful. (build/test/logs/.)
This test in particular has found at least a couple issues.
Btw, you know that you can run just this test via:
ant -Dtestcase=AsyncHammerTest test-core-java
which might save you some time.
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Patrick Hunt
> Priority: Critical
> Fix For: 3.3.1
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Patrick Hunt (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12859008#action_12859008 ]
Patrick Hunt commented on ZOOKEEPER-690:
----------------------------------------
The Observer test is failing, and it looks like one of the observer servers might be the issue (11247 is the fifth server afaict), Henry any insight?
[junit] 2010-04-20 10:48:21,936 - INFO [QuorumPeer:/0.0.0.0:11247:QuorumPeer@619] - OBSERVING
[junit] 2010-04-20 10:49:42,593 - INFO [main:ClientBase@536] - Found match with array:[389, 389, 389, 389, 0]
[junit] 2010-04-20 10:49:42,594 - INFO [main:JUnit4ZKTestRunner$LoggedInvokeMethod@53] - TEST METHOD FAILED testObserversHammer
[junit] java.lang.AssertionError: node count not consistent expected:<389> but was:<0>
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Patrick Hunt
> Priority: Critical
> Fix For: 3.3.1
>
> Attachments: jstack-201004201053.txt, nohup-201004201053.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Alan Cabrera (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12858689#action_12858689 ]
Alan Cabrera commented on ZOOKEEPER-690:
----------------------------------------
{code}
junit.run:
[junit] Running org.apache.zookeeper.test.AsyncHammerTest
[junit] Tests run: 2, Failures: 0, Errors: 2, Time elapsed: 32.958 sec
[junit] Test org.apache.zookeeper.test.AsyncHammerTest FAILED
BUILD FAILED
/Users/acabrera/dev/hadoop/zookeeper/build.xml:818: Tests failed!
Total time: 46 seconds
[acabrera-md:zookeeper 620]$ java -version
java version "1.6.0_17"
Java(TM) SE Runtime Environment (build 1.6.0_17-b04-248-9M3125)
Java HotSpot(TM) 64-Bit Server VM (build 14.3-b01-101, mixed mode)
{code}
Mac OS X 10.5.8
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Patrick Hunt
> Priority: Critical
> Fix For: 3.3.1
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Flavio Paiva Junqueira (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Flavio Paiva Junqueira updated ZOOKEEPER-690:
---------------------------------------------
Status: Open (was: Patch Available)
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Henry Robinson
> Fix For: 3.4.0
>
> Attachments: jstack-201004201053.txt, jstack-201004291409.txt, jstack-201004291527.txt, jstack-AsyncHammerTest-201004301209.txt, nohup-201004201053.txt, nohup-201004291409.txt, nohup-201004291527.txt, nohup-AsyncHammerTest-201004301209.txt, nohup-QuorumPeerMainTest-201004301209.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Flavio Paiva Junqueira (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12864040#action_12864040 ]
Flavio Paiva Junqueira commented on ZOOKEEPER-690:
--------------------------------------------------
Hi Alan, It does not block for me, with or without the patch. My configuration is pretty much like your "fine" one.
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Henry Robinson
> Fix For: 3.4.0
>
> Attachments: jstack-201004201053.txt, jstack-201004291409.txt, jstack-201004291527.txt, jstack-AsyncHammerTest-201004301209.txt, nohup-201004201053.txt, nohup-201004291409.txt, nohup-201004291527.txt, nohup-AsyncHammerTest-201004301209.txt, nohup-QuorumPeerMainTest-201004301209.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Alan Cabrera (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12864024#action_12864024 ]
Alan Cabrera commented on ZOOKEEPER-690:
----------------------------------------
I understand that it works for him. Just wondering if he needs the patch to get it to work.
Fine:
MacBook Pro - Intel Core 2 Duo 2.53 GHz, 4 GB
Mac OS X 10.5.8 - all updates applied
Java(TM) SE Runtime Environment (build 1.6.0_17-b04-248-9M3125)
Java HotSpot(TM) 64-Bit Server VM (build 14.3-b01-101, mixed mode)
Using WiFi through Cisco VPN
Not so good:
MacPro - 2 Quad-Core Intel Xeon 2.26GHz, 24GB
Mac OS X 10.5.8 - all updates applied
Java(TM) SE Runtime Environment (build 1.6.0_17-b04-248-9M3125)
Java HotSpot(TM) 64-Bit Server VM (build 14.3-b01-101, mixed mode)
One Ethernet port used
If you're interested I might be able to clean this machine and put it in a DMZ for you all to tinker with.
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Henry Robinson
> Fix For: 3.4.0
>
> Attachments: jstack-201004201053.txt, jstack-201004291409.txt, jstack-201004291527.txt, jstack-AsyncHammerTest-201004301209.txt, nohup-201004201053.txt, nohup-201004291409.txt, nohup-201004291527.txt, nohup-AsyncHammerTest-201004301209.txt, nohup-QuorumPeerMainTest-201004301209.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Alan Cabrera (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alan Cabrera updated ZOOKEEPER-690:
-----------------------------------
Attachment: TEST-org.apache.zookeeper.test.AsyncHammerTest.txt
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Patrick Hunt
> Priority: Critical
> Fix For: 3.3.1
>
> Attachments: TEST-org.apache.zookeeper.test.AsyncHammerTest.txt
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Alan Cabrera (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alan Cabrera updated ZOOKEEPER-690:
-----------------------------------
Attachment: jstack-201004291527.txt
nohup-201004291527.txt
Still locks up. (BTW, I applied the latest patch to a clean checkout of zookeeper)
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Henry Robinson
> Priority: Blocker
> Fix For: 3.3.1, 3.4.0
>
> Attachments: jstack-201004201053.txt, jstack-201004291409.txt, jstack-201004291527.txt, nohup-201004201053.txt, nohup-201004291409.txt, nohup-201004291527.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Alan Cabrera (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12862398#action_12862398 ]
Alan Cabrera commented on ZOOKEEPER-690:
----------------------------------------
Test does not lock up but it does not pass.
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Henry Robinson
> Priority: Blocker
> Fix For: 3.3.1, 3.4.0
>
> Attachments: jstack-201004201053.txt, nohup-201004201053.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log, ZOOKEEPER-690.patch
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Alan Cabrera (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12863888#action_12863888 ]
Alan Cabrera commented on ZOOKEEPER-690:
----------------------------------------
The test still locks up but it seems that it's Ant that freezes. This process then becomes a zombie process. This zombie process holds on to ports that the tests use and so subsequent runs fail. Notably, QuorumPeerMainTest takes 15 minutes to fail.
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Henry Robinson
> Priority: Blocker
> Fix For: 3.3.1, 3.4.0
>
> Attachments: jstack-201004201053.txt, jstack-201004291409.txt, jstack-201004291527.txt, jstack-AsyncHammerTest-201004301209.txt, nohup-201004201053.txt, nohup-201004291409.txt, nohup-201004291527.txt, nohup-AsyncHammerTest-201004301209.txt, nohup-QuorumPeerMainTest-201004301209.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Henry Robinson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12863082#action_12863082 ]
Henry Robinson commented on ZOOKEEPER-690:
------------------------------------------
Hi Alan - Do you think you might have a chance to test whether this patch improves things?
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Henry Robinson
> Priority: Blocker
> Fix For: 3.3.1, 3.4.0
>
> Attachments: jstack-201004201053.txt, jstack-201004291409.txt, jstack-201004291527.txt, nohup-201004201053.txt, nohup-201004291409.txt, nohup-201004291527.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch, ZOOKEEPER-690.patch
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (ZOOKEEPER-690) AsyncTestHammer test fails on
hudson.
Posted by "Henry Robinson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Henry Robinson updated ZOOKEEPER-690:
-------------------------------------
Attachment: ZOOKEEPER-690.patch
I have found what I hope is the problem.
Because QuorumPeers duplicate their 'LearnerType' in two places there's the possibility that they may get out of sync. This is what was happening here - it was a test bug. Although the Observers knew that they were Observers, the other nodes did not. This affected the leader election protocol as other node did not know to reject an Observer.
I feel like we should refactor the QuorumPeer.QuorumServer code so as not to duplicate information, but for the time being I think this patch will work.
I have also taken the opportunity to standardise the naming of 'learnertype' throughout the code (in some places it was called 'peertype' adding to the confusion).
Tests pass on my machine, but I can't guarantee that the problem is fixed as I could never recreate the error.
Thanks to Flavio for catching the broken invariant!
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Henry Robinson
> Priority: Blocker
> Fix For: 3.3.1, 3.4.0
>
> Attachments: jstack-201004201053.txt, nohup-201004201053.txt, TEST-org.apache.zookeeper.test.AsyncHammerTest.txt, zoo.log, ZOOKEEPER-690.patch
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (ZOOKEEPER-690) AsyncTestHammer test
fails on hudson.
Posted by "Alan Cabrera (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ZOOKEEPER-690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12858689#action_12858689 ]
Alan Cabrera edited comment on ZOOKEEPER-690 at 4/19/10 5:36 PM:
-----------------------------------------------------------------
Odd. Didn't hang this time.
{code}
junit.run:
[junit] Running org.apache.zookeeper.test.AsyncHammerTest
[junit] Tests run: 2, Failures: 0, Errors: 2, Time elapsed: 32.958 sec
[junit] Test org.apache.zookeeper.test.AsyncHammerTest FAILED
BUILD FAILED
/Users/acabrera/dev/hadoop/zookeeper/build.xml:818: Tests failed!
Total time: 46 seconds
[acabrera-md:zookeeper 620]$ java -version
java version "1.6.0_17"
Java(TM) SE Runtime Environment (build 1.6.0_17-b04-248-9M3125)
Java HotSpot(TM) 64-Bit Server VM (build 14.3-b01-101, mixed mode)
{code}
Mac OS X 10.5.8
was (Author: maguro):
{code}
junit.run:
[junit] Running org.apache.zookeeper.test.AsyncHammerTest
[junit] Tests run: 2, Failures: 0, Errors: 2, Time elapsed: 32.958 sec
[junit] Test org.apache.zookeeper.test.AsyncHammerTest FAILED
BUILD FAILED
/Users/acabrera/dev/hadoop/zookeeper/build.xml:818: Tests failed!
Total time: 46 seconds
[acabrera-md:zookeeper 620]$ java -version
java version "1.6.0_17"
Java(TM) SE Runtime Environment (build 1.6.0_17-b04-248-9M3125)
Java HotSpot(TM) 64-Bit Server VM (build 14.3-b01-101, mixed mode)
{code}
Mac OS X 10.5.8
> AsyncTestHammer test fails on hudson.
> -------------------------------------
>
> Key: ZOOKEEPER-690
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-690
> Project: Zookeeper
> Issue Type: Bug
> Reporter: Mahadev konar
> Assignee: Patrick Hunt
> Priority: Critical
> Fix For: 3.3.1
>
> Attachments: TEST-org.apache.zookeeper.test.AsyncHammerTest.txt
>
>
> the hudson test failed on http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h1.grid.sp2.yahoo.net/2/testReport/. There are huge set of cancelledkeyexceptions in the logs. Still going through the logs to find out the reason for failure.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.