You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Yiqun Lin (JIRA)" <ji...@apache.org> on 2016/06/28 13:26:57 UTC
[jira] [Comment Edited] (HDFS-10336) TestBalancer failing
intermittently because of not reseting UserGroupInformation completely
[ https://issues.apache.org/jira/browse/HDFS-10336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15352976#comment-15352976 ]
Yiqun Lin edited comment on HDFS-10336 at 6/28/16 1:25 PM:
-----------------------------------------------------------
Thanks [~rakeshr] for review.
{quote}
Increasing timeout is one approach, but am interested to know the reason behind 300000millis timeout. Did you see any specific case for exceeding the current value?
{quote}
I tested many times in my local, it seems good and runs quickly. I'm not so sure for the case that exceeding the 30s now. Post the patch for addressing your comments.
was (Author: linyiqun):
Thanks [~rakeshr] for review.
{quote}
Increasing timeout is one approach, but am interested to know the reason behind 300000millis timeout. Did you see any specific case for exceeding the current value?
{quote}
I tested many times in my local, it seems good and runs quickly. I'm not so sure for the case that exceeding the 30s now. Post the patch for your the comments.
> TestBalancer failing intermittently because of not reseting UserGroupInformation completely
> -------------------------------------------------------------------------------------------
>
> Key: HDFS-10336
> URL: https://issues.apache.org/jira/browse/HDFS-10336
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
> Affects Versions: 3.0.0-alpha1
> Reporter: Yiqun Lin
> Assignee: Yiqun Lin
> Attachments: HDFS-10336.001.patch, HDFS-10336.002.patch
>
>
> The unit test {{TestBalancer}} failed sometimes.
> I looked for the reason. I found two main reasons causing this.
> * 1st. The test {{TestBalancer#testBalancerWithKeytabs}} executed timeout.
> {code}
> org.apache.hadoop.hdfs.server.balancer.TestBalancer
> testBalancerWithKeytabs(org.apache.hadoop.hdfs.server.balancer.TestBalancer) Time elapsed: 300.41 sec <<< ERROR!
> java.lang.Exception: test timed out after 300000 milliseconds
> at java.lang.Thread.sleep(Native Method)
> at org.apache.hadoop.hdfs.server.balancer.Dispatcher.waitForMoveCompletion(Dispatcher.java:1122)
> at org.apache.hadoop.hdfs.server.balancer.Dispatcher.dispatchBlockMoves(Dispatcher.java:1096)
> at org.apache.hadoop.hdfs.server.balancer.Dispatcher.dispatchAndCheckContinue(Dispatcher.java:1060)
> at org.apache.hadoop.hdfs.server.balancer.Balancer.runOneIteration(Balancer.java:635)
> at org.apache.hadoop.hdfs.server.balancer.Balancer.run(Balancer.java:689)
> at org.apache.hadoop.hdfs.server.balancer.TestBalancer.testUnknownDatanode(TestBalancer.java:1098)
> at org.apache.hadoop.hdfs.server.balancer.TestBalancer.access$000(TestBalancer.java:125)
> {code}
> * 2nd. The test {{TestBalancer#testBalancerWithKeytabs}} reset the {{UGI}} not completely sometimes in the finally block. And this affected the other unit tests threw {{IOException}}, like this:
> {code}
> testBalancerWithNonZeroThreadsForMove(org.apache.hadoop.hdfs.server.balancer.TestBalancer) Time elapsed: 0 sec <<< ERROR!
> java.io.IOException: Running in secure mode, but config doesn't have a keytab
> at org.apache.hadoop.security.SecurityUtil.login(SecurityUtil.java:300)
> {code}
> And there were not only one test will be affected by this. We should add a line to do before doing reset {{UGI}} operation and can avoid the potenial exception happens.
> {code}
> UserGroupInformation.reset();
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org