You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Dmitro Lisnichenko <dl...@hortonworks.com> on 2015/07/08 11:44:14 UTC

Review Request 36303: yarn app fails with cant renew delegation token error in a cluster being downgraded

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36303/
-----------------------------------------------------------

Review request for Ambari, Jonathan Hurley and Vitalyi Brodetskyi.


Bugs: AMBARI-12276
    https://issues.apache.org/jira/browse/AMBARI-12276


Repository: ambari


Description
-------

It's a bit updated patch from https://reviews.apache.org/r/36169/

Failed Jobs:
JobId	Alias	Feature	Message	Outputs
job_1435740542268_0001	A	MAP_ONLY	Message: java.io.IOException: Failed to run job : Failed to renew token: Kind: MR_DELEGATION_TOKEN, Service: 172.31.37.125:10020, Ident: (owner=hrt_qa, renewer=yarn, realUser=oozie/ip-172-31-37-116.ec2.internal@EXAMPLE.COM, issueDate=1435740512336, maxDate=1436345312336, sequenceNumber=193, masterKeyId=2)
at org.apache.hadoop.mapred.YARNRunner.submitJob(YARNRunner.java:301)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:536)


Diffs
-----

  ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDescription.java 0cd5352 
  ambari-server/src/main/java/org/apache/ambari/server/checks/MapReduce2JobHistoryStatePreservingCheck.java PRE-CREATION 
  ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py 0b1bec5 
  ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py df8d7ce 
  ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/configuration-mapred/mapred-site.xml PRE-CREATION 
  ambari-server/src/test/java/org/apache/ambari/server/checks/MapReduce2JobHistoryStatePreservingCheckTest.java PRE-CREATION 
  ambari-server/src/test/python/stacks/2.0.6/YARN/test_historyserver.py 13d9ea9 

Diff: https://reviews.apache.org/r/36303/diff/


Testing
-------

Results :

Failed tests: 
  ClusterDeadlockTest.testDeadlockWhileRestartingComponents:285 [Deadlocked Thread:
  
  Tests run: 3112, Failures: 1, Errors: 0, Skipped: 28

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1:51:59.241s
[INFO] Finished at: Tue Jul 07 23:34:57 EEST 2015
[INFO] Final Memory: 50M/243M


Test failure seems to be unrelated and intermittent


Thanks,

Dmitro Lisnichenko


Re: Review Request 36303: yarn app fails with cant renew delegation token error in a cluster being downgraded

Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36303/#review90908
-----------------------------------------------------------



ambari-server/src/main/java/org/apache/ambari/server/checks/MapReduce2JobHistoryStatePreservingCheck.java (line 39)
<https://reviews.apache.org/r/36303/#comment144064>

    The UpgradeCheckGroup can be `CONFIGURATION_WARNING` instead of `DEFAULT`


Why are there so many new properties in ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/configuration-mapred/mapred-site.xml ... seems like we'd only need a few new ones (the ones added to the pre-req check).

- Jonathan Hurley


On July 8, 2015, 5:44 a.m., Dmitro Lisnichenko wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36303/
> -----------------------------------------------------------
> 
> (Updated July 8, 2015, 5:44 a.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-12276
>     https://issues.apache.org/jira/browse/AMBARI-12276
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> It's a bit updated patch from https://reviews.apache.org/r/36169/
> 
> Failed Jobs:
> JobId	Alias	Feature	Message	Outputs
> job_1435740542268_0001	A	MAP_ONLY	Message: java.io.IOException: Failed to run job : Failed to renew token: Kind: MR_DELEGATION_TOKEN, Service: 172.31.37.125:10020, Ident: (owner=hrt_qa, renewer=yarn, realUser=oozie/ip-172-31-37-116.ec2.internal@EXAMPLE.COM, issueDate=1435740512336, maxDate=1436345312336, sequenceNumber=193, masterKeyId=2)
> at org.apache.hadoop.mapred.YARNRunner.submitJob(YARNRunner.java:301)
> at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:536)
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDescription.java 0cd5352 
>   ambari-server/src/main/java/org/apache/ambari/server/checks/MapReduce2JobHistoryStatePreservingCheck.java PRE-CREATION 
>   ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py 0b1bec5 
>   ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py df8d7ce 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/configuration-mapred/mapred-site.xml PRE-CREATION 
>   ambari-server/src/test/java/org/apache/ambari/server/checks/MapReduce2JobHistoryStatePreservingCheckTest.java PRE-CREATION 
>   ambari-server/src/test/python/stacks/2.0.6/YARN/test_historyserver.py 13d9ea9 
> 
> Diff: https://reviews.apache.org/r/36303/diff/
> 
> 
> Testing
> -------
> 
> Results :
> 
> Failed tests: 
>   ClusterDeadlockTest.testDeadlockWhileRestartingComponents:285 [Deadlocked Thread:
>   
>   Tests run: 3112, Failures: 1, Errors: 0, Skipped: 28
> 
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 1:51:59.241s
> [INFO] Finished at: Tue Jul 07 23:34:57 EEST 2015
> [INFO] Final Memory: 50M/243M
> 
> 
> Test failure seems to be unrelated and intermittent
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>


Re: Review Request 36303: yarn app fails with cant renew delegation token error in a cluster being downgraded

Posted by Vitalyi Brodetskyi <vb...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36303/#review91076
-----------------------------------------------------------

Ship it!


Ship It!

- Vitalyi Brodetskyi


On Липень 8, 2015, 3:47 після полудня, Dmitro Lisnichenko wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36303/
> -----------------------------------------------------------
> 
> (Updated Липень 8, 2015, 3:47 після полудня)
> 
> 
> Review request for Ambari, Jonathan Hurley and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-12276
>     https://issues.apache.org/jira/browse/AMBARI-12276
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> It's a bit updated patch from https://reviews.apache.org/r/36169/
> 
> Failed Jobs:
> JobId	Alias	Feature	Message	Outputs
> job_1435740542268_0001	A	MAP_ONLY	Message: java.io.IOException: Failed to run job : Failed to renew token: Kind: MR_DELEGATION_TOKEN, Service: 172.31.37.125:10020, Ident: (owner=hrt_qa, renewer=yarn, realUser=oozie/ip-172-31-37-116.ec2.internal@EXAMPLE.COM, issueDate=1435740512336, maxDate=1436345312336, sequenceNumber=193, masterKeyId=2)
> at org.apache.hadoop.mapred.YARNRunner.submitJob(YARNRunner.java:301)
> at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:536)
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDescription.java 0cd5352 
>   ambari-server/src/main/java/org/apache/ambari/server/checks/MapReduce2JobHistoryStatePreservingCheck.java PRE-CREATION 
>   ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py 77092c1 
>   ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py df8d7ce 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/configuration-mapred/mapred-site.xml PRE-CREATION 
>   ambari-server/src/test/java/org/apache/ambari/server/checks/MapReduce2JobHistoryStatePreservingCheckTest.java PRE-CREATION 
>   ambari-server/src/test/python/stacks/2.0.6/YARN/test_historyserver.py 13d9ea9 
> 
> Diff: https://reviews.apache.org/r/36303/diff/
> 
> 
> Testing
> -------
> 
> Results :
> 
> Failed tests: 
>   ClusterDeadlockTest.testDeadlockWhileRestartingComponents:285 [Deadlocked Thread:
>   
>   Tests run: 3112, Failures: 1, Errors: 0, Skipped: 28
> 
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 1:51:59.241s
> [INFO] Finished at: Tue Jul 07 23:34:57 EEST 2015
> [INFO] Final Memory: 50M/243M
> 
> 
> Test failure seems to be unrelated and intermittent
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>


Re: Review Request 36303: yarn app fails with cant renew delegation token error in a cluster being downgraded

Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36303/
-----------------------------------------------------------

(Updated July 9, 2015, 12:45 p.m.)


Review request for Ambari, Jonathan Hurley and Vitalyi Brodetskyi.


Changes
-------

Fixed comments at jira


Bugs: AMBARI-12276
    https://issues.apache.org/jira/browse/AMBARI-12276


Repository: ambari


Description
-------

It's a bit updated patch from https://reviews.apache.org/r/36169/

Failed Jobs:
JobId	Alias	Feature	Message	Outputs
job_1435740542268_0001	A	MAP_ONLY	Message: java.io.IOException: Failed to run job : Failed to renew token: Kind: MR_DELEGATION_TOKEN, Service: 172.31.37.125:10020, Ident: (owner=hrt_qa, renewer=yarn, realUser=oozie/ip-172-31-37-116.ec2.internal@EXAMPLE.COM, issueDate=1435740512336, maxDate=1436345312336, sequenceNumber=193, masterKeyId=2)
at org.apache.hadoop.mapred.YARNRunner.submitJob(YARNRunner.java:301)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:536)


Diffs (updated)
-----

  ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDescription.java 0cd5352 
  ambari-server/src/main/java/org/apache/ambari/server/checks/MapReduce2JobHistoryStatePreservingCheck.java PRE-CREATION 
  ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py 77092c1 
  ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py df8d7ce 
  ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/configuration-mapred/mapred-site.xml PRE-CREATION 
  ambari-server/src/test/java/org/apache/ambari/server/checks/MapReduce2JobHistoryStatePreservingCheckTest.java PRE-CREATION 
  ambari-server/src/test/python/stacks/2.0.6/YARN/test_historyserver.py 13d9ea9 

Diff: https://reviews.apache.org/r/36303/diff/


Testing
-------

Results :

Failed tests: 
  ClusterDeadlockTest.testDeadlockWhileRestartingComponents:285 [Deadlocked Thread:
  
  Tests run: 3112, Failures: 1, Errors: 0, Skipped: 28

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1:51:59.241s
[INFO] Finished at: Tue Jul 07 23:34:57 EEST 2015
[INFO] Final Memory: 50M/243M


Test failure seems to be unrelated and intermittent


Thanks,

Dmitro Lisnichenko


Re: Review Request 36303: yarn app fails with cant renew delegation token error in a cluster being downgraded

Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36303/
-----------------------------------------------------------

(Updated July 8, 2015, 3:47 p.m.)


Review request for Ambari, Jonathan Hurley and Vitalyi Brodetskyi.


Changes
-------

fixed


Bugs: AMBARI-12276
    https://issues.apache.org/jira/browse/AMBARI-12276


Repository: ambari


Description
-------

It's a bit updated patch from https://reviews.apache.org/r/36169/

Failed Jobs:
JobId	Alias	Feature	Message	Outputs
job_1435740542268_0001	A	MAP_ONLY	Message: java.io.IOException: Failed to run job : Failed to renew token: Kind: MR_DELEGATION_TOKEN, Service: 172.31.37.125:10020, Ident: (owner=hrt_qa, renewer=yarn, realUser=oozie/ip-172-31-37-116.ec2.internal@EXAMPLE.COM, issueDate=1435740512336, maxDate=1436345312336, sequenceNumber=193, masterKeyId=2)
at org.apache.hadoop.mapred.YARNRunner.submitJob(YARNRunner.java:301)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:536)


Diffs (updated)
-----

  ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDescription.java 0cd5352 
  ambari-server/src/main/java/org/apache/ambari/server/checks/MapReduce2JobHistoryStatePreservingCheck.java PRE-CREATION 
  ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py 77092c1 
  ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py df8d7ce 
  ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/configuration-mapred/mapred-site.xml PRE-CREATION 
  ambari-server/src/test/java/org/apache/ambari/server/checks/MapReduce2JobHistoryStatePreservingCheckTest.java PRE-CREATION 
  ambari-server/src/test/python/stacks/2.0.6/YARN/test_historyserver.py 13d9ea9 

Diff: https://reviews.apache.org/r/36303/diff/


Testing
-------

Results :

Failed tests: 
  ClusterDeadlockTest.testDeadlockWhileRestartingComponents:285 [Deadlocked Thread:
  
  Tests run: 3112, Failures: 1, Errors: 0, Skipped: 28

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1:51:59.241s
[INFO] Finished at: Tue Jul 07 23:34:57 EEST 2015
[INFO] Final Memory: 50M/243M


Test failure seems to be unrelated and intermittent


Thanks,

Dmitro Lisnichenko