You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Robert Kanter (JIRA)" <ji...@apache.org> on 2012/11/01 20:45:12 UTC
[jira] [Created] (MAPREDUCE-4765) Restarting the JobTracker
programmatically can cause DelegationTokenRenewal to throw an exception
Robert Kanter created MAPREDUCE-4765:
----------------------------------------
Summary: Restarting the JobTracker programmatically can cause DelegationTokenRenewal to throw an exception
Key: MAPREDUCE-4765
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4765
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: jobtracker, mrv1
Affects Versions: 1.1.0
Reporter: Robert Kanter
Assignee: Robert Kanter
Priority: Minor
Fix For: 1.2.0
The DelegationTokenRenewal class has a global Timer; when you stop the JobTracker by calling {{stopTracker()}} on it (or {{stopJobTracker()}} in MiniMRCluster), the JobTracker will call {{close()}} on DelegationTokenRenewal, which cancels the Timer. If you then start up the JobTracker again by calling {{startTracker()}} on it (or {{startJobTracker()}} in MiniMRCluster), the Timer won't necessarily be re-created; and DelegationTokenRenewal will later throw an exception when it tries to use the Timer again (because you can't reuse a canceled Timer).
DelegationTokenRenewal doesn't seem to be used in trunk, so we only need this for branch-1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4765) Restarting the JobTracker
programmatically can cause DelegationTokenRenewal to throw an exception
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13491698#comment-13491698 ]
Alejandro Abdelnur commented on MAPREDUCE-4765:
-----------------------------------------------
+1
> Restarting the JobTracker programmatically can cause DelegationTokenRenewal to throw an exception
> -------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4765
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4765
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker, mrv1
> Affects Versions: 1.1.0
> Reporter: Robert Kanter
> Assignee: Robert Kanter
> Priority: Minor
> Fix For: 1.2.0
>
> Attachments: MAPREDUCE-4765_branch-1.patch, MAPREDUCE-4765_branch-1.patch
>
>
> The DelegationTokenRenewal class has a global Timer; when you stop the JobTracker by calling {{stopTracker()}} on it (or {{stopJobTracker()}} in MiniMRCluster), the JobTracker will call {{close()}} on DelegationTokenRenewal, which cancels the Timer. If you then start up the JobTracker again by calling {{startTracker()}} on it (or {{startJobTracker()}} in MiniMRCluster), the Timer won't necessarily be re-created; and DelegationTokenRenewal will later throw an exception when it tries to use the Timer again (because you can't reuse a canceled Timer).
> DelegationTokenRenewal doesn't seem to be used in trunk, so we only need this for branch-1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4765) Restarting the JobTracker
programmatically can cause DelegationTokenRenewal to throw an exception
Posted by "Robert Kanter (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Kanter updated MAPREDUCE-4765:
-------------------------------------
Attachment: MAPREDUCE-4765_branch-1.patch
The new patch addresses Alejandro's comments. I've checked for null before canceling and I've made the test call testDTRenewal() before closing the DTR timer (I had assumed it would always run before my new test, but that may not always be the case).
> Restarting the JobTracker programmatically can cause DelegationTokenRenewal to throw an exception
> -------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4765
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4765
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker, mrv1
> Affects Versions: 1.1.0
> Reporter: Robert Kanter
> Assignee: Robert Kanter
> Priority: Minor
> Fix For: 1.2.0
>
> Attachments: MAPREDUCE-4765_branch-1.patch, MAPREDUCE-4765_branch-1.patch
>
>
> The DelegationTokenRenewal class has a global Timer; when you stop the JobTracker by calling {{stopTracker()}} on it (or {{stopJobTracker()}} in MiniMRCluster), the JobTracker will call {{close()}} on DelegationTokenRenewal, which cancels the Timer. If you then start up the JobTracker again by calling {{startTracker()}} on it (or {{startJobTracker()}} in MiniMRCluster), the Timer won't necessarily be re-created; and DelegationTokenRenewal will later throw an exception when it tries to use the Timer again (because you can't reuse a canceled Timer).
> DelegationTokenRenewal doesn't seem to be used in trunk, so we only need this for branch-1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4765) Restarting the JobTracker
programmatically can cause DelegationTokenRenewal to throw an exception
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13488986#comment-13488986 ]
Hadoop QA commented on MAPREDUCE-4765:
--------------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12551752/MAPREDUCE-4765_branch-1.patch
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2979//console
This message is automatically generated.
> Restarting the JobTracker programmatically can cause DelegationTokenRenewal to throw an exception
> -------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4765
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4765
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker, mrv1
> Affects Versions: 1.1.0
> Reporter: Robert Kanter
> Assignee: Robert Kanter
> Priority: Minor
> Fix For: 1.2.0
>
> Attachments: MAPREDUCE-4765_branch-1.patch
>
>
> The DelegationTokenRenewal class has a global Timer; when you stop the JobTracker by calling {{stopTracker()}} on it (or {{stopJobTracker()}} in MiniMRCluster), the JobTracker will call {{close()}} on DelegationTokenRenewal, which cancels the Timer. If you then start up the JobTracker again by calling {{startTracker()}} on it (or {{startJobTracker()}} in MiniMRCluster), the Timer won't necessarily be re-created; and DelegationTokenRenewal will later throw an exception when it tries to use the Timer again (because you can't reuse a canceled Timer).
> DelegationTokenRenewal doesn't seem to be used in trunk, so we only need this for branch-1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4765) Restarting the JobTracker
programmatically can cause DelegationTokenRenewal to throw an exception
Posted by "Robert Kanter (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Kanter updated MAPREDUCE-4765:
-------------------------------------
Attachment: MAPREDUCE-4765_branch-1.patch
> Restarting the JobTracker programmatically can cause DelegationTokenRenewal to throw an exception
> -------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4765
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4765
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker, mrv1
> Affects Versions: 1.1.0
> Reporter: Robert Kanter
> Assignee: Robert Kanter
> Priority: Minor
> Fix For: 1.2.0
>
> Attachments: MAPREDUCE-4765_branch-1.patch
>
>
> The DelegationTokenRenewal class has a global Timer; when you stop the JobTracker by calling {{stopTracker()}} on it (or {{stopJobTracker()}} in MiniMRCluster), the JobTracker will call {{close()}} on DelegationTokenRenewal, which cancels the Timer. If you then start up the JobTracker again by calling {{startTracker()}} on it (or {{startJobTracker()}} in MiniMRCluster), the Timer won't necessarily be re-created; and DelegationTokenRenewal will later throw an exception when it tries to use the Timer again (because you can't reuse a canceled Timer).
> DelegationTokenRenewal doesn't seem to be used in trunk, so we only need this for branch-1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4765) Restarting the JobTracker
programmatically can cause DelegationTokenRenewal to throw an exception
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13491094#comment-13491094 ]
Hadoop QA commented on MAPREDUCE-4765:
--------------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12552191/MAPREDUCE-4765_branch-1.patch
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2989//console
This message is automatically generated.
> Restarting the JobTracker programmatically can cause DelegationTokenRenewal to throw an exception
> -------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4765
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4765
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker, mrv1
> Affects Versions: 1.1.0
> Reporter: Robert Kanter
> Assignee: Robert Kanter
> Priority: Minor
> Fix For: 1.2.0
>
> Attachments: MAPREDUCE-4765_branch-1.patch, MAPREDUCE-4765_branch-1.patch
>
>
> The DelegationTokenRenewal class has a global Timer; when you stop the JobTracker by calling {{stopTracker()}} on it (or {{stopJobTracker()}} in MiniMRCluster), the JobTracker will call {{close()}} on DelegationTokenRenewal, which cancels the Timer. If you then start up the JobTracker again by calling {{startTracker()}} on it (or {{startJobTracker()}} in MiniMRCluster), the Timer won't necessarily be re-created; and DelegationTokenRenewal will later throw an exception when it tries to use the Timer again (because you can't reuse a canceled Timer).
> DelegationTokenRenewal doesn't seem to be used in trunk, so we only need this for branch-1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4765) Restarting the JobTracker
programmatically can cause DelegationTokenRenewal to throw an exception
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alejandro Abdelnur updated MAPREDUCE-4765:
------------------------------------------
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Thanks Robert. Committed to branch-1
> Restarting the JobTracker programmatically can cause DelegationTokenRenewal to throw an exception
> -------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4765
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4765
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker, mrv1
> Affects Versions: 1.1.0
> Reporter: Robert Kanter
> Assignee: Robert Kanter
> Priority: Minor
> Fix For: 1.2.0
>
> Attachments: MAPREDUCE-4765_branch-1.patch, MAPREDUCE-4765_branch-1.patch
>
>
> The DelegationTokenRenewal class has a global Timer; when you stop the JobTracker by calling {{stopTracker()}} on it (or {{stopJobTracker()}} in MiniMRCluster), the JobTracker will call {{close()}} on DelegationTokenRenewal, which cancels the Timer. If you then start up the JobTracker again by calling {{startTracker()}} on it (or {{startJobTracker()}} in MiniMRCluster), the Timer won't necessarily be re-created; and DelegationTokenRenewal will later throw an exception when it tries to use the Timer again (because you can't reuse a canceled Timer).
> DelegationTokenRenewal doesn't seem to be used in trunk, so we only need this for branch-1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4765) Restarting the JobTracker
programmatically can cause DelegationTokenRenewal to throw an exception
Posted by "Robert Kanter (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Kanter updated MAPREDUCE-4765:
-------------------------------------
Status: Patch Available (was: Open)
> Restarting the JobTracker programmatically can cause DelegationTokenRenewal to throw an exception
> -------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4765
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4765
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker, mrv1
> Affects Versions: 1.1.0
> Reporter: Robert Kanter
> Assignee: Robert Kanter
> Priority: Minor
> Fix For: 1.2.0
>
> Attachments: MAPREDUCE-4765_branch-1.patch
>
>
> The DelegationTokenRenewal class has a global Timer; when you stop the JobTracker by calling {{stopTracker()}} on it (or {{stopJobTracker()}} in MiniMRCluster), the JobTracker will call {{close()}} on DelegationTokenRenewal, which cancels the Timer. If you then start up the JobTracker again by calling {{startTracker()}} on it (or {{startJobTracker()}} in MiniMRCluster), the Timer won't necessarily be re-created; and DelegationTokenRenewal will later throw an exception when it tries to use the Timer again (because you can't reuse a canceled Timer).
> DelegationTokenRenewal doesn't seem to be used in trunk, so we only need this for branch-1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4765) Restarting the JobTracker
programmatically can cause DelegationTokenRenewal to throw an exception
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13490930#comment-13490930 ]
Alejandro Abdelnur commented on MAPREDUCE-4765:
-----------------------------------------------
The close should check the timer is not NULL before attempting to cancel it. The testcase does not seem entirely correct, it assumes the DTR timer has been already initialized.
(In trunk and Hadoop 2, there is a renewer service, thus it has a defined lifecycle taking care of this)
> Restarting the JobTracker programmatically can cause DelegationTokenRenewal to throw an exception
> -------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4765
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4765
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker, mrv1
> Affects Versions: 1.1.0
> Reporter: Robert Kanter
> Assignee: Robert Kanter
> Priority: Minor
> Fix For: 1.2.0
>
> Attachments: MAPREDUCE-4765_branch-1.patch
>
>
> The DelegationTokenRenewal class has a global Timer; when you stop the JobTracker by calling {{stopTracker()}} on it (or {{stopJobTracker()}} in MiniMRCluster), the JobTracker will call {{close()}} on DelegationTokenRenewal, which cancels the Timer. If you then start up the JobTracker again by calling {{startTracker()}} on it (or {{startJobTracker()}} in MiniMRCluster), the Timer won't necessarily be re-created; and DelegationTokenRenewal will later throw an exception when it tries to use the Timer again (because you can't reuse a canceled Timer).
> DelegationTokenRenewal doesn't seem to be used in trunk, so we only need this for branch-1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira