You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Devaraj K (JIRA)" <ji...@apache.org> on 2012/10/17 13:50:03 UTC
[jira] [Created] (YARN-164) Race condition in Fair Scheduler
Devaraj K created YARN-164:
------------------------------
Summary: Race condition in Fair Scheduler
Key: YARN-164
URL: https://issues.apache.org/jira/browse/YARN-164
Project: Hadoop YARN
Issue Type: Bug
Components: scheduler
Affects Versions: 2.0.2-alpha
Reporter: Devaraj K
Assignee: Devaraj K
Priority: Critical
{code:xml}
Thread updateThread = new Thread(new UpdateThread());
updateThread.start();
initialized = true;
{code}
In the above code, making the initialized as true after starting the UpdateThread.
{code:xml}
private class UpdateThread implements Runnable {
public void run() {
while (initialized) {
try {
Thread.sleep(UPDATE_INTERVAL);
update();
preemptTasksIfNecessary();
} catch (Exception e) {
LOG.error("Exception in fair scheduler UpdateThread", e);
}
}
}
}
{code}
In this run method of UpdateThread, it is checking for the initialized and exiting if it is not true. Here most of the times initialized is getting true after exiting the UpdateThread and the thread functionality is missing, due to that all the submitted applications are hanging without making any progress.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-164) Race condition in Fair Scheduler
Posted by "Karthik Kambatla (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/YARN-164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13495791#comment-13495791 ]
Karthik Kambatla commented on YARN-164:
---------------------------------------
Nice catch. I have a minor comment: {{updateThread.start()}} should probably be in a try-catch, and catch should set {{initialized}} to false. Otherwise, +1
> Race condition in Fair Scheduler
> --------------------------------
>
> Key: YARN-164
> URL: https://issues.apache.org/jira/browse/YARN-164
> Project: Hadoop YARN
> Issue Type: Bug
> Components: scheduler
> Affects Versions: 2.0.2-alpha
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Critical
> Attachments: YARN-164.patch
>
>
> {code:xml}
> Thread updateThread = new Thread(new UpdateThread());
> updateThread.start();
> initialized = true;
> {code}
> In the above code, making the initialized as true after starting the UpdateThread.
> {code:xml}
> private class UpdateThread implements Runnable {
> public void run() {
> while (initialized) {
> try {
> Thread.sleep(UPDATE_INTERVAL);
> update();
> preemptTasksIfNecessary();
> } catch (Exception e) {
> LOG.error("Exception in fair scheduler UpdateThread", e);
> }
> }
> }
> }
> {code}
> In this run method of UpdateThread, it is checking for the initialized and exiting if it is not true. Here most of the times initialized is getting true after exiting the UpdateThread and the thread functionality is missing, due to that all the submitted applications are hanging without making any progress.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-164) Race condition in Fair Scheduler
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/YARN-164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492373#comment-13492373 ]
Tom White commented on YARN-164:
--------------------------------
+1. I've just triggered Jenkins for this patch. The lack of a unit test is OK since it's very hard to test for the absence of a race condition.
> Race condition in Fair Scheduler
> --------------------------------
>
> Key: YARN-164
> URL: https://issues.apache.org/jira/browse/YARN-164
> Project: Hadoop YARN
> Issue Type: Bug
> Components: scheduler
> Affects Versions: 2.0.2-alpha
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Critical
> Attachments: YARN-164.patch
>
>
> {code:xml}
> Thread updateThread = new Thread(new UpdateThread());
> updateThread.start();
> initialized = true;
> {code}
> In the above code, making the initialized as true after starting the UpdateThread.
> {code:xml}
> private class UpdateThread implements Runnable {
> public void run() {
> while (initialized) {
> try {
> Thread.sleep(UPDATE_INTERVAL);
> update();
> preemptTasksIfNecessary();
> } catch (Exception e) {
> LOG.error("Exception in fair scheduler UpdateThread", e);
> }
> }
> }
> }
> {code}
> In this run method of UpdateThread, it is checking for the initialized and exiting if it is not true. Here most of the times initialized is getting true after exiting the UpdateThread and the thread functionality is missing, due to that all the submitted applications are hanging without making any progress.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-164) Race condition in Fair Scheduler
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/YARN-164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13492385#comment-13492385 ]
Hadoop QA commented on YARN-164:
--------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12549489/YARN-164.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-YARN-Build/138//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/138//console
This message is automatically generated.
> Race condition in Fair Scheduler
> --------------------------------
>
> Key: YARN-164
> URL: https://issues.apache.org/jira/browse/YARN-164
> Project: Hadoop YARN
> Issue Type: Bug
> Components: scheduler
> Affects Versions: 2.0.2-alpha
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Critical
> Attachments: YARN-164.patch
>
>
> {code:xml}
> Thread updateThread = new Thread(new UpdateThread());
> updateThread.start();
> initialized = true;
> {code}
> In the above code, making the initialized as true after starting the UpdateThread.
> {code:xml}
> private class UpdateThread implements Runnable {
> public void run() {
> while (initialized) {
> try {
> Thread.sleep(UPDATE_INTERVAL);
> update();
> preemptTasksIfNecessary();
> } catch (Exception e) {
> LOG.error("Exception in fair scheduler UpdateThread", e);
> }
> }
> }
> }
> {code}
> In this run method of UpdateThread, it is checking for the initialized and exiting if it is not true. Here most of the times initialized is getting true after exiting the UpdateThread and the thread functionality is missing, due to that all the submitted applications are hanging without making any progress.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-164) Race condition in Fair Scheduler
Posted by "Sandy Ryza (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/YARN-164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13505757#comment-13505757 ]
Sandy Ryza commented on YARN-164:
---------------------------------
YARN-184 went in, so marking this as duplicate.
> Race condition in Fair Scheduler
> --------------------------------
>
> Key: YARN-164
> URL: https://issues.apache.org/jira/browse/YARN-164
> Project: Hadoop YARN
> Issue Type: Bug
> Components: scheduler
> Affects Versions: 2.0.2-alpha
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Critical
> Attachments: YARN-164.patch
>
>
> {code:xml}
> Thread updateThread = new Thread(new UpdateThread());
> updateThread.start();
> initialized = true;
> {code}
> In the above code, making the initialized as true after starting the UpdateThread.
> {code:xml}
> private class UpdateThread implements Runnable {
> public void run() {
> while (initialized) {
> try {
> Thread.sleep(UPDATE_INTERVAL);
> update();
> preemptTasksIfNecessary();
> } catch (Exception e) {
> LOG.error("Exception in fair scheduler UpdateThread", e);
> }
> }
> }
> }
> {code}
> In this run method of UpdateThread, it is checking for the initialized and exiting if it is not true. Here most of the times initialized is getting true after exiting the UpdateThread and the thread functionality is missing, due to that all the submitted applications are hanging without making any progress.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-164) Race condition in Fair Scheduler
Posted by "Sandy Ryza (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/YARN-164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13496643#comment-13496643 ]
Sandy Ryza commented on YARN-164:
---------------------------------
I noticed this independently and fixed it in my patch for YARN-184. If this one is used, starting the UpdateThread should happen after creating the QueueManager, and there's no need to check the initialized variable in the Update thread because it will always be true.
> Race condition in Fair Scheduler
> --------------------------------
>
> Key: YARN-164
> URL: https://issues.apache.org/jira/browse/YARN-164
> Project: Hadoop YARN
> Issue Type: Bug
> Components: scheduler
> Affects Versions: 2.0.2-alpha
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Critical
> Attachments: YARN-164.patch
>
>
> {code:xml}
> Thread updateThread = new Thread(new UpdateThread());
> updateThread.start();
> initialized = true;
> {code}
> In the above code, making the initialized as true after starting the UpdateThread.
> {code:xml}
> private class UpdateThread implements Runnable {
> public void run() {
> while (initialized) {
> try {
> Thread.sleep(UPDATE_INTERVAL);
> update();
> preemptTasksIfNecessary();
> } catch (Exception e) {
> LOG.error("Exception in fair scheduler UpdateThread", e);
> }
> }
> }
> }
> {code}
> In this run method of UpdateThread, it is checking for the initialized and exiting if it is not true. Here most of the times initialized is getting true after exiting the UpdateThread and the thread functionality is missing, due to that all the submitted applications are hanging without making any progress.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-164) Race condition in Fair Scheduler
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/YARN-164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13505640#comment-13505640 ]
Hadoop QA commented on YARN-164:
--------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12549489/YARN-164.patch
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/179//console
This message is automatically generated.
> Race condition in Fair Scheduler
> --------------------------------
>
> Key: YARN-164
> URL: https://issues.apache.org/jira/browse/YARN-164
> Project: Hadoop YARN
> Issue Type: Bug
> Components: scheduler
> Affects Versions: 2.0.2-alpha
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Critical
> Attachments: YARN-164.patch
>
>
> {code:xml}
> Thread updateThread = new Thread(new UpdateThread());
> updateThread.start();
> initialized = true;
> {code}
> In the above code, making the initialized as true after starting the UpdateThread.
> {code:xml}
> private class UpdateThread implements Runnable {
> public void run() {
> while (initialized) {
> try {
> Thread.sleep(UPDATE_INTERVAL);
> update();
> preemptTasksIfNecessary();
> } catch (Exception e) {
> LOG.error("Exception in fair scheduler UpdateThread", e);
> }
> }
> }
> }
> {code}
> In this run method of UpdateThread, it is checking for the initialized and exiting if it is not true. Here most of the times initialized is getting true after exiting the UpdateThread and the thread functionality is missing, due to that all the submitted applications are hanging without making any progress.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (YARN-164) Race condition in Fair Scheduler
Posted by "Devaraj K (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/YARN-164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Devaraj K updated YARN-164:
---------------------------
Attachment: YARN-164.patch
Attaching the straightforward patch to fix this issue.
> Race condition in Fair Scheduler
> --------------------------------
>
> Key: YARN-164
> URL: https://issues.apache.org/jira/browse/YARN-164
> Project: Hadoop YARN
> Issue Type: Bug
> Components: scheduler
> Affects Versions: 2.0.2-alpha
> Reporter: Devaraj K
> Assignee: Devaraj K
> Priority: Critical
> Attachments: YARN-164.patch
>
>
> {code:xml}
> Thread updateThread = new Thread(new UpdateThread());
> updateThread.start();
> initialized = true;
> {code}
> In the above code, making the initialized as true after starting the UpdateThread.
> {code:xml}
> private class UpdateThread implements Runnable {
> public void run() {
> while (initialized) {
> try {
> Thread.sleep(UPDATE_INTERVAL);
> update();
> preemptTasksIfNecessary();
> } catch (Exception e) {
> LOG.error("Exception in fair scheduler UpdateThread", e);
> }
> }
> }
> }
> {code}
> In this run method of UpdateThread, it is checking for the initialized and exiting if it is not true. Here most of the times initialized is getting true after exiting the UpdateThread and the thread functionality is missing, due to that all the submitted applications are hanging without making any progress.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira