You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by AlexanderShoshin <gi...@git.apache.org> on 2017/01/27 12:01:54 UTC
[GitHub] zeppelin pull request #1954: [ZEPPELIN-525] Test failing in zeppelin-interpr...
GitHub user AlexanderShoshin opened a pull request:
https://github.com/apache/zeppelin/pull/1954
[ZEPPELIN-525] Test failing in zeppelin-interpreter
`RemoteSchedulerTest.test` was fall when CPU switched `RemoteScheduler` execution (which executes a job in a separate thread) back to test main thread execution after changing job status to `Status.FINISHED` but before updating running jobs list:
```
job.setStatus(lastStatus);
// if thread execution is switched here to the main thread test will fall
synchronized (queue) {
running.remove(job);
queue.notify();
}
```
Test checked job status, saw that job was terminated and expected to see an empty jobs runing list.
```
while (!job.isTerminated() && cycles < MAX_WAIT_CYCLES) {
Thread.sleep(TICK_WAIT);
cycles++;
}
// this assert will fail because a job was not removed from running list yet
assertEquals(0, scheduler.getJobsRunning().size());
```
But `scheduler.running` list still contained the last job and the assertion failed.
### What is this PR for?
The goal is to synchronize updating of a job status and a scheduler running jobs list when a job is terminated.
### What type of PR is it?
Bug Fix
### What is the Jira issue?
[ZEPPELIN-525](https://issues.apache.org/jira/browse/ZEPPELIN-525)
### How should this be tested?
You may place a `Thread.sleep(1000);` code to the `RemoteScheduler.java` class after the `job.setStatus(lastStatus);` (line number 354). This will increase probability of threads switching after this line. Test should not fail.
### Questions:
* Does the licenses files need update? **no**
* Is there breaking changes for older versions? **no**
* Does this needs documentation? **no**
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/AlexanderShoshin/zeppelin ZEPPELIN-525
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/1954.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1954
----
commit 8a0f4afbdf2286748cff4cc415401670de904700
Author: Alexander Shoshin <al...@epam.com>
Date: 2017-01-27T08:30:11Z
synchronized a job status and a list of running jobs
synchronized changing of a job status and updating a list of running jobs after job termination
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #1954: [ZEPPELIN-525] Test failing in zeppelin-interpr...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/zeppelin/pull/1954
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1954: [ZEPPELIN-525] Test failing in zeppelin-interpreter
Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/1954
look reasonable.
CI is green, merging if no more comment
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1954: [ZEPPELIN-525] Test failing in zeppelin-interpreter
Posted by jongyoul <gi...@git.apache.org>.
Github user jongyoul commented on the issue:
https://github.com/apache/zeppelin/pull/1954
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #1954: [ZEPPELIN-525] Test failing in zeppelin-interpr...
Posted by AlexanderShoshin <gi...@git.apache.org>.
GitHub user AlexanderShoshin reopened a pull request:
https://github.com/apache/zeppelin/pull/1954
[ZEPPELIN-525] Test failing in zeppelin-interpreter
`RemoteSchedulerTest.test` was fall when CPU switched `RemoteScheduler` execution (which executes a job in a separate thread) back to test main thread execution after changing job status to `Status.FINISHED` but before updating running jobs list:
```
job.setStatus(lastStatus);
// if thread execution is switched here to the main thread test will fall
synchronized (queue) {
running.remove(job);
queue.notify();
}
```
Test checked job status, saw that job was terminated and expected to see an empty jobs runing list.
```
while (!job.isTerminated() && cycles < MAX_WAIT_CYCLES) {
Thread.sleep(TICK_WAIT);
cycles++;
}
// this assert will fail because a job was not removed from running list yet
assertEquals(0, scheduler.getJobsRunning().size());
```
But `scheduler.running` list still contained the last job and the assertion failed.
### What is this PR for?
The goal is to synchronize updating of a job status and a scheduler running jobs list when a job is terminated.
### What type of PR is it?
Bug Fix
### What is the Jira issue?
[ZEPPELIN-525](https://issues.apache.org/jira/browse/ZEPPELIN-525)
### How should this be tested?
You may place a `Thread.sleep(1000);` code to the `RemoteScheduler.java` class after the `job.setStatus(lastStatus);` (line number 354). This will increase probability of threads switching after this line. Test should not fail.
### Questions:
* Does the licenses files need update? **no**
* Is there breaking changes for older versions? **no**
* Does this needs documentation? **no**
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/AlexanderShoshin/zeppelin ZEPPELIN-525
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/1954.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1954
----
commit 8a0f4afbdf2286748cff4cc415401670de904700
Author: Alexander Shoshin <al...@epam.com>
Date: 2017-01-27T08:30:11Z
synchronized a job status and a list of running jobs
synchronized changing of a job status and updating a list of running jobs after job termination
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #1954: [ZEPPELIN-525] Test failing in zeppelin-interpr...
Posted by AlexanderShoshin <gi...@git.apache.org>.
Github user AlexanderShoshin closed the pull request at:
https://github.com/apache/zeppelin/pull/1954
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---