You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by khalidhuseynov <gi...@git.apache.org> on 2017/01/25 01:54:14 UTC
[GitHub] zeppelin pull request #1941: [ZEPPELIN-2009] Cron job isn't executed after c...
GitHub user khalidhuseynov opened a pull request:
https://github.com/apache/zeppelin/pull/1941
[ZEPPELIN-2009] Cron job isn't executed after couple of times
### What is this PR for?
This is to solve the problem with cron job scheduling. basically after https://github.com/apache/zeppelin/commit/6177c819b1edb76cfaa8f6249dc9041771ce6da9 all empty paragraphs are skipped when executing, but if the paragraph had status `READY` that status will be stayed same and then the note won't be considered as terminated in [here](https://github.com/apache/zeppelin/blob/master/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/Notebook.java#L866) resulting on keeping cron threads up running. So after all 10 threads in the threadpool are exhausted, new jobs are not scheduled. Here i change the status of paragraph to `FINISHED` when it's empty and skipping run.
### What type of PR is it?
Bug Fix
### Todos
* [x] - set status to `FINISHED`
### What is the Jira issue?
[ZEPPELIN-2009](https://issues.apache.org/jira/browse/ZEPPELIN-2009)
### How should this be tested?
try to schedule note using cron expression (e.g. `0/5 * * * * ?` every 5 secs) before and after this PR
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/khalidhuseynov/incubator-zeppelin fix/cron-scheduling
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/1941.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1941
----
commit 54e5ce9d7b37cf6fd5c067c0e1455c901a9a10a3
Author: Khalid Huseynov <kh...@gmail.com>
Date: 2017-01-25T01:45:55Z
set status FINISHED on empty paragraph
this is required since otherwise notebook wouldn't be isTerminated()
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1941: [ZEPPELIN-2009] Cron job isn't executed after couple o...
Posted by minahlee <gi...@git.apache.org>.
Github user minahlee commented on the issue:
https://github.com/apache/zeppelin/pull/1941
@khalidhuseynov Seems like this PR breaks `NotebookTest.testRunBlankParagraph`. Could you fix it?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1941: [ZEPPELIN-2009] Cron job isn't executed after couple o...
Posted by minahlee <gi...@git.apache.org>.
Github user minahlee commented on the issue:
https://github.com/apache/zeppelin/pull/1941
Another test is failing in multiple profiles, do you mind to fix them too?
```
Results :
Failed tests:
NotebookRestApiTest.testRunParagraphJob:108 expected:<FINISHED> but was:<READY>
Tests run: 82, Failures: 1, Errors: 0, Skipped: 1
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1941: [ZEPPELIN-2009] Cron job isn't executed after couple o...
Posted by minahlee <gi...@git.apache.org>.
Github user minahlee commented on the issue:
https://github.com/apache/zeppelin/pull/1941
Tested and CI is green now. Merge to branch-0.7 and master if there is no more discussion.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #1941: [ZEPPELIN-2009] Cron job isn't executed after c...
Posted by khalidhuseynov <gi...@git.apache.org>.
GitHub user khalidhuseynov reopened a pull request:
https://github.com/apache/zeppelin/pull/1941
[ZEPPELIN-2009] Cron job isn't executed after couple of times
### What is this PR for?
This is to solve the problem with cron job scheduling. basically after https://github.com/apache/zeppelin/commit/6177c819b1edb76cfaa8f6249dc9041771ce6da9 all empty paragraphs are skipped when executing, but if the paragraph had status `READY` that status will be stayed same and then the note won't be considered as terminated in [here](https://github.com/apache/zeppelin/blob/master/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/Notebook.java#L866) resulting on keeping cron threads up running. So after all 10 threads in the threadpool are exhausted, new jobs are not scheduled. Here i change the status of paragraph to `FINISHED` when it's empty and skipping run.
### What type of PR is it?
Bug Fix
### Todos
* [x] - set status to `FINISHED`
### What is the Jira issue?
[ZEPPELIN-2009](https://issues.apache.org/jira/browse/ZEPPELIN-2009)
### How should this be tested?
try to schedule note using cron expression (e.g. `0/5 * * * * ?` every 5 secs) before and after this PR
### Screenshots (if appropriate)
### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/khalidhuseynov/incubator-zeppelin fix/cron-scheduling
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/1941.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1941
----
commit 54e5ce9d7b37cf6fd5c067c0e1455c901a9a10a3
Author: Khalid Huseynov <kh...@gmail.com>
Date: 2017-01-25T01:45:55Z
set status FINISHED on empty paragraph
this is required since otherwise notebook wouldn't be isTerminated()
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1941: [ZEPPELIN-2009] Cron job isn't executed after couple o...
Posted by khalidhuseynov <gi...@git.apache.org>.
Github user khalidhuseynov commented on the issue:
https://github.com/apache/zeppelin/pull/1941
plz help review /cc @Leemoonsoo @minahlee @astroshim
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1941: [ZEPPELIN-2009] Cron job isn't executed after couple o...
Posted by minahlee <gi...@git.apache.org>.
Github user minahlee commented on the issue:
https://github.com/apache/zeppelin/pull/1941
@khalidhuseynov I am going to merge this one in few hours, but do you mind to create test cast for this in another PR?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1941: [ZEPPELIN-2009] Cron job isn't executed after couple o...
Posted by khalidhuseynov <gi...@git.apache.org>.
Github user khalidhuseynov commented on the issue:
https://github.com/apache/zeppelin/pull/1941
@minahlee thanks for review, just fixed it in 48fd94e03de3e4347ef41cdc5f464087108d5ce5
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1941: [ZEPPELIN-2009] Cron job isn't executed after couple o...
Posted by khalidhuseynov <gi...@git.apache.org>.
Github user khalidhuseynov commented on the issue:
https://github.com/apache/zeppelin/pull/1941
@minahlee yeah definitely, i was also thinking to have test case as a follow up
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #1941: [ZEPPELIN-2009] Cron job isn't executed after c...
Posted by khalidhuseynov <gi...@git.apache.org>.
Github user khalidhuseynov closed the pull request at:
https://github.com/apache/zeppelin/pull/1941
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #1941: [ZEPPELIN-2009] Cron job isn't executed after couple o...
Posted by khalidhuseynov <gi...@git.apache.org>.
Github user khalidhuseynov commented on the issue:
https://github.com/apache/zeppelin/pull/1941
@minahlee sorry for that, just addressed
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #1941: [ZEPPELIN-2009] Cron job isn't executed after c...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/zeppelin/pull/1941
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---