You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by NohSeho <gi...@git.apache.org> on 2017/02/23 16:23:22 UTC
[GitHub] zeppelin pull request #2060: [ZEPPELIN-2094] Decrease npm install retry time
GitHub user NohSeho opened a pull request:
https://github.com/apache/zeppelin/pull/2060
[ZEPPELIN-2094] Decrease npm install retry time
### What is this PR for?
It\u2019s too delayed for npm install when computer do not connected any networks.
Beacause when npm install, it has too long retry timeout.
This PR is to decrease retry timeout when npm install.
### What type of PR is it?
Improvement
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-2094
### How should this be tested?
Line 197 In zeppelin-zengine org.apache.zeppelin.helium.HeliumBundleFactory.java
First set with
`String npmCommand = "install \u2014loglevel=error\u201d;`
and You don\u2019t connect any ethernet or wireless internet.
build & run
and set with
`String npmCommand = "install \u2014fetch-retries=2 \u2014fetch-retry-factor=1 \u2014fetch-retry-mintimeout=5000 \u2014loglevel=error\u201d;`
also don\u2019t connect any networks, build & run.
WHY
retries = 2
factor = 1
mintimeout = 5(sec)?
npm use [retry](https://github.com/tim-kos/node-retry) module to retry.
It refers [this article](http://dthain.blogspot.kr/2009/02/exponential-backoff-in-distributed.html) for retry algorithms.
It is a math which structured _Math.min(Math.round(random * minTimeout * Math.pow(factor, attempt)), maxTimeout)_.
So before, zeppelin use npm's default setting
random = False = 1
retry = 2
minTimeout = 10(sec)
maxTimeout = 60(sec)
factor = 10
First retry
_Math.min(1 * 10(sec) * Math.pow(10, 1), 60(sec))_
then, _Math.min(10(sec) * 10^1, 60(sec))_ -> _Math.min(100(sec), 60(sec))_ = _100sec_ wating for first retry
and Second
_Math.min(1 * 10(sec) * Math.pow(10, 2), 60(sec))_
then, _Math.min(10(sec) * 10^2, 60(sec))_ -> _Math.min(10(sec) * 100, 60(sec))_ -> _Math.min(1000(sec), 60(sec))_ = _60sec_ wating for second retry
approximately 160sec, 2min 40sec wating for two times retries.
But it's too long to run server when no networks.
So decrease like that you see above.
First retry
_Math.min(1 * 5(sec) * Math.pow(1, 1), 60(sec))_
then, _Math.min(5(sec) * 1^1, 60(sec))_ -> _Math.min(5(sec), 60(sec)_ = _5sec_ wating for first retry
and Second
_Math.min(1 * 5(sec) * Math.pow(1, 2), 60(sec))_
then, _Math.min(5(sec) * 1^2, 60(sec))_ -> _Math.min(5(sec), 60(sec)_ = _5sec_ wating for second retry
approximately 10sec waiting for two times retries.
You can check like this below Screenshots.
### Screenshots
<img width="1077" alt="2017-02-24 12 32 06" src="https://cloud.githubusercontent.com/assets/1144643/23267951/9deaec6e-fa2f-11e6-9171-5792f24de76d.png">
<img width="1081" alt="2017-02-24 12 37 10" src="https://cloud.githubusercontent.com/assets/1144643/23267954/a12c0c0a-fa2f-11e6-99cd-335deef607ac.png">
### Questions:
* Does the licenses files need update? N/A
* Is there breaking changes for older versions? N/A
* Does this needs documentation? N/A
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/NohSeho/zeppelin ZEPPELIN-2094
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/2060.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2060
----
commit b3fe1886df3d8d64c4aced5672d0f98bb5ad4241
Author: NohSeho <ia...@sehonoh.kr>
Date: 2017-02-23T15:59:57Z
[ZEPPELIN-2094] Decrease npm install retry time
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #2060: [ZEPPELIN-2094] Decrease npm install retry time
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/zeppelin/pull/2060
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2060: [ZEPPELIN-2094] Decrease npm install retry time
Posted by Leemoonsoo <gi...@git.apache.org>.
Github user Leemoonsoo commented on the issue:
https://github.com/apache/zeppelin/pull/2060
Thanks @NohSeho for the improvement.
Looks good to me.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #2060: [ZEPPELIN-2094] Decrease npm install retry time
Posted by NohSeho <gi...@git.apache.org>.
GitHub user NohSeho reopened a pull request:
https://github.com/apache/zeppelin/pull/2060
[ZEPPELIN-2094] Decrease npm install retry time
### What is this PR for?
It\u2019s too delayed for npm install when computer do not connected any networks.
Beacause when npm install, it has too long retry timeout.
This PR is to decrease retry timeout when npm install.
### What type of PR is it?
Improvement
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-2094
### How should this be tested?
you must enable any one helium before test
Line 197 In zeppelin-zengine org.apache.zeppelin.helium.HeliumBundleFactory.java
First set with
`String npmCommand = "install \u2014loglevel=error\u201d;`
and You don\u2019t connect any ethernet or wireless internet.
build & run
and set with
`String npmCommand = "install \u2014fetch-retries=2 \u2014fetch-retry-factor=1 \u2014fetch-retry-mintimeout=5000 \u2014loglevel=error\u201d;`
also don\u2019t connect any networks, build & run.
WHY
retries = 2
factor = 1
mintimeout = 5(sec)?
npm use [retry](https://github.com/tim-kos/node-retry) module to retry.
It refers [this article](http://dthain.blogspot.kr/2009/02/exponential-backoff-in-distributed.html) for retry algorithms.
It is a math which structured _Math.min(Math.round(random * minTimeout * Math.pow(factor, attempt)), maxTimeout)_.
In retry source code, between two retries. First retry doesn't care _Math.min()_, just _Math.round(random * minTimeout * Math.pow(factor, attempt))_)
Description | Before | After
------- | ------- | -------
Condition | npm's default setting<br>random = False = 1<br>retry = 2<br>minTimeout = 10 (sec)<br>maxTimeout = 60 (sec)<br>factor = 10 | custom setting<br>random = False = 1<br>retry = 2<br>minTimeout = 5 (sec)<br>maxTimeout = 60 (sec)<br>factor = 1<br>
First retry | Math.round(1 * 10 (sec) * 10^1)) | Math.round(1 * 5 (sec) * 1^1))
First retry result (Approximately) | 100 (sec) | 5 (sec)
Second retry | Math.min(Math.round(1 * 10 (sec) * 10^2), 60 (sec)) | Math.min(Math.round(1 * 5 (sec) * 1^2), 60 (sec))
Second retry result (Approximately) | 60 (sec) | 5 (sec)
Total waiting time (Approximately) | 160 (sec) | 10 (sec)
You can check like this below Screenshots.
### Screenshots
Before | After
-------|-------
<img width="1077" alt="2017-02-24 12 32 06" src="https://cloud.githubusercontent.com/assets/1144643/23267951/9deaec6e-fa2f-11e6-9171-5792f24de76d.png"> | <img width="1081" alt="2017-02-24 12 37 10" src="https://cloud.githubusercontent.com/assets/1144643/23267954/a12c0c0a-fa2f-11e6-99cd-335deef607ac.png">
### Questions:
* Does the licenses files need update? N/A
* Is there breaking changes for older versions? N/A
* Does this needs documentation? N/A
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/NohSeho/zeppelin ZEPPELIN-2094
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/2060.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2060
----
commit 89ce1688cc866fed67eaf3ee8db086f7dc2b4c73
Author: NohSeho <ia...@sehonoh.kr>
Date: 2017-03-05T06:33:42Z
[ZEPPELIN-2094] Decrease npm install retry time
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2060: [ZEPPELIN-2094] Decrease npm install retry time
Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on the issue:
https://github.com/apache/zeppelin/pull/2060
CI passed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #2060: [ZEPPELIN-2094] Decrease npm install retry time
Posted by NohSeho <gi...@git.apache.org>.
Github user NohSeho closed the pull request at:
https://github.com/apache/zeppelin/pull/2060
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2060: [ZEPPELIN-2094] Decrease npm install retry time
Posted by Leemoonsoo <gi...@git.apache.org>.
Github user Leemoonsoo commented on the issue:
https://github.com/apache/zeppelin/pull/2060
Merge to master if no further discussions.
@NohSeho I'd like to merge this improvement to `branch-0.7` as well, so it can be included in `0.7.1` release. However, in `branch-0.7`, there's [HeliumVisualizationFactory.java](https://github.com/apache/zeppelin/blob/branch-0.7/zeppelin-zengine/src/main/java/org/apache/zeppelin/helium/HeliumVisualizationFactory.java) instead of [HeliumBundleFactory.java](https://github.com/apache/zeppelin/blob/master/zeppelin-zengine/src/main/java/org/apache/zeppelin/helium/HeliumBundleFactory.java), so this PR can't be merged.
Do you mind make another PR for branch-0.7 with the same improvement?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2060: [ZEPPELIN-2094] Decrease npm install retry time
Posted by 1ambda <gi...@git.apache.org>.
Github user 1ambda commented on the issue:
https://github.com/apache/zeppelin/pull/2060
Hi @echarles. I can install local (not published) spell modules even in current master. Could you describe more about problem?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2060: [ZEPPELIN-2094] Decrease npm install retry time
Posted by NohSeho <gi...@git.apache.org>.
Github user NohSeho commented on the issue:
https://github.com/apache/zeppelin/pull/2060
@Leemoonsoo why not, I will be right back :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #2060: [ZEPPELIN-2094] Decrease npm install retry time
Posted by NohSeho <gi...@git.apache.org>.
Github user NohSeho closed the pull request at:
https://github.com/apache/zeppelin/pull/2060
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2060: [ZEPPELIN-2094] Decrease npm install retry time
Posted by NohSeho <gi...@git.apache.org>.
Github user NohSeho commented on the issue:
https://github.com/apache/zeppelin/pull/2060
@khalidhuseynov great! thanks alot :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2060: [ZEPPELIN-2094] Decrease npm install retry time
Posted by khalidhuseynov <gi...@git.apache.org>.
Github user khalidhuseynov commented on the issue:
https://github.com/apache/zeppelin/pull/2060
@NohSeho closing and opening PR may help
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin pull request #2060: [ZEPPELIN-2094] Decrease npm install retry time
Posted by NohSeho <gi...@git.apache.org>.
GitHub user NohSeho reopened a pull request:
https://github.com/apache/zeppelin/pull/2060
[ZEPPELIN-2094] Decrease npm install retry time
### What is this PR for?
It\u2019s too delayed for npm install when computer do not connected any networks.
Beacause when npm install, it has too long retry timeout.
This PR is to decrease retry timeout when npm install.
### What type of PR is it?
Improvement
### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-2094
### How should this be tested?
you must enable any one helium before test
Line 197 In zeppelin-zengine org.apache.zeppelin.helium.HeliumBundleFactory.java
First set with
`String npmCommand = "install \u2014loglevel=error\u201d;`
and You don\u2019t connect any ethernet or wireless internet.
build & run
and set with
`String npmCommand = "install \u2014fetch-retries=2 \u2014fetch-retry-factor=1 \u2014fetch-retry-mintimeout=5000 \u2014loglevel=error\u201d;`
also don\u2019t connect any networks, build & run.
WHY
retries = 2
factor = 1
mintimeout = 5(sec)?
npm use [retry](https://github.com/tim-kos/node-retry) module to retry.
It refers [this article](http://dthain.blogspot.kr/2009/02/exponential-backoff-in-distributed.html) for retry algorithms.
It is a math which structured _Math.min(Math.round(random * minTimeout * Math.pow(factor, attempt)), maxTimeout)_.
In retry source code, between two retries. First retry doesn't care _Math.min()_, just _Math.round(random * minTimeout * Math.pow(factor, attempt))_)
Description | Before | After
------- | ------- | -------
Condition | npm's default setting<br>random = False = 1<br>retry = 2<br>minTimeout = 10 (sec)<br>maxTimeout = 60 (sec)<br>factor = 10 | custom setting<br>random = False = 1<br>retry = 2<br>minTimeout = 5 (sec)<br>maxTimeout = 60 (sec)<br>factor = 1<br>
First retry | Math.round(1 * 10 (sec) * 10^1)) | Math.round(1 * 5 (sec) * 1^1))
First retry result (Approximately) | 100 (sec) | 5 (sec)
Second retry | Math.min(Math.round(1 * 10 (sec) * 10^2), 60 (sec)) | Math.min(Math.round(1 * 5 (sec) * 1^2), 60 (sec))
Second retry result (Approximately) | 60 (sec) | 5 (sec)
Total waiting time (Approximately) | 160 (sec) | 10 (sec)
You can check like this below Screenshots.
### Screenshots
Before | After
-------|-------
<img width="1077" alt="2017-02-24 12 32 06" src="https://cloud.githubusercontent.com/assets/1144643/23267951/9deaec6e-fa2f-11e6-9171-5792f24de76d.png"> | <img width="1081" alt="2017-02-24 12 37 10" src="https://cloud.githubusercontent.com/assets/1144643/23267954/a12c0c0a-fa2f-11e6-99cd-335deef607ac.png">
### Questions:
* Does the licenses files need update? N/A
* Is there breaking changes for older versions? N/A
* Does this needs documentation? N/A
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/NohSeho/zeppelin ZEPPELIN-2094
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/2060.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2060
----
commit b7296ca0d60ba99d6c874831330a823ccea2f2b0
Author: NohSeho <ia...@sehonoh.kr>
Date: 2017-02-23T15:59:57Z
[ZEPPELIN-2094] Decrease npm install retry time
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2060: [ZEPPELIN-2094] Decrease npm install retry time
Posted by echarles <gi...@git.apache.org>.
Github user echarles commented on the issue:
https://github.com/apache/zeppelin/pull/2060
I am adding a `json` file in the `helium` folder with the artifact that points to a full path on my local drive. In that location I have the spell project. It was working 3 days ago. Today, I updated, and it does not work anymore unless I revert the HeliumBundleFactory class to its previous version.
But if it works at your side, it must be my env that needs to be fixed. I will double check and report back. Thx again.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2060: [ZEPPELIN-2094] Decrease npm install retry time
Posted by echarles <gi...@git.apache.org>.
Github user echarles commented on the issue:
https://github.com/apache/zeppelin/pull/2060
I think this change breaks the installation of spell modules that are not yet published in npmjs repository.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2060: [ZEPPELIN-2094] Decrease npm install retry time
Posted by NohSeho <gi...@git.apache.org>.
Github user NohSeho commented on the issue:
https://github.com/apache/zeppelin/pull/2060
I successfully restart build but it does not change.
How can I do this?
https://travis-ci.org/NohSeho/zeppelin/builds/204825105
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2060: [ZEPPELIN-2094] Decrease npm install retry time
Posted by 1ambda <gi...@git.apache.org>.
Github user 1ambda commented on the issue:
https://github.com/apache/zeppelin/pull/2060
Thanks for improvement! I will review and comment soon.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2060: [ZEPPELIN-2094] Decrease npm install retry time
Posted by NohSeho <gi...@git.apache.org>.
Github user NohSeho commented on the issue:
https://github.com/apache/zeppelin/pull/2060
I will update the pr description soon.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2060: [ZEPPELIN-2094] Decrease npm install retry time
Posted by 1ambda <gi...@git.apache.org>.
Github user 1ambda commented on the issue:
https://github.com/apache/zeppelin/pull/2060
Tested and works as described.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] zeppelin issue #2060: [ZEPPELIN-2094] Decrease npm install retry time
Posted by Leemoonsoo <gi...@git.apache.org>.
Github user Leemoonsoo commented on the issue:
https://github.com/apache/zeppelin/pull/2060
Thanks for the contribution.
Please take a look 'Continuous Integration" section http://zeppelin.apache.org/contribution/contributions.html#continuous-integration .
Once you have setup, make sure you turned on "Build Pushes"
Once everything is done, if you push a new commit in this PR then build status will be displayed correctly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---