You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zeppelin.apache.org by felixcheung <gi...@git.apache.org> on 2016/02/12 01:51:53 UTC
[GitHub] incubator-zeppelin pull request: Travis - spark script should stop...
GitHub user felixcheung opened a pull request:
https://github.com/apache/incubator-zeppelin/pull/710
Travis - spark script should stop on error
### What is this PR for?
There has been a few cases Travis fails to download Spark release but doesn't stop. Stopping would make it easier to track down and check the exit code.
### What type of PR is it?
Improvement
### Todos
* [x] - Update spark download & start/stop script
### Is there a relevant Jira issue?
N/A
### How should this be tested?
Run Travis CI
### Screenshots (if appropriate)
N/A
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/felixcheung/incubator-zeppelin checkdownload
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-zeppelin/pull/710.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #710
----
commit 3c64db88e24ab3b5a5f45aeee7bee5eee9dfd3f3
Author: Felix Cheung <fe...@hotmail.com>
Date: 2016-02-12T00:48:10Z
spark script stop on error
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: Travis - spark script should stop...
Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/710#issuecomment-183495276
Another download timeout
```
+wget -q http://mirror.sdunix.com/apache/spark/spark-1.3.1/spark-1.3.1-bin-hadoop2.3.tgz
No output has been received in the last 10 minutes, this potentially indicates a stalled build or something wrong with the build itself.
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: Travis - spark script should stop...
Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/710#issuecomment-184963957
another download error https://s3.amazonaws.com/archive.travis-ci.org/jobs/109754384/log.txt:
```
SPARK_HOME is /home/travis/build/apache/incubator-zeppelin/spark-1.3.1-bin-hadoop2.3
tar (child): spark-1.3.1-bin-hadoop2.3.tgz: Cannot open: No such file or directory
tar (child): Error is not recoverable: exiting now
tar: Child returned status 2
tar: Error is not recoverable: exiting now
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: Travis - spark script should stop...
Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/710#issuecomment-185425799
Let's merge this if no more discussion. At least we could isolate download or spark start failure with this change.
As for container base I could investigate that further. There is one pending PR that is including sudo commands though that would not work with it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: Travis - spark script should stop...
Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/710#issuecomment-183597881
Right, ideally. My understanding is though caching is available only for the travis-ci.com which they charge for and container based infra cannot use `sudo` - neither seems to work with us at this point?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: Travis - spark script should stop...
Posted by felixcheung <gi...@git.apache.org>.
GitHub user felixcheung reopened a pull request:
https://github.com/apache/incubator-zeppelin/pull/710
Travis - spark script should stop on error
### What is this PR for?
There has been a few cases Travis fails to download Spark release but doesn't stop. Stopping would make it easier to track down and check the exit code.
### What type of PR is it?
Improvement
### Todos
* [x] - Update spark download & start/stop script
### Is there a relevant Jira issue?
N/A
### How should this be tested?
Run Travis CI
### Screenshots (if appropriate)
N/A
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/felixcheung/incubator-zeppelin checkdownload
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-zeppelin/pull/710.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #710
----
commit 3c64db88e24ab3b5a5f45aeee7bee5eee9dfd3f3
Author: Felix Cheung <fe...@hotmail.com>
Date: 2016-02-12T00:48:10Z
spark script stop on error
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: Travis - spark script should stop...
Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung closed the pull request at:
https://github.com/apache/incubator-zeppelin/pull/710
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: Travis - spark script should stop...
Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung closed the pull request at:
https://github.com/apache/incubator-zeppelin/pull/710
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: Travis - spark script should stop...
Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/710#issuecomment-184012287
@prabhjyotsingh this seems to fail quite a bit:
```
20:01:31,655 ERROR org.apache.zeppelin.ZeppelinIT:198 - Exception in ZeppelinIT while testAngularDisplay
org.openqa.selenium.ElementNotVisibleException: Element is not currently visible and so may not be interacted with
Command duration or timeout: 10 milliseconds
Build info: version: '2.48.2', revision: '41bccdd10cf2c0560f637404c2d96164b67d9d67', time: '2015-10-09 13:08:06'
System info: host: 'testing-worker-linux-docker-77898bf2-3380-linux-5', ip: '172.17.0.196', os.name: 'Linux', os.arch: 'amd64', os.version: '3.13.0-40-generic', java.version: '1.7.0_76'
Session ID: b2de8969-6562-4b77-9902-910c5df7750e
Driver info: org.openqa.selenium.firefox.FirefoxDriver
Capabilities [{platform=LINUX, acceptSslCerts=true, javascriptEnabled=true, cssSelectorsEnabled=true, databaseEnabled=true, browserName=firefox, handlesAlerts=true, nativeEvents=false, webStorageEnabled=true, rotatable=false, locationContextEnabled=true, applicationCacheEnabled=true, takesScreenshot=true, version=31.0}]
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.openqa.selenium.remote.ErrorHandler.createThrowable(ErrorHandler.java:206)
at org.openqa.selenium.remote.ErrorHandler.throwIfResponseFailed(ErrorHandler.java:158)
at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:647)
at org.openqa.selenium.remote.RemoteWebElement.execute(RemoteWebElement.java:326)
at org.openqa.selenium.remote.RemoteWebElement.sendKeys(RemoteWebElement.java:121)
at org.apache.zeppelin.ZeppelinIT.testAngularDisplay(ZeppelinIT.java:169)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:264)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:124)
at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:200)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:153)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
Caused by: org.openqa.selenium.ElementNotVisibleException: Element is not currently visible and so may not be interacted with
Build info: version: '2.48.2', revision: '41bccdd10cf2c0560f637404c2d96164b67d9d67', time: '2015-10-09 13:08:06'
System info: host: 'testing-worker-linux-docker-77898bf2-3380-linux-5', ip: '172.17.0.196', os.name: 'Linux', os.arch: 'amd64', os.version: '3.13.0-40-generic', java.version: '1.7.0_76'
Driver info: driver.version: unknown
at <anonymous class>.fxdriver.preconditions.visible(file:///tmp/anonymous7743485075556311307webdriver-profile/extensions/fxdriver@googlecode.com/components/command-processor.js:9981)
at <anonymous class>.DelayedCommand.prototype.checkPreconditions_(file:///tmp/anonymous7743485075556311307webdriver-profile/extensions/fxdriver@googlecode.com/components/command-processor.js:12517)
at <anonymous class>.DelayedCommand.prototype.executeInternal_/h(file:///tmp/anonymous7743485075556311307webdriver-profile/extensions/fxdriver@googlecode.com/components/command-processor.js:12534)
at <anonymous class>.DelayedCommand.prototype.executeInternal_(file:///tmp/anonymous7743485075556311307webdriver-profile/extensions/fxdriver@googlecode.com/components/command-processor.js:12539)
at <anonymous class>.DelayedCommand.prototype.execute/<(file:///tmp/anonymous7743485075556311307webdriver-profile/extensions/fxdriver@googlecode.com/components/command-processor.js:12481)
Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 151.157 sec <<< FAILURE! - in org.apache.zeppelin.ZeppelinIT
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: Travis - spark script should stop...
Posted by felixcheung <gi...@git.apache.org>.
GitHub user felixcheung reopened a pull request:
https://github.com/apache/incubator-zeppelin/pull/710
Travis - spark script should stop on error
### What is this PR for?
There has been a few cases Travis fails to download Spark release but doesn't stop. Stopping would make it easier to track down and check the exit code.
### What type of PR is it?
Improvement
### Todos
* [x] - Update spark download & start/stop script
### Is there a relevant Jira issue?
N/A
### How should this be tested?
Run Travis CI
### Screenshots (if appropriate)
N/A
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/felixcheung/incubator-zeppelin checkdownload
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-zeppelin/pull/710.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #710
----
commit 3c64db88e24ab3b5a5f45aeee7bee5eee9dfd3f3
Author: Felix Cheung <fe...@hotmail.com>
Date: 2016-02-12T00:48:10Z
spark script stop on error
commit d89cde9347aece3f4a0efcb54715828fad38a74b
Author: Felix Cheung <fe...@hotmail.com>
Date: 2016-02-14T04:58:49Z
fix version check
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: Travis - spark script should stop...
Posted by felixcheung <gi...@git.apache.org>.
GitHub user felixcheung reopened a pull request:
https://github.com/apache/incubator-zeppelin/pull/710
Travis - spark script should stop on error
### What is this PR for?
There has been a few cases Travis fails to download Spark release but doesn't stop. Stopping would make it easier to track down and check the exit code.
### What type of PR is it?
Improvement
### Todos
* [x] - Update spark download & start/stop script
### Is there a relevant Jira issue?
N/A
### How should this be tested?
Run Travis CI
### Screenshots (if appropriate)
N/A
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/felixcheung/incubator-zeppelin checkdownload
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-zeppelin/pull/710.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #710
----
commit 3c64db88e24ab3b5a5f45aeee7bee5eee9dfd3f3
Author: Felix Cheung <fe...@hotmail.com>
Date: 2016-02-12T00:48:10Z
spark script stop on error
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: Travis - spark script should stop...
Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung closed the pull request at:
https://github.com/apache/incubator-zeppelin/pull/710
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: Travis - spark script should stop...
Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/710#issuecomment-183224046
Good call, this might be better? http://stackoverflow.com/questions/7577615/parallel-wget-in-bash?answertab=votes#tab-top
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: Travis - spark script should stop...
Posted by Leemoonsoo <gi...@git.apache.org>.
Github user Leemoonsoo commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/710#issuecomment-183588484
Another approach would be https://docs.travis-ci.com/user/caching/. It seems travis-ci capable of cache directory when it is configured to [use container based infrastructure](https://docs.travis-ci.com/user/workers/container-based-infrastructure/).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: Travis - spark script should stop...
Posted by Leemoonsoo <gi...@git.apache.org>.
Github user Leemoonsoo commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/710#issuecomment-183611903
Not 100%sure, but my understanding is `sudo: false` enables container based infrastructure as well as cache.
For example, Apache Flink project looks like using cache capability of travis-ci https://github.com/apache/flink/blob/master/.travis.yml
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: Travis - spark script should stop...
Posted by felixcheung <gi...@git.apache.org>.
GitHub user felixcheung reopened a pull request:
https://github.com/apache/incubator-zeppelin/pull/710
Travis - spark script should stop on error
### What is this PR for?
There has been a few cases Travis fails to download Spark release but doesn't stop. Stopping would make it easier to track down and check the exit code.
### What type of PR is it?
Improvement
### Todos
* [x] - Update spark download & start/stop script
### Is there a relevant Jira issue?
N/A
### How should this be tested?
Run Travis CI
### Screenshots (if appropriate)
N/A
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/felixcheung/incubator-zeppelin checkdownload
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-zeppelin/pull/710.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #710
----
commit 3c64db88e24ab3b5a5f45aeee7bee5eee9dfd3f3
Author: Felix Cheung <fe...@hotmail.com>
Date: 2016-02-12T00:48:10Z
spark script stop on error
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: Travis - spark script should stop...
Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung closed the pull request at:
https://github.com/apache/incubator-zeppelin/pull/710
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: Travis - spark script should stop...
Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung closed the pull request at:
https://github.com/apache/incubator-zeppelin/pull/710
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: Travis - spark script should stop...
Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung closed the pull request at:
https://github.com/apache/incubator-zeppelin/pull/710
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: Travis - spark script should stop...
Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/710#issuecomment-183138824
Download from Apache mirror timeout still seem to be a big source of issues,
https://api.travis-ci.org/jobs/108686270/log.txt?deansi=true
```
+wget -q http://mirror.tcpdiag.net/apache/spark/spark-1.6.0/spark-1.6.0-bin-hadoop2.3.tgz
No output has been received in the last 10 minutes, this potentially indicates a stalled build or something wrong with the build itself.
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: Travis - spark script should stop...
Posted by prabhjyotsingh <gi...@git.apache.org>.
Github user prabhjyotsingh commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/710#issuecomment-184513238
Thanks @felixcheung for pointing it out. I was neither able to reproduce it on my local, nor was able to do it on local travis. Hence, raise this PR https://github.com/apache/incubator-zeppelin/pull/719. I'll try and fix it there.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: Travis - spark script should stop...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-zeppelin/pull/710
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: Travis - spark script should stop...
Posted by felixcheung <gi...@git.apache.org>.
GitHub user felixcheung reopened a pull request:
https://github.com/apache/incubator-zeppelin/pull/710
Travis - spark script should stop on error
### What is this PR for?
There has been a few cases Travis fails to download Spark release but doesn't stop. Stopping would make it easier to track down and check the exit code.
### What type of PR is it?
Improvement
### Todos
* [x] - Update spark download & start/stop script
### Is there a relevant Jira issue?
N/A
### How should this be tested?
Run Travis CI
### Screenshots (if appropriate)
N/A
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/felixcheung/incubator-zeppelin checkdownload
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-zeppelin/pull/710.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #710
----
commit 3c64db88e24ab3b5a5f45aeee7bee5eee9dfd3f3
Author: Felix Cheung <fe...@hotmail.com>
Date: 2016-02-12T00:48:10Z
spark script stop on error
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: Travis - spark script should stop...
Posted by prabhjyotsingh <gi...@git.apache.org>.
Github user prabhjyotsingh commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/710#issuecomment-183186549
I have a hack, what if instead of
`wget -q http://mirror.tcpdiag.net/apache/spark/spark-1.6.0/spark-1.6.0-bin-hadoop2.3.tgz`
we use
wget -q -r -np -N http://mirror.tcpdiag.net/apache/spark/spark-1.6.0/spark-1.6.0-bin-hadoop2.3.tgz &
wget -q -r -np -N http://mirror.tcpdiag.net/apache/spark/spark-1.6.0/spark-1.6.0-bin-hadoop2.3.tgz &
wget -q -r -np -N http://mirror.tcpdiag.net/apache/spark/spark-1.6.0/spark-1.6.0-bin-hadoop2.3.tgz &
wget -q -r -np -N http://mirror.tcpdiag.net/apache/spark/spark-1.6.0/spark-1.6.0-bin-hadoop2.3.tgz &
wget -q -r -np -N http://mirror.tcpdiag.net/apache/spark/spark-1.6.0/spark-1.6.0-bin-hadoop2.3.tgz
Which will make wget download using 5 simultaneous connections, which may result in downloading it quicker.
or
THREADS=5
WGETFLAGS=’-r -np -N’
URL="http://mirror.tcpdiag.net/apache/spark/spark-1.6.0/spark-1.6.0-bin-hadoop2.3.tgz"
for i in `seq 1 $((THREADS-1))`
do
Command=”$Command””wget $WGETFLAGS $URL”‘ & ‘
done
Command=”$Command””wget $WGETFLAGS $URL”
eval $Command
References: http://blog.netflowdevelopments.com/2011/01/24/multi-threaded-downloading-with-wget/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: Travis - spark script should stop...
Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung commented on the pull request:
https://github.com/apache/incubator-zeppelin/pull/710#issuecomment-183138082
https://s3.amazonaws.com/archive.travis-ci.org/jobs/107893675/log.txt
```
[0K$ ./testing/startSparkCluster.sh $SPARK_VER $HADOOP_VER
SPARK_HOME is /home/travis/build/apache/incubator-zeppelin/spark-1.3.1-bin-hadoop2.3
gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now
./testing/startSparkCluster.sh: line 59: /home/travis/build/apache/incubator-zeppelin/spark-1.3.1-bin-hadoop2.3/sbin/start-master.sh: No such file or directory
./testing/startSparkCluster.sh: line 63: /home/travis/build/apache/incubator-zeppelin/spark-1.3.1-bin-hadoop2.3/sbin/start-slave.sh: No such file or directory
travis_time:end:2d16bb10:start=1454972345135573912,finish=1454972360543622554,duration=15408048642
[0K
[31;1mThe command "./testing/startSparkCluster.sh $SPARK_VER $HADOOP_VER" failed and exited with 127 during .[0m
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: Travis - spark script should stop...
Posted by felixcheung <gi...@git.apache.org>.
Github user felixcheung closed the pull request at:
https://github.com/apache/incubator-zeppelin/pull/710
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: Travis - spark script should stop...
Posted by felixcheung <gi...@git.apache.org>.
GitHub user felixcheung reopened a pull request:
https://github.com/apache/incubator-zeppelin/pull/710
Travis - spark script should stop on error
### What is this PR for?
There has been a few cases Travis fails to download Spark release but doesn't stop. Stopping would make it easier to track down and check the exit code.
### What type of PR is it?
Improvement
### Todos
* [x] - Update spark download & start/stop script
### Is there a relevant Jira issue?
N/A
### How should this be tested?
Run Travis CI
### Screenshots (if appropriate)
N/A
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/felixcheung/incubator-zeppelin checkdownload
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-zeppelin/pull/710.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #710
----
commit 3c64db88e24ab3b5a5f45aeee7bee5eee9dfd3f3
Author: Felix Cheung <fe...@hotmail.com>
Date: 2016-02-12T00:48:10Z
spark script stop on error
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-zeppelin pull request: Travis - spark script should stop...
Posted by felixcheung <gi...@git.apache.org>.
GitHub user felixcheung reopened a pull request:
https://github.com/apache/incubator-zeppelin/pull/710
Travis - spark script should stop on error
### What is this PR for?
There has been a few cases Travis fails to download Spark release but doesn't stop. Stopping would make it easier to track down and check the exit code.
### What type of PR is it?
Improvement
### Todos
* [x] - Update spark download & start/stop script
### Is there a relevant Jira issue?
N/A
### How should this be tested?
Run Travis CI
### Screenshots (if appropriate)
N/A
### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/felixcheung/incubator-zeppelin checkdownload
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-zeppelin/pull/710.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #710
----
commit 3c64db88e24ab3b5a5f45aeee7bee5eee9dfd3f3
Author: Felix Cheung <fe...@hotmail.com>
Date: 2016-02-12T00:48:10Z
spark script stop on error
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---